On Sep 27, 2008, at 11:15 AM, Jerome Velociter wrote:
Vincent Massol wrote:
Hi,
Since we really need to have automated tests for the new WYSIWYG from the onset here are some ideas I'd like to have:
1) It should use Selenium and we need to run it on different browsers (FF2, FF3, IE6, IE7, IE8 minimum).
+1, It sounds good, but we should be careful of the overhead of making such browser-low-level tests being cross browser. Worth reading http://blogs.atlassian.com/developer/2007/08/selenium_is_the_pain_worth_it.h...
Very interesting read but several things: 1) It was written in 2007 and since then there have been several releases of Selenium which have addressed several of the issues mentioned 2) Read all the comments they are very interesting. He admits it's getting easier now. They've also built some wrapper methods for making it easier to write tests 3) There are no better tool I know of. I have tested a few in the past and they all had problems (in general worse than selenium) 4) We need it cross browser but currently we're only running on a single browser so this is going to be challenging for sure 5) Even on one browser our tests are still failing so we need to fix them. JV, any idea (since you looked into that)? :) Thanks -Vincent
My 2 Euro cents, Jerome.
2) We need to develop a complete DSL for entering data in the editor (typeText, moveCursorLeft, clickBold, insertLink, etc). These commands sent in the test should be independent of the editor technology, which means I should be able to run the same test in the current TinyMCE editor or in the new GWT editor or in any future editor. This is important so that we can capitalize on our editor test suite.
Note: The goal is not to run the tests on the Tiny MCE editor (although that would be possible) but more to ensure that our tests are as solid as possible for the future and as easy to write as possible.
3) As a consequence the asserts done in the test should be on the generated XHTML (i.e. including the HTMLCleaner for the GWT editor) since otherwise there would be too many differences between editors. 4) Each editor should have its test suite where it can specify which test to run (since some features are available in the new GWT editor for ex but not in the tinyMCE one) 5) Writing a new test should take less than 5 minutes without even the need to run the test in the editor to prepare it. This means the DSL should be simple and expressive. For example here's an example of a test:
testVerifyCanDeleteList() { typeText("a"); selectBulletedList(); moveCursorLeft(); typeBackspace(); assertXHTML("<p>a</p>"); }
Note: This test is not taken at random since it currently doesn't pass ;)
6) Tests must be deterministic (i.e each action should verify that the action has worked before continuing. For example a typeText should have a wait on the type text in the page)
WDYT?
Thanks -Vincent