[xwiki-devs] [Brainstorming] Should we support only one skin in our functional tests or not?
Hi devs, ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri). Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo. After his changes are made our framework won’t work anymore with Colibri. The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations. One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc. Any opinion? Thanks -Vincent
On 08/19/2014 12:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Tests should work on different skins. At some point we did have some support for skins, in the old xe-test-selenium module [1], but this feature didn't survive the pageobjects revolution. [1] https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/... -- Sergiu Dumitriu http://purl.org/net/sergiu
On 19 Aug 2014 at 19:53:54, Sergiu Dumitriu ([email protected](mailto:[email protected])) wrote:
On 08/19/2014 12:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Tests should work on different skins. At some point we did have some support for skins, in the old xe-test-selenium module [1], but this feature didn't survive the pageobjects revolution.
Yes (I coded it). The main reason it didn’t survive is because several skins didn’t survive and in the end we’ve been supporting only 1 skin for years. The question are: 1) whether we’re going to support several skins in the future or not? We’ve already said we’re going drop Colibri when Flamingo is ready. Supporting several skins in the test framework is a lot of effort. 2) if we want to support several skins in the test fwk when is the right time. ATM we’re late for Flamingo as the default skin for 6.2 and doing this in 6.2 would mean postponing 6.2’s release by at least 1 to 2 weeks. Thanks -Vincent
[1] https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/...
-- Sergiu Dumitriu
Hi, What I don`t fully understand is why changing a skin should affect the UI tests of an application? Page objects should be specific to the application's UI elements that are custom coded to reflect the state of and to perform operations on the UI elements of the tested application only. These UI elements should generally be located inside the document's rendered content and should be minimally impacted by a skin change. For me, an application should be limited as much as possible to the document's rendered content. When we are not testing applications, but the UI of XWiki itself, then indeed, perhaps we should have both skin-specific page-objects and tests. So my question is: How much is the current skin change affecting our UI tests and at which level? Are application UI tests affected as well? Thanks, Eduard On Tue, Aug 19, 2014 at 9:02 PM, [email protected] <[email protected]> wrote:
On 19 Aug 2014 at 19:53:54, Sergiu Dumitriu ([email protected](mailto: [email protected])) wrote:
On 08/19/2014 12:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Tests should work on different skins. At some point we did have some support for skins, in the old xe-test-selenium module [1], but this feature didn't survive the pageobjects revolution.
Yes (I coded it). The main reason it didn’t survive is because several skins didn’t survive and in the end we’ve been supporting only 1 skin for years.
The question are: 1) whether we’re going to support several skins in the future or not? We’ve already said we’re going drop Colibri when Flamingo is ready. Supporting several skins in the test framework is a lot of effort. 2) if we want to support several skins in the test fwk when is the right time. ATM we’re late for Flamingo as the default skin for 6.2 and doing this in 6.2 would mean postponing 6.2’s release by at least 1 to 2 weeks.
Thanks -Vincent
[1]
https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/...
-- Sergiu Dumitriu
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On 20 Aug 2014 at 10:51:51, Eduard Moraru ([email protected](mailto:[email protected])) wrote:
Hi,
What I don`t fully understand is why changing a skin should affect the UI tests of an application?
Page objects should be specific to the application's UI elements that are custom coded to reflect the state of and to perform operations on the UI elements of the tested application only. These UI elements should generally be located inside the document's rendered content and should be minimally impacted by a skin change. For me, an application should be limited as much as possible to the document's rendered content.
When we are not testing applications, but the UI of XWiki itself, then indeed, perhaps we should have both skin-specific page-objects and tests.
So my question is: How much is the current skin change affecting our UI tests and at which level? Are application UI tests affected as well?
You can check by yourself from the commits done by GD yesterday and from the mail I sent yesterday about the failing tests in platform. I haven’t counted them. Still there are lots of UI tests about the XWiki UI so the question remains. Thanks -Vincent
Thanks, Eduard
On Tue, Aug 19, 2014 at 9:02 PM, [email protected] wrote:
On 19 Aug 2014 at 19:53:54, Sergiu Dumitriu ([email protected](mailto: [email protected])) wrote:
On 08/19/2014 12:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Tests should work on different skins. At some point we did have some support for skins, in the old xe-test-selenium module [1], but this feature didn't survive the pageobjects revolution.
Yes (I coded it). The main reason it didn’t survive is because several skins didn’t survive and in the end we’ve been supporting only 1 skin for years.
The question are: 1) whether we’re going to support several skins in the future or not? We’ve already said we’re going drop Colibri when Flamingo is ready. Supporting several skins in the test framework is a lot of effort. 2) if we want to support several skins in the test fwk when is the right time. ATM we’re late for Flamingo as the default skin for 6.2 and doing this in 6.2 would mean postponing 6.2’s release by at least 1 to 2 weeks.
Thanks -Vincent
[1]
https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/...
-- Sergiu Dumitriu
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Just to give one example, AllDocsTest from the Index app is failing. One reason is that it logs the user out to verify that the LT doesn’t display the actions reserved for admins. So it does a: // Logs out to be guest page.logout(); And this in turn will use a skin UI element (the logout button at the top right). Now you’re right that for this precise example, we could modify the test so that instead of clicking the logout button we log out the user by using TestUtils, i.e. not make it part of what we test. It could be cleaner. Of course we would need to have some other functional tests to prove that logout works fine when clicked but we do have that in this case. But as I mentioned we still have a lot of skin-related test code and this pushes my question to this code. Thanks -Vincent On 20 Aug 2014 at 10:57:19, [email protected] ([email protected](mailto:[email protected])) wrote:
On 20 Aug 2014 at 10:51:51, Eduard Moraru ([email protected](mailto:[email protected])) wrote:
Hi,
What I don`t fully understand is why changing a skin should affect the UI tests of an application?
Page objects should be specific to the application's UI elements that are custom coded to reflect the state of and to perform operations on the UI elements of the tested application only. These UI elements should generally be located inside the document's rendered content and should be minimally impacted by a skin change. For me, an application should be limited as much as possible to the document's rendered content.
When we are not testing applications, but the UI of XWiki itself, then indeed, perhaps we should have both skin-specific page-objects and tests.
So my question is: How much is the current skin change affecting our UI tests and at which level? Are application UI tests affected as well?
You can check by yourself from the commits done by GD yesterday and from the mail I sent yesterday about the failing tests in platform.
I haven’t counted them.
Still there are lots of UI tests about the XWiki UI so the question remains.
Thanks -Vincent
Thanks, Eduard
On Tue, Aug 19, 2014 at 9:02 PM, [email protected] wrote:
On 19 Aug 2014 at 19:53:54, Sergiu Dumitriu ([email protected](mailto: [email protected])) wrote:
On 08/19/2014 12:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Tests should work on different skins. At some point we did have some support for skins, in the old xe-test-selenium module [1], but this feature didn't survive the pageobjects revolution.
Yes (I coded it). The main reason it didn’t survive is because several skins didn’t survive and in the end we’ve been supporting only 1 skin for years.
The question are: 1) whether we’re going to support several skins in the future or not? We’ve already said we’re going drop Colibri when Flamingo is ready. Supporting several skins in the test framework is a lot of effort. 2) if we want to support several skins in the test fwk when is the right time. ATM we’re late for Flamingo as the default skin for 6.2 and doing this in 6.2 would mean postponing 6.2’s release by at least 1 to 2 weeks.
Thanks -Vincent
[1]
https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/...
-- Sergiu Dumitriu
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
I'm fine with running the functional tests only for the default skin. Note that although we support many browsers we run the functional tests only for Firefox. So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more). Thanks, Marius On Tue, Aug 19, 2014 at 7:12 PM, [email protected] <[email protected]> wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea ([email protected](mailto:[email protected])) wrote:
I'm fine with running the functional tests only for the default skin. Note that although we support many browsers we run the functional tests only for Firefox.
Which is bad and not something we want ;) Thanks -Vincent
So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
Thanks, Marius
On Tue, Aug 19, 2014 at 7:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent
On Wed, Aug 20, 2014 at 4:01 PM, [email protected] <[email protected]> wrote:
On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea ( [email protected](mailto:[email protected])) wrote:
I'm fine with running the functional tests only for the default skin. Note that although we support many browsers we run the functional tests only for Firefox.
Which is bad and not something we want ;)
We want to have test on all browser with as many skin as possible, including test of all combination of migrated version, and ... well, what we want is not really compatible with true reality :) So, taking into account our team, I do not think we can afford the support of more than one skin. In practice, I doubt those that see our new skin will want to stick with colibri for long, and those that will, will also stick with the old xwiki version they have. The support for both colibri and flamingo in parallel is also short term, so unless I over estimate the workload, I am +1 to only support one skin in functional tests, and to also put some reserves on the usage of colibri in upcoming versions. Thanks,
Thanks -Vincent
So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
Thanks, Marius
On Tue, Aug 19, 2014 at 7:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO
Hi, +1 for only testing on Flamingo. Thanks, Manuel On Thu, Aug 21, 2014 at 12:57 PM, Denis Gervalle <[email protected]> wrote:
On Wed, Aug 20, 2014 at 4:01 PM, [email protected] <[email protected]> wrote:
On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea ( [email protected](mailto:[email protected])) wrote:
I'm fine with running the functional tests only for the default skin. Note that although we support many browsers we run the functional tests only for Firefox.
Which is bad and not something we want ;)
We want to have test on all browser with as many skin as possible, including test of all combination of migrated version, and ... well, what we want is not really compatible with true reality :)
So, taking into account our team, I do not think we can afford the support of more than one skin. In practice, I doubt those that see our new skin will want to stick with colibri for long, and those that will, will also stick with the old xwiki version they have. The support for both colibri and flamingo in parallel is also short term, so unless I over estimate the workload, I am +1 to only support one skin in functional tests, and to also put some reserves on the usage of colibri in upcoming versions.
Thanks,
Thanks -Vincent
So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
Thanks, Marius
On Tue, Aug 19, 2014 at 7:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s
3
buttons. Etc.
Any opinion?
Thanks -Vincent
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Just for the record: anyone saying +1 for only testing on Flaming must realize that they are basically saying that we will NOT support more than 1 skin (Because manual testing doesn’t scale). Thanks -Vincent On 21 Aug 2014 at 14:22:38, Manuel Smeria ([email protected](mailto:[email protected])) wrote:
Hi,
+1 for only testing on Flamingo.
Thanks, Manuel
On Thu, Aug 21, 2014 at 12:57 PM, Denis Gervalle wrote:
On Wed, Aug 20, 2014 at 4:01 PM, [email protected] wrote:
On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea ( [email protected](mailto:[email protected])) wrote:
I'm fine with running the functional tests only for the default skin. Note that although we support many browsers we run the functional tests only for Firefox.
Which is bad and not something we want ;)
We want to have test on all browser with as many skin as possible, including test of all combination of migrated version, and ... well, what we want is not really compatible with true reality :)
So, taking into account our team, I do not think we can afford the support of more than one skin. In practice, I doubt those that see our new skin will want to stick with colibri for long, and those that will, will also stick with the old xwiki version they have. The support for both colibri and flamingo in parallel is also short term, so unless I over estimate the workload, I am +1 to only support one skin in functional tests, and to also put some reserves on the usage of colibri in upcoming versions.
Thanks,
Thanks -Vincent
So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
Thanks, Marius
On Tue, Aug 19, 2014 at 7:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s
3
buttons. Etc.
Any opinion?
Thanks -Vincent
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On 21 Aug 2014 at 11:58:01, Denis Gervalle ([email protected](mailto:[email protected])) wrote:
On Wed, Aug 20, 2014 at 4:01 PM, [email protected] wrote:
On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea ( [email protected](mailto:[email protected])) wrote:
I'm fine with running the functional tests only for the default skin. Note that although we support many browsers we run the functional tests only for Firefox.
Which is bad and not something we want ;)
We want to have test on all browser with as many skin as possible, including test of all combination of migrated version, and ... well, what we want is not really compatible with true reality :)
You’re mixing a bit everything in the same bag here :) There are 2 aspects: - It’s specifically because we don’t have that much manpower that we must have our CI automatically test on various browsers/databases/OSes. This works well because the tests don’t need to be modified (in theory at least but mostly true) to be executed on various browsers/databases/OSes. - for the various skins it’s different because you need to write several test frameworks (one per skin) so it’s indeed more work.
So, taking into account our team, I do not think we can afford the support of more than one skin. In practice, I doubt those that see our new skin will want to stick with colibri for long, and those that will, will also stick with the old xwiki version they have. The support for both colibri and flamingo in parallel is also short term, so unless I over estimate the workload, I am +1 to only support one skin in functional tests, and to also put some reserves on the usage of colibri in upcoming versions.
It depends if we’re able to fulfil a large majority of use cases with only 1 skin or not. The fact that we’ve been able to support only one skin shows how complex writing an XWiki skin is. If you check on extensions.xwiki.org you’ll see we have very few skins, especially compared to other tools like wordpress, drupal, etc. We’ve been saying this for years but we haven’t made progress. Quite the opposite, over the years, we’ve made writing skins more complex (for example right now if you want to write a new skin you need to make sure to implement all Extension Points in your skin if you want your skin to behave properly with extensions!). Now one solution could be to have only 1 (or a few) base skins and improve Skin Themes so that users can do more and more in a Theme, thus allowing just to modify Themes and not a full skin. We started with Color Themes, added the ability to change the logo, and we are now moving to Themes with the ability to also change fonts. This is a step in the right direction I believe. In some future, we could maybe also support more Theme features. At some point we might need to fold default Panels layout (1 column, 2 columns, 3 columns, etc) and Panels selection into Themes too. Etc. If we push this to a level where having 1 skin and using various Themes allow XWiki users to implement the UI they need for their project then we won’t need to support more than 1 skin. But I fear this is just pushing the testing problem to Themes (but maybe to a lesser degree, depending on the freedom we give to Themes) ;) And if we wish to support several Themes we’ll need to test them… Anyway, for 6.2 I agree we don’t have the time to make our UI test framework multiskin-enabled. However, I believe we’ll need to refactor our test framework to allow supporting testing several UI. That will be useful when we need to do that but also for our users writing new skins and needing tests for them. I don’t think it’s that complicated to do. Thanks -Vincent
Thanks,
Thanks -Vincent
So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
Thanks, Marius
On Tue, Aug 19, 2014 at 7:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
On Thu, Aug 21, 2014 at 4:52 PM, [email protected] <[email protected]> wrote:
On 21 Aug 2014 at 11:58:01, Denis Gervalle ([email protected](mailto: [email protected])) wrote:
On Wed, Aug 20, 2014 at 4:01 PM, [email protected] wrote:
On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea ( [email protected](mailto:[email protected])) wrote:
I'm fine with running the functional tests only for the default skin. Note that although we support many browsers we run the functional tests only for Firefox.
Which is bad and not something we want ;)
We want to have test on all browser with as many skin as possible, including test of all combination of migrated version, and ... well, what we want is not really compatible with true reality :)
You’re mixing a bit everything in the same bag here :) There are 2 aspects:
- It’s specifically because we don’t have that much manpower that we must have our CI automatically test on various browsers/databases/OSes. This works well because the tests don’t need to be modified (in theory at least but mostly true) to be executed on various browsers/databases/OSes.
We could not say that test does not require some maintenance effort, and supporting multiple skin will increase it.
- for the various skins it’s different because you need to write several test frameworks (one per skin) so it’s indeed more work.
I have not the feeling we have the time for that, there are many other aspect that also require our attention.
So, taking into account our team, I do not think we can afford the support of more than one skin. In practice, I doubt those that see our new skin will want to stick with colibri for long, and those that will, will also stick with the old xwiki version they have. The support for both colibri and flamingo in parallel is also short term, so unless I over estimate the workload, I am +1 to only support one skin in functional tests, and to also put some reserves on the usage of colibri in upcoming versions.
It depends if we’re able to fulfil a large majority of use cases with only 1 skin or not.
I agree that if flavor came with different skins, we will need to support testing those. This does not seems to be the plan at the moment. We have taken the direction of supporting mobile using the same skin than desktop, so the exact opposite of having multiple skins for different purposes.
The fact that we’ve been able to support only one skin shows how complex writing an XWiki skin is. If you check on extensions.xwiki.org you’ll see we have very few skins, especially compared to other tools like wordpress, drupal, etc. We’ve been saying this for years but we haven’t made progress. Quite the opposite, over the years, we’ve made writing skins more complex (for example right now if you want to write a new skin you need to make sure to implement all Extension Points in your skin if you want your skin to behave properly with extensions!).
Comparing XWiki skins ans wordpress, drupal, etc... themes, is really unfair. In fact, those products are simply not fully "skinable", they have a single static skin with many options, and they have themes, which is not exactly the same. The Skin is a very powerful feature of XWiki, since it allows a complete flexibility of the XWiki UI when need arise.
Now one solution could be to have only 1 (or a few) base skins and improve Skin Themes so that users can do more and more in a Theme, thus allowing just to modify Themes and not a full skin. We started with Color Themes, added the ability to change the logo, and we are now moving to Themes with the ability to also change fonts. This is a step in the right direction I believe. In some future, we could maybe also support more Theme features. At some point we might need to fold default Panels layout (1 column, 2 columns, 3 columns, etc) and Panels selection into Themes too. Etc.
I completely agree that we should improve that aspect, and Flamingo will really help. It should be able to take bootstrap themes and apply them to XWiki. Changing and supporting multiple theme is only a matter of CSS, and should not affect the functionalities, so testing different themes is not really required IMO.
If we push this to a level where having 1 skin and using various Themes allow XWiki users to implement the UI they need for their project then we won’t need to support more than 1 skin.
Users can always customize the skin, this does not change anything for us.
But I fear this is just pushing the testing problem to Themes (but maybe to a lesser degree, depending on the freedom we give to Themes) ;) And if we wish to support several Themes we’ll need to test them…
We should not put in Themes what is a matter of Skin. The skin could have many options, like other product does. Supporting multiple skin is actually out of scope for the small theme we are.
Anyway, for 6.2 I agree we don’t have the time to make our UI test framework multiskin-enabled.
However, I believe we’ll need to refactor our test framework to allow supporting testing several UI. That will be useful when we need to do that but also for our users writing new skins and needing tests for them. I don’t think it’s that complicated to do.
I have nothing against, I just do not see it as a priority.
Thanks -Vincent
Thanks,
Thanks -Vincent
So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
Thanks, Marius
On Tue, Aug 19, 2014 at 7:12 PM, [email protected] wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each
Page
Object to have various implementations.
One example of difference is Hover vs Clicking for the top level
menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO
Hi, On Wed, Aug 20, 2014 at 4:55 PM, Marius Dumitru Florea <[email protected]> wrote:
I'm fine with running the functional tests only for the default skin.
+1
Note that although we support many browsers we run the functional tests only for Firefox. So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
+1 for this as well. Andreea has been doing manual tests on Colibri in parallel with Flamingo and will probably continue to do so for a few more versions. Thanks, Silvia
Thanks, Marius
On Tue, Aug 19, 2014 at 7:12 PM, [email protected] <[email protected]> wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
I'm fine with running the functional tests only for the default skin.
+1
Note that although we support many browsers we run the functional tests only for Firefox. So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
+1 for this as well. Andreea has been doing manual tests on Colibri in parallel with Flamingo and will probably continue to do so for a few more versions.
I agree with Silvia on this. So, I'm +1. Thanks, Andreea On Thu, Aug 21, 2014 at 11:33 AM, Silvia Rusu <[email protected]> wrote:
Hi,
On Wed, Aug 20, 2014 at 4:55 PM, Marius Dumitru Florea <[email protected]> wrote:
I'm fine with running the functional tests only for the default skin.
+1
Note that although we support many browsers we run the functional tests only for Firefox. So I think it's ok if we continue supporting the Colibri skin for a few versions even if we stop running the functional tests on it (because it's not the default skin any more).
+1 for this as well. Andreea has been doing manual tests on Colibri in parallel with Flamingo and will probably continue to do so for a few more versions.
Thanks, Silvia
Thanks, Marius
On Tue, Aug 19, 2014 at 7:12 PM, [email protected] <[email protected]>
wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
+1 for supporting only one skin in tests On Tue, Aug 19, 2014 at 6:12 PM, [email protected] <[email protected]> wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
+1 as well, since we already went in that direction with the tests and of obvious resource limitation. Thanks, Eduard On Wed, Aug 27, 2014 at 11:13 AM, Thomas Mortagne <[email protected]
wrote:
+1 for supporting only one skin in tests
On Tue, Aug 19, 2014 at 6:12 PM, [email protected] <[email protected]> wrote:
Hi devs,
ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
After his changes are made our framework won’t work anymore with Colibri.
The question is: * Is this ok? * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
Any opinion?
Thanks -Vincent _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
participants (9)
-
Andreea Popescu -
Denis Gervalle -
Eduard Moraru -
Manuel Smeria -
Marius Dumitru Florea -
Sergiu Dumitriu -
Silvia Rusu -
Thomas Mortagne -
vincent＠massol.net