i.e. it should not find its way in WEB-INF/lib. extensions in WEB-INF (for example inside WEB-INF/extensions) and have XWiki install them when it starts the first time (as a DW step for example). as core extensions, and we define a mechanism to bundle third-party extensions inside an XWiki WAR. we have access to the permanent directory. But we could also leave it as part of a DW step. Note that an alternative would be to map WEB-INF/extensions as an additional extension repository. However, an important issue with this would be that a WAR file should be considered as read only by servlet containers (they can do whatever they want with it, expand it in some custom directory, etc) and thus it would not be possible to remove/update extensions. This is why I proposed that WEB-INF/extensions would contain uninstalled extensions that XWiki would need to install. Thanks -Vincent mariusdumitru.florea(a)xwiki.com> wrote: the contrib, didn't ends enterprise-8.2.1/xwiki-enterprise-ui/xwiki-enterprise-ui-common/pom.xml# L168 on enterprise-8.2.1/xwiki-enterprise-web/pom.xml#L837 generated would UI enterprise-8.2.1/xwiki-enterprise-web/pom.xml#L837 enterprise-8.2.1/xwiki-enterprise-data/pom.xml#L145 and _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

On 22 Aug 2016, at 11:58, Vincent Massol <vincent(a)massol.net> wrote: Thanks Guillaume. We mentioned this but it doesn’t work for the WAR packaging.

Thanks -Vincent > > 2016-08-16 21:03 GMT+02:00 Vincent Massol &lt;vincent(a)massol.net&gt;et>: > >> >> >>> On 16 Aug 2016, at 19:18, Vincent Massol &lt;vincent(a)massol.net&gt; wrote: >>> >>> Interesting problem :) >>> >>> So the CKEditor extension should not be considered a core extension, >> i.e. it should not find its way in WEB-INF/lib. >>> >>> One idea to achieve this is to allow bundling some uninstalled >> extensions in WEB-INF (for example inside WEB-INF/extensions) and have >> XWiki install them when it starts the first time (as a DW step for example). >>> >>> Basically they’re considered as bundled third-party extensions and not >> as core extensions, and we define a mechanism to bundle third-party >> extensions inside an XWiki WAR. >>> >>> For the Jetty/HSQLDB distribution, we could install them directly since >> we have access to the permanent directory. But we could also leave it as >> part of a DW step. >>> >>> WDYT? >> >> Note that an alternative would be to map WEB-INF/extensions as an >> additional extension repository. However, an important issue with this >> would be that a WAR file should be considered as read only by servlet >> containers (they can do whatever they want with it, expand it in some >> custom directory, etc) and thus it would not be possible to remove/update >> extensions. This is why I proposed that WEB-INF/extensions would contain >> uninstalled extensions that XWiki would need to install. >> >> Thanks >> -Vincent >> >>> >>> Thanks >>> -Vincent >>> >>>> On 01 Aug 2016, at 14:45, Marius Dumitru Florea < >> mariusdumitru.florea(a)xwiki.com&gt; wrote: >>>> >>>> Hi devs, >>>> >>>> Thomas raised this issue http://jira.xwiki.org/browse/XE-1570 . One of >> the >>>> reasons we decided to keep the CKEditor Integration extension in >> contrib, >>>> with its own release cycle, was to allow the users to upgrade it without >>>> the need to upgrade the entire XWiki distribution. >>>> >>>> There wouldn't be any problem if the CKEditor Integration extension >> didn't >>>> had any JAR dependencies. But it depends on the CKEditor WebJar which >> ends >>>> up in the XWiki WAR and thus is considered a core extension, and core >>>> extensions cannot be upgraded. >>>> >>>> Here's what happens: >>>> * xwiki-enterprise-ui-common depends on ckeditor-ui ( >>>> https://github.com/xwiki/xwiki-enterprise/blob/xwiki- >> enterprise-8.2.1/xwiki-enterprise-ui/xwiki-enterprise-ui-common/pom.xml# >> L168 >>>> ) >>>> * both xwiki-enterprise-ui-mainwiki and xwiki-enterprise-ui-wiki depend >> on >>>> xwiki-enterprise-ui-common >>>> * xwiki-enterprise-web depends on both xwiki-enterprise-ui-mainwiki and >>>> xwiki-enterprise-ui-wiki ( >>>> https://github.com/xwiki/xwiki-enterprise/blob/xwiki- >> enterprise-8.2.1/xwiki-enterprise-web/pom.xml#L837 >>>> ) in order to "transitively include all JAR dependencies in the >> generated >>>> WAR" >>>> >>>> So the ckeditor-webjar ends up in the XE WAR, thus it becomes a core >>>> extension. In order to fix this Thomas has proposed two solutions: >>>> >>>> (1) Build 2 XE WARs: one that bundles the transitive JAR dependencies of >>>> the UI (what we have currently) and one that doesn't bundle them. We >> would >>>> offer only the later for download on xwiki.org, >>>> knowing that the transitive JAR dependencies will be installed when the >> UI >>>> is installed (with the Distribution Wizard for instance). The first WAR >>>> would be used only for building the Jetty+HSQLDB distribution. >>>> >>>> The downside of this solution is: >>>> * we don't fix the problem for the Jetty+HSQLDB distribution >>>> * the ui-all XAR would work only with the "all" WAR (the first one that >>>> bundles the XAR dependencies) >>>> >>>> (2) Exclude the JARs we don't want as core extensions from >>>> https://github.com/xwiki/xwiki-enterprise/blob/xwiki- >> enterprise-8.2.1/xwiki-enterprise-web/pom.xml#L837 >>>> , one by one. This would fix the WAR-based installations but not the >>>> Jetty+HSQLDB distribution which uses the Import Mojo ( >>>> https://github.com/xwiki/xwiki-enterprise/blob/xwiki- >> enterprise-8.2.1/xwiki-enterprise-data/pom.xml#L145 >>>> ) to generate the distribution data folder and thus won't get the >>>> ckeditor-webjar. For this we would need to introduce a new Install Mojo >> and >>>> explicitly install the JAR dependecies we want (ckeditor-webjar in this >>>> case). >>>> >>>> The problem with this solution is the maintenance cost. We'll forget for >>>> sure to exclude/install the JAR in XE's pom whenever we add a new JAR >>>> dependency to one of the contrib extensions that are bundled in XE >>>> (CKEditor and Tour for the moment). >>>> >>>> Do you have any other ideas? >>>> >>>> Solution (1) seems the best so far. >>>> >>>> Thanks, >>>> Marius

On 29 Aug 2016, at 10:25, Guillaume Delhumeau &lt;guillaume.delhumeau(a)xwiki.com&gt; wrote: 2016-08-22 12:00 GMT+02:00 Vincent Massol &lt;vincent(a)massol.net&gt;et>: I don't see where you explain "why" you would prefer an other solution. Actually, I don't see any drawback with my current implementation: - for Jetty/HSQLDB, my plugin installs the extension correctly, both in the "data/extensions" directory and in the database. - for the WAR packaging, Distribution Wizard is already started during the first execution.

Bundling the dependencies in some directory inside the WAR has not much sense here, since we already don't do it for XAR extensions. Except if we want to be able to run DW without an Internet connection, and in this case we should also include all needed XARs. But again, it does not make much sense: we won't bundle all possible flavors. What we only needs is to put my plugin in contrib or in platform, but for doing it properly we need to do some refactoring with Thomas. Thanks, > > Would be great to have some opinions on the few proposals I made. I think > we need to move on quite quickly on this topic. It’s going to be a problem > for our users quite soon now. > > Thanks > -Vincent > >> >> Thanks >> -Vincent >> >>> >>> 2016-08-16 21:03 GMT+02:00 Vincent Massol &lt;vincent(a)massol.net&gt;et>: >>> >>>> >>>> >>>>> On 16 Aug 2016, at 19:18, Vincent Massol &lt;vincent(a)massol.net&gt; wrote: >>>>> >>>>> Interesting problem :) >>>>> >>>>> So the CKEditor extension should not be considered a core extension, >>>> i.e. it should not find its way in WEB-INF/lib. >>>>> >>>>> One idea to achieve this is to allow bundling some uninstalled >>>> extensions in WEB-INF (for example inside WEB-INF/extensions) and have >>>> XWiki install them when it starts the first time (as a DW step for > example). >>>>> >>>>> Basically they’re considered as bundled third-party extensions and not >>>> as core extensions, and we define a mechanism to bundle third-party >>>> extensions inside an XWiki WAR. >>>>> >>>>> For the Jetty/HSQLDB distribution, we could install them directly > since >>>> we have access to the permanent directory. But we could also leave it > as >>>> part of a DW step. >>>>> >>>>> WDYT? >>>> >>>> Note that an alternative would be to map WEB-INF/extensions as an >>>> additional extension repository. However, an important issue with this >>>> would be that a WAR file should be considered as read only by servlet >>>> containers (they can do whatever they want with it, expand it in some >>>> custom directory, etc) and thus it would not be possible to > remove/update >>>> extensions. This is why I proposed that WEB-INF/extensions would > contain >>>> uninstalled extensions that XWiki would need to install. >>>> >>>> Thanks >>>> -Vincent >>>> >>>>> >>>>> Thanks >>>>> -Vincent >>>>> >>>>>> On 01 Aug 2016, at 14:45, Marius Dumitru Florea < >>>> mariusdumitru.florea(a)xwiki.com&gt; wrote: >>>>>> >>>>>> Hi devs, >>>>>> >>>>>> Thomas raised this issue http://jira.xwiki.org/browse/XE-1570 . One > of >>>> the >>>>>> reasons we decided to keep the CKEditor Integration extension in >>>> contrib, >>>>>> with its own release cycle, was to allow the users to upgrade it > without >>>>>> the need to upgrade the entire XWiki distribution. >>>>>> >>>>>> There wouldn't be any problem if the CKEditor Integration extension >>>> didn't >>>>>> had any JAR dependencies. But it depends on the CKEditor WebJar which >>>> ends >>>>>> up in the XWiki WAR and thus is considered a core extension, and core >>>>>> extensions cannot be upgraded. >>>>>> >>>>>> Here's what happens: >>>>>> * xwiki-enterprise-ui-common depends on ckeditor-ui ( >>>>>> https://github.com/xwiki/xwiki-enterprise/blob/xwiki- >>>> enterprise-8.2.1/xwiki-enterprise-ui/xwiki- > enterprise-ui-common/pom.xml# >>>> L168 >>>>>> ) >>>>>> * both xwiki-enterprise-ui-mainwiki and xwiki-enterprise-ui-wiki > depend >>>> on >>>>>> xwiki-enterprise-ui-common >>>>>> * xwiki-enterprise-web depends on both xwiki-enterprise-ui-mainwiki > and >>>>>> xwiki-enterprise-ui-wiki ( >>>>>> https://github.com/xwiki/xwiki-enterprise/blob/xwiki- >>>> enterprise-8.2.1/xwiki-enterprise-web/pom.xml#L837 >>>>>> ) in order to "transitively include all JAR dependencies in the >>>> generated >>>>>> WAR" >>>>>> >>>>>> So the ckeditor-webjar ends up in the XE WAR, thus it becomes a core >>>>>> extension. In order to fix this Thomas has proposed two solutions: >>>>>> >>>>>> (1) Build 2 XE WARs: one that bundles the transitive JAR > dependencies of >>>>>> the UI (what we have currently) and one that doesn't bundle them. We >>>> would >>>>>> offer only the later for download on xwiki.org, >>>>>> knowing that the transitive JAR dependencies will be installed when > the >>>> UI >>>>>> is installed (with the Distribution Wizard for instance). The first > WAR >>>>>> would be used only for building the Jetty+HSQLDB distribution. >>>>>> >>>>>> The downside of this solution is: >>>>>> * we don't fix the problem for the Jetty+HSQLDB distribution >>>>>> * the ui-all XAR would work only with the "all" WAR (the first one > that >>>>>> bundles the XAR dependencies) >>>>>> >>>>>> (2) Exclude the JARs we don't want as core extensions from >>>>>> https://github.com/xwiki/xwiki-enterprise/blob/xwiki- >>>> enterprise-8.2.1/xwiki-enterprise-web/pom.xml#L837 >>>>>> , one by one. This would fix the WAR-based installations but not the >>>>>> Jetty+HSQLDB distribution which uses the Import Mojo ( >>>>>> https://github.com/xwiki/xwiki-enterprise/blob/xwiki- >>>> enterprise-8.2.1/xwiki-enterprise-data/pom.xml#L145 >>>>>> ) to generate the distribution data folder and thus won't get the >>>>>> ckeditor-webjar. For this we would need to introduce a new Install > Mojo >>>> and >>>>>> explicitly install the JAR dependecies we want (ckeditor-webjar in > this >>>>>> case). >>>>>> >>>>>> The problem with this solution is the maintenance cost. We'll forget > for >>>>>> sure to exclude/install the JAR in XE's pom whenever we add a new JAR >>>>>> dependency to one of the contrib extensions that are bundled in XE >>>>>> (CKEditor and Tour for the moment). >>>>>> >>>>>> Do you have any other ideas? >>>>>> >>>>>> Solution (1) seems the best so far. >>>>>> >>>>>> Thanks, >>>>>> Marius

On 29 Aug 2016, at 10:46, Thomas Mortagne &lt;thomas.mortagne(a)xwiki.com&gt; wrote: I agree with Guillaume here. The subject of the discussion started by Marius is no to package anything in the WAR, the subject is to have CKEditor fully being seen as an extension like XAR extensions are so that it can be upgraded. Like Guillaume said XAR extensions are not packaged in the WAR so there is no reason to decide that it's mandatory for JARs extensions. Marius already proposed an easy solution for WAR based installs (1: remove the dependencies on XARs in the WAR project which essentially means removing a hack). Guillaume's plugins fix the issue for jetty/hsqldb so it seems to me we have everything we need. The priority IMO is to fix the issue for WAR installs (A.K.A. "real installs") quickly and then finish standardize a bit more what Guillaume started. On Mon, Aug 29, 2016 at 10:31 AM, Vincent Massol &lt;vincent(a)massol.net&gt; wrote: -- Thomas Mortagne _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

On Mon, Aug 29, 2016 at 11:56 AM, Vincent Massol &lt;vincent(a)massol.net&gt; wrote: we were talking at cross-purpose. So here’s my summary on this topic: Thomas: XE pom that bundles the XAR dependencies) for sure to exclude/install the JAR in XE's pom whenever we add a new JAR (CKEditor and Tour for the moment). best and asked for other ideas. No It's not what I answered. I answered that the RESULT of 2 is better. But as Marius said it's too much of a pain to maintain which is why I said we should do 1 for now and see if we can find a better solution for the jetty/hsqldb (which is what Guillaume proposed). You should differentiated what we distribute and how we end up with it. Marius is only detailed the "how to" here but in both solutions he proposed you get the exact same thing for WAR installs: a WAR without the jars we want to be seen as installed extensions instead of core extensions that will come as dependencies of the flavor. In your case you assumed that we want to have everything in the WAR and all your proposals are following this assumption (and that's also the reason why you rejected Guillaume's idea) but I really don't think it's a requirement here. solution was perfect according to Marius), I proposed Solution 3. extensions as non-core extensions (based on a WEB-INF/extensions mechanism) Solution 1 at the time hence our misunderstanding!), providing an answer for problem 1) A). should do now and that problem 1) B) is not a blocker? Fo we have a solution for it? better solution? In this case is 2) better? Is solution 3) needed (I agree it’s better to not implement it if we can do without). wrote: wrote: guillaume.delhumeau(a)xwiki.com&gt; wrote: wrote: packager instead answer for solution. in the during the in the WAR since the WAR doesn’t package the data dir… So I don’t see how your solution helps at all. extensions in our packagings (and distribute those packagings). The main packaging is the WAR one and it needs to be supported. much if we case much for think problem wrote: extension, have and not third-party leave it this servlet some . One extension which and core xwiki-enterprise-ui-mainwiki core solutions: them. We when first one not the the Install in forget new JAR XE -- Thomas Mortagne _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

guillaume.delhumeau(a)xwiki.com&gt; wrote: You mean (4) (since mine was the (3) one ;)). of I don’t know how many users use that but that’s probably low indeed. My only problem with this is that I think this is how users who are offline can install XWiki easily, no? I think we do need to offer a way to install XWiki offline. It doesn’t have to be through the ui-all XAR though. What do you suggest?

IMO, the XWiki Core Dev Team will only propose one real flavor (the replacement of XE - I call it “Knowledge Base” but I don’t remember if we’ve finished our brainstorming on this topic, probably not) so it shouldn’t be an issue. Since Jetty/HSQLDB is for trial we shouldn’t have a package for the base flavor (which IMO is only for OEM and thus don’t require a Jetty/HSQLDB packaging). Thanks -Vincent [snip] _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs