Hi Jeremie, On Dec 1, 2011, at 4:51 PM, jerem wrote:

Hello xwiki community !

Since a long time, I work on a MailArchive application, and try to find time to post it to the community ... It's used in my team and is satisfying, but as I'm working on it currently I think it's time to unveil what to expect from it, and have your feedbacks before I (eventually) post a first version ...

What's implemented so far : - "eats" mails from specified mail server / account / folder - creates Mails and Topics pages and objects in wiki. Algorithm for topics tries to be "hijack-proof" ;) - Custom Livetable and timeline view for archive, table/threads/forum views for topics - persists main parts of mails (html content, main headers, attachments, attached mails, images, detects encrypted mails), with a clean display - configurable mailing-lists (independant of mail server) upon pattern matching, with auto-tagging of created pages - statistics page (top posters, …)

This is great! I had been discussing this need for such an application with Thomas Mortagne. It's really cool that you're working on it :) If/When it's solid enough we could try using it as a bridge with our xwiki.org mailing lists.

What I would like to implement before first release : - reply button and sheet - configurable auto-typing of mails (for example, Mail/Newsletter/Annoucement ...) (it already exists but it's hard-coded ...) - test it with other server than exchange (ie gmail at least) - configurable ldap integration : auto-creation of mail users from ldap (by matching emails). Already implemented but not inside the app itself. It's a very cool feature for enterprise level mailing-lists. Real users are also automatically assigned as mails/topics authors/creators in wiki pages, as well as mail dates. - remove everything that's hard-coded and dependent of our usage - put more simply, make it generic.

What I expect to add in next releases : - multi-mail-server configuration - improve admin pages - ratings - for now it dumps mails from a folder, meaning it does not act as a mail server by itself. This means that some common features are not available (validation of mails ...). For now it's more a simple dumped "clone" of a mailing-list, but it does not interact with users / mails.

Some open questions / remarks: - considering last point, I don't know how mailing-lists and mailing-lists archives do work (ie xwiki-users and robot, nabble …),

For nabble and the like they just register on the list and then archive all mails they receive on the registered email.

so I don't really know how I could implement such features inside xwiki, or even if it would be something useful for users ... For now the whole application is a XAR and I don't want to bring too much complexity to it.

IMO for it to be clean and maintainable on the long term you'll need a clean part in Java as components.

Anyway I'd like to keep both functionality. - for "reply" feature, I assume to use the global mail server configuration parameters of the wiki, and the email from connected xwiki user profile ("from").

Yep

To simplify I would ask user for his password when he clicks on "send" (and the form is valid), because depending on mail server configured, you might need to be authentified to send a mail. There could be an option to use or not the xwiki login for credentials. I don't know what 'd be the better for this ... Maybe I should add specific configuration for outgoing mail server in mail archive app ? Of course with a "real" mailing-list (with a robot), you don't have this kind of problem as I believe it acts as a mail server itself (kind of proxy).

Feel free to comment, manifest your enthusiasm (or your un-interest :) ) …

I'm eager to see a first version in action :) Thanks -Vincent

Jeremie

Thanks ! :) For Java I understand, even if I'm not eager to "verbosify" the cool groovy classes I use into painful java ... :) (I must say here how much I love groovy) Performances are quite good so far, really. At least for now it scales well with 2300 topics and 5500 mails. -- View this message in context: http://xwiki.475771.n2.nabble.com/Your-feedbacks-on-a-possible-MailArchive-a... Sent from the XWiki- Users mailing list archive at Nabble.com.

Of course Vincent, I completely agree with you ... It's more a joke hiding some kind of lazyness from my side, having to rewrite parts as components (and having to gain information about how to do that most of all ...) * How do you autocomplete in wiki pages I'm so used not having this feature when working in wiki pages that I don't even bother... * How do you have quick access to all your code scattered in several pages all visible from within your IDE My IDE is XWiki ... :/ Never found an easy way to manage wiki pages in link with IDE / SCM tools. * How do you write unit tests * How do you document and generate javadoc Of course there is none of that for now ... :( * How do you handle registration/unregistration of components Well, that's a question for me : for now I used the application manager to generate a Xar and set a version to it. I don't know if this kind of thing is/will be managed by extension manager ? Of course for now it's not a component at all. On another side working directly inside XWiki pages has some great advantage : you can test directly (no deployment or compilation phases). In fact it's too late to create the component before putting too much code in wiki pages, because almost all the app is already coded (in wiki pages ...) :) And what is left to code is mainly display ... But business logic is mainly located in one groovy class in one page (the mail loading, is the heart of the application). What's left in other pages are some utility methods (that need some refactoring), the ldap bridge, and the threading algorithm maybe. So I agree with you that it should not be a very big deal to refactor and transform this groovy class into an xwiki java component, and it has added-value. I'll add that in the pre-requisites for the first version. Indeed, I will ask you to be a little patient :) Side-question : is it possible / advisable to write this xwiki java component in groovy ? At the end it can be compiled into java bytecode, and as most groovy dependencies are already part of the xwiki distrib, it should work, what do you think ? Or do you recommend to write it in Java ? Jeremie -- View this message in context: http://xwiki.475771.n2.nabble.com/Your-feedbacks-on-a-possible-MailArchive-a... Sent from the XWiki- Users mailing list archive at Nabble.com.

Hi Jérémyie, I don't have much feedback regarding the code, but I'm definitely looking forward to the application. I've been thinking for a while that users should be able to create content easily in the wiki from an email and I'm very glad to see someone tackling the problem :-) Good luck, I'm looking forward to the result! Guillaume On Fri, Dec 2, 2011 at 12:12 PM, Thomas Mortagne <[email protected]>wrote:

On Fri, Dec 2, 2011 at 11:52 AM, jerem <[email protected]> wrote:

...

Of course Vincent, I completely agree with you ... It's more a joke hiding some kind of lazyness from my side, having to rewrite parts as components (and having to gain information about how to do that most of all ...)

* How do you autocomplete in wiki pages I'm so used not having this feature when working in wiki pages that I don't even bother... * How do you have quick access to all your code scattered in several pages all visible from within your IDE My IDE is XWiki ... :/ Never found an easy way to manage wiki pages in link with IDE / SCM tools. * How do you write unit tests * How do you document and generate javadoc Of course there is none of that for now ... :( * How do you handle registration/unregistration of components Well, that's a question for me : for now I used the application manager to generate a Xar and set a version to it. I don't know if this kind of thing is/will be managed by extension manager ? Of course for now it's not a component at all.

On another side working directly inside XWiki pages has some great advantage : you can test directly (no deployment or compilation phases).

In fact it's too late to create the component before putting too much code in wiki pages, because almost all the app is already coded (in wiki pages ...) :) And what is left to code is mainly display ... But business logic is mainly located in one groovy class in one page (the mail loading, is the heart of the application). What's left in other pages are some utility methods (that need some refactoring), the ldap bridge, and the threading algorithm maybe.

So I agree with you that it should not be a very big deal to refactor and transform this groovy class into an xwiki java component, and it has added-value. I'll add that in the pre-requisites for the first version. Indeed, I will ask you to be a little patient :)

Side-question : is it possible / advisable to write this xwiki java component in groovy ? At the end it can be compiled into java bytecode, and as most groovy dependencies are already part of the xwiki distrib, it should work, what do you think ?

Yes but there is an important drawback: there is no way for your groovy component to be registered automatically at startup, you need to external action to explicitly register it (either by hand or some cron which regularly check if your component is still registered). It could sound minor like that but it's a real pain to always have to remember to register you groovy component when you restard or worst when XWiki automatically restart because it was stuck.

A good example of Goovy component with regsitration page is the IRC bot. See https://github.com/xwiki/xwiki-platform/tree/master/xwiki-platform-core/xwik... .

This should be fixed by wiki component (see http://dev.xwiki.org/xwiki/bin/view/Design/WikiComponents) but the way XWiki database initialization is done make it difficult to cleanly find and intialize the wiki component at startup, need a bit of refactoring. An example of specialized wiki components is wiki macro but the way theses are initialized is very crappy and we are trying to avoid doing the same thing for components in general.

...

Or do you recommend to write it in Java ?

Jeremie

-- View this message in context: http://xwiki.475771.n2.nabble.com/Your-feedbacks-on-a-possible-MailArchive-a... Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

-- Thomas Mortagne _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

On Fri, Dec 2, 2011 at 12:43 PM, jerem <[email protected]> wrote:

Thanks for pointing this init problem Thomas ... But in my case the app relies on a scheduled job (in fact, for now it's in linux crontab but I'll move it to a scheduled job of course), so maybe it would be easy to workaround : the job could check if component is registered and if not register it, before loading new mails.

Yep.

-- View this message in context: http://xwiki.475771.n2.nabble.com/Your-feedbacks-on-a-possible-MailArchive-a... Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users

-- Thomas Mortagne

Hi Jeremie, On Dec 2, 2011, at 2:06 PM, jerem wrote:

By the way, in the Component tutorial : http://platform.xwiki.org/xwiki/bin/view/DevGuide/WritingComponents

... there is nothing clear about this needed registration step ... As it's described, deploying a component in order to use it seems as easy as copying the .jar somewhere. Maybe some more details could be added in the tutorial ?

There's this line: "And now, the components.txt file, in which component implementations present in this jar are specified for the ComponentManager to register them." This combined with Annotations makes registration.

In fact in the tutorial it seems we only need to create a component.txt file and everything is auto-magically registered by the ComponentManager …

Yes that's what is called registration (with annotations) :) There are some more details here (referenced from the tutorial: "You should start by reading the Reference document on XWiki Components."): http://extensions.xwiki.org/xwiki/bin/view/Extension/Component+Module So for registration 2 solutions are possible: 1) automatically with annotations and components.txt 2) manually using the ComponentManager Thanks -Vincent

Some news about this ... I'm slowly working on it. I'm implementing the admin page(s), some operations page (purge, split, merge ...), and just started writing the component (but it's really the beginning). For configuration parameters, I wonder if I'd better directly access my AdminClass to grab the parameters where needed (from component, from Admin pages and from view pages and sheets), or if I should hide this access inside the component API ? -- View this message in context: http://xwiki.475771.n2.nabble.com/Your-feedbacks-on-a-possible-MailArchive-a... Sent from the XWiki- Users mailing list archive at Nabble.com.