October 2009 - xwiki-devs@xwiki.org

[xwiki-devs] [Proposal] Wiki macros improvement ideas
by Vincent Massol 09 Dec '09

09 Dec '09

Hi, Wiki macros currently require programming rights and thus are limited and in addition cannot be created in subwikis by the majority of users. Since I've now almost finished implementing component isolation I'd like to agree on what we want for wiki macros. Proposal: * Modify the wiki macro class to add a "Visibility" field with 2 values: - visible only in the current wiki - visible to everyone * Visible to everyone will require admin rights * Make the macro available to a user only if that user has view rights to the wiki macro page. This is to allow making a wiki macro private (when testing for example). It also allows making a macro visible only to a given group of users. Note: I considered having a "visible only to the current user" value for the Visibility field but I don't think it means much since if someone else edit the page and saves it, then suddenly the macro becomes visible only to that person... WDYT? Thanks -Vincent

2 5

[xwiki-devs] [Discussion] Annotations improvements (wider annotations target) and roadmap update
by Anca Luca 30 Nov '09

30 Nov '09

Hi devs, following a discussion with Fabio about the second desired feature for the annotations, namely the ability to add annotations on any document, no matter how its content is generated, we came up with the solution described at http://dev.xwiki.org/xwiki/bin/view/Design/AnnotationFeature#HSolution1stor… , the main idea being that annotations would be defined by their selected text and a context (as opposed to offsets) and would be identified to be rendered in a document on a serialization of the transformed XDOM of the document, this way taking into account any macro rendering, document inclusion, etc. WDYT about this solution? Also, because the implementation of this, though relatively localized, comes together with refactor and cleanup of the annotations module (update everything so that annotations don't store and use offsets anymore, remove classes & functions which are not needed in this simplified process), I propose to include this improvement in version 1.0 of the annotations module (so that we don't cleanup and release what we know for sure we'll delete) and push the 1.0 version further to mid to end December. here's my +1 for this, WDYT? Happy coding, Anca

7 16

[xwiki-devs] [Proposal] Component isolation implementations
by Vincent Massol 27 Nov '09

27 Nov '09

Hi, We have the need to isolate groups of components. For ex a wiki macro created in a subwiki should only be visible in that subwiki by default. Here's an implementation proposal that I'm planning to implement: * There's a Root Component Manager (the current CM) * There are 3 components which implement the ComponentManager role and with 3 hints: "wiki", "user" and "all". There's a CompositeComponentManager class that allows chaining CM and the "all" CM chains the "default" (root CM), "wiki" CM and "user" CM. This works the same as with the configuration module. * Other components can have CMs injected as they want (if not specified then it's the default, etc). For ex: @Requirement("all") private ComponentManager cm * Creation process. As for now the user creates the root CM and then the annotation loader will create the descriptors for the other CMs and register them against the root CM. They'll get instantiated once (singleton) the first time they're looked up. * In order to register a component for, say, a given "enterprise" wiki, we need to add a new property to the ComponentDescriptor: get/ setAdditionalData(Object data). For example: wikiCM.registerComponent(CD mycd) where cd.setAdditionalData("enterprise"). * Last, Guice uses Modules to isolate component definitions so it should be possible and relatively easy to port the implementation to Guice (even though Guice uses static Modules we can make them dynamic). WDYT? Thanks -Vincent

5 17

[xwiki-devs] Reduce heap exhaustion during export operation (and partly import for now)
by Denis Gervalle 26 Nov '09

26 Nov '09

Hi, As you knows, Import/Export operation on large XAR file cause problem. We felt on an even worse situation, where a single XWiki document is not properly exported due to heap exhaustion during the built of its XML DOM or due to large attachments. Having a look at the source, I have notice that many optimizations on the way the export is produce could be quite easily introduced. Currently, XWiki stored the exported document several times in memory during the operation, which impact performance negativly and is uselessly heavy on memory usage, even for reasonable documents. Therefore, I have started a large patch to avoid these caveats, here is the strategies I have followed: 1) The current implementation mostly build a DOM in memory for immediately serialized it into a stream. So I have remove the intermediate DOM and provide direct streaming of Element content by: 1.1) extending org.dom4j.XMLWriter to allow direct streaming of Element content into the output stream, as is, or Base64 encoded. Accessorily, my extension also ensure proper pairing of open/close tag. 1.2) writing a minimal DOMXMLWriter which extends my XMLWriter and could be used with the same toXML() code to build a DOMDocument to provide the toXMLDocument() methods to support the older implementation unchanged if ever needed. 1.3) using the above, minimal change to the current XML code was required 1.3.1) replacing element.add(Element) by either writer.writeElement) or writer.writeOpen(Element) 1.3.2) for large content, use my extensions, either writer.write(Element, InputStream) or writer.writeBase64(Element, InputStream) which use the InputStream for the element content 2) The current implementation for binary data such as attachments and export zip file is mostly based on in memory byte[] passing from function to function while these data initially came from a request.getInputStream() or are written to a response.getOutputStream (). So I have change these to passover the stream instead of the data: 2.1) using IOUtils.copy when required 2.2) using org.apache.commons.codec.binary.Base64OutputStream for base64 encoding when required 2.3) using an extension of ZipInputStream to cope with unexpected close() 2.4) avoid buffer duplication in favor of stream filters 3) Since most oftently used large data came from the database through an attachment content, it would be nice to have these attachment streamed from the database when they are too large. However, I feel that it is still too early to convert our binary into a blob, mainly because HSQLDB and MySQL still does not really support blob, just an emulation. These are also used to be cached in the document cache, and this will require improvement to support blob. However I propose to take the occasion to go in the direction of the blob by: 3.1) deprecating setContent(byte[]) and Byte[] getContent() in favor of newly created setContent(InputStream, int), InputStream getContentInputStream() and getSize() 3.2) Begin to use these new function as much as possible as 2) implied 3.3) this also open the ability to store attachment in another repository that support better the streaming aspect (ie: a filesystem) I am currently testing the above changes onto our client project, and I expect to provide a patch really soon. It will require an upgrade of org.apache.commons.codec to version 1.4, to have access to Base64OutputStream. I feel it will be a first step into the right direction, further improvement should be to: - import XML using a SAXParser without building a DOM in memory - manage JRCS Archive better, the way they are built and store raise the same issue then attachment - manage the recycle bin better for the same reason - improve caching to avoid caching very large stuffs WDYT ? Denis Gervalle -- SOFTEC sa http://www.softec.st

4 5

[xwiki-devs] [VOTE] What to do with RawBlock in plain text renderer ?
by Thomas Mortagne 24 Nov '09

24 Nov '09

Hi devs, We need to decide what to do with html macro content (or any other RawBlock content) in plain text renderer. For me the rule of plain text renderer should be to print anything we are able to print which mean printing any pure text content and skip styles for which obviously we don't have syntax. RawBlock content is pure text so we should print it. Note that first version of plain text renderer was printing html content and it was lost when introducing RawBlock. Here is my +1 for printing RawBlock content in plain text renderer. WDYT ? -- Thomas Mortagne

3 4

[xwiki-devs] Feed Plugin: Simplify the Feed Plugin Api Interface
by Andreas Schaefer 20 Nov '09

20 Nov '09

Hi Based on this JIRA report: http://jira.xwiki.org/jira/browse/XWIKI-4430 it would be necessary to create a new set of methods just to be able to pass an additional parameter to the calls. Because this makes it harder and harder to deal with Plugin I would suggest to use a Parameter Container which could contain: - Feed Type (Blog, Web, etc) - Type (Query, List or Feed ) - List or - Query, Count and Start or - Feed - Metadata - Object Class Name - Output Type (RSS, Atom etc) Then we would only need 2 major methods: - SyncdFeed getFeed( FeedParams ) - String getFeedOutput( FeedParams ) This is what I came up so far (without the Feed Type): /** * @see #getBlogFeed(String, int, int, Map) * @see #getFeedOutput(SyndFeed, String) */ public String getBlogFeedOutput( FeedParams params) { SyndFeed feed = getBlogFeed( params ); String ret = getFeedOutput( feed, params.getOutputType() ); return ret; } public SyndFeed getBlogFeed( FeedParams params) { Map<String, Object> myMetaData = Collections.emptyMap(); if( params.getMetaData() != null ) { myMetaData = params.getMetaData(); } Map<String, Object> blogMappings = null; if( params.getObjectClassName() == null ) { blogMappings = BLOG_FIELDS_MAPPING; } else { blogMappings = new HashMap<String, Object>(); blogMappings.put(SyndEntryDocumentSource.FIELD_TITLE, params.getObjectClassName() + "_title"); blogMappings.put (SyndEntryDocumentSource.FIELD_DESCRIPTION, params.getObjectClassName () + "_content"); blogMappings.put (SyndEntryDocumentSource.FIELD_CATEGORIES, params.getObjectClassName() + "_category"); blogMappings.put(SyndEntryDocumentSource.CONTENT_LENGTH, new Integer(400)); } SyndFeed blogFeed = null; if( params.getType() == FeedParams.Type.QUERY ) { String query = params.getQuery(); if (query == null) { XWikiRequest request = getXWikiContext().getRequest(); String category = request.getParameter("category"); if (category == null || category.equals("")) { query = ", BaseObject as obj where obj.name=doc.fullName and obj.className='" + BLOG_POST_CLASS_NAME + "' and obj.name<>'" + BLOG_POST_TEMPLATE_NAME + "' order by doc.creationDate desc"; } else { query = ", BaseObject as obj, DBStringListProperty as prop join prop.list list where obj.name=doc.fullName and obj.className='" + BLOG_POST_CLASS_NAME + "' and obj.name<>'" + BLOG_POST_TEMPLATE_NAME + "' and obj.id=prop.id.id and prop.id.name='category' and list = '" + category + "' order by doc.creationDate desc"; } } blogFeed = getFeed( query, params.getCount(), params.getStart(), getSyndEntrySource(SyndEntryDocumentSource.class.getName(), blogMappings), Collections.EMPTY_MAP, fillBlogFeedMetadata(myMetaData) ); } else { blogFeed = getFeed( params.getEntries(), getSyndEntrySource (SyndEntryDocumentSource.class.getName(), blogMappings), Collections.EMPTY_MAP, fillBlogFeedMetadata(myMetaData) // params.getEntries(), getSyndEntrySource (SyndEntryDocumentSource.class.getName(), blogMappings), myMetaData, fillBlogFeedMetadata(myMetaData) ); } if (blogFeed != null) { blogFeed.setImage(getDefaultFeedImage()); } return blogFeed; } public static class FeedParams { public enum Type { QUERY, LIST } private static final String DEFAULT_OUTPUT_TYPE = "rss_2.0"; private static final int DEFAULT_COUNT = 10; private static final int DEFAULT_START = 0; private Type type; private List<Object> entries; private String query; private int count = DEFAULT_COUNT; private int start = DEFAULT_START; private Map<String, Object> metaData; private String outputType = DEFAULT_OUTPUT_TYPE; private String objectClassName; public FeedParams( List<Object> entries, String objectClassName, Map<String, Object> metaData, String outputType ) { this.type = Type.LIST; this.entries = entries; this.metaData = metaData; this.outputType = outputType == null ? DEFAULT_OUTPUT_TYPE : outputType; this.objectClassName = objectClassName; } public FeedParams( String query, Integer count, Integer start, String objectClassName, Map<String, Object> metaData, String outputType ) { this.type = Type.QUERY; this.count = count == null ? DEFAULT_COUNT : count; this.start = start == null ? DEFAULT_START: start; this.metaData = metaData; this.outputType = outputType == null ? DEFAULT_OUTPUT_TYPE : outputType; this.objectClassName = objectClassName; } public Type getType() { return type; } public String getQuery() { return query; } public int getCount() { return count; } public int getStart() { return start; } public List<Object> getEntries() { return entries; } public Map<String, Object> getMetaData() { return metaData; } public String getOutputType() { return outputType; } public String getObjectClassName() { return objectClassName; } } What do you think? -Andy

3 2

[xwiki-devs] Proposal to redesign code.xwiki.org
by Ludovic Dubost 13 Nov '09

13 Nov '09

Hi, I started working on a proposal to redesign code.xwiki.org http://dev.xwiki.org/xwiki/bin/view/Drafts/XWikiExtensionRepositoryRedesign Please comment and/or modify this document. The main thing would be to agree that there should be one "unique" repository for anything related to XWiki. If we agree on that, I would like to start a vote for the name. The name could be I think any combination of a "prefix" or "suffix" or a magical unrelated name. * Possible Prefix o Add-On o Extension o App o Store o Modules o Plugins * Possible Suffix o Directory o Repository o Store o Catalog I believe we should make this not feel "technical" but understandable by non technical users. Any other ideas of prefixes or suffixes or other names before we start a vote ? Ludovic -- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost

8 8

[xwiki-devs] [Proposal][UX] User Profile and Preferences
by Ecaterina Valica 13 Nov '09

13 Nov '09

Hi devs, With the new ActionMenu Proposal<http://incubator.myxwiki.org/xwiki/bin/view/Improvements/ActionMenuProposal>I've made a separation between: - Profile - things related to the user - http://incubator.myxwiki.org/xwiki/bin/view/Improvements/UserProfileProposal - Preferences - things related to user's account - http://incubator.myxwiki.org/xwiki/bin/view/Improvements/UserPreferencesPro… Please give feedback about: 1. the utility of the functionality added: Groups, Recent Activity 2. Default Fields: 1. is there a way to have dynamic fields in user class? 2. what fields should be default in the static case? 1. fields web oriented 2. fields business oriented 3. which are those fields? Thanks, Caty

6 22

[xwiki-devs] [VOTE] Following Web Content Accessibility Guidelines (WCAG)
by Vincent Massol 10 Nov '09

10 Nov '09

Hi devs, We've always tried to follow standards in the XWiki project. I'd like to propose that we start adhering to the WCAG ones to make XWiki even more accessible. The reason I'm proposing this is because XWiki SAS has been contacted by some Dutch organization promoting the usage of standards in Netherlands and they're interested in using XWiki for their site. As such they need that the wiki they use also follows the guidelines... They've found XWiki since it was one of the wikis out there that follows most closely the guidelines. This is our opportunity to meet the WCAG guidelines and go even beyond (since the Dutch guidelines go slightly further I believe, see http://www.webrichtlijnen.nl/english/guidelines/) . We would be the first wiki to meet this! If we agree we'll need to find some ways to automate the execution of the verifications. Right now we can test pages using this validator: http://www.webrichtlijnen.nl/english/test/ I've started verifying this page: http://playground.xwiki.org/xwiki/bin/view/Main/ Result: http://www.webrichtlijnen.nl/english/test/report/74721/179372/ Some examples of failing stuff: 1) Style attribute forbidden. Everything from CSS. Ex from home page: <div id="globallinks"> <form action="/xwiki/bin/view/Main/WebSearch" style="margin: 0; padding: 0;"> <div id="globalsearch"> <input id="globalsearchinput" type="text" name="text" value="search..." size="15" class="withTip"/><input class="button" value="Go" type="image" src="/xwiki/resources/icons/xwiki/search.png"/> </div> </ form> </div> ... <div class="hidden" id="spSpaceCreateForm"><form action="/xwiki/bin/ view/Main/?xpage=create" method="post"> <div> <input name="tocreate" type="hidden" value="space"></input> <input class="panelinput" name="title" onblur="if(this.value=='') this.value='Space name'" onfocus="if(this.value=='Space name') this.value=''" style="margin:auto;" type="text" value="Space name"></input> <input type="submit" value="Create"></input> </div> </form> </div> ... <p style="font-size:0.75em;padding-left:8px;"><a href="/xwiki/bin/ inline/Panels/QuickLinks">(Edit this panel)</a></p> 2) Need to run CSS Level 2.1 validation. Lots of error. For ex on home page: http://bit.ly/4x9YlM 3) Apparently we made a mistake by choosing to have 2 H1 (one for title and for headers). "The document hierarchy contains more that one h1 tags .". Before going further I'd like to ensure that we all agree to follow these guidelines and put in our best practices (and ensure we verify with some automated tests later on). Here's my +1 Thanks -Vincent

9 20

[xwiki-devs] Lucene search as default search?
by Vincent Massol 10 Nov '09

10 Nov '09

Hi, Should we make the lucene search the default search or not? In the default XAR the Lucene search is marked as experimental. This has been the case for a long time now. I think we need to do something about this. Do you think the Lucene search is working well enough to be set as the default? WDYT? Thanks -Vincent

5 6