- xwiki-devs@xwiki.org - lists.xwiki.org

[xwiki-devs] [Proposal] Adding sibling information to rendering Block API + get/setNextSibling()
by Vincent Massol 25 Oct '10

25 Oct '10

Hi, I'm adding sibling information to the Block API in the rendering module. There are 3 reasons for this: 1) It'll make a lot of existing method implementations much faster since right now in order to insert or find blocks we have to traverse the list/tree. 2) There's a bug in the current implementation for SpaceBlock since it's immutable and we have a single instance of it. Thus when there are several SpaceBlock in a list doing a list.indexOf(SpaceBlock) will always return the first space block even if you're looking for a subsequent space block. 3) I'm implementing a generic emoticon Transformation and I need to get the sibling of a block. Just shout if you think there's a better way of doing it. I'm implementing it now. Thanks -Vincent

3 3

[xwiki-devs] Velocity in title
by [Ricardo Rodriguez] eBioTIC. 22 Oct '10

22 Oct '10

Hi all! Please, is possible to include Velocity script in documents' title of a 1.3.8295 installation? Thanks! Cheers, Ricardo -- Ricardo Rodríguez CTO eBioTIC. Life Sciences, Data Modeling and Information Management Systems

4 7

[xwiki-devs] [ANN] XWiki Enterprise and XWiki Enterprise Manager 2.4.4 Released
by Thomas Mortagne 22 Oct '10

22 Oct '10

The XWiki development team is pleased to announce the release of XWiki Enterprise and XWiki Enterprise Manager 2.4.4. Important bug fixed: * XWIKI-5542 - Different rights in view and rest mode * XWIKI-5597 - File upload plugin doesn't strip the file path in IE * XWIKI-5598 - When importing a XAR, ignoring translated documents does not work * XWIKI-5309 - rest api query called by XE JumpToPage causes performance problem and even deadlock * XWIKI-5387 - Apache commons URIUtil is potentially unsafe * XWIKI-4366 - Blockquote is badly parsed when multiple lines are styled together * XWIKI-5525 - DBList request level cache is ignored * XWIKI-5591 - HTML to Wiki Syntax 2 looses color in certain cases * XWIKI-5599 - In the XAR importer UI, the initial space document count is wrong whenever that space has translated documents * XWIKI-5530 - LDAP module: Impossible to login with the same UID if your DN changes * XWIKI-5523 - NPE during parsing of a sequence table/style/paragraphe/paragraphe * XAANNOTATIONS-32 - Highlight does not disappear after deleting the last annotation from a page For more information see the Releases notes at: http://www.xwiki.org/xwiki/bin/view/Main/ReleaseNotesXWikiEnterprise244 and http://www.xwiki.org/xwiki/bin/view/Main/ReleaseNotesXEM244 Thanks -The XWiki dev team

1 0

[xwiki-devs] [VOTE] Release XE and XEM 2.5
by Sergiu Dumitriu 22 Oct '10

22 Oct '10

Hi devs, I'd like to start releasing 2.5 this Friday, so that users can test the build until Monday. Are there still blocking issues that you'd like to fix? -- Sergiu Dumitriu http://purl.org/net/sergiu/

7 8

[xwiki-devs] Work on converting from synthax 1.0 to 2.0
by Stefan abageru 21 Oct '10

21 Oct '10

Hello guys ! I am starting working on converting the documents in space Panels. In case anybody is working on it right now, please let me know :) Stefan

4 3

[xwiki-devs] [proposal] New database table binarychunk, interface c.x.x.store.BinaryStore, and interface c.x.x.doc.BinaryObject
by Caleb James DeLisle 21 Oct '10

21 Oct '10

Because the storage of large attachments is limited by database constraints and the fact that the JDBC does not allow us to stream content out of the database, I propose we add a new database table binarychunk. The mapping will read as follows: <class name="com.xpn.xwiki.store.hibernate.HibernateBinaryStore$BinaryChunk" table="binarychunk"> <composite-id unsaved-value="undefined"> <key-property name="id" column="id" type="integer" /> <key-property name="chunkNumber" column="chunknumber" type="integer" /> </composite-id> <property name="content" type="binary"> <column name="content" length="983040" not-null="true"/> </property> </class> Notice the maximum length (983040 bytes) is a number which is divisible by many common buffer sizes and is slightly less than the default max_packet_size in mysql which means that using the xwikibinary table, we could store attachments of arbitrary size without hitting mysql default limits. com.xpn.xwiki.store.BinaryStore will contain: @param toLoad a binary object with an id number set, will be loaded. void loadObject(BinaryObject toLoad) @param toStore a binary object, if no id is present then it will be given one upon successful store, if id is present then that id number will be used. void storeObject(BinaryObject toStore) This will be implemented by: com.xpn.xwiki.store.hibernate.HibernateBinaryStore com.xpn.xwiki.doc.BinaryObject will contain: void setContent(InputStream content) OutputStream setContent() InputStream getContent() void getContent(OutputStream writeTo) Note: The get function and set functions will be duplicated with input or output streams to maximize ease of use. This will be implemented by com.xpn.xwiki.doc.TempFileBinaryObject which will store the binary content in a temporary FileItem (see Apache commons fileupload). + This will be able to provide a back end for not only attachment content, but for attachment archive and document archive if it is so desired. + I have no intent of exposing it as public API at the moment. WDYT? Caleb

5 5

[xwiki-devs] [VOTE] SkinAccessBridge new API: getIconURL()
by Vincent Massol 21 Oct '10

21 Oct '10

Hi, In order to implement support for icons/symbols in the rendering (see ), we need to add an API to return an icon URL based on the icon name. public interface SkinAccessBridge { ... /** * @param iconName the standard name of an icon (it's not the name of the file on the filesystem, it's a generic * name, for example "success" for a success icon * @return the URL to the icon resource * @since 2.6M1 */ String getIconURL(String iconName); } +1 from me. Thanks -Vincent

9 9

[xwiki-devs] [Rendering] WikiModel API change for merging getAttachmentURL and getImageURL?
by Vincent Massol 21 Oct '10

21 Oct '10

Hi, We currently have 2 methods introduced in 2.5 timeframe in the WikiModel class: String getAttachmentURL(ResourceReference attachmentReference); String getImageURL(ResourceReference attachmentReference, Map<String, String> parameters); I think we should merge them into a single method in charge of returning the URL of any resource reference: getResourceURL(ResourceReference reference, Map<String, String> parameters); Note that this would assume that all references have a URL associated to them. It's not always true (it's true for documents, attachments, url, interwiki and the future icon/symbol but false for path and mailto). We could return null for resource types that have no associated URLs. The reason I'm proposing this because in order to implement support for symbol/icon I'd need to add a new method to WikiModel: getIconURL(ResourceReference iconReference) but I feel it's better to have a single getResourceURL(). WDYT? Thanks -Vincent

4 8

Re: [xwiki-devs] [discussion] Filesystem vs. database attachment storage. Re: [proposal] New database table binarychunk, interface c.x.x.store.BinaryStore, and interface c.x.x.doc.BinaryObject
by Ludovic Dubost 21 Oct '10

21 Oct '10

Le 20/10/10 12:03, Caleb James DeLisle a écrit : > > On 10/20/2010 05:33 AM, Ludovic Dubost wrote: >> Hi, >> >> We do want the availability of file attachment storage (Sergiu has done an implementation during the >> summer hackathon), but as Guillaume said it should be to the choice of the administrator. >> >> Now concerning database storage, about Hibernate, does it means streams are not available at all in >> Hibernate or does it mean they don't always work ? >> If streams are available for the databases that support it, which ones support it ? > They are available, they require use of the blob type so we would have to add a column. I was warned > about incompatibility issues. I understand that mysql and mssql stream the content back onto the > heap before saving which ruins any memory savings. Postgres seems to support blobs but I was warned > about strange issues, specifically this: http://in.relation.to/Bloggers/PostgreSQLAndBLOBs > I was told that oracle has the best streaming support but I also read that oracle blob support > requires the use of proprietary api. > > This is why I had opted for chinking the data instead of using blobs. > Indeed if we are positive that mysql will use the heap to store the full BLOB then there is no point to this solution since it is our main database. >> Concerning your proposal, it's interesting as indeed if we use streams for everything else, we do >> get rid of the memory consuption issue for attachments. >> Now I have a few concerns: >> >> - complexity and management of the data. What happens if we have a corrupted DB and one of the >> chunks fails to save. We might end up with invalid content. > I had planned on committing the transaction only after all chunks are saved, if the database has > memory issues with large commits, another possibility would be to verify after saving and throw an > exception if that fails. > That might indeed help if everything is in one transation except that MyISAM is not transactional so we can end up with incomplete data. We do need a way to verify the coherency. We could consider that if the size is incorrect we don't accept the result. >> - we also have to solve other large items (like attachment history or recycle bin of attachments) > This is why I favor a generic BinaryStore rather than a change to XWikiHibernateAttachmentStore. > Another issue which will have to be addressed is the memory consumption of JRCS for AttachmentArchive. At the same time we should avoid mixing apples and oranges. We should not have data with different meanings in different tables. For Attachment Archive, I'm not against a solution which stops doing RCS. It has never been efficient anyway. >> On a side note concerning the max_allowed_packet issue in MySQL, I was able to change that value at >> runtime (from the mysql console). If this also works using a remote connection, maybe we could hack >> and force a big value at runtime. >> This would be really great because the max_allowed_packet is killing us. XWiki does not report it >> well in many cases and almost no customers reads the documentation and sets the value properly. We >> also have seen in many cases, where the database is shared with other applications, and there is >> little access to the database configuration and to the ability of restart. To make it short, the >> max_allowed_packet issue is a major issue when operating XWiki. > ``little access to the database configuration'' This may also mean the xwiki user does not have > permission to change the setting at runtime. What I meant is not being allowed to restart it. >> Before we go into large fixes for that problem, could we maybe at least check that we report errors >> properly (on a 2.0.5 we were not for sure at least for attachment saving failure). > The fix to http://jira.xwiki.org/jira/browse/XWIKI-5405 has changed attachments so that the content > and meta data are all saved in a single transaction and http://jira.xwiki.org/jira/browse/XWIKI-5474 > prevents documents from being cached on save so we should have no more attachments which dissapear > when the cache is purged. Great. This will at least make the problem show up right away. Does 5405 protect us from having the attachment in the attachment list and have no content ? >> We should also >> make sure we can always delete even when we cannot read the data in memory. This is also not the >> case when we cannot read the data because it's too big or because one of the tables does not have >> any data. > Sounds like a test ;) > You mean a test for you ? a test in the code ? or an XWiki test suite ? It'a bit of a complex test which requires to screw up attachment data in all way possible and prove that you can still delete everything that is left. Ludovic > Caleb > >> Ludovic >> >> Le 18/10/10 19:55, Caleb James DeLisle a écrit : >>> I talked with the Hibernate people about using streams and was told that it is not supported by all >>> databases. >>> >>> As an alternative to the proposal below I would like to propose a filesystem based storage mechanism. >>> The main advantage of using the database to store everything is that administrators need only use >>> mysql_dump and they have their entire wiki backed up. >>> >>> If we are to abandon that requirement, we can have much faster attachment storage by using the >>> filesystem. For this, I propose BinaryStore interface remains the same but >>> com.xpn.xwiki.doc.BinaryObject would contain: >>> >>> void addContent(InputStream content) >>> >>> OutputStream addContent() >>> >>> void clear() >>> >>> InputStream getContent() >>> >>> void getContent(OutputStream writeTo) >>> >>> clear() would clear the underlying file whereas addContent would always append to it. >>> >>> >>> The added column would look like this: >>> >>> <class name="com.xpn.xwiki.store.doc.FilesystemBinaryObject" table="filesystembinaryobject"> >>> <id name="id" column="id"> >>> <generator class="native" /> >>> </id> >>> >>> <property name="fileURI" type="string"> >>> <column name="fileuri" length="255" not-null="true"/> >>> </property> >>> </class> >>> >>> >>> This would as with the original proposal be useful for not only storing attachments but attachment >>> history, deleted attachments and even document history or deleted documents. >>> >>> >>> WDYT? >>> >>> Caleb >>> >>> >>> On 10/15/2010 04:21 PM, Caleb James DeLisle wrote: >>>> Because the storage of large attachments is limited by database constraints and the fact that the >>>> JDBC does not allow us to stream content out of the database, I propose we add a new database table >>>> binarychunk. >>>> >>>> The mapping will read as follows: >>>> >>>> <class name="com.xpn.xwiki.store.hibernate.HibernateBinaryStore$BinaryChunk" table="binarychunk"> >>>> <composite-id unsaved-value="undefined"> >>>> <key-property name="id" column="id" type="integer" /> >>>> <key-property name="chunkNumber" column="chunknumber" type="integer" /> >>>> </composite-id> >>>> >>>> <property name="content" type="binary"> >>>> <column name="content" length="983040" not-null="true"/> >>>> </property> >>>> </class> >>>> >>>> Notice the maximum length (983040 bytes) is a number which is divisible by many common buffer sizes >>>> and is slightly less than the default max_packet_size in mysql which means that using the >>>> xwikibinary table, we could store attachments of arbitrary size without hitting mysql default >>>> limits. >>>> >>>> >>>> com.xpn.xwiki.store.BinaryStore will contain: >>>> >>>> @param toLoad a binary object with an id number set, will be loaded. >>>> void loadObject(BinaryObject toLoad) >>>> >>>> @param toStore a binary object, if no id is present then it will be given one upon successful >>>> store, if id is present then that id number will be used. >>>> void storeObject(BinaryObject toStore) >>>> >>>> This will be implemented by: com.xpn.xwiki.store.hibernate.HibernateBinaryStore >>>> >>>> >>>> com.xpn.xwiki.doc.BinaryObject will contain: >>>> >>>> void setContent(InputStream content) >>>> >>>> OutputStream setContent() >>>> >>>> InputStream getContent() >>>> >>>> void getContent(OutputStream writeTo) >>>> >>>> Note: The get function and set functions will be duplicated with input or output streams to maximize >>>> ease of use. >>>> >>>> This will be implemented by com.xpn.xwiki.doc.TempFileBinaryObject which will store the binary >>>> content in a temporary FileItem (see Apache commons fileupload). >>>> >>>> >>>> >>>> + This will be able to provide a back end for not only attachment content, but for attachment >>>> archive and document archive if it is so desired. >>>> + I have no intent of exposing it as public API at the moment. >>>> >>>> >>>> WDYT? >>>> >>>> Caleb >>>> >>>> _______________________________________________ >>>> devs mailing list >>>> devs(a)xwiki.org >>>> http://lists.xwiki.org/mailman/listinfo/devs >>>> >>> _______________________________________________ >>> devs mailing list >>> devs(a)xwiki.org >>> http://lists.xwiki.org/mailman/listinfo/devs >>> >> > -- Ludovic Dubost Blog: http://blog.ludovic.org/ XWiki: http://www.xwiki.com Skype: ldubost GTalk: ldubost

2 1

[xwiki-devs] [Proposal] Supporting symbols/emoticons
by Vincent Massol 21 Oct '10

21 Oct '10

Hi devs, I'd like to propose 3 things: 1) Add support for symbols/emoticons using our new system in XWiki Syntax 2.1, i.e. image:symbol:<symbol name> Another possibility is: image:icon:<icon name> ex: image:symbol:success (for the success symbol) 2) Use the existing silk icon library and be able to reference all icons using their file names as the symbol name. For ex, image:symbol:thumb_up 3) Modify the Box macro to accept a ResourceReference as its image parameter so that all valid image resource references can be specified (including the new symbol/icon scheme). WDYT? This will solve the emoticon/symbol need + the generic way of adding a box with any symbol. Open questions: A) Do we need to handle icon size? If so, how? (answer: with an image parameter) B) Do we want to expose all silk icons? What if we use another library later on, will we be able to support all existing silk symbols? Thus, do we need to pick a subset and settle on names independent from the underlying library? For A) IMO we don't need it now and we can always use an image param later on if need be. For B) I'm tempted to think that a set of well-known symbols independent from the underlying library is better (but more work since we need to define that set and the names). Thanks -Vincent

5 7