[xwiki-devs] Building Xwiki core - xwiki-devs@xwiki.org

3 Mar 2008

Hello,

I am trying to learn how to build XWiki from the source.

I checked out the source code in Eclipse 3.3 using Sunclipse and then I
created a Java Project out of the XWiki Core/src folder.
Now, I am trying to follow directions from the links -
http://dev.xwiki.org/xwiki/bin/view/Community/Building
I downloaded maven-2.1-SNAPSHOT and create dthe .m2/settings.xml.
I also created M2_HOME varibale and set the PATH variable with M2_HOME.
Set the JAVA_HOME to the jdk installed on my system.

But, when I try "mvn install" from the directory where I have the
XWiki/src checked out (Eclipse workspace), it says that JAVA_HOME is not set
and gives me the following error:

ERROR: JAVA_HOME not found in your environment.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation

I have JAVA_HOME set and it shows up in the list of env. variables in my
system like this:

JAVA_HOME = C:\abc\Java\jdk1.5.0_14
PATH = C:\WINDOWS;C:\WINDOWS\System32;C:\Program
Files\Subversion\bin;C:\Program Files\TortoiseSVN\bin;C:\Program Files\Quest
Software\PuTTY\;C:\Program Files\QuickTime\QTSystem\;C:\Program Files\Quest
Software\PuTTY\;%JAVA_HOME%\bin;%M2_HOME%\bin

Could you please tell me what I am doing wrong and what I should change to
get this right!

Also, am I on the irght path when I I say I am trying to buildthe src of
XWiki -core? and nothing else! As an eclipse project, it is all compiled and
does not show errors implying that it has all the files that it needs for it
to complie.

Thanks

On 3/3/08, devs-request(a)xwiki.org &lt;devs-request(a)xwiki.org&gt; wrote:
...

 Send devs mailing list submissions to
        devs(a)xwiki.org

 To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.xwiki.org/mailman/listinfo/devs
 or, via email, send a message with subject or body 'help' to
        devs-request(a)xwiki.org

 You can reach the person managing the list at
        devs-owner(a)xwiki.org

 When replying, please edit your Subject line so it is more specific
 than "Re: Contents of devs digest..."

 Today's Topics:

   1. Re: Profiling: Why do attachments require so      much    memory
      (Sergiu Dumitriu)
   2. [VOTE] Change the attachment archive storage (Sergiu Dumitriu)
   3. Re: [VOTE] Change the attachment archive storage (Vincent Massol)
   4. Re: [VOTE] Change the attachment archive storage (Sergiu Dumitriu)
   5. [Proposal] XE 1.3 final release date (Vincent Massol)
   6. Re: Xwiki Enterprise 1.3 RC1 release date? (Vincent Massol)
   7. Re: About GSoC 2008 (Asiri Rathnayake)
   8. Re: Xwiki Enterprise 1.3 RC1 release date? (Sergiu Dumitriu)

 ----------------------------------------------------------------------

 Message: 1
 Date: Mon, 03 Mar 2008 17:28:21 +0100
 From: Sergiu Dumitriu &lt;sergiu(a)xwiki.com&gt;
 Subject: Re: [xwiki-devs] Profiling: Why do attachments require so
        much    memory
 To: XWiki Developers &lt;devs(a)xwiki.org&gt;
 Message-ID: &lt;47CC2725.40707(a)xwiki.com&gt;
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed

 Vincent Massol wrote:
  Nice work Sergiu. We should transform this into a
jira issue to not
 forget it.

 We should vote for it first.

  One other idea: store attachments on the file
system and not in the DB.

 Thanks
 -Vincent

 On Feb 27, 2008, at 3:48 PM, Sergiu Dumitriu wrote:

> Hi devs,
>
> Last night I checked what happens when uploading a file, and why does
> that action require huge amounts of memory.
>
> So, whenever uploading a file, there are several places where the file
> content is loaded into memory:
> - as an XWikiAttachment as byte[] ~= filesize
> - as an XWikiAttachmentArchive as Base64 encoded string ~=
> 2*4*filesize
> - as hibernate tokens that are sent to the database, clones of the
> XWikiAttachment and XWikiAttachmentArchive data ~= 9*filesize
> - as Cached attachments and attachment archive, clones of the same 2
> objects ~= 9*filesize
>
> Total: ~27*filesize bytes in memory.
>
> So, out of a 10M file, we get at least 270M of needed memory.
>
> Worse, if this is not the first version of the attachment, then the
> complete attachment history is loaded in memory, so add another
> 24*versionsize*versions of memory needed during upload.
>
> After the upload is done, most of these are cleared, only the cached
> objects will remain in memory.
>
> However, a problem still remains with the cache. It is a LRU cache
> with
> a fixed capacity, so even if the memory is full, the cached
> attachments
> will not be released.
>
> Things we can improve:
> - Make the cache use References. This will allow cached attachments to
> be removed from memory when there's a need for more memory
> - Do a better attachment archive system. I'm not sure it is a good
> idea
> to have diff-based versioning of attachments. In theory, it saves
> space
> when versions are much alike, but it does not really work in practice
> because it does a line-diff, and a base64 encoded string does not have
> newlines. What's more, the space gain would be efficient when there
> are
> many versions, as one version alone takes 4 times more space than a
> binary dump of the content.
>
> Suppose we switch to a "one version per table row" for attachment
> history, with direct binary dump, then the memory needed for uploading
> would be 6*filesize, which is much less. 

 --
 Sergiu Dumitriu
 http://purl.org/net/sergiu/

 ------------------------------

 Message: 2
 Date: Mon, 03 Mar 2008 17:34:07 +0100
 From: Sergiu Dumitriu &lt;sergiu(a)xwiki.com&gt;
 Subject: [xwiki-devs] [VOTE] Change the attachment archive storage
 To: XWiki Developers &lt;devs(a)xwiki.org&gt;
 Message-ID: &lt;47CC287F.9090609(a)xwiki.com&gt;
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed

 Hi devs,

 As detailed in another mail
 (http://lists.xwiki.org/pipermail/devs/2008-February/005344.html), the
 current attachment archive mechanism is very inefficient. We should
 write a new one, which stores attachment versions as plain binary data,
 and see if the current core is pluggable enough to allow the old
 mechanism to be preserved as a plugin, and possibly define other storage
 mechanisms, like a filesystem based one.

 Artem, do you think you can help, as this is something related to what
 you've been working on?
 --
 Sergiu Dumitriu
 http://purl.org/net/sergiu/

 ------------------------------

 Message: 3
 Date: Mon, 3 Mar 2008 17:40:39 +0100
 From: Vincent Massol &lt;vincent(a)massol.net&gt;
 Subject: Re: [xwiki-devs] [VOTE] Change the attachment archive storage
 To: XWiki Developers &lt;devs(a)xwiki.org&gt;
 Message-ID: &lt;D5354D29-6FB9-41CB-A450-C77DE4E9BB54(a)massol.net&gt;
 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes

 On Mar 3, 2008, at 5:34 PM, Sergiu Dumitriu wrote:

  Hi devs,

 As detailed in another mail
 (http://lists.xwiki.org/pipermail/devs/2008-February/005344.html), the
 current attachment archive mechanism is very inefficient. We should
 write a new one, which stores attachment versions as plain binary
 data,
 and see if the current core is pluggable enough to allow the old
 mechanism to be preserved as a plugin, and possibly define other
 storage
 mechanisms, like a filesystem based one.

 Artem, do you think you can help, as this is something related to what
 you've been working on? 
 My only worry is that last time we changed the database format we
 spent several months stabilizing it... since there were lots of
 problems. I'm not even sure we've finished stabilizing it fully...

 So is there any chance that this would be simpler? :)

 Thanks
 -Vincent

 ------------------------------

 Message: 4
 Date: Mon, 03 Mar 2008 18:00:12 +0100
 From: Sergiu Dumitriu &lt;sergiu(a)xwiki.com&gt;
 Subject: Re: [xwiki-devs] [VOTE] Change the attachment archive storage
 To: XWiki Developers &lt;devs(a)xwiki.org&gt;
 Message-ID: &lt;47CC2E9C.8070404(a)xwiki.com&gt;
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed

 Vincent Massol wrote:
  On Mar 3, 2008, at 5:34 PM, Sergiu Dumitriu
wrote:

  Hi devs,

 As detailed in another mail
 (http://lists.xwiki.org/pipermail/devs/2008-February/005344.html), the
 current attachment archive mechanism is very inefficient. We should
 write a new one, which stores attachment versions as plain binary
 data,
 and see if the current core is pluggable enough to allow the old
 mechanism to be preserved as a plugin, and possibly define other
 storage
 mechanisms, like a filesystem based one.

 Artem, do you think you can help, as this is something related to what
 you've been working on? 
 My only worry is that last time we changed the database format we
 spent several months stabilizing it... since there were lots of
 problems. I'm not even sure we've finished stabilizing it fully...

 So is there any chance that this would be simpler? :)

 Thanks
 -Vincent

 It should be simpler, as attachments are just binary blobs, while the
 XML history is much too fragile.
 --
 Sergiu Dumitriu
 http://purl.org/net/sergiu/

 ------------------------------

 Message: 5
 Date: Mon, 3 Mar 2008 18:18:18 +0100
 From: Vincent Massol &lt;vincent(a)massol.net&gt;
 Subject: [xwiki-devs] [Proposal] XE 1.3 final release date
 To: XWiki Developers &lt;devs(a)xwiki.org&gt;
 Message-ID: &lt;FE2B2F15-5128-47CC-90B7-CAF96398BADE(a)massol.net&gt;
 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes

 Hi,

 I'm proposing to revise the 1.3 Final release date to this Friday 7th
 of March 2007.

 We've already fixed the major issues raised in 1.3RC1 but I think we
 need a few days to let people report any issues they might have had
 with 1.3RC1.

 Let me know what you think and especially if you don't agree. I'm
 proposing to do the release.

 Thanks
 -Vincent

 ------------------------------

 Message: 6
 Date: Mon, 3 Mar 2008 18:23:39 +0100
 From: Vincent Massol &lt;vincent(a)massol.net&gt;
 Subject: Re: [xwiki-devs] Xwiki Enterprise 1.3 RC1 release date?
 To: XWiki Developers &lt;devs(a)xwiki.org&gt;
 Message-ID: &lt;161441E7-DC86-48ED-A91A-A8F143A2FC4B(a)massol.net&gt;
 Content-Type: text/plain; charset="us-ascii"

 On Feb 27, 2008, at 3:17 PM, Everitt, Glenn wrote:

  Is the plan to provide a 
xwiki-enterprise-web1-1.3-rc1.war file on
 the Xwiki Download page?  When do you think this would be
 available?  Would it be better to pull it directly from svn?  If I
 get it from svn how is the RC1 release marked?
  It's been released a few days ago...
  I noticed that you were looking at using GWT Html
editor.  Doesn't
 GWT Html editor just wrap the FCKEditor?  What is the advantage of
 using GWT Html Editor instead of just using the FCKEditor?
  I have no idea. Anyone knows?

 Thanks
 -Vincent