On 27 Aug 2016, at 15:44, Paul Libbrecht
<paul(a)hoplahup.net> wrote:
Thanks for having this extra thread.
I think this thread is much more important than starting to design
something already.
Issues with the XAR format
======================
* XML is not an easy to edit format and doesn’t allow use a specific
editor to edit content
* XML also requires content to be XML-encoded and thus is really not
easy to make modification (there’s a risk of breaking the XML easily)
I completely
disagree with these two statements.
I probably didn’t use the right words because that’s the reason why I think that's you
started the xinclude proposal :)
AFAIK you started the xinclude proposal because you wanted to be able to edit content with
specific editor (js, css, etc)… which is exactly what the second part of the first point
is about.
Let me rephrase the first sentence:
* XML is not an easy to edit format for human beings (it’s very verbose and easy to make
mistakes: missing encoding, etc). It’s also very hard to edit with a plain text editor.
As for the 2nd sentence, I don’t see how you can disagree since it’s a fact...
XML is easy to edit and is supported by very very many
editors and IDEs.
It can also be validated.
What you’re saying is very theoretical. The practice (that we’ve had for 10 years of using
the XAR format) is that our XML format that is hard to edit and can break easily (as was
proven numerous times by our committers and contributors). It’s actually so dangerous that
we’ve had to develop a process which goes like this:
* Never edit the xml directly
* Always import it into a running XWiki instance, make the modifications there and export
from the wiki into XAR
* Then unpack the XAR into the source directory and run mvn xar:format to go back to the
original format.
Nobody is using a pure XML editor with validation. We are all using Java IDEA (IntelliJ
IDEA, Eclipse, etc) and they all allow you to edit XML as plain text and that’s what we’re
doing. No developer I know of is using an XML editor in a structured way (just too painful
and complex to navigate the structure).
(see below)
XML can be written in a very elegant and readable
fashion if you care
for it.
Generally however, XML is produced form other structured information in
a way that does not help readability.
Can you see more issues?
The problem is
how we put *everything* into XML.
(you get the same horror if you use Caleb's tools xardump and do not
tune anything: the resulting javascript is horrible.)
Use cases for an alternative filesystem format
===================================
(some UC taken from
http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiApp…)
* UC1: the structure should be (as) easy (as possible) to navigate in
an IDE style view
* UC2: it should be easy to add content (a new script or attachment on
an existing structure). It should allow using specific editors for
different content types, e.g. if a page content is in markdown, it
should be editable with a MD editor, js and css should be editable
with web editors, etc.
UC2.1: Attachments should also be present as standalone
files.
* UC3: It should be possible to build a packaged
version of the
sources with Maven
* UC4: It should be possible to import the packaged version into a
running XWiki instance
* UC5: It should be possible to export a portion of a running XWiki
instance in this format
* UC6: This format should be able to fully replace the XAR format .
The new format should support at least all features supported by the
XAR format (versioned, etc). Note: XE will need to be refactor a bit
so that the XAR format can be swapped out by introducing extension
points/APIs. The idea would be to deprecate the XAR format and
introduce this new format instead, and the 2 formats should be avle to
cohabit next to each other in XWiki.
* UC7: When importing in a wiki and exporting again (without making
any change in the wiki), it should generate an identical structure and
content, with no difference.
I do believe that UC7 is not doable in full
generality.
Why not, this is what we have with the XAR format actually and UC7 is actually already
contained in UC6 (but it’s better to be explicit)?
Any more?
UC8: the core representation should be using a syntax that is widely
spread and completely specified (i.e. we should not invent another
syntax for this)
This is not a requirement for me. The syntax should be easy to write into, especially
using a plain text editor. YAML for example is a perfectly valid syntax for me.
UC9: the system should make an archival process a
widespread practice,
in the form of zipped files probably.
That’s UC3 for me. I hesitated to put ZIP in the requirement for UC3 since I didn’t want
to limit us to that but it’s probably going to be zip anyway.
UC10: developers should have the discretion to decide
when a component
needs to be in a separate file or not. That is, small text fragments and
even small attachments should be includable within bigger files
I don’t agree with this one. I’d like convention over flexibility, i.e. a fixed format on
which tools can easily build upon. This is similar to Maven vs Ant.
This is why in my proposal for a format I’ve proposed fixed file names.
Allowing discretion means everyone will do it differently and we’ll need to define best
practices and best practices are hard to enforce and always cause problems.
UC11: there should be the possibility to share content
of a file between
several files or components (e.g. creator elements)
I wouldn’t phrase it this way. I’d prefer to say that it should be possible to apply
default values to missing elements in metadata.
The way I see this for example for the format I’ve proposed in the other thread, is by
having default properties that can be put on the filesystem, for example in
default.properties file) so that when an element is missing the default would be used.
Now I’m not sure we want this requirement and for me it’s an optional requirement and not
a mandatory one. It makes it much harder to develop an exporter.
UC12: (transformation) simple search and replace
operation should be
supported by the build mechanism, especially when dependencies are followed.
Could you explain more, I don’t understand?
UC13: it would be good that the format can be
specified by a grammer
against which one can validate (e.g. RelaxNG)
I don’t agree in the way it’s phrased since it’s too limiting. I’d change it to: it should
be possible for the packager tool to validate the content (what xar:verify does right now
but that could be extended to verify that the required metadata are defined). I don’t
think we need a formal grammar. The important part is that we have validation.
Thanks
-Vincent
Paul
(FYI UC10, UC11, and UC12 follow the architecture recommendation to be
composable vs contextual so as to give us greater flexibility)