[xwiki-devs] [Proposal] Using the browser's print feature instead of our PDF Export for the UI
Hi devs, We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it. I’d like to propose the following: * Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts. WDYT? Thanks -Vincent
Interesting, however, I see at least 2-3 possible limitations: * we will drop cover and table of contents * browsers (operating systems?) that don't have print to pdf option, will not have this working anymore * less customizations will be possible on this PDF (e.g. can we control header / footer of the browser's print to PDF? Margins size, etc) * multipage pdf export (that we currently have in multiple forms) will have to be re-thought * will links continue to work properly as links inside the pdf ( I will need to test - actually, I was talking about links that link to another place in the PDF, but while testing I realized that my Firefox did not export any link as link in pdf - internal or external - it only exported text) * as resulting from the previous small test but easy to generalize: the result of this feature will start to be browser dependent . For the problems of Apache FOP, does anyone have any idea what libraries do other software use for this kind of functionality? (maybe there is a more cool lib that we haven't found out about) Anca On Wed, Jan 18, 2017 at 2:54 PM, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
On 01/18/2017 12:05 PM, Anca Luca wrote:
For the problems of Apache FOP, does anyone have any idea what libraries do other software use for this kind of functionality? (maybe there is a more cool lib that we haven't found out about)
There are commercial alternatives, but those are out of the question due to the license. There's the programmatic PDF generation option with PDFBox, but that will require a MAJOR redesign of the export. Then there's the option of building our own patched version of FOP with https://issues.apache.org/jira/browse/FOP-2469 included. It wouldn't be the first time we use a patched library, and it seems to be the solution with the least amount of work needed. -- Sergiu Dumitriu http://purl.org/net/sergiu
On 18 Jan 2017, at 21:42, Sergiu Dumitriu <[email protected]> wrote:
On 01/18/2017 12:05 PM, Anca Luca wrote:
For the problems of Apache FOP, does anyone have any idea what libraries do other software use for this kind of functionality? (maybe there is a more cool lib that we haven't found out about)
There are commercial alternatives, but those are out of the question due to the license.
There's the programmatic PDF generation option with PDFBox, but that will require a MAJOR redesign of the export.
Then there's the option of building our own patched version of FOP with https://issues.apache.org/jira/browse/FOP-2469 included.
The patch in FOP-2469 is possibly not good enough which could explain why it’s not committed yet. So there might be more work to do before we could use it. Analysing and understand it is also quite some work I guess.
It wouldn't be the first time we use a patched library, and it seems to be the solution with the least amount of work needed.
There are 24 open issues with PDF export right now (and 10 bugs): http://jira.xwiki.org/issues/?jql=component%20%3D%20%22Old%20Core%20-%20PDF%... We could take them one by one and see if using the print button would solve some of them. Thanks -Vincent
-- Sergiu Dumitriu http://purl.org/net/sergiu
Hi Anca,
On 18 Jan 2017, at 18:05, Anca Luca <[email protected]> wrote:
Interesting, however, I see at least 2-3 possible limitations:
* we will drop cover and table of contents
Actually we won’t. The idea would be to have to a Print button inside XWiki (right now we have Print Preview) and we would add support for cover + TOC in print.vm.
* browsers (operating systems?) that don't have print to pdf option, will not have this working anymore
I think all OSes support this. Do you have any meaningful OS/Browser in mind? We could still leave it as an option to use our PDF Export if users need it.
* less customizations will be possible on this PDF (e.g. can we control header / footer of the browser's print to PDF? Margins size, etc)
Yes we can. This is controlled by the Browser which allows you to customise this. For the content of header/footer that’s easy, that would be the print.vm controlling them.
* multipage pdf export (that we currently have in multiple forms) will have to be re-thought
Yes indeed. We would still be able to support this if we want by having our print.vm support it.
* will links continue to work properly as links inside the pdf ( I will need to test - actually, I was talking about links that link to another place in the PDF, but while testing I realized that my Firefox did not export any link as link in pdf - internal or external - it only exported text)
Actually that should work better than what we have now ;) Note that we can do anything we want since we control the print.vm so we can use any XWikiServletURLFactory we want.
* as resulting from the previous small test but easy to generalize: the result of this feature will start to be browser dependent .
So we control the layout completely. Then indeed it’s up to the browser to print. I don’t know if various browser print thing differently and how much difference there are between them. All I know is that they’ve spent way more men/hours than we’ve done on making a nice print ;) So again my goal in this proposal is to benefit from the Print feature of browsers, externalise the feature as much as possible (similar to what we’ve done with CKEditor). I wouldn’t want to drop our PDF export since it has some use cases, for example for generating export from scripts. I’m only referring to the basic Print option here and to transform the Print Preview into a Print one and tune it a bit to look as nice as possible. And make the Export to PDF optional and off by default. Thanks -Vincent
For the problems of Apache FOP, does anyone have any idea what libraries do other software use for this kind of functionality? (maybe there is a more cool lib that we haven't found out about)
Anca
On Wed, Jan 18, 2017 at 2:54 PM, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
On Fri, Jan 20, 2017 at 2:10 PM, Vincent Massol <[email protected]> wrote:
Hi Anca,
On 18 Jan 2017, at 18:05, Anca Luca <[email protected]> wrote:
Interesting, however, I see at least 2-3 possible limitations:
* we will drop cover and table of contents
Actually we won’t. The idea would be to have to a Print button inside XWiki (right now we have Print Preview) and we would add support for cover + TOC in print.vm.
* browsers (operating systems?) that don't have print to pdf option, will not have this working anymore
I think all OSes support this. Do you have any meaningful OS/Browser in mind? We could still leave it as an option to use our PDF Export if users need it.
* less customizations will be possible on this PDF (e.g. can we control header / footer of the browser's print to PDF? Margins size, etc)
Yes we can. This is controlled by the Browser which allows you to customise this.
For the content of header/footer that’s easy, that would be the print.vm controlling them.
* multipage pdf export (that we currently have in multiple forms) will have to be re-thought
Yes indeed. We would still be able to support this if we want by having our print.vm support it.
* will links continue to work properly as links inside the pdf ( I will need to test - actually, I was talking about links that link to another place in the PDF, but while testing I realized that my Firefox did not export any link as link in pdf - internal or external - it only exported text)
Actually that should work better than what we have now ;)
Note that we can do anything we want since we control the print.vm so we can use any XWikiServletURLFactory we want.
I'm not sure we fully understand eachother... I printed to pdf with Firefox on Ubuntu the standard Sandbox.WebHome page and none of the links we have in there are actually clickable, they are plain text (in the Links section, with external links, or in the Table of contents section, with internal links). I don't understand exactly how XWikiServletURLFactory would help us control links inside the pdf itself, if the pdf is generated by the client-side software (it's true that html anchors seem to work for chrome, so that is one way of doing it, but apparently not on firefox). For the other points I would need to test more to see if they actually answer what I am thinking of.. Anca
* as resulting from the previous small test but easy to generalize: the result of this feature will start to be browser dependent .
So we control the layout completely. Then indeed it’s up to the browser to print. I don’t know if various browser print thing differently and how much difference there are between them.
All I know is that they’ve spent way more men/hours than we’ve done on making a nice print ;)
So again my goal in this proposal is to benefit from the Print feature of browsers, externalise the feature as much as possible (similar to what we’ve done with CKEditor). I wouldn’t want to drop our PDF export since it has some use cases, for example for generating export from scripts.
I’m only referring to the basic Print option here and to transform the Print Preview into a Print one and tune it a bit to look as nice as possible. And make the Export to PDF optional and off by default.
Thanks -Vincent
For the problems of Apache FOP, does anyone have any idea what libraries do other software use for this kind of functionality? (maybe there is a more cool lib that we haven't found out about)
Anca
On Wed, Jan 18, 2017 at 2:54 PM, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
On Fri, Jan 20, 2017 at 2:10 PM, Vincent Massol <[email protected]> wrote:
Hi Anca,
On 18 Jan 2017, at 18:05, Anca Luca <[email protected]> wrote:
Interesting, however, I see at least 2-3 possible limitations:
* we will drop cover and table of contents
Actually we won’t. The idea would be to have to a Print button inside XWiki (right now we have Print Preview) and we would add support for cover + TOC in print.vm.
* browsers (operating systems?) that don't have print to pdf option, will not have this working anymore
I think all OSes support this. Do you have any meaningful OS/Browser in mind? We could still leave it as an option to use our PDF Export if users need it.
* less customizations will be possible on this PDF (e.g. can we control header / footer of the browser's print to PDF? Margins size, etc)
Yes we can. This is controlled by the Browser which allows you to customise this.
For the content of header/footer that’s easy, that would be the print.vm controlling them.
Could you please link me to some documentation (or element name or something) that would help me understand more precisely what technology we could use to achieve this? Thanks
* multipage pdf export (that we currently have in multiple forms) will have to be re-thought
Yes indeed. We would still be able to support this if we want by having our print.vm support it.
* will links continue to work properly as links inside the pdf ( I will need to test - actually, I was talking about links that link to another place in the PDF, but while testing I realized that my Firefox did not export any link as link in pdf - internal or external - it only exported text)
Actually that should work better than what we have now ;)
Note that we can do anything we want since we control the print.vm so we can use any XWikiServletURLFactory we want.
* as resulting from the previous small test but easy to generalize: the result of this feature will start to be browser dependent .
So we control the layout completely. Then indeed it’s up to the browser to print. I don’t know if various browser print thing differently and how much difference there are between them.
All I know is that they’ve spent way more men/hours than we’ve done on making a nice print ;)
So again my goal in this proposal is to benefit from the Print feature of browsers, externalise the feature as much as possible (similar to what we’ve done with CKEditor). I wouldn’t want to drop our PDF export since it has some use cases, for example for generating export from scripts.
I’m only referring to the basic Print option here and to transform the Print Preview into a Print one and tune it a bit to look as nice as possible. And make the Export to PDF optional and off by default.
Thanks -Vincent
For the problems of Apache FOP, does anyone have any idea what libraries do other software use for this kind of functionality? (maybe there is a more cool lib that we haven't found out about)
Anca
On Wed, Jan 18, 2017 at 2:54 PM, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
On Fri, Jan 20, 2017 at 2:10 PM, Vincent Massol <[email protected]> wrote: Hi Anca,
On 18 Jan 2017, at 18:05, Anca Luca <[email protected]> wrote:
Interesting, however, I see at least 2-3 possible limitations:
* we will drop cover and table of contents
Actually we won’t. The idea would be to have to a Print button inside XWiki (right now we have Print Preview) and we would add support for cover + TOC in print.vm.
* browsers (operating systems?) that don't have print to pdf option, will not have this working anymore
I think all OSes support this. Do you have any meaningful OS/Browser in mind? We could still leave it as an option to use our PDF Export if users need it. Windows <10 does not provide built-in print to PDF in IE Firefox for Windows/Linux needs an add-on to print to PDF These are just two examples, I am afraid you are optimistic about the availability of that feature. I am a Mac user like you, and I completely understand your point of view, but OS X is still the only OS fully integrating PDF support. Chrome browser has also a good deal of PDF support. But I am afraid we should also consider other situations. Maybe your proposal will be fine in a few years… but still I am sure to like it. I would like for the advantage in maintenance/development. But my current feeling is that our PDF generation server side is currently a differentiator for XWiki compared to competition. I am not sure, even if it was working smoothly that printing to PDF (which might just require a good CSS, not necessarily a custom page) really compare with downloading a server-side crafted PDF for end users. I am even convinced that a good PDF generation feature, with more easy customization, or even the ability to create Flipbooks would be a nice feature to have… However, I am no more confident that this would be easy to find. I have quickly made some research on currently available PDF libraries, and the conclusions are not matching my expectations. The most promising one was PDF Clown (http://www.stefanochizzolini.it/en/projects/clown/), which advertised a 2.0 version with what we really would like to have… but it had never been released and no commit since more than a year :( PDFBox which is a very mature PDF library is actually missing all the layout aspects, and experimentation made around it to convert HTML (see for example https://github.com/Jahia/html2pdf for example) does not look simple and so powerful. Wrapping wkhtmltopdf (which use Qt WebKit for rendering) is probably not a portable enough option, why not directly using LibreOffice to generate our PDF in that case. There is also pdfkit (http://pdfkit.org/) which is an emerging JavaScript library for generating PDF, but it is currently missing many layout aspect, just like PDFBox, and so not better. iText was already available when we choose FOP, it does not look like a better option today. So, even if this is not what I had expected, it seems that FOP is not that bad. Therefore, the proposal from Anca of applying the table patch might be an interesting option. We could also revamp the way we use FOP, in particular we might want to find better way to apply CSS to it, improve the validity of the generated XML and that way ensures better error reporting and recovery. So IMO, we should decide if we abandon our PDF feature or if we make some effort to make it right. I am +0 to make it right (not +1 simply because I will not have time for doing it even if I would like to see this happening).
* less customizations will be possible on this PDF (e.g. can we control header / footer of the browser's print to PDF? Margins size, etc)
Yes we can. This is controlled by the Browser which allows you to customise this. For the content of header/footer that’s easy, that would be the print.vm controlling them.
* multipage pdf export (that we currently have in multiple forms) will have to be re-thought
Yes indeed. We would still be able to support this if we want by having our print.vm support it.
* will links continue to work properly as links inside the pdf ( I will need to test - actually, I was talking about links that link to another place in the PDF, but while testing I realized that my Firefox did not export any link as link in pdf - internal or external - it only exported text)
Actually that should work better than what we have now ;) Note that we can do anything we want since we control the print.vm so we can use any XWikiServletURLFactory we want.
* as resulting from the previous small test but easy to generalize: the result of this feature will start to be browser dependent .
So we control the layout completely. Then indeed it’s up to the browser to print. I don’t know if various browser print thing differently and how much difference there are between them. All I know is that they’ve spent way more men/hours than we’ve done on making a nice print ;) So again my goal in this proposal is to benefit from the Print feature of browsers, externalise the feature as much as possible (similar to what we’ve done with CKEditor). I wouldn’t want to drop our PDF export since it has some use cases, for example for generating export from scripts. I’m only referring to the basic Print option here and to transform the Print Preview into a Print one and tune it a bit to look as nice as possible. And make the Export to PDF optional and off by default. Between improving the print feature and making the PDF better, I think that we might have a benefit doing just the opposite of what you propose, since having a good PDF implies having a good way to print. Of course, it implies that we do the improvement I mentioned above. So our preview could be the PDF export, and this would probably be nicer and more in line with what others do currently when it comes to printing. I have never been really happy with browser printing so far, it might improve, but today it is still really fragile. Thanks -Vincent
For the problems of Apache FOP, does anyone have any idea what libraries do other software use for this kind of functionality? (maybe there is a more cool lib that we haven't found out about)
Anca
On Wed, Jan 18, 2017 at 2:54 PM, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent -- Denis Gervalle SOFTEC sa - CEO
I think this is a good idea for the future, but not right now. There are several things that current browsers don't support yet, such as the ability to specify page size and orientation, setting custom headers and footers, and we'd need to work more to generate the cover and ToC. With FOP we can control exactly how things work in all browsers, while leaving this part to the browser is guaranteed to lead to inconsistent and buggy output. On 01/18/2017 08:54 AM, Vincent Massol wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
-- Sergiu Dumitriu http://purl.org/net/sergiu
On 18 Jan 2017, at 21:23, Sergiu Dumitriu <[email protected]> wrote:
I think this is a good idea for the future, but not right now. There are several things that current browsers don't support yet, such as the ability to specify page size and orientation
I’ve just tested it and I could select the orientation, paper size, margins, whether header/footers are printed, etc.
, setting custom headers and footers, and we'd need to work more to generate the cover and ToC. With FOP we can control exactly how things work in all browsers, while leaving this part to the browser is guaranteed to lead to inconsistent and buggy output.
Well that’s the problem and the reason why I sent this email: right now FOP doesn’t allow us to control much because it’s limited and buggy. We have plenty of open jira issues related to FOP and we’ve worked hard (too hard) to fix issues in the past and we’re stuck on several of them (I can think of table auto layout). When I’ve tried to export a wiki page both with the browser print button and with our PDF export, I’ve had a better result (by better I mean that corresponds more to what the user is viewing) with the browser print button. Of course our print.css is lacking a lot of styles but that’s easily fixable (BTW I don’t understand why we don’t use view styles by default, most of them should work just fine). The other advantage is that browsers work all the time to improve this and thus we delegate the work to them (we reduce our maintenance). Note that I’m not suggesting to drop what we’re doing now but to not make it the default since it’s of lower quality than the browser print result. What we could do right now is: * Run some factual tests on various wiki pages and compare them with both solutions and see which are best. * Quickly try to improve our print.css to match our style.css more. Thanks -Vincent
On 01/18/2017 08:54 AM, Vincent Massol wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
-- Sergiu Dumitriu http://purl.org/net/sergiu
On 01/18/2017 03:43 PM, Vincent Massol wrote:
On 18 Jan 2017, at 21:23, Sergiu Dumitriu <[email protected]> wrote:
I think this is a good idea for the future, but not right now. There are several things that current browsers don't support yet, such as the ability to specify page size and orientation
I’ve just tested it and I could select the orientation, paper size, margins, whether header/footers are printed, etc.
It's not about selecting them manually from the browser, but specifying it in the CSS. The CSS print module contains rules for this, but not all browsers implement them.
, setting custom headers and footers, and we'd need to work more to generate the cover and ToC. With FOP we can control exactly how things work in all browsers, while leaving this part to the browser is guaranteed to lead to inconsistent and buggy output.
Well that’s the problem and the reason why I sent this email: right now FOP doesn’t allow us to control much because it’s limited and buggy. We have plenty of open jira issues related to FOP and we’ve worked hard (too hard) to fix issues in the past and we’re stuck on several of them (I can think of table auto layout).
When I’ve tried to export a wiki page both with the browser print button and with our PDF export, I’ve had a better result (by better I mean that corresponds more to what the user is viewing) with the browser print button. Of course our print.css is lacking a lot of styles but that’s easily fixable (BTW I don’t understand why we don’t use view styles by default, most of them should work just fine).
The other advantage is that browsers work all the time to improve this and thus we delegate the work to them (we reduce our maintenance).
Note that I’m not suggesting to drop what we’re doing now but to not make it the default since it’s of lower quality than the browser print result.
"Better quality" is subjective... But with enough work on either side (pdf.css or print.css) we can improve the look of the PDF. The main problem is that nobody considered it a high enough priority to spend time on making the PDF look nice. At a hackathon during 2012's XWiki seminar, IIRC, there was some work for this, but I don't think that ever got committed. Long term, it does make sense to stop spending time on the FOP export and into browser's print stylesheet.
What we could do right now is: * Run some factual tests on various wiki pages and compare them with both solutions and see which are best. * Quickly try to improve our print.css to match our style.css more.
print.css shouldn't be an alternative to style.css, but something on top of it, improving certain things for printing.
Thanks -Vincent
On 01/18/2017 08:54 AM, Vincent Massol wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
-- Sergiu Dumitriu http://purl.org/net/sergiu
-- Sergiu Dumitriu http://purl.org/net/sergiu
Hi everyone, Thanks to everyone who answered on this topic and help brainstorm about it. I made a mistake sending this as a proposal; I should have sent it as a brainstorming instead. Note that I’ve also discussed with Anca yesterday and she told me that I haven’t been nice in the way I answered her questions (not explaining enough and throwing off her questions). I wanted to publicly apologise for this since this wasn’t my intention and when reading my answers again I can see why this was felt this way. Let me try to improve below :) I also failed to explain properly the context that led me to send this email: our roadmap for 9.0/9.1/9.2 had the table issue to fix (there was 2 days allocated to it) and Guillaume worked on analysing it and the conclusion was that it would take more than 2 days and would need to send some PR (or fork) FOP and thus this issue is now dropped for 9.0/9.1/9.2. I was trying to think out of the box and in order to provide a solution to Olivier Seres (who raised the issue recently) I proposed to him that he used the browser’s print feature and it did work better for this specific use case. So I thought to myself: maybe there’s some value in using this more. So let me try to recap what was said and reach a conclusion on this thread. Features needed: ============== * Table of contents. This is a difficult one. We can have a TOC generated in our print.vm. The hard part is finding out the corresponding page numbers for the use case when the user wants to print the PDF. It could maybe be solved by setting fixed margins/paper size/orientation. But it doesn’t look easy. I haven’t checked how it’s done in the PDF export though so I could be wrong. * Page headers/footers. This is also a difficult one because even though this is part of the css3-page spec, it’s not yet implemented by browsers and especially by chrome. There are plenty of people trying to do this when I googled it with various more or less successful options. A promising possibility was the position:fixed option but it has its own issues (chrome said they fixed the issue but not sure it’s true) and we’d need to make sure that the text really goes in the margin and not in the content. It’s possible that a combination of various solutions depending on the browser could work but it’s not perfect and it’s complex. * Margin sizes/orientation/paper size. This seems supported by the @page CSS element and I saw it was mentioned to be supported by chrome but I didn’t get to test it nor across browsers. To be tested. * Local links. This should be ok I think since in the print.vm we can control which XWikiServletURLFactory to use and generate anchors for local links. This is essentially what we’re already doing for HTML export in the ExportURLFactory class. However Anca raised the point that it doesn’t seem to work on FF. Would need to be checked further. * External links. I don’t think there’s any real issue here. I tried PDF-printing a web page with links and it worked fine. I think the reason it doesn’t work when doing this in xwiki could be because of our print.css which removes the display of “a”: #commentform, .xwikitabbar li, #Historyshortcut, #Informationshortcut, .separator, .commentheader .commenttools a, a.tag-delete , .tag-add a { display: none !important; } * Browser-dependent result. True. In addition we’d need to ensure that browser offer a print to PDF feature. I know this is true on Mac and also on FF in unix (on Thomas’s computer and I think Thomas told me he didn’t have any add on). Haven’t checked other combinations. Denis mentioned that it’s not the default in Windows < 10. Research ======== The following alternatives to FOP exist but have their own limitations (see Denis answer for details): * PDF Clown * PDFBox * wkhtmltopdf * pdfkit * iText * Flying saucer, see http://flyingsaucerproject.github.io/flyingsaucer/r8/guide/users-guide-R8.ht... Conclusions ========= I feel there are several main use cases: * Wanting to print the current page. The browser print feature could work fine for this if we were to improve a bit our print.css (inherit from view.css for example) * Wanting to have some professional quality PDF export. There’s no doubt that the best solution for this is ATM our FOP-based PDF export and w * Scripted PDF exports (to generate invoices automatically for example). Again our FOP-based PDF export is the best answer. So I agree with all of you who said that CSS3-page is not ready yet. The FOP2 spec has been stopped and the future seems to be CSS3-page but it’s not moving fast right now. So I propose that we: * Marginally improve our print.css for page printing * Continue to improve our FOP-based solution and possibly send PR to this project and if they’re not applied quick-enough, then fork it and took some ownership of it. However we just need to acknowledge that this is going to be substantial work and we still have a sizeable number of open issue for the PDF generation. Thanks -Vincent
On 18 Jan 2017, at 14:54, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
Hello Vincent, all Thanks for these clear responses. See below some more info, from my experience. On Thu, Jan 26, 2017 at 11:15 AM, Vincent Massol <[email protected]> wrote:
Hi everyone,
Thanks to everyone who answered on this topic and help brainstorm about it. I made a mistake sending this as a proposal; I should have sent it as a brainstorming instead.
Note that I’ve also discussed with Anca yesterday and she told me that I haven’t been nice in the way I answered her questions (not explaining enough and throwing off her questions). I wanted to publicly apologise for this since this wasn’t my intention and when reading my answers again I can see why this was felt this way. Let me try to improve below :)
I also failed to explain properly the context that led me to send this email: our roadmap for 9.0/9.1/9.2 had the table issue to fix (there was 2 days allocated to it) and Guillaume worked on analysing it and the conclusion was that it would take more than 2 days and would need to send some PR (or fork) FOP and thus this issue is now dropped for 9.0/9.1/9.2. I was trying to think out of the box and in order to provide a solution to Olivier Seres (who raised the issue recently) I proposed to him that he used the browser’s print feature and it did work better for this specific use case. So I thought to myself: maybe there’s some value in using this more.
So let me try to recap what was said and reach a conclusion on this thread.
Features needed: ==============
* Table of contents. This is a difficult one. We can have a TOC generated in our print.vm. The hard part is finding out the corresponding page numbers for the use case when the user wants to print the PDF. It could maybe be solved by setting fixed margins/paper size/orientation. But it doesn’t look easy. I haven’t checked how it’s done in the PDF export though so I could be wrong.
As far as I remember, this is relying on an xsl-fo feature, the same one that we use to obtain the number of printed pages (when we print pagination, in footer, like 1/3, 2/3).
* Page headers/footers. This is also a difficult one because even though this is part of the css3-page spec, it’s not yet implemented by browsers and especially by chrome. There are plenty of people trying to do this when I googled it with various more or less successful options. A promising possibility was the position:fixed option but it has its own issues (chrome said they fixed the issue but not sure it’s true) and we’d need to make sure that the text really goes in the margin and not in the content. It’s possible that a combination of various solutions depending on the browser could work but it’s not perfect and it’s complex.
* Margin sizes/orientation/paper size. This seems supported by the @page CSS element and I saw it was mentioned to be supported by chrome but I didn’t get to test it nor across browsers. To be tested.
* Local links. This should be ok I think since in the print.vm we can control which XWikiServletURLFactory to use and generate anchors for local links. This is essentially what we’re already doing for HTML export in the ExportURLFactory class. However Anca raised the point that it doesn’t seem to work on FF. Would need to be checked further.
* External links. I don’t think there’s any real issue here. I tried PDF-printing a web page with links and it worked fine. I think the reason it doesn’t work when doing this in xwiki could be because of our print.css which removes the display of “a”:
#commentform, .xwikitabbar li, #Historyshortcut, #Informationshortcut, .separator, .commentheader .commenttools a, a.tag-delete , .tag-add a { display: none !important; }
* Browser-dependent result. True. In addition we’d need to ensure that browser offer a print to PDF feature. I know this is true on Mac and also on FF in unix (on Thomas’s computer and I think Thomas told me he didn’t have any add on). Haven’t checked other combinations. Denis mentioned that it’s not the default in Windows < 10.
Research ========
The following alternatives to FOP exist but have their own limitations (see Denis answer for details): * PDF Clown * PDFBox * wkhtmltopdf * pdfkit * iText * Flying saucer, see http://flyingsaucerproject.github.io/flyingsaucer/r8/ guide/users-guide-R8.html#pdf
Conclusions =========
I feel there are several main use cases:
* Wanting to print the current page. The browser print feature could work fine for this if we were to improve a bit our print.css (inherit from view.css for example) * Wanting to have some professional quality PDF export. There’s no doubt that the best solution for this is ATM our FOP-based PDF export and w * Scripted PDF exports (to generate invoices automatically for example). Again our FOP-based PDF export is the best answer.
So I agree with all of you who said that CSS3-page is not ready yet. The FOP2 spec has been stopped and the future seems to be CSS3-page but it’s not moving fast right now.
So I propose that we: * Marginally improve our print.css for page printing * Continue to improve our FOP-based solution and possibly send PR to this project and if they’re not applied quick-enough, then fork it and took some ownership of it. However we just need to acknowledge that this is going to be substantial work and we still have a sizeable number of open issue for the PDF generation.
For tables layouting (XWIKI-5627), which seems to be our biggest problem right now, i've seen implemented, in the past, some control the sizes of the columns, but "manually" - as in, the size would not be computed automatically depending on the content of the table. Set in a certain way (I need to remember exactly how), the sizes of the columns can be imposed to the pdf. So, if table autolayout is such a pain (and there is serious proof to support that it is), maybe we can offer a intermediate solution, such as a "helper" for the user preparing the document: something like a macro or even a wiki syntax snippet or something, that users could use to control the sizes of the columns of a specific table. This would obviously not cover all cases, but I think it could cover the usecase of Olivier Seres (afaik the usecase is something like this: "he prepared a document for print, he wanted to print it and realized that the tables were incorrect on print". From my point of view, using this macro or snippet could be part of the preparation of a document for print, and we could have it by default in the wiki - if it's a macro - or as an FAQ if it's a snippet). Lemme know if you're interested in this idea, as I will need to try to find the code we used to do this. Thanks, Anca
Thanks -Vincent
On 18 Jan 2017, at 14:54, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
On 01/26/2017 05:59 AM, Anca Luca wrote:
Hello Vincent, all
Thanks for these clear responses. See below some more info, from my experience.
On Thu, Jan 26, 2017 at 11:15 AM, Vincent Massol <[email protected]> wrote:
Hi everyone,
Thanks to everyone who answered on this topic and help brainstorm about it. I made a mistake sending this as a proposal; I should have sent it as a brainstorming instead.
Note that I’ve also discussed with Anca yesterday and she told me that I haven’t been nice in the way I answered her questions (not explaining enough and throwing off her questions). I wanted to publicly apologise for this since this wasn’t my intention and when reading my answers again I can see why this was felt this way. Let me try to improve below :)
I also failed to explain properly the context that led me to send this email: our roadmap for 9.0/9.1/9.2 had the table issue to fix (there was 2 days allocated to it) and Guillaume worked on analysing it and the conclusion was that it would take more than 2 days and would need to send some PR (or fork) FOP and thus this issue is now dropped for 9.0/9.1/9.2. I was trying to think out of the box and in order to provide a solution to Olivier Seres (who raised the issue recently) I proposed to him that he used the browser’s print feature and it did work better for this specific use case. So I thought to myself: maybe there’s some value in using this more.
So let me try to recap what was said and reach a conclusion on this thread.
Features needed: ==============
* Table of contents. This is a difficult one. We can have a TOC generated in our print.vm. The hard part is finding out the corresponding page numbers for the use case when the user wants to print the PDF. It could maybe be solved by setting fixed margins/paper size/orientation. But it doesn’t look easy. I haven’t checked how it’s done in the PDF export though so I could be wrong.
As far as I remember, this is relying on an xsl-fo feature, the same one that we use to obtain the number of printed pages (when we print pagination, in footer, like 1/3, 2/3).
https://www.w3.org/TR/css-gcpm-3/#cross-references will do that, but no browser supports it yet.
* Page headers/footers. This is also a difficult one because even though this is part of the css3-page spec, it’s not yet implemented by browsers and especially by chrome. There are plenty of people trying to do this when I googled it with various more or less successful options. A promising possibility was the position:fixed option but it has its own issues (chrome said they fixed the issue but not sure it’s true) and we’d need to make sure that the text really goes in the margin and not in the content. It’s possible that a combination of various solutions depending on the browser could work but it’s not perfect and it’s complex.
* Margin sizes/orientation/paper size. This seems supported by the @page CSS element and I saw it was mentioned to be supported by chrome but I didn’t get to test it nor across browsers. To be tested.
* Local links. This should be ok I think since in the print.vm we can control which XWikiServletURLFactory to use and generate anchors for local links. This is essentially what we’re already doing for HTML export in the ExportURLFactory class. However Anca raised the point that it doesn’t seem to work on FF. Would need to be checked further.
No, what Anca said (and I can confirm) is that links don't work at all in a PDF printed by Firefox/Linux. They aren't links, they're just underlined plain text, unclickable. When printed from Chrome the URLs work fine, and there's no need to use a special URLFactory, Chrome correctly transforms relative URLs to absolute URLs. Firefox most likely optimized its code for paper printing, with the option to make an actual PDF that can be displayed on screen arriving late, so it's a second class citizen, built on top of a process that doesn't intend to generate an interactive document.
* External links. I don’t think there’s any real issue here. I tried PDF-printing a web page with links and it worked fine. I think the reason it doesn’t work when doing this in xwiki could be because of our print.css which removes the display of “a”:
No it doesn't, those are interactive JS-only tools that don't make sense in a PDF. It's not hiding all <a> elements, just the following special cases: .commentheader .commenttools a a.tag-delete .tag-add a
#commentform, .xwikitabbar li, #Historyshortcut, #Informationshortcut, .separator, .commentheader .commenttools a, a.tag-delete , .tag-add a { display: none !important; }
* Browser-dependent result. True. In addition we’d need to ensure that browser offer a print to PDF feature. I know this is true on Mac and also on FF in unix (on Thomas’s computer and I think Thomas told me he didn’t have any add on). Haven’t checked other combinations. Denis mentioned that it’s not the default in Windows < 10.
Research ========
The following alternatives to FOP exist but have their own limitations (see Denis answer for details): * PDF Clown * PDFBox * wkhtmltopdf * pdfkit * iText * Flying saucer, see http://flyingsaucerproject.github.io/flyingsaucer/r8/ guide/users-guide-R8.html#pdf
Conclusions =========
I feel there are several main use cases:
* Wanting to print the current page. The browser print feature could work fine for this if we were to improve a bit our print.css (inherit from view.css for example) * Wanting to have some professional quality PDF export. There’s no doubt that the best solution for this is ATM our FOP-based PDF export and w * Scripted PDF exports (to generate invoices automatically for example). Again our FOP-based PDF export is the best answer.
So I agree with all of you who said that CSS3-page is not ready yet. The FOP2 spec has been stopped and the future seems to be CSS3-page but it’s not moving fast right now.
So I propose that we: * Marginally improve our print.css for page printing * Continue to improve our FOP-based solution and possibly send PR to this project and if they’re not applied quick-enough, then fork it and took some ownership of it. However we just need to acknowledge that this is going to be substantial work and we still have a sizeable number of open issue for the PDF generation.
For tables layouting (XWIKI-5627), which seems to be our biggest problem right now, i've seen implemented, in the past, some control the sizes of the columns, but "manually" - as in, the size would not be computed automatically depending on the content of the table. Set in a certain way (I need to remember exactly how), the sizes of the columns can be imposed to the pdf. So, if table autolayout is such a pain (and there is serious proof to support that it is), maybe we can offer a intermediate solution, such as a "helper" for the user preparing the document: something like a macro or even a wiki syntax snippet or something, that users could use to control the sizes of the columns of a specific table. This would obviously not cover all cases, but I think it could cover the usecase of Olivier Seres (afaik the usecase is something like this: "he prepared a document for print, he wanted to print it and realized that the tables were incorrect on print". From my point of view, using this macro or snippet could be part of the preparation of a document for print, and we could have it by default in the wiki - if it's a macro - or as an FAQ if it's a snippet). Lemme know if you're interested in this idea, as I will need to try to find the code we used to do this.
Thanks, Anca
Thanks -Vincent
On 18 Jan 2017, at 14:54, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
-- Sergiu Dumitriu http://purl.org/net/sergiu
On 26 Jan 2017, at 17:21, Sergiu Dumitriu <[email protected]> wrote:
On 01/26/2017 05:59 AM, Anca Luca wrote:
Hello Vincent, all
Thanks for these clear responses. See below some more info, from my experience.
On Thu, Jan 26, 2017 at 11:15 AM, Vincent Massol <[email protected]> wrote:
Hi everyone,
Thanks to everyone who answered on this topic and help brainstorm about it. I made a mistake sending this as a proposal; I should have sent it as a brainstorming instead.
Note that I’ve also discussed with Anca yesterday and she told me that I haven’t been nice in the way I answered her questions (not explaining enough and throwing off her questions). I wanted to publicly apologise for this since this wasn’t my intention and when reading my answers again I can see why this was felt this way. Let me try to improve below :)
I also failed to explain properly the context that led me to send this email: our roadmap for 9.0/9.1/9.2 had the table issue to fix (there was 2 days allocated to it) and Guillaume worked on analysing it and the conclusion was that it would take more than 2 days and would need to send some PR (or fork) FOP and thus this issue is now dropped for 9.0/9.1/9.2. I was trying to think out of the box and in order to provide a solution to Olivier Seres (who raised the issue recently) I proposed to him that he used the browser’s print feature and it did work better for this specific use case. So I thought to myself: maybe there’s some value in using this more.
So let me try to recap what was said and reach a conclusion on this thread.
Features needed: ==============
* Table of contents. This is a difficult one. We can have a TOC generated in our print.vm. The hard part is finding out the corresponding page numbers for the use case when the user wants to print the PDF. It could maybe be solved by setting fixed margins/paper size/orientation. But it doesn’t look easy. I haven’t checked how it’s done in the PDF export though so I could be wrong.
As far as I remember, this is relying on an xsl-fo feature, the same one that we use to obtain the number of printed pages (when we print pagination, in footer, like 1/3, 2/3).
https://www.w3.org/TR/css-gcpm-3/#cross-references will do that, but no browser supports it yet.
* Page headers/footers. This is also a difficult one because even though this is part of the css3-page spec, it’s not yet implemented by browsers and especially by chrome. There are plenty of people trying to do this when I googled it with various more or less successful options. A promising possibility was the position:fixed option but it has its own issues (chrome said they fixed the issue but not sure it’s true) and we’d need to make sure that the text really goes in the margin and not in the content. It’s possible that a combination of various solutions depending on the browser could work but it’s not perfect and it’s complex.
* Margin sizes/orientation/paper size. This seems supported by the @page CSS element and I saw it was mentioned to be supported by chrome but I didn’t get to test it nor across browsers. To be tested.
* Local links. This should be ok I think since in the print.vm we can control which XWikiServletURLFactory to use and generate anchors for local links. This is essentially what we’re already doing for HTML export in the ExportURLFactory class. However Anca raised the point that it doesn’t seem to work on FF. Would need to be checked further.
No, what Anca said (and I can confirm) is that links don't work at all in a PDF printed by Firefox/Linux. They aren't links, they're just underlined plain text, unclickable. When printed from Chrome the URLs work fine, and there's no need to use a special URLFactory, Chrome correctly transforms relative URLs to absolute URLs.
Firefox most likely optimized its code for paper printing, with the option to make an actual PDF that can be displayed on screen arriving late, so it's a second class citizen, built on top of a process that doesn't intend to generate an interactive document.
* External links. I don’t think there’s any real issue here. I tried PDF-printing a web page with links and it worked fine. I think the reason it doesn’t work when doing this in xwiki could be because of our print.css which removes the display of “a”:
No it doesn't, those are interactive JS-only tools that don't make sense in a PDF. It's not hiding all <a> elements, just the following special cases:
.commentheader .commenttools a a.tag-delete .tag-add a
Ok then that seals it! The print to PDF is only for the print use case where links are not useful. The only discussion left now is the new one I posted about in the jira issue regarding flying saucer vs FOP: http://jira.xwiki.org/browse/XWIKI-5627?focusedCommentId=93983&page=com.atla... Thanks -Vincent
#commentform, .xwikitabbar li, #Historyshortcut, #Informationshortcut, .separator, .commentheader .commenttools a, a.tag-delete , .tag-add a { display: none !important; }
* Browser-dependent result. True. In addition we’d need to ensure that browser offer a print to PDF feature. I know this is true on Mac and also on FF in unix (on Thomas’s computer and I think Thomas told me he didn’t have any add on). Haven’t checked other combinations. Denis mentioned that it’s not the default in Windows < 10.
Research ========
The following alternatives to FOP exist but have their own limitations (see Denis answer for details): * PDF Clown * PDFBox * wkhtmltopdf * pdfkit * iText * Flying saucer, see http://flyingsaucerproject.github.io/flyingsaucer/r8/ guide/users-guide-R8.html#pdf
Conclusions =========
I feel there are several main use cases:
* Wanting to print the current page. The browser print feature could work fine for this if we were to improve a bit our print.css (inherit from view.css for example) * Wanting to have some professional quality PDF export. There’s no doubt that the best solution for this is ATM our FOP-based PDF export and w * Scripted PDF exports (to generate invoices automatically for example). Again our FOP-based PDF export is the best answer.
So I agree with all of you who said that CSS3-page is not ready yet. The FOP2 spec has been stopped and the future seems to be CSS3-page but it’s not moving fast right now.
So I propose that we: * Marginally improve our print.css for page printing * Continue to improve our FOP-based solution and possibly send PR to this project and if they’re not applied quick-enough, then fork it and took some ownership of it. However we just need to acknowledge that this is going to be substantial work and we still have a sizeable number of open issue for the PDF generation.
For tables layouting (XWIKI-5627), which seems to be our biggest problem right now, i've seen implemented, in the past, some control the sizes of the columns, but "manually" - as in, the size would not be computed automatically depending on the content of the table. Set in a certain way (I need to remember exactly how), the sizes of the columns can be imposed to the pdf. So, if table autolayout is such a pain (and there is serious proof to support that it is), maybe we can offer a intermediate solution, such as a "helper" for the user preparing the document: something like a macro or even a wiki syntax snippet or something, that users could use to control the sizes of the columns of a specific table. This would obviously not cover all cases, but I think it could cover the usecase of Olivier Seres (afaik the usecase is something like this: "he prepared a document for print, he wanted to print it and realized that the tables were incorrect on print". From my point of view, using this macro or snippet could be part of the preparation of a document for print, and we could have it by default in the wiki - if it's a macro - or as an FAQ if it's a snippet). Lemme know if you're interested in this idea, as I will need to try to find the code we used to do this.
Thanks, Anca
Thanks -Vincent
On 18 Jan 2017, at 14:54, Vincent Massol <[email protected]> wrote:
Hi devs,
We have plenty of existing limitations in our PDF export (table autoresize, etc) which comes from FOP and our usage of it. And it’s hard to improve it.
I’d like to propose the following:
* Promote the browser’s print to PDF feature instead of our PDF Export by triggering the browser’s print feature when clicking on Export > PDF in the UI. * Work on our print.css so that it has all the styles we have in view mode (right now for ex, info boxes for ex do not have a nice style). * Move our PDF Export java code out of xwiki-platform and into an optional extension that we move into xwiki-contrib. The use case for it would be when you need to generate PDF from scripts.
WDYT?
Thanks -Vincent
-- Sergiu Dumitriu http://purl.org/net/sergiu
participants (4)
-
Anca Luca -
Denis Gervalle -
Sergiu Dumitriu -
Vincent Massol