[xwiki-devs] [GSOC Update 3] RedPen Integration
Hi all! This is the third update regarding the progress of the Redpen integration project. Some of the info can also be accessed in the design page over here: <http://design.xwiki.org/xwiki/bin/view/Proposal/RedPenIntegration> I have created a UI on XWiki. The two main parts are such: First, the application homepage will allow users to key in possible spelling and expression errors, and their recommended corrections. I have not started on building the backend in Java to process this database yet, but I'll start on it within the next couple of weeks. Second, there is a setting page within the global administration wiki that allows the user to key in basic settings parameters e.g. setting maximum sentence length. The functionality is still restricted to validating a wiki page on document save, but I will be implementing the function to run redpen validation as a Job next, right after i finish fixing and stabilising some issues within the code. Any comments? -- View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255... Sent from the XWiki- Dev mailing list archive at Nabble.com.
Hi DeSheng,
On 27 Jun 2017, at 07:10, deshengchuan <[email protected]> wrote:
Hi all! This is the third update regarding the progress of the Redpen integration project. Some of the info can also be accessed in the design page over here: <http://design.xwiki.org/xwiki/bin/view/Proposal/RedPenIntegration>
I have created a UI on XWiki. The two main parts are such: First, the application homepage will allow users to key in possible spelling and expression errors, and their recommended corrections. I have not started on building the backend in Java to process this database yet, but I'll start on it within the next couple of weeks. Second, there is a setting page within the global administration wiki that allows the user to key in basic settings parameters e.g. setting maximum sentence length.
The functionality is still restricted to validating a wiki page on document save, but I will be implementing the function to run redpen validation as a Job next, right after i finish fixing and stabilising some issues within the code. Any comments?
Thanks. I think you’re on the right track and that you understand well the objectives and what needs to be done :) Some questions/remarks: * I still need to review your code in the coming few days * Does it work? Am I able to get a RedPen validation when I save a page? (I need to test your extension now) * In the ConfigurableClass, you should use a static list of Strings that lets the user enter the exclusions so that they can enter regex. Something similar to http://extensions.xwiki.org/xwiki/bin/view/Extension/Multiselect%20custom%20... I think this would be more powerful. IMO you should also have includes and not just excludes to cover all use cases. * I see that you have implemented a UI to allow entering some words to be corrected. Does it mean that the RedPen execution not only validates content but is also able to modify it? Thanks for the update! -Vincent
-- View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255... Sent from the XWiki- Dev mailing list archive at Nabble.com.
vmassol wrote
Some questions/remarks: * I still need to review your code in the coming few days * Does it work? Am I able to get a RedPen validation when I save a page? (I need to test your extension now) * In the ConfigurableClass, you should use a static list of Strings that lets the user enter the exclusions so that they can enter regex. Something similar to http://extensions.xwiki.org/xwiki/bin/view/Extension/Multiselect%20custom%20... I think this would be more powerful. IMO you should also have includes and not just excludes to cover all use cases. * I see that you have implemented a UI to allow entering some words to be corrected. Does it mean that the RedPen execution not only validates content but is also able to modify it?
Thanks for the update! -Vincent
Hi! Thanks in advance for the review :) I am still working through issues in my code one at a time, and I don't think it would be working today. I will keep you updated on it, and if need be I can commit an earlier version, without much of the UI capabilities, which validates plain text on a page on save and appends the results at the bottom of the page. After getting my code to work I'll edit the configurable class. I can see that includes and excludes would cover all use cases. I will work on editing the class such that the user can choose to include or exclude. RedPen can take in a dictionary of words or expressions(in JSON format), but it doesn't alter the entries directly. Rather, it would output a message along with the other validation messages, with this one suggesting the word or expression replacement. However, I think it is possible to implement a replacement function. Since the validation message provides the location of the error, we can extract the erroreous word or expression out and replace it with the corrected one. What do you think? -- View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255... Sent from the XWiki- Dev mailing list archive at Nabble.com.
On 27 Jun 2017, at 10:29, deshengchuan <[email protected]> wrote:
vmassol wrote
Some questions/remarks: * I still need to review your code in the coming few days * Does it work? Am I able to get a RedPen validation when I save a page? (I need to test your extension now) * In the ConfigurableClass, you should use a static list of Strings that lets the user enter the exclusions so that they can enter regex. Something similar to http://extensions.xwiki.org/xwiki/bin/view/Extension/Multiselect%20custom%20... I think this would be more powerful. IMO you should also have includes and not just excludes to cover all use cases. * I see that you have implemented a UI to allow entering some words to be corrected. Does it mean that the RedPen execution not only validates content but is also able to modify it?
Thanks for the update! -Vincent
Hi! Thanks in advance for the review :)
I am still working through issues in my code one at a time, and I don't think it would be working today. I will keep you updated on it, and if need be I can commit an earlier version, without much of the UI capabilities, which validates plain text on a page on save and appends the results at the bottom of the page.
After getting my code to work I'll edit the configurable class. I can see that includes and excludes would cover all use cases. I will work on editing the class such that the user can choose to include or exclude.
RedPen can take in a dictionary of words or expressions(in JSON format), but it doesn't alter the entries directly. Rather, it would output a message along with the other validation messages, with this one suggesting the word or expression replacement. However, I think it is possible to implement a replacement function. Since the validation message provides the location of the error, we can extract the erroreous word or expression out and replace it with the corrected one. What do you think?
I suggest that we focus on validation for the time being and get that to work nicely. When this is done, we can think about auto-fixing stuff. This would be doable in the listener, by changing the doc’s content before the save. The part that worries me the most ATM is how to display nicely to the user the result of the validation. I don’t see how we can do that for now. Do you have an idea? I think this could go hand in hand with a feature we’ve been wanting for a long time (can’t fine the jira right now): the ability to capture logs when rendering a page and display them as notifications in the UI. So the idea could be the following: * The RedPen listener would issue warnings for example in the logs * In the future when we implement the feature I mentioned, the user will be able to see them in the UI Do you have a better idea? I also think you need to include the concept of Validation Severity (i.e. warning, error). And when it’s a warning it should never stop saving the page. Only errors would do that. WDYT? Thanks -Vincent
-- View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255... Sent from the XWiki- Dev mailing list archive at Nabble.com.
I’ve started reviewing the code and there are some issues I believe. For example we need to agree on how to handle the configuration. Generally speaking and in order to make sure you go in the right direction, I think you should regularly send implementation proposals to this list about how you’re planning to implement something so that we can help you and tell how how to do it in XWiki. Regarding configuration, I’ve tried to put some info in comments. Thanks -Vincent
On 28 Jun 2017, at 16:35, Vincent Massol <[email protected]> wrote:
On 27 Jun 2017, at 10:29, deshengchuan <[email protected]> wrote:
vmassol wrote
Some questions/remarks: * I still need to review your code in the coming few days * Does it work? Am I able to get a RedPen validation when I save a page? (I need to test your extension now) * In the ConfigurableClass, you should use a static list of Strings that lets the user enter the exclusions so that they can enter regex. Something similar to http://extensions.xwiki.org/xwiki/bin/view/Extension/Multiselect%20custom%20... I think this would be more powerful. IMO you should also have includes and not just excludes to cover all use cases. * I see that you have implemented a UI to allow entering some words to be corrected. Does it mean that the RedPen execution not only validates content but is also able to modify it?
Thanks for the update! -Vincent
Hi! Thanks in advance for the review :)
I am still working through issues in my code one at a time, and I don't think it would be working today. I will keep you updated on it, and if need be I can commit an earlier version, without much of the UI capabilities, which validates plain text on a page on save and appends the results at the bottom of the page.
After getting my code to work I'll edit the configurable class. I can see that includes and excludes would cover all use cases. I will work on editing the class such that the user can choose to include or exclude.
RedPen can take in a dictionary of words or expressions(in JSON format), but it doesn't alter the entries directly. Rather, it would output a message along with the other validation messages, with this one suggesting the word or expression replacement. However, I think it is possible to implement a replacement function. Since the validation message provides the location of the error, we can extract the erroreous word or expression out and replace it with the corrected one. What do you think?
I suggest that we focus on validation for the time being and get that to work nicely. When this is done, we can think about auto-fixing stuff. This would be doable in the listener, by changing the doc’s content before the save.
The part that worries me the most ATM is how to display nicely to the user the result of the validation. I don’t see how we can do that for now. Do you have an idea? I think this could go hand in hand with a feature we’ve been wanting for a long time (can’t fine the jira right now): the ability to capture logs when rendering a page and display them as notifications in the UI.
So the idea could be the following: * The RedPen listener would issue warnings for example in the logs * In the future when we implement the feature I mentioned, the user will be able to see them in the UI
Do you have a better idea?
I also think you need to include the concept of Validation Severity (i.e. warning, error). And when it’s a warning it should never stop saving the page. Only errors would do that.
WDYT?
Thanks -Vincent
-- View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255... Sent from the XWiki- Dev mailing list archive at Nabble.com.
Hi, sorry for the late reply, was spending the time doing up the configuration handler over the rest of last week. vmassol wrote
The part that worries me the most ATM is how to display nicely to the user the result of the validation. I don’t see how we can do that for now. Do you have an idea? I think this could go hand in hand with a feature we’ve been wanting for a long time (can’t fine the jira right now): the ability to capture logs when rendering a page and display them as notifications in the UI.
So the idea could be the following: * The RedPen listener would issue warnings for example in the logs * In the future when we implement the feature I mentioned, the user will be able to see them in the UI
I was looking around, and do you think for now, we could use a modal popup to indicate the errors and validation messages, aside from issuing warnings in the logs? vmassol wrote
I also think you need to include the concept of Validation Severity (i.e. warning, error). And when it’s a warning it should never stop saving the page. Only errors would do that.
As far as I know, whether the validation is a warning or an error depends on the ValidatorConfiguration object itself, for example, a SentenceLength validator gives out a warning message while a SuccessiveWord validator gives out an error. I will work on it right away to split the error-giving validators and the warning-giving validators. For now, I'll just put all the validation messages on the logs. Is that okay? -- View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255... Sent from the XWiki- Dev mailing list archive at Nabble.com.
participants (2)
-
deshengchuan -
Vincent Massol