[xwiki-devs] [Brainstorming] Avoid / and \ in URLs to be nice with default Tomcat security? - xwiki-devs@xwiki.org

Thomas Mortagne

16 Nov 16 Nov

10:29 a.m.

I really don't like Option 2, it will only lead to way too much complexity and it's impossible to do it clean. URL already have encoding syntax and Tomcat should follow it as it's supposed to. It's much better and not very hard to finally add a first validator page in the DW to check things like Tomcat setting, memory allocation etc. On Mon, Nov 16, 2015 at 10:21 AM, vincent(a)massol.net <vincent(a)massol.net> wrote:

...

Hi guys, I think we need to an agreement on how to handle the default Tomcat security which disables the usage of / and \ in URLs (even URL-encoded). See http://www.tomcatexpert.com/blog/2011/11/02/best-practices-securing-apache-… We have 2 main options: * Option 1: Tell users to disable this security feature of Tomcat: http://tomcat.apache.org/tomcat-7.0-doc/config/systemprops.html#Security. In this case we just need to review our code to ensure we’re not subject to directory traversal attacks (see https://en.wikipedia.org/wiki/Directory_traversal_attack). * Option 2: Decide to make it easy for Tomcat users (since it’s probably the typical servlet container used by our users) and to not use / and \ in our URLs. Option 2 means modifying our code. There are various possibilities: * A) Replace the “/“ and “\” characters by other characters in URLs and modify our URL Serialization code (implementations of XWikiURLFactory) and our URL parsing code (URL modules). * B) Use a different encoding. Marius has used Base64 encoding for http://jira.xwiki.org/browse/XWIKI-11528. However this cannot be a generic solution since it leads to large URLs and also makes the URL not legible anymore. So this solution could only be for internal URLs. * Other? For A), it could b a character like ‘|' for ‘/' (and thus “||" if you want to have a real ‘|') and ‘~’ for ‘\’ (and “~~” if you want to have a real ‘\’). So there are 2 questions in this thread: * Do we want to be Tomcat-friendly? * If so, what strategy do we apply? WDYT? Thanks -Vincent _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

-- Thomas Mortagne

Reply

Thomas Mortagne

11:01 a.m.

On Mon, Nov 16, 2015 at 10:33 AM, vincent(a)massol.net <vincent(a)massol.net> wrote:

...

On 16 Nov 2015 at 10:29:11, Thomas Mortagne (thomas.mortagne(a)xwiki.com) wrote: I really don't like Option 2, it will only lead to way too much complexity and it's impossible to do it clean. URL already have encoding syntax and Tomcat should follow it as it's supposed to. It's much better and not very hard to finally add a first validator page in the DW to check things like Tomcat setting, memory allocation etc.

...

Yes indeed but we must not forget the security implications. This is not related to Tomcat actually. Tomcat simply does this to avoid directory traversal attacks in our code. So if we turn it off we need to review our code to make sure we’re not subject to this attack (i.e that we don’t use parts of the URL to construct a File object). Which we should do anyway since we support containers other than Tomcat… ;)

Yes we should do and we do in many places and it's not Tomcat job to take care of this.

...

Thanks -Vincent On Mon, Nov 16, 2015 at 10:21 AM, vincent(a)massol.net <vincent(a)massol.net> wrote:

Hi guys, I think we need to an agreement on how to handle the default Tomcat security which disables the usage of / and \ in URLs (even URL-encoded). See http://www.tomcatexpert.com/blog/2011/11/02/best-practices-securing-apache-… We have 2 main options: * Option 1: Tell users to disable this security feature of Tomcat: http://tomcat.apache.org/tomcat-7.0-doc/config/systemprops.html#Security. In this case we just need to review our code to ensure we’re not subject to directory traversal attacks (see https://en.wikipedia.org/wiki/Directory_traversal_attack). * Option 2: Decide to make it easy for Tomcat users (since it’s probably the typical servlet container used by our users) and to not use / and \ in our URLs. Option 2 means modifying our code. There are various possibilities: * A) Replace the “/“ and “\” characters by other characters in URLs and modify our URL Serialization code (implementations of XWikiURLFactory) and our URL parsing code (URL modules). * B) Use a different encoding. Marius has used Base64 encoding for http://jira.xwiki.org/browse/XWIKI-11528. However this cannot be a generic solution since it leads to large URLs and also makes the URL not legible anymore. So this solution could only be for internal URLs. * Other? For A), it could b a character like ‘|' for ‘/' (and thus “||" if you want to have a real ‘|') and ‘~’ for ‘\’ (and “~~” if you want to have a real ‘\’). So there are 2 questions in this thread: * Do we want to be Tomcat-friendly? * If so, what strategy do we apply? WDYT? Thanks -Vincent _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

-- Thomas Mortagne _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

-- Thomas Mortagne

Reply

vincent＠massol.net

11:53 a.m.

On 16 Nov 2015 at 11:35:27, Caleb James DeLisle (cjd(a)cjdns.fr) wrote: +1 for Option 2 or 2A As I understand, we're only talking about URL escaped / which both Tomcat and Apache httpd will "do things with", tomcat blocks it, Apache either blocks or *unescapes* it depending on setup. To be clear is both “/“ and “\” for Tomcat (see my mail). I don’t know about Apache HTTPD’s defaults (you have a pointer?). -Vincent Thanks, Caleb On 16/11/15 10:21, vincent(a)massol.net wrote:

...

Hi guys, I think we need to an agreement on how to handle the default Tomcat security which disables the usage of / and \ in URLs (even URL-encoded). See http://www.tomcatexpert.com/blog/2011/11/02/best-practices-securing-apache-… We have 2 main options: * Option 1: Tell users to disable this security feature of Tomcat: http://tomcat.apache.org/tomcat-7.0-doc/config/systemprops.html#Security. In this case we just need to review our code to ensure we’re not subject to directory traversal attacks (see https://en.wikipedia.org/wiki/Directory_traversal_attack). * Option 2: Decide to make it easy for Tomcat users (since it’s probably the typical servlet container used by our users) and to not use / and \ in our URLs. Option 2 means modifying our code. There are various possibilities: * A) Replace the “/“ and “\” characters by other characters in URLs and modify our URL Serialization code (implementations of XWikiURLFactory) and our URL parsing code (URL modules). * B) Use a different encoding. Marius has used Base64 encoding for http://jira.xwiki.org/browse/XWIKI-11528. However this cannot be a generic solution since it leads to large URLs and also makes the URL not legible anymore. So this solution could only be for internal URLs. * Other? For A), it could b a character like ‘|' for ‘/' (and thus “||" if you want to have a real ‘|') and ‘~’ for ‘\’ (and “~~” if you want to have a real ‘\’). So there are 2 questions in this thread: * Do we want to be Tomcat-friendly? * If so, what strategy do we apply? WDYT? Thanks -Vincent _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Reply

vincent＠massol.net

1:23 p.m.

Hi Clements, On 16 Nov 2015 at 12:36:08, Clemens Klein-Robbenhaar (c.robbenhaar@espresto.com(mailto:c.robbenhaar@espresto.com)) wrote:

...

From: Vincent | at: Mon, 16.11.2015, 11:53

On 16 Nov 2015 at 11:35:27, Caleb James DeLisle (cjd(a)cjdns.fr) wrote: +1 for Option 2 or 2A As I understand, we're only talking about URL escaped / which both Tomcat and Apache httpd will "do things with", tomcat blocks it, Apache either blocks or *unescapes* it depending on setup. To be clear is both “/“ and “\” for Tomcat (see my mail). I don’t know about Apache HTTPD’s defaults (you have a pointer?). -Vincent

Actually by default apache httpd does not allow encoded slashes at all: https://httpd.apache.org/docs/2.4/en/mod/core.html#allowencodedslashes Personally I prefer a third option: - disallow '/' and '\' in page names completely when creating / renaming pages - in the UI, if the user enters these characters to add (or rename), scrap them from the page name, or replace then by '-' (they will still show up in the title) and create a new page with the name without the slashes - make this configureable, like 'xwiki.pagename.forbiddenchars=/\\' and then people who are able to set up their servlet container to allow slashes can empty that config variable.

This covers only a portion of all the use cases, unless you also want to disable “.”, “:”, “@“, “^” characters in page names because if you don’t then they’ll be escaped in serialized references which can surface in the URL (as shown by http://jira.xwiki.org/browse/XWIKI-11528 and by the URL format I’m proposing for Zip URLs. The reference URL scheme is also affected obviously, see http://design.xwiki.org/xwiki/bin/view/Design/AlternateURLScheme). Thanks -Vincent

...

just an idea Clemens

Thanks, Caleb On 16/11/15 10:21, vincent(a)massol.net wrote:

Hi guys, I think we need to an agreement on how to handle the default Tomcat security which disables the usage of / and \ in URLs (even URL-encoded). See http://www.tomcatexpert.com/blog/2011/11/02/best-practices-securing-apache-… We have 2 main options: * Option 1: Tell users to disable this security feature of Tomcat: http://tomcat.apache.org/tomcat-7.0-doc/config/systemprops.html#Security. In this case we just need to review our code to ensure we’re not subject to directory traversal attacks (see https://en.wikipedia.org/wiki/Directory_traversal_attack). * Option 2: Decide to make it easy for Tomcat users (since it’s probably the typical servlet container used by our users) and to not use / and \ in our URLs. Option 2 means modifying our code. There are various possibilities: * A) Replace the “/“ and “\” characters by other characters in URLs and modify our URL Serialization code (implementations of XWikiURLFactory) and our URL parsing code (URL modules). * B) Use a different encoding. Marius has used Base64 encoding for http://jira.xwiki.org/browse/XWIKI-11528. However this cannot be a generic solution since it leads to large URLs and also makes the URL not legible anymore. So this solution could only be for internal URLs. * Other? For A), it could b a character like ‘|' for ‘/' (and thus “||" if you want to have a real ‘|') and ‘~’ for ‘\’ (and “~~” if you want to have a real ‘\’). So there are 2 questions in this thread: * Do we want to be Tomcat-friendly? * If so, what strategy do we apply? WDYT? Thanks -Vincent _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs _______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________ devs mailing list devs(a)xwiki.org http://lists.xwiki.org/mailman/listinfo/devs

Reply

Clemens Klein-Robbenhaar

1:50 p.m.

From: Vincent: At 16.11.2015, 13:23 [...]

...

Personally I prefer a third option: - disallow '/' and '\' in page names completely when creating / renaming pages - in the UI, if the user enters these characters to add (or rename), scrap them from the page name, or replace then by '-' (they will still show up in the title) and create a new page with the name without the slashes - make this configureable, like 'xwiki.pagename.forbiddenchars=/\\' and then people who are able to set up their servlet container to allow slashes can empty that config variable.

This covers only a portion of all the use cases, unless you also want to disable “.”, “:”, “@“, “^” characters in page names because if you don’t then they’ll be escaped in serialized references which can surface in the URL (as shown by http://jira.xwiki.org/browse/XWIKI-11528 and by the URL format I’m proposing for Zip URLs. The reference URL scheme is also affected obviously, see http://design.xwiki.org/xwiki/bin/view/Design/AlternateURLScheme).

I didn't mean to say that with my proposal no URL Encoding or other escaping need to happen at all, or no escaping in entity references. Of course something like that will need to happen, unless one has a pretty long list of "forbidden" stuff, which will not be user-friendly. The only part I tried to address is the problem that by default tomcat and apache's httpd do not allow slashes - encoded or not - in the URL. I am not sure why the other chars come in here now. If they get properly escaped / unescaped they should not cause problems. Clemens

Reply

Clemens Klein-Robbenhaar

2:10 p.m.

From: Vincent At: 16.11.2015, 13:55

...

On 16 Nov 2015 at 13:50:52, Clemens Klein-Robbenhaar (c.robbenhaar@espresto.com(mailto:c.robbenhaar@espresto.com)) wrote:

From: Vincent: At 16.11.2015, 13:23 [...]

Personally I prefer a third option: - disallow '/' and '\' in page names completely when creating / renaming pages - in the UI, if the user enters these characters to add (or rename), scrap them from the page name, or replace then by '-' (they will still show up in the title) and create a new page with the name without the slashes - make this configureable, like 'xwiki.pagename.forbiddenchars=/\\' and then people who are able to set up their servlet container to allow slashes can empty that config variable.

This covers only a portion of all the use cases, unless you also want to disable “.”, “:”, “@“, “^” characters in page names because if you don’t then they’ll be escaped in serialized references which can surface in the URL (as shown by http://jira.xwiki.org/browse/XWIKI-11528 and by the URL format I’m proposing for Zip URLs. The reference URL scheme is also affected obviously, see http://design.xwiki.org/xwiki/bin/view/Design/AlternateURLScheme).

I didn't mean to say that with my proposal no URL Encoding or other escaping need to happen at all, or no escaping in entity references. Of course something like that will need to happen, unless one has a pretty long list of "forbidden" stuff, which will not be user-friendly.

What I meant is that your proposal won’t help unless you also forbid those characters! :) On a personal note, I wouldn’t like to forbid characters in space or page names. We’ve worked hard to not have to do this.

I do not like it much either; that is why I proposed to have it configureable, and only contain the characters which are "absolutely necessary". It could even be empty and people have to turn it on if they do not want to configure their servlet containers properly.

...

The only part I tried to address is the problem that by default tomcat and apache's httpd do not allow slashes - encoded or not - in the URL. I am not sure why the other chars come in here now. If they get properly escaped / unescaped they should not cause problems.

I think you missed the fact that these extra characters that I mentioned need to be escaped when present in serialized references and the escape character in references is “\” :)

Yes, this part is what I have been missing. Indeed, if you want to use entity references in the URL and them to be "tomcat friendly", you need a different escape character in the URL, which means we are back at proposal 2.

...

Thanks -Vincent

Cheers Clemens

Reply