[xwiki-users] Usage and placement of robots.txt file

Hi, in the Xwiki documentation for the robots.txt file it says to put it in the webserver configuration. http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances#HRobots.txt On Tomcat where would this go? - Directly on the webapps/ROOT/ directory?? Also the directives used it claims: # It could be also usefull to block certain spaces from crawling, # especially if this spaces doesn't provide new content Should the: /xwiki/bin/view/Photos/ portion also be excluded?? Just as a last thing, what kind of performance benefits would be adhered to by stopping crawlers? I am imagining: CPU, RAM, Network B/W........ Regards, Kaya