[xwiki-users] Using a Google search appliance with xwiki
I was able to quickly integrate our lab environment's Google Mini Appliance with xwiki today... The set-up of the appliance was simple (after some experimentation on what to filter out to reduce redundancy and confusion); 1. List of urls to crawl (e.g http://hostname.domain:8080/xwiki ) 2. List of patterns to follow (e.g. hostname.domain:8080/xwiki) 3. List of patterns to NOT crawl - I added to the default list the following a. contains:?viewer=code b. contains:?format=rtf c. contains:?format=pdf d. contains:?tag= e. contains:?xpage=print f. contains:?rev= and then let it start crawling... I added the following to a page: {image:Google.gif} ---- <!-- Search Google Appliance --> <form method="get" action="http://googleappliance.domain/search"> <table> <tr> <td> <input type="text" name="q" size="25" maxlength="255" value=""/> <input type="submit" name="btnG" value="Google Search"/> <input type="hidden" name="site" value="default_collection"/> <input type="hidden" name="client" value="default_frontend"/> <input type="hidden" name="proxystylesheet" value="default_frontend"/> <input type="hidden" name="output" value="xml_no_dtd"/> </td> </tr> </table> </form> <!-- Search Google Appliance--> ... and it was up and running with an inline Google Search. It indexed the pages and attachments quickly, and it's a pleasure to use. The Lucene engine will still be useful to us, because it permits filtering and understands the structure (spaces, authors) and we have adapted it for our reputation engine. -- Jim Dowson CTO, Global Services, EMC Corporation Linx: (617) 598-0505
Hi Jim, thanks for the feedback.
I was able to quickly integrate our lab environment's Google Mini Appliance with xwiki today…
The set-up of the appliance was simple (after some experimentation on what to filter out to reduce redundancy and confusion);
1. List of urls to crawl (e.g http://hostname.domain:8080/xwiki )
2. List of patterns to follow (e.g. hostname.domain:8080/xwiki)
3. List of patterns to NOT crawl – I added to the default list the following
a. contains:?viewer=code
b. contains:?format=rtf
c. contains:?format=pdf
d. contains:?tag=
e. contains:?xpage=print
f. contains:?rev=
I don't think there's a risk, but you may want to add "contains:delete", "contains:edit", "contains:inline" & "contains:?editor=" to your list... At worse it will make indexing faster. Guillaume
Cool Jim! Thanks for sharing this. It would be great if you could add a cod snippet on code.xwiki.org that describes this. Thanks -Vincent On Mar 13, 2008, at 2:06 PM, Guillaume Lerouge wrote:
Hi Jim, thanks for the feedback. I was able to quickly integrate our lab environment's Google Mini Appliance with xwiki today…
The set-up of the appliance was simple (after some experimentation on what to filter out to reduce redundancy and confusion);
1. List of urls to crawl (e.g http://hostname.domain:8080/ xwiki )
2. List of patterns to follow (e.g. hostname.domain:8080/xwiki)
3. List of patterns to NOT crawl – I added to the default list the following
a. contains:?viewer=code
b. contains:?format=rtf
c. contains:?format=pdf
d. contains:?tag=
e. contains:?xpage=print
f. contains:?rev=
I don't think there's a risk, but you may want to add "contains:delete", "contains:edit", "contains:inline" & "contains:? editor=" to your list... At worse it will make indexing faster.
Guillaume
Guillaume Lerouge [mailto:[email protected]] writes:
I don't think there's a risk, but you may want to add "contains:delete", "contains:edit", "contains:inline" & "contains:?editor=" to your list... At worse it will make indexing faster.
I understand the ?editor= pattern. I'm concerned that first three might wildcard exclude too much. Can you provide additional guidance on the url pattern so that a page called "delete%20an%object" is not included, for example? Thanks, Guillaume. -- Jim Dowson CTO, Global Services, (821-76814) Linx: (617) 598-0505
Re,
I don't think there's a risk, but you may want to add
"contains:delete", "contains:edit", "contains:inline" & "contains:?editor=" to your list... At worse it will make indexing faster.
I understand the ?editor= pattern.
I'm concerned that first three might wildcard exclude too much.
Can you provide additional guidance on the url pattern so that a page called "delete%20an%object" is not included, for example?
delete, edit & inline are usually used in urls such as "xwiki/bin/delete/SpaceName/PageName", so you could use "contains: bin/edit" bin/delete , bin/inline , bin/objectremove and so on... If using the "/" sign doesn't break the googlesearch syntax it should work fine. Guillaume
participants (3)
-
Dowson_Jim＠emc.com -
Guillaume Lerouge -
Vincent Massol