[xwiki-devs] Lucene - external indexing
Would it be possible to add an external directory for lucene to crawl and index? I want to be able to search wiki content, including the attachments, and also incorporate external search results as well. I was taking a look at Solr, and might try to go that route, despite being intimidated a bit. It looks extremely cool and well suited. Gracias! ________________________________ This communication (including all attachments) is intended solely for the use of the person(s) to whom it is addressed and should be treated as a confidential AAA communication. If you are not the intended recipient, any use, distribution, printing, or copying of this email is strictly prohibited. If you received this email in error, please immediately delete it from your system and notify the originator. Your cooperation is appreciated.
Hi Dan, On Aug 17, 2012, at 1:56 AM, Jones, Dan wrote:
Would it be possible to add an external directory for lucene to crawl and index?
I want to be able to search wiki content, including the attachments, and also incorporate external search results as well. I was taking a look at Solr, and might try to go that route, despite being intimidated a bit. It looks extremely cool and well suited.
Note that attachments are already indexed. Now the way we use Lucene is by having an indexing thread with a Queue and when a wiki page is modified it's put on the queue for indexing. To support an external directory containing arbitrary data you'll need to write code to index that data and integrate it with the XWiki Lucene plugin code. Note that we have a GSOC in progress about integrating SOLR: http://dev.xwiki.org/xwiki/bin/view/GoogleSummerOfCode/SOLRsearchcomponent Thanks -Vincent
Dan, Le 17 août 2012 à 09:16, Vincent Massol a écrit :
Note that we have a GSOC in progress about integrating SOLR: http://dev.xwiki.org/xwiki/bin/view/GoogleSummerOfCode/SOLRsearchcomponent
if you're ready to go on development, I'd start with the solr plugin. Though I may be wrong and it is probably configurable, I think the solr plugin is indexing separately attachments and pages so you're close to what you want. And with some great chance an attachments folder with the file-system-storage could actually be populated with any file. You're left with detecting changed files and requesting re-indexing. Alternatively, but that requires diving into some code, Solr has a few crawling utilities, in particular DIH, and I think this would satisfy your need. However this would put "non xwiki" documents into the Lucene index, so the results display would need to be adjusted to take this into account. Paul
participants (3)
-
Jones, Dan -
Paul Libbrecht -
Vincent Massol