[xwiki-devs] [Feedback] Search Requirements
Hi, We introduced in XWiki 4.3 a new experimental search based on Apache Solr [A] . Right now our Solr Search [B] is marked as experimental and I don't know how many of you got a chance to play a bit with it. In order for it to become default we need to make sure we are covering the major use cases. So in this mail we should provide some feedback on what is really needed from a relevance, user experience, performance, etc. point of view. I am very interested also in some general feedback on the Lucene Search: major problems you had with it and limitations. The following questions are mostly related to Solr Search: 1) The Solr implementation adds a 'Filtered Search'. Are all filters that are available now needed (wiki, space, type, filetype)? Are we missing some? What is the most important one? Should we have multiple select?
From a technical point of view we could add a lot of things (object properties, query boost, etc.) but I'm more interested, in production, what are some advanced or common use cases of search usage. IMO our Lucene implementation is too simplistic, but it would be a shame to add useless complexity to the Solr one if we really don't need it. On the other hand having a customizable search is a really nice thing.
The customization part is kind of complex, but also the most powerful. I want to know how much would we need the ability to explicitly mention the types of results we want vs. excluding things, etc. 2) The Solr search adds a 'Sort' feature. What do you think about that? 3) Search result metadatas: should we display the relevance? What other information should be displayed besides name, location, type? 4) Other mentions. Having a generic search is more complex, than knowing exactly what your search needs to return. We need to make sure we don't add too many niche things while keeping the functionality satisfiable for general use cases. Thanks, Caty --------------- [A] http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki43#HExperi... [B] http://extensions.xwiki.org/xwiki/bin/view/Extension/Solr+Search+Application
Hi Caty, thanks for your email. Here is a list of the main complaints I have heard recently concerning the existing search: - We enter a word and it does not find similar words: for instance searching for "café" does not return docs containing "cafe" or "cafés" - An attached document is ranked before the XWiki document for a given search - We cannot control the weight of various page elements: how can we give more weight to the title and tags versus the content of a page or an attachment? - We search for a word that we know is in a page and yet that page doesn't show up -> usually after an indexing issue. Making sure the indexing is always up and doesn't break for new or updated documents is a priority - We want to get results only for a subset of pages: only XWiki documents, only attachments, everything but images, only in a given space => this means the ability to use and mix filters - The search is returning docs with a lot of noise: maybe we remove the results with a very low relevance score (under 5% for instance)? - About relevance: I don't think displaying it is really required. Intuitively, users know that the ranking of results is a measure of relevance (most relevant docs are on the first page, near the top) That's it for a first list. What would be needed to investigate more are real-life examples of queries that users were expecting to work but that didn't. I'll try to find some and provide them to you. Thanks, Guillaume On Wed, Apr 24, 2013 at 10:52 AM, Ecaterina Moraru (Valica) < [email protected]> wrote:
Hi,
We introduced in XWiki 4.3 a new experimental search based on Apache Solr [A] .
Right now our Solr Search [B] is marked as experimental and I don't know how many of you got a chance to play a bit with it.
In order for it to become default we need to make sure we are covering the major use cases. So in this mail we should provide some feedback on what is really needed from a relevance, user experience, performance, etc. point of view.
I am very interested also in some general feedback on the Lucene Search: major problems you had with it and limitations.
The following questions are mostly related to Solr Search:
1) The Solr implementation adds a 'Filtered Search'. Are all filters that are available now needed (wiki, space, type, filetype)? Are we missing some? What is the most important one? Should we have multiple select?
From a technical point of view we could add a lot of things (object properties, query boost, etc.) but I'm more interested, in production, what are some advanced or common use cases of search usage. IMO our Lucene implementation is too simplistic, but it would be a shame to add useless complexity to the Solr one if we really don't need it. On the other hand having a customizable search is a really nice thing.
The customization part is kind of complex, but also the most powerful. I want to know how much would we need the ability to explicitly mention the types of results we want vs. excluding things, etc.
2) The Solr search adds a 'Sort' feature. What do you think about that?
3) Search result metadatas: should we display the relevance? What other information should be displayed besides name, location, type?
4) Other mentions.
Having a generic search is more complex, than knowing exactly what your search needs to return. We need to make sure we don't add too many niche things while keeping the functionality satisfiable for general use cases.
Thanks, Caty
--------------- [A]
http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki43#HExperi... [B]
http://extensions.xwiki.org/xwiki/bin/view/Extension/Solr+Search+Application _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Guillaume write a perfect synthesis of the mains complains / improvments suggestions I have personnaly experimented while using XWiki / Lucene search in an Wiki running XWiki 3.1 . regards, Le 24/04/2013 15:16, Guillaume Lerouge a écrit :
Hi Caty,
thanks for your email. Here is a list of the main complaints I have heard recently concerning the existing search:
- We enter a word and it does not find similar words: for instance searching for "café" does not return docs containing "cafe" or "cafés" - An attached document is ranked before the XWiki document for a given search - We cannot control the weight of various page elements: how can we give more weight to the title and tags versus the content of a page or an attachment? - We search for a word that we know is in a page and yet that page doesn't show up -> usually after an indexing issue. Making sure the indexing is always up and doesn't break for new or updated documents is a priority - We want to get results only for a subset of pages: only XWiki documents, only attachments, everything but images, only in a given space => this means the ability to use and mix filters - The search is returning docs with a lot of noise: maybe we remove the results with a very low relevance score (under 5% for instance)? - About relevance: I don't think displaying it is really required. Intuitively, users know that the ranking of results is a measure of relevance (most relevant docs are on the first page, near the top)
That's it for a first list. What would be needed to investigate more are real-life examples of queries that users were expecting to work but that didn't. I'll try to find some and provide them to you.
Thanks,
Guillaume
On Wed, Apr 24, 2013 at 10:52 AM, Ecaterina Moraru (Valica) < [email protected]> wrote:
Hi,
We introduced in XWiki 4.3 a new experimental search based on Apache Solr [A] .
Right now our Solr Search [B] is marked as experimental and I don't know how many of you got a chance to play a bit with it.
In order for it to become default we need to make sure we are covering the major use cases. So in this mail we should provide some feedback on what is really needed from a relevance, user experience, performance, etc. point of view.
I am very interested also in some general feedback on the Lucene Search: major problems you had with it and limitations.
The following questions are mostly related to Solr Search:
1) The Solr implementation adds a 'Filtered Search'. Are all filters that are available now needed (wiki, space, type, filetype)? Are we missing some? What is the most important one? Should we have multiple select?
From a technical point of view we could add a lot of things (object properties, query boost, etc.) but I'm more interested, in production, what are some advanced or common use cases of search usage. IMO our Lucene implementation is too simplistic, but it would be a shame to add useless complexity to the Solr one if we really don't need it. On the other hand having a customizable search is a really nice thing.
The customization part is kind of complex, but also the most powerful. I want to know how much would we need the ability to explicitly mention the types of results we want vs. excluding things, etc.
2) The Solr search adds a 'Sort' feature. What do you think about that?
3) Search result metadatas: should we display the relevance? What other information should be displayed besides name, location, type?
4) Other mentions.
Having a generic search is more complex, than knowing exactly what your search needs to return. We need to make sure we don't add too many niche things while keeping the functionality satisfiable for general use cases.
Thanks, Caty
--------------- [A]
http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki43#HExperi... [B]
http://extensions.xwiki.org/xwiki/bin/view/Extension/Solr+Search+Application _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Hi, I generally agree with Guillaume. I've heard this too a few times recently: "We search for a word that we know is in a page and yet that page doesn't show up". Additionally it would be great to be able to filter results by language. People have asked me before why the search results were mixed up and how they could perform a search only for a certain language. One other thing I'd find useful is the ability to filter results by author / person who edited the document. This way if I am particularly interested in a document created / modified by someone I can easily find it. Thanks, Silvia On Fri, May 10, 2013 at 10:24 AM, Ldm Public <[email protected]> wrote:
Guillaume write a perfect synthesis of the mains complains / improvments suggestions I have personnaly experimented while using XWiki / Lucene search in an Wiki running XWiki 3.1 .
regards,
Le 24/04/2013 15:16, Guillaume Lerouge a écrit :
Hi Caty,
thanks for your email. Here is a list of the main complaints I have heard recently concerning the existing search:
- We enter a word and it does not find similar words: for instance searching for "café" does not return docs containing "cafe" or "cafés" - An attached document is ranked before the XWiki document for a given search - We cannot control the weight of various page elements: how can we give more weight to the title and tags versus the content of a page or an attachment? - We search for a word that we know is in a page and yet that page doesn't show up -> usually after an indexing issue. Making sure the indexing is always up and doesn't break for new or updated documents is a priority - We want to get results only for a subset of pages: only XWiki documents, only attachments, everything but images, only in a given space => this means the ability to use and mix filters - The search is returning docs with a lot of noise: maybe we remove the results with a very low relevance score (under 5% for instance)? - About relevance: I don't think displaying it is really required. Intuitively, users know that the ranking of results is a measure of relevance (most relevant docs are on the first page, near the top)
That's it for a first list. What would be needed to investigate more are real-life examples of queries that users were expecting to work but that didn't. I'll try to find some and provide them to you.
Thanks,
Guillaume
On Wed, Apr 24, 2013 at 10:52 AM, Ecaterina Moraru (Valica) < [email protected]> wrote:
Hi,
We introduced in XWiki 4.3 a new experimental search based on Apache Solr [A] .
Right now our Solr Search [B] is marked as experimental and I don't know how many of you got a chance to play a bit with it.
In order for it to become default we need to make sure we are covering the major use cases. So in this mail we should provide some feedback on what is really needed from a relevance, user experience, performance, etc. point of view.
I am very interested also in some general feedback on the Lucene Search: major problems you had with it and limitations.
The following questions are mostly related to Solr Search:
1) The Solr implementation adds a 'Filtered Search'. Are all filters that are available now needed (wiki, space, type, filetype)? Are we missing some? What is the most important one? Should we have multiple select?
From a technical point of view we could add a lot of things (object properties, query boost, etc.) but I'm more interested, in production, what are some advanced or common use cases of search usage. IMO our Lucene implementation is too simplistic, but it would be a shame to add useless complexity to the Solr one if we really don't need it. On the other hand having a customizable search is a really nice thing.
The customization part is kind of complex, but also the most powerful. I want to know how much would we need the ability to explicitly mention the types of results we want vs. excluding things, etc.
2) The Solr search adds a 'Sort' feature. What do you think about that?
3) Search result metadatas: should we display the relevance? What other information should be displayed besides name, location, type?
4) Other mentions.
Having a generic search is more complex, than knowing exactly what your search needs to return. We need to make sure we don't add too many niche things while keeping the functionality satisfiable for general use cases.
Thanks, Caty
--------------- [A]
http://www.xwiki.org/xwiki/**bin/view/ReleaseNotes/** ReleaseNotesXWiki43#**HExperimentalSolrbasedsearchen**gine<http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki43#HExperimentalSolrbasedsearchengine> [B]
http://extensions.xwiki.org/**xwiki/bin/view/Extension/Solr+** Search+Application<http://extensions.xwiki.org/xwiki/bin/view/Extension/Solr+Search+Application> ______________________________**_________________ devs mailing list [email protected] http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
______________________________**_________________ devs mailing list [email protected] http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
______________________________**_________________ devs mailing list [email protected] http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
participants (4)
-
Ecaterina Moraru (Valica) -
Guillaume Lerouge -
Ldm Public -
Silvia Rusu