Hi, Ideas similar to the one described below were discussed when working on XWiki's new search interface. Basically, I could summarize by saying that the things you label as "limitations" are actually conscious design choices ;-) Here are my comments: On Wed, Jun 17, 2009 at 10:40 AM, Vincent Massol <[email protected]> wrote:

Hi,

I think the new search results page (the main search one not the lucene one) has some limitations that could be improved: - it doesn't scale with large number of documents (if you do a search that returns a lot of documents)

Indeed. I've suggested a solution below

- it doesn't allow refining your search easily

Caty & I investigated this question. There are various options that allow tackling this issue. Here are a couple: - Add filters in columns (what the livetable currently does) - Add a single filter for all data (what iTunes currently does) - Add a "search within results" field (what Google currently does) It's interesting to note than when using that later field in Google, it runs a query that adds your additional words to the first query you ran: 1. You run a search on "XWiki Development" 2. You choose to search for "API" within those results 3. Google will return results for a search on "XWiki Development API" Users are currently familiar with the concept of refining their search query when the first batch of results they get isn't satisfying enough. Thus we thought that the best option was to provide users with the interface and the way of searching they were most familiar with. I definitely agree on the fact that as a power user that's not my ideal way of searching (I like the iTunes paradigm much better), but the default search interface aims at the majority of users and - the information presented take up more vertical space than needed

(currently 3 lines instead of one) and thus less results can be presented

This is not correct. Most of the time, people click on one of the top 3 search results they get on a page. This means they do not scroll much. 2 things really matter: - Each result has to encapsulate enough of the right information to let the user know whether the result matches his/her expectations - Each result be clearly identifiable and separated from the others This was not the case with the previous table-based interface. A table makes it harder to identify the different results. The current view improves contrast by turning each result in an easily scannable nugget of information. However, I do agree that the current view isn't enough. What we wanted to do was to add an extract of the page's relevant content (content matching the initial query), the way Google does it (they recently beefed up that feature to display longer content extracts btw). This will make it easier for an user to know whether the page behind a given result is what he / she is looking for. The "vertical space" argument assumes that the search will return a lot of results that the user will scroll through. The best way to answer it is to provide more relevant results on top to avoid the need of scrolling altogether. - it could be reused by other pages which require listing documents

such as the SpaceIndex which lists all pages in a given space. It's actually currently reused but as a consequence the SpaceIndex page isn't nice at all and doesn't scale.

About using the page to display results, I think the right approach to is is actually to build several data displayers that can be called by an application whenever it needs it. It would mean splitting the current XWiki.Results in several different pages that will address different needs. The way results are best presented in a search interface is not necessarily the way results are presented best. Therefore I think what we really need to do is to write different displayers: - A livetable displayer (macro) for data where the livetable is the best option - A list results displayer (macro) for search results - We could actually go as far as writing a search macro that would take object & space as parameters and return results only for documents having a given object / located in a given space -> that would be pretty useful for application developers - A widget displayer (macro) to display 5 items in a panel-like fashion - etc... This way users & developers can choose the displayer that makes the more sense to the application they're using / writing instead of having a single-view-fits-all-use-cases approach. Would it make sense to:

- have a livetable - display all docs by default when you arrive on the search page (ie no filter set)

2 comments: - The index page already does this and I think it's better to keep it separated from the search - The user expectation when arriving on a search page is that no results should be present when no query has been entered rather than the other way round - make the current search field a field bound to the livetable and

that filters that table when text is typed inside

That's what I referred to as the "iTunes-like" way of doing things above. Though I personally like it, Laurent convinced me that it's not what the majority of people are expecting online, specifically when it comes to search.

- remove the need for the spaces combo box since it would be a filter field in the livetable already

- same for the wiki field Ideally, results returned by the search right away are relevant enough to remove the need to filter by space / wiki. However, since we don't live in an ideal world, if we do not use the livetable we'd have to keep those fields. Pros:

- all the limitations listed above are resolved - the search becomes live (that's a + compared to other solutions since I haven't seen other apps do this) - it unifies the way we use livetable across XE - if we do this we could also modify the AllDocs page to have the global search field. Basically wherever we display documents we could also include this search field.

What about the benefits to end users? Searching is about finding. I think that what we must strive to achieve is to provide relevant search results on top for any given query. Advanced search options are nice, especially for power users, but they're not used frequently by most users and they do not meet the need of mainstream users. Thus I think we'd better fix search before building a fancy live interface :-) Main Question:

- would the live search be costly? With Lucene search probably not, with the DB search, it could (for ex if document content isn't indexed).

WDYT?

Thanks -Vincent

_______________________________________________

devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs

My personal view is that rather than building a new fancy interface based on the livetable, there's 2 things we'd better do first: 1. Make XWiki's Lucene Search Backend work reliably 2. Merge both search pages to keep only a Lucene-based one This would solve the issues Vincent listed above: - Scalability (the lucene search is paginated) - Relevance (lucene search puts most relevant results up first) Plus it matches current users' expectations when it comes to search. Let met be clear: I'm all for experimenting with new, nicer, more powerful way to interact with data and filter it and I'm a livetable fan. However, I don't think it's the best choice for our default search interface right now and I think we've got other search-related issues to fix first before working in that direction. Guillaume -- Guillaume Lerouge Product Manager - XWiki Skype: wikibc Twitter: glerouge http://guillaumelerouge.com/