New subject: XWiki Application Help?

12 Jul 2005

Hi Jie,

Thanks for your email.

For XWiki the way I would do it:

- Have the FAQBot hosted on a wiki page. This page would have a text input
field for letting users enter queries and it would list answers on the page.
We should also provide other interfaces like IRC/chat/mailing list.
- Have some macros that can be used in the same manner as the {excerpt}
macro is used by Confluence. One macro would be used to designate a
question, another one for an answer and yet another one for a srai.
- Have some pages that are AIML sets. Mark them using a XWiki object class
and load them when the FAQBot page is loaded.
- Create on the fly AIML sets by aggregating all information scattered in
the different pages (using the macros)

About the reinforcement. It could be done by different ways. One idea: 

- On the FAQ bot page we could have a "teach" button where we would ask the
user to enter his question and the answer he would have liked. 
- All these user questions/answers will be added automatically to a User
AIML set (a xwiki page containing AIML). This AIML set will also be loaded
by the FAQBot page (same as the other AIML sets).
- Whenever this user AIML set page is changed a diff email will be sent to
the XWiki dev mailing list. The idea is that XWiki community will provide
oversight about new questions/answers. The reason for the oversight is
double:
  - to verify users do not enter stupid questions/answers
  - to possibly refactor the existing AIML sets or pages containing custom
tags by removing the user-entered information and placing it where it is
more appropriate.

Again, all this should be packaged as a XWiki application. Jeremi Joslin
should be able to help you about the format (he's the one who's created the
notion of XWiki applications).

Last, we need to start deciding on the project directory structure in SVN.
Could you please come up with a proposal? Also, we're doing a generic FAQBot
so I'd like it to remain as separate from XWiki as possible. The generic
part and the XWiki specific ones should be cleanly separated in this
directory structure. Could you please come up with an architecture diagram
fro this?

Thanks
-Vincent

PS: I'm cc-ing the xwiki-dev mailing list. They may have some
ideas/remarks/feedback.

PPS: I'd need a formal updated proposition from you listing deliverables and
planning so that we're clean WRT Google. Including the architecture diagram
in the proposal would be nice.

...
  -----Original Message-----
 From: Jie Tang [mailto:jietang@fas.harvard.edu]
 Sent: lundi 11 juillet 2005 08:01
 To: Vincent Massol
 Subject: RE: [SOC] Faqbot (ping?)

 Hi Vincent,

 Hope everything is going well with the baby, and congratulations. Take
 whatever
 time you need to get settled...I'll be ready whenever you are.

 First off, I still think that the Faqbot is something I'd like to do. I've
 been
 thinking over how to implement some of the ideas for the bot this weekend,
 in
 addition to building some elementary aiml sets.

 The idea behind the Xwiki plugin is that an xwiki page can have a section
 designated as an faq (through some specially formatted comment?). When
 this
 page is updated by a user, the new faq content is run through the macro,
 which
 will transform the faq text into aiml expressions, which can then be
 loaded by
 the faqbot dynamically. Thus, in theory it wouldn't require a webmaster to
 run
 any extra utilities to have the faqbot incorporate the extra knowledge.
 The
 algorithm could have the bot read in the question part of the FAQ, looking
 for
 keywords that it already knows how to process. It would then try to
 generate a
 bunch of questions similar in wording to the new question, to allow the
 bot to
 provide answers to questions not worded exactly the same as the original
 query.
 It would associate the questions with the proper answer. The answer part
 of the
 FAQ would have to be processed too, to find key phrases which the bot
 already
 understands. This set of phrases would also be associated with the new faq
 entry, and the bot would then ask one of these associated questions
 depending
 on the users response.

 The tough part would be to figure out how exactly each entry refers to the
 other; i.e., where the bot should go next after receiving a certain
 response
 from the user. One possibility would be extra formatting and markup,
 describing
 which other faq entries to look at for related knowledge, what to do if
 so-and-so is the response, etc., within the XWiki page. While this would
 be
 easier to implement, the more markup is required the less useful this will
 be
 over writing new aiml files entirely.

 The other idea is to implement some sort of reinforcement/learning in the
 bot.
 (this is more appealing to me, though I'm unsure how feasible it is).
 Ideally,
 a user would be able to rate the usefulness of the bot's responses to
 their
 queries, or suggest new responses; the bot would then be able to adjust
 its
 responses accordingly. This could be accomplished through creating
 entirely new
 aiml files, modifying existing aiml files (i.e., using a <random> element
 and
 weight certain responses more than others), or an exta system which causes
 the
 bot to learn and unlearn aiml. This would mimic a simple neural network
 algorithm, implemented in aiml.

 I've a feeling that I'll be working on the aiml sets for awhile. I look
 forward
 to hearing your thoughts on the new ideas?

 Thanks, and take care,

 Jie Tang 
[snip]