This issue has been created
There are 3 updates.
 
 
LLM AI Integration / cid:jira-generated-image-avatar-67f7174e-02ad-4a9c-9620-df3cf7cbfaf3 LLMAI-78 Open

Translate context documents in the benchmark and use combinations of languages
 
View issue   ยท   Add comment
 

Issue created

 
cid:jira-generated-image-avatar-26d835c3-a651-4f8d-be0e-ac0d201497ef Michael Hamann created this issue on 27/May/24 15:10
 
Summary: Translate context documents in the benchmark and use combinations of languages
Issue Type: cid:jira-generated-image-avatar-67f7174e-02ad-4a9c-9620-df3cf7cbfaf3 Improvement
Affects Versions: 0.3.1
Assignee: Unassigned
Created: 27/May/24 15:10
Priority: cid:jira-generated-image-static-major-84497fec-a491-4114-92bc-937d77595062 Major
Reporter: Michael Hamann
Description:

At the moment, all context documents are in English while questions are in English, German and French. It would be interesting to also translate all context documents to German and French and to evaluate all combinations of context and question language to better understand the limitations of the different models in terms of multilingual capabilities.
 
 

3 updates

 
cid:jira-generated-image-avatar-26d835c3-a651-4f8d-be0e-ac0d201497ef Changes by Michael Hamann on 27/May/24 15:11
 
Fix Version: 0.4
Description: At the moment, all context documents are in English while questions are in English, German and French. It would be interesting to also (automatically) translate all context documents to German and French and to evaluate all combinations of context and question language to better understand the limitations of the different models in terms of multilingual capabilities.
Assignee: Paul Pantiru
 
 
This message was sent by Atlassian Jira (v9.3.0#930000-sha1:287aeb6) Atlassian logo
If image attachments aren't displayed, see this article.