[xwiki-notifications] [xwiki-contrib/ai-llm-benchmark] a43361: LLMAI-61: Implement an evaluation framework

Branch: refs/heads/master Home: https://github.com/xwiki-contrib/ai-llm-benchmark Commit: a43361fb91d770dc560c24011ae2c9468e005f7d https://github.com/xwiki-contrib/ai-llm-benchmark/commit/a43361fb91d770dc56… Author: Paul Pantiru <paul.pantiru(a)xwiki.com> Date: 2024-04-23 (Tue, 23 Apr 2024) Changed paths: M .gitignore A CollectData.py A Eval.py A IndexContext.py R Jenkinsfile A MakePlots.py M README.md A context_data/collections/collection.json A context_data/collections/collection2.json A context_data/documents/EvalDoc2.json A context_data/documents/document.json A context_data/documents/document.txt A drafts/eval_draft.py A example.env A input/input.json A input/questions/1.json A input/questions/2.json A input/questions/3.json A input/questions/4.json A input/questions/5.json A input/splitToFiles.py A output/1.json A output/2.json A output/3.json A output/4.json A output/5.json A plots/accuracy_scores.png A plots/relevance_scores.png A request.json A results/1_result.json A results/2_result.json A results/3_result.json A results/4_result.json A results/5_result.json Log Message: ----------- LLMAI-61: Implement an evaluation framework * Added Indexing script * Added data collection script (which runs the input data against the ai server and collects the results * Added ROUGE score and relevacty based on cosin similarity (still in testing phase) * Example data To unsubscribe from these emails, change your notification settings at https://github.com/xwiki-contrib/ai-llm-benchmark/settings/notifications