Branch: refs/heads/master Home: https://github.com/xwiki-contrib/ai-llm-benchmark Commit: 76c3e7798775bef8cbb1521b149bd3c9ca8f7b39 https://github.com/xwiki-contrib/ai-llm-benchmark/commit/76c3e7798775bef8cb… Author: Paul Pantiru <paul.pantiru(a)xwiki.com> Date: 2024-07-16 (Tue, 16 Jul 2024) Changed paths: M README.md M environment.yml M scripts/evaluation_scripts/eval_rag_qa.py M scripts/evaluation_scripts/eval_summary.py M scripts/evaluation_scripts/eval_text_generation.py Log Message: ----------- LLMAI-80: Evaluate the output language of the LLM in the benchmark Commit: 927828effbd857111cb568f6932d86be987b82a3 https://github.com/xwiki-contrib/ai-llm-benchmark/commit/927828effbd857111c… Author: Paul Pantiru <paul.pantiru(a)xwiki.com> Date: 2024-07-16 (Tue, 16 Jul 2024) Changed paths: M context_data/collections/collection.json M context_data/collections/collection2.json M context_data/collections/collection3.json R context_data/documents/urls.txt A input/urls.txt Log Message: ----------- LLMAI-78: Translate context documents in the benchmark and use combinations of languages * Update collections * Moved urls.txt to input folder Commit: 38ff2305f276df226f1bd22b4debf7e4a1a981e6 https://github.com/xwiki-contrib/ai-llm-benchmark/commit/38ff2305f276df226f… Author: Paul Pantiru <paul.pantiru(a)xwiki.com> Date: 2024-07-17 (Wed, 17 Jul 2024) Changed paths: M scripts/context_indexing/index_data.py M scripts/evaluation_scripts/calculate_average_power.py M scripts/output_generation/collect_model_responses.py Log Message: ----------- LLMAI-76: Measure energy consumption for the benchmark * Update measurments Commit: 3161263f09d0efbc5bd94cf1c420e050491792f1 https://github.com/xwiki-contrib/ai-llm-benchmark/commit/3161263f09d0efbc5b… Author: Paul Pantiru <paul.pantiru(a)xwiki.com> Date: 2024-07-17 (Wed, 17 Jul 2024) Changed paths: M README.md M Snakefile M dag.png M scripts/results_visualization/create_plots.py A scripts/results_visualization/create_report.py Log Message: ----------- LLMAI-103: Add pdf report generation for the evaluation framework Compare: https://github.com/xwiki-contrib/ai-llm-benchmark/compare/fa0d23bae708...31… To unsubscribe from these emails, change your notification settings at https://github.com/xwiki-contrib/ai-llm-benchmark/settings/notifications