evaluation repositories
Trending repositories tagged evaluation, ranked by trending score. Updated daily.
-
1
NVIDIA/SkillEvaluator
Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
GitHub · Python · 4,260 stars · 554 forks · +794 stars in 24 hours · Trending score 5.61
-
2
Tencent/WeKnora
Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.
GitHub · Go · 37,901 stars · 5,286 forks · +151 stars in 24 hours · Trending score 4.32
-
3
langfuse/langfuse
🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.
GitHub · TypeScript · 37,677 stars · 4,281 forks · +85 stars in 24 hours · Trending score 3.85
-
4
promptfoo/promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
GitHub · TypeScript · 27,494 stars · 2,802 forks · +65 stars in 24 hours · Trending score 3.79
-
5
mlflow/mlflow
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
GitHub · Python · 28,914 stars · 6,647 forks · +40 stars in 24 hours · Trending score 3.23
-
6
alchaincyf/darwin-skill
达尔文.skill —— 一个让你的Skill无限进化的系统:评估→改进→测试→保留或回滚 | Autoresearch-inspired autonomous skill optimization for Claude Code. Evaluate, improve, test, keep or revert.
GitHub · HTML · 7,006 stars · 726 forks · +39 stars in 24 hours · Trending score 3.08
-
7
comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
GitHub · Java · 23,476 stars · 1,966 forks · +10 stars in 24 hours · Trending score 2.61
-
8
zaxiu88888/issue-rag
RAG over GitHub issues, with a hand-reviewed eval set and an audit of the judge itself. Reports the interval, not just the best number.
GitHub · Python · 485 stars · 6 forks · +16 stars in 24 hours · Trending score 2.33
-
9
BUAASubnet/UBAA
更好的智慧北航. Make BUAA Great Again!
GitHub · Dart · 368 stars · 26 forks · +13 stars in 24 hours · Trending score 2.29
-
10
open-compass/opencompass
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
GitHub · Python · 7,601 stars · 915 forks · +17 stars in 24 hours · Trending score 2.26
-
11
modelscope/evalscope
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
GitHub · Python · 3,716 stars · 586 forks · +5 stars in 24 hours · Trending score 2.01
-
12
didi/helixharness
HelixHarness 提供完整的 AI Coding 工程化基础能力,还支持不同企业和团队结合自身的业务知识、工程规范和研发流程,持续构建和演进属于自己的专属 Harness
GitHub · Python · 453 stars · 62 forks · +8 stars in 24 hours · Trending score 1.99
-
13
lmnr-ai/lmnr
Laminar - open-source observability platform purpose-built for AI agents. YC S24.
GitHub · TypeScript · 3,513 stars · 262 forks · +8 stars in 24 hours · Trending score 1.85
-
14
Privatris/AgentLeak
AgentLeak (IEEE Access 2026): benchmark for internal-channel privacy leakage in multi-agent LLM systems. Inter-agent messages leak 68.8% vs 27.2% in final outputs; output-only audits miss 41.7%. 1,000 scenarios, 7 channels, 5 LLMs. Tool & MCP server: github.com/yagobski/agentleak
GitHub · Python · 45 stars · 5 forks · +12 stars in 24 hours · Trending score 1.79
-
15
Valen-Team/Roller
Turn agent runs into training data. Roller rolls out Claude Code, Codex, OpenClaw and four other Harnesses in disposable Docker containers, grades benchmark tasks in place, and exports SFT-ready trajectories that keep reasoning, images and compacted-context segments.
GitHub · Python · 112 stars · 0 forks · +37 stars in 24 hours · Trending score 1.78
-
16
run-llama/ParseBench
ParseBench - A Document Parsing Benchmark for AI Agents
GitHub · Python · 648 stars · 140 forks · +5 stars in 24 hours · Trending score 1.76
-
17
NVIDIA-NeMo/Gym
Evaluate and improve models and agents using environments
GitHub · Python · 1,346 stars · 486 forks · +3 stars in 24 hours · Trending score 1.72
-
18
MCPJam/inspector
Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.
GitHub · TypeScript · 2,331 stars · 303 forks · +3 stars in 24 hours · Trending score 1.57
-
19
langwatch/langwatch
Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.
GitHub · TypeScript · 5,048 stars · 434 forks · +3 stars in 24 hours · Trending score 1.55
-
20
strands-agents/evals
A comprehensive evaluation framework for AI agents and LLM applications.
GitHub · Python · 261 stars · 87 forks · +3 stars in 24 hours · Trending score 1.47
-
21
InternScience/SGI-Bench
Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
GitHub · Python · 173 stars · 4 forks · +7 stars in 24 hours · Trending score 1.36
-
22
intel/dokimi
Dokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.
GitHub · Python · 39 stars · 13 forks · +1 stars in 24 hours · Trending score 1.30
-
23
beipo-ai/NorthCUA
Turn intent into work. Open-source Desktop/Web agent harness with long-horizon memory, live evaluation and recoverable batches.
GitHub · Python · 128 stars · 7 forks · +4 stars in 24 hours · Trending score 1.28
-
24
Helicone/helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
GitHub · TypeScript · 6,278 stars · 704 forks · 0 stars in 24 hours · Trending score 1.25