evaluation repositories

Trending repositories tagged evaluation, ranked by trending score. Updated daily.

  1. 1
    NVIDIA/SkillEvaluator

    Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.

    GitHub · Python · 4,260 stars · 554 forks · +794 stars in 24 hours · Trending score 5.61

  2. 2
    Tencent/WeKnora

    Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.

    GitHub · Go · 37,901 stars · 5,286 forks · +151 stars in 24 hours · Trending score 4.32

  3. 3
    langfuse/langfuse

    🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

    GitHub · TypeScript · 37,677 stars · 4,281 forks · +85 stars in 24 hours · Trending score 3.85

  4. 4
    promptfoo/promptfoo

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    GitHub · TypeScript · 27,494 stars · 2,802 forks · +65 stars in 24 hours · Trending score 3.79

  5. 5
    mlflow/mlflow

    The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

    GitHub · Python · 28,914 stars · 6,647 forks · +40 stars in 24 hours · Trending score 3.23

  6. 6
    alchaincyf/darwin-skill

    达尔文.skill —— 一个让你的Skill无限进化的系统:评估→改进→测试→保留或回滚 | Autoresearch-inspired autonomous skill optimization for Claude Code. Evaluate, improve, test, keep or revert.

    GitHub · HTML · 7,006 stars · 726 forks · +39 stars in 24 hours · Trending score 3.08

  7. 7
    comet-ml/opik

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    GitHub · Java · 23,476 stars · 1,966 forks · +10 stars in 24 hours · Trending score 2.61

  8. 8
    zaxiu88888/issue-rag

    RAG over GitHub issues, with a hand-reviewed eval set and an audit of the judge itself. Reports the interval, not just the best number.

    GitHub · Python · 485 stars · 6 forks · +16 stars in 24 hours · Trending score 2.33

  9. 9
    BUAASubnet/UBAA

    更好的智慧北航. Make BUAA Great Again!

    GitHub · Dart · 368 stars · 26 forks · +13 stars in 24 hours · Trending score 2.29

  10. 10
    open-compass/opencompass

    OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

    GitHub · Python · 7,601 stars · 915 forks · +17 stars in 24 hours · Trending score 2.26

  11. 11
    modelscope/evalscope

    A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

    GitHub · Python · 3,716 stars · 586 forks · +5 stars in 24 hours · Trending score 2.01

  12. 12
    didi/helixharness

    HelixHarness 提供完整的 AI Coding 工程化基础能力,还支持不同企业和团队结合自身的业务知识、工程规范和研发流程,持续构建和演进属于自己的专属 Harness

    GitHub · Python · 453 stars · 62 forks · +8 stars in 24 hours · Trending score 1.99

  13. 13
    lmnr-ai/lmnr

    Laminar - open-source observability platform purpose-built for AI agents. YC S24.

    GitHub · TypeScript · 3,513 stars · 262 forks · +8 stars in 24 hours · Trending score 1.85

  14. 14
    Privatris/AgentLeak

    AgentLeak (IEEE Access 2026): benchmark for internal-channel privacy leakage in multi-agent LLM systems. Inter-agent messages leak 68.8% vs 27.2% in final outputs; output-only audits miss 41.7%. 1,000 scenarios, 7 channels, 5 LLMs. Tool & MCP server: github.com/yagobski/agentleak

    GitHub · Python · 45 stars · 5 forks · +12 stars in 24 hours · Trending score 1.79

  15. 15
    Valen-Team/Roller

    Turn agent runs into training data. Roller rolls out Claude Code, Codex, OpenClaw and four other Harnesses in disposable Docker containers, grades benchmark tasks in place, and exports SFT-ready trajectories that keep reasoning, images and compacted-context segments.

    GitHub · Python · 112 stars · 0 forks · +37 stars in 24 hours · Trending score 1.78

  16. 16
    run-llama/ParseBench

    ParseBench - A Document Parsing Benchmark for AI Agents

    GitHub · Python · 648 stars · 140 forks · +5 stars in 24 hours · Trending score 1.76

  17. 17
    NVIDIA-NeMo/Gym

    Evaluate and improve models and agents using environments

    GitHub · Python · 1,346 stars · 486 forks · +3 stars in 24 hours · Trending score 1.72

  18. 18
    MCPJam/inspector

    Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.

    GitHub · TypeScript · 2,331 stars · 303 forks · +3 stars in 24 hours · Trending score 1.57

  19. 19
    langwatch/langwatch

    Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.

    GitHub · TypeScript · 5,048 stars · 434 forks · +3 stars in 24 hours · Trending score 1.55

  20. 20
    strands-agents/evals

    A comprehensive evaluation framework for AI agents and LLM applications.

    GitHub · Python · 261 stars · 87 forks · +3 stars in 24 hours · Trending score 1.47

  21. 21
    InternScience/SGI-Bench

    Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

    GitHub · Python · 173 stars · 4 forks · +7 stars in 24 hours · Trending score 1.36

  22. 22
    intel/dokimi

    Dokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.

    GitHub · Python · 39 stars · 13 forks · +1 stars in 24 hours · Trending score 1.30

  23. 23
    beipo-ai/NorthCUA

    Turn intent into work. Open-source Desktop/Web agent harness with long-horizon memory, live evaluation and recoverable batches.

    GitHub · Python · 128 stars · 7 forks · +4 stars in 24 hours · Trending score 1.28

  24. 24
    Helicone/helicone

    🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

    GitHub · TypeScript · 6,278 stars · 704 forks · 0 stars in 24 hours · Trending score 1.25