ai-evaluation repositories

Trending repositories tagged ai-evaluation, ranked by trending score. Updated daily.

  1. 1
    microsoft/PyRIT

    Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.

    GitHub · Python · 5,001 stars · 1,098 forks · +51 stars in 24 hours · Trending score 3.46

  2. 2
    ifixai-ai/iFixAi

    Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.

    GitHub · Python · 46,552 stars · 1,828 forks · −189 stars in 24 hours · Trending score 2.05

  3. 3
    microsoft/scope

    Scope | Agentic Experience evaluation platform (Research preview)

    GitHub · TypeScript · 90 stars · 32 forks · +7 stars in 24 hours · Trending score 1.45

  4. 4
    huggingface/cadgenbench

    A benchmark for AI-driven CAD generation and editing

    GitHub · Python · 156 stars · 14 forks · 0 stars in 24 hours · Trending score 1.06

  5. 5
    buildinghumanetech/humanebench

    Open-source benchmark that measures whether AI chatbots support human wellbeing. Code Apache-2.0, data CC BY 4.0. Maintained by Building Humane Tech, PBC.

    GitHub · Python · 31 stars · 7 forks · +2 stars in 24 hours · Trending score 0.89

  6. 6
    DawitLemmaBalcha/code-review-bench

    Review benchmark for model-generated code patches: six tasks across Python, JavaScript, Go and Java with hidden tests, a fixed rubric and a Docker runner.

    GitHub · Python · 13 stars · 2 forks · +2 stars in 24 hours · Trending score 0.57

  7. 7
    adhit-r/fairmind

    Open-source AI assurance workbench for models and agents: map governance frameworks (AIUC-1, NIST AI RMF, EU AI Act, DPDP) to controls, collect evaluation evidence, review findings and produce audit-ready evidence trails.

    GitHub · Python · 17 stars · 16 forks · 0 stars in 24 hours · Trending score 0.49

  8. 8
    brida-ai/reflexbench

    ReflexBench — open benchmark and evaluation harness for System One models and typed decision engines

    GitHub · Python · 7 stars · 0 forks · +1 stars in 24 hours · Trending score 0.46

  9. 9
    SE-Emam/EMO-X

    EMO-X: adaptive, execution-based evaluation OS for AI agents.

    GitHub · Python · 9 stars · 0 forks · +1 stars in 24 hours · Trending score 0.46

  10. 10
    mahmoudrabie/agentic-ai

    Agentic AI research papers, benchmarks, frameworks, and tools curated across 24 domains.

    GitHub · 192 stars · 7 forks · 0 stars in 24 hours · Trending score 0.38

  11. 11
    ARSI-LAB/ARSI-Evaluator

    Many ways to evaluate. One framework to bring them together. Unify benchmarks, agent evaluations, and performance testing across coding, terminal, office, web, and more.

    GitHub · TypeScript · 41 stars · 8 forks · 0 stars in 24 hours · Trending score 0.36

  12. 12
    awelist/awesome-ai-benchmarks-evaluation

    A curated list of evaluation tools, benchmark datasets, leaderboards, frameworks, and resources for assessing model performance.

    GitHub · Python · 14 stars · 21 forks · 0 stars in 24 hours · Trending score 0.33

  13. 13
    OpenAgentHQ/openagent-eval

    Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

    GitHub · Python · 16 stars · 32 forks · 0 stars in 24 hours · Trending score 0.32

  14. 14
    rudycelekli/rudycelekli

    Profile: proof-driven AI systems, evaluation, and agent reliability

    GitHub · Python · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.20

  15. 15
    shsridhar-beep/svgap

    Evidence profiles for AI-generated RTL that expose what functional benchmarks miss. Seeking collaborators, design partners, and validation

    GitHub · Python · 21 stars · 10 forks · 0 stars in 24 hours · Trending score 0.10

  16. 16
    Pro-GenAI/Agent-Action-Guard

    🛡️ Safe AI Agents through Action Classifier

    GitHub · Python · 7 stars · 8 forks · 0 stars in 24 hours · Trending score 0.10

  17. 17
    Claire1217/benchmark-radar

    Track newly released AI benchmarks and their real-world adoption momentum.

    GitHub · HTML · 11 stars · 0 forks · −1 stars in 24 hours · Trending score 0.09

  18. 18
    humane-intelligence/hi-oss-ai-red-teaming

    A platform for running human-centered red teaming evaluations on AI models and systems.

    GitHub · Python · 19 stars · 3 forks · 0 stars in 24 hours · Trending score 0.09

  19. 19
    Exponential-OS/prompt-engineering-in-action

    Co-Dialectic: catch your AI agreeing with you to please you, not because you're right. Sycophancy detection, cross-family judging, and prompt coaching that runs during the conversation rather than auditing after. Free and open source. Works with Claude, ChatGPT, Gemini.

    GitHub · TypeScript · 11 stars · 3 forks · 0 stars in 24 hours · Trending score 0.09

  20. 20
    SRCarlo/genai-course

    A hands-on GenAI Engineering roadmap from LLM fundamentals to production AI systems. Covers prompt engineering, RAG, Agentic AI, LLM evaluation, observability, AI security, LLMOps, production deployment, CI/CD, multimodal AI, MCP, cloud, and AI system design through practical projects and real-world architecture.

    GitHub · JavaScript · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.08

  21. 21
    MrKyaw/MrKyaw

    Principal AI Engineer & Forward-Deployed AI Architect | Enterprise & Solution Architecture | Agentic AI, Multi-Agent Systems, RAG & MCP | MLOps/LLMOps, Evaluation, Observability & Responsible AI | Customer-focused discovery, hands-on delivery, measurable business outcomes. PRIME: build, measure, operate.

    GitHub · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.07

  22. 22
    techtheist/knowledgedrift

    KnowledgeDrift - Second Generation AI Memory benchmark

    GitHub · Rust · 17 stars · 1 forks · 0 stars in 24 hours · Trending score 0.07

  23. 23
    ARTPARK-SAHAI-ORG/calibrate

    Core engine behind Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations

    GitHub · JavaScript · 24 stars · 7 forks · 0 stars in 24 hours · Trending score 0.04

  24. 24
    cvs-health/uqlm

    [JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification

    GitHub · Python · 1,217 stars · 140 forks · 0 stars in 24 hours · Trending score 0.01