ai-evaluation repositories
Trending repositories tagged ai-evaluation, ranked by trending score. Updated daily.
-
1
microsoft/PyRIT
Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.
GitHub · Python · 5,001 stars · 1,098 forks · +51 stars in 24 hours · Trending score 3.46
-
2
ifixai-ai/iFixAi
Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.
GitHub · Python · 46,552 stars · 1,828 forks · −189 stars in 24 hours · Trending score 2.05
-
3
microsoft/scope
Scope | Agentic Experience evaluation platform (Research preview)
GitHub · TypeScript · 90 stars · 32 forks · +7 stars in 24 hours · Trending score 1.45
-
4
huggingface/cadgenbench
A benchmark for AI-driven CAD generation and editing
GitHub · Python · 156 stars · 14 forks · 0 stars in 24 hours · Trending score 1.06
-
5
buildinghumanetech/humanebench
Open-source benchmark that measures whether AI chatbots support human wellbeing. Code Apache-2.0, data CC BY 4.0. Maintained by Building Humane Tech, PBC.
GitHub · Python · 31 stars · 7 forks · +2 stars in 24 hours · Trending score 0.89
-
6
DawitLemmaBalcha/code-review-bench
Review benchmark for model-generated code patches: six tasks across Python, JavaScript, Go and Java with hidden tests, a fixed rubric and a Docker runner.
GitHub · Python · 13 stars · 2 forks · +2 stars in 24 hours · Trending score 0.57
-
7
adhit-r/fairmind
Open-source AI assurance workbench for models and agents: map governance frameworks (AIUC-1, NIST AI RMF, EU AI Act, DPDP) to controls, collect evaluation evidence, review findings and produce audit-ready evidence trails.
GitHub · Python · 17 stars · 16 forks · 0 stars in 24 hours · Trending score 0.49
-
8
brida-ai/reflexbench
ReflexBench — open benchmark and evaluation harness for System One models and typed decision engines
GitHub · Python · 7 stars · 0 forks · +1 stars in 24 hours · Trending score 0.46
-
9
SE-Emam/EMO-X
EMO-X: adaptive, execution-based evaluation OS for AI agents.
GitHub · Python · 9 stars · 0 forks · +1 stars in 24 hours · Trending score 0.46
-
10
mahmoudrabie/agentic-ai
Agentic AI research papers, benchmarks, frameworks, and tools curated across 24 domains.
GitHub · 192 stars · 7 forks · 0 stars in 24 hours · Trending score 0.38
-
11
ARSI-LAB/ARSI-Evaluator
Many ways to evaluate. One framework to bring them together. Unify benchmarks, agent evaluations, and performance testing across coding, terminal, office, web, and more.
GitHub · TypeScript · 41 stars · 8 forks · 0 stars in 24 hours · Trending score 0.36
-
12
awelist/awesome-ai-benchmarks-evaluation
A curated list of evaluation tools, benchmark datasets, leaderboards, frameworks, and resources for assessing model performance.
GitHub · Python · 14 stars · 21 forks · 0 stars in 24 hours · Trending score 0.33
-
13
OpenAgentHQ/openagent-eval
Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.
GitHub · Python · 16 stars · 32 forks · 0 stars in 24 hours · Trending score 0.32
-
14
rudycelekli/rudycelekli
Profile: proof-driven AI systems, evaluation, and agent reliability
GitHub · Python · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.20
-
15
shsridhar-beep/svgap
Evidence profiles for AI-generated RTL that expose what functional benchmarks miss. Seeking collaborators, design partners, and validation
GitHub · Python · 21 stars · 10 forks · 0 stars in 24 hours · Trending score 0.10
-
16
Pro-GenAI/Agent-Action-Guard
🛡️ Safe AI Agents through Action Classifier
GitHub · Python · 7 stars · 8 forks · 0 stars in 24 hours · Trending score 0.10
-
17
Claire1217/benchmark-radar
Track newly released AI benchmarks and their real-world adoption momentum.
GitHub · HTML · 11 stars · 0 forks · −1 stars in 24 hours · Trending score 0.09
-
18
humane-intelligence/hi-oss-ai-red-teaming
A platform for running human-centered red teaming evaluations on AI models and systems.
GitHub · Python · 19 stars · 3 forks · 0 stars in 24 hours · Trending score 0.09
-
19
Exponential-OS/prompt-engineering-in-action
Co-Dialectic: catch your AI agreeing with you to please you, not because you're right. Sycophancy detection, cross-family judging, and prompt coaching that runs during the conversation rather than auditing after. Free and open source. Works with Claude, ChatGPT, Gemini.
GitHub · TypeScript · 11 stars · 3 forks · 0 stars in 24 hours · Trending score 0.09
-
20
SRCarlo/genai-course
A hands-on GenAI Engineering roadmap from LLM fundamentals to production AI systems. Covers prompt engineering, RAG, Agentic AI, LLM evaluation, observability, AI security, LLMOps, production deployment, CI/CD, multimodal AI, MCP, cloud, and AI system design through practical projects and real-world architecture.
GitHub · JavaScript · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.08
-
21
MrKyaw/MrKyaw
Principal AI Engineer & Forward-Deployed AI Architect | Enterprise & Solution Architecture | Agentic AI, Multi-Agent Systems, RAG & MCP | MLOps/LLMOps, Evaluation, Observability & Responsible AI | Customer-focused discovery, hands-on delivery, measurable business outcomes. PRIME: build, measure, operate.
GitHub · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.07
-
22
techtheist/knowledgedrift
KnowledgeDrift - Second Generation AI Memory benchmark
GitHub · Rust · 17 stars · 1 forks · 0 stars in 24 hours · Trending score 0.07
-
23
ARTPARK-SAHAI-ORG/calibrate
Core engine behind Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations
GitHub · JavaScript · 24 stars · 7 forks · 0 stars in 24 hours · Trending score 0.04
-
24
cvs-health/uqlm
[JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification
GitHub · Python · 1,217 stars · 140 forks · 0 stars in 24 hours · Trending score 0.01