llm-evals repositories

Trending repositories tagged llm-evals, ranked by trending score. Updated daily.

  1. 1
    itsdecodingai/building-a-coding-agent-from-scratch-course

    Learn harness engineering by building Claude Code from scratch. Free, open-source course: 8 articles, 6 videos, 1 codebase.

    GitHub · Python · 1,229 stars · 284 forks · +544 stars in 24 hours · Trending score 4.81

  2. 2
    jackulau/coding.sh

    Open-source, local-first observability and evals for AI agents: see what Claude Code, Codex, Cursor and others did and what it cost.

    GitHub · TypeScript · 149 stars · 6 forks · +21 stars in 24 hours · Trending score 2.75

  3. 3
    darkrishabh/agent-skills-eval

    A test runner for agentskills.io-style AI agent skills

    GitHub · TypeScript · 819 stars · 44 forks · +2 stars in 24 hours · Trending score 0.78

  4. 4
    abhixhek/jevcal

    Stop guessing confidence thresholds: calibrate, threshold, and drift-check typed decision models (TypeSafe Jev) against an LLM teacher.

    GitHub · Python · 12 stars · 0 forks · +2 stars in 24 hours · Trending score 0.73

  5. 5
    elm-os/rightmodeler

    Cheaper or newer model swaps for production AI, validated on your own traces and opened as draft PRs.

    GitHub · TypeScript · 28 stars · 5 forks · 0 stars in 24 hours · Trending score 0.51

  6. 6
    LinuxDevil/lousho

    TypeScript AI agent SDK: offline CI evals, durable runs, human approvals, MCP. Runs in your own Node process.

    GitHub · TypeScript · 10 stars · 0 forks · 0 stars in 24 hours · Trending score 0.23

  7. 7
    vonavikon/qa-agent-evals

    Acceptance testing for LLM systems via role subagents (Claude Code plugin)

    GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.08

  8. 8
    ARTPARK-SAHAI-ORG/calibrate

    Core engine behind Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations

    GitHub · JavaScript · 24 stars · 7 forks · 0 stars in 24 hours · Trending score 0.04

  9. 9
    kivo360/OmoiOS

    Turn feature specs into merged PRs with a self-supervising swarm of coding agents — parallel execution, isolated sandboxes, DAG dependencies. Open-source, self-hostable, model-agnostic (Claude / Gemini / Codex).

    GitHub · Python · 80 stars · 6 forks · 0 stars in 24 hours · Trending score 0.00

  10. 10
    BejeweledMe/codex-pro-agent-skills

    Niche high-signal skills for professional Codex agents

    GitHub · Shell · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00