llm-testing repositories

Trending repositories tagged llm-testing, ranked by trending score. Updated daily.

  1. 1
    faiscadev/fakecloud

    Free, open-source AWS emulator. LocalStack alternative: 107 services, 7,729 operations, true 100% Smithy conformance (260,001/260,001 variants pass). No account, no auth token, no paid tier.

    GitHub · Rust · 798 stars · 63 forks · 0 stars in 24 hours · Trending score 0.63

  2. 2
    rhesis-ai/rhesis

    The collaboration layer for teams building AI agents. Test, simulate, and trace your agents, while domain experts annotate behavior and engineers improve the agent from what they find.

    GitHub · Python · 405 stars · 43 forks · 0 stars in 24 hours · Trending score 0.39

  3. 3
    TaimoorKhan10/replayd

    Turn failed AI agent runs into replayable regression tests. Catch regressions before you ship.

    GitHub · Python · 19 stars · 2 forks · 0 stars in 24 hours · Trending score 0.28

  4. 4
    ProofAgent-ai/proofagent-harness

    Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

    GitHub · Python · 40 stars · 10 forks · 0 stars in 24 hours · Trending score 0.18

  5. 5
    AltairaLabs/PromptKit

    Test, red-team, and deploy LLM applications with confidence. Multi-provider support (OpenAI, Anthropic, Gemini), MCP integration, self-play testing, and production SDK.

    GitHub · Go · 12 stars · 3 forks · 0 stars in 24 hours · Trending score 0.10

  6. 6
    vonavikon/qa-agent-evals

    Acceptance testing for LLM systems via role subagents (Claude Code plugin)

    GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.08

  7. 7
    tastewise/mcp-whichtool

    Does the model pick the right tool? Tool-routing tests for your MCP server: run them on Claude and OpenAI models, see where each request lands, and fix the descriptions that lose.

    GitHub · TypeScript · 13 stars · 0 forks · 0 stars in 24 hours · Trending score 0.06

  8. 8
    Free-AI-Things/g4f-working

    g4f-working is a daily-updated list of working no-auth AI providers and models from @xtekky/gpt4free. It helps developers, testers, and AI enthusiasts instantly find which models are currently online and accessible without any API keys, tokens, or cookies.

    GitHub · Python · 153 stars · 18 forks · 0 stars in 24 hours · Trending score 0.05

  9. 9
    nano-step/eval-harness

    Typed evaluation for OpenCode skills and AI agents: capability, regression and product checks; provider-neutral evidence grading, provenance, and CI reports.

    GitHub · Shell · 15 stars · 6 forks · 0 stars in 24 hours · Trending score 0.00

  10. 10
    vieval-dev/vieval

    🧪💥 Evaluation framework based on Vitest, the testing framework you familiar with, for agents, models, and more.

    GitHub · TypeScript · 71 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  11. 11
    zgrai-com/llm-api-test

    A tool for testing and comparing the performance of different Large Language Model APIs. 一个用于测试和比较不同大语言模型API性能的工具。

    GitHub · JavaScript · 48 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00

  12. 12
    evalshift/evalshift-cli

    Open-source LLM migration and regression testing for AI agents. Compare models, detect tool-call regressions, and gate model changes in CI.

    GitHub · Python · 8 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00

  13. 13
    EvilFreelancer/tgfake

    Offline Telegram Bot API for testing bots, Mini Apps and LLM integrations.

    GitHub · Go · 18 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00