Skip to content

Make your model decision with live evidence

LocalArena runs the models you choose against the tasks you choose, using live responses from your machine or an explicitly configured API endpoint.

Get started Star LocalArena on GitHub

The shortest path

Install the Python 3.10+ package:

pip install localarena

With Ollama running and a model already available, make a real request, score the response, save portable JSON evidence, and render a standalone HTML report:

localarena quickstart ollama qwen3:0.6b

LocalArena never replaces an unavailable endpoint with recorded output. A failed call remains visible in the result instead of silently shrinking the experiment.

What stays honest

Live execution

Use llama.cpp, Ollama, LM Studio, OpenRouter, OpenAI, or any compatible endpoint with an explicit base URL.

Private by default

Your tasks can stay on your machine. Prompt text, answers, references, and detailed errors are excluded from saved results unless you opt in.

Portable evidence

Keep the complete model × task matrix, failures, coverage, latency, task-pack identity, and uncertainty in a versioned result.

Choose your next step

  • Follow Getting started for a first live score with each provider profile.
  • Read Providers and tasks for endpoint, authentication, privacy, and task-execution boundaries.
  • Read Evaluation methodology for failure-aware scoring, Bradley–Terry ratings, confidence intervals, and inconclusive outcomes.
  • Prefer JavaScript? Install localarena from npm and use the matching dependency-free API:
npm install localarena