Evaluation
Lighteval
Evaluate language models across inference backends and inspect sample-level results.
2.6K GitHub starsAs of Checked
Where it fits
What you can use it for
Teams comparing models with configurable tasks and metrics.
Test model outputs and agent behaviour against defined criteria.
Before you adopt
What to check for your team
Pin tasks, datasets and runtime versions. Verify platform support and review the licenses of benchmark data.
Get help evaluating the fitLicence and scope
MIT
Applies to this repository. Dependencies, model weights, datasets and hosted services can have separate terms.
Read the upstream licenceSupport the work
Go directly to the maintainers
Use the project's contribution guidance to report issues, help with documentation or contribute code. Follow any published funding links if you want to support the maintainers financially.
Find contribution guidanceKeep exploring
