Evaluation
LMMs-Eval
Evaluate multimodal models across text, image, video and audio tasks.
4.5K GitHub starsAs of Checked
Where it fits
What you can use it for
Researchers testing models beyond text-only benchmarks.
Test model outputs and agent behaviour against defined criteria.
Before you adopt
What to check for your team
The pipeline uses MIT and added model/task code uses Apache-2.0. Datasets and model weights carry their own terms.
Get help evaluating the fitLicence and scope
MIT + Apache-2.0
The main pipeline is MIT. Added model and task code is Apache-2.0. Benchmark datasets and model weights have separate licenses.
Read the upstream licenceSupport the work
Go directly to the maintainers
Use the project's contribution guidance to report issues, help with documentation or contribute code. Follow any published funding links if you want to support the maintainers financially.
Find contribution guidanceKeep exploring
