Google DeepMind has released a new benchmark that tests whether AI agents can handle work that depends on images, layouts and charts. The test, called the Multimodal Knowledge Agent benchmark, or MKA, was built with AI benchmarking firm Artificial Analysis.
The company announced the benchmark on November 10, 2026. Google DeepMind said its own Gemini 4 Carbon model ranked second in quality among the models tested.
Google said most knowledge work benchmarks can be solved with text alone. A model can often pass them without ever looking at an image or a page layout.
Many real jobs work differently. Architects cross-check blueprints, engineers trace schematics, and analysts turn findings into decks, reports and spreadsheets.
How the Tasks Were Built
To build the test, Google worked with experts across 10 industries and 83 jobs where daily work is visual. The group included architects, biologists, underwriters and data scientists.
These specialists wrote a set of 200 tasks. Each one is a multi-step project that takes an experienced worker three to eight hours to finish.
Every task comes with a realistic prompt and a bundle of files in different formats. It also includes an expert-written scoring guide with about 70 yes-or-no checks.
Those checks are weighted 75% on factual accuracy and 25% on visual design. That means a correct answer can still lose points if the final file is poorly laid out.
All models run in the same open-source agent setup, called Stirrup. Each one gets the same tools, sandbox and limit on turns.
Grading Handled by a Non-Google Model
Grading is done by an automated agent built on a model that Google does not make. It opens every file, renders pages, checks spreadsheet formulas and compares the work against the expert guide.
Google said domain experts also graded model outputs by hand. Their scores matched the automated grader 96% of the time.
Artificial Analysis runs the full 200-task set and keeps it private. This is meant to stop the tasks from leaking into training data or being found by a model through search.
A separate group of 10 public tasks has been posted on Hugging Face. Anyone can use them to see the format and test their own agents.
Google also published a technical report on its methods and results. It found that leading models worked in very different ways, using between 30 and 130 turns on average per task.
The report said strong vision skills are needed to solve the tasks. It also showed models trying and failing to make up for it when vision was taken away.
Gemini 4 Carbon placed second in quality on the benchmark. Google said it finished tasks at six times lower cost on average than the top-scoring model, which its post did not name.
The model sits on what Google calls the pareto frontier for quality and price on the test. That means no other model tested beat it on both quality and cost at once.
Developers can now test their own models on the benchmark through Artificial Analysis. Gemini 4 Carbon is available today through the Gemini API in Google AI Studio.
