Ai2 stated Molmo 2 improves on its earlier fashions regardless of its compact dimension. | Supply: Ai2
The Allen Institute for AI, also referred to as Ai2, final week launched Molmo 2, its newest multimodel suite able to exact spatial and temporal understanding of video, picture, and multi-image units. Constructing on the primary Molmo platform, Molmo 2 has superior capabilities in video pointing, multi-frame reasoning, and object monitoring.
Molmo 2 is an 8B-parameter mannequin that surpasses final yr’s 72B-parameter Molmo in accuracy, temporal understanding, and pixel-level grounding. Ai2 stated it additionally bests proprietary fashions like Gemini 3 on key rising expertise like video monitoring.
In the case of picture and multi-image reasoning, Ai2 claimed the Molmo 2 4B variant outperforms open fashions resembling Qwen 3-VL-8B whereas utilizing fewer parameters. Expertise like these assist the mannequin, and any software or system constructed on prime of it, to know what is going on, the place it’s taking place, and what it means.
Molmo 2 can be educated on far much less knowledge than comparable fashions — 9.19 million movies in contrast with 72.5 million for Meta’s PerceptionLM.
“With a fraction of the information, Molmo 2 surpasses many frontier fashions on key video understanding duties,” stated Ali Farhadi, the CEO of Ai2. ‘We’re excited to see the immense influence this mannequin can have on the AI panorama, including one other piece to our absolutely open mannequin ecosystem.”
Ai2 is a Seattle-based nonprofit AI analysis institute with the mission of constructing AI to resolve the world’s largest issues. Based in 2014 by late Microsoft co-founder Paul G. Allen, Ai2 stated it develops foundational AI analysis and new purposes by means of large-scale open fashions, open knowledge, robotics, conservation platforms, and extra.
Molmo 2 affords new capabilities
Deep video understanding is essential to constructing fashions that may perceive and act on sensor streams for robotics. Nonetheless, most fashions right this moment both lack video understanding capabilities or are locked behind proprietary methods with out transparency into the information. Ai2 stated it’s giving researchers entry to superior video grounding, monitoring, and multi-frame reasoning, all with open weights and knowledge.
Molmo 2 can determine precisely the place and when occasions happen, observe a number of objects by means of complicated scenes, and join actions to frame-level timelines. The firm stated these capabilities assist safer automation, extra correct real-world methods, and open analysis the worldwide group can examine, reproduce, and construct upon.
Ai2 listed key capabilities:
- Body-level spatial and temporal grounding: Molmo 2 goes past description. It returns exact pixel coordinates, object positions, and timestamps for occasions throughout a video.
- Sturdy multi-object monitoring and counting: The mannequin maintains constant object identities throughout occlusions, scene adjustments, and lengthy clips, enabling purposes in robotics, inspection, transportation, and trade.
- Dense long-form video captioning and anomaly detection: Molmo 2 produces extremely detailed, searchable descriptions and flags uncommon occasions in lengthy sequences.
Molmo 2 delivers on main open-weight benchmarks, says Ai2
Molmo 2 delivers outcomes on main open-weight benchmarks and is on par with main proprietary methods on real-world video duties. The mannequin meets main open-weight efficiency on short-video understanding benchmarks resembling MVBench, MotionQA, and NextQA.
It affords enhancements in video grounding accuracy, usually doubling or tripling the scores of earlier open fashions and surpassing proprietary APIs on a number of pointing and counting duties, Ai2 claimed. The mannequin additionally affords monitoring outcomes throughout multi-domain benchmarks, outperforming sturdy open baselines and several other business closed fashions.
As well as, Molmo 2 options picture and multi-image reasoning that rivals or exceeds bigger open-weight methods regardless of utilizing fewer parameters. Ai2 asserted that human desire evaluations confirmed that Molmo 2 is on par with or higher than a number of proprietary methods on real-world video QA and captioning duties.
Ai2 affords open knowledge and recipes
For transparency and reproducibility, all of the coaching sources for Molmo 2 are supplied within the technical report. Ai2 can be releasing a set of 9 new open datasets used to coach Molmo 2, totaling greater than 9 million multimodal examples throughout dense video captions, long-form QA, grounding, monitoring, and multi-image reasoning.
The captioning corpus alone spans greater than 100,000 movies with detailed descriptions that common greater than 900 phrases every. The info combine covers video pointing, multi-object monitoring, artificial grounding, and long-video reasoning. Collectively, they kind one of the crucial full open video knowledge collections out there right this moment, claimed Ai2.
Molmo 2 is available in three fundamental variants: Molmo 2 (4B), Molmo2 (8B), and Molmo 2-O (7B), which makes use of Ai2’s absolutely open Olmo spine for the entire end-to-end mannequin stream. Variations tuned particularly for pointing and monitoring are additionally out there.
All fashions, datasets, and analysis instruments are actually publicly out there on GitHub, Hugging Face, and the Ai2 Playground for interactive testing. The corporate plans to launch the coaching code quickly.
