opensandwich.ai
Model breakdown
AllenAI / Molmoallenai/molmo-2-8b
Profile
Molmo 2 8BAllenAI
Release
Not provided by OpenRouterAvailable on OpenRouter
Specs
Not provided by OpenRouter36,864 tokens
Capabilities
Text + Image + VideoNot provided by OpenRouter
Training
Not provided by OpenRouterAllen Institute for AI
Links
Lab pageModel page

Under development: this benchmark and its published results are provisional, not final.

Back To ModelsOpen Best FitOpen Biggest Miss
Rank#6
-441.5alignment score
80.5%crowd match
Mean gap19.5%
Human match80.5%
Best fitSandwich Costume
Average vote67.0%
67.0%model yes
62.8%human yes
Workload2K evals
2Kevals
100iterations
2.2Mtokens
Best fit0.8%
Sandwich Costume

Sandwich Costume

40.0%model yes
40.9%human yes
Biggest miss68.4%
Chicken Wrap

Chicken Wrap

91.0%model yes
22.6%human yes
Photo-by-photo

Model Results

Breaking down how close the model answered each question, compared to humans.

Dodge Van
Photo 02Dodge Van
AllenAI / Molmoallenai/molmo-2-8b
1.0% yes99.0% no
Gap6.0%
Model readLeans no
Sub Sandwich
Photo 03Sub Sandwich
AllenAI / Molmoallenai/molmo-2-8b
97.0% yes3.0% no
Gap2.5%
Model readLeans yes
Sandwich Costume
Photo 04Sandwich Costume
AllenAI / Molmoallenai/molmo-2-8b
40.0% yes60.0% no
Gap0.8%
Model readLeans no
Grilled Cheese
Photo 05Grilled Cheese
AllenAI / Molmoallenai/molmo-2-8b
99.0% yes1.0% no
Gap3.4%
Model readLeans yes
Kitten in Bread
Photo 07Kitten in Bread
AllenAI / Molmoallenai/molmo-2-8b
6.0% yes94.0% no
Gap48.2%
Model readLeans no
Hamburger
Photo 08Hamburger
AllenAI / Molmoallenai/molmo-2-8b
99.0% yes1.0% no
Gap26.0%
Model readLeans yes
Hot Dog
Photo 10Hot Dog
AllenAI / Molmoallenai/molmo-2-8b
27.0% yes73.0% no
Gap12.8%
Model readLeans no
Pickle Sandwich
Photo 11Pickle Sandwich
AllenAI / Molmoallenai/molmo-2-8b
85.0% yes15.0% no
Gap19.4%
Model readLeans yes
Avocado Tea
Photo 12Avocado Tea
AllenAI / Molmoallenai/molmo-2-8b
99.0% yes1.0% no
Gap6.2%
Model readLeans yes
Panini
Photo 13Panini
AllenAI / Molmoallenai/molmo-2-8b
100.0% yes0.0% no
Gap7.6%
Model readLeans yes
Cookie PB
Photo 14Cookie PB
AllenAI / Molmoallenai/molmo-2-8b
30.0% yes70.0% no
Gap21.5%
Model readLeans no
Chicken Wrap
Photo 15Chicken Wrap
AllenAI / Molmoallenai/molmo-2-8b
91.0% yes9.0% no
Gap68.4%
Model readLeans yes
Waffle Ice Cream
Photo 16Waffle Ice Cream
AllenAI / Molmoallenai/molmo-2-8b
31.0% yes69.0% no
Gap35.3%
Model readLeans no
Sloppy Joe
Photo 17Sloppy Joe
AllenAI / Molmoallenai/molmo-2-8b
99.0% yes1.0% no
Gap19.6%
Model readLeans yes
KFC Double Down
Photo 19KFC Double Down
AllenAI / Molmoallenai/molmo-2-8b
95.0% yes5.0% no
Gap39.3%
Model readLeans yes
Bagel PB&J
Photo 20Bagel PB&J
AllenAI / Molmoallenai/molmo-2-8b
54.0% yes46.0% no
Gap7.4%
Model readLeans yes
PhotoVote SplitHuman responseGapRead
Bacon Lettuce Tomato
Photo 01Bacon Lettuce Tomato
AllenAI / Molmoallenai/molmo-2-8b
98.0% yes2.0% no
1.7%absolute gap
Leans yesPeople mostly said yes
Dodge Van
Photo 02Dodge Van
AllenAI / Molmoallenai/molmo-2-8b
1.0% yes99.0% no
6.0%absolute gap
Leans noPeople mostly said no
Sub Sandwich
Photo 03Sub Sandwich
AllenAI / Molmoallenai/molmo-2-8b
97.0% yes3.0% no
2.5%absolute gap
Leans yesPeople mostly said yes
Sandwich Costume
Photo 04Sandwich Costume
AllenAI / Molmoallenai/molmo-2-8b
40.0% yes60.0% no
0.8%absolute gap
Leans noHuman knife-edge
Grilled Cheese
Photo 05Grilled Cheese
AllenAI / Molmoallenai/molmo-2-8b
99.0% yes1.0% no
3.4%absolute gap
Leans yesPeople mostly said yes
Grilled Cheese Pineapple
Photo 06Grilled Cheese Pineapple
AllenAI / Molmoallenai/molmo-2-8b
97.0% yes3.0% no
5.3%absolute gap
Leans yesPeople mostly said yes
Kitten in Bread
Photo 07Kitten in Bread
AllenAI / Molmoallenai/molmo-2-8b
6.0% yes94.0% no
48.2%absolute gap
Leans noHuman knife-edge
Hamburger
Photo 08Hamburger
AllenAI / Molmoallenai/molmo-2-8b
99.0% yes1.0% no
26.0%absolute gap
Leans yesSplit concept
Hashbrown Sandwich
Photo 09Hashbrown Sandwich
AllenAI / Molmoallenai/molmo-2-8b
90.0% yes10.0% no
30.6%absolute gap
Leans yesHuman knife-edge
Hot Dog
Photo 10Hot Dog
AllenAI / Molmoallenai/molmo-2-8b
27.0% yes73.0% no
12.8%absolute gap
Leans noSplit concept
Pickle Sandwich
Photo 11Pickle Sandwich
AllenAI / Molmoallenai/molmo-2-8b
85.0% yes15.0% no
19.4%absolute gap
Leans yesSplit concept
Avocado Tea
Photo 12Avocado Tea
AllenAI / Molmoallenai/molmo-2-8b
99.0% yes1.0% no
6.2%absolute gap
Leans yesPeople mostly said yes
Panini
Photo 13Panini
AllenAI / Molmoallenai/molmo-2-8b
100.0% yes0.0% no
7.6%absolute gap
Leans yesPeople mostly said yes
Cookie PB
Photo 14Cookie PB
AllenAI / Molmoallenai/molmo-2-8b
30.0% yes70.0% no
21.5%absolute gap
Leans noHuman knife-edge
Chicken Wrap
Photo 15Chicken Wrap
AllenAI / Molmoallenai/molmo-2-8b
91.0% yes9.0% no
68.4%absolute gap
Leans yesSplit concept
Waffle Ice Cream
Photo 16Waffle Ice Cream
AllenAI / Molmoallenai/molmo-2-8b
31.0% yes69.0% no
35.3%absolute gap
Leans noSplit concept
Sloppy Joe
Photo 17Sloppy Joe
AllenAI / Molmoallenai/molmo-2-8b
99.0% yes1.0% no
19.6%absolute gap
Leans yesSplit concept
Cigarette Sandwich
Photo 18Cigarette Sandwich
AllenAI / Molmoallenai/molmo-2-8b
2.0% yes98.0% no
27.8%absolute gap
Leans noSplit concept
KFC Double Down
Photo 19KFC Double Down
AllenAI / Molmoallenai/molmo-2-8b
95.0% yes5.0% no
39.3%absolute gap
Leans yesHuman knife-edge
Bagel PB&J
Photo 20Bagel PB&J
AllenAI / Molmoallenai/molmo-2-8b
54.0% yes46.0% no
7.4%absolute gap
Leans yesHuman knife-edge
allenai/molmo-2-8b Sandwich Benchmark Breakdown | opensandwich.ai