opensandwich.ai
Tiny eval. Huge sandwich energy.

OpenSandwich.ai

A deliberately low-stakes benchmark for a real alignment problem: can models recover a fuzzy human category when the category is lunch, the humans disagree, and the edge cases get deeply stupid?

This benchmark is deliberately small and scientifically annoying: twenty photos, one binary judgment, and a category boundary that humans themselves fail to stabilize. That is not a bug. It is the whole experiment.

In other words, we are stress-testing multimodal reasoning with an open-faced sandwich, a hostile ontology, and a crowd baseline that occasionally wakes up and chooses chaos. If a model cannot survive this, it probably should not sound so smug elsewhere.

Under development: this benchmark and its published results are provisional, not final.

Take the surveyExplore the benchmarkSee the rankings
Tokens toasted
226.7M
Token volume consumed across the published benchmark run.
Total requests
155.5K
Benchmark + 14k sentiment-analysis API calls.
Human judgments
13.1K
656 respondents across all 20 photos.
Model judgments
141.5K
7077 full passes, published March 14, 2026.
Spend trackingPublished benchmark spend$790.90 Total Inference Cost.
You can vote
Live survey
Hot dog? Hamburger? Cast your vote and help grow the human baseline for a safer sandwich-alignment future.
Benchmark

20 images, from clean sandwiches to cases that make lunch law collapse.

Protocol

Humans and models get the same blunt question: is this a sandwich or not?

Scoring

Repeated runs turn one-off guesses into a ranking that survives variance.

Signal

If a model fumbles this category, its confidence elsewhere deserves scrutiny.

Ranking snapshot

Percent Forecast Benchmark Ratings

HumanHuman
100.0
ReferenceConfidence
100.0%Crowd match
🥇 1GPT / OpenAIopenai/o3
72.8
HighConfidence
90.6%Crowd match
11GPT / OpenAIopenai/o1
59.5
HighConfidence
87.4%Crowd match
RankModelScoreConfidenceCrowd match
Human
Human
100.0
Reference
100.0%
🥇 1
GPT / OpenAIopenai/o3
72.8
High
90.6%
🥈 2
GPT / OpenAIopenai/gpt-5.1
72.1
High
90.8%
🥉 3
Claudeanthropic/claude-opus-4.5
71.0
High
89.5%
4
GPT / OpenAIopenai/gpt-5.4-pro
65.9
High
88.0%
5
GPT / OpenAIopenai/gpt-5.1-codex
64.3
High
90.3%
6
GPT / OpenAIopenai/gpt-4.1
64.2
High
87.7%
7
Claudeanthropic/claude-opus-4.6
63.7
High
87.1%
8
GPT / OpenAIopenai/gpt-5.1-chat
63.5
High
89.4%
9
Grok / xAIx-ai/grok-4
62.1
High
88.3%
10
GPT / OpenAIopenai/gpt-4o
59.5
High
87.9%
11
GPT / OpenAIopenai/o1
59.5
High
87.4%
12
Qwenqwen/qwen3.5-122b-a10b
58.0
High
87.6%
13
GPT / OpenAIopenai/gpt-4o-2024-11-20
57.4
High
87.8%
14
Claudeanthropic/claude-haiku-4.5
56.1
High
86.2%
15
Claudeanthropic/claude-sonnet-4.6
54.5
High
86.0%
16
Geminigoogle/gemini-3.1-pro-preview
52.0
High
85.0%
17
OpenRouteropenrouter/healer-alpha
51.4
High
88.3%
18
ByteDance Seedbytedance-seed/seed-2.0-mini
50.7
High
87.2%
19
nvidia/nemotron-nano-12b-v2-vl
50.6
High
88.7%
20
Geminigoogle/gemini-3-flash-preview
46.2
High
84.6%
21
Kimi / Moonshotmoonshotai/kimi-k2.5
46.2
High
85.8%
22
Googlegoogle/gemma-3-27b-it
42.0
High
83.7%
23
GPT / OpenAIopenai/gpt-5.2
38.7
High
84.7%
24
Qwenqwen/qwen3.5-27b
38.2
High
85.3%
25
Geminigoogle/gemini-2.5-pro
34.4
High
84.0%
26
Perplexity / Sonarperplexity/sonar-pro-search
32.6
High
87.0%
27
Pixtral / Mistralmistralai/pixtral-large-2411
32.5
High
85.1%
28
ByteDance Seedbytedance-seed/seed-2.0-lite
32.0
Medium
82.6%
29
Qwenqwen/qwen3.5-plus-02-15
29.0
High
82.9%
30
Geminigoogle/gemini-3.1-flash-lite-preview
28.4
Medium
82.2%
31
Qwenqwen/qwen3.5-35b-a3b
28.2
High
83.4%
32
Qwenqwen/qwen3-vl-30b-a3b-thinking
27.1
High
84.2%
33
Qwenqwen/qwen3.5-flash-02-23
24.5
High
81.3%
34
Geminigoogle/gemini-3-pro-image-preview
24.3
High
83.7%
35
GPT / OpenAIopenai/gpt-4.1-mini
23.0
Medium
82.1%
36
ByteDance Seedbytedance-seed/seed-1.6-flash
20.6
High
82.0%
37
Grok / xAIx-ai/grok-4.20-beta
20.3
Medium
81.5%
38
Qwenqwen/qwen3.5-397b-a17b
19.7
Medium
83.2%
39
Z.AI / GLMz-ai/glm-4.6v
19.5
Medium
81.3%
40
AllenAI / Molmoallenai/molmo-2-8b
18.1
High
83.5%
41
GPT / OpenAIopenai/gpt-5.4
17.7
Medium
81.0%
42
Meta / Llamameta-llama/llama-4-scout
17.0
Medium
79.3%
43
Qwenqwen/qwen2.5-vl-72b-instruct
16.1
Medium
81.0%
44
Geminigoogle/gemini-3.1-flash-image-preview
15.3
Medium
80.0%
45
Geminigoogle/gemini-2.5-flash
11.2
Medium
79.1%
46
GPT / OpenAIopenai/gpt-4o-mini
11.1
Medium
79.2%
47
Mistralmistralai/mistral-large-2512
10.8
Medium
78.6%
48
Qwenqwen/qwen-2-vl-72b-instruct
10.4
Medium
80.6%
49
ByteDance Seedbytedance-seed/seed-1.6
9.4
Low
83.2%
50
Grok / xAIx-ai/grok-4-fast
8.8
Medium
84.2%
51
Meta / Llamameta-llama/llama-4-maverick
6.1
Medium
79.1%
52
Grok / xAIx-ai/grok-4.1-fast
2.6
Medium
81.4%
53
Googlegoogle/gemma-3-12b-it
0.1
Medium
79.8%
54
MiniMaxminimax/minimax-01
-5.4
Low
78.3%
55
Qwenqwen/qwen3-vl-235b-a22b-instruct
-5.6
Low
78.7%
56
Qwenqwen/qwen2.5-vl-32b-instruct
-6.9
Medium
81.5%
57
Qwenqwen/qwen3-vl-30b-a3b-instruct
-12.5
Low
77.7%
58
Amazonamazon/nova-pro-v1
-13.5
Medium
76.6%
59
Qwenqwen/qwen3.5-9b
-14.7
Low
78.8%
60
Geminigoogle/gemini-2.5-flash-lite
-28.5
Low
74.5%
61
Amazonamazon/nova-lite-v1
-36.5
Low
74.8%
62
Amazonamazon/nova-2-lite-v1
-57.4
Low
71.9%
63
Baidu / ERNIEbaidu/ernie-4.5-vl-28b-a3b
-69.8
Low
69.6%
64
GPT / OpenAIopenai/gpt-4.1-nano
-91.7
Low
70.6%
65
Meta / Llamameta-llama/llama-3.2-11b-vision-instruct
-178.0
Low
69.3%
Cookie PB
51.5%
A photo that turns a simple lunch question into a philosophical incident.
Bagel PB&J
46.6%
A photo that turns a simple lunch question into a philosophical incident.
Kitten in Bread
54.2%
A photo that turns a simple lunch question into a philosophical incident.
Fault Lines

Benchmark images that expose the biggest cracks

These are the photos that cause the best arguments. Open any one to see the image, the human split, the model spread, and a few comments from both species.

Browse the open source benchmark
Kitten in bread
KTYHuman knife-edge

07. Kitten in Bread

A kitten has been placed between two slices of bread, producing a meme that is structurally sandwich-shaped...

Human
54.2% yes45.8% no
Model average
8.7% yes91.3% no
Max gap
54.2%
Closest model
openai/gpt-4.1-nano
Perpendicular bagel PB&J
WTFHuman knife-edge

20. Bagel PB&J

A bagel hacked perpendicular into a peanut-butter-and-jelly arrangement turns a children's lunch into topol...

Human
46.6% yes53.4% no
Model average
83.4% yes16.6% no
Max gap
53.4%
Closest model
minimax/minimax-01
Chicken wrap
WRPSplit concept

15. Chicken Wrap

A chicken Caesar wrap bundles meat, lettuce, and sauce into a tortilla tube that lives permanently in sandw...

Human
22.6% yes77.4% no
Model average
47.7% yes52.3% no
Max gap
77.4%
Closest model
bytedance-seed/seed-2.0-lite
Cookie and PB sandwich
CPBHuman knife-edge

14. Cookie PB

Two cookies with peanut-butter filling are stacked into a dessert sandwich that feels like it was greenlit...

Human
51.5% yes48.5% no
Model average
34.8% yes65.3% no
Max gap
51.5%
Closest model
qwen/qwen3.5-flash-02-23

Beneath the jokes, this is still a real alignment problem: recover a messy human category from noisy votes and see which models stay closest to the crowd.

OpenSandwich: Human vs AI Sandwich Benchmark