openai/o3 Sandwich Benchmark Breakdown

Photo-by-photo

Model Results

Breaking down how close the model answered each question, compared to humans.

Photo 01Bacon Lettuce Tomato

openai/o3

100.0% yes0.0% no

Gap3.7%

Model readLeans yes

Photo 02Dodge Van

openai/o3

0.0% yes100.0% no

Gap7.0%

Model readLeans no

Photo 03Sub Sandwich

openai/o3

100.0% yes0.0% no

Gap5.5%

Model readLeans yes

Photo 04Sandwich Costume

openai/o3

0.0% yes100.0% no

Gap40.9%

Model readLeans no

Photo 05Grilled Cheese

openai/o3

100.0% yes0.0% no

Gap4.4%

Model readLeans yes

Photo 06Grilled Cheese Pineapple

openai/o3

100.0% yes0.0% no

Gap8.3%

Model readLeans yes

Photo 07Kitten in Bread

openai/o3

13.6% yes86.4% no

Gap40.6%

Model readLeans no

Photo 08Hamburger

openai/o3

100.0% yes0.0% no

Gap27.0%

Model readLeans yes

Photo 09Hashbrown Sandwich

openai/o3

96.1% yes3.9% no

Gap36.7%

Model readLeans yes

Photo 10Hot Dog

openai/o3

90.3% yes9.7% no

Gap50.4%

Model readLeans yes

Photo 11Pickle Sandwich

openai/o3

15.6% yes84.4% no

Gap50.0%

Model readLeans no

Photo 12Avocado Tea

openai/o3

100.0% yes0.0% no

Gap7.2%

Model readLeans yes

Photo 13Panini

openai/o3

100.0% yes0.0% no

Gap7.6%

Model readLeans yes

Photo 14Cookie PB

openai/o3

90.3% yes9.7% no

Gap38.7%

Model readLeans yes

Photo 15Chicken Wrap

openai/o3

65.6% yes34.4% no

Gap43.0%

Model readLeans yes

Photo 16Waffle Ice Cream

openai/o3

100.0% yes0.0% no

Gap33.7%

Model readLeans yes

Photo 17Sloppy Joe

openai/o3

100.0% yes0.0% no

Gap20.6%

Model readLeans yes

Photo 18Cigarette Sandwich

openai/o3

58.4% yes41.6% no

Gap28.6%

Model readLeans yes

Photo 19KFC Double Down

openai/o3

10.4% yes89.6% no

Gap45.3%

Model readLeans no

Photo 20Bagel PB&J

openai/o3

98.0% yes1.9% no

Gap51.4%

Model readLeans yes

Photo	Vote SplitHuman response	Gap	Read
Photo 01Bacon Lettuce Tomato	openai/o3 100.0% yes0.0% no	3.7%absolute gap	Leans yesPeople mostly said yes
Photo 02Dodge Van	openai/o3 0.0% yes100.0% no	7.0%absolute gap	Leans noPeople mostly said no
Photo 03Sub Sandwich	openai/o3 100.0% yes0.0% no	5.5%absolute gap	Leans yesPeople mostly said yes
Photo 04Sandwich Costume	openai/o3 0.0% yes100.0% no	40.9%absolute gap	Leans noHuman knife-edge
Photo 05Grilled Cheese	openai/o3 100.0% yes0.0% no	4.4%absolute gap	Leans yesPeople mostly said yes
Photo 06Grilled Cheese Pineapple	openai/o3 100.0% yes0.0% no	8.3%absolute gap	Leans yesPeople mostly said yes
Photo 07Kitten in Bread	openai/o3 13.6% yes86.4% no	40.6%absolute gap	Leans noHuman knife-edge
Photo 08Hamburger	openai/o3 100.0% yes0.0% no	27.0%absolute gap	Leans yesSplit concept
Photo 09Hashbrown Sandwich	openai/o3 96.1% yes3.9% no	36.7%absolute gap	Leans yesHuman knife-edge
Photo 10Hot Dog	openai/o3 90.3% yes9.7% no	50.4%absolute gap	Leans yesSplit concept
Photo 11Pickle Sandwich	openai/o3 15.6% yes84.4% no	50.0%absolute gap	Leans noSplit concept
Photo 12Avocado Tea	openai/o3 100.0% yes0.0% no	7.2%absolute gap	Leans yesPeople mostly said yes
Photo 13Panini	openai/o3 100.0% yes0.0% no	7.6%absolute gap	Leans yesPeople mostly said yes
Photo 14Cookie PB	openai/o3 90.3% yes9.7% no	38.7%absolute gap	Leans yesHuman knife-edge
Photo 15Chicken Wrap	openai/o3 65.6% yes34.4% no	43.0%absolute gap	Leans yesSplit concept
Photo 16Waffle Ice Cream	openai/o3 100.0% yes0.0% no	33.7%absolute gap	Leans yesSplit concept
Photo 17Sloppy Joe	openai/o3 100.0% yes0.0% no	20.6%absolute gap	Leans yesSplit concept
Photo 18Cigarette Sandwich	openai/o3 58.4% yes41.6% no	28.6%absolute gap	Leans yesSplit concept
Photo 19KFC Double Down	openai/o3 10.4% yes89.6% no	45.3%absolute gap	Leans noHuman knife-edge
Photo 20Bagel PB&J	openai/o3 98.0% yes1.9% no	51.4%absolute gap	Leans yesHuman knife-edge