How to log food with a photo: what the camera gets right and wrong

How accurate is photo calorie logging?

Three recent studies put numbers on it.

• A 2026 study across three photo datasets tested current vision-language models on 1,463 US meal photos (the SNAPMe benchmark), 691 Japanese meals with dietitian-validated values and about 1,000 packaged-food images. The best model's mean energy error on the US meals was about 117 calories per meal, protein was off by about 6 g, and packaged foods were estimated more accurately than plated meals. Salt was estimated badly across the board.
• A 2026 comparison with registered dietitians photographed 15 standardised hospital meals under controlled conditions and weighed them for ground truth. The top models matched dietitians on energy and carbohydrate (correlations above 0.8, often within 10 percent) but were weaker on protein and over-estimated fat by more than 20 percent on average.
• A review of image-based dietary assessment from before the current generation of models made the same two points: images capture what memory misses, and portion size is the hard part.

For context, the classic study of self-reported intake found people under-reported what they ate by 47 percent on average. A photo taken at the table does not forget the fries.

Photo logging is good at	Photo logging is bad at
Naming what is on the plate	Oil, butter and dressing you cannot see
Energy and carbohydrate of a plated meal	Fat, which models tend to over-read
Packaged food with a visible label	What is in a drink (milk type, sugar, alcohol)
Catching the snack you would have omitted	Mixed dishes: curries, casseroles, stews, burrito
Consistency, meal after meal	Portion size with nothing for scale

The four photos that fail

1. The dressed salad. The leaves are 30 calories; the dressing and the cheese are 300. Say "with ranch" or "dressing on the side, used half".
2. The bowl. A chili, a curry, a pasta bake: the camera sees the top layer. Name the dish and the rough amount ("about two cups of the chicken curry").
3. The drink. A latte with oat milk, a smoothie, a glass of wine: colour tells the model very little. One word about the milk or the size fixes it.
4. The cooked-in fat. Stir-fries and anything pan-fried carry oil the photo cannot show. "Cooked in about a tablespoon of oil" is the most valuable six words in food logging.

How to take the photo

• Shoot from above or at 45 degrees with the whole plate in frame; include a fork or a hand if nothing else gives scale.
• One photo per plate, before you start. If you do not finish, follow up with "ate half".
• For anything with a label, photograph the barcode instead of the food: in focus, the bars filling a fifth of the frame or more, no glare across them. The label numbers are exact; the model's estimate is not.
• For a chain restaurant, type the order rather than photographing it. Chains publish their nutrition, and a text like "chipotle chicken bowl, brown rice, black beans, no sour cream" can be matched to the published components; a photo of the same bowl is a guess.
• Add a sentence when the photo cannot carry it. The combination of a photo and five words is better than either alone.

What a good photo logger does with the picture

The difference between a toy and a tool is what happens after the photo. A good one proposes an entry you can confirm or correct rather than silently saving a guess, pulls exact numbers from label data when it can, and learns your portions so the same bowl of oats is right the second time.

Bay does this over text. You send the photo to a normal iMessage, RCS or SMS thread, it replies with a proposal (what it thinks the meal is, the calories and macros, and how confident it is), and a 👍 or "yes" confirms it. Behind that reply:

• Barcodes are decoded on the server from the full-resolution photo, so a label photo logs the product's exact numbers from Open Food Facts or USDA data without the model guessing. A barcode that will not read is reported as such, with a tip to retake it, rather than estimated.
• Chain orders use published menus. Typed orders at 160-plus US chains, including build-your-own components at Chipotle and sized drinks at Starbucks, are matched to the chains' own nutrition data.
• Your portions are remembered. Every confirmed item updates a running average for that food. Say "that was two scoops" once and the correction wins from then on; a proposal that disagrees with your established portion by more than a quarter gets flagged so it can ask.
• Each entry says what it is based on: how many items were verified from a label or menu, how many came from your history, how many were estimated. Confidence is set by the data, not by how sure the model sounded.
• "Half of this", "same as yesterday" and a voice note instead of a photo all work.

Bay is free right now. For the texting side of logging (no photo at all), see how to track calories by texting; for photo-first apps and how they compare, see the Cal AI alternatives guide. If you would rather not use an app or a camera at all, five methods that last covers the alternatives, and eating at restaurants while losing weight covers the orders that are hardest to photograph.