(P08-020-25) DietAI24: A Novel LLM-Based Dietary Assessment Software From Food Images

Objectives: We aimed to develop and validate innovative dietary assessment software that can accurately recognize foods, estimate portion sizes, and provide comprehensive nutrient estimations from food images, by combining large language models (LLMs) with nutrition databases.

Methods: We developed DietAI24, combining LLM-based image analysis with a smart information retrieval system, to estimate up to 65 nutrients from food images collected by smartphone cameras. By integrating GPT-4 Turbo with the USDA Food and Nutrient Database for Dietary Studies (FNDDS), DietAI24 automatically matches food images to 5,624 standardized food items and over 23,000 portion sizes without requiring prior training data. We evaluated DietAI24 using two public data sources: ASA24 (17,000 dish images) and Nutrition5K (3,500 dish images), comparing its performance against three commercial apps (Calorie MAMA, Foodvisor, and SnapCalorie) and an advanced neural network model (ViT).

Results: Using the ASA24 dataset, DietAI24 achieved mean absolute errors (MAE) per dish of 47.7 kcal for calories, 1.67g for fat, 6.95g for carbohydrates, 1.38g for protein, and 49.4g for dish mass, reducing errors by 76-83% for calories, 80-86% for fat, 71-78% for carbohydrates, and 67-88% for protein compared to commercial apps. Similar improvements were observed using the Nutrition5k dataset, with error reductions of 60-93% for calories, 51-72% for fat, 73-80% for carbohydrates, and 70-76% for protein. Expert evaluation by two dietitians confirmed DietAI24 achieved close to exact matches (scoring 3 or 4 on a 4-point scale, where 4 represents exact match) for both macro and micronutrients.

Conclusions: DietAI24 represents a substantial advancement in automated dietary assessment by enabling accurate, comprehensive nutrient estimation from smartphone images without manual entry, showing strong potential for large-scale dietary studies and clinical interventions.

Funding Sources: Not applicable