Kroger is using visual recognition AI to help people make better use of the leftovers in their fridge.
The grocery chain and agency 360i recently launched a Twitter account called Chefbot that can identify up to three foods in user-submitted images and search a database of recipes for the best match. The makers claim the bot is trained to recognize 2,000 ingredients and pulls from a log of around 20,000 recipes.
While the tool is fairly rudimentary for now, Kroger has big plans to grow it into something more sophisticated over time, along with 360i and tech partners Coffee Labs and Clarifai. The creators plan to incorporate every interaction the bot has into its training data in order to make future iterations more accurate at identifying foods and customizing recipes.
“Over time, Chefbot will continuously learn and improve, and we’re excited about how the experience will evolve and can eventually integrate into Kroger’s mobile app,” Menno Kluin, chief creative officer at 360i, said. “This tool makes such good sense for Kroger.”
The tool may have a ways to go before it is ready for such app integration and use beyond a fun Twitter diversion. The current iteration of the bot is rarely able to guess all three ingredients in a picture—although it can usually identify at least one.
The chefbot’s prompt for users to confirm the ingredients after it attempts to identify them can also give Kroger valuable labeled data as it trains future iterations of the project.
Kroger is also billing the project in part as a way to cut down on food waste, part of its social good campaign around sustainability and grocery accessibility. “It aligns with their mission to reduce food waste, while providing a much-needed utility for people who are spending more time cooking at home than ever before and looking to maximize their purchases,” Kluin said.
Other companies are also exploring how AI and computer vision in particular might be used in the kitchen. McCormick spices teamed up with IBM last year on a project that used AI to invent new flavor combinations, and Facebook’s research lab has been working on a tool that guesses recipe ingredients with AI image recognition of a meal.