“A tool to better understand the client and help in decision making.” This is how the Technological Institute of Aragon (ITA) defines the sensory table that they have devised to capture and analyze, using artificial intelligence, the emotional reactions that the presentation of a dish or variations in its composition provoke in diners. Information that, once processed, can be taken into account by those responsible for the kitchen and dining room to improve the experience they offer in their premises.
At first glance, this ‘smart’ table does not differ from any other table except for one small detail: the 360º camera camouflaged in a kind of vase that, from the center of the tablecloth, makes an initial biometry of the faces of those gathered there and He captures their reactions throughout the meal. Once on the computer, these expressions are analyzed in real time to detect barometers of anger, disgust, fear or joy, among others. With these graphs, the program itself makes a final report in which it puts in black and white the evolution of the diners’ sensations throughout the agape.
“The final objective is to quantify the emotions detected,” says Rafael del Hoyo, technological coordinator of the Artificial Intelligence department of the ITA. “A chef can design a dish or an explanation with the intention of provoking surprise, nostalgia or tension in the diner, and this tool allows him to verify on a scientific basis whether or not his expectations are met,” he adds.
From their team, which also includes Gorka Labata, José Ignacio Calvo and Marcos Caballero, they say that their project draws on the sources of neuromarketing that large multinationals have been applying for decades to measure the reactions of potential customers to a certain product. Until recently, they were too expensive technologies, but their spectacular reduction in cost has democratized their use, to the point that they can now be used by an SME, such as a restaurant.
One of the biggest challenges until reaching this point has been educating the system so that it learned to correctly detect and decipher each of the emotions: the expression that is put on when seeing the presentation of the dish, the tension generated by hearing certain ingredients (blood or organ meats, for example), the reaction to the first spoonful… “There are previous models for analyzing expressions, but this is the first specific one trained to evaluate people eating,” they say from their offices in Zaragoza.
To do this, they have developed their own database for more than a year with recordings of different people, including them, tasting dishes that generated different emotions and labeling them. It has not been a simple task, since the act of eating involves a series of gestures (looking at the plate, opening the mouth or chewing) that can lead to mistakes, so they have had to train the models specifically to ignore or understand the reason for these grimaces. They have also paid a gustatory toll: on the table there were delicious things, yes, but also tortillas with extra salt or very spicy rice that produced extreme reactions. “Sometimes it’s been hard,” they joke.
In developing the prototype, the ITA has worked with the Gastronomic Innovation Center of Aragón and the Provincial Association of Hospitality and Tourism Entrepreneurs of Huesca. Likewise, they have collaborated closely with well-known restaurants in the region -Callizo, Yaín or Gente Rara, among others-, who advised them and incorporated their vision as experts and who also agreed to participate in its debut in the latest edition of Madrid Fusion.
Looking to the immediate future, their plans are to continue with more tests to increase the database and thus further refine their analyzes and eliminate biases. For now, they have what they call a “minimum viable product,” and the ultimate goal would be to be able to market it and take it to restaurants so that they can try different versions of their menus before offering them. “It will be able to contribute to perfecting the dishes and the service in the room and to changing work systems when necessary,” says del Hoyo.
