Artificial intelligence can associate ideas in a similar way to how a human mind does. Two scientists, one of them from the Pompeu Fabra University (UPF) in Barcelona, ??have achieved that an artificial neural network is capable of combining new concepts with previously known ones, a challenge considered impossible in the last 35 years. The advance, published this Wednesday in the journal Nature, opens the possibility of training AI models in a more economical and accessible way.
The artificial intelligences developed so far learn by brute force. In training, they are provided with a huge amount of data so that they are able to understand and execute each of the instructions they receive. However, this training limits the systems to repeating what they have learned, preventing them from having sufficient flexibility to link new concepts with others that they already knew.
If, for example, we taught an artificial intelligence to “take a step twice” and “jump,” the machine would be unable to understand the instruction “jump twice.” In 1988 it was proposed that an artificial intelligence would never have this ability to apply known instructions to new concepts, an inherently human ability known as systematic generalization, so the field rejected that artificial neural models could in any way replicate our minds and our way of learning.
The challenge has survived 35 years of advances in the field, until an article published this Wednesday in the journal Nature dared to challenge it. And the two scientists behind the paper, who work in the United States and Barcelona respectively, claim to have found a solution that does not involve developing an extremely novel model, but rather changing the way the machine learns. In fact, they have used an architecture identical to that used by ChatGPT, but on a smaller, simpler scale.
“The network does not learn by repeating examples and remembering, but rather it learns to generalize,” explains Marco Baroni, researcher at the Department of Translation and Language Sciences at the Pompeu Fabra University (UPF) in Barcelona and co-author of the study, in conversation with La Vanguardia. The learning they have subjected the machine to focuses, ultimately, on teaching it to organize concepts in a logical order, instead of memorizing a static database.
That is, instead of teaching the machine the meaning of “take a step twice” and “jump twice” separately, they have shown it what it means to “take a step”, “take a step twice” and “hop”. Then they asked him to “jump twice” and compared the action he carried out with what they expected to see. By doing this with many different exercises, in a way the AI ??understands that whenever a known slogan appears after any word, whether it knows it or not, it must apply the action to the word. You can even combine instructions you’ve never seen together.
To evaluate the success of their model, the researchers analyzed how it solves simple algebraic operations compared to humans. In the experiment, they invented a set of words and associated each with a color or an action (such as reversing the order of the next two colors, for example). They then asked the participants and the machine to interpret the sentences made with those words, and transform them into the corresponding color code.
With a success rate of around 82%, the machine achieved slightly better results than the participants. In comparison, “the most recent version of ChatGPT correctly solved about 58% of the problems we presented,” Baroni notes, “much worse” than his model.
Despite the revolutionary nature of the work, scientists point out that its success is relative as it occurs in a very limited system, that is, in an artificial intelligence that does not have a wealth and variety of concepts that allow it to be useful in the real world. Generalization to more complex systems should not be too problematic, Baroni argues, although he recognizes that things are never as simple as they seem.
Be that as it may, the objective is not so much that this ability to link new concepts with already acquired knowledge can be applied to already developed systems such as ChatGPT or Dall·e, but rather that it allows the creation of new AI models that require fewer resources. In the end, the UPF expert points out, his proposal allows artificial neural networks to learn in a more efficient way, which can facilitate the appearance of open source models developed by and for researchers from public centers.
