Imagine you're a child hearing an ice cream truck's horn for the first time: you're intrigued, but without any particular expectations. Soon after, your parents surprise you with a delicious ice cream. The ice cream truck returns every day, and each time, your parents give you ice cream. As this scene repeats itself, the ice cream becomes less and less surprising, and the sound of the horn eventually becomes pleasant because it announces the reward. This well-known phenomenon, where a neutral stimulus (the horn) becomes associated with a response (the pleasure of ice cream), is called Pavlovian conditioning, named after the researcher who first studied it in dogs.
Classical theories describe the formation of associations between two events as proportional to the prediction error, that is, the difference between what is expected and what actually happens. In the ice cream truck example, on the first day, the horn doesn't yet predict the arrival of the ice cream: the error between the prediction and the reward is therefore very large, and the association increases. Later, when the ice cream becomes predictable, this error is reduced, and the association stabilizes.
Dopamine, a reflection of a "prediction error"
“ In the 1990s, researchers discovered that the dopamine, a neurotransmitter (chemical messenger in the brain, editor's note), reflects this prediction error. Experiments measuring dopamine have shown that when the prediction error is large, the dopamine peak is large; conversely, when the association is learned and the error becomes zero, the dopamine peak disappears. It has even been observed that when one expects to receive a reward but it does not materialize, which corresponds to a negative prediction error, a decrease in dopamine is observed. explains Noé Hamou, a neuroscience researcher at University College London (Great Britain) and author of the work published in the journal Nature Communications, in collaboration with researchers from Harvard and Princeton Universities (USA).
"This discovery is one of the great 'success stories' of modern neuroscience: it makes it possible to link a biological molecule and a psychological quantity, the prediction error."" , continues Noé Hamou. These prediction errors have allowed for a more precise analysis of our learning mechanisms, but have also inspired artificial intelligence algorithms. The resulting reinforcement learning models have made it possible to beat the best chess players or, more recently, of Goand all make use of these prediction errors.
Integrating the temporal dimension of learning
As research continues, some experimental results have proven inconsistent with this classical theory based on prediction errors. Indeed, this model fails to explain experiments showing that learning is extremely sensitive to time.
Several experimental psychology laboratories have demonstrated that the time lapse between events (signal-reward or reward-reward) has a significant impact on learning ability. By changing the duration of these intervals, the learning speed also changes. This temporal dimension is a central element of learning, and yet it remains absent from classical models., emphasizes Noé Hamou. We have therefore developed a new model that incorporates both this temporal dimension and prediction errors, in order to get as close as possible to the experimental results.
The model proposed by these researchers therefore incorporates a “temporal distribution of intervals” between stimulus and reward (at what “distance in time” rewards occur after a given stimulus) and the causal probability that a certain stimulus is the “cause” of the reward.
The probability of response depends on both a temporal component and a causal component between stimulus and reward. Credits: Hamou et al., 2025
In other words, the mathematical formula is as follows:
The probability of response P(response) is the sum over the set of stimuli considered i, of the temporal distribution probability P(Ti) multiplied by the probability of the causal association of the stimulus P(Ci).
Causal learning
But what makes this model particularly innovative? According to our model, learning requires both a predictive temporal signal, based on classical prediction errors, and a retrospective signal, which reflects the causality between two events."That sums it up," says Noé Hamou. In other words: "Given that I received a reward, what was the probability that a signal preceded it?" or even: "I had indigestion; I'm going to try to remember what could have caused it, and next time I'll probably avoid that food."
It now remains to discover where, in the brain, this causal learning mechanism is hidden. Other neurotransmitters such as acetylcholine and serotonin are among the suspects… but nothing has been decided yet.
English 
French