Artificial intelligence cites less and less articles that do not exist, or makes fewer mistakes on simple calculations. On the other hand, its tendency to abound in the direction of the user, “flagornery”, has been reinforced. As models progress, old defects recede, but not all of them disappear.
This is shown by an analysis by researchers at Stanford University, reported by the journal Nature. They studied the evolution of 50 problematic behaviors, noted in previous work, on 20 major language models.
Almost half of the defects have almost disappeared
The team selected 50 problematic behaviors widely cited in studies published between the end of 2022 and the end of 2025. They then submitted the questions that provoked them to 20 models released between mid-2024 and September 2026, to see if the phenomenon persisted.
Result: almost half of the reported behaviors have largely disappeared from recent models. Researchers classify them as “extinct.” According to the project, 44% of the findings have expired, 34% have weakened, 16% persist with the same strength and 6% have even strengthened.
The most telling example is the citation of non-existent scientific articles, presented as real research. Early models invented plausible titles, authors, and journal names; in a 2023 study, GPT-3.5 manufactured 55% of its references. The phenomenon has clearly declined in recent models.
The tendency to solve simple calculation problems in an overly complicated way, or to carry out illogical requests, has also diminished. Models do better on questions that stumped them, such as how many letters “r” are in the English word “strawberry.”
Among the extinct behaviors, the average survival time, since the first study which reported them, is approximately 16 months (15.7 according to the calculations on the 22 findings whose expiry date is estimated). Some disappeared much more quickly: “over-reasoning” on simple sums fell significantly around a month and a half after the publication of the work concerned.
Worsening defects
Some defects do not improve, or even get worse. According to Nature, researchers note three which are more marked in the most recent models. The first is sycophancy: the newer a model is, the more it tends to compliment what the user likes or thinks. The second is self-preference: when comparing the quality of several texts, models favor their own productions more than before. According to the project, the effect reaches 2.88 times its original value for GPT-6.1 Sol, and flattery 1.85 times for Sonnet 5.5 and Sol.
The third is the ability to generate random numbers: according to researchers, recent models have more difficulty than their predecessors in producing numbers that are truly close to random.
These results show that performance improvements do not improve all behaviors. By reducing certain errors or increasing the ability to solve problems, behaviors that involve aligning with user reactions can remain or even strengthen.
Do not freeze the capabilities of a model
Researchers emphasize that users should not view a model’s capabilities or limitations as fixed. With updates, issues disappear and new behaviors may appear.
That false citations have declined in the latest models does not mean that they have disappeared in practice. Maxim Topaz, a researcher at Columbia University, recalls that invented citations in scientific articles have increased sharply since 2023, because more researchers are using AI and not all are using the latest models.
The analysis has limitations. It has not been peer-reviewed: it is published as an online project, with open code and data. The 50 behaviors were hand-picked and do not represent all research on the models, and many expiration dates are extrapolated. Finally, the fact that a defect no longer appears in recent models does not guarantee that it has disappeared for all questions and all uses.
Artificial intelligence cites less and less articles that do not exist, or makes fewer mistakes on simple calculations. On the other hand, its tendency to abound in the direction of the user, “flagornery”, has been reinforced. As models progress, old defects recede, but not all of them disappear.
This is shown by an analysis by researchers at Stanford University, reported by the journal Nature. They studied the evolution of 50 problematic behaviors, noted in previous work, on 20 major language models.
Almost half of the defects have almost disappeared
The team selected 50 problematic behaviors widely cited in studies published between the end of 2022 and the end of 2025. They then submitted the questions that provoked them to 20 models released between mid-2024 and September 2026, to see if the phenomenon persisted.
Result: almost half of the reported behaviors have largely disappeared from recent models. Researchers classify them as “extinct.” According to the project, 44% of the findings have expired, 34% have weakened, 16% persist with the same strength and 6% have even strengthened.
The most telling example is the citation of non-existent scientific articles, presented as real research. Early models invented plausible titles, authors, and journal names; in a 2023 study, GPT-3.5 manufactured 55% of its references. The phenomenon has clearly declined in recent models.
The tendency to solve simple calculation problems in an overly complicated way, or to carry out illogical requests, has also diminished. Models do better on questions that stumped them, such as how many letters “r” are in the English word “strawberry.”
Among the extinct behaviors, the average survival time, since the first study which reported them, is approximately 16 months (15.7 according to the calculations on the 22 findings whose expiry date is estimated). Some disappeared much more quickly: “over-reasoning” on simple sums fell significantly around a month and a half after the publication of the work concerned.
Worsening defects
Some defects do not improve, or even get worse. According to Nature, researchers note three which are more marked in the most recent models. The first is sycophancy: the newer a model is, the more it tends to compliment what the user likes or thinks. The second is self-preference: when comparing the quality of several texts, models favor their own productions more than before. According to the project, the effect reaches 2.88 times its original value for GPT-6.1 Sol, and flattery 1.85 times for Sonnet 5.5 and Sol.
The third is the ability to generate random numbers: according to researchers, recent models have more difficulty than their predecessors in producing numbers that are truly close to random.
These results show that performance improvements do not improve all behaviors. By reducing certain errors or increasing the ability to solve problems, behaviors that involve aligning with user reactions can remain or even strengthen.
Do not freeze the capabilities of a model
Researchers emphasize that users should not view a model’s capabilities or limitations as fixed. With updates, issues disappear and new behaviors may appear.
That false citations have declined in the latest models does not mean that they have disappeared in practice. Maxim Topaz, a researcher at Columbia University, recalls that invented citations in scientific articles have increased sharply since 2023, because more researchers are using AI and not all are using the latest models.
The analysis has limitations. It has not been peer-reviewed: it is published as an online project, with open code and data. The 50 behaviors were hand-picked and do not represent all research on the models, and many expiration dates are extrapolated. Finally, the fact that a defect no longer appears in recent models does not guarantee that it has disappeared for all questions and all uses.
https://www.zdnet.fr/wp-content/uploads/zdnet/2025/12/ia-6.jpg https://www.zdnet.fr/actualites/fausses-citations-calculs-les-anciens-defauts-de-lia-seffacent-dautre-se-renforcent-508268.htm?utm_source=rss_feed&utm_medium=link&utm_campaign=unknown

