Advanced Reasoning Lacks Reliability

This is a news story, published by Wired, that relates primarily to AI news.

News about Ai research

For more Ai research news, you can click here:

more Ai research news

Wired news

For more news from Wired, you can click here:

more news from Wired

About the Otherweb

Otherweb, Inc is a public benefit corporation, dedicated to improving the quality of news people consume. We are non-partisan, junk-free, and ad-free. We use artificial intelligence (AI) to remove junk from your news feed, and allow you to select the best tech news, business news, entertainment news, and much more. If you like this article about Ai research, you might also like this article about

reliable mathematical reasoning capabilities

. We are dedicated to bringing you the highest-quality news, junk-free and ad-free, about your favorite topics. Please come every day to read the latest reasoning processes news, inherent mathematical reasoning news, news about Ai research, and other high-quality news about any topic that interests you. We are working hard to create the best news aggregator on the web, and to put you in control of your news feed - whether you choose to read the latest news through our website, our news app, or our daily newsletter - all free!

complex reasoning capabilities

Wired

•

A new study shows that advanced AI models' "reasoning" can be extremely brittle and unreliable

Summary

Nutrition label

83% Informative

New study from six Apple engineers shows AI "reasoning" can be brittle and unreliable in the face of seemingly trivial changes to benchmark problems.

Apple researchers modified the GSM-Symbolic benchmark by adding "seemingly relevant but ultimately inconsequential statements" to the questions.

Results also showed high variance across 50 separate tests with different names and values.

"Current LLMs are not capable of genuine logical reasoning," the researchers hypothesize.

The GSM-Symbolic paper suggests LLMs don't actually perform formal reasoning and instead mimic it with probabilistic pattern-matching of the closest similar data seen in their vast training sets.

With enough training data and computation, the AI industry will likely reach what you might call "the illusion of understanding" with AI video synthesis.

VR Score

Informative language

Neutral language

Article tone

semi-formal

Language

English

Language complexity

Offensive language

not offensive

Hate speech

not hateful