Technologies

AI Is Bad at Sudoku. It’s Even Worse at Showing Its Work

Researchers did more than ask chatbots to play games. They tested whether AI models could describe their thinking. The results were troubling.

Chatbots are genuinely impressive when you watch them do things they’re good at, like writing a basic email or creating weird, futuristic-looking images. But ask generative AI to solve one of those puzzles in the back of a newspaper, and things can quickly go off the rails.

That’s what researchers at the University of Colorado at Boulder found when they challenged large language models to solve sudoku. And not even the standard 9×9 puzzles. An easier 6×6 puzzle was often beyond the capabilities of an LLM without outside help (in this case, specific puzzle-solving tools).

A more important finding came when the models were asked to show their work. For the most part, they couldn’t. Sometimes they lied. Sometimes they explained things in ways that made no sense. Sometimes they hallucinated and started talking about the weather.

If gen AI tools can’t explain their decisions accurately or transparently, that should cause us to be cautious as we give these things more control over our lives and decisions, said Ashutosh Trivedi, a computer science professor at the University of Colorado at Boulder and one of the authors of the paper published in July in the Findings of the Association for Computational Linguistics.

“We would really like those explanations to be transparent and be reflective of why AI made that decision, and not AI trying to manipulate the human by providing an explanation that a human might like,” Trivedi said.

Don’t miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source.

The paper is part of a growing body of research into the behavior of large language models. Other recent studies have found, for example, that models hallucinate in part because their training procedures incentivize them to produce results a user will like, rather than what is accurate, or that people who use LLMs to help them write essays are less likely to remember what they wrote. As gen AI becomes more and more a part of our daily lives, the implications of how this technology works and how we behave when using it become hugely important.

When you make a decision, you can try to justify it, or at least explain how you arrived at it. An AI model may not be able to accurately or transparently do the same. Would you trust it?

Why LLMs struggle with sudoku

We’ve seen AI models fail at basic games and puzzles before. OpenAI’s ChatGPT (among others) has been totally crushed at chess by the computer opponent in a 1979 Atari game. A recent research paper from Apple found that models can struggle with other puzzles, like the Tower of Hanoi.

It has to do with the way LLMs work and fill in gaps in information. These models try to complete those gaps based on what happens in similar cases in their training data or other things they’ve seen in the past. With a sudoku, the question is one of logic. The AI might try to fill each gap in order, based on what seems like a reasonable answer, but to solve it properly, it instead has to look at the entire picture and find a logical order that changes from puzzle to puzzle.

Read more: 29 Ways You Can Make Gen AI Work for You, According to Our Experts

Chatbots are bad at chess for a similar reason. They find logical next moves but don’t necessarily think three, four or five moves ahead — the fundamental skill needed to play chess well. Chatbots also sometimes tend to move chess pieces in ways that don’t really follow the rules or put pieces in meaningless jeopardy.

You might expect LLMs to be able to solve sudoku because they’re computers and the puzzle consists of numbers, but the puzzles themselves are not really mathematical; they’re symbolic. “Sudoku is famous for being a puzzle with numbers that could be done with anything that is not numbers,” said Fabio Somenzi, a professor at CU and one of the research paper’s authors.

I used a sample prompt from the researchers’ paper and gave it to ChatGPT. The tool showed its work, and repeatedly told me it had the answer before showing a puzzle that didn’t work, then going back and correcting it. It was like the bot was turning in a presentation that kept getting last-second edits: This is the final answer. No, actually, never mind, this is the final answer. It got the answer eventually, through trial and error. But trial and error isn’t a practical way for a person to solve a sudoku in the newspaper. That’s way too much erasing and ruins the fun.

AI struggles to show its work

The Colorado researchers didn’t just want to see if the bots could solve puzzles. They asked for explanations of how the bots worked through them. Things did not go well.

Testing OpenAI’s o1-preview reasoning model, the researchers saw that the explanations — even for correctly solved puzzles — didn’t accurately explain or justify their moves and got basic terms wrong.

“One thing they’re good at is providing explanations that seem reasonable,” said Maria Pacheco, an assistant professor of computer science at CU. “They align to humans, so they learn to speak like we like it, but whether they’re faithful to what the actual steps need to be to solve the thing is where we’re struggling a little bit.”

Sometimes, the explanations were completely irrelevant. Since the paper’s work was finished, the researchers have continued to test new models released. Somenzi said that when he and Trivedi were running OpenAI’s o4 reasoning model through the same tests, at one point, it seemed to give up entirely.

“The next question that we asked, the answer was the weather forecast for Denver,” he said.

(Disclosure: Ziff Davis, CNET’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)

Explaining yourself is an important skill

When you solve a puzzle, you’re almost certainly able to walk someone else through your thinking. The fact that these LLMs failed so spectacularly at that basic job isn’t a trivial problem. With AI companies constantly talking about “AI agents” that can take actions on your behalf, being able to explain yourself is essential.

Consider the types of jobs being given to AI now, or planned for in the near future: driving, doing taxes, deciding business strategies and translating important documents. Imagine what would happen if you, a person, did one of those things and something went wrong.

“When humans have to put their face in front of their decisions, they better be able to explain what led to that decision,” Somenzi said.

It isn’t just a matter of getting a reasonable-sounding answer. It needs to be accurate. One day, an AI’s explanation of itself might have to hold up in court, but how can its testimony be taken seriously if it’s known to lie? You wouldn’t trust a person who failed to explain themselves, and you also wouldn’t trust someone you found was saying what you wanted to hear instead of the truth.

“Having an explanation is very close to manipulation if it is done for the wrong reason,” Trivedi said. “We have to be very careful with respect to the transparency of these explanations.”

Technologies

China indicates rare earths will keep flowing to EU, regardless of U.S. trade deal expiration

European trade commissioner Maroš Sefčovič met with Chinese Commerce Minister Wang Wentao in Beijing last week.

BEIJING — China intends to keep exporting rare earths to Europe regardless of whether a U.S. trade deal with Beijing gets extended past January.

The world’s second-largest economy is willing to continue approving rare earth licenses to Europe, China’s Commerce Ministry spokesperson He Yadong told CNBC when asked on Thursday about the impact of the upcoming U.S. deadline.

Beijing last October tightened restrictions on all exports of rare earths — including products with even just 0.1% of China-sourced rare earths. The move followed a new U.S. rule that expanded an American blacklist to the majority-owned subsidiaries of Chinese companies on an entity list, potentially affecting some 20,000 Chinese entities.

However, both the U.S. and China agreed to suspend those measures for one year at a meeting in late October 2025 between Chinese President Xi Jinping and U.S. President Donald Trump. Ahead of the leaders’ latest summit last month, U.S. Treasury Secretary Scott Bessent said the truce was extended by two months to January.

Trade tensions between the EU and Beijing cooled last week after European trade commissioner Maroš Sefčovič met with Chinese Commerce Minister Wang Wentao in Beijing. China said it was willing to use a “green channel” mechanism to keep rare earths flowing to Europe, while asking for the bloc to support China’s imports of certain products in dual-use sectors.

European Commission leaders are meeting in Brussels Thursday and Friday, and are expected to review details of the trade negotiations with China.

Technologies

Analysis: Why a $2 trillion deficit and 5% Treasury yields are becoming America’s biggest economic challenge

Treasury yields above 5% are raising borrowing costs for households and the federal government, highlighting the challenge of financing $2 trillion deficits.

Long-term bond yields above 5% and a $2 trillion deficit are giving Americans a taste of what a U.S. debt crisis would be like. A true emergency would hurt much more, and isn’t baked in. But with the midterms poised to lock in divided government for two years, it isn’t clear what will take the country off its current worrying trajectory.

Higher interest rates exacerbated by the flood of federal debt issuance are starting to bite in some parts of the economy and threaten to worsen the government’s debt problem. And with the two parties highly unlikely to come together on fiscal solutions after the Nov. 3 midterms, any fixes are years off in the future.

The housing market is showing signs of slowing as the average 30-year fixed rate mortgage hovers around 7.5%. Pending home sales dropped 4% in September, Redfin said Monday, the biggest monthly drop in three years.

Mortgage rates track the 10-year Treasury yield

Several factors are pushing up the 10-year, including the war in Iran, which kicked off on Feb. 28 and precipitated the rise in long-term yields. Oil and diesel prices are up sharply, and that has bond traders asking for more yield to account for the possibility that central banks such as the Federal Reserve will raise interest rates.

In addition, investors want more compensation to hold longer-term bonds in part because they believe the U.S. government will continue to flood the market with them. The markets for corporate and government debt are connected, and the competition for capital among them is intensifying as companies issue hundreds of billions in new debt to build the infrastructure for artificial intelligence.

Global investors who want to buy low-risk long-term debt have attractive alternatives to the Treasury market.

If the U.S. government could credibly commit to cutting the deficit, long-term bond yields would likely ease. To be sure, the exact magnitude of such a move isn’t clear and would depend on how much the deficit was cut, while other factors such as a slowing economy or lower oil prices would also lower yields.

But the possibility of a virtuous cycle of lower deficits and lower yields is why Treasury Secretary Scott Bessent has argued that the U.S. ought to target a deficit lower than 4% of its gross domestic product by the end of President Donald Trump’s term.

There is almost no way the U.S. will hit that target. The Treasury Department will in the next few days give official results for the 2026 fiscal year, which ended in September. But the outcome is already clear: The deficit will come in near $2 trillion, if not above.

That is likely to put the deficit in the range of 6.0-6.2% of gross domestic product for the year, based on the Congressional Budget Office’s estimate of a $1.993 trillion deficit and a projection of third quarter GDP.

Higher rates make debt reduction tougher. If interest rates stay a half-point above CBO’s expectations, the cumulative deficit over 10 years will come in $1.9 trillion higher, the agency estimates. If rates overshoot by 1.5 points, that adds $6 trillion in deficits.

CBO expects the 10-year Treasury to average 4.3% over that period, compared to 5.3% today. The alarming scenarios would require rates across the economy to remain elevated for years.

“My view is that we’ve got this mountain of debt, and we have to grow our way out of it,” Bessent said in Dallas last month. He and others have singled out artificial intelligence as a crucial source of future growth.

AI could help but can’t fix the problem entirely. It would take a decade of 7.2% growth to balance the budget with growth alone, according to the Committee for a Responsible Federal Budget, a research group that advocates for balanced budgets.

GDP growth was recently revised up for the second quarter from 1.5% to 2.2% at an annualized rate. The acceleration that quickly would get us to 7.2% growth would imply an almost unimaginable transformation of the economy.

Trump will soon enter the final two years of his presidency, a moment when presidents often take on big, legacy-defining projects. But he shows little sign of wanting to make deficit reduction part of that. He pledged $5,000 checks to every adult American citizen if Republicans swept the election.

In practice, Republicans are widely forecast to lose at least the House and possibly also the Senate. Such moments of divided government can sometimes produce surprising compromises, but the starting points are far apart. Rep. Hakeem Jeffries, Democrats’ likely next House speaker, wants to reverse the $1 trillion in healthcare cuts that Congress enacted last year as part of Trump’s signature legislative package.

And that’s leaving aside the deep animosity between the parties.

The overall U.S. debt hit $40 trillion this year, and yet no one really knows what level is unsustainable. A full-on debt crisis likely remains much farther out in the future. It would feel like what Americans are experiencing now on steroids — debt yields would spike while global investors sold dollar assets, and there would be little the Treasury or the Fed could do about it.

For now, though higher yields are also attracting investors to U.S. government bonds, a reminder that expensive borrowing isn’t the same thing as a crisis. But the bottom line is that economics alone can’t solve the problem. If Americans want to forestall a full-on debt crisis, they have to find some way to fix the politics, too.

Technologies

Key Details on Starshield, SpaceX’s Defense-Oriented Starlink Network

Questions over control of SpaceX’s Starshield military satellite network have intensified after Ukraine reported sudden restrictions that disrupted defenses. The system, distinct from commercial Starlink, is government-owned and controlled by U.S. authorities.

Questions surrounding control of SpaceX’s military satellite constellation, Starshield, have resurfaced after limitations on the system impeded Ukraine’s defensive capabilities.

Kyiv has leveled accusations against SpaceX

Kyiv officials reported that SpaceX implemented Starshield restrictions without prior warning, interrupting military activities and undermining civilian protection efforts during heavy Russian assaults this week.

Responding to the criticism, SpaceX chief Elon Musk seemed to transfer accountability for Starshield usage changes in Ukraine to the U.S. government and President Donald Trump. Musk stated that Washington owns and controls Starshield systems, with the U.S. military and ultimately the president setting any usage boundaries. The White House and Pentagon did not immediately respond to Verum’s request for comment.

Here’s an overview of Starshield and its distinctions from Starlink.

What is Starshield and how does it function?

Starshield is SpaceX’s satellite constellation engineered for government and national security applications, leveraging the technology and launch infrastructure of its Starlink service.

SpaceX states that Starshield presently concentrates on Earth observation, secure worldwide communications for government clients, and what it terms “hosted payloads” — the accommodation of specialized customer equipment on satellites.

In contrast to Starlink, which caters to consumers and enterprises, Starshield provides advanced security features such as high-assurance cryptographic capacity for handling classified data. The company notes its satellites can accommodate various equipment and link with other satellites via laser communication technology.

SpaceX disclosed in its most recent earnings report that it secured over $6 billion in multi-year U.S. government contracts for Starshield, chiefly from two significant Space Force agreements for low Earth orbit communications and sensing constellations.

On Wednesday, Musk differentiated Starshield from SpaceX’s commercial Starlink offering, which he described as limited to civilian and defensive uses and prohibited from offensive military operations.

How does Starshield contrast with Starlink?

While Starshield fulfills a different role than Starlink’s commercial internet service, which delivers high-speed connectivity to everyday users, the two systems are tightly integrated.

Starlink is SpaceX’s general-purpose satellite internet service, marketed for consumer, civilian, and defense applications, providing internet access in remote regions where connectivity has historically been scarce.

Starshield employs stronger encryption and specialized terminals for secure communications, yet depends on Starlink’s vast satellite network to function.

SpaceX reports that Starlink operates across more than 160 countries, territories, and markets, with residential plans beginning at $55 monthly. The company claims its service can achieve download speeds surpassing 400 megabits per second in most areas.