AI can give a perfect answer. Based on the wrong numbers.

It's easy to get AI to deliver an impressive financial analysis. Upload an income statement, ask a few questions and get a well-written assessment of profitability, liquidity and risk. The problem is that the model may have misunderstood the numbers, missed an obligation or drawn conclusions on an incomplete basis. The answer can still sound completely right.

That's the difference between a chatbot that can talk about finance and a financial system you can trust.

The language model is not the financial system

Imagine a CEO asking: "Can we afford to hire two people next year?" A language model can quickly put together an answer. But a good answer requires far more than reading the income statement. The system has to understand the cost structure, liquidity, existing obligations and how new payroll costs affect the finances. It has to distinguish between profit and cash flow, handle accruals and calculate consequences under different assumptions. This is not primarily a language task, but accounting knowledge, data processing and mathematics.

In a technical article on context engineering from September 2025, Anthropic describes why it isn't enough to give a language model access to large amounts of information. It needs the right information, in the right context, when it needs it, and this is especially important in finance. A business can have thousands of transactions and several years of accounting history. But to assess liquidity for the next three months, the system has to identify the relevant data, understand the relationships and carry out the right calculations.

The wrong data gives the wrong analysis, no matter how good the language model is.

AI should explain. The calculations should be verifiable.

A language model is good at understanding questions, interpreting relationships and explaining complex information. That doesn't mean it should perform all the calculations. Key figures, cash flow and financial consequences should be calculated using defined rules that can be tested and verified. AI can then use the results to explain the development and recommend actions. If a system warns that liquidity will turn negative in three months, we need to be able to check why. Which numbers were used? What assumptions apply? What happens if revenue falls by ten percent? This shouldn't be hidden inside the language model's answer.

The calculations must be verifiable. The recommendations must be understandable. Otherwise we're asking the user to trust a conclusion without being able to check its basis.

A good demo is not a reliable product

In January 2026, Anthropic published an article on testing AI agents. The challenge is that agents can take different approaches to solve the same task. That's why it isn't enough to look at what the agent says it did. We have to test what the actual outcome was: which numbers were used, what was calculated and what was carried out. Anthropic warns against requiring a specific sequence of steps, and recommends assessing what the agent actually produced. The tests also have to be repeatable when the system changes or a new language model is introduced.

In financial management, this is crucial. An agent tasked with identifying a business's biggest risks must also catch the serious problems. It's of little help if it delivers five good analyses but misses an upcoming liquidity problem.

And when agents are able to take actions, the requirements increase further. There's a big difference between recommending changed payment terms and actually changing them in a system. OWASP, an organisation that works on software security, published its own top ten list of risks for agentic AI systems in December 2025, precisely because an agent with access to tools can do more than give a wrong answer.

Just because an AI agent can perform an action doesn't mean it should be allowed to. Access rights, control mechanisms and human approval must be matched to the consequences of what the agent does.

The hardest part of an AI CFO isn't the language model

Language models keep getting better. They can understand more, reason better and solve more complex tasks. But access to a good language model is not a lasting competitive advantage. We use many different language models from several providers ourselves, and that's exactly why I believe what matters is what you build around them. Does the system understand the accounting data? Can it identify the relevant relationships? Are the calculations correct? Can the results be verified? And does this work reliably across businesses and accounting systems? These are the challenges we've been working on at Capassa. We've built Capa as an AI CFO, not as a chatbot with access to accounting figures. Capa analyses a business's finances, identifies risks and opportunities, explains the relationships and recommends actions. Along the way, Capa shows what it's doing and which language models are being used, so the user doesn't just get a conclusion, but can see how it was reached.

The simple part is what the user sees. The demanding part lies behind it: turning accounting data into financial insight that can be used to make decisions.

For a business leader, that means a better overview of what needs attention. For an accounting firm, it means the analysis work can be done for far more clients without a corresponding increase in time spent. That's where technology creates value. Not by producing more reports, but by making financial insight available when decisions are being made.

We don't need AI that sounds like a CFO. We need technology that does a CFO's job. And then a convincing answer isn't enough.

It has to be right.