Can a company forget why it makes its decisions?

Could a company reach a point where it no longer knows why it makes certain decisions? At first glance, it seems unlikely: we've never had so much capacity to record what happens inside an organization. But there's a less obvious possibility: that the exact opposite happens.

As we hand different agents the ability to observe, decide, and act, a decision can stop happening in a single place. What one agent decides changes the context for the next. The organization keeps making decisions, possibly better and faster ones, but the reasoning behind them starts to get scattered across systems.

We could build companies capable of recording every action and, at the same time, unable to clearly explain some of their own decisions.

The genealogy of a decision

At 9:17 on an ordinary Tuesday, an agent decides to raise the price of a product. The change is within authorized margins; nobody gets an alert. Fourteen minutes and five decisions later, not a single meeting had been called, and two days later the results were good.

Fourteen minutes, five decisions, zero meetings:

• 9:17 Pricing raises the price
• 9:18 Promotion is scaled back
• 9:21 Demand forecast changes
• 9:24 Inventory adjusts replenishment
• 9:31 Purchasing modifies an order

During the sales meeting, someone looks at the dashboard and asks: “Why did we raise the price?” The team walks back up the chain only to end up, absurdly, right back at pricing. There are no errors. Every agent acted within its permissions. They know exactly what happened. What they can't explain as easily is why the company decided to do it. And it was a good decision.

1. Who made the decision?

For years, automating meant getting a machine to carry out a decision someone had defined beforehand. The decision had a recognizable origin. Agentic AI changes that relationship: when several agents take part in the same process, some agents' decisions become part of the context other agents decide on.

Pricing executed the price change, but it was reacting to information another system had modified. Each agent made a reasonable local decision, and none of them, on its own, contains the full explanation of the outcome. Nobody made that decision the way it ultimately happened. The company did.

Orphan decision: an organizational decision that emerges from several legitimate agentic decisions, but whose full rationale doesn't belong to any of the actors who took part in it.

2. The system passed the audit

What did the system do? We have an answer. Who carried out each action? We have that too. Was it authorized? Yes. Did it work? Yes. Why did the organization end up making that decision? That's where the answer starts getting complicated. Traceability and understanding tend to show up side by side when we talk about control, but they're not the same thing: logs preserve events, while understanding a decision requires reconstructing the relationships between them.

3. The problem may lie between the agents

In the example, we wouldn't find the problem by taking a closer look at the pricing agent, or at inventory, or at purchasing. The phenomenon lives between them: demand changes a forecast, inventory interprets it, pricing changes a price, marketing modifies a promotion, and that promotion affects demand all over again. Each step can make sense locally, and still the reason for the outcome can end up spread across the whole system.

Pricing aims to protect margin, inventory to reduce stock-outs, marketing to maximize conversion, purchasing to optimize costs. There's nothing strange about those goals: the problem shows up when they interact and compete.

• Pricing: protect margin
• Inventory: reduce stock-outs
• Marketing: maximize conversion
• Purchasing: optimize costs

4. From the log to the genealogy

The genealogy of a decision reconstructs which evidence, goals, and prior decisions made a later decision possible. A log lets you ask what happened before; a genealogy should let you reconstruct why what happened before ended up producing this decision and not another one. It's not about asking every agent to write an essay justifying its actions: what's worth preserving is the relevant structure: evidence, goals, dependencies, changes, and consequences.

5. The human needs more than the last word

A common response to the risk of automating decisions is to keep a person in the loop. But picture the person who gets the alert after twenty agents have made hundreds of related decisions over the last six hours: technically, they still have the last word, but in practice they've just been handed a problem whose history they didn't live through.

In 1983, Lisanne Bainbridge described one of the ironies of automation: the more a normal operation is automated, the more we may depend on the human precisely in the abnormal situations they're least prepared for. That's why handing a decision back to a person doesn't, by itself, guarantee human control. You have to hand back the context too.

6. What will we audit tomorrow?

As agents move from executing tasks to taking part in connected decisions, we'll have to broaden what we mean by control. The unit of control is no longer just the agent: it also becomes the decision that flows between them. Maybe organizations will end up auditing the genealogy of their decisions as naturally as they audit processes, transactions, or models today.

Not just being able to prove that their agents acted correctly, but still knowing why the organization decided what it decided.

Sources

• Bainbridge, L. (1983). “Ironies of Automation”. Automatica, 19(6), 775–779.
• Berberian, B. (2018). “Out-of-the-loop Performance Problem”. Neuroergonomics, ch. 75.
• Zhu, J. et al. (2026). “Interpreting Agentic Systems”. arXiv:2601.17168.
• NASA. “Crew Interfaces”, Human Integration Design Handbook.