AI Reality Check
Who Is Responsible When AI Gets It Wrong?
By Joseph Rose, founder of MetaBinary. 20+ years building enterprise software. Built the world's first instant online metal-casting costing engine.
Published
When Air Canada’s website chatbot gave a customer the wrong refund rules, the airline argued it could not be held liable for what the chatbot said. The tribunal called that “a remarkable submission,” and Air Canada paid. So far courts agree on this: a person or company answers for what its AI does. That is why a good AI build lets agents read and report on their own, and makes anything that sends, buys or changes a record wait for a person to say yes.
The courts have already picked an answer
People still argue about who should carry the blame when AI makes a mistake. It could be the company that built the model, the person who set it up, the person who typed the question, or the tool itself. It is a fair debate, and it will keep philosophers busy for years.
You don’t need to wait for them to finish. In the cases that have reached a judge or tribunal so far, the answer has been the same. The business or person that chose to use the AI answers for what it did. The tool never takes the blame.
This is not legal advice. It is what the rulings say, in plain English, and how I build because of them.
Deciding who checks the AI is part of the work around the model, and that work is where most of the effort in an AI project goes. I explain the rest of it in why an AI project costs real money.
Two cases every business owner should know
Moffatt v. Air Canada (2024). In November 2022, after a grandmother died, Jake Moffatt used the chatbot on Air Canada’s website while looking at flights. The chatbot said a bereavement fare could be claimed after booking. (A bereavement fare is a lower price for people travelling because a family member died.) Air Canada’s own rules did not allow that, and its staff refused the refund.
At the tribunal, Air Canada said it could not be held liable for information given by any of its “agents, servants, or representatives – including a chatbot.” Tribunal member Christopher Rivers wrote: “In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions. This is a remarkable submission.” He went on: “It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot.” Air Canada was ordered to pay $812.02 in damages, interest and fees.
Mata v. Avianca (2023). A man was suing the airline Avianca. One of his lawyers used ChatGPT for research, and the court filing that followed cited court cases that did not exist. In the words of Judge P. Kevin Castel’s ruling, the lawyers “submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question.” The judge ordered two lawyers and their firm to pay a $5,000 penalty.
The judge had no problem with the tool. He wrote that “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.” The lawyers were punished for sending work they had not checked.
AI can’t own its work either
Blame is one side of the coin. Credit is the other, and courts handle it the same way.
Stephen Thaler built an AI system called DABUS and applied for UK patents with DABUS named as the inventor. In December 2023, the UK Supreme Court ruled that “an inventor within the meaning of the 1977 Act must be a natural person.” A natural person is a human being.
Thaler also tried to register a copyright for a picture made by another of his AI systems. In March 2025, a US appeals court ruled that the Copyright Act “requires all work to be authored in the first instance by a human being.” In March 2026, the US Supreme Court declined to hear his appeal, so that ruling stands.
So the law will not let an AI take the credit for an invention, and it will not let a company hand the blame to a chatbot.
The EU AI Act writes it into law
For businesses in Ireland and the rest of the EU, the AI Act sets this down in writing. Article 14 says high-risk AI systems must be built so “that they can be effectively overseen by natural persons during the period in which they are in use.” In plain terms, a person has to be able to watch the system and step in.
“High-risk” covers a set list of uses. Examples include AI that filters job applications, and AI built into safety parts of regulated products such as machinery. In July 2026 the EU passed the Digital Omnibus on AI, Regulation (EU) 2026/1744. It moved the start of these high-risk duties to 2 December 2027 for listed uses like hiring, and to 2 August 2028 for AI inside regulated products.
Most everyday business AI, like asking questions about your own orders or stock, is not on the high-risk list. The direction is still clear, and it matches what the courts have already said.
Keeping a person in charge is about who answers for the result
Maybe one day AI will be a true co-worker. Someone will still sign off on its work, because a customer, a court or a regulator will ask who did.
The same goes for any tool that does sums for your business. The tool does the maths, and your people still know the parts and the customers. Every price that goes out is still your price.
AI does not know which customer is touchy about price or which order must not move this week. Your people do, which is why AI should make them faster instead of pushing them aside. So I build every system the same way:
- Agents read and report on their own. Pulling numbers, checking them and writing a summary needs no sign-off.
- Anything that acts waits for a yes. Sending an email, placing an order or changing a record waits for a person to approve it.
- Every answer shows its source. The person checking can see which record it came from. This depends on the tools the AI uses more than on the model, as I explain in why good tools beat a smarter AI.
The checker should be the person who already knows the job. If that know-how lives in one person’s head, write it down first, so more than one person can check the AI’s work. Process Capture is how we do that.
Some jobs should stay with a person entirely, because one mistake would cost too much. I cover those in when AI is the wrong answer. You can see the setup I describe here under “AI You Can Verify” on the AI Builds page.
Start with one job you can check
The safest first AI project is one where a wrong answer is easy to spot and cheap to fix. Pick a question your team answers every week. Let the AI draft the answer and show its sources, and have the person who answers it today check each one. Once you trust it, let it do more.
Want a second pair of eyes on it? Book a call from the AI Builds page. Tell me the job, and I’ll tell you where a person should stay in the loop.