Blog
When AI Gets It Wrong, Your Business Pays the Bill
Sam CarrUpdated
In almost every case, the business that used the AI output is liable, not the AI vendor. Air Canada tried to distance itself from its own chatbot in front of a small claims adjudicator in British Columbia after the bot told a passenger he could apply for a bereavement fare discount retroactively within 90 days, and it lost: as one analysis of the case puts it, "Liability is not avoided by automating the actions in question." The vendor is unlikely to pick up the bill, because OpenAI's terms state that its services are provided "AS IS", with no warranties except to the extent prohibited by law. Your insurance may not close the gap either, since many professional indemnity policies do not explicitly cover AI-related errors or omissions, which makes human review and a record of it your real protection.
Wrong AI output covers more than made-up facts: bad figures, bad advice, bad summaries
"Wrong information" from an AI tool is broader than the famous case of a chatbot inventing a fact. In practice it covers four things, and all four can land on your desk as a business decision.
Made-up facts and sources. This is the classic hallucination: the tool produces something that looks like a real citation, statistic or reference and it simply does not exist. In one US case reported by The Register, a court found lawyers had "abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations" generated by AI. In a separate case involving lawyers defending Alabama's prison system, the judge sanctioned them over fake ChatGPT cases in filings, writing that "Fabricating legal authority is serious misconduct that demands a serious sanction".
Bad figures. A number that is transposed, mis-totalled or pulled from the wrong row is wrong output too, even though nothing was invented. If an AI tool reads an invoice, a spreadsheet or a supplier quote and returns a figure you then act on, the error is the same size as the decision it feeds.
Bad advice. Here the output is a recommendation rather than a fact: what your notice period is, whether a clause is enforceable, whether a claim is worth pursuing. The US Federal Trade Commission's action against DoNotPay is the sharpest example. According to the FTC, the company never tested the legal accuracy of the chatbot's output. That matters because advice-shaped output invites reliance without any obvious way for the user to check it.
Bad summaries. The most common in day-to-day operations and the hardest to spot. The tool reads a long email chain, a contract or a set of meeting notes and drops a detail, flips a negative, or merges two customers into one. Nothing in the summary looks odd. That is the problem.
The practical test is not "did the AI lie". It is: did the output change what someone in the business did, said or signed? If yes, it is wrong information in the sense that matters for liability, whether it arrived as a fabricated citation, a mistyped total or a tidy paragraph that left out the one clause you needed.
The vendor's terms almost always push the risk back onto you
Before you rely on an AI tool's output, read what the supplier actually promises. In most cases the answer is: very little.
Take OpenAI, the maker of ChatGPT. Its consumer Terms of Use state that the services are provided "AS IS", and that OpenAI, its affiliates and its licensors make no warranties, "EXCEPT TO THE EXTENT PROHIBITED BY LAW". The business-facing OpenAI Services Agreement uses near-identical wording: the services are provided "AS IS", to the extent permitted by law and except as expressly stated in the agreement.
"As is" is the important bit. In plain English, it means you take the product in the state it arrives in. The supplier is not promising the output is accurate, complete, or fit for your particular job. So if the tool tells your customer the wrong delivery date, quotes the wrong price, or invents a clause in a contract, the terms are not written to leave that with the vendor. They are written to leave it with you.
Two phrases are worth noting, because they cut the other way slightly. Both documents qualify the disclaimer with a reference to what the law allows. UK law limits how far a supplier can contract out of certain liabilities, so an "as is" clause is not a complete shield. But you should not plan your business around arguing that point after the damage is done.
The practical takeaway for a small business:
- Assume the vendor's terms put accuracy risk on you, and check whether yours say anything different.
- Look for any section where the supplier does make an express promise. In OpenAI's business agreement, the disclaimer is expressly subject to what is "expressly stated in the agreement", so anything written in there still counts.
- If a wrong answer would cost you real money or a customer, put a human check between the AI and the outside world. The terms will not do that for you.
The business carries the liability, not the member of staff who typed the prompt
If a member of your staff asks an AI tool to draft a quote, a policy or an email, and the tool gets it wrong, the customer's complaint lands on the business. Not on the person who typed the prompt.
The reason is vicarious liability, sometimes called respondeat superior: an employer answers for what its employees do in the course of their work. Cornell's Legal Information Institute notes that a court will apply the doctrine to an employer regardless of how closely the employer was monitoring the employee. So "we told them to check it" is not, on its own, a defence.
The AI does not change the analysis much. Neathouse Partners describes a common scenario as reliance on unverified AI output that is then sent to a client or third party, causing loss, and says the legal focus falls on the user's conduct rather than the tool. In other words, nobody sues the chatbot. They look at what your person did with the output, and you stand behind that person.
When it does shift to the individual
There are limits. Lawyer Monthly notes that liability "may shift where an employee acts outside their role, breaches clear internal policies, or uses unauthorised tools." That is the practical argument for having a written AI policy that names which tools are approved and what must be checked before anything goes out. Without one, there is no clear internal policy to breach, and no such thing as an unauthorised tool.
Courts have long drawn a line between a minor deviation from assigned duties and a genuine departure from them, the distinction legal scholarship calls a detour versus something more serious. A sales assistant using an approved AI tool badly is squarely inside the job. Someone pasting your client list into a random free tool you have never heard of, against a rule you wrote down, is closer to the edge.
Either way, the money question for an owner is the same. Assume the business pays first and argues later. Then decide what you are willing to let an AI tool touch unchecked.
Professional indemnity may cover AI errors, cyber insurance usually will not
Insurance is the first place most business owners look after a mistake. It is also where the AI question gets murky.
Start with professional indemnity, the cover that pays out when your advice or work causes a client a loss. Kennedys Law notes that "many professional indemnity policies may not explicitly cover AI-related errors or omissions." That silence cuts both ways. A claim arising from bad AI output may still be covered as a straightforward professional error, because nothing in the wording rules it out. But nothing in the wording promises it either, and that is a poor position to discover after the event.
Cyber insurance is a narrower tool than people assume. It is built around attacks and data breaches, not around your own tools getting things wrong. Buchalter observes that most cyber policies do not yet contain express AI exclusions, and that cover may apply where an AI system exposes sensitive information. ACA Group draws the sharper line: policies generally continue to cover cyber attacks that use AI, while losses caused by an organisation's own autonomous AI systems or weak AI governance sit in a different and less comfortable category.
So the rough rule is this. If someone attacks you with AI, cyber cover is likely in play. If your own AI tool invents a price, a deadline or a legal position, cyber cover is probably the wrong policy and professional indemnity is where the argument will happen.
Expect this to keep moving. Fenwick describes coverage fragmenting across cyber, technology errors and omissions, directors and officers, and employment practices liability as insurers respond to AI risk. In practice that means the gaps between your policies matter as much as what each one says.
Two things to do before you rely on any of it. Ask your broker, in writing, whether errors produced by AI tools are covered under your current professional indemnity wording. And tell them which tools you actually use and for what, because a non-disclosed change in how you work is the kind of detail insurers return to at claim time.
Human checks, logging and clear limits cut most of the exposure
None of this is exotic. The controls that reduce exposure are the ones you would put around a new member of staff who is fast, confident and occasionally wrong.
Put a human in the loop where the answer matters. The ICO's guidance on human review treats meaningful human review and checks as a control measure for AI decisions, where appropriate. In practice that means deciding in advance which outputs go straight to a customer and which need a person to read them first. Quotes, prices, contract terms, refund and policy statements, anything about someone's rights: those are the ones to check.
Log the review. The ICO also advises logging human review decisions, including where a reviewer challenges or overrides automated decision making and the considerations behind it. That log is your evidence later. If a customer says your tool told them something, you want a record of what was generated, what was sent, and who signed it off.
Set clear limits on what the tool will answer. If the AI is not allowed to state prices, it cannot misstate one. Narrow the scope, then route anything outside it to a human.
Tell people when AI is involved. The AI Consultancy's 2026 UK compliance checklist advises updating privacy notices to disclose AI use where material, particularly where AI processes personal data in a way that is not obvious to the person concerned.
And the point to keep in mind while you do all of it, from the analysis of the Moffatt v. Air Canada chatbot case: "Liability is not avoided by automating the actions in question." Automation moves the work. It does not move the responsibility.
The first 48 hours after a bad AI answer reaches a customer decide how bad it gets
An AI error becomes expensive when a business argues with the customer instead of fixing it. In the Air Canada case, the airline's chatbot told a passenger he could apply for a bereavement fare discount retroactively within 90 days. He took a screenshot of the exchange and booked his flight. Air Canada accepted the information was "misleading" but still contested his right to a refund, and it tried to distance itself from its own chatbot in front of a small claims adjudicator in British Columbia, an argument the adjudicator did not accept. The wrong answer was cheap. The fight over it was not.
So treat the first two days as damage control, not as a legal dispute.
- Get the evidence before it disappears. Save the full chat transcript, the timestamp, and the exact wording the tool used. Your customer probably already has a screenshot. In the Air Canada case the passenger screenshotted the exchange, and that record is what the claim was built on.
- Work out what the customer did because of the answer. Did they book, buy, cancel, or miss a deadline? The loss usually sits in the action taken, not the words themselves.
- Decide fast whether to simply honour it. If the cost of doing what your chatbot promised is smaller than the cost of arguing, honour it and move on. Air Canada disputed a single fare and ended up as a case study in AI liability.
- Do not tell the customer they should not have believed your own tool. That was the airline's position and it failed. Once you publish an AI answer to a customer, treat it as your answer.
- Turn the tool off for that question. If the error is repeatable, take the topic out of the bot's scope or route it to a human until it is fixed. Every further customer who gets the same answer widens your exposure.
- Check whether the wrong answer is a marketing or sales claim, not just a service slip. In the UK, the unfair commercial practices regime now sits in the Digital Markets, Competition and Consumers Act 2024, so misleading statements to consumers carry consequences beyond a refund.
- Log it. Date, question asked, answer given, who was affected, what you did. If a pattern emerges later, you want the paper trail to show you acted each time.
- Tell whoever bought or built the tool. Vendors cannot fix what they do not hear about, and your record of reporting matters if you later argue the fault was theirs.
None of this needs a lawyer on day one. It needs a transcript, a decision, and someone senior enough to say yes to the refund.
Common questions
Does it help to tell customers we are using AI?
Being open about it is treated as good practice, not an admission of weakness. The AI Consultancy's 2026 UK compliance checklist advises updating privacy notices to disclose AI use where material, particularly where AI processes personal data in a way that is not obvious to the person concerned.
What if the customer has a screenshot of what the AI told them?
Assume the screenshot is the evidence and act accordingly. In the Air Canada case the passenger took a screenshot of the chatbot exchange and booked his flight, and the airline later accepted the information was "misleading" while still contesting his right to a refund.
Does it make a difference whether we used a free account or a paid business plan?
Both OpenAI's consumer Terms of Use and its business-facing Services Agreement provide the services "AS IS". The difference is that the business agreement's disclaimer is expressly subject to what is "expressly stated in the agreement", so the contract you actually signed is worth reading.
Can we hold the employee responsible if they used a tool we never approved?
It can change the picture, but do not count on it. Lawyer Monthly notes that liability "may shift where an employee acts outside their role, breaches clear internal policies, or uses unauthorised tools", and courts have long distinguished a minor deviation from assigned duties from a genuine departure from them.
Is this only a contract problem, or do consumer rules bite too?
Misleading information given to consumers can also engage the unfair commercial practices regime, which in the UK now sits in the Digital Markets, Competition and Consumers Act 2024. That is a separate route to trouble from a customer simply suing on the promise.
Drafted by our blog writer. Read, checked and published by Sam.
Sources
- DoNotPay
Wikipedia (DoNotPay)
- This article is more than 1 year old
The Register
- Judge sanctions lawyers defending Alabama’s prison system for using fake ChatGPT cases in filings
Yahoo News (AP)
- DoNotPay
Wikipedia (DoNotPay)
- OpenAI Services Agreement | OpenAI
OpenAI
- Terms of Use | OpenAI
OpenAI
- Who Is Liable When AI Causes Harm at Work?
Lawyer Monthly
- respondeat superior | Wex | US Law | LII / Legal Information Institute
Cornell LII (Wex)
- Acting with AI: An Interaction-Based Framework for Agentic Tort Liability
arXiv (legal scholarship)
- Artificial Intelligence: When is an employer vicariously liable? | Neathouse Partners
Neathouse Partners
- AI Conversations & Chatbot Accountability Under Scrutiny: The Case of the (Too) Helpful Chatbot | BD&P
BD&P
- AI Mistakes: Could Your Business Be Liable? - Butcher & Barlow LLP
Butcher & Barlow LLP
- Who is liable if AI goes wrong? UKJT statement on AI and civil liability published - Burges Salmon
Burges Salmon
- BC Tribunal Confirms Companies Remain Liable for Information Provided by AI Chatbot
American Bar Association
- Negligent Misrepresentation in Moffatt v Air Canada
UBC Law Review
- Fake Cases, Real Consequences: Misuse of ChatGPT ...
Goldberg Segalla (NY Litigator)
- Mata v. Avianca, Inc.
Wikipedia
- How can I mislead you? Air Canada found liable for chatbot's bad advice on bereavement rates | CBC News
CBC News
- Silent AI cover: the unforeseen risks for insurers
Kennedys Law
- Are You Covered for the Risk? Existing Insurance Policies and AI Exposure | Buchalter
Buchalter
- The End of ‘Silent AI’? Emerging AI Exclusions, Coverage Fragmentation, and Practical Implications for Policyholders | Fenwick
Fenwick
- How AI Is Changing the Rules of Cyber Insurance - ACA Group
ACA Group
- "We signed what?!": The Hidden Hazards of Vendor AI Terms and Conditions | Bennett Jones
Bennett Jones
- Unfair Contract Terms Act 1977
legislation.gov.uk (UCTA 1977)
- "We signed what?!": The Hidden Hazards of Vendor AI Terms and Conditions | Bennett Jones
Bennett Jones
- Digital, Commerce & Creative 101: All’s ‘fair’ in love and contracts – a brief look at the Unfair Contract Terms Act 1977
Lewis Silkin
- 2026 UK AI Compliance Checklist for Businesses
The AI Consultancy
- Human review | ICO
ICO
- Human review | ICO
ICO
- Moffatt v. Air Canada: A Misrepresentation by an AI Chatbot
McCarthy Tétrault
- When AI Gets It Wrong: Who’s Legally Responsible? | Sprintlaw UK
Sprintlaw UK
- Why Air Canada’s Chatbot Cost Them a Lawsuit and Changed AI Liability Forever
The Decision Case Library (Substack)
- How can I mislead you? Air Canada found liable for chatbot's bad advice on bereavement rates | CBC News
CBC News
- Air Canada chatbot case highlights AI liability risks
Pinsent Masons (Out-Law)