# When Is AI the Wrong Answer? Source: https://metabinary.ie/blog/when-ai-is-the-wrong-answer [← Back to blog](/blog) AI Reality Check # When Is AI the Wrong Answer? By [Joseph Rose](/about), founder of MetaBinary. 20+ years building enterprise software. Built the world's first instant online metal-casting costing engine. Published 2026-10-22 “If stock drops below 10, reorder” needs a formula. Put AI on it and you get a slower, costlier way to sometimes get it wrong. AI is the wrong answer more often than vendors admit: when the rule is fixed, when the data doesn’t exist yet, or when a mistake costs too much to risk. It earns its place when the question is messy, the data lives in several places, and a person checks the result. A good partner tells you which one you have before you spend anything. Most of the cost of an AI project is the work around the model, as I explain in [why an AI project costs real money](/blog/why-ai-projects-cost-money). That work is only worth paying for when AI is the right tool. This post is about the times it is not. ## Fixed rules belong in plain automation A fixed rule is one where the same facts always give the same answer. Reorder when stock hits 10. Send the invoice when the job is marked done. Flag any invoice that is more than 30 days unpaid. Plain automation handles rules like these. Automation here means software that does a set task with no guessing: a spreadsheet formula, a scheduled report, or a step in your ERP (the main system you use for orders, stock and jobs). It runs in a split second, costs almost nothing, and gives the same answer every time, as long as the rule and the data behind it are right. AI works differently. It makes a best guess from patterns. A best guess is useful when the question is messy. For a fixed rule, it adds cost, slows things down, and leaves room for a wrong answer where there should be none. Google’s engineers say the same thing in their guide, [Rules of Machine Learning](https://developers.google.com/machine-learning/guides/rules-of-ml), written by Martin Zinkevich. Machine learning is the kind of AI that learns from past data. Rule #1 of the guide reads: “Don’t be afraid to launch a product without machine learning.” It adds: “If machine learning is not absolutely required for your product, don’t use it until you have data.” So the first thing I check with any idea is whether someone can write the rule on one line. If they can, it needs a formula, and I’ll say so. ## No data means no AI AI can only work from records that exist. Some businesses price jobs from memory, keep delivery dates on sticky notes, or let each estimator run a private spreadsheet. In those cases there is nothing for the AI to read. The RAND Corporation [interviewed 65 experienced data scientists and engineers](https://www.rand.org/pubs/research_reports/RRA2680-1.html) about why AI projects fail. One of the five main causes they found was that “the organization lacks the necessary data to adequately train an effective AI model.” Another was a focus on “the latest and greatest technology” over solving real problems for the people who use it. The fix comes before any AI. Start keeping the records you would need. When the know-how lives in one person’s head, [Process Capture](/automation) writes it down as clear steps. Those steps pay off on their own. New staff can follow them, and you stop depending on one person. Later, they give AI something to work from. Writing the rules down is often most of the work. Once they are on paper, much of it turns out to be plain maths a computer can follow, with no AI needed. Your experts still judge the odd jobs the rules don’t cover. ## Some mistakes cost too much to automate Every AI system gets some answers wrong. How often depends on the task, your data, and the tools built around it. You only find out by testing it on your own records, which is why [AI rarely gets it right the first time](/blog/why-ai-gets-it-wrong-first-time). So ask what one wrong answer would cost you. A wrong draft email takes a minute to fix. A wrong price on a large quote, a wrong machine setting, or a wrong promise to a customer can cost far more. For decisions like those, a person decides. AI can gather the facts and lay them out side by side, and a named person makes the call. Who carries the blame when an AI answer goes wrong is a question of its own. I cover it in [who is responsible when AI gets it wrong](/blog/who-is-responsible-when-ai-gets-it-wrong). ## AI fits messy questions with a person checking AI earns its place when three things are true at the same time: - **The question is messy.** It cannot be written as one rule. “Which open jobs look like the ones that ran over budget?” is messy. So is “What is this customer asking for in this long email?” - **The data is spread out.** The answer needs facts from the ERP, a spreadsheet and an inbox, and pulling them together by hand takes hours. - **A person checks the result.** Someone who knows the work looks at the answer before anyone acts on it, and a wrong answer is easy to spot. When all three are true, AI can save your team hours every week. It still needs to be told what a good answer looks like. Someone has to set the question, connect the data, and test the answers. “AI” also covers several different kinds of tool. Some read text, some forecast numbers, and some take actions in your systems. Picking the right kind matters as much as picking the right problem, as I explain in [is all AI the same thing](/blog/is-all-ai-the-same). ## Hearing “not this one” is a good sign Someone who sells AI builds has a reason to say yes to every idea. I would rather tell you early when a formula, a report, or better record-keeping will do the job. You save money, and when I do say AI fits, you have a reason to believe me. That is why every build on the [AI Builds page](/ai) starts small. We build a small working version on your real data, and you try it. If it is not useful, we stop there and you keep what we learned. If a forecast cannot beat the way you guess today, you hear that before you spend more. This also means AI work is not for everyone. If your problem is a fixed rule, you don’t need an AI build. Ask whoever looks after your ERP or spreadsheets to set up the formula. If your records are thin, start with the record-keeping and come back later. ## Find out which one you have Most businesses have a mix. Some questions need a formula, some need better records first, and a few are a good fit for AI. Sorting them takes one conversation, and it can save you from paying for the wrong build. **Not sure which one you have?** Book a call from the [AI Builds page](/ai). Tell me a question your team spends hours answering, and I’ll tell you if it needs AI, a formula, or better records first. ## Common questions ### What's the difference between AI and automation? Automation follows a fixed rule you write, the same way every time. AI makes a best guess from patterns, which helps with messy questions that cannot be written as one rule. Many good systems use both: automation for the fixed steps, AI for the messy ones, and a person to check the result. ### Can AI work with bad data? A little. It can read messy text and cope with small gaps. It cannot find records that were never kept. If the answer you want depends on facts nobody wrote down, fix the record-keeping first. ### How do I know if my problem needs AI? Try to write the rule on one line. If you can, use a formula or plain automation. If the question is messy, the data is spread across systems, and a person can check the answer, AI may fit. A small test on your real data tells you for sure. ## Where to go next - [If AI Can Do Anything, Why Does an AI Project Cost Real Money?Running an AI model costs little. The money goes into picking the right question, connecting your data, building the tools the AI uses, and checking its answers.](/blog/why-ai-projects-cost-money) - [Why Didn't AI Get It Right the First Time?AI gets general jobs right first time. Jobs that need your own rules fail until someone writes those rules down and tests the AI against real answers.](/blog/why-ai-gets-it-wrong-first-time) - [Is All AI the Same Thing?No. Business AI comes in three kinds: AI for numbers, AI for patterns, and AI for language. The question you need answered picks the kind.](/blog/is-all-ai-the-same) - [AI BuildsWhat we build, and how a build runs.](/ai)