AI Reality Check
Why Can't My Team Just Use ChatGPT for This?
By Joseph Rose, founder of MetaBinary. 20+ years building enterprise software. Built the world's first instant online metal-casting costing engine.
Published
Ask a fresh ChatGPT account which of your suppliers were late last quarter and it can’t tell you. It has never seen your records. For emails, summaries and first drafts, ChatGPT or Copilot is often all you need, so keep it. For questions about your own jobs, prices and suppliers, the AI needs a safe connection to your systems and tools that return real figures. Building that connection is what a custom AI build is. Most businesses end up with both.
Chat tools are enough for everyday writing
I’ll be fair to the chat tools first, because they are very good. ChatGPT and Microsoft Copilot can draft an email to a customer, sum up a long report, tidy a job advert, or turn rough notes into a clear procedure. They do it in seconds. If your team writes a lot, a paid seat each is money well spent.
For this kind of work, you don’t need anything custom. The text is in front of the AI, the task is plain, and a person reads the result before it goes anywhere. If it gets something wrong, you spot it and fix it.
So if most of what you want from AI is help with writing, stop here. Buy the seats, show people how to use them, and save your money.
Your records are out of their reach
The trouble starts when the question is about your business, such as which jobs ran over budget, which customers owe you money, or which suppliers keep missing dates. The answers live in your ERP (the main system that tracks your orders, stock and invoices), in spreadsheets, and in people’s heads.
A chat tool has read a huge part of the public internet. It has never read any of that. As I explain in why an AI project costs real money, the AI doesn’t know what you mean until someone shows it. It doesn’t know your part numbers, your price rules, or what your team means when they call an order “late.”
Both tools can now link to some business systems. Microsoft lists over 100 ready-made Copilot connectors, including ones for SAP, Dynamics 365 and common databases. ChatGPT’s business plans can link to tools like SharePoint and Google Drive. These change every few months, so check what yours can do today.
Two gaps remain. If your system isn’t on the list, Microsoft’s own guide says that building a connector “requires a developer to define a schema, register the connection in Microsoft Entra ID, and write code to pull and push data.” That work is a small build in its own right. Finding your records is a different job from adding them up. To say which suppliers were late, something has to compare every promised date with every delivery date and count them. A search that finds a few matching records can’t promise you it found them all.
Uploading a spreadsheet creates new problems
The quick fix most teams try is to export a spreadsheet and drop it into the chat. Sometimes that works well for a one-off look. As a habit, it causes three problems.
The copy goes stale. The file shows last Tuesday. Your ERP has moved on since. Nobody can tell which version the AI used when it gave its answer.
There is no source. When the answer says a supplier was late six times, you can’t click through to the six orders. You have to trust it or redo the work by hand.
Your data spreads. Each upload is one more copy of your prices or customer list, sitting in someone’s chat history. Business plans have settings to protect it. Personal plans have different ones. When a dozen people do this every week, it gets hard to know where your data has gone.
A build adds live data, real tools and a person in charge
A custom build is made to close those gaps. It has three parts.
A safe, live link to your data. The AI reads from your systems directly, with read-only access, so it sees your current figures. It only sees what the person asking is allowed to see.
Tools that return real figures. These are small programs the AI calls to do the exact work: pull the orders, compare the dates, count the late ones. The AI decides what to ask for. The tool does the sums. I wrote more about why this matters in why good tools beat a smarter AI.
A good build gives your team real numbers from your own records, every time, so nobody starts from a blank sheet. It doesn’t replace them. They still check the answers and make the calls.
A person in charge. Every answer shows where it came from, so someone can check it. A named person signs off before it is used for anything that matters, such as a quote or a payment. I explain who carries the blame when it goes wrong in who is responsible when AI gets it wrong.
Use both
Most businesses I talk to end up with both, and that is the sensible place to land. Chat seats for everyone, for the writing and thinking work they already do well. A build for the few questions about your own data that cost you the most each month.
Pick those questions with care. A good one gets asked every week, takes hours to answer by hand, and has data behind it that already exists somewhere. A wrong answer should be easy to spot. If you can’t name a question like that, a build isn’t for you yet, and the chat seats are enough for now.
It also helps to know that “AI” covers very different kinds of tool. The chat tool, the search over your files and the custom build are three separate things. I break that down in is all AI the same thing. If you want to see the kinds of builds we do, the AI Builds page lists them.
Start with the wall your team keeps hitting
If your team already uses ChatGPT or Copilot and keeps hitting the same wall, that wall is a good place to start. Write down the question they can’t get answered.
When you have it, book a call from the AI Builds page. I’ll tell you if the tools you already pay for can answer it, or if it needs a build.