Blog

Update April 2026: this article is now partially obsolete. These days, all major search engines include AI models that will search the internet for you and summarize the information in the results.

To adapt this article’s lesson to today, I’d say: go ahead and use those AI search engines, but make sure you also follow the included citations to see where the AI is getting its information from. Use your judgement as to whether or not the sources support what the AI is telling you: sometimes, they still just make stuff up, or use sources that are unrelated to what you’re actually asking about.

The unedited original article follows below.

My previous article was all about how AI can’t be trusted. Maybe you think this means I’m a luddite, raging against the new technology of the day.

That’s pretty far from the truth. If anything, being at the cutting edge of technology gives me a pretty good view of what new developments are and are not good at. I maintain that AI chatbots are, as of right now, somewhat questionable at giving you reliable answers to questions, or explanations of topics you’re not already familiar with, particularly in niche fields or in areas where the details matter. You can use them for this if you want, but be careful and double-check the answers, which usually means going to a search engine to verify whatever it tells you.

But AI chatbots have one major strength: you can put a lot of words into them, and they’ll generally understand those words.

This is something that’s unprecedented in the history of computing, and it lets you do the opposite of what you do with a search engine. Instead of giving it a term and asking it to explain it, give it an explanation of what you want, and ask it to suggest you the terms to look into.

If you’ve been using AI chatbots in the past year or two, hopefully you’ve realized that they’re often pathological liars. Here’s a funny example of that from the legal world:

A lawyer asked ChatGPT for examples of cases that supported an argument they were trying to make.

ChatGPT, as it often does, hallucinated wildly—it invented several supporting cases out of thin air.

When the lawyer was asked to provide copies of the cases in question, they turned to ChatGPT for help again—and it invented full details of those cases, which they duly screenshotted and copied into their legal filings.

Now, I’m not convinced these kinds of lies are that harmful. So long as people know that they need to double-check what the AI says, these kinds of issues won’t come up very often.

A much bigger issue is subtle lies. Sometimes AIs will say things that are mostly correct, but slightly wrong, in a way that can’t easily be cross-checked.

And one of these subtle lies, one that I’ve seen multiple chatbots say, is that they’re programmed.

Let me explain why this is a lie, in a way that’s hopefully understandable by non-programmers.

There’s an idea in software product development called “3X”, standing for Explore, Expand, and Extract. Each is supposed to represent a different stage of a software product’s journey.

This was originally proposed by Kent Beck, one of the original pioneers of the Agile Software Development movement. He has a lot of writing about how the model works and some of its consequences, but here’s my quick description of it.

Imagine you were looking for convincing explanations as to why stealing is wrong. Let’s use the example of stealing from your local mom-and-pop store.

You ask a therapist, and they make an appeal to your empathy. Think about how that would make the owners feel: they rely on selling their goods to make a living, and if too much is stolen they may not be able to make ends meet. You have a good amount of empathy, and so you find this argument reasonably convincing.

You ask an economist, and they tell you that theft imposes costs on everyone else. In order to compensate for the loss of income from selling that good, the store will have to raise prices. They might even have to invest in security systems and cameras, and raise prices further to cover the cost. You don’t like it when goods get more expensive, so you find this reasonably convincing.

You ask a judge, and they you it’s good to live under the rule of law. If everyone goes around just stealing whatever they want, it leads to chaos, as people take to more desperate measures to secure the property they need to live their lives and run their businesses. You like living in a society with trustworthy laws, so you find this reasonably convincing.

Then you ask one of the more uneducated Christians in your neighbourhood, and they tell you that stealing is wrong because Jesus said so.¹

In my day job, I build a platform: that is, software that lets other people build things. The tools we build are used by people both inside and outside the company to make some pretty neat stuff.

Since we’re co-workers, though, the internal folks using our platform get the privilege of talking directly to me and my team on a regular basis. As a result, I deal with a lot of teams on that are in this kind of situation:

• They have some product they want to build using our tech.

• In order to build this product, they need some additional features from the platform.

The people that do this tend to be great product developers that know their intended audience well, and can build something really compelling. They have a keen sense of what they’re looking for, and are very talented.

But the trouble is, there’s a subtle but important mindset shift that’s needed when switching between product development and platform development.