Using AI is more than... well, using AI...

Seven years ago I wrote a post about the Mountain of Test Automation Doom. Most-read thing I've published on this blog. And every time I re-read it, I see the same shape standing behind a different acronym.

A lot of managers want their engineers to pick up AI now. Same well-intentioned shrug. They're often willing to give them time to experiment with the latest tool (Cursor? Claude Code? Whatever Microsoft is calling Copilot this week?).

Now, when a manager thinks "use AI" they probably imagine the engineer picking up a tool or two, learning the prompts, and getting code, tests, or analysis back faster than before. A chat window open in one tab, the work in another, the model doing some of the heavier lifting.

For some of us, that's mostly what it is.

The problem though is that on a sliding scale, engineers tend to end up in two camps. The ones that have been poking at LLMs since GPT-3 was a fun toy, who burned through OpenAI credits in 2023 experimenting, who keep at least half an eye on the model zoo from week to week, and who already have opinions about which tool fits which job.

Then there's the ones that have never seriously used AI for work... well... ever...

( Of course there's a sliding scale in between these two groups that people fall into, but a fair few tend to end up more towards one of the two groups mentioned above. It's a gray zone, not black and white.)

If I look to myself, I've been using ChatGPT since the day it launched in November 2022. Originally as a fun exercise to try and prove it wrong, but pushing myself to use it as much as possible to find where it actually worked. Three and a half years on, that experiment has grown into a personal knowledge base with hybrid search and a data constitution, a personal assistant I built and later killed once the data showed I didn't actually use it, an eval framework with sandboxed fixtures, a LoRA fine-tuning pipeline on local models, and MCP servers wired into my own data. Claude Code as the main driver these days, ChatGPT still in the mix for plan review, with me sitting in the middle as the architect.

So when I'm asked to pick up a new AI tool, then well... it's mostly limited to learning that new tool and how it's best being used for my context.

But when an engineer who has never used AI for real work hears this, then they see this:

"The Mountain of Doom"

Or at least that's what I called it back in 2019. Same construction, different decade. But when I look closer at this one, I'm not sure mountain is the right shape. A mountain at least lets you see the peak, even when it looks brutal and impossibly far away. There's a direction. You climb.

This thing has no peak. It has trees. Fog between them. Things moving around in the dark that you can't quite make out.

"The Forest of AI Doom" might be closer.

It looks dense, with tangled paths going off in directions that all look equally plausible, mysterious things (agents? hallucinations? something about tokens?) crashing about between the trees, and surely we're not seeing the whole thing right?

Photo by Jeff Pratley on Unsplash

What I'm trying to get at is that for an engineer that's never used AI in their work there is... well... more than just "using AI" that needs to be learned.

Let's say we're working in our normal web-shop again, writing tests or features around some GUI and a REST API back-end, and the manager has decided AI should now be part of how we work. Now that person has to be familiar with:

• Picking a tool from a list that gets longer every Tuesday. ChatGPT, Claude, Gemini, Copilot, Cursor, Perplexity, Mistral, Grok... and the honest answer to "which is best?" is "it depends" and "ask me again in three weeks."
• Subscription tiers. Free vs Plus vs Pro vs Team vs Enterprise. Which model do I actually get on this tier? Why did the free version suddenly feel worse? Why is Pro twice as expensive but apparently the API would be cheaper if I knew how to use the API?
• Prompt engineering. Not the silly "you are an expert prompt engineer, take a deep breath" stuff that goes viral on LinkedIn, but actually learning what context helps, what doesn't, how to iterate, when to start a fresh chat instead of trying to rescue a derailed one.
• Context windows. What is one? Why doesn't "I pasted the whole codebase" always work? Why does the model seem to "forget" things I told it earlier?
• Hallucinations. How do I spot them, why is the model so confident when it just invented an API endpoint that doesn't exist, and why did it cite a paper that turns out to be entirely fictional?
• Model differences. Reasoning vs non-reasoning. When does "thinking" actually help, why is one model better at code, another at writing, another at long documents.
• Knowledge cutoffs. Why is it suggesting library versions from two years ago?
• Tool use, agents, browser-using agents, computer-using agents. What's the difference between any of these and a chat? What does "agentic" even mean? And why is everyone yelling about it on LinkedIn?
• MCP servers. Oh, there's a protocol now for connecting AI to your tools? Cool. Now what's a connector, and which ones can I trust with company data?
• Custom GPTs vs Projects vs System Prompts vs Custom Instructions vs Memory. What's the difference and which one am I supposed to use?
• Wait... I can hook the AI up to our Jira / Slack / Google Drive / GitHub? Is that... is that allowed?
• Speaking of allowed, what does Legal say? What does the security team say? GDPR? The EU AI Act? The customer's NDA? Can I paste this code in? Can I paste this bug report? Probably not? Maybe not? Who decides?
• API vs chat interface. I keep hearing "just use the API" but I'm not a backend developer, and an API key is a thing I have to keep track of now.
• Tokens. What are they, why do they cost money, and why does my favourite tool burn through them so frighteningly fast the moment I let it run in "agent" mode.
• IDE integrations. Copilot, Cursor, Claude Code, Windsurf, Cline. Different paradigms, different costs, different superpowers, different ways of accidentally checking in something you shouldn't have.
• "Vibe coding." The developers are doing it. I'm not sure if it's brilliant or terrifying.
• For testing specifically, should I use AI to generate test ideas? To write automation? To analyse failures? To triage bug reports? To pair as a sounding board during exploratory testing? All of those? Where does it actually help, and where does the output just look like it helps?
• Critical thinking about plausible-looking output. The model produces something that sounds confident, uses the right jargon, references the right concepts. Is it actually right? Now I have to be more critical, not less, even though it feels like I should be done.
• Skill atrophy. If I let the model write everything for six months, do I still know how to write it myself? Do I want to be the person who can only ship work when the AI is having a good day?
• Keeping up. Every week there's a new model, a new tool, a new "this changes everything" post. When am I supposed to actually do my job?

As you can see, for someone that's never seriously used AI in their work, or maybe just dabbled with ChatGPT to write a wedding speech once, there is so much more than... well... just "using AI."

That's not a tooling problem. That's a literacy problem on top of a tooling problem on top of an accountability problem.

And underneath all of that, the one that matters most.

The model writes me 200 lines of test code, in a language I half-know, against an API I don't fully understand. How do I know it's any good? How do I know the assertions aren't just "it returned 200, so it must be fine"? And six months from now, when it goes wrong, who explains why it was written that way?

This is THE problem. Not "is the AI useful?" but "can anyone defend what it produced?"

The thing is, most of the time a manager is not aware of any of this and only thinks "use AI." The employee is nervous and doesn't even know where to start, doesn't want to show that they don't know, doesn't want to admit they don't really understand what an "agent" is or why everyone seems to have such strong opinions about MCP. That forest sure looks impenetrable.

Photo by Mohammad Alizade on Unsplash

Perhaps you're lucky and have fantastic colleagues who help you with a smile and don't make you feel stupid for asking what a token is, or whether it's okay to paste a snippet of the production database schema into the chat. Maybe you don't have to have a tense conversation with someone about why "just let the AI write the tests" isn't actually a strategy, and why the tests it wrote yesterday all pass for the wrong reasons.

But it also requires you to be willing to ask for help, to show that you don't know something, and to push back when an executive says "we're an AI-first company now." That requires a team and environment that is a safe place where we feel confident asking these questions. They sure aren't a given in any workplace, and they're arguably less of a given right now, when everyone is anxious about being judged for using AI too little, or too much, or wrong.

So as a manager, the next time you want your engineers to pick up AI, just know that it's more than just "using AI", and you need to be willing to put aside the time for it, and to respect that maybe not everyone wants to or can pick up all the things mentioned above (and I haven't even mentioned the work that isn't getting done while the team learns the new toy, or the very real possibility that for some tasks, the AI is making things worse, not better). And remember that different people will start off from different places and have more or less to learn before they can get to the "use AI" part.

And remember the part that doesn't go away no matter how good the model gets: the model didn't choose to use AI on your sprint. You did. The accountability for that choice doesn't disappear just because the output looks confident.

It's incredibly hard for a person to pick up all the things necessary for using AI well in the sprint, alongside all the other work we do concurrently, the features that need to be tested and shipped that our developers deliver to us. Expecting the employee to pick it up on their own time outside of work is just unrealistic and not even remotely fair to people who have families, private lives, dinners to make and so on. (It is also, incidentally, how you end up with people who pretend to use AI well while quietly being terrified of it, which is worse than them just not using it at all.)

There are ways to make that forest a little less intimidating, let some light through the trees, and make the whole problem easier to approach. For the people walking into it, and for the people who sent them in there in the first place. I plan to come back to those in coming articles. For now just remember.

Photo by Ales Krivec on Unsplash

"Using AI is more than just using AI."