Your AI Can Write Code.
But Can It Actually Run It?

A few years ago, we were amazed that AI could write an email. Today, we're watching AI tools build applications, browse the internet, fix bugs, and even run commands on our computers.

If you've used tools like OpenAI's Codex or Anthropic's Claude Code, you've probably experienced this yourself. You describe what you want, and the AI starts working. It opens files, writes code, runs tests, encounters errors, fixes them, and sometimes comes back with a message saying the task is complete.

It's impressive.

But something about this is worth questioning.

How did a language model, something originally designed to generate text, suddenly learn to operate a computer?

Did we train it to use a terminal? Does it somehow understand where files are stored on our computers? How does it know which command to execute or which file to modify?

And perhaps the most interesting question: Is Codex itself an LLM, or is there something else going on behind the scenes?

To understand this, we need to go back to where it all started.

Let's forget about AI agents for a moment.

Imagine I give you a sentence:

"The weather outside is..."

You might finish it with "beautiful," "terrible," or "cold."

Your answer depends on what you've read, heard, and experienced.

Now imagine a machine trained on enormous amounts of text, code, books, and other information. During training, it learns patterns in how words relate to one another, how sentences are formed, how programming languages work, and even how people approach different kinds of problems.

That's roughly the foundation of a Large Language Model (LLM).

Of course, the actual technology is far more complicated. Modern LLMs use neural networks with billions of learned parameters, and their capabilities go far beyond finishing simple sentences.

Underneath all that complexity, however, there's a surprisingly simple idea.

An LLM receives input and generates output.

That output is produced one token at a time. A token might be a word, part of a word, or even a symbol.

For example, when you ask an LLM to write a Python function, it generates a sequence of tokens intended to form Python code. That code can still contain syntax errors or logic mistakes.

It has learned enough about programming to recognize patterns, understand requirements, and generate solutions.

But here's where things get interesting.

Writing code and executing code are two completely different things.

Suppose I ask an LLM:

"Write a Python program that calculates the average of three numbers."

Without execution tools, it might produce:

numbers = [10, 20, 30]

average = sum(numbers) / len(numbers)

print(average)

Perfectly reasonable code.

But did the LLM execute it?

No.

Did it create a Python file on my computer?

No.

Did it even check whether Python was installed?

Again, no.

It simply generated text that happened to be Python code.

Now let's make the request slightly more interesting.

"Open my project, find the bug, fix it, and run the tests."

The LLM might understand exactly what I'm asking. It might even be capable of identifying the bug if I showed it the relevant code.

But there's a small problem.

It has no idea what's inside my project because nobody has shown it the files.

And even if I pasted every file into the conversation, the model itself still wouldn't have the ability to modify those files on my computer.

This isn't necessarily a limitation of its intelligence.

It's a limitation of what it can access and what it's allowed to do.

Think about it this way.

Imagine having a brilliant software engineer sitting in a room with no computer. You can ask them any programming question, and they'll happily explain the solution.

But ask them to fix a bug in your application, and they'll probably say:

"Sure. Can I see the code?"

Give them the code, and they might identify the problem.

But unless you also give them access to a computer, they can't actually change anything.

An LLM faces a similar problem.

And this is where the story starts getting interesting.

What if we gave the LLM access to tools?

We already have a model capable of understanding programming problems.

Instead of asking it to explain how to fix something, what if we gave it a way to interact with our computer?

Let's start with something simple.

Suppose we have a Python function that reads a file.

def read_file(path):
    with open(path) as file:
        return file.read()

Nothing fancy. Just an ordinary Python function.

Now imagine we build an application that tells the LLM:

"Hey, whenever you need to read a file, you can request this function."

Suddenly, something changes.

Instead of responding with "Please paste your code here," the model could generate a request that looks something like this:

Here is an illustrative format; actual tool-call schemas depend on the application.

{
  "tool": "read_file",
  "arguments": {
    "path": "app.py"
  }
}

Pretty interesting, right?

But look closely.

The LLM still hasn't read the file.

All it did was generate a structured request asking for a function to be executed.

Something else needs to receive that request, recognize the function name, execute the Python code, and collect the result.

That result then needs to be sent back to the LLM.

Let's say the file contains:

def divide(a, b):
    return a / b

The application executes read_file("app.py") and passes the contents back to the model.

Now the LLM can actually see the code.

It might notice that the function raises an error for division by zero. Whether that is a bug depends on the intended behavior; it could suggest adding a check if the application needs one.

But what if we want it to make the change itself?

We could give it another tool.

def write_file(path, content):
    with open(path, "w") as file:
        file.write(content)

Now the model can request that a file be modified.

Of course, real applications need permission checks, safe file handling, and other protections. We probably don't want an AI assistant overwriting arbitrary files on our computer.

But ignoring those details for a moment, something important has happened.

We've taken a model that could only generate answers and connected it to functions that can perform real actions.

And there's something even more interesting.

We didn't need to retrain the entire LLM to teach it how to read a file.

We made a tool available to a model that supports tool calling and described how that tool should be used. The model may already have been trained to select tools and produce structured calls; adding this particular function usually does not require retraining it.

The LLM generates the request. Our application executes it.

But now we've created another problem.

What happens if the model needs to use multiple tools?

Who's actually managing all of this?

Let's return to our bug-fixing example.

I tell the AI:

"Find the problem in my application and fix it."

First, it needs to inspect the files.

Then it might need to search for a function.

After finding a potential bug, it needs to modify the code.

Then it should probably run the tests.

And what if the tests fail?

It needs to read the error, understand what went wrong, make another change, and run the tests again.

Notice how this is no longer a single question followed by a single answer.

We're describing a process that involves multiple decisions and actions.

Something needs to keep track of all this.

Something needs to tell the model which tools are available, execute its requests, return the results, and maintain enough context for the model to decide what to do next.

And that brings us to one of the most important, yet often overlooked, parts of modern AI systems.

The agent harness.

The name might sound complicated, but the idea is straightforward.

A harness is the software surrounding the LLM that manages its interactions with the outside world.

Think of an LLM as a chef.

The chef knows thousands of recipes, understands ingredients, and can explain how to prepare an amazing meal.

But put that chef in an empty room without a kitchen, ingredients, or utensils.

You'll get great cooking advice.

You won't get dinner.

Now give the chef a kitchen, ingredients, and the equipment needed to cook.

Suddenly, they can do much more than explain recipes.

The harness plays a similar role for an LLM.

It provides the environment and coordinates the tools that allow the model's decisions to turn into actions.

Here's a simplified picture of what happens when we ask an AI agent to fix a bug.

The important thing to notice is that the LLM isn't directly opening files or running commands.

It's generating requests for actions.

The harness is responsible for executing those actions through the available tools and returning the results.

This interaction can happen repeatedly.

In fact, that's exactly what makes modern AI agents so interesting.

The model doesn't necessarily have to stop after generating one response.

It can receive the result of an action, examine it, decide what to do next, and request another action.

This repeated process is often called an agent loop.

And once you understand this, something about Codex and Claude Code starts making a lot more sense.

So, are Codex and Claude Code actually LLMs?

This is where I think a lot of confusion comes from.

We often hear people say things like:

"Codex is better at coding than Claude."

Or:

"Claude Code fixed a bug that another model couldn't."

There's nothing inherently wrong with comparing these products.

But there's an important distinction that's easy to miss.

Claude is a family of AI models. Claude Code is a coding agent built around those models.

Similarly, the Codex product is a coding agent that combines models with a harness and development tools. OpenAI has also used the Codex name for models, so it helps to distinguish the product from the model powering it.

When you ask Claude Code to fix a bug, the underlying model helps understand the problem and decide what to do.

But Claude Code also provides the surrounding software needed to interact with your project.

It can make tools available for reading files, editing code, executing terminal commands, and inspecting results.

Codex follows a similar general idea.

The details of their implementations differ, but the underlying concept is the same.

And here's something worth thinking about.

Imagine two AI coding agents using the exact same LLM.

The first agent can only read files and suggest changes.

The second can read files, modify them, execute tests, inspect errors, and retry failed changes.

Which one would you expect to be more useful for fixing a bug?

Probably the second.

But why?

The underlying model is identical.

The difference is the software surrounding it.

This is one reason a coding agent's performance cannot be understood by looking at the underlying LLM alone.

The quality of the harness matters too.

A poorly designed harness might repeatedly send unnecessary information to the model, lose track of previous results, or execute tools inefficiently.

A better harness might retrieve only the relevant files, manage context carefully, handle failed commands, and provide useful feedback after every action.

None of this means a good harness can magically turn a weak model into a brilliant one.

Both the model and the surrounding engineering matter.

But it does explain why two products using similarly capable models can behave very differently.

And it also explains why we're hearing more about something called harness engineering.

Building a capable AI application isn't just about choosing a powerful model and writing a clever prompt.

It's about designing the entire system that allows the model to work.

But there's still a question we haven't answered.

We've given our AI access to local files and terminal commands.

What happens when we want it to interact with other applications?

Our AI can read files. Now what about GitHub?

Suppose our coding assistant has successfully fixed a bug.

Now I ask:

"Can you create a GitHub issue describing the problem?"

The LLM knows what a GitHub issue is.

It can write a title and description.

But how does it actually create the issue?

Well, GitHub provides something called an API.

An Application Programming Interface (API) allows one software application to communicate with another.

Think about a food delivery app.

When you place an order, the app needs to communicate with systems that handle restaurants, payments, and deliveries.

It doesn't magically know what's happening inside those systems.

It sends requests through defined interfaces and receives responses.

That's the basic idea behind APIs.

GitHub provides an API that allows applications to create issues, manage repositories, read pull requests, and perform many other operations.

For example, creating an issue might involve an HTTP request like this:

POST /repos/example/project/issues

With data such as:

{
  "title": "Login button not working",
  "body": "Clicking the login button does nothing."
}

GitHub processes the request and creates the issue, assuming the request is authenticated and authorized.

Nothing particularly AI-specific is happening here.

Developers have been using APIs long before LLMs became popular.

So we could simply write a function that calls GitHub's API and make that function available to our LLM.

Problem solved.

Or is it?

Because now I want my assistant to check Gmail.

And Google Calendar.

And search Google Drive.

And retrieve information from a database.

Each service has its own API, authentication requirements, request formats, and responses.

But wait.

Why is that a problem? Isn’t communicating with different APIs something software already does?

Yes. An AI agent can use those APIs too.

We could write an integration that creates GitHub issues, another that reads Gmail, and another that checks Google Calendar. Our application would handle the service-specific requests and return the results to the model.

That would work. We already have everything needed to connect our agent to those services.

And the model wouldn’t necessarily need to understand every HTTP header or authentication detail. Our integration code could handle those, just as it does in ordinary software.

So different APIs aren’t the missing piece. What problem is still left to solve?

Then why not use APIs for everything and stop there?

For one application, that might be exactly the right choice.

But suppose we want the same GitHub integration to work with several different AI applications.

The GitHub API stays the same. Yet each application may have its own way of registering tools, describing their inputs, and passing back results. We might reuse the code that calls GitHub and still need extra work to connect it to each application.

What if those applications agreed on a common way to discover and call that integration?

This is where another acronym enters the picture.

And if you've been following discussions about AI agents, you've probably heard it everywhere.

MCP.

Wait, isn't MCP just another API?

MCP stands for Model Context Protocol.

I'll admit, the name isn't particularly beginner-friendly.

But the idea behind it is easier to understand than it sounds.

Let's go back to GitHub.

We already know that our AI assistant can interact with GitHub through its API.

To make that possible, we could build our own custom integration.

We'd write code that calls GitHub, describe the available functions to the LLM, handle authentication, and process the responses.

That works perfectly well.

But what if someone else has already built a reusable integration?

And what if multiple AI applications could communicate with that integration using a common protocol?

That's essentially the problem MCP helps solve.

MCP defines a standard way for AI applications to discover and interact with external capabilities.

That’s the problem MCP helps address.

It doesn’t remove GitHub’s API or make Gmail and GitHub behave identically. It gives compatible AI applications a shared way to connect to capabilities exposed by a server.

Someone still has to handle the underlying API. The model still needs to understand what a tool does and what information it requires.

MCP changes how the integration connects to the AI application—not whether the underlying API is needed.

So using APIs directly isn’t an inferior approach. MCP becomes useful when that common connection makes an integration easier to share and reuse.

A useful analogy is USB.

Think about how frustrating it would be if every keyboard, mouse, and storage device required a completely different connector.

Standardized interfaces make it easier to connect devices without designing an entirely new connection every time.

MCP aims to provide a similar kind of standardization for AI applications.

It's not a perfect comparison, because software protocols are more complicated than physical connectors.

But the basic idea holds.

Instead of every AI application needing a completely different way to discover and call external tools, MCP provides a common communication protocol.

And this brings us to something called an MCP server.

Despite its name, an MCP server doesn't necessarily mean a giant computer running somewhere on the internet.

It can simply be a program running on your own computer.

Its job is to expose capabilities that an AI application can use.

For example, a GitHub MCP server might expose tools that allow an AI assistant to search repositories, create issues, or read pull requests.

A filesystem MCP server might expose tools that allow an assistant to read files from permitted directories.

A database MCP server might provide access to certain database operations.

The AI application connects to these servers through MCP clients.

The application itself is called the MCP host.

Let's visualize it.

Now look at that diagram again.

Do you notice something?

The GitHub API is still there.

MCP didn't replace it.

The MCP server is simply providing a standardized interface through which the AI application can access GitHub functionality.

And that's the distinction I wish someone had explained to me when I first started hearing about MCP.

An API allows software systems to communicate.

MCP defines a common way for AI applications to connect with tools and other sources of context.

They aren't competing technologies.

In fact, they often work together.

Here's another way to think about it.

Imagine a restaurant with its own ordering system.

That's similar to an API. The restaurant defines how orders must be submitted.

Now imagine a common interface that helps an AI assistant discover what can be ordered, understand the required information, and submit an order through that restaurant's existing system.

That's closer to the role MCP plays.

The restaurant's ordering system still exists.

We've simply added a common interface for the AI assistant.

And here's an important detail.

You don't actually need MCP to build an AI agent.

You can connect tools directly using ordinary functions and APIs.

MCP becomes useful when you want standardized, reusable integrations across applications.

It also supports more than tools. MCP servers can expose resources and reusable prompts, giving AI applications other ways to access information and capabilities.

So MCP isn't the thing that suddenly makes AI intelligent.

It isn't a replacement for APIs.

And it isn't an LLM.

It's a communication protocol.

That's all.

The interesting part is what becomes possible when applications agree on that protocol.

Does this mean an LLM with MCP is automatically an AI agent?

Not necessarily.

Let's say we give a chatbot access to a weather tool through MCP.

We ask:

"What's the weather in Dallas?"

The model requests the tool, receives the result, and generates an answer.

That's a useful tool interaction.

But it doesn't automatically mean we've created a sophisticated autonomous agent.

An AI agent generally involves a system in which a model can work toward a goal by selecting actions, observing results, and deciding what to do next.

The distinction isn't perfectly defined across the industry, but it's useful.

Consider these two requests.

First:

"What's the weather in Dallas?"

The system checks the weather and responds.

Now consider:

"Look at my application, find the performance bottleneck, improve it, run benchmarks, and tell me whether your changes actually helped."

That second request is very different.

The agent doesn't know in advance exactly how many steps it will need.

It might inspect the source code, run profiling tools, identify a slow database query, modify the query, and run benchmarks.

What if performance gets worse?

It should recognize that the change didn't help.

Maybe it needs to revert the modification and try another approach.

Now we're describing a system that repeatedly uses feedback from the environment to guide its next action.

That's one of the defining ideas behind agentic workflows.

But there's something else worth mentioning.

An agent doesn't become reliable simply because it can call tools repeatedly.

What if it misunderstands the benchmark results?

What if it modifies the wrong file?

What if it confidently claims the application is faster even though the measurements show otherwise?

Giving a model more tools also gives it more opportunities to make mistakes.

That's why the software surrounding the model matters so much.

A well-designed harness needs to think about permissions, error handling, context management, and ways to verify whether a task was actually completed.

You don't want an AI assistant deleting your database because it thought that was the easiest way to resolve a problem.

More capability also means more responsibility for the systems we build around the model.

And this is where the AI world becomes fascinating.

So far, we've talked mostly about coding assistants.

But there's no reason these ideas should be limited to software development.

Imagine you're building an AI assistant that helps people apply for jobs.

You tell it:

"Find software engineering positions that match my experience, customize my resume, and help me submit applications."

Think about how much work is hidden inside that one sentence.

The assistant needs to find job listings.

It needs to understand your experience and compare it with job descriptions.

It may need to generate a customized resume.

Then it needs to navigate application websites, fill in forms, upload documents, and handle unexpected questions.

Finally, it should keep track of which applications were submitted.

That's not something a language model can accomplish merely by generating a paragraph of text.

But combine a capable model with the right tools and a well-designed harness, and much of this workflow becomes possible.

The model might use a search API to retrieve job listings.

It might use document-generation tools to prepare resumes.

It might use browser automation to interact with application forms.

Some of those tools could be exposed through MCP servers.

And the harness would coordinate the process, including asking the user for approval before submitting applications.

Notice how all these pieces fit together.

The LLM doesn't need to know the internal implementation of every application it interacts with.

It needs clear information about what capabilities are available, how to request them, and what results come back.

The surrounding software takes care of the execution.

Of course, building this reliably is much harder than the simple explanation makes it sound. Websites change, tools fail, models make incorrect decisions, and unexpected situations appear.

But that's precisely what makes this field interesting.

We're no longer just asking whether an AI model can generate a good answer.

We're asking whether an entire AI system can complete a real task.

And those are two very different engineering problems.

So what are AI companies really competing on?

Whenever a new model is released, people immediately start comparing it with other models.

Which one is smarter?

Which one writes better code?

Which one has better reasoning capabilities?

Which one scores higher on benchmarks?

These are useful questions.

But as AI applications become more agentic, I think we need to ask another question.

How good is the system built around the model?

Imagine two coding assistants using the same underlying LLM.

One constantly forgets relevant context, repeatedly reads the same files, and struggles to recover when a command fails.

The other provides relevant information, executes tools efficiently, tracks changes, and verifies its work.

The difference in their performance could be substantial, even though they're using the same model.

And that's why I find the idea of harness engineering so interesting.

We're beginning to realize that making AI systems more capable isn't only about training larger or smarter models.

Sometimes, the surrounding software is just as important.

Better tool interfaces, more useful context, safer execution environments, and reliable feedback loops can dramatically change what an AI system is able to accomplish.

This doesn't mean models are becoming less important.

A poor model can still make poor decisions, regardless of how sophisticated its harness is.

But it means intelligence alone isn't enough.

A model needs a way to interact with the world.

And that interaction needs to be designed carefully.

The funny thing is, we ended up where we started.

Remember our first example?

We asked an LLM to write a simple Python program.

It generated the code.

Nothing more.

Then we gave it a function to read files.

Now it could inspect our project.

We added another function to write files.

Now it could request modifications.

We introduced a harness.

Now the system could execute a sequence of requested actions, return the results to the model, and continue working toward a goal.

We connected external services through APIs.

Now it could interact with applications beyond our computer.

We introduced MCP.

Now some of those capabilities could be exposed through a common protocol, making integrations easier to reuse.

And somewhere along the way, what started as a system that generated text became part of an application capable of performing real work.

Yet there's something fascinating about the whole process.

The underlying LLM is still receiving input and generating output.

What's changed is what those outputs mean to the surrounding software.

Sometimes the output is a paragraph explaining a concept.

Sometimes it's a block of Python code.

And sometimes it's a structured request that tells the harness which tool to execute next.

The difference is that the application can interpret certain outputs as requests for actions.

That's what makes the combination so powerful.

And it also raises an interesting question.

When an AI agent successfully builds an application or fixes a complicated bug, how much of that achievement comes from the intelligence of the model, and how much comes from the engineering around it?

There isn't a simple answer.

Both matter.

A brilliant model with no useful tools is limited in what it can accomplish.

A sophisticated harness with a model that consistently makes poor decisions won't be particularly useful either.

The real power comes from combining them.

And once you understand that, tools like Codex and Claude Code start looking a little different.

They're not simply chatbots that became better at programming.

They're complete software systems designed to turn a model's decisions into useful actions.

One last thought

The next time you see an AI demo where someone says, "Look, the AI built an entire application by itself," don't just think about how intelligent the model must be.

Think about what's happening behind the scenes.

How is it reading the files?

Who's executing the terminal commands?

How are the results getting back to the model?

What happens when something fails?

And how does the system know when the task is actually complete?

These might sound like small implementation details.

But they're a huge part of what makes modern AI agents possible.

And perhaps that's what makes the current AI landscape so exciting.

We're not just building models that can generate increasingly intelligent answers.

We're building systems that can take those answers and turn them into actions.

The model is only the beginning of the story.

If you're curious to explore further, the Model Context Protocol documentation ↗, Anthropic's Claude Code documentation ↗, and OpenAI's articles on harness engineering ↗ and the Codex harness ↗ are good places to start.