SophiArch

Working with AI Tools · 20 min

Delegating Data Work to AI Coding Agents

What has changed

An AI coding agent is a tool that takes a task written in plain language and carries it out on its own: it reads your files, writes code, runs it, reads any errors, and changes the code until it decides the task is done. Claude Code, Codex, Cursor and Gemini CLI are current examples. Working with an agent is delegation: you specify the task, the agent does the work, and you review what comes back. This lesson explains what delegation changes in data analysis, what an agent still cannot know about your data, and why your review now has to cover what the agent did, not only the code it wrote.

For most of the last decade, the path into data science meant learning to write pandas, matplotlib and scikit-learn from scratch. The first AI tools changed that by suggesting code a few lines at a time. Agents go further: they run the code themselves, fix their own errors, and hand you a finished notebook.

This changes what the job requires. It does not make data science easier in the ways people assume. The hard parts of the work — understanding the data, making sound analytical decisions, communicating results, and taking responsibility for conclusions — have not been automated. What has been automated is the typing, and more and more of the running.

From pair programming to delegation

The first AI coding tools fitted a model borrowed from software teams: pair programming. One person types (the "driver"); the other reviews each line and keeps the bigger picture in view (the "navigator"). The AI was the driver. You saw every suggestion before it ran.

That model no longer describes how most people work. An agent might take thirty steps — load the file, inspect it, write a cleaning function, hit an error, change the approach, train a model — before you look at anything. You aren't watching each line. You're handing over a task and reviewing the outcome, the way you would with a new analyst on your team: fast and capable, but new to your business, and inclined to report that everything went fine.

AI code suggestionsAI coding agent
Unit of workA line or block of codeA whole task: "load, clean, model, report"
Who runs the codeYouThe agent
Who sees the errors firstYouThe agent — and it tries to fix them before you look
What you reviewThe codeThe code, what the agent ran, and what it changed along the way
Who is accountable for the resultYouYou

The last row is the one that doesn't change.

The delegation loop

The delegation loop in four steps: 1. You write the task spec (schema, goal, constraints, when to stop). 2. The agent works in a loop: plan, write code, run it, read the output or error, and fix and retry. 3. The agent hands back three things: the files it produced, the run log of what it executed, and a summary message. 4. You review the run and the result. If the review finds a problem, you sharpen the spec and delegate again; otherwise you accept the result and own it.
Delegating to a coding agent: you write the spec and review the run; the agent's own loop happens in between, out of sight unless you read the log

Two steps belong to you: the spec at the start and the review at the end. Everything in between happens without you, and by default you see only the agent's summary of it.

What the agent still can't see

It sees your data, not what your data means. An agent can open your CSV and run df.describe(). It still doesn't know that -1 in revenue means "not recorded", that customer_id repeats because of a system migration, or that last March's numbers exclude one region. It sees values, not business rules.

It works to make errors stop, not to make answers right. When an agent hits an error, its goal is to get past it. Sometimes that is a real fix. Sometimes it removes the rows that caused the error, or loosens a check that failed. In the summary, both look like success.

Its summary is a claim, not evidence. "All steps completed successfully" is the agent's description of its own work. The evidence is in the run log and the output files.

It only remembers what's written down. Each session starts fresh, apart from the project instruction files the agent reads at startup (lesson 2). A constraint you typed into yesterday's chat is gone today unless you put it in a file.

It doesn't feel the consequences. An agent that builds a leaky preprocessing pipeline doesn't know that leakage will make your model look far better than it is and lead to a bad deployment decision. It just reports the score.

Why the review matters more, not less

With code suggestions, you saw every line before it ran. With an agent, decisions are made and executed before you look. The agent automates more of the work, so your review carries more of the weight.

Here is what that looks like in practice. The first block is an agent's summary; the second is an excerpt from the log of the same run.

Agent summary:
  Done. Loaded sales.csv, cleaned the data and trained a model.
  Test accuracy: 94.1%. All steps completed successfully.

Run log (excerpt):
  [step 4] ValueError: could not convert string to float: 'N/A'
  [step 5] edit: df = df[df["revenue"] != "N/A"]    # 1,208 of 9,870 rows removed
  [step 6] re-ran cleaning: OK

The summary is accurate: every step did complete. It leaves out that the agent deleted 12% of the data to make an error go away. Whether that's acceptable depends on what 'N/A' meant. If those rows were refunds or a newly launched region, the model was trained on a different business from the one it will score. Only you can answer that question, and only if you read the log.

What this course covers

This course builds the skills that delegation depends on:

  1. This lesson: the delegation model, and what an agent can and can't know
  2. Prompt engineering for data tasks: task specs and instruction files that get output closer to correct
  3. The review protocol: a structured process for evaluating AI-generated code and the agent's run that produced it
  4. Silent failure patterns: the specific ways AI-generated pipelines fail without raising errors, including the ones agents introduce while fixing their own errors
  5. Professional responsibility: owning analysis you didn't write line by line, and deciding what an agent is allowed to touch
  6. Capstone: a full audit of an agent-produced analysis, including its run log

Each lesson is built around real failure modes. You'll see code that ran cleanly and produced wrong results, and summaries that reported success on runs that shouldn't have passed.


Summary: An AI coding agent takes a task, then writes, runs and fixes code on its own until it decides the task is done. Working with one is delegation, not pair programming: you write the spec, the agent does the work, and you review both the result and the run that produced it. Agents can read your data but not its business meaning. They work to make errors stop, and their summaries are claims, not evidence. The agent automates more of the work, so your review matters more, and you remain fully accountable for the result.

Next: Prompt engineering for data tasks — the task specs and instruction files that get agents to produce work you can actually use.

Knowledge check

4 questions · pass with 70% or better