Software engineering is unusually easy for AIs

Many people look at the breakneck pace of AI progress on software engineering tasks and expect that:

1. Software engineering tasks will shortly be ~fully automated
2. White collar work will be automated ~shortly after software engineering tasks.

I don't think #2 is true.

Why expect white collar automation?

When AIs are ~as smart as humans in all respects they will shortly afterwards take over almost all white collar work. Disembodied AIs can use computers by clicking around, and can talk to people on the phone, so they should be able to do most white collar things.

I don't expect the replacement of human workers to take long in this scenario, I'm not one of those economists who doesn't believe in competitive markets.

What am I not claiming?

One route for software engineering task automation to automate all white collar jobs would be if the automation of those tasks led to rapid improvements in AI capabilities, and the automation of AI R&D.

I don't think that's true, but I'm not arguing against it here. A brief sketch of my claim against it would be that the automation of AI R&D requires the automation of tasks AIs are bad at right now (see below) and we need to fix those deficiencies before we get full automation.

AIs are still really bad at some things

A lot of these are my impressions rather than proven things, feel free to mentally add or subtract your own.

1. AIs do not have enough 9's of reliability when completing tasks. Neither do most humans when they first start doing a task! But humans have more generic common sense, and will reach out to others for help when they see they're going wrong. LLMs often merrily go off a cliff. This is worse on longer time horizon tasks where early errors compound.
2. AIs are too insecure. The current crop is manifestly unsuited for processing untrusted data e.g. in customer service, or even in searching the internet. This gets worse with more context, on longer time horizons and with more autonomy.
3. AIs don't learn well on the job. They can only add to a giant context window, constantly refer to written documentation or get expensively re-trained. This makes them unsuitable for niche tasks, tasks without much training data, tasks which change often and tasks which rely on implicit unwritten knowledge.
4. AIs lack a certain autonomy or agency. I'm loathe to use these words as they are overused on Twitter, and this is the weakest of my arguments. But it feels like they lack a certain drive that humans have, towards improvement and long term goals. They feel like they just love completing little tasks when asked, instead of doing a whole job and being motivated to explore how to do that whole job.

Software engineering is weird

I claim that software engineering tasks are much easier for AIs to do than many other white collar tasks. Software engineering lets them sidestep their limitations in a way that other work does not. It has a number of properties that make it easier for AIs:

Checkable work

The output of a software engineering task can be checked much more easily than e.g. producing a good legal contract. We can check that the code compiles, that it passes some pre-written tests, that it can be run and produce some effect (e.g. show a website to the user).

The instant feedback of a compiler error keeps early unreliability from ruining the work. This is hidden from you as a programmer these days - the LLM will hallucinate a method name, and try to use it, only to receive a compiler error and correct itself automatically.

Checkable work also helps reinforcement learning as (in theory) the AI can be rewarded as it goes along instead of simply on the final output.

Abstractions allow reduced context

Software engineering can lay out abstractions like "The Backend Server should supply an API method to GetUserDetails and the Frontend Server should call that method to display UserDetails in some HTML."

A worker can develop a broad plan for how everything fits together and then work on little bits separately. When you are working on the Backend Server you don't have to care about the Frontend at all.

AIs therefore don't need to keep the whole context of the project in their minds at once. But many white collar jobs need this. When a product manager is working on a project presentation, they should be thinking about the whole project, the team, other involved teams, where the project was in the past and where it needs to be in the future, who they want to impress, what are shortcomings they're trying to subtly hide etc.

This is an idealistic view of course - codebases are full of hidden dependencies - but I do think it's mostly true.

There is a lot of data for training

There is a huge amount of code on the internet ready to be slurped up by companies training AIs. And you can check if that code runs to know it's not worthless. And you can generate synthetic code relatively easily.

Most white collar tasks don't have this much data. Worse, some of the knowledge is quite implicit. So just on the data quantity, it seems like the AIs would be better at software tasks than other tasks.

Similar tasks

Relatedly, many software engineering tasks are very similar ("Make me an API server to expose the data in this database"). The AI has been pre-trained on a lot of tasks like this and doesn't need to learn anything new.

White collar work often involves learning "how we do things here", which may not exist in the training set.

Changes can be made rapidly, and rolled back rapidly

When writing code you can make a change to something, run the code in a real world scenario and then change the code back, all in the space of a few minutes.

This helps with instant feedback, and also means that any changes can (in theory) be quickly rolled back. A rapid rollback means that you can take more risks, and be less reliable as a worker.

Compare that to an AI which makes a mistake in e.g. ordering the right supplies for a construction site. Resolving that mistake might take days or weeks. So greater reliability is required.

You work alone

AIs freely browsing the internet, or interacting with clients or customers, can be prompt injected fairly easily. You do not want an AI salesperson and AI legal team to interact with customers right now, because they will get prompt injected into selling your product for nothing. Also the current crop of models is designed to be nice and accommodating to people, but that could presumably be trained out of them fairly easily.

Coding tasks allow AIs to work alone, on a trusted codebase with trusted documentation.

There is one exception to this...

Your work is reviewed

Software engineering has a slightly peculiar culture whereby we do not trust our coworkers, and therefore almost all code being used has been written by one person and reviewed by another.

Unreliable workers are less bad when someone else (mostly humans for now) is checking for mistakes.

Much white collar work is not like this. Call center staff responding to customer queries do not generally get their calls reviewed in real time, they just resolve the customer's problem alone.