People have been predicting the "end of programming" since the dawn of time. It's one of those tired old tropes that refuses to die.  In the mid-1990s it was all about CASE tools, closely followed by 4GLs. Then "low code" tools were going to empower "citizen developers" to build applications without engineers. Now it's the turn of generative AI and large language models (LLMs) to provide fuel for this lazy old cliché.

The general idea is that generative AI will form a new abstraction that greatly simplifies building applications. It can be seen in terms of a long line of evolution running from assembly code through to more modern languages and tools. A common thread of this evolution has been the emergence of more powerful development environments and robust engineering practices that improve productivity. Now we are on the threshold of a new generation of AI-powered tools that will transform the role of an engineer.

Some have even gone as far to say that engineers will become redundant. After all, AI can be scaled horizontally, doesn't need time off, and can be trained more quickly than an engineer. The tools in this space will only become more sophisticated, eventually leading to teams of AI-powered "agents" that generate code for human prompters.

Some of the emerging tools are undeniably impressive. Github Copilot can feel like Stack Overflow on steroids. Instead of looking up how to build something you can get a chatbot to tailor it to your use case. It feels like a step-change where usable boilerplate can easily be generated for everything from integration pipelines to unit test suites. However, there is a world of difference between creating productivity tools and replacing the role of software engineer.

Are generative AIs good at coding?

Given how impressive these tools can be, it can be tempting to yield to magical thinking and interpret their capabilities in terms of human intelligence. LLMs are good at writing boilerplate because they have been trained against millions of lines of code that implement working examples. Once you try to build a full system using an LLM then you come up against some of the natural limitations of these models.

Generative AIs are hard to work with as they are not deterministic or even that predictable. They forget things, give inconsistent responses, occasionally hallucinate, and are easily duped. Although their capabilities are growing all the time it may be a mistake to assume that they are on an inevitable path towards being able to build systems without human intervention.

We may even have already passed "peak AI" for software engineering. A recent paper found significant degradation in the problem-solving performance between GPT3 and GPT4. This included the accuracy of generated code, where less code could be executed without modification. Recent research also suggests that Github Copilot has an adverse effect on code quality, mainly due to an increase in code churn and cut-and-paste.

AI-generated code may become unsustainable over the long term if humans stop coding. A potential shortage of accessible training data is a known problem for generative AI. Feeding models with synthetic data can cause "model collapse" where performance degrades over successive iterations. Model Autophagy Disorder is like a form of in-breeding for generative AI where models "eat themselves" after being trained on model-generated content. If Stack Overflow becomes a ghost town, how are we going to train the next generation of coding agents?

AI is not good software...

Generative AI tools always demo well, which can make sober assessment of their capabilities that much more difficult. Simple use cases are usually executed on point and to rapturous applause. Once you inject real-world complexity and the need for nuanced interpretation into the mix, they quickly become less impressive.

The idea that prompt engineering can be a "low code" tool that empowers a new generation of "citizen developers" may deny the reality of building systems. "Low code" tools make it easy to build a basic application, but they often make complex problems much more difficult to solve. Generative AI is no different in that respect. It's easy to get lost in a world of over-complex prompts to the point where you might as well write the code yourself.

Software engineering is so much more than blandly interpreting a bunch of written instructions. It requires knowledge of the wider domain, an appreciation of how the software will be used, and no small amount of creative problem solving. Interpretation of intent and understanding of context are vital. Generative AI can do surprising things, but it never achieves a nuanced understanding of circumstance.

Ethan Mollick suggested that "AI is not good software, it is pretty good people". This quote is based on an acceptance that generative AI systems do not work like repeatable and standardised software. We are accustomed to deterministic systems that work in the same way every time and we are prone to misunderstanding generative AI. There's no manual to help us figure out how these systems work, and we can only understand their capabilities by using them.

This suggests that generative AI tends to be more useful in those areas where you already have some expertise and can provide guidance. In this sense, you are treating them as people that you can delegate work to. Generative AI is something that we can collaborate with, but it's unlikely to replace us.

Has the revolution already happened?

Perhaps the revolution in programming has already happened. After all, we don't really talk about "computer programmers" anymore. It's all about software engineers and has been for some time. This is an important nuance. It implies a greater role for creative problem-solving and lateral thinking, which generative AI is notoriously poor at.

Software development as a profession became accessible to people without computer science degrees some time ago. Many software engineers spent their student years learning how to critically assess written sources and form coherent arguments as opposed to writing code. This has given rise to greater diversity of thought within the industry, which can only be a good thing. The idea of using generative AI to build systems denies the growing emphasis on collaboration, effective communication, and self-reflection in software engineering.

Generative AI does have a big part to play in building future systems. It may give rise to some new abstractions that accelerate existing trends in software engineering, helping us to be more productive and build more reliable systems. Simpler use cases may even become more accessible to people who do not have a software engineering background. I guess that doesn't make for quite as good click-bait as "the end of programming"...

Filed under Architecture, Strategy, Development process and Rants.