Will AI Replace Software Engineers?

AI has been a growing thing for a while now. Once Github Copilot released, and ChatGPT released, a recurring question has continued to appear all over the software industry and that’s if AI will replace software engineers, and if we’ll all be out of a career in the near future. There’s also a lot of misinformation as well, and a mixture of naivety about what a software engineer really does. I’ve heard and talked to folks from various places in their career, but usually if you break it down far enough, people realize the root truth. Once you remove the marketing, the hype, the anxiety, and you really try it for yourself outside of trivial things, you start to realize the real state of where AI is in the industry. To put it simply, our careers aren’t going anywhere. But there’s a lot of insights into why, and where to go from here.

Context is critical, and where AI falls short often

Have you ever tried to maintain a long running thread in ChatGPT and you slowly realize that it’s forgetting what you told it earlier? That’s a frustrating but common issue. If AI can’t maintain context, then it has no nope of truly replacing an engineer. Having and maintaining context is one of the most important skills of an engineer, including context around code, the current feature you’re working on, sibling features that will be affected, other unrelated features, business context, user context, etc. That’s all crucial to our jobs.

I was having a ChatGPT thread about getting a desktop and mobile menu HTML and CSS setup to streamline a new header navigation I wanted to make. I told it I needed it to be a horizontal navigation on desktop, and a hamburger menu on mobile that when you click it show and hides the menu on mobile. I would try to refine the output slowly to add an animation, add an active state, etc. and it slowly would forget things. And as we’ll get to later, it didn’t ever work correctly either. I spent more time trying to get ChatGPT to give me the correct result than if I just did it myself, or if I took its first response which was about 60% of what I needed and finished the rest on my own.

Another example is I was having ChatGPT auto-generate a new custom post type for this website, and I specifically asked for it to generate the arrays using bracket syntax. But it didn’t, it used a different syntax. I told it after to use bracket syntax, and it did. I then asked for it to alter the custom post type to add a new field, and it reverted back to the syntax I didn’t want, in one simple step.

Micro-level and macro-level

AI and things like ChatGPT, Copilot, etc. are amazing when you’re looking for things on the smallest of scale. Want to fire off an API for a URL, and grab specific data from the response? Want to parse a quick file? Forgot how to setup a new custom post type in WordPress? In these cases, AI is great! You’ll get explanations, and code really quickly. All engineers should be incorporating AI into their toolkit and workflow.

But on the macro level, it falls short. When you need to provide a lot of detail and a lot more context, the resulting quality drops a lot. I’ve given ChatGPT a SFC (Single File Component) in VueJS and told it to generate unit tests for the code, and I specifically gave it the context of our testing libraries. I tried to use the generated tests, but it failed when they ran. Similarly, we had a small team at our company spend a week trying to streamline this and automate the generation of unit tests with SFCs in Vue, and had the same problems both in ChatGPT 3.5 and 4. It simply always failed to include the context it was specifically provided time and again and the unit tests would fail to build and fail to run. The test code for each individual test looked fine though! It was simply all of the tooling, context, and builds that would fail. Despite a week of working on it with a small team, we decided that it simply wasn’t ready for it.

So for now, we use AI to help generate code on a small scale, but still take the time to be mindful of what it generates and how to incorporate it into a broader system. In that way, it’s really no different than Stack Overflow, except that it’s possibly more efficient in terms of time.

Bugs are a common problem

A common complaint is simply that the code generated has bugs. If you use it often enough, you’ll quickly learn this, and learn to not simply copy and paste code generated from AI into your project. Again, it’s really no different than getting code from Stack Overflow or somewhere else. You really need to take the time to understand what it’s doing too.

I tried to use ChatGPT to read a PHP file, which was a custom post type in WordPress, that had many different layouts and fields in it, with the file being around 20,000 lines of code long. I wanted it to read the file, and write a script that would split the custom post type into different files so it would be easier to maintain. It didn’t work immediately, but with some tweaking the code executed and it generated different files. But the names didn’t work as expected. But the sneaky thing was, it didn’t generate all of the definitions! It wasn’t apparent though at first. We imported the files, and it worked until we ran into a few pages that just flat broke. After researching, we found that definitions were missing, and it wasn’t clear at all why they got skipped.

You’ll find many other experiences like that, and I have others too. So like any generated or community code, take it with a grain of salt, test it, and fully understand it before committing to it.

Bigger picture cannot be replaced easily

All that we have talked about so far is mostly related to code creation, maintainability, etc. But another key aspect of being an engineer is to understand technology on a greater scale than a function, or a class. You have to understand how a whole feature is put together, how that feature may exist in a monolith, a microservice, or use messaging systems to communicate. And how to connect features to other features in meaningful ways that fulfill a business need. And do so with knowing the balance of using the right tools for the job, for the company, and for your team. All while being flexible with changing requirements, changing technology, working with different departments like sales, support, operations, management, marketing, and more.

We’re nowhere near a place where a business analyst can sit down and tell ChatGPT to create a new SaaS application with all of the various features, networking, database, unit tests, pipelines, deployments, and then continually evolve that product over time simply through a small interface.

Beware of AI company marketing

Another often overlooked element is the power of marketing. Many AI companies are trying to make a name for themselves and race into the AI sector. There’s a lot of money to be gained in that vertical. And that means a lot of hype, and over-promises are all over the place right now with AI.

I spent a week analyzing BI tools that promised AI features to streamline and automate how business analysis gain insights from their data. I reviewed and tested many software suites, and the results were a large mix of AI that was promised to exist that simply didn’t, to AI that really just automated clicking a few buttons in a naive and unhelpful way due to how error prone it was, to actually impressive analysis created through a sentence.

So when you see or hear how AI is transforming the industry and engineers are in danger of losing their careers, take it with a mountain of salt because it’s likely you’re being sold a service that won’t live up to those promises.

So what is the outlook for AI and engineers short term?

The short term outlook is that engineers aren’t in any danger from being replaced with AI. What engineers should be doing is evolving their workflow to lean on AI to help them do their job, but do so very carefully. There are a lot of pitfalls to using AI right now, but there are benefits too.

What is the outlook for AI and engineers long term?

AI will continue to evolve. What does that mean in 20 years from now? Who knows. Technology has evolved drastically in the last 20 years, and engineers are still here. We still don’t have flying cars, but we do have robots that make food, and stuff like that. So who knows. So there’s no value in worrying over something that far ahead, and something you don’t control.

Do what you can to make yourself a better engineer today, and enjoy the ride.