Are We Really Engineers?

This is part one of the Crossover Project. Part two is here and part three is here. A conference talk based on this work is now available here.

I sat in front of Mat, idly chatting about tech and cuisine. Before now, I had known him mostly for his cooking pictures on Twitter, the kind that made me envious of suburbanites and their 75,000 BTU woks. But now he was the test subject for my new project, to see if it was going to be fruitful or a waste of time.

“What’s your job?”

“Right now I’m working on microservices for a social media management platform.”

“And before that?”

“Geological engineering. A lot of open pit mining, some amount of underground tunnel work. Hydropower work. Earth embankment dams because they come along with mines.”

He told me a story about his old job. His firm was hired to analyze a block cave in British Columbia. Block caves are a kind of mining project where you dig tunnels underneath the deposit to destabilize it. The deposit slowly collapses and leaks material into the tunnels, and then “you just print money”, as Mat called it. The big problem here? The block cave was a quarter mile under a rival company’s toxic waste dump. “In the event of an earthquake, could the waste flood the mine and kill everyone?” He had to prove it was safe. A different kind of work than what he was doing now.

As for another difference? “My personal blog has better security than some $100 million mining projects.”

This wasn’t going to be a waste of time, after all.

---

Is software engineering “really” engineering? A lot of us call ourselves software engineers. Do we deserve that title? Are we mere pretenders to the idea of engineering? This is an important question, and like all important questions, it regularly sparks arguments online. On one hand you have the people who say we are not engineers because we do not live up to “engineering standards”. These people point to things like The Coming Software Apocalypse as evidence that we don’t have it together. They say that we need things like certification, licensing, and rigorous design if we want to earn the title engineer.

On the other end of the horseshoe, we have people like Pete McBreen and Paul Graham who say that we are not engineers because engineering cannot apply to our domain. Engineers work on predictable projects with a lot of upfront planning and rigorous requirements. Software is dynamic, constantly changing, unpredictable. If we try to apply engineering practice to software then software would be 10 times as expensive and stuck in 1970.

For a long time I refused to call myself a software engineer and thought of other people with that title as poseurs. It was reading a short story against engineering that made me question my core assumptions. It was A Bridge to Nowhere, one of the Codeless Code stories. The author argues that the techniques of bridge building don’t apply to software, since software clients can change their requirements and software gravity can sometimes reverse itself.

The predictability of a true engineer’s world is an enviable thing. But ours is a world always in flux, where the laws of physics change weekly. If we did not quickly adapt to the unforeseen, the only foreseeable event would be our own destruction.

That’s when I realized something about everybody involved in all of these arguments.

They’ve never built a bridge.

Nobody I read in these arguments, not one single person, ever worked as a “real” engineer. At best they had some classical training in the classroom, but we all know that looks nothing like reality. Nobody in this debate had anything more than stereotypes to work with. The difference between the engineering in our heads and in reality has been noticed by others before, most visibly by Glenn Vanderburg. He read books on engineering to figure out the difference. But I wanted to go further.

If I wanted to know how software development compares and contrasts with “real” engineering, then I’d have to talk to “real” engineers. But thinking about that, I realized another problem: while “real” engineers could tell me what they did, they couldn’t tell me how their work differed from mine. Just as we make sweeping statements about real engineering without any evidence, they could make sweeping statements about software without knowing anything about it either. Even if they use a bit of software in the day-to-day work, that’s not the same as developing software as a job. Only a person who’s done both software development and “real” engineering can truthfully speak to the differences between them.

So that’s what I set out to find: people who used to be professional engineers and then became professional software developers. I call these people crossovers, hybrids between the two worlds. I interviewed 17 crossovers on common software misconceptions, how the two worlds relate to each other, whether we can truthfully call what we do engineering, and what the different fields can teach and learn from each other.

There’s a lot I want to talk about here, much more than can comfortably fit in one blog post. I divided the write up into three parts, dealing with my three core topics. Part one is about the term “engineering”. Is what we do engineering, and can we honestly call ourselves engineers?

Why is this a Question?

So first some ground rules: “software engineering” is a real field. Most people would agree that the software running on spacecraft counts as “real engineering”. The debate is about the rest of the field. Are the people who build websites engineers? What about embedded hardware? I found in my research that people defending the title “engineer” rarely coherently define what an engineer is, usually boiling it down to “engineers solve problems”. The arguments against the title tend to be a little more developed, as are the arguments about transcending it.

The backlash against “software engineering” comes from two places. First, there are the gatekeepers. While at first we called ourselves programmers and “developers”, after 1985 more and more people started using the title “software engineer”. This “title bloat” is one of the things that led to the backlash: people using it as a title without trying to live up to some form of standards.

The other side of the backlash comes from the new software movements of the 90s and early 00s. Movements like Extreme Programming and Agile were rejections of applying standard project management techniques to software development. To them, engineering represented the old way of doing things, incompatible with the new way of software development. Software engineering was the past, a failed dead end, and artisanal software was the future.

Both of these backlashes come from the connotations of engineering. In the case of gatekeepers, it’s the positive connotations they believe we have not earned. In the case of artisanal craftship, it’s the negative baggage they believe we don’t deserve. Both of these are agenda-driven viewpoints, arguments based on how they want software to be. This is good for advocacy but doesn’t help us figure out where software is right now. And their agendas are based on a reference point they don’t understand themselves. None of the people arguing for or against software engineering as engineering have worked as engineers.

Common Misconceptions

Let’s start with common arguments that people make about whether to include software in engineering. Most people have an informal concept of what engineering is but not a strict definition. Like pornography, people know what engineering is when they see it. When I asked people to explicitly list the qualities of what makes something engineering, here are the most common responses:

1. Making something complex as part of a team.
2. Making something physical.
3. Making something according to a rigorous spec with strict principles.
4. Using mathematical principles in their design.
5. Working on situations with high consequences, like loss of life.
6. Doing work with a certification and license.

The first quality is too broad: almost all human professions involve making something complex in a team. It doesn’t capture “engineering”. The second quality is too restrictive: not all forms of engineering result in physical processes. In particular, industrial engineering rarely does. The third quality will be discussed in the next essay.

That leaves three claims to discuss here. All three are used to say software can’t be engineering. “We don’t use any math, unlike real engineers. Software doesn’t matter, unlike real engineers. We aren’t licensed, unlike real engineers.” As we’ll see, none of these quite hold up.

Engineering is mathematical

This one I can address directly. The claim is that engineering involves a lot of hard math, while software involves very little math. The confusion here comes from our misunderstanding of mathematics. Much of the math that mechanical engineers use is continuous math. This is where we work over a continuous domain, like real numbers. Things like calculus, trigonometry, and differential equations are in this category. This is what most people in the US learn in high school, codifying it as what they think of as “math”.

In software, we don’t use these things, leading to the conception that we don’t use math. But we actually use discrete math, where we deal exclusively with non-continuous numbers. This includes things like graph theory, logic, and combinatorics. You might not realize that you are using these, but you do. They’re just so internalized in software that we don’t see them as math! In fact most of computer science is viewable as a branch of mathematics. Every time you simplify a conditional or work through the performance complexity of an algorithm, you are using math. Just because there are no integrals doesn’t mean we are mathless.

This falls in line with the rest of engineering. Different branches use different kinds of math in different ways. Industrial engineers are concerned with very different things than mechanical engineers are. Just because we use a different branch of math doesn’t mean we’re not doing engineering.

Engineering is high-consequence

I’m not sure what first disabused me of this notion. It might have been the conversation with Mike, an ex-mechanical engineer who now designs glaciology sensors. One of his last projects before the crossover was as part of a team working on a classified medical device. Most of it went smoothly, but then came the big blocker:

The one thing that gave us the most grief was the handle, which was a bent bit of wire with some plastic molded around it. Yeah, getting the wires to bend correctly and the plastic onto this shiny piece of anodized aluminium wire. […] That was the thing that took three months in the project.

Three months of the project dedicated to getting the handle working right.

This is not to say the work wasn’t important. It does mean that the engineers spent a lot of time on something that’s low-consequence or non-safety critical. In retrospect, this shouldn’t have surprised me. The world is vast and our industry is huge. Someone needs to design the bridges, yes, but someone also needs to design all of the little things we use in our day-to-day life. Much of the engineering there is low-stakes, low-consequence, just like much software is.

There’s a huge difference between if you’re making a cabinet for Bluetooth speakers, like ones that are on my desk, or you’re making an assembly for the landing gear of 737. You’re using some of the same tools, but your approach is wildly different. - Nathan (mechanical)

«But there are still some things that lead to loss of life!» Yes, and the same is true of software. An integer overflow in the Intrado software lead to a several hour 911 outage for millions of people. A biased algorithm unfairly sent people to jail. Just like with “real” engineering, there’s also a wide swath of software that won’t kill people but will lead to millions of dollars in damages. This can be everything from Amazon going down to a game being unplayable.

Engineers are licensed

This is the claim I’ve heard most: licensure. There’s a difference between doing engineering and being an engineer, just as people practice medicine at home without being doctors. Maybe software engineering is possible, but without the licenses we are not software engineers. In Canada you can’t even call yourself a “Software Engineer” unless you’re accredited!

At least that’s what people in the United States say. Several Europeans I spoke to said the exact same thing about US system. Everybody seems to think the worst about their own system and the best about others. And about half of the engineers I spoke to weren’t licensed but were still considered professional engineers by their peers. They didn’t have the “certification”, but they had the skills.

In the US, you don’t need a license to practice any kind of engineering. You need a license to be a “principal engineer”, a.k.a. the person who formally signs off on plans as valid. But the engineers working under the principal engineer don’t need to be accredited and often are not. In fact many of them don’t even have formal training as engineers.

Here’s the problem with deciding engineering based on licenses: licenses are a political and social construct, not a fact of nature. Societies adopt licensure for reasons that are as much political as technical. To see this, let’s discuss some of the history of licensure in the United States.

As recently as 1906, not a single state in the US required licenses for any project. Wyoming was the first state to instate a licensure policy, entirely because irrigation projects kept blowing the budget. They theorized accredited engineers could give better estimates on project costs. It took 40 more years for all fifty states to get on board.

This all means that licensure isn’t part of the federal government: it’s a requirement by the states. Among other things, licenses aren’t necessarily transferable between states. If you get a license in Texas, you have to go through a process to be able to practice in California, in what’s called “comity”.

While licensure originated as part of cost overruns, it expanded so rapidly for an entirely different reason. Regulations are written in blood. Fields become regulated when the lack of regulation kills people. Until that happens on a wide scale with arbitrary programs, it’s unlikely that we’ll ever see the same licensure requirements for software. In the places where software has killed people, like Therac-25 and aircraft accidents, we see stricter regulations. Whether this leads to broader licensure requirements for software engineers remains to be seen.

You could argue that it’s immoral for us to not be licensed. This is an argument I’m sympathetic to. But it’s a normative argument, not a positive one. By saying “we should be licensed”, you are saying something about how the world ought to be. You are trying to answer the question “should we be held to higher standards?” But that’s not the question here. I don’t care right now where we should be going; I just want to know where we are right now. Whether or not we are engineers is irrelevant to whether or not we are good engineers.

In conclusion: licenses exist because we are part of society and have legal requirements, not because they are essential to what it means to do engineering. So while you might want to make software more licensed, as it stands the licensing question doesn’t change the essence of our work.

The Truth

This leaves us back where we started: there aren’t any qualities we can point to and say “this is engineering, this is not”. It’s a standard Wittgenstein game problem: human constructs do not neatly fall into precise definitions. Rather something like “engineering” is a family of related concepts, where we judge whether something belongs based on how much it resembles other things in the family. In other words, “engineering” is “what engineers do”. In other other words, something becomes engineering if enough engineers say it’s engineering.

Consider chemical engineering. Chemical engineering is unlike mechanical, civil, or electrical engineering. Chemical engineers create processes to manufacture products at scale, often using experimentation and iteration. But nobody would disagree that it is engineering. Chemical engineering started in the late 1800s, before states licensed engineers. If chemical engineering started now, people would refuse to call it engineering. And they’d be wrong to refuse.

Once I realized this, my interviewing process changed a little. Instead of asking how they felt about certain engineering topics, I just asked them point blank. “Do you consider software engineering actually engineering?”

Of the 17 crossovers I talked to, 15 said yes.¹

That’s not the answer I expected going in. I assumed we weren’t engineers, that we’re actually very far from being engineers. But then again, I was never a “real” engineer. I don’t know what it’s like to be a “real” engineer, and so can’t compare software engineering to other forms. I don’t have the experience. These people did, and they considered software engineering real engineering.

Even the product owners and project managers are in a sense engineers […] everyone is kind of an engineer, an engineer of sorts. -Kate (chemical)

It makes no sense to use “real” engineering in contrast to “software” engineering. Going forward I will instead use the term “trad” engineering.

Craft vs Engineering

I’m gonna respond in a slightly different way. Not every software developer is a software engineer, just like not every single person who works in construction is an engineer. An engineer is a very specific set of skills. […] Software [engineering] is skill plus understanding, all the processes and life cycle and the consequences and all the things that you should be aware of [and] avoid. -Dawn (Mechanical & Chemical)

That said, many of the crossovers also added an additional qualification: software engineering is real engineering, but a lot of people who write software aren’t doing software engineering. This is not a problem with them, rather a problem with our field: we don’t have a rich enough vocabulary to talk about what these developers do. Not everybody who works with electricity is going to be an electrical engineer; many will be electricians. And this is okay. Electrical engineering is a very narrow skill set in the broader field of electric professions and plenty of people have other important skills in that space. But we use things like “programmer”, “software engineer”, and “software developer” interchangeably. What is the difference between a software engineer and a software developer?²

Some people propose the word “software craftsman”. This term comes from the book Software Craftsmanship: The New Imperative, by Pete McBreen. In it he argues that software is not a kind of engineering, being much more free-form creative and flexible. We’re not line workers but artisans, artists, people who take pride in the craft we do and the flexibility of our states. As he puts it:

Software development is all about the unknown. The production process for software is trivially easy—just copy a disk or CD. The software engineering metaphor fails because we understand production, a mechanical task, much better than we understand design, an intellectual task. - Software Craftsmanship

Many people have asked me why I care so much about this project. Why does it matter whether or not software is “really” engineering? Why can’t we just say that “software is software”? It’s because of these misconceptions. People have a stereotyped notion of what engineering looks like. Because software doesn’t look like the stereotype, they assume that we are wholly unlike engineering. The engineering disciplines have nothing to teach us. We are breaking pristine ground and have no broader history to guide us.

In contrast, if we are doing engineering, then we have a landmark. We can meaningfully compare and contrast the work we do as software engineers from the work that others do as traditional engineers. We can adapt their insights and watch for their pitfalls. We can draw upon the extant knowledge of our society to make better software.

I believe everything McBreen said about software is fairly reasonable, about how hard it is to predict things and how it’s intensely personally created. What he gets wrong is the assumption that engineering is not this way. Engineering is much richer, more creative, and more artistic than he thought. But of course he would have an imperfect view: he is, after all, not a traditional engineer.

---

Here’s my final take. This is the belief I’ve settled on in synthesizing all the interviews I did and does not necessarily reflect how the crossovers think. I went into this thinking that software wasn’t really engineering. Maybe there were a few people who could count themselves as that but most of us were far below that threshold. I still believe that most of us are not engineers, because we’re working in domains that people don’t see as engineering. Most people don’t consider a website “engineered”. However, and this is a big however, there’s a much smaller gap between “software development” and “software engineering” than there is between “electrician” and “electrical engineer”, or between “trade” and “engineering” in all other fields. Most people can go between “software craft” and “software engineering” without significant retraining. We are separated from engineering by circumstance, not by essence, and we can choose to bridge that gap at will.

In the next essay, we’ll talk about the similarities and differences between traditional engineering and software engineering, and how they’re actually not all that different in the end.

Part two, We Are Not Special, is available here. If you enjoyed these essays, I have a weekly newsletter and am on Twitter.

Thanks to Glenn Vanderburg, Chelsea Troy, Will Craft, and Dan Luu for feedback, and to all of the engineers whom I interviewed.

Appendix: Methodology

I only searched out people who work professionally for at least one year in each field. In practice, work experience in trad engineering ranged from a year and a half on the low end to over 15 years on the high end. Interviews ranged from half an hour to two hours, leading to a total of 24 hours of recorded interview. Two of the interviewees were not recorded, but I took notes. The breakdown of the specialties is:

• Civil engineers work on buildings. One person I talked to was a classic civil engineer, one specialized in designing mines, and two worked on oil rigs.
• Mechanical engineers design physical machines.
• Electrical engineers design circuits and electronics. Two worked on chipsets, while one was embedded in a submarine.
• Chemical engineers create processes to make chemicals at scale. They are generally not creating entirely new chemicals or chemical products, but work on how to produce extant ones. “Chemicals” is a broad category here, covering everything from clean water to toothpaste.
• Industrial Engineers create holistic systems and procedures in an industry. One designed data center layouts and the other worked on integrating disparate air traffic systems.

These are very broad summaries of the specialties. Most engineers work in a subdomain of a specialty, such as automotive engineering or circuit design. There were also some fields of engineering I didn’t cover in my interviews. This includes aerospace and nuclear engineering. I suspect (suspect) that aerospace would be roughly similar to mechanical engineering and nuclear engineering would be roughly similar to civil engineering. But it remains a threat to validity nonetheless. ³

The other two threats to validity are geographic location and crossover type. The majority of the interviewees were either in the US or the UK, with the rest being from the EU or Canada. I did not get a chance to interview anybody from Latin America, South America, Africa, or Asia. Everybody interviewed crossed from traditional engineering to software engineering. I was not able to find anyone who crossed the other way.