What We Know We Don't Know: Empirical Software Engineering
This version of the talk was given at DDD Europe, 2024.
Technology is a multitrillion dollar industry, but we know almost nothing about how it’s best practiced. Empirical Software Engineering, or ESE, is the study of what works in software and why. Instead of trusting our instincts we collect data, run studies, and peer-review our results. This talk is all about how we empirically find the facts in software and some of the challenges we face, concluding with a guide on how to find existing research and an overview on some things we’ve learned about DDD.
Slides are here.
Sources
I referenced a bunch of papers in my talk. These are links so you can read them yourself:
Intro
- Big Data is slower than laptops
Why we care
- Section references
Methods
- Controlled Trials
Comparing syntax highlightings and their effects on code comprehension
- Natural Experiments
- Natural Experiments Gone Rogue
A Large Scale Study of Programming Languages and Code Quality in Github (original faulty study)
On the Impact of Programming Languages on Code Quality (replication study)
- Observational Studies
Fixing Faults in C and Java Source Code: Abbreviated vs. Full-Word Identifier Names (preprint)
DDD
- Survey Paper
- Interesting Papers
Design, Monitoring, and Testing of Microservices Systems: The Practitioners’ Perspective
Refactoring with domain-driven design in an industrial context
Note that there are other interesting papers in the survey paper, these are just the ones I brought up in the talk.
Additional Sources
- Recommended Reading
- Free research