What We Know We Don't Know: Empirical Software Engineering
Empirical Software Engineering is the study of what actually works in programming. Instead of trusting our instincts we collect data, run studies, and peer-review our results. This talk is all about how we empirically find the facts in software and some of the challenges we face, with a particular focus on software defects and productivity.
Talk doesn’t seem to be online yet; in the meantime, you can see a recording of an older version of the talk here. Slides are here.
Sources
I referenced a bunch of papers in my talk. These are links so you can read them yourself:
Intro
- Big Data is slower than laptops
Why we care
- Section references
Methods
- Controlled Trials
Comparing syntax highlightings and their effects on code comprehension
- Natural Experiments
- Observational Studies
-
Errata: In the talk I said it studied “cats on keyboards”, but the paper was actually on cats on tablets (digital projections).
Defects
- Programming Language Effects
A Large Scale Study of Programming Languages and Code Quality in Github (original faulty study)
On the Impact of Programming Languages on Code Quality (replication study)
- Code Review
Best Kept Secrets of Peer Code Review
An Empirical Study of the Impact of Modern Code Review Practices on Software Quality
First Come First Served: The Impact of File Position on Code Review
- Human Factors
-
Impact of a Night of Sleep Deprivation on Novice Developers Performance
Sleep deprivation: Impact on cognitive performance
Additional Sources
- Recommended Reading
- Free research