This is my writeup of all the talks I saw at Strangeloop, written on the train ride back, while the talks were still fresh in my mind. Now that all the talks are online I can share it!
Next year’s the last Strange Loop. You can buy tickets here.
Topic: What we empirically know about how experts write code.
Ultimately found a lot of it pretty conventional. It was the usual stuff like “experts think about the problem, they don’t just rush into a solution” and “experts try to figure out how the bug conflicts with their mental model of the code.” I’m guessing there are a lot of people for whom this was important and insightful, but I’m deep enough in the expertise whole that I already knew most of it. I do like how it emphasized qualitative research over quantitative research.
Never Work in Theory Miniconf
This was a special “mini conference” on empirical research in software engineering, run by the It Never Work In Theory people. As you might know, this is a topic really interesting to me, so I spent a session on it. It was two sessions long and in a lightning talk format, being eight talks total. First half was on how bugs enter code, second half was on how to fix bugs in existing code.
I only stayed for the first half. Covered topics:
- How “learning culture” actually manifests in organizations. Often, it doesn’t: people say “it’s okay to make mistakes” and then rip each other to shreds during code reviews.
- Refactor commits take longer to approve than feature commits. It helps to list 1) the point of the refactor, 2) what changes you made, and 3) why this affects the goals.
- Looking for code smells in the training set for AI-based code generators, and then triggering similar code smells in generated code, like unsanitized inputs.
- How often automatic bug-finding tools find actionable bugs. Only one third of raised bugs actually mattered. They got much better results by instead looking for common patterns in API usage and flagging code that didn’t follow those patterns. See https://sarahnadi.org/smr/api-misuse/
Unsurprisingly, I found it all pretty interesting. I’m giving a talk on ESE in November and a lot of this is good stuff to consider adding. I liked how a lot of it was humble and tightly scoped— we’ve seen the problems of grandiose research projects crashing and burning and it’s important to for the field to start small. Utimately I’m pessimistic about the state of research, but I’m also not too cynical as to believe SE is impossible to study.
Slides were too small, couldn’t read
(Disclosure, I didn’t actually see this talk live, I watched Felienne Hermans rehearse and gave feedback. Also, she’s a friend of mine.)
Topic: a “gradual programming language designed for teaching.” Based on the speaker’s experience teaching children who were fed up with Scratch but found that syntax had too much cognitive overhead. Hedy consists of progressive syntactic levels, where earlier versions are more limited but easier to understand. By the end, the student is working with full Python.
I’m a little biased, but Felienne is a fantastic speaker and delivered her passion project well. I was constantly struck by the challenges involved with teaching children Python and the challenges involved in teaching programmers TLA+ (as well as our respective solutions). I can see the value of a gradual programming language.
The most interesting takeaway: the difference between how scratch is designed for learning while Hedy is designed for teaching. Scratch is very different from other programming languages, so it’s hard for a teacher to build a mental model! Hedy’s gradual progression also keeps learners in a “tight band” of ability: you don’t have a few students racing a million miles ahead and writing Tiktok killers while other students are still on the basics. It’s a good look into the constraints of teaching classrooms.
Topic: Paper is really good for organizing thoughts, but it’s static: it doesn’t provide information back to us. Computers are good for that, but it’s hard to think in the same way with manipulating formal languages. Speaker wants to create “digital ink”, where you can draw things and compute directly from the drawings. It’s easier to explain if you see their demos, so:
Ultimately, I have mixed feelings on this. I do very strongly believe in the “mental augmentation” value of computers, and the demos were very cool. But I’ve used a lot of different “digital ink” programs and they all have the same problem: handwriting recognition. My handwriting is garbage, and even when I make an effort to be clear, the tools have no idea what I’m writing. That’s just for converting handwriting to text: extracting semantics will be a lot harder. Someone I was chatting with afterwards quipped “A great demo is easier than a good product.”
The one good “digital ink” I’ve used: browser mouse gestures. Those worked because the gestures mapped my scrawlings into changes in cardinal directions, eg both a box and a lowercase
b would be RDLU. This was less expressive but much more tractable. Maybe making general purpose “digital ink” is too ambiguous, and we should focus on the easier things first.
(I also wonder if “digital ink” might be the wrong mental model. “Improving pen&paper with computers” is working against the natural computer I/O instead of with it. Consider how much faster you can write with the “200 buttons” vim model vs what you can do by hand. What would 200 buttons I/O give you for mental modeling?)
The Early Days of id Software (John Romero)
Topic: A history of id from the creator of Doom and Quake. How he got started writing games, how they founded id, the processes and principles they had while making some of the most important games in history.
My main takeaway was “if you have a really good team of aligned people, you can do anything.” Romero talked about how they’d work full days at their real jobs, and then come together and work full days writing games. And, at least in the beginning, all four the id people were perfectly in sync. Same age, same life outlooks, they all got along, none of them needed money, etc. A lot of the problems that come from large companies are rooted in 1) the difficulties of communicating, and 2) conflicting interests.
IIRC eventually Romero and Carmack had a falling out and went their separate ways, and that kind of killed the magic. Carmack went on to turn id into more of a game engine company, and Romero went on to make Daikatana. But this is all being shared thirdhand! I don’t know much about the actual history, and I don’t think Romero was giving us the full picture. Still a very entertaining talk, though.
(Disclosure, also a friend with Jean Yang.
Topic: The constraints in real-world development teams that make observability hard to adopt, like not knowing all the endpoints of the system or having employee turnover.
I was expecting a lot more about how Akita (Jean’s company) solved these problems, but only 3 minutes or so of the 33 minute talk were about what they did. The rest was about the problems themselves and the state of observability today. Could be just wanting to not seem like a sales pitch. She mentioned eBPF-based traffic listening and building an API spec from existing traffic. I was left wanting more about the details.
Topic: modern languages are stuck in the past, based on backwards compatibility with legacy contexts. We should stop working with these constraints and start making languages that match modern capabilities.
The summary: the speaker was extremely good at presenting but also incredibly arrogant and his presentation was beautiful and masterfully paced but filled with errors, half-truths, and oversimplifications. That’s all I’ll say here. Shoutout to Marianne Bellotti who watched it with me and let me rant for fifteen minutes after.
Topic: Testing distributed systems is extremely difficult. Formal methods is more thorough, but programmers have trouble using it. P helps bridge that gap by being an FM that looks like a programming language. Developers model their system as message-passing state machines, which can then be model checked.
Probably the most useful talk for me. I’ve had my eye on P for a while, but never went deep. What surprised me most is that the specifications are also state machines. Unlike most FMs, where the invariant can be a mathematical expression, you have to write an implementation of the invariant. I feel like as an experienced specifier I’d find that extremely limiting, but if it makes things more approachable for programmers that might be a worthwhile tradeoff. The talk certainly inspired me to play more with it.
After the talk I asked him why he named it P. He said it was because it was one of the only one-letter languages remaining. He now regrets the name.
Topic: There are lots of ways to subvert a piece of software via a dependency attack. What can we do as developers to make our software more trustworthy to use as a dependency? Goes through several levels of SLSA clearance, including signing commits, using isolated build environments, and providing a Software Bill of Materials (“sbom”).
I’ll admit, I knew nothing about Kelsey before seeing this talk. Was he kubernetes or something? After this it was pretty obvious how he got so big. Fantastic speaker, and he did everything live. The most interesting idea to me was how Go encodes build metadata into the binary. IE if someone gives you a go binary, you can extract when it was built and what dependencies it uses.
Another important takeaway: Kelsey said there will come a time when regulations will prevent people from working on certain OS projects due to their nationality. And it’s coming sooner than we think.
And that’s all the talks for Strangeloop 2022! Hope to see you for the 2023 finale!