Is AI like Robin Hood, robbing the knowledgeable to make the less-skilled smarter? Or is AI like compound interest, where the smarter you are the smart AI will make you?
Somewhat depressingly, this recent MIT study argues that AI is not an equalizer.
Top performers who used AI doubled their output. The bottom third saw little change. The difference? More skilled performers were better at evaluating AI outputs.
I want to believe that AI is an equalizer. It surely is in some domains. But my experience with AI programming tools so far shows a similar "rich get richer" dynamic.
My most skillful friends are doing 3-5 days of work in 1 day. But several have told me privately that AI is destroying collaboration on their teams.
Less skilled engineers are blasting buggy, incoherent-but-correct-looking code at warp speed.
Tools like Cursor (or, my favorite, Claude 3.5 Fast Edit in Zed!) offer insane productivity gains.
But they can also be weapons of mass destruction in a codebase.
A few clicks inserts hundreds of lines of good-looking but wrong code. It's like the stereotype of the bad Management Consultant that says obviously false things at 110% confidence.
Without good taste, nonsense looks correct. So what do about taste?
Taste is hard to train. It's hard to explain. It's classic "tacit knowledge" that seems to arise from repeated exposure to many patterns. (Ironically, not unlike how AI models are trained.)
Let's assume AGI doesn't replace software engineering entirely. How can teams use AI tools, support junior engineers, and ship great code? I can think of a few implications:
PR reviews will become even more important but also more laborious because problems will hide better. (At least until AI can do the reviews, too.)
Testing will grow in importance and net test coverage will increase. It's easier than ever to generate tests. And they're insurance against good-looking-but-wrong AI code. (Especially end-to-end and integration testing, where broad system context determines correctness.)
Mentorship will matter more for juniors because architecture, style, and hard-to-explain tacit "taste" for code will be a bigger constraint than the quantity of code an engineer can generate. (Everyone can generate infinite code).
Startups will hire smaller, more senior teams. In theory, this would maximize productivity gains from AI while minimizing the "collaboration cost" of using these tools. Teams will stay smaller for longer.
On the bright side, there's never been a better time for an engineer to bootstrap or build a side project. AI handles so much drudgery!