Are 10% of your software engineers lazy?

Nearly 10% of all software engineers are “ghosts” who “do virtually nothing” but collect $300K salaries. This, according to research from Stanford University. The paper takes particular aim at engineers who work remotely, declaring that 14% of them apparently spend more time gardening than tending to their Git repositories. Controversial? Yes. Accurate? Almost certainly not—at least, not based on this particular analysis of more than 50,000 engineers. The analysis makes for great sound bites but poor understanding of how software development works.

Do you believe in ghosts?

The point of the research is apparently to suggest that cutting these 9.5% “ghosts” would save $90 billion, as Stanford researcher Yegor Denisov-Blanch notes. It is, of course, very possible (even probable) that 10% of the developers within any company are low performers. Any job category will have a tier of relatively low-performing people. But it’s not at all reasonable to use code commits, as the researchers do, to paint performers as good or bad.

After all, writing code is not a software developer’s most important role—not for senior developers, anyway. Honeycomb CTO Charity Majors argues, “Being a senior engineer is not primarily a function of your ability to write code.” Instead, she continues, “It has far more to do with your ability to understand, maintain, explain, and manage a large body of software in production over time, as well as the ability to translate business needs into technical implementation.” As the Stack Overflow team puts it, the “hardest part of building software is not coding, [it’s figuring out] requirements.” The best engineers will figure out what to build and reduce that to as little code as possible.

In other words, Denisov-Blanch’s contention that less code is a strong indicator of poor performance might signal the opposite. At the least, it doesn’t confirm his and the other researchers’ finger-pointing at low levels of Git commits as dispositive proof of developers “ghosting” their employers. Nor does it confirm his “don’t-quote-me-on-this” argument that the research also shows that “the top 25% of engineers contributed about 50% to 60% of the output,” though that finding may be more intuitively correct, given the 80/20 rule.)

Less code may mean more productivity

Counting code commits, while an understandable approach, is flawed. Yes, the approach is a bit more sophisticated than that, but not as much as the researchers seem to think. For example, Nvidia Senior Engineering Manager Aaron Erickson points out that the researchers might find “another 10% of engineers who do add code, but it’s useless abstractions or vanity rework that adds negative value and confusion.” Stanford’s research would say that these are valuable engineers, but in reality, they might be doing more harm than good. Their employers would be better off if they decided to ghost instead of committing worse-than-useless code. The research doesn’t account for bad contributions, by Denisov-Blanch’s admission. They just expect bad commits are resolved during review.

All of this is a long way of saying the research may not say what the researchers believe. This wouldn’t be a big deal except that the headline is clearly meant to drive employers to revisit how they measure engineering productivity. (Denisov-Blanch says he did the research because he believes “software engineering could benefit from transparency, accountability, and meritocracy and [he] is trying to find a solution.”) That’s a great goal, but what about all the CEOs who may see the headline and demand that their ghost engineers be fired? Using code commits as the only metric could end up removing some of a company’s top engineers, not necessarily their worst ones.