Explaining tech's notion of talent scarcity
I was recently asked to explain what it means when people say that, for example, “There are only [10-200] people in the world who can do what [highly-paid AI researcher] does.” Why can’t more people be trained to do these jobs?
The notion that only some engineers or researchers in the world can do certain types of work – i.e., nobody can learn how to be Linus Torvalds or Andrej Karpathy at a coding bootcamp – feels very intuitive to me, but apparently this is not necessarily intuitive, or even valued, among other industries. That made me wonder how much this implicit belief drives tech culture.
Why does software have this phenomenon, while other industries don’t? Are companies that hire for exceptional talent organized differently from those that don’t? And what do we mean by “exceptional talent,” anyway?
I ended up with a framework for talking about different types of talent distribution (normal, Pareto, and bimodal) and how they influence corporate cultures, which helped me answer these questions. Enjoy!
P.S. In other news, I’m spending time this summer at Summer of Protocols, a program funded by the Ethereum Foundation (and run by the one and only Venkatesh Rao) to explore deeper research questions around the sociological theory of protocols. I’ll be looking at the spread, transmission, and defection (?!) of social protocols.
In addition to the topic itself, I’m excited to participate in a para-academic experiment in bringing a bunch of independent researchers together around the same topic. I’m planning to write about the experience from a field-building perspective, and particularly how it compares to similar efforts to catalyze research fields in academia. Stay tuned! And in the meantime, send me all your wild musings and unanswered questions about protocols.
Explaining tech's notion of talent scarcity
TLDR: Most conversations about “top talent” assume Pareto distribution; however, a closer examination suggests that different corporate cultures benefit from different types of talent distribution (normal, Pareto, and a third option – bimodal) according to the problem they’re trying to solve. Bimodal talent distribution is rare but more frequently observed in creative industries, including some types of software companies.
While Pareto companies compete for A-players (“high-IQ generalists”), bimodal companies compete for linchpins (those who are uniquely gifted at a task that few others can do). These differences account for variations in management style and corporate cultures.
It was a group of consultants at McKinsey & Company who coined the “war for talent” in their 1998 report and subsequent book of the same name, propelling the term “top talent” into the corporate executive hive-mind for the next two decades. While McKinsey refrained from offering a precise definition of talent, they thought that a shortage of “smart, energetic, ambitious individuals” was coming, and that it would lead companies to fight to attract and retain the very best.
In software, there is a related but distinct notion of the “10x developer,” which dates at least as far back as a 1968 study that accidentally uncovered individual differences in programmer performance, and was further popularized by Fred Brooks’ 1975 book, The Mythical Man-Month. The definition of a 10x developer is similarly vague, and its existence is frequently contested. Depending on who you ask, a 10x developer might be someone who can write code 10x faster; is 10x better at understanding product needs; makes their team 10x more effective; or is 10x as good at finding and resolving issues in their code.
Despite the similarity between these two concepts, McKinsey’s notion of top talent and software’s 10x developer reveal subtle cultural differences. Both are concerned with identifying the best people to work with, but the McKinsey version defines the best as the top percentile in their field, whereas the 10x developer is often a singular, talented individual whose magic is difficult to explain or replicate.
For example, in conversations about hiring AI researchers, many people have said something to the effect of “There are only [10-200] people in the world who can do what [highly-paid AI researcher] does.” This is a very different statement from, say, “We are trying to hire top AI researchers.” In the latter case, “top” means the highest-performing slice of all AI researchers, but in the former, the assumption is that there are only a handful of people who can perform the job at all. While this idea is intuitive among software engineers, it is rarely seen in other industries.
Why can’t more people be trained to do certain tasks in software? Why aren’t there more Linus Torvaldses or John Carmacks? Will there only be 100 people, ever, who can do what some AI researchers do?
After exploring these questions, I identified three distinct models of talent distribution, which correlate strongly to industry, but vary even within industries, depending on what the company does and how mature it is:
Normal distribution: Talent follows a normal distribution. Companies succeed not by attracting and retaining “top talent,” but by the strength of their processes, to which all employees are expected to conform. Frequently seen among manufacturing, construction, and logistics companies.
Pareto distribution: Talent follows a Pareto distribution, skewed towards the top nth percentile. Companies benefit from attracting, retaining, and cultivating “A-players,” who are expected to demonstrate exceptional individual performance. Frequently seen among knowledge work and sales-centric companies.
Bimodal distribution: Talent follows a bimodal distribution, where companies benefit from identifying, hiring, and retaining “linchpins,” who make up a fraction of headcount, but drive most of the company’s success. Frequently seen in creative industries (ex. entertainment, fashion, design), as well as software companies solving difficult technical problems (ex. infrastructure).
A company’s distribution type also shapes their organizational culture, which lives downstream of the types of talent they are most incentivized to seek out and hire. Most notably, we can understand the difference between what I’ll call McKinsey and Silicon Valley mindsets by understanding differences in their respective definitions of “top talent.”