Knowing what great looks like

A lot of zeitgeist management wisdom on hiring, if you look at it hard enough, is really about avoiding bad hires. And for good reason. Hiring the wrong person destroys value, infects culture, and is an expensive, generally unpleasant experience for everyone involved. But the ultimate objective of hiring, to put it in the words of Apple and Netflix, is to hire people who are both insanely great and stunning colleagues. [1] [2] But avoiding crappy hires and making great ones involve different skills, strategies, and tactics. And you need to do both.

What you can actually see

To hire great people, you have to be able to find them. And to find them, you need to know what you’re looking for. But can humans effectively judge another person’s competency or capabilities if they don’t have first-hand experience doing the things they’re hiring someone else to do?

Surprisingly, yes. But you first have to recognize where you’re an expert and where you aren’t. As Stephen Hawking said, “The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge.” Except he didn’t say that. The real quote is, “The greatest obstacle to discovery is not ignorance—it is the illusion of knowledge.” And it’s from historian Daniel Boorstin. [3] A quote about the illusion of knowledge being itself a beacon of mutation and misattribution is more than a bit on the nose, right? Delightful stuff, really. Thankfully, we have Kahneman & Klein, “True experts, it is said, know when they don’t know.” [4]

To figure out what we know, what we don’t know, to create a map for deciding what we need to know more of, and decide when enough is enough, we have to understand two different types of expertise. There’s our ability to Do the Thing, known as contributory expertise, which I call “operating expertise.” Then there’s our ability to Know Stuff About the Thing and Talk About the Thing, known as interactional expertise (because you acquire it by interacting with other people), which I call “observing expertise.” [5] Each of these types of expertise can be developed on a scale from novice (1) to competent (3) to expert (5). [6] And your ability to hire someone to do something you’ve yet to master depends on the relationship between these two types of expertise.

Developing operating expertise takes years of time, time a founder doesn’t have. Observing expertise is what a film critic, food journalist, or biographer develops. A CEO of a software company may be at a one or a two in operating expertise with accounting, but to be effective, they’ll need to be at least a three in observing expertise. And while the evidence can’t recommend a firm target for each, my recommendation is that for any role you’re hiring, shoot for a 2 in doing and at least a 3 in talking.

Thankfully, research shows through conversation alone, people can develop the ability to judge quality, spot fakes, and effectively weigh arguments. Developing observing expertise is worth it and the payoff curve is steep, then flat. As you’d expect, in experiments undergraduates given three-paragraph briefings (executive summary, anyone?) forecasting world events had judgment worse than chance. [7] But well-read outsiders who’d developed fluency, which we don’t have a formal definition of, but let’s estimate it at a three or “competent,” judge as well as career specialists. [7]

To develop from novice to competent doesn’t require spending a semester at Harvard, either. My recommendation: run two to three Calibration Calls with people actively succeeding at the job, then another two to three with people who have hired for or worked with the truly great, so you get a few angles on greatness. The research says the gains from additional advisors flatten fast [8], beyond six calls you’re hitting diminishing returns.

The primary way we evaluate candidates in a hiring process is through conversation. Given that, interviews primarily collect evidence for observing expertise. But the job you’re hiring for isn’t a conversation; it requires operating expertise. And that’s why interviews are imperfect. Yes, people who can Do the Thing definitely Know Stuff About the Thing, but that doesn’t always mean they’re great at Talking About the Thing.

Smooth talkers get over-priced and masters often get dismissed for being quiet or—god forbid—weird. People trust more confident advisors even when track records are identical. [9] [10] A sociologist once answered seven emailed gravitational-wave questions; his answers were judged side-by-side with a real physicist’s answers by gravitational-wave physicists. Asked to pick out the real physicist, seven of the nine couldn’t tell, and the other two picked the sociologist. [5] True masters are dismissed for two types of weirdness. The first because many arrived at their field of mastery via non-standard paths. [11] The second because their expertise is so dense that they know which rules to break or bend in order to be more effective. And less competent people judge them for that. [12]

Observing expertise has limits, then. But the thing about limits, once you know them, is that they can be worked around. Unfortunately, Collins and Evans argue we can only reliably judge people below our level in a given domain. [5]

This very limit can cause real problems in two specific scenarios. First, when hiring an executive for a seat on the exec team in a domain the rest of the team doesn’t have operating expertise in, the hiring manager (the CEO) may develop observing expertise, but the rest of the executive team does not—and they still want to be part of the hiring process. Second, in any hiring process, many managers include junior employees in the interview process to be fair, equitable, and inclusive. Inoculate yourself by being deliberate about assigning specific people to evaluate competencies they are adequately skilled in. This may be something as simple as asking a junior to evaluate whether the candidate treated them with respect, explained things clearly, and inspired confidence. But it is not asking a junior to evaluate a VP’s domain-specific competence.

And what about AI? Use it to augment and improve your expertise, but never surrender your thinking to it. [13] Artificial intelligence can act as a terrific always-on teacher. It can help you learn the vocabulary, learn about failure modes, and brainstorm questions for Calibration Calls. But each answer a model gives you will be a compression of the market consensus. [14] No model can give you a localized standard of excellence for your stage, team, company values, and your personal preferences as a leader. Not yet, at least. If you’re not convinced, take yourself out of the equation. Would you rather hire a VP of Engineering who regularly calibrated greatness in engineering with actively working high performing engineers or one who just talked to Claude?

This is why you hire specialist functional leaders, after all, and it’s also why you should not hesitate to call in an external expert you trust to help interview key hires. If your judgment is limited, import better judgment.

Dimensions of greatness

To be great at learning what greatness is in any given field, you have to know what you’re looking for. Greatness itself has a shape. It has dimensions. A novice knows about one or two. The expert knows many of them, and the master has internalized each. I’ve found two main groups: what greatness is and what the best judges of greatness do. Here’s what I’ve found so far.

To learn what greatness looks like requires knowing it has both a level and a slope. Its level is its state at a fixed moment in time. Its slope is how fast it’s changing and in which direction, either improving or decaying quickly or slowly. Too many hiring processes measure level obsessively and don’t measure slope at all. And at a startup, there’s arbitrage in buying slope. [15] My friend Julian calls this “putting people in play,” prospecting for high-slope but low-level young people, and taking them off the metaphorical bench, giving them a shot to show the world what they’re made of.

One of the most desirable aspects of greatness is a person’s ability to independently deliver or captaincy. These people can take something from conception to launch, owning all of the coordination in the middle. Keith Rabois calls these people “barrels,” contrasted to “ammunition,” because each additional barrel you have means you can now run more work in parallel. [16] I think of barrels like ship captains. With one captain, you have a ship. With eight, you have a fleet. Regardless of term, barrels or captains are devastatingly rare.

Counterintuitively, glimmers of greatness can be spotted early. The entire field of college admissions is built on this idea, but its track record of mistaking polish for potential [17] demonstrates how imprecise our tools for consistently spotting greatness early are. Marc Andreessen wrote, in 2007, how desirable traits like self-motivation, curiosity, and integrity can be spotted early. [18] Self-motivated people create stuff. They start lemonade stands, student organizations, businesses, products, and projects. Curious people relate to books, magazines, podcasts, and lectures (knowledge!) as if they were oxygen.

And look, there is evidence that people become more conscientious, driven, and responsible with age. [19] But, there’s also evidence that position relative to peers is moderately stable while young. [20] People change, people grow, but rank reshuffling primarily happens amongst the young. All the more reason to find young talented stars before the market has figured out how to adequately value them.

The best people at this also know that greatness is localized to the specific outcomes your company needs each hire to achieve over the next 12-18 months. [21] Founders and new managers have a habit of generalizing, ironically, a person’s specialization. They see people as functions, design, sales, marketing, finance, possibly adding a second layer of seniority level: junior/senior individual contributor, manager, director, VP, and so on. But a person’s ability to deliver on their assigned objectives depends on whether they have operating expertise in more dimensions, which I call The Competency Stack. Domain specific knowledge is insufficient. Ask yourself, is this person great as compared to peers at their seniority level? Do they share your company’s values? Do they have experience at this size and stage of company (see: The Terrain Test)? Will they complement the other members of the team well? Will they need to invest significant time in developing observing expertise (e.g. a salesperson who sold SaaS but is now selling medical devices has a lot to learn).

Greatness is time-indexed. When it comes to skills, knowledge, and experience, time is a devastatingly effective thief. What great looked like in any given field in 2016 is different from what it is in 2026, and you should expect the standard of greatness to be similarly (or even more) different by 2036. The tools change, the best practices shift, and the best people in the world stay on top of what works. [22] Even if the field stays static, which it doesn’t, people’s skills decay over time. [22] Given that, someone great at learning what great looks like, as an outsider, needs to meet people who are actively succeeding at The Thing today. Not yesterday. Not in 1996. Today.

The hunt for what great looks like in a given domain requires a search for both specific greatness and general greatness. General greatness looks like curiosity, self-motivation, and high agency. [18] They are open-minded, reflective, and they take responsibility instead of deflecting or blaming. [23] They give and take feedback well (most of the time), and they’re good to be around. They can be disagreeable or agreeable and they embrace conflict. They keep up with their field and tools, and they identify with craftsmanship. They work hard, often to the edge of obsession, because they enjoy their work. [24] And they hold an internal locus of control. [25]

But isn’t that just a list of generally admirable human traits? Yes. To learn what greatness looks like in the field means acknowledging that great people have weaknesses. I know, sad. Every person who is great, specifically and generally, does something uncommonly well. [26] When learning what greatness looks like, it is your job to hunt down what specific “spikes” in competence correlate to effectiveness against the set of outcomes the person you’re hiring needs to achieve. [21] Do not expect “spiky people” to be generally great at every aspect of general greatness, but rather expect them to be weak on one, two, or several. More importantly, make sure their specific spike matches your needs.

But we do have to be careful. People great at finding and evaluating greatness separate the person from the context surrounding them. The psychologists Lee Ross and Richard Nisbett wrote a book called “The Person and the Situation.” [27] It’s about how humans make a category of mistake Ross coined in 1977 called “fundamental attribution error,” where we attribute things to a person when their cause has more to do with a situation. Some people, early in their career, hopped on a rocketship. They worked at a small company that became huge, or they worked at Goldman Sachs. And because of their situation, we inaccurately infer the specific person’s contribution. Stars don’t “travel” between companies as well as people think. [28] Apply this to operating expertise, and you have to sort out how much expertise was in the Kool-Aid in the break room where the person worked and how much the individual specifically contributed and gained. Remember: observed performance is always person × system, and that applies to the people you learn from, to your own scorecard, and to candidates. But great people also get good at picking the environments they’ll be great in. A rocketship on a résumé may be luck, but it also may be a sign of great judgment. It’s your job to figure out which.

When exploring what it looks like to be great at something you’re hiring for, it’s easy to get stumped because you don’t know anyone who’s great at what you need. The good news is, people who are great at stuff cluster, preferring to spend time with other people who are great at stuff, even if it’s different stuff. The key is to ask anyone talented at anything, “Who do you know who might know a bunch of people who might be great at X?” Referrals from high performers result in higher profit per worker ($4,190) than those from weak performers ($1,063), and referrals from weak performers are worse than a non-referred candidate who passed a screen ($1,756). [29] A mediocre network may cost you more than no network at all.

If you’re human, at this point, I suspect this is all intimidating, if not despair-inducing. Think of what I’ve laid out here for you as a map, not a shopping list. Your job, now, is to determine where the market has mispriced against the map, where you can find the high-slope young person, the weird-shaped expert, the quiet master. And to differentiate those people from the rocket-ship passengers. The market for talent is a blunt instrument, mistaking cheap noise like levels, logos, and credentials for signal. [17] Your advantage is in proportion to how few people will read the map to find the arbitrage. This is especially relevant when you’re small, at pre-seed or seed. You need to shop for mispriced talent because it may be the only segment you can afford vs. incumbents with Scrooge McDuck’s swimming pools of cash to deploy.

The master of discovering what greatness looks like, then, studies greatness, hunts for it, and does so because they know it is why they will win. They do not have a mythically golden gut. Believe it or not, when it comes to gut, hundreds of hires’ worth of experience may not sharpen it at all. [30] Because for experience to turn into intuition, it needs a feedback mechanism. [4] I’ve yet to meet a manager who’s kept a spreadsheet of every hire they’ve made, the outcomes they defined at hiring, and scored whether or not they hit them. Great pickers, in a world of lossy data, then, become masters through method, not talent.

At Management Craft, we chart each competency on an abridged version of the Dreyfus Model, a system developed in the 1980s to chart how humans develop skills. The original model has five rungs (novice, advanced beginner, competent, proficient, and expert); a sixth, mastery, was added later. For simplicity’s sake, each competency has a simplified rubric with only three levels: novice, competent, and expert. Use the rubric below to self-assess, use it as a team to calibrate each others’ capabilities, or use it as a conversational aid in the role definition process.

Decide to improve. Price the value of doing so.

1. Realize and accept you need to improve. Getting feedback is rarely easy. If you’re reading this, it’s probably because someone suggested that you have room to improve here.

2. Price the consequences of staying the same. Kurt Lewin sketched the foundational model for how people change back in the 1940s, and Edgar Schein later built it into the version managers still use [31]. To dust off those nasty old habits, our motivation to change must outweigh the comfort of staying the same. If you’re spending $150K / year on a hire, you’re expecting a return on that investment. Learning what great looks like increases your odds of getting an above average return on your investment in this new team member. Marc Andreessen’s rule of thumb: if you know what you’re doing, the odds of a given executive hire working out will be about 50/50. If you don’t know what you’re doing, your failure rate will be closer to 100%. [32]

Build social and human capital

1. Get to know more people who are great at anything. The more time you spend around people who are excellent at their work, the more you’ll start to intuit the general principles of greatness. Meeting these people can be intimidating at first. But it’s incredible how accessible great people can be if you’re genuinely interested. If you love someone’s writing, design work, open source projects, or you’re generally impressed with their work, tell them. Send them a DM, an email. Ask if they’d be open to getting a coffee. You’ll get some no’s, but a few will say yes, and you’ll be better off for it. This is not transactional networking in the Dale Carnegie sense; talented people make great friends, and a friendship built on real admiration for someone’s work is a foundation that makes warm introductions easy later.

2. Go work somewhere with a high talent density. If you’re hiring right now, you don’t need a job. But if you’re early in your career, one of the best investments you can make in yourself is to go work somewhere with a reputation for hiring insanely great people. That will make #1 easy, too.

3. Go work for a truly great manager. Again, if you’re a leader hiring for a role, this one is irrelevant. But at any point in your life, if you’re feeling like you need to recalibrate what greatness looks like, go find someone great to work for. You’ll learn a lot along the way.

Practice using tools

1. Practice Defining roles. Great pickers are great definers with great process. If you’re a leader, make sure you use the MOC to define roles. As you do so, you’ll realize where you’re not calibrated on what great looks like.

2. Run Calibration Calls. Go to investors, founders, and people whose judgment you trust. Ask them who is best in class and who knows people who are. Talk to two or three people actively succeeding in the job, then another two or three who have hired for the role or worked alongside someone excellent at it. A few angles on greatness beats one. These conversations will take you from novice to competent, at least in observing expertise.

3. Pay attention to and use the Competency Stack and the Terrain Test. A design job at a magazine is different from one at a startup. A five person company is wildly different from a five hundred person one. The difference between a manager and a VP is material and your company’s values are critical to score against.

4. Learn to ask evocative questions. The one question experts suck at answering is the one you need the answer to: “what does great look like?” Instead, ask them about what makes a person fail in this role. [33] [34] Ask “How much of this person’s greatness is due to them working in a great environment?” Ask them how they interview for this role. Ask them what a true master would do in this role that would look surprising to the untrained eye. [12] Ask “Who’s the most underrated person you’ve ever seen do this work, and why do you think others missed it?” And finally, learn to ask your own questions that draw out dimensions of general and specific greatness along with ways to evaluate whether candidates have either.

5. Be aware of sample sizes. Graham Duncan, the so-called “Talent Whisperer,” [35] speaking on reference calls, recommends asking how many people in this role have you actually seen up close? Call the best a 100/100; where’s this person? [36] But the sample-size check is relevant whether you’re calibrating what great looks like, too. When doing a Calibration Call with someone who isn’t actively doing the job, but rather someone who hired for it successfully or worked alongside someone great, find out how big their sample size is. That will allow you to weigh everything else they say.

6. Keep a Hire Log. Peter Drucker argued the only way to find out if your judgment is any good is to write down your decisions and revisit them 9-12 months later. He called it “feedback analysis.” [37] Apply this to hiring. Create a spreadsheet for yourself as a leader to keep a record of your hires. Twelve months later, score each hire: success, OK, or failure. When you log a miss, make sure to appropriately attribute the miss: did their role change unfairly, did their manager quit? If you’re creating a role definition document like an MOC, you should already have this data ready to harvest. Drucker cited the average hiring manager as getting it right one third of the time, but believed the best could bat close to 1,000. [21] Marc Andreessen, on the other hand, thinks people who work at it can get to 50/50. [32] Either way, I’m confident .333 is not a ceiling. I dare you to do better.

Your goal is to make great hires. To do so you do need to avoid making bad ones. The primary failure modes of this competency lie in deficiency and excess (thank you, Aristotle) [38], but also in miscalibration. If you never spend a minute learning what the best people look like and do for any given role you’re hiring for, you will not hire great people. Full stop. That’s deficiency. On the other hand, excess involves spending excess time or holding an unreachable standard. You could easily spend all day calibrating and calibrating and calibrating. And calibrating. Did I mention calibrating? As for an unreachable standard, remember: great people have flaws, accept that, plan for it, and be extremely clear about what strengths are non-negotiable and which weaknesses you can accept. Finally, it’s too easy to tune into the wrong signal. That’s miscalibration.

Deficiency

1. Naive practice. Getting in “reps” as a hiring manager only teaches you something if there’s a feedback mechanism, a scorecard. If you’re not tracking your wins and losses, you’re tricking yourself into thinking you’re gaining experience. [22] The antidote here is to use the Hire Log described above.

2. Failure to calibrate against opportunistic hires. In those rare moments where you discover a diamond in seat 4C on the flight to Newark, don’t cowboy hire them. Run a compressed version of calibrating what great looks like. Clear the decks if you need to. This failure mode lies in thinking your gut is good enough. It’s not.

3. Beware of the timeline trap. The right person in two or three months is infinitely better than the wrong person in three weeks. Writing a great role definition doc, doing Calibration Calls, checking the Competency Stack and the Terrain Test, it is deeply seductive to think of all these things as extra credit. They are not. Moving a few things around on your calendar, while unpleasant in the moment, is far more preferable to the mountain of costs a failed hire will exert.

Excess

1. The vain socialite. Spending time around truly talented people is a lot of fun. But beware the pull to become a socialite, attending all the hot events. Remember your end goal: calibrating what greatness looks like so you can hire a stunning colleague who gets the job done.

2. Impossible role design. It’s common for leaders to define an impossible role at first. They want one person to do the job of two or three. If you notice you’re getting feedback as you calibrate that nobody can do the job you’re looking for, that greatness does not exist, stop calibrating and redesign the role. [21]

Miscalibration

1. Overlooking weirdness. Some forms of weird are red flags and others are artifacts of a non-standard path taken to greatness. Differentiate between the two or risk passing over outliers. Let a track record of impressive outcomes be your guide.

2. Calibrating inside one network. Five calls with people part of one social circle is a fast way to trick yourself into thinking you’re getting five opinions when it’s only one. Make sure not to calibrate from an echo chamber.

3. Failure to calibrate with true experts. You could make a hundred calibration calls with mediocre people (don’t), you’ll be left with a high-definition picture of mediocrity.

4. Privilege proof over confidence and fame. People follow confident advisors more. In studies, humans actually prefer overconfident advisors to accurately-confident ones. [9] Confidence is not an accurate cue. Fame also inversely correlates with accuracy. The more famous an expert, the worse their forecasting. [7] Look for proof: someone actively succeeding on objective standards, not the biggest name with the most confident opinions.

5. Beware the “one-big-idea advisor”. Some people are a man with a hammer who thinks all things are a nail. In practice, this will sound like “Look, it all comes down to pipeline; that’s all that matters,” or “Only hire people from Stripe.” Research found that so-called “foxes” (people who synthesize many small ideas) beat hedgehogs (one organizing idea) on calibration. [7] Even worse, hedgehogs get worse with expertise, because sophisticated people can build sophisticated justifications for their one lens. [7]

6. Beware of listening hazard. The more you know, the more you’re prone to egocentric discounting, the act of “discounting” advice because we think we’re smarter than the advice. [39] When an expert’s take conflicts with yours, especially after you’ve gone from novice to advanced beginner or competent, know you’re likely to discount it. Remember their reps (hundreds over your three), argue their position back to test it, not to defend yours.

A friend of mine runs a business that puts him in the room with the world’s top business leaders on a regular basis. I asked him what goes through his head at these events. His reply, “You belong here.” The team you build is a reflection of the standards you hold for yourself and your company. Did I belong in a baseball stadium drinking Diet Coke with a man who’d just sold his company for a billion dollars? Who even decides something like that. You do. You decide. Just don’t be an asshole about it.