Our 10 Step Engineering Hiring Process at Mixer Labs
Before my company, Mixer Labs, was acquired by Twitter, I spent a big chunk of every day on hiring. A lot of entrepreneurs I help out ask about the process we used to hire a very strong engineering team and so I thought I would share the steps we would take in this post.
1. Resume screen.
We would source people from friends, engineering meetups, LinkedIn, basically anywhere we could find someone who seemed good. I think of approximately 100 resumes or profiles we looked at closely we hired 1 person.
2. Phone screen 1 - culture fit and "reasonable to set up technical call"
After choosing a subset of profiles or referrals to follow up on, I would do an initial "culture fit / is this person worth talking to" phone call or a "sell" call to people not actively looking. In some cases people did not want to leave their job (if I was reaching out to them), or they might be a bad fit to begin with. I would screen for things like:
What motivates this person? What do they care about?
What are they exceptional at?
Do they think of themselves as a generalist engineer or a specialist (e.g. FE, mobile, etc.)? We were looking for generalists.
Do they seem like a really strong engineer?
Are they excited to join a startup?
Will they be a good culture fit?
I would simultaneously try to sell the person on the opportunity, to get them excited about our company. Remember, all interviews go both ways as the candidate is also interviewing you as an employer.
These interviews would literally take 15-30 minutes and I set expectations up fron that the call would take 15-20 minutes (if someone is asked to block 15 minutes on their calendar, they usually block 30 minutes anyhow, so we never ran out of time). These short blocks of time minimized the time spent on this first screen and often I would know within 5-10 minutes if the person was worth a second conversation.
I would meet the really promising people in person for a coffee if they were somebody I was trying to convince to leave a comfortable existing job.
Of 100 people we started with, maybe 15-20 people got an initial phone calls or coffees.
3. Phone screen 2 - engineering interview.
This second phone screen focused more on engineering questions and ensuring the person was strong technically and merited an in person interview. This would take 30-45 minutes and be performed by a rotating set of engineers on our team. Different engineers had different filters / set points (e.g. some were much harder "graders" then others, so I would take this into account when scheduling in person interviews, asking the candidate to do an additional phone screen (if they seemed marginal), or dinging them.
Of the 100 people we started with, 7-8 would probably make it through the phone screen to an in person interview.
4. In person interviews.
People would come in and be interviewed by each member of the technical team for ~45 minutes. We would sometime put an extra 15 minute buffer for engineers who tended to take longer with candidates.
We asked the interviewers not to let each other know what they thought of the candidate as they came out of the interview, so the next interviewer would not go in biased. Studies have shown that sharing your opinion of person A with person B (if B has not met A) biases the interactions B will then have with A.
I asked each interviewer what they thought of the candidate over IM immediately after each interview (basic thumbs up or down). If enough people gave a thumbs down, I would cut off the rest of the interviews to save engineering time. I.e. the time of our engineering team was too precious to waste on candidates that had people think they were not strong enough for the team.
Before we had multiple engineers on the team, we would ask out strongest technical friends to come by and interview people to help us. This allowed us to ensure we always had ~4 engineers interview every candidate, even if we were just 2 founders to start with. As we scaled to past 4 people, we still had every engineer on the team met every hire.
If one of our team had worked directly with the engineer before, we would skip them past phone screens directly to an in person interview. We had *everyone* interview with everyone on the team, even truly exceptional engineers we had worked with in person before.
Of the 100 people we started with, 2-4 made it through the in person interview.
5. Discuss the candidate
We had each person write down their thoughts of the candidate before we discussed them and then asked each person for their opinion. We also asked people how they would rank the candidate against all the other candidates we had interviewed. I.e. below a certain bar they were not worth considering further.
For a while, we used to graph the stack ranked interview feedback vs years of experience to look for outliers who are stronger then average. We would pursue these outliers aggressively.
6. Half to full day work with team
We asked a subset of the people we moved forward with to come in to work with the team for a half day or full day to test their working style, how much they could get done in a day, design/engineering choices they would make, and also whether they were interested enough in us to actually take the time off to meet with us.
We would always give the candidate the same coding problem, and we would always through away they code at the end. The purpose was to test their skills and fit, rather then have something productive come of the day. At the end of the day, we would all gather around and the engineer would show us his or her work.
3 out of 4 people who came in to work with us we moved forward with to the beer test. In some cases, having someone come in to work with us made us more excited about them as a candidate (i.e. it was someone we were going to ding based on interviews, whom we kept due to their performance coding). They may have been medium on the in person interviews, but great coders who just got a lot done and were good to work with.
7. "Beer test"
We would take candidates out for dinner or beers to see if they were a good culture fit. I.e. would our team enjoy having the person around, drinking beer with them etc. In a small organization culture fit is paramount.
We actually dinged a really exceptional engineer that made a lot of grossly off color comments in the beer test. He was one of the strongest engineers we met, but felt he would be a bad fit culturally, and this only came out during the beer test.
1-2 out of ~100 people whose profiles we started with would make it through the beer test to be given an offer.
8. Reference checks
We would check ~5 reference per candidate. Only one candidate who made it to the reference check phase got dinged because of his references (we actually always tried to back channel to references the candidate did not provide, and in this case the engineer had some politically driven conflicts with others, so we decided not to hire him). As a small organization we did not want to take the culture risk, although that engineer went on to thrive elsewhere.
Great reference checking will be the subject of a future post.
9. Offer
We would make the offer to the candidate in person or on the phone, depending on our prior relationship to the candidate. I may get into more details on making an offer in a future blog post.
10. Close
Of the people we made offers too, about 2/3 to 3/4 of them accepted. Most of the people who did not join Mixer Labs, tended to stay with their existing employer. We would use a number of tools to close the candidate, including e.g. having them meet with our investors to get our investors view of why we were a good company, or we would invite the candidate to events we had at the company, such as our speaker series (even when tiny, we invited CEOs and other industry execs or entrepreneurs in to our company to give talks on their experiences. This was a really cool company perk everyone enjoyed).
Hiring is a numbers game
All numbers quite rough (e.g. 100 profiles to 1 hire is more or less what happened with us but I am pulling all the numbers from memory so they are rough estimates. I actually had big spreadsheets with all the details but can not find them) - but hiring is very much a big numbers game. You need to set up a funnel and optimize each step.
All else being equal, we chose to optimize for false negatives over false positives. We kept our bar really high, and due to this grew (painfully) slowly. But it is was worth it as we ended up with a pretty extraordinary team.
Does your company do something differently that works really well? Let me know in the comments what you do differently - I always love to hear how people approach hiring and what works and does not work.
You can follow me on Twitter here.
.