Chris Dragan's Corner

However easy it may seem, interviewing software engineers is hard.

If you have a little experience in developing software, you are probably participating in the hiring process once in a while. It goes like this: you come up with a few questions, see how candidates tackle them and give your opinion whether the candidates will fit into the position you are interviewing for or not. Sounds easy.

But how do you know that you can measure anything with your questions? If you are a novice interviewer, you will make mistakes. You will sometimes make bad judgements even if you are a seasoned interviewer. This is why most companies throw several interviewers at each candidate. If all interviewers agree about a candidate, the hiring manager has an easy task. If there is too much disagreement between interviewers, the manager will either dismiss the candidate or recommend him to another team. Also sound easy. But it’s far from perfect, let’s look how the process works and what could be improved.

How hiring works at most companies

The first steps of the hiring process involve getting candidates, e.g. by advertising open job positions. When the candidates send in their resumés, they go through the HR filter, which is typically based on keywords. For this reason, if you are looking for a job, you should put in as many keywords and acronyms as you can come up with into your “experience” section, to just get through the Great Filter. If you are a C++ developer applying for a C++ position, but you have used C# in the past, just throw it in, even if you are not interested in a C# position or have very little experience with it – to an HR person you may appear as more valuable than other potential candidates.

The great fault of this process is not that bad candidates slip in through the filter – those who have no experience but a pretty resumé – but that the filter potentially rejects good candidates. Some candidates may be missing resumé writing skills, but could make great employees.

Then there is the interview series, where multiple interviewers question each candidate, as described above. Usually it is easy to dismiss weak candidates. For example, most candidates are unable to answer simple programming questions, hiring a person who does not care about programming won’t make a great employee, unless the goal is to hire a bunch of drones for a thoughtless job, which is always a bad idea – such approach is usually a sign of bad management.

The fault here is not that this sieves through the sea of candidates lacking competence for this particular job. The problem is that the process fails to distinguish between great hires and mediocre hires and also usually fails to match candidates for a particular position (about this later).

The last step is salary negotiation, which I am not going to cover here, although this is also an interesting topic.

Questions

Probably everybody has their own favorite types of questions. So there are the technical questions, which focus on knowledge. For a long time this was my favorite kind, the problem was that almost no candidate was able to answer them all. For example I would ask about five various C++ keywords. I could count on the fingers of one hand the candidates who actually knew them all. Most candidates are able to guess a few keywords.

I am asking, why are you applying for a position, which specifies a particular language you will be using, and you are not even willing to learn what all its keywords are for? OK, C++ is one of the most difficult languages and has over 60 keywords. I have seen several candidates claiming being “advanced” or “experienced” in C++, and still not able to even guess some keywords. You might say, some of these keywords are rarely used or not useful, or it’s easy to look them up, but would you expect the candidate to do his job seriously with attention to details if he is not even willing to prepare for the interview? (BTW. “I could look it up” is also not an entirely bad answer)

Indeed, too technical questions do not help us to determine how good a fit a particular candidate is for the job. Knowing rare keywords by heart does not help with good design, thorough testing or finishing the job at all.

For years I would indulge myself and abuse the fact that there were other interviewers interviewing the candidates as well, so they would compensate for my being too harsh. I would also throw in a piece of C code where the candidate would have to track what number gets printed in the end. Very tedious. Not everybody knows that 1.0f is represented as 0x3F800000U. But I was not satisfied, in the end knowledge != wisdom. I realised that I am looking for something more than just knowledge in the candidates.

So there are the riddle kinds of questions. Some companies are famous or excel at giving sophisticated riddles. The proponents of riddles claim this is the best way to hire smart people. So do you really want to hire smart people who don’t know anything about the technology they are going to be working with in the coming years? Or people who are extremely smart, but don’t make good team players? Or nobody wants to work with them because they are rude and don’t respect others’ opinions? I guess the riddle questions are not the best kind.

Two kinds of employees

There are two kinds of employees. It is a simplification, but it boils down to this – the process of recruiting should take this into account, because everything else and the whole “lifetime” of an employee at one company is centered around this. If an employee of one kind takes a position that requires an employee of the other kind, the employment will not be successful and in the best case scenario will be terminated sooner than later, no matter how good the employee really is. This classification applies not only to software engineering, but to most, if not all jobs.

The first kind is more common. These employees must be told what to do. They are not able to find problems and fix them on their own, somebody else must find work to do for them. They are usually not proactive. Novice employees of this kind will not be able to complete tasks without guidance, will not research problems thoroughly, will not look for potential issues. When faced with obstacles, they will be hopelessly stuck until somebody pushes them in the right direction. After many years of work they will finally learn how to cope with most of these problems, they will still need to be told what and when to do and they will still not pursue new potential tasks on their own. Be careful not to dismiss them, they are very valuable when assigned the right job and they can handle many if not most common tasks. They are good are performing repetitive or simple tasks, after long training even sophisticated ones. They are great to have if the direction is clear. They always need supervision.

The second kind is much less common. These employees are the drivers. They go out looking for problems, find solutions and fix them. They are proactive and often don’t require supervision. They should not be micro managed, because they will feel they don’t have control and will not be able to perform and will not be satisfied with their job. These employees are often good at researching difficult problems and driving things to completion. It is always good to have at least one on a team.

There are groups within each kind and there is a special group in the second kind, let’s call them high achievers for now. I admire them the most. They constantly strive to improve themselves, they treat their job almost like an art, they are the craftsmen. Whatever problem you will throw at them, they will always solve it.

Now don’t be fooled, you don’t want to assemble a team of high achievers. The perfect team is a balance of people of each kind, so the entire team can make forward progress as well as solve problems as they are encountered. The last thing you want is to assemble a team of all smart gurus who won’t want to communicate with each other and will rip the team apart.

This is true not only in software development but in most kinds of jobs. For example there have been many attempts of building all-star teams in sports, but eventually these teams were mediocre, because the top players were not able to work together.

The question is, how do you determine during an interview to which group the candidate belongs? Throw in a riddle? People from both groups can be smart and able to solve riddles. Technical questions are not good either. To be frank, I don’t know yet how to distinguish between the two kinds of candidates yet.

Two kinds of employees, second try

To make it easier, somebody suggested to me another classification: those who are hard-working and those who are not. This is orthogonal to the other method of classification. Hard working in this case does not mean willing to do overtime, but it simply means: bent on finishing tasks, finding solutions and being honest about the work. Even people who need constant guidance can be split into those who just don’t care about the job and those who are willing to learn and build their careers.

The nice thing about this classification is that it is easy to devise simple questions with which you can judge with a high level of certainty whether a person is hard-working or not. You can start with asking details about their previous assignments and judge how willing they were to complete them and how competent they are.

I like to think of these two classifications as of two dimensions of employee classification scale. There are other dimensions as well, such as the technical knowledge dimension, which is also important but trivial to assess.

Other dimensions such as “fit for the team” are often overrated and over-advertised by HR or management. What matters is whether a person is a team player or not, but sometimes even this does not matter – if you need a “high achiever” to tackle the difficult problems your team comes across, it does not matter much whether he will be a team player or not, but how effective he will be, although you still need to back other tasks in the project with team players.

The bottom line

The bottom line is whether the candidate will be able to perform the job or not. Will he write good code or not? We he care about the code he writes or not? Will he be willing to improve his skills or not?

Currently I am focusing on presenting the candidates with real world problems and asking them how they would solve them. For example, what would they do if they faced a badly written code? Would they leave it alone or try to fix it and why yes or why not? It still gives me some ideas about the candidates and allows me to compare them, discussing the solutions with them also allows me to judge their experience. I also like to give them some simple (but not too trivial) piece of code to write and see how they cope with the task.

I know this is not a perfect way to interview, I am still looking for a better way.