Testing and Hiring Developers - You need to put some effort in too.

This is a topic that has been covered by a number of other websites before now, but I just wanted to add my thoughts into the mix. It’s a difficult question...How do you test and hire a developer?

Unfortunately, unlike many other job roles, time and experience does not have a direct correlation as to how good a developer may be. I’ve met developers who have only been in the field professionally for a year or so, yet have outstanding skills. Conversely, I’ve met developers with nine or ten years experience that are nowhere near as good.

So, as a hiring manager, how do you figure out if you’re hiring a duff or a diamond? Obviously, the answer is that you have to formulate some sort of testing methodology to separate the wheat from the chaff. However, this is a monumental minefield in itself that inspires heated debates whenever the subject is mentioned.

So what can you do?

Firstly, before you even consider testing a potential hire, make sure you are doing the following.

Hire for the right position

This sounds pretty obvious, but I’ve often seen companies recruiting for a ‘PHP Developer’ when they’re actually looking for a Web Architect or a similar level position where a high skill level in engineering and application design and architecture are far more important than solid PHP skills. Whilst it’s possible that a good PHP developer may be able to do this, you are far more likely to get more relevant applications if the job title is appropriately named.

Spend some time on the job spec.

The majority of job specs that I’ve seen are useless. They often have very little detail, are very vague and don’t really give much information. If the job is going to be difficult and requires a high level of skill make sure the spec communicates this. I know it’s probably quite tempting to rush the spec and give it to an agency and let them wade through the hundreds of CVs sent for the role but in doing this there is inherently a higher chance in candidates that aren’t of the required level getting through to an interview and consequently wasting your (and their own) time.

A higher level sounding spec will entice a higher level developer.

Don’t expect people to know everything

Don’t put things like the following (which I actually found on a job spec recently) in your spec:

”Must be highly proficient in PHP, ASP.NET, C#, Java, Apache, HTML, CSS, JavaScript, jQuery, Ajax, XML, XSLT, Linux, MySQL, MSSQL, Python and Perl.”

Now, whilst some people do have a decent knowledge of technologies and languages outside of their core set of skills (most PHP developers should have a fair knowledge of HTML, CSS, Apache and Linux for example) expecting people to be “highly proficient” in such a wide range of technologies is asking for trouble. If somebody did happen to have knowledge of all of those, chances are they would not be particularly brilliant at any in particular - the old phrase “jack of all trades, master of none” springs to mind.

Much better is to focus your skill requests on the real core aspects of the job. Think what the job couldn’t be done without. Don’t just spam a load of keywords at your agency or HR department as it will almost certainly degrade the quality of applications for the position. I know of developers who would deliberately not apply for any spec that had this kind of listing in it.

The Testing

Once you’ve got your spec sorted and have some candidates for interview you’re going to need to formulate some kind of test for them. Developers hate tests by default. We dislike having to prove our skills to someone else. However, we also understand that a degree of testing, particularly for a Senior or higher level job is a necessary evil.

There isn’t really a standard qualification that proves someone is a great developer (though if you’re in the Java world the SCJP would do so, but it is expensive and many can’t afford to pay for the SCJP even if they possess the skills to pass).

PHP has the Zend Certification, but this doesn’t really prove that someone is a good developer as in my experience it hinges too much on testing peoples memory and knowledge of the PHP manual rather than their general programming and development skills.

This isn’t a dig at the Zend Certification, if someone has passed it then there’s a high chance that they’re pretty solid at PHP and it’s not easy to formulate a programming certification that will test peoples development ability above knowledge of a particular language.

In answer to this, we’ve seen a few ‘online testing’ websites and companies spring up (BrainBench is one such example) and to a hiring manager, these must seem great. Cheap, easy, don’t take much time to set up and if you’re using an agency then they can do it all for you and it takes no time at all. There’s one problem with this though, the tests are mostly meaningless.

I’ve taken a couple of tests on one such website and apparently, I’m among the top 3% of PHP developers in the world....Whilst I like to think I’m pretty good at it, that’s pushing it a bit far! I’m also in the top 10% of MySQL users in the world. My SQL skills, compared to a hardcore DBA would be average at best.

Why are they bad? Again, they rely far too heavily on testing the tiny details of syntax. Most questions are ‘which line of this code has a syntax error’ or ‘what function would you use to perform x task’. Contrary to popular belief, us developers don’t sit in our chairs all day memorizing the PHP manual. Nor do we sit on the train on the way home reading it for ‘fun’.

Have someone from your technical team involved in the whole process. They and the potential hire can willingly speak “geek” to each other and you won’t have to worry about what the hell they’re talking about. You can get your tech guy to translate after the interview.

What kind of test should you give a developer?

There are a few ways you could go about this. As I’ve said before, there is no real “winner” when it comes to the best way of testing developers.

Sit them down with a piece of paper and a pen describe some simple program and have them write it down. Do not ask them to do this in any particular language or check it for syntax errors or correct function names. They may write it in a particular language and that is fine. The script would not need to run for them to be successful at this task. This kind of test should be used purely to see how their brain works out programming logic and communicates it to you. See the post on FizzBuzz at Coding Horror for something similar to this kind of test (although it seems to be language / syntax specific).

Give them a very open ended, real world task to complete using a computer with internet access and a limited timescale (say 1 or 2 hours). Produce a specification for a basic program and let them go and build it in their natural environment.

That involves having a computer with internet access. I’m sorry to break any notions you have of developers brains, but we use google a lot. Don’t make this test something that is very niche or would require knowledge of your company code base to complete. Give them a coffee and make them feel comfortable and make it clear that not completing the spec is OK. In fact it should probably be a bigger task than can reasonably be completed in the allotted time. You should be using this kind of test to see how they would go about working on a project in terms of prioritizing sections. It should also give you some idea about the general quality of their coding skills.

Get some print outs of code and see if they can figure out what it’s doing and explain it to you. You could also field questions on what might be able to be done to improve the code. Don’t use this kind of test to ask the candidate to spot potential syntax errors. We are not walking compilers/parsers.

Get some non programming, but logic related questions to ask the candidate, but let them answer them in a non pressured way. Maybe email them to the candidate before the interview. One such question would be:

• The number of false statements here is one.
• The number of false statements here is two.
• The number of false statements here is three.
• The number of false statements here is four.

Which of the above statements is the true one?

Such questions can test one’s mental aptitude to a degree. I would advise that you don’t use the exact question above as it’s now fairly well known.

Summary

Having covered the details in a fair amount of depth in this post we could summarize by saying that you should be putting a fair amount of effort into your recruitment and testing processes if you want to attract and hire good developers. If you hastily throw together a spec and farm it out to an agency the chances are the calibre of your applicants will be lower. A few of the best developers I know actually refuse to be recruited through an agency, something to bear in mind if you’re looking for the best.

I hope some of you find this useful in improving the way in which you recruit your development team members, please leave any thoughts or comments.