The topic of 'hiring' always generates a lot of discussion. And why not? Talking about hiring is a convenient way to pass judgment on large groups of people while keeping a professional, detached demeanor.. Ouch! But the topic has enough technical basis to warrant the interest of experienced developers, so here we are.

This post is for those who handle the technical evaluation necessary to hire candidates, especially in the Ruby and Rails scenes, although the overall strategies are language-agnostic (though I'd hope if you're hiring folks to work on missiles and nuclear power plants, all bets are off).

This is a guest post by Tim Goh of Trikeapps, an Australian software development company. More info at the bottom of this post.

Some Structure

While hiring processes differ, I'm organizing my advice by the following components that, assumedly, most processes include:

• the job posting itself
• the coding test
• the phone screen

The Job Posting

You need a permanent "Careers" page

You should have a permanent careers page on your company site and constantly correspond with candidates, even if you're not hiring at the time. The goal is to populate a queue of developers you can subsequently start to contact the moment you have an opening. This way you hire only after the need, while identifying talent before the need.

You can also continuously refine and test your hiring process against more candidates in this way as well as get a more accurate picture of who's looking and who's available, instead of peeking out your window to see who's there only when you're hiring.

Include a filter question

You want to easily distinguish the genuinely interested from those who are simply applying everywhere (blasting resumes out semi-automatically has become very common in these shaky times). With the right question you can design a mail filter around the answer and automate away those who aren't paying attention.

Such a question might be as simple as:

What is the MD5 hash of the string 'I am not a resume spammer'?

Make the question more difficult if you're getting too many resumes that qualify at this stage. The Dropbox Challenges and Facebook Puzzles pages have good examples of non-trivial coding questions.

Providing a larger set of questions and making some optional allows candidates to signal their desire, and gives you an easy sorting function to use on your incoming applications. If you want to hire someone who's particularly creative, their response to the question may also be an indicator of this.

Include more information about your company

Qualified applicants will be more choosy and select a small group of job ads to send their resumes to. Help them make that decision by providing as much information about your company and the role you're hiring for as possible. Just as you would balk at a resume that solely said 'I've been coding in Ruby since 2008', a job description of 'develop web applications using Rails' would deny valuable data to the talent you're looking for.

In days of yore it was popular to include the Joel Test in job postings. This is a high signal way to communicate about your company's processes, and you could follow a similar template.

Job boards may not allow for the best presentation of your company (Ed: Tim is lying, the Ruby Inside Jobs Board is undoubtedly awesome! ;-)). Keep the big information dumps on specific landing pages on your site that you link to — this way you can use analytics to determine how often your ad is read by candidates who are really interested, not to mention which job site sends over more of those candidates.

The Matasano careers page is an excellent template to follow. It goes into exquisite detail and serves as a permanent ad for working at the company. Bonus points for describing their hiring process and even including an FAQ.

It is particularly important to describe what your existing developers are opinionated and passionate about. While you don't want to turn away qualified candidates, being upfront about potential deal-breakers avoids wasting their time and yours.

Take a look at this YC W11 startup job ad. Although the name of the company and what it does is not revealed, you get a very good idea of what they're like and who they're looking for.

A Philosophical Interlude

The tips for the coding test and phone screen assume a set of principles. It is important to have some guiding philosophy when it comes to personnel at your company, lest your culture becomes scattershot and non-discerning.

Most of the time your principles can be boiled down to a single catchphrase, a motto if you will. Perhaps the most famous for developers is Spolsky's 'smart and gets things done'. Google has the Lake Wobegon strategy Investment banks and law firms often follow some variant of 'hire alpha male assholes'.

The philosophy I go by when hiring is: test for the first derivative.

Eric Sink's post on Career Calculus describes a formula C = G + LT as follows:

C is Cluefulness. It is the measure of how valuable you are to an employer.

G is Gifting. It is defined as the amount of natural cluefulness you were given 'at the factory'.

L is Learning. It is defined as the rate at which you gain (or lose) cluefulness over time.

T is Time. It is on the horizontal axis of your career graph.

My adaptation: C is redefined 'cluefulness about your company's technology stack'. G then naturally becomes 'existing knowledge of your stack'.

A High-L candidate is a better investment than a High-G

Unless you have a short-term need you have to absolutely fill ASAP, you should always prioritize L. So you should open your positions to Ruby experts who don't know web development too well, or a highly experienced web developer who does not know Ruby. The latter is particularly valuable because they'll bring a wealth of experience of alternate solutions for common web development problems.

Testing for G is easy — it's a pre-defined set of knowledge determined by what you use. The coding test tips and phone screen tips that follow are geared more towards finding L, which dominates the final productivity of the candidate (ie the area under the graph) over time.

Another way of stating this principle is that you're testing for second-order effects. What a candidate knows may be useful information; but we're interested in actionable information: what a candidate can do with what they know.

The Coding Test(s)

Do not hire without seeing code written under time constraints. Code samples and open source contributions are great, but not entirely reliable: they often do not give an adequate idea of timeframe involved, plus most developers will naturally choose to code what they like/know.

In the same way you extract frameworks from an app, interview questions should be extracted from actual problems you encountered at work. Chances are you haven't ever had to write atoi or quicksort recently. And I've certainly never output 'Fizz' in any production code. By basing interview questions on a recently-solved problem, you already have a in-house solution to test against, and it is as accurate a simulation of real work you can get.

Note that when picking a recently-solved problem to include in an interview you have the benefit of context and hindsight, both of which the candidate will lack. Test your question on someone unfamiliar with it to gauge its difficulty.

The goal is to test for as many of the following as you can include in a short question:

• data munging
• corner case identification
• working with existing code
• understanding of complexity

Data Munging

One of the most common programming tasks is to take data in one structure/format and convert it into another. Some developers will do this elegantly in one line, chaining appropriate Enumerable/etc methods. Others will introduce intermediate variables, open up a core class or two ('because my_array.extract_hash_elements just reads better!'), use a Tempfile, and hit S3 before their curtain call to the xmltojsonfordummies.com API.

A simple way to test this is to have a candidate access a web API that returns xml/json and have them munge the result into a particular format. Code smells to look out for: populating a Hash by hand instead of with inject, unnecessary use of nested arrays, clumsiness with string manipulation.

Here's a template you can use to base such questions around:

Take the result from http://api.twitter.com/version/statuses/public_timeline.json and output an array that meets the following criteria:

• has usernames as elements
• that has the text 'Ruby' somewhere in the tweet
• ordered by follower count of user who posted

If a candidate bloats a simple data munge in an interview, imagine the obesity they can inflict on a code base they work on daily. No hire.

Corner case identification

Even more destructive than a bloaty munger is the eternal optimist coder, who lives in a fantasy land where every input is sanitized, HTTP calls never time out, and exceptions are never raised.

Not being able to identify and deal with edge cases is a straight No Hire. Such developers are a tax on the rest of the team's focus.

You shouldn't have to gear a question specifically towards seeing how well candidates do in this regard — most questions have them for free. Just be sure to check for error handling when you review their answers.

Working with existing code

It is much easier to write new code than to grok and work with code you didn't write. We need to test if a candidate can figure out where to add/remove code, given someone else's coding style and assumptions.

This is one area that open source contributions predict very well, but should the candidate not have those available, there are many ways to test this:

• debugging
• refactoring
• extending code to add a new feature

The code snippet used here should be similar to the code they'll encounter on a regular basis. A simple source for such questions is from your company's internal repository of custom patches for dependencies that never made it upstream.

Sample Questions:

• Modify Sinatra's send_file method to optionally support Nginx's X-Accel-Redirect feature
• You are trying to profile the load times of gems to figure out why your app takes so long to load. Show a diff of how you would modify Bundler and/or Rubygems to report the metrics you would need.

Understanding of complexity

The ideal half-hour code test question has an obvious n^2 or greater solution, and a not so obvious linear solution. This may be difficult to fit in with the other goals (especially if you're trying not to reuse well-known questions), so it's something you may have to test elsewhere.

I generally use algorithmic complexity as a proxy for theoretical knowledge in general. The industry is full of autodidacts, which is perfectly fine, but they need to have had the discipline to spend time grokking the fundamentals that would have been taught it a quality computer science curriculum instead of flitting between surface learning of the newest technologies.

The Phone Screen

Eventually you'll want to get the candidate on the phone to get a more direct, "in-person" feel for what they're like and how they express themselves. Welcome to the good old 'phone screen.'

Structure the interview

Research (PDF) has shown that structured interviews provide a better indication of future job performance. For each question you have, have an established scale that the person conducting the phone screen scores candidates against immediately after each answer.

Phone screens are typically conducted by one person, so structuring the interview reduces bias and gives a better basis for comparing candidates screened by different developers.

Ask open-ended questions

While its meta-commentary on Ruby's popularity is dated, Steve Yegge's 2006 post Interviewing Ruby Programmers is still a good source for questions. You'll notice that most of his questions are very open-ended. This is no coincidence. This is what Stevey has to say on syntax questions:

Language syntax trivia doth NOT a goode programmer maketh. Hiring based on syntax trivia is the last, desperate resort of a weak interviewer. Just say No.

When you ask a specific question for which you have already determined the answer, you are limiting the amount of information you get from their answer and worse still, capping the candidate's ability to your own.

The basic rule of thumb:

Never test something that takes less than a minute of experimentation to find out in a REPL.

Sample questions:

• if you were writing a gem to do mocking and stubbing in Ruby, what methods do you think you would use most in your implementation?
• how would you implement session handling for a feature where site admins can log in as a specific user to test their actions on the site?
• Jesse Robbins's favorite interview question

Have conversations about trade-offs

This is something that happens in regular work all the time, and few interviews cater for it. Pick a problem, describe some solutions, and have candidates enumerate trade-offs, and situations in which they would pick one solution over another.

Many website features can be implemented at different layers, and this makes for a rich source of such questions.

Sample questions (note that these aren't all mutually exclusive):

• Rails's use of before_save and after_save advice versus having developers override save.
• config file in YAML versus configuration in Ruby
• mocking and stubbing in tests versus hitting the database
• ESI/SSI versus fragmented caching in templates
• pre-rendering HTML in a template and toggling display versus adding/removing via Javascript
• using the Accept-Language header versus IP address geolocation for i18n

Prefer why to how

This is actually a generalization of the trade-off conversation, but more geared towards specific code idioms in your language/framework.

Knowing why they do something is what differentiates quality developers from those who cargo cult. It shows that they deliberate on the code they're writing and spend time understanding the internals.

Here's an example of a 'why' question on database guidelines design. In the same vein you can ask what they think of an ORM's naming conventions and the failure modes that it is designed to avoid.

Test protocol knowledge

HTTP is the most important protocol to test for, though you should also touch lightly on a candidate's knowledge of DNS and SMTP. Candidates should know what request and response headers look like, and what the more common headers are.

Some quick and easy questions that serve as a minimum bar:

On one of your sites, clicking a link to an image results in a downloading the file instead of displaying an image. How would you fix this?

What is your favorite tool for inspecting HTTP traffic? Describe a situation you encountered where you needed to look at the HTTP headers to debug.

Test candidates outside of the framework you use

For many Ruby companies, this means Rails. I've encountered a lot of candidates who can do absolutely nothing without their security blanket of ActiveSupport. They can't perform calculations on datetimes, can't do metaprogramming without constantize, and generally flounder the instant they have to do something outside of a web request.

Conclusion

Hiring isn't that painful as people make it out to be. By interviewing all the time and opening up positions to experienced developers unfamiliar with your stack, you should have a much larger pool of people to cull from.

Just make sure you evaluate candidates with the -Wall option enabled. Err towards 'No hire' the moment any warning flags are raised.

And while I have you..

Have excellent answers to the questions in this post? My company, TrikeApps, is always looking for Ruby developers in Melbourne, Australia :-)

This post was contributed to Ruby Inside by Tim Goh — Tim Goh has been the tech lead on web-based Python and Ruby projects since 2006. He goes by 'keyist' on Hacker News and Twitter.

Picture credit: Milton's Red Stapler and Driving! by Mike "Dakinewavamon" Kline - CC Attribution 2.0 Generic (CC BY 2.0) license.