How Stripe, Weebly And Cue Make Programming Challenges That Are Good For Recruiting

Editor’s note: Robby Walker is co-founder and CTO of Cue. His previous company, Zenter, sold to Google in 2007. Follow him on Twitter @rwalker.

Startups like Stripe, Weebly, and Cue have spent weeks of valuable engineering time building programming challenges. And tens of thousands of engineers spend their valuable personal time playing them. Why? Because programming challenges require coding ability (just like startups). Also, challenges are fun because the participant gets to solve problems quickly (just like startups) unhindered by anything other than their own ability (unlike big companies).

Programming challenges are a fantastic way to connect great people with great jobs, particularly great jobs at startups. For example, more than half of the hires at Cue have come from our two programming challenges.

Competition: Stripe’s Capture The Flag

Stripe has run two massive capture the flag contests: extremely elaborate security challenges that test an attacker’s ability to discover and exploit security holes. CTFs are really hard. Really. Hard. So hard, in fact, that out of 10,000 entrants in the first CTF, there were only 200 completions.

Greg Brockman, who ran the CTFs, remembers that it took a lot of work. “We pulled all-nighters to get it ready, and then had to deal with babysitting the machines as things were getting forkbombed.” Greg noted, “we were very conscious of security and about separating the CTF from Stripe itself.”

The CTF took on a life of its own. “People reimplemented the levels and hosted them elsewhere… we expected 10-100 people to poke around… and then O(10k) people did it,” Greg said.

Stripe’s second CTF featured a leaderboard and a pre-announced start time, making it a race to the finish. “We couldn’t go to sleep until someone had solved it, otherwise maybe it’s just too hard” recalled Greg, but finally a user, identified as “wgrant,” solved the challenge, and the Stripe team “went home and slept.”

Stripe attracted people who loved to code by making their challenge really hard (also by giving them t-shirts). The company has made several hires through the CTF.

Curiosity: Weebly’s Easter Egg

On the other side of the programming challenge spectrum is the single, innocuous line in Weebly’s job listing for a front-end web engineer: There is a puzzle embedded in our jobs page…

This is nerd sniping at its finest â€" a thread that many engineers can’t help but tug. Solving the puzzle requires skills that are necessary to work at Weebly, including using a JavaScript console and debugging HTTP traffic.

Weebly’s first engineering hire found Weebly because of the puzzle, and every subsequent hire must complete it as a prerequisite of being hired.

Despite being a requirement, most people don’t see it that way. CEO David Rusenko notes, “It’s seen less as a gatekeeper and more as a fun thing. It attracts great people instead of cutting down on the applications.”

Nostalgia: The Colossal Cue Adventure

Last month at Cue, we released The Colossal Cue Adventure: It’s part programming challenge and part homage to text-adventure games of the 70s and 80s; and it’s all bad jokes.

One of the things we have learned about how to make a successful programming challenge is to make it fun.  When the Cue Adventure hit Hacker News, the comments section quickly evolved into a nostalgia board for Zork, one of the games that the adventure emulates. From a programming perspective our adventure is pretty easy â€" we added a bonus level for the diehards â€" but we mostly just wanted to strike up a conversation with like-minded people. A person who spends time writing code to complete an old-school game is a person we want to talk to.

<Your Challenge Here>

The best startups create an environment where each team member is their own limiting factor. Not politics, overwrought processes or organizational apathy. This is why talented people join startups, forsaking giant salaries and free massages for the opportunity to ship amazing solutions to hard problems on a daily basis. It’s this kind of person who is attracted to a programming challenge.

If you’re an engineer looking for a new opportunity, consider trying a few programming challenges to see what you can learn about your potential employer. If you’re a founder considering launching a challenge, do it. Make sure it stands out somehow. Each of the above challenges stood out in some way â€" nostalgia, competition, or curiosity.

Weebly, Stripe and Cue still get results from traditional hiring methods, like recruiters and employee referral bonuses. Comparatively, a programming challenge may seem like an incredibly large investment of time and effort. Devoting a week or more to creating a challenge is a difficult decision â€" especially when you’re already short-staffed! While programming challenges are admittedly high effort, they are also high reward (just like startups).