How We’ve Used Crowdsourcing to Split Test (And Why You Should Too)

crowdIf you’ve been following us for awhile, you’ve no doubt heard us mention split testing a couple of times. We love having the ability to be pragmatic in our efforts, trying multiple approaches to achieve the best result. And we use split testing in various aspects of what we do, from trying different pieces of copy to generate website traffic in different regions, to trying out different email messages with subsets of the same audience segment. There’s a time for split testing nearly everything in marketing, but one of the most interesting and effective use we’ve found for split testing comes through crowdsourcing, which was a recent topic of discussion in a Preneur Podcast featuring Alec Lynch, co-founder of DesignCrowd.

Over the years, we’ve leveraged crowdsourcing in a few ways. We’ve used extensively to help us find designs for product packaging elements and Web properties that we’ve launched over the years.

And it’s with good reason that we’ve turned to the crowd.An article by Peter Gasca, of Wild Creations, calls out six compelling reasons for trying crowdsourcing.

Gasca’s list includes:

  • Reducing costs

  • Eliminating overhead

  • Minimising management

  • Maximising options

  • Optimising creativity

  • Creating buzz

As you might notice, all of these are welcome occurrences in your business (not to mention that each of those reasons coincides with one or more of the 7 Levers of Business). All of the above factors have led us at Preneur HQ to love crowdsourcing, and once you’ve given it a try, you’re likely to love it, too.

With that in mind, in this essay, we’ll look at the following:

  • Ways we’ve used crowdsourcing as a form of split testing

  • How you can use crowdsourcing to get better results

  • Outsourcing beyond design with a split testing ethos

Our goal here is for you to understand why we’ve used split testing by way of crowdsourcing, and that you will leave here knowing how to do it yourself.

Let’s get started!

Ways We’ve Used Crowdsourcing as a Form of Split Testing

Let’s start with how we used crowdsourcing to split test designs for a product. We’ve used the site a few times.

One instance came when we posted a job asking designers to revamp the homepage for Infiniti Telecommunications. For that job, we fielded 89 designs to pick the winning design (which served as the homepage there until our most recent major site update).

89 designs were submitted for the Infiniti homepage redesign.

89 designs were submitted for the Infiniti homepage redesign.


Another task for which we turned to 99designs was a redesign of For that task, we chose one design of more than 70 entries, and even chanced the selection of a competent designer who was able to help us improve the usability of our site through her own tweaks to our design brief.

The winning design for Discount Mobiles.

The winning design for Discount Mobiles.

We relaunched the ‘Own the G’ campaign, in 2010. As the video at that link explains, the squeeze page for the relaunch was actually three different split-tested pages. We created three different headlines, three videos, and even three bits of copy around the opt-in form/button itself. We then used Google Website Optimiser to randomly assign variations of the content to each visitor who landed on the page. From the resulting analytics, we were able to work out what combination worked best.

We received more than thirty entries for the Own the G relaunch.

We received more than thirty entries for the Own the G relaunch.


Ultimately, we found that one particular combination resulted in upwards of six times the opt-ins. While the whole exercise of relaunching the ‘Own the G’ campaign is an example of split testing, the part of it that we’ll focus on, is the redesign of the collectible certificate that accompanied each piece of the timber from the G’s Ponsford Stand, which was one the bonuses added-on to the sale of each piece of limited edition crested carpet from the MCG.

The winning design from 99designs.

The winning design from 99designs.

To get a great design for the certificate, we turned to 99designs, where we prompted the crowd to contribute designs for the backer, with $100 offered for the winning designer. We received submissions, admittedly of varying quality, but we we ultimately able to narrow it down to a clear winner fairly quickly.

The thinking behind using 99designs for projects such as these is as follows:

Rather than plunking down $500 or more for a single design from a freelance design marketplace like eLance or oDesk, we were able to spend a comparable amount for each project at 99designs, while benefiting from having more than one freelance designer (and, therefore, a multitude of designs) to choose from. This is split testing, potentially split 99 different ways!

And that exposes the underlying reason for split testing in the first place: directing your dollar spend towards only those processes that generate the best result. In our case, a wealth of options is typically the desired result, and turning to crowdsourcing gives us plenty of designs to choose from.

News Report on How Pete First Sold the G:

How You Can Use Crowdsourcing to Get Better Results

So, do you feel like you’re ready to get in the crowdsourcing game?

We’ve chiefly used crowdsourcing for graphic and Web design, but there are many other tasks that platforms like 99designs (and have to offer.

Here are some other common crowdsourcing tasks:

  • App design

  • Packaging design

  • Stationery design

  • Signage

  • Tee shirt design

  • Photo editing

  • WordPress design/development/administration

  • PowerPoint creation

And there’s a lot more – pretty much anything related to ‘design,’ in all its various forms. Basically, any task for which you’d find value in having a wealth of options is a task that’s suited for crowdsourcing.

Swipe + Deploy!

Now, let’s dive right into a Swipe + Deploy strategy for outsourcing a job. Once you’ve set up an account at 99designs, you’ll want to create your first job brief. We’ve included the one from which all the briefs for the design jobs we’ve done have been derived. The idea here is that you can take what we’ve done and simply fill in the blanks with your needs.

Here’s our outsourced job template:


I am looking for an experienced …


Describe what needs to happen…


Give an overview of how/where this work will be used…


You will be provided with…


Describe the desired result…

For a working example, let’s look at how we filled in some of the important parts of an EHS sales letter at DesignCrowd.

Here’s an excerpt from the actual advert.


I am looking for an alternative design for our product sales letter (webpage) at which will be used as an A/B Split Test to hopefully increase conversions.


We are currently driving traffic to the site via Facebook Adverts, and strategic affiliates in the cycling and triathlon niche. Most visitors to the site have no pre-awareness of the brand or product, so it’s typically a cold visitor, seeing the product for the first time, driven by ‘safety focused’ advertising messages.


– All the copy which you see currently on the sales letter / webpage is needed, so please incorporate all copy. (If you feel some should be left out etc, please message and explain your reasoning etc.)

– A range of product images are available here:

[](I am also having a photographer take new product shots this week, so if you feel a specific type of image is needed, just let me know, and we can have it shot.)

– We also have an account at if other images are needed, which you can’t / won’t supply.


– Development ready PSD (with fonts etc)


– The site does NOT have to be parallax in nature, like the current design.

– The video and be in a pop-up, or embedded on page

– The call to action (But Now) section of the current site definitely needs a redesign. (Attached is something we are about to replace that section of the current design with. So please incorporate that ‘value style’ message. )

As you can tell from the brief, we’ve striven to be clear in our instructions, even letting the outsourcing candidates in on the fact that their task is to create a page for the purposes of split testing. The brief for a crowdsourcing job is a lot like a regular job posting. But, unlike a regular job advert, the crowdsourcing brief should succinctly describe the singular task you need done, rather than macro-level educational and skill requirements. In a crowdsourcing brief, the task at hand is the sole focus, so keep that in mind.

Crowdsourcing also opens you up to social proof: voting for the best design.

Crowdsourcing also opens you up to social proof: voting for the best design. Here, the crowd votes for designs for the EHS project.

Outsourcing Beyond Design with Split Testing Ethos

You may be wondering how you can split test other outsourced tasks, beyond those tasks that generally qualify as design.

James-Tuckerman-plus-ANT-180x180Our good friend, James Tuckerman, founder of Anthill Online, uses split testing in all of his hiring decisions. When James hires a new team member for his family of websites, he’s pooling talent from freelance marketplaces like eLance. But, despite that site not strictly adhering to the crowdsourcing model, James designs his job briefs to harness the exact same benefit we do: more options.

When James needs to add a new journalist to his stable of regular freelancers, he does more than just take a proverbial shot in the dark on the freelance marketplace. He posts a job advert and hires the top three candidates to apply for the gig.

James assigns all three to complete the same task – and he pays them all their stated rate for the work, whether it is ultimately accepted or not. James knows that of those three, more often than not, one will fail to deliver the project altogether, one will do it poorly, and one will set himself or herself apart from the others by delivering quality work. This approach is so effective that every employee of Anthill Online currently employed there has made it through this process.

So, from James’ example, you can expand the crowdsourcing ethos to the freelance marketplace, allowing you to vet workers (and, thus, work itself) for more than just design-related tasks. (Pete also presented a webinar along with James that covered a lot of insights relating to this topic a while back).

Although you might be able to split test every possible task in your business, by this logic you can split test work relating to the following:

  • Programming

  • Mobile development

  • Writing

  • Editing and proofreading

  • Accounting

  • Consulting

  • Search Engine Optimization (SEO)

  • Lead generation

  • Administration

And this list goes on, as you can imagine. The idea here is to take an already wide field of freelancers and get even more value from every dollar you spend to get a given task done, therein applying the crowdsourcing mindset to outsourcing in a broader sense.

Are You Using Crowdsourcing to Your Advantage?

We’ve had great results with crowdsourcing over the years, and the ability to use it for split testing in various ways has helped us narrow our focus (and our spend). And, in the ongoing quest to raise profitability, narrowness of focus is always a plus.

PS – Keep your eyes peeled for a forthcoming article that hones in on how split testing itself can be used to increase your success in various marketing contexts.

Pete Williams is an entrepreneur, author, and marketer from Melbourne, Australia.

Before being honored “Australia’s Richard Branson” in media publications all over the continent, Pete was just 21 years old when he sold Australia’s version of Yankee Stadium, The Melbourne Cricket Ground For Under $500! Don’t believe it? You will! Check out the story in the FAQ section (it really is our most asked question).

Since then, he’s done some cool stuff like write the international smash hit ‘How to Turn Your Million-Dollar Idea Into a Reality’ (+ the upcoming ‘It’s Not About the Product‘) and he’s created a bunch of companies including Infiniti Telecommunications, On Hold Advertising, Simply Headsets and Preneur Group.

Lots of other people think he’s pretty good too! He’s been announced as the Global Runner-Up in the JCI Creative Young Entrepreneur Awards for 2009, the Southern Region Finalist in the Ernst & Young 2010 Entrepreneur of the Year, and a member of SmartCompany’s Top 30 Under 30.

Download Pete’s First Book For FREE!

“Now that I've got the rights back from the publishers, if you enter your email address below, you can instantly download a free audiobook version of my first book, How to Turn Your Million-Dollar Idea Into a Reality.” - Pete
100% Secure ... We will NEVER sell or rent your details to anyone.