Thursday, May 17, 2012

Next thing we knew, we had 40,000 friends at the CloudSpokes Party!

We wanted to take a moment to thank all the folks that have supported CloudSpokes over the past 15 months. All of you spreading the word, registering, competing, and winning challenges is what has made the site possible. Ever since we launched, the core mission of CloudSpokes has remained the same: combining cloud technologies and crowdsourcing so customers and developers can work together like never before. Developers decide what technologies and projects they want to work on, discover new technologies in practical ways, compete to demonstrate and be recognized for their skills (badges) and win ($) based on producing great results. This can be a great hobby or a way to create independence in your career.

But, like many rapid growth spurts, there have been some growing pains for the community along the way. The volume of contests and participation from both developers and sponsors has led to a spike in demand for judging and support for answering questions on the contests that are running in a timely manner. And of course we want to make sure our developers get paid as efficiently as possible - not always the simplest thing with participants from more than 65 countries! How will we make sure we make it through the growing pains of adolescence as quickly and smoothly as possible? With all of you of course!

Over the next few months we will continue to build out the capabilities of CloudSpokes through a number of contests that help add in transparency and understanding of what is occurring.

Before we get to some of that, we’d like to share some statistics on CloudSpokes to date:
  • We have run 330 contests.  Of those:
    • 243 have completed and paid successfully (73.6%), 
    • 24 are open or being reviewed (7.3%), 
    • 51 had zero submissions (15.5%), 
    • 10 had no valid submissions (3.0%),
    • *2 were cancelled (0.6%) (*for both, we offered to compensate participants who had started working on submissions)
  • We have paid prize money on 100% of challenges that have received at least one valid submission
  • We have paid 62 bonuses (totalling $16,190), either because we received more good submissions than we had prizes, or as compensation for doing some extra work to polish up a submission
  • In total, we’ve given away $453,640 to CloudSpokers from at least 23 different countries
  • Average time to judge a contest during the past three months is 6 days past the “Winner Announced” date posted on the challenge. Average time to judge a contest during the past 30 days has been 2 days past the “Winner Announced” date posted on the challenge
  • The number of submissions per contest have increased by 19.7% in the last three months. In 2011, the average number of submissions per contest was 2.94, and that has risen to 3.52 in 2012 (so there’s clearly an appetite for even more contests!)
To manage this growth, here are a few of the things we will be doing over the next few months - with your help!
  • New dashboards - Views that let you see the status of judging for challenges you have entered, and the status of your payment for the challenges you have won. In addition, we will provide summary views of challenges (number of registrants, submissions, valid submissions, winners) to show the progression, but also to give you a sense of what it may take to win challenges on average
  • Enhancements to badging and more challenges - We will be expanding our usage of badges on the site to give folks credit for challenges they did not win. In some cases, challenges draw an unusually high number of valid submissions that greatly exceeds the number of prizes. While we would love to pay everyone, it is a contest, so we guarantee paying the winners (best entries), not necessarily all the submissions. We will address this by using badges to allow people to see your accomplishments and by ramping up the number of total challenges so this happens less often
  • Enhanced guide to scoring on each challenge - For every challenge, we directly indicate the judging criteria that will be used to evaluate the submissions (scorecards). Sometimes our judging has been delayed because it has not been possible to easily deploy and test the submission, so we plan to add criteria on deployment readiness to the scorecards. In addition, as we grow, we want to give proven CloudSpokes participants the ability to also judge contests they are not competing in (and earn money for their efforts). To support this we will be creating guidelines for this program to ensure fairness and avoid conflicts of interest

CloudSpokes has the potential to revolutionize development by providing independence to a generation of developers by focusing on what they can accomplish instead of forcing them into an hourly rate.   And it can show customers that the old school model of hiring hundreds of consultants isn’t the only way in the cloud. So please send us your stories of how its helped you, your suggestions on what we can do in the future and even your complaints on the things we need to fix today. It’s how we become better. We look forward to announcing some very cool new ways of giving back to the community over the coming months - so stay tuned!

- mess


  1. Dave, thanks for the wonderful post writing about all the good things Cloudspokes has accomplished over the past 15 months. I have to say, I rediscovered the developer in me via Cloudspokes, learnt some new technologies and made some some smart new friends along the way. The fact that I got paid prizes for all of this... you just can't beat that. It's a win-win situation for all. This model is scalable and democratic. Let's keep up the good work.

  2. Thanks for sharing these statistics. I think it's been a fabulous journey for CloudSpokes over the last couple of years and I have no doubt that it will only get better with time. I look forward to the mentioned enhancements - especially badging and judging a few interesting CloudSpokes challenges when I don't have the time to participate :D

    You might also want to consider things such as DreamSpokes (aka Dreamforce for CloudSpokes - perhaps virtual) event, CloudSpokes MVPs, participant hangouts etc. for future.

    IMO, CloudSpokes is a testimony that crowdsourcing is a disruptive phenomenon and I think every developer who's participated will be glad to be a part of this rock and roll ride.