But, like many rapid growth spurts, there have been some growing pains for the community along the way. The volume of contests and participation from both developers and sponsors has led to a spike in demand for judging and support for answering questions on the contests that are running in a timely manner. And of course we want to make sure our developers get paid as efficiently as possible - not always the simplest thing with participants from more than 65 countries! How will we make sure we make it through the growing pains of adolescence as quickly and smoothly as possible? With all of you of course!
Over the next few months we will continue to build out the capabilities of CloudSpokes through a number of contests that help add in transparency and understanding of what is occurring.
Before we get to some of that, we’d like to share some statistics on CloudSpokes to date:
- We have run 330 contests. Of those:
- 243 have completed and paid successfully (73.6%),
- 24 are open or being reviewed (7.3%),
- 51 had zero submissions (15.5%),
- 10 had no valid submissions (3.0%),
- *2 were cancelled (0.6%) (*for both, we offered to compensate participants who had started working on submissions)
- We have paid prize money on 100% of challenges that have received at least one valid submission
- We have paid 62 bonuses (totalling $16,190), either because we received more good submissions than we had prizes, or as compensation for doing some extra work to polish up a submission
- In total, we’ve given away $453,640 to CloudSpokers from at least 23 different countries
- Average time to judge a contest during the past three months is 6 days past the “Winner Announced” date posted on the challenge. Average time to judge a contest during the past 30 days has been 2 days past the “Winner Announced” date posted on the challenge
- The number of submissions per contest have increased by 19.7% in the last three months. In 2011, the average number of submissions per contest was 2.94, and that has risen to 3.52 in 2012 (so there’s clearly an appetite for even more contests!)
To manage this growth, here are a few of the things we will be doing over the next few months - with your help!
- New dashboards - Views that let you see the status of judging for challenges you have entered, and the status of your payment for the challenges you have won. In addition, we will provide summary views of challenges (number of registrants, submissions, valid submissions, winners) to show the progression, but also to give you a sense of what it may take to win challenges on average
- Enhancements to badging and more challenges - We will be expanding our usage of badges on the site to give folks credit for challenges they did not win. In some cases, challenges draw an unusually high number of valid submissions that greatly exceeds the number of prizes. While we would love to pay everyone, it is a contest, so we guarantee paying the winners (best entries), not necessarily all the submissions. We will address this by using badges to allow people to see your accomplishments and by ramping up the number of total challenges so this happens less often
- Enhanced guide to scoring on each challenge - For every challenge, we directly indicate the judging criteria that will be used to evaluate the submissions (scorecards). Sometimes our judging has been delayed because it has not been possible to easily deploy and test the submission, so we plan to add criteria on deployment readiness to the scorecards. In addition, as we grow, we want to give proven CloudSpokes participants the ability to also judge contests they are not competing in (and earn money for their efforts). To support this we will be creating guidelines for this program to ensure fairness and avoid conflicts of interest
CloudSpokes has the potential to revolutionize development by providing independence to a generation of developers by focusing on what they can accomplish instead of forcing them into an hourly rate. And it can show customers that the old school model of hiring hundreds of consultants isn’t the only way in the cloud. So please send us your stories of how its helped you, your suggestions on what we can do in the future and even your complaints on the things we need to fix today. It’s how we become better. We look forward to announcing some very cool new ways of giving back to the community over the coming months - so stay tuned!