The goal of this contest is to collaborate with your other contestants to build the “worst app server ever” (WASE) , and use it to complete one or more challenge computations. The challenge computation(s) and their input data-set(s) will be announced and posted next week on Thursday, November 12. The contest will remain open until Monday, November 16th at 6pm PST. Winners will be announced within the following week.
(Update: We think the rule-set below is now complete, but we still welcome any suggestions or tweaks that you might have.)
There will be three prizes.
- The first prize (a Motorola DROID and $1,000 of Engine Yard Cloud credit) goes to the person who completes our challenge task correctly first.
- Second prize (a DROID and $500 of cloud credit) goes to the person who builds the most popular WASE endpoint (the one used the most often in the most submissions).
- Third prize (a DROID) goes to the “best” WASE endpoint written in Ruby (as determined by us). The contest DROIDS are full price, non-contract-linked, US models.
How WASE Works
Why is WASE the worlds worst app server technology? Well, instead of a sane message bus like AMQP, WASE uses Twitter as its message bus. Instead of a proper message router, WASE uses a list of twitter accounts as its program listing. And instead of encapsulating data with each message, WASE messages only contain a reference to JSON objects or arrays at input and output location(s) specified by a bit.ly.
“This Sounds Like the World’s Worst App Server. Tell Me More.”
Well here is an example of how a sample computation might work. Let’s say @engineyard wants to take a JSON file containing an array of names, and get a list of the top quartile of names after sorting the array. We know that there are two Twitter accounts (which we will henceforth call WASEpoints) @ey-sort and @ey-firsthalf that can be useful to us. We know @ey-sort takes an input data set, sorts it and outputs the result. We also know that @ey-firsthalf takes an input data set, and outputs the first “half” of the dataset. To perform a computation, we set up URI’s for the program listing, the input data and the output data, and kick off the computation with an appropriate message. (For those of you with dataflow or actor-based programming experience, WASE should look like a vague, but disreputable cousin.)
So let’s go through the message flow:
We’ll put our program listing at:
www.engineyard.com/top-quartile-sorted-list.json, (bit.ly/7yQK6) whose body contents are a JSON array:
["@ey-sort", "@ey-firsthalf", "@ey-firsthalf", "@engineyard"]
We put our input data here: www.engineyard.com/unsortedmegalist.json (bit.ly/3kl0xs)
And set up a location for our output data here: www.engineyard.com/top25percentofmymegalist.json (bit.ly/2uhGcl)
Or to summarize the bit.ly’s,
Program listing: bit.ly/7yQK6 (read with a http: GET)
Output location: bit.ly/2uhGcl (written with a http: PUT)
Input location: bit.ly/3kl0xs (read with a http: GET)
To perform the computation, we’d simply send the following twitter message from our @engineyard account: @ey-sort #wase, 0, bit.ly/7yQK6, 1256850843, bit.ly/2uhGcl, bit.ly/3kl0xs
So the message format of a WASTE message is: [WASEpoint], [WASE hashtag] [Program Counter (0 initially)], [Program listing URI], [Unix Timestamp], [Output URI] [,Input URI (optional)] [, Input URI 2 (optional)]
In the case of this computation, the message and computation sequence would look like:
- @engineyard sends: “@ey-sort #wase, 0, bit.ly/7yQK6, 1256850843, bit.ly/2uhGcl, bit.ly/3kl0xs”
…. @ey-sort reads the message from @engineyard in its twitter list and parses the message. First it GETs the program listing from bit.ly/7yQK6, GETS the input data set from bit.ly/3kl0xs, sorts it, PUTS the output to bit.ly/2uhGcl, then looks for the 0+1 WASEpoint in the program listing (@ey-sort) and then..
- @ey-sort sends: “@ey-firsthalf #wase, 1, bit.ly/7yQK6, 1256850875, bit.ly/2uhGcl”
…. @ey-firsthalf reads the message from @ey-sort in its twitter list and parses the message. First it GETs the program listing from bit.ly/7yQK6, GETS the input data set from bit.ly/2uhGcl, halves it, then PUTS the output to bit.ly/2uhGcl, looks for the 1+1 WASEpoint in the program listing (@ey-firsthalf) and then..
- @ey-firsthalf sends (to itself): “@ey-firsthalf #wase, 2, bit.ly/7yQK6, 1256850885, bit.ly/2uhGcl”
… etc. …
- @ey-firsthalf sends: “@engineyard #wase, 3, bit.ly/7yQK6, 1256850899, bit.ly/2uhGcl”
— finally @engineyard receives this message with the pointer to the final location of output data.
A few new things here. There’s a program counter that tells the WASEpoint where in the program listing the computation is, and there’s a Unix timestamp (could be useful for discarding messages that get held up in the twitterverse?). If no input URI is specified, then the WASEpoint should use the Output URI as both Input and Output locations. One restriction that we will enforce for the contest is that a WASEPOINT MAY NOT DECREMENT A PROGRAM COUNTER: to avoid infinite looping. (Update: And a WASEpoint must conserve the program listing and output URI’s from the input to the output message.)
Hey, maybe we should have some basic error handling. Hmm, let’s say @ey-firsthalf is expecting a standard JSON object but the input data fails to parse properly. Let’s have it send the following message:
- @ey-firsthalf sends: “@engineyard, #wase, -1, bit.ly/7yQK6, 1256850885, bit.ly/2uhGcl
So the error message structure is: [Update: first WASEpoint in program listing], [WASE hashtag], [Negative of Program Counter], [Program listing URI], [Unix Timestamp], [Output URI] [, Input URI (optional)] [, Input URI 2 (optional)]
Note that there are no type declarations in the message format because the only data-type supported by WASE are JSON objects and arrays.
What are Guidelines for the Contest?
Apart from the message format and data guidelines above — here are additional guidelines:
1. Each contestant may register no more than 5 WASE endpoints/twitter accounts. WASEpoints must be registered here to be eligible for use. Your WASEpoints must follow @eycontest. This is also where you should go to pick and choose good WASEpoints for constructing your app. Each contest entry must use a minimum of 10 WASEpoints from at least four separate contestants, where each WASEpoints performs functionally significant data operations. You must supply your own Output URI!
2. Please do not submit WASEpoints whose twitter accounts you do not own :-) We do not want to encourage the business of spamming Ashton Kutcher with mysterious messages. We will test each submitted WASEpoint with a DM to make sure they are legitimate.
3. Source code for your WASEpoint must be posted to a public repository (e.g codaset, github, sourceforge, kenai) for other contestants to inspect :-) If observed behavior deviates from the posted code (aka you have filed a prank WASEpoint), then your entry and all your WASEpoints will be removed from the registered list.
4. A WASEpoint must not store state, and may not rely on any state data other than the input data (of course, it’s easy to generate a private data set programmatically, but this will also be considered state). Trivial WASEpoints (e.g. identity) will be disqualified, although triviality is hard to define, you know it when you see it.
5. You must use bit.ly as your URL shortener (to make everyone’s job building parsers easier—and bit.ly has a http: interface.
7. UPDATE: Challenge calculations submissions must be in the form of a RETWEET to @engineyard of the first message in your WASE program listing from your home twitter account. The final WASEpoint in your program listing should be @eycontest. You must be following @engineyard with your home account in order to enter.
8. We must be able to reproduce your computation using your program listing and our own output URI.
9. We STRONGLY encourage people to write their WASEpoints in Ruby, but we’ll also accept Perl, Scala and Python. Although, be prepared for people avoiding your WASEpoint since the common denominator among people reading this blog is the fact that they know Ruby!
10. We may alter these guidelines along the way, based on your input and feedback, although the spirit and philosophy of them will remain.
Loading ...
Watch a Live Demo of Engine Yard AppCloud
The Engine Yard Newsletter
Maybe I'm just being stupid… But it doesn't look to me like there is enough information here to really give anyone a start. Can we have hints as to what the WASTEpoints might need to do? Or what kind of data they'll be dealing with, etc.? As it is I think it's just too wide open.
Well, we wanted to give you the basics first so you can get started on the parsing/http/twitter foundations without giving people who happened to read the post early, first dibs on interesting WASEpoints. But without getting into too specifics, one of the challenge computations will be math based, and one will be text based.
I'm sure you're aware of the connection with the WASTE name and Nullsoft – I'm sure they won't appreciate this; the last group that encouraged [open source] development of a networking app called WASTE — even as a joke was forced to remove it; http://www.nullsoft.com/free/waste/
Hi Lee. It seems as if the link you point to was about someone posting an unauthorized copy of WASTE as free software. WASTE seems to be in continuing development: http://bit.ly/2dcsAC. But since WASTE is in the same general area as the contest, we'll make things simpler and change the terminology to WASE "Worst App Server Ever"
@engineyard – yeah you're right, except that was Nullsoft's site – AOL own the rights to the name (afaik), and it was a Nullsoft release; anyway – happy hacking on the competition everyone!
Can you please clarify the message syntax? There are discrepancies between the format shown and the examples (especially regarding comma placement), and the error message format spec omits the hashtag placeholder.
Hi CBE — latest update should have cleaned up the last inconsistencies.
I assume the first example message should have bit.ly/2uhGcl instead of bit.ly/7yQK6 as the output URI.
You are correct — this is now corrected
Can you explain how the output POST should be accepted? is it multipart/form-data? is it application/x-www-form-urlencoded? or it is just the sole json string delivered in the POST body? It would be important for all WASTEpoints to be POSTing in the same way.
Urg. Reviewing oversight — the POSTs, should, of course, be PUTs — since the WASEpoint is asking to replace the existing resource at that location.
Urg. Reviewing oversight — the POSTs, should, of course, be PUTs — since the WASEpoint is asking to replace the existing resource at that location.
aw, so a scala solution really won't be accepted?
If we can get an engineyard developer willing to review scala entries we'll add it. Stay tuned.
ok John Crosby has volunteered to review any Scale winner! Scala is in!
One could easily craft their points such that they are the only ones that can use them by requiring their input be signed by their key. To use the other points it would be as simple as making a point that takes an input and a key, then signs it before dispatching it to the next in the list. If their twitter feed is private the key need not ever be broadcast to a different node nor included in their point's source code.
Also, as written one could easily solve it with their own point and spend the rest of the required performing useless operations such as using a point that gets the first half of an array, a point that gets the second half of an array, and one that combines 2 arrays into the same input.
Hi anonymous, we have to be able to verify the winning computation with our own kickoff message, so the first part won't be an issue. The second part yes might be an issue, but the computation will be difficult enough that people will need to use multiple points for the solution. We will also more definition on the scope of a WASE point.
It's stated explicitly that WASEpoints may not decrement the program counter – but can they increment it by more than 1?
Specifically, I'm thinking of a way WASEpoints could accept arguments. Let's say I've got a WASEpoint called @ey-add which, when called from position _n_ in the program listing, adds to the output the value of program listing position _n+1_, and then increments by 2 the program counter and sends control to the WASEpoint there.
In concrete terms, this example above would be used with a program listing like ["@ey-identity", "@ey-add", 10, "@ey-identity"], which would add 10 to whatever the input was.
Incrementing & forward jumps is ok.
So, PHP is a no go?
sorry, looks like no PHP
Also, how does each point decide what values belong in the [Output URI] [,Input URI (optional)] [, Input URI 2 (optional)] fields? In your example:
* @engineyard sends: “@ey-sort #wase, 0, bit.ly/7yQK6, 1256850843, bit.ly/2uhGcl, bit.ly/3kl0xs”
* @ey-sort sends: “@ey-firsthalf #wase, 1, bit.ly/7yQK6, 1256850875, bit.ly/2uhGcl”
How does @ey-sort know to strip off the optional input parameter? Does this mean that only the first WASEpoint can handle input or will the functionality for passing input URIs be handled by a third party WASEpoint? Just trying to come up with a convention everyone can follow.
Well after you parse the message:
If there are three bitly's, then they are output, input1, and input 2
If there are two bitly's, then they are output, input
If there is one bitly, then that bitly is both input and output
Each WASEpoint is free to decide whether it will accept multiple inputs, or emit a message with multiple inputs. You have to read the doc or code for a WASEpoint to decide whether you can use it or not.
Are there any rules against a WASEpoint generating new documents/bit.ly URIs for any of the IO files associated with a request? For example, say one developed a WASEpoint which redirects a request and generates a new [Program listing URI]? The request could then be tracked using the [Program Counter]. Would this be considered "storing state"? I can give a more in depth explanation after I register the end point :-)
Well I would guess this is a neat way of implementing a subroutine that assembles a chain of other wasepoints, but I guess you're going to try to hack the program counter so that you get a hint that an incoming message is actually a return from a sub-routine? This would not be kosher. I will add restrictions on PC modification that makes this more explicit.
After a bit of hacking I've created a basic framework for creating a Wase Endpoint Daemon.
Install and usage docs at: http://github.com/dougal/wase_endpoint/
Pull requests with features and bug fixes welcome.
So, according to http://www.engineyard.com/contests/wase there was only a single user who registered WASE points, do you plan on extending the competition?
Feedback seems to be the contest was too complex for people to grok easily. We're not extending the current contest – but plan to make the next one a bit easier to approach
I wish I would have seen this post weeks ago. I have so many ideas for the worst app server ever. I totally could have won this competition. Its my fault for now keeping up on your posts.