Cloud APIs allow engineers to control all layers of the cloud, from provisioning new VMs to altering specific software services. There are several competing cloud APIs in existence today. Learning how they came to be, and how they’re faring against one another, can give a deeper insight into the challenges presently faced by cloud computing in general. In this post we take a detailed look at one such API developed by the Open Grid Forum.
De Facto vs De Jure
There are two models for designing APIs: standardise from the code, or code from a standard. The first is a de facto approach, and the second is de jure.
Cloud APIs are no exception to this characterisation.
The leading de facto API is that of the Amazon Web Services (AWS) suite. This emerged as a proprietary interface to Amazon’s software. The first AWS service launched in 2004, though both S3 and EC2 were launched in 2006.
The leading de jure API is the Open Cloud Computing Interface or OCCI. Work on OCCI began at the Open Grid Forum in 2009. This effort was partly driven by the fact that numerous competitors to AWS were by now available, and that they all used divergent and usually proprietary APIs. This enabled OCCI to be created with the advantage of learning from those many existing APIs—especially EC2, ElasticHosts, FlexiScale, GoGrid, and Sun Cloud.
OCCI was primarily intended to target IaaS, encompassing the low-level infrastructure and especially concentrating on VM management. It was also designed to be extensible, however, and can therefore also work with PaaS and SaaS frameworks.
How OCCI Works
OCCI is a REST framework, which means it uses HTTP extensions to provide the extra metadata that it needs. These extensions are entirely compatible with existing HTTP clients, so for example you can issue commands to your machines using a command line tool such as curl. Whatever you use to make an HTTP request, the principle is similar for each operation supported by OCCI. The trivial example is just doing a GET on a VM path:
GET /us-east/webapp/vm01 HTTP/1.1
In which case you get an OVF file back with some extra header metadata. These headers contain information such as whether the VM is startable. When a system wants to return even richer metadata, it can do so using a variety of properties encapsulated as values within an OCCI extension header field:
X-OCCI-Attribute: occi.compute.cores=”1”
Other things that you can do through OCCI calls include creating or listing VMs, executing start and stop actions on a VM, creating a block storage volume, creating or listing security groups, and allocating floating IP addresses.
Popularity
Many corporations have contributed to the development of OCCI, including big names such as Rackspace, Oracle, and Cisco. In deployment terms, there are now OCCI interfaces to a range of software, including Apache CloudStack, OpenNebula, and OpenStack.
But how popular is OCCI? Well according to OpenNebula, OCCI had a 26% market share against 36% for AWS in 2013. An OpenStack user survey conducted later the same year was more bleak. EC2 was enabled on 30% (122) of the surveyed instances. OCCI, by contrast, was enabled on just 0.74% (3). OCCI has been available for OpenStack for a while now (since at least 2012), so the fact that relatively few people are using it in OpenStack can’t be due to recent implementation. It is hard to explain why so many more people seem to use OCCI in OpenNebula than they do in OpenStack. One possibility arises from the fact that OpenNebula is less enterprise facing than OpenStack, at least in part due to its European Commission research funding and consequent use in academia and High Performance Computing. This means that their users, which could include a large ex-EGEE contingent, may be more minded towards open standards.
And what about CloudStack? Here’s what David Nalley, a CloudStack committer, had to say about OCCI and a related competing standard, CIMI:
“There are really two problems with these standards. The first problem is that neither has massive adoption. Which in turn means that the number of libraries and third-party tools that support these APIs is pretty small. In many ways this defeats the purpose of using a single, widely adopted, standard. The second issue is that because of their wide applicability the APIs are pretty generic. Turning on or off a virtual machine is easy, but if you want to use the latest coolest feature of a specific IaaS platform it might not be exposed at all by CIMI or OCCI.”
Nalley’s fellow committer Sebastien Goasguen, who recently added OCCI capabilities to CloudStack, voiced much the same concerns.
One counter argument to the second claim, that standardisation focuses on the lowest common denominator, is that OCCI was actually designed as extensible from the outset, and can therefore be made to support any feature set. But the first claim, that of relatively low adoption, is hard to dispute. In 2009, when OCCI was launching, some were of the opinion that it could turn out to be the ubiquitous control model in cloud computing. That didn’t happen, and uptake has not matched that of AWS. Yet with a nice design, major backers, and plenty of available implementations, the reason for this slow OCCI uptake is harder to explain.
One possibility is that AWS is so pervasive that open standardisation is seen as redundant and counter-zeitgeist. Yet there are clearly inherent dangers to such an approach. Nalley, who is critical of OCCI, also criticises the AWS approach, saying that anybody who tracks the AWS API must also track Amazon’s peculiarities and whims because their API is neither generic nor vendor-neutral. And if you simply choose not to track Amazon, then you may run into “some potential problems with your fidelity to the API.”
Other areas of the marketplace may have their own indigenous de facto APIs. For example, AWS just built a plug-in for the VMware vCenter API set.
Alternatives
There are some potential alternatives to the AWS vs OCCI dichotomy. One is to use an adaptor to make OCCI compatible with AWS itself, translating OCCI calls into AWS calls. This has been available since 2012, but it’s not clear how widely used it is. One advantage would be that if you wanted to move your cloud operations away from AWS, you could do so more easily if your calls are already using OCCI, since then you would be able to switch to any other OCCI compatible platform. Of course, many alternative platforms also track the AWS API anyway, so this may not be as great an advantage as it first seems.
Another potential alternative is to use something like Apache Libcloud. This approach sits somewhere between the de jure approach of standards like OCCI, and the de facto approach of code like AWS. Apache Libcloud is a code-based generic interface to many cloud platforms. In other words, it’s like a standardisation effort written in code. There are also a few other alternatives in this area, such as Fog, and—as Darko Androcec points out—Deltacloud, Apache jclouds, and the Dasein Cloud API. These adapters allow you to build to them, acting as a cloud abstraction layer, when writing multi-cloud connectors that are independent of the middleware.
Conclusion
Cloud API standardisation attempts started out with the ambition to provide vendor-neutral interfaces to all layers of the cloud ecosystem, promoting interoperability and reuse. To a great extent these aims were met intrinsically by the results of their efforts, but this did not result in them becoming as popular as had been initially hoped or expected.
Though OCCI in particular has some adoption, the AWS API is still an industry leader and the de facto standard, and that position looks unlikely to undergo any radical change in the short term. Despite this, lessons from OCCI can be learned, and the AWS API increasingly contains features specific to AWS. If competition amongst cloud platforms such as AWS, GCE, and Azure increases in the future, the value that OCCI or other standardisation efforts provides could increase in proportion.
P.S. Do you use OCCI in your daily work, or perhaps a competing standard such as CIMI? Are you able to compare them with AWS, the CloudStack native API, or another well known cloud API? If so, please share your insights in a comment below! We’d love to hear from you.
