OCCI interface to CloudStack

CloudStack has its own API. Cloud wrappers like libcloud and jclouds work well with this native API, but CloudStack does not expose any standard API like OCCI and CIMI. We (Isaac Chiang really, I just tested and pointed him in the right direction) started working on a CloudStack backend for rOCCI using our CloudStack ruby gem. The choice of rOCCI was made due to the existence of an existing Opennebula backend and the adoption of OCCI in the European Grid Initiative Federated cloud testbed.

Let's get started with installing the rOCCI server, this work has not yet been merged upstream so you will need to work from Isaac Chiang's fork.

    git clone https://github.com/isaacchiang/rOCCI-server.git
    bundle install
    cd etc/backend
    cp cloudstack/cloudstack.json default.json

Edit the defautl.json file to contain the information about your CloudStack cloud (e.g apikey, secretkey, endpoint). Start the rOCCI server:

    bundle exec passenger start

The server should be running on http://0.0.0.0:3000 and run the tests:

    bundle exec rspec

This was tested with the CloudStack simulator and a basic zone configuration, help us test it in production clouds.

You can also try an OCCI client. Install the rOCCI client from Github:

    git clone https://github.com/gwdg/rOCCI-cli.git

    cd rOCCI-cli
    gem install bundler
    bundle install
    bundle exec rake test
    rake install

You will then be able to use the OCCI client:

    occi --help

Test it against the server that you are started previously. You will need a running CloudStack cloud. Either a production one or a dev instance using DevCloud. The credentials and the endpoint to this cloud will have been entered in `default.json` file that you created in the previous section. Try a couple OCCI client command:

    $ occi --endpoint http://0.0.0.0:3000/ --action list --resource os_tpl

    Os_tpl locations:
     os_tpl#6673855d-ce9b-4997-8613-6830de037a8f

    $ occi --endpoint http://0.0.0.0:3000/ --action list --resource resource_tpl

    Resource_tpl locations:
     resource_tpl##08ba0343-bd39-4bf0-9aab-4953694ae2b4
     resource_tpl##f78769bd-95ea-4139-ad9b-9dfc1c5cb673
     resource_tpl##0fd364a9-7e33-4375-9e10-bb861f7c6ee7

You will recognize the `uuid` from the templates and service offerings that you have created in CloudStack. To start an instance:

    $ occi --endpoint http://0.0.0.0:3000/ --action create --resource compute 
           --mixin os_tpl#6673855d-ce9b-4997-8613-6830de037a8f 
           --mixin resource_tpl#08ba0343-bd39-4bf0-9aab-4953694ae2b4
           --resource-title foobar

A handle on the resource created will be returned. That's it !

We will keep on improving this driver to provide a production quality OCCI interface to users who want to use a standard. In all fairness we will also work on a CIMI implementation. Hopefully some of the clouds in the EGI federated cloud will pick CloudStack and help us improve this OCCI interface. In CloudStack we aim to provide the interfaces that the users want and keep them up to date and of production quality so that users can depend on it.

About those Cloud APIs....

There has been a lot of discussions lately within the OpenStack community on the need for an AWS API interface to OpenStack nova. I followed the discussion from far via a few tweets, but I am of the opinion that any IaaS solution does need to expose an AWS interface. AWS is the leader in Cloud and has been since 2006 -yes that's seven years- Users are accustomed to it and the AWS API is the de-factor standard.

When Eucalyptus started, it's main goal was to become an AWS clone and in 2012 signed an agreement with Amazon to offer seamless AWS support in Eucalyptus. Opennebula has almost always offered an AWS bridge and CloudStack has too, even though in total disclosure the interface was broken in the Apache CloudStack 4.1 release. Thankfully the AWS interface is now fixed in 4.1.2 release and will also be in the upcoming 4.2 release. To avoid breaking this interface we are developing a jenkins pipeline which will test it using the Eucalyptus testing suite.

Opennebula recently ran a survey to determine where to best put its efforts in API development. The results where clear with 47% of respondents asking for better AWS compatibility. There are of course developing official standards from standard organizations, most notably OCCI from OGF and CIMI from DMTF. The opennebula survey seems to indicate a stronger demand for OCCI than CIMI, but IMHO this is due to historical reasons: Opennebula early efforts in being the first OCCI implementation and Opennebula user base especially within projects like HelixNebula.

CIMI was promising and probably still is but it will most likely face an up-hill battle since RedHat announced it's scaling back on supporting Apache DeltaCloud. I recently heard about a new CIMI implementation project for Stratuslab from some of my friends at Sixsq, it is interesting and fun because written in Clojure and I hope to see it used with Clostack to provide a CIMI interface to CloudStack. We may be couple weeks out :)

While AWS is the de-facto standard, I want to make sure that CloudStack offers choices for its users. If someone wants to use OCCI and CIMI or AWS or the native CloudStack API they should be able to. I will be at the CloudPlugfest Interoperability week in Madrid Sept 18-20 and I hope to demonstrate a brand new OCCI interface to CloudStack using rOCCI and CloudStack ruby gem. A CloudStack contributor from Taiwan has been working on it.

The main issue with all these "standard" interfaces is that they will never give you the complete API of a given IaaS implementation. They by nature provide the lowest common denominator. That roughly means that the user facing APIs could be standardized but the administrator API will always remain hidden and non-standard. In CloudStack for instance, there are over 300 API calls. While we can expose a compatible interface, this will always only cover a subset of the overall API. It also brings the question of all the other AWS services: EMR, Beanstalk, CloudFormation... Standardizing on those will be extremely difficult if not impossible.

So yes, we should expose an AWS compatible interface, but we should also have OCCI, CIMI and of course our native API. Making those bridges is not hard, what's hard is the implementation behind it.

All of this would leave us with Google Compute Engine (GCE), and I should be able to bring back some good news by the end of september, stay tuned !!!