We are coming down the stretch to submit translations for the Apache CloudStack Documentation. The 4.1 release should be cut on March 22nd, we need the translations in before that. It is a huge task, we need your help, here is how:
I am trying bitcoin :) need a little donation: 18Wbz8mRkCd84mFud9QKn8HKJzVbxSYvWy
Saturday, February 23, 2013
Tuesday, February 12, 2013
SDN in CloudStack
Software Defined Networking (SDN) has seen a lot of uptick in momentum since VMware acquired Nicira last summer, three months after Google announced that they were using OpenFlow to optimize their internal backbone. In this post we look at "SDN" support in CloudStack.
First, let's try to define SDN in a short paragraph. It is, was it spells out to be (like the french adage: "c'est comme le Port-Salut c'est écrit dessus" :)) a way to configure the network using software. This means that the network definition (routing, switching), optimization (load-balancing, firewall, etc) becomes a software problem with SDN. If you have seen the wikipedia page and read other articles, you most likely have read that SDN decouples the control plan and the data plane. This is short for saying that the forwarding tables used in switches/routers will be controlled by a software/applications that can be remote. Part of the SDN landscape is OpenFlow. OpenFlow is a standard defined by the ONF that allows you to implement a SDN solution. It defines the protocol used by the control plane to send forwarding information (a.k.a flow rules) to network devices. However while some early SDN companies embraced and led the development of openflow (e.g BigSwitch), an SDN solution may not use the OpenFlow protocol. This leads to a key point of todays SDN solutions: SDN != OpenFlow.
The academic in me can't help but point out the GENI initiative, which aims to re-design the internet and start from a clean slate. SDN research happens on GENI but it is also seen as a way to instrument the network, isolate experiments and dynamically reconfigure the network. Something impossible (especially over wide are networks) before SDN. With FutureGrid and Grid5000 being testbeds for IaaS solutions, GENI is a real-life testbed for future networking solutions. I had a chance to work a little bit on GENI while at Clemson. We modified the NOX OpenFlow controller to integrate it with OpenNebula and provide Security Groups as well as Elastic IP functionality.
While virtualization was a key enabler of IaaS, virtual switches are key enablers of SDN based networks. Open Virtual Switch (OVS) is the leading virtual switch. OVS is now used in most private and public clouds, it replaces the standard linux bridge and is used to connect virtual machine network interfaces to the physical network. An OVS can be connected with an OpenFlow controller and receive flow rules from the controller. However it does not need to, an SDN solution could talk directly to OVS. The main issue being that a single openflow controller may not be fast enough to process all "control" decisions on a large networks. We would have to see a distributed OpenFlow controller to be able to reach extremely large scale (It might exist, I just have not found references for it). OVS can do many things among them: VLAN tagging, QoS, Generic Routing Encapsulation(GRE) and Stateless Transport Tunneling (STT) tunnels. To know more about the difference between GRE and STT see this blog by Bruce Davie and Andrew Lambeth.
So where does SDN help IaaS ? Anyone who has worked on networking of virtual machines (VM) knows how complex this can get. VMs from multiple tenants need to be isolated from each other, they have private IP addresses but may need to be accessed form the public internet, VMs can be migrated within a single broadcast domain but one may want to migrate across domains. VMs from multiple data centers may need to be in the same subnet and broadcast domain (Layer2) etc. Networking is really complex issue in IaaS. Even more so that it is hard to understand conceptually. One really needs to think in terms of logical networks and forget about the physical network (at least at a high level). To enable all these things and reach large scale we need to be able to control the network devices from the application layer. This is where a significant shift is happening. The application developers are now going to describe the network they need and provision it on-demand. Yet again SDN is Cloud. On-demand and Elasticity in the network thanks to SDN.
The previous snapshot on the left shows the CloudStack GUI when you are creating an advanced zone. You need to specify the type of isolation. Traditional isolation would be using VLANs, but you see two other types of isolations: GRE and STT. These are protocols used to create tunnels between OVS bridges (logical switches). The GRE isolation type will be used with what I call the "native SDN solution" in CloudStack. It is a SDN controller built-in the CloudStack code that creates GRE tunnels using OVS (Currently only supported with Xen, but KVM support should be in 4.1 if not 4.2 this summer). The wiki has an extensive functional specification titled OVS tunnel manager. The slides below are also a great presentation of this solution:
Choosing the STT isolation type will you guessed it use STT tunnels between all the virtual switches. This is currently only being used by the Nicira NVP plugin described in our documentation. Hugo Trippaers (@Spark404) from Schuberg Phillis is the author of the plugin, he recently presented about the integration at a Build a Cloud Day workshop and talked about the upcoming features in the CloudStack 4.1 release (KVM support and Layer 3 routing). See his slides below:
We are seeing two more SDN "solutions" being integrated in CloudStack. First is Big Virtual Switch from BigSwitch. Development is happening right now, and the commits made the 4.1 code freeze. So expect to see it in the 4.1 release at the end of March. Expect to see open source OpenFlow controllers being used with CloudStack this summer. The last one is Midonet from Midokura. While documentation has been posted on slideshare (see below and skip the first page if you don't read japanese), the commits have not yet been made. So look at 4.2 release for Midonet support in CloudStack.
This is only the beginning. While these solutions are used to provide multi-tenant isolation, we can bet that SDN will be used to provide load-balancing, elastic IPs, security groups, migration support, dynamic leasing and optimization of network. SDN brings network intelligence to your IaaS.
Friday, February 08, 2013
Build A Cloud Day, Ghent Feb1st Summary
Last friday we had a full day workshop (Build A Cloud Day) in Ghent, Belgium. It was co-located with Puppet Camp. All the logistics was planned by our friends from Inuits led by Kris Buytaert (@KrisBuytaert). The BACD had approximately 50 people in attendance throughout the day, with things winding down by 4pm, when people started heading to Brussels for Europe's biggest Open Source event: FOSDEM. We had an exciting day with terrific speakers that showed the complete range of the CloudStack ecosystem. Here are all the slides, enjoy.
I started the day with an introduction talk about CloudStack, bringing some high level vision about Clouds and how CloudStack fits in. I also presented the "Apache Way", what you can expect in terms of releases, and I also highlighted the main components of CloudStack. I introduced the rest of the agenda with a theme of covering the entire "stack" of Cloud computing and seeing how CloudStack is the core backend of it.
Hugo Trippaers (@Spark404) from Schuberg Phillis then presented the integration of the Nicira Private Gateway with CloudStack. Which brings an SDN solution (STT tunnels meshes for isolation of tenants) to ACS and complements the native SDN controller that can build meshes of GRE tunnels. Hugo is a committer and PMC member of CloudStack and the lead engineer for operation of the CloudStack private cloud at Schuberg Phillis.
Once we learned about SDN and advanced networking for multi-tenant isolation, we heard from Wido Den Hollander from PCExtreme. Wido is a committer and PMC member of CloudStack as well, and he is most notably the man behind the CloudStack/Ceph integration. Ceph has received a lot of attention for the last couple years as it offers an alternative to traditional parallel distributed file systems and builds a highly scalable object store and efficient storage for virtual machines with the so-called Rados Block Device (RBD). Wido had few slides but filled many questions on the board.
Three talks and the excitement rose learning about the latest features in networking and storage for CloudStack. A short lunch and we came back in to talk about the API exposed by CloudStack. When discussing APIs, everyone mentions standards. CloudStack API is not a standard and I doubt it will ever be (for good reasons). But what we have seen in the field is a de-facto standard in AWS APIs (which is supported in CloudStack to some extent) and some emerging standards from OGF and DMTF. While limited in scope they do provide some assurance against vendor lock-in. The talk by RedHat Oved Ourfali (@ovedou) was about CIMI, the DMTF standard. We were particularly interested to hear about CIMI to see how we could integrate it in CloudStack and potentially write a deltacloud driver.
Having learned about the backend networking and storage solutions, plus the API used to manage and access your cloud, it was now time to hear about an exciting use case: Spotify the on-line music service. Noa Resare (@blippie) has been active on the CloudStack mailing list and is now helping out with packaging. He presented us Spoticloud a private cloud built for their engineers/developers to allow them to "be developers". He gave us some great feedback with pointy details like adding a pod with the wrong id, and asked for features that we are working on, like removing the secondary storage VM. Also exciting was that Spotify is hiring a cloudstack engineer: http://spoti.fi/cloudstack
Next up was Brian Amedro (@brianamedro) from Activeeon, I really wanted Brian to talk because he represents a SaaS application making use of CloudStack. His company Activeeon has a very interesting application: ProActive, that offers a powerful IDE, a workflow engine and a resource manager to automate the parallelization of compute intensive tasks. It is used in the automotive industry, pharmacy, finance and other fields that have a need for long running computing analysis. Where CloudStack comes in is at the resource management layer, Brian integrated ProActive with CloudStack using the API, it allows ProActive to dynamically provision machines on cloud providers and run the workflows. Check the video as well.
To wrap up our day we had Charles Moulliard (@cmoulliard), (RedHat), an Apache committer on several projects and very active on the CloudStack mailing list. Charles is not shy to bring up issues with the API, DevCloud or thrid party clients like jclouds. Charles introduced us to Karaf and Fuse Fabric. By deploying Karaf on multiple cloud providers you can create a coordinated PaaS. All the providers coordinate with a Zookeeper instance, creating a "Fuse Fabric" that eases deployment of software on all cloud nodes. It gave me an idea to use FuseFabric to deploy a hadoop cluster in the cloud...
And that was it for BACD Ghent. If you were there you learned a ton :), if you missed it...well you can always come to the next one, but it may not come with Belgium beer :)
Monday, February 04, 2013
What a month !
What a month January 2013 was for Apache CloudStack mailing lists. I have barely recovered from what was our most intense month since CloudStack was donated to the Apache Software Foundation. As some of you know I have been keeping a close eye on our mailing lists, trying to extract valuable information for our community. This was triggered by Qingye Jiang study about CloudStack, OpenStack, Eucalyptus and Opennebula. Since all decisions happen on the mailing list at the ASF, it seemed like a very reasonable method to check the health of our community
I posted some early results in a prior post. With the whirlwind of emails in January I wanted to get a closer look at this past month alone.
We had a total of 5144 emails on the developers list and a total of 775 emails on the users list. 2036 of the 5144 where from JIRA.
First 5144 emails in a month is quite big to put it mildly. Second 2036 from JIRA is a lot. JIRA generates emails automatically, but what we have seen this month is an increase in conversation between developers happening within tickets. This is actually a mode of operation that is very familiar to the HDFS community for instance. I suspect that as we go through graduation, we may find that working tickets directly in JIRA will be very valuable and that JIRA emails will represent valid communication emails.
The developers list saw 175 unique contributors and the users list saw 145 unique contributors. I still need to clean up a few duplicates, but I don't expect it to represent more than 5% of the total number of contributors. That seems to me like a pretty robust community for a single month. If we compare both set of contributors we see that 76 are common to both list. This means that 48% of the contributors on the users list are not participating in the developers list, they are "true" users.
While affiliation of individual contributors is not recognized within the ASF, I can't help but mention that 57 domains were represented in the developers list versus 63 domains in the users lists. It would not be a stretch to associate an email domain with a company.
The graph below shows our daily number of contributors. First thing to notice is that thankfully we take a break on week-ends. The dips clearly show a lower number of contributors on saturday and sunday. The developers list approximately peaks at 60 contributors a day on Tuesday or Wednesday. The users list peaks at roughly 20 contributors a day mid-week.
In my previous post I also started a social network analysis. I did it again for this month alone. Below is the social graph of our developers list. To obtain this graph I group emails by thread and create a connection between two contributors if they exchange emails within a thread. Every time the same two contributors exchange an email within a thread, the "strength" of their connection increases. This is represented by the thickness of the line between all nodes. Also the size of the node/contributor matters. The more central a node is within our community, the bigger the node. Centrality is defined as being the shortest path between any two nodes in the network. The graph clearly shows that Chip Childers had the biggest centrality in our developers list in January. That means that for any two contributors to talk, the shortest path was through Chip (Sorry Chip, looks like your inbox is going to get even bigger). Other notables is Chiradeep Vittal, Alex Huang, David Nalley and Animesh Chaturvedi. Nodes that had a small number of connections were filtered out (but their contribution is very much appreciated. I only applied the filter to get a clearer picture). My boss would like to see a dynamic version of this graph, were we can see the evolution of the social network over time...:) Will see, I am just not sure how to automate that...
And below is the social graph of our users list. We see that Geoff Higginbottom from Shapeblue is the most influential node on the users list. Closely followed by Ahmad Emneina and Pranav Saxena from Citrix. A new comer is also showing up this month, Geoff's protege, Paul Angus from Shapeblue. It is great to see the leading CloudStack integrator taking leadership in helping users of Apache CloudStack. We also see a new user coming on board strongly: Nux also know as Lucian that I had the pleasure to meet in London. Interestingly we see that the thickness of the connections between the nodes is less than on the developers list and the network is larger. This just indicates that the most influential nodes answer a lot of questions from many different people but that they don't tend to exchange within a large number of threads. Also note that the traffic on the users list is much less than on the dev list.
And that's it for January, our highest traffic so far on the mailing lists. We see some strong and increasing participation every day, a clear technical leadership has emerged and some dedicated folks are helping out users.
Monday, January 21, 2013
Social Network Analysis of Apache CloudStack
Apache is about building communities of developers and users around an open source software. As such they can be analyzed with social networking tools to identify patterns of communications (communication networks), sub-communities within groups and bring up the most influential nodes. This type of analysis can be done over time using data from the Apache mailing lists.
Since I am a new Apache committer on CloudStack I wanted to have a look at the health of our community and thought a social network analysis (SNA) would do it. A little googling led me to this very nice research paper on SNA of the R mailing lists. I have not done all the analysis mentioned in the paper, especially the content based analysis but I wanted to post my early results.
Methodology: To get the graphs I grabbed the emails archive from Apache. I used Python to load the mbox files into single Mongo collections. I cleaned the data to avoid replications of senders as well as remove JIRA and Review Board entries. Then with a little bit of PyMongo I made the queries and build the graph with NetworkX. Finished up with the graph visualization and calculations using Gephi. Since there are thousands of emails and threads, there is still some work to pre-process the data, avoid duplicates and match individuals to multiple email addresses.
Using Gephi, I manipulated the graphs. Computing the degree of each node (i.e the number of direct connection to other nodes), the betweeness centrality (i.e a measure of how often a node serves as a bridge between the shortest path between two nodes. In other terms: is a node the best "proxy" between two other nodes ?), I then partitioned the graphs with a color code, trying to identify sub-communities. Finally for clarity I filtered nodes by degree. In CloudStack filtering is especially important since the list has grown quite large of late (This may actually be an indirect sign that it is time to split the dev list).
The graph of the cloudstack-dev mailing list can be seen below:
What stands out right away are the largest nodes, or the most influential nodes according to betweeness centrality. Chip, David, Edison, Chiradeep, Hugo, Wido, Alex are all members of the PMC and exhibit a high centrality. Prasanna and Rohit also exhibit a high centrality but are not currently in the PMC. Also of interest is that this graph is valid since CloudStack joined Apache in April 2012, we can identify contributors who are not active currently but once where and thus are still part of the overall communication network. The color code highlights communities within the community. There seems to be 4 to 5 sub-communities (green, blue, red, cyan, yellow, more investigation is necessary to give interesting meanings to these sub-communities. You will also notice that the edges have all the same thickness. This means that they have the same weight. Once two people exchange an email, an edge is drawn between the two nodes. If they communicate again, the edge is not modified. I will add edge weighting in a future study, this will show us "pathways" between community members and will also affect the influence of the nodes.
Update January 22nd: I added weight to the edges. In english this means that everytime two people communicated in a thread I increases their connectidness by 1. The graph below shows edges with a different thickness. Nodes and Edges were filtered to highlight the strongest connection. This clearly shows the "PMC" of ACS.
The graph of the cloudstack users mailing list can be seen below:
What stands out the most in this graph is that some of the PMC members are still influential (Chiradeep, David, Alex and Edison for instance). But new influential nodes have appeared. Most notably: mcirauqui, geoff.higginbottom and ahmad.emneina. Chip Childers is still present but his influence in this users community is much less. Based on this I am ready to campaign for mcirauqui and geoff to become committers, as they are clear contributors of the CloudStack users community :)
For comparison I checked the HDFS dev mailing list (note that this is fairly restrictive since Hadoop is a very large ecosystem with many mailing lists), followed the same process and obtained the following graph. Maybe the HDFS community can help me analyze it and see if this gives the right picture of their dev community :)
Tuesday, January 08, 2013
A Mahout Cluster across France and Luxembourg Using CloudStack
Early December I attended the Grid5000 Winter school held at the Ecole des Mines de Nantes (EMN) and organized by Adrien Lebre. Grid 5000 is
While in Nantes I met with Alexandra Carpen Amarie an INRIA research engineer who developed an amazing tool G5k campaign. G5k campaign allows any user of Grid5000 (G5k) to book nodes on the infrastructure, deploy machines with bare-metal provisioning and then deploy their favorite Cloud IaaS framework (currently CloudStack, Opennebula and Nimbus). G5k scripts are available via git, of interest are some Chef recipes. Heavily tailored for Alexandra's scripts they could be useful for the CloudStack community. Alexandra held a tutorial on deploying a IaaS and PaaS on G5k. For the tutorial the PaaS was Apache Mahout. Lets' not get into a discussion about whether Mahout is a PaaS or not, the point is that a IaaS can be used to deploy and managed a set of nodes that run Hadoop and Mahout on top, to provide a high level functionality. In this case machine learning algorithms to analyze large data-sets. It was attended by approximately 30 people. How did it work exactly?
One thing about G5k and I believe the French research computing community is that they are very prolific in creating great tools. Unfortunately few people know about them. The clusters of G5k are operated like regular batch processing clusters. A batch scheduler is used to access the nodes. Tool #1: OAR a PBS/MOAB like equivalent. Once the nodes are allocated they are provisioned using Tool #2: Kadeploy a crowbar like equivalent. Of great interest is Tool #3 KaVLAN, a tool to lease the VLANs configured on G5k. While not currently used in Alexandra's G5k campaign, I hope the Apache CloudStack community can start making use of it to test Advanced Zones.
The beauty of G5k campaign is that Alexandra's has hidden most of the complexity of the provisioning and configuring. You only need to write a YAML configuration file for your deployment. Specifying the sites and the number of nodes that you want to run on/at, for example:
deployment:
engine:
name: CloudStack
customization_type: multisiteChef
walltime: 2:00:00
sites:
rennes:
nodes: 10
subnet: slash_22=1
nancy:
nodes: 10
subnet: slash_22=1
sophia:
nodes: 10
subnet: slash_22=1
ssh:
user: username
Launch your campaign and wait for the nodes to be allocated, provisioned and then configured with your IaaS. Depending on the number of nodes requested, you could have a Cloud working within 20 minutes. You can then interact with it. In the case of CloudStack, using the API, Alexandra developed some wrappers to manage VMs. She did it before CloudMonkey came out and is not needed now even though still a great exercise. Couple days after the tutorial I asked Alexandra to deploy CloudStack across several sites. Within 24 hours I had those snapshots in my inbox. A 5 sites cloud, one basic zone per physical site and 97 physical hosts setup, 800 cores and 100 VMs deployed running Mahout. It took 30 minutes to deploy the nodes, one hour to configure the hosts in CloudStack (serially, Alexandra is working on adding parallel configuration in her tool). The 100 VMs were deployed in roughly 10 minutes. The 3 physical nodes missing were due to bare metal provisioning problems.
The snapshot below shows the infrastructure/zone view of the CloudStack deployment. Five basic zones were configured at Rennes, Toulouse, Nancy, Sophia and Luxembourg. All cities connected via the RENATER fiber network.
Below the infrastructure view, shows five zones, 97 hosts, 10 system VMs (console proxy and secondary storage) and 4 virtual routers (One router was not started at the time of the snapshot).
A small detail that you may have seen from the YAML configuration file is that this is all based on ssh. Access to G5k is via ssh keys and not via a PKI infrastructure. Having worked on TeraGrid. This was a nice surprised. Using PKI across different organizations and managing authorization can be extremely complex. This was a sore point in the TeraGrid. It is also used in the LHC grid with more success but still requires a lot of work. In G5k the user base is smaller and more trusted. SSH keys are distributed among sites using a basic NFS setup on the private RENATER network. This makes it easy for users to access all sites.
Looking ahead, the basic question one might ask is whether it makes sense to run mahout within virtual machines. In cases where the dataset is not very large the use of HDFS as a large scale distributed storage systems is not the issue. Rather the time spent running the machine learning algorithms is. There the cpu overhead of virtualization is the main performance factor. Alexandra pointed me to a paper she wrote on performance of map-reduce in the Cloud. I asked her to do some more analysis specific to a CloudStack based Cloud, stay tuned for the results :).
Tuesday, December 18, 2012
Thoughts from the Frontline
I am stealing the title of this blog from John Mauldin an analyst who has a great newsletter. For almost five years he has been very bearish on the economy, calling it a "muddle through" economy. He gets on my nerves lately because he calls for the end of the euro...
Anyway :) I dont' think he knows anything about Cloud Computing except maybe the value of certain tech companies and trends in the sector. What's the link with Cloud you are asking ? Well, the link is that while we were in full blown sub-prime mortgage crisis back in 2007/2008 (with Bear Stearns crumbling in March 2008) the world of Cloud Computing was in full innovation mode.
What really triggered this post is a talk by Marten Mickos from Eucalyptus at Linux Con Europe in Barcelona. In it, he called CloudStack, Eucalyptus, OpenNebula and OpenStack the four sisters. He called them the four sisters apparently in response to VMware prince charming blog. The point is not to start a polemic on this, but rather that I thought Marten missed at least two other ones: Nimbus and VCL.
Looking back at those six sisters, I wanted to see when they started. CloudStack started as VMops in early 2008, Nimbus was announced in March 2008, OpenNebula first release came in July 2008. Eucalyptus started also around April 2008 with a tech report filed at UCSB and a paper later published at the CCGRID conference in 2009. VCL started much earlier via the IBM Virtual Computing Initiative ( VCI ) out of Research Triangle Park. OpenStack first release -Austin- came in October 2010, but NASA nova came out of the Nebula project which was started in 2008 as well ( I believe through stimulus money, but I could not find confirmation).
So Bear Stearns was crumbling, portfolios were taking a huge hit but Cloud was being created.
What is also striking with those "six sisters" is that three came out of educational institutions, and two (Nimbus and Openstack nova) came out of a federal lab (even though NASA IT isn't really a research lab). Eucalyptus came out of Rich Wolski's group at UCSB, OpenNebula came out of Ignacio de Lorente and Ruben Montero's group at University of Madrid, Nimbus came out of Kate Keahey's group out of Argonne National Lab and VCL came out of North Carolina State University under Mladen Vouk's leadership. Only CloudStack stands out as being 100% enterprise driven.
The research on Cloud started much earlier but did not have the Cloud tag on it. It started as virtualization research and then application of virtualization to build distributed systems. I could list hundred of references on the subject but I will just point to my friends from University of Florida and tell you to read "A case for Grid Computing on Virtual Machines.". You may have to order it or find a free version. This paper is dated 2003, it lays the foundation for how virtualization is going to allow us to build dynamic homogenous systems over a distributed set of physical resources: in one word, the Cloud. Yes it is compute centric, but remember that the Amazon Dynamo paper only came out late in 2007, while it's true that the Google File System was presented in 2003 as well.
Eucalyptus, OpenNebula, Nimbus and VCL all published description of their systems around 2009. Take into account at least two/three years of work leading to the actual presentation of the paper.
VCL is now a top level Apache project, having graduated from incubation last June. VCL is almost unknown in the Cloud community despite running software, great features and vision. Nimbus is still going strong in academic research, specifically in support of science through the science clouds but has yet to see wide adoption in national research infrastructure projects like OSG. These two seem out of the discussion when we talk about enterprise solution and large open source communities. One can wonder though, what is the differentiator that makes similar ideas successful or not.
We are left with the four sisters then - CloudStack, Eucalyptus, OpenNebula, OpenStack - by alphabetical order :)
Rather oddly OpenNebula has not received any venture capital or large support from a hardware vendor (at least none that I know of). It is a shame for a great project and software that works very well. Perhaps the fact that it is located in Europe makes it harder. Europe is more conservative when it comes to innovation and taking risks. The remaining three for that matter are all located or originated in California. Surf and computers seem to do well together :)
From a momentum stand point, it is difficult to argue that OpenStack is not leading. It has gathered a lot of interest and support from key hardware vendors. In retrospect though it was the last project to start -~summer 2010-. In my opinion the reason why it has gained a lot of user/developer support is that there was a vacuum. CloudStack was not yet in Apache and cloud.com operated silently. OpenNebula was an academic European project and Eucalyptus was going through growing pains of migrating from an academic project to an enterprise solution. OpenStack seemed the only choice.
Now that CloudStack is incubating in Apache with tons of enterprise experience and focus, Eucalyptus is re-focusing and OpenNebula still going strong hoping for the euro to survive, users have plenty of choices. This is good news.
This is good news because instead of being forced into the only potential solution on the market. Users will be able to test, deploy, benchmark, stress test, integrate, develop, take part in the community, and then make an educated choice. In my view this is what was in the core of the Gartner note. When making such an important choice for your IT, you need to look at available choices and make an educated technical decision based on your local constraints.
deploy & test != twitter & prod
Friday, December 07, 2012
Using Transifex client to translate CloudStack docs
In a previous post I showed how to translate CloudStack documentation from the Web UI of the transifex website. Afterwards I sensed paranormal activities suggesting that developers would not use a Web UI and I better show how to do it from the good old command line. And yes, it is possible. Here we go:
The key is the almost magical transifex client. Install it, learn to use it, submit translations and the CloudStack committers will be able to pull your translations and build the documentation in the translated language.
Transifex can be a little overwhelming a first. At least I was. But the easiest way to get started is to actually start translating and not worry about setting up the projects. So jump right away to this part of the tx guide.
You will need to enter a few commands to get things setup:
- Get an account on transifex
- Get the transifex client:
sudo easy_install transifex-client - Initialize the project:
tx init - Point to the CloudStack Transifex project:
tx set --auto-remote https://www.transifex.com/projects/p/ACS_DOCS/ - Pull all the resources:
tx pull -s - Pull all the existing translations:
tx pull -a - Pick a resource and make a copy with the language code of your target language, for example:
cp translations/ACS_DOCS.pod-add/en.po translations/ACS_DOCS.pod-add/fr.po - Enter your translation in the msgstr lines:
vi translations/ACS_DOCS.pod-add/fr.po - Push your tranlation (replace the language code and the resource name with yours):
tx push -l fr -r ACS_DOCS.pod-add -t
That's it. You can go back to the Transifex project and you will see that your translation has been received. If you want to get a little more fancy, check out this guide, it contains a description of the push and pull commands. Get translating !
Parting thoughts: Using tx for translation only is actually relatively straightforward, once you have read the docs :) Setting a documentation project up for translation is a little more challenging. At CloudStack we use publican. Publican makes it easy to prepare a project for translation. Freezing the documentation, setting it up for translation and pulling everything back together before a release, might however, be a significant challenge.
Friday, November 30, 2012
Testing CloudStack 4.1 with DevCloud
DevCloud is the sandbox for CloudStack, it can be used in different mode:
- Full sandbox
- Development environment
- Cloud testbed
The full sandbox runs the CloudStack management server and acts as a host, using nested virtualization to start instances within DevCloud. This is great for testing and training.
The development environment is used when developers want to develop CloudStack, modify the source, build locally and deploy on a working setup. In this use case, they push their current development to DevCloud.
In the testbed version, you run the management server locally and use DevCloud as a host and NFS server. Of course multiple variations can be done: adding hosts, different storage backends (possibly) and adding different physical machines.
In the screencast below I demo the testbed setup, using a host-only interface, running the management server on OSX (mac book air) and starting tinylinux instances within DevCloud.
Below is the screencast that will say more than I can write in this blog
Testing CloudStack 4.1 with DevCloud from sebastien goasguen on Vimeo.
If you are interested in the first release of CloudStack: 4.0 Incubating, watch the screencast below, which shows you the testing procedure we followed to vote on the release.
CloudStack 4.0 testing procedure from sebastien goasguen on Vimeo.
Wednesday, November 28, 2012
Translating Apache CloudStack docs with Transifex
One of the big changes leading to the release of Apache CloudStack 4.0 (ACS) has been converting the existing documentation to docbook XML format. The resulting books are built in various output formats using publican. The entire ACS 4.0 documentation is now in docbook XML format.
All the documentation is located in the /docs directory in the ACS 4.0 source release. Folks who are not Apache committers yet, can write documentation in docbook and submit patches. A good way to contribute is to start working on existing docs bugs (I should follow my own advice sometimes :) ).
One of the benefits of moving to docbook XML and publican is the ability to produce documentation in multiple languages. Publican can define Portable Objects (PO) that build a framework for translation of the original documentation resources. To translate them, ACS uses transifex. While it is possible to provide translation from the command line using the transifex client, the easiest way to get started is to go through the on-line interface of transifex. The following screenshots walk you through this process (click on them to enlarge).
First, create an account on transifex and login
You will be presented with your brand new transifex dashboard, search for CloudStack projects
You will see several CloudStack related projects, pick the one that interests you the most. Most likely the core documentation project , but you can also contribute to the runbook or the UI.
Once you are on the project page, you will see the various languages that are being worked on, as well as the percentage of completion of the translations. There is work to do :). By clicking on resources you will access all the resources that are available for translation. If you have checked out the source code, you will recognize the names of the docbook XML files in /docs/en-US.
Pick a resource that you would like to translate, you will be presented with that resource page. A Add New Translation icon is present on the right, click on it. You will then be able to select the language you want to translate to. Proceed by selecting the translate online button.
A form to enter your translation will be displayed. It is broken up in strings that make up that resource. Enter your translation for each string, save and exit.
You will then be returned to the resource home page and you should see that your translation has been added (i.e your language should be present). Once you return to the project Dashboard if the language your translated to was new, you should see it in the dashboard.
Once a significant portion of the translation is completed, one of the committers ( I volunteer David Nalley :) because he has nothing else to do ) will pull your translations using the transifex client and will build the new book with publican. Let's get translating...
Monday, November 26, 2012
How CERNVM uses CloudStack
CERNVM inception: CERNVM started in 2007 and entered development in 2008, it has been 4 years in the making and the project is now wrapping up. The main concept was to create a virtual machine appliance that scientists could use on their Desktop. The appliance would have the latest analysis software needed to analyze the data coming out of the LHC. Building appliances for LHC has now become routine, and CERNVM comes in various flavors: VirtuaBox, VMware, Xen, KVM and Hyper-V. What strikes you when you download CERNVM is its small size (x100MB). This is made possible through the use of the CERNVM file system or CVMFS an http based read only file system optimized to deliver software applications to the appliances. CVMFS is now used widely throughout the LHC community. This file system is really a side artifact of the project but a very valuable one. I heard that a micro CERNVM is under development, it would be ~6MB in size and the entire software needed would be streamed via CVMFS.
Contextualization: With a very mature building procedure to build VM appliances and a highly performant file system to help distribute software in-time, what CERNVM was lacking was a way to provision hundreds/thousands of instances on large clusters. Around 2009/2010, the CERNVM team developed a batch edition that could be used for batch processing in clusters used for analysis of LHC data. This appliance was tested successfully on LXCLOUD. The biggest challenge for the appliances was the configuration or what is known in this community as contextualization (a term often attributed to the Nimbus project). Basically it amounts to configuring the instance based on what it is supposed to do. The team developed a very advanced contextualization system. To business data center people, this is for example a way to tell an instance which Puppet profile it is supposed to use, where the Puppet master is and what are the other instances that it should be aware of. In the case of EC2, the way to contextualize an image is to pass variables through the EC2_USER_DATA entry. With Opennebula this is also done using scripts placed in an ISO attached to the instance.
CERNVM Cloud: I had not talked to Predrag about the latest development in a while, and I was impressed by how far they had gone. They totally automatized the contextualization process, creating a web frontend CERNVM On-line that users can use to fill the parameter of the contextualization and creating profiles for instances, including specifying service offerings that tie to CloudStack service offerings. The kicker is that they tied it to a Gateway, the CERNVM Gateway (they got their branding right ! ), that allows users to start instances and define clusters of instances using specific profiles. While enterprise people think of VM profiles as database or webserver profiles, here the profiles tend to be Condor clusters for batch processing, MPI clusters for parallel computing, and PROOF clusters for LHC analysis. The combination makes up their Cloud. What I really like is that they moved up the stack, from building VM images to providing an end to end service for users. A one-click stop shop to write definitions of infrastructure and instantiate it. I think of it as a marketplace and a cloud management system all at once.
Internals: What technologists will love is how they combined a IaaS to their Gateway. They built on XMPP, the former chat and now more general messaging protocol. Predrag and I had talked about XMPP some time back and a former student had developed an entire framework (Kestrel) for batch processing in virtual machines. Couple of the many interesting aspects of XMPP, is its scalability, the ability to federate servers and the ability to communicate with agents that are behind NAT. Of course we can argue about XMPP vs AMQP, but this would have to be another post. What CERNVM ended up doing is creating a complete XMPP based framework to do Cloud Federation and communicate with Cloud IaaS APIs. An XMPP agent sits within the premise of a Cloud Provider and is responsible to start the instances using the Cloud platform API. The instances then contextualize themselves using a pairing mechanism that ties them back to the CERVNM Cloud. Their Cloud can be made of Public clouds, private clouds and even Desktops. Brilliant !
And CloudStack in all of this ?:
Final thoughts:I came back from meeting with the CERNVM team thinking it was worth skipping the Turkey. It gave me a few ideas and showed me again the power of CloudStack and Open Source Software:
- Installing a Private of Public Cloud should be a non-event. Ioannis feedback really showed me that this was the case with CloudStack and that the hard work put forth by the community really paid off. From the source build with maven to the hosting of convenience binaries, installing CloudStack is a straightforward process (at least in a basic Zone configuration).
- An ecosystem is developing around CloudStack, we already knew this from the various partners that are participating in the Apache community and contributing, but we are also starting to see end-to-end scenarios like the CERNVM Cloud. I encouraged them to open source their XMPP framework and enhance it to generalize the profile definition. They could easily make it a marketplace for Puppet or Chef recipes and use non-CERNVM images that would pair with these recipes for automatic configuration, that would be even more exciting.
- One other aspect that also struck me, is that they developed a very enterprise looking system. Leveraging the Bootstrap javascript framework and Django, their UI is extremely intuitive, interactive and pleasing. Something rather rare in R&D projects focused on batch processing.
- Finally, I proposed to identify CloudStack based providers that would be willing to install their Gateway agent and share their resources with them. Participating in a Cloud Federation to find the God particle is a worth while endeavor !
Thursday, November 15, 2012
Why I love CloudStack Devcloud ?
Apache CloudStack 4.0 incubating has been released couple weeks ago now. The testing procedure used by the community to vote on the release candidate included using the new CloudStack sandbox DevCloud.
It's not too late to go through the testing procedure, just follow the steps defined on the wiki page If you want a shortcut, just watch my screencast and enjoy the french accent.
You will see that one of the first things to do is to install DevCloud: A virtualbox appliance based on Ubuntu 12.04 and running a Xen Kernel. The CloudStack management server is pre-installed with a basic toy data center being configured. Thanks to nested virtualization this allows users to start virtual machine instances within the devcloud sandbox.
The key ingredient is nested virtualization. It is really nice for testing things but most likely less so if your are concerned with performance, even though I have not seen benchmarks on nested virtualization.
DevCloud was not created solely for the release candidate testing procedure. It was developed by Citrix developer Edison Su to act as a development environment, giving developers the ability to deploy their own cloud testbed on their own machine. Of course, this does not allow for testing of all CloudStack features especially advanced networking features like VPC and other SDN solutions, but it enables anyone to do some quick smoke tests...and learn CloudStack.
The most compelling fact in favor of DevCloud is that it lowers the barrier of entry for folks to give CloudStack a go: access a working GUI, start instances, take snapshots, access the systemVMs, play with a XCP host, and learn the API. I used it in a tutorial at Linuxcon EU in Barcelona last week. I handed out several USB sticks to attendees, they loaded the appliance in VirtualBox and where on the go with cloudStack. The FOSS event coming up in India at the end of the month will also feature a CloudStack tutorial using DevCloud.
I love DevCloud because it's a great teaching and training tool. It helps you to discover CloudStack with no investment. You can get going on your laptop. Hey, I run it on my mb Air and it runs super fast. It can also expand into a research tool if you want to get adventurous with networking. A few of the folks in the CloudStack community are now using it in a host-only mode running the CloudStack management server on the localhost, together with the mysql database needed, and using devcloud as a host on which to run the VMs. It leverages the host-only interface of VirtualBox. This means that additional Vbox instances will be able to communicate with DevCloud.
This also means that something like the Virtual Distributed Ethernet VDE switch could be used as well. This would open the door to make use of the VLAN features in CloudStack and link to other hosts.
VirtualBox is great, but a DevCloud image for other hypervisors would be nice as well. Assuming they support nested virtualization. The ACIS research group at the University of FLorida is working on creating a KVM appliance for DevCloud this would open yet more doors...
To wrap up, DevCloud is a terrific new tool for CloudStack, in my view it has three basic modes:
- Fully contained sandbox, to give CloudStack a try
- Dev environment, compiling the CloudStack source outside DevCloud and pushing the build inside of it for testing.
- A Dev/Research environment where you can link DevCloud to other resources in your computing infrastructure via advanced networking features.
DevCloud = Awesome !
Wednesday, November 07, 2012
High Performance Computing and CloudStack
I was asked the other day what was the connection between High Performance Computing (HPC) and Clouds, so I thought I would try to post an answer here. Let's first talk a little bit about HPC.
High Performance Computing is about finding every single flops and every single iops on the largest machine you can get your hands on, in order to run your code as fast as possible. It is about batch processing on as many cores as you can get, so you can solve the largest problem you are facing. For a while, supercomputers, were large shared memory machines but in the late nineties distributed memory systems appeared, they were cheaper and you could assemble lots of nodes to get hundreds of cpus. Today the Top500 supercomputers are ranked every 6 months, this ranking is the theater of great technological battle between countries, vendors, research labs and programmers. In the latest ranking, Sequoia the supercomputer from Lawrence Livermore National lab topped the ranking at 16.32 PetaFlop/s and 1,572,864 cores. Weather modeling, atomic weapons simulation, molecular dynamics, genomics and high energy physics are among those that benefit from HPC.
There is big difference however within HPC itself. It is the difference between applications that rely heavily on inter-process communication and need a low latency network for message passing, and applications where each process runs an independent task, the so-called embarrassingly parallel applications. (e.g Map-reduce is an example of how to express an embarrassingly parallel problem ). High Throughput Computing (HTC) defines the type of application where access to a large number of cores over a specific amount of time is needed. Protein folding popularized by the Folding@home project running on PS3 as well as desktops is a good example. Financial simulation such as stock price forecasting and portfolio analysis also tend to fall in that category due to their statistical nature. Graphics rendering for animated movies also falls under HTC. HTC cares less about performance -as measured by FLOPS- and more about productivity -as in processing lots of jobs-.
The HPC quest for performance seems totally antagonist with the IaaS layer of clouds, at least when one thinks of true HPC workload that consumes every flop. Virtualization, the key enabler of IaaS, introduces overhead, both in cpus and network latency, and thus has been deemed "evil" for true HPC. Despite directed I/O, pass thrus, VM pinning and other tuning possibilities to reduce the overhead of virtualization, you might think that this would be it, no connection between HPC and Clouds. However according to a recent academic study of hypervisor performance from a group at Indiana University, this may not be entirely true and it would also be forgetting the users and their specific workloads.
In november 2010 a new player in the Top 500 arrived: Amazon EC2. Amazon submitted a benchmark result which placed an EC2 cluster 233rd on the top 500 list. By June 2011, this cluster was down to rank 451. Yet it proved a point: that a Cloud based cluster could do High Performance computing, raking up 82.5 TFlops peak using VM instances and 10GigE network. In november 2011, Amazon followed with a new EC2 cluster ranked 42nd with 17,023 cores and 354 TFlops peak. This cluster is made of "Cluster Compute Eight Extra Large" instances with 16 cores, 60 GB of RAM and 10 GigE interconnect and now ranked 72nd. For $1000 per hour this allows users to get an on-demand HPC cluster that itself ranks in the top500. This is done on-demand and provides users with their personal cluster.
A leading HTC software company CycleComputing also demonstrated last April, the ability to provision a 50,000 cores cluster on all availability zones of AWS EC2. In such a setup, the user is operating in a HTC mode with little or no need for low latency networks and interprocess communication. Cloud resources seem to be able to fulfill both the traditional HPC need and the HTC need.
The high-end HPC users are few, they are the ones fighting for every bit of performance, but you also have the more "mundane" HPC user, the one who has a code that runs in parallel but who only needs a couple hundred cores, and the one who can tolerate a ~10% performance hit, especially if that means that he can run on hundreds of different machines across the world and thus reach a new scale in the number of tasks he can tackle. This normal HPC user tend to have an application that is embarrassingly parallel, expressed in a master worker paradigm where all the workers are independent. The workload may not be extremely optimized, it may wait for I/O quite a bit, it may be written in scripting languages, it is not a traditional HPC workload but it needs an HPC resource. This user wants on-demand/elastic HPC if his application and pricing permits and he needs to run his own operating system not the one imposed by the supercomputer operator. HPC as a Service if you wish. AWS has recognized this need and offered a service. The ease of use balances out the performance hit. For those users, the Cloud can help a lot.
What you do need, is a well optimized hypervisor, potentially operated without multi-tenancy for dedicated access to network cards or GPUs, or a quick way to re-image part of a larger cluster with bare-metal provisioning. You also need a data center orchestrator that can scale to tens of thousands of hosts and manage part of the data centers in a hybrid fashion. All features are present in CloudStack, which leads me to believe that it's only a matter of time before we see our first CloudStack based virtual cluster in the Top500 list. This would be an exciting time for Apache CloudStack.
If you are already using CloudStack for HPC use cases, I would love to hear about it.
CloudStack vs. Opennebula
Prior to joining Citrix's community team for Apache CloudStack, I had worked with OpenNebula (ONE) since 2009. Like CloudStack, ONE is an Apache-licensed IaaS solution, though they take different approaches.
To compare two systems like these, the best way would be to go through a deployment and then an evaluation period. We could also do a one by one comparison of features. (See the feature pages for CloudStack and OpenNebula, respectively.) You might also want to compare the CloudStack API and OpenNebula API.
Both systems have notable commonalities: A rich GUI, support for multiple hypervisors, and a philosophy to be hypervisor agnostic, an EC2 compatible interface as well as a native API, support for multiple zones and virtual data centers with relatively fine grained access control lists. CloudStack and OpenNebula also offer an image/template repository, a storage backend supporting NFS, GlusterFS, iSCSI, and LVM. Finally, both systems have a vibrant community contributing to the projects.
From a software perspective the big difference is that CloudStack is written in Java, while ONE has a C++ core with significant Ruby scripting as well as Bash script used for drivers. It does feel like CloudStack is more on the Dev side of DevOps while ONE is more on the Ops side, but this is very much a personal opinion.
ONE does boast a few interesting characteristics that I think CloudStack could benefit from: Support for Hybrid cloud (i.e. the ability to add an EC2 like site as a cloud bursting capability), a virtual appliance marketplace (i.e the ability for the community to share images between sites), as well as tools to test-drive the software without having to do a full install.
Some of this is underway already. The marketplace concept is being discussed on the CloudStack mailing list , while DevCloud was recently announced and is the perfect Sandbox to give CloudStack a try. What could be interesting is to set up a CloudStack Public cloud, for users to access and test the GUI and API.
What CloudStack brings, however, is terrific scalability which outperforms ONE at the moment. CloudStack also has the ability to do bare metal provisioning in addition to "traditional" virtual machine provisioning, and amazing network management with the ability to configure hardware networking devices like the Juniper SRX and Citrix Netscaler as well as new features like Nicira integration.
Though I've found some differences between CloudStack and OpenNebula, I think they're both great projects.
Subscribe to:
Posts (Atom)