Setting up on-demand, configurable big data service on your cloud

October 19, 2015
Words by XLAB

GWDG, the IT provider for the University of Götingen and for the Max Planck Society and partner in MIKELANGELO project, wants to build a system that will enable resource providers to deploy an on-demand big data service on their cloud.

From a technical point of view there are a couple of hurdles to overcome when setting up such on-demand, configurable big data service, as described by Peter Chronz, the technical leader of MIKELANGELO at GWDG:

The first hurdle is the management of big data clusters in a cloud. Here OpenStack Sahara provides a good start, but more elaborate features are required to allow for an end-user driven deployment of big data clusters. Furthermore, since customers have diverse requirements for big data cluster, a high degree of flexibility needs to be allowed. This in turn leads to the requirements for end-users to configure their system by themselves, even without deep expertise in system administration. In addition, we expect that our customers will require a high degree of security, especially with regards to sensitive data. The last technical hurdle is one of efficiency. Although currently it is possible to run a big data cluster in a public cloud, the deployment will be inefficient. There are multiple reasons for this inefficiency, which in the end boil down to low I/O efficiency and poor cloud agility.

These issues lie at the core of MIKELANGELO’s motivation – to improve responsiveness, agility and security of the virtual infrastructure, using the lean guest operating system OSv and I/O-optimised hypervisor sKVM.

An elaboration on the technical hurdles, the concrete technical limitations of current clouds and virtual infrastructures in general and how to overcome them – gain I/O efficiency, faster bursting, better application management, and improved security – described in MIKELANGELO use cases report:

First Virtualised Big Data Use Case Implementation Strategy

 

---

MIKELANGELO is funded by the European Union’s flagship research program and innovation program Horizon 2020 and coordinated by XLAB. It involves a consortium of partners with research and technical expertise in Cloud and virtualized environments from Europe and worldwide; Huawei, IBM, Intel, Cloudius, HLRS, GWDG, Ben Gurion University and Pipistrel.



Mikelangelo-logo

Project No. 645402

© Copyright Beneficiaries of the MIKELANGELO Project





Social media

Keep up with what we do on our social media.

Keep up with XLAB on Facebook. Keep up with XLAB on Facebook.
Keep up with XLAB on Facebook. Keep up with XLAB on Facebook.
Keep up with XLAB on Facebook. Keep up with XLAB on Facebook.
Keep up with XLAB on Facebook. Keep up with XLAB on Facebook.