OpenStack - Openstack Tutorial

OpenStack is a free and open-source software platform for cloud computing, mostly deployed as an infrastructure-as-a-service (IaaS). The software platform consists of interrelated components that control diverse, multi-vendor hardware pools of processing, storage, and networking resources throughout a data center. Users either manage it through a web-based dashboard, through command-line tools, or through a RESTful API. OpenStack.org released it under the terms of the Apache License.

OpenStack began in 2010 as a joint project of Rackspace Hosting and NASA. As of 2016, it is managed by the OpenStack Foundation, a non-profit corporate entity established in September 2012 to promote OpenStack software and its community. More than 500 companies have joined the project.

The OpenStack community collaborates around a six-month, time-based release cycle with frequent development milestones. During the planning phase of each release, the community gathers for an OpenStack Design Summit to facilitate developer working sessions and to assemble plans.

Recent OpenStack Summits have taken place in Austin on 25â€"29 April 2016, and Barcelona on 25â€"28 October 2016.

Earlier OpenStack Summits have taken place also in Tokyo in October 2015, Vancouver in May 2015, and Paris in November 2014. The summit in May 2014 in Atlanta drew 4,500 attendees â€" a 50% increase from the Hong Kong summit six months earlier.

History

In July 2010, Rackspace Hosting and NASA jointly launched an open-source cloud-software initiative known as OpenStack. The OpenStack project intended to help organizations offer cloud-computing services running on standard hardware. The community's first official release, code-named Austin, appeared four months later, with plans to release regular updates of the software every few months. The early code came from NASA's Nebula platform as well as from Rackspace's Cloud Files platform.

In 2011, developers of the Ubuntu Linux distribution adopted OpenStack with an unsupported technology preview of the OpenStack "Bexar" release for Ubuntu 11.04 "Natty Narwhal". Ubuntu's sponsor Canonical then introduced full support for OpenStack clouds, starting with OpenStack's Cactus release.

OpenStack became available in Debian Sid from the Openstack "Cactus" release in 2011, and the first release of Debian including OpenStack was Debian 7.0 (code name "Wheezy"), including OpenStack 2012.1 (code name: "Essex").

In October 2011, SUSE announced the public preview of the industry's first fully configured OpenStack powered appliance based on the "Diablo" OpenStack release. In August 2012, SUSE announced its commercially supported enterprise OpenStack distribution based on the "Essex" release.

In 2012, Red Hat announced a preview of their OpenStack distribution, beginning with the "Essex" release. After another preview release, Red Hat introduced commercial support for OpenStack with the "Grizzly" release, in July 2013.

In July 2013, NASA released an internal audit citing lack of technical progress and other factors as the agency's primary reason for dropping out as an active developer of the project and instead focus on the use of public clouds. This report is contradicted in part by remarks made by Ames Research Center CIO, Ray Obrien.

In December 2013, Oracle announced it had joined OpenStack as a Sponsor and planned to bring OpenStack to Oracle Solaris, Oracle Linux, and many of its products. It followed by announcing Oracle OpenStack distributions for Oracle Solaris and for Oracle Linux using Icehouse on 24 September 2014.

In May 2014, HP announced HP Helion and released a preview of HP Helion OpenStack Community, beginning with the IceHouse release. HP has operated HP Helion Public Cloud on OpenStack since 2012.

At the 2014 Interop and Tech Field Day, software-defined networking was demonstrated by Avaya using Shortest path bridging and OpenStack as an automated campus, extending automation from the data center to the end device, removing manual provisioning from service delivery.

As of March 2015, NASA still makes use of OpenStack private cloud and has RFPs out for OpenStack public cloud support.

Components

OpenStack has a modular architecture with various code names for its components.

Compute (Nova)

OpenStack Compute (Nova) is a cloud computing fabric controller, which is the main part of an IaaS system. It is designed to manage and automate pools of computer resources and can work with widely available virtualization technologies, as well as bare metal and high-performance computing (HPC) configurations. KVM, VMware, and Xen are available choices for hypervisor technology (virtual machine monitor), together with Hyper-V and Linux container technology such as LXC.

It is written in Python and uses many external libraries such as Eventlet (for concurrent programming), Kombu (for AMQP communication), and SQLAlchemy (for database access). Compute's architecture is designed to scale horizontally on standard hardware with no proprietary hardware or software requirements and provide the ability to integrate with legacy systems and third-party technologies.

Due to its widespread integration into enterprise-level infrastructures, monitoring OpenStack performance in general, and Nova performance in particular, at scale has become an increasingly important issue. Monitoring end-to-end performance requires tracking metrics from Nova, Keystone, Neutron, Cinder, Swift and other services, in addition to monitoring RabbitMQ which is used by OpenStack services for message passing.

Networking (Neutron)

OpenStack Networking (Neutron) is a system for managing networks and IP addresses. OpenStack Networking ensures the network is not a bottleneck or limiting factor in a cloud deployment, and gives users self-service ability, even over network configurations.

OpenStack Networking provides networking models for different applications or user groups. Standard models include flat networks or VLANs that separate servers and traffic. OpenStack Networking manages IP addresses, allowing for dedicated static IP addresses or DHCP. Floating IP addresses let traffic be dynamically rerouted to any resources in the IT infrastructure, so users can redirect traffic during maintenance or in case of a failure.

Users can create their own networks, control traffic, and connect servers and devices to one or more networks. Administrators can use software-defined networking (SDN) technologies like OpenFlow to support high levels of multi-tenancy and massive scale. OpenStack networking provides an extension framework that can deploy and manage additional network servicesâ€"such as intrusion detection systems (IDS), load balancing, firewalls, and virtual private networks (VPN).

Block Storage (Cinder)

OpenStack Block Storage (Cinder) provides persistent block-level storage devices for use with OpenStack compute instances. The block storage system manages the creation, attaching and detaching of the block devices to servers. Block storage volumes are fully integrated into OpenStack Compute and the Dashboard allowing for cloud users to manage their own storage needs. In addition to local Linux server storage, it can use storage platforms including Ceph, CloudByte, Coraid, EMC (ScaleIO, VMAX, VNX and XtremIO), GlusterFS, Hitachi Data Systems, IBM Storage (IBM DS8000, Storwize family, SAN Volume Controller, XIV Storage System, and GPFS), Linux LIO, NetApp, Nexenta, Nimble Storage, Scality, SolidFire, HP (StoreVirtual and 3PAR StoreServ families) and Pure Storage. Block storage is appropriate for performance sensitive scenarios such as database storage, expandable file systems, or providing a server with access to raw block level storage. Snapshot management provides powerful functi onality for backing up data stored on block storage volumes. Snapshots can be restored or used to create a new block storage volume.

Identity (Keystone)

OpenStack Identity (Keystone) provides a central directory of users mapped to the OpenStack services they can access. It acts as a common authentication system across the cloud operating system and can integrate with existing backend directory services like LDAP. It supports multiple forms of authentication including standard username and password credentials, token-based systems and AWS-style (i.e. Amazon Web Services) logins. Additionally, the catalog provides a queryable list of all of the services deployed in an OpenStack cloud in a single registry. Users and third-party tools can programmatically determine which resources they can access.

Image (Glance)

OpenStack Image (Glance) provides discovery, registration, and delivery services for disk and server images. Stored images can be used as a template. It can also be used to store and catalog an unlimited number of backups. The Image Service can store disk and server images in a variety of back-ends, including Swift. The Image Service API provides a standard REST interface for querying information about disk images and lets clients stream the images to new servers.

Glance adds many enhancements to existing legacy infrastructures. For example, if integrated with VMware, Glance introduces advanced features to the vSphere family such as vMotion, high availability and dynamic resource scheduling (DRS). vMotion is the live migration of a running VM, from one physical server to another, without service interruption. Thus, it enables a dynamic and automated self-optimizing datacenter, allowing hardware maintenance for the underperforming servers without downtimes.

Other OpenStack modules that need to interact with Images, for example Heat, must communicate with the images metadata through Glance. Also, Nova can present information about the images, and configure a variation on an image to produce an instance. However, Glance is the only module that can add, delete, share, or duplicate images.

Object Storage (Swift)

OpenStack Object Storage (Swift) is a scalable redundant storage system. Objects and files are written to multiple disk drives spread throughout servers in the data center, with the OpenStack software responsible for ensuring data replication and integrity across the cluster. Storage clusters scale horizontally simply by adding new servers. Should a server or hard drive fail, OpenStack replicates its content from other active nodes to new locations in the cluster. Because OpenStack uses software logic to ensure data replication and distribution across different devices, inexpensive commodity hard drives and servers can be used.

In August 2009, Rackspace started the development of the precursor to OpenStack Object Storage, as a complete replacement for the Cloud Files product. The initial development team consisted of nine developers. SwiftStack, an object storage software company, is currently the leading developer for Swift with significant contributions from HP, Red Hat, NTT, NEC, IBM and more.

Dashboard (Horizon)

OpenStack Dashboard (Horizon) provides administrators and users with a graphical interface to access, provision, and automate deployment of cloud-based resources. The design accommodates third party products and services, such as billing, monitoring, and additional management tools. The dashboard is also brand-able for service providers and other commercial vendors who want to make use of it. The dashboard is one of several ways users can interact with OpenStack resources. Developers can automate access or build tools to manage resources using the native OpenStack API or the EC2 compatibility API.

Orchestration (Heat)

Heat is a service to orchestrate multiple composite cloud applications using templates, through both an OpenStack-native REST API and a CloudFormation-compatible Query API.

Workflow (Mistral)

Mistral is a service that manages workflows. User typically writes a workflow using workflow language based on YAML and uploads the workflow definition to Mistral via its REST API. Then user can start this workflow manually via the same API or configure a trigger to start the workflow on some event.

Telemetry (Ceilometer)

OpenStack Telemetry (Ceilometer) provides a Single Point Of Contact for billing systems, providing all the counters they need to establish customer billing, across all current and future OpenStack components. The delivery of counters is traceable and auditable, the counters must be easily extensible to support new projects, and agents doing data collections should be independent of the overall system.

Database (Trove)

Trove is a database-as-a-service provisioning relational and non-relational database engine.

Elastic Map Reduce (Sahara)

Sahara is a component to easily and rapidly provision Hadoop clusters. Users will specify several parameters like the Hadoop version number, the cluster topology type, node flavor details (defining disk space, CPU and RAM settings), and others. After a user provides all of the parameters, Sahara deploys the cluster in a few minutes. Sahara also provides means to scale a preexisting Hadoop cluster by adding and removing worker nodes on demand.

Bare Metal (Ironic)

Ironic is an OpenStack project that provisions bare metal machines instead of virtual machines. It was initially forked from the Nova Baremetal driver and has evolved into a separate project. It is best thought of as a bare-metal hypervisor API and a set of plugins that interact with the bare-metal hypervisors. By default, it will use PXE and IPMI in concert to provision and turn on and off machines, but Ironic supports and can be extended with vendor-specific plugins to implement additional functionality.

Messaging (Zaqar)

Zaqar is a multi-tenant cloud messaging service for Web developers. The service features a fully RESTful API, which developers can use to send messages between various components of their SaaS and mobile applications by using a variety of communication patterns. Underlying this API is an efficient messaging engine designed with scalability and security in mind. Other OpenStack components can integrate with Zaqar to surface events to end users and to communicate with guest agents that run in the "over-cloud" layer.

Shared File System (Manila)

OpenStack Shared File System (Manila) provides an open API to manage shares in a vendor agnostic framework. Standard primitives include ability to create, delete, and give/deny access to a share and can be used standalone or in a variety of different network environments. Commercial storage appliances from EMC, NetApp, HP, IBM, Oracle, Quobyte, and Hitachi Data Systems are supported as well as filesystem technologies such as Red Hat GlusterFS.

DNS (Designate)

Designate is a multi-tenant REST API for managing DNS. This component provides DNS as a Service and is compatible with many backend technologies, including PowerDNS and BIND. It doesn't provide a DNS service as such as its purpose is to interface with existing DNS servers to manage DNS zones on a per tenant basis.

Search (Searchlight)

Searchlight provides advanced and consistent search capabilities across various OpenStack cloud services. It accomplishes this by offloading user search queries from other OpenStack API servers by indexing their data into ElasticSearch. Searchlight is being integrated into Horizon and also provides a Command-line interface.

Key Manager (Barbican)

Barbican is a REST API designed for the secure storage, provisioning and management of secrets. It is aimed at being useful for all environments, including large ephemeral Clouds.

Historical names

Several OpenStack projects changed names due to trademark issues.

• Neutron was formerly known as Quantum.
• Sahara used to be called Savanna.
• Designate was previously known as Moniker.
• Trove was formerly known as RedDwarf.
• Zaqar was formerly known as Marconi.

Compatibility with other cloud APIs

OpenStack does not strive for compatibility with other clouds APIs. However, there is some amount of compatibility driven by various members of the OpenStack community for whom such things are important.

• The EC2 API project aims to provide compatibility with Amazon EC2
• The GCE API project aims to provide compatibility with Google Compute Engine

Governance

OpenStack is governed by a non-profit foundation and its board of directors, a technical committee and a user committee. The board of directors is made up of eight members from each of the eight platinum sponsors, eight members from the 24 defined maximum allowed Gold sponsors, and eight members elected by the Foundation individual members.

Users

OpenStack has a wide variety of users, from a number of different sectors. Notable users include:

Deployment models

As the OpenStack project has matured, vendors have pioneered multiple ways for customers to deploy OpenStack:

• OpenStack-based Public Cloud: A vendor provides a public cloud computing system based on the OpenStack project.
• On-premises distribution: In this model, a customer downloads and installs an OpenStack distribution within their internal network. See Distributions.
• Hosted OpenStack Private Cloud: A vendor hosts an OpenStack-based private cloud: including the underlying hardware and the OpenStack software.
• OpenStack-as-a-Service: A vendor hosts OpenStack management software (without any hardware) as a service. Customers sign up for the service and pair it with their internal servers, storage and networks to get a fully operational private cloud.
• Appliance based OpenStack: Nebula was a vendor that sold appliances that could be plugged into a network which spawned an OpenStack deployment.

Distributions

• Bright Computing
• Canonical
• HPE
• IBM
• Mirantis
• Oracle OpenStack for Oracle Linux, or O3L
• Oracle OpenStack for Oracle Solaris
• Red Hat
• Stratoscale
• SUSE
• VMware Integrated OpenStack (VIO)

Release history