Written by Wissem EL-Khlifi

Introduction

Being in the IT industry, we have been going through a tedious and long process of acquisition of the infrastructure (hardware software and licenses). Typical projects are delayed due to infrastructure delays in definition, contracts, support and licenses. Many projects have been cancelled due to infrastructure under estimation, non-ability of escalation, difficulties in upgrades and downgrades and high cost. Cloud Computing addresses all these difficulties and beyond. Teams now can focus in more important aspects of the projects: business, functionalities, applications and minor efforts in the infrastructure which is totally (if they wish) or partially managed by the cloud provider. Everything from storage Petabyte of data to high compute capacity is at your disposals. You have the software, hardware, everything about the infrastructure all on demand and at your disposals everywhere and anytime with great capacity of escalation and at lower cost. In this article, we will introduce the concept of Infrastructure as a code (IaaC). We will see how Amazon Web Services makes easier the DevOps role by providing the right tools for the IaaC. Later, we will give an example of how to automate the Oracle RDS creation using the AWS tools.

Cloud Services Automation

When the Cloud computing offers flexibility and easy management of the services, the project deadlines are made shorter and management team request fast deliveries of the whole stack (webservers, databases, DNS and so on). As a consequence, Operations team need to automate processes that once took months to complete and instead focus on their customer service rather than managing IT infrastructure. The infrastructure automation is to:

  • avoid human errors in Infrastructure creation
  • avoid repetitive, time consuming tasks.
  • offer version controlling

Amazon Web Services (AWS), the cloud leader in the market, provides application programming interface (API) to control AWS services and resources. AWS users can call the API using:

  • SDK (Python, Java, Ruby, JavaScript…)
  • AWS Management Console
  • AWS Command Line Interface

The API brings new concepts of the Infrastructure management and automation; the IaaC.

IaaC (Infrastructure as a code)

The idea of controlling and managing the Infrastructure using programming language is called Infrastructure as a code (IaaS). DevOps is an approach of bringing development and operations into a single profile. The goal is to assure quality of the delivered stack through automated unit plus integrations tests and validation processes. The repetitive and time consuming tasks need to be avoided by automation and this is the role of development operations (DevOps). As far the IaaC is involved, AWS offers a tool for DevOps called CloudFormation.

AWS CloudFormation

AWS CloudFormation Overview

AWS CloudFormation is a tool that gives DevOps an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion. The CloudFormation uses templates. The template is a description of the resources and services used by your Infrastructure. In the template the DevOps describes in a JSON or YAML format the Infrastructure components and how they are connected. No costs are added in using CloudFormation.

AWS CloudFormation templates

A template can be used repeatedly to create identical copies of the same stack (or to use as a foundation to start a new stack). A stack is a collection of resources that result from instantiating a template. You create a stack by supplying a template and any required parameters to AWS CloudFormation. Based on the template and any dependencies specified in it, AWS CloudFormation determines what AWS resources need to be created and in which order.

AWS CloudFormation template sections

Templates include the following sections. The Resources section is the only required section. Some sections in a template can be in any order. However, as you build your template, it might be helpful to use the logical ordering of the following list, as values in one section might refer to values from a previous section. The list gives a brief overview of each section.

  • Format Version (optional): Specifies the AWS CloudFormation template version.
     
  • Description (optional): A description of the template; It is good way to document what your template is about. This section must always follow the template format version section.
     
  • Metadata (optional): Objects that provide additional information about the template.
     
  • Parameters (optional): Specifies values that you can pass in to your template at runtime (when you create or update a stack). Example of parameters can be InstanceTypeParameter which indicates the EC2 instance type (t2. micro, m1. small, m1. large etc.)
     
  • Mappings (optional): A mapping of keys and associated values that you can use to specify conditional parameter values, similar to a lookup table. Find more on Mapping here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/mappings-section-structure.html
     
  • Conditions (optional): Defines conditions that control whether certain resources are created or whether certain resource properties are assigned a value during stack creation or update. For example, you could conditionally create a resource that depends on whether the stack is for a production or test environment. More on conditions can be found here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/conditions-section-structure.html
     
  • Resources (required): declare the AWS resources that you want as part of your stack, such as an Amazon EC2 instance or an RDS or an Amazon S3 bucket. You must declare each resource separately; however, you can specify multiple resources of the same type. If you declare multiple resources, separate them with commas. Example of resources in JSON look like below:

    "Resources" : {

"MyEC2Instance" : {

   "Type" : "AWS::EC2::Instance",

   "Properties" : {

     "ImageId" : "ami-2f731146"

   }

}

}

 

AWS CloudFormation Designer

AWS CloudFormation Designer is a graphic tool accessible from the AWS Management Console. It allows the creation, modification and view of AWS CloudFormation templates. The tool allows the DevOps to diagram AWS resources and their relations. The output of template is in JSON format. In this article we will walk through the steps for Oracle RDS creation using the AWS CloudFormation.

Oracle RDS Stack Creation from the AWS console

Amazon calls the creation of an infrastructure from a template a stack. A template is a blueprint or set of instructions to build a specific infrastructure. A stack is a self-contained component that contains properties and instructions needed to make a certain infrastructure operational. The Stack can be composed of webservers, databases, DNS, Load Balancer and the relationship between all of those components.

To make the example simple, we will focus only on a stack containing a single Oracle RDS instance in the cloud.

  • Login to the AWS console: Open the AWS Management Console and login to http://console.aws.amazon.com
  • Click on the CloudFormation link from the Console as shown from the image below:

  • There 3 options: we will choose to Design a template using the AWS CloudFormation Designer

  • Click on the resource Types and choose RDS- Oracle. You drag and drop the resource type to the designer and add the following properties as shown below: You notice we choose an EC2 small instance with an Oracle Enterprise Edition installed (Typically a 12c Oracle database) and with 10GB of storage available to the instance.

  • Click on validate template for syntax verification:

  • Once the template is validated. The template JSON looks like below: let’s save locally or on AWS S3 the JSON:




  • We keep the JSON example simple, but you may add to the template user data options like running a gather database statistic after Oracle RDS gets created or a run a simple SQL to verify the installation (example: query the DBA_REGISTRY to check the components installed are VALID).
  • Once the template is created and validated. We can now proceed with the stack creation. Let’s choose now to create a New Stack:

  • Select Choose a file locally and browse to the JSON template:

  • Give a stack name like testOracleRDS:

  • You may tag your resources in the stack for later identification. All resources in the stack will be tagged automatically with these tags. you can add stack notifications and IAM role for CloudFormation. This step is optional and you can leave all as default.

  • This step displays a summary of the stack before creation. You click on “Create” to start the stack creation process.

  • Once you submit the stack creation, the AWS console will show all the stack creation events starting by CREATE_IN_PROGRESS until CREATE_COMPLETE if all the steps complete successfully.

  • You can open a new AWS console window and monitor the Oracle RDS instance creation from the RDS entry.



  • After about 15 minutes to 20 minutes waiting, the Stack is completed successfully and the Oracle RDS instance is available as shown in the images below:



 

Oracle RDS Stack Creation from the AWS Cli

An alternative to the AWS CloudFormation Management console, you can use the AWS Client (AWS Cli) to create a stack from a JSON template. The command looks like below:

aws cloudformation create-stack --stack-name testOracleRDS --template-body file://myOracleRDS.template

We can later check the stack creation progress by running the AWS cli command:

aws cloudformation describe-stack-events --stack-name testOracleRDS

 

Stack deletion

To delete the stack, you need to select “Delete Stack” action from the AWS console.

Conclusion

In this article we have introduced the concept of Infrastructure as a code (IaaC); a concept to automate and program the cloud infrastructure creation for fast delivery. Later, we have explained the AWS CloudFormation tool for DevOps to automate the infrastructure creation. As an example, we show how to automate the Oracle RDS stack creation from templates using the AWS CloudFormation tool and AWS Client.