Showing posts with label AWS. Show all posts
Showing posts with label AWS. Show all posts

Saturday, April 12, 2025

Use case study of CloudFormation and Terraform

April 12, 2025 0

 

Scope: CloudFormation is very powerful because it is developed and supported directly by AWS, but Terraform has a great community that always works at a fast pace to ensure new resources, and features are implemented for providers quickly.
Type: CloudFormation is a managed service by AWS, but Terraform has a CLI tool that can run from your workstation, a server, or a CI/CD system (such as Jenkins, GitHub Actions, etc.) or Terraform Cloud (a SaaS automation solution from HashiCorp).
License and support: CloudFormation is a native AWS service, and AWS Support plans cover it as well. Terraform is an enterprise product and an open source project. HashiCorp offers 24/7 support, but at the same time, the huge Terraform community and provider developers are always helpful.
Syntax/language: CloudFormation supports both JSON and YAML formats. Terraform uses HashiCorp Configuration Language (HCL), which is human-readable as well as machine-friendly.
Architecture: CloudFormation is an AWS-managed service to which you send/upload your templates for provisioning; on the other hand, Terraform is a decentralized system with which you can provision infrastructure from any workstation or server.
Modularization: In CloudFormation, nested stacks and cross-stack references can be used to achieve modularization, while Terraform is capable of creating reusable and reproducible modules.
User experience/ease of use: In contrast to CloudFormation, which is limited to AWS services, Terraform spans multiple cloud service providers such as AWS, Azure, and Google Cloud Platform, among others. This flexibility allows Terraform to provide a unified approach to managing cloud infrastructure across multiple providers, making it a popular choice for organizations that use more than one cloud provider.
Life cycle and state management: CloudFormation stores the state and manages it with the use of stacks. Terraform stores the state on disk in JSON format and allows you to use a remote state system, such as an AWS S3 bucket, that gives you the capability of tracking versions.
Import from existing infrastructure: It is possible to import resources into CloudFormation, but only a few resources are supported. It is possible to import all resources into Terraform state, but it does not generate configuration in the process; you need to handle that. But there are third-party tools that can generate configuration, too.
Verification steps: CloudFormation uses change sets to verify the required changes. Terraform has a powerful plan for identifying changes and allows you to verify your changes to existing infrastructure before applying them.
Rolling updates and rollbacks: CloudFormation automatically rolls back to the last working state. Terraform has no feature for rolling updates or rollbacks, but you can build a rollback system using a CI/CD system.
Multi-cloud management: CloudFormation is AWS-only, but Terraform supports multiple cloud providers and many more services.
Compliance integration: CloudFormation is built by AWS, so compliance is already assured, but for Terraform, you need to implement third-party tools yourself to achieve compliance.
Deployment type: CloudFormation has a built-in CI/CD system that takes care of everything concerning deployment and rollbacks. Terraform can be deployed from any system, but you need to build your CI/CD workflow or adopt a service that can fill the gaps.
Drift detection: Both tools have drift detection by default.
Cost: Using AWS CloudFormation does not incur any additional charges beyond the cost of the AWS resources that are created, such as Amazon EC2 instances or Elastic Load Balancing load balancers. In contrast, Terraform is an open source project that can be used free of charge. However, to obtain enterprise-level features such as CI/CD automation and state management, you may need to consider using additional services and systems provided by HashiCorp or third-party service providers. These additional services may come with their own costs.

Saturday, March 1, 2025

Creating AWS Load Balancer Controller under EKS in the AWS environment

March 01, 2025 0


 AWS Load Balancer Controller:

Architecture diagram


Associates an OIDC provider with your EKS cluster:

eksctl is a CLI tool for EKS cluster in AWS. We can able to map the existing OIDC provider into EKS cluster through below CLI command.

#eksctl utils associate-iam-oidc-provider --cluster test-demo-cluster  --approve --region us-east-2

Created an IAM role for the EKS cluster:

An Amazon EKS cluster IAM role is required for each cluster. Kubernetes clusters managed by Amazon EKS use this role to manage nodes and the legacy Cloud Provider uses this role to create load balancers with Elastic Load Balancing for services.

Creating the Amazon EKS cluster role:

You can use the AWS Management Console or the AWS CLI to create the cluster role.
AWS Management Console
Open the IAM console at https://console.aws.amazon.com/iam/.
Choose Roles, then Create role.
Under Trusted entity type, select AWS service.
From the Use cases for other AWS services dropdown list, choose EKS.
Choose EKS - Cluster for your use case, and then choose Next.
On the Add permissions tab, choose Next.
For Role name, enter a unique name for your role, such as eksClusterRole.
For Description, enter descriptive text such as Amazon EKS - Cluster role.
Choose Create role.

AWS CLI
a) Copy the following contents to a file named EKS-loadbalancer-policy.json.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "eks.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }

b) Create an IAM policy:

#aws iam create-role \
  --role-name AWSLoadBalancerControllerIAMPolicy  \
  --assume-role-policy-document file://"EKS-loadbalancer-policy.json"

Set up an IAM service account in an EKS cluster, allowing the AWS Load Balancer Controller to manage AWS Load Balancers on behalf of the Kubernetes cluster.

  • Creates a Kubernetes ServiceAccount named aws-load-balancer-controller.
  • Associates it with an IAM Role (AmazonEKSLoadBalancerControllerRole).
  • Attaches the AWSLoadBalancerControllerIAMPolicy.
  • Allows Kubernetes to use AWS IAM for authentication.

eksctl create iamserviceaccount \
  --cluster=alb-demo-cluster \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --attach-policy-arn=arn:aws:iam::<aws-account-id>:policy/AWSLoadBalancerControllerIAMPolicy \
  --region us-east-2 \
  --approve

Validated the controller:
#kubectl get deployment -n kube-system aws-load-balancer-controller

Step 2: Install AWS Load Balancer Controller:

Install the AWS Load Balancer Controller.
Installs the AWS Load Balancer Controller in the kube-system namespace.
Links it to the existing aws-load-balancer-controller service account.
#helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=alb-demo-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

Step 3: Validated the load balancer:

#kubectl get deployment -n kube-system aws-load-balancer-controller

Thursday, September 26, 2024

AWS - Auto Scaling

September 26, 2024 0

 


We can scale up or scale down the resources through Auto scaling in AWS.

Auto scaling is classified as 3 parts:

* Auto Scaling group
* Launch Configuration/Template
* Scaling Policies 

Auto Scaling group : We can define a minimum and maximum require resources through Auto scaling group.

Launch Configuration/Template - Defined a AMI and other require resource parameter for the EC2 instance.

Scaling Polices: It is classified as 3 types:

* Manual - We can update the minimum and maximum resource requirement by manually or through CLI,
* Scheduling : We an schedule the desire resource requirement through crontab.
* Dynamic - It will increase the resource based on metrics.




Wednesday, September 25, 2024

AWS - Load Balancer

September 25, 2024 0

 


Load balance is a method of distributing the network traffic equally across the connection/network pool that support an application.

AWS is provided a load balancing called ELB [Elastic Load balancer]

ELB is classified as below:

* Network Load balancer
* Application Load balancer
* Classic Load balancer 
* Gateway Load balancer

Network Load balancer is routing the traffic through TCP/IP [Layer 4] & Application Load balancer is routing the traffic through https/http [Layer 7]

* ELB should have a minimum 1 listener is in active state, so that it will listen the traffic and routing into target group.


Tuesday, September 24, 2024

AWS - VPC

September 24, 2024 0

 


Basic Networks:

IPv4 is connectless protocol that use multi packet routing to break data into smaller blocks to send across the internet.  IPv4 is a series of of four eight-bit-binary numbers separated by a decimal point. 

IPv4 has a different type of classes:

Class A - Address range [1 to 126] - Subnetmasking [255.0.0.0] - Used for large number of hosts
Class B - Address range [128 to 191] - Subnetmasking [255.255.0.0] - Used for medium size network
Class C - Address range [192 to 223] - Subnetmasking [255.255.255.0] - Used for local area network
Class D - Address range [224 to 239] - NA - Reserve for multi tasking
Class E - Address range [240 to 254] - NA - This is class is reserved for research and development purpose. 

The RFC1918 address is an IP address that is assigned by an enterprise organization to an Internal host, these IP address are used in private networks which is not available or reachable from internet.

10.0.0.0 - 10.255.255.255 [10/8 prefix]
172.16.0.0 - 172.31.255.255 [172.16/12 prefix]
192.168.0.0 - 192.168.255.255 [192.168/16 prefix]

We cannot use of first four ip address and last IP address in the network segments. It will be reserve to use for it.

Example:

Network segment of 172.31.0.0

172.31.0.1 - Reserved by AWS for the VPC router.
172.31.0.2 - Reserved by AWS for DNS server
172.31.0.3 - Reserved for the future use.
172.31.0.255 - Network broadcast address. We don't support the broadcast in a VPC, therefore we reserve this address.

Creating the VPC network:

Login into AWS console and navigate into VPC.

1)      Select the VPC only option.



We will get a more network option while selecting the VPC and more option. It will display the availability zone along with network CIDR.

2)      Select the IPv4 CIDR manual input in the IPv4 CIDR block section.



If there is an Amazon VPC IP address Manager (IPAM) address pool available in this region, you can get a CIDR from an IPAM pool. If you select an IPAM pool, the size of CIDR is limited by the allocation rules on the IPAM pool. The same will be applicable for IPv6 as well.

3)      Select a default zone, we can specific a different Tenant as well.



4)      Create a tag for grouping purpose and click on create VPC button.



We need to create a sub netmask according your requirement and mapped into VPC.

Click on subnet from the VPC dashboard.


Click on create subnet button and select the VPC which you want to create a sub netmask.



Define a subnet name and availability zone as per your requirement.



I can able to get 11 number of IP address as per my sub netmask and 5 IP address goes for reservation.


We need to create a internet gateway incase of access this machine from your system or outside world.

Creating an Internet Gateway:

Navigate into Internet gateway from left hand side of the VPC dashboard and clicked create Internet gateway button.





Sunday, September 8, 2024

AWS - Introduction

September 08, 2024 0

 


Amazon Web Services is a subsidiary of Amazon that provides on-demand cloud computing platform and API's to Individuals, Companies and Governments on a metered Pay-as-You-go basis.

AWS Global Infrastructure as of 2024:

The AWS Cloud has 108 Availability Zones and 34 launched Regions across the world.  They will plan to extend another 18 more Availability Zones and Six more Regions in Mexico, New Zealand,  the kingdom of Saudi Arabia, Thailand, Taiwan and the AWS European Sovereign Cloud.



Amazon cloud computing resources are hosted in multiple location across world wide.  These location are composed of AWS Regions, Availability Zones and local Zones.  

AWS Regions: Region is a logical name which represent some of the Geographic location.  Each AWS Region has multiple, isolated location knows as Availability Zones.

Local Zones: We can place resources such as compute & storage in multiple locations closer to end user. Local zones are designed to bring core services closer  to end users.

AWS has a multiple resource components for various purpose, we will walk through one by one now.

EC2 - Elastic compute cloud

EC2 provides scalable compute capacity in the AWS cloud. We can create a multiple virtual machines through EC2.  We can scale up or down the resources easily through EC2.

EC2 Instance Purchasing Options:

On-Demand:

* It is very expensive and very flexible purchase options

* We are charged only when instance is running (billed by hour)

* We can terminate or provision at any time.

Reserved:

* Allows us to purchase an instance for a specific time period.

* We can get a special discounts while purchasing it.

Spot:

* Amazon will sell the unused resources with lower price.  

* We can bid on an instances type and get those instance if bid match with our price or go below to our bid price.

* Spot price will be fluctuate based on supply and demand in the market.

The price of EC2 instance will vary based on Instance type, Region, EBS and storage resources.

Amazon Machine Image:

* A preconfigured a package image (ex. ISO) along with necessary software's which require to launch an EC2 instance.

AMI comes with 3 categories as below:

1) Community AMI
2) AWS Market place AMI
3) My AMI [User created image]

EC2 instance Family:

General purpose: It is combination of CPU and memory with common purpose depends upon the request[family type is t2, m4, m3]. Example of common websites, web application, micro service and code repos.

Compute Optimized: It will provide more compute power to host. It will require for high end customers [family type is c3, c4, cc2]. For example of web servers, batch processing and analytics.

Storage Optimized: It will provide a more IOPS while access the data from the storage[family type is d2, i2 and i3]. For example data warehousing and NoSQL

Memory Optimized: It will give a more random memory while access the application [family type is r3 and r4]. It is mainly used for Hadoop and SAP Hanna application.

Allocation the IP address to EC2 Instance:

Private IP : By default every EC2 instance will be provided with a private IP address.
Public IP: EC2 instance can be launched with or without public IP address.  It will allow the instance to communicate from Network.
Elastic IP: Static public IP address for the instance. It is chargeable.

Storage service:

EBS - Elastic block store - The data will access in block level. we can able to read or write concurrently.
EFS - Elastic File store - The data will access a serial level.
S3  - Simple storage service - The data will save as object. We can get a unique ID and endpoint after uploaded the data.

EC2 and EBS should be in same availability zone.