This article shows how we can change Hadoop data node storage on the fly without data loss. This will be achieved by integrating Hadoop with LVM(Logical Volume Manager).

What I’ll do
1.Configure Hadoop master with data node
2. Create an LVM partition.
3. Mount this partition to the data node directory
4. Now, we can resize the partition, and It’ll be reflected in Hadoop.

As simple as it is…

Configure Hadoop

Here, I have Configured my Hadoop Cluster on AWS EC2.

My Hadoop Master Config. is as Follows…

What Is Ansible?

• Ansible is an Automation Engine.
• It can automate cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs.
• It uses no agents and no additional custom security infrastructure.
• It uses the YAML Syntax.
• Ansible includes hundreds of modules to support a wide variety of integrations, including AWS, Cisco, Docker, Jenkins, Spuck, Windows and many more.

How Ansible is Helping Industries?

Here are some of the Cases Studies to understand it better..

1. Microsoft

Microsoft needed to address increasing complexity across their corporate network infrastructure — comprised of tens of thousands of endpoints — that connects Microsoft locations worldwide. “We have thousands of devices of…

What Is Docker?

Docker is an open-source project based on Linux containers. It Helps Creating Containers on the top of the Operating System.

A container provides operating-system-level virtualization. Containers are almost same as VMs. Both isolate an application and its dependencies into a self-contained unit that can run anywhere. They have private space for processing, can execute commands as root, have a private network interface and IP address, allow custom routes and IPtable rules, can mount file systems, and etc.

Installing Docker

Docker Community has An Awesome Installations Documentation. Here

Running Your Python Code On Docker

1. Download Docker Official Images for Python: Here

docker pull python:3

~ We can Specify version…

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

Prerequisites

• A 64-bit version of Windows XP or later.
• Admin rights to install software

Download AWS CLI v2 on Your System

Windows: Download_Link
Linux: Read_Here
Mac: Read_Here

Configure AWS CLI

You Need Your AWS Access_key and Secret Key to configure AWS_CLI.

What is access_key or secret_key?
Access keys consist of an access key ID and secret access key, which are used to sign programmatic requests that you make to AWS. …

What We’ll Do ??

We will create a web portal for our company with all the security as much as possible. We will be using WordPress software with a dedicated database server i.e. MySQL Server. This MySQL Server will run in my Private Subnet and WordPress in Public Subnet Attached with A NAT gateway. Only WordPress Site Should be Public To The World.

How We’ll Do ??

1. We’ll Create a VPC For Our Web Portal.
2. In that VPC we have to create 2 subnets:

-Public Subnet [ Accessible for Public World! ]

-Private Subnet [ Restricted for Public World! ]

3. Create a public-facing internet gateway and attach…

I always had this thought what if I can automate the infrastructure? as a CS engineer always try to make automate all the processes. Always and I love to work with AWS as my primary cloud service provider using AWS with Terraform. For Continuous Integration, I cant always log in to the AWS console and provisioning the servers. So I made an Infrastructure as code using Terraform.

What is Terraform?

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions. …

Netflix is one of the Biggest customer of AWS and it uses AWS for nearly all its computing and storage needs, including databases, analytics, recommendation engines, video transcoding, and more hundreds of functions that in total use more than 100,000 server instances on AWS.

Let See How Netflix Started Using AWS Services for growth and scalability.

How AWS Can Help Any Company or Startup?

Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform, offering over 175 fully-featured services from data centers globally.

For almost all Startups rapid growth and scalability are the keys to success. One of the biggest obstacles startups face is…

Each minute of every day on the Internet

➡Every second, on average, around 6,000 tweets are tweeted on Twitter, which corresponds to over 350,000 tweets sent per minute.

➡300 hours of video are uploaded to YouTube every minute! Almost 5 billion videos are watched on YouTube every single day.

➡300,000 photos and videos are shared on Instagram per minute.

➡Over 3.5 Billion Google searches are conducted worldwide each minute.

➡Every minute on Facebook: 510,000 comments are posted, 350,000 statuses are updated, and 250,000 photos are uploaded.

with a bit of Calculation, we can see the amount of data created on the internet each day. …