Learnitguide.net - Learn Linux, DevOps and Cloud

Python Pandas Tutorial: Basics to Advanced Guide

2026-03-11T07:54:00.000-07:00

Before reading this post, make sure you have a good understanding of Python basics - variables, data types, loops, functions, and lists. You should also have basic knowledge of NumPy, since Pandas is built on top of it. If you have not covered NumPy yet, go through the NumPy tutorial post first and then come back here.

This tutorial works on both VS Code and Google Colab. NumPy and Pandas must be installed on your system before running any code in this post.

Pandas is one of the most widely used Python libraries in data science and data analysis. Whether you are cleaning data, analysing trends, preparing datasets for machine learning, or generating reports - Pandas is the tool that gets the job done.

In this post, we are going to cover Pandas from scratch - creating Series and DataFrames, loading real data, inspecting it, filtering, sorting, grouping, merging, and exporting - all with real code and real output. By the end of this post you will have a solid understanding of how Pandas works and how to use it in real projects.

You can also watch this tutorial in youtube.

What is Pandas?

Pandas stands for Panel Data. It is an open source Python library built for data manipulation and analysis. It was originally developed for financial data analysis and later adopted across the entire data science community.

The simplest way to understand Pandas is to think of it as Excel inside Python - but far more powerful. You can handle millions of rows, automate repetitive tasks, connect to databases, read from CSV and Excel files, and perform complex transformations - all in a few lines of code.

Installing and Importing Pandas

If you are using VS Code, open your terminal and run:

pip install pandas

If you are using Google Colab, Pandas is already available. No installation is needed.

At the top of every Python file or notebook, always import Pandas and NumPy like this:

import pandas as pd
import numpy as np

The alias pd is the universal convention for Pandas. Use it in every project.

Pandas Series

A Series is a one-dimensional labelled array. Think of it as a single column from a spreadsheet - with an index on the left and values on the right.

import pandas as pd

s = pd.Series([10, 20, 30, 40, 50])
print(s)

Output:

0    10
1    20
2    30
3    40
4    50
dtype: int64

You can create a Series with custom labels instead of default numeric index:

scores = pd.Series([85, 90, 78, 95], index=['Maths', 'Science', 'English', 'History'])
print(scores)

Output:

Maths      85
Science    90
English    78
History    95
dtype: int64

Accessing values from a Series:

print(scores['Science'])    # Access by label
print(scores[scores > 85])  # Boolean filtering

Output:

90

Science    90
History    95
dtype: int64

Creating a Series from a dictionary:

data = {'Monday': 200, 'Tuesday': 350, 'Wednesday': 300}
sales = pd.Series(data)
print(sales)

Output:

Monday       200
Tuesday      350
Wednesday    300
dtype: int64

Pandas DataFrame

A DataFrame is a two-dimensional table with rows and columns. This is the main data structure you will use in almost every Pandas project. Think of it as a full spreadsheet or a database table.

data = {
    'Name':   ['Alice', 'Bob', 'Charlie', 'Diana', 'Eve'],
    'Age':    [25, 30, 35, 28, 22],
    'City':   ['Mumbai', 'Delhi', 'Chennai', 'Pune', 'Hyderabad'],
    'Salary': [50000, 70000, 65000, 80000, 45000]
}

df = pd.DataFrame(data)
print(df)

Output:

      Name  Age       City  Salary
0    Alice   25     Mumbai   50000
1      Bob   30      Delhi   70000
2  Charlie   35    Chennai   65000
3    Diana   28       Pune   80000
4      Eve   22  Hyderabad   45000

Accessing a single column:

print(df['Name'])

Output:

0      Alice
1        Bob
2    Charlie
3      Diana
4        Eve
Name: Name, dtype: object

Accessing multiple columns:

print(df[['Name', 'Salary']])

Output:

      Name  Salary
0    Alice   50000
1      Bob   70000
2  Charlie   65000
3    Diana   80000
4      Eve   45000

Loading Data from Files

In real projects you will load data from external files rather than creating DataFrames manually.

# Load from CSV
df = pd.read_csv('sales_data.csv')

# Load from Excel
df = pd.read_excel('sales_data.xlsx')

# Load from JSON
df = pd.read_json('data.json')

The read_csv() function is the most commonly used. One line is all it takes to load an entire file into a DataFrame.

Inspecting Your Data

After loading a dataset, the first step is always to inspect it before making any changes.

df.head()       # First 5 rows
df.tail()       # Last 5 rows
df.shape        # (rows, columns)
df.columns      # List of column names
df.dtypes       # Data type of each column
df.info()       # Full summary
df.describe()   # Statistical summary

Running df.info() on the sample dataset:

df.info()

Output:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 5 entries, 0 to 4
Data columns (total 4 columns):
 #   Column  Non-Null Count  Dtype
---  ------  --------------  -----
 0   Name    5 non-null      object
 1   Age     5 non-null      int64
 2   City    5 non-null      object
 3   Salary  5 non-null      int64
dtypes: int64(2), object(2)

Running df.describe():

df.describe()

Output:

             Age        Salary
count   5.000000      5.000000
mean   28.000000  62000.000000
std     4.848       13600.000
min    22.000000  45000.000000
25%    25.000000  50000.000000
50%    28.000000  65000.000000
75%    30.000000  70000.000000
max    35.000000  80000.000000

Indexing with loc and iloc

Pandas provides two ways to access specific rows and columns.

iloc - access by position (row number, column number):

print(df.iloc[0])           # First row
print(df.iloc[2, 3])        # Row 2, Column 3
print(df.iloc[0:3, :])      # First 3 rows, all columns

Output:

Name      Alice
Age          25
City     Mumbai
Salary    50000
Name: 0, dtype: object

65000

      Name  Age     City  Salary
0    Alice   25   Mumbai   50000
1      Bob   30    Delhi   70000
2  Charlie   35  Chennai   65000

loc - access by label (row label and column name):

print(df.loc[0, 'Name'])
print(df.loc[1:3, ['Name', 'Salary']])

Output:

Alice

      Name  Salary
1      Bob   70000
2  Charlie   65000
3    Diana   80000

Filtering Data

Rows can be filtered based on conditions without writing any loop.

# Salary greater than 60000
high_earners = df[df['Salary'] > 60000]
print(high_earners)

Output:

      Name  Age     City  Salary
1      Bob   30    Delhi   70000
2  Charlie   35  Chennai   65000
3    Diana   28     Pune   80000

Multiple conditions using & for AND and | for OR:

# Salary > 60000 AND Age < 32
filtered = df[(df['Salary'] > 60000) & (df['Age'] < 32)]
print(filtered)

Output:

    Name  Age   City  Salary
1    Bob   30  Delhi   70000
3  Diana   28   Pune   80000

Using isin() to filter by a list of values:

selected = df[df['City'].isin(['Mumbai', 'Pune'])]
print(selected)

Output:

    Name  Age    City  Salary
0  Alice   25  Mumbai   50000
3  Diana   28    Pune   80000

Using str.contains() for text matching:

result = df[df['Name'].str.contains('a', case=False)]
print(result)

Output:

      Name  Age     City  Salary
0    Alice   25   Mumbai   50000
2  Charlie   35  Chennai   65000
3    Diana   28     Pune   80000

Adding, Renaming and Dropping Columns

Adding new columns to a DataFrame:

df['Tax'] = df['Salary'] * 0.10
df['Net Salary'] = df['Salary'] - df['Tax']
print(df[['Name', 'Salary', 'Tax', 'Net Salary']])

Output:

      Name  Salary     Tax  Net Salary
0    Alice   50000  5000.0     45000.0
1      Bob   70000  7000.0     63000.0
2  Charlie   65000  6500.0     58500.0
3    Diana   80000  8000.0     72000.0
4      Eve   45000  4500.0     40500.0

Renaming columns:

df.rename(columns={'Name': 'Employee', 'Salary': 'Gross Salary'}, inplace=True)
print(df.columns)

Dropping a column:

df.drop(columns=['Tax'], inplace=True)

Dropping a row by index:

df.drop(index=2, inplace=True)

Sorting Data

# Sort by Salary ascending
df_sorted = df.sort_values('Salary')
print(df_sorted)

# Sort by Salary descending
df_sorted_desc = df.sort_values('Salary', ascending=False)
print(df_sorted_desc)

# Sort by multiple columns
df_multi = df.sort_values(['City', 'Salary'], ascending=[True, False])
print(df_multi)

# Reset index after sorting
df_sorted = df.sort_values('Salary').reset_index(drop=True)
print(df_sorted)

GroupBy Operations

GroupBy allows you to split data into groups, apply a function to each group, and combine the results.

data = {
    'Department': ['HR', 'Tech', 'HR', 'Tech', 'Finance', 'Finance', 'Tech'],
    'Employee':   ['Alice', 'Bob', 'Charlie', 'Diana', 'Eve', 'Frank', 'Grace'],
    'Salary':     [50000, 90000, 55000, 95000, 70000, 72000, 85000],
    'Experience': [3, 7, 4, 9, 5, 6, 8]
}

df3 = pd.DataFrame(data)

# Average salary by department
print(df3.groupby('Department')['Salary'].mean())

Output:

Department
Finance    71000.0
HR         52500.0
Tech       90000.0
Name: Salary, dtype: float64

Multiple aggregations at once using agg():

result = df3.groupby('Department').agg(
    Avg_Salary=('Salary', 'mean'),
    Max_Salary=('Salary', 'max'),
    Total_Employees=('Employee', 'count')
)
print(result)

Output:

            Avg_Salary  Max_Salary  Total_Employees
Department
Finance      71000.0       72000             2
HR           52500.0       55000             2
Tech         90000.0       95000             3

Merging and Joining DataFrames

In real projects data is often spread across multiple tables. Pandas provides merge() to combine them - similar to SQL joins.

employees = pd.DataFrame({
    'EmpID':  [1, 2, 3, 4],
    'Name':   ['Alice', 'Bob', 'Charlie', 'Diana'],
    'DeptID': [101, 102, 101, 103]
})

departments = pd.DataFrame({
    'DeptID':   [101, 102, 103],
    'DeptName': ['HR', 'Tech', 'Finance']
})

# Inner join
merged = pd.merge(employees, departments, on='DeptID', how='inner')
print(merged)

Output:

   EmpID     Name  DeptID DeptName
0      1    Alice     101       HR
1      3  Charlie     101       HR
2      2      Bob     102     Tech
3      4    Diana     103  Finance

Other join types:

pd.merge(employees, departments, on='DeptID', how='left')   # Left join
pd.merge(employees, departments, on='DeptID', how='right')  # Right join
pd.merge(employees, departments, on='DeptID', how='outer')  # Outer join

Stacking DataFrames vertically using concat():

df_new = pd.DataFrame({
    'EmpID': [5, 6],
    'Name':  ['Eve', 'Frank'],
    'DeptID': [102, 101]
})

df_combined = pd.concat([employees, df_new], ignore_index=True)
print(df_combined)

Output:

   EmpID     Name  DeptID
0      1    Alice     101
1      2      Bob     102
2      3  Charlie     101
3      4    Diana     103
4      5      Eve     102
5      6    Frank     101

Apply Function

The apply() function lets you apply any custom logic to a column row by row.

def salary_category(sal):
    if sal < 55000:
        return 'Low'
    elif sal < 75000:
        return 'Medium'
    else:
        return 'High'

df3['Salary_Category'] = df3['Salary'].apply(salary_category)
print(df3[['Employee', 'Salary', 'Salary_Category']])

Output:

  Employee  Salary Salary_Category
0    Alice   50000             Low
1      Bob   90000            High
2  Charlie   55000          Medium
3    Diana   95000            High
4      Eve   70000          Medium
5    Frank   72000          Medium
6    Grace   85000            High

Using a lambda function for quick one-line operations:

df3['Salary_in_Lakhs'] = df3['Salary'].apply(lambda x: round(x / 100000, 2))
print(df3[['Employee', 'Salary', 'Salary_in_Lakhs']])

Output:

  Employee  Salary  Salary_in_Lakhs
0    Alice   50000             0.50
1      Bob   90000             0.90
2  Charlie   55000             0.55
3    Diana   95000             0.95
4      Eve   70000             0.70
5    Frank   72000             0.72
6    Grace   85000             0.85

Pivot Tables

data = {
    'Region':   ['North', 'South', 'North', 'East', 'South', 'East'],
    'Product':  ['A', 'B', 'A', 'B', 'A', 'A'],
    'Sales':    [200, 150, 300, 250, 180, 220],
    'Quantity': [20, 15, 30, 25, 18, 22]
}

df4 = pd.DataFrame(data)

pivot = pd.pivot_table(df4,
                       values='Sales',
                       index='Region',
                       columns='Product',
                       aggfunc='sum',
                       fill_value=0)
print(pivot)

Output:

Product  A    B
Region
East   220  250
North  500    0
South  180  150

Exporting Data

# Export to CSV
df.to_csv('output.csv', index=False)

# Export to Excel
df.to_excel('output.xlsx', index=False)

# Export to JSON
df.to_json('output.json', orient='records')

Setting index=False prevents the row index from being written into the output file. This is almost always the preferred option.

Quick Command Reference

pd.Series() - Create a Series
pd.DataFrame() - Create a DataFrame
pd.read_csv() - Load CSV file
df.head() - First 5 rows
df.tail() - Last 5 rows
df.info() - Column info and data types
df.describe() - Statistical summary
df.shape - Dimensions of DataFrame
df.dtypes - Data type of each column
df.iloc[] - Access by position
df.loc[] - Access by label
df[condition] - Filter rows
df.isin() - Filter by list of values
df.sort_values() - Sort rows
df.groupby() - Group and aggregate
df.agg() - Multiple aggregations
pd.merge() - Join DataFrames
pd.concat() - Stack DataFrames
df.apply() - Apply custom function
pd.pivot_table() - Create pivot table
df.to_csv() - Export to CSV
df.to_excel() - Export to Excel
df.rename() - Rename columns
df.drop() - Drop rows or columns
df.reset_index() - Reset row index

Key Takeaways

Pandas is built on NumPy - understanding NumPy first makes Pandas much easier to learn
Series is a single column, DataFrame is a full table - these are the two core data structures
Always inspect your data with head(), info(), and describe() before making any changes
Use loc for label-based access and iloc for position-based access
GroupBy and merge are the most powerful tools for real-world data analysis
Use apply() to run any custom logic across rows or columns without writing a loop

All the code examples in this post are also covered in detail in the YouTube video. The link is available at the top of this page. If you have any questions, drop them in the comments section below.

Python NumPy Tutorial: Basics to Advanced Guide

2026-03-11T07:41:00.000-07:00

Before reading this post, make sure you have basic knowledge of Python programming. You should be comfortable with variables, data types, loops, and functions. If you are new to Python, go through a Python basics tutorial first and then come back to this post.

You should also have Python installed on your system. This tutorial works on both VS Code and Google Colab.

If you are working with data in Python, sooner or later you will come across NumPy. Almost every data science and machine learning library in Python - Pandas, TensorFlow, PyTorch, Scikit-learn - is built on top of NumPy. So understanding NumPy properly gives you a very strong foundation for everything else that comes after it.

In this post, we are going to cover NumPy from the basics all the way to advanced concepts - with real code and real output at every step. By the end of this post you will know how to create arrays, perform operations, reshape data, and use advanced techniques like broadcasting and boolean masking.

You can also watch this tutorial in youtube.

What is NumPy?

NumPy stands for Numerical Python. It is an open source library built specifically for numerical computation in Python. The core feature of NumPy is the ndarray - an N-dimensional array - which allows you to store and operate on large amounts of numerical data very efficiently.

The biggest difference between a Python list and a NumPy array is speed and efficiency. A Python list can hold mixed data types - strings, numbers, objects - all in one list. Because of this, Python has to check the type of every element before performing any operation. That takes time.

A NumPy array holds only one data type. So operations are performed directly without any type checking. This makes NumPy significantly faster than regular Python lists for numerical work - sometimes 50 times faster on large datasets.

Installing NumPy

If you are using VS Code, open your terminal and run the following command:

pip install numpy

If you are using Google Colab, NumPy is already installed. You can directly import it.

To import NumPy in your Python file or notebook, always use this line:

import numpy as np

The alias np is a universal convention. Every data scientist and machine learning engineer in the world uses np for NumPy. So follow this same convention in all your projects.

Creating NumPy Arrays

The most basic thing you can do in NumPy is create an array. Let us start with a simple one-dimensional array:

import numpy as np

a = np.array([10, 20, 30, 40, 50])
print(a)

Output:

[10 20 30 40 50]

A two-dimensional array - which is like a table with rows and columns - is created like this:

b = np.array([[1, 2, 3], [4, 5, 6]])
print(b)

Output:

[[1 2 3]
 [4 5 6]]

NumPy also provides shortcut functions to create arrays without typing every value manually:

# Array of all zeros
print(np.zeros((3, 3)))

# Array of all ones
print(np.ones((2, 4)))

# Range of numbers with a step
print(np.arange(0, 10, 2))

# Evenly spaced numbers between two values
print(np.linspace(0, 1, 5))

# Random values between 0 and 1
print(np.random.rand(3, 3))

Output:

[[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]

[[1. 1. 1. 1.]
 [1. 1. 1. 1.]]

[0 2 4 6 8]

[0.   0.25 0.5  0.75 1.  ]

[[0.37 0.95 0.73]
 [0.60 0.15 0.86]
 [0.71 0.02 0.97]]

Array Properties

Every NumPy array has properties that describe its structure. These are checked constantly when working with real data.

a = np.array([[1, 2, 3], [4, 5, 6]])

print(a.shape)   # Dimensions
print(a.ndim)    # Number of dimensions
print(a.size)    # Total number of elements
print(a.dtype)   # Data type of elements

Output:

(2, 3)
2
6
int64

The shape property is especially important. Shape mismatches are the most common source of errors in machine learning code. Always check .shape on your arrays before and after every operation.

Indexing and Slicing

Accessing specific elements or ranges from a NumPy array works as follows:

a = np.array([10, 20, 30, 40, 50])

print(a[0])     # First element
print(a[-1])    # Last element
print(a[1:4])   # Elements at index 1, 2, 3

Output:

10
50
[20 30 40]

For two-dimensional arrays:

b = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

print(b[1, 2])    # Row 1, Column 2
print(b[:, 1])    # All rows, Column 1
print(b[0:2, :])  # First 2 rows, all columns

Output:

6
[2 5 8]
[[1 2 3]
 [4 5 6]]

Math Operations

All basic math operations in NumPy are element-wise by default. This means the operation is applied to each pair of elements automatically - no loop required.

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

print(a + b)       # Addition
print(a * b)       # Multiplication
print(a - b)       # Subtraction
print(a / b)       # Division
print(np.dot(a, b))  # Dot product

Output:

[5 7 9]
[ 4 10 18]
[-3 -3 -3]
[0.25 0.4  0.5 ]
32

Broadcasting

Broadcasting is one of the most powerful features of NumPy. It allows operations to be performed between arrays of different shapes without writing any extra code.

For example, adding a single number to every element of a two-dimensional array:

a = np.array([[1, 2, 3], [4, 5, 6]])
print(a + 10)

Output:

[[11 12 13]
 [14 15 16]]

NumPy automatically stretches the value 10 across every element in the array. This is called broadcasting. It is used in normalisation, feature scaling, and image processing.

Broadcasting also works between arrays of compatible shapes:

a = np.array([[1, 2, 3], [4, 5, 6]])
b = np.array([10, 20, 30])
print(a + b)

Output:

[[11 22 33]
 [14 25 36]]

Reshaping and Transposing

The shape of an array can be changed using reshape(). The total number of elements must remain the same.

a = np.arange(12)
print(a)

b = a.reshape(3, 4)
print(b)

Output:

[ 0  1  2  3  4  5  6  7  8  9 10 11]

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]]

Transposing - swapping rows and columns - is done with .T:

print(b.T)

Output:

[[ 0  4  8]
 [ 1  5  9]
 [ 2  6 10]
 [ 3  7 11]]

Advanced NumPy

Fancy Indexing

Multiple specific elements can be extracted at once by passing a list of indices:

a = np.array([10, 20, 30, 40, 50])
print(a[[0, 2, 4]])

Output:

[10 30 50]

Boolean Masking

Elements that satisfy a condition can be filtered in a single line:

a = np.array([3, 7, 1, 9, 4, 6])
print(a[a > 5])

Output:

[7 9 6]

Stacking Arrays

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

print(np.vstack([a, b]))   # Vertical stack
print(np.hstack([a, b]))   # Horizontal stack

Output:

[[1 2 3]
 [4 5 6]]

[1 2 3 4 5 6]

Statistical Functions

a = np.array([[1, 2, 3], [4, 5, 6]])

print(np.mean(a))           # Overall mean
print(np.sum(a, axis=0))    # Sum down columns
print(np.sum(a, axis=1))    # Sum across rows
print(np.max(a))            # Maximum value
print(np.min(a))            # Minimum value
print(np.std(a))            # Standard deviation

Output:

3.5
[5 7 9]
[ 6 15]
6
1
1.707

The axis parameter controls the direction of the operation. axis=0 runs down the rows (column-wise). axis=1 runs across the columns (row-wise).

np.where()

A conditional replacement can be applied across an entire array at once:

a = np.array([3, 7, 1, 9, 4, 6])
result = np.where(a > 5, 'high', 'low')
print(result)

Output:

['low' 'high' 'low' 'high' 'low' 'high']

np.unique() and np.sort()

a = np.array([3, 1, 4, 1, 5, 9, 2, 6, 5])

print(np.unique(a))   # Unique values
print(np.sort(a))     # Sorted array

Output:

[1 2 3 4 5 6 9]
[1 1 2 3 4 5 5 6 9]

Quick Command Reference

np.array() - Create an array from a list
np.zeros() - Array of zeros
np.ones() - Array of ones
np.arange() - Range of values with step
np.linspace() - Evenly spaced values
np.random.rand() - Random values
.shape - Dimensions of the array
.dtype - Data type of elements
.reshape() - Change array shape
.T - Transpose the array
np.dot() - Dot product
np.vstack() - Vertical stack
np.hstack() - Horizontal stack
np.where() - Conditional replacement
np.unique() - Unique values
np.sort() - Sort array
np.mean(), np.sum(), np.std() - Statistical functions

Key Takeaways

NumPy arrays are faster and more efficient than Python lists for numerical operations
Always check .shape when debugging - shape mismatches are the most common error
Broadcasting removes the need for loops when performing operations on arrays of different sizes
Use boolean masking to filter data without writing any loop
NumPy is the foundation of Pandas, TensorFlow, PyTorch, and Scikit-learn

All the code examples in this post are also covered in detail in the YouTube video. The link is available at the top of this page. If you have any questions, drop them in the comments section below.

Python Pandas Data Cleaning & Null Values Guide

2026-03-11T06:10:00.000-07:00

Every data science project starts with one unavoidable step - cleaning the data.

Most beginners want to jump straight into analysis, building charts, or training machine learning models. But here is the reality: if the data going in is broken, the results coming out will be wrong. And the frustrating part is - the code will not always throw an error to warn you. It will just give you a number that looks correct but is not.

That is what makes dirty data dangerous.

In this post, we are going to work through the full data cleaning process using Python and Pandas - step by step, with real code and real output. By the end, you will know how to find missing values, remove duplicates, fix data types, and fill nulls using the right strategy for each column type.

You can also watch this tutorial in youtube.

What Is Dirty Data?

Dirty data is any dataset that contains errors, inconsistencies, or missing information that can affect the accuracy of your analysis.

The three most common problems in real-world datasets are:

Missing values - cells that are empty or contain NaN
Duplicate rows - the same record appearing more than once
Wrong data types - a numeric column stored as text, for example

None of these are rare. In production datasets - whether they come from databases, forms, APIs, or CSV exports - all three problems tend to show up together.

Setting Up the Environment

This tutorial works in both VS Code and Google Colab.

VS Code: Create a new file data_cleaning.py and run it from the integrated terminal.
Google Colab: Open colab.research.google.com, create a new notebook, and run each cell block by block.

Start every session with these two imports:

Code

import pandas as pd
import numpy as np

Step 1 - Load the Data and Spot the Problems

Before fixing anything, the first job is to see what you are working with. Below is a sample student dataset with intentional problems built in:

Code

import pandas as pd
import numpy as np

data = {
    'Name':  ['Alice', 'Bob', 'Charlie', 'Bob', 'Diana', 'Eve'],
    'Age':   [22.0, None, 21.0, None, 23.0, 25.0],
    'Score': [88.0, 92.0, None, 92.0, 79.0, None],
    'Grade': ['A', 'B', None, 'B', 'C', None]
}

df = pd.DataFrame(data)
print(df)

Output

Name Age Score Grade 0 Alice 22.0 88.0 A 1 Bob NaN 92.0 B 2 Charlie 21.0 NaN None 3 Bob NaN 92.0 B 4 Diana 23.0 79.0 C 5 Eve 25.0 NaN None

Problems visible at a glance:

Row 1 and Row 3 are exact duplicates - Bob appears twice
Age is missing in rows 1 and 3
Score is missing in rows 2 and 5
Grade is missing in rows 2 and 5

Step 2 - Find All Null Values

Before deciding how to handle missing values, you need to know exactly where they are and how many there are.

Code

print(df.isnull())

Output

Name Age Score Grade 0 False False False False 1 False True False False 2 False False True True 3 False True False False 4 False False False False 5 False False True True

Use .sum() to get a count per column:

Code

print(df.isnull().sum())

Output

Name 0 Age 2 Score 2 Grade 2 dtype: int64

For large datasets, check the percentage of missing values:

Code

print(df.isnull().sum() / len(df) * 100)

Output

Name 0.00 Age 33.33 Score 33.33 Grade 33.33 dtype: float64

33% of Age, Score, and Grade is missing. That is too significant to ignore.

Step 3 - Remove Duplicate Rows

Duplicates quietly inflate your numbers. If Bob appears twice, his score gets counted twice in every average or aggregation - without a single warning.

Code

# Count duplicates
print(df.duplicated().sum())

Output

Code

# See which row is duplicate
print(df[df.duplicated()])

Output

Name Age Score Grade 3 Bob NaN 92.0 B

Code

# Remove duplicates
df.drop_duplicates(inplace=True)
print(df.shape)

Output

(5, 4)

Additional options for drop_duplicates():

Code

df.drop_duplicates(keep='last', inplace=True)          # Keep last occurrence
df.drop_duplicates(keep=False, inplace=True)           # Remove all copies
df.drop_duplicates(subset=['Name'], inplace=True)      # Check by column only

Step 4 - Drop or Fill?

Once duplicates are handled, you need to deal with null values. You have two options - drop or fill.

Situation	Recommended Action
Very few rows have nulls, large dataset	Drop rows
Nulls are common or rows are valuable	Fill with mean / median / mode
Entire column is mostly null	Drop the column

Step 5 - Dropping Nulls

Code

# Drop any row with at least one null
df_dropped = df.dropna()
print(df_dropped)

Output

Name Age Score Grade 0 Alice 22.0 88.0 A 4 Diana 23.0 79.0 C

Code

df.dropna(axis=1)        # Drop columns with any null
df.dropna(thresh=3)      # Keep rows with at least 3 non-null values

Step 6 - Filling Nulls in Numeric Columns

For numeric columns, fill with mean when there are no significant outliers, or median when outliers exist.

Code

# Fill Age with mean
df['Age'].fillna(df['Age'].mean(), inplace=True)
print(df['Age'])

Output

0 22.00 1 22.75 2 21.00 4 23.00 5 25.00 Name: Age, dtype: float64

Code

# Fill Score with median
df['Score'].fillna(df['Score'].median(), inplace=True)
print(df['Score'])

Output

0 88.0 1 92.0 2 88.5 4 79.0 5 88.5 Name: Score, dtype: float64

Step 7 - Filling Nulls in Categorical Columns

For categorical columns like Grade or City - filling with mean makes no sense. Use the mode - the most frequently occurring value.

Code

# Fill Grade with mode
df['Grade'].fillna(df['Grade'].mode()[0], inplace=True)
print(df['Grade'])

Output

0 A 1 B 2 B 4 C 5 B Name: Grade, dtype: object

💡 Note: mode()[0] is used because .mode() returns a Series. The [0] extracts the first - most frequent - value from it.

Step 8 - Checking and Fixing Data Types

A very common issue with CSV files is that numeric columns get loaded as strings when the column contains mixed content.

Code

data2 = {
    'Product': ['Laptop', 'Phone', 'Tablet', 'Watch'],
    'Price':   ['999', '499', 'NaN', '199'],
    'Stock':   [50, 30, 20, 15]
}

df2 = pd.DataFrame(data2)
print(df2.dtypes)

Output

Product object Price object Stock int64 dtype: object

Code

# Convert Price to numeric
df2['Price'] = pd.to_numeric(df2['Price'], errors='coerce')
print(df2.dtypes)
print(df2)

Output

Product object Price float64 Stock int64 Product Price Stock 0 Laptop 999.0 50 1 Phone 499.0 30 2 Tablet NaN 20 3 Watch 199.0 15

Code

df2['Stock']   = df2['Stock'].astype(int)     # To integer
df2['Product'] = df2['Product'].astype(str)   # To string

Step 9 - Complete End-to-End Cleaning Workflow

Below is the full seven-step workflow in a single block. This is the process to follow every time a new dataset lands on your desk.

Code

import pandas as pd
import numpy as np

# Step 1 - Load
df = pd.read_csv("students.csv")
print("Raw Data:")
print(df.head())

# Step 2 - Inspect
print("\nShape:", df.shape)
print(df.info())

# Step 3 - Check nulls
print("\nNull Counts:")
print(df.isnull().sum())
print("\nNull Percentage:")
print(df.isnull().sum() / len(df) * 100)

# Step 4 - Remove duplicates
print("\nDuplicates:", df.duplicated().sum())
df.drop_duplicates(inplace=True)

# Step 5 - Fix data types
df['Age']   = pd.to_numeric(df['Age'],   errors='coerce')
df['Score'] = pd.to_numeric(df['Score'], errors='coerce')

# Step 6 - Fill nulls by column type
df['Age'].fillna(df['Age'].mean(),        inplace=True)   # Numeric → mean
df['Score'].fillna(df['Score'].median(),  inplace=True)   # Numeric → median
df['Grade'].fillna(df['Grade'].mode()[0], inplace=True)   # Categorical → mode

# Step 7 - Confirm clean
print("\nNull counts after cleaning:")
print(df.isnull().sum())
print("\nClean Dataset:")
print(df)

Output

Null Counts: Name 0 Age 2 Score 1 Grade 1 Duplicates: 1 Null counts after cleaning: Name 0 Age 0 Score 0 Grade 0 Clean Dataset: Name Age Score Grade 0 Alice 22.00 88.0 A 1 Bob 22.00 88.5 B 2 Charlie 21.00 88.5 A 4 Diana 23.00 79.0 C

Quick Command Reference

Task	Command
Check nulls per cell	`df.isnull()`
Count nulls per column	`df.isnull().sum()`
Null percentage	`df.isnull().sum() / len(df) * 100`
Drop rows with nulls	`df.dropna()`
Drop columns with nulls	`df.dropna(axis=1)`
Fill with fixed value	`df.fillna(0)`
Fill numeric with mean	`df['col'].fillna(df['col'].mean())`
Fill numeric with median	`df['col'].fillna(df['col'].median())`
Fill categorical with mode	`df['col'].fillna(df['col'].mode()[0])`
Count duplicates	`df.duplicated().sum()`
Remove duplicates	`df.drop_duplicates()`
Check data types	`df.dtypes`
Convert to numeric	`pd.to_numeric(df['col'], errors='coerce')`
Convert type manually	`df['col'].astype(int)`

Docker Installation on Ubuntu 22.04 | Step-by-Step Guide

2025-07-25T22:25:00.000-07:00

In this guide, we are going to learn how to install Docker on Ubuntu with step-by-step complete explanation. We will also verify every step to make sure Docker is properly installed and running on your system.

Topics Covered:

How to update and prepare your Ubuntu system
How to install Docker securely using Docker’s official repository
How to verify Docker installation
How to test Docker by running a container

My Setup:

I'm using Ubuntu 22.04 as my operating system for this demo.

You can also watch this tutorial demo on our Youtube Channel

Step 1: Update System Packages

We need to update the package list and upgrade any outdated packages.

sudo apt update
sudo apt upgrade -y

Command Output:

Trimmed
Processing triggers for dbus (1.14.10-4ubuntu4.1) ...
Processing triggers for install-info (7.1-3build2) ...
Processing triggers for libc-bin (2.39-0ubuntu8.5) ...
Processing triggers for man-db (2.12.0-4build2) ...
Processing triggers for initramfs-tools (0.142ubuntu25.5) ...
update-initramfs: Generating /boot/initrd.img-6.8.0-64-generic
Scanning processes...
Scanning candidates...
Scanning linux images...

Running kernel seems to be up-to-date.

Restarting services...
systemctl restart multipathd.service polkit.service ssh.service udisks2.service

Service restarts being deferred:
systemctl restart ModemManager.service
/etc/needrestart/restart.d/dbus.service
systemctl restart getty@tty1.service
systemctl restart systemd-logind.service
systemctl restart unattended-upgrades.service

No containers need to be restarted.

User sessions running outdated binaries:
skuma @ session #2: apt[1490], sshd[891]
skuma @ session #4: sshd[893]
skuma @ user manager service: systemd[898]

No VM guests are running outdated hypervisor (qemu) binaries on this host.
skuma@server1:~$

This makes sure we are working with the latest versions of all dependencies.

Step 2: Install Required Packages

Install some necessary packages that help us securely install Docker.

sudo apt install ca-certificates curl gnupg -y

Command Output:

skuma@server1:~$ sudo apt install ca-certificates curl gnupg -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ca-certificates is already the newest version (20240203).
ca-certificates set to manually installed.
curl is already the newest version (8.5.0-2ubuntu10.6).
gnupg is already the newest version (2.4.4-2ubuntu17.3).
gnupg set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
skuma@server1:~$

These tools are very important for managing repositories and downloading packages over HTTPS.

Step 3: Add Docker’s GPG Key

To make sure we install Docker from a trusted source, we need to add Docker’s official GPG key.

First, create the directory:

sudo mkdir -p /etc/apt/keyrings

Then download and store the GPG key securely using curl:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg \
--dearmor -o /etc/apt/keyrings/docker.gpg

Step 4: Add Docker’s Repository

Now we’ll add Docker’s official repository.

echo \
"deb [arch=$(dpkg --print-architecture) \
signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

This command automatically detects your system architecture and Ubuntu version, and configures the Docker repo accordingly.

Step 5: Update the Package List Again

Now that Docker's repo is added, we need to refresh the package list again.

sudo apt update

Command Output:

skuma@server1:~$ sudo apt update
Get:1 https://download.docker.com/linux/ubuntu noble InRelease [48.8 kB]
Get:2 https://download.docker.com/linux/ubuntu noble/stable amd64 Packages [28.5 kB]
Hit:3 http://in.archive.ubuntu.com/ubuntu noble InRelease
Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease
Hit:5 http://in.archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:6 http://in.archive.ubuntu.com/ubuntu noble-backports InRelease
Fetched 77.3 kB in 2s (47.0 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
4 packages can be upgraded. Run 'apt list --upgradable' to see them.

Step 6: Install Docker Engine

Now comes the main part, installing Docker.

sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

Command Output:

skuma@server1:~$ sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
docker-ce-rootless-extras libslirp0 pigz slirp4netns
Suggested packages:
cgroupfs-mount | cgroup-lite docker-model-plugin
The following NEW packages will be installed:
containerd.io docker-buildx-plugin docker-ce docker-ce-cli docker-ce-rootless-extras docker-compose-plugin libslirp0 pigz slirp4netns
0 upgraded, 9 newly installed, 0 to remove and 4 not upgraded.
Need to get 103 MB of archives.
After this operation, 429 MB of additional disk space will be used.
Output Trimmed
Running kernel seems to be up-to-date.

Restarting services...

Service restarts being deferred:
/etc/needrestart/restart.d/dbus.service
systemctl restart getty@tty1.service
systemctl restart systemd-logind.service
systemctl restart unattended-upgrades.service

No containers need to be restarted.

User sessions running outdated binaries:
skuma @ session #2: sshd[891]
skuma @ session #4: sshd[893]
skuma @ user manager service: systemd[898]

No VM guests are running outdated hypervisor (qemu) binaries on this host.
skuma@server1:~$

This installs:

Docker Engine

Docker CLI tools

Docker Compose plugin

Step 7: Verify Docker Installation

Check Docker version:

docker --version

Command Output:

skuma@server1:~$ docker --version
Docker version 28.3.2, build 578ccf6
skuma@server1:~$

If you see the version number, Docker is installed correctly.

Step 8: Check Docker Service Status

Verify whether Docker is running or not.

sudo systemctl status docker

Command Output:

skuma@server1:~$ sudo systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; preset: enabled)
Active: active (running) since Sat 2025-07-26 05:23:22 UTC; 1min 6s ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 16697 (dockerd)
Tasks: 8
Memory: 21.6M (peak: 21.8M)
CPU: 483ms
CGroup: /system.slice/docker.service
└─16697 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Jul 26 05:23:21 server1 dockerd[16697]: time="2025-07-26T05:23:21.669297451Z" level=info msg="detected 127.0.0.53 nameserver, assuming systemd-resolved, so using re>
Jul 26 05:23:21 server1 dockerd[16697]: time="2025-07-26T05:23:21.708933067Z" level=info msg="Creating a containerd client" address=/run/containerd/containerd.sock >
Jul 26 05:23:21 server1 dockerd[16697]: time="2025-07-26T05:23:21.824117058Z" level=info msg="Loading containers: start."
Jul 26 05:23:22 server1 dockerd[16697]: time="2025-07-26T05:23:22.404568741Z" level=info msg="Loading containers: done."
Jul 26 05:23:22 server1 dockerd[16697]: time="2025-07-26T05:23:22.436489743Z" level=info msg="Docker daemon" commit=e77ff99 containerd-snapshotter=false storage-dri>
Jul 26 05:23:22 server1 dockerd[16697]: time="2025-07-26T05:23:22.436901615Z" level=info msg="Initializing buildkit"
Jul 26 05:23:22 server1 dockerd[16697]: time="2025-07-26T05:23:22.501542109Z" level=info msg="Completed buildkit initialization"
Jul 26 05:23:22 server1 dockerd[16697]: time="2025-07-26T05:23:22.518178178Z" level=info msg="Daemon has completed initialization"
Jul 26 05:23:22 server1 dockerd[16697]: time="2025-07-26T05:23:22.518243119Z" level=info msg="API listen on /run/docker.sock"
Jul 26 05:23:22 server1 systemd[1]: Started docker.service - Docker Application Container Engine.

You should see something like active (running) in green. That means Docker is up and running.

Step 9: Run a Test Container

Let’s run a simple test container using the hello-world image:

sudo docker run hello-world

Command Output:

skuma@server1:~$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
e6590344b1a5: Pull complete
Digest: sha256:ec153840d1e635ac434fab5e377081f17e0e15afab27beb3f726c3265039cfff
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/

For more examples and ideas, visit:
https://docs.docker.com/get-started/

skuma@server1:~$

This downloads a test image and runs a small container.

If everything is working, you'll see a friendly message from Docker confirming the setup is successful.

That’s It!

We have successfully installed Docker on Ubuntu 22.04! See you in the next tutorial.

Kubernetes RBAC Explained with Hands-On Demo

2025-07-22T22:28:00.000-07:00

In this post, I will help you to explain what is RBAC in Kubernetes? and help you understand the concept in a very simple way with hands-on examples.

So, if you want to know how to give access to multiple users with different permissions in your Kubernetes cluster, then this tutorial will definitely help you.

For example:

Admin access for some users
Read-only access for some users in specific namespaces
Full access for some users in specific namespaces

We will start with understanding what is RBAC and why we need it.

Topics Covered

What is RBAC and why do we need it?
How Kubernetes users are authenticated and authorized
Types of RBAC components:

▸ Role
▸ ClusterRole
▸ RoleBinding
▸ ClusterRoleBinding

Creating admin and read-only users
Mapping them using aws-auth
And finally, a demo with examples using different AWS profiles:

🔹 user1-ro – Read-only access to a single namespace
🔹 user2-full – Full access to a single namespace
🔹 user3-admin – Full access to the entire cluster

Since this is for Kubernetes, I use AWS EKS. It may vary if you are using other cloud providers like Google Cloud, etc. But here, I’ll be using AWS IAM for authentication.

You can also watch this tutorial demo on our Youtube Channel

What is RBAC?

RBAC stands for Role-Based Access Control. It is a method to define who can do what on your Kubernetes cluster.

To do this, RBAC uses four main components:

Role
ClusterRole
RoleBinding
ClusterRoleBinding

Each component plays a key role in defining and controlling access.

Authentication & Authorization

Kubernetes always checks two things when a request is made:

Authentication – Who is making the request?
In EKS, the identity is verified using AWS IAM.
Authorization – What is that user allowed to do?
This is controlled using RBAC.

For example, if a user tries to list pods, Kubernetes checks: is this user authenticated? Then, is the user allowed to list pods?

Creating IAM Users

So I have already created three IAM users for this demo:

user1-ro
user2-full
user3-admin

I’ve added them with the AdministratorAccess policy just for the sake of demo.

Each user has their own access key ID and secret, which is configured using AWS CLI on my system.

Updating aws-auth ConfigMap

For mapping AWS IAM users to Kubernetes clusters, We have to use aws-auth ConfigMap.

kubectl -n kube-system get cm aws-auth -o yaml

So export this aws-auth configmap to a file, which will be easy for managing access

kubectl -n kube-system get cm aws-auth -o yaml > aws-auth.yaml

You can verify the aws-auth.yaml file using cat command.

skuma@server1:~/kubernetes/auth$ cat aws-auth.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
data:
mapRoles: |
- rolearn: arn:aws:iam::591874054217:role/eksctl-demo-cluster-nodegroup-demo-NodeInstanceRole-cW2EtCUZvNc7
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes

API Resources

Before creating actual roles and bindings, we must know the list of resources we can use in RBAC for access:

kubectl api-resources

Output:

You will see list of all Kubernetes resources, that supports to manage using RBAC including whether they are namespaced controlled or not.

Use long names like services, pods, deployments, configmaps. Don’t use short names like svc, po, deploy.

Create Roles & ClusterRole

Now let’s create the role and cluster role.

1. Read-only Role for `dev` namespace

$ cat role-dev-readonly.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: dev
name: dev-readonly
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["pods","services","configmaps"]
verbs: ["get", "watch", "list"]

2. Full-access Role for `dev` namespace

$ cat role-dev-fullaccess.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: dev
name: dev-fullaccess
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["*"]
verbs: ["*"]

3. Cluster Admin Role

$ cat clusterrole-cluster-admin-full.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-admin-full
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["*"]
verbs: ["*"]

Once we have created role, cluster role with required permissions including resources, verbs. Now we need to create bindings for each role and clusterrole.

RoleBinding and ClusterRoleBinding

RoleBinding read-only access for namespace "dev":

$ cat rolebinding-dev-readonly.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rolebinding-dev-readonly
namespace: dev
subjects:
- kind: Group
name: dev-readonly
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: dev-readonly
apiGroup: rbac.authorization.k8s.io

RoleBinding Full-access for namespace "dev":

$ cat rolebinding-dev-fullaccess.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rolebinding-dev-fullaccess
namespace: dev
subjects:
- kind: Group
name: dev-fullaccess
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: dev-fullaccess
apiGroup: rbac.authorization.k8s.io

ClusterRoleBinding Admin access for entire cluster:

$ cat rolebinding-cluster-admin-full.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: rolebinding-cluster-admin-full
subjects:
- kind: Group
name: cluster-admin-full
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: cluster-admin-full
apiGroup: rbac.authorization.k8s.io

Map users in aws-auth configmap

Open aws-auth yaml file using some editor and add all your users with respective groups that we have created in the rolebdindings as below.

$ cat aws-auth.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
data:
mapRoles: |
- rolearn: arn:aws:iam::591874054217:role/eksctl-demo-cluster-nodegroup-demo-NodeInstanceRole-cW2EtCUZvNc7
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
mapUsers: |
- groups:
- dev-readonly
userarn: arn:aws:iam::591874054217:user/user1-ro
username: user1-ro
- groups:
- dev-fullaccess
userarn: arn:aws:iam::591874054217:user/user2-full
username: user1-full
- groups:
- cluster-admin-full
userarn: arn:aws:iam::591874054217:user/user3-admin
username: user3-admin

Testing Access with AWS CLI Profiles

Ask each users to update kubeconfig into the system, then only they can manage your kubernetes cluster.

aws eks update-kubeconfig --region us-east-1 --name demo-cluster

Once you have got your kubeconfig file. Test the access.

Switch to each user and run different commands to check the appropriate permission are set or not.

kubectl get pods -n dev # ✅ for user1-ro
kubectl delete pod podname # ❌ for user1-ro
kubectl get ns # ❌ for user1-ro

Finally,

user1-ro can only view resources in dev namespace.
user2-full can manage everything in dev namespace but cannot touch prod or cluster-wide objects.
user3-admin has full access across the entire cluster.

That’s how we control access in Kubernetes using RBAC. I hope you understood the concept.

Deploy AWS EKS Cluster on Private Subnets with Terraform

2025-07-18T21:20:00.000-07:00

In the previous article, I showed you how to create an AWS EKS cluster with a custom VPC using Terraform modules. That setup was great for a non-production or testing environment.

But here, we’ll take things one step further. We will convert the same setup into a production-ready environment. The key changes? We will deploy our AWS EKS cluster into private subnets and connect it through a NAT Gateway. This is a widely recommended best practice for securing production workloads on AWS.

🛠 Prerequisites

Before you start, make sure these tools are installed and configured on your system.

Terraform – Used to provision infrastructure as code.

👉 Install: terraform command installation

👉 Check with terraform -v

AWS CLI – Needed to interact with AWS services and fetch kubeconfig.

👉 Download: aws.amazon.com/cli

👉 Configure with aws configure

kubectl – To interact with your EKS Kubernetes cluster.

👉 Download: kubernetes.io

You can also watch this tutorial demo on our Youtube Channel

📁 Setting Up the Terraform Project

I’m using Visual Studio Code. I copied the previous project folder eks-terraform-demo and renamed it to eks-terraform-demo-prod. This way we reuse all the Terraform files we created earlier and just tweak them for production.

$ ls -lrt eks-terraform-demo
total 7
-rw-r--r-- 1 skuma 197609 136 Jul 8 20:25 outputs.tf
-rw-r--r-- 1 skuma 197609 43 Jul 9 08:05 provider.tf
-rw-r--r-- 1 skuma 197609 467 Jul 9 12:09 vpc.tf
-rw-r--r-- 1 skuma 197609 559 Jul 9 13:40 eks.tf

skuma@hp MINGW64 /d/Learning/codes/Terraform
$ ls -lrt eks-terraform-demo-prod/
total 76
-rw-r--r-- 1 skuma 197609 136 Jul 8 20:25 outputs.tf
-rw-r--r-- 1 skuma 197609 43 Jul 9 08:05 provider.tf
-rw-r--r-- 1 skuma 197609 650 Jul 9 13:42 vpc.tf
-rw-r--r-- 1 skuma 197609 627 Jul 9 19:29 eks.tf

🏗️ VPC Changes for Production

Inside vpc.tf, I made several important changes:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "flipkart-vpc-prod" #changed name to match prod.
  cidr = "10.0.0.0/16"
  azs  = ["us-east-1a", "us-east-1b"]

  public_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  private_subnets = ["10.0.11.0/24", "10.0.12.0/24"]

  map_public_ip_on_launch = true
  enable_nat_gateway = true #changed to true
  single_nat_gateway = true #added for prod

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  #Added for prod
  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }

  tags = {
    Terraform = "true"
  }
}

VPC Name changed to flipkart-vpc-prod

map_public_ip_on_launch = true (kept as-is)

enable_nat_gateway = true – this allows private subnets to access the internet

single_nat_gateway = true – avoids creating multiple NATs to save cost

Added tags to private subnets – this is required for Kubernetes to provision internal load balancers

These changes ensure that our worker nodes will be launched in private subnets and still have internet access via the NAT Gateway — a common and secure setup in production environments.

⚙️ EKS Cluster Changes

In eks.tf, I made the following updates:

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.0"

  cluster_name    = "flipkart-eks-prod" #changed name to match prod
  cluster_version = "1.31"

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets #changed to private subnets

  cluster_endpoint_public_access = true
  enable_cluster_creator_admin_permissions = true

  eks_managed_node_groups = {
    default = {
      instance_types = ["t3.medium"]
      min_size       = 1
      max_size       = 2
      desired_size   = 1
    }
  }

  tags = {
    Environment = "prod" #changed to prod
  }
}

Cluster Name changed to flipkart-eks-prod

Subnet IDs updated to use private subnets instead of public ones

Tag updates to reflect environment as "prod"

The cluster endpoint and public access settings were kept the same for now. You can make them private if needed.

🔧 Terraform Apply Configuration

It's Time to deploy the infrastructure using terraform apply command, but I have initialized the terraform, so first i will initiate it.

terraform init
terraform apply -auto-approve

Once done, you’ll see the output: your cluster endpoint and name.

✅ Verifying EKS Cluster Setup

Go to the AWS Console → EKS → You’ll see the new cluster flipkart-eks-prod in "Active" status.

Check the EC2 instances – a new instance should appear, deployed into your private subnet (e.g., CIDR 10.0.11.0/24).

🔗 Connect AWS EKC Cluster using kubectl

Configure your local kubeconfig to interact with the new cluster:

aws eks update-kubeconfig --region us-east-1 --name flipkart-eks-prod

Then test the connection:

kubectl get nodes

You should see one worker node up and ready.

🌐 Deploy Test App Deployment

Now let’s deploy a basic Apache web app and expose it.

kubectl run apache --image=httpd --port=80
kubectl get pods

Expose it using a LoadBalancer:

kubectl expose pod apache --type=LoadBalancer --port=80 --target-port=80
kubectl get svc

You’ll get a LoadBalancer hostname (likely internal, since it’s running in a private subnet).

🔓 Allow HTTP Temporarily (For Testing Only)

If needed, open port 80:

Go to EC2 → Security Groups → Find the EKS node group SG

Add an inbound rule: HTTP (80), source: 0.0.0.0/0

Visit the LoadBalancer URL in your browser (use http://, not https://). You might get a "Not Secure" warning — just proceed for this test.

If it loads the Apache default page, then everything is working!

📌 Bonus Tip: Avoid One Service = One LoadBalancer

For a real production workload, exposing each pod with its own LoadBalancer is not ideal. You’ll end up with high cost and complex routing.

Instead, deploy an Ingress Controller (like NGINX or HAProxy) in front of your cluster. We’ve already covered how to deploy Ingress Controllers — check out our earlier tutorials for that.

This is now a production-ready Kubernetes environment on AWS using Terraform.

Create EKS Cluster on AWS Using Terraform Modules

2025-07-11T23:54:00.000-07:00

In this guide, I'm going to walk you through how to create a Kubernetes cluster on AWS (EKS) using Terraform modules. And it's not just about the Kubernetes cluster. we will also create the VPC required for it. This is going to be a complete Kubernetes environment, and by the end, you’ll have a strong understanding of both VPC and AWS EKS setup.

Finally, we'll do a deployment of a simple test application like Apache to verify that the cluster is running and accessible.

You can also watch this tutorial demo on our Youtube Channel

AWS EKS + VPC Architecture

This is the architecture we are going to create:

First, we need to create a VPC with two public subnets and two private subnets.

Since this is a non-production environment, we’ll deploy the Kubernetes cluster on public subnets.

If you're just getting started with Kubernetes on AWS, this is the best way to begin. In a follow-up, I’ll show you how to deploy on private subnets for a full production setup.

I assume, you have already installed the required tools.

kubectl – https://kubernetes.io/docs/tasks/tools

AWS CLI – https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html

- Run aws configure to set your AWS access and secret keys.

Terraform – https://developer.hashicorp.com/terraform/downloads

I’ve created a directory called eks-terraform-demo. Inside it, I’ve already defined the necessary Terraform files and blocks.

Let’s go through them one by one.

provider.tf

In the provider.tf file, I’ve defined the AWS provider, and the region is set to us-east-1.

provider "aws" {
  region = "us-east-1"
}

Creating VPC with Terraform Module:

vpc.tf

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "flipkart-vpc-nonprod"
  cidr = "10.0.0.0/16"
  azs  = ["us-east-1a", "us-east-1b"]

  public_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  private_subnets = ["10.0.11.0/24", "10.0.12.0/24"]

  map_public_ip_on_launch = true
  enable_nat_gateway = false

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  tags = {
    Terraform = "true"
  }
}

In vpc.tf, I’ve used the official Terraform VPC module. You can find it by searching “Terraform modules” in your browser and heading to terraform-aws-modules. There, you’ll find a module for VPC and another for EKS.

I’ve copied the VPC example and made some modifications based on the requirement for integrating the AWS VPC with AWS EKS Module.

Mentioned Two public subnets and two private subnets across us-east-1a and us-east-1b for better high availability.

I also disabled the NAT gateway to save cost, since we’re not deploying anything on the private subnets for now.

I faced issues using the latest VPC module version (v6.x) with the EKS module, so I used version 5.x instead—it worked perfectly.

The VPC is named flipkart-vpc-nonpro, just as an example.

Creating AWS EKS with Terraform Module

eks.tf

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.0"

  cluster_name    = "flipkart-eks-nonprod"
  cluster_version = "1.31"

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.public_subnets

  cluster_endpoint_public_access = true
  enable_cluster_creator_admin_permissions = true

  eks_managed_node_groups = {
    default = {
      instance_types = ["t3.medium"]
      min_size       = 1
      max_size       = 2
      desired_size   = 1
    }
  }

  tags = {
    Environment = "nonprod"
  }
}

In eks.tf, I used the EKS managed node group example from the Terraform registry. I prefer managed node groups as they are fully controlled by AWS.

Here are some key details:

Cluster name: flipkart-eks-nonpro

VPC ID is passed from the VPC module

Subnets used are public subnets only

cluster_endpoint_public_access = true to allow access from outside (like your laptop)

Managed node group with:

Instance type: t3.medium
Desired capacity: 1
Min size: 1
Max size: 2

Initialize and Apply Terraform Configuration:

Once all the files are ready:

terraform init
terraform apply -auto-approve

After a successful apply, I check the output values for cluster_name and cluster_endpoint. Because i have added the below output blocks to get printed the values on the terminal.

outputs.tf

output "cluster_name" {
  value = module.eks.cluster_name
}

output "cluster_endpoint" {
  value = module.eks.cluster_endpoint
}

Update KubeConfig

First, we need the kubeconfig locally:

aws eks update-kubeconfig --region us-east-1 --name flipkart-eks-nonprod

Then run:

kubectl get nodes

If you see the node listed, your setup is working.

Deploying Test Application

We now test by deploying a simple Apache pod.

Now deploy Apache directly:

kubectl run apache --image=httpd --port=80

Now Check pod:

kubectl get pods

Expose it via a LoadBalancer:

kubectl expose pod apache --type=LoadBalancer --port=80

Now check:

kubectl get svc

You’ll see a LoadBalancer service created with an external IP (provided by AWS ELB).

Allowing Access (Temporary for Testing)

By default, port 80 won’t be open in the security group. So:
Go to EKS → Networking → Security Groups
Open the additional node group security group
Add an inbound rule for HTTP (port 80) from anywhere (0.0.0.0/0)
Save the rule
Now access the Load Balancer URL from your browser using HTTP (not HTTPS). If it redirects to HTTPS, remove the "s" and try again.
You should see the Apache default page.

This setup is perfect for learning and testing purposes. It will give you better understanding of how Terraform integrates with AWS services like EKS and VPC.

In the next guide, I’ll show you how to set up a production-ready cluster using private subnets, proper NAT gateways, and restricted access.

Understanding Terraform Provisioners with Real-Time EC2 Demo

2025-07-01T20:59:00.000-07:00

Hello friends, this is our 9th post in our Terraform tutorial series.

We’ve already covered all the important topics from the basics, step by step, in the previous topics. In this post, I’ll explain about Terraform provisioners, including:

1. What are Terraform provisioners?
2. Types of provisioners
3. What is file provisioner?
4. What is remote-exec provisioner?

You can also Watch this tutorial demo on our YouTube Channel

What are Terraform Provisioners?

Provisioners are like post-setup scripts. They help you perform additional tasks after Terraform has applied the configuration.

For example:

Copying files
Running installation scripts
Configuring services
Executing any custom commands

We have a few types of provisioners:

file
local-exec
remote-exec

In this post, we’ll primarily focus on file and remote-exec type provisioners, because when using Terraform for infrastructure provisioning, we typically need to set up everything on the remote server after the resource is created.

What is File Provisioner?

As the name suggests, the file provisioner copies files from your local machine to a remote server.

The file must exist locally.
Terraform will copy it to the remote machine using a connection.
The connection type must be defined — either SSH for Linux or WinRM for Windows.

provisioner "file" {
  source = "index.html"
  destination = "/home/ubuntu/index.html"
}

This provisioner is used only to copy files or directories, nothing more. So it will have source and destination parameters to define the file to be copied from source and destination location.

What is Remote-Exec Provisioner?

This provisioner helps you run commands on the remote server. It’s useful for post-setup tasks like:

Installing packages
Configuring services
Running shell scripts

provisioner "remote-exec" {
  inline = [ 
    "sudo apt-get update",
    "sudo apt-get -y install apache2",
    "sudo cp -rf /home/ubuntu/index.html /var/www/html/index.html",
    "sudo systemctl start apache2"
   ]
}

For both types, we need connection block to establish a connection to the target resources.

connection {
  type = "ssh"
  user = "ubuntu"
  private_key = file("demov2.pem")
  host = aws_instance.server1.public_ip
}    

Provisioners and connection blocks must be inside your resource block, not outside it.

Steps:

Define the connection block
Add the file provisioner to copy the index.html
Add the remote-exec provisioner to install Apache and move the file

main.tf

provider "aws" {
  region     = "us-east-1"
}

resource "aws_instance" "server1" {
  ami                         = "ami-0e86e20dae9224db8"
  instance_type               = "t2.micro"
  subnet_id                   = "subnet-0105b1aef1e7755cd"
  key_name                    = "demov2"
  ebs_block_device {
    device_name = "/dev/sda1"
    volume_size = 10
  }
  associate_public_ip_address = "true"
  vpc_security_group_ids      = ["sg-04dd813e22c5a0b2f"]
  tags = {
    Name = "server1"
  }

connection {
  type = "ssh"
  user = "ubuntu"
  private_key = file("demov2.pem")
  host = aws_instance.server1.public_ip
}

provisioner "file" {
  source = "index.html"
  destination = "/home/ubuntu/index.html"
}

provisioner "remote-exec" {
  inline = [ 
    "sudo apt-get update",
    "sudo apt-get -y install apache2",
    "sudo cp -rf /home/ubuntu/index.html /var/www/html/index.html",
    "sudo systemctl start apache2"
   ]
}

}

Above file will deploy an ec2 instance on aws and copy index.html to target location.

Then it will setup a apache2 webserver for us as post setup scripts.

Create index.html in your current directory and keep your SSH key for making connection.

Once things are ready, you can apply the terraform configuration.

terraform init
terraform apply --auto-approve

They’re useful for lightweight configurations, especially during resource creation. For long-term, robust service management, consider integrating with config management tools.

Terraform Workspaces: Managing Multiple Environments Efficiently

2025-06-26T19:38:00.000-07:00

We are going to learn about Terraform Workspaces and, most importantly, how to use them to manage multiple environments like Dev, QA, Staging, and Prod efficiently. Previously we have covered the topic - Terraform State File Management Tutorial

You can also Watch this tutorial demo on our YouTube Channel

The Problem: Traditional Environment Management

Usually, when we manage different environments, we will either create separate Git branches in a repository (like a Dev branch, QA, Prod, and so on) or we'll create separate directories like this.

So, the Dev environment will have all the Terraform resource files. The Prod environment will have similar Terraform resource files. Similarly, for other environments, I will copy the same resource file and keep it in a different directory. This is how we generally manage multiple environments, with each directory for each environment having these resource files.

For example, in this Dev directory, which is managing my Dev environment, if you see this main.tf, I have a resource section which is nothing but the creation of an EC2 server in the Dev environment. The same code, the same directory, I will copy to create another environment.

Let's say I want to have a different environment for Prod, then I'll copy this directory and rename the directory to prod, and I will make the changes accordingly. So here, in this Prod directory as well, I have a main.tf which also has the same code. Similarly, if you want to manage multiple environments, this is what we do.

Here is the actual problem: If I want to make a small change, like in the Dev environment I want my instance to be T2.micro, and I want all my environments' resource types to be the same. In this case, if I want to use T2.micro or T3.medium, I need to go and update each main.tf file in every directory, update the value, and then I need to apply it.

But, if you want to manage these environments smartly, where you can use the same resource file and apply it for different environments, this is exactly where Terraform Workspaces come in and make our life easier.

Introducing Terraform Workspaces

A Workspace in Terraform is like an isolated environment that will have its own state file, but it will share the same codebase. If you see this diagram, the Terraform codebase will be the same, but for each environment, we will create a workspace. Each workspace will have a dedicated state file for that particular workspace. Here, we can consider each workspace as an environment. This is the beauty of using Terraform Workspaces.

So, think of it like this: we want to create and manage multiple environments like Dev, Prod, QA. Instead of creating these kinds of different directories and copy-pasting the same code, I have created a single directory called workspace-demo, and I have copied the same code here only once.

Creating and Switching Workspaces

Now, we'll see how to create workspaces, how to switch between different workspaces, and how to apply configurations to them.

This is the directory I'm going to use. Let me first initialize Terraform. Even if you have not created any workspace, by default Terraform has created a workspace called default.

terraform workspace show

Here, by default, Terraform has created a default workspace for you. This is the one you have been using even if you don't know it.

Let's create a new workspace for our environments. How can we create it?

terraform workspace new dev

For example, dev. This is nothing but we are creating a workspace for the dev environment. Once this is created, this will be switched into the newly created workspace. How would you confirm it?

terraform workspace show

Now you are in dev.

Similarly, we'll create another workspace, prod.

terraform workspace new prod

Now you are switched into prod. Again, I will create one more, qa.

terraform workspace new qa

If I run terraform workspace show, this will show me the workspace I'm currently in.

If you want to get a list of workspaces available in your current directory, I can give:

terraform workspace list

So we have this many, and I'm in qa now.

Now, if I want to switch over between any workspaces, I can use terraform workspace select. I can switch to any workspace I want. For example, if I want to get into dev, I can use the workspace name:

terraform workspace select dev

Now I have switched into the dev workspace. You can use list or show to confirm this information.

So, this is how we switch between workspaces. Let's see how we apply the configuration for each workspace.

Applying Configuration Across Workspaces

Now, I have a similar Terraform configuration. I have a provider where I have mentioned my provider is AWS and the region is US East-1. Also, I have a Terraform resource configuration file main.tf. Here, I have defined my resource to create an EC2 instance. You can create multiple resources as you want. For this demo, I'll be using this one resource.

Let's say if you apply this configuration from dev, the server will be created with a name like "server one". This is what we generally do from the default workspace. But smartly, what we can do is dynamically update the value, just like how we use variables. Similarly, we can use the Terraform workspace dynamically. For example, here, in the prefix or suffix, you can define it like:

resource "aws_instance" "server1" {
ami = "ami-0e86e20dae9224db8"
instance_type = "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov2"
ebs_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = "server1-${terraform.workspace}"
}
}

This is my code. So, my server name will be created like server-one-dev. If you want, you can put it in the prefix; it's up to you.

Let's apply this configuration. So right now, I am in dev.

terraform apply -auto-approve

The configuration is applied now, and we are in the dev workspace. So, this configuration is applied for the dev environment. Let me go to my EC2 console and refresh my page.

So, server-one-dev is created. We can think like these resources are dedicatedly created for the dev environment. Similarly, you can create multiple resources. As we have only one single resource, this is fine.

Now, if I want to apply the same code to multiple workspaces, I need to switch over into that particular workspace and then apply it. The point is, I'm using the same Terraform code for multiple environments.

Let's select qa now.

terraform workspace select qa
terraform apply -auto-approve

So the configuration is applied.

Again, I will switch over to the prod environment.

terraform workspace select prod
terraform apply -auto-auto-approve

Okay. Now, if you take this example, we used the same Terraform resource files. I have created multiple workspaces to manage my different environments. I switched over to each workspace and applied the same configuration. If you go to EC2 and refresh the page, now you see server-one-dev, server-one-qa, server-one-prod. So you have servers for each environment.

Understanding State File Management

The same code is used, but the resource gets created independently in each environment. How is it maintaining its state file for each workspace? Because we need a state file, but if you see here, we don't see a TF state file directly. Instead, Terraform has created a directory called .terraform/terraform.tfstate.d. If you expand this, you can see it created a directory for each workspace, and it has saved the state file within that directory. So, it is maintaining the TF state file dedicatedly for each workspace.

Dynamic Configuration with Workspaces

Okay, one more thing: we can dynamically update the values based on the environment's needs. For example, for a lower environment, I can use a small instance type. For a production environment, I can go with a little higher instance type. So, how can we achieve this, especially since we use a single resource configuration? We can use conditions. This topic is already covered in a separate video about conditions, so go through it if you need a refresher.

So, what I can do for the instance type:

resource "aws_instance" "server1" {
ami = "ami-0e86e20dae9224db8"
instance_type = terraform.workspace == "prod" ? "t3.medium" : "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov2"
ebs_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = "server1-${terraform.workspace}"
}
}

This is how we generally define conditions. So, terraform.workspace == "prod" is the condition. If this condition is true (meaning the workspace is prod), then it will pick the first value (t3.medium). If it is false (meaning any other workspace like dev or qa), then it will pick up the second value (t2.micro) as the instance type.

Let me start from dev workspace.

terraform workspace select dev
terraform apply -auto-approve

Here, the condition is false because the workspace is dev. That is why T2.micro will be set. It is already set, so that is why it is refreshing.

Now I will switch over to qa and I will apply the configuration.

terraform workspace select qa
terraform apply -auto-approve

Okay. Now I will switch over to prod then apply the configuration.

terraform workspace select prod
terraform apply -auto-approve

This time, the condition will become true. terraform.workspace will be prod, so it will pick T3.medium as the type. Initially, the instance type was T2.micro (from a previous run), so now a change is detected, and it will apply the new change. That is why we see this "modifying" message.

The configuration is applied in all workspaces. So, let's go and refresh the EC2 page.

So here, if you see, for dev and qa, the instance type is T2.micro, but for production, it is T3.medium.

Deleting Workspaces

Terraform Workspaces help us manage multiple environments using the same set of code. Each workspace has its own isolated state file. We can dynamically use the workspace name to change tags, names, instance types, and other parameters.

Okay, let's say we want to delete a workspace. You can simply run:

terraform workspace delete [workspace_name]

But the condition here is: if you are already in the workspace, you cannot delete it because it is an active workspace.

So, let me switch over to qa.

terraform workspace select qa

From here, I will try to delete the prod workspace. This is what I want to show you: if you are planning to delete a workspace, the first thing you need to do is destroy all the resources within that workspace using terraform destroy. Only if you do a destroy and make it clean, then only you will be able to delete a workspace.

So first, I will destroy all the resources, then I will delete the workspace. As we have only one resource, it is deleting that one resource. If you have multiple resources, it will destroy them accordingly.

Now I will switch over to default.

terraform workspace select default

From here, I can delete all the workspaces. First, I need to switch over to another workspace. From here, I will destroy the resources. Then we'll use prod again. We'll destroy it. Now we'll switch over to dev. Destroy the resources.

terraform workspace list

From here, I will switch over to select dev. Sorry, I have to get into default. From here, I will delete the workspaces one by one. So this is how I need to delete it.

terraform workspace delete dev
terraform workspace delete qa
terraform workspace delete prod

Now we don't have any workspaces other than default. This is how we use terraform workspace delete command.

Hope this article was helpful for you to understand Terraform Workspaces in a simple and practical way. See you in the next topic. Till then, keep practicing and have fun!

Terraform State File Management Tutorial

2025-06-19T22:00:00.000-07:00

If you’re working with Terraform, you’ve probably seen this file pop up: terraform.tfstate.

What is it? Why does Terraform need it? Should you be scared of it? 😅

I’m going to explain everything about the Terraform state file — in simple, everyday language — and show you how to manage it safely. We’ll walk through examples, look at common commands, and even talk about remote state.

In the previous posts, we have covered the topic - "Terraform Conditionals, Loops & Built-in Functions Explained"

Let’s get into it!"

You can also Watch this tutorial demo on our YouTube Channel

What is Terraform State File?

"So first off — what even is this state file?

When you run terraform apply, Terraform actually builds your infrastructure and then writes down what it created.

It keeps track of things like:

what resources it made
their IDs
any settings it used

All of that gets saved in a file called terraform.tfstate.

Think of it like Terraform’s memory. Without it, it forgets what it’s done."

Why Terraform state file is important?

"Now you might be wondering — why does Terraform need to remember anything?

Let’s say you created an EC2 instance on AWS.

Next week, you change the instance type in your code.

Without the state file, Terraform has no idea that the instance even exists — so it might try to create a new one, or worse, delete something by mistake.

The state helps Terraform figure out what to change, and what to leave alone."

"By default, Terraform saves the state file right in your project folder — it's just a normal file called terraform.tfstate.

Let me show you real quick:"

terraform init
terraform apply

"Once that runs, boom — there’s your terraform.tfstate file.

You can open it, but… fair warning — it’s just a big chunk of JSON. Not fun to read. 😅"

How to Inspect Terraform State?

"Thankfully, you don’t have to read that JSON yourself. Terraform gives us easy commands to peek inside the state. Like these:"

terraform show
terraform state list
terraform state show <resource>

"terraform show gives you a summary of everything.

terraform state list will just list all the resources Terraform is tracking.

And terraform state show lets you look at one resource in detail."

Understanding Terraform State Commands 'mv' and 'rm':

"Okay, now let’s talk about two state commands that people get confused about — and they’re super useful once you get the hang of them.

That’s terraform state mv and terraform state rm."

🔄 terraform state mv – Rename a Resource

"This command is like telling Terraform,

‘Hey, I renamed this thing in my code, but don’t worry — it’s still the same resource.’

Here's an example:"

# Old name

resource "aws_instance" "web1" {
  ami                         = "ami-0e86e20dae9224db8"
  instance_type               = "t2.micro"
  subnet_id                   = "subnet-0105b1aef1e7755cd"
  key_name                    = "demov2"
  ebs_block_device {
    device_name = "/dev/sda1"
    volume_size = 10
  }
  associate_public_ip_address = "true"
  vpc_security_group_ids      = ["sg-04dd813e22c5a0b2f"]
  tags = {
    Name = "Web2"
  }
}

# You change it to:

resource "aws_instance" "web2" {
ami = "ami-0e86e20dae9224db8"
instance_type = "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov2"
ebs_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = "Web2"
}
}

"Now, if you don’t do anything, Terraform thinks the old one is gone and the new one is something totally different — so it tries to destroy and recreate it.

We don’t want that.

Instead, run this:"

terraform state mv aws_instance.web1 aws_instance.web2

"Now Terraform knows:

'Ah, okay — you just renamed it. Got it.'

No resources get recreated. Everything stays the same in the cloud. You just updated the state."

🗑️ terraform state rm – Remove from State

"Alright, now let’s talk about terraform state rm. This one removes a resource from the state file.

That means Terraform forgets that the resource exists — but it doesn’t delete it from your cloud provider.

For example:"

terraform state rm aws_instance.web2

"After this, Terraform no longer tracks that ec2 web2 instance.

It still exists in AWS — you can see it in the console — but Terraform won’t manage it anymore.

Why would you do this?

Maybe you’re importing that resource into a different project
Or maybe you want to recreate it cleanly
Or maybe you just don’t want Terraform managing it anymore

Just keep in mind — if the resource is still in your .tf code, and you run terraform apply again, Terraform might try to create it again. So use this carefully."

How to create Terraform Remote Backend?

"Now, what if you don’t want the state file sitting in your folder? Maybe you’re on a different machine, or want to back it up.

You can use something called a remote backend — and the easiest one is S3."

terraform {
backend "s3" {
bucket = "skumarsumar0710"
key = "terraform_demo/terraform.tfstate"
region = "us-east-1"
}
}

"This will store the state file in an S3 bucket. When you run terraform init, it’ll ask you if you want to migrate your local state to remote — say yes.

Terraform Conditionals, Loops & Built-in Functions Explained

2025-06-16T21:21:00.000-07:00

There are some scenario will come to apply the terraform infrastructure based on conditions, functions and loops. On such cases, we can use these feature in terraform codes efficiently. In this post, I will show you how to use terraform conditionals, terraform built-in functions and terraform loops with examples.

In the previous posts, we have covered the topic - "What is Terraform Modules? Explained with Examples".

Let's get started.

You can also Watch this tutorial demo on our YouTube Channel

Terraform Conditionals:

Terraform conditionals can be used to perform some actions based on some conditions, It is very similar to if else condition that we use in any programming languages.

Terraform conditional expression is as follows:

condition ? true_value : false_value

Where,

condition - It is the expression or condition we need to define, Based on the condition result, further value will be set. it generally return Boolean value ie, true or false.

true_value - This value will be set if the condition is true.

false_value - This value will be set if the condition is false.

Lets take an example that if I want to create one aws ec2 resource with instance type "t2.micro" using terraform for my dev environment as below.

variable "environment" {
default = "dev"
}

resource "aws_instance" "web1" {
ami = "ami-0e86e20dae9224db8"
instance_type = "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov2"
ebs_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = "Web Server1"
Environment = var.environment
}
}

What if I want to create one more ec2 resources for production environment where I want my AWS EC2 with instance type "t3.medium" or "t3.large". In general, we will copy the resource section and modify the values accordingly for production environment. But with terraform conditionals, we can simplify the code as below.

variable "environment" {
default = "dev"
}

resource "aws_instance" "web1" {
ami = "ami-0e86e20dae9224db8"
instance_type = var.environment == "prod" ? "t3.large" : "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov2"
ebs_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = "Web Server1"
Environment = var.environment
}
}

Above condition is defined like, if the environment is "prod", "t3.large" value will be set for instance_type. If any other environment except "prod", then "t2.micro" instance_type will be set.

Either, we can modify the variable environment from "dev" to "prod" and apply the terraform or we can use "-var" options along with terraform command to override the environment variable.

terraform apply -var 'environment=prod'

its not only, you have to use terraform conditionals on instance_type. Wherever you think, the conditional logic can be applied, You can give a try.

Terraform Loops:

In terraform, We can achieve the terraform loops concept using for_each and count method.

Terraform loops are generally used when you have multiple values, but you need to use single resources.

For example, Lets say I need to create 10 AWS ec2 instance. On that case, we need to create 10 aws ec2 resource section. Instead, We can create these values in list and run through terraform loops.

Using for_each:

variable "environment" {
default = "dev"
}
variable "ec2_names" {
default = ["web1", "app1", "db1"]
}
resource "aws_instance" "server" {
for_each = toset(var.ec2_names)
ami = "ami-0e86e20dae9224db8"
instance_type = var.environment == "prod" ? "t3.large" : "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov2"
ebs_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = each.value
Environment = var.environment
}
}

This will create each instances through loops based on each values.

Using Count:

variable "environment" {
default = "dev"
}
resource "aws_instance" "server" {
count = 3
ami = "ami-0e86e20dae9224db8"
instance_type = var.environment == "prod" ? "t3.large" : "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov2"
ebs_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = "ec2-${count.index + 1}"
}
}

if we define the value in count parameter, accordingly the resources will be looped and created.

Built-in functions:

Terraform built-in functions are pre defined functions that will help us to achieve or perform some actions. Terraform has lot of built-in functions to simply the code. You can find lot of functions from official terraform website - https://developer.hashicorp.com/terraform/language/functions

List of some Built-in functions:

upper()

lower()

max()

replace()

join()

For example, if I want to change my Aws ec2 names into uppercase. We can use this upper() function to achieve the same as below.

variable "environment" {
default = "dev"
}
resource "aws_instance" "server" {
count = 3
ami = "ami-0e86e20dae9224db8"
instance_type = var.environment == "prod" ? "t3.large" : "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov2"
ebs_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = upper("ec2-${count.index + 1}")
}
}

That's it for this post. Keep practicing and have fun. Leave your comments if any.

What is Terraform Modules? Explained with Examples

2024-09-29T05:37:00.000-07:00

This post will help you to understand what is terraform modules and explained terraform modules with examples. In the previous posts, we have covered the below topics to create AWS VPC using Terraform Resources. So we can create such infrastructure using terraform resources or terraform modules based on our tasks.

If you are interested in learning, Request you to go through the below recommended tutorial.

DevOps Full Course Tutorial for Beginners - DevOps Free Training Online
Docker Full Course Tutorial for Beginners - Docker Free Training Online
Kubernetes Full Course Tutorial for Beginners - Kubernetes Free Training Online
Ansible Full Course Tutorial for Beginners - Ansible Free Training Online
Openstack Full Course Tutorial for Beginners - Openstack Free Training Online

Let's Get Started.

You can also Watch this tutorial demo on our YouTube Channel

What is Terraform Modules?

Terraform modules are reusable units that are very well organized collections of Terraform configurations with infrastructure resources. They really help us to manage any complex infrastructure by grouping related resources, allowing for scalability and maintainence easily. By using modules, we can simplify our code, promote best practices and share the configurations across any different projects. Think of them like building blocks for your cloud infrastructure.

Lets take an example of creating AWS VPC using Terraform Modules.

In the last post, we have created AWS VPC using individual terraform resources. For creating such AWS VPC, We identified around 9 components form VPC architecture. Also we created 1 security group and deployed 2 AWS EC2 on Public Subnet and Private Subnet to test it.

1. AWS VPC (Virtual Private Cloud)

2. AWS Internet Gateway (IGW)

3. AWS Public Subnet 1

4. AWS Private Subnet 1

5. AWS Public Route table 1

6. AWS Private Route table 1

7. AWS Public Route table 1 association with Public Subnet 1

8. AWS Private Route table 1 association with Private Subnet 1

9. AWS Security Group 1

10. AWS EC2 - Web1 (Public Subnet 1)

11. AWS EC2 - DB1 (Private Subnet 1)

For all these list of components, We have found respective terraform resources from terraform registry and created resource section on vpc.tf. You can refer the code here.

If we use terraform modules for the same AWS VPC architecture. we can create the same setup with less than 10 lines.

You can find all terraform modules from here - https://registry.terraform.io/browse/modules

We need to identify the correct terraform modules that will suit for our tasks. So I will be using this terraform provided aws modules called vpc - https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/latest

Here is my vpc.tf where i have copied examples from terraform modules and updated as per my requirement.

1. VPC

2. 2 Public Subnets

3. 2 Private Subnets

4. Security group

5. AWS EC2

With less number of lines in code, it creates a entire AWS VPC. Here you go.

vpc.tf

provider "aws" {
  region = "us-east-1"
}

module "vpc1" {
  source = "terraform-aws-modules/vpc/aws"

  name = "flipkart-vpc-dev"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24"]

  tags = {
    Terraform   = "true"
    Environment = "dev"
  }
}

resource "aws_security_group" "sg1" {
  name = "flipkart-vpc-sg1"
  #   description = "Allow TLS inbound traffic and all outbound traffic"
  vpc_id = module.vpc1.vpc_id

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["115.99.14.198/32"]

  }

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]

  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = {
    Name       = "flipkart-sg1"
    Managed_by = "terraform"
  }
}

resource "aws_instance" "web1" {
  ami                         = "ami-0e86e20dae9224db8"
  instance_type               = "t2.micro"
  subnet_id                   = module.vpc1.public_subnets[0]
  key_name                    = "demov1"
  associate_public_ip_address = "true"
  vpc_security_group_ids      = [aws_security_group.sg1.id]

  tags = {
    Name = "Web1"
  }
}

output "my_web1_public_ip" {
  value = aws_instance.web1.public_ip

}

output "my_web1_private_ip" {
  value = aws_instance.web1.private_ip

}

After making all required changes to vpc.tf manifest files. we have to initialize the directory and apply it.

terraform init
terraform apply --auto-approve

Simply by copying the same modules multiple times, we can create multiple AWS VPC easily for different purposes.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Create AWS VPC using Terraform - AWS Cloud Provisioning

2024-09-09T06:46:00.000-07:00

This post will help you with How to Create AWS VPC using Terraform from basics. In the previous posts, we have covered below topics which are very important for this topic in order to create such AWS cloud provisioning.

Introduction to Terraform

Terraform Basics and Initial Configuration

What is Terraform Resources, Variables & Outputs

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

You can also Watch this tutorial demo on our YouTube Channel

How to Create AWS VPC using Terraform?

If you want to create AWS VPC using terraform, We must have proper cloud infrastrcuture architecture diagram or documents that should show the resources that are required for our AWS Cloud infrastructure. Based on it, we can start creating the Terraform manifests files one by one to provision AWS Cloud.

Lets take an example of above architecture diagram for creating AWS VPC architecture with Terraform..

First, List all the resources that are required to achieve the tasks easily.

1. AWS VPC (Virtual Private Cloud)

2. AWS Internet Gateway (IGW)

3. AWS Public Subnet 1

4. AWS Private Subnet 1

5. AWS Public Route table 1

6. AWS Private Route table 1

7. AWS Public Route table 1 association with Public Subnet 1

8. AWS Private Route table 1 association with Private Subnet 1

9. AWS Security Group 1

10. AWS EC2 - Web1 (Public Subnet 1)

11. AWS EC2 - DB1 (Private Subnet 1)

Second, Identify the correct terraform resources from terraform registry and go through each resources and its example. Copy the example block from each resources.

Adjust the values and fields as per our requirement.

Here is the complete terraform manifest files that will create AWS VPC architecture as per the diagram.

Note: AWS Secret Key and Access Key are configured through AWS CLI, so it will not be available in manifests files, Please use similar approach for AWS authentication.

vpc.tf

# 1. VPC
resource "aws_vpc" "vpc1" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name       = "${var.client_name}-vpc"
    Managed_by = "${var.managed_by}"
  }
}


# 2. Internet Gateway

resource "aws_internet_gateway" "igw1" {
  vpc_id = aws_vpc.vpc1.id

  tags = {
    Name       = "${var.client_name}-igw1"
    Managed_by = "${var.managed_by}"
  }
}

# 3. Public Subnet 1

resource "aws_subnet" "pub_subnet1" {
  vpc_id     = aws_vpc.vpc1.id
  cidr_block = "10.0.1.0/24"

  tags = {
    Name       = "${var.client_name}-pub_subnet1"
    Managed_by = "${var.managed_by}"
  }
}

# 4. Private Subnet 1

resource "aws_subnet" "pri_subnet1" {
  vpc_id     = aws_vpc.vpc1.id
  cidr_block = "10.0.2.0/24"

  tags = {
    Name       = "${var.client_name}-pri_subnet1"
    Managed_by = "${var.managed_by}"
  }
}

# 5. Public RT 1

resource "aws_route_table" "pub_rt1" {
  vpc_id = aws_vpc.vpc1.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw1.id
  }

  tags = {
    Name       = "${var.client_name}-pub_rt1"
    Managed_by = "${var.managed_by}"
  }
}
# 6. Private RT 1

resource "aws_route_table" "pri_rt1" {
  vpc_id = aws_vpc.vpc1.id

  tags = {
    Name       = "${var.client_name}-pri_rt1"
    Managed_by = "${var.managed_by}"
  }
}

# 7. Public subnet 1 association
resource "aws_route_table_association" "pubsub1_rt1" {
  subnet_id      = aws_subnet.pub_subnet1.id
  route_table_id = aws_route_table.pub_rt1.id
}

# 8. Private Subnet 1 association

resource "aws_route_table_association" "prisub1_rt1" {
  subnet_id      = aws_subnet.pri_subnet1.id
  route_table_id = aws_route_table.pri_rt1.id
}

# 9. Security Group 1

resource "aws_security_group" "sg1" {
  name = "${var.client_name}-sg1"
  #   description = "Allow TLS inbound traffic and all outbound traffic"
  vpc_id = aws_vpc.vpc1.id

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["115.99.14.198/32", aws_vpc.vpc1.cidr_block]

  }

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]

  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }



  tags = {
    Name       = "${var.client_name}-sg1"
    Managed_by = "${var.managed_by}"
  }
}

# 10. Ec2 - web1

resource "aws_instance" "web1" {
  ami                         = "ami-0e86e20dae9224db8"
  instance_type               = var.my-instance-type
  subnet_id                   = aws_subnet.pub_subnet1.id
  key_name                    = "demov1"
  associate_public_ip_address = "true"
  vpc_security_group_ids      = [aws_security_group.sg1.id]


  tags = {
    Name       = "${var.client_name}-web1"
    Managed_by = "${var.managed_by}"
  }
}

# 11. Ec2 - DB1

resource "aws_instance" "db1" {
  ami           = "ami-0e86e20dae9224db8"
  instance_type = var.my-instance-type
  subnet_id     = aws_subnet.pri_subnet1.id
  key_name      = "demov1"

  vpc_security_group_ids = [aws_security_group.sg1.id]


  tags = {
    Name       = "${var.client_name}-db1"
    Managed_by = "${var.managed_by}"
  }
}

output "my_web1_public_ip" {
  value = aws_instance.web1.public_ip

}

output "my_web1_private_ip" {
  value = aws_instance.web1.private_ip

}

output "my_db1_private_ip" {
  value = aws_instance.db1.private_ip

}

Above vpc.tf manifest file is provided each resources and blocks to create all AWS VPC resources that is listed earlier.

Also I have used the advatanges of variables and updated the variable.tf and terraform.tfvars accordingly.

provider.tf

provider "aws" {
  region = "us-east-1"
}

variable.tf

variable "my-instance-type" {
  type    = string
  default = "t2.micro"
}

variable "client_name" {
  default = "my-default"
}
variable "managed_by" {
  default = "devops"
}

terraform.tfvars

my-instance-type = "t2.micro"
client_name      = "coke"
managed_by       = "terraform"

Once all these terraform manifests files are ready, we can plan and apply the terraform codes as below.

terraform plan
terraform apply --auto-approve

After terraform apply is completed, you can see the output of IP address as we have used output blocks in vpc.tf. Also you can login into AWS Cloud and verify all the resources are created as expected.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

What is Terraform Resources, Variables & Outputs

2024-08-31T10:06:00.000-07:00

This post will help you to understand what is terraform resources, terraform variables and terraform outputs. Each topic is explained with examples. In the previous post, we have already covered Introduction to Terraform and Terraform Basics and Initial Configuration.

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

Watch this tutorial demo on our YouTube Channel

What is Terraform Resources?

Terraform resources are the important section that we define in terraform manifest files. These terraform resources will be used to manage specific services or applications.

resource "resource_name" "name1" {
key1 = value1
key2 = value2
}

Lets take an example. If we want to create EC2 instance in AWS cloud. Then we have to use the terraform resources which will help us to manage AWS EC2 on AWS along with attributes like AMI ID, Instance type, Subnet ID, Security Group and so on.

resource "aws_instance" "web1" {
ami = "ami-0e86e20dae9224db8"
instance_type = "t2.micro"
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov1"
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = "Web Server1"
}
}

Where,

1. "aws_instance" is the resource type provided by terraform to manage AWS EC2 instance. Similarly we may have more resources types to manage different resources like AWS S3, AWS EBS, AWS VPC, etc,.

We have to find the correct resource types that is suitable to create the terraform resources as per the request. These resources can be find in terraform registry.

2. "web1" is the unique resource name should be used, this will be useful for terraform to manage the resources. For example, if we want to create multiple AWS EC2 instance, we cannot provide name like just "web". We have to call each resource with unique names like web1, web2, user1, user2.

3. Within resource section, lot of attributes are defined that are required to create the specific resources.

Lets say, we want to create AWS S3 bucket. On this case, we have to use the terraform resources called "aws_s3_bucket". We found these resource name from terraform registry and its supporting attributes.

resource "aws_s3_bucket" "my_bucket1" {
bucket = "my-unique-bucket-name-123456"
acl = "private"

tags = {
Name = "MyBucket"
}
}

Once we have defined all resources that we need to manage, then we can go for "terraform plan" or "terraform apply" command to proceed further.

Lets move on to Terraform Variables.

Terraform Variables

Generally, variables are used to define a key value pair that will make our codes more flexible and reusable. Once variable is defined, we can refer the variable many times in all the terraform manifests files whereever we want.

For example, if i create 10 AWS ec2 instances with multiple resource sections with different EBS volume size and AMI ID for our applications. And all our instance types are same type like t2.micro. On this case, we dont want to specify t2.micro in all resource section. Instead. we can refer the variable name that we create for instance type.

Even this will help us to change the instance type only on variable if we want to modify it. If we dont have such variable option. We have to modify in all resource sections.

Lets see how to define terraform variables. In order to define variables in terraform, we have to create a variable section in any manifests file or create a dedicated variable.tf file.

variable "instance_type" {
description = "Our basic AWS EC2 instance type"
type = string
default = "t2.micro"
}

Where,

"instance_type" is the variable name. This can be used in any resources sections whereever we want.

description is used to provide some information about the variable.

type is used to define the type of the resources. Eg, it may be string, integer, number, boolean, list, tuples.

default is the actual value where we define the value for the variable. This will be default value used for the variable, if no variable values is overridden. because, we can override variables using multiple ways, will talk about this in later section.

resource "aws_instance" "web1" {
ami = "ami-0e86e20dae9224db8"
instance_type = var.instance_type
subnet_id = "subnet-0105b1aef1e7755cd"
key_name = "demov1"
associate_public_ip_address = "true"
vpc_security_group_ids = ["sg-04dd813e22c5a0b2f"]
tags = {
Name = "Web Server1"
}
}

In the above example, if you see the instance_type attribute value, we used "var.instance_type". If you want to refer variable names, then we have to use the format like "var." then variable name.

As I said before, variable section that we specify in manifest file is the default value will be used. There are different ways to define variables also it has priority in order to override the variables. I will list the same in ascending order.

1. Command-Line Flags: We can pass terraform variables while running the terraform commands like below. This has the high priority.

terraform apply -var="instance_type=t2.micro"

2. Environment Variables: We can set values for variables using environment variables like below. As we use environment variables features of operating system, we have to use the prefix like TF_VAR to differentiate the terraform variables from other environment variables.

export TF_VAR_instance_type="t2.micro"

3. Terraform variables (.tfvars): We can create dedicated file with extension .tfvars where we can define variables in key value pair format like below.

instance_type = "t2.micro"

Lets move on to Terraform output.

Terraform Output

Terraform outputs are used to display the attribute value of any resource after it is created or updated. For example, if we create AWS EC2 isntance using terraform. We are not aware of the the attributes like public ip, private ip before creating the resources. These values will be created on AWS provider. So these values will be issued to terraform only after it is applied. So I want to know the such attributes, we have to login into AWS console and get the information. Instead we can use terraform output block to get the attribute.

output "instance_id" {
value = aws_instance.web1.id
}

output "instance_public_ip" {
value = aws_instance.web1.public_ip
}

Above output block will just display the instance ID and Public IP of the resources on the terminal itself.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Terraform Basics and Initial Configuration

2024-08-07T20:23:00.000-07:00

This post will help you to understand the terraform basics and how to do the terraform initial configurations. In the previous post, we have covered the terraform introduction, you can refer it if you want to start from terraform introduction?

Topics Covered:

Terraform Basic Files
Terraform Providers
How Terraform Works?
Terraform Workflow with Commands

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

You can also watch this entire tutorial on our Youtube channel with demo.

Terraform Basic Files

In terraform, we do everything through files. So we should have knowledge on what are the types of terraform files we have and its purpose. Lets understand the terraform files first.

TF files (.tf)

Terraform files are created with .tf extension. Filename can be anything but .tf extension is must. Terraform will read only .tf files and other terraform files, It won't read any other file types.

Examples:

main.tf

ec2.tf

vpc.tf

busapp_vpc.tf

iam_developer.tf

State files (.tfstate)

Terraform always maintains a state file called terraform.tfstate whenever you apply your changes. This file will track the current state of your target infrastructure resources and store it in terraform.tfstate fille.

Example:

terraform.tfstate

terraform.tfstate.backup (This file is backup of previous terraform.tfstate file.)

Variable files (.tfvars)

Terraform provides a dedicated files to define any variables that is needed for our infrastructure. These variables can be called in any of our terraform files wherever needed. We can create our variables files with extension .tfvars.

Examples:

variables.tfvars

dev_env.tfvars

production.tfvars

Terraform Providers

Terraform providers are the most important component that tells the terraform to interact with cloud providers, API's, services. So providers are responsible to manage the resources on the target infrastructure.

Obviously we will write terraform files to manage some target infrastructure. That infrastructure is providers. We need to specify the providers (eg. AWS, Google Cloud, Azure, VMware) in the terraform files.

Terraform supports lot of providers. That is why terraform is so popular.

Here is the link to find all supported terraform providers - Browse Providers | Terraform Registry

Lets say, you have decided to manage your AWS infrastructure. In that case, your provider is AWS.

If we define the below provider section in terraform files, terraform will make sure to download all the necessary plugins and codes to talk to aws cloud api's so that we would be able to create or manage any resources within the AWS cloud environment. Also we have to create CLI based token on AWS like aws_access_key and aws_secret_key to manage resource through terraform tools. For any providers you choose, it should support such authentication, then only you can manage through cli.

provider "aws" {
region = "us-west-2"
access_key = "my-access-key"
secret_key = "my-secret-key"
}

Once terraform provider is defined, we can start writing further terraform files to create or manage any resources of the infrastructure. We will talk about how to create terraform resources in the next post.

How Terraform Works?

Terraform reads all your terraform files, starting with provider block, followed by all resources blocks.

Then it builds a plan to create, update or delete resources based on the terraform configurations.

We need to provide instructions to terraform for the workflow. Lets understand the terraform workflow with commands.

Terraform Workflow

1. First, Write terraform files.

2. Initialize terraform directory - "terraform init"

3. Validate terraform configuration - "terraform validate"

4. Generate execution plan and identify the changes - "terraform plan"

5. Apply terraform changes - "terraform apply"

Terraform Commands

terraform init - This command will initialize the current terraform directory and will download all necessary plugins, source codes that are needed to manage the infrastructure. It is based on the provider block you have defined in terraform files.

Once if you initialize your terraform directory, terraform will recognize all the terraform files.

terraform validate - This terraform validate command helps you to validate for syntax errors in your terraform files.

terraform plan - This command will be like a dry run. It will generate the execution plan and will show you the list of resources that are going to be affected if we apply the changes. So before applying any changes, its good to run the terraform plan command to understand the impact.

terraform apply - This terraform apply command will be the main command that will apply the changes to your infrastructure based on the terraform configuration files you have created.

terraform destroy - This command will delete all your resources on your infrastructure that is created through your terraform files.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Introduction to Terraform - Terraform Tutorial for Beginners

2024-07-29T22:51:00.000-07:00

In this post, we will take you through the Introduction to Terraform. This post will help you to understand the below topics.

Topics Covered:

What is Infrastructure as a Code (IaC)?
List of IaC Tools
Introduction to Terraform
Why should we use Terraform and Its Benefits?
How Terraform works with Key Concepts
Terraform Installation on Windows & Linux

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

You can also watch this entire tutorial on our youtube channel with demo.

What is Infrastructure as a Code (IaC)?

Infrastructure as Code (IaC) helps us to automates the infrastructure provisioning by using codes, scripts or any tools instead of creating the infrastructure manually. Anything you create with codes, files, scripts or using any tools, those will be considered as Infrastructure as Code (IaC).

We can provision, manage or destroy our entire infrastructure using Infrastructure as code (IaC). It enables our organization to completely develop, deploy and scale cloud applications with greater speed, less risk and reduced cost.

List of IaC Tools

Here are some of the most popular IaC Tools:

Terraform
AWS CloudFormation
Azure Resource Manager
Google Cloud Deployment Manager
Ansible
Chef
Puppet
SaltStack
Vagrant

Introduction to Terraform?

Terraform is an open-source infrastructure as code (IaC) tool created by HashiCorp for automating or provisioning infrastructure easily.

Users has to create files, define resources, modules and provision entire data center infrastructure easily, securely and efficiently using declarative configuration language known as HashiCorp Configuration Language (HCL), or optionally JSON.

Even Terraform can manage our existing and any popular service providers as well as custom in house solutions.

Why should we use Terraform and Its Benefits?

Infrastructure as Code (IaC) - Terraform Manages our entire infrastructure through codes, making our task easier to create, update and destroy it.
Consistency - Terraform ensure the same configuration defined in terraform files are applied across multiple environments like development, staging and production.
Automation - Terraform reduces human errors, syntax related issues and manual setup by automating infrastructure provisioning.
Version Control - Terraform has its inbuilt code tracking feature, so our infrastructure changes will be tracked in code, making easier to review and revert changes at any time.
Collaboration - Team members can work together on the same infrastructure configurations that improves collaboration from every team members and it increases the productivity.
Scalability - Using Terraform, We can easily scale our entire infrastructure up or down based on our project requirements.
Multi-Cloud - Terraform can manage the resources available across any different cloud providers from single terraform configuration.

How Terraform works with Key Concepts

Create files: Firstly, Create our infrastructure resources in .tf files using HashiCorp Configuration Language (HCL).
Terraform Init: Execute the command 'terraform init' to initialize the working directory and download the required providers plugins to connect with providers using API.
Terraform Plan: Execute the command 'terraform plan' to validate and review the changes that is going to be affected before applying any changes. Its like a dry run.
Terraform Apply: Execute the command 'terraform apply' to apply the desired changes to the infrastructure.
Review State: Once applied, state files will be created for tracking the desired state. It will hold the current configuration state of the infrastructure.
Terraform Destroy: Execute the command 'terraform destroy' to completely remove the infrastructure managed by Terraform codes.

Terraform Installation on Windows & Linux

Installing terraform is a straight forward procedure. We have to update the terraform official repository and we can install the terraform packages. I have provided the commands and instruction on how to install terraform on specific platforms.

How to Install Terraform on Ubuntu?

Download terraform gpg key, add terraform repository for the respective ubuntu releases and install terraform.

wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

How to Install Terraform on RHEL / CentOS?

Install required dependency packages and update terraform repository to install terraform.

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
sudo yum -y install terraform

How to Install Terraform on Windows?

Step 1: Use the below official links to download the respective binary files based on your windows platform.

for i386 - https://releases.hashicorp.com/terraform/1.9.3/terraform_1.9.3_windows_386.zip

for amd64 - https://releases.hashicorp.com/terraform/1.9.3/terraform_1.9.3_windows_amd64.zip

If link not worked, go to Terraform official website and download the respective binary file from the link directly.

https://www.terraform.io/downloads.html

Step 2: Unzip the downloaded zip file and copy the executable file to binary locations.

C:\Windows\System32

That's it for this post. Keep practicing and have fun. Leave your comments if any.

How to Configure Docker Containers as Jenkins Build Agents?

2024-06-06T06:26:00.000-07:00

This tutorial will show you the entire step by step procedure to configure docker containers as jenkins build agents for our jenkins build.

Jenkins build agents or build slaves usually setup for distributing the jenkins jobs and also resource utilization of these jenkins slaves are very less, These jenkins slave agents will run only when builds are scheduled. Else it will be removed.

Lets assume you have already a running jenkins server. If not please go through this link How to install jenkins on Ubuntu?

Server Specification:

Operating System: Ubuntu 20.04

Jenkins Server IP: 192.168.2.21

Docker Server IP: 192.168.2.22

As we are going to setup the agent on docker. We need to install docker on server. For this purpose, I use the server (192.168.2.22)

First, lets install and configure the docker server ready for jenkins..

Install Docker on Ubuntu 20.04:

Use apt command to update the repository and install it as below.

sudo apt-get update
sudo apt-get install docker.io

Configure Docker Remote Host:

Edit the docker service file and change the line to allow the remote API for Docker host.

Search for ExecStart and replace that line with given line.

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock

Reload and restart docker service:

sudo systemctl daemon-reload
sudo service docker restart

Validate Docker Remote Host API

Use curl command to easily validate the docker remote host API from localhost.

curl http://localhost:4243/version

Output:

{"Platform":{"Name":""},"Components":[{"Name":"Engine","Version":"24.0.5","Details":{"ApiVersion":"1.43","Arch":"amd64","BuildTime":"2023-08-21T19:50:14.000000000+00:00","Experimental":"false","GitCommit":"24.0.5-0ubuntu1~20.04.1","GoVersion":"go1.20.3","KernelVersion":"5.4.0-163-generic","MinAPIVersion":"1.12","Os":"linux"}},{"Name":"containerd","Version":"1.7.2","Details":{"GitCommit":""}},{"Name":"runc","Version":"1.1.7-0ubuntu1~20.04.2","Details":{"GitCommit":""}},{"Name":"docker-init","Version":"0.19.0","Details":{"GitCommit":""}}],"Version":"24.0.5","ApiVersion":"1.43","MinAPIVersion":"1.12","GitCommit":"24.0.5-0ubuntu1~20.04.1","GoVersion":"go1.20.3","Os":"linux","Arch":"amd64","KernelVersion":"5.4.0-163-generic","BuildTime":"2023-08-21T19:50:14.000000000+00:00"}

Login into the Jenkins server and validate the docker remote host to make sure there is a connection between jenkins and docker host.

curl http://192.168.2.22:4243/version

Output:

{"Platform":{"Name":""},"Components":[{"Name":"Engine","Version":"24.0.5","Details":{"ApiVersion":"1.43","Arch":"amd64","BuildTime":"2023-08-21T19:50:14.000000000+00:00","Experimental":"false","GitCommit":"24.0.5-0ubuntu1~20.04.1","GoVersion":"go1.20.3","KernelVersion":"5.4.0-163-generic","MinAPIVersion":"1.12","Os":"linux"}},{"Name":"containerd","Version":"1.7.2","Details":{"GitCommit":""}},{"Name":"runc","Version":"1.1.7-0ubuntu1~20.04.2","Details":{"GitCommit":""}},{"Name":"docker-init","Version":"0.19.0","Details":{"GitCommit":""}}],"Version":"24.0.5","ApiVersion":"1.43","MinAPIVersion":"1.12","GitCommit":"24.0.5-0ubuntu1~20.04.1","GoVersion":"go1.20.3","Os":"linux","Arch":"amd64","KernelVersion":"5.4.0-163-generic","BuildTime":"2023-08-21T19:50:14.000000000+00:00"}

At this point, docker host is ready.

In order to run docker container as jenkins build agent, we need to create our docker images for jenkins build agents.

This docker image should be ready with all required packages for jenkins projects. For example, if our projects depends on git, maven, ant, artifactory. On that case, we have to install everything along with docker images.

So here is my basic dockerfile that I use for jenkins build agents.

FROM ubuntu:20.04
RUN apt-get update && \
apt-get install -qy openjdk-13-jdk && \
apt-get install -qy git && \
apt-get install -qy maven && \
apt-get install -qy openssh-server && \
mkdir -p /var/run/sshd && \
apt-get -qy autoremove && \
useradd -m jenkins -s /bin/bash && \
mkdir -p /home/jenkins/.ssh/
COPY id_rsa.pub /home/jenkins/.ssh/authorized_keys
RUN chown -R jenkins:jenkins /home/jenkins/.ssh/
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]

Above dockerfile will build the docker image from Ubuntu 20 and will install required packages, user creations and other configurations required.

Also there is a line mentioned to copy SSH public key to a SSH folder. This key will be used by jenkins while launching the docker container to login and perform actions as per the jenkins build. So lets create SSH key.

selva@docker-host:~$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/selva/.ssh/id_rsa): Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/selva/.ssh/id_rsa
Your public key has been saved in /home/selva/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:1lEHdHxUMF8rJDdfky9wsrMXQTcLkPrh4AsAtL+eyec selva@docker-host
The key's randomart image is:
+---[RSA 3072]----+
| .. +*O*=O|
| .. o*oB=O|
| .. o * *o|
| .. + oo + .|
| .. S = .o o |
| .o . o. . |
| . . . . |
| o o. . |
| =oE |
+----[SHA256]-----+
selva@docker-host:~$

At this point, we will have both public and privay key in the same directory. So we can build the docker images.

Build Docker Image for Jenkins Build Agents:

As you are in the same directory, we can build the docker image using the below command.

sudo docker build -t learnitguide/jenkins-slave-agent:1.0 .

Replace with your docker hub repository name (learnitguide/jenkins-slave-agent) and tag (1.0) as per your need.

Our docker image is ready.

Push Docker Image to Docker Registry:

Login into docker hub registry using below command and push the image to docker hub. Then only our image will be pulled from jenkins to launch the docker container.

sudo docker login

Use "docker push" command to push the docker image to docker hub registry.

sudo docker push learnitguide/jenkins-slave-agent:1.0

Now the jenkins build agent docker image is pushed to our docker image registry. This image will be used on jenkins to build the agents.

Lets go to jenkins and configure the Jenkins Build agents.

Configure Jenkins Build Agents:

It required more steps to be done, so you can refer the below youtube video.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

Accessing Kubernetes Service from Localhost

Helm: Render Chart Templates Locally

How to Configure DNS for Applications Deployed on Kubernetes?

How to Install Jenkins on CentOS 8?

2023-10-17T19:36:00.003-07:00

This article will provide you the step-by-step guide on how to install Jenkins on CentOS 8:

In the previous post, We have covered below topics on Jenkins.

1. Jenkins Introduction, Advantages, CI/CD Workflow Explained

2. How to Install and Configure Jenkins on Ubuntu 20.04?

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

Prerequisites:

1. Centos 8 or CentOS 8 Steam Installed Server

2. Internet must be enabled to fetch and install required packages.

3. Admin privileges (root or sudo access)

Steps to Install Jenkins on CentOS 8

Step 1: Update the System

Before installing Jenkins, it's essential to ensure your CentOS system is up to date. Open your terminal and run the following commands:

sudo yum update

Step 2: Install Java

Jenkins requires Java to run. You can install OpenJDK, which is open-source and compatible with Jenkins:

sudo yum -y install java-11-openjdk-devel

After the installation is complete, verify the Java version:

java -version

Step 3: Add Jenkins Repository

To install the latest version of Jenkins, add the Jenkins repository:

sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat/jenkins.io-2023.key

Step 4: Install Jenkins

Now, you can install Jenkins using the following command:

sudo yum -y install jenkins

Step 5: Start and Enable Jenkins

Once Jenkins is installed, start the service and enable it to start on boot:

sudo systemctl start jenkins
sudo systemctl enable jenkins

Step 6: Check Jenkins Status

To verify that Jenkins is running, use the following command:

sudo systemctl status jenkins

You should see a message indicating that Jenkins is active and running.

Step 7. Allow Port 8080 on Firewall:

If you have firewalld enabled on your server, then we must allow our Jenkins default port 8080 to make it accessible from network, if not we can ignore this step.

To allow incoming traffic on port 8080, run:

sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload

Step 8: Unlock Jenkins

Jenkins is initially locked, and you need to retrieve the initial admin password to unlock it. Run this command to get the password:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

This command will display the password you need to unlock Jenkins. Copy it.

Step 9: Access Jenkins Web Interface

Open a web browser and enter your server's IP address followed by port 8080, like this: http://your_server_ip:8080.

You will be prompted to enter the initial admin password obtained in Step 7. Paste the password and click "Continue."

Step 10: Install Recommended Plugins

Select the "Install suggested plugins" option. Jenkins will begin downloading and installing the required plugins.

Step 11: Create an Admin User

After the plugins are installed, you'll be prompted to create an admin user. Fill in the required details.

Step 12: Start Using Jenkins

Once you've completed the setup, Jenkins is ready to use. You can start creating and managing your Jenkins jobs to automate your development workflows.

By following these steps, you've successfully installed Jenkins on your CentOS 8 server, and you're now ready to harness its automation capabilities for your projects.

Additional Configurations on Jenkins:

Any Users created in jenkins console are only to manage jenkins through web browser. Bydefault, user 'jenkins' only runs everything in background even if you have loggedin using any users on portal and this user "jenkins" is created during the installation at the Operating system level, but it is a non-root user.

cat /etc/passwd | grep -i jenkins

Output:

jenkins:x:114:118:Jenkins,,,:/var/lib/jenkins:/bin/bash

So When user "jenkins" doesnt have sufficient previleges at the Operating system level, this user cannot perform all administrative level tasks. So we must configure sudo to gain the administrative level access.

SUDO Access:

Edit /etc/sudoers file and add below entry to allow user "jenkins" to use sudo with nopasswd prompt and also disable requiretty option.

jenkins ALL=(ALL) NOPASSWD: ALL
Defaults !requiretty

Change Jenkins user shell:

Verify the "jenkins" user shell. If it is already specified with "/bin/bash", You can ignore this. If not, we must change the user shell, this would help us to perform any tasks using CLI.

cat /etc/passwd | grep -i jenkins

Output:

jenkins:x:995:992:Jenkins Automation Server:/var/lib/jenkins:/bin/bash

Restart Service:

sudo systemctl restart jenkins

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

How to Configure Jenkins Mail Notification on Build Failures

How to Configure Jenkins behind Proxy to Install Plugins

How to Reset Forgotten Jenkins Admin User's Password

How to Install & Configure Jenkins on Linux Easily

How to Install and Configure Jenkins on Ubuntu 20.04?

2023-10-17T02:09:00.005-07:00

If you're running Ubuntu 20.04 and want to install Jenkins easily, you're in the right place. In this guide, we'll walk you through the process of How to Install and Configure Jenkins on Ubuntu 20.04, step by step, along with commands and examples to make the journey smoother.

In the previous post, We have covered below topics, You can refer the link "Jenkins Introduction, Advantages, CI/CD Workflow Explained".

1. Introduction to Jenkins

2. Advantages of Jenkins

3. CI/CD Workflow Explained with Examples.

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

Prerequisites:

1. Ubuntu 20.04 Installed Server

2. Internet must be enabled to fetch and install required packages.

3. Admin privileges (root or sudo access)

Steps to Install Jenkins on Ubuntu 20.04

Step 1: Update the System

The first step is to ensure your Ubuntu system is up to date. Open a terminal and run the following commands:

sudo apt-get update

This will update the package lists and upgrade the installed packages to their latest versions.

Step 2: Install Java

Jenkins requires Java to run. You can install OpenJDK, which is open-source and compatible with Jenkins:

sudo apt-get -y install openjdk-11-jdk

After the installation is complete, verify the Java version:

java -version

Output:

openjdk version "11.0.20.1" 2023-08-24
OpenJDK Runtime Environment (build 11.0.20.1+1-post-Ubuntu-0ubuntu120.04)
OpenJDK 64-Bit Server VM (build 11.0.20.1+1-post-Ubuntu-0ubuntu120.04, mixed mode, sharing)

Step 3: Add Jenkins Repository

You need to add the Jenkins repository to your system to install the latest version. Import the GPG key and add the Jenkins repository:

wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'

Step 4: Install Jenkins

Now, you can update the repositories to fetch latest packages:

sudo apt-get update

If you noticed below GPG error, We must add the key manually to make it work. If not, Ignore this step.

Reading package lists... Done
W: GPG error: https://pkg.jenkins.io/debian-stable binary/ Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 5BA31D57EF5975CA
E: The repository 'http://pkg.jenkins.io/debian-stable binary/ Release' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Resolve the above GPG error using below command.

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5BA31D57EF5975CA

Output:

Executing: /tmp/apt-key-gpghome.uLnySvtJYM/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys 5BA31D57EF5975CA
gpg: key 5BA31D57EF5975CA: public key "Jenkins Project <jenkinsci-board@googlegroups.com>" imported
gpg: Total number processed: 1
gpg: imported: 1

Now try installing Jenkins using apt-get command.

sudo apt-get -y install jenkins

Step 5: Start and Enable Jenkins

Once Jenkins is installed, start the service and enable it to start on boot:

sudo systemctl start jenkins
sudo systemctl enable jenkins

Step 6: Check Jenkins Status

To verify that Jenkins is running, use the following command:

sudo systemctl status jenkins

You should see a message indicating that Jenkins is active and running.

Step 7: Unlock Jenkins

Jenkins is initially locked, and you need to retrieve the initial admin password to unlock it. Run this command to get the password:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

This command will display the password you need to unlock Jenkins. Copy it.

Step 8: Access Jenkins Web Interface

Open a web browser and enter your server's IP address followed by port 8080, like this: http://your_server_ip:8080.

You will be prompted to enter the initial admin password obtained in Step 7. Paste the password and click "Continue."

Step 9: Install Recommended Plugins

Select the "Install suggested plugins" option. Jenkins will begin downloading and installing the required plugins.

Step 10: Create an Admin User

After the plugins are installed, you'll be prompted to create an admin user. Fill in the required details.

Step 11: Start Using Jenkins

Once you've completed the setup, Jenkins is ready to use. You can start creating and managing your Jenkins jobs to automate your development workflows.

Additional Configurations on Jenkins:

cat /etc/passwd | grep -i jenkins

Output:

jenkins:x:114:118:Jenkins,,,:/var/lib/jenkins:/bin/bash

SUDO Access:

Edit /etc/sudoers file and add below entry to allow user "jenkins" to use sudo with nopasswd prompt and also disable requiretty option.

jenkins ALL=(ALL) NOPASSWD: ALL
Defaults !requiretty

Change Jenkins user shell:

Verify the "jenkins" user shell. If it is already specified with "/bin/bash", You can ignore this. If not, we must change the user shell, this would help us to perform any tasks using CLI.

cat /etc/passwd | grep -i jenkins

Output:

jenkins:x:995:992:Jenkins Automation Server:/var/lib/jenkins:/bin/bash

Restart Service:

sudo systemctl restart jenkins

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

How to Configure Jenkins Mail Notification on Build Failures

How to Configure Jenkins behind Proxy to Install Plugins

How to Reset Forgotten Jenkins Admin User's Password

How to Install & Configure Jenkins on Linux Easily

Jenkins Introduction, Advantages, CI/CD Workflow Explained

2023-10-16T08:52:00.001-07:00

Introduction to Jenkins:

Jenkins is an open-source automation server and integration tool that helps us for continuous integration and continuous delivery (CI/CD) processes in software development. It is widely used in the software industry to automate various aspects of the software development lifecycle, from building and testing code to deploying applications.

It is not only can be used in software delivery, it can be used in any other automation like system administration tasks, infrastructure provisioning. And it is written on java so it can be installed and used from any operating system where java is installed. So it supports multiple platforms.

If you are interested in learning, Request you to go through the below recommended tutorial.

You can also watch this tutorial video on our youtube channel.

Advantages of Jenkins:

Continuous Integration (CI):

Jenkins automates the process of integrating code changes into a shared repository, allowing for frequent and automated testing of code. It detects and reports integration issues early in the development process, improving software quality. CI in Jenkins reduces the risk of integration problems during later stages of development.

Continuous Delivery (CD):

In addition to CI, Jenkins can be extended to support CD. CD involves automating the deployment of code changes to various environments, such as development, staging, and production, in a consistent and reliable manner. Jenkins can help manage and orchestrate the entire CD pipeline.

Open Source and Community:

Jenkins is an open-source project with a strong community of contributors and users. This community continuously develops and maintains plugins and extensions, offering support through forums, documentation, and third-party resources.

Automation:

Jenkins enables the automation of repetitive and time-consuming tasks in the software development process. It allows developers to define and execute jobs, which can include building applications, running tests, and deploying to different environments, all without manual intervention.

Plugin Architecture:

A huge ecosystem of plugins is available from Jenkins to increase its capability. These plugins help integrate Jenkins with various tools, technologies, and platforms. The plugin architecture allows you to tailor Jenkins to your specific project requirements.

Easy to Use:

Jenkins provides a user-friendly web-based interface that simplifies configuration and job management. It has a low learning curve, making it accessible to both developers and non-technical team members.

Supports Multiple Platforms:

Jenkins is platform-agnostic and can run on various operating systems, including Windows, Linux, macOS, and more. This cross-platform compatibility ensures flexibility in choosing your development and deployment environment.

Distributed Builds:

Jenkins can be set up with a master-slave architecture, allowing you to distribute build and testing workloads across multiple machines. Distributed builds improve performance, reduce build times, and enhance scalability.

Customizable:

Jenkins is highly customizable, allowing you to define and configure build and deployment pipelines as code using Jenkinsfiles or Pipeline DSL. You can tailor Jenkins to meet your specific project requirements and workflows.

Robust Security:

Jenkins offers security features to protect your CI/CD pipelines and sensitive data. It supports user authentication, role-based access control, and integration with external security systems. This ensures that your CI/CD processes remain secure.

Comprehensive Testing:

Jenkins integrates seamlessly with various testing tools and frameworks, allowing you to perform a wide range of tests, including unit tests, integration tests, and functional tests.

Comprehensive testing helps identify and fix issues early, ensuring software quality.

Scalability:

Jenkins is highly scalable and can handle the needs of both small and large organizations. With distributed builds, you can scale horizontally by adding more build agents as your projects grow.

Web Interface:

Jenkins provides a web-based user interface for configuration, job management, and monitoring. This makes it accessible to both developers and operations teams, with support for role-based access control.

CI/CD workflow Example:

Here's a simple example of a CI/CD workflow, from code development to deployment stage that involves various stages.

Code Development:

Developers write and commit code to a version control system (e.g., Git). Code changes are typically made on feature branches.

Continuous Integration (CI) Phase:

When code is committed, the CI server (e.g., Jenkins) detects the changes. The CI server automatically triggers a build process.

Build Stage:

The CI server checks out the latest code from the repository. It compiles the code and creates an executable or artifact. The build may include unit tests to ensure code quality. If the build or tests fail, the team is notified.

Continuous Delivery (CD) Phase:

In a CD setup, the CI server automatically moves to the deployment phase.

Deployment Stage:

The CI/CD pipeline deploys the artifact to a staging environment. In the staging environment, more extensive testing (integration, performance, etc.) may occur. If the staging tests fail, the team is alerted.

Automated Testing:

Automated testing scripts are executed in the staging environment. These tests ensure that the application works as expected and performs well.

Manual Testing (if required):

Some organizations might have a manual testing step where the QA team performs additional tests.

Approval (if required):

In some cases, manual approval may be needed before proceeding to production.

Deployment to Production:

Once all tests pass and any required approvals are obtained, the CI/CD pipeline deploys the artifact to the production environment.

Continuous Monitoring:

After deployment, the application is continuously monitored for performance, security, and other metrics. Any issues detected trigger alerts for the operations team.

Rollback (if necessary):

If issues arise in production, an automated or manual rollback process may be initiated to revert to the previous stable version.

Logging and Reporting:

The CI/CD pipeline logs all activities and results, providing a record for auditing and analysis.

This is a simplified overview of a CI/CD workflow, and the actual implementation can vary depending on the organization, tools used, and specific requirements. CI/CD promotes a streamlined and automated process from code development to deployment, enabling rapid and reliable software releases while maintaining high quality and reducing manual intervention.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

How to Configure Jenkins Mail Notification on Build Failures

How to Configure Jenkins behind Proxy to Install Plugins

How to Reset Forgotten Jenkins Admin User's Password

How to Install & Configure Jenkins on Linux Easily

10 Practical Examples of Rsync Command in Linux

2023-10-12T00:44:00.006-07:00

Rsync Command in Linux:

'Rsync' Command is a powerful and versatile tool for file synchronization in Linux. It is not only efficiently copies files and directories but also synchronizes them in various scenarios. Whether you're looking to backup your data, mirror directories, or maintain remote server synchronization, rsync can do all of that.

These 10 practical commands provide you with a solid foundation for efficiently managing your data, whether it's for backups, remote synchronization, or data mirroring.

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

You can also watch this tutorial video on our youtube channel.

1. Basic Local Directory Synchronization:

The most basic use of rsync involves synchronizing the contents of one directory with another on the same system. The -av options enable archive mode and verbose output, ensuring that file attributes are preserved during synchronization.

rsync -av /source/directory/ /destination/directory/

This command efficiently copies and synchronizes the contents of the source directory with the destination directory.

Example:

rsync -av /usr/local/app-data /backup/app-data

2. Synchronize with a Remote Server over SSH:

One of the standout features of rsync is its ability to synchronize data between local and remote systems securely. Using SSH for secure communication, you can sync a local directory with a remote directory on a different server.

rsync -avz -e ssh /local/source/ user@remote-server:/remote/destination/

The -e ssh option specifies the use of SSH for secure data transfer, making it ideal for remote server synchronization.

Example:

rsync -avz -e ssh /usr/local/app-data congo@remote-server:/usr/local/app-data

3. Dry Run (Preview) Mode:

Before making actual changes, it's often a good practice to perform a dry run or preview of the synchronization to see what changes would be made. This is helpful in avoiding unexpected results.

rsync -av --dry-run /source/directory/ /destination/directory/

The --dry-run flag shows you what changes rsync would make without actually applying them.

Example:

rsync -av --dry-run /usr/local/app-data /backup/app-data

4. Synchronize and Delete Extra Files on the Destination:

Sometimes, you may want to ensure that the destination directory exactly mirrors the source, including removing any extra files not present in the source. The --delete option comes to the rescue.

rsync -av --delete /source/directory/ /destination/directory/

With this command, rsync will synchronize the source with the destination and remove any files in the destination that are not in the source.

Example:

rsync -av --delete /usr/local/app-data /backup/app-data

5. Exclude Files and Directories:

There might be situations where you need to exclude specific files or directories from the synchronization process. You can achieve this using the --exclude option.

rsync -av --exclude 'file.txt' /source/directory/ /destination/directory/

The --exclude option allows you to specify files or directories that should be skipped during synchronization.

Example:

rsync -av --exclude 'file.txt' /usr/local/app-data /backup/app-data

6. Show Progress and Transfer Speed:

If you want to keep an eye on the progress of your synchronization, as well as the transfer speed, use the --progress option.

rsync -av --progress /source/directory/ /destination/directory/

This command provides you with real-time updates on the progress of your rsync operation and the speed at which data is being transferred.

Example:

rsync -av --progress /usr/local/app-data /backup/app-data

7. Preserve File Permissions and Ownership:

Preserving file permissions and ownership is crucial in many scenarios, especially when working with system files. The --chown option allows you to set the ownership of the copied files.

rsync -av --chown=USER:GROUP /source/directory/ /destination/directory/

With this command, you can ensure that file permissions and ownership are retained in the destination directory.

Example:

rsync -av --chown=congo:administrator /usr/local/app-data /backup/app-data

8. Use Rsync Daemon for Remote Sync:

rsync can also operate in daemon mode, allowing for remote synchronization through a custom port. This is particularly useful when working with remote servers.

rsync -avz /source/directory/ rsync://remote-server:8873/destination/

In this example, we use rsync's daemon mode to synchronize data with a remote server over port 6262.

Example:

rsync -avz /usr/local/app-data rsync://remote-server:6262/backup/app-data

9. Backup Rotation with Hard Links:

Creating backups with rsync can be optimized for efficiency and storage space. By using hard links, you can create incremental backups while preserving disk space.

rsync -av --link-dest=/backup/prev-backup /source/directory/ /backup/new-backup/

This command creates a new backup while linking to the previous one, saving space by only copying changes.

Example:

rsync -av --link-dest=/backup/app-data-old /usr/local/app-data /backup/app-data

10. Exclude Multiple Patterns from Sync:

When you need to exclude multiple files or patterns from synchronization, you can utilize an exclude file (e.g., exclude.txt) that lists the items to be excluded.

rsync -av --exclude-from=exclude.txt /source/directory/ /destination/directory/

This approach allows you to maintain a list of excluded files and patterns for easy management.

Example:

rsync -av --exclude-from=exclude.txt /usr/local/app-data /backup/app-data

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

Accessing Kubernetes Service from Localhost

Helm: Render Chart Templates Locally

How to Configure DNS for Applications Deployed on Kubernetes?

How to Expose Single REST API in Kubernetes Cluster?

2023-05-17T20:36:00.002-07:00

This post will help you to understand different methods is to expose single REST API in Kubernetes Cluster to the outside world and will provide step by step instructions. Because this is one common requirement for us to expose single REST API to allow external access to specific services within Kubernetes Cluster.

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

Topics:

Using NodePort Service
Using LoadBalancer Service
Using Ingress Controller
Choosing the Right Method
Step by Step Instructions

Using NodePort Service:

NodePort service type helps us to expose services on specific port on each node in kubernetes cluster. With this NodePort Service, Our service can be accessed from externally by using respective node's IP address and allocated port. This NodePort Service is a basic and straight inbuilt feature available in kubernetes cluster, But it may not best for realtime production scenarios as it requires managing port mappings manually.

Using LoadBalancer Service:

LoadBalancer service type is another feature that uses cloud provider and it supports external load balancers. Kubernetes will automatically create one loadbalancer and will assign one external IP address or CNAME record. This will route or forward the network traffic to the service. This Loadbalancer service method is more suitable for production realtime environments with high availability and automatic load balancing.

Using Ingress Controller:

Ingress controller is a additional controller to be installed on kubernetes cluster. This load balancer operates at the application layer (Layer 7) and it provides more advanced routing capabilities. It uses kubernetes ingress resources to define or create rules for routing incoming requests to different services based on paths, hostnames or patterns. This Ingress controller method is highly stuitable and it allows for sophisticated routing configurations on kubernetes cluster.

Choosing the Right Method:

When selecting right method to expose single REST API in Kubernetes cluster, consider the application, deployments and environment requirements. NodePort is the easiest and simple method but it doesnt have automation and scalability. LoadBalancer is a good option for cloud-based environemtn but may have additional costs involved. Ingress Controller is the most advanced features but it requires additional configuration and supported Ingress controller implementation.

Step by Step Instructions:

Let's see the step by step instructions for exposing single REST API using NodePort service type:

Step 1: Create Kubernetes deployment and service using Kubectl command:

$ kubectl create deployment my-api --image=my-api-image
$ kubectl expose deployment my-api --port=80 --target-port=8080 --type=NodePort

Step 2: Find NodePort IP assigned to service:

$ kubectl get service my-api

Step 3: Access REST API externally using node's IP address and NodePort number:

http://<node-ip>:<node-port>

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

Accessing Kubernetes Service from Localhost

Helm: Render Chart Templates Locally

How to Configure DNS for Applications Deployed on Kubernetes?

Accessing Kubernetes Service from Localhost

2023-05-17T20:06:00.001-07:00

This post will help you with accessing kubernetes service from localhost.

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

Hope you have running Kubernetes cluster and kubectl is already installed on your local to manage your kubernetes cluster. if you dont have existing kubernetes cluster, please refer this link How to Install & Configure Kubernetes Cluster on CentOS 7 / RHEL 7 or How to Install & Configure Kubernetes Cluster on Ubuntu

Step 1: Find Kubernetes Service IP Address

Firstly, we need to find the IP address allocated to your kubernetes services that you want to access. Use default kubectl command with service option to find service IP address.

kubectl get services

This command will list all the running services in your Kubernetes cluster along with their IP addresses. Specifically if you want to show the service ip address of kubernetes namespace. use 'n' option along with namespace.

kubectl get services -n test-namespace

Step 2: Create Port Forwarding

Let's create port forwarding from your local machine to Kubernetes service. Use kubectl command with port-forward option.

kubectl port-forward <service-name> <local-port>:<service-port>

Where, Replace <service-name> with the name of your Kubernetes service that you want to access, <local-port> with the port number on your local machine & <service-port> with the port number of the Kubernetes service.

For example, if your service name is my-service and running on port 8080 and you want to access the Kubernetes service, Use the below kubectl command with port forward command option to create port forwarding to port 8081 on your local machine.

kubectl port-forward my-service 8081:8080

Step 3: Access Kubernetes Service

Once the port forwarding is created, it will start listening. So you can access your Kubernetes service from your local machine by just visiting the URL http://localhost:<local-port> in any of your web browser.

For example, if port-forwarding is created for port 8081. Then you can access Kubernetes service from localhost with the URL http://localhost:8081.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

How to Configure DNS for Applications Deployed on Kubernetes?

Helm: List Installed Charts

Helm: Render Chart Templates Locally

How to Configure DNS for Applications Deployed on Kubernetes?

2023-05-17T18:19:00.002-07:00

This post will help you with How to configure DNS for Applications deployed on Kubernetes. DNS configuration on Kubernetes is very important to make sure the applications can be accessed through DNS names. So the other part of resolution will be taken care by DNS or Domain Name System, because it is responsible for resolving domain names to IP addresses.

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

Prerequisites:

A Kubernetes cluster with one or more nodes.
kubectl command-line tool installed on your local machine.

Lets jump into the step by step instruction to configure DNS on Kubernetes cluster.

Step 1: Create Namespace on Kubernetes Cluster

I assume, you may have already a dedicated kubernetes namespace created for your application. If not, use kubectl command with namespace option to create namespace on kubernetes cluster for your application. Because, Kubernetes Namespace will help our application by organizing and isolating resources within Kubernetes cluster.

$ kubectl create namespace <namespace-name>

Replace <namespace-name> with your preferred name.

Step 2: Deploy Application on Kubernetes Cluster

I assume again, you already have application deployed on kubernetes cluster. If not, create deployment resource to deploy the application using the below command easily.

$ kubectl create deployment <deployment-name> --image=<image-name> --namespace=<namespace-name>

Replace <deployment-name> with the preferred name, <image-name> with the name of the Docker image that you need to deploy and <namespace-name> with the name of the namespace you created in Step 1.

Step 3: Expose Deployment

Once application is deployed and running in state, we need to expose the application. So users can access it from outside the kubernetes cluster. Use the below command to expose deployment.

$ kubectl expose deployment <deployment-name> --type=LoadBalancer --port=<port-number> --namespace=<namespace-name>

Replace <deployment-name> with the name of the deployment you created in Step 2, <port-number> with the port number your application is listening on and <namespace-name> with the name of the namespace you created in Step 1.

Step 4: Configure DNS

Here is the the final step to configure DNS for your application. You can use Kubernetes ingress to configure DNS as follows: Just create a kubernetes ingress yaml file and apply it.

Make sure the domain name record is created and mapped with correct CNAME record.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: <ingress-name>
  namespace: <namespace-name>
spec:
  rules:
  - host: <domain-name>
    http:
      paths:
      - backend:
          serviceName: <service-name>
          servicePort: <port-number>

Replace <ingress-name> with the name you want to give your ingress, <domain-name> with the domain name you want to use to access your application, <service-name> with the name of the service you created in Step 3 and <port-number> with the port number your application is listening on.

After configuring DNS, you should be able to access the application running on the kubernetes cluster using the domain name you specified in the ingress.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

Helm: List Installed Charts

Helm: Render Chart Templates Locally

Helm Upgrade: Update Chart Values with Examples

Kubectl: Force Delete Namespace Stuck in Terminating

Helm: List Repos & Charts in Repo

2023-05-17T10:46:00.001-07:00

This post will help you to understand how to use Helm command to list repositories and how to list charts from repositories with step by step instruction along with helm command example.

Helm has the inbuild feature to list repositories and helm charts available in the repositories. This functionality helps users to go through the available charts and install it easily if they find that suits for them.

If you are interested in learning, Request you to go through the below recommended tutorial.

Let's Get Started.

Listing Repositories:

Lets see, how to list repositories using Helm. Basically, Repositories holds all the collection of charts that are already packaged completely and made publicly available for installation.

Use the below Helm command to list helm repositories.

helm repo list

Step-by-Step Instructions:

Here is the steps to list repositories in Helm:

Step 1: Open terminal or command prompt.

Step 2: Hope you have Helm installed already and properly configured.

Step 3: Run helm repo list command

Step 4: Helm command will fetch the repositories list configured and will display the results on the screen as helm command output.

Example:

$ helm repo list
NAME        	URL                                             
stable      	https://charts.helm.sh/stable
local       	http://localhost:8080/charts

Above output shows the list of repositories available in your Helm environment along with the repository name and URL of repositories configured.

Listing Charts in Repository:

Once you have listed repositories from the Helm environment, now we have to find the helm charts available in specific repositories. Helm command provides a search option to list the helm charts within repository.

Use the below helm command to list helm charts within repository.

helm search repo <repository-name>

Step-by-Step Instructions:

Here is the steps to list helm charts within repository:

Step 1: Open terminal or command prompt.

Step 2: Run helm command helm search repo <repository-name>, where replace <repository-name> with the repository name that you want to list.

Step 3: Now, Helm command will search the mentioned repository and will display all the available helm charts along with helm chart versions and helm chart descriptions.

Example:

$ helm search repo stable
NAME                           	CHART VERSION	APP VERSION	DESCRIPTION                                   
stable/wordpress               	10.2.3       	5.4.1      	Web publishing platform for building blogs a...
stable/mysql                   	1.6.8        	5.7.34     	Fast, reliable, scalable, and easy to use op...
stable/nginx-ingress           	1.41.3       	0.48.1     	An nginx Ingress controller that uses ConfigM...
...

Above output shows helm chart name, helm chart version, helm app version and description of helm charts of specified repository.

That's it for this post. Keep practicing and have fun. Leave your comments if any.

Related Searches and Questions asked:

Helm: List Installed Charts

Helm: Render Chart Templates Locally

Helm Upgrade: Update Chart Values with Examples

Kubectl: Force Delete Namespace Stuck in Terminating

Learnitguide.net - Learn Linux, DevOps and Cloud

Python Pandas Tutorial: Basics to Advanced Guide

What is Pandas?

Installing and Importing Pandas

Pandas Series

Pandas DataFrame

Loading Data from Files

Inspecting Your Data

Indexing with loc and iloc

Filtering Data

Adding, Renaming and Dropping Columns

Sorting Data

GroupBy Operations

Merging and Joining DataFrames

Apply Function

Pivot Tables

Exporting Data

Quick Command Reference

Key Takeaways

Python NumPy Tutorial: Basics to Advanced Guide

What is NumPy?

Installing NumPy

Creating NumPy Arrays

Array Properties

Indexing and Slicing

Math Operations

Broadcasting

Reshaping and Transposing

Advanced NumPy

Fancy Indexing

Boolean Masking

Stacking Arrays

Statistical Functions

np.where()

np.unique() and np.sort()

Quick Command Reference

Key Takeaways

Python Pandas Data Cleaning & Null Values Guide

What Is Dirty Data?

Setting Up the Environment

Step 1 - Load the Data and Spot the Problems

Step 2 - Find All Null Values

Step 3 - Remove Duplicate Rows

Step 4 - Drop or Fill?

Step 5 - Dropping Nulls

Step 6 - Filling Nulls in Numeric Columns

Step 7 - Filling Nulls in Categorical Columns

Step 8 - Checking and Fixing Data Types

Step 9 - Complete End-to-End Cleaning Workflow

Quick Command Reference

Docker Installation on Ubuntu 22.04 | Step-by-Step Guide

Topics Covered:

My Setup:

Step 1: Update System Packages

Step 2: Install Required Packages

Step 3: Add Docker’s GPG Key

Step 4: Add Docker’s Repository

Step 5: Update the Package List Again

Step 6: Install Docker Engine

Step 7: Verify Docker Installation

Step 8: Check Docker Service Status

Step 9: Run a Test Container

Kubernetes RBAC Explained with Hands-On Demo

Topics Covered

What is RBAC?

Authentication & Authorization

Creating IAM Users

Updating aws-auth ConfigMap

API Resources

Create Roles & ClusterRole

1. Read-only Role for dev namespace

2. Full-access Role for dev namespace

3. Cluster Admin Role

RoleBinding and ClusterRoleBinding

RoleBinding read-only access for namespace "dev":

RoleBinding Full-access for namespace "dev":

ClusterRoleBinding Admin access for entire cluster:

Map users in aws-auth configmap

Testing Access with AWS CLI Profiles

Deploy AWS EKS Cluster on Private Subnets with Terraform

1. Read-only Role for `dev` namespace

2. Full-access Role for `dev` namespace