Mahesh Info

Dapper vs Entity Framework Core in .NET – Complete End-to-End Guide

2026-09-22T17:22:42.491+05:30

Introduction

When developing modern ASP.NET Core Web API applications, one of the most important decisions is how the application communicates with the database.

Two popular approaches in the .NET ecosystem are:

Entity Framework Core (EF Core)
Dapper

Both are powerful, but they solve the database-access problem differently.

EF Core is a full-featured Object-Relational Mapper (ORM), while Dapper is a lightweight micro-ORM that provides a thin layer over ADO.NET.

A common interview question is:

"What is the difference between Dapper and Entity Framework Core, and when would you use each?"

This article explains the differences from beginner to production level.

1. What is Entity Framework Core?

Entity Framework Core (EF Core) is Microsoft's modern, cross-platform ORM for .NET.

ORM stands for:

Object Relational Mapping

It allows developers to work with database tables using C# classes and LINQ instead of writing SQL for every operation.

For example, suppose we have a database table:

CREATE TABLE Employees
(
    Id INT PRIMARY KEY IDENTITY,
    Name NVARCHAR(100),
    Department NVARCHAR(100),
    Salary DECIMAL(18,2)
);

In EF Core, we can create:

public class Employee
{
    public int Id { get; set; }

    public string Name { get; set; }

    public string Department { get; set; }

    public decimal Salary { get; set; }
}

Then query the database using LINQ:

var employees = await dbContext.Employees
    .Where(x => x.Department == "IT")
    .ToListAsync();

EF Core generates the appropriate SQL behind the scenes.

2. What is Dapper?

Dapper is a lightweight micro-ORM for .NET.

It was originally created by the Stack Overflow team and is widely used for high-performance database access.

Unlike EF Core, Dapper does not attempt to provide a complete ORM framework.

You normally write the SQL yourself:

var sql = """
          SELECT Id, Name, Department, Salary
          FROM Employees
          WHERE Department = @Department
          """;

var employees = await connection.QueryAsync<Employee>(
    sql,
    new { Department = "IT" });

Dapper executes the SQL and maps the returned rows to the Employee C# class.

3. Dapper vs EF Core – High-Level Difference

Feature	Dapper	EF Core
Type	Micro ORM	Full ORM
SQL	Developer writes SQL	EF can generate SQL
LINQ	Limited/direct SQL approach	Extensive LINQ support
Change Tracking	No built-in change tracker	Yes
Migrations	No	Yes
Relationships	Manual SQL/mapping	Built-in relationship mapping
Lazy Loading	No built-in ORM-style lazy loading	Supported with configuration
Performance	Generally very fast	Generally good, but more abstraction
Control over SQL	Very high	Medium/High
Learning Curve	Relatively low	Higher
CRUD	SQL-based	Object-based/LINQ
Transactions	ADO.NET transactions	`DbContext` transactions
Complex queries	Excellent when SQL is known	Excellent with LINQ, but SQL knowledge still useful
Stored Procedures	Excellent	Supported
Change Tracking	No	Yes
Database-first	Manual	Supported
Code-first	No EF-style migrations	Supported
Best suited for	SQL-centric/high-performance applications	Domain/application-centric applications

4. Architecture Difference

The biggest conceptual difference is the abstraction level.

Dapper

Application
     |
     v
Repository
     |
     v
Dapper
     |
     v
ADO.NET
     |
     v
SQL Server

You control the SQL.

EF Core

Application
     |
     v
Repository / Service
     |
     v
DbContext
     |
     v
EF Core
     |
     v
LINQ
     |
     v
SQL Generation
     |
     v
SQL Server

EF Core provides more abstraction.

5. Basic EF Core Setup

Suppose we are developing:

ASP.NET Core Web API
        |
        v
EmployeeService
        |
        v
EmployeeRepository
        |
        v
EF Core
        |
        v
Azure SQL / SQL Server

Install the SQL Server provider:

dotnet add package Microsoft.EntityFrameworkCore.SqlServer

For migrations:

dotnet add package Microsoft.EntityFrameworkCore.Tools

6. Create DbContext

public class EmployeeDbContext : DbContext
{
    public EmployeeDbContext(
        DbContextOptions<EmployeeDbContext> options)
        : base(options)
    {
    }

    public DbSet<Employee> Employees { get; set; }
}

7. Configure Connection String

{
  "ConnectionStrings": {
    "DefaultConnection":
      "Server=localhost;Database=EmployeeDb;Trusted_Connection=True;TrustServerCertificate=True"
  }
}

builder.Services.AddDbContext<EmployeeDbContext>(options =>
    options.UseSqlServer(
        builder.Configuration.GetConnectionString("DefaultConnection")));

8. EF Core CRUD Operations

Create

var employee = new Employee
{
    Name = "John",
    Department = "IT",
    Salary = 75000
};

dbContext.Employees.Add(employee);

await dbContext.SaveChangesAsync();

EF Core generates an INSERT statement.

9. Read

var employee = await dbContext.Employees
    .FirstOrDefaultAsync(x => x.Id == 10);

10. Read Multiple Records

var employees = await dbContext.Employees
    .Where(x => x.Department == "IT")
    .ToListAsync();

11. Update

var employee = await dbContext.Employees
    .FirstOrDefaultAsync(x => x.Id == 10);

if (employee != null)
{
    employee.Salary = 85000;

    await dbContext.SaveChangesAsync();
}

EF Core detects the modified property using change tracking.

12. Delete

var employee = await dbContext.Employees
    .FindAsync(10);

if (employee != null)
{
    dbContext.Employees.Remove(employee);

    await dbContext.SaveChangesAsync();
}

13. Dapper Setup

Install Dapper:

dotnet add package Dapper

For SQL Server:

dotnet add package Microsoft.Data.SqlClient

14. Dapper Connection

using Microsoft.Data.SqlClient;
using System.Data;

public class EmployeeRepository
{
    private readonly string _connectionString;

    public EmployeeRepository(IConfiguration configuration)
    {
        _connectionString =
            configuration.GetConnectionString("DefaultConnection")!;
    }

    private IDbConnection CreateConnection()
    {
        return new SqlConnection(_connectionString);
    }
}

15. Dapper SELECT

public async Task<IEnumerable<Employee>> GetEmployeesAsync()
{
    using var connection = CreateConnection();

    const string sql = """
        SELECT Id, Name, Department, Salary
        FROM Employees
        """;

    return await connection.QueryAsync<Employee>(sql);
}

16. Dapper SELECT By ID

public async Task<Employee?> GetEmployeeAsync(int id)
{
    using var connection = CreateConnection();

    const string sql = """
        SELECT Id, Name, Department, Salary
        FROM Employees
        WHERE Id = @Id
        """;

    return await connection.QuerySingleOrDefaultAsync<Employee>(
        sql,
        new { Id = id });
}

Notice:

WHERE Id = @Id

and:

new { Id = id }

Dapper maps the parameter safely.

17. Dapper INSERT

public async Task<int> CreateEmployeeAsync(Employee employee)
{
    using var connection = CreateConnection();

    const string sql = """
        INSERT INTO Employees
        (
            Name,
            Department,
            Salary
        )
        VALUES
        (
            @Name,
            @Department,
            @Salary
        );

        SELECT CAST(SCOPE_IDENTITY() AS INT);
        """;

    return await connection.ExecuteScalarAsync<int>(
        sql,
        employee);
}

18. Dapper UPDATE

public async Task<int> UpdateEmployeeAsync(Employee employee)
{
    using var connection = CreateConnection();

    const string sql = """
        UPDATE Employees
        SET
            Name = @Name,
            Department = @Department,
            Salary = @Salary
        WHERE Id = @Id
        """;

    return await connection.ExecuteAsync(sql, employee);
}

19. Dapper DELETE

public async Task<int> DeleteEmployeeAsync(int id)
{
    using var connection = CreateConnection();

    const string sql = """
        DELETE FROM Employees
        WHERE Id = @Id
        """;

    return await connection.ExecuteAsync(
        sql,
        new { Id = id });
}

20. Query vs Execute in Dapper

This is an important interview topic.

Query

Use Query when you expect rows back.

var employees =
    await connection.QueryAsync<Employee>(sql);

Execute

Use Execute for INSERT, UPDATE, DELETE.

var affectedRows =
    await connection.ExecuteAsync(sql);

ExecuteScalar

Use when you want one value.

var employeeId =
    await connection.ExecuteScalarAsync<int>(sql);

21. Entity Framework Core Change Tracking

One of EF Core's major features is change tracking.

For example:

var employee =
    await dbContext.Employees
        .FirstAsync(x => x.Id == 10);

employee.Salary = 100000;

await dbContext.SaveChangesAsync();

EF Core knows that Salary changed.

It generates the UPDATE statement.

22. Dapper Does Not Track Changes

Dapper does not maintain an entity state.

For example:

var employee = await connection.QuerySingleAsync<Employee>(
    sql,
    new { Id = 10 });

employee.Salary = 100000;

Nothing happens automatically.

You must execute an UPDATE yourself:

await connection.ExecuteAsync(
    updateSql,
    employee);

23. AsNoTracking in EF Core

When you only need to read data, change tracking may not be required.

Use:

var employees = await dbContext.Employees
    .AsNoTracking()
    .ToListAsync();

This can reduce tracking overhead for read-only queries.

This is especially useful in:

Reporting
Dashboards
Search APIs
Read-only endpoints
Large result sets

24. Relationships in EF Core

Suppose we have:

Department
    |
    +---- Employee
    +---- Employee
    +---- Employee

Entities:

public class Department
{
    public int Id { get; set; }

    public string Name { get; set; }

    public ICollection<Employee> Employees { get; set; }
        = new List<Employee>();
}

Employee:

public class Employee
{
    public int Id { get; set; }

    public string Name { get; set; }

    public int DepartmentId { get; set; }

    public Department Department { get; set; }
}

EF Core can configure the relationship.

modelBuilder.Entity<Employee>()
    .HasOne(e => e.Department)
    .WithMany(d => d.Employees)
    .HasForeignKey(e => e.DepartmentId);

25. Dapper Relationships

Dapper doesn't automatically manage relationships like EF Core.

You normally write SQL joins.

SELECT
    e.Id,
    e.Name,
    d.Id,
    d.Name
FROM Employees e
INNER JOIN Departments d
    ON e.DepartmentId = d.Id

Then map the result manually using Dapper's multi-mapping capabilities or project into DTOs.

26. EF Core Include

EF Core provides Include.

var employees = await dbContext.Employees
    .Include(x => x.Department)
    .ToListAsync();

This tells EF Core to load the related department data.

27. Dapper Stored Procedures

Dapper works very well with stored procedures.

Example:

CREATE PROCEDURE GetEmployeesByDepartment
    @Department NVARCHAR(100)
AS
BEGIN
    SELECT
        Id,
        Name,
        Department,
        Salary
    FROM Employees
    WHERE Department = @Department;
END

Dapper:

var employees =
    await connection.QueryAsync<Employee>(
        "GetEmployeesByDepartment",
        new
        {
            Department = "IT"
        },
        commandType: CommandType.StoredProcedure);

28. EF Core Stored Procedures

EF Core can also execute stored procedures.

For example, raw SQL can be used:

var employees =
    await dbContext.Employees
        .FromSqlInterpolated(
            $"EXEC GetEmployeesByDepartment {"IT"}")
        .ToListAsync();

EF Core also provides APIs for executing SQL commands where appropriate.

The important point is:

Choosing EF Core does not mean you can never use SQL or stored procedures.

29. SQL Injection

Both Dapper and EF Core can be used safely.

The important thing is parameterization.

Dapper

const string sql = """
    SELECT *
    FROM Employees
    WHERE Department = @Department
    """;

var employees = await connection.QueryAsync<Employee>(
    sql,
    new { Department = department });

Avoid building SQL using string concatenation:

var sql =
    "SELECT * FROM Employees WHERE Department = '" +
    department +
    "'";

30. EF Core Parameterization

LINQ queries are parameterized by EF Core.

var employees = await dbContext.Employees
    .Where(x => x.Department == department)
    .ToListAsync();

EF Core generates parameterized SQL.

31. Transactions with Dapper

Dapper uses ADO.NET transactions.

using var connection = CreateConnection();

connection.Open();

using var transaction = connection.BeginTransaction();

try
{
    await connection.ExecuteAsync(
        insertEmployeeSql,
        employee,
        transaction);

    await connection.ExecuteAsync(
        updateDepartmentSql,
        department,
        transaction);

    transaction.Commit();
}
catch
{
    transaction.Rollback();
    throw;
}

32. Transactions with EF Core

EF Core provides transaction support through Database.

await using var transaction =
    await dbContext.Database.BeginTransactionAsync();

try
{
    dbContext.Employees.Add(employee);

    await dbContext.SaveChangesAsync();

    department.Budget += 1000;

    await dbContext.SaveChangesAsync();

    await transaction.CommitAsync();
}
catch
{
    await transaction.RollbackAsync();
    throw;
}

33. EF Core Migrations

One of the major advantages of EF Core is migrations.

For example:

dotnet ef migrations add InitialCreate

Then:

dotnet ef database update

If you add a property:

public string Email { get; set; }

you can create another migration:

dotnet ef migrations add AddEmployeeEmail

Then:

dotnet ef database update

EF Core manages schema changes through migrations.

34. Does Dapper Support Migrations?

No.

Dapper itself is not a database schema migration framework.

You can use tools such as:

SQL scripts
DbUp
FluentMigrator
Flyway
Liquibase
CI/CD database migration processes

For example:

Application
    |
    +---- Dapper
    |
    +---- SQL Scripts
    |
    +---- Migration Tool
    |
    v
SQL Server

35. Performance Comparison

A common misconception is:

"Dapper is always faster than EF Core."

That is too simplistic.

Dapper has a smaller abstraction layer and often performs very well for straightforward SQL queries.

EF Core has additional capabilities such as:

Change tracking
Identity resolution
LINQ translation
Relationship handling
Model metadata
State management

These features have overhead.

However, properly written EF Core queries can perform very well.

For example:

var employees = await dbContext.Employees
    .AsNoTracking()
    .Where(x => x.Department == "IT")
    .Select(x => new EmployeeDto
    {
        Id = x.Id,
        Name = x.Name
    })
    .ToListAsync();

This avoids retrieving unnecessary columns and avoids tracking.

36. Performance Depends on SQL and Database Design

Database performance is not determined only by Dapper or EF Core.

Other factors include:

Indexes
Query design
Execution plan
Database schema
Network latency
Connection pooling
Number of records
Pagination
Locking
Blocking
CPU
Memory
I/O
Database configuration

For example, a poorly written Dapper query can be slower than a well-written EF Core query.

37. Pagination with Dapper

const string sql = """
    SELECT Id, Name, Department, Salary
    FROM Employees
    ORDER BY Id
    OFFSET @Offset ROWS
    FETCH NEXT @PageSize ROWS ONLY
    """;

var employees = await connection.QueryAsync<Employee>(
    sql,
    new
    {
        Offset = (pageNumber - 1) * pageSize,
        PageSize = pageSize
    });

38. Pagination with EF Core

var employees = await dbContext.Employees
    .AsNoTracking()
    .OrderBy(x => x.Id)
    .Skip((pageNumber - 1) * pageSize)
    .Take(pageSize)
    .ToListAsync();

39. Repository Pattern with EF Core

public interface IEmployeeRepository
{
    Task<Employee?> GetByIdAsync(int id);

    Task<IEnumerable<Employee>> GetAllAsync();

    Task AddAsync(Employee employee);

    Task UpdateAsync(Employee employee);

    Task DeleteAsync(int id);
}

Implementation:

public class EmployeeRepository : IEmployeeRepository
{
    private readonly EmployeeDbContext _context;

    public EmployeeRepository(EmployeeDbContext context)
    {
        _context = context;
    }

    public async Task<Employee?> GetByIdAsync(int id)
    {
        return await _context.Employees
            .FirstOrDefaultAsync(x => x.Id == id);
    }
}

40. Repository Pattern with Dapper

public class EmployeeRepository : IEmployeeRepository
{
    private readonly string _connectionString;

    public EmployeeRepository(IConfiguration configuration)
    {
        _connectionString =
            configuration.GetConnectionString("DefaultConnection")!;
    }

    public async Task<Employee?> GetByIdAsync(int id)
    {
        using var connection =
            new SqlConnection(_connectionString);

        const string sql = """
            SELECT Id, Name, Department, Salary
            FROM Employees
            WHERE Id = @Id
            """;

        return await connection.QuerySingleOrDefaultAsync<Employee>(
            sql,
            new { Id = id });
    }
}

41. CQRS with Dapper and EF Core

A very common enterprise architecture is:

                 API
                  |
             Application
                  |
             +----+----+
             |         |
         Commands    Queries
             |         |
             v         v
          EF Core    Dapper
             |         |
             v         v
          Database   Database

For example:

Command

Create employee:

POST /employees
       |
       v
Command Handler
       |
       v
EF Core
       |
       v
SQL Server

Query

Get employees:

GET /employees
       |
       v
Query Handler
       |
       v
Dapper
       |
       v
SQL Server

This approach can combine the strengths of both technologies.

42. Real-Time Enterprise Example

Consider a BPO workforce-management application.

The system contains:

Forecast Service
Workforce Service
Capacity Service
Hiring Service
Employee Service
Reporting Service

Suppose the Reporting Service needs a complicated dashboard query:

Employee
    |
Department
    |
Skill
    |
Shift
    |
Forecast
    |
Actual Volume
    |
Utilization

The query may involve:

Multiple JOINs
Aggregations
Window functions
CTEs
Stored procedures
Complex filtering

Dapper can be a strong choice when the team wants precise control over the SQL.

For standard application CRUD, EF Core may provide faster development.

Therefore:

CRUD / Domain Logic
       |
     EF Core

Complex Reporting
       |
     Dapper

This is a hybrid approach.

43. Can We Use Dapper and EF Core in the Same Application?

Yes.

There is no requirement to choose only one.

For example:

ASP.NET Core API
       |
       +----------------+
       |                |
   EF Core           Dapper
       |                |
       +-------+--------+
               |
          SQL Server

Use EF Core for:

Create
Update
Delete
Domain operations
Change tracking
Relationships

Use Dapper for:

Reports
Complex queries
Read-heavy APIs
Stored procedures
Performance-sensitive queries

44. Hybrid Repository Example

public class EmployeeService
{
    private readonly EmployeeDbContext _context;
    private readonly IDbConnection _connection;

    public EmployeeService(
        EmployeeDbContext context,
        IDbConnection connection)
    {
        _context = context;
        _connection = connection;
    }
}

You can then use:

// EF Core
await _context.Employees.AddAsync(employee);
await _context.SaveChangesAsync();

and:

// Dapper
var result = await _connection.QueryAsync<EmployeeReport>(
    reportSql);

45. Advantages of Dapper

1. Lightweight

Dapper has a small abstraction layer.

2. High SQL Control

Developers write the SQL directly.

3. Excellent for Complex Queries

SQL features such as:

CTE
JOIN
Window Functions
Stored Procedures
Temporary Tables
Aggregation
Database-specific features

can be used naturally.

4. Good Performance

Dapper avoids many ORM features that introduce additional processing.

5. Easy to Integrate

It works directly with ADO.NET connections and transactions.

46. Disadvantages of Dapper

1. More SQL Code

Developers must write SQL manually.

2. No Change Tracking

You must explicitly update the database.

3. No Built-in Migrations

A separate migration solution is required.

4. Relationship Handling

Complex relationships require additional SQL and mapping.

5. Maintenance

Large applications can end up with many SQL strings.

For example:

Repository
   |
   +-- SQL Query 1
   +-- SQL Query 2
   +-- SQL Query 3
   +-- SQL Query 4
   +-- SQL Query 5
   ...

Maintaining these queries requires discipline.

47. Advantages of EF Core

1. Productivity

CRUD operations require relatively little code.

2. LINQ

You can query using strongly typed C# expressions.

var employees = await context.Employees
    .Where(x => x.Salary > 50000)
    .OrderBy(x => x.Name)
    .ToListAsync();

3. Change Tracking

EF Core automatically tracks entity changes.

4. Relationships

Relationships are modeled naturally.

5. Migrations

Database schema changes can be managed through migrations.

6. Strong Typing

Many errors can be caught at compile time.

7. Integration with .NET

EF Core integrates naturally with:

ASP.NET Core
Dependency Injection
LINQ
Async/Await
Configuration
Logging
Testing

48. Disadvantages of EF Core

1. Abstraction Overhead

EF Core performs more work than a lightweight data-access library.

2. Generated SQL

Developers may not have direct control over every aspect of generated SQL.

3. Learning Curve

You need to understand:

DbContext
Change Tracking
LINQ
Relationships
Loading strategies
Migrations
Transactions
Query translation

4. N+1 Query Problems

Poorly designed loading can result in many database queries.

5. Complex SQL

Some highly database-specific SQL can be more straightforward to express directly using SQL/Dapper.

49. What is N+1 Query Problem?

Suppose we load departments:

var departments =
    await context.Departments.ToListAsync();

Then separately query employees for each department.

If there are 100 departments:

1 query
+
100 queries
=
101 queries

This is the N+1 problem.

It can cause serious performance issues.

Possible solutions include:

Include()

or projection:

.Select(...)

or using an optimized SQL query/Dapper.

50. EF Core Projection

Instead of loading the entire entity:

var employees = await context.Employees
    .Select(x => new EmployeeDto
    {
        Id = x.Id,
        Name = x.Name
    })
    .ToListAsync();

This can generate SQL that selects only the required columns.

51. Security Comparison

Both Dapper and EF Core can be secure.

The developer must follow secure coding practices.

Important practices:

Parameterized queries
Least-privilege database accounts
Secrets management
Azure Key Vault
Managed Identity
Input validation
Authorization
HTTPS
Auditing

Never put passwords directly into source code.

Bad:

Server=myserver;
User Id=admin;
Password=MyPassword123;

In Azure environments, consider:

Managed Identity
        |
        v
Azure SQL

and/or secure secret management where credentials are required.

52. Azure SQL with EF Core

A typical Azure architecture:

Internet
   |
Azure Front Door / Application Gateway
   |
API Management
   |
ASP.NET Core API
   |
EF Core
   |
Azure SQL Database

53. Azure SQL with Dapper

Dapper fits the same architecture:

Internet
   |
API Management
   |
ASP.NET Core API
   |
Dapper
   |
Azure SQL Database

The difference is primarily how the application accesses the database.

54. Testing EF Core

EF Core repositories/services can be tested using approaches such as:

Real SQL Server
SQL Server test container
SQLite where appropriate
Mocked abstractions in specific cases

For realistic database behavior, integration tests against a real SQL Server-compatible database are often more representative than relying exclusively on mocks.

55. Testing Dapper

Dapper-based repositories are also commonly tested using:

SQL Server test database
Testcontainers
Docker
Integration tests

Example architecture:

Integration Test
       |
       v
Dapper Repository
       |
       v
SQL Server Container

This provides realistic SQL execution.

56. Dependency Injection

EF Core:

builder.Services.AddDbContext<EmployeeDbContext>(
    options =>
        options.UseSqlServer(connectionString));

Dapper:

builder.Services.AddScoped<IDbConnection>(
    _ => new SqlConnection(connectionString));

In modern applications, it is also common to create a connection factory abstraction rather than injecting a raw connection everywhere.

57. Which One Should You Choose?

There is no universal answer.

Consider the application's requirements.

Choose EF Core when:

CRUD-heavy application
Domain-driven design
Complex entity relationships
Change tracking required
Rapid application development
Code-first migrations
LINQ-heavy application
Strong domain model

Choose Dapper when:

SQL-heavy application
Complex reporting
Existing stored procedures
Fine SQL control required
Read-heavy APIs
Database-specific SQL
Very lightweight data-access layer desired

Choose both when:

Business/domain operations -> EF Core

Complex queries/reports -> Dapper

58. Interview Question: Is Dapper faster than EF Core?

A good interview answer:

Dapper generally has less abstraction and less ORM overhead, so it can perform very well for SQL-centric operations. However, it is not correct to say that Dapper is always faster. Actual performance depends on query design, indexing, database workload, network latency, result size, tracking, and application architecture. Properly optimized EF Core queries can also provide excellent performance.

59. Interview Question: Does Dapper track changes?

No.

Dapper does not provide EF Core-style change tracking.

The developer explicitly executes INSERT, UPDATE, or DELETE statements.

60. Interview Question: Does EF Core always generate efficient SQL?

No.

EF Core generates SQL based on the LINQ expression and model configuration, but developers still need to understand the SQL being generated.

For performance-critical queries, inspect:

Generated SQL
Execution Plan
Indexes
Query duration
Database waits
Logical reads

61. Interview Question: Can Dapper call stored procedures?

Yes.

Example:

await connection.QueryAsync<Employee>(
    "GetEmployees",
    parameters,
    commandType: CommandType.StoredProcedure);

62. Interview Question: Can EF Core use stored procedures?

Yes.

EF Core supports raw SQL and stored procedure-related database operations.

However, the exact API depends on whether you are querying entities, executing commands, or using EF Core's newer modification stored-procedure capabilities.

63. Interview Question: Can Dapper perform transactions?

Yes.

Dapper uses ADO.NET transaction support.

using var transaction =
    connection.BeginTransaction();

64. Interview Question: Can EF Core and Dapper use the Same Database?

Yes.

For example:

                     SQL Server
                         |
             +-----------+-----------+
             |                       |
          EF Core                 Dapper
             |                       |
       Write operations         Read operations

They can coexist in the same application.

65. Interview Question: Which is better for Microservices?

Neither is universally better.

The decision should depend on the microservice's responsibilities.

For example:

Order Service
    |
    +-- EF Core

because it contains domain relationships and transactional business operations.

Reporting Service
    |
    +-- Dapper

may be appropriate when it is primarily SQL/reporting focused.

Different microservices can use different data-access technologies.

66. Recommended Enterprise Architecture

For a large ASP.NET Core application, a practical architecture can be:

                    Client
                      |
                      v
                API Management
                      |
                      v
                ASP.NET Core
                      |
              +-------+-------+
              |               |
          Commands          Queries
              |               |
              v               v
           EF Core          Dapper
              |               |
              +-------+-------+
                      |
                  SQL Server

The important principle is:

Select the data-access technology based on the use case rather than choosing a technology simply because it is popular.

67. Dapper vs EF Core – Final Comparison

Area	Dapper	EF Core
Abstraction	Low	High
SQL Control	Excellent	Good
CRUD Productivity	Good	Excellent
Change Tracking	No	Yes
LINQ	Limited	Excellent
Relationships	Manual	Excellent
Migrations	External	Built-in
Stored Procedures	Excellent	Supported
Complex SQL	Excellent	Good
Performance	Very good	Very good
Learning	Easier initially	More concepts
Domain Modeling	Manual	Strong
Read Models	Excellent	Excellent
Write Models	Good	Excellent
Reporting	Excellent	Good/Excellent depending on query
Large Enterprise Application	Very good	Very good
Hybrid Usage	Yes	Yes

68. Real-World Recommendation

For a typical enterprise .NET application, don't think:

Dapper OR EF Core

Think:

Where does each technology provide the most value?

For example:

                   ASP.NET Core
                         |
                 Application Layer
                         |
             +-----------+-----------+
             |                       |
        Business Commands          Queries
             |                       |
             v                       v
          EF Core                 Dapper
             |                       |
             +-----------+-----------+
                         |
                     SQL Server

EF Core can handle:

Aggregate updates
CRUD
Transactions
Relationships
Change tracking
Domain operations

Dapper can handle:

Reports
Dashboards
Complex joins
Read models
Stored procedures
SQL-specific operations
Performance-sensitive queries

69. Final Conclusion

Both Dapper and Entity Framework Core are excellent database-access technologies for .NET, but they have different philosophies.

Dapper

Think:

"I want control over my SQL and a lightweight mapping layer."

EF Core

Think:

"I want an ORM that manages entities, relationships, tracking, LINQ and database schema evolution."

For many enterprise applications, the best architecture is not necessarily choosing one exclusively.

A hybrid approach can be very effective:

             ASP.NET Core Application
                       |
          +------------+------------+
          |                         |
       EF Core                    Dapper
          |                         |
    Business/Data               Reporting/
    modifications               Read models
          |                         |
          +------------+------------+
                       |
                    SQL Server

The most important skill for a senior .NET developer is not simply knowing Dapper vs EF Core, but understanding:

How SQL is executed
How connections are managed
How transactions work
How indexes affect queries
How to avoid N+1 queries
How to optimize EF Core queries
How to write efficient Dapper SQL
How to analyze execution plans
How to handle concurrency
How to design repositories and services
How to secure database access
How to monitor production database performance

Ultimately, database design, SQL quality, indexing, query patterns, and architecture usually have a much larger impact on application performance than simply selecting Dapper or EF Core.

Quick Interview Summary

If an interviewer asks:

"Dapper or EF Core — which one would you choose?"

A strong answer is:

"I would choose based on the application's requirements. EF Core is useful when we need an ORM with LINQ, change tracking, relationships, migrations and strong domain modeling. Dapper is useful when we need fine-grained SQL control, lightweight data access, complex reporting queries or existing stored procedures. In an enterprise application, I can also use both — EF Core for transactional/domain operations and Dapper for optimized read models and reporting. I would validate the decision using actual query performance, execution plans and application requirements rather than assuming one technology is always faster."

This is structured to work as a technical blog article and also as an interview-preparation reference. If you want, I can next turn this into a Dapper vs EF Core Part-2 with a complete ASP.NET Core Web API project, SQL Server database, Repository + Unit of Work + CQRS, and production-ready code.

MongoDB with Microsoft Azure and ASP.NET Core – Complete End-to-End Guide with Real-Time E-Commerce Example

2026-09-17T08:53:13.999+05:30

Introduction

Modern applications often need a database that can handle:

Large volumes of data
Rapidly changing data structures
High read/write workloads
Horizontal scalability
JSON-based application data
Microservices architectures
Event-driven applications

MongoDB is a popular NoSQL document database designed for these types of workloads.

When MongoDB is deployed using MongoDB Atlas on Microsoft Azure, we can build a cloud-native architecture where:

Angular / React / Mobile
          |
          v
     Azure Front Door
          |
          v
 Azure API Management
          |
          v
 ASP.NET Core Web API
          |
          v
     MongoDB Atlas
       on Azure

MongoDB Atlas is MongoDB's fully managed cloud database service, and Atlas supports Microsoft Azure as a cloud provider. (MongoDB)

1. What is MongoDB?

MongoDB is a NoSQL document-oriented database.

Unlike SQL Server, where data is normally organized as:

Database
   |
   +-- Tables
          |
          +-- Rows

MongoDB organizes data as:

Database
   |
   +-- Collections
          |
          +-- Documents

A MongoDB document looks similar to JSON:

{
  "customerId": "CUST1001",
  "name": "Ravi Kumar",
  "email": "ravi@example.com",
  "city": "Hyderabad"
}

Internally MongoDB stores documents using BSON, which extends JSON with additional data types.

2. SQL Server vs MongoDB

SQL Server	MongoDB
Database	Database
Table	Collection
Row	Document
Column	Field
Primary Key	`_id`
JOIN	`$lookup` / application-level modeling
Stored Procedure	Application/service logic
Relational	Document-oriented
Fixed schema commonly used	Flexible document schema
SQL	MongoDB Query API

For example, SQL Server might contain:

Customers
-------------------------
CustomerId
Name
Email
City

MongoDB could contain:

{
  "_id": "CUST1001",
  "name": "Ravi Kumar",
  "email": "ravi@example.com",
  "city": "Hyderabad"
}

3. What is MongoDB Atlas?

MongoDB Atlas is the managed cloud service for MongoDB.

Instead of installing and maintaining MongoDB servers yourself, Atlas manages much of the database infrastructure.

Conceptually:

Your Application
       |
       | MongoDB Driver
       |
       v
+----------------------+
|   MongoDB Atlas      |
|                      |
|  MongoDB Cluster     |
|                      |
|  Database            |
|    |                 |
|  Collection          |
|    |                 |
|  Documents           |
+----------------------+

Atlas can deploy MongoDB clusters on Microsoft Azure. (MongoDB)

4. Why MongoDB on Azure?

A common enterprise architecture is:

Azure
│
├── Azure Front Door
│
├── API Management
│
├── AKS / App Service
│       │
│       └── ASP.NET Core APIs
│
├── Key Vault
│
├── Application Insights
│
└── MongoDB Atlas
        │
        └── MongoDB Cluster

Advantages include:

Scalability

MongoDB can scale horizontally using sharding.

High availability

Atlas supports replica-set based deployments, and Azure availability zones can be used for supported regions. (MongoDB)

Managed infrastructure

Atlas manages much of the operational database infrastructure.

Flexible document model

Different documents can contain different fields when your application requires a flexible schema.

Cloud integration

Applications running on Azure can connect to MongoDB Atlas using standard or private networking approaches.

5. Real-Time Example

Let's take an E-Commerce Order Management System.

Suppose customers place orders through an Angular application.

Architecture:

                Customer
                   |
                   v
            Angular Application
                   |
                   v
           Azure API Management
                   |
                   v
          ASP.NET Core Web API
                   |
       +-----------+-----------+
       |                       |
       v                       v
 MongoDB Atlas             Azure Service Bus
       |                       |
       v                       v
 Orders Collection       Other Microservices

The Order Service needs to store:

Customer information
Order information
Products
Quantity
Price
Shipping address
Payment status
Order status
Created date

MongoDB is well suited to representing this as a document.

6. MongoDB Data Model

Our database can be:

ECommerceDB
   |
   +-- Orders
   |
   +-- Customers
   |
   +-- Products

For this example:

Database:
ECommerceDB

Collection:
Orders

7. Sample MongoDB Document

An order could look like this:

{
  "_id": "ORD10001",
  "customer": {
    "customerId": "CUST1001",
    "name": "Ravi Kumar",
    "email": "ravi@example.com"
  },
  "items": [
    {
      "productId": "P1001",
      "productName": "Laptop",
      "quantity": 1,
      "unitPrice": 75000
    },
    {
      "productId": "P1002",
      "productName": "Mouse",
      "quantity": 2,
      "unitPrice": 1500
    }
  ],
  "shippingAddress": {
    "street": "Hitech City",
    "city": "Hyderabad",
    "state": "Telangana",
    "country": "India",
    "postalCode": "500081"
  },
  "paymentStatus": "Paid",
  "orderStatus": "Confirmed",
  "totalAmount": 78000,
  "createdAt": "2026-09-17T08:30:00Z"
}

Notice that customer, items, and shipping information can be represented naturally inside the document.

8. MongoDB Terminology

Understanding the terminology is important.

MongoDB
   |
   +-- Database
          |
          +-- Collection
                  |
                  +-- Document
                          |
                          +-- Field

For our application:

MongoDB
   |
   +-- ECommerceDB
          |
          +-- Orders
          |
          +-- Customers
          |
          +-- Products

9. Create MongoDB Atlas on Azure

Go to MongoDB Atlas and create an Atlas organization/project.

When creating your cluster, select:

Cloud Provider:
Microsoft Azure

Then select an appropriate Azure region.

MongoDB currently lists Azure regions including Central India, South India and West India, among many other supported Azure regions. Availability and cluster-tier support vary by region. (MongoDB)

For an application primarily running in India, an appropriate India region can reduce network latency, subject to your application's requirements and the cluster tier available there.

10. Create Database User

Create a MongoDB database user such as:

Username:
ecommerce-api-user

Use a strong password.

Do not hard-code the password in source code.

Bad:

var connectionString =
    "mongodb+srv://user:password@cluster.mongodb.net";

Instead, use:

Azure Key Vault
       |
       v
ASP.NET Core
       |
       v
MongoDB Atlas

11. Configure Network Access

Atlas provides network controls that determine which clients can connect.

For development, you may configure an appropriate IP access rule.

For production, consider private networking where appropriate.

A production architecture could be:

Azure VNet
   |
   +-----------------------+
   |                       |
   v                       v
AKS                    Private Endpoint
   |                       |
   |                       v
   +--------------> MongoDB Atlas

MongoDB documents different connection approaches, including standard connections, peering and private endpoints depending on how the application network is configured. (MongoDB)

12. Get MongoDB Connection String

From Atlas, obtain the application connection string.

It generally resembles:

mongodb+srv://<username>:<password>@cluster0.xxxxx.mongodb.net/

The MongoDB .NET driver uses the connection URI together with MongoClient to establish the connection. (MongoDB)

Never publish your real connection string in GitHub or your blog.

13. Create ASP.NET Core Web API

Create the project:

dotnet new webapi -n Ecommerce.Api

Move into the project:

cd Ecommerce.Api

Install the MongoDB .NET driver:

dotnet add package MongoDB.Driver

MongoDB provides an official .NET/C# driver for connecting .NET applications to MongoDB deployments including Atlas. (MongoDB)

14. Configure appsettings.json

Create:

{
  "MongoDB": {
    "ConnectionString": "mongodb+srv://<username>:<password>@cluster0.xxxxx.mongodb.net/",
    "DatabaseName": "ECommerceDB",
    "OrdersCollectionName": "Orders"
  }
}

For production, don't store the actual secret directly in appsettings.json.

Instead:

Azure Key Vault
      |
      v
ASP.NET Core Configuration
      |
      v
MongoDB Driver

MongoDB's own REST API tutorial demonstrates storing the MongoDB URI, database name, and collection name in application configuration. (MongoDB)

15. Create MongoDB Settings Class

Create:

Configuration/
    MongoDbSettings.cs

public class MongoDbSettings
{
    public string ConnectionString { get; set; } = string.Empty;

    public string DatabaseName { get; set; } = string.Empty;

    public string OrdersCollectionName { get; set; } = string.Empty;
}

16. Create Order Model

using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;

public class Order
{
    [BsonId]
    public string Id { get; set; } = string.Empty;

    public Customer Customer { get; set; } = new();

    public List<OrderItem> Items { get; set; } = new();

    public ShippingAddress ShippingAddress { get; set; } = new();

    public decimal TotalAmount { get; set; }

    public string PaymentStatus { get; set; } = string.Empty;

    public string OrderStatus { get; set; } = string.Empty;

    public DateTime CreatedAt { get; set; }
}

17. Customer Model

public class Customer
{
    public string CustomerId { get; set; } = string.Empty;

    public string Name { get; set; } = string.Empty;

    public string Email { get; set; } = string.Empty;
}

18. OrderItem Model

public class OrderItem
{
    public string ProductId { get; set; } = string.Empty;

    public string ProductName { get; set; } = string.Empty;

    public int Quantity { get; set; }

    public decimal UnitPrice { get; set; }
}

19. ShippingAddress Model

public class ShippingAddress
{
    public string Street { get; set; } = string.Empty;

    public string City { get; set; } = string.Empty;

    public string State { get; set; } = string.Empty;

    public string Country { get; set; } = string.Empty;

    public string PostalCode { get; set; } = string.Empty;
}

20. Create MongoDB Service

Create:

Services/
    OrderService.cs

using MongoDB.Driver;

public class OrderService
{
    private readonly IMongoCollection<Order> _orders;

    public OrderService(IConfiguration configuration)
    {
        var connectionString =
            configuration["MongoDB:ConnectionString"];

        var databaseName =
            configuration["MongoDB:DatabaseName"];

        var collectionName =
            configuration["MongoDB:OrdersCollectionName"];

        var client = new MongoClient(connectionString);

        var database = client.GetDatabase(databaseName);

        _orders = database.GetCollection<Order>(collectionName);
    }
}

The basic MongoDB .NET architecture is:

MongoDB Connection String
          |
          v
     MongoClient
          |
          v
      Database
          |
          v
     Collection
          |
          v
      Documents

21. Insert Order

Add:

public async Task CreateAsync(Order order)
{
    await _orders.InsertOneAsync(order);
}

Complete example:

public async Task CreateAsync(Order order)
{
    order.Id = $"ORD{DateTime.UtcNow.Ticks}";

    order.CreatedAt = DateTime.UtcNow;

    await _orders.InsertOneAsync(order);
}

MongoDB creates the document in the Orders collection.

22. Insert Sample Data

You can insert this document:

{
  "_id": "ORD10001",
  "customer": {
    "customerId": "CUST1001",
    "name": "Ravi Kumar",
    "email": "ravi@example.com"
  },
  "items": [
    {
      "productId": "P1001",
      "productName": "Laptop",
      "quantity": 1,
      "unitPrice": 75000
    },
    {
      "productId": "P1002",
      "productName": "Mouse",
      "quantity": 2,
      "unitPrice": 1500
    }
  ],
  "shippingAddress": {
    "street": "Hitech City",
    "city": "Hyderabad",
    "state": "Telangana",
    "country": "India",
    "postalCode": "500081"
  },
  "totalAmount": 78000,
  "paymentStatus": "Paid",
  "orderStatus": "Confirmed",
  "createdAt": "2026-09-17T08:30:00Z"
}

23. Retrieve All Orders

public async Task<List<Order>> GetAllAsync()
{
    return await _orders
        .Find(_ => true)
        .ToListAsync();
}

Here:

.Find(_ => true)

means:

Return all documents.

24. Retrieve Order by ID

public async Task<Order?> GetByIdAsync(string id)
{
    return await _orders
        .Find(x => x.Id == id)
        .FirstOrDefaultAsync();
}

25. Search Orders by Customer

public async Task<List<Order>> GetByCustomerAsync(
    string customerId)
{
    return await _orders
        .Find(x => x.Customer.CustomerId == customerId)
        .ToListAsync();
}

This demonstrates querying a nested document property.

26. Search Orders by City

public async Task<List<Order>> GetByCityAsync(
    string city)
{
    return await _orders
        .Find(x => x.ShippingAddress.City == city)
        .ToListAsync();
}

For example:

GET /api/orders/city/Hyderabad

27. Update an Order

Suppose an order changes from:

Confirmed

to:

Shipped

We can write:

public async Task UpdateStatusAsync(
    string id,
    string status)
{
    var filter =
        Builders<Order>.Filter.Eq(x => x.Id, id);

    var update =
        Builders<Order>.Update
            .Set(x => x.OrderStatus, status);

    await _orders.UpdateOneAsync(
        filter,
        update);
}

28. Delete an Order

public async Task DeleteAsync(string id)
{
    await _orders.DeleteOneAsync(
        x => x.Id == id);
}

29. Create Controller

using Microsoft.AspNetCore.Mvc;

[ApiController]
[Route("api/[controller]")]
public class OrdersController : ControllerBase
{
    private readonly OrderService _service;

    public OrdersController(OrderService service)
    {
        _service = service;
    }

    [HttpGet]
    public async Task<IActionResult> GetAll()
    {
        var orders = await _service.GetAllAsync();

        return Ok(orders);
    }

    [HttpGet("{id}")]
    public async Task<IActionResult> GetById(string id)
    {
        var order = await _service.GetByIdAsync(id);

        if (order == null)
            return NotFound();

        return Ok(order);
    }

    [HttpPost]
    public async Task<IActionResult> Create(Order order)
    {
        await _service.CreateAsync(order);

        return Ok(order);
    }

    [HttpDelete("{id}")]
    public async Task<IActionResult> Delete(string id)
    {
        await _service.DeleteAsync(id);

        return NoContent();
    }
}

30. Register MongoDB Service

In Program.cs:

builder.Services.AddSingleton<OrderService>();

Then:

var app = builder.Build();

app.MapControllers();

app.Run();

A simplified architecture becomes:

HTTP Request
     |
     v
OrdersController
     |
     v
OrderService
     |
     v
MongoClient
     |
     v
MongoDB Atlas
     |
     v
Orders Collection

31. POST Request Example

Send:

POST /api/orders
Content-Type: application/json

Request body:

{
  "customer": {
    "customerId": "CUST1002",
    "name": "Suresh",
    "email": "suresh@example.com"
  },
  "items": [
    {
      "productId": "P2001",
      "productName": "Mobile Phone",
      "quantity": 1,
      "unitPrice": 45000
    }
  ],
  "shippingAddress": {
    "street": "Madhapur",
    "city": "Hyderabad",
    "state": "Telangana",
    "country": "India",
    "postalCode": "500081"
  },
  "totalAmount": 45000,
  "paymentStatus": "Paid",
  "orderStatus": "Confirmed"
}

32. GET Request

GET /api/orders

Response:

[
  {
    "_id": "ORD10001",
    "customer": {
      "customerId": "CUST1001",
      "name": "Ravi Kumar"
    },
    "totalAmount": 78000,
    "orderStatus": "Confirmed"
  }
]

33. MongoDB Query Example

MongoDB shell query:

db.Orders.find({
    "customer.customerId": "CUST1001"
})

Another example:

db.Orders.find({
    "shippingAddress.city": "Hyderabad"
})

Find paid orders:

db.Orders.find({
    "paymentStatus": "Paid"
})

34. MongoDB Operators

MongoDB provides many query operators.

Greater Than

db.Orders.find({
    "totalAmount": {
        $gt: 50000
    }
})

Less Than

db.Orders.find({
    "totalAmount": {
        $lt: 50000
    }
})

Greater Than or Equal

db.Orders.find({
    "totalAmount": {
        $gte: 50000
    }
})

IN

db.Orders.find({
    "orderStatus": {
        $in: ["Confirmed", "Shipped"]
    }
})

35. MongoDB Indexes

Indexes are extremely important for production applications.

Suppose we frequently search:

CustomerId

Create an index:

db.Orders.createIndex({
    "customer.customerId": 1
})

For city:

db.Orders.createIndex({
    "shippingAddress.city": 1
})

For order status:

db.Orders.createIndex({
    "orderStatus": 1
})

The application should create indexes based on actual query patterns rather than creating indexes on every field.

MongoDB's .NET driver documentation includes dedicated guidance for creating and managing indexes. (MongoDB)

36. Compound Index

Suppose the application frequently queries:

Customer + Order Status

We can create:

db.Orders.createIndex({
    "customer.customerId": 1,
    "orderStatus": 1
})

This can support queries such as:

db.Orders.find({
    "customer.customerId": "CUST1001",
    "orderStatus": "Confirmed"
})

37. MongoDB Aggregation

Aggregation is one of MongoDB's powerful features.

Suppose we want:

Total sales by city

We can use:

db.Orders.aggregate([
    {
        $group: {
            _id: "$shippingAddress.city",
            totalSales: {
                $sum: "$totalAmount"
            }
        }
    }
])

Example result:

[
  {
    "_id": "Hyderabad",
    "totalSales": 2500000
  },
  {
    "_id": "Bangalore",
    "totalSales": 1800000
  }
]

38. Aggregation Pipeline

MongoDB aggregation works as a pipeline:

Documents
    |
    v
   $match
    |
    v
  $group
    |
    v
  $sort
    |
    v
 Result

Example:

db.Orders.aggregate([
    {
        $match: {
            "paymentStatus": "Paid"
        }
    },
    {
        $group: {
            _id: "$shippingAddress.city",
            totalSales: {
                $sum: "$totalAmount"
            }
        }
    },
    {
        $sort: {
            totalSales: -1
        }
    }
])

39. Consuming MongoDB Data from Angular

The frontend does not normally connect directly to MongoDB.

Instead:

Angular
   |
   | HTTP
   v
ASP.NET Core Web API
   |
   | MongoDB Driver
   v
MongoDB Atlas

Angular service:

@Injectable({
  providedIn: 'root'
})
export class OrderService {

  private apiUrl = 'https://api.example.com/api/orders';

  constructor(private http: HttpClient) {}

  getOrders() {
    return this.http.get<any[]>(this.apiUrl);
  }

  getOrder(id: string) {
    return this.http.get<any>(
      `${this.apiUrl}/${id}`
    );
  }
}

Component:

export class OrdersComponent {

  orders: any[] = [];

  constructor(
    private orderService: OrderService
  ) {}

  ngOnInit() {

    this.orderService
      .getOrders()
      .subscribe(data => {
        this.orders = data;
      });

  }
}

40. Complete Data Flow

When the Angular application requests orders:

1. User opens Orders screen
             |
             v
2. Angular sends HTTP GET
             |
             v
3. Azure API Management
             |
             v
4. ASP.NET Core API
             |
             v
5. OrderService
             |
             v
6. MongoDB Driver
             |
             v
7. MongoDB Atlas
             |
             v
8. Orders Collection
             |
             v
9. MongoDB returns documents
             |
             v
10. ASP.NET Core returns JSON
             |
             v
11. Angular displays data

41. Production Azure Architecture

For a larger enterprise system, the architecture could look like this:

                         Internet
                            |
                            v
                  +-------------------+
                  | Azure Front Door  |
                  +-------------------+
                            |
                            v
                  +-------------------+
                  | Azure API         |
                  | Management        |
                  +-------------------+
                            |
                            v
                  +-------------------+
                  | Azure Application |
                  | / AKS             |
                  +-------------------+
                            |
               +------------+------------+
               |                         |
               v                         v
        Order Microservice       Customer Service
               |                         |
               v                         v
        MongoDB Atlas              MongoDB Atlas
               |
               v
       Orders Collection

Supporting services:

Azure Key Vault
Azure Monitor
Application Insights
Azure Service Bus
Azure Container Registry
Azure DevOps

42. MongoDB with Microservices

MongoDB works particularly well with microservice architectures when each service owns its data.

For example:

Order Service
     |
     v
OrderDB
     |
     +-- Orders

Customer Service
     |
     v
CustomerDB
     |
     +-- Customers

Product Service
     |
     v
ProductDB
     |
     +-- Products

Instead of:

All Microservices
       |
       v
One Shared Database

we can use:

Order Service --------> Order Database

Customer Service -----> Customer Database

Product Service ------> Product Database

This helps maintain service ownership and reduces tight database coupling.

43. MongoDB and Azure Service Bus

Consider an order workflow:

Customer
   |
   v
Order API
   |
   v
MongoDB
   |
   v
Order Created
   |
   v
Azure Service Bus
   |
   +-----------> Inventory Service
   |
   +-----------> Payment Service
   |
   +-----------> Notification Service

For example:

{
  "eventType": "OrderCreated",
  "orderId": "ORD10001",
  "customerId": "CUST1001",
  "totalAmount": 78000
}

MongoDB stores the order state while Azure Service Bus can be used for asynchronous communication between services.

44. Security Architecture

A production architecture should avoid exposing database credentials.

Recommended:

Azure Key Vault
       |
       | Secret
       v
ASP.NET Core
       |
       | TLS
       v
MongoDB Atlas

Instead of:

appsettings.json
       |
       +-- username
       +-- password

Use:

Key Vault
   |
   v
Managed Identity
   |
   v
ASP.NET Core

Also consider:

TLS encryption
Network restrictions
Private connectivity where appropriate
Least-privilege database users
Secret rotation
Application-level authentication and authorization
Audit and monitoring

45. Azure Deployment Flow

A typical CI/CD pipeline could be:

Developer
    |
    v
Git Repository
    |
    v
Azure DevOps Pipeline
    |
    +---- Build
    |
    +---- Unit Tests
    |
    +---- Security Scan
    |
    +---- Docker Build
    |
    +---- Push to ACR
    |
    v
Azure AKS
    |
    v
ASP.NET Core Container
    |
    v
MongoDB Atlas

46. Docker Example

A simplified Dockerfile:

FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base

WORKDIR /app

EXPOSE 8080

FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build

WORKDIR /src

COPY . .

RUN dotnet restore

RUN dotnet publish \
    -c Release \
    -o /app/publish

FROM base AS final

WORKDIR /app

COPY --from=build /app/publish .

ENTRYPOINT ["dotnet", "Ecommerce.Api.dll"]

The container can then be deployed to Azure services such as AKS or another appropriate Azure hosting platform.

47. Performance Considerations

When using MongoDB in production, pay attention to:

1. Indexes

Create indexes for frequently executed queries.

2. Document Size

Avoid unnecessarily huge documents.

3. Query Projection

Retrieve only the fields you need when appropriate.

4. Connection Management

Reuse MongoClient rather than creating a new client for every request.

5. Pagination

Don't return millions of records in a single API request.

Example:

GET /api/orders?page=1&pageSize=50

6. Monitoring

Monitor:

CPU
Memory
Disk
Connections
Query performance
Latency
Operations per second

48. Pagination Example

A simple MongoDB pagination query:

public async Task<List<Order>> GetPagedAsync(
    int page,
    int pageSize)
{
    return await _orders
        .Find(_ => true)
        .Skip((page - 1) * pageSize)
        .Limit(pageSize)
        .ToListAsync();
}

For example:

page = 1
pageSize = 20

MongoDB returns the first 20 documents.

49. MongoDB Repository Pattern

For larger applications, you can introduce a repository layer:

Controller
    |
    v
Application Service
    |
    v
Repository
    |
    v
MongoDB

Example:

public interface IOrderRepository
{
    Task<List<Order>> GetAllAsync();

    Task<Order?> GetByIdAsync(string id);

    Task CreateAsync(Order order);

    Task UpdateAsync(Order order);

    Task DeleteAsync(string id);
}

Implementation:

public class OrderRepository : IOrderRepository
{
    private readonly IMongoCollection<Order> _orders;

    public OrderRepository(IMongoDatabase database)
    {
        _orders = database.GetCollection<Order>("Orders");
    }

    public async Task<List<Order>> GetAllAsync()
    {
        return await _orders
            .Find(_ => true)
            .ToListAsync();
    }

    public async Task<Order?> GetByIdAsync(string id)
    {
        return await _orders
            .Find(x => x.Id == id)
            .FirstOrDefaultAsync();
    }

    public async Task CreateAsync(Order order)
    {
        await _orders.InsertOneAsync(order);
    }

    public async Task UpdateAsync(Order order)
    {
        await _orders.ReplaceOneAsync(
            x => x.Id == order.Id,
            order);
    }

    public async Task DeleteAsync(string id)
    {
        await _orders.DeleteOneAsync(
            x => x.Id == id);
    }
}

50. Recommended Enterprise Project Structure

A clean architecture could look like:

Ecommerce.Api
│
├── Controllers
│     └── OrdersController.cs
│
├── Models
│     ├── Order.cs
│     ├── Customer.cs
│     └── OrderItem.cs
│
├── DTOs
│     ├── CreateOrderRequest.cs
│     └── OrderResponse.cs
│
├── Services
│     └── OrderService.cs
│
├── Repositories
│     ├── IOrderRepository.cs
│     └── OrderRepository.cs
│
├── Configuration
│     └── MongoDbSettings.cs
│
├── Program.cs
│
└── appsettings.json

51. MongoDB vs SQL Server – When to Use Which?

MongoDB can be useful when:

Data is naturally document-oriented
Schema changes frequently
High-scale workloads require horizontal scaling
Application data is naturally represented as JSON-like documents
Microservices need independently owned data
Flexible document structures are valuable

SQL Server can be preferable when:

Strong relational modeling is central
Complex joins dominate the workload
Existing enterprise applications rely heavily on relational features
Strong transactional relational constraints are fundamental

The choice should be based on workload and data requirements rather than simply choosing NoSQL because it is newer.

52. Real-Time End-to-End Example

Let's put everything together.

Suppose a customer purchases a laptop.

Step 1 – Customer

Customer submits:

Laptop
Quantity: 1
Price: ₹75,000

Step 2 – Angular

Angular sends:

POST /api/orders

Step 3 – Azure API Management

APIM handles API gateway responsibilities.

Step 4 – ASP.NET Core

The Order Controller receives the request.

Step 5 – Order Service

The service validates the order.

Step 6 – MongoDB

The Order Service saves:

{
  "_id": "ORD10001",
  "customer": {
    "customerId": "CUST1001"
  },
  "items": [
    {
      "productId": "P1001",
      "quantity": 1,
      "unitPrice": 75000
    }
  ],
  "totalAmount": 75000,
  "orderStatus": "Confirmed"
}

Step 7 – Event

The service publishes:

OrderCreated

to Azure Service Bus.

Step 8 – Inventory

Inventory Service receives the event.

Laptop Stock
100 → 99

Step 9 – Payment

Payment Service processes payment.

Step 10 – Notification

Notification Service sends the order confirmation.

Complete flow:

Angular
   |
   v
Azure API Management
   |
   v
Order API
   |
   v
Order Service
   |
   +--------------------+
   |                    |
   v                    v
MongoDB Atlas       Azure Service Bus
   |                    |
   |             +------+------+
   |             |             |
   |             v             v
   |        Inventory      Payment
   |                         |
   |                         v
   |                    Notification
   |
   v
Order Status

53. Important Production Best Practices

Database

Use appropriate indexes.
Design documents around access patterns.
Avoid unnecessarily large documents.
Monitor slow queries.
Use appropriate replication/high availability configuration.

Application

Reuse MongoClient.
Use asynchronous APIs.
Implement pagination.
Use DTOs rather than exposing internal models directly.
Validate incoming requests.

Security

Never commit passwords.
Use Azure Key Vault.
Use least-privilege database accounts.
Restrict network access.
Use encrypted connections.

Azure

Use Application Insights/Azure Monitor.
Use managed identities where applicable.
Use private networking where appropriate.
Use CI/CD.
Use autoscaling based on workload requirements.

Architecture

Keep database ownership with the appropriate service.
Use Service Bus for asynchronous communication where appropriate.
Don't make every microservice directly access every other service's database.

54. Complete Architecture Summary

The complete enterprise architecture can be represented as:

                         USERS
                           |
                           v
                    Angular / Mobile
                           |
                           v
                  +-------------------+
                  | Azure Front Door  |
                  +-------------------+
                           |
                           v
                  +-------------------+
                  | Azure API         |
                  | Management        |
                  +-------------------+
                           |
                           v
                +-----------------------+
                | Azure AKS / App       |
                | Service               |
                +-----------------------+
                           |
                           v
                 ASP.NET Core Web API
                           |
              +------------+-------------+
              |                          |
              v                          v
       MongoDB Atlas              Azure Service Bus
              |                          |
              v                    +-----+------+
       ECommerceDB                 |            |
              |                    v            v
          Collections          Inventory     Payment
              |
       +------+------+
       |             |
    Orders        Customers
       |
       v
   Products
       
Supporting Services:
       
Azure Key Vault
Azure Monitor
Application Insights
Azure Container Registry
Azure DevOps

55. Conclusion

MongoDB provides a document-oriented approach to data storage that can be particularly useful for modern cloud-native and microservices applications.

When combined with MongoDB Atlas on Microsoft Azure, an ASP.NET Core application can follow a cloud architecture such as:

Frontend
   ↓
Azure API Management
   ↓
ASP.NET Core
   ↓
MongoDB Atlas on Azure

For an enterprise application, this can be extended with:

Azure Front Door
       ↓
API Management
       ↓
AKS
       ↓
Microservices
       ↓
MongoDB Atlas
       +
Azure Service Bus
       +
Azure Key Vault
       +
Application Insights

The official MongoDB .NET driver provides the APIs needed to connect to Atlas and perform CRUD, aggregation, indexing, and other database operations. (MongoDB)

Useful official references

MongoDB Atlas on Microsoft Azure
MongoDB .NET/C# Driver documentation
.NET Driver – Get Started
MongoDB Atlas Driver Connection Guide

Terraform: Complete Guide with Real-Time Azure and .NET Microservices Example -2026

2026-09-16T08:31:06.303+05:30

Introduction

In modern cloud-based application development, manually creating and configuring infrastructure through the Azure Portal, AWS Console, or other cloud management portals can become difficult as applications grow.

A typical enterprise .NET application may require multiple infrastructure components such as:

Azure Virtual Network
Azure Kubernetes Service (AKS)
Azure Container Registry (ACR)
Azure SQL Database
Azure Service Bus
Azure Key Vault
Azure API Management
Application Gateway
Application Insights
Azure Monitor
Storage Accounts
Managed Identities

Creating all these resources manually can be time-consuming and error-prone.

This is where Terraform becomes extremely useful.

Terraform allows us to define infrastructure using code and manage that infrastructure consistently across different environments such as:

Development
QA
UAT
Production

Terraform is an Infrastructure as Code (IaC) tool developed by HashiCorp.

Instead of manually creating infrastructure, we describe what infrastructure we need in Terraform configuration files, and Terraform creates and manages those resources for us.

1. What Is Terraform?

Terraform is an Infrastructure as Code tool used to provision and manage infrastructure using configuration files.

For example, instead of opening the Azure Portal and manually creating a Resource Group, we can write:

resource "azurerm_resource_group" "oneview" {
  name     = "rg-oneview-dev"
  location = "East US"
}

Terraform interprets this configuration and creates the corresponding Azure Resource Group.

The basic idea is:

Terraform Configuration
        |
        v
    Terraform
        |
        v
   Azure Provider
        |
        v
 Azure Infrastructure

Terraform can manage infrastructure from many platforms and services.

Examples include:

Microsoft Azure
Amazon Web Services
Google Cloud
Kubernetes
GitHub
Databases
Monitoring platforms
SaaS applications

2. What Is Infrastructure as Code?

Infrastructure as Code means managing infrastructure through machine-readable configuration files instead of manually configuring infrastructure.

Traditional Approach

Suppose we need to create an AKS cluster.

An engineer may manually:

Open Azure Portal
Create Resource Group
Create Virtual Network
Create Subnet
Create AKS
Configure networking
Configure identity
Configure permissions
Configure monitoring
Connect AKS with ACR

This process can be repeated for every environment.

Developer
    |
    v
Azure Portal
    |
    +--> Resource Group
    +--> VNet
    +--> Subnet
    +--> AKS
    +--> ACR
    +--> SQL
    +--> Service Bus

The problem is that manual configuration can introduce inconsistencies.

For example:

DEV  -> 2 AKS nodes
QA   -> 3 AKS nodes
UAT  -> 4 AKS nodes
PROD -> 6 AKS nodes

Some differences may be intentional, while others may happen because of manual configuration.

Infrastructure as Code Approach

With Terraform:

Terraform Code
      |
      v
Terraform Plan
      |
      v
Terraform Apply
      |
      v
Azure Infrastructure

The infrastructure configuration is stored in Git just like application source code.

3. Why Do We Need Terraform?

Terraform solves several infrastructure management problems.

3.1 Automation

Infrastructure can be created automatically.

3.2 Repeatability

The same infrastructure configuration can be reused for multiple environments.

3.3 Version Control

Terraform files can be stored in Git.

For example:

Git
 |
 +-- Terraform Code
 |
 +-- Version History
 |
 +-- Pull Requests
 |
 +-- Code Reviews

3.4 Consistency

Dev, QA, UAT, and Production can use standardized infrastructure modules.

3.5 Disaster Recovery

Infrastructure can be recreated from code when appropriate.

3.6 Collaboration

Multiple developers, architects, and DevOps engineers can work with the same infrastructure definition.

4. Terraform Architecture

The Terraform architecture can be represented as follows:

                Terraform Configuration
                         |
                         v
                  Terraform CLI
                         |
          +--------------+--------------+
          |                             |
          v                             v
   Terraform State                 Terraform Provider
                                        |
                                        v
                                  Azure APIs
                                        |
                                        v
                              Azure Infrastructure

The major components are:

Terraform CLI
Terraform Configuration
Providers
Resources
Variables
Data Sources
State
Backend
Modules
Outputs

Let's understand each one.

5. Terraform Configuration Files

Terraform configuration files generally use the .tf extension.

For example:

main.tf
variables.tf
outputs.tf
provider.tf
network.tf
aks.tf
sql.tf
servicebus.tf

Terraform reads all .tf files in the working directory.

Therefore, we don't have to put everything into one file.

A large enterprise project can be organized into multiple files.

6. HCL — HashiCorp Configuration Language

Terraform configurations are generally written using HCL.

For example:

resource "azurerm_resource_group" "oneview" {
  name     = "rg-oneview-dev"
  location = "East US"
}

The syntax is:

resource "RESOURCE_TYPE" "RESOURCE_NAME" {
    configuration
}

In this example:

resource
   |
   +-- azurerm_resource_group
   |
   +-- oneview

7. Terraform Providers

A provider allows Terraform to communicate with an external platform or service.

For Azure, we commonly use:

azurerm

Example:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
}

provider "azurerm" {
  features {}
}

The provider acts as the bridge between Terraform and Azure.

Terraform
    |
    v
Azure Provider
    |
    v
Azure API
    |
    v
Azure Resources

8. Installing Terraform

After installing Terraform, verify the installation:

terraform version

You should see the installed Terraform version.

For Azure development, Azure CLI is also useful.

Check Azure CLI:

az version

az login

You can then select the required Azure subscription.

az account set --subscription "<subscription-id>"

9. Real-Time Project Example

Let's consider a real-world .NET application called:

OneView Workforce Planning Platform

Suppose OneView contains the following microservices:

Forecast Service
Workforce Service
Capacity Service
Hiring Service

A possible architecture is:

                         Users
                           |
                           v
                  Application Gateway
                           |
                           v
                  Azure API Management
                           |
          +----------------+----------------+
          |                |                |
          v                v                v
    Forecast API     Workforce API     Capacity API
          |                |                |
          +----------------+----------------+
                           |
                           v
                  Azure Service Bus
                           |
                           v
                     Hiring Service
                           |
                           v
                      Azure SQL

Supporting Azure services might include:

AKS
ACR
Azure SQL
Service Bus
Key Vault
API Management
Application Gateway
Application Insights
Azure Monitor
Virtual Network
Managed Identity

Terraform can provision much of this infrastructure.

10. Creating the Terraform Project

Let's create the following project structure:

oneview-infrastructure/
│
├── provider.tf
├── variables.tf
├── resource-group.tf
├── network.tf
├── acr.tf
├── aks.tf
├── servicebus.tf
├── outputs.tf
└── terraform.tfvars

For a larger enterprise project, we can later convert these resources into reusable modules.

11. Configure Azure Provider

Create:

provider.tf

Add:

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
}

provider "azurerm" {
  features {}
}

This tells Terraform that the project uses the Azure Resource Manager provider.

12. Create Azure Resource Group

Create:

resource-group.tf

resource "azurerm_resource_group" "oneview" {
  name     = var.resource_group_name
  location = var.location

  tags = {
    Environment = var.environment
    Application = "OneView"
    ManagedBy   = "Terraform"
  }
}

Instead of hardcoding values, we are using variables.

13. Terraform Variables

Create:

variables.tf

variable "resource_group_name" {
  description = "Azure Resource Group name"
  type        = string
}

variable "location" {
  description = "Azure region"
  type        = string
  default     = "East US"
}

variable "environment" {
  description = "Deployment environment"
  type        = string
  default     = "dev"
}

variable "aks_node_count" {
  description = "Number of AKS nodes"
  type        = number
  default     = 2
}

Now create:

terraform.tfvars

resource_group_name = "rg-oneview-dev"
location            = "East US"
environment         = "dev"
aks_node_count      = 2

14. Terraform Init

Navigate to the project directory:

cd oneview-infrastructure

Run:

terraform init

Terraform downloads the required provider and initializes the working directory.

Typical flow:

Terraform Project
       |
       v
terraform init
       |
       v
Download Providers
       |
       v
Initialized

15. Terraform Validate

Before creating infrastructure, validate the configuration:

terraform validate

This checks the Terraform configuration for syntax and configuration errors.

16. Terraform Format

Terraform provides a formatting command:

terraform fmt

This automatically formats Terraform files according to Terraform's standard formatting rules.

17. Terraform Plan

Now execute:

terraform plan

Terraform calculates what changes are required.

For example:

Plan: 1 to add, 0 to change, 0 to destroy.

The important point is:

terraform plan previews changes.

It is normally used before applying infrastructure changes.

18. Terraform Apply

To create the infrastructure:

terraform apply

Terraform displays the planned changes and asks for confirmation.

Enter:

yes

Terraform then creates the resources.

You can also use:

terraform apply -auto-approve

However, automatic approval should be used carefully, especially for production environments.

19. Creating Azure Virtual Network

Now let's create networking.

Create:

network.tf

resource "azurerm_virtual_network" "oneview" {
  name                = "vnet-oneview"
  location            = azurerm_resource_group.oneview.location
  resource_group_name = azurerm_resource_group.oneview.name
  address_space       = ["10.10.0.0/16"]

  tags = {
    Environment = var.environment
    Application = "OneView"
    ManagedBy   = "Terraform"
  }
}

Create an AKS subnet:

resource "azurerm_subnet" "aks" {
  name                 = "snet-aks"
  resource_group_name  = azurerm_resource_group.oneview.name
  virtual_network_name = azurerm_virtual_network.oneview.name
  address_prefixes     = ["10.10.1.0/24"]
}

20. Creating Azure Container Registry

.NET microservices can be packaged as Docker containers.

Those container images can be stored in Azure Container Registry.

Create:

acr.tf

resource "azurerm_container_registry" "oneview" {
  name                = "acroneviewdev123"
  resource_group_name = azurerm_resource_group.oneview.name
  location            = azurerm_resource_group.oneview.location

  sku = "Standard"

  admin_enabled = false

  tags = {
    Environment = var.environment
    Application = "OneView"
    ManagedBy   = "Terraform"
  }
}

The application deployment flow becomes:

.NET Source Code
       |
       v
Docker Build
       |
       v
Container Image
       |
       v
Azure Container Registry
       |
       v
AKS

21. Creating Azure Kubernetes Service

Now let's create AKS.

Create:

aks.tf

Example:

resource "azurerm_kubernetes_cluster" "oneview" {

  name                = "aks-oneview-dev"
  location            = azurerm_resource_group.oneview.location
  resource_group_name = azurerm_resource_group.oneview.name
  dns_prefix          = "oneview-dev"

  default_node_pool {
    name       = "system"
    node_count = var.aks_node_count
    vm_size    = "Standard_D2s_v5"
  }

  identity {
    type = "SystemAssigned"
  }

  tags = {
    Environment = var.environment
    Application = "OneView"
    ManagedBy   = "Terraform"
  }
}

Terraform now understands that AKS belongs to the resource group.

22. Terraform Dependency Management

Terraform automatically identifies many dependencies by analyzing resource references.

For example:

resource_group_name = azurerm_resource_group.oneview.name

Terraform understands:

Resource Group
       |
       v
      AKS

Therefore, Terraform creates the Resource Group before AKS.

This is called an implicit dependency.

23. Explicit Dependencies

Sometimes Terraform cannot determine a dependency automatically.

In those cases, we can use:

depends_on = [
  azurerm_resource_group.oneview
]

Example:

resource "some_resource" "example" {

  # configuration

  depends_on = [
    azurerm_resource_group.oneview
  ]
}

It is generally preferable to rely on implicit dependencies whenever possible.

24. Connect AKS with ACR

AKS needs permission to pull container images from ACR.

We can assign the AcrPull role to the AKS kubelet identity.

Example:

resource "azurerm_role_assignment" "aks_acr" {

  principal_id = azurerm_kubernetes_cluster.oneview
    .kubelet_identity[0]
    .object_id

  role_definition_name = "AcrPull"

  scope = azurerm_container_registry.oneview.id

  skip_service_principal_aad_check = true
}

The relationship is:

Azure Container Registry
          |
          | AcrPull
          v
         AKS
          |
          v
      Kubernetes
          |
          v
       .NET Pods

25. Azure Service Bus with Terraform

In a microservices architecture, Azure Service Bus can be used for asynchronous communication.

Create:

servicebus.tf

Example:

resource "azurerm_servicebus_namespace" "oneview" {

  name                = "sb-oneview-dev"
  location            = azurerm_resource_group.oneview.location
  resource_group_name = azurerm_resource_group.oneview.name

  sku = "Standard"

  tags = {
    Environment = var.environment
    Application = "OneView"
    ManagedBy   = "Terraform"
  }
}

Create a topic:

resource "azurerm_servicebus_topic" "forecast_events" {

  name         = "forecast-events"
  namespace_id = azurerm_servicebus_namespace.oneview.id
}

Create a subscription:

resource "azurerm_servicebus_subscription" "workforce" {

  name               = "workforce-subscription"
  topic_id           = azurerm_servicebus_topic.forecast_events.id
  max_delivery_count = 10
}

The communication flow becomes:

Forecast Service
       |
       | ForecastCreated
       v
forecast-events
       |
       v
workforce-subscription
       |
       v
Workforce Service

26. Terraform Outputs

Create:

outputs.tf

output "resource_group_name" {
  value = azurerm_resource_group.oneview.name
}

output "aks_cluster_name" {
  value = azurerm_kubernetes_cluster.oneview.name
}

output "acr_login_server" {
  value = azurerm_container_registry.oneview.login_server
}

After applying the configuration:

terraform output

Terraform can display the output values.

27. Terraform State

One of the most important concepts in Terraform is state.

Terraform maintains information about infrastructure it manages.

The local state file is commonly:

terraform.tfstate

Conceptually:

Terraform Configuration
          |
          v
     Desired State
          |
          |
Terraform State
          |
          v
     Actual Azure

Terraform uses state to understand relationships between configuration and infrastructure.

28. Why Remote State Is Important

In a team environment, storing state only on an individual developer's machine is usually not appropriate.

Imagine:

Developer A
     |
Developer B
     |
Developer C
     |
Azure DevOps
     |
     v
Shared Terraform State

For Azure, an Azure Storage Account can be used as a remote backend.

Example:

terraform {
  backend "azurerm" {
    resource_group_name  = "rg-terraform-state"
    storage_account_name = "tfstateoneview"
    container_name       = "tfstate"
    key                  = "oneview-dev.tfstate"
  }
}

The backend infrastructure should generally be bootstrapped separately before configuring the project to use it.

29. Terraform State Locking

Consider this situation:

Developer A --------+
                    |
Developer B --------+----> Terraform State
                    |
CI/CD Pipeline -----+

If multiple operations modify the same state concurrently, problems can occur.

Remote backends can provide state locking or concurrency controls, depending on the backend.

Therefore, enterprise Terraform implementations should carefully design:

Remote state
State isolation
Locking
Access control
Backup
Recovery

30. Terraform Modules

As Terraform projects become larger, we should avoid putting hundreds or thousands of lines into a single file.

Terraform supports reusable modules.

Example:

terraform/
│
├── modules/
│   ├── network/
│   ├── aks/
│   ├── acr/
│   ├── sql/
│   ├── servicebus/
│   ├── keyvault/
│   └── apim/
│
└── environments/
    ├── dev/
    ├── qa/
    ├── uat/
    └── prod/

A module can encapsulate a reusable infrastructure component.

31. Example Terraform Module

Suppose we have an AKS module:

modules/
└── aks/
    ├── main.tf
    ├── variables.tf
    └── outputs.tf

The root configuration can call it:

module "aks" {

  source = "./modules/aks"

  resource_group_name = azurerm_resource_group.oneview.name
  location            = var.location
  node_count          = var.aks_node_count
}

The same module can then be reused by different environments.

32. Multiple Environments

A real enterprise application normally has:

DEV
QA
UAT
PROD

We can structure Terraform as:

environments/
│
├── dev/
│   ├── main.tf
│   └── terraform.tfvars
│
├── qa/
│   ├── main.tf
│   └── terraform.tfvars
│
├── uat/
│   ├── main.tf
│   └── terraform.tfvars
│
└── prod/
    ├── main.tf
    └── terraform.tfvars

For example:

Development

environment    = "dev"
aks_node_count = 2

Production

environment    = "prod"
aks_node_count = 5

The infrastructure module can remain reusable while environment-specific values are separated.

33. Terraform Data Sources

A Terraform resource creates or manages infrastructure.

A data source reads information about existing infrastructure.

For example:

data "azurerm_resource_group" "shared" {
  name = "rg-shared-services"
}

We can then access:

data.azurerm_resource_group.shared.location

This is particularly useful when an enterprise already has shared infrastructure.

For example:

Existing Shared Infrastructure
             |
      +------+------+
      |             |
      v             v
   Shared VNet   Key Vault
      |
      v
Application Terraform

Terraform doesn't necessarily need to create every resource itself.

34. Terraform Drift

Drift occurs when infrastructure changes outside Terraform.

Suppose Terraform defines:

AKS Node Count = 3

But someone manually changes Azure:

AKS Node Count = 5

Now:

Desired State = 3

Actual State = 5

This is infrastructure drift.

Running:

terraform plan

can reveal differences Terraform detects between the configuration/state and the infrastructure.

35. Terraform Import

Suppose an Azure resource already exists:

Azure Portal
     |
     v
Existing Resource

But Terraform doesn't currently manage it.

Terraform supports importing existing resources into Terraform state.

Conceptually:

Existing Azure Resource
          |
          v
    Terraform Import
          |
          v
   Terraform State

After import, you should ensure the Terraform configuration accurately represents the resource.

Modern Terraform also supports declarative import blocks for import workflows.

36. Terraform Lifecycle

Terraform provides lifecycle controls.

For example:

lifecycle {
  prevent_destroy = true
}

This can be useful for resources where accidental destruction would be especially undesirable.

Another lifecycle option is:

lifecycle {
  ignore_changes = [
    tags
  ]
}

Lifecycle rules should be used carefully because they can change Terraform's normal reconciliation behavior.

37. Terraform with Docker

Terraform and Docker solve different problems.

Docker

Docker packages applications into containers.

.NET Application
       |
       v
Docker Build
       |
       v
Container Image

Terraform

Terraform provisions infrastructure.

Terraform
    |
    +--> AKS
    +--> ACR
    +--> VNet
    +--> SQL
    +--> Service Bus

Together:

.NET Application
       |
       v
Docker Image
       |
       v
ACR
       |
       v
AKS
       ^
       |
 Terraform

38. Terraform with Kubernetes

Terraform and Kubernetes YAML also have different responsibilities.

Terraform can provision:

Azure
 |
 +--> VNet
 +--> AKS
 +--> ACR
 +--> SQL
 +--> Service Bus

Kubernetes can manage:

AKS
 |
 +--> Deployment
 +--> Service
 +--> Ingress
 +--> ConfigMap
 +--> Secret
 +--> HPA

Therefore:

Terraform
    |
    v
Infrastructure
    |
    v
AKS
    |
    v
Kubernetes
    |
    v
.NET Microservices

Terraform can also manage Kubernetes resources, but organizations often separate infrastructure provisioning from application deployment for operational clarity.

39. Terraform with Azure DevOps

Terraform becomes especially powerful when integrated with CI/CD.

A typical pipeline looks like:

Developer
    |
    v
Git Commit
    |
    v
Azure DevOps
    |
    +--> terraform fmt
    |
    +--> terraform validate
    |
    +--> terraform plan
    |
    v
Approval
    |
    v
terraform apply
    |
    v
Azure Infrastructure

40. Example Azure DevOps Terraform Pipeline

A simplified example:

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:

- task: TerraformInstaller@1
  inputs:
    terraformVersion: 'latest'

- script: |
    terraform init
  displayName: 'Terraform Init'

- script: |
    terraform fmt -check
  displayName: 'Terraform Format Check'

- script: |
    terraform validate
  displayName: 'Terraform Validate'

- script: |
    terraform plan
  displayName: 'Terraform Plan'

- script: |
    terraform apply -auto-approve
  displayName: 'Terraform Apply'

A production pipeline should additionally consider:

Azure service connections
Secure authentication
Remote state
Plan artifacts
Approval gates
Environment protection
Separate plan/apply stages
Secret management
Policy checks

41. Terraform and Security

Security is extremely important when managing infrastructure.

Never hardcode sensitive credentials:

password = "MyPassword123"

Instead, use mechanisms such as:

Azure Key Vault
Managed Identity
Workload Identity
Azure DevOps secret variables
Secure pipeline variables
Environment variables

For example:

Terraform
    |
    v
Azure Identity
    |
    v
Key Vault
    |
    v
Secrets

42. Sensitive Variables

Terraform supports sensitive variables.

Example:

variable "database_password" {
  type      = string
  sensitive = true
}

This helps prevent the value from being displayed in normal Terraform output.

However, marking a value as sensitive does not automatically remove it from Terraform state. Therefore, state itself must be securely stored and access-controlled.

43. Terraform Workspaces

Terraform supports workspaces.

For example:

terraform workspace new dev

terraform workspace new qa

terraform workspace new prod

Select a workspace:

terraform workspace select dev

Workspaces can be useful in certain scenarios, but for larger enterprise environments, separate environment directories and isolated remote states can sometimes provide clearer isolation.

44. Important Terraform Commands

Command	Description
`terraform init`	Initializes Terraform
`terraform fmt`	Formats Terraform files
`terraform validate`	Validates configuration
`terraform plan`	Shows proposed changes
`terraform apply`	Creates/updates infrastructure
`terraform destroy`	Destroys managed infrastructure
`terraform output`	Displays outputs
`terraform show`	Displays state/plan information
`terraform state list`	Lists resources in state
`terraform state show`	Shows a resource from state
`terraform providers`	Displays configured providers
`terraform workspace list`	Lists workspaces

45. Terraform Destroy

Terraform can also remove infrastructure that it manages.

terraform destroy

Terraform shows the resources it intends to destroy.

For example:

Plan: 0 to add, 0 to change, 10 to destroy.

After confirmation, Terraform removes the managed resources.

This command should be used with extreme care in production.

46. Terraform vs ARM Templates vs Bicep

For Azure architects, this is an important interview topic.

Feature	Terraform	ARM Templates	Bicep
Provider	HashiCorp	Microsoft	Microsoft
Azure	Yes	Yes	Yes
AWS	Yes	No	No
GCP	Yes	No	No
Multi-cloud	Strong	No	No
Syntax	HCL	JSON	Bicep
Modules	Yes	Yes	Yes
Azure Native	No	Yes	Yes
CI/CD	Yes	Yes	Yes

Bicep is an Azure-native Infrastructure as Code language.

Terraform is particularly attractive when an organization wants a common IaC approach across multiple providers or already has a mature Terraform ecosystem.

47. Terraform vs Ansible

Terraform and Ansible are also frequently compared.

Terraform

Primarily focuses on infrastructure provisioning.

Terraform
    |
    +--> Network
    +--> VM
    +--> AKS
    +--> Database
    +--> Storage

Ansible

Primarily focuses on configuration and automation.

Ansible
    |
    +--> Install packages
    +--> Configure servers
    +--> Deploy configuration
    +--> Execute operational tasks

Simplified:

Terraform
    |
    v
Provision Infrastructure

Ansible
    |
    v
Configure / Automate Systems

48. Terraform vs Docker vs Kubernetes

These three technologies solve different problems.

Terraform
    |
    v
Infrastructure Provisioning

Docker
    |
    v
Application Containerization

Kubernetes
    |
    v
Container Orchestration

For a .NET microservices platform:

Terraform
    |
    +--> Azure VNet
    +--> AKS
    +--> ACR
    +--> SQL
    +--> Service Bus
    |
    v
Docker
    |
    v
.NET Container
    |
    v
ACR
    |
    v
Kubernetes / AKS
    |
    v
Running Microservices

49. Complete OneView Terraform Architecture

Let's put everything together.

                         Internet
                            |
                            v
                  Application Gateway
                            |
                            v
                   Azure API Management
                            |
                            v
                           AKS
                            |
          +-----------------+-----------------+
          |                 |                 |
          v                 v                 v
    Forecast API      Workforce API     Capacity API
          |                 |                 |
          +-----------------+-----------------+
                            |
                            v
                    Azure Service Bus
                            |
                            v
                     Hiring Service
                            |
                            v
                       Azure SQL


Supporting Azure Services
-------------------------

        +-----------------------------+
        | Azure Container Registry    |
        | Azure Key Vault             |
        | Application Insights        |
        | Azure Monitor               |
        | Managed Identity             |
        | Virtual Network             |
        +-----------------------------+

                 ^
                 |
              Terraform
                 |
                 v
          Infrastructure as Code

50. End-to-End Terraform Deployment Flow

A complete enterprise deployment can look like:

                     Developer
                         |
                         v
                  Git Repository
                         |
                         v
                  Azure DevOps
                         |
            +------------+------------+
            |                         |
            v                         v
       Terraform                  Application
          Code                       Code
            |                         |
            v                         v
    terraform validate          Docker Build
            |                         |
            v                         v
      terraform plan               ACR
            |                         |
            v                         v
        Approval                    AKS
            |
            v
    terraform apply
            |
            v
      Azure Infrastructure
            |
     +------+------+------+
     |      |      |      |
     v      v      v      v
    VNet   AKS    SQL   Service Bus

51. Enterprise Terraform Repository Structure

A production-style Terraform repository can look like:

terraform-infrastructure/
│
├── modules/
│   │
│   ├── network/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   │
│   ├── aks/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   │
│   ├── acr/
│   ├── sql/
│   ├── servicebus/
│   ├── keyvault/
│   ├── apim/
│   └── monitoring/
│
├── environments/
│   │
│   ├── dev/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── terraform.tfvars
│   │
│   ├── qa/
│   │
│   ├── uat/
│   │
│   └── prod/
│
└── pipelines/
    ├── terraform-plan.yml
    └── terraform-apply.yml

This structure provides:

Reusable Modules
       +
Environment Isolation
       +
CI/CD Automation
       +
Version Control

52. Terraform Best Practices

1. Use Remote State

Use an appropriate remote backend instead of relying on local state for team-managed infrastructure.

2. Use Modules

Create reusable modules for common infrastructure components.

3. Never Hardcode Secrets

Use Key Vault, managed identities, and secure CI/CD mechanisms.

4. Use Git

Store Terraform code in source control.

5. Review Terraform Plans

Use:

terraform plan

before applying important infrastructure changes.

6. Use Environment Isolation

Separate development and production state appropriately.

7. Use Naming Standards

For example:

rg-oneview-dev
rg-oneview-qa
rg-oneview-uat
rg-oneview-prod

8. Use Resource Tags

tags = {
  Application = "OneView"
  Environment = "Production"
  Owner       = "PlatformTeam"
  ManagedBy   = "Terraform"
}

9. Implement CI/CD

Terraform should be integrated with your organization's DevOps process.

10. Protect Production

Production infrastructure should have:

Approval processes
Restricted permissions
Remote state
Secure credentials
Policy validation
Monitoring
Backup/recovery processes

53. Terraform in a .NET Solution Architect's Architecture

From a Solution Architect perspective, Terraform is not responsible for writing the .NET application.

Instead, it manages the infrastructure required to run the application.

For example:

                   .NET Application
                         |
                         v
                    Docker Image
                         |
                         v
                         ACR
                         |
                         v
                        AKS
                         |
       +-----------------+----------------+
       |                 |                |
       v                 v                v
 Forecast           Workforce         Capacity
 Service             Service           Service
       |                 |                |
       +-----------------+----------------+
                         |
                         v
                  Service Bus
                         |
                         v
                   Hiring Service
                         |
                         v
                    Azure SQL


Terraform manages:
-------------------

VNet
AKS
ACR
SQL
Service Bus
Key Vault
APIM
Application Gateway
Monitoring
Identity

This separation is important.

Application code describes what the application does.

Terraform describes what infrastructure the application needs.

54. Real-World Scenario

Suppose a company wants to create a new production environment for OneView.

Without Terraform:

Engineer
   |
   +--> Create Resource Group
   +--> Create VNet
   +--> Create Subnets
   +--> Create ACR
   +--> Create AKS
   +--> Create SQL
   +--> Create Service Bus
   +--> Create Key Vault
   +--> Configure APIM
   +--> Configure Monitoring
   +--> Configure Permissions

With Terraform:

Terraform Repository
        |
        v
terraform plan
        |
        v
Approval
        |
        v
terraform apply
        |
        v
Azure Infrastructure

The infrastructure definition becomes repeatable, reviewable, and version controlled.

55. Terraform Workflow — Summary

The standard Terraform workflow is:

             Write Terraform Code
                      |
                      v
              terraform init
                      |
                      v
             terraform fmt
                      |
                      v
           terraform validate
                      |
                      v
              terraform plan
                      |
                      v
                 Review
                      |
                      v
             terraform apply
                      |
                      v
             Azure Resources
                      |
                      v
             Monitor / Maintain
                      |
                      v
               Code Changes
                      |
                      v
             terraform plan

56. Important Terraform Concepts for Interviews

If you are preparing for a .NET Solution Architect, Azure Architect, or DevOps interview, the following Terraform topics are particularly important:

Terraform Fundamentals

What is Terraform?
What is Infrastructure as Code?
What is HCL?
What is a provider?
What is a resource?
What is a data source?
What is a variable?
What is an output?

State Management

What is Terraform state?
Why is state required?
What is remote state?
What is state locking?
What is state drift?
How do you secure Terraform state?

Modules

What is a Terraform module?
Why use modules?
How do you create reusable modules?
How do you pass variables into modules?
How do modules expose outputs?

Enterprise

Terraform with Azure DevOps
Terraform with AKS
Terraform with ACR
Terraform with Azure SQL
Terraform with Service Bus
Terraform with Key Vault
Terraform with API Management
Terraform with Application Gateway
Terraform with Managed Identity
Terraform with Azure Monitor

57. Key Takeaways

Terraform provides a consistent way to manage cloud infrastructure through code.

The most important concepts are:

Terraform
   |
   +--> Infrastructure as Code
   |
   +--> Providers
   |
   +--> Resources
   |
   +--> Variables
   |
   +--> Data Sources
   |
   +--> State
   |
   +--> Remote Backend
   |
   +--> Modules
   |
   +--> Outputs
   |
   +--> Plan
   |
   +--> Apply
   |
   +--> Destroy

For a modern .NET microservices application:

Terraform
    |
    +--> Azure Infrastructure
             |
             +--> VNet
             +--> AKS
             +--> ACR
             +--> Azure SQL
             +--> Service Bus
             +--> Key Vault
             +--> APIM
             +--> Application Gateway
             +--> Monitoring

The application itself can then be built using:

.NET
  |
  v
Docker
  |
  v
ACR
  |
  v
AKS

Therefore, Terraform plays an important role in creating a repeatable and automated cloud foundation for modern .NET applications.

Conclusion

Terraform is much more than a tool for creating Azure resources.

In an enterprise environment, Terraform can become an important part of the overall Cloud Infrastructure and DevOps strategy.

A typical modern architecture can combine:

.NET
 +
Docker
 +
Kubernetes / AKS
 +
Azure
 +
Terraform
 +
Azure DevOps

where:

.NET develops the business applications
Docker packages applications into containers
AKS orchestrates containers
Azure provides cloud infrastructure and managed services
Terraform provisions and manages infrastructure as code
Azure DevOps automates the CI/CD lifecycle

The result is a repeatable, automated, version-controlled infrastructure platform that can support development through production environments.

Terraform + Azure + .NET + Kubernetes + DevOps is therefore a powerful combination for building and operating modern enterprise applications.

Azure Red Hat OpenShift (ARO) with .NET Core — Complete Guide with Real-Time Example

2026-08-31T13:26:13.185+05:30

Important clarification: Azure Red Hat OpenShift (ARO) is not mandatory for .NET Core/.NET applications. It is an enterprise container platform that can be an excellent choice when an organization wants OpenShift/Kubernetes capabilities with Azure-managed infrastructure. Microsoft describes ARO as a fully managed OpenShift service jointly operated and supported by Microsoft and Red Hat. (Microsoft Learn)

1. What is Azure Red Hat OpenShift?

Azure Red Hat OpenShift (ARO) is Microsoft's managed Red Hat OpenShift service running on Azure.

In simple terms:

.NET Application
       ↓
Docker Container
       ↓
OpenShift
       ↓
Azure Red Hat OpenShift
       ↓
Microsoft Azure

OpenShift extends Kubernetes with additional enterprise capabilities and developer/operator tooling. ARO gives you those OpenShift capabilities without requiring your team to manually operate the underlying OpenShift control plane and infrastructure. Microsoft and Red Hat jointly engineer, operate, and support the service. (Microsoft Learn)

ARO clusters are deployed into your Azure subscription, while Microsoft and Red Hat handle major platform-management responsibilities such as patching and monitoring of the managed cluster components. (Microsoft Learn)

2. Why do we need OpenShift?

Imagine an organization has 50 microservices:

                    API Gateway
                        |
        +---------------+---------------+
        |               |               |
   Order Service   Payment Service   Customer Service
        |               |               |
      SQL DB          SQL DB          Redis
        |
   Notification
      Service

Each service may have:

Different deployments
Different versions
Different resource requirements
Multiple replicas
Health checks
Networking requirements
Secrets
Configuration
Logging
Monitoring
Autoscaling

Managing all of this manually becomes difficult.

A container orchestration platform can manage these workloads.

OpenShift provides Kubernetes-based orchestration plus additional platform capabilities. ARO packages this as a managed Azure service. (Microsoft Learn)

3. Is Azure OpenShift mandatory for .NET Core?

❌ No.

This is one of the most important points.

A .NET application can run on:

IIS
Azure App Service
Azure Container Apps
Azure Kubernetes Service
Azure Red Hat OpenShift
Windows Services
Linux
Docker
Virtual Machines
On-premises servers

For example:

ASP.NET Core Web API
       |
       +----> IIS
       |
       +----> Azure App Service
       |
       +----> Docker
       |
       +----> AKS
       |
       +----> ARO

So .NET does not require OpenShift.

The decision depends on your organization's architecture, operational requirements, existing platform standards, compliance requirements, and team expertise.

4. Why would an enterprise choose ARO?

ARO becomes particularly interesting when an organization wants:

1. Kubernetes

OpenShift is built on Kubernetes.

2. Enterprise OpenShift platform

Organizations that already standardize on Red Hat OpenShift can use ARO while keeping their workloads on Azure.

3. Managed platform

Microsoft and Red Hat manage important parts of the platform, reducing the amount of cluster administration required from application teams. (Microsoft Learn)

4. Containerized microservices

.NET applications can be packaged as containers and deployed as independent workloads.

5. Scaling

Multiple replicas of a service can run simultaneously.

6. Security and identity

ARO supports integration with Microsoft Entra ID and Kubernetes RBAC. (Microsoft Learn)

7. CI/CD

It can be integrated with enterprise CI/CD pipelines.

8. Hybrid-cloud strategy

OpenShift can be useful for organizations that want consistency between OpenShift environments across different infrastructure.

5. ARO Architecture

A simplified architecture looks like this:

                   Internet
                      |
                      ↓
                Azure Front Door
                      |
                      ↓
             Application Gateway
                      |
                      ↓
             Azure Red Hat OpenShift
              ┌───────────────────┐
              │    OpenShift      │
              │     Cluster       │
              │                   │
              │  ┌─────────────┐  │
              │  │   Router    │  │
              │  └──────┬──────┘  │
              │         │         │
              │  ┌──────┴──────┐  │
              │  │   Service   │  │
              │  └──────┬──────┘  │
              │         │         │
              │   ┌─────┴─────┐   │
              │   │   Pods    │   │
              │   │           │   │
              │   │ .NET API  │   │
              │   │ .NET API  │   │
              │   │ .NET API  │   │
              │   └───────────┘   │
              └───────────────────┘
                       |
                Azure SQL Database

ARO provides single-tenant, high-availability OpenShift clusters on Azure. (Microsoft Learn)

6. Important OpenShift Terminology

Before deploying .NET, understand these concepts.

Cluster

The complete OpenShift environment.

ARO Cluster
   |
   +-- Control Plane
   |
   +-- Worker Nodes
   |
   +-- Networking
   |
   +-- Storage
   |
   +-- Operators

Node

A node is a machine that runs workloads.

Worker Node
   |
   +-- Pod
   +-- Pod
   +-- Pod

Pod

A Pod is the basic Kubernetes/OpenShift unit in which containers run.

For a simple .NET API:

Pod
 |
 +-- .NET Container

If you have three replicas:

Pod 1 → .NET API
Pod 2 → .NET API
Pod 3 → .NET API

Deployment

Defines how your application should be deployed.

For example:

replicas: 3

means we want three instances.

Service

Provides stable networking to the Pods.

             Service
                |
       +--------+--------+
       |        |        |
      Pod      Pod      Pod

Route

OpenShift Route exposes an application outside the cluster.

Internet
   |
   ↓
OpenShift Route
   |
   ↓
Service
   |
   ↓
Pods

7. Real-Time .NET Microservices Example

Let's design an e-commerce application.

We have:

E-Commerce
    |
    +-- Product Service
    |
    +-- Order Service
    |
    +-- Payment Service
    |
    +-- Inventory Service
    |
    +-- Notification Service

Suppose the Order Service is built using:

ASP.NET Core Web API
.NET 10
C#
Entity Framework Core
SQL Server
Docker

We want:

                 Client
                   |
                   ↓
              OpenShift
                   |
             Order Service
                   |
        +----------+----------+
        |          |          |
        ↓          ↓          ↓
    Payment    Inventory   Notification

8. Create the .NET API

Create the project:

dotnet new webapi -n OrderService
cd OrderService

Example API:

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllers();

var app = builder.Build();

app.MapControllers();

app.Run();

Controller:

using Microsoft.AspNetCore.Mvc;

namespace OrderService.Controllers;

[ApiController]
[Route("api/orders")]
public class OrdersController : ControllerBase
{
    [HttpGet("{id}")]
    public IActionResult GetOrder(int id)
    {
        return Ok(new
        {
            OrderId = id,
            Product = "Laptop",
            Quantity = 2,
            Status = "Confirmed"
        });
    }
}

Now:

GET /api/orders/1001

returns:

{
  "orderId": 1001,
  "product": "Laptop",
  "quantity": 2,
  "status": "Confirmed"
}

9. Containerize the .NET Application

OpenShift runs containerized workloads, so we can create a Docker image for our API.

Example Dockerfile:

FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build

WORKDIR /src

COPY . .

RUN dotnet restore

RUN dotnet publish -c Release \
    -o /app/publish \
    --no-restore

FROM mcr.microsoft.com/dotnet/aspnet:10.0

WORKDIR /app

COPY --from=build /app/publish .

EXPOSE 8080

ENTRYPOINT ["dotnet", "OrderService.dll"]

The basic flow becomes:

C# Code
   ↓
dotnet publish
   ↓
Docker Image
   ↓
Container Registry
   ↓
OpenShift
   ↓
Pod

10. Build the Docker Image

docker build -t orderservice:1.0 .

Test locally:

docker run -p 8080:8080 orderservice:1.0

Then:

http://localhost:8080/api/orders/1001

11. Push Image to Container Registry

In an enterprise environment, you could use a container registry such as:

Azure Container Registry

Conceptually:

docker tag orderservice:1.0 \
    myregistry.azurecr.io/orderservice:1.0

docker push \
    myregistry.azurecr.io/orderservice:1.0

Now the image is available to OpenShift.

12. Deploy the .NET Application to OpenShift

A Kubernetes/OpenShift Deployment could look like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
spec:
  replicas: 3

  selector:
    matchLabels:
      app: order-service

  template:
    metadata:
      labels:
        app: order-service

    spec:
      containers:
        - name: order-service

          image: myregistry.azurecr.io/orderservice:1.0

          ports:
            - containerPort: 8080

          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"

            limits:
              cpu: "500m"
              memory: "512Mi"

Notice:

replicas: 3

OpenShift will run three Pods.

             Order Service
                   |
       +-----------+-----------+
       |           |           |
       ↓           ↓           ↓
     Pod 1       Pod 2       Pod 3

If one Pod becomes unavailable, the platform can maintain the desired replica state.

13. Create an OpenShift Service

Create:

apiVersion: v1
kind: Service

metadata:
  name: order-service

spec:
  selector:
    app: order-service

  ports:
    - port: 80
      targetPort: 8080

Now:

Service
   |
   +---- Pod 1
   |
   +---- Pod 2
   |
   +---- Pod 3

The application doesn't need to know which Pod receives the request.

14. Expose the API using an OpenShift Route

Example:

apiVersion: route.openshift.io/v1
kind: Route

metadata:
  name: order-service

spec:
  to:
    kind: Service
    name: order-service

  port:
    targetPort: 8080

The resulting flow is:

Client
  |
  | HTTPS
  ↓
OpenShift Route
  |
  ↓
Service
  |
  +--------+--------+
  |        |        |
 Pod 1    Pod 2    Pod 3

15. Deploy Using `oc`

OpenShift provides the oc command-line client.

For example:

oc login <your-cluster>

Create/select a project:

oc new-project ecommerce

Deploy:

oc apply -f deployment.yaml

Service:

oc apply -f service.yaml

Route:

oc apply -f route.yaml

Check Pods:

oc get pods

Example:

NAME                             READY   STATUS
order-service-7c8d9f7c5d-abc12   1/1     Running
order-service-7c8d9f7c5d-def34   1/1     Running
order-service-7c8d9f7c5d-ghi56   1/1     Running

16. What Happens When Traffic Increases?

Suppose:

Normal traffic
   ↓
3 Pods

During a festival sale:

Traffic increases
       ↓
Autoscaling
       ↓
5 Pods
       ↓
10 Pods

Conceptually:

              Service
                 |
       +---------+---------+
       |         |         |
      Pod       Pod       Pod
       |         |         |
       +---------+---------+
                 |
            More traffic
                 ↓
          Additional Pods

This is one of the major advantages of container orchestration.

17. Health Checks

A production .NET application shouldn't simply be considered healthy because its process is running.

ASP.NET Core provides health-check support.

Example:

builder.Services.AddHealthChecks();

var app = builder.Build();

app.MapHealthChecks("/health");

app.MapControllers();

app.Run();

Now:

GET /health

can be used by the platform to determine application health.

18. Configure Kubernetes/OpenShift Probes

For example:

livenessProbe:
  httpGet:
    path: /health
    port: 8080

  initialDelaySeconds: 10
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /health
    port: 8080

  initialDelaySeconds: 5
  periodSeconds: 5

There are two important concepts:

Liveness

"Is my application alive?"

Readiness

"Can my application receive traffic?"

This distinction is extremely important in microservices.

19. Configuration Management

Don't hard-code:

var connectionString =
    "Server=production-server;Database=Orders...";

Instead use configuration:

var connectionString =
    builder.Configuration.GetConnectionString("OrdersDb");

Environment-specific values can then be supplied through OpenShift configuration mechanisms.

For example:

env:
  - name: ASPNETCORE_ENVIRONMENT
    value: Production

  - name: ConnectionStrings__OrdersDb
    valueFrom:
      secretKeyRef:
        name: orders-db-secret
        key: connection-string

20. Secrets

Never put passwords directly into:

Dockerfile
Git repository
deployment.yaml
source code

Instead use a Secret.

Example:

oc create secret generic orders-db-secret \
  --from-literal=connection-string="YOUR-CONNECTION-STRING"

Then the application can consume it through an environment variable.

For enterprise Azure environments, you can also integrate workloads with Azure identity capabilities rather than relying on long-lived credentials. ARO supports managed and workload identities. (Microsoft Learn)

21. Scaling a .NET Microservice

Suppose:

Order Service
Current replicas = 3

You can scale manually:

oc scale deployment/order-service --replicas=5

Now:

Order Service
     |
     +-- Pod 1
     +-- Pod 2
     +-- Pod 3
     +-- Pod 4
     +-- Pod 5

In production, autoscaling can be configured based on resource utilization and other supported metrics.

22. Rolling Deployment

Suppose version 1.0 is running:

Pod 1 → v1
Pod 2 → v1
Pod 3 → v1

You deploy:

v2

The platform can perform a rolling update rather than stopping every instance simultaneously.

Conceptually:

Pod 1 → v2
Pod 2 → v1
Pod 3 → v1

Pod 1 → v2
Pod 2 → v2
Pod 3 → v1

Pod 1 → v2
Pod 2 → v2
Pod 3 → v2

This helps reduce application downtime during deployments.

23. Real-Time E-Commerce Architecture

Now let's put everything together.

                         Customers
                             |
                             ↓
                    Azure Front Door
                             |
                             ↓
                    OpenShift Route
                             |
                             ↓
                      API Gateway
                             |
       +---------------------+---------------------+
       |                     |                     |
       ↓                     ↓                     ↓
 Order Service        Product Service       Customer Service
       |                     |                     |
       ↓                     ↓                     ↓
 Payment Service       Inventory Service      Redis
       |
       ↓
 Notification Service
       |
       ↓
 Azure Service Bus

Each microservice can have:

.NET API
   ↓
Docker Image
   ↓
Container Registry
   ↓
OpenShift Deployment
   ↓
Pods
   ↓
Service
   ↓
Route

24. How .NET and OpenShift Work Together

The relationship is important to understand:

.NET
 ↓
Application Framework

Docker
 ↓
Application Packaging

Kubernetes
 ↓
Container Orchestration

OpenShift
 ↓
Enterprise Kubernetes Platform

Azure Red Hat OpenShift
 ↓
Managed OpenShift on Azure

So ARO isn't replacing .NET.

It provides the platform on which your containerized .NET applications can run.

25. OpenShift vs AKS

This is a very common interview question.

Feature	AKS	Azure Red Hat OpenShift
Technology	Kubernetes	OpenShift/Kubernetes
Azure managed service	Yes	Yes
Microsoft	Managed service	Jointly operated with Red Hat
Red Hat ecosystem	Not central	Strong
OpenShift tooling	No	Yes
Kubernetes workloads	Yes	Yes
.NET support	Yes	Yes
Enterprise OpenShift standard	No	Yes
Best fit	Azure/Kubernetes environments	Organizations standardized on OpenShift

Microsoft explicitly describes ARO as an Azure-managed OpenShift service jointly engineered, operated, and supported by Microsoft and Red Hat. (Microsoft Learn)

26. OpenShift vs Azure App Service

These solve different problems.

Azure App Service

Best when:

I have a web application/API
        ↓
I want managed hosting
        ↓
I don't need Kubernetes

ARO

Better suited when:

I have many containerized services
        ↓
I need Kubernetes/OpenShift
        ↓
I need enterprise container orchestration

Don't introduce OpenShift simply because you are using .NET.

27. OpenShift vs Docker

Another common misunderstanding:

Docker and OpenShift are not competitors in the same sense.

Docker packages the application.

OpenShift orchestrates containerized workloads.

.NET Application
       ↓
Docker Image
       ↓
OpenShift
       ↓
Pods
       ↓
Services
       ↓
Routes

28. Why .NET Applications Work Well on OpenShift

Red Hat provides supported .NET container images and OpenShift guidance for .NET applications. For example, Red Hat's documentation describes dotnet SDK images and corresponding runtime/ASP.NET runtime image streams for OpenShift. (Red Hat Documentation)

That means an enterprise can standardize on:

.NET
+
Red Hat Enterprise Linux/UBI-based containers
+
OpenShift
+
Azure

For example:

.NET 10
   ↓
ASP.NET Core
   ↓
Red Hat UBI-based container
   ↓
OpenShift
   ↓
Azure Red Hat OpenShift

Red Hat's current documentation includes guidance for running .NET 10 on OpenShift and provides dotnet, dotnet-runtime, and dotnet-aspnet image streams. (Red Hat Documentation)

29. CI/CD Pipeline

A typical enterprise pipeline might look like:

Developer
    |
    ↓
Git Repository
    |
    ↓
CI Pipeline
    |
    +-- Build
    +-- Unit Tests
    +-- SonarQube
    +-- Security Scan
    +-- Docker Build
    |
    ↓
Container Registry
    |
    ↓
CD Pipeline
    |
    ↓
ARO
    |
    ↓
OpenShift Deployment
    |
    ↓
Pods

For example:

docker build -t orderservice:1.0 .
docker push myregistry.azurecr.io/orderservice:1.0

oc apply -f deployment.yaml

In a real enterprise pipeline, image tags would normally be immutable version/build identifiers rather than simply latest.

30. What Happens When a Pod Fails?

Suppose:

Order Service

Pod 1 → Running
Pod 2 → Running
Pod 3 → Failed

The desired state says:

replicas = 3

The orchestration layer can create another Pod:

Pod 1 → Running
Pod 2 → Running
Pod 3 → Failed
Pod 4 → Starting

Eventually:

Pod 1 → Running
Pod 2 → Running
Pod 4 → Running

The application therefore doesn't have to manually detect and recreate failed instances.

31. Production Architecture

A mature enterprise architecture could look like:

                    Internet
                       |
                       ↓
                Azure Front Door
                       |
                       ↓
              WAF / Load Balancing
                       |
                       ↓
              Azure Red Hat OpenShift
                       |
        +--------------+--------------+
        |              |              |
        ↓              ↓              ↓
   API Gateway      Services       Workers
        |              |              |
        |        +-----+-----+        |
        |        |     |     |        |
        ↓        ↓     ↓     ↓        ↓
     Orders   Payment Inventory Customer
        |
        ↓
  Azure Service Bus
        |
        ↓
 Notification Worker
        |
        ↓
 External Systems

Supporting services:

Azure SQL
Azure Cache for Redis
Azure Key Vault
Azure Container Registry
Application monitoring
Centralized logging
CI/CD

32. Is ARO Mandatory in Microservices?

Again:

❌ No.

Microservices can run on:

AKS
ARO
Azure Container Apps
Docker Compose
ECS
GKE
Self-managed Kubernetes
Virtual Machines
Other container platforms

The correct architecture decision is:

Business Requirements
        ↓
Non-functional Requirements
        ↓
Platform Requirements
        ↓
Choose Hosting Platform

Not:

.NET
 ↓
Must use ARO ❌

33. When Should You Choose ARO?

Choose ARO when your organization needs things such as:

Enterprise OpenShift standardization

Your company already uses OpenShift across environments.

Kubernetes + OpenShift ecosystem

You need OpenShift-specific platform capabilities and workflows.

Managed Azure deployment

You want OpenShift deployed into Azure while Microsoft and Red Hat handle important platform operations. (Microsoft Learn)

Large microservices platform

You have many containerized services and need centralized orchestration.

Hybrid-cloud consistency

You want an OpenShift-based application platform that can fit into broader hybrid-cloud strategies.

Enterprise security/governance

You need platform-level identity, RBAC, networking, policy, and operational controls.

34. When Should You NOT Choose ARO?

If you have:

One ASP.NET Core API

and your requirement is simply:

Host the API

ARO could be unnecessary complexity.

You might choose:

Azure App Service

instead.

Likewise, if your team already has strong Azure Kubernetes expertise and doesn't need OpenShift, AKS may be a more natural choice.

35. ARO Interview Question

Q: Is Azure Red Hat OpenShift mandatory for .NET Core?

Answer:

No. Azure Red Hat OpenShift is not mandatory for .NET Core applications. .NET applications can run on Azure App Service, Azure Container Apps, AKS, virtual machines, Docker, and other platforms. ARO is a managed OpenShift platform on Azure and is mainly selected when an organization needs enterprise OpenShift/Kubernetes capabilities, OpenShift standardization, hybrid-cloud consistency, or specific operational and governance requirements.

That's the answer I'd recommend giving in a .NET Architect interview.

36. The Most Important Concept

Remember this hierarchy:

             .NET
              │
              │ builds
              ↓
        ASP.NET Core API
              │
              │ packaged as
              ↓
          Docker Image
              │
              │ deployed to
              ↓
          OpenShift
              │
              │ hosted on
              ↓
       Azure Red Hat OpenShift
              │
              ↓
           Azure

ARO is the hosting/orchestration platform—not a requirement of the .NET framework.

37. Final Summary

Azure Red Hat OpenShift combines Azure + Red Hat OpenShift + Kubernetes-based container orchestration + managed platform operations. Microsoft says ARO provides fully managed OpenShift clusters, with Microsoft and Red Hat jointly engineering, operating, and supporting the service. (Microsoft Learn)

For a .NET microservices application:

.NET Microservice
       ↓
Docker Container
       ↓
Container Registry
       ↓
Azure Red Hat OpenShift
       ↓
OpenShift Deployment
       ↓
Pods
       ↓
Service
       ↓
Route
       ↓
Users

The key takeaway is:

Don't choose ARO because you use .NET. Choose ARO when your application's operational, architectural, enterprise, or organizational requirements justify an OpenShift platform.

Official references

Azure Red Hat OpenShift — Microsoft Learn

Azure Red Hat OpenShift documentation

.NET on OpenShift — Red Hat documentation

Create an Azure Red Hat OpenShift cluster

If you're publishing this as a blog, a strong title would be “Azure Red Hat OpenShift (ARO) with .NET 10: Complete Guide to Deploying ASP.NET Core Microservices with Real-Time E-Commerce Example”.

Worker Services in .NET — Complete Guide

2026-08-31T12:59:32.286+05:30

1. What is a Worker Service?

A Worker Service is a .NET application designed to run continuously in the background.

Unlike an ASP.NET Core Web API:

Web API
   ↓
HTTP Request
   ↓
Controller
   ↓
Business Logic
   ↓
HTTP Response

A Worker Service generally works like:

Worker Service
      ↓
Background Process
      ↓
Read Message / Timer / Event
      ↓
Business Logic
      ↓
Database / External API
      ↓
Continue Processing

A Worker Service can run as:

Windows Service
Linux systemd service
Docker container
Kubernetes Pod
Azure Container Apps
Azure App Service background process in appropriate hosting models
VM-hosted process
Kubernetes CronJob for scheduled/batch work

2. Why do we need Worker Services?

Consider an e-commerce application.

A customer places an order:

Customer
   ↓
Order API
   ↓
Create Order
   ↓
Return response

But after creating the order, many things may need to happen:

Order Created
    │
    ├── Send Email
    ├── Generate Invoice
    ├── Update Inventory
    ├── Send Notification
    ├── Create Shipment
    └── Update Analytics

Doing all of these inside the API request can make the API slow.

Instead:

                 ┌───────────────┐
Customer ───────►│   Order API   │
                 └───────┬───────┘
                         │
                         │ OrderCreated
                         ▼
                 ┌───────────────┐
                 │ Message Queue │
                 └───────┬───────┘
                         │
          ┌──────────────┼──────────────┐
          ▼              ▼              ▼
   Email Worker    Inventory Worker  Invoice Worker

This is where Worker Services become extremely useful.

3. Worker Service vs BackgroundService

These two terms are related but not exactly the same.

Worker Service

A Worker Service is a .NET project/application template designed for long-running background workloads.

You can create one using:

dotnet new worker -n OrderProcessing.Worker

It normally contains:

OrderProcessing.Worker
│
├── Program.cs
├── Worker.cs
└── appsettings.json

BackgroundService

BackgroundService is a base class provided by .NET for implementing long-running background work.

For example:

public class Worker : BackgroundService
{
    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            Console.WriteLine("Worker is running...");

            await Task.Delay(
                TimeSpan.FromSeconds(5),
                stoppingToken);
        }
    }
}

So:

Worker Service
      │
      └── Hosting Model
             │
             └── BackgroundService
                    │
                    └── ExecuteAsync()

4. Basic Worker Service Architecture

A typical Worker Service looks like this:

┌─────────────────────────────────────┐
│         .NET Worker Service         │
│                                     │
│  Generic Host                       │
│       │                             │
│       ├── Dependency Injection      │
│       ├── Configuration             │
│       ├── Logging                   │
│       ├── Configuration             │
│       └── Hosted Services           │
│                  │                  │
│                  ▼                  │
│          BackgroundService          │
│                  │                  │
│                  ▼                  │
│           ExecuteAsync()            │
│                  │                  │
│                  ▼                  │
│          Business Processing        │
└─────────────────────────────────────┘

5. Creating a Worker Service

Create the project:

dotnet new worker -n OrderProcessing.Worker

Move into the project:

cd OrderProcessing.Worker

Run it:

dotnet run

The default template will create something similar to:

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;

    public Worker(ILogger<Worker> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            _logger.LogInformation(
                "Worker running at: {time}",
                DateTimeOffset.Now);

            await Task.Delay(
                1000,
                stoppingToken);
        }
    }
}

6. Understanding ExecuteAsync()

This is the most important method.

protected override async Task ExecuteAsync(
    CancellationToken stoppingToken)

The Worker Service calls this method when the application starts.

For example:

Application Starts
       ↓
Host Starts
       ↓
Worker Starts
       ↓
ExecuteAsync()
       ↓
while loop
       ↓
Background processing

7. Why CancellationToken is important

Suppose your worker is running:

Worker
  ↓
Processing
  ↓
Processing
  ↓
Processing

Now Kubernetes sends a termination signal.

The worker should stop gracefully.

That's why we use:

CancellationToken stoppingToken

Example:

while (!stoppingToken.IsCancellationRequested)
{
    await ProcessOrderAsync(stoppingToken);
}

When cancellation occurs:

Kubernetes
     ↓
SIGTERM
     ↓
CancellationToken
     ↓
Worker stops

This is especially important in production microservices.

8. Real-Time Example — Order Processing Microservice

Let's build a realistic architecture.

Imagine an e-commerce system:

Customer
   ↓
Order API
   ↓
Azure Service Bus
   ↓
Order Processing Worker
   ↓
Order Database

The API doesn't need to process everything synchronously.

9. Step 1 — Order API

The API receives:

POST /api/orders

Request:

{
  "customerId": 1001,
  "productId": 5001,
  "quantity": 2
}

The API creates the order.

[ApiController]
[Route("api/orders")]
public class OrdersController : ControllerBase
{
    [HttpPost]
    public async Task<IActionResult> CreateOrder(
        CreateOrderRequest request)
    {
        var order = new Order
        {
            Id = Guid.NewGuid(),
            CustomerId = request.CustomerId,
            ProductId = request.ProductId,
            Quantity = request.Quantity,
            Status = "Pending"
        };

        // Save order

        // Publish OrderCreated event

        return Accepted(order);
    }
}

Notice:

return Accepted(order);

The API doesn't need to wait for the complete background processing.

10. Step 2 — Create an Event

Create:

public class OrderCreatedEvent
{
    public Guid OrderId { get; set; }

    public int CustomerId { get; set; }

    public int ProductId { get; set; }

    public int Quantity { get; set; }
}

The API publishes:

OrderCreatedEvent

to a message broker.

For example:

Azure Service Bus

Architecture:

Order API
    │
    │ OrderCreatedEvent
    ▼
Azure Service Bus
    │
    ▼
Order Processing Worker

11. Step 3 — Create Worker Service

Create:

dotnet new worker -n OrderProcessing.Worker

Install the Azure Service Bus package:

dotnet add package Azure.Messaging.ServiceBus

12. Configure Service Bus

appsettings.json:

{
  "ServiceBus": {
    "ConnectionString": "YOUR_CONNECTION_STRING",
    "QueueName": "orders"
  }
}

In production, don't put secrets directly into appsettings.json.

Use:

Azure Key Vault
Managed Identity
Environment variables
Kubernetes Secrets

13. Create Worker

public class OrderWorker : BackgroundService
{
    private readonly ILogger<OrderWorker> _logger;
    private readonly ServiceBusProcessor _processor;

    public OrderWorker(
        IConfiguration configuration,
        ILogger<OrderWorker> logger)
    {
        _logger = logger;

        var connectionString =
            configuration["ServiceBus:ConnectionString"];

        var queueName =
            configuration["ServiceBus:QueueName"];

        var client = new ServiceBusClient(connectionString);

        _processor = client.CreateProcessor(queueName);
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        _processor.ProcessMessageAsync += ProcessMessage;
        _processor.ProcessErrorAsync += ProcessError;

        await _processor.StartProcessingAsync(
            stoppingToken);

        try
        {
            await Task.Delay(
                Timeout.Infinite,
                stoppingToken);
        }
        catch (OperationCanceledException)
        {
            // Application shutting down
        }

        await _processor.StopProcessingAsync();
    }

    private async Task ProcessMessage(
        ProcessMessageEventArgs args)
    {
        var messageBody =
            args.Message.Body.ToString();

        _logger.LogInformation(
            "Received Order: {Message}",
            messageBody);

        // Process order

        await args.CompleteMessageAsync(args.Message);
    }

    private Task ProcessError(
        ProcessErrorEventArgs args)
    {
        _logger.LogError(
            args.Exception,
            "Error processing message");

        return Task.CompletedTask;
    }
}

14. Register Worker in Program.cs

var builder = Host.CreateApplicationBuilder(args);

builder.Services.AddHostedService<OrderWorker>();

var host = builder.Build();

host.Run();

This is the key line:

builder.Services.AddHostedService<OrderWorker>();

It tells .NET:

Start this Worker when the application starts.

15. Complete Processing Flow

Now the entire flow becomes:

             CUSTOMER
                 │
                 ▼
          ┌──────────────┐
          │   Order API  │
          └──────┬───────┘
                 │
                 │ Save Order
                 ▼
          ┌──────────────┐
          │ Azure SQL DB │
          └──────────────┘
                 │
                 │ Publish Event
                 ▼
        ┌───────────────────┐
        │  Azure Service Bus│
        │      orders       │
        └─────────┬─────────┘
                  │
                  ▼
        ┌─────────────────────┐
        │ Order Worker Service│
        └──────────┬──────────┘
                   │
          ┌────────┼─────────┐
          ▼        ▼         ▼
       Inventory  Payment   Notification

This is a very common microservice architecture.

16. Worker Service with Dependency Injection

Don't put all business logic inside:

Worker.cs

Instead:

Worker
  │
  ▼
OrderProcessor
  │
  ├── OrderRepository
  ├── InventoryService
  ├── PaymentService
  └── NotificationService

Example:

public interface IOrderProcessor
{
    Task ProcessAsync(
        OrderCreatedEvent order,
        CancellationToken cancellationToken);
}

Implementation:

public class OrderProcessor : IOrderProcessor
{
    private readonly ILogger<OrderProcessor> _logger;

    public OrderProcessor(
        ILogger<OrderProcessor> logger)
    {
        _logger = logger;
    }

    public async Task ProcessAsync(
        OrderCreatedEvent order,
        CancellationToken cancellationToken)
    {
        _logger.LogInformation(
            "Processing Order {OrderId}",
            order.OrderId);

        // Business logic

        await Task.CompletedTask;
    }
}

builder.Services.AddScoped<IOrderProcessor, OrderProcessor>();
builder.Services.AddHostedService<OrderWorker>();

17. Important Issue — Scoped Services

This is a very important interview question.

A Worker Service itself is generally effectively singleton-like because it lives for the lifetime of the host.

But services such as:

DbContext
Repository
UnitOfWork

are usually:

Scoped

You shouldn't inject a scoped DbContext directly into a long-lived singleton worker.

Instead use:

IServiceScopeFactory

Example:

public class OrderWorker : BackgroundService
{
    private readonly IServiceScopeFactory _scopeFactory;

    public OrderWorker(
        IServiceScopeFactory scopeFactory)
    {
        _scopeFactory = scopeFactory;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            using var scope =
                _scopeFactory.CreateScope();

            var processor =
                scope.ServiceProvider
                    .GetRequiredService<IOrderProcessor>();

            await processor.ProcessAsync(
                stoppingToken);

            await Task.Delay(
                TimeSpan.FromSeconds(10),
                stoppingToken);
        }
    }
}

This creates a new DI scope for each processing cycle.

18. Worker + Entity Framework Core

For example:

public class OrderProcessor : IOrderProcessor
{
    private readonly ApplicationDbContext _db;

    public OrderProcessor(ApplicationDbContext db)
    {
        _db = db;
    }

    public async Task ProcessAsync(
        CancellationToken cancellationToken)
    {
        var orders = await _db.Orders
            .Where(x => x.Status == "Pending")
            .ToListAsync(cancellationToken);

        foreach (var order in orders)
        {
            order.Status = "Processed";
        }

        await _db.SaveChangesAsync(
            cancellationToken);
    }
}

builder.Services.AddDbContext<ApplicationDbContext>(
    options =>
        options.UseSqlServer(
            builder.Configuration.GetConnectionString(
                "DefaultConnection")));

19. Worker Service for Scheduled Processing

Workers aren't limited to queues.

You can also execute tasks periodically.

Example:

public class ReportWorker : BackgroundService
{
    private readonly ILogger<ReportWorker> _logger;

    public ReportWorker(
        ILogger<ReportWorker> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        using var timer = new PeriodicTimer(
            TimeSpan.FromMinutes(5));

        while (await timer.WaitForNextTickAsync(
            stoppingToken))
        {
            await GenerateReportAsync(
                stoppingToken);
        }
    }

    private async Task GenerateReportAsync(
        CancellationToken cancellationToken)
    {
        _logger.LogInformation(
            "Generating report at {Time}",
            DateTimeOffset.Now);

        await Task.CompletedTask;
    }
}

20. Worker Service for File Processing

Another real-world example:

Customer uploads CSV
       ↓
Blob Storage
       ↓
Queue
       ↓
Worker
       ↓
Read CSV
       ↓
Validate records
       ↓
Insert database
       ↓
Move file to Processed

This is excellent for Worker Services because processing can take several minutes.

21. Worker Service for Email Processing

Architecture:

Application
     │
     ▼
Email Queue
     │
     ▼
Email Worker
     │
     ├── Read message
     ├── Validate
     ├── Send email
     └── Complete message

This prevents email processing from slowing down the main API.

22. Worker Service in Microservices

Suppose your system contains:

Order Service
Payment Service
Inventory Service
Notification Service
Shipping Service

You can have:

Order API
     │
     ▼
Order Worker

Payment Worker

Inventory Worker

Notification Worker

Shipping Worker

Each worker can be independently deployed and scaled.

This follows an important microservice principle:

A microservice should own a specific business capability.

23. Scaling Workers

Suppose there are:

100 messages/hour

One worker might be enough.

But suppose:

100,000 messages/hour

You can run:

             Queue
               │
       ┌───────┼───────┐
       ▼       ▼       ▼
    Worker 1 Worker 2 Worker 3

In Kubernetes:

Deployment
     │
     ├── Pod 1
     ├── Pod 2
     ├── Pod 3
     └── Pod 4

Multiple workers consume messages from the same queue.

The broker distributes messages between consumers.

24. Worker Service + Kubernetes

A Worker can run as a container.

Dockerfile:

FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base
WORKDIR /app

FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /src

COPY . .

RUN dotnet restore

RUN dotnet publish \
    -c Release \
    -o /app/publish

FROM base AS final

WORKDIR /app

COPY --from=build /app/publish .

ENTRYPOINT ["dotnet", "OrderProcessing.Worker.dll"]

Then:

Docker Image
      ↓
Azure Container Registry
      ↓
AKS
      ↓
Worker Pod

25. Worker Deployment in Kubernetes

Example:

apiVersion: apps/v1
kind: Deployment

metadata:
  name: order-worker

spec:
  replicas: 3

  selector:
    matchLabels:
      app: order-worker

  template:
    metadata:
      labels:
        app: order-worker

    spec:
      containers:
        - name: order-worker

          image: myregistry/order-worker:1.0

          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"

            limits:
              cpu: "500m"
              memory: "512Mi"

Now Kubernetes runs:

Order Worker
   │
   ├── Pod 1
   ├── Pod 2
   └── Pod 3

26. Worker Service Failure Handling

Production workers must handle failures.

For example:

Worker
   ↓
Process Order
   ↓
Payment API
   ↓
FAILED

Don't simply crash the entire application.

Use:

Retry
 ↓
Retry
 ↓
Retry
 ↓
Still Failed
 ↓
Dead Letter Queue

For example:

Orders Queue
     │
     ▼
Worker
     │
     ├── Success → Complete
     │
     └── Failure
           │
           ▼
       Retry
           │
           ▼
      Max Retries
           │
           ▼
    Dead Letter Queue

27. Idempotency — Extremely Important

Imagine the worker receives:

OrderId = 1001

It processes it.

But before acknowledgement:

Worker crashes

The message may be delivered again.

Now:

Order 1001
     ↓
Processed
     ↓
Worker crashes
     ↓
Message delivered again
     ↓
Order 1001 processed again

Therefore your worker should be idempotent.

For example:

var alreadyProcessed =
    await _db.ProcessedMessages
        .AnyAsync(
            x => x.MessageId == messageId,
            cancellationToken);

if (alreadyProcessed)
{
    return;
}

Then:

Message
  ↓
Check MessageId
  ↓
Already processed?
  │
  ├── Yes → Skip
  │
  └── No → Process

This is extremely important in distributed systems.

28. Worker + Saga Pattern

Workers are also commonly used with Saga-based microservices.

Example:

Order Created
     ↓
Order Worker
     ↓
Payment
     ↓
Inventory
     ↓
Shipping

If inventory fails:

Order
  ↓
Payment SUCCESS
  ↓
Inventory FAILED
  ↓
Compensation
  ↓
Refund Payment
  ↓
Cancel Order

Workers can process these asynchronous commands/events.

29. Worker Service vs API

Feature	Web API	Worker Service
HTTP endpoint	Yes	No
Long-running process	Not ideal	Excellent
Queue consumer	Possible	Excellent
Scheduled jobs	Possible	Excellent
Background processing	Limited	Excellent
Request/response	Yes	No
Kubernetes	Yes	Yes
Docker	Yes	Yes
Microservices	Yes	Yes

30. Alternatives to Worker Services

Worker Services are not the only solution.

Depending on your requirement, you can use several alternatives.

1. Hangfire

Excellent for:

Background Jobs
Scheduled Jobs
Retries
Recurring Jobs
Dashboard

Architecture:

API
 ↓
Hangfire
 ↓
Background Job
 ↓
Database

Good when you need:

Run every day at 2 AM

or:

Run this job in background

31. Quartz.NET

Quartz.NET is useful for sophisticated scheduling.

Example:

Daily
Hourly
Weekly
Cron
Complex schedules

Architecture:

Application
     ↓
Quartz Scheduler
     ↓
Job
     ↓
Business Logic

Good for complex scheduling requirements.

32. Azure Functions

If you're already using Azure, Azure Functions can be an excellent alternative.

For example:

Service Bus
    ↓
Azure Function
    ↓
Process Order

Or:

Timer
 ↓
Azure Function
 ↓
Generate Report

Advantages:

Serverless
Automatic scaling
Event-driven
Less infrastructure management

33. Azure Service Bus Trigger

A common architecture:

Order API
     ↓
Azure Service Bus
     ↓
Azure Function
     ↓
Process Order

Instead of maintaining a continuously running Worker Service, Azure manages the execution environment.

34. Kubernetes CronJob

For batch jobs that run at a specific time:

Every night 2 AM
       ↓
Kubernetes CronJob
       ↓
Create Pod
       ↓
Execute job
       ↓
Pod completes

This is better than keeping a Worker continuously running for a job that only needs to execute once per day.

35. Azure Logic Apps

For workflow/integration scenarios:

Trigger
  ↓
Logic App
  ↓
Service A
  ↓
Service B
  ↓
Email

Useful for integration workflows rather than complex application business logic.

36. Azure Data Factory

For data movement/ETL:

SQL Server
     ↓
Azure Data Factory
     ↓
Transform
     ↓
Azure SQL

If your requirement is:

ETL
Data migration
Data integration
Scheduled data processing

ADF may be better than a Worker.

37. Azure WebJobs

For applications already hosted in Azure App Service, WebJobs can be useful for background execution.

Architecture:

Azure App Service
       │
       ├── Web API
       │
       └── WebJob

38. Which One Should You Choose?

A simple decision matrix:

Requirement	Recommended
Long-running background process	Worker Service
Queue consumer	Worker Service / Azure Functions
Complex scheduled jobs	Quartz.NET
Simple background jobs	Hangfire
Serverless event processing	Azure Functions
Kubernetes scheduled batch	CronJob
ETL/Data movement	Azure Data Factory
Azure App Service background task	WebJobs
Complex integration workflow	Logic Apps

39. Worker Service vs Azure Function

This is a common interview question.

Worker Service

You manage:
    Application
    Container/VM
    Deployment
    Scaling

Azure Function

Azure manages:
    Infrastructure
    Scaling
    Runtime

Worker:

Queue
 ↓
Worker
 ↓
Process

Function:

Queue
 ↓
Azure Function Trigger
 ↓
Process

40. Worker Service vs Hangfire

Worker Service

Better for:

Continuous processing
Queue consumers
Long-running workloads
Microservice background processes

Hangfire

Better for:

Scheduled jobs
Fire-and-forget jobs
Recurring jobs
Retry management
Job dashboard

41. Worker Service vs Kubernetes CronJob

Use Worker:

Continuous
 ↓
Consume messages
 ↓
Process continuously

Use CronJob:

2 AM
 ↓
Start
 ↓
Process
 ↓
Exit

For example:

Order queue consumer:

Worker Service

Daily database cleanup:

Kubernetes CronJob

42. Recommended Microservice Architecture

For a production e-commerce system, I'd typically consider:

                     ┌──────────────┐
                     │   Angular UI │
                     └──────┬───────┘
                            │
                            ▼
                    ┌───────────────┐
                    │ API Management│
                    └───────┬───────┘
                            │
            ┌───────────────┼────────────────┐
            ▼               ▼                ▼
       Order API       Payment API      Customer API
            │
            ▼
      Azure Service Bus
            │
     ┌──────┼─────────────┐
     ▼      ▼             ▼
 Order    Inventory    Notification
 Worker     Worker        Worker
     │        │             │
     ▼        ▼             ▼
 Azure     Azure          Email/
 SQL       SQL            SMS

And deploy using:

.NET Worker
     ↓
Docker
     ↓
Azure Container Registry
     ↓
AKS
     ↓
Pods
     ↓
Horizontal Scaling

43. Production Best Practices

When implementing Worker Services, remember these:

1. Use CancellationToken

await ProcessAsync(cancellationToken);

2. Use structured logging

_logger.LogInformation(
    "Processing Order {OrderId}",
    orderId);

3. Don't inject scoped dependencies directly

Use:

IServiceScopeFactory

4. Make processing idempotent

Prevent duplicate processing.

5. Implement retry

Transient failures should be retried.

6. Use Dead Letter Queue

Don't retry permanently bad messages forever.

7. Configure health monitoring

Monitor:

Worker status
Message count
Processing latency
Failure count
Retry count
DLQ count

8. Use distributed tracing

For microservices:

API
 ↓
Service Bus
 ↓
Worker
 ↓
Database

You should be able to trace the complete transaction/correlation flow.

44. Most Important Concept

The biggest idea to remember is:

Worker Services are not simply "another type of API." They are long-running background processes designed to execute work independently of incoming HTTP requests.

In microservices, they are particularly valuable for:

          EVENT
            │
            ▼
      Message Broker
            │
            ▼
     Worker Service
            │
     ┌──────┼──────┐
     ▼      ▼      ▼
    DB     API    Events

This allows your APIs to remain:

Fast
Stateless
Scalable
Responsive

while the Worker handles:

Long-running
Asynchronous
Retryable
Queue-based
Scheduled
Background

processing.

Recommended architecture for your .NET/Azure stack

Given a typical .NET + Microservices + Azure + Docker + AKS architecture, a strong production combination is:

                    CLIENT
                      │
                      ▼
                 API Management
                      │
                      ▼
                 .NET Web API
                      │
               Publish Event
                      │
                      ▼
              Azure Service Bus
                      │
          ┌───────────┼───────────┐
          ▼           ▼           ▼
     .NET Worker  .NET Worker  .NET Worker
       Order       Inventory    Notification
          │           │           │
          ▼           ▼           ▼
       Azure SQL   Azure SQL    External API

with:

Docker
  ↓
Azure Container Registry
  ↓
AKS
  ↓
Worker Pods
  ↓
Horizontal Scaling

That gives you a clean event-driven microservice architecture with independent scaling and asynchronous processing.

Working with Windows Services in .NET — Complete Guide from Scratch to Production

2026-08-31T12:40:41.695+05:30

A Windows Service is a long-running application that runs in the background without requiring a user to keep a console window open. In modern .NET, the recommended approach is to build a Worker Service using BackgroundService and configure it to run under the Windows Service infrastructure. Microsoft documents this approach for .NET 8 and later. (Microsoft Learn)

Below is a blog-ready article with a real-time example, architecture, code, deployment, logging, configuration, error handling, recovery, and troubleshooting.

Windows Services in .NET: Complete Guide with Real-Time Example

Introduction

In enterprise applications, we often have requirements where some processing must happen continuously in the background without a user manually starting the application.

For example:

Process pending orders
Generate invoices
Send notification emails
Read files from a folder
Synchronize data between systems
Process messages from a queue
Generate reports
Monitor application health
Clean up temporary files
Synchronize data with a third-party API

A normal console application is not ideal for these scenarios because someone has to manually start it.

A Windows Service solves this problem.

A Windows Service can start automatically when Windows starts and continue running in the background.

Modern .NET provides the Worker Service template and BackgroundService for implementing these applications.

1. What is a Windows Service?

A Windows Service is an application designed to run in the background under the control of the Windows Service Control Manager (SCM).

Unlike a normal desktop application:

Normal Console Application

User
  |
  v
Start Application
  |
  v
Console Window
  |
  v
Application Running

A Windows Service works more like:

Windows Operating System
        |
        v
Service Control Manager
        |
        v
Order Processing Service
        |
        v
Background Worker
        |
        +----> Database
        |
        +----> APIs
        |
        +----> File System
        |
        +----> Message Queue

The Service Control Manager can start, stop and monitor the service.

2. Why Do We Need Windows Services?

Imagine an e-commerce application.

Customers place orders throughout the day.

The application stores orders in SQL Server.

Instead of processing everything inside the Web API request, we can have a background service that periodically checks for pending orders.

Customer
   |
   v
Angular Application
   |
   v
.NET Web API
   |
   v
SQL Server
   |
   | Pending Orders
   v
Windows Service
   |
   +---- Process Order
   |
   +---- Generate Invoice
   |
   +---- Update Status
   |
   +---- Send Notification

This is a common enterprise architecture.

3. Real-Time Example

Let's build a real-world application called:

Order Processing Windows Service

Our requirement is:

Every 30 seconds, the Windows Service should check SQL Server for pending orders and process them.

The workflow will be:

SQL Server
    |
    | Pending Orders
    v
Windows Service
    |
    v
Read Order
    |
    v
Process Order
    |
    v
Generate Invoice
    |
    v
Update Order Status
    |
    v
Log Result

4. Worker Service vs Windows Service

These two terms are often confused.

Worker Service

A Worker Service is a .NET application designed for long-running background processing.

Windows Service

A Windows Service is the way the operating system hosts and manages a background application.

Therefore:

Worker Service
      +
UseWindowsService()
      |
      v
Windows Service

The modern .NET approach is to create a Worker Service and configure it to run as a Windows Service.

Microsoft recommends using the Worker Service template with BackgroundService for this scenario. (Microsoft Learn)

5. Prerequisites

You need:

Windows OS
.NET SDK
Visual Studio or VS Code
SQL Server if using the database example
Administrator privileges for installing the Windows Service

Microsoft's current Windows Service documentation uses .NET 8 or later as the prerequisite baseline. (Microsoft Learn)

6. Create the Worker Service

Using the .NET CLI:

dotnet new worker -n OrderProcessingService

Move into the project:

cd OrderProcessingService

Run the application:

dotnet run

The Worker Service template creates a background worker application.

The template can also be created from Visual Studio by selecting:

Create a new project
        |
        v
Worker Service

7. Project Structure

Our project can look like this:

OrderProcessingService
│
├── Program.cs
├── Worker.cs
├── appsettings.json
│
├── Models
│   └── Order.cs
│
├── Services
│   ├── IOrderProcessor.cs
│   └── OrderProcessor.cs
│
└── Data
    └── OrderRepository.cs

A good architecture separates:

Worker
  |
  v
Business Service
  |
  v
Repository
  |
  v
Database

The Worker should not contain all business logic.

8. Install Windows Service Package

Install:

dotnet add package Microsoft.Extensions.Hosting.WindowsServices

This package provides Windows Service integration for the .NET hosting infrastructure. (NuGet)

9. Understanding BackgroundService

The main class of our Worker Service will inherit from:

BackgroundService

Example:

public class Worker : BackgroundService
{
    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            // Background processing

            await Task.Delay(
                TimeSpan.FromSeconds(30),
                stoppingToken);
        }
    }
}

The important method is:

ExecuteAsync()

This is where the background processing happens.

10. Understanding CancellationToken

A Windows Service must be able to stop gracefully.

Suppose Windows sends a stop command.

We don't want the application to suddenly terminate in the middle of an operation.

Therefore:

CancellationToken stoppingToken

is provided.

We can check:

while (!stoppingToken.IsCancellationRequested)

This means:

Continue processing until Windows tells the service to stop.

11. Program.cs

Now configure our application.

using Microsoft.Extensions.Hosting;

var builder = Host.CreateApplicationBuilder(args);

builder.Services.AddWindowsService(options =>
{
    options.ServiceName = "Order Processing Service";
});

builder.Services.AddHostedService<Worker>();

var host = builder.Build();

host.Run();

The important line is:

builder.Services.AddWindowsService();

This configures the application to work with Windows Service lifetime management. Microsoft's current documentation uses this approach with Host.CreateApplicationBuilder. (Microsoft Learn)

12. Worker.cs

Create our worker:

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;

    public Worker(ILogger<Worker> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        _logger.LogInformation(
            "Order Processing Service started.");

        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                _logger.LogInformation(
                    "Checking for pending orders.");

                await ProcessOrders(stoppingToken);
            }
            catch (Exception ex)
            {
                _logger.LogError(
                    ex,
                    "Error while processing orders.");
            }

            await Task.Delay(
                TimeSpan.FromSeconds(30),
                stoppingToken);
        }

        _logger.LogInformation(
            "Order Processing Service stopped.");
    }

    private async Task ProcessOrders(
        CancellationToken cancellationToken)
    {
        // Order processing logic

        await Task.CompletedTask;
    }
}

13. Why Use Task.Delay?

Suppose we want to execute our process every 30 seconds.

We can use:

await Task.Delay(
    TimeSpan.FromSeconds(30),
    stoppingToken);

The flow becomes:

Service Starts
      |
      v
Process Orders
      |
      v
Wait 30 Seconds
      |
      v
Process Orders
      |
      v
Wait 30 Seconds
      |
      v
Continue...

14. Don't Use Thread.Sleep

Avoid:

Thread.Sleep(30000);

Prefer:

await Task.Delay(
    TimeSpan.FromSeconds(30),
    stoppingToken);

Why?

Thread.Sleep() blocks the thread.

Task.Delay() allows asynchronous waiting and can respond to cancellation.

15. Create Order Model

Create:

Models/Order.cs

namespace OrderProcessingService.Models;

public class Order
{
    public int Id { get; set; }

    public string OrderNumber { get; set; } = string.Empty;

    public decimal Amount { get; set; }

    public string Status { get; set; } = string.Empty;

    public DateTime CreatedDate { get; set; }
}

16. Database Table

Suppose we have this SQL Server table:

CREATE TABLE Orders
(
    Id INT IDENTITY PRIMARY KEY,

    OrderNumber VARCHAR(50) NOT NULL,

    Amount DECIMAL(18,2) NOT NULL,

    Status VARCHAR(20) NOT NULL,

    CreatedDate DATETIME2 NOT NULL
);

Insert sample records:

INSERT INTO Orders
(
    OrderNumber,
    Amount,
    Status,
    CreatedDate
)
VALUES
(
    'ORD1001',
    2500,
    'Pending',
    GETDATE()
);

INSERT INTO Orders
(
    OrderNumber,
    Amount,
    Status,
    CreatedDate
)
VALUES
(
    'ORD1002',
    3500,
    'Pending',
    GETDATE()
);

17. Create Repository

Create:

Data/OrderRepository.cs

For demonstration, we can use ADO.NET.

using Microsoft.Data.SqlClient;
using OrderProcessingService.Models;

public class OrderRepository
{
    private readonly string _connectionString;

    public OrderRepository(string connectionString)
    {
        _connectionString = connectionString;
    }

    public async Task<List<Order>> GetPendingOrdersAsync(
        CancellationToken cancellationToken)
    {
        var orders = new List<Order>();

        using var connection =
            new SqlConnection(_connectionString);

        await connection.OpenAsync(cancellationToken);

        var command = new SqlCommand(
            """
            SELECT TOP 10
                   Id,
                   OrderNumber,
                   Amount,
                   Status,
                   CreatedDate
            FROM Orders
            WHERE Status = 'Pending'
            ORDER BY Id
            """,
            connection);

        using var reader =
            await command.ExecuteReaderAsync(
                cancellationToken);

        while (await reader.ReadAsync(cancellationToken))
        {
            orders.Add(new Order
            {
                Id = reader.GetInt32(0),
                OrderNumber = reader.GetString(1),
                Amount = reader.GetDecimal(2),
                Status = reader.GetString(3),
                CreatedDate = reader.GetDateTime(4)
            });
        }

        return orders;
    }
}

18. Register Repository Using Dependency Injection

Instead of creating the repository manually inside the Worker, use Dependency Injection.

Example:

builder.Services.AddSingleton<OrderRepository>();

However, for production applications, it is usually better to register database-related services according to their actual lifetime and dependency behavior.

For example:

builder.Services.AddScoped<IOrderProcessor, OrderProcessor>();

For scoped services inside a BackgroundService, create an explicit scope using IServiceScopeFactory.

19. Business Service

Create:

Services/IOrderProcessor.cs

public interface IOrderProcessor
{
    Task ProcessAsync(
        CancellationToken cancellationToken);
}

Implementation:

public class OrderProcessor : IOrderProcessor
{
    private readonly OrderRepository _repository;
    private readonly ILogger<OrderProcessor> _logger;

    public OrderProcessor(
        OrderRepository repository,
        ILogger<OrderProcessor> logger)
    {
        _repository = repository;
        _logger = logger;
    }

    public async Task ProcessAsync(
        CancellationToken cancellationToken)
    {
        var orders =
            await _repository.GetPendingOrdersAsync(
                cancellationToken);

        foreach (var order in orders)
        {
            _logger.LogInformation(
                "Processing order {OrderNumber}",
                order.OrderNumber);

            // Business logic

            _logger.LogInformation(
                "Order {OrderNumber} processed successfully.",
                order.OrderNumber);
        }
    }
}

20. Worker with Dependency Injection

Now our Worker becomes cleaner.

public class Worker : BackgroundService
{
    private readonly IServiceScopeFactory _scopeFactory;
    private readonly ILogger<Worker> _logger;

    public Worker(
        IServiceScopeFactory scopeFactory,
        ILogger<Worker> logger)
    {
        _scopeFactory = scopeFactory;
        _logger = logger;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        _logger.LogInformation(
            "Order Processing Service started.");

        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                using var scope =
                    _scopeFactory.CreateScope();

                var processor =
                    scope.ServiceProvider
                        .GetRequiredService<IOrderProcessor>();

                await processor.ProcessAsync(
                    stoppingToken);
            }
            catch (OperationCanceledException)
                when (stoppingToken.IsCancellationRequested)
            {
                break;
            }
            catch (Exception ex)
            {
                _logger.LogError(
                    ex,
                    "Unexpected error.");
            }

            await Task.Delay(
                TimeSpan.FromSeconds(30),
                stoppingToken);
        }

        _logger.LogInformation(
            "Order Processing Service stopped.");
    }
}

This gives us a clean architecture:

Worker
  |
  v
IOrderProcessor
  |
  v
OrderProcessor
  |
  v
OrderRepository
  |
  v
SQL Server

21. appsettings.json

Configuration should not be hard-coded.

Create:

{
  "ConnectionStrings": {
    "DefaultConnection": "Server=localhost;Database=OrderDb;Trusted_Connection=True;TrustServerCertificate=True"
  },

  "WorkerSettings": {
    "IntervalSeconds": 30,
    "BatchSize": 10
  },

  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  }
}

22. Using Configuration

Create:

public class WorkerSettings
{
    public int IntervalSeconds { get; set; }

    public int BatchSize { get; set; }
}

builder.Services.Configure<WorkerSettings>(
    builder.Configuration.GetSection("WorkerSettings"));

Now the interval can be changed without modifying the code.

23. Complete Program.cs

Our final Program.cs can look like:

using Microsoft.Extensions.Hosting;

var builder = Host.CreateApplicationBuilder(args);

builder.Services.AddWindowsService(options =>
{
    options.ServiceName = "Order Processing Service";
});

builder.Services.Configure<WorkerSettings>(
    builder.Configuration.GetSection("WorkerSettings"));

var connectionString =
    builder.Configuration.GetConnectionString(
        "DefaultConnection");

builder.Services.AddSingleton(
    new OrderRepository(connectionString!));

builder.Services.AddScoped<IOrderProcessor, OrderProcessor>();

builder.Services.AddHostedService<Worker>();

var host = builder.Build();

host.Run();

24. Logging

Logging is extremely important for Windows Services because there is no console window when the service is running in production.

We can use:

_logger.LogInformation(
    "Order {OrderNumber} processed.",
    order.OrderNumber);

Other levels include:

_logger.LogDebug("Debug information");

_logger.LogInformation("Information");

_logger.LogWarning("Warning");

_logger.LogError("Error");

_logger.LogCritical("Critical failure");

25. Windows Event Viewer

When running as a Windows Service, logs can be written to the Windows Event Log.

Microsoft's Windows Service hosting integration supports Event Log logging, and UseWindowsService/AddWindowsService configures Windows Service behavior and Event Log integration. (Microsoft Learn)

You can open:

Start
  |
  v
Event Viewer
  |
  v
Windows Logs
  |
  v
Application

Then search for events generated by your application.

26. Important Difference: Console vs Windows Service

During development:

dotnet run

The application runs as a console process.

After installation:

Windows
   |
   v
Service Control Manager
   |
   v
Order Processing Service
   |
   v
Worker

The same application can therefore be useful both locally and as a Windows Service.

27. Publish the Application

Before installing the service, publish the application.

For a Windows x64 deployment:

dotnet publish -c Release -r win-x64 --self-contained true

You can also publish as a single executable.

Microsoft recommends publishing a Worker Service as a single-file executable for Windows Service deployment because it reduces deployment-file complexity. (Microsoft Learn)

Example:

dotnet publish -c Release -r win-x64 --self-contained true /p:PublishSingleFile=true

The published application will be under a path similar to:

bin\Release\net9.0\win-x64\publish\

The exact target framework depends on the .NET version used by your project.

28. Installing the Windows Service

Open:

PowerShell

or:

Command Prompt

as Administrator.

Navigate to the published executable.

Then:

sc.exe create "Order Processing Service" binPath= "C:\Services\OrderProcessingService\OrderProcessingService.exe"

If successful, Windows reports:

[SC] CreateService SUCCESS

Microsoft documents sc.exe create as the native Service Control Manager approach for creating the service. (Microsoft Learn)

29. Start the Service

Run:

sc.exe start "Order Processing Service"

Or open:

Services

Find:

Order Processing Service

Then:

Right Click
    |
    v
Start

30. Service Lifecycle

The complete lifecycle looks like:

Windows Boot
     |
     v
Service Control Manager
     |
     v
Start Service
     |
     v
.NET Host
     |
     v
BackgroundService
     |
     v
ExecuteAsync()
     |
     v
Process Orders
     |
     v
Wait
     |
     +--------+
              |
              v
        Process Again

When Windows stops the service:

Windows
   |
   v
Stop Request
   |
   v
CancellationToken
   |
   v
ExecuteAsync exits
   |
   v
Host shuts down
   |
   v
Service stopped

31. Stop the Service

sc.exe stop "Order Processing Service"

Or use:

Services
  |
  v
Order Processing Service
  |
  v
Stop

32. Delete the Service

If you want to completely remove it:

sc.exe stop "Order Processing Service"

sc.exe delete "Order Processing Service"

Microsoft notes that a service should be stopped before deleting it. (Microsoft Learn)

33. Configure Automatic Startup

A production service normally should start automatically.

Use:

sc.exe config "Order Processing Service" start= auto

Now Windows can start the service automatically during system startup.

34. Configure Service Recovery

One of the biggest advantages of Windows Services is recovery configuration.

Imagine:

Order Processing Service
        |
        v
Unexpected Error
        |
        v
Process Terminates

We want:

Service Failure
      |
      v
Windows Service Manager
      |
      v
Restart Service

Microsoft provides sc.exe failure for configuring service recovery actions. (Microsoft Learn)

Example:

sc.exe failure "Order Processing Service" reset= 86400 actions= restart/60000/restart/60000/run/1000

This can configure restart actions after failures.

35. Why Recovery Is Important

Consider a production server:

2:00 AM
   |
   v
Service crashes
   |
   v
Windows detects failure
   |
   v
Service automatically restarts
   |
   v
Processing continues

Without recovery:

Service crashes
     |
     v
Processing stops
     |
     v
Manual intervention required

With recovery:

Service crashes
     |
     v
Automatic restart
     |
     v
Processing continues

36. Handling Exceptions

Never allow an unexpected exception to bring down the entire worker loop unnecessarily.

Bad:

while (true)
{
    await ProcessOrders();
}

Better:

while (!stoppingToken.IsCancellationRequested)
{
    try
    {
        await ProcessOrders(stoppingToken);
    }
    catch (Exception ex)
    {
        _logger.LogError(
            ex,
            "Error processing orders.");
    }

    await Task.Delay(
        TimeSpan.FromSeconds(30),
        stoppingToken);
}

However, exception handling should be designed carefully. Some failures indicate that the application should stop rather than endlessly retry.

37. Graceful Shutdown

Suppose an order is currently being processed:

Processing Order 1001
        |
        |
Windows Stop Request
        |
        v
CancellationToken

The service should stop accepting new work and allow the current operation to finish when appropriate.

Use:

CancellationToken

throughout the call chain:

Worker
  |
  v
Processor
  |
  v
Repository
  |
  v
Database

For example:

await connection.OpenAsync(
    cancellationToken);

and:

await command.ExecuteReaderAsync(
    cancellationToken);

38. Avoid Long Blocking Operations

Avoid:

Thread.Sleep(...)

Avoid synchronous network calls when asynchronous APIs are available.

Prefer:

await httpClient.GetAsync(
    url,
    cancellationToken);

Prefer asynchronous database operations:

await command.ExecuteNonQueryAsync(
    cancellationToken);

This makes the service more responsive and easier to shut down gracefully.

39. Calling a Third-Party API

Suppose after processing an order we need to notify an external payment service.

Use HttpClient through Dependency Injection.

builder.Services.AddHttpClient(
    "PaymentApi",
    client =>
    {
        client.BaseAddress =
            new Uri("https://api.example.com/");
    });

Then:

public class PaymentService
{
    private readonly IHttpClientFactory _httpClientFactory;

    public PaymentService(
        IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
    }

    public async Task NotifyPaymentAsync(
        int orderId,
        CancellationToken cancellationToken)
    {
        var client =
            _httpClientFactory.CreateClient("PaymentApi");

        await client.PostAsJsonAsync(
            "payments/process",
            new
            {
                OrderId = orderId
            },
            cancellationToken);
    }
}

40. Real Production Architecture

A production implementation could look like:

                 ┌─────────────────────┐
                 │      SQL Server     │
                 │                     │
                 │ Pending Orders      │
                 └──────────┬──────────┘
                            │
                            ▼
                 ┌─────────────────────┐
                 │  Windows Service    │
                 │                     │
                 │ BackgroundService   │
                 └──────────┬──────────┘
                            │
                            ▼
                 ┌─────────────────────┐
                 │ Order Processor     │
                 │                     │
                 │ Business Logic      │
                 └──────────┬──────────┘
                            │
             ┌──────────────┼──────────────┐
             ▼              ▼              ▼
       SQL Server       Payment API    Notification API
             │              │              │
             └──────────────┼──────────────┘
                            ▼
                       Logging
                            │
                            ▼
                     Event Viewer

41. Windows Service vs Web API

These applications solve different problems.

Feature	Web API	Windows Service
User request	Yes	No
HTTP endpoint	Yes	Not required
Long-running background work	Not ideal	Excellent
Runs continuously	Usually hosted continuously	Yes
Trigger	HTTP request	Timer/event/message
UI required	No	No
Windows Service support	Possible	Native scenario
Background processing	Limited/specialized	Excellent

A Web API is usually request-driven.

A Windows Service is usually background-driven.

42. Windows Service vs BackgroundService

They are not exactly the same thing.

BackgroundService
       |
       +---- Console Application
       |
       +---- Windows Service
       |
       +---- Container
       |
       +---- Other Host

BackgroundService is an abstraction for implementing long-running hosted background work.

Windows Service hosting is one way to host that worker.

43. Timer-Based Processing

For periodic work, another pattern is using PeriodicTimer.

Example:

protected override async Task ExecuteAsync(
    CancellationToken stoppingToken)
{
    using var timer =
        new PeriodicTimer(
            TimeSpan.FromSeconds(30));

    while (await timer.WaitForNextTickAsync(
        stoppingToken))
    {
        await ProcessOrders(
            stoppingToken);
    }
}

This can make periodic processing easier to read.

44. Important Production Consideration: Duplicate Processing

Suppose:

Worker
   |
   v
Gets Order 1001
   |
   v
Starts Processing

Before updating the status, the service crashes.

After restart:

Worker
   |
   v
Gets Order 1001 again

Now the same order could be processed twice.

This is a major production concern.

We need an idempotent processing strategy.

45. Use Status Transitions

Instead of:

Pending
  |
  v
Processed

use:

Pending
   |
   v
Processing
   |
   v
Completed

If something fails:

Processing
     |
     v
Failed

Example:

Pending
   |
   v
Processing
   |
   +------> Failed
   |
   v
Completed

This allows us to understand exactly where the order is.

46. Database Transaction

Critical operations can use transactions.

Conceptually:

BEGIN TRANSACTION

Get Pending Order

Change Status = Processing

Perform Database Operations

Change Status = Completed

COMMIT

If an operation fails:

ROLLBACK

However, transactions should not normally be held open across slow external API calls. For distributed workflows, patterns such as idempotency, outbox/inbox, queues, and Saga may be more appropriate.

47. Windows Service + Message Queue

In a larger architecture, instead of polling SQL Server:

SQL Server
   |
   v
Windows Service

we might use:

Web API
   |
   v
Azure Service Bus / RabbitMQ
   |
   v
Windows Service
   |
   v
Order Processor

The Worker waits for messages.

This is often better when work should be processed asynchronously and reliably.

48. Windows Service + Azure Service Bus

For example:

Customer
   |
   v
Web API
   |
   v
Azure Service Bus
   |
   v
Windows Service
   |
   v
Order Processing

The service can consume messages continuously.

Conceptually:

while (!stoppingToken.IsCancellationRequested)
{
    var message =
        await ReceiveMessageAsync(
            stoppingToken);

    await ProcessMessageAsync(
        message,
        stoppingToken);
}

For enterprise systems, this approach can provide better decoupling than repeatedly querying the database.

49. Health Monitoring

Production Windows Services should be monitored.

Useful information includes:

Service Status
Last Successful Processing
Last Failure
Number of Records Processed
Processing Duration
Database Connectivity
External API Availability

For example:

Order Processing Service

Status: Running

Orders Processed: 15,230

Last Successful Run:
2026-08-31 12:15:00

Last Error:
None

Average Processing Time:
1.8 seconds

50. Configuration by Environment

Avoid hard-coding production settings.

Use:

appsettings.json
appsettings.Development.json
appsettings.Production.json

Example:

{
  "WorkerSettings": {
    "IntervalSeconds": 30
  }
}

Development:

{
  "WorkerSettings": {
    "IntervalSeconds": 10
  }
}

Production:

{
  "WorkerSettings": {
    "IntervalSeconds": 60
  }
}

51. Security

Do not store passwords directly inside source code.

Bad:

var connectionString =
    "Server=...;User Id=admin;Password=12345";

Better approaches include:

Windows authentication where appropriate
Environment-specific configuration
Secret management
Azure Key Vault for Azure-hosted workloads
Restricted service accounts

The service should run with only the permissions it actually needs.

Avoid giving unnecessary administrator privileges.

52. Service Account

A Windows Service runs under an account.

Common options include:

Local System
Local Service
Network Service
Custom Service Account

For production applications, use an appropriately restricted service identity rather than automatically granting broad privileges.

The identity should have only the permissions required for:

Database
File System
Network
APIs
Certificates
Logs

53. File Access

If your service processes files:

C:\Input
C:\Output
C:\Archive

make sure the Windows Service account has appropriate permissions.

A common mistake is:

Console Application
     |
     v
Works perfectly

but:

Windows Service
     |
     v
Access Denied

Why?

Because the console application and Windows Service may be running under different user accounts.

54. Current Directory Problem

When running interactively, developers sometimes use:

Directory.GetCurrentDirectory()

But Windows Services may have a different working directory.

Prefer application-relative paths based on:

AppContext.BaseDirectory

Microsoft's Windows Service hosting integration sets the content root appropriately when running as a Windows Service. (Microsoft Learn)

55. Deployment Process

A typical deployment process is:

Developer
   |
   v
Git Repository
   |
   v
CI/CD Pipeline
   |
   v
dotnet build
   |
   v
dotnet test
   |
   v
dotnet publish
   |
   v
Deployment Server
   |
   v
Stop Service
   |
   v
Copy New Version
   |
   v
Start Service
   |
   v
Verify Logs

For enterprise applications, this process can be automated using Azure DevOps or another CI/CD platform.

56. Updating the Service

Suppose version 1 is installed:

OrderProcessingService v1

You release:

OrderProcessingService v2

Typical deployment:

sc.exe stop "Order Processing Service"

Deploy the new files.

Then:

sc.exe start "Order Processing Service"

Always plan deployment carefully if the service is processing critical work.

57. Troubleshooting

Problem 1: Service doesn't start

Check:

Event Viewer

Also verify:

Executable path
Service account permissions
Configuration
Connection strings
Required files
.NET runtime

Problem 2: Service starts and immediately stops

Possible causes:

Unhandled exception
Invalid configuration
Missing dependency
Database connection failure
Invalid executable
Startup exception

Check Event Viewer and application logs.

Problem 3: Works with dotnet run but not as a service

Common reasons:

Different service account
File permission issue
Different working directory
Environment configuration
Missing configuration file
Database authentication
Network permissions

Problem 4: Database connection fails

Check:

SQL Server availability
Connection string
Authentication
Firewall
Service account permissions
Database permissions

Problem 5: Service keeps restarting

Check:

Event Viewer
Application logs
Service recovery configuration
Unhandled exceptions
Memory/CPU issues
External dependency failures

58. Complete Architecture

A mature implementation might look like:

                  ┌─────────────────────┐
                  │ Windows Server      │
                  │                     │
                  │ Service Control     │
                  │ Manager             │
                  └──────────┬──────────┘
                             │
                             ▼
                  ┌─────────────────────┐
                  │ Order Worker        │
                  │                     │
                  │ BackgroundService   │
                  └──────────┬──────────┘
                             │
                             ▼
                  ┌─────────────────────┐
                  │ Order Processor      │
                  │                     │
                  │ Business Rules      │
                  └───────┬───────┬─────┘
                          │       │
                ┌─────────┘       └─────────┐
                ▼                           ▼
        ┌──────────────┐             ┌──────────────┐
        │ SQL Server   │             │ External API │
        └──────────────┘             └──────────────┘
                          │
                          ▼
                    ┌───────────┐
                    │ Event Log │
                    └───────────┘

59. End-to-End Flow

Let's summarize the entire application.

Step 1 — Windows starts

Windows Server

Step 2 — Service Control Manager starts the service

SCM
 |
 v
Order Processing Service

Step 3 — .NET Host starts

Host
 |
 v
Dependency Injection
 |
 v
BackgroundService

Step 4 — Worker starts

Worker.ExecuteAsync()

Step 5 — Worker retrieves orders

SQL Server
 |
 v
Pending Orders

Step 6 — Business logic executes

OrderProcessor

Step 7 — Database gets updated

Pending
   |
   v
Processing
   |
   v
Completed

Step 8 — Logging occurs

Event Log

Step 9 — Worker waits

30 seconds

Step 10 — Processing starts again

Process
  |
  v
Wait
  |
  v
Process
  |
  v
Wait

Step 11 — Windows sends stop signal

CancellationToken

Step 12 — Worker exits gracefully

ExecuteAsync()
     |
     v
Host Shutdown
     |
     v
Service Stopped

60. Complete Worker Example

Here is a simplified final version:

using Microsoft.Extensions.Hosting;

public class Worker : BackgroundService
{
    private readonly IServiceScopeFactory _scopeFactory;
    private readonly ILogger<Worker> _logger;

    public Worker(
        IServiceScopeFactory scopeFactory,
        ILogger<Worker> logger)
    {
        _scopeFactory = scopeFactory;
        _logger = logger;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        _logger.LogInformation(
            "Order Processing Service started.");

        using var timer =
            new PeriodicTimer(
                TimeSpan.FromSeconds(30));

        try
        {
            while (await timer.WaitForNextTickAsync(
                stoppingToken))
            {
                try
                {
                    using var scope =
                        _scopeFactory.CreateScope();

                    var processor =
                        scope.ServiceProvider
                            .GetRequiredService<IOrderProcessor>();

                    await processor.ProcessAsync(
                        stoppingToken);
                }
                catch (OperationCanceledException)
                    when (stoppingToken.IsCancellationRequested)
                {
                    break;
                }
                catch (Exception ex)
                {
                    _logger.LogError(
                        ex,
                        "Error occurred while processing orders.");
                }
            }
        }
        catch (OperationCanceledException)
            when (stoppingToken.IsCancellationRequested)
        {
            // Expected during shutdown.
        }

        _logger.LogInformation(
            "Order Processing Service stopped.");
    }
}

61. Complete Program Example

using Microsoft.Extensions.Hosting;

var builder = Host.CreateApplicationBuilder(args);

builder.Services.AddWindowsService(options =>
{
    options.ServiceName = "Order Processing Service";
});

builder.Services.Configure<WorkerSettings>(
    builder.Configuration.GetSection("WorkerSettings"));

var connectionString =
    builder.Configuration.GetConnectionString(
        "DefaultConnection");

builder.Services.AddSingleton(
    new OrderRepository(connectionString!));

builder.Services.AddScoped<IOrderProcessor, OrderProcessor>();

builder.Services.AddHostedService<Worker>();

var host = builder.Build();

host.Run();

62. Commands Cheat Sheet

Create project

dotnet new worker -n OrderProcessingService

Add Windows Service support

dotnet add package Microsoft.Extensions.Hosting.WindowsServices

Build

dotnet build

Run locally

dotnet run

Publish

dotnet publish -c Release -r win-x64 --self-contained true /p:PublishSingleFile=true

Create service

sc.exe create "Order Processing Service" binPath= "C:\Services\OrderProcessingService\OrderProcessingService.exe"

Configure automatic startup

sc.exe config "Order Processing Service" start= auto

Start

sc.exe start "Order Processing Service"

Stop

sc.exe stop "Order Processing Service"

Delete

sc.exe delete "Order Processing Service"

Check failure configuration

sc.exe qfailure "Order Processing Service"

63. Best Practices

For production Windows Services, follow these practices:

Use BackgroundService.
Use Dependency Injection.
Use asynchronous APIs.
Pass CancellationToken throughout the processing pipeline.
Use structured logging.
Don't hard-code connection strings.
Use secure secret management.
Run using a least-privilege service account.
Configure service recovery.
Make processing idempotent.
Use database transactions where appropriate.
Avoid processing the same record twice.
Monitor service health.
Monitor CPU and memory.
Keep the Worker class thin.
Put business logic into separate services.
Use configuration for intervals and batch sizes.
Test the application as both a console application and Windows Service.
Automate deployment through CI/CD where practical.
Use a message broker when continuous database polling is no longer appropriate.

64. When Should You Use a Windows Service?

Windows Services are particularly useful when:

Continuous Background Processing
          +
Windows Server Environment
          +
No User Interaction Required

Examples:

File Processing

Input Folder
     |
     v
Windows Service
     |
     v
Validate File
     |
     v
Process File
     |
     v
Archive File

Order Processing

Database
   |
   v
Windows Service
   |
   v
Process Orders

Data Synchronization

System A
   |
   v
Windows Service
   |
   v
System B

Scheduled Reports

Windows Service
      |
      v
Generate Report
      |
      v
Save PDF
      |
      v
Send Notification

65. When Should You NOT Use a Windows Service?

A Windows Service may not be the best choice when:

The workload is already event-driven through a managed cloud messaging platform.
You need massive horizontal scaling.
The workload is better suited to serverless functions.
The application must expose HTTP endpoints as its primary responsibility.
The environment is Linux-only.
A managed cloud service can perform the same task more reliably.

For cloud-native systems, alternatives can include:

Azure Functions
Azure Container Apps
AKS
Azure Service Bus
Cloud-hosted Worker Services

The right choice depends on the workload and operational requirements.

66. Interview Questions

Q1. What is a Windows Service?

A Windows Service is a background application managed by the Windows Service Control Manager.

Q2. What is BackgroundService?

BackgroundService is a .NET base class used to implement long-running background tasks.

Q3. How do you convert a Worker Service into a Windows Service?

Install:

Microsoft.Extensions.Hosting.WindowsServices

and configure:

builder.Services.AddWindowsService();

Q4. What is ExecuteAsync?

It is the main asynchronous method where the background work is implemented.

Q5. Why use CancellationToken?

It allows the worker to respond to shutdown requests gracefully.

Q6. How do you install a Windows Service?

Using:

sc.exe create

Q7. How do you start it?

sc.exe start

Q8. How do you stop it?

sc.exe stop

Q9. How do you remove it?

sc.exe delete

Q10. Where can you check Windows Service errors?

Use:

Event Viewer
→ Windows Logs
→ Application

Q11. How do you automatically restart a failed service?

Configure Windows Service recovery actions using Service Control Manager settings or sc.exe failure.

Q12. How do you avoid duplicate processing?

Use techniques such as:

Idempotency
Status transitions
Database constraints
Transactions
Outbox/Inbox patterns
Message deduplication

depending on the architecture.

67. Final Takeaway

A modern .NET Windows Service is not simply a program containing an infinite loop.

A production-quality implementation should have:

                 Windows Service
                       |
                       v
                BackgroundService
                       |
                       v
                Dependency Injection
                       |
                       v
                 Business Service
                       |
             +---------+---------+
             |                   |
             v                   v
         Database            External API
             |                   |
             +---------+---------+
                       |
                       v
                    Logging
                       |
                       v
                 Monitoring

The most important concepts to remember are:

Worker Service
      ↓
BackgroundService
      ↓
ExecuteAsync()
      ↓
CancellationToken
      ↓
Dependency Injection
      ↓
Business Processing
      ↓
Logging
      ↓
Publish
      ↓
Windows Service
      ↓
Service Control Manager
      ↓
Recovery + Monitoring

This architecture provides a clean foundation for building background processing applications such as order processors, file processors, synchronization services, scheduled jobs, notification services, and enterprise integration services.

Conclusion

Windows Services continue to be useful for long-running background workloads on Windows servers.

With modern .NET, the preferred approach is to build the application using the Worker Service/BackgroundService model and then integrate it with Windows Service hosting.

The important distinction is:

BackgroundService implements the background work, while Windows Service hosting allows Windows to manage the application's lifecycle.

Once this foundation is understood, the same Worker Service concepts can be extended to SQL Server processing, REST APIs, Azure Service Bus, RabbitMQ, file processing, scheduled jobs, monitoring, and other enterprise workloads.

Official Microsoft references

If you want, I can next turn this into a professional blog thumbnail + architecture diagram, or convert the complete article into Telugu.

Service Life Cycle in .NET Core Application

2026-08-21T09:49:56.147+05:30

Transient vs Scoped vs Singleton with Real-Time E-Commerce Example

Dependency Injection (DI) is one of the most important concepts in ASP.NET Core. It helps us create loosely coupled, maintainable, testable, and scalable applications.

One of the most frequently asked .NET interview questions is:

What is Service Lifetime in .NET Core? Explain Transient, Scoped, and Singleton with a real-time example.

To understand this properly, we need to understand what happens to a service from the moment it is requested until the moment it is destroyed.

1. What is Service Life Cycle?

A Service Life Cycle defines:

When an object is created
How long that object lives
Whether the same object is reused
When the object is destroyed

In ASP.NET Core, the built-in Dependency Injection container provides three primary service lifetimes:

Transient
Scoped
Singleton

The basic idea is:

Application Starts
       |
       v
DI Container Created
       |
       v
Request Comes
       |
       v
Controller / Service Requests Dependency
       |
       v
DI Container Checks Service Lifetime
       |
       +-------------------+
       |                   |
   Transient           Scoped
       |                   |
New instance          One per Request
       |                   |
       +---------+---------+
                 |
              Singleton
                 |
          One per Application

2. Dependency Injection in ASP.NET Core

Suppose we have an e-commerce application.

A request comes to:

GET /api/orders/1001

The request reaches the controller:

public class OrdersController : ControllerBase
{
    private readonly IOrderService _orderService;

    public OrdersController(IOrderService orderService)
    {
        _orderService = orderService;
    }
}

ASP.NET Core needs to create IOrderService.

It looks into the DI container and asks:

How is IOrderService registered?
What is its lifetime?
Should I create a new instance?
Can I reuse an existing instance?

For example:

builder.Services.AddScoped<IOrderService, OrderService>();

This tells ASP.NET Core:

Create one OrderService instance for each HTTP request and reuse it throughout that request.

3. The Three Service Lifetimes

The three important registrations are:

services.AddTransient<IService, Service>();

services.AddScoped<IService, Service>();

services.AddSingleton<IService, Service>();

Their behavior is different.

Lifetime	Instance Creation	Lifetime
Transient	New instance every time requested	Very short
Scoped	One instance per HTTP request	Request lifetime
Singleton	One instance for application	Application lifetime

Let's understand each one with a real-time example.

4. Transient Lifetime

Transient means:

A new object is created every time the service is requested from the DI container.

Registration:

builder.Services.AddTransient<IEmailService, EmailService>();

Suppose:

public interface IEmailService
{
    void SendEmail();
}

Implementation:

public class EmailService : IEmailService
{
    public EmailService()
    {
        Console.WriteLine("EmailService Created");
    }

    public void SendEmail()
    {
        Console.WriteLine("Email Sent");
    }
}

Now imagine:

public class OrderService
{
    private readonly IEmailService _emailService;

    public OrderService(IEmailService emailService)
    {
        _emailService = emailService;
    }
}

Another service also requests:

public class NotificationService
{
    private readonly IEmailService _emailService;

    public NotificationService(IEmailService emailService)
    {
        _emailService = emailService;
    }
}

Because EmailService is transient:

OrderService
     |
     +---- EmailService Instance #1

NotificationService
     |
     +---- EmailService Instance #2

Two different objects are created.

5. Real-Time Use Cases for Transient

Transient is suitable for lightweight, stateless services.

Examples:

Email Formatter
Message Formatter
DTO Mapper
Validation Service
Small Calculation Service
Data Transformation Service

For example:

builder.Services.AddTransient<IPriceCalculator, PriceCalculator>();

Every time IPriceCalculator is requested, a new instance can be created.

6. Scoped Lifetime

Scoped is probably the most commonly used lifetime in ASP.NET Core Web API applications.

Registration:

builder.Services.AddScoped<IOrderService, OrderService>();

Scoped means:

One instance is created for a particular scope. In ASP.NET Core Web API, the scope normally corresponds to one HTTP request.

Consider:

POST /api/orders

The request enters ASP.NET Core.

ASP.NET Core creates a request scope.

HTTP Request
     |
     v
Request Scope Created
     |
     +---- OrderService
     |
     +---- CustomerService
     |
     +---- Repository
     |
     +---- DbContext
     |
     v
Request Completed
     |
     v
Request Scope Destroyed

All scoped services requested during that request can share their scoped instance.

7. Real-Time Example – E-Commerce Order

Consider an e-commerce application.

The request:

POST /api/orders

requires:

OrdersController
       |
       v
OrderService
       |
       v
OrderRepository
       |
       v
ApplicationDbContext

Registration:

builder.Services.AddScoped<IOrderService, OrderService>();
builder.Services.AddScoped<IOrderRepository, OrderRepository>();
builder.Services.AddScoped<ApplicationDbContext>();

During Request #1:

Request #1

OrderService       ---> Instance A
OrderRepository    ---> Instance A
DbContext          ---> Instance A

Another request arrives:

POST /api/orders

Request #2 gets different scoped objects:

Request #2

OrderService       ---> Instance B
OrderRepository    ---> Instance B
DbContext           ---> Instance B

Therefore:

Request #1
   |
   +-- OrderService A
   +-- Repository A
   +-- DbContext A


Request #2
   |
   +-- OrderService B
   +-- Repository B
   +-- DbContext B

This isolation is one of the major reasons why DbContext is normally registered as Scoped.

8. Why DbContext is Usually Scoped

Consider an order transaction:

Create Order
     |
     +-- Insert Order
     |
     +-- Insert Order Items
     |
     +-- Update Inventory
     |
     +-- SaveChanges()

The same DbContext can track entities involved in that request.

For example:

public class OrderService
{
    private readonly ApplicationDbContext _context;

    public OrderService(ApplicationDbContext context)
    {
        _context = context;
    }

    public async Task CreateOrder(Order order)
    {
        _context.Orders.Add(order);

        await _context.SaveChangesAsync();
    }
}

Using a scoped context helps keep the database unit of work associated with the request.

9. Singleton Lifetime

Singleton means:

Only one instance of the service is created for the application's DI container lifetime, and that instance is reused wherever the service is requested.

Registration:

builder.Services.AddSingleton<IApplicationConfiguration, ApplicationConfiguration>();

Conceptually:

Application Starts
       |
       v
Singleton Instance Created
       |
       +-----------------------+
       |                       |
Request 1                  Request 2
       |                       |
       +----------+------------+
                  |
          Same Singleton

The same object can be reused across requests.

10. Real-Time Singleton Example

Suppose our application has application-wide configuration.

public interface IApplicationConfiguration
{
    string ApplicationName { get; }
}

Implementation:

public class ApplicationConfiguration : IApplicationConfiguration
{
    public string ApplicationName => "E-Commerce API";
}

Registration:

builder.Services.AddSingleton<
    IApplicationConfiguration,
    ApplicationConfiguration>();

Now multiple requests can use the same instance.

Request 1 ----+
              |
Request 2 ----+----> ApplicationConfiguration
              |
Request 3 ----+

11. Real-Time Example – Product Cache

Suppose an application frequently reads product categories.

Instead of querying the database repeatedly, we may maintain an application-level cache.

Conceptually:

public class ProductCache
{
    private readonly Dictionary<int, string> _products
        = new();

    public void Add(int id, string name)
    {
        _products[id] = name;
    }

    public string? Get(int id)
    {
        return _products.TryGetValue(id, out var name)
            ? name
            : null;
    }
}

Registration:

builder.Services.AddSingleton<ProductCache>();

The same cache object can be shared by requests.

However, when using mutable shared state, thread safety must be considered carefully. A singleton should not simply contain an ordinary mutable collection and assume concurrent requests are safe.

12. Complete E-Commerce Example

Let's build a simplified dependency chain.

OrdersController
       |
       v
OrderService
       |
       +---- OrderRepository
       |
       +---- PaymentService
       |
       +---- EmailService
       |
       +---- ApplicationCache
       |
       v
ApplicationDbContext

Possible registrations:

builder.Services.AddControllers();

builder.Services.AddScoped<IOrderService, OrderService>();

builder.Services.AddScoped<IOrderRepository, OrderRepository>();

builder.Services.AddScoped<IPaymentService, PaymentService>();

builder.Services.AddTransient<IEmailService, EmailService>();

builder.Services.AddSingleton<ProductCache>();

builder.Services.AddDbContext<ApplicationDbContext>();

Now each service has an appropriate lifetime based on its responsibilities.

13. What Happens During an HTTP Request?

Let's follow a request step-by-step.

Request:

POST /api/orders

Step 1 – Request Arrives

The client sends:

POST /api/orders

ASP.NET Core receives the request.

Step 2 – Request Scope Is Created

ASP.NET Core creates a scope for the HTTP request.

Request Scope
     |
     +-------------------------+
     |                         |
     v                         v
Scoped Services          Other Dependencies

Step 3 – Controller Is Created

ASP.NET Core needs:

OrdersController

Its constructor requires:

IOrderService

The DI container resolves it.

Step 4 – Scoped OrderService Is Created

Because it is registered as:

AddScoped<IOrderService, OrderService>()

ASP.NET Core creates one instance for this request.

Step 5 – Dependencies Are Resolved

Suppose OrderService requires:

IOrderRepository
IPaymentService
IEmailService

The DI container resolves them.

OrderService
     |
     +-- OrderRepository    Scoped
     |
     +-- PaymentService     Scoped
     |
     +-- EmailService       Transient

Step 6 – DbContext Is Resolved

OrderRepository requires:

ApplicationDbContext

Because DbContext is scoped, the request receives its scoped instance.

OrderService
      |
      v
OrderRepository
      |
      v
ApplicationDbContext

Step 7 – Singleton Is Resolved

Suppose ProductCache is needed.

The DI container checks:

Is ProductCache already created?

If yes:

Use existing instance

If no:

Create singleton instance

Step 8 – Request Executes

The order is processed.

Validate Order
      |
      v
Check Product
      |
      v
Create Order
      |
      v
Process Payment
      |
      v
Save Database Changes
      |
      v
Send Notification

Step 9 – Request Completes

The HTTP request ends.

ASP.NET Core disposes the request scope.

Scoped and transient disposable services associated with that scope are disposed according to the DI container's lifetime management.

The singleton remains alive while its container remains alive.

14. Visualizing All Three Lifetimes

Imagine three HTTP requests.

                  APPLICATION
                      |
             Singleton Instance
                      |
          +-----------+-----------+
          |           |           |
       Request 1   Request 2   Request 3
          |           |           |
       Scoped A    Scoped B    Scoped C
          |           |           |
       Transient   Transient   Transient
          A           B           C

The important point is:

Singleton
   |
   +-- Same instance across requests

Scoped
   |
   +-- Same instance within one request
   +-- Different instance for another request

Transient
   |
   +-- New instance whenever resolved

15. Important Difference: Scoped vs Transient

This is a common interview question.

Suppose:

builder.Services.AddScoped<IService, MyService>();

and the same service is requested twice within one scope.

Conceptually:

Request
  |
  +-- Resolve IService ---> Instance A
  |
  +-- Resolve IService ---> Instance A

Same instance.

With:

builder.Services.AddTransient<IService, MyService>();

you can get:

Request
  |
  +-- Resolve IService ---> Instance A
  |
  +-- Resolve IService ---> Instance B

Different instances.

16. Important Difference: Scoped vs Singleton

Scoped:

Request 1 ---> Instance A
Request 2 ---> Instance B
Request 3 ---> Instance C

Singleton:

Request 1 ---+
Request 2 ---+---> Same Instance A
Request 3 ---+

Therefore, singleton services should be designed very carefully because multiple requests can access the same instance concurrently.

17. Can Singleton Depend on Scoped Service?

This is an important interview question.

Suppose:

builder.Services.AddSingleton<MySingleton>();
builder.Services.AddScoped<MyScoped>();

and:

public class MySingleton
{
    private readonly MyScoped _scoped;

    public MySingleton(MyScoped scoped)
    {
        _scoped = scoped;
    }
}

This creates a lifetime mismatch.

A singleton lives much longer than a scoped service.

Conceptually:

Singleton
     |
     v
Scoped Service

The scoped service cannot naturally live for the entire lifetime of the singleton.

ASP.NET Core's DI validation can detect such captive dependency problems in appropriate environments/configurations.

General rule:

Singleton
   ↓
Should not directly depend on
   ↓
Scoped

18. Can Scoped Depend on Singleton?

Yes.

For example:

Scoped OrderService
        |
        v
Singleton ApplicationConfiguration

This is generally valid because the singleton has a lifetime longer than the scoped service.

19. Can Transient Depend on Scoped?

Within a valid request scope, yes.

For example:

Scoped OrderService
        |
        v
Transient EmailFormatter

The transient object is created for the resolution and can use dependencies available in that scope.

20. Can Singleton Depend on Transient?

This requires careful consideration.

Technically, a singleton can resolve a transient dependency during its construction, but that transient instance then effectively becomes held by the singleton for as long as the singleton holds it.

Therefore, the dependency's effective lifetime can become much longer than intended.

This is sometimes called a captive dependency.

So don't choose lifetimes merely because the DI container allows the registration.

Choose them based on the object's state and responsibilities.

21. Service Lifetime and Thread Safety

This is especially important for Singleton services.

Imagine:

public class CounterService
{
    private int _count;

    public void Increment()
    {
        _count++;
    }
}

If registered as:

builder.Services.AddSingleton<CounterService>();

multiple requests may access the same object concurrently.

Therefore:

Request A ----+
              |
Request B ----+----> Same Singleton
              |
Request C ----+

The implementation must be safe for concurrent access.

For shared mutable state, use appropriate thread-safe techniques or concurrency-safe collections where necessary.

22. Service Lifetime in Microservices

In a microservices architecture, each microservice normally has its own DI container and process/application lifetime.

For example:

Order Service
     |
     +-- Singleton
     +-- Scoped
     +-- Transient

Payment Service
     |
     +-- Singleton
     +-- Scoped
     +-- Transient

Inventory Service
     |
     +-- Singleton
     +-- Scoped
     +-- Transient

A singleton in the Order Service is not automatically shared with the Payment Service.

Each application has its own process/container and its own DI registrations.

23. Service Lifetime vs Database Lifetime

Don't confuse these concepts.

Service lifetime:

Transient
Scoped
Singleton

Database connection lifetime is a separate concern.

For example:

Web API
   |
   v
DbContext
   |
   v
Database Provider
   |
   v
Database

DbContext is normally scoped, while the underlying database connection management is handled by the database provider and connection pooling mechanisms.

24. Common Real-Time Registration

A typical ASP.NET Core application might contain:

builder.Services.AddControllers();

builder.Services.AddDbContext<ApplicationDbContext>();

builder.Services.AddScoped<IOrderService, OrderService>();
builder.Services.AddScoped<IOrderRepository, OrderRepository>();

builder.Services.AddScoped<IPaymentService, PaymentService>();

builder.Services.AddTransient<IEmailTemplateService, EmailTemplateService>();

builder.Services.AddSingleton<IApplicationSettings, ApplicationSettings>();

A reasonable conceptual mapping is:

Service	Typical Lifetime	Reason
DbContext	Scoped	Request/unit-of-work oriented
Repository	Scoped	Works with DbContext
Business Service	Scoped	Request-level operation
Stateless Formatter	Transient	Lightweight/stateless
Application Configuration	Singleton	Shared application-level data
In-memory Cache	Singleton	Shared cache, if designed safely

These are common patterns, not absolute rules.

25. Service Life Cycle and IDisposable

Another important concept is disposal.

Suppose:

public class FileService : IDisposable
{
    public void Dispose()
    {
        Console.WriteLine("Disposed");
    }
}

If the DI container creates and owns a disposable service, it generally manages its disposal according to the service lifetime and scope.

For example, a scoped disposable service is normally disposed when its request scope ends.

A singleton disposable service is normally disposed when the application's DI container is disposed.

This is one reason you should generally let the DI container manage dependencies that it creates rather than manually disposing injected dependencies.

26. Service Lifetime and Background Services

A common mistake is trying to inject a scoped service directly into a long-running BackgroundService.

For example:

public class OrderBackgroundService : BackgroundService
{
    private readonly ApplicationDbContext _context;

    public OrderBackgroundService(ApplicationDbContext context)
    {
        _context = context;
    }
}

This is problematic because BackgroundService is effectively long-lived, while DbContext is scoped.

A better approach is to create a scope when processing each unit of work:

public class OrderBackgroundService : BackgroundService
{
    private readonly IServiceScopeFactory _scopeFactory;

    public OrderBackgroundService(IServiceScopeFactory scopeFactory)
    {
        _scopeFactory = scopeFactory;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        using var scope = _scopeFactory.CreateScope();

        var dbContext =
            scope.ServiceProvider
                 .GetRequiredService<ApplicationDbContext>();

        // Process work
    }
}

For repeated background work, create and dispose an appropriate scope for each unit of work rather than keeping one scoped dependency forever.

27. How DI Container Resolves a Service

Suppose:

builder.Services.AddScoped<IOrderService, OrderService>();

and:

public class OrderService : IOrderService
{
    private readonly IOrderRepository _repository;

    public OrderService(IOrderRepository repository)
    {
        _repository = repository;
    }
}

When the controller requests IOrderService, the DI container performs roughly this process:

1. Controller requests IOrderService
             |
             v
2. DI Container checks registration
             |
             v
3. Finds OrderService
             |
             v
4. Checks OrderService lifetime
             |
             v
5. Scoped
             |
             v
6. Checks current scope
             |
             v
7. Creates OrderService if not already created
             |
             v
8. Sees IOrderRepository dependency
             |
             v
9. Resolves IOrderRepository
             |
             v
10. Creates OrderService
             |
             v
11. Injects dependencies
             |
             v
12. Controller receives OrderService

This is the essence of Dependency Injection.

28. Real-Time Request Example

Let's assume:

Customer places an order

Request:

POST /api/orders

Flow:

Client
  |
  | HTTP Request
  v
ASP.NET Core
  |
  v
Middleware Pipeline
  |
  v
Request Scope
  |
  v
OrdersController
  |
  v
OrderService
  |
  +---- OrderRepository
  |          |
  |          v
  |      DbContext
  |
  +---- PaymentService
  |
  +---- EmailService
  |
  +---- ProductCache
  |
  v
Database

Lifetimes might be:

OrderService        -> Scoped
OrderRepository     -> Scoped
DbContext           -> Scoped
PaymentService      -> Scoped
EmailService        -> Transient
ProductCache        -> Singleton

At the end:

HTTP Request Ends
       |
       v
Request Scope Disposed
       |
       +-- Scoped services disposed
       +-- Request-owned transient disposables disposed
       |
       v
Singleton remains available

29. Common Interview Questions

Q1. What are the three service lifetimes?

Answer:

Transient
Scoped
Singleton

Q2. What is Transient?

A new service instance is created each time the service is requested from the DI container.

services.AddTransient<IEmailService, EmailService>();

Q3. What is Scoped?

One service instance is generally created per scope. In ASP.NET Core Web API, this normally means one instance per HTTP request.

services.AddScoped<IOrderService, OrderService>();

Q4. What is Singleton?

One service instance is reused for the lifetime of the DI container/application.

services.AddSingleton<ICache, Cache>();

Q5. Which lifetime is normally used for DbContext?

DbContext is normally registered as Scoped in ASP.NET Core applications.

Q6. Why shouldn't DbContext normally be Singleton?

Because DbContext is designed around a unit-of-work pattern and is not intended to be shared concurrently across unrelated requests.

Q7. Which lifetime is best for stateless lightweight services?

Often Transient, although Scoped can also be appropriate depending on the service's dependencies and design.

Q8. Which lifetime is best for shared application-wide state?

A Singleton can be appropriate, but shared mutable state must be designed for concurrent access and application lifetime.

Q9. Can Singleton depend on Scoped?

Generally, no direct dependency should be created, because it creates a lifetime mismatch/captive dependency.

Q10. Can Scoped depend on Singleton?

Yes. This is generally valid.

30. Common Mistakes

Mistake 1 – Making Everything Singleton

Avoid:

services.AddSingleton<OrderService>();
services.AddSingleton<OrderRepository>();
services.AddSingleton<ApplicationDbContext>();

This can create serious lifetime and concurrency problems.

Mistake 2 – Making Everything Transient

Using transient everywhere can cause unnecessary object creation and can undermine intentional request-level sharing.

Mistake 3 – Ignoring Thread Safety

Singleton services may be accessed concurrently.

Never assume:

Singleton = Automatically Thread Safe

It is not.

Mistake 4 – Injecting Scoped Services into Long-Lived Services

For example:

BackgroundService
       |
       v
DbContext

Instead, create a scope when processing the background operation.

31. Easy Way to Remember

Remember this formula:

Transient = Every Time

Scoped = Every Request

Singleton = Entire Application

Or:

Transient
    ↓
New Object

Scoped
    ↓
One Object Per Scope

Singleton
    ↓
One Object Per Container Lifetime

32. Final Comparison

Feature	Transient	Scoped	Singleton
New instance frequently?	Yes	No	No
Same instance within request?	Not necessarily	Yes	Yes
Same instance across requests?	No	No	Yes
Typical lifetime	Resolution	Scope/request	Application/container
Thread safety concern	Usually less shared state	Usually less shared across requests	High if mutable
DbContext	❌	✅	❌
Repository	Usually Scoped	✅	❌
Stateless lightweight service	✅	Sometimes	Sometimes
Application-wide cache	❌	❌	✅, if designed safely
Configuration service	Sometimes	Sometimes	Often

33. Complete Mental Model

When you see:

builder.Services.AddTransient<A>();
builder.Services.AddScoped<B>();
builder.Services.AddSingleton<C>();

think:

                    DI CONTAINER
                         |
        +----------------+----------------+
        |                |                |
        v                v                v
    TRANSIENT         SCOPED          SINGLETON
        |                |                |
     New Object       One per Scope    One per Container
        |                |                |
        |           HTTP Request       Application
        |                |                |
        v                v                v
     A1, A2...          B1              C1
                       B2              C1
                       B3              C1

The key idea is that service lifetime is not merely about object creation—it determines how long the object can retain state and who can share that state.

Conclusion

Service Lifetime is a fundamental part of Dependency Injection in ASP.NET Core.

The three primary lifetimes are:

Transient → New instance when resolved

Scoped → One instance per scope/request

Singleton → One instance for the DI container lifetime

For a real-world e-commerce Web API, a common design is:

Controller
    |
    v
OrderService        → Scoped
    |
    +---- Repository → Scoped
    |
    +---- DbContext  → Scoped
    |
    +---- Payment    → Scoped
    |
    +---- Email      → Transient
    |
    +---- Cache      → Singleton

Choosing the correct lifetime is important for:

Performance
Memory management
Thread safety
Database consistency
Resource management
Scalability
Application stability

The most important interview rule to remember is:

Transient = new instance, Scoped = one instance per scope/request, Singleton = one instance for the container lifetime.

Once you understand this concept, Dependency Injection, DbContext, middleware, background services, repositories, caching, and ASP.NET Core application architecture become much easier to understand.

JWT Authentication & Authorization — Complete Flow

2026-08-21T09:21:49.404+05:30

Imagine the application has:

Angular / React frontend
ASP.NET Core Web API
SQL Server database
JWT-based authentication

The overall flow is:

User
  |
  | 1. Login: username + password
  v
Frontend
  |
  | 2. POST /api/auth/login
  v
Web API
  |
  | 3. Validate credentials
  v
Database
  |
  | 4. User is valid
  v
Web API
  |
  | 5. Create JWT
  v
Frontend
  |
  | 6. Store JWT
  |
  | 7. Send JWT in Authorization Header
  v
Web API
  |
  | 8. Validate JWT
  |
  | 9. Authentication
  |
  | 10. Authorization
  v
Controller
  |
  v
Response

Now let's understand every step.

1. What problem does JWT solve?

Suppose a user logs into your application.

Username: mahesh
Password: ********

The API verifies the username and password.

But after login, the API needs to know:

"Who is making this next request?"

For example:

GET /api/orders

The API needs to know:

Which user?
Is the user authenticated?
What roles does the user have?
Is the user allowed to access orders?

JWT provides a way for the client to prove its identity on subsequent requests.

2. Where does the JWT flow start?

The JWT authentication flow starts when the user performs login.

For example:

POST /api/auth/login

Request:

{
    "username": "mahesh",
    "password": "Password123"
}

The request reaches the Authentication API.

3. Step 1 — User sends credentials

The frontend sends:

Username
Password

to:

POST /api/auth/login

For example:

Angular Application
       |
       | username + password
       v
ASP.NET Core Web API

The password should be transmitted over HTTPS, not plain HTTP.

4. Step 2 — API validates the user

The API receives the credentials.

It queries the database:

SELECT Id, UserName, PasswordHash, Role
FROM Users
WHERE UserName = 'mahesh'

The application should compare the supplied password against the stored password hash.

It should not store plain-text passwords.

If authentication fails:

401 Unauthorized

If authentication succeeds:

UserId       = 101
Username     = mahesh
Role         = Customer

Now the API can create a JWT.

5. Step 3 — JWT is created

A JWT normally looks like this:

xxxxx.yyyyy.zzzzz

There are three parts:

HEADER.PAYLOAD.SIGNATURE

For example:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
.
eyJzdWIiOiIxMDEiLCJyb2xlIjoiQ3VzdG9tZXIifQ
.
abc123xyz...

These three parts have different responsibilities.

6. JWT Header

The header contains information about the token.

Example:

{
  "alg": "HS256",
  "typ": "JWT"
}

`alg`

This tells the receiver which signing algorithm is being used.

For example:

HS256

or:

RS256

`typ`

This tells us the token type:

JWT

Conceptually:

HEADER
   |
   +-- Algorithm
   |
   +-- Token Type

7. JWT Payload

The payload contains claims.

For example:

{
  "sub": "101",
  "name": "Mahesh",
  "role": "Customer",
  "email": "mahesh@example.com",
  "exp": 1787220000
}

These are called claims.

Common claims include:

Claim	Meaning
`sub`	Subject/User ID
`name`	User name
`email`	Email
`role`	User role
`iss`	Token issuer
`aud`	Intended audience
`iat`	Issued-at time
`exp`	Expiration time

For example:

{
   "sub": "101",
   "role": "Admin"
}

means:

User ID = 101
Role = Admin

Important security point

The JWT payload is encoded, not encrypted, in a normal JWT.

Therefore, don't put sensitive information such as:

Password
Credit card number
Secret keys

inside the payload.

8. JWT Signature

This is the most important part for understanding JWT security.

Conceptually, the server takes:

Base64Url(Header)
+
"."
+
Base64Url(Payload)

and signs that data using a secret/private key.

For example, conceptually with HMAC:

Signature =
HMACSHA256(
    Base64Url(Header) + "." +
    Base64Url(Payload),
    SecretKey
)

The result becomes:

HEADER.PAYLOAD.SIGNATURE

9. Why do we need the Signature?

Imagine the original payload is:

{
    "userId": 101,
    "role": "Customer"
}

An attacker might try to change it to:

{
    "userId": 101,
    "role": "Admin"
}

But the attacker doesn't have the signing secret/private key.

Therefore, they cannot generate a valid signature for the modified payload.

When the API receives the token, it validates the signature.

If the token was modified:

Payload changed
      ↓
Signature no longer matches
      ↓
JWT validation fails
      ↓
401 Unauthorized

So the signature provides integrity/authenticity of the token, assuming the signing key is properly protected.

10. JWT is returned to the client

After successful login:

Web API
   |
   | JWT
   v
Frontend

For example:

{
    "accessToken": "eyJhbGciOiJIUzI1NiIs..."
}

Now the frontend has the access token.

11. What happens on the next API request?

Suppose the user wants to see orders.

Frontend sends:

GET /api/orders

But how does the API know who the user is?

The frontend sends the JWT using the HTTP Authorization header.

Authorization: Bearer eyJhbGciOiJIUzI1NiIs...

This is extremely important.

12. What is the Authorization Header?

The HTTP request looks like:

GET /api/orders HTTP/1.1
Host: api.example.com
Authorization: Bearer eyJhbGciOiJIUzI1NiIs...

There are two important pieces:

Authorization
      |
      +-- Bearer
      |
      +-- JWT Token

Bearer essentially means:

"The caller is presenting this access token as its credential."

13. Complete request flow

Now we can visualize the entire process:

                    LOGIN
                      |
                      v
              +---------------+
              |    Frontend   |
              +---------------+
                      |
                      | username/password
                      v
              +---------------+
              |   Auth API    |
              +---------------+
                      |
                      | Validate credentials
                      v
              +---------------+
              |   Database    |
              +---------------+
                      |
                      | User valid
                      v
              +---------------+
              |   Auth API    |
              +---------------+
                      |
                      | Create JWT
                      v
          +-------------------------+
          | HEADER.PAYLOAD.SIGNATURE|
          +-------------------------+
                      |
                      | JWT
                      v
              +---------------+
              |    Frontend   |
              +---------------+

Then:

                API REQUEST
                     |
                     v
              +-------------+
              |  Frontend   |
              +-------------+
                     |
                     | Authorization:
                     | Bearer JWT
                     v
              +-------------+
              |  Web API    |
              +-------------+
                     |
                     v
              JWT Middleware
                     |
             +-------+-------+
             |               |
          Invalid           Valid
             |               |
             v               v
           401          User.Identity
                           created
                              |
                              v
                       Authorization
                              |
                    +---------+---------+
                    |                   |
                 Allowed              Denied
                    |                   |
                    v                   v
                Controller             403

14. What happens inside ASP.NET Core?

Suppose we have:

[Authorize]
[HttpGet("orders")]
public IActionResult GetOrders()
{
    return Ok();
}

The request arrives:

GET /api/orders
Authorization: Bearer <JWT>

ASP.NET Core's JWT authentication middleware processes the token before the controller action executes.

Conceptually:

HTTP Request
     |
     v
ASP.NET Core Middleware
     |
     v
JWT Authentication Handler
     |
     v
Read Authorization Header
     |
     v
Extract Bearer Token
     |
     v
Validate JWT
     |
     v
Create ClaimsPrincipal
     |
     v
Authorization
     |
     v
Controller

15. JWT Validation

The API validates several things depending on its configuration.

For example:

Signature

Is the signature valid?

Issuer

Who issued this token?

Example:

https://my-auth-server

Audience

Is this token intended for my API?

Example:

my-ecommerce-api

Expiration

Has the token expired?

For example:

{
    "exp": 1787220000
}

If the current time is beyond the expiration time, the token is rejected.

16. Authentication vs Authorization

This is one of the most important interview questions.

Authentication

Authentication answers:

Who are you?

Example:

User logs in
     ↓
Username/password validated
     ↓
JWT issued
     ↓
JWT presented to API
     ↓
API validates JWT
     ↓
User is authenticated

Authentication establishes the user's identity.

17. Authorization

Authorization answers:

What are you allowed to do?

Suppose we have:

Admin
Customer
Manager

JWT:

{
    "sub": "101",
    "role": "Customer"
}

Then:

[Authorize]

means:

An authenticated user can access this endpoint.

But:

[Authorize(Roles = "Admin")]

means:

Only authenticated users with the Admin role can access this endpoint.

If a Customer calls it:

Authenticated? YES

Authorized? NO

Result:
403 Forbidden

18. 401 vs 403

This is another important interview question.

401 Unauthorized

Usually means:

Authentication failed / no valid authentication

Examples:

No token
Invalid token
Expired token
Invalid signature

Conceptually:

Who are you?
→ I can't authenticate you.

403 Forbidden

Means:

You are authenticated,
but you don't have permission.

Example:

User = Customer

Endpoint requires = Admin

Result:

403 Forbidden

Think:

401 = I don't know who you are.

403 = I know who you are,
      but you're not allowed.

19. Authentication + Authorization Example

Suppose:

[Authorize]
[HttpGet("profile")]
public IActionResult Profile()
{
    return Ok();
}

Any authenticated user can access it.

But:

[Authorize(Roles = "Admin")]
[HttpDelete("users/{id}")]
public IActionResult DeleteUser(int id)
{
    return Ok();
}

Only Admin users can access it.

The flow becomes:

JWT
 |
 v
Signature validation
 |
 v
Token valid?
 |
 +---- NO ----> 401
 |
 YES
 |
 v
Authentication successful
 |
 v
Read Claims
 |
 v
Role = Customer?
 |
 v
Endpoint requires Admin
 |
 v
Authorization fails
 |
 v
403 Forbidden

20. Where does the Role come from?

The role can be included as a JWT claim.

Example:

{
    "sub": "101",
    "name": "Mahesh",
    "role": "Admin"
}

ASP.NET Core converts JWT claims into a ClaimsPrincipal.

You can then access claims:

var userId = User.FindFirst("sub")?.Value;

or:

var role = User.FindFirst("role")?.Value;

Depending on configuration, role claims can also be accessed through:

User.IsInRole("Admin")

21. How does ASP.NET Core know which JWT to validate?

You configure JWT authentication in the application.

Conceptually:

builder.Services
    .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidateIssuer = true,
            ValidateAudience = true,
            ValidateLifetime = true,
            ValidateIssuerSigningKey = true,

            ValidIssuer = "...",
            ValidAudience = "...",
            IssuerSigningKey = ...
        };
    });

And:

app.UseAuthentication();
app.UseAuthorization();

The order is important:

UseAuthentication()
        ↓
UseAuthorization()
        ↓
MapControllers()

Authentication must establish the user's identity before authorization makes the access decision.

22. Why is the Header important?

There are actually two places where "header" can mean different things.

JWT Header

Inside the JWT:

{
   "alg": "HS256",
   "typ": "JWT"
}

This describes the token.

HTTP Authorization Header

Outside the JWT:

Authorization: Bearer <token>

This transports the JWT from the client to the API.

So:

JWT Header
     ↓
Describes JWT

HTTP Authorization Header
     ↓
Carries JWT to API

Don't confuse these two.

23. Complete E-Commerce Example

Let's imagine an e-commerce application.

User:

Mahesh
UserId = 101
Role = Customer

Step 1 — Login

POST /api/auth/login

{
   "username": "mahesh",
   "password": "********"
}

Step 2 — Database validation

Username exists?
        ↓
Password valid?
        ↓
Role = Customer
        ↓
YES

Step 3 — JWT creation

Payload:

{
   "sub": "101",
   "name": "Mahesh",
   "role": "Customer",
   "exp": "..."
}

JWT:

HEADER.PAYLOAD.SIGNATURE

Step 4 — JWT returned

Authentication API
       ↓
      JWT
       ↓
    Frontend

Step 5 — Get orders

GET /api/orders
Authorization: Bearer <JWT>

Step 6 — API validates JWT

Token exists?
     ↓
Signature valid?
     ↓
Issuer valid?
     ↓
Audience valid?
     ↓
Token expired?
     ↓
All valid

Step 7 — Authentication

User = Mahesh
UserId = 101
Role = Customer

Step 8 — Authorization

Suppose:

[Authorize]

Customer is allowed.

Therefore:

Controller executes

24. What if the JWT is modified?

Original:

{
   "userId": 101,
   "role": "Customer"
}

Attacker changes it:

{
   "userId": 101,
   "role": "Admin"
}

The attacker doesn't have the signing key.

Therefore:

Modified Payload
       ↓
Signature doesn't match
       ↓
JWT validation fails
       ↓
Authentication fails
       ↓
401

This is why the signature is critical.

25. What if the JWT is expired?

Suppose:

{
   "sub": "101",
   "exp": 1787220000
}

The API checks:

Current time > exp?

If yes:

Token expired
     ↓
Authentication fails
     ↓
401 Unauthorized

The frontend can then obtain a new access token using an appropriate token renewal mechanism, such as a refresh-token flow, depending on the authentication architecture.

26. Does every API call query the User table?

Not necessarily.

That's one of the major benefits of JWT.

For a properly configured self-contained JWT, the API can validate:

Signature
Issuer
Audience
Expiration
Claims

without querying the user database on every request.

For example:

Request
   ↓
JWT
   ↓
Signature validation
   ↓
Claims
   ↓
Authorization
   ↓
Controller

However, applications sometimes still consult a database/cache for things such as account status, revocation, permissions that must change immediately, or other business rules.

27. Where is the JWT stored?

This depends on the frontend architecture and security requirements.

A common browser approach is to use secure cookie-based mechanisms, especially when designed to mitigate token theft/XSS risks.

Another approach is storing an access token in browser storage, but storing long-lived authentication tokens in localStorage has important security trade-offs because JavaScript can access it.

For a production system, token storage should be designed together with:

HTTPS
XSS protection
CSRF protection
Token lifetime
Refresh-token strategy
Cookie settings

28. JWT Flow — Start to End

Here's the complete flow you can remember for interviews:

                 ┌──────────────┐
                 │     USER     │
                 └──────┬───────┘
                        │
                        │ Login
                        ▼
                 ┌──────────────┐
                 │   FRONTEND   │
                 └──────┬───────┘
                        │
                        │ username/password
                        ▼
                 ┌──────────────┐
                 │  AUTH API    │
                 └──────┬───────┘
                        │
                        │ Validate
                        ▼
                 ┌──────────────┐
                 │   DATABASE   │
                 └──────┬───────┘
                        │
                        │ Valid
                        ▼
                 ┌──────────────┐
                 │  JWT CREATE  │
                 └──────┬───────┘
                        │
                        │
                HEADER.PAYLOAD
                  .SIGNATURE
                        │
                        ▼
                 ┌──────────────┐
                 │   FRONTEND   │
                 └──────┬───────┘
                        │
                        │ Authorization:
                        │ Bearer JWT
                        ▼
                 ┌──────────────┐
                 │   WEB API    │
                 └──────┬───────┘
                        │
                        ▼
                JWT VALIDATION
                        │
             ┌──────────┴──────────┐
             │                     │
          Invalid                 Valid
             │                     │
             ▼                     ▼
            401              Authentication
                                  │
                                  ▼
                            Authorization
                                  │
                    ┌─────────────┴────────────┐
                    │                          │
                 Allowed                    Denied
                    │                          │
                    ▼                          ▼
               Controller                    403
                    │
                    ▼
                 Response

29. The most important distinction

Remember these three concepts:

Header

What algorithm/type is this JWT using?

Payload

Who is the user?
What claims/attributes are associated with the token?

Signature

Has the token been altered,
and can it be validated using the expected signing key?

And remember:

HTTP Authorization Header
        ↓
Carries JWT

JWT Header
        ↓
Describes JWT

JWT Payload
        ↓
Contains Claims

JWT Signature
        ↓
Protects token integrity/authenticity

30. One-line interview answer

If an interviewer asks:

"Explain JWT authentication flow."

A strong answer is:

"When a user logs in, the authentication service validates the credentials and creates a signed JWT containing claims such as user ID, issuer, audience, role and expiration. The client then sends that token with subsequent API requests in the HTTP Authorization header using the Bearer scheme. ASP.NET Core's JWT authentication middleware extracts and validates the token's signature, issuer, audience and lifetime and creates the authenticated ClaimsPrincipal. The authorization middleware then evaluates policies or roles, such as [Authorize(Roles = "Admin")]. If authentication fails, the API returns 401; if authentication succeeds but authorization fails, it returns 403. If both succeed, the request reaches the controller."

That is the complete JWT authentication → authorization flow from login to API response.

Saga Design Pattern – Complete Guide with E-Commerce Example

2026-08-20T19:58:24.958+05:30

1. What Problem Does Saga Solve?

Imagine an e-commerce application with these microservices:

                    E-Commerce Application
                            |
        +-------------------+-------------------+
        |                   |                   |
   Order Service       Payment Service     Inventory Service
        |                   |                   |
        +-------------------+-------------------+
                            |
                    Shipping Service

A customer places an order.

The business process might be:

Create Order
    ↓
Reserve Inventory
    ↓
Process Payment
    ↓
Create Shipment
    ↓
Order Completed

The problem is that each operation belongs to a different database.

For example:

Order Service       → OrderDB
Inventory Service   → InventoryDB
Payment Service     → PaymentDB
Shipping Service    → ShippingDB

We cannot normally use a single SQL transaction such as:

BEGIN TRANSACTION

OrderDB
InventoryDB
PaymentDB
ShippingDB

COMMIT

because these are independent microservices.

This is where Saga Pattern comes in.

2. What Is Saga Design Pattern?

A Saga is a sequence of local transactions where each microservice performs its own transaction.

If one transaction fails, previously completed transactions are compensated by executing corresponding compensating transactions.

Conceptually:

Transaction 1
     ↓
Transaction 2
     ↓
Transaction 3
     ↓
Transaction 4

If Transaction 3 fails:

Transaction 1 ✓
Transaction 2 ✓
Transaction 3 ✗

        ↓

Compensation 2
        ↓
Compensation 1

So instead of a traditional distributed ACID transaction, Saga provides eventual consistency using local transactions + compensation.

3. E-Commerce Example

Suppose customer places an order:

Order #1001

Product: Laptop
Quantity: 1
Price: ₹80,000

The workflow is:

Customer
   |
   ↓
Order Service
   |
   ↓
Inventory Service
   |
   ↓
Payment Service
   |
   ↓
Shipping Service
   |
   ↓
Order Completed

Let's define the transactions.

T1 – Create Order

Order Service:

Order Status = Pending

T2 – Reserve Inventory

Inventory Service:

Laptop Stock
100 → 99

T3 – Process Payment

Payment Service:

₹80,000 charged

T4 – Create Shipment

Shipping Service:

Shipment Created

Finally:

Order Status = Confirmed

4. What Happens If Payment Fails?

Suppose:

T1 Create Order        ✓
T2 Reserve Inventory   ✓
T3 Payment             ✗

We cannot simply rollback T1 and T2 using a normal database rollback because they happened in different databases.

Instead:

Payment Failed
      ↓
Release Inventory
      ↓
Cancel Order

So:

T1 Create Order ✓
       ↓
T2 Reserve Stock ✓
       ↓
T3 Payment ✗
       ↓
C2 Release Stock ✓
       ↓
C1 Cancel Order ✓

This is the core concept of Saga.

5. Saga Has Two Main Approaches

There are two major implementations.

Approach 1 – Choreography

Services communicate through events.

Order Service
     |
 OrderCreated
     ↓
Inventory Service
     |
InventoryReserved
     ↓
Payment Service
     |
PaymentCompleted
     ↓
Shipping Service

There is no central coordinator.

Approach 2 – Orchestration

A central Saga Orchestrator controls the workflow.

                 Saga Orchestrator
                        |
          +-------------+-------------+
          |             |             |
          ↓             ↓             ↓
      Order          Inventory      Payment
      Service         Service       Service
                                      |
                                      ↓
                                  Shipping

The orchestrator says:

Reserve inventory

then:

Process payment

then:

Create shipment

If something fails:

Release inventory
Cancel order

For an enterprise e-commerce application, orchestration is often easier to understand and manage, especially when the workflow becomes complex.

6. Which One Should We Use?

Feature	Choreography	Orchestration
Central controller	No	Yes
Simple workflows	Excellent	Good
Complex workflows	Difficult	Excellent
Debugging	Difficult	Easier
Business workflow visibility	Lower	Higher
Coupling	Event-based	Orchestrator-based
Failure handling	Distributed	Centralized
Large enterprise workflows	Can become complicated	Often preferable

For the example below, I'll use Saga Orchestration.

7. Overall Architecture

Let's design the system.

                         Client
                           |
                           ↓
                    API Gateway
                           |
                           ↓
                    Order Service
                           |
                           ↓
                  Saga Orchestrator
                           |
          +----------------+----------------+
          |                |                |
          ↓                ↓                ↓
     Inventory          Payment          Shipping
      Service           Service           Service
          |                |                |
      InventoryDB       PaymentDB       ShippingDB

Communication could use:

Azure Service Bus
Kafka
RabbitMQ

For an Azure-based .NET system, Azure Service Bus is a natural choice.

8. Database Design

An important Saga principle is:

Each microservice owns its own database.

Don't do this:

Order Service
       |
       ↓
Shared Database
       ↑
       |
Inventory Service

Instead:

Order Service
     ↓
OrderDB

Inventory Service
     ↓
InventoryDB

Payment Service
     ↓
PaymentDB

Shipping Service
     ↓
ShippingDB

9. Order Model

Order Service might have:

public class Order
{
    public Guid Id { get; set; }

    public Guid CustomerId { get; set; }

    public decimal TotalAmount { get; set; }

    public OrderStatus Status { get; set; }

    public DateTime CreatedAt { get; set; }
}

Status:

public enum OrderStatus
{
    Pending,
    InventoryReserved,
    PaymentProcessing,
    Confirmed,
    Failed,
    Cancelled
}

10. Order Items

public class OrderItem
{
    public Guid Id { get; set; }

    public Guid OrderId { get; set; }

    public Guid ProductId { get; set; }

    public int Quantity { get; set; }

    public decimal Price { get; set; }
}

11. Inventory Model

Inventory Service owns:

public class Inventory
{
    public Guid ProductId { get; set; }

    public int AvailableQuantity { get; set; }

    public int ReservedQuantity { get; set; }
}

Example:

Product       Available     Reserved

Laptop          100            0

After reservation:

Laptop           99            1

12. Inventory Reservation Model

We should maintain a separate reservation record.

public class InventoryReservation
{
    public Guid Id { get; set; }

    public Guid OrderId { get; set; }

    public Guid ProductId { get; set; }

    public int Quantity { get; set; }

    public ReservationStatus Status { get; set; }
}

Status:

public enum ReservationStatus
{
    Reserved,
    Released
}

Why?

Because Saga requires us to know:

What exactly should I compensate?

13. Payment Model

Payment Service:

public class Payment
{
    public Guid Id { get; set; }

    public Guid OrderId { get; set; }

    public decimal Amount { get; set; }

    public PaymentStatus Status { get; set; }

    public string TransactionReference { get; set; }
}

Status:

public enum PaymentStatus
{
    Pending,
    Completed,
    Failed,
    Refunded
}

14. Shipment Model

Shipping Service:

public class Shipment
{
    public Guid Id { get; set; }

    public Guid OrderId { get; set; }

    public string Address { get; set; }

    public ShipmentStatus Status { get; set; }
}

15. Saga State Model

The orchestrator should maintain Saga state.

For example:

public class OrderSaga
{
    public Guid SagaId { get; set; }

    public Guid OrderId { get; set; }

    public SagaStatus Status { get; set; }

    public bool OrderCreated { get; set; }

    public bool InventoryReserved { get; set; }

    public bool PaymentCompleted { get; set; }

    public bool ShipmentCreated { get; set; }

    public DateTime CreatedAt { get; set; }

    public DateTime UpdatedAt { get; set; }
}

This becomes very useful for:

monitoring
retries
recovery
debugging
compensation

16. Complete Saga Flow

Let's look at the complete process.

Customer
   |
   | Place Order
   ↓
Order Service
   |
   | Order Created
   ↓
Saga Orchestrator
   |
   | Reserve Inventory
   ↓
Inventory Service
   |
   | Inventory Reserved
   ↓
Saga Orchestrator
   |
   | Process Payment
   ↓
Payment Service
   |
   | Payment Completed
   ↓
Saga Orchestrator
   |
   | Create Shipment
   ↓
Shipping Service
   |
   | Shipment Created
   ↓
Saga Orchestrator
   |
   ↓
Order Confirmed

17. Step 1 – Customer Creates Order

Client:

POST /api/orders

Request:

{
  "customerId": "C001",
  "items": [
    {
      "productId": "P100",
      "quantity": 1
    }
  ]
}

Order Service creates:

OrderId = 1001
Status = Pending

Database:

Orders

1001 | C001 | 80000 | Pending

Then publish:

OrderCreated

Event:

public record OrderCreatedEvent(
    Guid OrderId,
    Guid CustomerId,
    decimal Amount);

18. Step 2 – Saga Starts

The orchestrator receives:

OrderCreated

It creates:

SagaId = S1001
OrderId = 1001
Status = Started

Then sends:

ReserveInventory

19. Step 3 – Inventory Reservation

Inventory Service receives:

{
  "sagaId": "S1001",
  "orderId": "1001",
  "productId": "P100",
  "quantity": 1
}

It executes a local database transaction.

For example:

BEGIN TRANSACTION

Check available stock

Available = Available - 1

Create Reservation

COMMIT

Database becomes:

Available = 99
Reserved = 1

Then publish:

InventoryReserved

20. Step 4 – Payment

Saga orchestrator receives:

InventoryReserved

Then sends:

ProcessPayment

Payment Service:

BEGIN TRANSACTION

Create Payment
Status = Processing

Call payment provider

Payment successful

Status = Completed

COMMIT

Then:

PaymentCompleted

21. Step 5 – Shipping

Saga receives:

PaymentCompleted

Then:

CreateShipment

Shipping Service creates:

ShipmentId = SH1001
OrderId = 1001
Status = Created

Then publishes:

ShipmentCreated

22. Step 6 – Complete Saga

Saga Orchestrator receives:

ShipmentCreated

Now:

OrderCreated       ✓
InventoryReserved  ✓
PaymentCompleted   ✓
ShipmentCreated    ✓

So:

Saga Status = Completed

And Order Service is instructed:

ConfirmOrder

Order:

1001 | Confirmed

23. What Happens When Payment Fails?

This is where Saga becomes interesting.

Suppose:

Order Created       ✓
Inventory Reserved  ✓
Payment             ✗

The orchestrator receives:

PaymentFailed

It knows:

InventoryReserved = true
PaymentCompleted = false

Therefore compensation is required.

24. Compensation Flow

The orchestrator sends:

ReleaseInventory

Inventory Service:

BEGIN TRANSACTION

Available = Available + 1

Reservation.Status = Released

COMMIT

Now:

Available = 100
Reserved = 0

Then orchestrator sends:

CancelOrder

Order Service:

Order.Status = Cancelled

Final state:

Order       = Cancelled
Inventory   = Released
Payment     = Failed
Saga        = Compensated

25. Important Point – Compensation Is NOT Rollback

This is one of the most important interview concepts.

Traditional transaction:

BEGIN

Operation A
Operation B
Operation C

ROLLBACK

Saga:

Operation A
Operation B
Operation C → FAILED

Compensation B
Compensation A

There is no global database rollback.

Instead:

A compensating transaction semantically reverses the business effect of a previous transaction.

26. Example: Payment Succeeds but Shipping Fails

Consider:

Order        ✓
Inventory    ✓
Payment      ✓
Shipping     ✗

Now we need to compensate.

Possible sequence:

Shipping Failed
       ↓
Refund Payment
       ↓
Release Inventory
       ↓
Cancel Order

So:

T1 Create Order       ✓
T2 Reserve Inventory  ✓
T3 Payment            ✓
T4 Shipping           ✗

C3 Refund Payment     ✓
C2 Release Inventory  ✓
C1 Cancel Order       ✓

Final:

Order = Cancelled
Inventory = Released
Payment = Refunded
Shipping = Failed

27. What If Compensation Also Fails?

This is a very important real-world scenario.

Suppose:

Payment Failed
      ↓
Release Inventory
      ↓
Inventory Service FAILED

Now:

Order = Pending
Inventory = Reserved
Payment = Failed

We cannot simply give up.

The Saga must retry the compensation.

ReleaseInventory
      ↓
FAILED
      ↓
Retry
      ↓
FAILED
      ↓
Retry
      ↓
SUCCESS

Therefore Saga implementations need:

Retry
Dead-letter queue
Idempotency
Timeout
Monitoring
Manual recovery

28. Retry Strategy

For example:

Attempt 1
   ↓
5 seconds
   ↓
Attempt 2
   ↓
30 seconds
   ↓
Attempt 3
   ↓
5 minutes

This is called exponential backoff.

For Azure Service Bus, failed messages can eventually be moved to a dead-letter queue.

29. Idempotency Is Extremely Important

Suppose:

ReserveInventory

message is delivered twice.

Without idempotency:

Message 1 → Reserve 1 item
Message 2 → Reserve another item

Incorrect:

Stock: 100 → 98

But we wanted:

Stock: 100 → 99

Therefore every command should have a unique identifier.

Example:

public class ProcessedMessage
{
    public Guid MessageId { get; set; }

    public DateTime ProcessedAt { get; set; }
}

Before processing:

Has MessageId already been processed?

If yes:

Ignore

Otherwise:

Process
Save MessageId

30. Better Idempotency Model

Instead of only MessageId, use a business operation ID.

Example:

SagaId = S1001
OrderId = 1001
Operation = ReserveInventory

Create unique constraint:

(SagaId, Operation)

Then duplicate commands cannot create duplicate reservations.

31. Transactional Outbox Pattern

There is another major problem.

Suppose Order Service does:

BEGIN TRANSACTION

Insert Order

COMMIT

Then:

Publish OrderCreated

What if the application crashes between these operations?

Database Insert ✓
Publish Event ✗

Now the order exists but Saga never receives the event.

This is where Transactional Outbox Pattern is commonly combined with Saga.

32. Outbox Table

Order Service database:

Orders
OutboxMessages

When creating the order:

BEGIN TRANSACTION

INSERT INTO Orders

INSERT INTO OutboxMessages

COMMIT

Both happen in the same local database transaction.

Example:

Orders

OrderId = 1001
Status = Pending

And:

OutboxMessages

MessageId = M1001
Type = OrderCreated
Payload = {...}
Published = false

A background publisher then reads:

Published = false

and sends the message.

After successful publishing:

Published = true

This greatly improves reliability.

33. Saga + Outbox Architecture

A robust architecture becomes:

                         Saga Orchestrator
                                |
                         Message Broker
                                |
          +---------------------+---------------------+
          |                     |                     |
          ↓                     ↓                     ↓
      Order Service        Inventory Service      Payment
          |                     |                     |
       OrderDB              InventoryDB           PaymentDB
          |                     |                     |
       Outbox                 Outbox                Outbox
          |                     |                     |
          +---------------------+---------------------+
                                |
                         Azure Service Bus

34. Commands vs Events

This distinction is important.

Command

A command tells another service:

Do something.

Examples:

CreateOrder
ReserveInventory
ProcessPayment
CreateShipment
RefundPayment
ReleaseInventory
CancelOrder

Event

An event says:

Something happened.

Examples:

OrderCreated
InventoryReserved
InventoryReservationFailed
PaymentCompleted
PaymentFailed
ShipmentCreated
ShipmentFailed

35. Example Command

public record ReserveInventoryCommand(
    Guid MessageId,
    Guid SagaId,
    Guid OrderId,
    Guid ProductId,
    int Quantity);

36. Example Event

public record InventoryReservedEvent(
    Guid MessageId,
    Guid SagaId,
    Guid OrderId);

Failure:

public record InventoryReservationFailedEvent(
    Guid MessageId,
    Guid SagaId,
    Guid OrderId,
    string Reason);

37. Saga State Machine

The orchestrator can be modeled as a state machine.

             OrderCreated
                  |
                  ↓
          InventoryPending
                  |
        +---------+---------+
        |                   |
     Success              Failure
        |                   |
        ↓                   ↓
 PaymentPending        CancelOrder
        |
   +----+----+
   |         |
Success     Failure
   |         |
   ↓         ↓
Shipping   ReleaseInventory
Pending       |
   |          ↓
Success    CancelOrder
   |
   ↓
Completed

This is a very good way to explain Saga in an interview.

38. Orchestrator Pseudocode

Conceptually:

public async Task Handle(OrderCreatedEvent message)
{
    await ReserveInventory(message);
}

When inventory succeeds:

public async Task Handle(InventoryReservedEvent message)
{
    await ProcessPayment(message);
}

Payment succeeds:

public async Task Handle(PaymentCompletedEvent message)
{
    await CreateShipment(message);
}

Shipment succeeds:

public async Task Handle(ShipmentCreatedEvent message)
{
    await ConfirmOrder(message);
}

Payment fails:

public async Task Handle(PaymentFailedEvent message)
{
    await ReleaseInventory(message);
    await CancelOrder(message);
}

39. Compensation Table

A useful way to design a Saga is to create a compensation matrix.

Forward Transaction	Compensation
Create Order	Cancel Order
Reserve Inventory	Release Inventory
Process Payment	Refund Payment
Create Shipment	Cancel Shipment
Apply Coupon	Restore Coupon
Allocate Loyalty Points	Return Loyalty Points

For every business transaction, ask:

If this succeeds and something later fails, how do I undo its business effect?

If you cannot answer that, your Saga design isn't complete.

40. Data Consistency

A common question is:

How does Saga maintain data consistency?

It does not provide immediate strong consistency across all databases like a single ACID transaction.

Instead it provides:

Eventual Consistency

For example:

Initially:

Order = Pending
Inventory = Reserved
Payment = Processing

After processing:

Order = Confirmed
Inventory = Reserved
Payment = Completed

Or if failure occurs:

Order = Cancelled
Inventory = Released
Payment = Failed

The system may temporarily have intermediate states, but eventually it reaches a valid business state.

41. Important: Don't Compensate Everything Blindly

Suppose:

Inventory reserved
Payment completed
Shipping failed

You shouldn't simply execute compensation commands without knowing the actual state.

Maintain Saga state:

InventoryReserved = true
PaymentCompleted = true
ShipmentCreated = false

Then compensation is based on completed steps.

if PaymentCompleted
    RefundPayment()

if InventoryReserved
    ReleaseInventory()

if OrderCreated
    CancelOrder()

42. Timeout Handling

Suppose Payment Service doesn't respond.

Payment Request
      ↓
Waiting...
      ↓
Waiting...
      ↓
Timeout

The orchestrator should not wait forever.

For example:

Payment timeout = 5 minutes

Then:

Payment Timeout
      ↓
Check payment status
      ↓
If unknown → retry/query provider
      ↓
If definitely failed → compensate

This is especially important with external payment gateways.

43. Why Payment Status Needs Special Care

Imagine:

Payment Service → Payment Gateway

Payment request is sent.

Gateway processes it.

But response is lost.

Your service sees:

Timeout

You must not automatically refund or retry blindly because the first payment might actually have succeeded.

You may accidentally charge the customer twice.

Instead use:

Idempotency Key

For example:

OrderId = 1001
PaymentAttempt = 1
IdempotencyKey = ORDER-1001-PAYMENT

The payment provider should treat repeated requests with the same key as the same logical operation where its API supports idempotency.

44. Handling Concurrent Orders

Suppose only one laptop remains.

Stock = 1

Two customers simultaneously order.

Customer A → Reserve
Customer B → Reserve

Inventory Service must use appropriate concurrency control.

For example:

UPDATE Inventory
SET AvailableQuantity = AvailableQuantity - 1
WHERE ProductId = @ProductId
AND AvailableQuantity >= 1;

Then check:

Rows affected = 1

Reservation succeeds.

If:

Rows affected = 0

reservation fails.

This prevents overselling.

45. Saga Failure Scenarios

Scenario 1

Order ✓
Inventory ✓
Payment ✓
Shipping ✓

Result:

Completed

Scenario 2

Order ✓
Inventory ✗

Compensation:

Cancel Order

Scenario 3

Order ✓
Inventory ✓
Payment ✗

Compensation:

Release Inventory
Cancel Order

Scenario 4

Order ✓
Inventory ✓
Payment ✓
Shipping ✗

Compensation:

Refund Payment
Release Inventory
Cancel Order

Scenario 5

Compensation fails

Solution:

Retry
 ↓
Retry
 ↓
Dead Letter Queue
 ↓
Operational Alert
 ↓
Manual Recovery

46. Azure Implementation

Since you're working with Azure/.NET, one possible architecture is:

ASP.NET Core
      |
      ↓
Order Service
      |
      ↓
Azure Service Bus
      |
      ↓
Saga Orchestrator
      |
      +----------------+
      |                |
      ↓                ↓
Inventory          Payment
Service            Service
      |                |
      ↓                ↓
 Azure SQL          Azure SQL

And:

Shipping Service
       |
       ↓
   Azure SQL

For monitoring:

Application Insights
Azure Monitor

For secrets:

Azure Key Vault

47. Azure Service Bus Structure

You might design:

Topic: ecommerce-events

Subscriptions:

order
inventory
payment
shipping
saga

Or use separate command queues:

inventory-commands
payment-commands
shipping-commands
order-commands

For orchestration, a command queue per service plus an event topic is often a clean conceptual model.

48. Message Flow

Example:

Order Service
     |
     | OrderCreated
     ↓
Service Bus
     |
     ↓
Saga Orchestrator
     |
     | ReserveInventory
     ↓
Inventory Queue
     |
     ↓
Inventory Service
     |
     | InventoryReserved
     ↓
Service Bus
     |
     ↓
Saga
     |
     | ProcessPayment
     ↓
Payment Queue

And so on.

49. Database Transactions Are Still Used

This is another important point.

Saga does not mean:

Don't use database transactions.

Each microservice should still use normal local transactions.

For example:

Inventory Service

BEGIN TRANSACTION

UPDATE Inventory

INSERT Reservation

INSERT OutboxMessage

COMMIT

This is a local ACID transaction.

Saga coordinates these local transactions.

50. The Golden Rule

Think about Saga like this:

Saga
 =
Multiple Local Transactions
 +
Messages
 +
State Machine
 +
Compensating Transactions
 +
Retry
 +
Idempotency
 +
Timeout Handling
 +
Observability

That is a much more accurate real-world definition than simply saying:

Saga is rollback for microservices.

It is not a distributed rollback mechanism.

51. Complete E-Commerce Flow

Here's the complete picture:

                         CUSTOMER
                            |
                            ↓
                       API Gateway
                            |
                            ↓
                     ORDER SERVICE
                            |
                       Create Order
                            |
                            ↓
                      OrderCreated
                            |
                            ↓
                   SAGA ORCHESTRATOR
                            |
                    Reserve Inventory
                            |
                            ↓
                   INVENTORY SERVICE
                            |
                   Inventory Reserved
                            |
                            ↓
                   SAGA ORCHESTRATOR
                            |
                     Process Payment
                            |
                            ↓
                    PAYMENT SERVICE
                            |
                     Payment Success
                            |
                            ↓
                   SAGA ORCHESTRATOR
                            |
                     Create Shipment
                            |
                            ↓
                   SHIPPING SERVICE
                            |
                    Shipment Created
                            |
                            ↓
                   SAGA ORCHESTRATOR
                            |
                            ↓
                     Confirm Order

Failure:

Payment Failed
      |
      ↓
Saga Orchestrator
      |
      +------→ Release Inventory
      |
      +------→ Cancel Order
      |
      ↓
Saga Compensated

52. Production-Grade Saga Checklist

When implementing Saga in a real application, consider all of these:

Saga ID
Correlation ID
Message ID
Idempotency
Saga state persistence
Local database transactions
Transactional Outbox
Reliable messaging
Retries
Exponential backoff
Timeouts
Dead-letter queues
Compensating transactions
Concurrency control
Optimistic/pessimistic locking where appropriate
Observability
Distributed tracing
Audit logging
Manual recovery
Poison message handling

53. Saga vs Two-Phase Commit

A common interview question is:

2PC

Coordinator
    |
    +--- DB1
    +--- DB2
    +--- DB3

Prepare
Prepare
Prepare

Commit
Commit
Commit

It attempts to provide distributed transactional atomicity, but can introduce blocking, coordination overhead, and operational complexity.

Saga:

Local Transaction
      ↓
Message
      ↓
Local Transaction
      ↓
Message

Failure:

Compensating Transaction

Saga is generally better suited to independently deployable microservices where business operations can be compensated.

54. Interview Answer

If an interviewer asks:

"Explain Saga Design Pattern with an e-commerce example."

A strong answer would be:

"Saga is a distributed transaction pattern used in microservices where a business transaction is divided into a sequence of local transactions. Each service commits its own transaction independently. If a later transaction fails, the Saga executes compensating transactions for the previously completed operations.
For example, in an e-commerce system, placing an order may involve creating the order, reserving inventory, processing payment, and creating a shipment. If payment fails after inventory has been reserved, the Saga doesn't perform a database rollback across services. Instead, it executes a compensating transaction to release the inventory and then cancels the order.
Saga can be implemented using choreography, where services communicate through events, or orchestration, where a central Saga orchestrator manages the workflow. In a production system, I would also use an outbox pattern, idempotent message processing, retries, timeouts, dead-letter queues, correlation IDs, and persistent Saga state to achieve reliable eventual consistency."

55. Recommended .NET Architecture

For a production .NET implementation, I would structure it approximately like this:

src
│
├── OrderService
│   ├── Controllers
│   ├── Domain
│   ├── Application
│   ├── Infrastructure
│   └── Messaging
│
├── InventoryService
│   ├── Domain
│   ├── Application
│   ├── Infrastructure
│   └── Messaging
│
├── PaymentService
│   ├── Domain
│   ├── Application
│   ├── Infrastructure
│   └── Messaging
│
├── ShippingService
│   ├── Domain
│   ├── Application
│   ├── Infrastructure
│   └── Messaging
│
└── OrderSaga
    ├── StateMachine
    ├── Commands
    ├── Events
    ├── Consumers
    └── Persistence

A particularly robust combination is:

Microservices
     +
Saga Orchestration
     +
Azure Service Bus
     +
Transactional Outbox
     +
Idempotent Consumers
     +
Retry/Timeout
     +
Dead Letter Queue
     +
Azure SQL
     +
Application Insights

The key idea to remember is:

Saga does not make multiple databases behave like one database. It coordinates independent local transactions and uses compensating actions to bring the overall business process to a consistent state.

Azure AI Search: Complete Guide with a Real-Time .NET Example

2026-08-20T12:26:51.854+05:30

Introduction

Modern applications generate and consume enormous amounts of information—PDFs, Word documents, web pages, product catalogs, support tickets, database records, manuals, and knowledge-base articles.

Traditional database queries work well when users know the exact keywords they are looking for. But modern applications increasingly need to understand meaning and intent, not just exact words.

This is where Azure AI Search becomes extremely useful.

Azure AI Search is a managed search and retrieval service from Microsoft Azure that supports traditional full-text search, vector search, hybrid search, semantic ranking, and generative-AI/RAG scenarios. (Microsoft Learn)

A typical modern architecture looks like this:

                User
                  |
                  v
        Angular / Web / Mobile
                  |
                  v
          ASP.NET Core API
                  |
        +---------+---------+
        |                   |
        v                   v
 Azure AI Search       Azure OpenAI
        |                   |
        |<--- Context ------|
        |
        v
   Search Results
        |
        +--------> LLM
                    |
                    v
              Final Answer

This article explains Azure AI Search from the fundamentals through an end-to-end ASP.NET Core + Azure AI Search + Azure OpenAI RAG application.

1. What Is Azure AI Search?

Azure AI Search is Microsoft's cloud-based search-as-a-service platform.

It allows applications to:

Store searchable documents
Search text
Search using keywords
Search by semantic meaning
Perform vector similarity searches
Perform hybrid searches
Apply filters
Sort and facet results
Re-rank search results using semantic ranking
Retrieve grounding information for RAG applications

Microsoft documentation currently describes Azure AI Search as supporting text, vector, and multimodal content for traditional and generative search scenarios. (Microsoft Learn)

Think of it as:

Traditional Database
        |
        | Exact data retrieval
        v
     SQL Query

Azure AI Search
        |
        | Intelligent information retrieval
        v
 Keyword + Semantic + Vector + Hybrid Search

2. Why Do We Need Azure AI Search?

Suppose a company has:

100,000 PDF documents
50,000 Word documents
1 million support tickets
500,000 product descriptions

A user asks:

"How can I reset my corporate VPN password?"

A traditional SQL query might search for:

WHERE Description LIKE '%VPN%'
AND Description LIKE '%password%'

But the user could ask:

"I forgot the credentials needed to connect remotely."

The words are completely different.

A semantic/vector search engine can recognize that:

"forgot credentials"

is related to:

"reset VPN password"

This is one of the major advantages of vector and semantic search.

3. Azure AI Search vs SQL Server

Azure AI Search does not replace SQL Server.

They solve different problems.

SQL Server	Azure AI Search
Transactional data	Search/retrieval
INSERT/UPDATE/DELETE	Indexing/search
Relationships	Search indexes
Joins	Search queries
ACID transactions	Relevance ranking
Structured data	Text/vector content
Business transactions	Information retrieval

A common architecture is:

SQL Server
   |
   | Product / Customer / Order data
   |
   v
ASP.NET Core
   |
   +------> SQL Server
   |
   +------> Azure AI Search

4. Important Azure AI Search Concepts

Before implementing Azure AI Search, understand these concepts.

4.1 Search Service

The Azure AI Search service is the Azure resource that provides the search engine.

For example:

my-company-search.search.windows.net

Applications communicate with this service through:

REST API
Azure SDK
.NET SDK
Python SDK
JavaScript/TypeScript SDK
Java SDK

Microsoft provides official SDKs and samples for these languages. (Microsoft Learn)

5. Search Index

A search index is similar conceptually to a database table, but it is designed specifically for search.

For example:

ProductIndex

could contain:

ProductId
ProductName
Description
Category
Price
Availability
DescriptionVector

Example:

{
  "ProductId": "P1001",
  "ProductName": "Laptop",
  "Description": "Business laptop with 16GB RAM",
  "Category": "Computer",
  "Price": 75000
}

6. Search Fields

Each index contains fields.

For example:

ProductId
ProductName
Description
Category
Price
DescriptionVector

Different fields can have different capabilities.

For example:

ProductName
    searchable

Description
    searchable

Category
    filterable

Price
    filterable
    sortable

DescriptionVector
    vector searchable

7. Documents

A document represents one searchable item inside an index.

For example:

{
    "id": "101",
    "title": "Azure AI Search",
    "content": "Azure AI Search provides intelligent search capabilities.",
    "category": "Azure"
}

Another document might represent:

PDF chunk
Product
Customer FAQ
Support ticket
Knowledge-base article
Web page

8. Full-Text Search

Traditional full-text search searches words contained in documents.

For example:

Search:
Azure Service Bus

Azure AI Search looks for matching terms in searchable fields.

This approach is particularly useful for:

Product codes
Names
Exact terminology
IDs
Dates
Specialized technical terms

Microsoft notes that keyword search can be particularly useful where exact matching matters. (Microsoft Learn)

9. Vector Search

Vector search is one of the most important capabilities for modern AI applications.

An embedding model converts text into a numerical representation called a vector.

For example:

"How do I reset my password?"

might become conceptually:

[0.021, -0.192, 0.442, ...]

The vector represents semantic information rather than simply storing individual keywords.

The user's question is also converted into a vector.

Then Azure AI Search finds documents whose vectors are mathematically similar.

Microsoft describes vector search as useful for conceptual similarity and multilingual or multimodal scenarios. (Microsoft Learn)

10. Why Vector Search Is Powerful

Consider these two sentences:

Document:
"Employees can change their credentials from the security portal."

User:
"Where can I update my password?"

There may be no exact keyword match for every word.

But semantically:

change credentials
        ≈
update password

Vector search can identify that relationship.

11. What Is an Embedding?

An embedding is a numerical representation of content.

For example:

Text
 |
 v
Embedding Model
 |
 v
Vector

Conceptually:

"Azure Functions"
        |
        v
[0.12, -0.44, 0.72, ...]

The actual vector has many dimensions.

The application stores this vector in a vector field in Azure AI Search.

12. Hybrid Search

Hybrid search combines:

Keyword Search
       +
Vector Search

in a single request.

Azure AI Search executes the text and vector searches and merges their results using Reciprocal Rank Fusion (RRF). (Microsoft Learn)

Conceptually:

                User Query
                    |
          +---------+---------+
          |                   |
          v                   v
     Keyword Search      Vector Search
          |                   |
          v                   v
      Results A           Results B
          |                   |
          +---------+---------+
                    |
                    v
                   RRF
                    |
                    v
             Combined Results

This is often a strong choice for enterprise search.

13. Semantic Ranking

Semantic ranking provides another relevance layer.

The initial results can be reranked using Microsoft's language-understanding models.

For example:

Query:
"How can I work remotely?"

Result A:
"Remote working policy..."

Result B:
"Office parking policy..."

Result C:
"VPN remote access instructions..."

Semantic ranking can identify which results are most relevant to the meaning of the query.

Microsoft describes semantic ranker as a secondary ranking stage over an initial BM25 or RRF result set. (Microsoft Learn)

14. Search Architecture

A modern Azure AI Search application can look like this:

                  Documents
                     |
       +-------------+-------------+
       |             |             |
       v             v             v
     PDF          Database       Website
       |             |             |
       +-------------+-------------+
                     |
                     v
              Data Processing
                     |
                     v
              Chunking / Cleaning
                     |
                     v
               Embeddings
                     |
                     v
             Azure AI Search
                     |
              Search Index
                     |
                     v
                 User Query
                     |
          +----------+----------+
          |                     |
          v                     v
       Keyword               Vector
       Search                Search
          |                     |
          +----------+----------+
                     |
                     v
                    RRF
                     |
                     v
              Semantic Ranker
                     |
                     v
                Top Results
                     |
                     v
                Azure OpenAI
                     |
                     v
               Final Response

15. What Is RAG?

RAG means:

Retrieval-Augmented Generation

Instead of asking an LLM to answer only from its trained knowledge, we first retrieve relevant information from our own data.

The process is:

User Question
      |
      v
Azure AI Search
      |
      v
Relevant Documents
      |
      v
Prompt + Retrieved Context
      |
      v
Azure OpenAI
      |
      v
Final Answer

Microsoft describes RAG as a pattern that grounds LLM responses in proprietary content. (Microsoft Learn)

16. Real-Time Example: Employee Knowledge Assistant

Let's build a realistic enterprise application.

Imagine a company has:

HR Policies
IT Policies
Leave Policies
Travel Policies
Insurance Documents
Employee Handbook
Security Guidelines

Employees can ask:

"How many days of annual leave can I take?"

or:

"What is the process for claiming travel expenses?"

or:

"How do I reset my VPN password?"

Instead of manually searching hundreds of documents, the application retrieves the relevant information automatically.

17. Complete Architecture

Our solution will use:

Angular
   |
   v
ASP.NET Core Web API
   |
   +--------------------+
   |                    |
   v                    v
Azure AI Search      Azure OpenAI
   |
   v
Search Index

Documents:

Azure Blob Storage
       |
       v
Document Processing
       |
       v
Chunking
       |
       v
Embeddings
       |
       v
Azure AI Search

18. Step 1 – Create Azure AI Search

In Azure Portal:

Azure Portal
     |
     v
Create a resource
     |
     v
Search service

Configure:

Subscription
Resource Group
Service Name
Region
Pricing Tier

After deployment, you will have an Azure AI Search service.

19. Step 2 – Create Azure OpenAI

Create an Azure OpenAI resource and deploy appropriate models for:

Chat / generation
+
Embeddings

The embedding model is used to convert content and queries into vectors.

The chat model generates the final response.

20. Step 3 – Store Documents

Suppose we have:

EmployeeHandbook.pdf
LeavePolicy.pdf
TravelPolicy.pdf
SecurityPolicy.pdf

A common architecture is:

Azure Blob Storage
        |
        v
Document Processing
        |
        v
Azure AI Search

Microsoft's .NET RAG tutorial similarly demonstrates a solution using Azure OpenAI, Azure AI Search, storage, and an application layer. (Microsoft Learn)

21. Step 4 – Chunk Documents

Large documents should normally be divided into smaller pieces.

For example:

LeavePolicy.pdf

        |
        v

Chunk 1
Introduction

Chunk 2
Annual Leave

Chunk 3
Sick Leave

Chunk 4
Approval Process

Chunk 5
Carry Forward Policy

Why?

Because sending an entire 100-page document to an LLM for every question is inefficient.

Instead, retrieve only the relevant chunks.

22. Example Search Document

Our index could contain:

{
  "id": "leave-001",
  "title": "Annual Leave Policy",
  "content": "Employees are entitled to annual leave according to company policy...",
  "category": "HR",
  "documentName": "LeavePolicy.pdf",
  "contentVector": [ ... ]
}

The important fields are:

id
title
content
category
documentName
contentVector

23. Create a .NET Project

Create an ASP.NET Core Web API:

dotnet new webapi -n EnterpriseSearch.Api

Install the Azure AI Search SDK:

dotnet add package Azure.Search.Documents

The official .NET SDK package is Azure.Search.Documents. (Microsoft Learn)

24. Configuration

appsettings.json:

{
  "AzureSearch": {
    "Endpoint": "https://YOUR-SEARCH-SERVICE.search.windows.net",
    "IndexName": "employee-knowledge"
  }
}

For production, avoid storing secrets directly in configuration files.

Prefer:

Managed Identity
+
Microsoft Entra ID
+
Azure Key Vault

Microsoft's current .NET RAG tutorial demonstrates managed identities for passwordless service-to-service authentication. (Microsoft Learn)

25. Create Search Service Class

Example:

using Azure;
using Azure.Search.Documents;
using Azure.Search.Documents.Models;

public class SearchService
{
    private readonly SearchClient _searchClient;

    public SearchService(
        IConfiguration configuration)
    {
        var endpoint =
            configuration["AzureSearch:Endpoint"];

        var indexName =
            configuration["AzureSearch:IndexName"];

        var credential =
            new Azure.Identity.DefaultAzureCredential();

        _searchClient =
            new SearchClient(
                new Uri(endpoint!),
                indexName,
                credential);
    }

    public async Task<List<SearchDocument>> SearchAsync(
        string query)
    {
        var options = new SearchOptions
        {
            Size = 5
        };

        options.Select.Add("id");
        options.Select.Add("title");
        options.Select.Add("content");
        options.Select.Add("documentName");

        var response =
            await _searchClient.SearchAsync<SearchDocument>(
                query,
                options);

        var results = new List<SearchDocument>();

        await foreach (var result in response.Value.GetResultsAsync())
        {
            results.Add(result.Document);
        }

        return results;
    }
}

26. Create the API Controller

[ApiController]
[Route("api/search")]
public class SearchController : ControllerBase
{
    private readonly SearchService _searchService;

    public SearchController(SearchService searchService)
    {
        _searchService = searchService;
    }

    [HttpGet]
    public async Task<IActionResult> Search(
        string query)
    {
        var results =
            await _searchService.SearchAsync(query);

        return Ok(results);
    }
}

Now the API can be called like:

GET /api/search?query=annual leave

27. Example Response

The API could return:

[
  {
    "title": "Annual Leave Policy",
    "content": "Employees are entitled to annual leave...",
    "documentName": "LeavePolicy.pdf"
  },
  {
    "title": "Employee Handbook",
    "content": "Leave requests must be submitted...",
    "documentName": "EmployeeHandbook.pdf"
  }
]

28. Adding Vector Search

For vector search, the index contains a vector field.

Conceptually:

content
contentVector

The process is:

User Query
     |
     v
Embedding Model
     |
     v
Query Vector
     |
     v
Azure AI Search
     |
     v
Similar Document Vectors

29. Hybrid Search Query

A hybrid query contains both:

search
+
vectorQueries

Conceptually:

{
  "search": "How do I apply for annual leave?",
  "vectorQueries": [
    {
      "kind": "vector",
      "vector": [ ... ],
      "fields": "contentVector",
      "k": 5
    }
  ],
  "top": 5
}

Azure AI Search executes both searches and combines their results using RRF. (Microsoft Learn)

30. Semantic Hybrid Search

For high-quality retrieval, you can combine:

Keyword
+
Vector
+
Semantic Ranking

Conceptually:

User Query
    |
    +----> Keyword Search
    |
    +----> Vector Search
              |
              v
             RRF
              |
              v
       Semantic Ranker
              |
              v
       Best Documents

This is an especially useful architecture for RAG applications. Microsoft documentation highlights hybrid search with semantic ranking as a strong relevance strategy. (Microsoft Learn)

31. Complete RAG Flow

Let's walk through an actual question.

User asks:

"How many days of annual leave can I take?"

Step 1

Angular sends:

POST /api/chat

Request:

{
  "question": "How many days of annual leave can I take?"
}

Step 2

ASP.NET Core receives the question.

Step 3

The application sends the question to the retrieval layer.

Step 4

The query is converted into a vector.

Step 5

Azure AI Search performs:

Keyword Search
+
Vector Search

Step 6

Results are merged using RRF.

Step 7

Semantic ranking can further improve ordering.

Step 8

Top relevant chunks are returned.

Example:

Annual Leave Policy
-------------------

Employees are entitled to 20 days
of annual leave per calendar year.

Step 9

The application builds a prompt:

Answer the user's question using
only the following company policy.

Context:
Employees are entitled to 20 days
of annual leave per calendar year.

Question:
How many days of annual leave can I take?

Step 10

Azure OpenAI generates:

According to the Annual Leave Policy,
employees are entitled to 20 days of
annual leave per calendar year.

32. Why RAG Is Better Than Sending Everything to the LLM

Without RAG:

User
 |
 v
LLM
 |
 v
Possible hallucination

With RAG:

User
 |
 v
Azure AI Search
 |
 v
Company Documents
 |
 v
Relevant Context
 |
 v
LLM
 |
 v
Grounded Answer

RAG doesn't automatically guarantee that every answer is correct, but it gives the model relevant source material from your organization's data.

33. Adding Citations

A production enterprise application should ideally tell users where an answer came from.

For example:

According to the Annual Leave Policy,
employees are entitled to 20 days of annual leave.

Source:
LeavePolicy.pdf
Page 4

Your index can store metadata such as:

documentName
pageNumber
chunkId
sourceUrl
department

Then your API can return:

{
  "answer": "Employees are entitled to 20 days of annual leave.",
  "sources": [
    {
      "document": "LeavePolicy.pdf",
      "page": 4
    }
  ]
}

This greatly improves trust and auditability.

34. Security in Azure AI Search

Enterprise search must consider authorization.

Suppose:

HR Documents
Finance Documents
Engineering Documents

An Engineering employee should not automatically retrieve confidential HR documents.

A production architecture should therefore consider:

User Identity
     |
     v
Authorization
     |
     v
Security Filter
     |
     v
Azure AI Search

Metadata can include:

department
userId
role
securityGroup
accessLevel

Then search filters can restrict what users are allowed to retrieve.

35. Azure AI Search and Managed Identity

For production applications, avoid this:

API Key stored in appsettings.json

Prefer:

ASP.NET Core
     |
     v
Managed Identity
     |
     v
Microsoft Entra ID
     |
     v
Azure AI Search

This eliminates the need to distribute long-lived secrets between Azure services.

36. Common Real-Time Use Cases

Azure AI Search can be used for:

Enterprise Knowledge Search

Company Documents
       |
       v
AI Search
       |
       v
Employee Assistant

E-Commerce

Products
   |
   v
AI Search
   |
   +--> Keyword
   +--> Semantic
   +--> Vector

Example:

"Find lightweight laptops for programming."

Customer Support

Support Tickets
Knowledge Base
Product Manuals
       |
       v
Azure AI Search
       |
       v
Support Copilot

Legal Document Search

Contracts
Policies
Agreements
       |
       v
Search + RAG

Healthcare Knowledge Systems

For appropriate authorized environments:

Clinical documents
Research documents
Policies
       |
       v
Search

Sensitive applications require strong access control, privacy, compliance, and human oversight.

37. Azure AI Search in Microservices Architecture

For a microservices application:

                    API Gateway
                         |
          +--------------+--------------+
          |              |              |
          v              v              v
     Product Service  Order Service  Customer Service
          |              |              |
          +--------------+--------------+
                         |
                         v
                  Search Service
                         |
                         v
                 Azure AI Search

I recommend keeping search concerns separate from transactional services.

For example:

Product Service
      |
      v
Product Database

Search Indexing Service
      |
      v
Azure AI Search

When product data changes:

Product Updated
      |
      v
Event / Message
      |
      v
Search Indexing Service
      |
      v
Azure AI Search

This avoids tightly coupling every service to the search engine.

38. Event-Driven Indexing

For large systems:

Product Service
      |
      v
Azure Service Bus
      |
      v
Search Index Worker
      |
      v
Azure AI Search

For example:

ProductUpdatedEvent

could contain:

{
  "productId": "P1001",
  "operation": "Updated"
}

The indexing worker then updates the corresponding Azure AI Search document.

39. Azure AI Search vs Elasticsearch

Feature	Azure AI Search	Elasticsearch
Azure integration	Excellent	Good
Microsoft ecosystem	Excellent	Good
Vector search	Yes	Yes
Hybrid search	Yes	Yes
Semantic ranking	Yes	Different approach
Azure OpenAI integration	Excellent	Possible
Managed Azure service	Yes	Depends on offering
.NET integration	Excellent	Excellent

If your application is already heavily invested in Azure, Azure AI Search is often a natural choice.

40. Azure AI Search vs Database LIKE

Bad approach for large-scale AI search:

SELECT *
FROM Documents
WHERE Content LIKE '%password%';

Better:

Azure AI Search
      |
      +-- Full Text
      +-- Vector
      +-- Hybrid
      +-- Semantic Ranking

41. Common Mistakes

Mistake 1 – Indexing entire documents as one chunk

Instead:

Document
   |
   v
Meaningful chunks

Mistake 2 – Using only vector search

Vector search is powerful, but exact keywords can be important.

For example:

INC-45872

A keyword search may be more useful than semantic similarity.

Therefore:

Keyword + Vector

is often preferable.

Mistake 3 – Sending too many search results to the LLM

Don't retrieve hundreds of chunks.

Instead:

Retrieve
   |
   v
Rank
   |
   v
Top relevant chunks
   |
   v
LLM

Mistake 4 – Ignoring security

Never assume:

"If the document is indexed,
every user can see it."

Security trimming and authorization must be designed into the solution.

Mistake 5 – Storing secrets in source code

Avoid:

const string apiKey = "xxxxxxxx";

Use:

Managed Identity
Azure Key Vault
Microsoft Entra ID

42. Performance Optimization

Important considerations include:

Chunk Size

Avoid extremely large chunks.

Use logically meaningful sections.

Top K

Retrieve a controlled number of candidates.

For example:

Top 5
Top 10
Top 20

depending on the workload.

Hybrid Search

Use hybrid retrieval when both exact matching and semantic matching matter.

Semantic Ranking

Use semantic ranking when better relevance justifies its additional cost/latency.

Filters

Apply metadata filters when possible.

For example:

department = 'IT'

or:

documentType = 'Policy'

Monitoring

Monitor:

Latency
Query volume
Failed queries
Search relevance
Token usage
LLM latency
Cost

43. Production Architecture

A more complete enterprise implementation might look like:

                    Users
                      |
                      v
                 Azure Front Door
                      |
                      v
                API Management
                      |
                      v
              ASP.NET Core API
                      |
          +-----------+-----------+
          |                       |
          v                       v
 Azure AI Search             Azure OpenAI
          |
          |
    +-----+------+
    |            |
    v            v
Vector Index   Text Index
    |
    v
Blob Storage
    |
    v
Documents

Supporting services:

Azure Key Vault
Azure Monitor
Application Insights
Azure Service Bus
Microsoft Entra ID

44. Monitoring

Use Azure monitoring tools to observe:

Search latency
Request count
Errors
Throttling
Application latency
AI model latency
Token usage

Application Insights can help trace:

HTTP Request
     |
     v
Search Query
     |
     v
Azure OpenAI
     |
     v
Response

This is especially valuable when debugging slow RAG applications.

45. How to Design a Production RAG Pipeline

A recommended pipeline is:

1. Collect documents
        |
2. Extract text
        |
3. Clean text
        |
4. Split into chunks
        |
5. Generate embeddings
        |
6. Store chunks + vectors
        |
7. Build search index
        |
8. User asks question
        |
9. Retrieve candidates
        |
10. Hybrid search
        |
11. Semantic reranking
        |
12. Security filtering
        |
13. Build context
        |
14. Call Azure OpenAI
        |
15. Return answer + sources

46. Classic RAG vs Agentic Retrieval

Azure AI Search now also provides agentic retrieval capabilities.

Classic RAG generally follows:

Question
   |
   v
Search
   |
   v
Results
   |
   v
LLM

Agentic retrieval can use an LLM to help understand and decompose more complex queries before retrieval.

Microsoft currently documents agentic retrieval as a newer retrieval approach, with some capabilities available in preview depending on the API/features used. (Microsoft Learn)

For a first production implementation, classic hybrid RAG is often easier to understand, test, and operate.

47. Recommended Technology Stack

For a Microsoft-based enterprise application:

Frontend
    Angular

Backend
    ASP.NET Core Web API

Search
    Azure AI Search

LLM
    Azure OpenAI

Embeddings
    Azure OpenAI embedding model

Storage
    Azure Blob Storage

Database
    Azure SQL

Messaging
    Azure Service Bus

Secrets
    Azure Key Vault

Identity
    Microsoft Entra ID

Monitoring
    Application Insights
    Azure Monitor

Deployment
    Azure App Service / AKS

48. Complete Request Flow

Let's summarize the complete request:

                    USER
                      |
                      v
                Angular App
                      |
                      v
              ASP.NET Core API
                      |
                      v
               User Question
                      |
                      v
             Query Processing
                      |
             +--------+--------+
             |                 |
             v                 v
        Keyword Search    Vector Search
             |                 |
             +--------+--------+
                      |
                      v
                     RRF
                      |
                      v
              Semantic Ranking
                      |
                      v
              Security Filtering
                      |
                      v
              Top Relevant Chunks
                      |
                      v
                 Prompt Builder
                      |
                      v
                Azure OpenAI
                      |
                      v
                Final Answer
                      |
                      v
             Answer + Citations
                      |
                      v
                    USER

49. Key Interview Questions

If you are preparing for a .NET/Azure interview, these are important questions.

Q1. What is Azure AI Search?

A managed Azure service for information retrieval supporting full-text, vector, hybrid, semantic, and generative-AI search scenarios.

Q2. What is vector search?

Searching for documents based on vector similarity rather than only exact keywords.

Q3. What is hybrid search?

Combining keyword/full-text and vector search in the same request and merging their rankings using RRF. (Microsoft Learn)

Q4. What is semantic ranking?

A second-stage relevance process that reranks an initial result set using language understanding models. (Microsoft Learn)

Q5. Why use Azure AI Search with Azure OpenAI?

Azure AI Search retrieves enterprise-specific information, while Azure OpenAI generates a natural-language response based on that retrieved context.

Q6. What is RAG?

Retrieval-Augmented Generation: retrieve relevant external knowledge and provide it to an LLM as context for generating an answer.

Q7. Why chunk documents?

To retrieve smaller, relevant pieces of information instead of sending entire documents to the LLM.

Q8. What is RRF?

Reciprocal Rank Fusion is used to combine rankings from multiple search result sets, including hybrid search results. (Microsoft Learn)

Q9. How do you secure Azure AI Search?

Use Microsoft Entra ID, managed identities, RBAC, and security-aware filtering/authorization appropriate to the application.

Q10. How would you integrate Azure AI Search into a .NET microservices architecture?

Use a dedicated search/indexing component, publish domain events through messaging when data changes, and update the search index asynchronously.

50. Final Takeaway

Azure AI Search is much more than a traditional search engine.

It can provide:

Traditional Search
       +
Vector Search
       +
Hybrid Search
       +
Semantic Ranking
       +
RAG Retrieval
       +
Enterprise Security

The most important architecture to remember is:

              YOUR DATA
                  |
                  v
          Azure AI Search
                  |
        +---------+---------+
        |                   |
   Keyword Search      Vector Search
        |                   |
        +---------+---------+
                  |
                 RRF
                  |
          Semantic Ranking
                  |
                  v
          Relevant Context
                  |
                  v
            Azure OpenAI
                  |
                  v
          Grounded Response

For a .NET developer, a particularly powerful enterprise stack is:

Angular
   ↓
ASP.NET Core
   ↓
Azure AI Search
   ↓
Azure OpenAI
   ↓
Azure SQL / Blob Storage
   ↓
Azure Service Bus
   ↓
Key Vault + Managed Identity
   ↓
Application Insights

This architecture can be used to build enterprise knowledge assistants, customer-support copilots, product search, document search, internal AI assistants, and RAG-based applications.

Microsoft also provides an official .NET RAG sample that combines Azure AI Search and Azure OpenAI, including document indexing and managed-identity-based authentication. (Microsoft Learn)

Official Microsoft References

In one sentence: Azure AI Search is the retrieval layer that helps your application find the right information, while Azure OpenAI is the generation layer that turns that information into a useful natural-language answer.

How to Migrate SQL Server Database from Local Server to Azure Cloud

2026-08-19T13:06:24.974+05:30

Complete Step-by-Step Guide for SQL Server to Azure Migration

Moving a SQL Server database from an on-premises/local server to Microsoft Azure is a common requirement when organizations want to modernize their infrastructure, improve scalability, increase availability, and reduce the operational effort involved in maintaining physical database servers.

In this article, we will walk through the complete migration process:

Local SQL Server → Azure Storage → Azure SQL → Application Connection

We will also discuss the different Azure SQL options, backup and restore, Azure Database Migration Service (DMS), validation, application connection-string changes, and post-migration activities.

1. What Are We Migrating?

Assume we currently have the following environment:

LOCAL / ON-PREMISES SERVER
        |
        | SQL Server
        |
        +-- CustomerDB
        +-- OrderDB
        +-- ProductDB
        |
        +-- ASP.NET Core Web API
        +-- Angular Application

We want to move the database to Azure:

                         AZURE CLOUD
                              |
                    +---------+---------+
                    |                   |
               Azure SQL          Azure Storage
                    |
              CustomerDB
              OrderDB
              ProductDB
                    |
              ASP.NET Core API
                    |
                 Angular

The migration does not necessarily mean simply copying the .mdf and .ldf files to Azure.

The correct migration approach depends on the Azure target you choose.

2. Azure SQL Deployment Options

Before migrating, the first and most important decision is:

Where should the SQL Server database run in Azure?

There are three major options.

Option 1 – Azure SQL Database

Azure SQL Database is a fully managed database platform.

Microsoft manages many infrastructure tasks such as:

Hardware
Operating system
Database patching
Backups
High availability
Infrastructure maintenance

It is generally the best choice when your application can work with a database-as-a-service model.

ASP.NET Core API
       |
       v
Azure SQL Database
       |
       v
Managed by Azure

However, Azure SQL Database is not identical to a traditional SQL Server instance, so compatibility should be assessed before migration.

3. Option 2 – Azure SQL Managed Instance

Azure SQL Managed Instance provides much greater compatibility with traditional SQL Server.

It is particularly useful when your existing application depends on SQL Server instance-level capabilities or you want a more lift-and-shift-oriented migration.

For example:

ON-PREMISES

SQL Server Instance
       |
       +-- Database A
       +-- Database B
       +-- Database C
       |
       +-- Logins
       +-- SQL Agent Jobs
       +-- Cross-database functionality

can be moved toward:

AZURE

SQL Managed Instance
       |
       +-- Database A
       +-- Database B
       +-- Database C

Microsoft describes SQL Managed Instance as a suitable target when you need maximum compatibility and when your applications depend on instance-level or cross-database functionality. (Microsoft Learn)

4. Option 3 – SQL Server on Azure Virtual Machine

This is the closest approach to your existing environment.

You create an Azure VM and install/run SQL Server on it.

Azure
 |
 +-- Virtual Network
      |
      +-- SQL Server VM
             |
             +-- SQL Server
                    |
                    +-- Database

This approach is useful when you need very high compatibility with the existing SQL Server environment.

It is essentially a lift-and-shift approach.

Microsoft's current migration tooling supports SQL Server migrations to SQL Server on Azure VMs using Azure Database Migration Service. (Microsoft Learn)

5. Which Azure Option Should You Choose?

A simplified decision table is:

Requirement	Recommended Target
Fully managed database	Azure SQL Database
High SQL Server compatibility	Azure SQL Managed Instance
Almost identical SQL Server environment	SQL Server on Azure VM
Cross-database functionality	Managed Instance / Azure VM
SQL Agent and instance-level dependencies	Managed Instance / Azure VM
Modern cloud-native application	Azure SQL Database
Lift-and-shift migration	Managed Instance / Azure VM

Do not choose the target only because it is easy to create.

First perform a compatibility and dependency assessment.

6. Migration Architecture

A typical migration architecture looks like this:

                 ON-PREMISES
              +----------------+
              | Local SQL      |
              | Server         |
              |                |
              | CustomerDB     |
              | OrderDB        |
              +-------+--------+
                      |
                      | Backup
                      v
              +---------------+
              | Azure Storage |
              | Blob          |
              +-------+-------+
                      |
                      v
              +---------------+
              | Azure Database|
              | Migration     |
              | Service       |
              +-------+-------+
                      |
                      v
              +---------------+
              | Azure SQL     |
              | Managed       |
              | Instance /    |
              | SQL Database  |
              +-------+-------+
                      |
                      v
              ASP.NET Core API
                      |
                      v
                 Angular App

For some migration scenarios, Azure DMS uses database backup files stored in Azure Storage. (Microsoft Learn)

7. Step 1 – Analyze the Existing SQL Server

Before moving anything, collect information about your existing SQL Server.

Check:

SQL Server version
SQL Server edition
Database size
Number of databases
Database compatibility level
SQL Agent jobs
Logins
Users
Linked servers
Stored procedures
Functions
Views
Triggers
SSIS packages
Cross-database queries
CLR dependencies
Encryption
External dependencies
Application connection strings

For example:

SELECT
    name,
    database_id,
    compatibility_level,
    state_desc
FROM sys.databases
ORDER BY name;

Check database size:

SELECT
    DB_NAME(database_id) AS DatabaseName,
    SUM(size) * 8 / 1024 AS SizeMB
FROM sys.master_files
GROUP BY database_id
ORDER BY SizeMB DESC;

This gives you an initial understanding of your environment.

8. Step 2 – Perform a Compatibility Assessment

This step is extremely important.

Do not immediately migrate a production database.

First determine whether your existing SQL Server features are supported by the Azure target.

Microsoft's current SSMS migration functionality can assess a SQL Server instance and recommend Azure SQL targets such as:

Azure SQL Database
Azure SQL Managed Instance
SQL Server on Azure VM

It can also identify compatibility considerations before migration. (Microsoft Learn)

A typical process is:

Discover
   |
   v
Assess
   |
   v
Identify blockers
   |
   v
Fix compatibility issues
   |
   v
Migrate

9. Step 3 – Create an Azure Account

Open the Azure Portal:

https://portal.azure.com

You need an Azure subscription.

10. Step 4 – Create a Resource Group

A Resource Group is a logical container for Azure resources.

For example:

Resource Group:
rg-production-database

You may place resources such as:

rg-production-database
 |
 +-- Azure SQL
 +-- Storage Account
 +-- Database Migration Service
 +-- Key Vault
 +-- Monitoring resources

For production environments, use a naming convention appropriate for your organization.

Example:

rg-prod-sql-eastus
rg-prod-storage-eastus

11. Step 5 – Create Azure Storage

Azure Storage can be used as an intermediate location for database backup files.

Create:

Storage Account
      |
      +-- Blob Container
             |
             +-- sqlbackups

For example:

https://mystorageaccount.blob.core.windows.net/sqlbackups/

You can place backup files here:

CustomerDB.bak
OrderDB.bak
ProductDB.bak

For migration projects using Azure DMS, Microsoft recommends using a dedicated storage account for migration-related backup files rather than sharing it with unrelated workloads. (Microsoft Learn)

12. Step 6 – Take a Full SQL Server Backup

On your local SQL Server, take a full backup.

Example:

BACKUP DATABASE CustomerDB
TO DISK = 'D:\SQLBackups\CustomerDB.bak'
WITH
    INIT,
    COMPRESSION,
    STATS = 10;

Verify that the backup completed successfully.

You can check backup history:

SELECT
    database_name,
    backup_start_date,
    backup_finish_date,
    type,
    backup_size
FROM msdb.dbo.backupset
WHERE database_name = 'CustomerDB'
ORDER BY backup_finish_date DESC;

13. Step 7 – Upload the Backup to Azure Storage

Upload:

CustomerDB.bak

to:

Azure Storage
     |
     +-- sqlbackups
           |
           +-- CustomerDB.bak

You can use Azure Storage tools or Azure-supported migration workflows.

For large databases, uploading the backup can take significant time, so estimate the transfer duration before scheduling production migration.

14. Step 8 – Create the Azure SQL Target

Now create the target.

For example, if you selected Azure SQL Managed Instance:

Azure Portal
    |
    +-- Azure SQL
          |
          +-- SQL Managed Instance

Configure:

Subscription
Resource Group
Region
Instance name
Compute
Storage
Networking
Authentication
Security settings

For Azure SQL Database, create:

Azure SQL Server
       |
       +-- Azure SQL Database

For SQL Server on Azure VM:

Azure VM
   |
   +-- SQL Server

15. Step 9 – Configure Networking

This is one of the most commonly overlooked parts of a database migration.

Your application must be able to reach the Azure database.

Typical architecture:

                    Internet
                       |
                       v
                Azure Front Door
                       |
                       v
                 Application
                       |
                       v
                ASP.NET Core API
                       |
                       v
                 Private Network
                       |
                       v
                Azure SQL

For production systems, consider:

Virtual Network
Private Endpoint / Private networking
Network Security Groups
Firewall rules
DNS configuration
VPN
ExpressRoute where required

Avoid exposing a production database unnecessarily to the public internet.

16. Step 10 – Choose Your Migration Method

There are several approaches.

Method A – BACPAC

Useful for certain Azure SQL Database migration scenarios.

Local SQL Server
       |
       | Export
       v
    .bacpac
       |
       v
Azure SQL Database

Azure SQL supports importing a BACPAC file into Azure SQL Database or SQL Managed Instance. (Microsoft Learn)

17. Method B – Backup and Restore

This is a traditional SQL Server migration approach.

Local SQL Server
       |
       | .bak
       v
Azure Storage
       |
       v
Azure SQL Managed Instance

This approach is especially useful for SQL Managed Instance.

Microsoft supports native backup/restore of SQL Server backups stored in Azure Storage for SQL Managed Instance. (Microsoft Learn)

Example:

RESTORE DATABASE CustomerDB
FROM URL =
'https://mystorageaccount.blob.core.windows.net/sqlbackups/CustomerDB.bak';

The exact credential and restore configuration depends on your target and storage authentication method.

18. Method C – Azure Database Migration Service

Azure Database Migration Service, or Azure DMS, is designed specifically for database migration scenarios.

The general workflow is:

Source SQL Server
       |
       v
Azure Database Migration Service
       |
       v
Azure SQL Target

Microsoft currently provides DMS migration paths for SQL Server to:

Azure SQL Database
Azure SQL Managed Instance
SQL Server on Azure VM. (Microsoft Learn)

19. Step 11 – Create Azure Database Migration Service

In the Azure Portal:

Portal
  |
  +-- Search
       |
       +-- Azure Database Migration Service

Select:

Create

Configure:

Subscription
Resource Group
Region
Service name
Networking

After deployment:

Azure Database Migration Service
              |
              +-- New Migration

20. Step 12 – Configure Source SQL Server

Provide your source SQL Server information.

Example:

Source Server:
192.168.1.50

Database:
CustomerDB

Authentication:
SQL Authentication / Windows Authentication

The exact connectivity setup depends on your migration scenario and network architecture.

If backup files are located on an on-premises network share, some DMS scenarios require a self-hosted integration runtime so the migration service can access the source environment and backup files. (Microsoft Learn)

21. Step 13 – Select the Target

Select the Azure destination.

For example:

Source:
SQL Server

Target:
Azure SQL Managed Instance

Then select:

Subscription
Resource Group
Managed Instance
Target Database

22. Step 14 – Configure Backup Storage

Tell the migration service where the backup files are located.

Example:

Storage Account
     |
     +-- sqlbackups
           |
           +-- CustomerDB.bak
           +-- CustomerDB_Log.trn

Depending on the migration mode, full and subsequent transaction-log backups may be involved.

23. Step 15 – Start the Migration

Once configuration is complete:

Validate
   |
   v
Start Migration

Azure DMS will perform the migration according to the selected migration scenario.

Monitor:

Migration Status
        |
        +-- Starting
        +-- In Progress
        +-- Validating
        +-- Completed
        +-- Failed

Microsoft's DMS documentation describes monitoring the migration from the DMS monitoring experience. (Microsoft Learn)

24. Step 16 – Monitor Migration

Do not immediately switch your application to Azure.

First verify the migration.

Check:

Database Status
Data Size
Tables
Indexes
Stored Procedures
Functions
Views
Triggers
Users
Permissions

Run:

SELECT
    name,
    state_desc
FROM sys.databases;

Then connect to the Azure target using SSMS.

25. Step 17 – Validate the Data

This is one of the most important steps.

Compare:

LOCAL DATABASE
       vs
AZURE DATABASE

Check table counts:

SELECT
    t.name AS TableName,
    SUM(p.rows) AS RowCount
FROM sys.tables t
INNER JOIN sys.partitions p
    ON t.object_id = p.object_id
WHERE p.index_id IN (0,1)
GROUP BY t.name
ORDER BY t.name;

Run the same query against the Azure database.

Compare:

Customer
Local: 1,250,000
Azure: 1,250,000

Orders
Local: 8,500,000
Azure: 8,500,000

Also validate:

Primary keys
Foreign keys
Indexes
Constraints
Stored procedures
Functions
Views
Triggers
Permissions

26. Step 18 – Test Application Connectivity

Suppose your existing ASP.NET Core application has:

{
  "ConnectionStrings": {
    "DefaultConnection": "Server=LOCALSERVER;Database=CustomerDB;Trusted_Connection=True;"
  }
}

After migration, the application must connect to the Azure database.

The new connection string depends on your selected Azure service and authentication method.

Conceptually:

{
  "ConnectionStrings": {
    "DefaultConnection": "Server=<AZURE-SQL-SERVER>;Database=CustomerDB;..."
  }
}

Do not hard-code production passwords into source code.

For production applications, consider Azure Key Vault and managed identity-based authentication where appropriate.

27. Step 19 – Update ASP.NET Core Configuration

For example:

builder.Services.AddDbContext<ApplicationDbContext>(options =>
    options.UseSqlServer(
        builder.Configuration.GetConnectionString("DefaultConnection")));

The application code does not necessarily need to change.

Usually, the major change is the database connection configuration.

This is one of the major advantages of using Entity Framework Core with SQL Server-compatible Azure targets.

28. Step 20 – Test CRUD Operations

After changing the connection string, test:

Create

Create Customer

Read

Get Customer

Update

Update Customer

Delete

Delete Customer

Also test:

Login
Search
Reports
Transactions
Batch jobs
Background services
Stored procedures
File uploads
Notifications

29. Step 21 – Performance Testing

Do not assume that migration automatically means better performance.

Measure:

Before Migration
----------------
API Response: 250 ms
DB Query: 100 ms

After Migration
---------------
API Response: 180 ms
DB Query: 70 ms

or identify queries that became slower.

Check:

CPU
Memory
DTU/vCore usage depending on Azure SQL offering
Query duration
Blocking
Deadlocks
Index usage
Database waits
Connection pool usage

Use Azure monitoring capabilities and SQL performance tools to investigate performance.

30. Step 22 – Security Configuration

Production databases should be secured properly.

Consider:

Azure SQL
   |
   +-- Firewall
   +-- Private Endpoint
   +-- Microsoft Entra authentication
   +-- Managed Identity
   +-- Encryption
   +-- Auditing
   +-- Defender for SQL
   +-- Key Vault

Never publish credentials in:

GitHub
appsettings.json
Source Code
Dockerfile
Azure DevOps YAML

For production, use secure secret-management mechanisms.

31. Step 23 – Configure Backups

One major advantage of Azure SQL services is that Azure provides managed backup capabilities.

However, you should still understand:

Backup
   |
   +-- Retention
   +-- Point-in-time restore
   +-- Long-term retention
   +-- Disaster recovery

Your backup strategy should match your organization's Recovery Point Objective (RPO) and Recovery Time Objective (RTO).

32. Step 24 – Configure Monitoring

Use Azure monitoring capabilities to observe your database.

Typical monitoring architecture:

Azure SQL
   |
   +-- Azure Monitor
   |
   +-- Log Analytics
   |
   +-- Application Insights
   |
   +-- Alerts

Create alerts for important conditions such as:

High CPU
High storage usage
Connection failures
Long-running queries
Availability problems

33. Step 25 – Application Cutover

Once testing is successful, perform the final cutover.

A typical production cutover is:

1. Notify users
        |
2. Stop application writes
        |
3. Complete final synchronization
        |
4. Validate Azure database
        |
5. Update connection string
        |
6. Deploy application
        |
7. Start application
        |
8. Perform smoke testing
        |
9. Monitor

For migration methods that maintain synchronization, Microsoft recommends ensuring the target is synchronized and validated before switching application traffic. (Microsoft Learn)

34. Example Production Migration

Suppose we have:

LOCAL SERVER

Server: SQLSERVER01

Database:
SalesDB

Size:
500 GB

Application:
ASP.NET Core Web API

Frontend:
Angular

We want:

AZURE

Azure SQL Managed Instance

Database:
SalesDB

The migration architecture becomes:

                 ON-PREMISES
                     |
                     |
              SQLSERVER01
                     |
                  SalesDB
                     |
                  Backup
                     |
                     v
              Azure Storage
                     |
                     v
        Azure Database Migration
                 Service
                     |
                     v
          Azure SQL Managed
                Instance
                     |
                     v
              ASP.NET Core API
                     |
                     v
                 Angular

35. Complete Migration Flow

The complete process can be summarized as:

              LOCAL SQL SERVER
                     |
                     v
             Database Assessment
                     |
                     v
             Compatibility Check
                     |
                     v
              Choose Azure Target
                     |
          +----------+----------+
          |          |          |
          v          v          v
       SQL DB       MI       Azure VM
          |          |          |
          +----------+----------+
                     |
                     v
              Create Azure
              Infrastructure
                     |
                     v
             Create Storage
                     |
                     v
              Take Backup
                     |
                     v
          Upload / Configure
              Backup Storage
                     |
                     v
            Azure DMS / Restore
                     |
                     v
             Migration Complete
                     |
                     v
             Validate Database
                     |
                     v
             Test Application
                     |
                     v
             Performance Test
                     |
                     v
                Cutover
                     |
                     v
              Azure Production

36. Common Migration Problems

Problem 1 – Unsupported SQL Server Feature

Your local SQL Server may use a feature that isn't available on the selected Azure target.

Solution

Perform compatibility assessment before migration.

Problem 2 – Application Cannot Connect

Possible causes:

Firewall
Networking
DNS
Authentication
Connection String
Private Endpoint
Credentials

Check each layer systematically.

Problem 3 – Database Is Very Large

For a very large database, BACPAC export/import may not be the best approach.

Consider:

Azure Database Migration Service
Backup/Restore
Managed Instance Link
Log Replay Service

depending on your target and downtime requirements.

Microsoft documents multiple migration paths for SQL Managed Instance, including DMS, Managed Instance link, Log Replay Service, and native restore. (Microsoft Learn)

37. What About Minimal Downtime?

Suppose your database is:

2 TB

and your application cannot be stopped for several hours.

A simple:

Backup
   ↓
Upload
   ↓
Restore

may result in too much downtime.

Instead, consider a migration method that supports ongoing synchronization/replication, depending on your target.

The architecture becomes:

LOCAL SQL SERVER
       |
       | Initial migration
       v
AZURE DATABASE
       ^
       |
       | Continuous changes
       |
LOCAL SQL SERVER

Then:

Stop application
      |
      v
Final synchronization
      |
      v
Validate target
      |
      v
Change connection string
      |
      v
Start application

This can significantly reduce the final cutover window.

For SQL Managed Instance, Microsoft documents DMS, Managed Instance link, and Log Replay Service for migration scenarios designed to reduce downtime. (Microsoft Learn)

38. What Happens to SQL Agent Jobs?

This depends heavily on your target.

If you move to SQL Server on Azure VM, you retain a traditional SQL Server environment.

With SQL Managed Instance, many SQL Server instance-level capabilities are supported, including SQL Agent-related workloads, although you should still assess individual dependencies.

With Azure SQL Database, you should not assume that every SQL Server instance-level feature will move directly.

Therefore:

SQL Server Agent Jobs
        |
        +-- Identify
        |
        +-- Assess
        |
        +-- Recreate / redesign where required

39. What Happens to Logins and Users?

Database users and server-level logins are different concepts.

You should inventory:

Server Logins
Database Users
Roles
Permissions
Application Accounts
Service Accounts

After migration, verify that every application identity has the correct permissions.

Do not simply grant:

db_owner

to every application account.

Use the minimum permissions required.

40. What Happens to Stored Procedures?

Stored procedures generally migrate well when supported by the selected target.

After migration, execute important procedures and compare results.

For example:

EXEC dbo.GetCustomerOrders
     @CustomerId = 1001;

Compare the results between:

Local SQL Server
vs
Azure SQL

41. What Happens to SQL Server Jobs and SSIS?

Create an inventory before migration.

SQL Agent Jobs
SSIS Packages
Linked Servers
Maintenance Plans
Database Mail
Cross Database Queries
CLR
Replication

Some workloads may need redesign when moving to a fully managed Azure SQL platform.

This is one reason why assessment before migration is critical.

42. Rollback Strategy

Never perform a production database migration without a rollback plan.

Before cutover:

LOCAL DATABASE
       |
       +-- Final backup
       |
       +-- Keep source available

If something goes wrong:

Azure
  |
  X
  |
Rollback
  |
  v
Local SQL Server

Do not immediately delete the original database after migration.

Keep the source environment available until:

Application validated
Data validated
Business users approved
Performance validated
Monitoring stable
Rollback window completed

43. Production Migration Checklist

Before migration:

☐ Inventory SQL Server
☐ Check database size
☐ Check SQL Server version
☐ Assess compatibility
☐ Identify dependencies
☐ Identify SQL Agent jobs
☐ Identify logins/users
☐ Identify linked servers
☐ Choose Azure target
☐ Create Azure subscription/resource group
☐ Configure networking
☐ Create target database/server
☐ Configure Azure Storage
☐ Take backup
☐ Test restore

During migration:

☐ Start migration
☐ Monitor migration
☐ Check errors
☐ Verify database state
☐ Compare row counts
☐ Validate schema
☐ Validate indexes
☐ Validate users
☐ Validate permissions

Before cutover:

☐ Test application
☐ Test APIs
☐ Test reports
☐ Test transactions
☐ Test performance
☐ Prepare rollback
☐ Notify stakeholders

After cutover:

☐ Update connection string
☐ Deploy application
☐ Smoke test
☐ Monitor database
☐ Monitor API
☐ Monitor errors
☐ Monitor performance
☐ Obtain business approval

44. Recommended Architecture for a .NET + Angular Application

For a modern application, the architecture could look like:

                       USERS
                         |
                         v
                    Angular SPA
                         |
                         v
                Azure Front Door
                         |
                         v
                Azure API Management
                         |
                         v
                 ASP.NET Core APIs
                         |
              +----------+----------+
              |                     |
              v                     v
       Azure Service Bus       Azure SQL
              |               Managed Instance
              |                     |
              v                     v
       Background Workers       Database
              |
              v
       External Services

Additional Azure services can be introduced based on requirements:

Azure Key Vault
Azure Monitor
Application Insights
Azure Storage
Azure Container Registry
AKS / App Service
Azure Functions

45. Azure SQL Database vs Managed Instance vs Azure VM

Feature	Azure SQL Database	SQL Managed Instance	SQL Server Azure VM
Fully managed	Yes	Yes	No
SQL Server compatibility	Moderate	High	Very High
OS management	Azure	Azure	Customer
SQL Server instance access	Limited	High	Full
Lift-and-shift	Sometimes	Excellent	Excellent
Cross-database scenarios	Limited compared with MI	Strong	Strong
SQL Agent	Not traditional SQL Agent	Supported scenarios	Full
Infrastructure control	Low	Medium	High
Administration effort	Lowest	Medium	Highest

Always verify the current feature support for your exact SQL Server version and Azure target before production migration.

46. Recommended Approach

For a typical enterprise application currently running:

ASP.NET Core
Angular
SQL Server
Azure Services

I recommend this decision process:

                 Existing SQL Server
                         |
                         v
                Compatibility Assessment
                         |
             +-----------+-----------+
             |                       |
       Minimal changes          Cloud modernization
             |                       |
             v                       v
     SQL Managed Instance       Azure SQL Database
             |
             |
       Very high SQL
       compatibility
             |
             v
     SQL Server Azure VM

For a large existing SQL Server environment with many instance-level dependencies, Azure SQL Managed Instance is often a strong candidate.

For a modern application that can adapt to Azure SQL Database capabilities, Azure SQL Database can reduce infrastructure administration.

For applications requiring maximum control and near-traditional SQL Server behavior, SQL Server on Azure VM is the closest lift-and-shift option.

47. Final Takeaway

Migrating SQL Server to Azure is not simply:

Copy Database
      ↓
Azure

A successful migration is:

ASSESS
   ↓
PLAN
   ↓
CHOOSE TARGET
   ↓
PREPARE AZURE
   ↓
BACKUP / MIGRATE
   ↓
VALIDATE
   ↓
TEST
   ↓
CUTOVER
   ↓
MONITOR
   ↓
OPTIMIZE

The most important rule is:

Never migrate a production database directly without first performing compatibility assessment, testing the migration, validating the data, and preparing a rollback plan.

Azure provides several migration approaches, and the correct choice depends on the target platform, database size, SQL Server features, network architecture, and acceptable downtime. Microsoft currently documents Azure DMS, native backup/restore, Managed Instance link, and Log Replay Service among the available migration approaches for relevant SQL Server-to-Azure scenarios. (Microsoft Learn)

Official Microsoft References

Retrieval-Augmented Generation (RAG): Complete Guide with Architecture, Real-Time Examples and .NET Implementation

2026-08-18T06:16:48.212+05:30

Introduction

Large Language Models (LLMs) such as GPT, Gemini, Claude, and other generative AI models can understand questions and generate remarkably useful answers. However, an LLM has an important limitation:

An LLM does not automatically know your organization's private, frequently changing, or newly created information.

For example, imagine a company has:

10,000 internal documents
HR policies
Product manuals
Customer records
Technical documentation
Financial reports
Support tickets
Project documents
Frequently changing business data

You could train or fine-tune a model on some of this information, but that can be expensive and does not solve the problem of constantly changing information.

This is where RAG — Retrieval-Augmented Generation becomes extremely useful.

RAG allows an AI application to:

Receive a user's question.
Search an external knowledge source.
Retrieve the most relevant information.
Give that information to an LLM.
Generate an answer grounded in the retrieved information.

In simple terms:

RAG = Search for relevant knowledge + Give it to the LLM + Generate an answer

1. What is RAG?

RAG stands for:

Retrieval-Augmented Generation

It combines two major capabilities:

Retrieval

Find relevant information from an external knowledge source.

Generation

Use an LLM to generate a natural-language answer using that retrieved information.

A simplified representation is:

User Question
      |
      v
   Retriever
      |
      v
Relevant Documents
      |
      v
   LLM / GPT
      |
      v
Generated Answer

For example:

User asks:

"What is our company's leave policy for employees with more than 5 years of service?"

The LLM itself may not know your company's policy.

A RAG system searches your company's HR documents, finds the relevant policy, and passes it to the LLM.

The LLM then answers:

"According to the company's leave policy, employees with more than five years of service are eligible for ..."

The important part is that the answer is based on your organization's data.

2. Who Invented RAG?

RAG was not created as a commercial product by a single company.

The term and a well-known formal RAG architecture were introduced in the research paper:

"Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks"

published in 2020.

The paper was authored by:

Patrick Lewis
Ethan Perez
Aleksandra Piktus
Fabio Petroni
Vladimir Karpukhin
Naman Goyal
Heinrich Küttler
Mike Lewis
Wen-tau Yih
Tim Rocktäschel
Sebastian Riedel
Douwe Kiela

The paper described RAG models that combine a pretrained sequence-to-sequence model with a dense vector index used as external non-parametric memory.

The original work was associated with the Facebook AI Research ecosystem, now part of Meta AI.

However, it is important to understand that the broader idea of retrieving external knowledge and combining it with language models existed before the 2020 RAG paper. For example, Google's REALM research also explored retrieval-augmented language modeling in 2020.

Therefore:

RAG is a research architecture/pattern, not a programming language or a single software product.

3. In Which Programming Language Was RAG Developed?

This is one of the most common misconceptions.

RAG is not a programming language.

It is an AI architecture/pattern.

You can implement RAG using many programming languages.

Common choices include:

Language	Typical Usage
Python	AI/ML, RAG experimentation, LangChain, LlamaIndex
C#	Enterprise .NET applications
Java	Enterprise applications
JavaScript/TypeScript	Node.js applications
Go	High-performance backend services
C++	High-performance AI infrastructure

Python is particularly popular in AI research because of its extensive machine-learning ecosystem.

But a company building an enterprise application using:

ASP.NET Core
Angular
Azure
SQL Server

can implement RAG using C#/.NET.

4. Why Was RAG Needed?

Traditional LLM architecture looks like this:

User
 |
 v
LLM
 |
 v
Answer

The model relies primarily on knowledge encoded in its parameters.

This creates several problems.

Problem 1 — Private Data

Suppose your company has:

EmployeePolicy.pdf
ProductManual.pdf
CustomerSupport.pdf
Architecture.docx
ProjectDocumentation.pdf

The public LLM does not automatically know these documents.

Problem 2 — Frequently Changing Data

Imagine asking:

"What is today's product inventory?"

The answer may change every hour.

You don't want to retrain an LLM every time inventory changes.

Problem 3 — Hallucination

An LLM can sometimes generate information that sounds convincing but is incorrect.

RAG can reduce this risk by supplying relevant source information to the model.

However:

RAG does not completely eliminate hallucinations.

Research continues to show that insufficient or poor-quality retrieved context can still cause incorrect answers.

5. Main Purpose of RAG

The primary purpose of RAG is:

To allow an LLM to use external, relevant and potentially up-to-date knowledge while generating an answer.

This provides several benefits:

1. Access private information

Example:

Company HR Documents
Company Technical Documents
Company Product Documents

2. Access frequently changing information

Example:

Inventory
Prices
Policies
News
Tickets
Orders

3. Reduce hallucination

The model can use retrieved evidence instead of relying entirely on its internal knowledge.

4. Provide source references

A well-designed RAG application can show:

Source:
Employee_Leave_Policy.pdf
Page 12

5. Avoid retraining for every document update

Instead of retraining the LLM whenever a document changes:

Update Document
      |
      v
Update Knowledge Index
      |
      v
RAG uses new information

6. RAG Architecture

A typical RAG system looks like this:

                  ┌──────────────────┐
                  │     User         │
                  └────────┬─────────┘
                           |
                           v
                  ┌──────────────────┐
                  │ User Question    │
                  └────────┬─────────┘
                           |
                           v
                  ┌──────────────────┐
                  │ Query Processing │
                  └────────┬─────────┘
                           |
                           v
                  ┌──────────────────┐
                  │ Embedding Model  │
                  └────────┬─────────┘
                           |
                           v
                  ┌──────────────────┐
                  │ Vector Database  │
                  └────────┬─────────┘
                           |
                           v
                  ┌──────────────────┐
                  │ Relevant Chunks  │
                  └────────┬─────────┘
                           |
                           v
                  ┌──────────────────┐
                  │ Prompt + Context │
                  └────────┬─────────┘
                           |
                           v
                  ┌──────────────────┐
                  │       LLM        │
                  └────────┬─────────┘
                           |
                           v
                  ┌──────────────────┐
                  │ Final Answer     │
                  └──────────────────┘

7. The Two Major Parts of RAG

A RAG system generally has two major workflows:

A. Data Ingestion

This happens before users ask questions.

Documents
   |
   v
Document Extraction
   |
   v
Chunking
   |
   v
Embeddings
   |
   v
Vector Database

B. Query Processing

This happens when the user asks a question.

User Question
   |
   v
Embedding
   |
   v
Vector Search
   |
   v
Relevant Chunks
   |
   v
LLM
   |
   v
Answer

Understanding these two pipelines is essential for understanding RAG.

8. What Is an Embedding?

An embedding converts text into a numerical representation called a vector.

For example:

"How can I reset my password?"

might be represented conceptually as:

[0.21, -0.45, 0.73, 0.11, ...]

Real embedding vectors can contain hundreds or thousands of dimensions depending on the embedding model.

The important concept is:

Similar meanings produce vectors that are close to each other in vector space.

For example:

"How do I change my password?"

and:

"What is the procedure for resetting my password?"

have different words but similar meaning.

Their embeddings should therefore be semantically similar.

9. Why Do We Need a Vector Database?

Suppose you have:

1,000 documents
10,000 documents
1 million documents

Searching all the text directly for every question can become inefficient.

A vector database stores embeddings and allows similarity searches.

Common technologies include:

Azure AI Search
PostgreSQL with pgvector
Elasticsearch
OpenSearch
Pinecone
Weaviate
Milvus
Qdrant
Chroma
Redis with vector search capabilities

The exact technology depends on your architecture and requirements.

10. What Is a Vector Database?

A vector database stores data such as:

Document ID
Chunk ID
Text
Embedding
Metadata

Example:

DocumentId:
EMP001

ChunkId:
EMP001-CHUNK-12

Text:
Employees are eligible for 20 days of annual leave...

Embedding:
[0.21, 0.52, -0.11, ...]

Metadata:
Department = HR
DocumentType = Policy
Year = 2026

11. Document Chunking

One of the most important steps in RAG is chunking.

Suppose a PDF contains 100 pages.

You generally should not send the entire PDF to the LLM for every question.

Instead, divide it into smaller pieces.

For example:

Document
   |
   +---- Chunk 1
   |
   +---- Chunk 2
   |
   +---- Chunk 3
   |
   +---- Chunk 4
   |
   +---- Chunk 5

A chunk might contain:

500–1000 tokens

The exact size should be determined experimentally based on the document type and retrieval quality.

12. Chunk Overlap

Sometimes important information crosses chunk boundaries.

For example:

Chunk 1:
Employees are eligible for annual leave after completing...

Chunk 2:
...one year of continuous service.

If there is no overlap, retrieval may lose context.

Therefore, systems may use overlapping chunks.

Example:

Chunk 1
---------------------
A B C D E F G H

Chunk 2
              E F G H I J K L

The overlap improves the chance that related information remains together.

13. Metadata

Metadata is extremely important in enterprise RAG.

Example:

{
  "documentId": "HR-2026-001",
  "department": "HR",
  "documentType": "LeavePolicy",
  "year": 2026,
  "region": "India"
}

Metadata allows filtering.

For example:

"Search only HR documents from 2026."

Instead of searching the entire knowledge base:

Vector Search
+
Department = HR
+
Year = 2026

This is called metadata filtering.

14. End-to-End RAG Pipeline

Let's understand the complete process.

Step 1 — Upload Documents

Example:

HRPolicy.pdf

Step 2 — Extract Text

The system extracts text from:

PDF
DOCX
TXT
HTML
Web pages
Database

For scanned documents, OCR may be required.

Step 3 — Chunk the Text

Example:

HRPolicy.pdf

       |
       +-- Chunk 1
       +-- Chunk 2
       +-- Chunk 3
       +-- Chunk 4

Step 4 — Generate Embeddings

Each chunk is converted into a vector.

Chunk 1
   |
Embedding Model
   |
Vector

Step 5 — Store in Vector Database

Vector
+
Text
+
Metadata

is stored.

15. Query-Time Process

Now the user asks:

"How many annual leave days can I take?"

The system performs:

Question
   |
   v
Embedding
   |
   v
Vector Search
   |
   v
Top Relevant Chunks

Suppose the database returns:

Chunk 12
Chunk 27
Chunk 31

These are passed to the LLM.

16. Prompt Augmentation

The application creates a prompt similar to:

You are an HR assistant.

Answer the question using only the provided context.

Context:
--------------------
Employees are entitled to 20 days
of annual leave per calendar year.

Leave must be requested through
the employee portal.

Question:
How many annual leave days can I take?

The LLM then generates:

Employees are entitled to 20 days
of annual leave per calendar year.

This is the generation part of RAG.

17. RAG vs Traditional LLM

Feature	Traditional LLM	RAG
General knowledge	Yes	Yes
Private company data	Limited	Yes
Dynamic data	Limited	Yes
External documents	Not automatically	Yes
Knowledge updates	Model-dependent	Update knowledge source/index
Source citations	Not guaranteed	Can be implemented
Hallucination risk	Exists	Can be reduced
Retraining required for every document	No/depends	Usually no
Enterprise knowledge assistant	Limited	Excellent fit

18. RAG vs Fine-Tuning

This is one of the most important concepts.

Fine-Tuning

Fine-tuning changes model behavior/weights using training examples.

Useful for:

Style
Behavior
Task specialization
Output format
Domain-specific behavior

RAG

RAG provides external knowledge at query time.

Useful for:

Private documents
Current information
Frequently changing information
Knowledge bases
Company policies
Product documentation

A simple rule:

Use RAG to give the model knowledge.

Use fine-tuning to change how the model behaves.

Sometimes enterprises use both.

19. Real-Time Example #1 — Company HR Assistant

Imagine a company has:

Employee Handbook
Leave Policy
Travel Policy
Insurance Policy
Work From Home Policy
Salary Policy

An employee asks:

"How many work-from-home days can I take?"

RAG:

User
 |
 v
Question
 |
 v
Embedding
 |
 v
Vector Search
 |
 v
HR Documents
 |
 v
Relevant Policy
 |
 v
LLM
 |
 v
Answer

The employee doesn't need to manually search hundreds of pages.

20. Real-Time Example #2 — Customer Support

Suppose an organization sells networking equipment.

Documents:

Router Manual
Switch Manual
Troubleshooting Guide
Warranty Policy
Installation Guide

Customer asks:

"My router is showing a red status light. What should I check?"

RAG retrieves the troubleshooting section.

The LLM generates a user-friendly answer based on the retrieved manual.

This is much better than asking the model to guess the troubleshooting procedure.

21. Real-Time Example #3 — Banking

A bank may have:

Loan Policy
Credit Card Policy
Interest Rate Policy
KYC Documentation
Account Rules
Product Terms

A customer asks:

"What documents are required for this loan?"

RAG retrieves the relevant policy.

The LLM summarizes it.

The application can also provide:

Source Document
Section
Page
Last Updated Date

This is particularly valuable for regulated environments.

22. Real-Time Example #4 — Software Development

Suppose your organization has:

Architecture Documents
API Documentation
Coding Standards
Database Documentation
Microservice Documentation
Deployment Documentation

A developer asks:

"How does the Customer Service communicate with the Order Service?"

RAG searches the architecture documentation.

It may retrieve:

Customer Service
      |
      v
Azure Service Bus
      |
      v
Order Service

The LLM can then explain the architecture.

23. Real-Time Example #5 — E-Commerce

Suppose an online store has:

Products
Prices
Inventory
Returns
Shipping Policies
Customer Orders

Customer asks:

"Can I return my order?"

RAG can retrieve the applicable return policy.

For dynamic information such as order status, the RAG application may also retrieve information directly from operational APIs or databases.

This leads to an important architecture:

LLM
 |
 +---- Vector Search
 |
 +---- SQL Database
 |
 +---- REST API
 |
 +---- Business Services

This is often more powerful than document-only RAG.

24. RAG With SQL Database

RAG does not mean everything has to be stored in a vector database.

Suppose you ask:

"How many orders did customer 10025 place last month?"

A vector database is not necessarily the right tool.

A better architecture may be:

User Question
      |
      v
Intent Detection
      |
      +------------------+
      |                  |
      v                  v
Document Search       SQL Query
      |                  |
      +--------+---------+
               |
               v
              LLM
               |
               v
            Answer

This is often called a hybrid/agentic architecture.

25. RAG + SQL Example

Question:

"What is our return policy?"

Use:

Vector Search

Question:

"How many orders were placed yesterday?"

Use:

SQL

Question:

"Why was customer 12345's order delayed?"

Potentially use:

SQL
+
Order API
+
Support tickets
+
RAG

The LLM can orchestrate the sources.

26. Semantic Search vs Keyword Search

Traditional search might search:

"password reset"

and look for exact words.

Semantic search understands meaning.

Question:

"I forgot my login credentials. How can I get back into my account?"

It can retrieve:

Password Reset Procedure

even though the exact phrase may not appear.

This is one of the major benefits of embeddings.

27. Hybrid Search

Modern enterprise RAG systems often combine:

Keyword Search
+
Vector Search

For example:

BM25 / keyword search
        +
Semantic vector search
        |
        v
Combined Results

Why?

Keyword search is excellent for exact identifiers such as:

INV-2026-00125
Customer ID 10045
Error E5001
API-123

Vector search is excellent for semantic meaning.

Combining both can improve retrieval quality.

28. Reranking

Retrieving the top 20 documents does not necessarily mean all 20 are equally relevant.

A reranker can evaluate them again.

Query
 |
 v
Retriever
 |
 v
Top 20 documents
 |
 v
Reranker
 |
 v
Top 5 documents
 |
 v
LLM

This can improve the quality of the context provided to the LLM.

29. Basic RAG Architecture for .NET

For your .NET ecosystem, an enterprise architecture could look like:

                    Angular
                       |
                       v
                 ASP.NET Core
                       |
             +---------+---------+
             |                   |
             v                   v
        RAG Service          Business APIs
             |
             v
      Embedding Service
             |
             v
       Azure AI Search
             |
             v
      Enterprise Documents
             |
             v
            LLM

Possible Azure components include:

Angular
   |
Azure App Service / Static Web Apps
   |
ASP.NET Core Web API
   |
Azure AI Search
   |
Azure OpenAI
   |
Blob Storage
   |
SQL Server / Azure SQL

The exact Azure services can vary depending on the application.

30. Simple C# RAG Flow

Conceptually:

public async Task<string> AskAsync(string question)
{
    var queryEmbedding =
        await embeddingService.CreateEmbeddingAsync(question);

    var documents =
        await vectorStore.SearchAsync(queryEmbedding, topK: 5);

    var context = string.Join(
        "\n\n",
        documents.Select(x => x.Content));

    var prompt = $"""
        Answer the question using only the context below.

        Context:
        {context}

        Question:
        {question}
        """;

    return await llm.GenerateAsync(prompt);
}

The important flow is:

Question
   ↓
Embedding
   ↓
Vector Search
   ↓
Relevant Documents
   ↓
Prompt
   ↓
LLM
   ↓
Answer

31. Document Ingestion in C#

Conceptually:

public async Task IndexDocumentAsync(Document document)
{
    var chunks = ChunkDocument(document.Content);

    foreach (var chunk in chunks)
    {
        var embedding =
            await embeddingService.CreateEmbeddingAsync(chunk);

        await vectorStore.AddAsync(new VectorDocument
        {
            DocumentId = document.Id,
            Content = chunk,
            Embedding = embedding,
            Metadata = document.Metadata
        });
    }
}

This creates the knowledge base.

32. RAG Application Using Angular + .NET

A practical enterprise solution could be:

                Angular
                   |
                   |
              HTTP / HTTPS
                   |
                   v
          ASP.NET Core Web API
                   |
           +-------+-------+
           |               |
           v               v
      RAG Service       Auth Service
           |
     +-----+------+
     |            |
     v            v
Embedding       Vector DB
Service
     |
     v
     LLM

Angular provides:

Chat UI
Document Upload
Source Display
Conversation History

ASP.NET Core provides:

Authentication
Authorization
RAG orchestration
Document processing
Business logic
API endpoints
Logging

33. Example API

A simple API could be:

POST /api/rag/ask

Request:

{
  "question": "What is the leave policy?"
}

Response:

{
  "answer": "Employees are eligible for annual leave...",
  "sources": [
    {
      "document": "LeavePolicy.pdf",
      "page": 12
    }
  ]
}

Angular can display:

Answer
--------------------------------
Employees are eligible for...

Sources
--------------------------------
LeavePolicy.pdf
Page 12

34. RAG Security

Enterprise RAG must take security seriously.

Suppose:

Employee A

should not access:

Employee B's salary information.

Simply storing everything in one vector index can create security problems.

The system should enforce:

User
 |
 v
Authentication
 |
 v
Authorization
 |
 v
Security Filter
 |
 v
Retrieval

Metadata can help:

{
  "department": "Finance",
  "classification": "Confidential",
  "allowedRoles": [
    "FinanceManager"
  ]
}

The retrieval layer should apply appropriate authorization filters before returning context.

35. RAG and JWT Authentication

In an ASP.NET Core enterprise application:

Angular
   |
   v
JWT
   |
   v
ASP.NET Core
   |
   v
User Claims
   |
   v
RAG Authorization
   |
   v
Filtered Retrieval

For example:

Role = HRManager
Department = HR

could result in:

Department = HR
AND
UserAuthorized = true

during retrieval.

36. RAG and Microservices

RAG fits naturally into microservice architecture.

Example:

                    API Gateway
                         |
          +--------------+--------------+
          |              |              |
          v              v              v
      User Service   Order Service   RAG Service
                                         |
                              +----------+----------+
                              |                     |
                              v                     v
                        Vector Search             LLM
                              |
                              v
                       Document Store

A dedicated RAG service can own:

Document ingestion
Chunking
Embedding
Retrieval
Reranking
Prompt construction
LLM interaction
Citation generation

37. RAG + Azure Service Bus

For large enterprise applications, document processing should not always happen synchronously.

For example:

User uploads PDF
       |
       v
Blob Storage
       |
       v
Azure Service Bus
       |
       v
Document Processing Service
       |
       v
Text Extraction
       |
       v
Chunking
       |
       v
Embedding
       |
       v
Azure AI Search

This provides an asynchronous ingestion pipeline.

38. RAG + Blob Storage

A common Azure architecture:

                    Blob Storage
                         |
                         v
                Document Processor
                         |
                    Chunking
                         |
                    Embeddings
                         |
                         v
                  Azure AI Search
                         |
                         v
                    RAG API
                         |
                         v
                    Azure OpenAI

Documents can remain in Blob Storage while searchable chunks and metadata are stored in the search system.

39. RAG Evaluation

Building a RAG application is not just about making it work.

You need to measure it.

Important metrics include:

Retrieval Precision

Did the system retrieve relevant documents?

Retrieval Recall

Did it retrieve the information needed to answer the question?

Faithfulness

Does the generated answer actually follow the retrieved context?

Answer Relevance

Does the answer address the user's question?

Latency

How long does the complete request take?

Cost

How many embedding and LLM tokens are being consumed?

40. Common RAG Problems

Problem 1 — Bad Chunking

If chunks are too large:

Too much irrelevant context

If chunks are too small:

Important context may be lost

Problem 2 — Poor Retrieval

If the retriever returns irrelevant documents:

Wrong Context
     ↓
LLM
     ↓
Poor Answer

Problem 3 — Hallucination

Even with RAG, an LLM may generate information not supported by the context.

Therefore prompts should clearly instruct the model:

Use the provided context.

If the answer cannot be found in
the context, say that the information
is not available.

Problem 4 — Too Much Context

Sending hundreds of irrelevant chunks to the LLM can:

Increase cost
Increase latency
Reduce answer quality

Therefore retrieval and reranking are important.

41. Advanced RAG

Basic RAG:

Question
   ↓
Vector Search
   ↓
LLM

Advanced RAG can look like:

User Question
      |
      v
Query Rewriting
      |
      v
Hybrid Retrieval
      |
      v
Metadata Filtering
      |
      v
Reranking
      |
      v
Context Compression
      |
      v
LLM
      |
      v
Citation / Validation
      |
      v
Answer

42. Agentic RAG

A newer approach is Agentic RAG.

Instead of performing one retrieval operation, an AI agent can determine what information it needs and use multiple tools.

For example:

User:
"Why was customer 10025's order delayed?"

The agent may decide to query:

Customer API
       +
Order Database
       +
Support Tickets
       +
Product Documentation

Then combine the results.

A modern Agentic RAG system may therefore contain:

              AI Agent
                  |
        +---------+---------+
        |         |         |
        v         v         v
      SQL       Search      API
        |         |         |
        +---------+---------+
                  |
                  v
                 LLM

Recent industry research is increasingly exploring agentic RAG architectures that use planning, routing and retrieval to improve grounded responses.

43. RAG vs Agentic RAG

Feature	Basic RAG	Agentic RAG
Retrieval	Usually fixed	Dynamic
Decision making	Limited	Agent-driven
Multiple tools	Limited	Yes
SQL/API integration	Possible	Strong
Query planning	Limited	Yes
Complexity	Lower	Higher
Cost	Lower	Potentially higher
Enterprise workflows	Good	Excellent for complex workflows

44. RAG Security Architecture

For enterprise applications:

                    User
                     |
                     v
              Authentication
                     |
                     v
              Authorization
                     |
                     v
               RAG API
                     |
              Security Filter
                     |
                     v
               Retrieval
                     |
                     v
                   LLM
                     |
                     v
             Output Validation
                     |
                     v
                  User

Important controls include:

Authentication
Authorization
Tenant isolation
Document-level permissions
Metadata filtering
Encryption
Audit logging
PII protection
Prompt-injection defenses
Output validation

45. Multi-Tenant RAG

Consider a SaaS application with:

Company A
Company B
Company C

The system must prevent:

Company A → Company B documents

A common approach is to include:

TenantId

in document metadata.

Example:

{
  "tenantId": "COMPANY-A",
  "documentId": "DOC-1001"
}

At query time:

WHERE TenantId = CurrentUser.TenantId

This is critical for enterprise SaaS systems.

46. RAG Cost Optimization

RAG does not automatically mean low cost.

Costs may come from:

Embedding generation
Vector search
LLM input tokens
LLM output tokens
Storage
Document processing
OCR
Reranking

Optimization techniques include:

1. Good chunking

Avoid unnecessary context.

2. Top-K optimization

Don't retrieve 100 chunks when 5 are enough.

3. Reranking

Retrieve candidates first, then select the best ones.

4. Cache embeddings

Don't regenerate embeddings unnecessarily.

5. Cache frequent questions

For repeated queries, response caching may help.

47. RAG Complete Architecture for an Enterprise .NET Application

A production architecture could look like:

                         Angular
                            |
                            v
                     API Management
                            |
                            v
                    ASP.NET Core API
                            |
                    +-------+-------+
                    |               |
                    v               v
                RAG Service     Business APIs
                    |
         +----------+----------+
         |          |           |
         v          v           v
     Embedding   Search      SQL/API
      Service     Index
         |          |
         |          v
         |      Vector Search
         |          |
         +----------+
                    |
                    v
                   LLM
                    |
                    v
              Answer + Sources

Document ingestion:

User/Admin
    |
    v
Blob Storage
    |
    v
Service Bus
    |
    v
Document Processor
    |
    v
Text Extraction
    |
    v
Chunking
    |
    v
Embedding
    |
    v
Search Index

48. Recommended Technology Stack for a .NET Developer

For an enterprise .NET developer, one possible stack is:

Layer	Technology
Frontend	Angular
Backend	ASP.NET Core
Authentication	Microsoft Entra ID / OAuth/OIDC
LLM	Azure OpenAI or another LLM provider
Search	Azure AI Search
Document Storage	Azure Blob Storage
Database	Azure SQL
Messaging	Azure Service Bus
Monitoring	Application Insights / Azure Monitor
Secrets	Azure Key Vault
API Gateway	Azure API Management
Containerization	Docker
Orchestration	AKS
CI/CD	Azure DevOps
Embeddings	Embedding model
Vector Store	Azure AI Search / another vector-capable store

This is particularly suitable for organizations already invested in Microsoft technologies.

49. RAG Request Flow

Let's follow one request from beginning to end.

User asks:

"What is our refund policy?"

Step 1

Angular sends:

POST /api/rag/ask

Step 2

ASP.NET Core authenticates the user.

Step 3

RAG service creates an embedding.

Step 4

Search system performs semantic/hybrid search.

Step 5

Relevant documents are retrieved.

RefundPolicy.pdf
Section 4
Section 7

Step 6

Reranker selects the best chunks.

Step 7

Application constructs the prompt.

Step 8

LLM generates the answer.

Step 9

Application returns:

{
  "answer": "Customers can request a refund within...",
  "sources": [
    {
      "document": "RefundPolicy.pdf",
      "section": "4"
    }
  ]
}

Step 10

Angular displays the answer and sources.

50. Is RAG a Replacement for an LLM?

No.

RAG normally works with an LLM.

Think of the responsibilities like this:

Vector Search
     |
     | Finds information
     v
Relevant Context
     |
     | Gives context
     v
LLM
     |
     | Understands and generates
     v
Human-readable Answer

The vector database does not replace the LLM.

The LLM does not replace the search engine.

They work together.

51. Is RAG the Same as ChatGPT?

No.

ChatGPT is an AI application/service that can use LLMs and various tools.

RAG is an architectural pattern.

A developer can build:

Custom RAG Application

using:

ASP.NET Core
+
Vector Database
+
Embedding Model
+
LLM

52. When Should You Use RAG?

RAG is a strong choice when:

You have private documents
        OR
You have frequently changing information
        OR
You need source-grounded answers
        OR
You need enterprise knowledge search
        OR
You need natural-language access to documentation

Examples:

HR Assistant
Customer Support Assistant
Legal Document Search
Technical Documentation Assistant
Product Assistant
Knowledge Management
Research Assistant
Enterprise Search

53. When RAG May Not Be the Best Solution

Don't automatically use RAG for everything.

For example:

"Calculate 125 × 75."

You don't need document retrieval.

Similarly:

"Sort this list."

RAG isn't necessary.

For structured business data:

SQL
API
Business Service

may be more appropriate.

The best enterprise architecture often combines:

LLM
+
RAG
+
SQL
+
APIs
+
Business Services
+
Tools

54. RAG vs Search Engine

Traditional search:

Question
   |
   v
Search Engine
   |
   v
10 Results

RAG:

Question
   |
   v
Search
   |
   v
Relevant Documents
   |
   v
LLM
   |
   v
Natural Language Answer

Traditional search gives you documents.

RAG can understand the retrieved information and synthesize an answer.

55. RAG vs Database

A database is designed primarily for structured data.

For example:

CustomerId
Name
OrderId
OrderDate
Amount

RAG is especially useful for unstructured/semi-structured knowledge such as:

PDFs
Manuals
Policies
Documentation
Emails
Knowledge articles
Text

Enterprise systems often use both.

56. Golden Rule for RAG

A very useful way to remember RAG is:

LLM = Brain

Embedding Model = Meaning Representation

Vector Database = Memory/Search

Retriever = Finds Memory

Prompt = Instructions

RAG = Brain + External Memory

This is only an analogy, but it makes the architecture easier to understand.

57. The Future of RAG

RAG is evolving beyond simple:

Search → Prompt → LLM

Modern systems increasingly explore:

Query Planning
+
Hybrid Search
+
Reranking
+
Knowledge Graphs
+
Multimodal Retrieval
+
Agentic Workflows
+
Tool Calling
+
SQL
+
APIs
+
Validation

Research is also exploring techniques for improving RAG efficiency and answer quality, including systems that use multiple retrieved document subsets and verification stages.

58. Complete RAG Mental Model

Remember this architecture:

                 ┌─────────────────┐
                 │      USER       │
                 └────────┬────────┘
                          |
                          v
                 ┌─────────────────┐
                 │     QUESTION    │
                 └────────┬────────┘
                          |
                          v
                 ┌─────────────────┐
                 │ QUERY PROCESSOR │
                 └────────┬────────┘
                          |
                          v
                 ┌─────────────────┐
                 │   EMBEDDING     │
                 └────────┬────────┘
                          |
                          v
              ┌───────────────────────┐
              │ VECTOR / HYBRID SEARCH│
              └───────────┬───────────┘
                          |
                          v
                 ┌─────────────────┐
                 │    RERANKER     │
                 └────────┬────────┘
                          |
                          v
                 ┌─────────────────┐
                 │ RELEVANT CONTEXT│
                 └────────┬────────┘
                          |
                          v
                 ┌─────────────────┐
                 │       LLM       │
                 └────────┬────────┘
                          |
                          v
                 ┌─────────────────┐
                 │ ANSWER + SOURCE │
                 └─────────────────┘

59. Key Takeaways

The most important points are:

RAG stands for Retrieval-Augmented Generation.
RAG is an architecture/pattern, not a programming language.
The well-known RAG architecture was introduced in a 2020 research paper by Patrick Lewis and collaborators.
RAG combines:
```
Retrieval + LLM Generation
```
Embeddings convert text into vectors representing semantic meaning.
Vector databases/search engines allow semantic retrieval.
Documents should normally be split into chunks.
Metadata is extremely important for filtering and security.

RAG can use:

PDFs
DOCX
Websites
Databases
APIs
Knowledge Bases

RAG can reduce hallucination but cannot guarantee that every answer is correct.
Fine-tuning and RAG solve different problems.
RAG is highly useful for enterprise applications.
.NET developers can build RAG applications using:

Angular
+
ASP.NET Core
+
Embedding Model
+
Vector Search
+
LLM
+
SQL
+
Azure Services

Advanced RAG can include:

Hybrid Search
Reranking
Metadata Filtering
Query Rewriting
Context Compression
Agentic Workflows
SQL
APIs
Knowledge Graphs

60. Final Conclusion

RAG is one of the most important architectural patterns for building practical enterprise generative-AI applications.

An LLM provides the reasoning and language-generation capability, while RAG provides a mechanism for accessing external knowledge.

The basic concept is simple:

             Traditional LLM

Question ──────────> LLM ──────────> Answer


             RAG Application

Question
   |
   v
Retrieve Knowledge
   |
   v
Relevant Context
   |
   v
LLM
   |
   v
Grounded Answer

The real power appears when RAG is combined with enterprise technologies:

Angular
   +
ASP.NET Core
   +
Azure OpenAI
   +
Azure AI Search
   +
Azure SQL
   +
Blob Storage
   +
Azure Service Bus
   +
API Management
   +
Key Vault
   +
Application Insights
   +
Docker / AKS
   +
Azure DevOps

This combination allows developers to build practical systems such as:

Enterprise Knowledge Assistant
Customer Support AI
HR Assistant
Technical Documentation Assistant
Product Assistant
Financial Document Assistant
Legal Document Search
Developer Copilot
Enterprise Search

The key idea to remember is:

RAG does not teach an LLM everything permanently. Instead, it retrieves the right information at the right time and gives that information to the LLM so it can generate a more relevant, grounded response.

That makes RAG one of the most useful bridges between traditional enterprise data and modern Generative AI.

References

The original RAG research paper is available on arXiv: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.

Google's research on REALM provides useful background on retrieval-augmented language modeling and explicit external knowledge retrieval.

Complete CI/CD Pipeline for .NET 9 Microservices on Azure Using Azure DevOps, Docker, ACR and AKS

2026-08-16T08:27:37.755+05:30

Complete CI/CD Pipeline for .NET 9 Microservices on Azure Using Azure DevOps, Docker, ACR and AKS

Introduction

Modern enterprise applications require a reliable and automated process for building, testing, packaging, and deploying applications across multiple environments.

In a typical microservices-based application, developers continuously make changes to individual services. Manually building and deploying these services is time-consuming and error-prone.

This is where CI/CD — Continuous Integration and Continuous Delivery/Deployment — becomes extremely important.

In this article, we will build a complete CI/CD pipeline for the following enterprise architecture:

Angular Web Application
.NET 9 Web API / Microservices
Docker
Azure Container Registry (ACR)
Azure Kubernetes Service (AKS)
Azure SQL Database
Azure Service Bus
Azure Key Vault
Azure DevOps
Kubernetes Ingress
DEV, QA, UAT and PRODUCTION environments

We will specifically use the Azure DevOps Classic/Visual UI approach, so you can understand how to configure the pipeline directly through the Azure DevOps portal without initially writing an azure-pipelines.yml file.

Note: Microsoft recommends YAML pipelines for new development, but Classic pipelines remain useful for understanding the Azure DevOps pipeline concepts and for organizations that still use the Classic UI.

1. What Are We Going to Build?

The overall CI/CD architecture will look like this:

Developer
    |
    | Git Push
    v
Azure Repos
    |
    v
+-----------------------------+
|       CI / Build Pipeline   |
|                             |
|  1. Restore                 |
|  2. Build                   |
|  3. Unit Test               |
|  4. Publish                 |
|  5. Docker Build            |
|  6. Docker Push             |
+-------------+---------------+
              |
              v
      Azure Container Registry
              |
              | Docker Image
              v
+-----------------------------+
|      CD / Release Pipeline  |
|                             |
|       DEV                   |
|         |                   |
|         v                   |
|       QA                    |
|         |                   |
|         v                   |
|       UAT                   |
|         |                   |
|     Approval                |
|         |                   |
|         v                   |
|     PRODUCTION              |
+-------------+---------------+
              |
              v
             AKS
              |
       +------+------+
       |      |      |
       v      v      v
    Order  Customer Product
     API     API     API

The basic principle is:

Code → Build → Test → Docker Image → ACR → Deploy → AKS

2. Prerequisites

Before creating the pipeline, we need several Azure and Azure DevOps resources.

Azure Resources

You should have:

Azure Subscription
Resource Group
Azure Container Registry
Azure Kubernetes Service
Azure SQL Database
Azure Service Bus
Azure Key Vault
Azure Monitor / Application Insights

Azure DevOps

You should have:

Azure DevOps Organization
Azure DevOps Project
Azure Repos
Pipeline permissions
Microsoft-hosted agent capability

Microsoft-hosted agents are suitable for many standard .NET builds. Self-hosted agents can be used when an organization requires custom tooling or private-network access.

3. Example Enterprise Application

Let's assume our solution has the following structure:

EnterpriseApp
│
├── EnterpriseApp.sln
│
├── src
│   ├── CustomerService
│   │   └── CustomerService.csproj
│   │
│   ├── OrderService
│   │   └── OrderService.csproj
│   │
│   └── ProductService
│       └── ProductService.csproj
│
├── tests
│   └── OrderService.Tests
│
├── Dockerfile
│
└── k8s
    ├── namespace.yaml
    ├── order-deployment.yaml
    ├── order-service.yaml
    ├── customer-deployment.yaml
    ├── customer-service.yaml
    └── ingress.yaml

For our first example, we will deploy:

OrderService

Once the process is understood, the same approach can be applied to:

CustomerService
ProductService
InventoryService
PaymentService
NotificationService
Other microservices

4. Step 1 — Create an Azure DevOps Project

Open your Azure DevOps organization and create a project.

For example:

Organization
    |
    └── EnterpriseProject

Inside the project, Azure DevOps provides services such as:

Boards
Repos
Pipelines
Test Plans
Artifacts

For our CI/CD implementation, the most important areas are:

Repos
Pipelines
Project Settings

5. Step 2 — Add Source Code to Azure Repos

Navigate to:

Repos
    |
    └── Files

Create or import your Git repository.

For example:

EnterpriseApp

The repository might contain:

EnterpriseApp.sln

src/
    CustomerService/
    OrderService/
    ProductService/

tests/
    OrderService.Tests/

Dockerfile

k8s/

Developers can then work with the repository using Git:

git clone <repository>

After making changes:

git add .
git commit -m "Update order validation"
git push

This Git push can eventually trigger the CI pipeline automatically.

6. Step 3 — Create Azure Container Registry

The Azure Container Registry (ACR) stores the Docker images generated by the CI pipeline.

In the Azure Portal:

Create a resource
        ↓
Container Registry

For example:

Registry Name:
enterpriseacr

The registry endpoint could be:

enterpriseacr.azurecr.io

Our Docker image can then be tagged as:

enterpriseacr.azurecr.io/order-service:1.0

or, preferably, with a unique build number:

enterpriseacr.azurecr.io/order-service:1234

Using a unique build identifier makes it easier to identify exactly which source version produced a container image.

7. Step 4 — Create Azure Kubernetes Service

Create an AKS cluster from the Azure Portal or Azure CLI.

For example:

EnterpriseAKS

An AKS cluster contains:

AKS Cluster
    |
    ├── Node Pools
    |
    ├── Kubernetes Control Plane
    |
    └── Workloads

After connecting your local environment to the cluster, you can verify the nodes:

kubectl get nodes

8. Step 5 — Connect ACR with AKS

AKS needs permission to pull Docker images from Azure Container Registry.

A common Azure CLI approach is:

az aks update \
  --resource-group EnterpriseRG \
  --name EnterpriseAKS \
  --attach-acr enterpriseacr

The relationship becomes:

Azure Container Registry
          |
          | Pull Docker Image
          v
         AKS

This is important because the CI pipeline pushes the image into ACR, while AKS pulls that image when deploying the application.

9. Step 6 — Create Azure DevOps Service Connection

Azure DevOps needs permission to access Azure resources.

Navigate to:

Azure DevOps
    ↓
Project Settings
    ↓
Service connections
    ↓
New service connection

Select:

Azure Resource Manager

Configure the connection according to your organization's authentication and security requirements.

For example:

Service Connection Name:

Azure-Enterprise-Connection

Conceptually:

Azure DevOps
       |
       | Service Connection
       v
Azure Subscription
       |
       +---- ACR
       +---- AKS
       +---- Other Azure Resources

Service connections are a critical security boundary, so they should be granted only the permissions required by the pipeline.

10. Step 7 — Create the CI Build Pipeline

Now we can create the Continuous Integration pipeline.

Navigate to:

Pipelines
    ↓
Builds

Depending on your Azure DevOps UI, select the option to create a new pipeline and choose the Classic editor.

The Classic editor allows you to configure the build process using the Azure DevOps UI rather than writing YAML.

Select:

Azure Repos Git

Then select:

Project:
EnterpriseProject

Repository:
EnterpriseApp

Branch:
main

Click:

Continue

11. Step 8 — Select Empty Job

Instead of allowing Azure DevOps to automatically create all the tasks, select:

Empty Job

You should now see something similar to:

Agent Job 1

This approach is useful when learning CI/CD because you can understand each pipeline task individually.

12. Step 9 — Configure the Build Agent

Select:

Agent Job 1

Choose an agent specification.

For example:

ubuntu-latest

or:

windows-latest

For Docker and Linux-based containers, Ubuntu is commonly convenient.

Our pipeline now starts with:

Agent Job
     |
     v
Use .NET SDK

13. Step 10 — Install / Select .NET SDK

Add a task:

Search for the .NET SDK task.

Configure the required .NET version.

For example:

SDK Version:

9.x

This ensures that the build agent uses the expected .NET SDK version.

14. Step 11 — Restore NuGet Packages

Add a .NET Core task.

Configure:

Command:

restore

For example:

Path to project:

EnterpriseApp.sln

The pipeline becomes:

Use .NET SDK
      |
      v
Restore

This downloads the NuGet dependencies required by the solution.

15. Step 12 — Build the Application

Add another .NET task.

Configure:

Command:

build

Project:

EnterpriseApp.sln

Arguments:

--configuration Release --no-restore

The pipeline is now:

Use .NET SDK
      |
      v
Restore
      |
      v
Build

The equivalent .NET CLI operation is:

dotnet build EnterpriseApp.sln \
    --configuration Release \
    --no-restore

16. Step 13 — Execute Unit Tests

Add another .NET task.

Configure:

Command:

test

Project:

tests/**/*.csproj

Arguments:

--configuration Release --no-build

The pipeline becomes:

Restore
   |
   v
Build
   |
   v
Unit Tests

If the unit tests fail:

Build ❌

the pipeline should stop.

This is an important CI principle:

Code should not progress toward deployment if automated tests are failing.

17. Step 14 — Publish the .NET Application

Add another .NET task.

Configure:

Command:

publish

Project:

src/OrderService/OrderService.csproj

Configuration:

Release

Output:

$(Build.ArtifactStagingDirectory)/order-service

The flow becomes:

Build
  |
  v
Test
  |
  v
Publish
  |
  v
Build Artifact

18. Step 15 — Build the Docker Image

Now we move from application build to containerization.

Add a Docker task.

Select:

Docker

Command:

Build

Configure the Azure Container Registry connection.

Repository:

order-service

Dockerfile:

$(Build.SourcesDirectory)/src/OrderService/Dockerfile

Tag:

$(Build.BuildId)

The resulting image could be:

enterpriseacr.azurecr.io/order-service:1234

19. Step 16 — Push Docker Image to ACR

Add another Docker task.

Select:

Docker

Command:

Push

Repository:

order-service

Tag:

$(Build.BuildId)

The CI process is now:

Git Push
    |
    v
Restore
    |
    v
Build
    |
    v
Unit Test
    |
    v
Publish
    |
    v
Docker Build
    |
    v
Docker Push
    |
    v
Azure Container Registry

This is the core Continuous Integration workflow.

20. Step 17 — Save and Run the CI Pipeline

Save the pipeline with a name such as:

CI-OrderService

Then select:

Save & Queue

Run the pipeline.

You should see tasks such as:

✔ Restore
✔ Build
✔ Unit Test
✔ Publish
✔ Docker Build
✔ Docker Push

If everything succeeds:

BUILD SUCCESSFUL

21. Step 18 — Verify the Docker Image in ACR

Open:

Azure Portal
    ↓
Container Registry
    ↓
Repositories

You should see:

order-service

Inside the repository:

order-service
    |
    └── 1234

The tag 1234 represents the pipeline build number.

22. Step 19 — Create the CD / Release Pipeline

The next step is Continuous Delivery/Deployment.

In the Classic pipeline model, the release pipeline is separate from the build pipeline.

Navigate to:

Pipelines
    ↓
Releases
    ↓
New pipeline

Select:

Empty Job

Rename the first stage:

DEV

A typical enterprise release pipeline can contain:

DEV
  |
  v
QA
  |
  v
UAT
  |
  v
PRODUCTION

23. Step 20 — Configure the Release Artifact

Add the output from the CI/build process as the release artifact.

For containerized applications, the key deployment input is the Docker image and its version/tag stored in ACR.

Conceptually:

CI Pipeline
     |
     v
Docker Image
     |
     v
ACR
     |
     v
Release Pipeline
     |
     v
AKS

24. Step 21 — Configure DEV Deployment

Open the DEV stage.

Add a Kubernetes deployment task.

Configure the Azure connection:

Connection Type:

Azure Resource Manager

Select:

Azure Subscription:

Azure-Enterprise-Connection

Then select the AKS cluster:

EnterpriseAKS

Use a namespace such as:

dev

This allows the same AKS cluster to host workloads for multiple environments when that is appropriate for the organization's architecture.

25. Kubernetes Deployment

A simplified Kubernetes Deployment might look like:

apiVersion: apps/v1
kind: Deployment

metadata:
  name: order-service

spec:
  replicas: 3

  selector:
    matchLabels:
      app: order-service

  template:
    metadata:
      labels:
        app: order-service

    spec:
      containers:
        - name: order-service

          image: enterpriseacr.azurecr.io/order-service:1234

          ports:
            - containerPort: 8080

The important part is:

image: enterpriseacr.azurecr.io/order-service:1234

When a new version is released, the image tag changes.

For example:

Old:

order-service:1233

becomes:

New:

order-service:1234

Kubernetes then performs a rollout to replace the old application version with the new one.

26. DEV Deployment Flow

The deployment now looks like:

ACR
 |
 | order-service:1234
 v
AKS
 |
 v
DEV Namespace
 |
 +---- Pod 1
 |
 +---- Pod 2
 |
 +---- Pod 3

Running multiple replicas provides application redundancy and enables Kubernetes to distribute traffic across available Pods.

27. Step 22 — Create QA Stage

Add another stage:

QA

Configure the deployment to target:

QA

and use the appropriate Kubernetes namespace:

qa

The pipeline becomes:

DEV
 |
 v
QA

Configure the QA stage to execute after successful DEV deployment.

28. Step 23 — Create UAT Stage

Create another stage:

UAT

The release flow becomes:

DEV
 |
 v
QA
 |
 v
UAT

UAT is generally used for business/user acceptance validation before production.

29. Step 24 — Create Production Stage

Create the final stage:

PRODUCTION

The complete release flow is now:

DEV
 |
 v
QA
 |
 v
UAT
 |
 v
PRODUCTION

30. Step 25 — Add Production Approval

Production deployments should normally have appropriate controls.

Instead of allowing:

Developer
   |
   v
DEV
   |
   v
QA
   |
   v
UAT
   |
   v
PRODUCTION

without any control, introduce an approval gate:

DEV
 |
 v
QA
 |
 v
UAT
 |
 v
+----------------------+
| Production Approval  |
|                      |
| Lead / Manager       |
+----------+-----------+
           |
           v
      PRODUCTION

In the Classic Release pipeline, configure:

PRODUCTION
    ↓
Pre-deployment conditions
    ↓
Pre-deployment approvals

Add the authorized reviewers according to your organization's release-management process.

31. Complete CI/CD Architecture

The complete architecture now looks like this:

                         DEVELOPER
                             |
                             | Git Push
                             v
                      +--------------+
                      | Azure Repos  |
                      +------+-------+
                             |
                             v
                  +----------------------+
                  |    CI / BUILD        |
                  |                      |
                  | Restore              |
                  | Build                |
                  | Unit Test            |
                  | Publish              |
                  | Docker Build         |
                  | Docker Push          |
                  +----------+-----------+
                             |
                             v
                  +----------------------+
                  | Azure Container      |
                  | Registry (ACR)       |
                  +----------+-----------+
                             |
                             v
                  +----------------------+
                  |    CD / RELEASE      |
                  +----------+-----------+
                             |
                             v
                        +---------+
                        |   DEV   |
                        +----+----+
                             |
                             v
                        +---------+
                        |   QA    |
                        +----+----+
                             |
                             v
                        +---------+
                        |   UAT   |
                        +----+----+
                             |
                          Approval
                             |
                             v
                     +---------------+
                     | PRODUCTION    |
                     +-------+-------+
                             |
                             v
                            AKS
                             |
             +---------------+---------------+
             |               |               |
             v               v               v
        Customer API    Order API       Product API

32. Where Does Ingress Fit?

Ingress is part of the Kubernetes application architecture. It is not a replacement for Azure DevOps.

A typical AKS architecture can look like:

                         Internet
                            |
                            v
                     +--------------+
                     |    Ingress   |
                     |   / Gateway  |
                     +------+-------+
                            |
              +-------------+-------------+
              |             |             |
              v             v             v
        Customer API    Order API    Product API
              |             |             |
              v             v             v
            Pods          Pods          Pods
              |             |             |
              +-------------+-------------+
                            |
                   +--------+--------+
                   |                 |
                   v                 v
              Azure SQL       Azure Service Bus

The important distinction is:

Azure DevOps automates application delivery.

Kubernetes/AKS runs the application.

Ingress/Gateway manages incoming application traffic.

33. Where Do Azure SQL and Azure Service Bus Fit?

Azure SQL and Azure Service Bus are generally infrastructure dependencies rather than resources that should be recreated every time the application pipeline runs.

A typical environment contains:

Azure Infrastructure
    |
    +── AKS
    |
    +── ACR
    |
    +── Azure SQL
    |
    +── Azure Service Bus
    |
    +── Key Vault
    |
    +── Monitoring

The CI/CD pipeline then deploys the application:

CI/CD
   |
   v
AKS

The application can communicate with:

Order API
    |
    +---- Azure SQL
    |
    +---- Azure Service Bus

34. Managing Secrets

Sensitive information should never be hardcoded into:

Source code
Dockerfiles
Kubernetes manifests
Pipeline scripts
Git repositories

Examples include:

SQL connection strings
Service Bus credentials
JWT secrets
API keys
Third-party credentials

A recommended architecture is:

                    Azure Key Vault
                          |
                          |
                    Managed Identity
                          |
                          v
                         AKS
                          |
                          v
                    Microservices

Azure DevOps variable/secret mechanisms can also be used where appropriate, but secrets should be managed according to your organization's security architecture.

35. What Happens When a Developer Commits Code?

Suppose a developer changes the OrderService validation logic.

The developer executes:

git add .

Then:

git commit -m "Update order validation"

Then:

git push origin main

The automated process becomes:

Git Push
   |
   v
CI Trigger
   |
   v
Restore
   |
   v
Build
   |
   v
Unit Tests
   |
   v
Docker Build
   |
   v
Docker Image
   |
   v
ACR
   |
   v
Release Pipeline
   |
   v
DEV
   |
   v
QA
   |
   v
UAT
   |
   v
Approval
   |
   v
PRODUCTION

This is the essence of CI/CD.

36. Continuous Integration vs Continuous Delivery

Understanding the difference between CI and CD is important.

Continuous Integration

Continuous Integration focuses on validating the application whenever code changes.

Code
 |
 v
Restore
 |
 v
Build
 |
 v
Test
 |
 v
Package
 |
 v
Docker Image
 |
 v
ACR

The primary question is:

"Is my code buildable, testable and packageable?"

Continuous Delivery / Deployment

Continuous Delivery/Deployment focuses on moving the validated application version through environments.

ACR
 |
 v
DEV
 |
 v
QA
 |
 v
UAT
 |
 v
Approval
 |
 v
PRODUCTION

The primary question is:

"Can this version be safely deployed to the required environments?"

Therefore:

CI = Build, test and package the application

CD = Deliver/deploy the application through environments

37. Classic Pipeline vs YAML Pipeline

The Classic UI is an excellent way to learn Azure DevOps because you can visually see:

Agent
Tasks
Stages
Artifacts
Approvals
Deployment

However, for modern enterprise projects, YAML pipelines are often preferred because the pipeline definition can be stored alongside the source code.

A modern architecture can look like:

Azure DevOps
      |
      v
YAML Pipeline
      |
      +---- CI
      |
      +---- Build
      |
      +---- Unit Tests
      |
      +---- Docker
      |
      +---- Security
      |
      +---- ACR
      |
      +---- CD
             |
             +---- DEV
             |
             +---- QA
             |
             +---- UAT
             |
             +---- PROD

The key advantage is Pipeline as Code.

The pipeline itself becomes version-controlled, reviewable, and reproducible.

38. Recommended Enterprise Improvements

The basic pipeline described above can be extended significantly for production environments.

Security

Add:

Azure Key Vault
Managed Identity
Microsoft Entra ID
Container image scanning
Dependency scanning
Secret scanning
Least-privilege service connections

Quality

Add:

Unit tests
Integration tests
Code coverage
SonarQube/SonarCloud
API testing
Performance testing

Deployment

Add:

Rolling deployments
Health probes
Readiness probes
Liveness probes
Deployment strategies
Automatic rollback
Environment approvals

Observability

Add:

Application Insights
Azure Monitor
Log Analytics
Kubernetes monitoring
Alerts
Dashboards

39. Production-Ready Microservices Flow

A mature enterprise architecture could eventually look like:

                         Developer
                             |
                             v
                        Azure Repos
                             |
                             v
                    Azure DevOps CI
                             |
          +------------------+------------------+
          |                  |                  |
          v                  v                  v
      Build/Test        Security Scan       Code Quality
          |                  |                  |
          +------------------+------------------+
                             |
                             v
                         Docker Build
                             |
                             v
                           ACR
                             |
                             v
                    Deployment Pipeline
                             |
            +----------------+----------------+
            |                |                |
            v                v                v
           DEV              QA               UAT
                                             |
                                         Approval
                                             |
                                             v
                                         PROD
                                             |
                                             v
                                            AKS
                                             |
                         +-------------------+-------------------+
                         |                   |                   |
                         v                   v                   v
                    Customer API        Order API          Product API
                         |                   |                   |
                         +-------------------+-------------------+
                                             |
                    +------------------------+----------------------+
                    |                                               |
                    v                                               v
                Azure SQL                                  Azure Service Bus
                    |
                    v
               Application
                 Insights
                    |
                    v
               Azure Monitor

40. Important Takeaways

The complete deployment journey can be remembered as:

Developer
   ↓
Git
   ↓
Azure DevOps
   ↓
CI
   ↓
Build
   ↓
Test
   ↓
Docker
   ↓
ACR
   ↓
CD
   ↓
DEV
   ↓
QA
   ↓
UAT
   ↓
Approval
   ↓
PRODUCTION
   ↓
AKS

The responsibilities of the major components are:

Component	Responsibility
Azure Repos	Source-code management
Azure DevOps	CI/CD automation
Docker	Application containerization
ACR	Docker image storage
AKS	Container orchestration
Kubernetes	Application workload management
Ingress/Gateway	Incoming traffic routing
Azure SQL	Relational database
Azure Service Bus	Asynchronous messaging
Key Vault	Secret management
Application Insights	Application telemetry
Azure Monitor	Monitoring and alerting

41. Final Conclusion

Implementing CI/CD for a .NET microservices application on Azure provides a repeatable and controlled way to move software from development to production.

The complete lifecycle is:

Code
  ↓
Commit
  ↓
Build
  ↓
Unit Test
  ↓
Docker Image
  ↓
Azure Container Registry
  ↓
AKS Deployment
  ↓
DEV
  ↓
QA
  ↓
UAT
  ↓
Production Approval
  ↓
PRODUCTION

For learning Azure DevOps, the Classic UI pipeline is a very useful starting point because every stage and task is visible through the portal.

For a new enterprise implementation, however, it is worth moving toward YAML-based pipelines, infrastructure as code, automated security scanning, managed identities, automated testing, deployment strategies, and comprehensive monitoring.

The ultimate goal is not simply to automate deployment.

The goal is to create a secure, repeatable, observable, and reliable software delivery process.

Quick Reference

CI

Git
 ↓
Restore
 ↓
Build
 ↓
Test
 ↓
Publish
 ↓
Docker Build
 ↓
Docker Push
 ↓
ACR

CD

ACR
 ↓
DEV
 ↓
QA
 ↓
UAT
 ↓
Approval
 ↓
PRODUCTION
 ↓
AKS

Application Architecture

Internet
   ↓
Ingress / Gateway
   ↓
Microservices
   ↓
Azure SQL
   +
Azure Service Bus
   +
Key Vault
   +
Application Insights

This architecture provides a strong foundation for deploying modern .NET 9 microservices applications on Azure using Azure DevOps, Docker, ACR and AKS.

SignalR vs Azure Service Bus vs Kafka vs RabbitMQ: Complete Guide with Real-Time Examples and C# Code

2026-08-14T12:18:51.425+05:30

SignalR vs Azure Service Bus vs Kafka vs RabbitMQ: Complete Guide with Real-Time Examples and C# Code

Modern enterprise applications often use multiple applications, microservices, background workers, and front-end clients that need to communicate with each other. Choosing the right communication technology is therefore an important architectural decision.

Four technologies frequently considered in .NET and microservices architectures are:

ASP.NET Core SignalR
Azure Service Bus
RabbitMQ
Apache Kafka

Although they are sometimes compared with each other, they are designed for different communication patterns.

The most important principle is:

SignalR is primarily for real-time client communication, Azure Service Bus and RabbitMQ are message brokers, while Kafka is primarily an event-streaming platform.

1. Introduction

Consider an enterprise e-commerce application:

                    Angular Application
                           |
                           v
                     Order Web API
                           |
             +-------------+-------------+
             |             |             |
             v             v             v
        Payment        Inventory     Notification
        Service          Service        Service

Now consider the following requirements:

Notify the customer's browser immediately when the order status changes.
Send an order-processing message reliably to another microservice.
Route messages to different queues based on business requirements.
Store large volumes of events and allow multiple applications to process them independently.
Replay historical events for analytics or recovery.

One technology is not necessarily the best solution for all these requirements.

This is where SignalR, Azure Service Bus, RabbitMQ and Kafka come into the picture.

2. Quick Comparison

Technology	Primary Purpose	Best Use Case
SignalR	Real-time client communication	Notifications, chat, live dashboards
Azure Service Bus	Enterprise messaging	Reliable microservice communication
RabbitMQ	Message broker	Queues, routing, work distribution
Kafka	Event streaming	High-volume events, analytics, event pipelines

3. SignalR

SignalR is an ASP.NET Core library designed for real-time communication between a server and connected clients.

Instead of the client repeatedly asking:

Is my order ready?

Is my order ready?

Is my order ready?

the server can push an update immediately:

Order #1001 has been shipped.

SignalR supports WebSockets and fallback transports such as Server-Sent Events and Long Polling.

Typical SignalR use cases

Real-time notifications
Chat applications
Live dashboards
Order tracking
Stock/price updates
Monitoring applications
Progress notifications
Collaborative applications

4. SignalR Architecture

                ASP.NET Core Server
                       |
                       |
                    SignalR
                       |
              WebSocket Connection
                       |
          +------------+------------+
          |            |            |
          v            v            v
       Browser 1    Browser 2    Browser 3

The important point is that SignalR is generally about connected clients.

It is not intended to replace a durable enterprise message broker.

5. SignalR C# Example

Install the SignalR package if required:

dotnet add package Microsoft.AspNetCore.SignalR

Create a Hub:

public class NotificationHub : Hub
{
    public async Task SendNotification(string message)
    {
        await Clients.All.SendAsync(
            "ReceiveNotification",
            message);
    }
}

Configure the Hub:

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSignalR();

var app = builder.Build();

app.MapHub<NotificationHub>("/notificationHub");

app.Run();

A client can connect to:

/notificationHub

The server can then send:

await hubContext.Clients.All.SendAsync(
    "ReceiveNotification",
    "Order #1001 has been shipped.");

All connected clients receive the notification.

6. SignalR with Angular

Install the SignalR client:

npm install @microsoft/signalr

Create a connection:

import * as signalR from '@microsoft/signalr';

const connection =
    new signalR.HubConnectionBuilder()
        .withUrl('https://localhost:7000/notificationHub')
        .build();

connection.on('ReceiveNotification', message => {
    console.log(message);
});

await connection.start();

The communication looks like:

ASP.NET Core
     |
     | SignalR
     | WebSocket
     v
Angular Application

7. Azure Service Bus

Azure Service Bus is a fully managed enterprise message broker.

It is designed for reliable communication between applications and services.

Microsoft describes Azure Service Bus as a cloud messaging system supporting queues, topics, subscriptions and enterprise messaging scenarios.

Consider:

Order API
    |
    | OrderCreated
    v
Azure Service Bus
    |
    +------> Payment Service
    |
    +------> Inventory Service
    |
    +------> Notification Service

The producer and consumer don't need to execute at exactly the same time.

The broker provides the decoupling layer.

8. Azure Service Bus Queue

A queue generally represents a point-to-point communication model.

Producer
   |
   v
+----------------+
| Order Queue    |
+----------------+
   |
   v
Consumer

For example:

Order API
    |
    v
OrderQueue
    |
    v
Payment Service

The message can remain available until a consumer successfully processes it.

This is particularly useful for background processing and microservice workflows.

9. Azure Service Bus C# Producer

Install:

dotnet add package Azure.Messaging.ServiceBus

Producer:

using Azure.Messaging.ServiceBus;
using System.Text.Json;

string connectionString = "...";
string queueName = "orders";

await using var client =
    new ServiceBusClient(connectionString);

ServiceBusSender sender =
    client.CreateSender(queueName);

var order = new
{
    OrderId = 1001,
    CustomerId = 5001,
    Amount = 2500
};

string json = JsonSerializer.Serialize(order);

var message = new ServiceBusMessage(json);

await sender.SendMessageAsync(message);

Architecture:

Order API
    |
    | SendMessageAsync()
    v
Azure Service Bus
    |
    v
Order Queue

10. Azure Service Bus Consumer

ServiceBusProcessor processor =
    client.CreateProcessor(queueName);

processor.ProcessMessageAsync += async args =>
{
    string message =
        args.Message.Body.ToString();

    Console.WriteLine(message);

    await args.CompleteMessageAsync(args.Message);
};

processor.ProcessErrorAsync += args =>
{
    Console.WriteLine(args.Exception);
    return Task.CompletedTask;
};

await processor.StartProcessingAsync();

The consumer processes the message and explicitly completes it.

11. Azure Service Bus Topic

A topic is useful when the same business event needs to reach multiple subscribers.

                    Order API
                       |
                       v
                 OrderCreated
                       |
                       v
                Azure Service Bus
                     Topic
                       |
          +------------+------------+
          |            |            |
          v            v            v
     Subscription  Subscription  Subscription
       Payment       Inventory    Notification

For example:

OrderCreated
      |
      +---- Payment Service
      |
      +---- Inventory Service
      |
      +---- Notification Service
      |
      +---- Audit Service

Azure Service Bus topics and subscriptions provide a publish/subscribe model.

12. RabbitMQ

RabbitMQ is a popular message broker.

One of its most important architectural concepts is the exchange.

The typical flow is:

Producer
   |
   v
Exchange
   |
   | Routing
   |
   +--------+--------+
   |        |        |
   v        v        v
Queue A   Queue B   Queue C
   |        |        |
   v        v        v
Consumer  Consumer  Consumer

RabbitMQ supports different exchange types, including:

Direct
Topic
Fanout
Headers

This provides flexible message-routing capabilities.

13. RabbitMQ C# Producer

A commonly used .NET package is:

dotnet add package RabbitMQ.Client

Example:

using RabbitMQ.Client;
using System.Text;

var factory = new ConnectionFactory
{
    HostName = "localhost"
};

using var connection =
    await factory.CreateConnectionAsync();

using var channel =
    await connection.CreateChannelAsync();

await channel.QueueDeclareAsync(
    queue: "orders",
    durable: true,
    exclusive: false,
    autoDelete: false);

string message = "Order #1001 created";

byte[] body =
    Encoding.UTF8.GetBytes(message);

await channel.BasicPublishAsync(
    exchange: "",
    routingKey: "orders",
    body: body);

The message is sent to the RabbitMQ broker.

14. RabbitMQ Consumer

var consumer =
    new AsyncEventingBasicConsumer(channel);

consumer.ReceivedAsync += async (sender, args) =>
{
    string message =
        Encoding.UTF8.GetString(args.Body.ToArray());

    Console.WriteLine(message);

    await channel.BasicAckAsync(
        args.DeliveryTag,
        multiple: false);
};

await channel.BasicConsumeAsync(
    queue: "orders",
    autoAck: false,
    consumer: consumer);

The acknowledgement tells RabbitMQ that the message has been successfully processed.

15. Kafka

Apache Kafka is primarily an event-streaming platform.

This is an important distinction.

Kafka is not simply:

Producer -> Queue -> Consumer

Instead, Kafka uses:

Producer
    |
    v
Kafka Topic
    |
    +--- Partition 0
    |
    +--- Partition 1
    |
    +--- Partition 2

Kafka topics are divided into partitions, allowing data to be distributed and processed in parallel. Kafka consumers use offsets to track their position in the stream.

16. Kafka Consumer Groups

This is one of Kafka's most important concepts.

Suppose:

OrderCreated

is published to Kafka.

Multiple independent systems can consume it.

                    Kafka Topic
                        |
              +---------+---------+
              |         |         |
              v         v         v
          Payment    Analytics   Fraud
           Group       Group      Group

Each consumer group maintains its own position.

Therefore, the same event can be processed independently by multiple systems.

17. Kafka C# Producer

Install:

dotnet add package Confluent.Kafka

Producer:

using Confluent.Kafka;

var config = new ProducerConfig
{
    BootstrapServers = "localhost:9092"
};

using var producer =
    new ProducerBuilder<string, string>(config)
        .Build();

await producer.ProduceAsync(
    "orders",
    new Message<string, string>
    {
        Key = "1001",
        Value = "Order #1001 created"
    });

The architecture is:

Order API
    |
    v
Kafka Producer
    |
    v
orders topic

18. Kafka Consumer

var config = new ConsumerConfig
{
    BootstrapServers = "localhost:9092",
    GroupId = "payment-service",
    AutoOffsetReset = AutoOffsetReset.Earliest
};

using var consumer =
    new ConsumerBuilder<string, string>(config)
        .Build();

consumer.Subscribe("orders");

while (true)
{
    var result = consumer.Consume();

    Console.WriteLine(
        $"Received: {result.Message.Value}");
}

The consumer group is:

payment-service

Kafka uses offsets to keep track of which events have been processed.

19. The Most Important Difference: Queue vs Event Stream

This is one of the most frequently asked interview questions.

Traditional Message Queue

Conceptually:

Producer
   |
   v
Queue
   |
   v
Consumer

The primary purpose is to deliver work/messages to consumers.

Examples:

Azure Service Bus Queue
RabbitMQ Queue

Kafka Event Stream

Kafka is based around a durable event log:

Producer
   |
   v
+-----------------------------+
| Kafka Topic                 |
|                             |
| E1 E2 E3 E4 E5 E6 E7 ...    |
+-----------------------------+
       |
       +---- Consumer Group A
       |
       +---- Consumer Group B
       |
       +---- Consumer Group C

Events can be retained according to Kafka's retention configuration, and consumers can maintain their own offsets.

This makes Kafka particularly powerful for:

Event streaming
Event replay
Analytics
Data pipelines
Audit/event history
High-volume processing

20. SignalR vs Azure Service Bus

These technologies solve very different problems.

SignalR

Server
   |
   | Real-time
   v
Browser

Azure Service Bus

Service A
   |
   v
Azure Service Bus
   |
   v
Service B

Feature	SignalR	Azure Service Bus
Real-time browser updates	Excellent	No
WebSockets	Yes	No
Service-to-service messaging	Not primary	Excellent
Durable messaging	Not primary	Yes
Queue	No	Yes
Topic/Subscription	Client groups	Yes
Chat	Excellent	Not primary
Background processing	Not primary	Excellent
Enterprise workflows	Not primary	Excellent

21. Azure Service Bus vs RabbitMQ

These are much closer competitors.

Feature	Azure Service Bus	RabbitMQ
Message queue	Yes	Yes
Pub/Sub	Yes	Yes
Routing	Good	Excellent
Managed Azure service	Yes	No, unless using a managed RabbitMQ offering
Self-hosting	No	Yes
Azure integration	Excellent	Good
Exchanges	No	Yes
Routing keys	No	Yes
Enterprise messaging	Excellent	Excellent
Operational overhead	Lower	Higher if self-managed

If your application is heavily based on Azure, Azure Service Bus is often a natural choice because it is a managed Azure service.

RabbitMQ becomes attractive when you need broker-level control, flexible routing, self-hosting, or a platform-independent messaging layer.

22. RabbitMQ vs Kafka

These are also frequently compared.

Feature	RabbitMQ	Kafka
Primary purpose	Message broker	Event streaming
Queue	Excellent	Different model
Routing	Excellent	Good
Event streaming	Limited compared with Kafka	Excellent
Event retention	Not its primary model	Core capability
Event replay	Not primary	Excellent
Consumer groups	Different model	Core capability
Partitioning	No Kafka-style partition model	Core capability
High-volume streams	Good	Excellent
Work queues	Excellent	Possible
Complex routing	Excellent	Less central
Analytics pipelines	Good	Excellent

23. Kafka vs Azure Service Bus

Feature	Kafka	Azure Service Bus
Event streaming	Excellent	Good
Traditional queues	Possible	Excellent
Event replay	Excellent	Different model
Consumer groups	Core feature	Different model
Partitioning	Core feature	Different scaling model
Enterprise messaging	Excellent	Excellent
Analytics	Excellent	Good
Data pipelines	Excellent	Good
Azure-native applications	Good	Excellent
Operational complexity	Higher	Lower
Event history	Excellent	Different model

24. Real-World E-Commerce Architecture

Let's combine these technologies in a realistic enterprise application.

                        Angular
                           |
                           | HTTPS
                           v
                     Order Web API
                           |
                           |
                    +------+------+
                    |             |
                    v             v
              Azure Service      Kafka
                  Bus              |
                    |              |
          +---------+---------+    +----------+
          |         |         |    |          |
          v         v         v    v          v
       Payment   Inventory Notification Analytics
       Service     Service    Service
                                     
                           |
                           v
                       SignalR
                           |
                           v
                     Angular Browser

Let's understand the flow.

25. Step 1 — Customer Places Order

Angular calls:

POST /api/orders

The Order API creates the order.

Angular
   |
   v
Order API
   |
   v
Order Created

26. Step 2 — Azure Service Bus

The Order API sends a business message:

OrderCreated

to Azure Service Bus.

Order API
    |
    v
Azure Service Bus
    |
    +---- Payment Service
    |
    +---- Inventory Service

This gives the services loose coupling.

27. Step 3 — Kafka

The system can also publish an event:

OrderCreated

to Kafka.

                    Kafka
                      |
          +-----------+-----------+
          |           |           |
          v           v           v
      Analytics      Fraud      Reporting

These systems can independently process the event.

28. Step 4 — SignalR

Once the order status changes:

Order Processing
       |
       v
Order Shipped

the backend sends a SignalR notification:

Order Service
      |
      v
SignalR Hub
      |
      v
Angular

The customer immediately sees:

Your order has been shipped!

29. Complete Enterprise Architecture

                         +----------------+
                         |    Angular     |
                         +-------+--------+
                                 |
                                 | REST
                                 v
                         +---------------+
                         |   API Layer   |
                         +-------+-------+
                                 |
                         +-------+-------+
                         |               |
                         v               v
                +----------------+   +----------+
                | Azure Service  |   |  Kafka   |
                |     Bus        |   |          |
                +-------+--------+   +-----+----+
                        |                  |
             +----------+----------+      |
             |          |          |      |
             v          v          v      v
          Payment   Inventory  Notification Analytics
          Service     Service      Service
             |
             |
             v
       Business Processing
             |
             v
         SignalR Hub
             |
             v
          Angular UI

This architecture demonstrates why these technologies should not automatically be treated as interchangeable.

30. When Should You Use SignalR?

Use SignalR when the requirement is:

"I need to push information from my server to connected clients immediately."

Examples:

Live notification
Live chat
Order status
Real-time dashboard
Progress updates
Monitoring
Collaboration

31. When Should You Use Azure Service Bus?

Use Azure Service Bus when the requirement is:

"I need reliable enterprise messaging between applications or microservices."

Examples:

Order processing
Payment processing
Inventory processing
Background jobs
Business workflows
Microservice communication
Enterprise integration

32. When Should You Use RabbitMQ?

Use RabbitMQ when the requirement is:

"I need a flexible message broker with sophisticated routing and queue-based processing."

Examples:

Work queues
Task distribution
Microservice messaging
Complex routing
Pub/Sub
Self-hosted messaging infrastructure

33. When Should You Use Kafka?

Use Kafka when the requirement is:

"I need a high-throughput, durable event stream that can be consumed independently by many applications."

Examples:

IoT events
Clickstream
Analytics
Data pipelines
Event-driven architecture
Audit events
Financial events
Real-time processing
Large-scale event ingestion

34. Can We Use All Four Together?

Yes.

In a large enterprise system, using multiple technologies can be completely valid.

For example:

                 Customer
                    |
                    v
                Angular
                    |
                    v
                ASP.NET
                    |
          +---------+---------+
          |                   |
          v                   v
    Azure Service Bus       Kafka
          |                   |
          v                   v
   Business Services      Analytics
          |
          v
       SignalR
          |
          v
       Angular

RabbitMQ could also be introduced for a specialized workload where its routing or queueing model is advantageous.

The important thing is not to add technologies unnecessarily.

35. Common Architectural Mistakes

Mistake 1: Using SignalR as a message broker

Don't design:

Order API
    |
    v
SignalR
    |
    v
Payment Service

just because SignalR can send messages.

SignalR is primarily intended for real-time client communication.

Mistake 2: Using Kafka for every small background job

Kafka is extremely powerful, but that doesn't mean every background task requires Kafka.

For a simple:

Order API
    |
    v
Process Invoice

a traditional queue such as Azure Service Bus or RabbitMQ may be simpler.

Mistake 3: Choosing Kafka because it is "fast"

Architecture should not be:

Kafka is fast
       ↓
Use Kafka everywhere

Instead ask:

Do I need event streaming?
Do I need retention?
Do I need replay?
Do I need many consumer groups?
Do I need high-volume event processing?

If yes, Kafka becomes a strong candidate.

36. Interview Question: Are SignalR, Kafka, RabbitMQ and Service Bus Alternatives?

Answer: Not completely.

They overlap in some areas, but their primary purposes are different.

SignalR
   ↓
Real-time client communication

Azure Service Bus
   ↓
Enterprise message broker

RabbitMQ
   ↓
Message broker + flexible routing

Kafka
   ↓
Distributed event streaming

37. Interview Question: Kafka vs RabbitMQ?

A good answer:

RabbitMQ is primarily a message broker focused on queues, routing and message delivery, whereas Kafka is primarily an event-streaming platform designed around durable event streams, partitions, consumer groups and high-throughput processing. I would typically consider RabbitMQ for work queues and complex routing, and Kafka when I need scalable event streaming, retention, replay and multiple independent consumers.

38. Interview Question: SignalR vs Kafka?

A good answer:

SignalR is designed for real-time communication between servers and connected clients, typically browsers or mobile applications. Kafka is designed for durable, scalable event streaming between backend systems. In an enterprise application, I might use Kafka to distribute an OrderShipped event internally and SignalR to push the resulting status update to the customer's browser.

39. Interview Question: Service Bus vs RabbitMQ?

A good answer:

Both are message brokers. Azure Service Bus is a managed Azure messaging service with strong integration into the Azure ecosystem, while RabbitMQ provides a flexible broker with exchanges, bindings and routing capabilities and can be self-hosted. If the application is Azure-centric and we want minimal infrastructure management, Service Bus is attractive. If we require greater broker-level control or specific RabbitMQ routing capabilities, RabbitMQ can be a better choice.

40. Interview Question: Can SignalR and Azure Service Bus Be Used Together?

Absolutely.

For example:

Order Service
     |
     | OrderCompleted
     v
Azure Service Bus
     |
     v
Notification Service
     |
     v
SignalR
     |
     v
Browser

The Service Bus provides reliable backend messaging.

SignalR provides real-time delivery to the user.

This is often a cleaner architecture than trying to use SignalR for both responsibilities.

41. Final Decision Matrix

Requirement	Recommended Technology
Real-time browser notification	SignalR
Chat	SignalR
Live dashboard	SignalR
Microservice command	Azure Service Bus / RabbitMQ
Enterprise business messaging	Azure Service Bus
Complex broker routing	RabbitMQ
Background work queue	Azure Service Bus / RabbitMQ
High-volume event stream	Kafka
Event replay	Kafka
Data/analytics pipeline	Kafka
Multiple independent consumers	Kafka
Azure-native messaging	Azure Service Bus
Self-hosted broker	RabbitMQ / Kafka
Server-to-browser communication	SignalR

42. Conclusion

SignalR, Azure Service Bus, RabbitMQ and Kafka are powerful technologies, but they solve different problems.

The easiest way to remember the difference is:

+---------------------------------------------------+
|                   COMMUNICATION                   |
+---------------------------------------------------+
|                                                   |
|  SignalR                                          |
|  Server ---> Connected Clients                   |
|                                                   |
|  Azure Service Bus                                |
|  Application ---> Reliable Business Message      |
|                                                   |
|  RabbitMQ                                         |
|  Producer ---> Broker ---> Routed Queue           |
|                                                   |
|  Kafka                                            |
|  Producer ---> Durable Event Stream ---> Consumers|
|                                                   |
+---------------------------------------------------+

In one sentence:

Use SignalR for real-time client communication, Azure Service Bus for reliable Azure enterprise messaging, RabbitMQ for flexible broker-based messaging and routing, and Kafka for scalable, durable event streaming.

For a modern .NET 9/10 + Angular + Microservices + Azure enterprise application, a combination such as Azure Service Bus + Kafka + SignalR can be very effective when each technology is assigned a clear responsibility.

ASP.NET Core SignalR: Complete Guide with Real-Time Examples

2026-08-13T15:26:17.579+05:30

ASP.NET Core SignalR: Complete Guide with Real-Time Examples

Introduction

In traditional web applications, the client usually sends a request to the server and waits for a response.

For example:

Angular Application
       |
       | HTTP Request
       v
ASP.NET Core Web API
       |
       | HTTP Response
       v
Angular Application

This works very well for normal CRUD operations.

But consider applications where information needs to appear immediately:

💬 Chat applications
🔔 Real-time notifications
📊 Live dashboards
🚚 Order/shipment tracking
🏦 Stock-price updates
🛒 E-commerce order status
🏭 IoT monitoring
🎮 Multiplayer applications
👥 Collaboration applications
📢 System alerts

Polling the server repeatedly is one possible solution, but it is inefficient.

This is where ASP.NET Core SignalR becomes useful.

1. What is SignalR?

SignalR is a real-time communication library for ASP.NET Core applications.

It allows the server to push data to connected clients without requiring the client to continuously request new data.

The communication can look like this:

                 ASP.NET Core Server
                        |
                        |
                 SignalR Hub
                 /     |      \
                /      |       \
               /       |        \
          Client A   Client B   Client C

When something happens on the server, the server can immediately notify connected clients.

For example:

Order Created
     |
     v
ASP.NET Core
     |
     v
SignalR Hub
     |
     +-----------> Customer 1
     |
     +-----------> Customer 2
     |
     +-----------> Admin Dashboard

No client polling is required.

2. Why Do We Need SignalR?

Suppose an Angular application displays an order status.

The order status changes on the server:

Order #1001
Status = Processing

A few seconds later:

Order #1001
Status = Shipped

Without SignalR, the Angular application might repeatedly call:

GET /api/orders/1001

For example:

Every 5 seconds
     |
     +---- GET /api/orders/1001
     |
     +---- GET /api/orders/1001
     |
     +---- GET /api/orders/1001
     |
     +---- GET /api/orders/1001

This is called polling.

It creates unnecessary HTTP requests.

With SignalR:

Server detects status change
           |
           v
      SignalR Hub
           |
           v
     Connected Client
           |
           v
     UI updates immediately

3. SignalR Architecture

A simplified SignalR architecture looks like this:

             Angular / React / JavaScript
                         |
                         |
                 SignalR Connection
                         |
                         v
                  ASP.NET Core
                         |
                         v
                   SignalR Hub
                         |
              +----------+----------+
              |                     |
          Client A               Client B

The major components are:

Client
SignalR Connection
Hub
Hub Methods
Server-to-Client Calls
Client-to-Server Calls
Groups
Connection Management

4. What is a SignalR Hub?

A Hub is the central communication point between clients and the server.

A Hub is similar to a controller in the sense that it exposes methods, but its purpose is real-time communication rather than traditional HTTP request/response processing.

Example:

public class NotificationHub : Hub
{
    public async Task SendNotification(string message)
    {
        await Clients.All.SendAsync("ReceiveNotification", message);
    }
}

Here:

Clients.All

means all connected clients.

And:

SendAsync("ReceiveNotification", message)

invokes a client-side method named:

ReceiveNotification

5. Creating a SignalR Project

Suppose we have an ASP.NET Core Web API application.

Install SignalR if necessary:

dotnet add package Microsoft.AspNetCore.SignalR

Depending on the ASP.NET Core version and project type, SignalR server functionality may already be included through the ASP.NET Core shared framework.

6. Create a SignalR Hub

Create a folder:

Hubs

Then create:

NotificationHub.cs

Code:

using Microsoft.AspNetCore.SignalR;

public class NotificationHub : Hub
{
    public async Task SendNotification(string message)
    {
        await Clients.All.SendAsync(
            "ReceiveNotification",
            message);
    }
}

This is our first SignalR Hub.

7. Configure SignalR in Program.cs

In ASP.NET Core:

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllers();

builder.Services.AddSignalR();

var app = builder.Build();

app.UseHttpsRedirection();

app.MapControllers();

app.MapHub<NotificationHub>("/notificationHub");

app.Run();

The important line is:

builder.Services.AddSignalR();

This registers SignalR.

The following line maps the Hub:

app.MapHub<NotificationHub>("/notificationHub");

The client can connect to:

/notificationHub

For example:

https://localhost:7001/notificationHub

8. Understanding SendAsync()

Consider:

await Clients.All.SendAsync(
    "ReceiveNotification",
    message);

There are three important pieces.

Clients.All

Sends the message to every connected client.

ReceiveNotification

This is the client-side method name.

message

This is the data being sent.

Conceptually:

Server

Clients.All.SendAsync(
      "ReceiveNotification",
      "Order Created"
)

              |
              v

Client A -> ReceiveNotification("Order Created")

Client B -> ReceiveNotification("Order Created")

Client C -> ReceiveNotification("Order Created")

9. JavaScript Client

Install the SignalR JavaScript client:

npm install @microsoft/signalr

Then:

import * as signalR from '@microsoft/signalr';

Create a connection:

const connection =
    new signalR.HubConnectionBuilder()
        .withUrl("https://localhost:7001/notificationHub")
        .withAutomaticReconnect()
        .build();

Start the connection:

connection.start()
    .then(() => {
        console.log("SignalR connected");
    })
    .catch(error => {
        console.error(error);
    });

10. Receiving Messages

The server sends:

await Clients.All.SendAsync(
    "ReceiveNotification",
    message);

The JavaScript client listens for:

connection.on(
    "ReceiveNotification",
    (message) => {
        console.log(message);
    });

Now the complete flow is:

ASP.NET Core

Clients.All.SendAsync(
    "ReceiveNotification",
    message
)

            |
            v

JavaScript

connection.on(
    "ReceiveNotification",
    message => {
        console.log(message);
    }
)

The method names must match:

ReceiveNotification

11. Complete Real-Time Notification Example

Let's build a realistic notification system.

Imagine an e-commerce application.

When an order is created:

Customer places order
       |
       v
Order API
       |
       v
Database
       |
       v
SignalR Hub
       |
       v
Customer/Admin Dashboard

The UI immediately displays:

🔔 New Order Created

Order Number: ORD-1001
Status: Processing

12. Notification Hub

using Microsoft.AspNetCore.SignalR;

public class NotificationHub : Hub
{
    public async Task SendNotification(
        string title,
        string message)
    {
        await Clients.All.SendAsync(
            "ReceiveNotification",
            new
            {
                Title = title,
                Message = message,
                CreatedAt = DateTime.UtcNow
            });
    }
}

Now we can send an object rather than just a string.

13. Calling SignalR from an API Controller

Suppose we have:

[ApiController]
[Route("api/orders")]
public class OrdersController : ControllerBase
{
    private readonly IHubContext<NotificationHub> _hubContext;

    public OrdersController(
        IHubContext<NotificationHub> hubContext)
    {
        _hubContext = hubContext;
    }

    [HttpPost]
    public async Task<IActionResult> CreateOrder(
        CreateOrderRequest request)
    {
        // Save order into database

        var orderId = 1001;

        await _hubContext.Clients.All.SendAsync(
            "OrderCreated",
            new
            {
                OrderId = orderId,
                Status = "Processing"
            });

        return Ok(new
        {
            OrderId = orderId
        });
    }
}

This is an important real-world pattern.

We don't need to call a Hub method directly.

Instead, our Web API can inject:

IHubContext<NotificationHub>

and send messages to clients.

14. What is IHubContext?

IHubContext allows application code outside the Hub to communicate with connected clients.

For example:

Controller
Service
Background Worker
Azure Function
Message Consumer
     |
     v
IHubContext
     |
     v
SignalR Clients

Example:

private readonly IHubContext<NotificationHub> _hubContext;

Then:

await _hubContext.Clients.All.SendAsync(
    "ReceiveNotification",
    "New order created");

15. SignalR Client Types

SignalR provides several ways to target clients.

Clients.All

Send to everyone.

await Clients.All.SendAsync(
    "ReceiveNotification",
    message);

Clients.Caller

Send only to the client that made the request.

await Clients.Caller.SendAsync(
    "ReceiveNotification",
    message);

Clients.Others

Send to everyone except the caller.

await Clients.Others.SendAsync(
    "ReceiveNotification",
    message);

Clients.Client

Send to a specific connection.

await Clients.Client(connectionId)
    .SendAsync(
        "ReceiveNotification",
        message);

16. SignalR Groups

Groups are extremely useful in real-world applications.

Imagine an enterprise application with:

Sales Team
Support Team
Managers
Administrators

We don't want every user to receive every notification.

Instead, we can create groups.

SignalR

+----------------------+
| Sales Group          |
| User A               |
| User B               |
+----------------------+

+----------------------+
| Support Group        |
| User C               |
| User D               |
+----------------------+

+----------------------+
| Manager Group        |
| User E               |
| User F               |
+----------------------+

17. Joining a Group

public async Task JoinGroup(string groupName)
{
    await Groups.AddToGroupAsync(
        Context.ConnectionId,
        groupName);
}

A client can call:

JoinGroup("Sales")

Now the client belongs to the Sales group.

18. Sending to a Group

await Clients.Group("Sales")
    .SendAsync(
        "ReceiveNotification",
        "New sales order received");

Only clients in the Sales group receive the message.

This is very useful for:

Department notifications
Tenant-based applications
Project-based collaboration
Team dashboards
Chat rooms
Order-specific updates

19. Real-Time Order Tracking

Let's create a more realistic example.

Suppose an e-commerce order moves through:

Order Created
     |
     v
Processing
     |
     v
Packed
     |
     v
Shipped
     |
     v
Out for Delivery
     |
     v
Delivered

The server can notify the customer whenever the status changes.

Example:

await _hubContext.Clients
    .Group($"order-{orderId}")
    .SendAsync(
        "OrderStatusChanged",
        new
        {
            OrderId = orderId,
            Status = "Shipped"
        });

Only users watching that order receive the update.

20. Joining an Order Group

public async Task JoinOrderGroup(int orderId)
{
    await Groups.AddToGroupAsync(
        Context.ConnectionId,
        $"order-{orderId}");
}

Client:

await connection.invoke(
    "JoinOrderGroup",
    orderId);

Then listen for:

connection.on(
    "OrderStatusChanged",
    (order) => {
        console.log(
            `Order ${order.orderId} status: ${order.status}`
        );
    }
);

21. Angular + SignalR

SignalR works very well with Angular.

Install:

npm install @microsoft/signalr

Create:

signalr.service.ts

Example:

import { Injectable } from '@angular/core';
import * as signalR from '@microsoft/signalr';

@Injectable({
  providedIn: 'root'
})
export class SignalRService {

  private connection!: signalR.HubConnection;

  startConnection(): void {

    this.connection =
      new signalR.HubConnectionBuilder()
        .withUrl(
          'https://localhost:7001/notificationHub'
        )
        .withAutomaticReconnect()
        .build();

    this.connection
      .start()
      .then(() => {
        console.log('SignalR connected');
      })
      .catch(error => {
        console.error(error);
      });
  }

  onNotification(
    callback: (message: any) => void
  ): void {

    this.connection.on(
      'ReceiveNotification',
      callback
    );
  }
}

22. Using SignalR Service in Angular Component

import { Component, OnInit } from '@angular/core';
import { SignalRService } from './signalr.service';

@Component({
  selector: 'app-dashboard',
  templateUrl: './dashboard.component.html'
})
export class DashboardComponent
  implements OnInit {

  notification: any;

  constructor(
    private signalRService: SignalRService
  ) {}

  ngOnInit(): void {

    this.signalRService.startConnection();

    this.signalRService.onNotification(
      (message) => {

        this.notification = message;

        console.log(
          'Notification received:',
          message
        );
      }
    );
  }
}

23. SignalR Connection Lifecycle

A SignalR connection can have different states:

Disconnected
     |
     v
Connecting
     |
     v
Connected
     |
     v
Reconnecting
     |
     +----------+
     |          |
     v          v
 Connected   Disconnected

You can monitor connection events.

connection.onreconnecting(error => {
    console.log('Connection lost. Reconnecting...');
});

connection.onreconnected(connectionId => {
    console.log(
        'Reconnected:',
        connectionId
    );
});

connection.onclose(error => {
    console.log('SignalR connection closed');
});

24. Automatic Reconnection

SignalR supports automatic reconnect.

const connection =
    new signalR.HubConnectionBuilder()
        .withUrl('/notificationHub')
        .withAutomaticReconnect()
        .build();

You can also specify retry intervals:

.withAutomaticReconnect([
    0,
    2000,
    5000,
    10000
])

Meaning:

First retry   -> Immediately
Second retry  -> 2 seconds
Third retry   -> 5 seconds
Fourth retry  -> 10 seconds

25. WebSockets and SignalR

SignalR can use different transports.

The primary transport is:

WebSockets

Other supported transports can include:

Server-Sent Events
Long Polling

Conceptually:

SignalR
   |
   +---- WebSockets
   |
   +---- Server-Sent Events
   |
   +---- Long Polling

SignalR selects an appropriate transport based on the environment and client/server capabilities.

Therefore, developers normally don't need to implement WebSocket management manually.

26. SignalR vs WebSocket

A common interview question is:

"What is the difference between WebSockets and SignalR?"

WebSocket

WebSocket is a communication protocol that provides full-duplex communication between client and server.

SignalR

SignalR is a higher-level framework/library that provides real-time communication and can use WebSockets when available, while also supporting fallback transports.

Feature	WebSocket	SignalR
Communication	Real-time	Real-time
Abstraction	Low-level	High-level
Automatic reconnect	Implement yourself	Supported
Groups	Implement yourself	Built-in
Client management	Manual	Built-in abstractions
ASP.NET Core integration	Manual	Excellent
Hub concept	No	Yes

27. SignalR vs Polling

Polling

Client
  |
  | Request
  v
Server
  |
  | Response
  v
Client

Wait

Client
  |
  | Request
  v
Server

This generates repeated requests.

SignalR

Client
   |
   | Establish connection
   v
SignalR Server
   |
   | Connection remains available
   |
   | <--- Real-time message
   |
   | <--- Real-time message
   |
   | <--- Real-time message

This is much more suitable for real-time communication.

28. Authentication with SignalR

Enterprise applications usually need authentication.

For example:

Angular
   |
   | JWT
   v
ASP.NET Core
   |
   v
SignalR Hub

Configure authentication:

builder.Services.AddAuthentication(
    JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        // JWT configuration
    });

Then:

builder.Services.AddAuthorization();

Hub:

[Authorize]
public class NotificationHub : Hub
{
}

Now only authenticated users can connect to the Hub.

29. Accessing the Current User

Inside a Hub:

var userId =
    Context.User?.FindFirst("sub")?.Value;

You can also access:

Context.ConnectionId

and:

Context.User

This is useful for user-specific notifications.

30. User-Specific Notifications

Suppose User 1001 receives an order update.

We can send a message to that specific user:

await _hubContext.Clients
    .User("1001")
    .SendAsync(
        "ReceiveNotification",
        "Your order has been shipped.");

This is different from:

Clients.All

because only the specified user receives the notification.

31. Real-Time Enterprise Architecture

A typical enterprise architecture might look like:

                Angular
                   |
                   |
             API Management
                   |
          +--------+--------+
          |                 |
       Web API          SignalR
          |                 |
          |                 |
     Microservices      SignalR Hub
          |                 |
          |          +------+------+
          |          |             |
          v          v             v
       Database   Client A      Client B
          |
          v
      Message Bus

For larger systems, events can flow through a message broker.

For example:

Order Service
     |
     v
Azure Service Bus / Kafka
     |
     v
Notification Service
     |
     v
SignalR
     |
     +------> Angular Client
     |
     +------> Admin Dashboard

This provides a scalable architecture.

32. SignalR with Background Services

SignalR is not limited to controllers.

A background service can also send notifications.

For example:

public class OrderMonitoringService
    : BackgroundService
{
    private readonly IHubContext<NotificationHub>
        _hubContext;

    public OrderMonitoringService(
        IHubContext<NotificationHub> hubContext)
    {
        _hubContext = hubContext;
    }

    protected override async Task ExecuteAsync(
        CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            await _hubContext.Clients.All.SendAsync(
                "ReceiveNotification",
                "Background process completed",
                stoppingToken);

            await Task.Delay(
                TimeSpan.FromMinutes(1),
                stoppingToken);
        }
    }
}

builder.Services.AddHostedService<OrderMonitoringService>();

33. SignalR in Microservices

In a microservices architecture, avoid making every microservice directly responsible for client connections.

A better architecture can be:

              Angular
                 |
                 v
           SignalR Service
                 ^
                 |
          Notification Service
                 ^
                 |
        Azure Service Bus / Kafka
                 ^
                 |
       +---------+---------+
       |         |         |
    Order     Payment   Shipping
   Service     Service   Service

For example:

Order Service
     |
     | OrderCreated event
     v
Azure Service Bus
     |
     v
Notification Service
     |
     v
SignalR
     |
     v
Angular

This creates better separation of responsibilities.

34. Scaling SignalR

This is very important for production systems.

Imagine:

100,000 users

connected to your application.

You may have multiple application instances:

                    Load Balancer
                         |
          +--------------+--------------+
          |              |              |
          v              v              v
      Server 1        Server 2        Server 3
          |              |              |
       SignalR        SignalR        SignalR

The problem is that clients can be connected to different servers.

For example:

Client A ---> Server 1

Client B ---> Server 2

If Server 1 sends a message locally, Server 2 may not automatically know about it.

For scaled-out SignalR deployments, you typically need a backplane or managed SignalR service depending on the architecture.

35. Azure SignalR Service

For Azure-based applications, Azure SignalR Service can handle SignalR connection scaling.

Architecture:

              Angular Clients
             /       |       \
            /        |        \
           v         v         v
      Azure SignalR Service
                 |
                 |
                 v
          ASP.NET Core API
                 |
                 v
          Microservices

This can reduce the responsibility of managing large numbers of persistent client connections in your application servers.

For enterprise applications deployed on Azure, Azure SignalR Service is an important option to evaluate.

36. SignalR in AKS

If your application runs in AKS:

                    Azure
                      |
               Application Gateway
                      |
                  Ingress
                      |
          +-----------+-----------+
          |           |           |
          v           v           v
        Pod 1       Pod 2       Pod 3
          |           |           |
       SignalR     SignalR     SignalR

Persistent connections introduce additional considerations when scaling.

For large-scale deployments, Azure SignalR Service can simplify the connection-management layer.

37. Real-World Use Case: Live Dashboard

Imagine an operations dashboard:

+---------------------------------------+
|       REAL-TIME OPERATIONS DASHBOARD  |
+---------------------------------------+
| Orders Today              12,542      |
| Successful Orders         11,934      |
| Failed Orders                108      |
| Pending Orders               500      |
+---------------------------------------+

Recent Events
-----------------------------------------
10:20:12  Order #1001 Created
10:20:15  Order #1002 Created
10:20:18  Order #1003 Shipped
10:20:22  Order #1004 Delivered

Instead of refreshing the browser:

Dashboard
    |
    | SignalR
    v
ASP.NET Core
    |
    v
Real-time events

The dashboard updates automatically.

38. SignalR Error Handling

Always consider connection failures.

Server:

try
{
    await Clients.All.SendAsync(
        "ReceiveNotification",
        message);
}
catch (Exception ex)
{
    // Log exception
}

Client:

connection.start()
    .catch(error => {
        console.error(
            'SignalR connection failed',
            error
        );
    });

Use proper logging rather than silently ignoring errors.

39. Important SignalR Best Practices

1. Don't put business logic inside the Hub

Avoid:

public async Task CreateOrder()
{
    // 500 lines of business logic
}

Prefer:

Hub
 |
 v
Application Service
 |
 v
Domain Service
 |
 v
Repository

2. Use strongly typed contracts

Instead of sending arbitrary objects everywhere, define DTOs.

public class OrderStatusNotification
{
    public int OrderId { get; set; }

    public string Status { get; set; } = string.Empty;
}

Then:

await _hubContext.Clients.All.SendAsync(
    "OrderStatusChanged",
    notification);

3. Use Groups

For large applications, groups are useful for:

Tenant
Department
Project
Order
Chat Room
Team

4. Enable Authentication

Don't expose sensitive real-time data to unauthenticated clients.

5. Handle Reconnection

Use:

.withAutomaticReconnect()

and monitor:

onreconnecting
onreconnected
onclose

6. Don't Send Excessive Data

Instead of:

Send entire database record

send only what the UI needs:

{
    "orderId": 1001,
    "status": "Shipped"
}

40. Complete SignalR Flow

The complete application flow can be represented as:

             Angular Application
                     |
                     |
             SignalR Connection
                     |
                     v
              NotificationHub
                     |
                     ^
                     |
             IHubContext
                     ^
                     |
              Order Service
                     |
                     v
                  Database

With an event-driven architecture:

Angular
   |
   v
SignalR
   ^
   |
Notification Service
   ^
   |
Azure Service Bus / Kafka
   ^
   |
Order Service
   |
   v
Database

41. SignalR Interview Questions

What is SignalR?

SignalR is a real-time communication library for ASP.NET Core applications that allows servers to push messages to connected clients.

What is a Hub?

A Hub is a high-level abstraction for communication between clients and the server.

What is IHubContext?

IHubContext allows application components outside a Hub to send messages to connected SignalR clients.

What is Clients.All?

It sends a message to all connected clients.

What is Clients.Caller?

It sends a message only to the client that initiated the Hub call.

What are SignalR Groups?

Groups allow connected clients to be logically grouped and targeted collectively.

Can SignalR work with Angular?

Yes. Microsoft provides the @microsoft/signalr JavaScript/TypeScript client.

Does SignalR always use WebSockets?

No. SignalR can use WebSockets when available and supports fallback transports.

How do you scale SignalR?

Depending on the deployment, options include a backplane or a managed service such as Azure SignalR Service.

Can SignalR be used with microservices?

Yes. A common pattern is to publish domain/application events through a message broker and have a notification component deliver relevant events to clients through SignalR.

42. SignalR vs REST API

Feature	REST API	SignalR
Communication	Request/Response	Real-time
Server Push	No	Yes
Persistent Connection	Usually no	Yes
CRUD	Excellent	Not primary purpose
Notifications	Limited	Excellent
Chat	Not ideal	Excellent
Live Dashboard	Polling required	Excellent
Order Tracking	Polling required	Excellent
WebSockets	Not inherent	Supported

SignalR does not replace REST APIs.

A modern application commonly uses both:

REST API
    |
    +---- CRUD operations

SignalR
    |
    +---- Real-time notifications
    +---- Live status
    +---- Events

43. Recommended Enterprise Architecture

For a large .NET + Angular application, a practical architecture could be:

                         Angular
                            |
             +--------------+--------------+
             |                             |
          REST API                      SignalR
             |                             |
             v                             v
       API Management              Azure SignalR
             |                             |
             v                             |
      ASP.NET Core APIs                    |
             |                             |
       +-----+------+                      |
       |            |                      |
       v            v                      |
    Orders       Payments                  |
       |            |                      |
       +-----+------+                      |
             |                             |
             v                             |
       Azure Service Bus ------------------+
                     |
                     v
              Notification Service

This architecture provides:

Separation of responsibilities
Real-time client communication
Event-driven integration
Better scalability
Better support for microservices
Reduced polling
Better user experience

44. Conclusion

SignalR is one of the most useful technologies in the ASP.NET Core ecosystem when an application needs real-time communication.

It is especially useful for:

🔔 Notifications
💬 Chat
📊 Live dashboards
🚚 Order tracking
📈 Monitoring
🏭 IoT applications
👥 Collaboration
⚡ Real-time enterprise applications

The key concepts to remember are:

SignalR
   |
   +-- Hub
   |
   +-- Clients
   |
   +-- IHubContext
   |
   +-- Groups
   |
   +-- Connections
   |
   +-- Authentication
   |
   +-- Automatic Reconnection
   |
   +-- Scaling
   |
   +-- Azure SignalR Service

For a simple application:

Angular
   |
   v
SignalR Hub
   |
   v
ASP.NET Core

For an enterprise application:

Angular
   |
   +------ REST API
   |
   +------ SignalR
             |
             v
      Azure SignalR Service
             |
             v
      Notification Service
             |
             v
   Azure Service Bus / Kafka
             |
             v
       Microservices

The biggest advantage of SignalR is that the application doesn't have to continuously ask:

"Has anything changed?"

Instead, the server can tell the client immediately when something important happens.

That makes SignalR a strong choice for building responsive, event-driven, real-time .NET applications.

The Ultimate Guide to AI Tools (2026)

2026-08-07T12:20:27.999+05:30

The Ultimate Guide to AI Tools (2026)

Complete Comparison of 50+ AI Tools, Features, Pricing, Industry Use Cases, Owners, Download Links & Expert Recommendations

Introduction

Artificial Intelligence (AI) has rapidly evolved from an experimental technology into an essential business tool. Today, organizations across software development, healthcare, banking, education, marketing, manufacturing, and entertainment rely on AI to improve productivity, automate repetitive tasks, generate insights, and create innovative products.

Whether you're a software engineer, solution architect, technical lead, business analyst, content creator, or entrepreneur, selecting the right AI tool can significantly improve efficiency and reduce costs.

This guide provides an overview of the most popular AI tools available in 2026, their owners, release dates, pricing models, industry applications, and key advantages.

Why AI Matters

Organizations are adopting AI because it helps them:

Increase employee productivity
Automate repetitive work
Improve customer experience
Reduce operational costs
Generate software code faster
Create images, videos, and presentations
Analyze large amounts of data
Make better business decisions

Top AI Tools Comparison

AI Tool	Owner	Released	Free Version	Primary Use	Best Industry	Major Advantage	Official Website
ChatGPT	OpenAI	Nov 2022	Yes	AI Assistant	Software, Education	Best all-round AI assistant	https://chat.openai.com
Claude	Anthropic	Mar 2023	Yes	AI Assistant	Enterprise, Legal	Excellent reasoning	https://claude.ai
Gemini	Google	Dec 2023	Yes	AI Assistant	Google Workspace	Deep Google integration	https://gemini.google.com
Microsoft Copilot	Microsoft	2023	Limited	Productivity	Microsoft 365	Office integration	https://copilot.microsoft.com
GitHub Copilot	GitHub (Microsoft)	2021	Limited	Coding	Software Development	AI pair programmer	https://github.com/features/copilot
Cursor AI	Anysphere	2023	Limited	Coding IDE	Software Engineering	Understands entire codebase	https://cursor.com
Perplexity AI	Perplexity	2022	Yes	Research	Research	Live web citations	https://perplexity.ai
NotebookLM	Google	2023	Yes	Knowledge Management	Education	Works with your own documents	https://notebooklm.google
Midjourney	Midjourney	2022	No	Image Generation	Design	Premium artistic images	https://midjourney.com
Adobe Firefly	Adobe	2023	Limited	Image Generation	Marketing	Commercial-safe AI images	https://firefly.adobe.com
Canva AI	Canva	2023	Limited	Design	Marketing	Easy graphics & presentations	https://canva.com
Runway ML	Runway	2022	Limited	Video Generation	Media	AI video creation	https://runwayml.com
ElevenLabs	ElevenLabs	2022	Yes	Voice AI	Podcasts	Natural voice generation	https://elevenlabs.io
Gamma	Gamma	2023	Limited	Presentations	Business	AI slide generation	https://gamma.app
Zapier AI	Zapier	2023	Limited	Automation	Enterprise	Workflow automation	https://zapier.com

Industry-wise Best AI Tools

Software Development

ChatGPT
GitHub Copilot
Cursor AI
Claude
DeepSeek

Benefits

Code generation
Debugging
Unit testing
API development
Documentation

Banking & Finance

Recommended Tools:

Claude
ChatGPT
Microsoft Copilot

Used For:

Financial analysis
Compliance
Fraud detection
Customer support

Healthcare

Recommended Tools:

ChatGPT
Claude
Microsoft Copilot

Applications:

Clinical documentation
Medical research
Patient assistance
Knowledge management

Education

Recommended Tools:

ChatGPT
Gemini
NotebookLM
Canva AI

Applications:

Lesson planning
Student learning
Research
Content generation

Digital Marketing

Recommended Tools:

ChatGPT
Canva AI
Midjourney
Adobe Firefly
Ideogram

Applications:

Blog writing
Social media
SEO
Image creation
Campaign planning

Video Production

Recommended Tools:

Runway ML
Pika
Synthesia
HeyGen

Applications:

AI videos
Marketing
Corporate training
YouTube content

Free vs Paid AI Tools

Completely Free (Basic Plans)

ChatGPT
Claude
Gemini
Perplexity
NotebookLM
ElevenLabs (limited)
DeepSeek

Limited Free Tier

GitHub Copilot
Cursor AI
Microsoft Copilot
Canva AI
Gamma
Zapier AI
Runway ML

Paid Only or Mostly Paid

Midjourney
Synthesia

Where Each AI Tool Fits in the Enterprise

User Interface Layer

ChatGPT
Gemini
Claude
Copilot

Development Layer

GitHub Copilot
Cursor AI
DeepSeek

Creative Layer

Midjourney
Canva AI
Firefly
Runway ML

Knowledge Layer

NotebookLM
Perplexity AI

Automation Layer

Zapier AI
Make AI

How to Choose the Right AI Tool

Before selecting an AI platform, ask these questions:

What problem am I solving?
Is a free version sufficient?
Does it integrate with my existing tools?
Is my data secure?
Can it scale as my business grows?
What is the total cost of ownership?
Does it provide enterprise support?

Major Advantages of AI Tools

Increase productivity
Save development time
Reduce operational costs
Improve customer experience
Generate high-quality content
Automate repetitive tasks
Enhance software quality
Improve decision making
Enable 24×7 availability
Accelerate innovation

Future of AI (2027 and Beyond)

The next generation of AI will focus on:

AI Agents performing complete business workflows
Multi-modal AI (text, image, voice, video)
Industry-specific foundation models
Autonomous software development
AI-assisted decision making
Hyper-personalized customer experiences
Secure enterprise AI platforms
Responsible and explainable AI

Conclusion

Artificial Intelligence is no longer optional—it is becoming a fundamental capability for modern organizations. The right AI tool depends on your goals, industry, budget, and workflow.

For software professionals, a combination of ChatGPT, GitHub Copilot, Cursor AI, Claude, and Microsoft Copilot provides an excellent productivity stack. Creative professionals benefit from tools such as Midjourney, Canva AI, Adobe Firefly, Runway ML, and ElevenLabs, while researchers and analysts can leverage Perplexity AI and NotebookLM for trusted information retrieval and knowledge management.

Start by exploring the free versions, evaluate how each tool fits your workflow, and gradually adopt those that deliver the greatest value.

SonarQube: The Complete Guide for .NET Developers (2026)

2026-08-06T20:38:06.331+05:30

Working with SonarQube: The Complete Guide for .NET Developers (2026)

Learn how to integrate SonarQube with .NET, Azure DevOps, GitHub, Jenkins, and CI/CD pipelines to improve code quality, security, and maintainability.

Introduction to SonarQube
Why SonarQube?
SonarQube Architecture
Editions of SonarQube
Installation
Dashboard Overview
Core Concepts
Quality Gates
Quality Profiles
Code Smells
Bugs
Vulnerabilities
Security Hotspots
Technical Debt
Code Coverage
Duplicated Code
Static Code Analysis
Analyzing .NET Applications
Integration with Visual Studio
SonarScanner for .NET
Azure DevOps Integration
GitHub Actions Integration
Jenkins Integration
Pull Request Analysis
Branch Analysis
Enterprise Best Practices
Real-Time Enterprise Example
Interview Questions
Best Practices
Common Mistakes

What is SonarQube?

SonarQube is an automated code quality inspection platform that continuously analyzes your source code.

It identifies:

Bugs
Security vulnerabilities
Code smells
Duplicated code
Code coverage
Maintainability issues

Think of SonarQube as a code reviewer that never sleeps.

Instead of waiting for senior developers to review code manually, SonarQube automatically checks thousands of coding rules.

Why SonarQube?

Without SonarQube

Developer
     |
     V

Writes Code

     |

Code Review

     |

Deployment

Problems:

Bugs reach production
Duplicate code
Security issues
Low code coverage
Poor maintainability

With SonarQube

Developer

     |

Writes Code

     |

SonarQube Analysis

     |

Quality Gate

     |

Deploy

Bad code never reaches Production.

Real Enterprise Example

Suppose Amazon has 500 developers.

Every day

250 Pull Requests
80 Projects
Millions of Lines of Code

Manual review is impossible.

Instead

Developer

↓

Push Code

↓

Azure DevOps

↓

Build

↓

SonarQube Scan

↓

Quality Gate

↓

Deploy

Coverage <80%

OR

New Bugs >0

OR

Critical Vulnerability Exists

Deployment is blocked.

SonarQube Architecture

                 Developers
                     |
                     |
             SonarScanner
                     |
                     |
              SonarQube Server
                     |
        -------------------------
        |                       |
 Elasticsearch             PostgreSQL

Components

SonarScanner

Collects source code.

SonarQube Server

Analyzes the code.

Database

Stores

Reports
Issues
Quality Gates
History

Elasticsearch

Provides fast searching.

Editions

Community

Free

Supports

C#
Java
JavaScript
Python
SQL

Developer

Adds

Branch analysis
Pull Request Analysis
Security Reports

Enterprise

Adds

Portfolio
Governance
Advanced Security

Data Center

Large organizations.

Supports clustering.

Installing SonarQube

Requirements

Java 21
PostgreSQL
SonarQube ZIP
SonarScanner

Extract

sonarqube/

bin/

conf/

logs/

extensions/

data/

Start

Windows

StartSonar.bat

Linux

./sonar.sh start

Default URL

http://localhost:9000

Dashboard

After login

Project

Coverage

Security

Maintainability

Reliability

Duplications

Technical Debt

Everything is visible in one dashboard.

What is a Quality Gate?

Quality Gate decides

Can this code be released?

Example

Condition	Value
Bugs	0
Vulnerabilities	0
Coverage	>80%
Duplicates	<3%
Code Smells	<10

If any condition fails

Quality Gate = Failed

Deployment stops.

Quality Profile

Quality Profile contains coding rules.

Example

Avoid Empty Catch

No Hardcoded Password

Dispose IDisposable

Avoid SQL Injection

Avoid Null Reference

Naming Convention

Each language has its own profile.

Bugs

Example

string s = null;

Console.WriteLine(s.Length);

SonarQube

Bug

Possible NullReferenceException

Code Smells

Example

if(a==true)

Better

if(a)

Another example

public void Method()
{
}

Unused methods are reported.

Vulnerabilities

Example

string sql =
"SELECT * FROM Users WHERE Name='" + user + "'";

SonarQube reports

SQL Injection

Better

command.Parameters.Add("@Name", user);

Security Hotspots

Example

SHA1.Create();

Not always vulnerable.

But

Needs developer review.

Code Coverage

Coverage tells

How much code is tested.

Total Lines =1000

Covered =850

Coverage =85%

Higher coverage means better confidence.

Duplicate Code

Bad

CalculateTax()

CalculateTax()

CalculateTax()

Repeated everywhere.

SonarQube detects duplication.

Technical Debt

Suppose

100 Code Smells

Estimated Fix Time

18 Hours

Technical Debt

18 Hours

Reliability Rating

A

B

C

D

E

A is best.

Maintainability Rating

Measures

Complexity
Duplication
Readability

Security Rating

Measures

Vulnerabilities
Security Hotspots

Cyclomatic Complexity

Bad

if()
{
 if()
 {
   while()
   {
      switch()
      {
      }
 }
}

Complexity increases.

Keep methods simple.

Cognitive Complexity

Measures

How difficult the code is to understand.

Example

if()

foreach()

while()

switch()

Higher nesting

Higher complexity.

SonarScanner for .NET

Install

dotnet tool install --global dotnet-sonarscanner

Begin Analysis

dotnet sonarscanner begin \
/k:"EmployeeAPI" \
/d:sonar.host.url="http://localhost:9000" \
/d:sonar.token="TOKEN"

Build

dotnet build

End

dotnet sonarscanner end \
/d:sonar.token="TOKEN"

Dashboard updates automatically.

Visual Studio Integration

Install

SonarLint extension.

Benefits

Live issue detection
Coding suggestions
Security warnings
Connected Mode with SonarQube

Developers can fix issues before committing code.

Azure DevOps Integration

Pipeline

trigger:
- main

pool:
  vmImage: windows-latest

steps:

- task: SonarQubePrepare@7
  inputs:
    SonarQube: 'SonarQube'
    scannerMode: 'dotnet'
    projectKey: 'EmployeeAPI'

- task: DotNetCoreCLI@2
  inputs:
    command: build
    projects: '**/*.csproj'

- task: SonarQubeAnalyze@7

- task: SonarQubePublish@7
  inputs:
    pollingTimeoutSec: '300'

If the Quality Gate fails, configure the pipeline to stop the release or deployment stage.

GitHub Actions Integration

name: Build

on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: windows-latest

    steps:

    - uses: actions/checkout@v4

    - uses: actions/setup-dotnet@v4
      with:
        dotnet-version: '9.0.x'

    - run: dotnet tool install --global dotnet-sonarscanner

    - run: |
        dotnet sonarscanner begin /k:"EmployeeAPI" /d:sonar.host.url="${{ secrets.SONAR_HOST_URL }}" /d:sonar.token="${{ secrets.SONAR_TOKEN }}"
        dotnet build
        dotnet sonarscanner end /d:sonar.token="${{ secrets.SONAR_TOKEN }}"

Jenkins Integration

Pipeline

Checkout

↓

Restore Packages

↓

Build

↓

Run Unit Tests

↓

Sonar Scanner

↓

Quality Gate

↓

Deploy

Pull Request Analysis

Every PR receives

Bugs
Security Issues
Code Coverage
Duplication
New Code Quality

Reviewers see issues before merging.

Branch Analysis

Analyze

Main

Develop

Release

Feature

Hotfix

Each branch has its own quality metrics (supported in commercial editions).

Enterprise Workflow

Developer

↓

Git

↓

Azure DevOps

↓

Build

↓

Unit Test

↓

SonarQube

↓

Quality Gate

↓

Container Build

↓

Docker

↓

AKS

↓

Production

This helps ensure only quality-checked builds are deployed.

Best Practices

Analyze every Pull Request.
Enforce a Quality Gate in CI/CD.
Aim for at least 80% unit test coverage (adjust based on project needs).
Fix new issues before addressing legacy issues.
Keep Quality Profiles consistent across projects.
Use Connected Mode with SonarLint.
Review Security Hotspots rather than ignoring them.
Track technical debt regularly.
Customize rules to match your team's coding standards.
Treat critical vulnerabilities as release blockers.

Common Mistakes

❌ Ignoring Quality Gate failures

❌ Disabling important rules

❌ Running analysis only before releases

❌ Ignoring code duplication

❌ Not writing unit tests

❌ Using outdated Quality Profiles

❌ Hardcoding secrets in source code

Azure Data Factory Connectors

2026-08-06T16:32:45.785+05:30

Azure Data Factory Connectors: A Complete Guide with Real-Time Examples

Introduction

One of the biggest strengths of Azure Data Factory (ADF) is its ability to connect to hundreds of different data sources without requiring custom integration code.

Whether your data is stored in:

SQL Server
Oracle
SAP
Azure Storage
AWS S3
Salesforce
REST APIs
Snowflake
MongoDB
PostgreSQL
MySQL

Azure Data Factory provides built-in Connectors that make it easy to move and transform data.

Think of a connector as a bridge between Azure Data Factory and an external data source or destination.

What is an Azure Data Factory Connector?

A connector is a built-in component that enables Azure Data Factory to communicate with external systems.

Without connectors, developers would have to write custom code to:

Authenticate
Read data
Write data
Handle errors
Manage connections

ADF connectors eliminate this complexity by providing a standardized way to access data.

How Connectors Work

          Azure Data Factory
                 │
      ------------------------
      │                      │
 Source Connector      Destination Connector
      │                      │
 SQL Server            Azure SQL Database
 Oracle                Azure Blob Storage
 REST API              Azure Data Lake
 SAP                   Synapse Analytics
 Salesforce            Snowflake

A connector can act as both a Source (reading data) and a Sink (writing data), depending on the service.

Types of Connectors

Azure Data Factory categorizes connectors into several groups.

1. Azure Connectors

These connect to Azure-native services.

Examples:

Azure SQL Database
Azure Blob Storage
Azure Data Lake Storage Gen2
Azure Synapse Analytics
Azure Cosmos DB
Azure Table Storage
Azure Files
Azure SQL Managed Instance
Azure Key Vault
Azure Database for PostgreSQL
Azure Database for MySQL

Real-Time Example

A retail company stores invoices in Azure Blob Storage.

ADF reads invoice files from Blob Storage and loads them into Azure SQL Database for reporting.

2. Database Connectors

Used to connect to relational databases.

Supported databases include:

SQL Server
Oracle
MySQL
PostgreSQL
IBM DB2
SAP HANA
MariaDB
Teradata
Vertica
Informix
Sybase

Real-Time Example

A bank stores customer accounts in Oracle.

ADF copies customer information every night into Azure Synapse Analytics.

3. File-Based Connectors

Used for files stored on-premises or in the cloud.

Supported formats include:

CSV
Excel
JSON
XML
Parquet
Avro
ORC
Text files

Storage Locations

Local File System
Azure Blob Storage
ADLS Gen2
Amazon S3
Google Cloud Storage
FTP
SFTP

Real-Time Example

Every hour, an SFTP server receives supplier CSV files.

ADF automatically:

Reads the files.
Validates the data.
Loads it into Azure SQL Database.
Archives the processed files.

4. Cloud Storage Connectors

ADF integrates with multiple cloud storage providers.

Examples:

Azure Blob Storage
Azure Data Lake Storage
Amazon S3
Google Cloud Storage
Oracle Cloud Storage

Real-Time Example

A company migrating from AWS to Azure uses ADF to copy files from Amazon S3 to Azure Data Lake Storage.

5. SaaS Application Connectors

ADF supports popular Software-as-a-Service applications.

Examples:

Salesforce
Dynamics 365
ServiceNow
HubSpot
Marketo
Shopify

Real-Time Example

A sales organization pulls Salesforce opportunity data every night into Azure Synapse for executive dashboards.

6. ERP Connectors

Enterprise Resource Planning systems are common in large organizations.

Supported systems include:

SAP ECC
SAP S/4HANA
SAP BW
SAP HANA

Real-Time Example

A manufacturing company extracts purchase orders from SAP every hour and loads them into a data warehouse for analytics.

7. CRM Connectors

Examples:

Dynamics 365
Salesforce
Zoho CRM

Real-Time Example

Marketing teams synchronize customer information from Dynamics 365 into Azure SQL for campaign analysis.

8. Big Data Connectors

Examples:

Apache Hive
Apache HBase
Apache Spark
Azure Databricks
Snowflake

Real-Time Example

ADF orchestrates a pipeline that copies raw IoT data to Azure Data Lake, triggers an Azure Databricks notebook for processing, and stores the results in Snowflake.

9. API Connectors

ADF can integrate with RESTful web services.

Supported APIs:

REST APIs
OData
HTTP endpoints
GraphQL (through HTTP/REST patterns)

Real-Time Example

An e-commerce application exposes order data through a REST API.

ADF retrieves new orders every 30 minutes and stores them in Azure SQL Database.

10. Messaging Connectors

Examples:

Azure Service Bus
Azure Event Hubs
Kafka (typically integrated through compatible services or custom approaches)

Real-Time Example

An online shopping application sends order events to Azure Event Hubs. ADF orchestrates downstream processing and stores aggregated data for reporting.

Commonly Used Enterprise Connectors

Connector	Source	Destination	Common Use Case
SQL Server	✔	✔	Transactional databases
Azure SQL Database	✔	✔	Cloud relational data
Oracle	✔	✔	Banking and ERP systems
Azure Blob Storage	✔	✔	File storage
ADLS Gen2	✔	✔	Data lakes
Amazon S3	✔	✔	Multi-cloud migration
Salesforce	✔	✔	CRM data
REST API	✔	✔	Third-party integrations
SAP	✔	✔	Enterprise ERP
Snowflake	✔	✔	Cloud data warehouse
PostgreSQL	✔	✔	Open-source databases
MySQL	✔	✔	Web applications

How ADF Uses a Connector

Suppose you need to move customer data from SQL Server to Azure SQL Database.

Step 1: Create a Linked Service

Source:

SQL Server

Destination:

Azure SQL Database

Step 2: Create Datasets

Customer Table

↓

Azure SQL Customer Table

Step 3: Create a Copy Activity

Source Dataset

↓

Copy Activity

↓

Destination Dataset

Step 4: Publish and Run

ADF automatically:

Connects to SQL Server.
Reads the customer records.
Transfers the data securely.
Writes the data to Azure SQL Database.
Logs execution details for monitoring.

No custom coding is required for the data movement.

Authentication Methods Supported by Connectors

Different connectors support different authentication mechanisms, including:

SQL Authentication
Windows Authentication
Azure Active Directory (Microsoft Entra ID)
Managed Identity
Service Principal
Shared Access Signature (SAS)
Storage Account Keys
OAuth 2.0
Anonymous Access (where applicable)

Best Practice: Use Managed Identity or Microsoft Entra ID whenever possible, and store secrets securely in Azure Key Vault instead of embedding credentials.

Best Practices for Using Connectors

Use parameterized Linked Services to avoid duplication.
Store secrets in Azure Key Vault.
Use Self-hosted Integration Runtime for on-premises systems.
Prefer Managed Identity for Azure resources.
Configure retries for transient network failures.
Enable monitoring and alerting for production pipelines.
Choose Incremental Loads over Full Loads for large datasets.
Validate connectivity before deploying to production.

Real Enterprise Scenario

A multinational retail company needs to integrate data from several systems:

System	Connector
SQL Server	SQL Server Connector
SAP	SAP Connector
Salesforce	Salesforce Connector
Oracle	Oracle Connector
Amazon S3	Amazon S3 Connector
Azure Blob Storage	Azure Blob Connector
REST APIs	REST Connector
Azure Synapse Analytics	Synapse Connector

Workflow:

SQL Server
     │
Oracle
     │
SAP
     │
Salesforce
     │
Amazon S3
     │
REST APIs
     │
Azure Data Factory
     │
Data Validation
     │
Azure Data Lake
     │
Azure Synapse Analytics
     │
Power BI Dashboard

This architecture enables a single, automated data integration platform that powers enterprise reporting, analytics, and machine learning while reducing manual effort and improving reliability.

Conclusion

Azure Data Factory connectors are the backbone of modern cloud data integration. They allow organizations to connect to a wide variety of on-premises, cloud, SaaS, and enterprise systems without writing complex integration code. By combining connectors with pipelines, activities, and Integration Runtime, organizations can build secure, scalable, and automated data workflows that support business intelligence, analytics, and AI initiatives.

Azure Data Factory (ADF) : A Complete Guide with Real-Time Examples

2026-08-06T16:25:57.007+05:30

Working with Azure Data Factory (ADF): A Complete Guide with Real-Time Examples

Introduction

In today's data-driven world, organizations collect data from multiple sources such as SQL databases, ERP systems, CRM applications, APIs, IoT devices, cloud storage, and third-party applications. However, collecting data is only the first step. The real challenge is moving, transforming, validating, and loading that data into a centralized repository for reporting, analytics, and machine learning.

This is where Azure Data Factory (ADF) comes into the picture.

Azure Data Factory is Microsoft's cloud-based data integration service that enables you to create, schedule, automate, and monitor data pipelines without writing large amounts of custom code.

In this article, we'll explore Azure Data Factory in detail, understand its architecture, components, and work through real-world enterprise examples.

What is Azure Data Factory?

Azure Data Factory (ADF) is a cloud ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) service provided by Microsoft Azure.

It allows organizations to:

Move data between systems
Transform data
Schedule workflows
Automate data processing
Integrate on-premises and cloud data
Build enterprise-grade data pipelines

Think of Azure Data Factory as the orchestrator of your data ecosystem.

Instead of manually copying files every day, ADF automates everything.

Why Do We Need Azure Data Factory?

Consider a retail company.

Every day data is generated from:

Sales Database
Inventory Database
SAP ERP
CRM
Excel Files
CSV Files
APIs
Azure Blob Storage
Amazon S3
Oracle Database

Management wants a dashboard every morning.

Without ADF:

Developers manually write scripts
Jobs fail frequently
No monitoring
Difficult maintenance

With ADF:

Automated pipelines
Error handling
Monitoring
Alerts
Scheduling
Scalability

Azure Data Factory Architecture

                SQL Server
                     |
                Oracle DB
                     |
                REST APIs
                     |
             Azure Blob Storage
                     |
             Amazon S3 Storage
                     |
          ------------------------
          | Azure Data Factory |
          ------------------------
             /      |       \
      Copy Data  Transform  Monitor
           |         |         |
        Azure SQL  Databricks  Logs
           |
     Power BI Reports

ADF acts as the central orchestrator.

Core Components of Azure Data Factory

1. Pipeline

A Pipeline is a logical grouping of activities.

Example:

Read CSV

Validate Data

Transform Data

Load into Azure SQL

Send Email

All these activities together form one pipeline.

2. Activity

Activities are individual tasks.

Examples:

Copy Activity
Lookup Activity
Stored Procedure Activity
Data Flow Activity
Execute Pipeline
Web Activity
Azure Function Activity

Think of activities as individual functions inside a program.

3. Linked Service

Linked Service stores connection information.

Example:

Azure SQL Connection

Server:
company.database.windows.net

Database:
SalesDB

Username

Password

ADF uses Linked Services to connect to data sources.

Supported sources include:

Azure SQL
SQL Server
Oracle
SAP
Blob Storage
ADLS
Snowflake
Amazon S3
REST APIs
MySQL
PostgreSQL
MongoDB

4. Dataset

Dataset represents the data you want to use.

Example:

Sales.csv

Customer Table

Blob Folder

Excel File

Linked Service = Connection

Dataset = Actual Data

5. Integration Runtime (IR)

Integration Runtime is the compute engine that moves data.

Three types:

Azure Integration Runtime

For cloud-to-cloud data movement.

Example:

Azure Blob → Azure SQL

Self-hosted Integration Runtime

Used when data is inside your company network.

Example:

On-Prem SQL Server → Azure SQL

Install Self-hosted IR on a local server.

Azure SSIS Integration Runtime

Used for running existing SSIS packages in Azure.

ETL vs ELT

ETL

Extract

↓

Transform

↓

Load

Transformation happens before loading.

ELT

Extract

↓

Load

↓

Transform

Transformation happens after loading.

Azure Data Factory supports both.

Real-Time Example 1

Daily Sales Data Migration

Company:

ABC Retail

Requirement:

Every night:

Read Sales data
Validate
Remove duplicates
Load into Data Warehouse

Pipeline:

Trigger

↓

Copy Sales Data

↓

Data Flow

↓

Remove Duplicates

↓

Load Azure SQL

↓

Success Email

Result:

Automated daily process.

Real-Time Example 2

Banking Industry

Every night:

Customer Accounts
Transactions
Loan Data
Credit Card Data

Need to move into Data Warehouse.

Pipeline:

Oracle

↓

ADF Copy

↓

Azure Data Lake

↓

Databricks

↓

Azure Synapse

↓

Power BI

Real-Time Example 3

Healthcare

Hospitals receive:

Patient Records
Laboratory Results
Pharmacy Data
Insurance Data

ADF:

SQL Server

↓

Validate

↓

Encrypt Sensitive Columns

↓

Store in Data Lake

↓

Analytics

Real-Time Example 4

E-Commerce

Every hour:

Orders
Payments
Delivery Status

Pipeline:

REST API

↓

ADF

↓

Blob Storage

↓

Azure SQL

↓

Dashboard

Copy Activity

Most commonly used activity.

Example:

Source:

SQL Server

Destination:

Azure Blob Storage

ADF copies data without writing code.

Data Flow

Data Flow performs transformations visually.

Operations include:

Filter
Sort
Join
Aggregate
Derived Columns
Lookup
Union
Pivot
Unpivot
Conditional Split
Window Functions

No coding required.

Parameters in Pipeline

Instead of hardcoding values:

Customer2025.csv

Use:

@pipeline().parameters.FileName

Now the same pipeline works for:

Customer2024.csv

Customer2025.csv

Customer2026.csv

Variables

Variables store temporary values.

Example:

CurrentDate

FileCount

Status

RowCount

Control Flow Activities

ADF provides many workflow activities:

If Condition
Switch
Until
ForEach
Wait
Execute Pipeline
Fail
Validation

Example:

If File Exists

↓

Copy

Else

Send Email

Triggers

Pipelines can execute automatically.

Types:

Schedule Trigger

Run every day.

Daily 2 AM

Tumbling Window

Useful for time-series processing.

Event Trigger

Automatically executes when a new file arrives.

Example:

Customer.csv uploaded

↓

Pipeline Starts

Monitoring

ADF provides monitoring dashboards showing:

Running Pipelines
Failed Pipelines
Duration
Success Rate
Retry Count
Error Messages

Alerts can be integrated with Azure Monitor.

Security

ADF supports enterprise-grade security:

Azure Key Vault integration
Managed Identity
RBAC
Private Endpoints
Encryption at Rest
Encryption in Transit

Avoid storing passwords directly in Linked Services. Use Azure Key Vault whenever possible.

Best Practices

Use parameterized pipelines.
Separate development, testing, and production environments.
Store secrets in Azure Key Vault.
Implement retry policies for transient failures.
Enable monitoring and alerts.
Use naming conventions for pipelines and datasets.
Optimize parallelism for large data loads.
Use Incremental Loading instead of Full Loading whenever possible.

Real Enterprise Scenario

A multinational retail company receives:

500 CSV files daily
SAP data every hour
Oracle data every night
CRM data through REST APIs
SQL Server transactions every 30 minutes

ADF orchestrates the process:

Blob Storage
        │
Oracle DB
        │
SQL Server
        │
REST API
        │
SAP
        │
   Azure Data Factory
        │
    Validation
        │
 Azure Data Lake
        │
 Azure Databricks
        │
 Azure Synapse Analytics
        │
     Power BI

Business users start their day with up-to-date dashboards, while data engineers benefit from automated, monitored, and scalable pipelines.

Advantages of Azure Data Factory

Fully managed cloud service
No infrastructure maintenance
Visual pipeline designer
Supports 100+ connectors
Highly scalable
Enterprise-grade security
Built-in scheduling and monitoring
Easy integration with Azure services
Supports hybrid cloud environments

Common Interview Questions

What is Azure Data Factory?

Azure Data Factory is a cloud-based data integration service that automates data movement and transformation across cloud and on-premises systems.

What is a Pipeline?

A pipeline is a logical container that groups one or more activities into a complete workflow.

What is the difference between Linked Service and Dataset?

A Linked Service defines the connection to a data source, while a Dataset represents the specific data (such as a table, file, or folder) within that source.

What is Integration Runtime?

Integration Runtime is the compute infrastructure used by Azure Data Factory to move and transform data. It is available as Azure Integration Runtime, Self-hosted Integration Runtime, and Azure SSIS Integration Runtime.

What is the difference between ETL and ELT?

ETL transforms data before loading it into the destination. ELT loads raw data first and performs transformations afterward, typically using the compute capabilities of the target system.

Conclusion

Azure Data Factory is one of the most powerful cloud-native data integration services available today. It enables organizations to build secure, scalable, and automated data pipelines that connect virtually any data source to modern analytics platforms. Whether you're migrating on-premises databases, orchestrating enterprise ETL workflows, integrating SaaS applications, or feeding business intelligence dashboards, ADF simplifies the entire process with minimal code and rich monitoring capabilities.

For modern data engineering teams working in Azure, Azure Data Factory is a foundational service that plays a key role in building reliable and enterprise-ready data platforms.

Azure Data Factory Connectors

How to Remove Duplicate Values from an Array in C#

2026-08-06T10:35:49.080+05:30

How to Remove Duplicate Values from an Array in C# – 6 Best Approaches with Examples

Removing duplicate values from an array is one of the most common tasks in C# programming. Whether you're processing user input, reading data from a database, or preparing data for reports, eliminating duplicate values improves data quality and application performance.

In this article, we'll explore multiple ways to remove duplicate values from an array in C#, ranging from beginner-friendly approaches to optimized solutions suitable for production applications and technical interviews.

Problem Statement

Suppose we have the following array:

int[] iArray = new int[] { 1, 1, 2, 2, 3, 3, 4, 5, 6, 6 };

Expected Output

1, 2, 3, 4, 5, 6

Method 1: Using LINQ Distinct() (Recommended)

The easiest and most readable approach is to use LINQ's Distinct() method.

using System;
using System.Linq;

class Program
{
    static void Main()
    {
        int[] iArray = { 1, 1, 2, 2, 3, 3, 4, 5, 6, 6 };

        int[] uniqueArray = iArray.Distinct().ToArray();

        Console.WriteLine(string.Join(", ", uniqueArray));
    }
}

Output

1, 2, 3, 4, 5, 6

Advantages

Very easy to write
Highly readable
Excellent for production code
Preserves the first occurrence order

Disadvantages

Requires LINQ
Slightly more memory usage than in-place algorithms

Time Complexity: O(n)

Space Complexity: O(n)

Method 2: Using HashSet (Fastest)

A HashSet<T> stores only unique values. Any duplicate values are automatically ignored.

using System;
using System.Collections.Generic;

class Program
{
    static void Main()
    {
        int[] iArray = { 1, 1, 2, 2, 3, 3, 4, 5, 6, 6 };

        HashSet<int> uniqueNumbers = new HashSet<int>(iArray);

        foreach (int item in uniqueNumbers)
        {
            Console.Write(item + " ");
        }
    }
}

Output

1 2 3 4 5 6

Advantages

Extremely fast
Automatically removes duplicates
Excellent for large collections

Disadvantages

Uses additional memory
Order is not guaranteed in all scenarios

Time Complexity: O(n)

Space Complexity: O(n)

Method 3: Using List Without LINQ

This approach is useful for beginners or interview situations where LINQ is not allowed.

using System;
using System.Collections.Generic;

class Program
{
    static void Main()
    {
        int[] iArray = { 1, 1, 2, 2, 3, 3, 4, 5, 6, 6 };

        List<int> result = new List<int>();

        foreach (int item in iArray)
        {
            if (!result.Contains(item))
            {
                result.Add(item);
            }
        }

        Console.WriteLine(string.Join(", ", result));
    }
}

Advantages

Easy to understand
No LINQ dependency

Disadvantages

Contains() performs a linear search
Not suitable for large datasets

Time Complexity: O(n²)

Method 4: Using Dictionary

A dictionary can also be used to keep track of values that have already been processed.

using System;
using System.Collections.Generic;

class Program
{
    static void Main()
    {
        int[] iArray = { 1, 1, 2, 2, 3, 3, 4, 5, 6, 6 };

        Dictionary<int, bool> dictionary = new Dictionary<int, bool>();

        foreach (int item in iArray)
        {
            if (!dictionary.ContainsKey(item))
            {
                dictionary.Add(item, true);
            }
        }

        foreach (var item in dictionary.Keys)
        {
            Console.Write(item + " ");
        }
    }
}

Advantages

Fast lookup
Good alternative to HashSet

Disadvantages

Stores unnecessary values (bool)
More verbose than HashSet

Time Complexity: O(n)

Method 5: Using Nested Loops (Without Collections)

This method is frequently asked during coding interviews because it demonstrates your understanding of the logic without relying on built-in collection classes.

using System;

class Program
{
    static void Main()
    {
        int[] iArray = { 1, 1, 2, 2, 3, 3, 4, 5, 6, 6 };

        for (int i = 0; i < iArray.Length; i++)
        {
            bool duplicate = false;

            for (int j = 0; j < i; j++)
            {
                if (iArray[i] == iArray[j])
                {
                    duplicate = true;
                    break;
                }
            }

            if (!duplicate)
            {
                Console.Write(iArray[i] + " ");
            }
        }
    }
}

Advantages

No LINQ
No collections
Demonstrates algorithmic thinking

Disadvantages

Slow for large datasets

Time Complexity: O(n²)

Space Complexity: O(1)

Method 6: Remove Consecutive Duplicates from a Sorted Array

If the array is already sorted, you only need to compare each element with the previous one.

using System;
using System.Collections.Generic;

class Program
{
    static void Main()
    {
        int[] iArray = { 1, 1, 2, 2, 3, 3, 4, 5, 6, 6 };

        List<int> result = new List<int>();

        if (iArray.Length > 0)
        {
            result.Add(iArray[0]);

            for (int i = 1; i < iArray.Length; i++)
            {
                if (iArray[i] != iArray[i - 1])
                {
                    result.Add(iArray[i]);
                }
            }
        }

        Console.WriteLine(string.Join(", ", result));
    }
}

Advantages

Very fast for sorted arrays
Simple implementation

Disadvantages

Works correctly only when duplicates are adjacent

Time Complexity: O(n)

Performance Comparison

Method	Time Complexity	Space Complexity	Best Use Case
LINQ Distinct()	O(n)	O(n)	General-purpose applications
HashSet	O(n)	O(n)	Large datasets and high performance
List + Contains()	O(n²)	O(n)	Learning and small datasets
Dictionary	O(n)	O(n)	Alternative to HashSet
Nested Loops	O(n²)	O(1)	Coding interviews
Sorted Array Comparison	O(n)	O(n)	Already sorted arrays

Real-World Use Cases

Removing duplicates is useful in many real-world applications, including:

Removing duplicate customer IDs
Eliminating repeated product codes
Cleaning imported CSV or Excel data
Processing unique email addresses
Generating distinct reports
Removing duplicate API response records
Preparing analytics data
Data migration and ETL processes

Interview Questions

1. What is the easiest way to remove duplicates from an array in C#?

Use Distinct() from LINQ.

2. Which collection automatically removes duplicates?

HashSet<T>.

3. What is the complexity of `Distinct()`?

Time Complexity: O(n)
Space Complexity: O(n)

4. Can we remove duplicates without using LINQ?

Yes. We can use:

HashSet
Dictionary
List
Nested loops

5. Which approach is preferred in production?

For most applications:

Distinct() for readability
HashSet<T> for maximum performance

6. Which approach is commonly asked in coding interviews?

The nested-loop solution is a popular interview question because it tests your understanding of algorithms without relying on built-in methods.

Best Practices

Use Distinct() for clean, readable, and maintainable code.
Use HashSet<T> when performance is critical.
Avoid List.Contains() for large datasets because it results in quadratic time complexity.
Understand the manual nested-loop solution for coding interviews.
Choose the approach that best matches your application's performance and readability requirements.

Conclusion

Removing duplicate values from an array is a common programming task in C#. The language offers several approaches, each with its own strengths.

Use Distinct() for simplicity and maintainability.
Use HashSet<T> for the best performance with large collections.
Learn the nested-loop solution to strengthen your problem-solving skills for interviews.
Select the right approach based on your application's size, performance requirements, and coding standards.

Mastering these techniques will help you write cleaner, more efficient C# code and prepare you for technical interviews with confidence.

If you'd like, I can also create:

an SEO-optimized version (targeting keywords like "Remove Duplicates from Array in C#"),
a professional feature image/thumbnail,
and schema-ready FAQ markup to improve your blog's Google search visibility.

ROW_NUMBER(), RANK(), DENSE_RANK(), NTILE(), PERCENT_RANK(), CUME_DIST() In Sql

2026-08-05T20:07:42.853+05:30

SQL Window Ranking Functions – ROW_NUMBER(), RANK(), DENSE_RANK(), NTILE(), PERCENT_RANK(), CUME_DIST()

These are Window Functions (Analytic Functions) introduced in SQL Server to perform calculations across a set of rows without grouping them.

Unlike GROUP BY, window functions do not reduce the number of rows. Every row remains in the result.

Why do we need Ranking Functions?

Suppose you have an Employee table.

Employee
-------------------------------------
EmpId | Name   | Department | Salary
-------------------------------------
1     | John   | IT         | 80000
2     | David  | IT         | 90000
3     | Smith  | IT         | 90000
4     | Alice  | HR         | 60000
5     | Bob    | HR         | 70000
6     | Mike   | HR         | 70000
7     | Tom    | Sales      | 50000

Suppose interviewer asks

Find highest salary employee.

Easy.

SELECT *
FROM Employee
ORDER BY Salary DESC;

But what if interviewer asks

Highest salary employee in each department
Top 3 salaries
Second highest salary
Remove duplicate records
Pagination
Rank employees

That's where ranking functions are used.

Syntax

FunctionName()
OVER
(
    PARTITION BY column
    ORDER BY column
)

There are two important clauses.

ORDER BY

Mandatory.

Determines ranking order.

Example

ORDER BY Salary DESC

PARTITION BY

Optional.

Splits data into groups before assigning ranks.

Example

PARTITION BY Department

Means every department ranking starts from 1.

1. ROW_NUMBER()

Assigns a unique sequential number to every row.

Even if salaries are equal,
numbers are different.

Example

SELECT
Name,
Salary,
ROW_NUMBER() OVER(ORDER BY Salary DESC) AS RowNum
FROM Employee;

Output

Name	Salary	ROW_NUMBER
David	90000	1
Smith	90000	2
John	80000	3
Bob	70000	4
Mike	70000	5
Alice	60000	6
Tom	50000	7

Notice

David and Smith have same salary.

Still

1
2

No ties.

Real-world use

Pagination

WITH CTE AS
(
SELECT *,
ROW_NUMBER() OVER(ORDER BY EmpId) RN
FROM Employee
)

SELECT *
FROM CTE
WHERE RN BETWEEN 11 AND 20;

This returns page 2.

Remove duplicates

Suppose duplicate emails exist.

WITH CTE AS
(
SELECT *,
ROW_NUMBER()
OVER(PARTITION BY Email ORDER BY EmpId) RN
FROM Customer
)

DELETE FROM CTE
WHERE RN>1;

Very common interview question.

2. RANK()

Assigns same rank to equal values.

But skips next rank.

Example

SELECT
Name,
Salary,
RANK() OVER(ORDER BY Salary DESC) RankNo
FROM Employee;

Output

Name	Salary	Rank
David	90000	1
Smith	90000	1
John	80000	3
Bob	70000	4
Mike	70000	4
Alice	60000	6
Tom	50000	7

Notice

1
1
3

Rank 2 is skipped.

Because two employees share rank 1.

Visualization

90000
David     Rank 1

90000
Smith     Rank 1

80000
John      Rank 3

Rank 2 missing.

Use Case

Sports Competition

100 Marks
Student A

100 Marks
Student B

95 Marks
Student C

Ranks

1
1
3

Exactly like Olympics.

3. DENSE_RANK()

Similar to RANK()

But

No gaps.

Example

SELECT
Name,
Salary,
DENSE_RANK()
OVER(ORDER BY Salary DESC)
AS DenseRank
FROM Employee;

Output

Name	Salary	Dense Rank
David	90000	1
Smith	90000	1
John	80000	2
Bob	70000	3
Mike	70000	3
Alice	60000	4
Tom	50000	5

Notice

No skipped numbers.

Visualization

90000
David     1

90000
Smith     1

80000
John      2

70000
Bob       3

70000
Mike      3

Difference between ROW_NUMBER(), RANK(), DENSE_RANK()

Salary	ROW_NUMBER	RANK	DENSE_RANK
90000	1	1	1
90000	2	1	1
80000	3	3	2
70000	4	4	3
70000	5	4	3
60000	6	6	4

Visual Difference

ROW_NUMBER()

90000   1
90000   2
80000   3
70000   4
70000   5

DENSE_RANK()

90000   1
90000   1
80000   2
70000   3
70000   3
60000   4

Which one should you use?

Need unique row number?

Use

ROW_NUMBER()

Examples

Pagination
Duplicate removal
Unique numbering

Need competition ranking?

Use

RANK()

Examples

Sports
Exam Ranking

Need consecutive ranking?

Use

DENSE_RANK()

Examples

Top N salaries
Product Ranking
Customer Ranking

PARTITION BY Example

Suppose

IT

John    80000
David   90000
Smith   90000

HR

Alice   60000
Bob     70000
Mike    70000

Query

SELECT
Department,
Name,
Salary,
DENSE_RANK()
OVER
(
PARTITION BY Department
ORDER BY Salary DESC
)
AS Rank
FROM Employee;

Output

Dept	Name	Salary	Rank
IT	David	90000	1
IT	Smith	90000	1
IT	John	80000	2
HR	Bob	70000	1
HR	Mike	70000	1
HR	Alice	60000	2

Notice ranking restarts for every department.

4. NTILE()

Splits rows into equal-sized groups (buckets).

Example

SELECT
Name,
Salary,
NTILE(4)
OVER(ORDER BY Salary DESC)
AS Quartile
FROM Employee;

If there are 20 rows,

Quartile 1

Top 25%

Quartile 2

Next 25%

Quartile 3

Next

Quartile 4

Lowest

Used in

Salary bands
Performance categories
Customer segmentation

5. PERCENT_RANK()

Returns the relative rank of a row as a value between 0 and 1.

Formula:

(RANK - 1) / (TotalRows - 1)

Example

SELECT
Name,
Salary,
PERCENT_RANK() OVER(ORDER BY Salary DESC) AS PercentRank
FROM Employee;

Sample output

Name	Salary	PercentRank
David	90000	0.00
Smith	90000	0.00
John	80000	0.33
Bob	70000	0.50
Mike	70000	0.50
Alice	60000	0.83
Tom	50000	1.00

Use cases:

Percentile analysis
Academic grading
Sales performance evaluation

6. CUME_DIST()

Returns the cumulative distribution, i.e., the proportion of rows with values less than or equal to the current row.

Example

SELECT
Name,
Salary,
CUME_DIST() OVER(ORDER BY Salary DESC) AS CumulativeDist
FROM Employee;

Sample output

Name	Salary	CUME_DIST
David	90000	0.29
Smith	90000	0.29
John	80000	0.43
Bob	70000	0.71
Mike	70000	0.71
Alice	60000	0.86
Tom	50000	1.00

Use cases:

Finding top performers
Statistical analysis
Distribution reporting

Common Interview Questions

1. Find the second highest salary.

Using DENSE_RANK():

WITH SalaryRank AS
(
    SELECT *,
           DENSE_RANK() OVER(ORDER BY Salary DESC) AS SalaryRank
    FROM Employee
)
SELECT *
FROM SalaryRank
WHERE SalaryRank = 2;

2. Find the top 3 highest salaries.

WITH SalaryRank AS
(
    SELECT *,
           DENSE_RANK() OVER(ORDER BY Salary DESC) AS SalaryRank
    FROM Employee
)
SELECT *
FROM SalaryRank
WHERE SalaryRank <= 3;

3. Find the highest-paid employee in each department.

WITH DeptRank AS
(
    SELECT *,
           DENSE_RANK() OVER(
               PARTITION BY Department
               ORDER BY Salary DESC
           ) AS DeptRank
    FROM Employee
)
SELECT *
FROM DeptRank
WHERE DeptRank = 1;

4. Remove duplicate rows.

WITH DuplicateRows AS
(
    SELECT *,
           ROW_NUMBER() OVER(
               PARTITION BY Email
               ORDER BY EmpId
           ) AS RN
    FROM Customer
)
DELETE FROM DuplicateRows
WHERE RN > 1;

5. Implement pagination (rows 21–30).

WITH OrderedEmployees AS
(
    SELECT *,
           ROW_NUMBER() OVER(ORDER BY EmpId) AS RN
    FROM Employee
)
SELECT *
FROM OrderedEmployees
WHERE RN BETWEEN 21 AND 30;

Quick Summary

Function	Duplicates Get Same Rank?	Gaps in Ranking?	Common Use Cases
`ROW_NUMBER()`	❌ No	❌ No	Pagination, duplicate removal, unique sequencing
`RANK()`	✅ Yes	✅ Yes	Competition ranking, exam results
`DENSE_RANK()`	✅ Yes	❌ No	Top N salaries, leaderboards
`NTILE(n)`	N/A	N/A	Divide rows into equal groups or quartiles
`PERCENT_RANK()`	Based on `RANK()`	May reflect rank gaps	Percentile calculations
`CUME_DIST()`	Includes ties together	N/A	Cumulative distribution and statistical reporting

Interview Tip

A very common interview question is:

"What is the difference between ROW_NUMBER(), RANK(), and DENSE_RANK()?"

A concise answer:

ROW_NUMBER() gives every row a unique sequential number, even when values are tied.
RANK() assigns the same rank to tied rows but skips the next rank(s).
DENSE_RANK() assigns the same rank to tied rows without leaving gaps.

Understanding these differences, along with PARTITION BY and ORDER BY, is essential for solving real-world SQL problems such as pagination, leaderboards, Top-N queries, duplicate removal, and departmental rankings.

Technical Lead Scenario-Based Interview Questions & Answers (.NET Core + Azure)

2026-08-05T20:07:20.136+05:30

Technical Lead Scenario-Based Interview Questions & Answers (.NET Core + Azure)

Scenario 1: Designing a New Enterprise Application

Interviewer

Your company wants to build a Loan Management System used by 500,000 customers. How would you design the architecture?

Expected Answer

First I gather functional and non-functional requirements.

Functional:

Customer Registration
Loan Application
Approval Workflow
Payment
Notifications

Non-functional:

High Availability
Scalability
Security
Performance
Disaster Recovery

Architecture:

Angular

      |

API Gateway (Azure APIM)

      |

-----------------------------

Identity Service

Loan Service

Payment Service

Notification Service

Document Service

-----------------------------

Azure Service Bus

Azure SQL

Blob Storage

Redis Cache

Application Insights

Patterns:

Clean Architecture
DDD
CQRS
Repository
Unit of Work
Event Driven

Azure Services

Azure App Service
Azure SQL
Azure Key Vault
Azure Service Bus
Azure Storage
Azure Monitor
Application Insights

This architecture supports independent deployment and scaling.

Scenario 2: Microservice Communication

Interviewer

Service A needs customer details from Service B.

Would you call it synchronously or asynchronously?

Answer

Depends on business requirement.

If immediate response required

Order Service

↓

Customer Service (REST API)

Use:

REST
gRPC

If eventual consistency acceptable

Order Created

↓

Azure Service Bus

↓

Customer Service

↓

Notification Service

↓

Audit Service

Benefits

Loose coupling
Retry
High availability
Independent scaling

Scenario 3: API Performance Issue

Interviewer

One API takes 18 seconds.

How will you investigate?

Answer

My investigation sequence:

Step 1

Application Insights

Check

Dependency calls
Exceptions
Slow SQL

Step 2

Database

Check

Execution Plan

Missing Index

Blocking

Deadlock

Step 3

Code

Look for

foreach(...)
{
   await repository.GetById();
}

N+1 problem.

Instead

Single Query

Step 4

Caching

Redis

Memory Cache

Step 5

Load Testing

Azure Load Test

JMeter

Finally optimize.

Scenario 4: Production is Down

Interviewer

Production suddenly becomes unavailable.

What do you do?

Answer

Never panic.

Follow Incident Process.

Step 1

Verify

Azure Health

App Service

SQL

Service Bus

Step 2

Rollback if deployment caused issue.

Step 3

Collect Logs

Application Insights

Azure Monitor

Step 4

Temporary Fix

Scale Out

Restart unhealthy instances

Disable problematic feature

Step 5

Root Cause Analysis

Document

Prevent recurrence.

Scenario 5: Azure SQL Database Reaches 100% CPU

Interviewer

Users complain application is slow.

Azure SQL CPU is 100%.

How do you solve it?

Answer

Check

Top Queries

Execution Plans

Missing Indexes

Deadlocks

Blocking Sessions

Large Table Scan

Optimize

Indexes

Pagination

Caching

Read-only replicas

Partitioning

Increase DTU/vCore only if required.

Scenario 6: Team Writes Duplicate Logic

Interviewer

Every developer writes validation differently.

What will you do?

Answer

As Technical Lead

I introduce

Coding Standards
Common Validation Library
Reusable NuGet Package
Static Code Analysis
Pull Request Guidelines

Every PR goes through review.

Consistency improves.

Scenario 7: Security Review

Interviewer

Security team found secrets inside appsettings.json.

How do you fix it?

Answer

Never store secrets.

Move

Connection String

API Keys

Certificates

Passwords

Azure Key Vault

Use

Managed Identity

Benefits

Secret Rotation
Encryption
Audit Trail

Scenario 8: API Authentication

Interviewer

How do you secure APIs?

Answer

Authentication

Azure AD

JWT

Authorization

Role Based Access

Policy Based Authorization

Claims

Other security

HTTPS

Rate Limiting

Input Validation

OWASP

Key Vault

APIM

Scenario 9: Scaling

Interviewer

Traffic increases 20 times during Black Friday.

How will your application survive?

Answer

Use

Stateless APIs

Redis Cache

Autoscaling

Azure App Service Scale Out

Azure SQL Elastic Pool

Azure Service Bus

CDN

Blob Storage

Horizontal scaling.

Scenario 10: Event Driven Architecture

Interviewer

Customer places an order.

Five services need notification.

How will you design?

Answer

Instead of

Order

↓

Inventory

↓

Payment

↓

Email

↓

Analytics

↓

CRM

Use

Order Service

↓

Publish Event

↓

Azure Service Bus Topic

↓

Inventory

↓

Payment

↓

Email

↓

Analytics

↓

CRM

Much more scalable.

Scenario 11: Code Review

Interviewer

What do you check during code review?

Answer

I check

✔ SOLID

✔ Naming

✔ Security

✔ Performance

✔ Exception Handling

✔ Logging

✔ Unit Tests

✔ Async

✔ Disposal

✔ SQL Injection

✔ Reusability

Scenario 12: Logging Strategy

Interviewer

How do you implement logging?

Answer

Levels

Information

Warning

Error

Critical

Use

ILogger

Serilog

Application Insights

Correlation ID

Never log passwords.

Scenario 13: Production Bug

Interviewer

A bug exists only in Production.

Not reproducible locally.

How do you investigate?

Answer

Check

Logs

Telemetry

Configuration

Environment Variables

Version

Feature Flags

Data Differences

Database

Application Insights Transaction Search.

Scenario 14: API Versioning

Interviewer

Old clients still use v1.

New features in v2.

How do you support both?

Answer

api/v1/customer

api/v2/customer

Deprecate older versions gradually.

Communicate timeline.

Scenario 15: CI/CD

Interviewer

Explain your deployment pipeline.

Answer

Developer

↓

Git

↓

Pull Request

↓

Build

↓

Unit Test

↓

SonarQube

↓

Publish Artifact

↓

Deploy Dev

↓

UAT

↓

Production Approval

↓

Production

Scenario 16: Production Deployment Failed

Interviewer

Half deployment completed.

Users affected.

Now what?

Answer

Blue-Green Deployment

Rollback

Database backward compatibility

Feature Flags

Health Check

Smoke Test

Scenario 17: Team Conflict

Interviewer

Two senior developers disagree on architecture.

What do you do?

Answer

Arrange technical discussion.

Compare

Performance

Maintainability

Scalability

Security

Development effort

Prototype if needed.

Decision based on facts.

Not opinions.

Scenario 18: Slow Microservice

Interviewer

One service delays every request.

Answer

Check

Dependency Calls

Database

Redis

Thread Pool

Memory

CPU

Distributed Tracing

Application Insights

Scenario 19: Legacy Monolith Migration

Interviewer

Company wants Microservices.

How will you migrate?

Answer

Use

Strangler Pattern

Extract modules one by one

Customer

Order

Payment

Notification

Eventually retire monolith.

Scenario 20: Observability

Interviewer

How do you monitor application health?

Answer

Metrics

CPU

Memory

Response Time

Availability

Dependency Failures

Exception Rate

Dashboard

Azure Monitor

Application Insights

Alerts

Teams

SMS

Scenario 21: Handling High Traffic During a Flash Sale

Interviewer

Your e-commerce site receives 10x normal traffic during a flash sale. Checkout APIs start timing out. What immediate and long-term actions would you take?

Answer

Immediate actions:

Scale out Azure App Service instances.
Increase Azure SQL compute if it's the bottleneck.
Enable/verify Redis caching for frequently accessed data.
Queue non-critical operations (emails, analytics) using Azure Service Bus.
Monitor live metrics in Application Insights.

Long-term improvements:

Optimize slow database queries and indexing.
Implement autoscaling rules.
Use CDN for static content.
Add load testing to CI/CD.
Introduce asynchronous processing where appropriate.

Scenario 22: Production Memory Leak

Interviewer

After a few days of uptime, your API becomes slow and eventually crashes with OutOfMemoryException. How do you investigate?

Answer

Check Application Insights memory metrics.
Capture memory dumps.
Analyze with Visual Studio Diagnostic Tools or dotMemory.
Look for:
- Undisposed IDisposable objects.
- Static collections growing indefinitely.
- Event handler leaks.
- Large object allocations.
Fix resource disposal and validate with load testing.

Scenario 23: Breaking Database Schema Changes

Interviewer

A database schema change is required, but older application versions are still running during deployment. How do you avoid downtime?

Answer

Follow backward-compatible database changes.
Add new columns before removing old ones.
Deploy application changes that support both schemas.
Migrate data in the background.
Remove deprecated schema only after all instances are updated.

Scenario 24: Third-Party API Failure

Interviewer

Your payment provider is intermittently unavailable. How do you prevent your application from failing?

Answer

Use resilience patterns:

Retry with exponential backoff.
Circuit Breaker.
Timeout policies.
Fallback responses where appropriate.
Queue requests if business rules allow.
Monitor failures and alert operations.

Common implementation: Polly in .NET.

Scenario 25: Mentoring a Junior Team

Interviewer

A junior developer repeatedly introduces bugs. As a Technical Lead, how do you handle it?

Answer

Review the code together instead of only rejecting PRs.
Explain the reasoning behind feedback.
Pair program on complex features.
Recommend learning resources.
Gradually increase responsibility.
Track improvement through regular one-on-one sessions.

This approach improves both code quality and team growth.

Technical Lead Interview Tips

Interviewers also evaluate how you think, not just what you know. When answering scenario questions:

Clarify assumptions before proposing a solution.
Consider scalability, security, performance, and maintainability together.
Mention trade-offs and why you chose a particular approach.
Include Azure services, design patterns, and DevOps practices where relevant.
Explain how you would communicate with stakeholders and lead the team during incidents.

These scenarios closely reflect the responsibilities you listed and are typical of Technical Lead interviews for enterprise .NET and Azure roles.

Apache Kafka Interview Questions & Answers

2026-08-02T17:40:39.762+05:30

Apache Kafka Interview Questions & Answers (Lead/Architect Level)

1. What is Apache Kafka?

Interview Answer

Apache Kafka is a distributed event streaming platform designed for high-throughput, fault-tolerant, scalable, and real-time data streaming. It allows applications to publish, store, and consume streams of events asynchronously.

Unlike traditional messaging systems that remove messages after they are consumed, Kafka stores messages for a configurable retention period, enabling multiple consumers to read the same events independently.

Kafka is widely used in:

Microservices
Event-Driven Architecture (EDA)
Real-time analytics
Log aggregation
IoT applications
Financial systems
E-commerce platforms

Real-Time Example

Amazon Order Processing

When a customer places an order:

Customer

↓

Order Service

↓

Kafka

↓

Payment Service

↓

Inventory Service

↓

Email Service

↓

Analytics Service

↓

Shipping Service

Every service receives the event independently.

Key Features

High Throughput
Fault Tolerance
Horizontal Scalability
Message Persistence
Event Replay
Distributed Architecture

Interview Tip

Kafka is an Event Streaming Platform, not just a Message Queue.

2. Explain Kafka Architecture

Kafka follows a distributed architecture.

                Producer

                    │

                    ▼

             Kafka Cluster

      ----------------------------

      Broker1

      Broker2

      Broker3

      ----------------------------

                    │

               Orders Topic

      ----------------------------

      Partition0

      Partition1

      Partition2

      ----------------------------

                    │

          Consumer Group

       Inventory Service

       Email Service

       Analytics Service

Components

Producer

Creates messages.

↓

Topic

Stores messages logically.

↓

Partition

Splits topic into multiple logs.

↓

Broker

Stores partitions.

↓

Consumer

Reads messages.

Flow

Angular

↓

.NET API

↓

Producer

↓

Topic

↓

Broker

↓

Consumer

↓

Database

Why Distributed?

If one broker crashes,

Other brokers continue serving requests.

3. What is a Broker?

A Broker is a Kafka server responsible for storing topic partitions and serving producer and consumer requests.

Example:

Kafka Cluster

Broker1

Broker2

Broker3

Each broker stores part of the data.

Responsibilities

Store messages
Handle producer requests
Serve consumers
Replicate data
Leader election
Offset management

Real Example

Topic has 6 partitions

Broker1

P0

P1

Broker2

P2

P3

Broker3

P4

P5

Interview Tip

Kafka Cluster = Collection of Brokers.

4. What is a Topic?

Topic is a logical category where messages are stored.

Example

Orders

Payments

Customers

Notifications

Producer writes

Orders Topic

Consumer reads

Orders Topic

Real Example

OrderCreated

↓

Orders Topic

Consumers

Inventory
Billing
Email
Analytics

All subscribe to the same topic.

Important

A Topic contains multiple partitions.

5. What is a Partition?

A partition is a physical subdivision of a topic.

Instead of

Orders

Kafka stores

Orders

Partition0

Partition1

Partition2

Why?

Allows parallel processing.

Suppose

1 Million Orders.

One partition

One Consumer

Slow.

Ten partitions

10 Consumers

Much faster.

Real Example

Partition0

Order1

Order4

Order7

Partition1

Order2

Order5

Order8

Partition2

Order3

Order6

Order9

Benefits

Scalability
Parallelism
Load balancing

6. What is an Offset?

Offset is the unique position of a message inside a partition.

Example

Partition0

Offset0

Offset1

Offset2

Offset3

Offset4

Consumer stores

Current Offset = 4

After restart,

Reads from Offset5.

Why Important?

Resume processing
Replay events
Fault recovery

Interview Tip

Offset is unique only within a partition, not across the entire topic.

7. Explain Consumer Groups

Consumer Group is a collection of consumers working together.

Example

Inventory Group

Consumer1

Consumer2

Consumer3

Topic

Partition0

Partition1

Partition2

Assignment

Consumer1 → Partition0

Consumer2 → Partition1

Consumer3 → Partition2

No duplicate processing occurs within the same group.

Multiple Groups

Orders Topic

↓

Inventory Group

↓

Email Group

↓

Analytics Group

Every group receives the same event independently.

Benefits

Scalability
Load balancing
Fault tolerance

8. Difference between Queue and Topic

Queue	Kafka Topic
Point-to-Point	Publish-Subscribe
One consumer receives a message	Multiple consumer groups receive the same message
Message is typically removed after processing	Message is retained for a configured period
Limited replay	Replay supported
Suitable for task distribution	Suitable for event streaming

Queue Example

Producer

↓

Queue

↓

Consumer A

Consumer B never receives the message.

Kafka Topic Example

Producer

↓

Orders Topic

↓

Inventory Group

↓

Email Group

↓

Analytics Group

Every group processes the event.

9. Kafka vs RabbitMQ

Kafka	RabbitMQ
Distributed Event Streaming	Traditional Message Broker
Stores messages in logs	Stores messages in queues
Very high throughput	Moderate throughput
Supports replay	Replay is limited
Excellent for analytics and streaming	Excellent for task queues and request/response
Partition-based scaling	Queue-based scaling

Real Use Cases

Kafka

Banking events
IoT
Real-time analytics
Clickstream processing

RabbitMQ

Email queues
Background jobs
Order processing tasks
RPC-style messaging

10. Why are partitions needed?

Partitions allow Kafka to scale horizontally.

Without partitions

1 Topic

↓

1 Consumer

With partitions

Partition0

Partition1

Partition2

↓

Consumer1

Consumer2

Consumer3

Benefits

Higher throughput
Better CPU utilization
Parallel processing
Horizontal scaling

11. How does Kafka guarantee ordering?

Kafka guarantees ordering only within a single partition.

Example

Partition0

Offset0 OrderCreated

Offset1 PaymentCompleted

Offset2 OrderShipped

Consumers read in offset order.

Across multiple partitions, there is no global ordering guarantee.

Best practice: If ordering is important for a business entity (for example, an order), use the same message key (OrderId) so Kafka routes all events for that key to the same partition.

12. What happens when a broker fails?

Kafka uses replication to maintain availability.

Broker1

Leader

↓

Broker2

Follower

↓

Broker3

Follower

If Broker1 fails:

Kafka elects a new leader from the in-sync replicas (ISR).
Producers and consumers continue working with the new leader.
When the failed broker comes back, it catches up and rejoins as a follower.

This minimizes downtime and prevents data loss when configured correctly.

13. What is replication factor?

The replication factor defines how many copies of each partition Kafka maintains.

Example:

Replication Factor = 3

Partition0

Leader → Broker1

Replica → Broker2

Replica → Broker3

If one broker fails, the replicas are available for leader election.

Trade-off:

Higher replication = better availability and durability.
Higher replication also requires more storage and network bandwidth.

14. What are producer acknowledgements (acks)?

The acks setting determines how many brokers must acknowledge a write before the producer considers it successful.

acks = 0

Producer does not wait.
Fastest.
Messages may be lost.

acks = 1

Waits for the leader broker only.
Good balance of speed and reliability.
Most common configuration.

acks = all (or -1)

Waits for all in-sync replicas.
Highest durability.
Slightly higher latency.

15. What is consumer lag?

Consumer lag is the difference between:

the latest offset written to a partition, and
the last offset processed by a consumer.

Example

Latest Offset = 1000

Consumer Offset = 900

Lag = 100

High lag may indicate:

Slow consumers
Insufficient consumer instances
Long-running processing
Resource bottlenecks

Monitoring consumer lag is essential in production systems.

16. What is an idempotent producer?

An idempotent producer ensures that retrying a message does not create duplicates.

Without idempotence:

Send Message

↓

Network Failure

↓

Retry

↓

Duplicate Message

With idempotence enabled:

Retry

↓

Kafka recognizes the duplicate

↓

Stores only one copy

In the .NET Confluent.Kafka client, this is enabled using:

var config = new ProducerConfig
{
    BootstrapServers = "localhost:9092",
    EnableIdempotence = true
};

This is especially important in payment and financial systems.

17. What is exactly-once processing?

Exactly-once processing (EOS) ensures that each event is processed only once, even in the presence of retries or failures.

Kafka achieves this through:

Idempotent producers
Transactions
Transaction-aware consumers

Example:

Transfer ₹100

↓

Kafka Event

↓

Retry

↓

Still processed only once

This is critical in banking, inventory, and billing systems where duplicate processing is unacceptable.

18. What is the role of Schema Registry?

Schema Registry centrally manages event schemas, commonly using Avro, Protobuf, or JSON Schema.

Benefits:

Producers and consumers agree on the event structure.
Supports schema evolution while maintaining compatibility.
Reduces message size (especially with Avro).
Prevents incompatible changes.

Example:

OrderCreated

OrderId

CustomerId

Amount

If a new field is added later, compatibility rules determine whether older consumers can still process the event.

19. How do you scale Kafka consumers?

Consumers scale through:

Increasing the number of partitions.
Adding more consumer instances to the same consumer group.

Example:

Topic

6 Partitions

↓

Consumer Group

Consumer1

Consumer2

Consumer3

Consumer4

Consumer5

Consumer6

Each consumer processes one partition.

Important rules:

A consumer group cannot process more partitions concurrently than the number of partitions available.
If there are more consumers than partitions, the extra consumers remain idle.

20. How do you integrate Kafka with .NET Core?

A common architecture is:

Angular

↓

ASP.NET Core Web API

↓

Kafka Producer

↓

Orders Topic

↓

Kafka Cluster

↓

Inventory Service

↓

Payment Service

↓

Email Service

↓

Analytics Service

Step 1: Install the Kafka client

dotnet add package Confluent.Kafka

Step 2: Configure the producer

var config = new ProducerConfig
{
    BootstrapServers = "localhost:9092"
};

using var producer =
    new ProducerBuilder<string, string>(config).Build();

Step 3: Publish an event

await producer.ProduceAsync(
    "orders",
    new Message<string, string>
    {
        Key = "1001",
        Value = JsonSerializer.Serialize(order)
    });

Step 4: Create a consumer

var config = new ConsumerConfig
{
    BootstrapServers = "localhost:9092",
    GroupId = "inventory-group",
    AutoOffsetReset = AutoOffsetReset.Earliest
};

using var consumer =
    new ConsumerBuilder<string, string>(config).Build();

consumer.Subscribe("orders");

while (true)
{
    var result = consumer.Consume();
    Console.WriteLine(result.Message.Value);
}

Angular Integration

Angular should not connect directly to Kafka.

Instead:

Angular

↓

HTTP POST

↓

ASP.NET Core Web API

↓

Kafka Producer

↓

Kafka Topic

The Web API validates the request, persists business data if needed, and publishes the event to Kafka.

Interview Summary (30-Second Answer)

"Apache Kafka is a distributed event-streaming platform used to build scalable, fault-tolerant, and asynchronous systems. Producers publish events to topics, which are split into partitions for parallel processing. Brokers store these partitions and replicate them for high availability. Consumer groups process events independently, while offsets track processing progress. In .NET Core, we typically use the Confluent.Kafka library to publish and consume events, exposing HTTP APIs for Angular clients while Kafka handles communication between backend microservices. This architecture improves scalability, resiliency, and loose coupling compared to direct service-to-service communication."

Apache Kafka

2026-08-02T17:36:31.180+05:30

Apache Kafka Complete Tutorial for .NET Core & Angular Developers (Lead/Architect Level)

This tutorial is designed for Senior .NET Developers, Technical Leads, Architects, and Microservices Developers.

By the end of this guide, you'll understand:

What is Kafka?
Why Kafka was created?
Kafka Architecture
Kafka Components
Topics vs Queue
Producers & Consumers
Consumer Groups
Partitions
Replication
Offset
Broker
ZooKeeper vs KRaft
Event Driven Architecture
Integrating Kafka with .NET Core
Integrating Angular with Kafka using Web API
Real-time Microservices Example
Deployment in Docker
Azure Integration
Best Practices
Interview Questions

Chapter 1 - Why Kafka?

Imagine Amazon.

Thousands of events happen every second.

Customer places order
↓

Payment Completed
↓

Inventory Updated
↓

Invoice Generated
↓

Email Sent
↓

SMS Sent
↓

Loyalty Points Added
↓

Analytics Updated

If every service directly called another service,

Order Service
      ↓
Payment Service
      ↓
Inventory Service
      ↓
Email Service
      ↓
Notification Service

Problems

Tight Coupling
Slow
Difficult to Scale
Single Point Failure

Instead

Order Service

↓

Kafka

↓

Payment

Inventory

Shipping

Analytics

Email

Notification

Everything becomes independent.

Chapter 2 - What is Kafka?

Kafka is

A Distributed Event Streaming Platform

Think of Kafka as

Post Office

Producer

↓

Post Office (Kafka)

↓

Consumer

Producer doesn't know who receives.

Consumer doesn't know producer.

Everything is asynchronous.

Chapter 3 - Kafka Architecture

               Producer

                  |

                  |

           Kafka Cluster

         -----------------

         Broker 1

         Broker 2

         Broker 3

         -----------------

          Topic

      Orders

     Partition-0

     Partition-1

     Partition-2

          |

          |

 Consumer Group

Kafka Components

Producer

Produces Message.

Example

Order API

Order Created

Producer sends

OrderCreated Event

Broker

Kafka Server.

Stores all messages.

Example

Broker 1

Broker 2

Broker 3

Kafka Cluster = Collection of Brokers.

Topic

Topic is a category.

Example

Orders

Payments

Inventory

Shipping

Notification

Each topic contains messages.

Partition

A Topic is divided into multiple partitions.

Example

Orders Topic

---------------------

Partition 0

Partition 1

Partition 2

Partition 3

Partitions enable

Parallel Processing
Scalability
High Throughput

Offset

Each message has an ID.

Kafka calls it Offset.

Example

Offset

0

1

2

3

4

5

Consumer remembers

Last Offset = 5

If application crashes

Restart

Continue from Offset 6.

Consumer

Reads messages.

Example

Inventory Service

Email Service

Analytics Service

Notification Service

Consumer Group

Multiple consumers work together.

Example

Consumer Group

Inventory-1

Inventory-2

Inventory-3

Kafka distributes partitions.

Example

Partition0 → Consumer1

Partition1 → Consumer2

Partition2 → Consumer3

No duplication.

Queue vs Topic

This is one of the most asked interview questions.

Traditional Queue

Producer

↓

Queue

↓

Consumer

Only ONE consumer gets message.

Example

Queue

Message

↓

Consumer A

Consumer B

Consumer C

Only Consumer A receives.

Kafka Topic

Producer

↓

Topic

↓

Consumer Group A

Consumer Group B

Consumer Group C

All groups receive same message.

Inside a group,

only one consumer receives.

Example

Order Created

↓

Orders Topic

↓

Inventory Group

↓

Email Group

↓

Analytics Group

↓

Shipping Group

Everyone gets copy.

Real Example

Customer Orders Mobile.

Order API

↓

Kafka Topic

↓

Payment

↓

Inventory

↓

Email

↓

SMS

↓

Analytics

↓

Recommendation Engine

Single Event

Many Consumers.

Queue vs Topic Comparison

Queue	Kafka Topic
One Consumer	Multiple Consumer Groups
Message Removed	Message Retained
Point-to-Point	Publish Subscribe
Low Scalability	Very High
Low Throughput	Millions/sec

Message Flow

Customer

↓

Angular

↓

.NET API

↓

Kafka Producer

↓

Orders Topic

↓

Broker

↓

Partition

↓

Consumer Group

↓

Inventory Service

↓

SQL

↓

Notification

↓

Email

Kafka Storage

Many developers ask

Where does Kafka save messages?

Kafka stores messages

Inside

Topic

↓

Partition

↓

Log Files

Example

orders-0.log

orders-1.log

orders-2.log

Messages are appended.

Kafka never inserts in middle.

Only append.

Offset 0

Offset 1

Offset 2

Offset 3

Kafka Message Format

Example

Key

OrderId=1001

Value

{
OrderId:1001,
Customer:"John",
Amount:2500
}

Replication

Suppose

Broker1 crashes.

Without replication

Data Lost.

With replication

Broker1

Leader

↓

Broker2

Follower

↓

Broker3

Follower

If Leader dies

Follower becomes Leader.

Producer Acknowledgement

acks=0

Fire and Forget

Fastest

Risky

acks=1

Leader confirms.

Most common.

acks=all

All replicas confirm.

Safest.

Delivery Guarantee

At Most Once

No Retry

May Lose.

At Least Once

Retry Enabled

Duplicate Possible.

Exactly Once

No Duplicate

No Loss

Used for Banking.

Kafka Ordering

Ordering guaranteed only

within a partition.

Example

Partition0

Order1

Order2

Order3

Always maintained.

Across partitions

No guarantee.

Why Partitions?

Imagine

10 Million Orders.

One partition

Single Consumer

Slow.

10 partitions

10 Consumers

10x faster.

.NET Core Integration

Architecture

Angular

↓

.NET API

↓

Kafka Producer

↓

Kafka Broker

↓

Inventory Service

↓

SQL Server

Install Package

dotnet add package Confluent.Kafka

Producer Example

using Confluent.Kafka;

var config = new ProducerConfig
{
    BootstrapServers="localhost:9092"
};

using var producer =
new ProducerBuilder<string,string>(config)
.Build();

await producer.ProduceAsync(
"orders",
new Message<string,string>
{
    Key="1001",
    Value="{OrderId:1001}"
});

Producer sends

Topic

orders

Consumer Example

var config =
new ConsumerConfig
{
BootstrapServers="localhost:9092",
GroupId="inventory-group",
AutoOffsetReset=AutoOffsetReset.Earliest
};

using var consumer =
new ConsumerBuilder<string,string>(config)
.Build();

consumer.Subscribe("orders");

while(true)
{
var result=consumer.Consume();

Console.WriteLine(result.Message.Value);
}

ASP.NET Core Web API

Angular

POST

/api/orders

Controller

[HttpPost]
public async Task<IActionResult> Create(OrderDto order)
{
await producer.Publish(order);

return Ok();
}

Producer Service

await _producer.ProduceAsync(
"orders",
message);

Angular

Order Service

createOrder(order:any){

return this.http.post(
"/api/orders",
order);

}

Component

submit(){

this.orderService
.createOrder(this.order)
.subscribe();
}

Angular never talks directly to Kafka.

Reason

Kafka is backend infrastructure.

Angular

↓

Web API

↓

Kafka

Complete Flow

Angular

↓

Web API

↓

Kafka Producer

↓

Orders Topic

↓

Broker

↓

Partition

↓

Consumer Group

↓

Inventory Service

↓

Database

↓

Notification

↓

Email

Kafka vs RabbitMQ

Kafka	RabbitMQ
Event Streaming	Message Queue
Huge Throughput	Moderate
Log Based	Queue Based
Message Retention	Message Deleted
Replay Possible	Difficult
Analytics	Task Processing

Kafka in Microservices

Example

Customer Created

↓

customer-created topic

↓

Billing

↓

CRM

↓

Analytics

↓

Notification

↓

Search Index

No service depends on another.

Docker

version: '3'

services:

  kafka:

    image: bitnami/kafka

Run

docker compose up

Azure Integration

Kafka can integrate with

Azure Event Hubs (Kafka-compatible endpoint)
Azure Kubernetes Service (AKS)
Azure Container Apps
Azure Virtual Machines
Azure Monitor
Azure Key Vault (Secrets)
Azure DevOps (CI/CD)

Best Practices

Use meaningful topic names (e.g., orders.created.v1)
Keep events immutable.
Prefer Avro or Protobuf with a Schema Registry over raw JSON for large systems.
Use keys that preserve ordering (for example, OrderId).
Avoid very large messages; store large files externally and publish references.
Configure retry, idempotent producers, and dead-letter handling where appropriate.
Monitor consumer lag and broker health.
Plan partition counts based on expected throughput and consumer parallelism.

Common Interview Questions

What is Apache Kafka?
Explain Kafka architecture.
What is a Broker?
What is a Topic?
What is a Partition?
What is an Offset?
Explain Consumer Groups.
Difference between Queue and Topic.
Kafka vs RabbitMQ.
Why are partitions needed?
How does Kafka guarantee ordering?
What happens when a broker fails?
What is replication factor?
What are producer acknowledgements (acks)?
What is consumer lag?
What is idempotent producer?
What is exactly-once processing?
What is the role of Schema Registry?
How do you scale Kafka consumers?
How do you integrate Kafka with .NET Core?

Complete E-Commerce Example

                Angular UI
                     │
                     ▼
         ASP.NET Core Web API
                     │
                     ▼
            Kafka Producer Service
                     │
                     ▼
         ┌──────────────────────────┐
         │      Orders Topic         │
         └──────────────────────────┘
            │       │        │
            ▼       ▼        ▼
     Inventory   Payment   Analytics
       Service    Service     Service
            │       │        │
            ▼       ▼        ▼
      SQL Server  Email   Data Warehouse
                     │
                     ▼
              Notification Service

A customer clicks Place Order in Angular. The Angular app calls the ASP.NET Core API. The API validates the request, stores the order (if following the Outbox pattern), and publishes an OrderCreated event to Kafka. Kafka persists the event in the orders topic, making it available to multiple consumer groups. Inventory reserves stock, Payment charges the customer, Analytics records the event, and Notification sends an email—all independently and asynchronously.

This decoupled architecture improves scalability, resilience, and maintainability because producers and consumers evolve independently.

For a production-grade implementation, the next topics to master are:

Kafka internals (segments, ISR, leader election, page cache)
KRaft architecture (ZooKeeper-free Kafka)
Schema Registry with Avro/Protobuf
Outbox Pattern with .NET and Entity Framework Core
Saga Pattern with Kafka
Retry topics and Dead Letter Topics (DLT)
Idempotent consumers and exactly-once semantics
Monitoring with Prometheus and Grafana
Deploying Kafka on Docker, Kubernetes, and Azure
End-to-end .NET 9 microservices with Angular 20 and Kafka using Clean Architecture and CQRS

These advanced topics are commonly expected in senior .NET Lead and Technical Architect interviews.

Interview Questions And Answers :

Apache Kafka Interview Questions And Answers

Azure API Management

2026-08-02T10:44:17.208+05:30

Azure API Management (APIM) — Complete Guide for .NET Lead

Azure API Management (APIM) is one of the most important Azure services for .NET Developers, Technical Leads, Solution Architects, and Cloud Engineers. It acts as a central gateway between clients and your backend APIs, providing security, routing, authentication, throttling, monitoring, versioning, transformation, caching, and developer onboarding.

Official Documentation

Azure API Management Documentation

What is Azure API Management?
Why Do We Need APIM?
APIM Architecture
Components of APIM
Real-Time Enterprise Example
Request Flow
Policies
Authentication & Authorization
Rate Limiting & Throttling
API Versioning
API Products & Subscriptions
Developer Portal
API Transformation
Caching
Monitoring
Azure DevOps Integration
C# Web API Example
Best Practices
Lead-Level Interview Questions

1. What is Azure API Management?

Azure API Management is a fully managed API Gateway that sits between API consumers and backend services.

Instead of exposing your APIs directly:

Client
   |
ASP.NET Core API

You expose them through APIM:

Client
    |
Azure API Management
    |
ASP.NET Core API

APIM becomes the single entry point for all APIs.

2. Why Do We Need APIM?

Imagine an enterprise has:

Mobile App
Angular Web App
Partner APIs
Internal APIs
Third-party integrations

Without APIM:

Mobile  ------\
Angular -------\
Partner --------> Order API
Portal --------/

Each client communicates directly with every API.

Problems:

No centralized security
Duplicate authentication logic
No rate limiting
Difficult monitoring
Hard API version management

With APIM:

Clients
    |
Azure API Management
    |
-------------------------------------
| Order API
| Payment API
| Inventory API
| Notification API
-------------------------------------

Everything is managed centrally.

3. Real-Time Banking Architecture

                 Mobile App
                      |
                 Internet Banking
                      |
                 Partner Portal
                      |
               Azure API Management
                      |
-------------------------------------------------
| Account API
| Payment API
| Card API
| Loan API
| Notification API
-------------------------------------------------
                      |
Azure SQL | Service Bus | Redis | Key Vault

4. APIM Components

Azure API Management
        |
+----------------------------+
| API Gateway                |
| Policies                   |
| Security                   |
| Analytics                  |
| Developer Portal           |
| Products                   |
| Subscriptions              |
+----------------------------+

5. Core Features

Azure API Management provides:

API Gateway
Authentication
Authorization
Rate Limiting
Throttling
Caching
API Versioning
API Revisions
Request/Response Transformation
Monitoring & Analytics
Developer Portal
Subscription Keys
OAuth 2.0 / OpenID Connect integration
JWT Validation
IP Filtering
Custom Domains
Certificates

6. Request Flow

Client
   |
HTTPS Request
   |
Azure API Management
   |
Validate JWT
Check Subscription
Apply Policies
Rate Limit
Cache Check
Transform Request
   |
ASP.NET Core API
   |
Azure SQL Database
   |
Response
   |
Transform Response
   |
Client

7. Real-Time Insurance Example

Customer submits claim.

Mobile App

↓

API Management

↓

JWT Validation

↓

Claim API

↓

Azure SQL

↓

Azure Service Bus

↓

Notification Service

Every request passes through APIM.

8. API Gateway

The gateway:

Receives requests
Validates security
Applies policies
Routes requests
Logs telemetry
Returns responses

Without changing backend code.

9. API Import

APIs can be imported from:

OpenAPI (Swagger)
ASP.NET Core Web API
Azure Functions
Logic Apps
WSDL (SOAP)
Existing HTTP APIs

10. Authentication

Supports:

Microsoft Entra ID
OAuth 2.0
OpenID Connect
JWT Tokens
Subscription Keys
Client Certificates

11. JWT Validation

Instead of validating JWT inside every API:

Client

↓

JWT

↓

API Management

↓

Valid Token

↓

Backend API

Invalid tokens are rejected before reaching the backend.

12. Subscription Keys

Each client receives:

Primary Key

Secondary Key

Clients include the subscription key in requests.

APIM validates the key before forwarding the request.

13. Rate Limiting

Suppose:

Customer sends

10,000 requests/sec

Without APIM:

API crashes.

With APIM:

Allowed

↓

100 requests/minute

↓

429 Too Many Requests

This protects backend services.

14. Throttling

Example Policy:

<rate-limit calls="100"
            renewal-period="60" />

Meaning:

100 requests

Every

60 seconds

15. API Versioning

Suppose API evolves.

v1

/api/customers

New version:

v2

/api/customers

Both versions can run simultaneously.

16. Revisions

Version:

Breaking changes

Revision:

Minor changes

Example:

Version 1

Revision 1

↓

Revision 2

↓

Revision 3

17. Products

A Product groups APIs.

Example:

Partner Product

↓

Customer API

Order API

Payment API

Partners subscribe to products rather than individual APIs.

18. Developer Portal

Developers can:

Browse APIs
Read documentation
Test APIs
Generate subscription keys
Download OpenAPI specs

19. Policies

Policies are XML-based rules executed by APIM.

Examples:

Validate JWT
Add Headers
Remove Headers
Rewrite URL
Rate Limit
IP Filter
Cache
Retry
Mock Response

Example:

<set-header name="X-App">
    <value>OneView</value>
</set-header>

20. Request Transformation

Client sends:

{
  "customer":"Mahesh"
}

APIM transforms:

{
  "CustomerName":"Mahesh"
}

No backend code changes required.

21. Response Transformation

Backend returns:

{
 "CustomerName":"Mahesh"
}

APIM transforms:

{
 "name":"Mahesh"
}

22. Caching

Without Cache:

Client

↓

API

↓

SQL

Every request hits SQL.

With Cache:

Client

↓

APIM Cache

↓

Response

Reduces latency and database load.

23. Monitoring

Integrated with:

Azure Monitor
Application Insights
Log Analytics

Monitor:

Requests
Response Time
Failures
Backend Latency
Subscription Usage

24. Azure DevOps Integration

Developer

↓

Git

↓

Azure DevOps

↓

Build

↓

Deploy API

↓

Azure API Management

↓

Production

API definitions and policies can be deployed as part of CI/CD pipelines.

25. ASP.NET Core API Example

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllers();

var app = builder.Build();

app.MapControllers();

app.Run();

Deploy the API to:

Azure App Service
Azure Kubernetes Service (AKS)
Azure Container Apps
Azure Functions

Then expose it securely through APIM.

26. Enterprise Architecture

Users
     |
Azure Front Door
     |
Azure API Management
     |
-----------------------------------------
| Order API
| Payment API
| Customer API
| Inventory API
-----------------------------------------
     |
Azure SQL
Azure Service Bus
Azure Storage
Azure Key Vault
Application Insights

27. Best Practices

Never expose backend APIs directly.
Use Microsoft Entra ID or OAuth 2.0.
Validate JWT at the gateway.
Enable rate limiting and quotas.
Use API versioning instead of breaking changes.
Apply caching for read-heavy endpoints.
Secure secrets with Azure Key Vault.
Monitor with Azure Monitor and Application Insights.
Use separate APIM instances or products for different environments when appropriate.
Automate deployments with Azure DevOps.

28. Frequently Asked Interview Questions

Basic

What is Azure API Management?
Why do we use APIM?
What is an API Gateway?
What are Products?
What are Subscription Keys?

Intermediate

Explain APIM policies.
How does JWT validation work?
Difference between Version and Revision?
How does rate limiting work?
What is request transformation?
What is response transformation?
What is API caching?

Lead-Level

Design an enterprise API architecture using APIM.
How would you secure hundreds of APIs?
How would you onboard external partners?
How would you implement zero-downtime API versioning?
How would you monitor API performance?
How would you implement CI/CD for APIM?
How would you integrate APIM with AKS and Azure Functions?
When would you choose APIM over exposing APIs directly?

29. Lead-Level Interview Answer

Interviewer: "How have you used Azure API Management in your project?"

Answer:

"In our enterprise .NET microservices platform, Azure API Management served as the single entry point for all client applications, including web, mobile, and partner integrations. We used APIM to enforce JWT authentication with Microsoft Entra ID, apply rate limiting and quotas, transform requests and responses, manage API versions, and expose APIs through Products and Subscription Keys. Backend services were hosted on Azure App Service and AKS, while Azure Monitor and Application Insights provided end-to-end observability. API deployments and policy updates were automated using Azure DevOps pipelines, ensuring secure, consistent, and zero-downtime releases across development, QA, and production environments."

30. APIM vs Azure Application Gateway vs Azure Front Door

Feature	Azure API Management	Application Gateway	Azure Front Door
Primary Purpose	API Gateway & API Lifecycle	Layer 7 Web Traffic Load Balancer	Global Entry Point & CDN
Authentication	✅ Yes	Limited	Limited
Rate Limiting	✅ Yes	No	Basic WAF/Rules
API Versioning	✅ Yes	No	No
Developer Portal	✅ Yes	No	No
Request/Response Transformation	✅ Yes	Limited	Limited
Global Routing	Regional (can be combined)	Regional	✅ Global
Typical Use	Secure and manage APIs	Route web traffic	Global traffic acceleration and failover

When to use each

Azure API Management: When you need API security, governance, versioning, throttling, analytics, and developer onboarding.
Azure Application Gateway: When you need Layer 7 load balancing, SSL termination, and Web Application Firewall (WAF) for web applications.
Azure Front Door: When you need global load balancing, edge acceleration, CDN capabilities, and multi-region failover.

For large enterprise systems, it's common to use them together:

Users
   |
Azure Front Door
   |
Application Gateway (optional, depending on architecture)
   |
Azure API Management
   |
ASP.NET Core APIs / Azure Functions / AKS

This layered approach provides global availability, web protection, and centralized API governance.