The given image explains the full process of PDF generation:

Let’s generate a complete PDF file with MigraDoc in ASP.NET Core version 10.0

MigraDoc Code Structure

The structure of MigraDoc contains 3 parts:

1. Document: it is the parent object which contains sections.
2. Section: all contents of a document are organized in sections. Sections contains other objects like table and paragraph.
3. PdfDocumentRenderer: it takes a Document object, formats it properly (layout, fonts, pages) and outputs a PDF file

The code structure is given below.

var document = new Document();
var section = document.AddSection();

var heading = section.AddParagraph("Test PDF");

var pdfRenderer = new PdfDocumentRenderer();

pdfRenderer.Document = document;

pdfRenderer.RenderDocument();
pdfRenderer.Save("SimpleDocument.pdf");

Generate PDF file with MigraDoc in ASP.NET Core version 10

To see how MigraDoc works we will generate a complete credit card statement of a bank customer in ASP.NET Core version 10.0. Once completed the PDF file will look as shown in the below image.

In Visual Studio create a new project and select the template called ASP.NET Core Web App (Model-View-Controller).

First, we need to install the package called PDFsharp-MigraDoc from NuGet.

Next, import the necessary namespaces on the controller.

using MigraDoc.Rendering;
using MigraDocTutorial.Models;
using PdfSharp.Fonts;
using MigraDoc.DocumentObjectModel;

Now, open the HomeController file and in Index action we add MigraDoc codes as shown below.

public class HomeController : Controller
{
    private IWebHostEnvironment hostingEnvironment;

    public HomeController(IWebHostEnvironment environment)
    {
        hostingEnvironment = environment;
    }

    public IActionResult Index()
    {
        string imagePath = Path.Combine(hostingEnvironment.WebRootPath, "Images");

        var document = new Document();
        var section = document.AddSection();

        GlobalFontSettings.UseWindowsFontsUnderWindows = true;

        var heading = section.AddParagraph("Your Credit Card Statement Report has been Generated");

        heading.Format.OutlineLevel = OutlineLevel.Level1;
        heading.Format.Font.Size = 25;

        // Add line below the heading
        heading.Format.Borders.Bottom.Width = 1;
        heading.Format.SpaceAfter = "30pt";

        // Add table.
        var table = section.AddTable();

        // Add first column.
        var columnA = table.AddColumn(Unit.FromCentimeter(6));

        // Add second column.
        var columnB = table.AddColumn(Unit.FromCentimeter(12));
        
        // Add first row.
        var row1 = table.AddRow();

        // Add paragraph to first cell of row1.
        var cellA1 = row1[0];

        document.ImagePath = imagePath;
        var image = cellA1.AddImage("woman.jpg");
        image.Width = Unit.FromPoint(150);
        image.Height = Unit.FromPoint(150);

        // Add paragraph to second cell of row1.
        var cellB1 = row1[1];
        cellB1.AddParagraph("Name: Mrs. Grace Kelly");
        cellB1.AddParagraph("Address: House 20, 31 drowning street, London (UK)");
        cellB1.AddParagraph("Occupation: Doctor");
        cellB1.AddParagraph("Age: 30");
        cellB1.Format.Font.Size = 25;
        cellB1.Format.Font.Color = Colors.Red;

        var heading1 = section.AddParagraph("This month's transation in your Credit Card !");
        heading1.Format.Font.Size = 20;
        heading1.Format.Font.Color = Colors.BurlyWood;

        // Add line below the heading
        heading1.Format.Borders.Bottom.Width = 1;
        heading1.Format.SpaceBefore = "30pt";
        heading1.Format.SpaceAfter = "30pt";

        var table2 = section.AddTable();
        table2.Borders.Visible = true;

        table2.AddColumn("3cm");
        table2.AddColumn("3cm");
        table2.AddColumn("3cm");
        table2.AddColumn("3cm");
        table2.AddColumn("3cm");

        var row1Table2 = table2.AddRow();
        row1Table2.HeadingFormat = true;
        row1Table2.Format.Font.Color = Colors.BlueViolet;
        row1Table2.Shading.Color = Colors.LightGray;

        row1Table2[0].AddParagraph("S.No");
        row1Table2[1].AddParagraph("Merchant");
        row1Table2[2].AddParagraph("Item");
        row1Table2[3].AddParagraph("Cost");
        row1Table2[4].AddParagraph("Date");

        var row2Table2 = table2.AddRow();
        row2Table2[0].AddParagraph("1");
        row2Table2[1].AddParagraph("NYC Junction");
        row2Table2[2].AddParagraph("Fruits");
        row2Table2[3].AddParagraph("$100.00");
        row2Table2[4].AddParagraph("June 1");

        var row3Table2 = table2.AddRow();
        row3Table2[0].AddParagraph("2");
        row3Table2[1].AddParagraph("David Store");
        row3Table2[2].AddParagraph("Napkins");
        row3Table2[3].AddParagraph("5.90");
        row3Table2[4].AddParagraph("June 3");

        var row4Table2 = table2.AddRow();
        row4Table2[0].AddParagraph("3");
        row4Table2[1].AddParagraph("Singhs");
        row4Table2[2].AddParagraph("Toys");
        row4Table2[3].AddParagraph("$99.99");
        row4Table2[4].AddParagraph("June 9");

        var row5Table2 = table2.AddRow();
        row5Table2[0].AddParagraph("4");
        row5Table2[1].AddParagraph("Seven 11");
        row5Table2[2].AddParagraph("Grocery");
        row5Table2[3].AddParagraph("$140.00");
        row5Table2[4].AddParagraph("June 15");

        var row6Table2 = table2.AddRow();
        row6Table2[0].AddParagraph("5");
        row6Table2[1].AddParagraph("Carlos Pharmacy");
        row6Table2[2].AddParagraph("Drugs");
        row6Table2[3].AddParagraph("$60.00");
        row6Table2[4].AddParagraph("June 25");

        var custName = section.AddParagraph("Hello Grace,");
        custName.Format.SpaceBefore = "30pt";
        custName.Format.SpaceAfter = "20pt";
        section.AddParagraph("Thank you for being our valuable customer. We hope our letter finds you in the best of health and wealth.\n\nYours Sincerely.\nICICI Bank");

        // Create a PDF renderer for the MigraDoc document.
        var pdfRenderer = new PdfDocumentRenderer();

        // Associate the MigraDoc document with a renderer.
        pdfRenderer.Document = document;

        // Layout and render document to PDF.
        pdfRenderer.RenderDocument();
        // Save the document.
        pdfRenderer.Save("SimpleDocument.pdf");

        return View();
    }
}

The above code is a complete code which will generate the Credit Card statement PDF file. Lets understand the code part by part.

Document and Section

In the above code we defined the Document and added a section to it. The code which does this work is given below.

var document = new Document();
var section = document.AddSection();

We then specified MigraDoc to use windows fonts by the below code:

GlobalFontSettings.UseWindowsFontsUnderWindows = true;

public HomeController(IWebHostEnvironment environment)
{
    hostingEnvironment = environment;
}

string imagePath = Path.Combine(hostingEnvironment.WebRootPath, "Images");

Heading

From the AddParagraph method we added the text – “Your Credit Card Statement Report has been Generated”. To make it big size we gave it OutlineLevel and font size 25. We also added a border of width 1pt below it and gave spacing of 30pt after it. See the below code.

var heading = section.AddParagraph("Your Credit Card Statement Report has been Generated");

heading.Format.OutlineLevel = OutlineLevel.Level1;
heading.Format.Font.Size = 25;

// Add line below the heading
heading.Format.Borders.Bottom.Width = 1;
heading.Format.SpaceAfter = "30pt";

The above code will generate the following as shown by the below image.

MigraDoc Table

Next, we defined a table which will contain 2 columns. The left column will contain the customer image and the right column will contain the customer name, address and other details. The below image shown the details.

The code which does this thing is given below.

// Add table.
var table = section.AddTable();

// Add first column.
var columnA = table.AddColumn(Unit.FromCentimeter(6));

// Add second column.
var columnB = table.AddColumn(Unit.FromCentimeter(12));

// Add first row.
var row1 = table.AddRow();

// Add paragraph to first cell of row1.
var cellA1 = row1[0];

document.ImagePath = imagePath;
var image = cellA1.AddImage("woman.jpg");
image.Width = Unit.FromPoint(150);
image.Height = Unit.FromPoint(150);

// Add paragraph to second cell of row1.
var cellB1 = row1[1];
cellB1.AddParagraph("Name: Mrs. Grace Kelly");
cellB1.AddParagraph("Address: House 20, 31 drowning street, London (UK)");
cellB1.AddParagraph("Occupation: Doctor");
cellB1.AddParagraph("Age: 30");
cellB1.Format.Font.Size = 25;
cellB1.Format.Font.Color = Colors.Red;

var heading1 = section.AddParagraph("This month's transation in your Credit Card !");
heading1.Format.Font.Size = 20;
heading1.Format.Font.Color = Colors.BurlyWood;

// Add line below the heading
heading1.Format.Borders.Bottom.Width = 1;
heading1.Format.SpaceBefore = "30pt";
heading1.Format.SpaceAfter = "30pt";

If we explain the above code, it starts by adding the table to the section. Then adding the 2 columns of width 6cms and 12cms to the table.

var table = section.AddTable();
var columnA = table.AddColumn(Unit.FromCentimeter(6));
var columnB = table.AddColumn(Unit.FromCentimeter(12));

After that we added a row and the first cell to the row. Note that the first cell has index 0 and second one has index 1.

var row1 = table.AddRow();
var cellA1 = row1[0];

The first cell will contain the image of the customer so we have to provide the image path i.e. wwwroot/Images. See below code.

document.ImagePath = imagePath;

Next, we add the image of the customer with dimension 150pt * 150pt.

var image = cellA1.AddImage("woman.jpg");
image.Width = Unit.FromPoint(150);
image.Height = Unit.FromPoint(150);

The second cell is also added in the same way where we have shown the customer details. See the below code.

var cellB1 = row1[1];
cellB1.AddParagraph("Name: Mrs. Grace Kelly");
cellB1.AddParagraph("Address: House 20, 31 drowning street, London (UK)");
cellB1.AddParagraph("Occupation: Doctor");
cellB1.AddParagraph("Age: 30");
cellB1.Format.Font.Size = 25;
cellB1.Format.Font.Color = Colors.Red;

After this we added another table which contains 5 columns to show the credit card transaction details of the customer. The below image shows this.

There is nothing new to this code and it is self explanatory.

The next part of the pdf contains the final message to the customer. The code which does this thing is given below.

var custName = section.AddParagraph("Hello Grace,");
custName.Format.SpaceBefore = "30pt";
custName.Format.SpaceAfter = "20pt";
section.AddParagraph("Thank you for being our valuable customer. We hope our letter finds you in the best of health and wealth.\n\nYours Sincerely.\nICICI Bank");

The below image shown this portion of the pdf.

MigraDoc PDF Rendering

We have added all the details to the pdf. We now have to save the pdf file. For this we use the PdfDocumentRenderer object to render the pdf. The pdf file will be named as “SimpleDocument.pdf” and will be saved on the root of the app.

Well that’s it we just completed the full pdf generation with MigraDoc. You can create any type of pdf with this library free of charge. Download the source codes from our GitHub repo (link at the top) and start using this library.

The post How to create PDF files in ASP.NET Core with MigraDoc appeared first on YogiHosting.

When building modern web applications with ASP.NET Core, especially in microservices architectures, direct communication between services can lead to tight coupling and reduced flexibility. RabbitMQ helps solve this by acting as an intermediary that manages message queues. Instead of services calling each other directly, they send messages to a queue, which are then consumed by other services when they are ready. This approach enhances fault tolerance and allows systems to scale independently.

Using RabbitMQ in ASP.NET Core typically involves producing messages (publishers) and consuming them (consumers). For example, an “Order” Microservice (Publisher) takes an order from a customer, and publishes a message to RabbitMQ about this order. RabbitMQ sends this Order message to the “Shipping” Microservice (Consumer), so that the shipping microservice can ship the order to the customer. This asynchronous workflow ensures that the whole app remains responsive while heavy tasks are handled separately.

ASP.NET Core Microservices example with RabbitMQ

In this .NET example we will use RabbitMQ as a message broker for Microservices communication. There will be 2 Microservices build on ASP.NET Core these are:

• Order Microservice (Publisher)- that will take the customer order and produce the message regarding the order for RabbitMQ. RabbitMQ will send this order message to the Shipping Mircoservice.
• Shipping Microservice (Consumer) – will get the Order message from RabbitMQ regarding the Order message, and will ship the product to the customer.

We have explained the whose process in the below image.

Installing RabbiMQ

We will install RabbitMQ through Docker. This is a very fast process. We just run the following command on command prompt or Powershell.

docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:4-management

Check the below image which shows this command.

In 1 to 2 minutes RabbitMQ image will be downloaded on our pc and will run from a docker container. Open the RabbitMQ url – http://localhost:15672/ on the browser. Then for both username and password enter “guest” and click the login button.

Once login we will see tabs for – Overview, Connections, Channels, Exchanged, Queues and Streams and Admin. Navigate to any of them shows the specific details.

In RabbitMQ, a channel is a lightweight virtual connection that runs on top of a real connection. It’s the main way your application actually interacts with RabbitMQ to send and receive messages.

A connection is a fundamental link established between your application (such as an ASP.NET Core MVC) and the RabbitMQ server.

If you navigate to Channels and Connections they will show nothing since we haven’t yes interacted with RabbitMQ with our ASP.NET Core MVC app.

In RabbitMQ, an exchange is the component responsible for receiving messages from producers and deciding how to route them to queues. It acts like a smart message router. An exchange receives messages from a producer (publisher), uses rules (called bindings) to determine where messages go and Sends messages to one or more queues. Note that producers never send messages directly to queues—they always send them to an exchange.

Navigate to “Exchanges” where you will find few exchanges already present, these are:

• Default Exchange : “AMQP default” usually refers to the default exchange that is automatically created by the broker when it starts. It’s part of the AMQP standard.
• Direct Exchange : Routes messages based on an exact match of the routing key.
• Fanout Exchange : Sends messages to all bound queues.
• Topic Exchange : Routes messages using pattern matching.
• Headers Exchange : Routes based on message headers instead of routing key.

Queues and Streams are two different ways of storing and delivering messages. They serve different use cases depending on how you want messages to be processed.

A queue:

• Stores messages until they are consumed.
• Delivers messages to consumers (usually one at a time).
• Removes messages once they are acknowledged.

Streams are a newer feature designed for high-throughput and event streaming scenarios.

A stream:

• Stores messages as a continuous log (like an event history).
• Messages are not deleted after consumption.
• Consumers can read messages multiple times.

On navigating to Queues and Streams tab there won’t be any since we haven’t created a message. We will do it in just a moment.

ASP.NET Core Publisher Microservice

Lets start with creating a Publisher Microservice. In visual studio create a new project by selecting the template called ASP.NET Core Web App (Model-View-Controller). We named this project MTTutorialP, you can name it anything. This microservices acts as a Publisher that can interact with other Consumer microservices via RabbitMQ.

We first add an Order.cs class to the project which is for the orders made by customers. We gave the namespace MTTutorialC.Models for this class. We will use the same class with the same namespace on the Consumer also. This is because RabbitMQ treats messages based on it’s namespaces. if the received and outgoing message are of different namespace (signatures), RabbitMQ would not recognize the Consumer.

namespace MTTutorialC.Models
{
    public class Order
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public int Quantity { get; set; }
    }
}

Installing MassTransit Packages

MassTransit is a free, open-source “distributed application framework” for .NET. At its core, it acts as a service bus—a layer of abstraction that sits on top of message brokers (like RabbitMQ, Azure Service Bus, or Amazon SQS) to make building message-based, loosely coupled applications much easier. Think of it as an “Object-Relational Mapper” (ORM) but for messaging. Just as Entity Framework abstracts the complexities of SQL, MassTransit abstracts the complexities of message brokers. So install the following 2 MassTransit packages to the project from NuGet:

Install-Package MassTransit
Install-Package MassTransit.RabbitMQ

After adding the MassTransit packages, we will have to configure it to work as a Publisher. Navigate to the Program.cs and add the following code that registers MassTransit as a service.

builder.Services.AddMassTransit(x =>
{
    x.UsingRabbitMq();
});

Note that in the above case the MassTransit will use default RabbitMQ username and password which is “guest”. If the username and password are different then you can specify them using the below configurations.

builder.Services.AddMassTransit(x =>
{
    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.Host("localhost", "/", h =>
        {
            h.Username("myusername");
            h.Password("mypassword");
        });
    });
});

Publisher: Send Message to RabbitMQ with MassTransit

Lets post an Order message to RabbitMQ with MassTransit. Open HomeController.cs file and inject ISendEndpointProvider object to the constructors. This object will be provided by the dependency injection.

We then use the ISendEndpointProvider to get the endpoint address and using it we send the order message to RabbitMQ. Check the below code.

public class HomeController : Controller
{
    private readonly ISendEndpointProvider sendEndpointProvider;

    public HomeController(ISendEndpointProvider sendEndpointProvider)
    {
        this.sendEndpointProvider = sendEndpointProvider;
    }
    
    public async Task<IActionResult> Index()
    {
        var endpoint = await sendEndpointProvider.GetSendEndpoint(new Uri("queue:OrderC"));

        await endpoint.Send(new Order {
            Id = 1,
            Name = "Football",
            Quantity = 10
        });

        return View();
    }
}

Notice the GetSendEndpoint method needs the RabbitMQ queue name which is given as queue:OrderC. Here “OrderC” is the name of the queue. Name of the queue can be anything.

The Order message containing the Order.cs class values of id=1, name=football and quantity=10 is send to RabbitMQ.

Lets test the working by running the project. The Index action of HomeController will execute automatically as it is the default route of the ASP.NET Core MVC project. Next, open RabbitMQ UI on browser and check the Connections and Channels where we will see new entry, this specifies that RabbitMQ has received the message.

The most important thing is the addition of a new exchange called “OrderC” which we can find on the Exchanges area. Check the below image.

Recall we gave the queue name “OrderC” in the URI of GetSendEndpoint() method.

Now go to Queues and Streams where we find a new entry. The message is waiting for a consumer to pick them up for processing. See the state of the message showing 0 for Ready, Unacked and Total which means message is waiting for a consumer. Check the below image:

Click on the message to see it details, we can see there are no consumers for the message. Check the below image.

In RabbitMQ, the terms Ready, Unacked, and Total describe the state of messages in a queue:

• Messages that are waiting in the queue
• Not yet delivered to any consumer
• Available to be consumed immediately

• Messages that have been delivered to a consumer
• But the consumer has not yet acknowledged (ACKed) them
• These are “in progress”
• If the consumer crashes or disconnects, RabbitMQ will requeue them

• The sum of Ready + Unacked
• Represents all messages currently in the queue

ASP.NET Core Consumer Microservice

Lets add a Consumer, so right click on the Solution and select add a new project. Select the same old template of ASP.NET Core Web App (Model-View-Controller), and name the project as “MTTutorialC”. Name is not important and you can choose your own name.

This .NET Microservice will be responsible for consuming the incoming messages from RabbitMQ. Do you remember the packages we installed earlier? Install the same one in this project too.

To this project add the Order.cs class that has the name namespace like the Order.cs defined in the publisher project. This is the requirement for RabbitMQ.

namespace MTTutorialC.Models
{
    public class Order
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public int Quantity { get; set; }
    }
}

Next, we configure the MassTransit in the program class as shown below.

builder.Services.AddMassTransit(x =>
{
    x.AddConsumer<OrderC>();

    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.ConfigureEndpoints(context);
    });
});

In the above code we added the consumer – x.AddConsumer<OrderC>(). OrderC is the consumer class which will receive the message.

Next, we configured the Endpoints of RabbitMQ for MassTransit using the below code:

x.UsingRabbitMq((context, cfg) =>
{
    cfg.ConfigureEndpoints(context);
});

Consumer: Receive Message from RabbitMQ with MassTransit

To receive messages from RabbitMQ using MassTransit, we need to define a class that will be the consumer for the message. It has to inherit the IConsumer&ltT> where T is the type of message which is “Order” for our case. The code of this class called OrderC.cs is given below.

namespace MTTutorialC.Models
{
    public class OrderC : IConsumer<Order>
    {
        public async Task Consume(ConsumeContext<Order> context)
        {
            var jsonMessage = JsonConvert.SerializeObject(context.Message);
            Console.WriteLine($"OrderCreated message: {jsonMessage}");
        }
    }
}

The message received is serialized by the Json.NET library which we can install by the below command.

Install-Package Newtonsoft.Json

Testing the Microservices

Let’s test our Microservices now. We need both the Microservices running in order to send and receive the messages. To enable Multiple Starup Projects, Right click on the solution and set the action of each project to “Start”. See the below image.

Place a breakpoint on the line “var jsonMessage” in the consumer. Now press the Run button on Visual Studio which will start both the projects.

The breakpoint will hit and we can see the message is received by the consumer. Check the below image.

Go to Queues and Streams on the RabbitMQ UI where we will find a new queue is formed. Check the below image.

Click the Queue to see it’s details check the consumer binding now present.

Publisher sends message Consumer unavailable

The consumer can be offline due to several reasons like server issues. Even if the consumer is offline, the publisher can still send messages to the RabbitMQ queue. Once the consumer comes back online, it can process any pending messages. That’s essentially the core idea behind message brokering—let’s take a closer look.

Change the startup to run only the Publisher project (and not Consumer). This mimics the scenario where the consumer if offline. If we run the publisher project the message goes to rabbitmq queue.

Now change the setup to run only the Consumer project. Put a breakpoint on the OrderC.cs class Consume method. Run the project in Visual Studio, we will see breakpoint hits telling the message is received from RabbitMQ.

In this article, we explored message brokers, RabbitMQ, its advantages, and how to integrate it with ASP.NET Core using MassTransit. We also built a small prototype application to demonstrate sending data through a RabbitMQ server. You can find the complete source code for this implementation here.

Feel free to share your questions and suggestions in the comments below. If you found this article helpful or learned something new, consider sharing it with your developer community. Happy coding!

The post How to use RabbitMQ with MassTransit for ASP.NET Core Microservices Communication appeared first on YogiHosting.

In this tutorial we will be building a simple real-world implementation to help understand jQuery DataTables to it’s fullest. The whole working is given by the below gif image. It will have paginations, to searching, to sorting and deleting.

Key Features of jQuery DataTables:

• Pagination – Automatically splits large tables into number based pages.
• Search / Filtering – Users can quickly search inside the table.
• Column Sorting – Click column headers to sort them in ascending/descending way.
• Responsive Tables – Works on mobile and desktop, responsiveness through Bootstrap.
• AJAX Data Loading – Load data dynamically from APIs or servers.
• Export Options – Export to CSV, Excel, PDF, etc.
• Custom Styling – Works with frameworks like Bootstrap.

Client-side Pagination vs Server-side Pagination

Pagination in a grid means splitting a large set of data into smaller pages so the user only sees a limited number of rows at a time instead of the entire dataset. Suppose a grid contains 1,000 records then instead of showing all 1,000 rows at once, pagination shows something like:

• Page 1: rows 1–10
• Page 2: rows 11–20
• Page 3: rows 21–30
• … and so on.

Advantage of Pagination are:

• Better performance – Loading fewer records at once makes the page faster.
• Better user experience – Easier to read smaller sets of data.
• Reduced server load – Especially when data comes from a database.
• Cleaner UI – Avoids very long scrolling tables.

Client-side Processing means all the records are loaded into the browser first, and the pagination (page switching) is handled using jQuery on the client (browser), instead of requesting new data from the server each time.

Server-side Processing means the server sends only a small portion of records (i.e. records for a particular page number) to the browser instead of sending the entire records at once. When the user moves to another page, the browser requests the records for that page from the server.

jQuery DataTables offers both Client and Server Side Processing. Client Side Processing works best when there are less than 1000 records to show on jQuery DataTables. If you have more than 1000 records then it is important to use Server Side Processing where only the records for the particular page are fetched from the server. With Server Side Processing you are drastically improving the load time (as the JQuery Datatable is loading just the records of the particular page and not each and every record), reducing the CPU and bandwidth usage.

ASP.NET Core Database driven app

In our ASP.NET Core app, we will read Employee records from the database using Entity Framework Core. Then these records are display in jQuery DataTables. First, add Employee.cs in the Models folder.

public class Employee
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string Gender { get; set; }
    public string Email { get; set; }
    public string Telephone { get; set; }
    public DateTime DateOfBirth { get; set; }
    public string Designation { get; set; }
}

We add the following packages to set up Entity Framework Core.

Install-Package Microsoft.EntityFrameworkCore
Install-Package Microsoft.EntityFrameworkCore.Tools
Install-Package Microsoft.EntityFrameworkCore.SqlServer
Install-Package Microsoft.EntityFrameworkCore.Design

Next, we add DbContext file called CompanyContext.cs inside the Models folder.

public class CompanyContext : DbContext
{
    public CompanyContext(DbContextOptions<CompanyContext> options) : base(options)
    {
    }

    public DbSet<Employee> Employee { get; set; }
}

After this add the connection string in the appsettings.json as given below.

"ConnectionStrings": {
  "DefaultConnection": "Data Source=(localdb)\\MSSQLLocalDB;Initial Catalog=Company;Integrated Security=True;Connect Timeout=30;Encrypt=False;TrustServerCertificate=true;ApplicationIntent=ReadWrite;MultiSubnetFailover=False"
}

With everything set, we perform the Migrations. Open up your package manager console and use the following commands.

add-migration Migration1
update-database

With the database ready, we add 1000 dummy records to the database. There are many free tools available to generate dummy data like – generatedata, mockaroo, etc. We generated 1000 records for Employee table in SQL Insert script and executed the script directly in the database.

Good news! We will be providing 1000 dummy records SQL Insert script for you to use. Check the GitHub repository.

Create HTML Table for Employee Records

In order to implement JQuery Datatable, lets define the structure of our HTML Table in a Razor View File. In the Views Folder, edit up the Index.cshtml to include the HTML table as shown below:

@{
    ViewData["Title"] = "Home Page";
}

<link href="//cdn.datatables.net/2.3.7/css/dataTables.dataTables.min.css" rel="stylesheet" />

<div class="text-center">
    <h1 class="display-4">Welcome</h1>
    <p>Learn about <a href="https://learn.microsoft.com/aspnet/core">building Web apps with ASP.NET Core</a>.</p>
</div>

<div class="container">
    <table id="employeeDataTable" class="table table-striped table-bordere">
        <thead>
            <tr>
                <th>ID</th>
                <th>First Name</th>
                <th>Last Name</th>
                <th>Gender</th>
                <th>Email</th>
                <th>Telephone</th>
                <th>DateOfBirth</th>
                <th>Designation</th>
            </tr>
        </thead>
    </table>
</div>

@section Scripts
{
    <script src="//cdn.datatables.net/2.3.7/js/dataTables.min.js"></script>

    <script>
        $(document).ready(function () {
            $('#employeeDataTable').dataTable({
            
            });
        });
    </script>
}

This will render jQuery DataTables as shown by the image below.

Although, we get the message “No data available in table” which is obvious since we have not applied the data to the DataTables till now. This we will do in just a moment.

Let’s understand the above code.

1. We created an HTML table to display Employee records.
2. On the top we provide the link to jQuery DataTables css.
3. On the bottom inside the script tag we provided the like of jQuery DataTables JavaScript file.
4. Both the CSS and JavaScript will be downloaded from CDN.
5. We invoke the DataTable() method by using the ID of the employee table which is employeeDataTable. This is how to initialize the datatable.

The below line initialize the jQuery DataTables.

<script>
    $(document).ready(function () {
        $('#employeeDataTable').dataTable({
            
        });
    });
</script>

How to use jQuery DataTables

Lets understand how to use DataTables by configuring it. Update the jQuery DataTables as shown below.

<script>
    $(document).ready(function () {
        $('#employeeDataTable').dataTable({
            processing: true,
            serverSide: true,
            "filter": true,
            "ajax": {
                "url": "/api/Employee/GetEmployees",
                "type": "POST",
                "datatype": "json"
            },
            "columns": [
                { "data": "id" },
                { "data": "firstName" },
                { "data": "lastName" },
                { "data": "gender" },
                { "data": "email" },
                { "data": "telephone" },
                { "data": "dateOfBirth"},
                { "data": "designation" }
            ]
        });
    });
</script>

In the above code we applied parameters to configure our jQuery DataTables. These parameters are:

• processing – A boolean property that is used to control the visibility of the “processing” indicator message.
• serverSide – This property enables server-side processing.
• filter – enables/disables the search bar.
• ajax – used to fetch the data from external sources using JavaScript AJAX. We will create Web API though which this data will be fetched.
• columns – columns declared in the DataTables.

See the “columns” options given as data: columnname. Let understand it deeper. When naming variables, it’s important to follow camelCase conventions. For example, firstName is correct, while FirstName is not. It might seem a bit unusual, but that’s how JavaScript works—so be sure to stick to camelCase when writing your scripts.

Also, keep in mind that the API returns a list of records under the key ‘data’. That’s why we reference fields as data.id, data.firstName, and so on when defining column values.

The ajax url is given as “url”: “/api/Employee/GetEmployees”. It is the url of the Web API that will return the employee records. Let’s create the Web API.

Web API to return data for jQuery DataTables

Add a new API Controller to the Controllers folder and name it EmployeeController.cs. We also need to install the package required for data sorting. So run the below install command on NuGet.

Install-Package System.Linq.Dynamic.Core

After installing the package, you can proceed with adding the necessary code to the EmployeeController.cs.

using jQueryDataTables.Models;
using Microsoft.AspNetCore.Mvc;
using System.Linq.Dynamic.Core;
using System.Text.Json;

namespace jQueryDataTables.Controllers
{
    [ApiController]
    [Route("api/[controller]")]
    public class EmployeeController : ControllerBase
    {
        private CompanyContext context;
        public EmployeeController(CompanyContext cc)
        {
            context = cc;
        }

        [HttpPost("GetEmployees")]
        public IActionResult GetEmployees()
        {
            var draw = Request.Form["draw"].FirstOrDefault();
            var start = Request.Form["start"].FirstOrDefault();
            var length = Request.Form["length"].FirstOrDefault();
            
            var colIndex = Request.Form["order[0][column]"];
            string sortColumn = Request.Form["columns[" + colIndex + "][data]"].FirstOrDefault();

            var sortColumnDirection = Request.Form["order[0][dir]"].FirstOrDefault();
            var searchValue = Request.Form["search[value]"].FirstOrDefault();
            int pageSize = length != null ? Convert.ToInt32(length) : 0;
            int skip = start != null ? Convert.ToInt32(start) : 0;
            int recordsTotal = 0;
            var employeeData = (from t in context.Employee select t);

            if (!(string.IsNullOrEmpty(sortColumn) && string.IsNullOrEmpty(sortColumnDirection)))
            {
                employeeData = employeeData.OrderBy($"{sortColumn} {sortColumnDirection}");
            }
            if (!string.IsNullOrEmpty(searchValue))
            {
                employeeData = employeeData.Where(m => m.FirstName.Contains(searchValue)
                                            || m.LastName.Contains(searchValue)
                                            || m.Email.Contains(searchValue));
            }
            recordsTotal = employeeData.Count();
            var data = employeeData.Skip(skip).Take(pageSize).ToList();
            var jsonData = new { draw = draw, recordsFiltered = recordsTotal, recordsTotal = recordsTotal, data = data };

            string jsonString = JsonSerializer.Serialize(jsonData);

            return Ok(jsonData);
        }
    }
}

The work of the APIController is to fetch the employee records from the database in page by page manner. It uses Entity Framework Core to do this task.

First, we get the values of draw, start & length and order[0][column] from Request.Form. These are used for create pagination in DataTables. We can see page size in the Dropdown of DataTables that says, ‘Showing n entries’ where n being the page size.

var draw = Request.Form["draw"].FirstOrDefault();
var start = Request.Form["start"].FirstOrDefault();
var length = Request.Form["length"].FirstOrDefault();

The 3 variables colIndex, sortColumn, sortColumnDirection get the values of necessary columns for performing sorting in DataTables. These are using Request.Form method to get these values from the html of the page.

var colIndex = Request.Form["order[0][column]"];
string sortColumn = Request.Form["columns[" + colIndex + "][data]"].FirstOrDefault();
var sortColumnDirection = Request.Form["order[0][dir]"].FirstOrDefault();

The searchValue variable contains the value to search in the DataTables.

Next, we added the code for fetching and showing the records of the current page.

int pageSize = length != null ? Convert.ToInt32(length) : 0;
int skip = start != null ? Convert.ToInt32(start) : 0;
int recordsTotal = 0;

With LINQ Skip and Take method we get these records and convert them to json. This json is send to jQuery DataTables where the records are displayed.

recordsTotal = employeeData.Count();
var data = employeeData.Skip(skip).Take(pageSize).ToList();
var jsonData = new { draw = draw, recordsFiltered = recordsTotal, recordsTotal = recordsTotal, data = data };

string jsonString = JsonSerializer.Serialize(jsonData);

Also check the codes that performs the sorting and searching (filtering) of records. Note that searching is performed on FirstName, LastName and Email fields.

if (!(string.IsNullOrEmpty(sortColumn) && string.IsNullOrEmpty(sortColumnDirection)))
{
    employeeData = employeeData.OrderBy($"{sortColumn} {sortColumnDirection}");
}

if (!string.IsNullOrEmpty(searchValue))
{
    employeeData = employeeData.Where(m => m.FirstName.Contains(searchValue)
                                || m.LastName.Contains(searchValue)
                                || m.Email.Contains(searchValue));
}

Well it’s time to see the working. Run the app and you can see the records displayed by the jQuery DataTables. You can see the page size dropdown on the top left, search box on the top right. At the bottom there are the page numbers for navigating between the records. Also see that clicking the column names will sort the records in ascending and descending manner.

In the below image we are showing the search/filtering feature in jQuery DataTables.

Delete a row from jQuery DataTables

Lets understand how to add delete record feature on jQuery DataTables. Once this feature is complete we get a delete button against every row of records, click on a delete button will delete the corresponding row of data. See the below image.

For this we need to add a new column for the “Delete” button that will show against each row of records. We will use columns.render option in the table initialization. This option accepts a function or a built-in renderer to transform the cell’s underlying data.

See the below code where we gave a delete button against the rows using render option.

"columns": [
    { "data": "id" },
    { "data": "firstName" },
    { "data": "lastName" },
    { "data": "gender" },
    { "data": "email" },
    { "data": "telephone" },
    { "data": "dateOfBirth"},
    { "data": "designation" },
    {
      "render": function (data, type, full, meta) {
        // 'full' contains the data for the entire row
        return '<button class="btn btn-danger" data-id="' + full.id + '">Delete</button>'; }
    }
]

<p>Next, we add click event to this button using the below JS. The JS calls a method named "deleteRecord".</p>

$('#employeeDataTable tbody').on('click', '.btn-danger', function () {
        // Get the record ID from the button's data attribute
        var recordId = $(this).data('id');

        // Get the specific table row (tr) that was clicked
        var row = $(this).closest('tr');

        // Confirm the deletion with the user (optional)
        if (confirm('Are you sure you want to delete this record?')) {
            deleteRecord(recordId, row);
        }
    });
});

We next add the function called “deleteRecord”. This function make use of the jQuery AJAX method to call another method of our Web API which will actually be deleting the records from the database. Check the below code.

function deleteRecord(id, row) {
    $.ajax({
        url: "/api/Employee/" + id, 
        type: "DELETE", 
        dataType: "json",
        success: function (response) {
            if (response) {
                var dt = new DataTable('#employeeDataTable');
                // Remove the row from the DataTables instance
                // row().remove() deletes the data and node from the browser memory
                // draw(false) updates the display without resetting the pagination
                dt.row(row).remove().draw(false);
                alert('Record deleted successfully!');
            } 
            else {
                alert('Error deleting record on the server.');
            }
        },
            error: function (xhr, status, error) {
            alert('An error occurred during the AJAX request: ' + error);
        }
    });
}

In the above code we are providing the row id in the url as url: “/api/Employee/” + id. Also note the DataTables method – row().remove() is used to deletes the data and node from the browser memory and draw(false) updates the display without resetting the pagination.

Finally we add the delete method to our Web Api which does the record deletion and returns a bool value of true (in json) once the deletion is successful.

[HttpDelete("{id}")]
public IActionResult Delete(int id)
{
    bool status = false;
    var entityToDelete = context.Employee.Where(e => e.Id == id).FirstOrDefault();
    if (entityToDelete != null)
    {
        context.Employee.Remove(entityToDelete);
        context.SaveChanges();
        status = true;
    }
    return new JsonResult(status);
}

Let’s test the deletion of record. We click the delete button against any row which we want to delete. We will get a successful alert message once the deletion is completed.

In this article, we explored everything you need to know about working with jQuery DataTables in ASP.NET Core using server-side processing. We covered the syntax, required files, and overall integration, and built a clean data table featuring paging, sorting, searching, and efficient server-side handling. You can find the complete source code for this implementation in GitHub repository (link given at the top).

I hope this article helped you gain a solid understanding of jQuery DataTables in ASP.NET Core. If you have any feedback or suggestions, feel free to share them in the comments section below. Don’t forget to share this article with your developer community. Thanks, and happy coding!

The post jQuery DataTables in ASP.NET Core with Server Side Processing appeared first on YogiHosting.

Install and Configure Prometheus

Download Prometheus for your OS from https://prometheus.io/download/ and extract the contents of the download. I am using Window 11 OS so I have downloaded the windows version and extracted the zip file contents to a folder in my PC. Inside this folder their are 2 important files prometheus.exe which starts prometheus and prometheus.yml which contains the configurations.

Open the prometheus.yml file to find the below lines. It specifies that Prometheus will run from http://localhost:9090 url in the browser.

static_configs:
  - targets: ["localhost:9090"]
   # The label name is added as a label `label_name=<label_value>` to any timeseries scraped from this config.
    labels:
      app: "prometheus"

Next, add the following code lines at the end of this file.

- job_name: 'MyASPNETApp'
  scrape_interval: 5s # Poll every 5 seconds
  static_configs:
    - targets: ["localhost:5284"]  ## Enter the HTTP port number of .NET app

In the above lines job_name is given MyASPNETApp which can be any name of your choice. Then for scrape_interval give 5s, which is the number of seconds at which Prometheus server scrapes metrics from the app. And finally for the targets specify the http url of the app which in my case is localhost:5284. Note that you have to enter HTTP url not HTTPS one. You will find the app’s http url inside launchsettings.json file.

Save this file and now double click on the prometheus.exe file to start Prometheus. A console window will open showing logs of Prometheus, this specifies that Prometheus is now running. Next, open url – http://localhost:9090 on the browser which opens Prometheus, see below screenshot.

We are going to integrate Prometheus in our app.

Integrate Prometheus in ASP.NET Core app

First, reference the OpenTelemetry packages. Use the NuGet Package Manager or command line to add the following NuGet packages.

dotnet add package OpenTelemetry.Exporter.Console
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
dotnet add package OpenTelemetry.Exporter.Prometheus.AspNetCore --prerelease
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.Http

Next, open Program.cs class and configure OpenTelemetry with the Prometheus provider.

var otel = builder.Services.AddOpenTelemetry();

// Configure OpenTelemetry Resources with the application name
otel.ConfigureResource(resource => resource
    .AddService(serviceName: builder.Environment.ApplicationName));

otel.WithMetrics(metrics => metrics
    .AddAspNetCoreInstrumentation()
    .AddMeter("Microsoft.AspNetCore.Hosting")
    .AddMeter("Microsoft.AspNetCore.Server.Kestrel")
    .AddMeter("System.Net.Http")
    .AddMeter("System.Net.NameResolution")
    .AddPrometheusExporter());

Through WithMetrics() method we are adding Metrics for ASP.NET Core. Metrics are numerical measurements about your application’s behavior and performance. They help you monitor how the app is running in production or during testing. Metrics are usually collected continuously and sent to monitoring tools like Prometheus, Grafana, or Azure Monitor.

Metrics provide quantitative data about your application such as:

• Number of requests
• Response time
• Error rates
• CPU or memory usage
• Database query duration

They help answer questions like:

• Is the API slow?
• How many users are hitting the server?
• Are errors increasing?

Finally, add the OpenTelemetry Prometheus Scraping Endpoint middleware. This means the app metrics are going to Prometheus where we can see their details.

// Configure the Prometheus scraping endpoint
app.MapPrometheusScrapingEndpoint();

Run the app in visual studio and visit the uri – “/metrics” which in our case is https://localhost:7285/metrics. Here all the metrics are shown, see the below image.

Next, on Prometheus go to Status > Target health where you will see the app’s metrics url listed. This means Prometheus is receiving the apps metrics input correctly. See the below image.

Adding Custom Metrics and export to Prometheus

Lets add a Custom Metrics to the app. Start by adding a new class called AppCustomMeter.cs whose code is given below. The class uses IMeterFactory to create a meter instance.

public class AppCustomMeter
{
    public static string name = "CustomMeter";

    private readonly Counter<int> productCounter;
    
    public AppCustomMeter(IMeterFactory meterFactory)
    {
        var meter = meterFactory.Create(name);
        productCounter = meter.CreateCounter<int>("Product.Sold");
    }
    public void ProductSold(string productName, int quantity)
    {
        productCounter.Add(quantity, new KeyValuePair<string, object?>("Product.Name", productName));
    }
}

Now on Program.cs register this custom metrics as a singleton.

builder.Services.AddSingleton<AppCustomMeter>();

Also add it to the OpenTelemetryBuilder with AddMeter method as shown in highlighted code below.

otel.WithMetrics(metrics => metrics
    .AddAspNetCoreInstrumentation()
    .AddMeter(AppCustomMeter.name)
    .AddMeter("Microsoft.AspNetCore.Hosting")
    .AddMeter("Microsoft.AspNetCore.Server.Kestrel")
    .AddMeter("System.Net.Http")
    .AddMeter("System.Net.NameResolution")
    .AddPrometheusExporter());

Now we can test the metrics in work. On the HomeController file, provide the custom metrics class object through DI.

private AppCustomMeter appCustomMeter;

public HomeController(AppCustomMeter appCustomMeter)
{
    this.appCustomMeter = appCustomMeter;
}

Then call the ProductSold method of the metrics to add 3 football quantity.

public IActionResult Index()
{
    appCustomMeter.ProductSold("Football", 3);
    
    return View();
}

Rerun the app, and check the metrics on Prometheus. Click the 3 dots given on the search text box. This will open a menu, select “Explore metrics”.

A Dialog opens where you will see “Product_Sold_total”, click the lens icon to explore it.

The “Product_Sold_total” metrics details opens up. Here click the Insert button which will add this metrics to the search text box of the previous dialog.

In this page, select the Graph and click the Execute button to see the graph. You can see the graph of the football product sold over time. Check the below image where we have explained this thing.

There are large number of metrics to explore. Example put http_ in the search text box to see the available metrics.

In the same way put kestrel in the search text box to see the kestrel related metrics.

Integrate Grafana in ASP.NET Core app

Download Grafana from https://grafana.com/oss/grafana/ and install it to your pc. On windows you get an installer to install Grafana. After installation open the grafana url – http://localhost:3000. You’ll need to log in; the default username and password are both “admin”.

Now, go to ‘Add new connection’ and search Prometheus for Data Sources then click Prometheus.

On the prometheus page click the “Add new data source” button. This will add prometheus to grafana.

Next, you will be taken to the prometheus configurations page. Here add prometheus server url which is http://localhost:9090. Then click “Save & test” button. This will show successful message along with a link “building a dashboard”. Click this link to start building the dashboard. Check the below image screenshot.

On the next page, click “Add visualization” button.

Next, click on “prometheus” for data source.

This will take you to Grafana Dashboard where you can select the metrics – “”Product_Sold_total” and click the “Run queries” button to see the graph of this metrics. Check the below image where we have shown this.

We have successfully integrated Grafana in ASP.NET Core app. We can design sophisticated dashboards that will track any number of metrics. Each metric in .NET can have dimensions, which are key-value pairs that can be used to partition the data.

In this tutorial we learned how to integrate Prometheus and Grafana in ASP.NET Core. We also learned how to create custom metrics and view it’s details in prometheus. At the end we also learned to create dashboards in Grafana where metrics are viewed in graphs. We hope you liked this tutorial, if any question ask them through the comments section below.

The post How to use Prometheus and Grafana in ASP.NET Core appeared first on YogiHosting.

Whether you are handling batch data imports, real-time telemetry, or large-scale synchronization, Entity Framework Extensions library bridges the gap between the productivity of an ORM and the raw speed of specialized data loading tools. Thus offers a “need for speed” approach, that can reduce execution times by up to 95%, all while maintaining seamless integration with your existing DbContext and entity configurations.

In this tutorial we will be implementing Entity Framework Extensions library features in our ASP.NET Core app to create popular features on bulk operations.

Integration of Entity Framework Extensions

In the ASP.NET Core app, open NuGet package manager and install the package Z.EntityFramework.Extensions.EFCore. Since Entity Framework Extensions works with EF Core therefore the app should also have Microsoft.EntityFrameworkCore.SqlServer package installed.

Below I have shown the screenshot of this library from NuGet Package Manager.

Bulk Inserting a large CSV file on SQL Server Database with Entity Framework Extensions

Lets build the bulk Inserting of records to a SQL Server Database with Entity Framework Extensions library. I will use a CSV file, containing 50,000 + records of the World Bank economic survey, for this bulk inserting work. You can find this CSV file on the GitHub repository along with the app.

Note that for working with CSV files, I will be using CsvHelper library, that can be installed from NuGet.

On the ASP.NET Core app, add the entity class called Survey.cs. This class maps the survey records on the CSV file.

public class Survey
{
    public string Year { get; set; }
    public string Industry_aggregation_NZSIOC { get; set; }
    public string Industry_code_NZSIOC { get; set; }
    public string Industry_name_NZSIOC { get; set; }
    public string Units { get; set; }
    public string Variable_code { get; set; }
    public string Value { get; set; }
    public string Variable_name { get; set; }
    public string Variable_category { get; set; }
    public string Industry_code_ANZSIC06 { get; set; }
}

Next, I add the razor view file called ImportCsv.cshtml file which contains a file upload control for uploading the csv file.

@{
    ViewData["Title"] = "Upload CSV";
}

<h1 class="bg-info text-white">Upload CSV</h1>

<h2>@ViewData["Message"]</h2>

<form method="post" enctype="multipart/form-data">
    <div class="form-group">
        <label for="Poster"></label>
        <input type="file" id="csvfile" name="csvfile" class="form-control" />
    </div>
    <button type="submit" class="btn btn-primary">Create</button>
</form>

The Controller code which performs the bulk data inserts in the database is given below.

public class AdminController : Controller
{
    private BulkContext context;
    private IWebHostEnvironment hostingEnvironment;
    public AdminController(BulkContext c, IWebHostEnvironment environment)
    {
        context = c;
        hostingEnvironment = environment;
    }

    public IActionResult ImportCsv()
    {
        return View();
    }

    [HttpPost]
    [ActionName("ImportCsv")]
    public async Task<IActionResult> ImportCsv_Post(IFormFile csvfile)
    {
        // By CsvReader Package
        string path = Path.Combine(hostingEnvironment.WebRootPath, "CSV/" + csvfile.FileName);
        using (var stream = new FileStream(path, FileMode.Create))
        {
            await csvfile.CopyToAsync(stream);
        }

        var config = new CsvConfiguration(CultureInfo.InvariantCulture)
        {
            PrepareHeaderForMatch = args => args.Header.ToLower(),
            Delimiter = ",",
            MissingFieldFound = null,
            BadDataFound=null
        };
        using (var reader = new StreamReader(path))
        {
            using (var csv = new CsvReader(reader, config))
            {
                var records = csv.GetRecords<Survey>();
                context.BulkInsert(records, options => options.InsertKeepIdentity = true);

            }
        }
        ViewData["Message"] = "Import Successful";
        return View();
    }

}

Explanation – Here CsvReader library is used to read the csv file which the user uploads. After preforming some initial configurations, the csv records are read in a Survey class object by the below code:

var records = csv.GetRecords<Survey>();

Next, all the records are bulk insert with Entity Framework Extensions. Just a single line of code to do this job.

context.BulkInsert(records, options => options.InsertKeepIdentity = true);

I have used Stopwatch class to measure the time it takes to complete this operation. Note that there are massive 50,000 + records to be inserted. Entity Framework Extensions completed this in 2183 milliseconds. This is indeed quite impressive. Check the below screenshot.

Need More Speed ? Use BulkInsertOptimized method. It is 25% more fast. See below code line.

context.BulkInsertOptimized(records, options => options.InsertKeepIdentity = true);

The Key difference between them are:

• BulkInsert: AutoMapOutputDirection = true by default. It returns values like identity keys but can generate slightly less optimized SQL.
• BulkInsertOptimized: AutoMapOutputDirection = false by default. It skips return values for maximum speed, unless you explicitly ask for them.

By the way this whole code is there on my GitHub repository which you can copy and use it freely.

Reading CSV files by old OleDbConnection manner

We can also read CSV files by old OleDbConnection class without using CsvHelper. The below code does this work.

Note that you have to install the package System.Data.OleDb from NuGet.

[HttpPost]
public async Task<IActionResult> ImportCsv_Old(IFormFile csvfile)
{
    // By old OleDbConnection way

    string path = Path.Combine(hostingEnvironment.WebRootPath, "CSV/" + csvfile.FileName);
    using (var stream = new FileStream(path, FileMode.Create))
    {
        await csvfile.CopyToAsync(stream);
    }

    string folderPath = Path.Combine(hostingEnvironment.WebRootPath, "CSV"); // The Data Source is the folder, not the file
    string connectionString = $@"Provider=Microsoft.ACE.OLEDB.12.0;Data Source={folderPath};Extended Properties=""text;HDR=YES;FMT=Delimited;IMEX=1;MaxScanRows=0""";

    using (var conn = new OleDbConnection(connectionString))
    {
        conn.Open();
        var query = $"SELECT * FROM [{csvfile.FileName}]"; // The file name is used in the query
        using (var adapter = new OleDbDataAdapter(query, conn))
        {
            var dataTable = new DataTable();
            adapter.Fill(dataTable);

            List<Survey> records = dataTable.AsEnumerable().Select(row => new Survey
            {
                Year = row.Field<string>("year"),// Use .Field<T>() for type safety and null handling
                Industry_aggregation_NZSIOC = row.Field<string>("vote_average"),
                Industry_code_NZSIOC = row.Field<string>("industry_code_NZSIOC"),
                // other fiels add heere
            }).ToList();

            context.BulkInsert(records, options => options.InsertKeepIdentity = true);
        }
    }

    return View();
}

Copying large DB data from one Table to another Table with Entity Framework Extensions

Entity Framework Extensions high speed is useful in making backups of the database tables. This can come handy in situations like copying a large DB table to another table.

Here I have a massive database containing 200k + movies records in my dabase table called BulkMovies. I will copy all its records to a new table called BulkMoviesCopy.

To do this I will have to add 2 classes for the movie object as shown below. Since we are making a copy therefore these classes fields are all same.

public class BulkMovies
{
    public int? Id { get; set; }
    public string Title { get; set; }
    public double? Vote_Average { get; set; }
    public int? Vote_Count { get; set; }
    public string Status { get; set; }
    public DateTime? Release_Date { get; set; }
    public long? Revenue { get; set; }
    public int? Runtime { get; set; }
    public string Adult { get; set; }
    public int? Budget { get; set; }
    public string Homepage { get; set; }
    public string Overview { get; set; }
    public string Tagline { get; set; }
    public string Genres { get; set; }
    public string Production_Companies { get; set; }
    public string Production_Countries { get; set; }
    public string Spoken_Languages { get; set; }
    public string Keywords { get; set; }
}

public class BulkMoviesCopy
{
    public int? Id { get; set; }
    public string Title { get; set; }
    public double? Vote_Average { get; set; }
    public int? Vote_Count { get; set; }
    public string Status { get; set; }
    public DateTime? Release_Date { get; set; }
    public long? Revenue { get; set; }
    public int? Runtime { get; set; }
    public string Adult { get; set; }
    public int? Budget { get; set; }
    public string Homepage { get; set; }
    public string Overview { get; set; }
    public string Tagline { get; set; }
    public string Genres { get; set; }
    public string Production_Companies { get; set; }
    public string Production_Countries { get; set; }
    public string Spoken_Languages { get; set; }
    public string Keywords { get; set; }
} 

Next, on a post action method, which is shown below, I have used BulkInsert method to perform the copying task.

[HttpPost]
[ActionName("BulkCopy")]
public IActionResult BulkCopy_Post()
{
    List<BulkMovies> bm = context.BulkMovies.AsQueryable().ToList();

    List<BulkMoviesCopy> bmc = bm.Select(t => new BulkMoviesCopy
    {
        Id = t.Id,
        Title = t.Title,
        Vote_Average = t.Vote_Average,
        Vote_Count = t.Vote_Count,
        Status = t.Status,
        Release_Date = t.Release_Date,
        Revenue = t.Revenue,
        Runtime = t.Runtime,
        Adult = t.Adult,
        Budget = t.Budget,
        Homepage = t.Homepage,
        Overview = t.Overview,
        Tagline = t.Tagline,
        Genres = t.Genres,
        Production_Companies = t.Production_Companies,
        Spoken_Languages = t.Spoken_Languages,
        Keywords = t.Keywords
    }).ToList();

    var clock = new Stopwatch();
    clock.Start();

    context.BulkInsert(bmc, options => options.InsertKeepIdentity = true);

    clock.Stop();
    var time = clock.ElapsedMilliseconds;

    return View();
}

It should be noted the BulkMoviesCopy object is mapped to the movies data contained by BulkMovies object called “bm” using the LINQ Select method as shown below.

List<BulkMovies> bm = context.BulkMovies.AsQueryable().ToList();
List<BulkMoviesCopy> bmc = bm.Select(t => new BulkMoviesCopy
{
    Id = t.Id,
    Title = t.Title,
    // include the fields that need to be copied
}).ToList();

Then with the BulkInsert method the full data is copied to the BulkMoviesCopy table by the below code line.

context.BulkInsert(bmc, options => options.InsertKeepIdentity = true);

Below is the result. It took just 9156 milliseconds to perform this job. Considering the records are massive and this little time to complete the job is indeed impressive.

Bulk Updates and Deletion

In Entity Framework Core we have Add and Update method to perform records Inserts and Updates. In Entity Framework Extensions there is a BulkMerge method which does both the work of records addition and records modification.

It matches the primary key or the composite key of the entity and then automatically decides which record needs to be inserted and which needs to be updated. This decision is made based on:

• If the DB table already has the row matching the record key (primary or composite) then this row is updated.
• If the DB table does not have any row matching the record’s key then this row is inserted to the table.

This certainly reduces the number of code lines if we have to perform multiple tasks like Inserts and Updates on the same method.

Now we are going to implement BulkMerge method in order to perform bulk update of the records.

Add Employee.cs class to the app which is given below:

public class Employee
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Designation { get; set; }
    public string Address { get; set; }
    public int Salary { get; set; }
}

Next, add an action method of HTTP POST type since we will be calling the codes on the button click. This action method is given below.

[HttpPost]
public async Task<IActionResult> BulkMerge_Post()
{
    List<Employee> emp = context.Employee.AsQueryable().ToList();

    List<Employee> updateEmployee = emp.Select(t => new Employee
    {
        Id = t.Id,
        Name = t.Name,
        Designation = t.Designation,
        Address = t.Address,
        Salary = t.Salary + 1000,
    }).ToList();

    context.BulkMerge(updateEmployee);

    return View();
}

The code above is increasing the salary of every employee by $1000. It is adding the 1000 to the current salary by – Salary = t.Salary + 1000 and then calling the BulkMerge method to perform this update.

In the same way we can perform the bulk inserts by modified code which is given below. This code inserts the Employees since their Id field is not provided.

List<Employee> insertEmployee = new List<Employee> {
    new Employee{Name="N1", Designation="D1", Address="A1", Salary=1000},
    new Employee{Name="N2", Designation="D2", Address="A2", Salary=2000},
    new Employee{Name="N3", Designation="D3", Address="A3", Salary=3000}
};

context.BulkMerge(insertEmployee);

Mostly in real work scenario you will come acrosss a situation where an excel / CSV file containing the updated data is provided to you. And you as a developer will have to update the database table with this new data on the file. In this situation make sure the key value (which here is Id) given on the excel / csv should match for the records which have to be updated while the key should be empty for the records that needs to be inserted.

See the below code which does exactly this thing.

[HttpPost]
public async Task<IActionResult> BulkMerge_Post()
{
    // read the excel/csv file code omitted for brevity

    List<BulkMoviesCopy> bmc = dataTable.AsEnumerable().Select(row => new    BulkMoviesCopy
{
    Id = row.Field<int>("id"), // Use .Field<T>() for type safety and null handling
    Name = row.Field<string>("name"),
    Designation = row.Field<string>("Designation"),
    Address = row.Field<string>("Address"),
    Salary = row.Field<int>("Salary"),
}).ToList();

    context.BulkMerge(updateEmployee);

    return View();
}

The excel file can contain tens of thousands of records and Entity Frameowork Extensions will complete this in a matter of seconds.

The bulk delete feature is provided through BulkDelete method which deletes large number of records in a fraction of seconds. Here also you can read the records that needs to be deleted from an excel or csv file and call this method as shown below.

context.BulkDelete(deleteEmployee);

The benefits of BulkDelete method:

1. Many ways to delete: custom keys, delete related entities and apply any conditions.
2. Super fast.
3. No need to load entities. No change tracking.

To wrap things up, Entity Framework Extensions serve as the “turbocharger” for the standard EF Core engine. While Microsoft has made great strides in optimizing EF Core, developers often hit a performance wall when dealing with large datasets or complex batch operations.

Here is a summary of why they remain a staple in the .NET ecosystem:

The Performance Edge:

The core value proposition is speed. By bypassing the traditional “one-at-a-row” processing model and utilizing efficient database-level commands, extensions can turn operations that take minutes into tasks that take seconds.

Key Takeaways:

Scalability: They are essential for enterprise-level applications where data growth is inevitable.

Developer Productivity: You get to keep the clean, LINQ-based syntax you love while gaining the performance of raw SQL.

Resource Efficiency: By reducing the number of round-trips to the database, you lower CPU and memory overhead on both the application server and the database.

Final Verdict:

Entity Framework Extensions are not a replacement for EF Core, but rather a necessary evolution for performance-critical applications. If your project involves data migrations, nightly syncs, or massive user-generated content, the investment in this library usually pays for itself in saved execution time and reduced infrastructure costs.

The post The Entity Framework Extensions: Performance-Focused (Need for Speed) when working with large Datasets appeared first on YogiHosting.

OpenTelemetry (often called OTel) is the global industry standard for how software applications generate and send data about their health and performance. It is an open source observability framework for cloud native software which provides a single set of APIs, libraries, agents, and collector services to capture distributed traces and metrics from your application.

What is Trace and Activity? Each time a new request is received by an app, it can be associated with a trace. In .NET, a trace are represented by instances of System.Diagnostics.Activity and it can be spanning across many distinct processes. The first Activity is the root of the trace tree and it tracks the overall duration and success/failure. Child activities can be created to subdivide the work into different steps that can be tracked individually.

What is Jaeger

Jaeger is an observability platforms to track requests going through a complex web of services. Jaeger acts as the connective tissue, mapping every hop and uncovering hidden delays or failures. By bridging the gaps between isolated components, Jaeger transforms fragmented data into a clear roadmap for troubleshooting, allowing engineers to pinpoint bottlenecks and boost reliability. It is a fully open-source, cloud-native powerhouse built to scale alongside the most demanding architectures.

Jaeger does the following things:

• Monitor distributed workflows
• Find & fix performance bottlenecks
• Track down root causes
• Analyze service dependencies

The below figure explains the Jaeger architecture where it receives the data from traced applications and write it directly to storage. The Jaeger UI is a web-based interface for visualizing and analyzing distributed tracing data, playing a critical role in microservices monitoring. It enables developers to search for specific traces, map request flows, analyze latency bottlenecks, compare trace data, and troubleshoot service errors.

First download Jaeger from the official website. I am using windows so I will proceed with the windows version. The download file is a compressed file which you need to extract to a folder. You will get jaeger.exe file, next navigate to this folder from command prompt and run the command – start jaeger. See the below image where I am starting jaeger from command prompt.

A new command prompt window will open which will show Jaeger logs – version, ports and configurations. This means Jaeger is in the running state.

Open the url – http://localhost:16686 on your browser which will open the Jaeger UI.

Integrate Jaeger in ASP.NET Core

First I need to add OpenTelemetry libraries and SDKs to add instrumentation into my ASP.NET Core app. These instrumentations automatically capture the traces, metrics, and logs we are interested in. These libraries need to be installed from NuGet:

dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.Http
dotnet add package OpenTelemetry.Exporter.Console

It’s time to configure some services in the program class.

Update the code of Program.cs file as given below.

var tracingOtlpEndpoint = builder.Configuration["OTLP_ENDPOINT_URL"];
var otel = builder.Services.AddOpenTelemetry();

otel.ConfigureResource(resource => resource.AddService(serviceName: builder.Environment.ApplicationName));

otel.WithTracing(tracing =>
{
    tracing.AddAspNetCoreInstrumentation();
    tracing.AddHttpClientInstrumentation();
    if (tracingOtlpEndpoint != null)
    {
        tracing.AddOtlpExporter(otlpOptions =>
        {
            otlpOptions.Endpoint = new Uri(tracingOtlpEndpoint);
        });
    }
    else
    {
        tracing.AddConsoleExporter();
    }
});

In the above code:

• AddOpenTelemetry – adds OpenTelemetry SDK services
• AddAspNetCoreInstrumentation – adds ASP.NET Core instrumentation
• AddHttpClientInstrumentation – adds HttpClient instrumentation for outgoing requests

I am reading the url of Jaeger endpoint from the appsettings.json from the code line:

var tracingOtlpEndpoint = builder.Configuration\["OTLP\_ENDPOINT\_URL"];

Then I am adding OpenTelemetry (OTLP) exporter to the trace provider which is Jaeger. The below code line does this work.

tracing.AddOtlpExporter(otlpOptions =>
{
    otlpOptions.Endpoint = new Uri(tracingOtlpEndpoint);
});

I will have to add the “OTLP_ENDPOINT_URL” in the appsettings.json as shown below.

"OTLP_ENDPOINT_URL": "http://localhost:4317/"

The url – http://localhost:4317/ is where Jaeger is listening to the traffic. This is given on the Jaeger console (which opened when I ran the start jaeger start command). Find the line where Jaeger is listening for OTLP traffic via gRPC to contain this url. The below given line is the one which contains this – “endpoint”: “127.0.0.1:4317”.

2026-03-07T11:42:39.708+0530    info    otlpreceiver@v0.145.0/otlp.go:120       Starting GRPC server    {"resource": {"service.instance.id": "171c6e00-b515-48e6-bf89-49c86e4bd3ce", "service.name": "jaeger", "service.version": "v2.15.0"}, "otelcol.component.id": "otlp", "otelcol.component.kind": "receiver", "endpoint": "127.0.0.1:4317"}

Lets run the app and check whether Jaeger is capturing the Traces or not. I will have to refresh the Jaeger UI tab.

Jaeger has captured my Apps traces. My apps name is “JaegerTutorial” and it is visible on the dropdown list, the below image shows it.

I click on the Find Traces button to see all the traces. Next I select the trace called – JaegerTutorial: GET {controller=Home}/{action=Index}/{id?} 4241af5. This shown the HTTP GET traces captured when the Home page of the app is called. Also notice the otel.scope.name as Microsoft.AspNetCore. Check the below image:

I have successfully integrated Jaeger to my ASP.NET Core app.

SQL Server Tracing in Jaeger + ASP.NET Core

To collect SQL Server traces and export to Jaeger, I first need to add the NuGet package called OpenTelemetry.Instrumentation.SqlClient. So I run the below given dotnet add package command:

dotnet add package OpenTelemetry.Instrumentation.SqlClient

Next, I enable Enable SqlClient Instrumentation in the Program class with the following code line – AddSqlClientInstrumentation(). Check the highlighted code below:

otel.WithTracing(tracing =>
{
    tracing.AddAspNetCoreInstrumentation();
    tracing.AddHttpClientInstrumentation();
    tracing.AddSqlClientInstrumentation();     
    if (tracingOtlpEndpoint != null)
    {
        tracing.AddOtlpExporter(otlpOptions =>
        {
            otlpOptions.Endpoint = new Uri(tracingOtlpEndpoint);
        });
    }
    else
    {
        tracing.AddConsoleExporter();
    }
});

This will record the following:

• error type
• namespace
• operation name
• query summary
• query text
• response status code
• stored procedure name
• system name
• server address
• server port

With this configuration in place, when the app fetched the records from the SQL Server database. The SQL Server traces are recorded and can be viewed on Jaeger. On Jaeger UI, I can see details like:

• db.system.name microsoft.sql\_server
• otel.scope.name OpenTelemetry.Instrumentation.SqlClient

To test it, I added a Read action method in the HomeController.cs which reads Employee records from the database. See below code.

public async Task<IActionResult> Read()
{
    var employees = context.Employee;
    return View(employees);
}

Next, on Jaeger I can see the traces along with the SQL Select query that ran against the database.

SELECT [e].[Id], [e].[DateOfBirth], [e].[Designation], [e].[Email], [e].[FirstName], [e].[Gender], [e].[LastName], [e].[Telephone]
FROM [Employee] AS [e]

Check the below Jaeger screenshot where I have shown this information.

This is very helpful in cases where there are databases errors like connection errors, sql query errors and so on. We can find the causes and location of these error and can quicky fix them. See the below image which shows error trace displayed on Jaeger. This database error came when trying to insert a record. Jaeger also displayed the url path in the app from where this db error was raised.

Create Custom Traces with ActivitySource class

We can create custom traces or Activities with ActivitySource class. An Activity has an operation name, an ID, a start time and duration, tags, and baggage. I added a class named AppActivitySource.cs inside the Models folder of the app, see it’s code below.

public class AppActivitySource : IDisposable
{
    public static readonly string ActivitySourceName = "AppCustomActivity";
    private readonly ActivitySource activitySource = new ActivitySource(ActivitySourceName);

    public Activity AppActivity(string activityName, ActivityKind kind = ActivityKind.Internal)
    {
        var activity = activitySource.StartActivity(activityName, kind);
        return activity;
    }

    public void Dispose()
    {
        activitySource.Dispose();
    }
}

The above class creates an ActivitySource object with name “AppCustomActivity”. The name is provided in a static variable so that it can be added to the TracerProviderBuilder object in the Program class.

See the AppActivity() method which creates a new Activity using the StartActivity() method.

Now, I register the AppActivitySource class as a Singleton in the Program.cs and add it to the Trace builder. The code for this is given below.

builder.Services.AddSingleton<AppActivitySource>();

otel.WithTracing(tracing =>
{
    tracing.AddAspNetCoreInstrumentation();
    tracing.AddHttpClientInstrumentation();
    tracing.AddSqlClientInstrumentation(); 
    tracing.AddEntityFrameworkCoreInstrumentation(); 
    tracing.AddSource(AppActivitySource.ActivitySourceName); // custom Activity

    if (tracingOtlpEndpoint != null)
    {
        tracing.AddOtlpExporter(otlpOptions =>
        {
            otlpOptions.Endpoint = new Uri(tracingOtlpEndpoint);
        });
    }
    else
    {
        tracing.AddConsoleExporter();
    }
});

I now create custom Activites in the Create action method of the HomeController.cs. This action method adds a new employee to the database. See the highlighted code below.

public class HomeController : Controller
{
    private CompanyContext context;
    private AppActivitySource appActivitySource;

    public HomeController(CompanyContext context, AppActivitySource appActivitySource)
    {
        this.context = context;
        this.appActivitySource = appActivitySource;
    }

    // other methods

    public IActionResult Create()
    {
        return View();
    }

    [HttpPost]
    public async Task<IActionResult> Create(Employee emp)
    {
        context.Add(emp);
        await context.SaveChangesAsync();

        using (Activity? activity = appActivitySource.AppActivity("CRUD Operations"))
        {
            activity?.SetTag("CREATE Employee", "Create Action HTTPOST");
        }

        ViewBag.Message = "Employee created successfully!";
        return View();
    }
}

After the employee is added I create a new custom Activity so that a new trace is created with the below code.

using (Activity? activity = appActivitySource.AppActivity("CRUD Operations"))
{
    activity?.SetTag("CREATE Employee", "Create Action HTTPOST");
}

On Jaeger this activity is recorded as shown by the below image.

Entity Framework Core Tracing in Jaeger + ASP.NET Core

To collect Entity Framework Core traces the process is similar to that of SQL Server. I need to add the package called OpenTelemetry.Instrumentation.EntityFrameworkCore. The command to run on NuGet is:

dotnet add package OpenTelemetry.Instrumentation.EntityFrameworkCore --version 1.15.0-beta.1

Next, on the Program class add the following code – AddEntityFrameworkCoreInstrumentation(). Check the highlighted code below.

otel.WithTracing(tracing =>
{
    tracing.AddAspNetCoreInstrumentation();
    tracing.AddHttpClientInstrumentation();
    tracing.AddSqlClientInstrumentation(); 
    tracing.AddEntityFrameworkCoreInstrumentation(); 

    if (tracingOtlpEndpoint != null)
    {
        tracing.AddOtlpExporter(otlpOptions =>
        {
            otlpOptions.Endpoint = new Uri(tracingOtlpEndpoint);
        });
    }
    else
    {
        tracing.AddConsoleExporter();
    }
});

Now, whenever EF core is used to communicate with the database, the traces are recorded and viewed on Jaeger. Check the below image where Jaeger is showing the traces Entity Framework Core code that fetched the records from the database.

Web API Tracing in Jaeger + ASP.NET Core

In order to collect HTTP Web API traces I need to enable AddHttpClientInstrumentation() on the program class.

otel.WithTracing(tracing =>
{
    tracing.AddAspNetCoreInstrumentation();
    tracing.AddHttpClientInstrumentation();
    tracing.AddSqlClientInstrumentation(); 
    tracing.AddEntityFrameworkCoreInstrumentation(); 

    if (tracingOtlpEndpoint != null)
    {
        tracing.AddOtlpExporter(otlpOptions =>
        {
            otlpOptions.Endpoint = new Uri(tracingOtlpEndpoint);
        });
    }
    else
    {
        tracing.AddConsoleExporter();
    }
});

Next, when a web api is called then it’s traces are viewable on Jaeger. See the below image showing HTTP Response Status Code of 200 received from the web api.

Redis Cache Tracing in Jaeger + ASP.NET Core

Redis and Jaeger are often used together in modern cloud-native architectures to manage application performance and visibility. Redis serves as a high-speed, in-memory cache to speed up data retrieval, while Jaeger provides distributed tracing to visualize how requests flow through systems, including the time spent accessing Redis.

Firstly I run Redis from a Docker Container. The command is given below:

docker run --name redis -d -p 6379:6379 redis

Next, I install the package OpenTelemetry.Instrumentation.StackExchangeRedis for Redis Instrumentation and Microsoft.Extensions.Caching.StackExchangeRedis for working with Redis on the app. The NuGet command is given below:

dotnet add package OpenTelemetry.Instrumentation.StackExchangeRedis --version 1.0.0-rc9.15
dotnet add package Microsoft.Extensions.Caching.StackExchangeRedis

After this, on the program class, I add IConnectionMultiplexer object to create a connection to the Redis server which is running from localhost:6379 url. I also register the redis cache as a service in the program class. See the below code.

IConnectionMultiplexer connectionMultiplexer = ConnectionMultiplexer.Connect("localhost:6379");

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.ConnectionMultiplexerFactory = () => Task.FromResult(connectionMultiplexer);
});

Now final thing is to add the Redis service to the trace builder. See the highlighted code below.

otel.WithTracing(tracing =>
{
    tracing.AddAspNetCoreInstrumentation();
    tracing.AddHttpClientInstrumentation();
    tracing.AddSqlClientInstrumentation(); 
    tracing.AddEntityFrameworkCoreInstrumentation(); 
    tracing.AddSource(AppActivitySource.ActivitySourceName); 
    tracing.AddRedisInstrumentation(connectionMultiplexer); 

    if (tracingOtlpEndpoint != null)
    {
        tracing.AddOtlpExporter(otlpOptions =>
        {
            otlpOptions.Endpoint = new Uri(tracingOtlpEndpoint);
        });
    }
    else
    {
        tracing.AddConsoleExporter();
    }
});

I now use the IDistributedCache interface that provides the methods to manipulate items in the distributed cache implementation which in my case is Redis cache. So I create a new class called DistributedCacheExtensions.cs which provides extension methods for IDistributedCache. These extension method will provide works like reading from Cache and adding values to cache. I have not provided the class code here but you will find this class on the source code of this app, check the “Models” folder where you will find this class.

public static class DistributedCacheExtensions
{
    // code
}

Let’s check the working of Redis Cache and find it’s instrumentation data on Jaeger. So I add a new action method called “ReadCache” in the HomeController.cs with the following code.

public async Task<IActionResult> ReadCache()
{
    var employees = await GetAll();
    return View(employees);
}

public async Task<List<Employee>> GetAll()
{
    var cacheKey = "employees";
    var cacheOptions = new DistributedCacheEntryOptions()
            .SetAbsoluteExpiration(TimeSpan.FromMinutes(20))
            .SetSlidingExpiration(TimeSpan.FromMinutes(2));
    var products = await cache.GetOrSetAsync(
        cacheKey,
        async () =>
        {
            return await context.Employee.ToListAsync();
        },
        cacheOptions)!;
    return products!;
}

When this action method executes, the first time there is no cache so database is called to get the records. These records are also saved on the Redis Cache so that for the next time, the records are fetched from the Cache itself.

Now on Jaeger, I can see the full Redis Cache traces. I have shown this in the below images.

Microservices – MassTransit – RabbitMQ Cache Tracing in Jaeger + ASP.NET Core

I will now implement tracing for MassTransit and RabbitMQ. I will send Messages to RabbitMQ with MassTransit so that RabbitMQ publishes the messages to the consumer.

First thing I have to do is to run RabbitMQ from a Docker Container. The command is given below.

docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:4-management

In the app I install the RabbitMQ OpenTelemetry NuGet package:

dotnet add package RabbitMQ.Client.OpenTelemetry --version 1.0.0-rc.2

Next, on the program class, I add AddRabbitMQInstrumentation() as shown below.

otel.WithTracing(tracing =>
{
    tracing.AddAspNetCoreInstrumentation();
    tracing.AddHttpClientInstrumentation();
    tracing.AddSqlClientInstrumentation(); 
    tracing.AddEntityFrameworkCoreInstrumentation(); 
    tracing.AddSource(AppActivitySource.ActivitySourceName); 
    tracing.AddRedisInstrumentation(connectionMultiplexer); 
    tracing.AddRabbitMQInstrumentation(); 

    if (tracingOtlpEndpoint != null)
    {
        tracing.AddOtlpExporter(otlpOptions =>
        {
            otlpOptions.Endpoint = new Uri(tracingOtlpEndpoint);
        });
    }
    else
    {
        tracing.AddConsoleExporter();
    }
});

To test it, I first add the NuGet Package RabbitMQ.Client with the following command:

dotnet add package RabbitMQ.Client

Next, I add SendMessage action in the HomeController which sends the message to RabbitMQ.

public async Task<IActionResult> SendMessage(Employee emp)
{
    var factory = new ConnectionFactory { HostName = "localhost" };
    using var connection = await factory.CreateConnectionAsync();
    using var channel = await connection.CreateChannelAsync();

    await channel.QueueDeclareAsync(queue: "hello", durable: false, exclusive: false, autoDelete: false,
        arguments: null);

    const string message = "Hello World!";
    var body = Encoding.UTF8.GetBytes(message);

    await channel.BasicPublishAsync(exchange: string.Empty, routingKey: "hello", body: body);

    return RedirectToAction("Index");
}

The traces of the messages are viewed in Jaeger. Check the below screenshot.

Microservices architecture is a more complex distributed trace. Take for example the case of an Online Shopping website, I am sending a request to get the customer’s cart. This request will first hit the API gateway, which proxies it to the Microservice that owns the data. Next this microservice calls another microservice to get the authorization information. And all of this leads to the distributed trace that Jaeger will display in a beautiful manner.

In this tutorial I covered how to use Jaeger in ASP.NET Core. I explained by to capture traces of SQL Server, Entity Framework Core, ActivitySource, Web API, Redis Cache, RabbitMQ and other things. You can download the source codes of this tutorial from my GitHub account, link given at the top.

The post How to integrate OpenTelemetry with Jaeger for Distributed Tracing in ASP.NET Core appeared first on YogiHosting.

Proactive Recovery: Systems like Kubernetes use these checks to identify “zombie” processes that are running but frozen, automatically restarting them to minimize downtime.

Smart Traffic Routing: Load balancers like Nginx use health checks to divert traffic away from struggling servers, ensuring users only hit healthy instances.

Early Detection: They catch issues—like a disconnected database or a full disk—before they escalate into a site-wide outage.

Add Health Checks in ASP.NET Core Apps

In ASP.NET Core app a basic heath check can be added to process requests (liveness) of the app. In the Program class import the namespace called “Microsoft.Extensions.Diagnostics.HealthChecks” then add builder.Services.AddHealthChecks() and app.MapHealthChecks("/healthz") as shown below.

using Microsoft.AspNetCore.Diagnostics.HealthChecks;
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHealthChecks();

var app = builder.Build();

app.MapHealthChecks("/healthz");

app.Run();

This creates a health check endpoint at /healthz. If we open this endpoint on the browser we will see Healthy text in plain text. See the below image.

Create Custom Health Checks in ASP.NET Core

To implement Health checks, create a class and implement the IHealthCheck interface. The interface method called CheckHealthAsync returns a HealthCheckResult that indicates the health of the app as Healthy, Degraded, or Unhealthy. The result is written as a plaintext response with a configurable status code. Check the below sample:

public class MyappHealthCheck : IHealthCheck
{
    public Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context, CancellationToken cancellationToken = default)
    {
        var isHealthy = true;

        // code to check the app and its necessary services. Set isHealthy variable to false if there is some service not working

        if (isHealthy)
        {
            return Task.FromResult(
                HealthCheckResult.Healthy("My App is Healthy"));
        }

        return Task.FromResult(
            new HealthCheckResult(
                context.Registration.FailureStatus, "My App is Un-Healthy"));
    }
}

If CheckHealthAsync throws an exception during the check, a new HealthCheckResult is returned with a FailureStatus message.

Health check probes are mechanisms used to determine the state of an application running inside a container. They help ensure reliability, automatic recovery, and proper traffic routing. There are 3 types of probes – Liveness Probe, Readiness and Startup. These are explained in the below table.

See the below chart to understand the comparison between these probes.

In order to add these 3 probes – Liveness Probe, Readiness and Startup, to our .NET app. We have to Register these health check probes in the Program.cs and then create health check endpoints for each of these 3 probes. Check the highlighted code below.

using Microsoft.AspNetCore.Diagnostics.HealthChecks;
var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddControllersWithViews();

builder.Services.AddHealthChecks()
    .AddCheck<MyHealthCheckStartup>(
        "Startup",
        tags: new[] { "sp_tag" });

builder.Services.AddHealthChecks()
    .AddCheck<MyHealthCheckLiveness>(
        "Liveness",
        tags: new[] { "lp_tag" });

builder.Services.AddHealthChecks()
    .AddCheck<MyHealthCheckReadiness>(
        "Readiness",
        tags: new[] { "rp_tag" });

var app = builder.Build();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Home/Error");
    app.UseHsts();
}

app.MapHealthChecks("/healthz/SP", new HealthCheckOptions()
{
    Predicate = (check) => check.Tags.Contains("sp_tag")
});
app.MapHealthChecks("/healthz/LP", new HealthCheckOptions()
{
    Predicate = (check) => check.Tags.Contains("lp_tag")
});
app.MapHealthChecks("/healthz/RP", new HealthCheckOptions()
{
    Predicate = (check) => check.Tags.Contains("rp_tag")
});

app.UseHttpsRedirection();
app.UseRouting();

app.UseAuthorization();

app.MapStaticAssets();

app.MapControllerRoute(
    name: "default",
    pattern: "{controller=Home}/{action=Index}/{id?}")
    .WithStaticAssets();

app.Run();

In the above code we create 3 endpoints for Startup, Liveness and Readiness probes. These endpoints are:

/healthz/SP 
/healthz/LP
/healthz/RP

These endpoints are implemented with 3 separate C# classes which are:

1. MyHealthCheckStartup.cs
2. MyHealthCheckLiveness.cs
3. MyHealthCheckReadiness.cs

By default, the Health Checks Middleware runs all registered health checks. To filter health checks for running a subset of health checks, provide a function that returns a boolean to the Predicate option. Here we applied filters to the health checks so that only those tagged with “sp_tag” runs for endpoint “/healthz/SP”, tagged with “lp_tag” runs for “/healthz/LP” and tagged with “rp_tag” runs for “/healthz/RP”. See the below codes:

app.MapHealthChecks("/healthz/SP", new HealthCheckOptions()
{
    Predicate = (check) => check.Tags.Contains("sp_tag")
});

app.MapHealthChecks("/healthz/LP", new HealthCheckOptions()
{
    Predicate = (check) => check.Tags.Contains("lp_tag")
});

app.MapHealthChecks("/healthz/RP", new HealthCheckOptions()
{
    Predicate = (check) => check.Tags.Contains("rp_tag")
});

Next, we add the following 3 classes to our app which implements the 3 probes – Startup, Liveness & Readiness.

Implementation of Startup Probe

An app takes some initial time to start because it does some initial tasks like running DB migrations, warm up caches, load large configurations, initialize background services, compile codes and so on. It is necessary to not send traffic to the app until it has fully started.

In the below class we have implemented the Startup Probe in a class which does the initial work, and only when this work is successfully finished the Healthy HealthCheckResult is returned.

public class MyHealthCheckStartup : IHealthCheck
{
    public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default)
    {
        if (StartupWork.DoWork())
        {
            return Task.FromResult(HealthCheckResult.Healthy("The startup task has completed."));
        }

        return Task.FromResult(HealthCheckResult.Unhealthy("That startup task is still running."));
    }
}

The StartupWork.DoWork() checks the status of the initial work and will get a bool value of true or false telling whether this work has completed or not. The StartupWork.cs class code is given below, it does the checking of the work progress. Although we are demonstrating this by applying a time delay of 10 seconds so that in these 10 seconds the work completes. During these first 10 seconds the startup probe will fail since it returns “Unhealthy” status and after 10 seconds it completes successfully because it will then return “Healthy” status.

In real world app, you have to add your own code which does the complete checks and returns bool value of true when the initial works have finished.

public class StartupWork
{
    private static readonly DateTime _startTime = DateTime.UtcNow;
    public static bool DoWork()
    {
        // here check the status of the initial work
        if ((DateTime.UtcNow - _startTime).TotalSeconds >= 10)
        {
            return true;
        }

        return false;
    }
}

The apps hosted on Kubernetes runs the startup probe first and until it succeeds the Liveness and Readiness probes are disabled.

Once Startup probe succeeds, then only the liveness/readiness probes begin. If the startup probe fails too many times → the container is restarted.

Implementation of Readiness Probe – Example of Database Probe

Readiness probes ensure the application is functional and ready to serve traffic, making them ideal for checking dependency availability. For example we should check whether the database is responding normally or not.

For checking the database function choose a query that returns quickly. Ideal query is – SELECT 1 since it completes quickly in the database. We defined MyHealthCheckReadiness class which performs the Readiness Health Check probe to test the database. The SeELECT 1 query is executed against the database, if successful we return HealthCheckResult.Healthy("The Database is working properly") else in case of failure we return HealthCheckResult.Unhealthy(ex.Message). Check the below code.

public class MyHealthCheckReadiness : IHealthCheck
{
    private readonly string _connectionString;

    public MyHealthCheckReadiness(IConfiguration configuration)
    {
        _connectionString = configuration.GetConnectionString("Database");
    }

    public Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        try
        {
            using var sqlConnection = new SqlConnection(_connectionString);

            sqlConnection.OpenAsync(cancellationToken);

            using var command = sqlConnection.CreateCommand();
            command.CommandText = "SELECT 1";

            command.ExecuteScalarAsync(cancellationToken);

            return Task.FromResult(HealthCheckResult.Healthy("The Database is working properly"));
        }
        catch (Exception ex)
        {
            return Task.FromResult(HealthCheckResult.Unhealthy(ex.Message));
        }
    }
}

We can also use an existing Health Check Libraries to perform this SQL Server health check. Include a package reference to the AspNetCore.HealthChecks.SqlServer NuGet package. Then on the program class add the below code.

var conStr = builder.Configuration.GetConnectionString("DefaultConnection");
builder.Services.AddHealthChecks()
    .AddSqlServer(conStr);

If the app is using Entity Framework Core then include a package reference to the Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore NuGet package. Next, add the following code to the program class.

builder.Services.AddDbContext<SampleDbContext>(options =>
    options.UseSqlServer(
        builder.Configuration.GetConnectionString("DefaultConnection")));

builder.Services.AddHealthChecks()
    .AddDbContextCheck<SampleDbContext>();

There are Health Check Libraries available for popular programs some examples are given below.

• SQL Server – AspNetCore.HealthChecks.SqlServer
• Postgres – AspNetCore.HealthChecks.Npgsql
• Redis – AspNetCore.HealthChecks.Redis
• RabbitMQ – AspNetCore.HealthChecks.RabbitMQ
• AWS S3 – AspNetCore.HealthChecks.Aws.S3
• SignalR – AspNetCore.HealthChecks.SignalR

The Readiness probe for RabbitMQ is given below.

builder.Services.AddHealthChecks().AddRabbitMQ(rabbitConnectionString)

Implementation of Liveness Probe

The Liveness health check probe should only check internal app health. It should figure out if the app is still alive, or is it stuck/broken.

Common problem for the Liveness probe to fail are due to:

• Deadlocks
• Infinite loops
• Crashed background workers
• Memory corruption
• Thread pool exhaustion

Below class does the Liveness probe. You can update this sample based on your specific needs.

public class MyHealthCheckLiveness : IHealthCheck
{
    public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default)
    {
        if (LivenessWork.DoWork())
        {
            return Task.FromResult(HealthCheckResult.Healthy("The startup task has completed."));
        }

        return Task.FromResult(HealthCheckResult.Unhealthy("That startup task is still running."));
    }
}

Health Checks in Docker and Kubernetes

Docker can detect if your application inside the container is running properly or not. In the Dockerfile we can add HEALTHCHECK as given below.

FROM mcr.microsoft.com/dotnet/aspnet:10.0
WORKDIR /app
COPY . .
ENTRYPOINT ["dotnet", "MyApp.dll"]

HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
  CMD curl -f http://localhost/health || exit 1

It says run a curl command – Every 30 seconds and wait for 5 seconds to check the app. If it fails 3 times → mark container unhealthy.

The below code is for docker-compose.yml file. Here start_period gives the app time to start before health checks begin.

services:
  myapp:
    image: myapp:latest
    ports:
      - "5000:8080"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s

To restart unhealthy container use restart policy.

docker run -d --restart=always --name my_container my_image

The Docker Compose version is:

version: '3'
services:
  myservice:
    image: my-image
    restart: always

In Kubernetes, health checks (probes) are mechanisms that let Kubernetes determine whether your container:

1. Has started correctly
2. Is ready to receive traffic
3. Is still running properly

Kubernetes uses probes to automatically manage container lifecycle and traffic routing, container restarts and so on. We have written a complete article on Kubernetes Liveness Readiness Startup Probes which you will find quite useful.

The Kubernetes yml file that defines the health checks are given below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-dep
  labels:
    app: aspnet-core-app
spec:
  replicas: 1
  selector:
    matchLabels:
      component: web
  template:
    metadata:
      labels:
        component: web
    spec:
      containers:
        - name: csimpleweb
          image: simpleweb
          imagePullPolicy: Never
          ports:
            - containerPort: 8080
          startupProbe:
            httpGet:
              path: /healthz/SP
              port: 80
            failureThreshold: 25
            periodSeconds: 10
          
          livenessProbe:
            httpGet:
              path: /healthz/LP
              port: 80
            initialDelaySeconds: 2
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 10

          readinessProbe:
            httpGet:
              path: /healthz/RP
              port: 80
            initialDelaySeconds: 2
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 10
            successThreshold: 5

Health Check Publisher

a Health Check Publisher is a background component that periodically runs health checks and publishes the results somewhere (logs, metrics system, monitoring tool, etc.). It runs automatically in the background.

The work of the Health Check Publisher is to:

• Push health status to monitoring
• Log unhealthy states automatically
• Send alerts
• Export metrics to Prometheus
• Publish to Azure Application Insights

The Health Check Publisher must implement IHealthCheckPublisher interface of Microsoft.Extensions.Diagnostics.HealthChecks namespace. See the below sample:

using Microsoft.Extensions.Diagnostics.HealthChecks;

public class SampleHealthCheckPublisher : IHealthCheckPublisher
{
    public Task PublishAsync(HealthReport report, CancellationToken cancellationToken)
    {
        if (report.Status == HealthStatus.Healthy)
        {
            // ...
        }
        else
        {
            // ...
        }

        return Task.CompletedTask;
    }
}

We also need to registers the health check publisher as a singleton and configures HealthCheckPublisherOptions:

builder.Services.Configure<HealthCheckPublisherOptions>(options =>
{
    options.Delay = TimeSpan.FromSeconds(20);       // Initial delay
    options.Period = TimeSpan.FromSeconds(100);     // Run every 100s
    options.Predicate = healthCheck => healthCheck.Tags.Contains("sample"); // run a subset of health checks i.e. here tagged with sample
});

builder.Services.AddSingleton<IHealthCheckPublisher, SampleHealthCheckPublisher>();

What You Get in ‘HealthReport’ object – Overall status, Individual check results, Duration, Exception details and Custom data. See the below code.

foreach (var entry in report.Entries)
{
    Console.WriteLine($"{entry.Key} - {entry.Value.Status}");
}

Formatting Health Checks Response

By default, the health check endpoint returns a simple string that represents the overall HealthStatus. To format it in order to see the messages in json which is easy to understand. You can provide a custom ResponseWriter. A ready-made implementation is available in the AspNetCore.HealthChecks.UI.Client library, which returns a more detailed and structured health report.

Install the NuGet package:

Install-Package AspNetCore.HealthChecks.UI.Client

Update the call to MapHealthChecks to use the ResponseWriter as shown below.

app.MapHealthChecks(
    "/healthz",
    new HealthCheckOptions
    {
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });

Now the response from the health check endpoint looks like:

{
  "status": "Unhealthy",
  "totalDuration": "00:00:00.987769",
  "entries": {
    "npgsql": {
      "data": {},
      "duration": "00:00:00.2424424",
      "status": "Healthy",
      "tags": []
    },
    "rabbitmq": {
      "data": {},
      "duration": "00:00:00.21412",
      "status": "Healthy",
      "tags": []
    },
    "myapp-sql": {
      "data": {},
      "description": "Unable to connect to the DB.",
      "duration": "00:00:00.2424242",
      "exception": "Unable to connect to the DB.",
      "status": "Unhealthy",
      "tags": []
    }
  }
}

Health checks in .NET applications are an essential part of building reliable, production-ready systems. They provide visibility into the state of your application and its dependencies, enabling platforms like Kubernetes, load balancers, and monitoring tools to respond appropriately to failures. In this tutorial we covered everything related to building a health check system for your .NET app. Hope you liked it and if you have any ideas make sure to post them by using the below given comments section.

The post How to perform Health checks in ASP.NET Core appeared first on YogiHosting.

1. Console – displays logs on console window, helpful in local development phase of the app for monitoring and debugging in real time.
2. Debug – writes logs using the System.Diagnostics.Debug class. Used mainly in local development of the app.
3. EventSource – this provider writes to a cross-platform event source with the name Microsoft-Extensions-Logging.
4. Windows EventLog – this provider writes logs to Windows Event Log.

All the logs created by default logging providers are displayed in console window and Debug output window when debugging. When running the application using the dotnet run command from a terminal, logs are displayed directly in the command shell.

In Visual Studio create a new .NET app by selecting ASP.NET Core Web App (Model-View-Controller) template.

Open the appsettings.development.json to find the default configurations in json format given as follows:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  }
}

It configures logging at Information level for every category and at Warning level for Microsoft.AspNetCore category. We will see more about it later on. If you run the app we can see the logs displayed on the console, see below image:

Here dbug (stands for Debug) and info (stands for Information) on the left, these are the log levels. Log level indicates the severity of the logged event. The middle part where we have Microsoft.Hosting.Lifetime is the category and 14 inside the bracket ([14]) is the Event ID. Each log message can specify an Event ID through which filtering on the ID can be done.

In the app we use an ILogger<TCategoryName> object on the constructor of the controller to inject ILogger object from dependency injection (DI). The TCategoryName is the log category which is a string that is associated with each log. It is usually the name of the class where the logs are written. The log category is useful for identifying, sorting, and filtering log messages.

Update the HomeController code, to inject ILogger object through DI and then log at the Information level.

public class HomeController : Controller
{
    private readonly ILogger<HomeController> logger;

    public HomeController(ILogger<HomeController> log)
    {
        logger = log;
    }

    public IActionResult Index()
    {
        logger.LogInformation("Testing logging");
        return View();
    }
}

When the Index action executes, you will find the logs as shown below:

info: LoggingExample.Controllers.HomeController[0]
      Testing logging

Here LoggingExample.Controllers.HomeController is the category and 0 inside the bracket ([0]) is the Event ID. Each log message can specify an Event ID through which filtering on the ID can be done.

See the below image where the logs are displayed on the Console window and Debug output window.

In the below code logging for the exception is done in the catch block. Here 400 is an event ID which can be anything of your liking.

try
{
   ...

   throw new Exception("Test exception");
}
catch (Exception ex)
{
    logger.LogWarning(400, ex.Message, DateTime.UtcNow);
}

Configure logging on appsettings.json file

We can provide logging configuration on the appsettings.json file. Note that for production we write the configurations on appsettings.json and for development the configurations goes to appsettings.development.json file.

Open the appsettings.development.json to find the default configurations in json format given as follows:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  }
}

The JSON data is written as name/value pairs. It consists of a field name (in double quotes), followed by a colon, followed by its value. The first field is “Logging” which contains child field inside it. Here it is “LogLevel”. The LogLevel specifies the minimum level to log for the categories i.e. logging is enabled at the specified level and higher (more severe). More about LogLevel in the below section.

See the LogLevel field on the json as given above. It contains 2 category fields – “Default” and “Microsoft.AspNetCore” and their values are given as “Information” and “Warning”. So this means logging is done at “Information” level for the “Default” category (i.e. every category other that Microsoft.AspNetCore), and at “Warning” level for the “Microsoft.AspNetCore” category.

It should also be noted that in the above case no provider is specified so this applies to every providers.

When LogLevel is given under a provider name then the configurations apply for that provider only. These settings overrides the non-provider log settings. Consider the following.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    },
    "EventLog": {
      "LogLevel": {
        "Default": "Warning"
      }
    }
  }
}

Here Windows EventLog provider is added so the configurations overrides the non-provider log settings. This means now for EventLog, logging for Default category will be done at Warning level and for Microsoft.AspNetCore category at Warning level. Note that since EventLog does not have “Microsoft.AspNetCore” category so it is inherited from the non-provider log settings given above.

Log Level

The Log Levels indicate the severity of the log. The LogLevel specifies the minimum level to log for the categories i.e. logging is enabled at the specified level and higher (more severe). These 7 log levels from 0 to 6 are given below.

Trace – trace contain the most detailed messages which also contain sensitive data that should not be made available on the production. So by default it is disabled. In the development phase of the app, it can be very useful for debugging purposes.

Debug – it is popularly used for debugging and development. Use with caution in production due to the high volume of messages logged.

Information – used for logging general flow of the app.

Warning – used for abnormal or unexpected events that do not cause the app to fail.

Errors – used for unhandled errors and exception that cause the app failure.

Critical – used for severe errors that requires immediate attention. Examples – data loss or out of disk space.

None – specifies not to write messages.

The important methods to log messages are:

logger.Log(LogLevel.Information, 400, "Testing"); // specify the log level as the first parameter followed by the event Id then at the last the message

Logger.LogTrace("Testing"); // log at trace level

Logger.LogDebug("Testing"); // log at debug level

Logger.LogInformation("Testing"); // log information

logger.LogWarning(400, "Test exception at {DT}", DateTime.UtcNow); // log at warning level with event id 400

Logger.LogError("Testing"); // log at error level

Logger.LogCritical("Testing"); // log at critical level

Logging Providers

The Logging Providers help to write logs on a given medium. ASP.NET Core includes the following logging providers by default:

1. Console
2. Debug
3. EventSource
4. Windows EventLog

Third party providers are also available and can be installed from NuGet. Some examples includes AzureAppServicesFile, ApplicationInsights, etc.

Logging providers are configured on appsettings.json and appsettings.development.json depending upon the environment.

Open the appsettings.development.json file to find the default configuration containing non-provider section, see below.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  }
}

We can activate other providers by adding their configurations to this file.

Console

The Console provider only displays logs on the console. The logs are not persisted once the app stops. The console logs are useful when running an app locally for monitoring and debugging in real time. On the appsettings.development.json file we can add the console section to configure console logging provider. See below.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    },
    "Console": {
      "LogLevel": {
        "Default": "Warning",
        "Microsoft.AspNetCore.Mvc.Razor.Razor": "Debug",
        "Microsoft.AspNetCore.Mvc.Razor": "Error"
      }
    }
  }
}

The configuration specifies that for Console, every category is set as “Warning” except for categories Microsoft.AspNetCore.Mvc.Razor.Razor at Debug level and Microsoft.AspNetCore.Mvc.Razor at Error level.

When a specific category is listed, the specific category overrides the default value.

If you now look to the console, very few logs are displayed. This is because Warning for Default category is quite high and lower level logs are not written on the console. These are given below.

dbug: Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine[1]
      => SpanId:f0377c2fb682381f, TraceId:debbe69a510e3a5336d79ce744429ac0, ParentId:0000000000000000 => ConnectionId:0HNJ9IFE8CN2C => RequestPath:/ RequestId:0HNJ9IFE8CN2C:00000001 => LoggingExample.Controllers.HomeController.Index (LoggingExample)
      View lookup cache miss for view 'Index' in controller 'Home'.
dbug: Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine[1]
      => SpanId:f0377c2fb682381f, TraceId:debbe69a510e3a5336d79ce744429ac0, ParentId:0000000000000000 => ConnectionId:0HNJ9IFE8CN2C => RequestPath:/ RequestId:0HNJ9IFE8CN2C:00000001 => LoggingExample.Controllers.HomeController.Index (LoggingExample)
      View lookup cache miss for view '_Layout' in controller 'Home'.

Change the “Default”: “Warning” to “Default”: “Trace” for the console provider. Now rerun the app and see on the console. You will find hundreds of log message. See below.

dbug: Microsoft.AspNetCore.Mvc.ModelBinding.ModelBinderFactory[12]
      Registered model binder providers, in the following order: Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BinderTypeModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.ServicesModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.HeaderModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.FloatingPointTypeModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.EnumTypeModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.DateTimeModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.SimpleTypeModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.TryParseModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.CancellationTokenModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.ByteArrayModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.FormFileModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.FormCollectionModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.KeyValuePairModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.DictionaryModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.ArrayModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.CollectionModelBinderProvider, Microsoft.AspNetCore.Mvc.ModelBinding.Binders.ComplexObjectModelBinderProvider
dbug: Microsoft.Extensions.Hosting.Internal.Host[1]
      Hosting starting
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[63]
      User profile is available. Using 'C:\Users\Avita\AppData\Local\ASP.NET\DataProtection-Keys' as key repository and Windows DPAPI to encrypt keys at rest.
dbug: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[37]
      Reading data from file 'C:\Users\Avita\AppData\Local\ASP.NET\DataProtection-Keys\key-e075d172-f7d1-4435-895d-14ee7bf094e4.xml'.
dbug: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[37]
      Reading data from file 'C:\Users\Avita\AppData\Local\ASP.NET\DataProtection-Keys\key-f24b36ed-9ad7-42f7-ba4e-18e72008ecf2.xml'.
dbug: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[18]
      Found key {e075d172-f7d1-4435-895d-14ee7bf094e4}.
dbug: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[18]
      Found key {f24b36ed-9ad7-42f7-ba4e-18e72008ecf2}.
dbug: Microsoft.AspNetCore.DataProtection.KeyManagement.DefaultKeyResolver[13]
      Considering key {e075d172-f7d1-4435-895d-14ee7bf094e4} with expiration date 2026-03-26 11:10:11Z as default key.
dbug: Microsoft.AspNetCore.DataProtection.TypeForwardingActivator[0]
      Forwarded activator type request from Microsoft.AspNetCore.DataProtection.XmlEncryption.DpapiXmlDecryptor, Microsoft.AspNetCore.DataProtection, Version=10.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60 to Microsoft.AspNetCore.DataProtection.XmlEncryption.DpapiXmlDecryptor, Microsoft.AspNetCore.DataProtection, Culture=neutral, PublicKeyToken=adb9793829ddae60
dbug: Microsoft.AspNetCore.DataProtection.XmlEncryption.DpapiXmlDecryptor[51]
      Decrypting secret element using Windows DPAPI.

Certainly this can be useful when you want to dig deep into the app’s debugging process.

Whatever logs we write from any logging methods (examples below), these are all displayed on the console. Provided the log level are equal or higher to that configured on the appsettings file.

logger.Log(LogLevel.Information,400, "Testing");
logger.LogTrace("TestT");
logger.LogDebug("TestD");
logger.LogInformation("Testing logging");

Debug

The Debug provider writes log output by using the System.Diagnostics.Debug class and only when the debugger is attached. We can use System.Diagnostics.Debug.WriteLine() method to write logs with Debug provider. On Linux, the Debug provider log location is one of the following:

/var/log/message
/var/log/syslog

This provider is useful for developers to monitor application flow, identify errors, and check variable states during development phase of the app.

In the below configuration we added the Debug provider block which sets the default category to Debug level.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    },
    "Console": {
      "LogLevel": {
        "Default": "Warning",
        "Microsoft.AspNetCore.Mvc.Razor.Razor": "Debug",
        "Microsoft.AspNetCore.Mvc.Razor": "Error"
      }
    },
    "Debug": { 
      "LogLevel": {
        "Default": "Information" 
      }
    }
  }
}

The Debug provider configuration will inherit the non-provider category “Microsoft.AspNetCore” value of Warning.

The below code line writes log message with the Debug provider.

System.Diagnostics.Debug.WriteLine("Test Debug Provider", "LoggingExample.Controllers.HomeController");

EventSource

The EventSource provider writes to a cross-platform event source with the name Microsoft-Extensions-Logging.

On the appsettings.development.json add the EventSource configurations as shown below.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information"

    },
    "Console": {
      "LogLevel": {
        "Default": "Warning",
        "Microsoft.AspNetCore.Mvc.Razor.Razor": "Debug",
        "Microsoft.AspNetCore.Mvc.Razor": "Error"
      }
    },
    "Debug": { 
      "LogLevel": {
        "Default": "Trace" 
      }
    },
    "EventSource": {
      "LogLevel": {
        "Default": "Information"
      }
    }
  }
}

This configuration tells ASP.NET Core to write logs that are at Information level with the EventSource provider. We can add more configurations to it but for the sake of simplicity I have kept things simple to understand.

To read the logs written by EventSource provider we use a tool called dotnet-trace. Lets understand how to use it.

dotnet-trace

To install dotnet-trace tool, open a terminal from the directory of the project file and run the below given command:

dotnet tool install --global dotnet-trace

Now to view the logs perform the following tasks:

1. Run the app from the CLI command – dotnet run. For this open a terminal from the directory of the project file (.csproj) and run the command from there.
2. Find the process identifier (PID) of the .NET app. For this you have to run dotnet-trace ps command. Note that the PID for the process has the same name as the app’s assembly.
3. Open another terminal and run the command – dotnet-trace collect -p %PID% –providers Microsoft-Extensions-Logging. Replace %PID% with the PID of the .NET app.

In the below image, I am running the dotnet run command from the terminal.

Once the app is running, from another terminal I run the dotnet-trace ps command which gave me the PID of the .NET app as 15348. I next run the dotnet-trace collect -p 15348 –providers Microsoft-Extensions-Logging command to see the logs. See the below image where I have shown this thing.

The full syntax of dotnet-trace command is given below.

dotnet-trace collect -p {PID} --providers Microsoft-Extensions-Logging:{KEYWORD}:{PROVIDER LEVEL}:FilterSpecs=\"{LOGGER CATEGORY 1}:{CATEGORY LEVEL 1}; {LOGGER CATEGORY 2}:{CATEGORY LEVEL 2}; ...
{LOGGER CATEGORY N}:{CATEGORY LEVEL N}\"

The following table defines the keyword {KEYWORD} placeholder.

The below table gives the provider levels.

FilterSpecs is used to filter the logs based on the categories and their log levels. Multiple FilterSpecs entries can be added with the ; semicolon character between them

Examples:

dotnet-trace collect -p %PID% --providers Microsoft-Extensions-Logging:4:2:FilterSpecs=\"Microsoft.AspNetCore.Hosting*:Debug\"
dotnet-trace collect -p %PID%  --providers Microsoft-Extensions-Logging:4:5:\"Microsoft.AspNetCore.Hosting*:Debug; Microsoft.AspNetCore.Mvc*:Information;\"

Windows EventLog

The Windows EventLog provider writes log to the Windows Event Log. Unlike the other providers, the EventLog provider doesn’t inherit the default non-provider settings. If EventLog log configurations aren’t specified, they default to LogLevel.Warning. In order to log lower than LogLevel.Warning, explicitly set them with the log level.

The following example sets the Event Log default log level to LogLevel.Information:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information"

    },
    "Console": {
      "LogLevel": {
        "Default": "Warning",
        "Microsoft.AspNetCore.Mvc.Razor.Razor": "Debug",
        "Microsoft.AspNetCore.Mvc.Razor": "Error"
      }
    },
    "Debug": {
      "LogLevel": {
        "Default": "Trace"
      }
    },
    "EventSource": {
      "LogLevel": {
        "Default": "Information"
      }
    },
    "EventLog": {
      "LogLevel": {
        "Microsoft": "Information"
      }
    }
  }
}

Open Event Viewer on your windows OS to view the logs saved there. Check the below image:

Note that when storing logs on Windows Event Log then makes sure only the high priority logs should be stored since it will unnecessary slow down the app and the server.

Third Party log providers

Various 3rd party providers writes logs to various mediums like Microsoft.Extensions.Logging.AzureAppServices writes logs to text files in an Azure App Service file system and blob storage.

Microsoft.Extensions.Logging.ApplicationInsights provider logs to Azure Application Insights.

Other popular log providers are NLog and Serilog.

HTTP logging in ASP.NET Core

We can also log incoming HTTP requests and HTTP responses to ASP.NET Core apps using HTTP logging. Some of the features of HTTP logging are:

• Log HTTP request, HTTP response, Headers and Body.
• Log only requests and responses that meet certain criteria and also select parts of request or response to log.
• For security redact sensitive information from the logs.

How to enable HTTP Logging in ASP.NET Core apps

In the Program class add the AddHttpLogging and UseHttpLogging methods as shown below.

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddControllersWithViews();

builder.Services.AddHttpLogging(o => { });

var app = builder.Build();

app.UseHttpLogging();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Home/Error");
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
    app.UseHsts();
}

...

Also add Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware to the appsettings.Development.json file at the “LogLevel”.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning",
      "Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware": "Trace"
    }
  }
}

The HTTP logs will now appear on the console as shown below.

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[1]
      Request:
      Protocol: HTTP/2
      Method: GET
      Scheme: https
      PathBase:
      Path: /
      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
      Host: localhost:7211
      User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36
      Accept-Encoding: gzip, deflate, br, zstd
      Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
      Cookie: [Redacted]
      Upgrade-Insecure-Requests: [Redacted]
      sec-ch-ua: [Redacted]
      sec-ch-ua-mobile: [Redacted]
      sec-ch-ua-platform: [Redacted]
      sec-fetch-site: [Redacted]
      sec-fetch-mode: [Redacted]
      sec-fetch-user: [Redacted]
      sec-fetch-dest: [Redacted]
      priority: [Redacted]

We can also configure logging through lambda expressions as shown below.

builder.Services.AddHttpLogging(logging =>
{
    logging.LoggingFields = HttpLoggingFields.All;
    logging.RequestHeaders.Add("sec-ch-ua");
    logging.ResponseHeaders.Add("MyResponseHeader");
    logging.MediaTypeOptions.AddText("application/javascript");
    logging.RequestBodyLogLimit = 4096;
    logging.ResponseBodyLogLimit = 4096;
    logging.CombineLogs = true;
});

Here HttpLoggingOptions.LoggingFields is an enum flag that configures specific parts of the request and response to log. RequestHeaders and ResponseHeaders are sets of HTTP headers that are logged. Header values are only logged for header names that are in these collections.

MediaTypeOptions provides configuration for selecting which encoding to use for a specific media type. Setting CombineLogs to true configures the middleware to consolidate all of its enabled logs for a request and response into one log at the end.

Http logging with redaction can be enabled by calling AddRedaction and AddHttpLoggingRedaction on the program class.

builder.Services.AddRedaction();
builder.Services.AddHttpLoggingRedaction(op => { });

Redaction helps you sanitize or mask sensitive information in logs, error messages, or other outputs. This keeps you compliant with privacy rules and protects sensitive data. It’s useful in apps that handle personal data, financial information, or other confidential data points.

In this tutorial we covered logging in ASP.NET Core from complete beginning till advanced levels. It will help you to debug various problems coming on .NET apps.

The post How to perform Logging in ASP.NET Core appeared first on YogiHosting.

First we will push our ASP.NET Core app to GitHub repository which will trigger the Workflow. The Workflow will basically do 3 tasks:

1. Build a Docker Image for the ASP.NET Core from the Dockerfile.
2. Push the Docker Image to Azure Container Registry (ACR).
3. Deploy the Azure Container Apps with the Docker Image in ACR.

This tutorial is a part of ASP.NET Core apps on Docker series.

• 1. Create first ASP.NET Core App in a Docker Container
• 2. Deploy a Docker based ASP.NET Core app to Azure
• 3. ASP.NET Core APP with HTTPS in Docker
• 4. Multi-Container ASP.NET Core App with Docker Compose
• 5. CRUD Operations in ASP.NET Core and SQL Server with Docker
• 6. Docker Volumes on ASP.NET Core App
• 7. Configuring Nginx as Reverse Proxy and Load Balancer for Dockerized ASP.NET Core apps
• 8. Deploy ASP.NET Core Dockerized app to Azure with GitHub Actions CI / CD

ASP.NET Core App

The .NET app is a Quiz based app which is shown below:

You can get this full app along with the GitHub Workflow from my GitHub Repository.

Create Azure Container Registry

First, we will create an Azure Container Registry which will store the docker image for this app. Later the GitHub Actions CI/CD will push the docker image to this repository. So create a new Azure Container Repository and call it QuizAppACR.

Note that we have used “Role assignment permissions mode” to “RBAC Registry Permissions”. We will create this permission next. Basically this permission is needed for pushing the Docker Image of the APP from GitHub CI / CD pipeline to ACR.

After the registry is created enable the Admin user from “Settings > Access keys”. Check the below screenshot.

Admin user is needed for using the Image for Container Apps when we create the app from Azure portal.

We now have to Create Credentials for Azure Authentication for GitHub Actions to push Docker Image of the app to Azure Container Registry. We have to run 3 commands on Azure CLI (given below). First login to azure using the Azure CLI command az login on the command prompt or terminal. Note that you have to install Azure CLI on your pc. Check the Download Link.

After login run the below 3 commands one by one on.

az group show --name <resource-group-name> --query id --output tsv

Change to your resource group name. In my case it is R1 so this below command I run, check the below image:

The command will give the resource ID.

/subscriptions/73435ad5-0765-7664-CV56-62hy02436a7d/resourceGroups/R1

We will use this resource id in the next command.

The next command is to create the service principal from the below command.

az ad sp create-for-rbac --scope $groupId --role Contributor --sdk-auth

Here change the $groupId with the resource Id we got in the previous step. In my case my command becomes:

az ad sp create-for-rbac --scope /subscriptions/73435ad5-0765-7664-CV56-62hy02436a7d/resourceGroups/R1 --role Contributor --sdk-auth

The command will give a JSON, save the JSON output because it’s used in a later step. Also, take note of the clientSecret and clientId, which you need to update the service principal in the next section.

{
  "clientId": "6334i778-5352-434c-a469-40332d7e63a9",
  "clientSecret": "HdY8Q~jj4EynmuGjgD5c7BCKMCI-U2XbUB2jqcm_",
  "subscriptionId": "73435ad5-0765-7664-CV56-62hy02436a7d",
  "tenantId": "404c4c76-767d-6s89-b7c9-ad800d5433ea",
  "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
  "resourceManagerEndpointUrl": "https://management.azure.com/",
  "activeDirectoryGraphResourceId": "https://graph.windows.net/",
  "sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
  "galleryEndpointUrl": "https://gallery.azure.com/",
  "managementEndpointUrl": "https://management.core.windows.net/"
}

Check the below image showing the output of this command.

We now get the ACR registry Id from the below command.

az acr show --name <registry-name> --resource-group <resource-group-name> --query id --output tsv

We change the <registry-name> and <resource-group-name> from our values. This is the updated command in our case.

az acr show --name QuizAppACR --resource-group R1 --query id --output tsv

Check the screenshot below.

Next, we update for registry authentication. We assign the AcrPush role, which gives push and pull access to the registry. The command is given below.

az role assignment create --assignee <ClientId> --scope $registryId --role AcrPush

Substitute the client ID of your service principal and registryId in the below command.

az role assignment create --assignee 6334i778-5343452-434c-a469-40332d7e63a9 --scope /subscriptions/74d23ad6-5454-4236-ab57-62b342dd2wtrrtd/resourceGroups/R1/providers/Microsoft.ContainerRegistry/registries/QuizAppACR --role AcrPush

We have now created our registry authentication, see the below image.

Create GitHub Actions Workflow

Create a GitHub Workflow file called dotnet.yml inside the .github\workflows folder of the app. Add the following code to this file:

# This workflow will build a .NET project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net
 
name: .NET
 
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]
 
jobs:
  build:
 
    runs-on: ubuntu-latest
 
    steps:
    - uses: actions/checkout@v4
    - name: Setup .NET
      uses: actions/setup-dotnet@v4
      with:
        dotnet-version: 10.0.x
    - name: Restore dependencies
      run: dotnet restore
    - name: Build
      run: dotnet build --no-restore
    - name: Test
      run: dotnet test --no-build --verbosity normal --logger trx --results-directory "TestResults"
    - name: Upload dotnet test results
      uses: actions/upload-artifact@v4
      with:
        name: dotnet-results-${{ matrix.dotnet-version }}
        path: TestResults
        # Use always() to always run this step to publish test results when there are test failures
      if: ${{ always() }}
     
    - name: Login via Azure CLI
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}
         
    - name: Build and push image'
      uses: azure/docker-login@v1
      with:
        login-server: ${{ secrets.REGISTRY_LOGIN_SERVER }}
        username: ${{ secrets.REGISTRY_USERNAME }}
        password: ${{ secrets.REGISTRY_PASSWORD }}
    - run: | 
        docker build -f Quiz/Dockerfile -t ${{ secrets.REGISTRY_LOGIN_SERVER }}/quizca:${{ github.sha }} .
        docker push ${{ secrets.REGISTRY_LOGIN_SERVER }}/quizca:${{ github.sha }}

The Workflow task includes setting a runner machine with .NET 10.0.

- name: Setup .NET
  uses: actions/setup-dotnet@v4
  with:
    dotnet-version: 10.0.x

Next, the tasks like running the project restore, building the project and running the test are done. These are done through dotnet commands – dotnet restore, dotnet build and dotnet test commands.

The next task performs the login to Azure through the JSON output from the service principal creation step. We will add this json to GitHub secret variable called “AZURE_CREDENTIALS”.

- name: Login via Azure CLI
  uses: azure/login@v1
  with:
    creds: ${{ secrets.AZURE_CREDENTIALS }}

The next task performs the building of the docker image and pushing the image to our Azure Container Registry. Here we used 3 GitHub secrets – REGISTRY_LOGIN_SERVER, REGISTRY_USERNAME and REGISTRY_PASSWORD.

The code is give below.

- name: Build and push image'
  uses: azure/docker-login@v1
  with:
    login-server: ${{ secrets.REGISTRY_LOGIN_SERVER }}
    username: ${{ secrets.REGISTRY_USERNAME }}
    password: ${{ secrets.REGISTRY_PASSWORD }}
- run: | 
    docker build -f Quiz/Dockerfile -t ${{ secrets.REGISTRY_LOGIN_SERVER }}/quizca:${{ github.sha }} .
    docker push ${{ secrets.REGISTRY_LOGIN_SERVER }}/quizca:${{ github.sha }}

Note the docker build command has -f flag where we have specified the location of Dockerfile. The image name will be quizappacr.azurecr.io/quizca followed by the GitHub sha as image tag.

Create GitHub Repository with Repository Secrets

We now create a GitHub repository with 4 GitHub secrets containing the values given below. Secrets can be added from Settings > Secrets and variables > Actions.

Its time to push the full code of our app to our GitHub repository. You can use git push command or GitHub Desktop to do this work. The Workflow execution will trigger automatically and it finishes successfully. See the below image of the workflow.

The Workflow builds the docker image in the runner virtual machine and pushes the image to the Azure Container Registry. Open the registry where you will find the image. Check the below image.

Create Azure Container Apps

We now create an Azure Container Apps that will use the Docker image from our Azure Container Registry. We named the app “quizapp” and selected “Azure Container Registry” for Image source. Then select the registry, Image and Image Tag which contains the docker image for our app.

For the Authentication type select “Managed identity” and for the “Managed identity” select System assigned Identity (environment). Check the below image where we have shown this thing.

Also check the Ingress check box and select the Ingress traffic value as “Accepting traffic from anywhere”.

The app is now ready and opening on the browser. See the below image:

You may be wondering that we have not yet applied the CI / CD pipeline on the app’s deployment. Although we have applied CI / CD on the docker image pushing from GitHub to ACR. Don’t worry this is the topic next.

Add GitHub Actions CI / CD for App Deployment to Azure

Add the below task on the Workflow file at the end. This will Deploy the app to Azure Container app with the image in ACR. The acrName is specified as the name of the ACR repository which is QuizAppACR, the containerAppName value is quizapp. Resource Group is given R1 for our case (choose your own resource group). Then on the imageToDeploy, we provide the latest image which is find out with the current github.sha code.

- name: Build and deploy Container App
  uses: azure/container-apps-deploy-action@v1
  with:
    acrName: QuizAppACR 
    containerAppName: quizapp
    resourceGroup: R1
    imageToDeploy: ${{ secrets.REGISTRY_LOGIN_SERVER }}/quizca:${{ github.sha }}

We also change the heading on the apps html to include “UPDATED” text. Push the new app file and the workflow file to GitHub once again. This will push the latest updated apps code. Which you can see in the below image.

So the confirms our CICD is working and it is now also covering the CI/CD pipeline for the app’s deployment to azure.

This is the full GitHub Workflow file code:

# This workflow will build a .NET project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net
 
name: .NET
 
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]
 
jobs:
  build:
 
    runs-on: ubuntu-latest
 
    steps:
    - uses: actions/checkout@v4
    - name: Setup .NET
      uses: actions/setup-dotnet@v4
      with:
        dotnet-version: 10.0.x
    - name: Restore dependencies
      run: dotnet restore
    - name: Build
      run: dotnet build --no-restore
    - name: Test
      run: dotnet test --no-build --verbosity normal --logger trx --results-directory "TestResults"
    - name: Upload dotnet test results
      uses: actions/upload-artifact@v4
      with:
        name: dotnet-results-${{ matrix.dotnet-version }}
        path: TestResults
        # Use always() to always run this step to publish test results when there are test failures
      if: ${{ always() }}
     
    - name: Login via Azure CLI
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}
         
    - name: Build and push image'
      uses: azure/docker-login@v1
      with:
        login-server: ${{ secrets.REGISTRY_LOGIN_SERVER }}
        username: ${{ secrets.REGISTRY_USERNAME }}
        password: ${{ secrets.REGISTRY_PASSWORD }}
    - run: | 
        docker build -f Quiz/Dockerfile -t ${{ secrets.REGISTRY_LOGIN_SERVER }}/quizca:${{ github.sha }} .
        docker push ${{ secrets.REGISTRY_LOGIN_SERVER }}/quizca:${{ github.sha }}
 
    - name: Build and deploy Container App
      uses: azure/container-apps-deploy-action@v1
      with:
        acrName: QuizAppACR 
        containerAppName: quizapp
        resourceGroup: R1
        imageToDeploy: ${{ secrets.REGISTRY_LOGIN_SERVER }}/quizca:${{ github.sha }}

In this tutorial we covered in full details how to create GitHub Action CI / CD pipeline for ASP.NET Core containerized app. The pipeline builds the docker image of the app in a virtual machine on GitHub Runner. Then pushes the docker image to ACR and deploys this ACR image to the app. I hope you liked this tutorial, if any questions then use the comments section below.

The post Deploy ASP.NET Core Dockerized app to Azure with GitHub Actions CI / CD appeared first on YogiHosting.

GitHub Action

GitHub Actions is a CI/CD platform which is integrated into your GitHub repository. This means you can run a CI/CD pipeline right from your GitHub repository. GitHub Actions are organized into Workflows, which are automated process that will run one or more jobs. A common example of a Workflow is to automatically build and deploy your app to Azure whenever the app is pushed to the GitHub repository.

The GitHub Actions kick off based on Events. Events can be anything like a push to the repository or a pull request, and so on.

Runners are the machines that execute jobs defined in a GitHub Actions Workflow. For example, a UBUNTU runner machine can clone your repository locally, install testing software, and then runs the tests.

In the below figure GitHub actions working is explained. An Event kicks the Runners that executes the different Jobs defined in the WorkFlows.

Workflows are defined by a YAML file checked in to the repository. Their location is .github/workflows directory located on the root of the repository.

The CICD process is explained in the below figure. We will push the app to the GitHub repository, this will trigger the Workflow. The Workflow will build the app and deploy the app’s published files to Azure App Services.

Example ASP.NET Core app

I have an ASP.NET Core Quiz app which asks the users 4 questions. The questions are shown inside a multi-page form. This is how this app looks.

You can download the full app along with the Workflow from my GitHub repository.

This app contains 2 projects:

1. ASP.NET Core MVC project – which contains the quiz.
2. Console project – which contains the unit tests.

The below image shows the full app with it’s 2 projects.

I will automate a CI/CD pipeline with GitHub Actions for this app and deploy this app to Azure App Services. Note that there are some unit tests which the Workflow will run on runner machines and will only deploy the app when all the tests pass. GitHub Actions will not deploy the app if any of the test fails.

Create an Azure App Services

Create a new App Service on the Azure account. Make sure to select the Publish to “Code” and Runtime stack to “.NET 10 (LTS)”. Check the below image.

After the app is created, I have to enable Basic Authentication for this app. This is a necessary step needed to Download publish profile. Go to Settings > Configuration inside the app section on Azure and click the “General settings” tab. Here select the option SCM Basic Auth Publishing Credentials and click the apply button. See the below image.

Now go to the app overview page and here you will see “Download publish profile” button at the top. Click it to download this file. It is an XML file containing authentication values which GitHub will use to authenticate itself when deploying the app to azure.

This is all we have to do on Azure, we now move to GitHub.

GitHub Repository and Workflow file

We first have to create a .GitHub folder on the root of the app. Inside this folder create another folder called workflows. Then inside this folder create a new file called dotnet.yml. The file name could be anything but should have an extension .yml or .yaml.

In the below image I have shown the workflows directory.

Let’s now add the Workflow codes to the dotnet.yml file. The full code of this file is given below.

# This workflow will build a .NET app and deploys it to Azure

name: .NET APP

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

env:
  AZURE_WEBAPP_NAME: "QUIZ"

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4
    - name: Setup .NET
      uses: actions/setup-dotnet@v4
      with:
        dotnet-version: 10.0.x
    - name: Restore dependencies
      run: dotnet restore
    - name: Build
      run: dotnet build --no-restore
    - name: Test
      run: dotnet test --no-build --verbosity normal --logger trx --results-directory "TestResults"
    - name: Upload dotnet test results
      uses: actions/upload-artifact@v4
      with:
        name: dotnet-results-${{ matrix.dotnet-version }}
        path: TestResults
        # Use always() to always run this step to publish test results when there are test failures
      if: ${{ always() }}
    - name: Publish
      run: dotnet publish ./Quiz/Quiz.csproj --configuration Release --output ./publish
    - name: Deploy
      uses: azure/webapps-deploy@v3
      with: 
        app-name: ${{ env.AZURE_WEBAPP_NAME }} 
        publish-profile: ${{ secrets.PUBLISH_SECRET }}
        package: ./publish

Lets go through it part by part so that it becomes easy for you to understand it.

# This workflow will build a .NET app and deploys it to Azure

name: .NET APP

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

In the above code we named the workflow name as “.NET APP”. Name can be anything of your choice. Then with on, I define the events which will trigger the workflow. The workflow will run on 2 events which are:

1. when a push is made to any main branch.
2. when a pull is made to any main branch.

Next, I define the environment variable using the “env” key. The environment variable is “AZURE_WEBAPP_NAME” and it’s value is given as “QUIZ”. We can access this variable using $env.AZURE_WEBAPP_NAME, this will be see later on.

env:
  AZURE_WEBAPP_NAME: "QUIZ"

A job is a set of steps in a workflow that is executed on the same runner machine. Jobs are defined by “jobs” key. I named this job as “build”, this name can be anything. Next, I define the virtual machine type which will run this job. This is the runner machine which in our case is Ubuntu. Note that the virtual machines are build new every single time.

jobs:
  build:

    runs-on: ubuntu-latest

A job contains a sequence of tasks called steps. Steps can run commands, setup tasks, or perform some action in your repository. I add the below code:

steps:
- uses: actions/checkout@v4
- name: Setup .NET
  uses: actions/setup-dotnet@v4
  with:
    dotnet-version: 10.0.x
- name: Restore dependencies
  run: dotnet restore
- name: Build
  run: dotnet build --no-restore

Let’s understand what this code does. The first task, which is actions/checkout@v4, says to downloads the code from your repository into the virtual machine (runner) where the workflow is executing.

Next, the second task is defined and given the name “Setup .NET” – name: Setup .NET. Then with the uses key, I define which action to run. This action is to setup dotnet on the runner – uses: actions/setup-dotnet@v4. I also defined the version of dotnet to setup using the with key – dotnet-version: 10.0.x.

The third task restores the dependencies of the ASP.NET Core app. It runs the dotnet restore CLI command that uses NuGet to download and install all the dependencies (packages and tools) required by the app.

The fourth task build the app with dotnet build command. The use of –no-restore flag tells to build a .NET project or solution without automatically performing a NuGet package restore operation first.

We now have 2 tasks that performs the testing of the app and upload the test result on a directory. These tasks are given below.

- name: Test
  run: dotnet test --no-build --verbosity normal --logger trx --results-directory "TestResults"
- name: Upload dotnet test results
  uses: actions/upload-artifact@v4
  with:
    name: dotnet-results-${{ matrix.dotnet-version }}
    path: TestResults
    # Use always() to always run this step to publish test results when there are test failures
  if: ${{ always() }}

The task named Test runs the tests but does not builds the project – dotnet test –no-build, this will make the process fast. It also sets the output level to normal with –verbosity normal. The –logger trx will generate a Visual Studio Test Results file (.trx). This XML-based file contains the test execution results. The .trx file will be created inside the “TestResults” directory, see the parameter – –results-directory “TestResults”. This task’s full code is:

- name: Test
  run: dotnet test --no-build --verbosity normal --logger trx --results-directory "TestResults"

The task named Upload dotnet test results uploads “TestResults” directory (artifacts) from the runner machine making them available to download after the workflow execution is completed. Through this you can download the test results on our pc to debug why a particular test has failed. The expression if: ${{ always() }} is used to ensure that a job or step runs regardless of whether previous steps or jobs in the workflow succeeded, failed, or were canceled. So the upload of the test artifact will always run.

- name: Upload dotnet test results
  uses: actions/upload-artifact@v4
  with:
    name: dotnet-results-${{ matrix.dotnet-version }}
    path: TestResults
    # Use always() to always run this step to publish test results when there are test failures
  if: ${{ always() }}

The next task publishes the app using dotnet publish command. The configuration is set to Release and output directory is given as “publish”.

- name: Publish
   run: dotnet publish ./Quiz/Quiz.csproj --configuration Release --output ./publish

The final task is to deploy the app to Azure App service. Here I set the app-name from the environment variable ${{ env.AZURE_WEBAPP_NAME }}. I also set the publish-profile to ${{ secrets.PUBLISH_SECRET }, this will give GitHub my Azure App Services credentials so that it can deploy the published codes on “publish” folder to Azure. See the next section where I will configure this thing. We also defined the package location of the app as package: ./publish. The code for this task is given below.

- name: Deploy
  uses: azure/webapps-deploy@v3
  with: 
    app-name: ${{ env.AZURE_WEBAPP_NAME }} 
    publish-profile: ${{ secrets.PUBLISH_SECRET }}
    package: ./publish

GitHub Secret containing Azure Publish Profile

Recall I earlier downloaded the publish profile from Azure. It contains an xml file containing the Azure App Services credentials. I will use this file so that GitHub can use these credentials to deploy the app. So on the GitHub repository, go to “Setting”. Then under Secrets and variables, click on “Actions” and create a new secret. Give the secret any name (I named it “Publish_Secret”) and paste the contents of publish profile file on the “Secret” text box. I have shown this on the below image.

With the secret in place the workflow deploy task will use it through the code line – publish-profile: ${{ secrets.PUBLISH_SECRET }}.

Push the code to GitHub Repository

Let’s push the code of my app to my GitHub repository. Run the following git commands one by one on Command Prompt window.

// initialize git

git init
git add .
git commit -m "FirstCommit"

// push the app to GitHub repository

git remote add origin https://github.com/yogyogi/Quiz.git
git branch -M main
git push -u origin main

After the push is completed, check the Actions tab to see the workflow automatically runs. When all the tasks finishes successfully (including the tests), the app is deployed to azure. See the below image.

My Workflow executed successfully. Now let’s can open the app’s url (which we get in Azure App Service). The app is now opening on the browser. See below image.

We can also download the artifact containing the test results from the Workflow. The download link will be provided. Check the below image.

We now added a new test to our app and this test fails. Next, I push the changed to the repository. Again see the workflow whose test task fails since the dotnet test cli command has failed. In this case the new changes to the .NET app are not deployed to Azure by the CI/CD pipeline. See the below image.

In this tutorial we learned how to create GitHub Actions CI/CD pipeline to deploy an ASP.NET Core app to Azure App Services. We also build the complete Workflow file and understood all the tasks it runs like creating runner, testing, building, publishing and deploying. I hope you liked this tutorial. If you face any difficult in creating your CICD then message me from the below comments section.

The post Deploy ASP.NET Core app to Azure with GitHub Actions CI/CD appeared first on YogiHosting.