Software And Architecture

Blazor component lifecycle events

2023-08-29T21:09:00.009+05:30

Lifecycle of Blazor Components

Blazor Component Lifecycle Events

Blazor components have a lifecycle that defines the various stages a component goes through from its creation to its disposal. Understanding the component lifecycle is crucial for managing state, optimizing performance, and responding to events. Here's an overview of the Blazor component lifecycle events along with sample code to illustrate each stage:

Understanding OnInitialized, OnInitializedAsync

Initialization: This is the initial stage where the component's parameters and dependencies are set. It happens before rendering.

@code {
    [Parameter] public string Message { get; set; }

    protected override void OnInitialized()
    {
        // Initialization logic
    }
}

<h3>@Message</h3>

Initialize a database connection
Load data from a server
Set up bindings
Create timers or other asynchronous tasks

Parameter Set: This stage occurs when component parameters are set. It can be useful for reacting to changes in parameters.

@code {
    [Parameter] public string Message { get; set; }

    protected override void OnParametersSet()
    {
        // Parameter set logic
    }
}

<h3>@Message</h3>

Update the component's title
Update the data it is displaying
Change its behavior based on the new parameters

Render: This is where the component's rendering logic occurs. It happens after parameters are set.

@code {
    [Parameter] public string Message { get; set; }

    protected override void BuildRenderTree(RenderTreeBuilder builder)
    {
        // Render logic
    }
}

<h3>@Message</h3>

OnAfterRender/OnAfterRenderAsync: These events occur after the component has been rendered and added to the UI. They are useful for interacting with the rendered DOM elements.

@code {
    [Parameter] public string Message { get; set; }

    protected override void OnAfterRender(bool firstRender)
    {
        if (firstRender)
        {
            // Logic to execute after first render
        }
    }
}

<h3>@Message</h3>

Update the component's state in response to user input
Make asynchronous calls
Handle errors

Update: This stage happens when the component needs to update due to changes in parameters or state.

@code {
    private int count = 0;

    private void IncrementCount()
    {
        count++;
    }

    protected override bool ShouldRender()
    {
        // Custom logic to determine whether to render
        return true;
    }
}

<button @onclick="IncrementCount">Increment</button>
<p>Count: @count</p>

Dispose: This is the final stage when the component is removed from the UI. It's useful for cleaning up resources.

@implements IDisposable

@code {
    // Component logic

    public void Dispose()
    {
        // Cleanup logic
    }
}

Blazor lifecycle methods, such as OnInitializedAsync, are crucial for managing state and optimizing performance in Blazor applications.

Remember that Blazor also has asynchronous versions of some lifecycle events, such as OnInitializedAsync, OnParametersSetAsync, and OnAfterRenderAsync. These can be used when you need to perform asynchronous operations during those stages.

It's important to note that you might not always need to use all of these lifecycle events for every component. Choose the ones that fit your needs and make your code efficient and maintainable.

Async implementation in Blazor

2023-08-28T19:17:00.000+05:30

Step-by-Step Guide to Achieving Async Flows in Blazor:

1. Understanding Asynchronous Programming:

Before delving into Blazor-specific async flows, it's crucial to understand asynchronous programming concepts like async and await. Asynchronous operations help improve the responsiveness of your UI by not blocking the main thread.

2. Blazor Component Lifecycle:

Blazor components have their lifecycle methods. The OnInitializedAsync, OnParametersSetAsync, and OnAfterRenderAsync methods allow you to implement asynchronous operations during various stages of a component's lifecycle.

3. Asynchronous API Calls:

Performing asynchronous API calls is a common scenario in web applications. You can use HttpClient to make HTTP requests asynchronously. For example, fetching data from a remote server:

@page "/fetchdata"

@inject HttpClient Http

@if (forecasts == null)
{
    <p><em>Loading...</em></p>
}
else
{
    <table>
        <!-- Display data here -->
    </table>
}

@code {
    private WeatherForecast[] forecasts;

    protected override async Task OnInitializedAsync()
    {
        forecasts = await Http.GetFromJsonAsync<WeatherForecast[]>("sample-data/weather.json");
    }
}

4. Using async and await in Event Handlers:

You can use async and await within event handlers, allowing you to perform asynchronous tasks in response to user actions.

<button @onclick="FetchData">Fetch Data</button>

<p>@data</p>

@code {
    private string data;

    private async Task FetchData()
    {
        data = "Fetching...";
        // Simulate async work
        await Task.Delay(1000);
        data = "Data fetched!";
    }
}

5. Handling Multiple Asynchronous Operations:

In some cases, you might need to await multiple asynchronous operations and proceed when all are completed. You can use Task.WhenAll for this purpose.

private async Task HandleMultipleAsyncOperations()
{
    var task1 = SomeAsyncMethod1();
    var task2 = SomeAsyncMethod2();

    await Task.WhenAll(task1, task2);

    // Both tasks completed
}

6. Error Handling:

When dealing with asynchronous operations, you need to handle exceptions properly. Use try-catch blocks to catch and handle exceptions that might occur during asynchronous tasks.

private async Task FetchData()
{
    try
    {
        // Async operation that might throw an exception
        data = await FetchDataFromServer();
    }
    catch (Exception ex)
    {
        // Handle the exception
        data = "Error fetching data: " + ex.Message;
    }
}

Architecture of WebAssembly

2023-08-27T16:12:00.003+05:30

WebAssembly Architecture and Browser Integration

WebAssembly (Wasm) is a revolutionary technology that enables efficient execution of code in web browsers at near-native speeds. Its architecture involves a combination of components within the browser that work together to compile, load, and execute WebAssembly modules. In this detailed article, we will delve into the architecture of WebAssembly and explore how it integrates with browser components to enable seamless execution of Wasm modules.

WebAssembly Architecture Overview

Source Code: Developers write code in high-level programming languages like C, C++, Rust, or even languages like Python through transpilers. This code is then compiled to WebAssembly binary format (.wasm files).
WebAssembly Binary Format: The WebAssembly binary format is a low-level, compact bytecode representation of the source code. It is designed for efficient and secure execution in web browsers. This binary format is portable and can be delivered over the internet.
Browser Components:
- Parser: When a Wasm module is fetched, the browser's parser interprets the binary data and converts it into an Abstract Syntax Tree (AST). This AST represents the structure and semantics of the WebAssembly code.
- Validator: The validator checks the validity and safety of the Wasm module. It ensures that the module adheres to the WebAssembly specification, preventing malicious or erroneous code from running.
- Compiler: Once validated, the compiler translates the WebAssembly code into machine code that is suitable for the target architecture. This machine code is optimized for efficient execution.
- Memory Management: WebAssembly operates in a linear memory model. Browsers allocate a memory buffer for the module to use. The module interacts with this memory using load and store instructions.
Execution Engine:
- JIT Compilation: Just-In-Time (JIT) compilation involves compiling the WebAssembly module into native machine code at runtime, just before execution. This compiled code is then executed by the browser's JavaScript engine.
- Ahead-of-Time Compilation (AOT): Some browsers offer the option to perform Ahead-of-Time compilation, where the Wasm module is compiled to native machine code ahead of time, reducing the runtime overhead.
Integration with JavaScript: WebAssembly can interact with JavaScript code through a mechanism called the WebAssembly JavaScript API. This API allows JavaScript and WebAssembly to call each other's functions, share data, and collaborate seamlessly.

Browser Components for Running WebAssembly

Parser and Validator:

The browser's parser reads the fetched WebAssembly binary and converts it into an AST.
The validator checks the module's structure, types, and control flow, ensuring it adheres to the WebAssembly specification.
Invalid or malicious code is rejected during validation to maintain security.

Compiler:

The compiler translates the validated WebAssembly code into machine code.
Browsers may use various compilation strategies, including JIT compilation or AOT compilation, to optimize execution speed.

Memory Management:

WebAssembly operates with a linear memory model, which is a contiguous block of memory accessible by the module.
The browser allocates memory for the WebAssembly module, and the module interacts with this memory using load and store instructions.
Typed arrays in JavaScript can be used to share data between JavaScript and WebAssembly.

Execution Engine:

The JIT compiler translates the WebAssembly module into native machine code, optimizing it for the target architecture.
The JIT-compiled code is executed by the browser's JavaScript engine, which manages memory, execution flow, and interactions with other browser components.

WebAssembly JavaScript API:

The WebAssembly JavaScript API allows JavaScript and WebAssembly code to communicate and collaborate.
JavaScript can call WebAssembly functions and vice versa, enabling seamless integration between the two.
This API also facilitates data sharing, allowing efficient exchange of data between JavaScript and WebAssembly.

Conclusion

WebAssembly's architecture involves a sequence of steps, from source code to execution, that work together to enable efficient and secure execution of code within web browsers. Browser components such as parsers, validators, compilers, memory managers, execution engines, and the WebAssembly JavaScript API collaborate to ensure that WebAssembly modules are safely and optimally executed. This integration of technologies empowers developers to build high-performance web applications that leverage the strengths of WebAssembly. As the technology continues to evolve, WebAssembly's impact on web development is expected to be significant and far-reaching.

WebAssembly vs Java Applets

2023-08-27T16:07:00.002+05:30

A Detailed Comparison: WebAssembly vs. Java Applets

WebAssembly and Java Applets are technologies aimed at enhancing web applications by enabling execution of code directly in the browser. However, they have significant differences in terms of architecture, performance, security, and modern web development practices. Let's compare these two technologies in-depth, along with scenarios and code samples.

WebAssembly

Architecture:

WebAssembly is a binary instruction format that allows high-level languages to be compiled into a low-level bytecode that can run efficiently in browsers.
It provides a secure sandboxed environment where code can execute without direct access to browser internals.
WebAssembly modules can be integrated into web applications, enabling performance-critical components to run faster than JavaScript.

Performance:

WebAssembly code can achieve near-native performance, making it suitable for applications requiring complex computations or real-time interactions.
Execution is highly optimized due to its low-level nature and efficient bytecode representation.

Security:

WebAssembly's sandboxed execution environment enhances security by isolating code from the host environment.
It limits direct access to the DOM and browser APIs, reducing the attack surface for potential vulnerabilities.

Modern Web Development:

WebAssembly aligns well with modern web development practices, where applications are often built using diverse technologies and languages.
Developers can use languages like C++, Rust, and C# and compile them to WebAssembly, offering a broader range of language choices.

Sample Code:

// WebAssembly C code example
#include <stdio.h>

int main() {
    printf("Hello, WebAssembly!\n");
    return 0;
}

Java Applets

Architecture:

Java Applets are small Java programs that run inside a browser using a Java Virtual Machine (JVM).
They were historically used for adding interactive elements to web pages, such as animations and mini-games.

Performance:

Java Applets can suffer from performance issues due to the overhead of running a Java Virtual Machine within the browser.
JVM startup and memory consumption could be relatively high, impacting the user experience.

Security:

Java Applets faced security challenges, leading to vulnerabilities. They could access system resources and pose security risks.
Modern browsers have largely deprecated support for Java Applets due to security concerns.

Modern Web Development:

Java Applets are not aligned with modern web development practices, as they require using Java, which is less common for web application development.

Sample Code:

// Java Applet example (deprecated)
import java.applet.Applet;
import java.awt.Graphics;

public class HelloWorldApplet extends Applet {
    public void paint(Graphics g) {
        g.drawString("Hello, Java Applet!", 50, 25);
    }
}

Scenarios and Comparison:

Performance-Critical Tasks:
- WebAssembly is better suited for performance-critical tasks due to its low-level compilation and near-native execution speed.
- Java Applets could suffer from overhead due to the JVM and might not provide the same level of performance as WebAssembly.
Security and Compatibility:
- WebAssembly's sandboxed environment enhances security by isolating code execution.
- Java Applets historically faced security issues, and modern browsers have largely deprecated support for them due to these concerns.
Language and Ecosystem:
- WebAssembly allows developers to use various programming languages and leverage existing ecosystems, making it more versatile.
- Java Applets are limited to Java, which might not align well with the diverse set of languages used in modern web development.
Integration with Modern Web Standards:
- WebAssembly aligns with modern web development practices, as it integrates well with HTML, CSS, and JavaScript.
- Java Applets do not integrate as seamlessly with these modern standards.
User Experience:
- WebAssembly can provide a smoother user experience with its potential for better performance.
- Java Applets could lead to slower page loads and less responsiveness due to JVM startup time.

In summary, while both WebAssembly and Java Applets aim to enhance web applications with executable code in the browser, WebAssembly has emerged as a more viable option due to its performance, security improvements, and alignment with modern web development practices. Java Applets, on the other hand, have largely fallen out of favor due to security concerns and their limitations in the modern web landscape.

Overview of WebAssembly

2023-08-27T16:02:00.006+05:30

WebAssembly: Revolutionizing Web Application Development

WebAssembly (Wasm) is a groundbreaking technology that enables developers to run high-performance code in web browsers at near-native speeds. It has the potential to significantly change the way web applications are built, offering new levels of performance, security, and cross-platform compatibility. In this article, we'll explore what WebAssembly is, how it affects web app development, and provide a detailed example of building an e-commerce application using C# and .NET Core.

What is WebAssembly?

WebAssembly is a binary instruction format designed as a portable compilation target for high-level programming languages like C, C++, Rust, and others. It aims to provide efficient and secure execution of code on web browsers by allowing developers to compile their applications to a bytecode format that can be executed in a sandboxed environment. This bytecode can be executed at near-native speeds, providing performance improvements over traditional JavaScript-based web applications.

How WebAssembly Affects Web App Development

1. Performance Boost

WebAssembly allows developers to write code that executes faster than equivalent JavaScript code. This speed improvement is crucial for applications that require heavy computations or real-time interactivity, such as games or complex simulations.

2. Language Diversity

With WebAssembly, developers are not limited to using just JavaScript for web development. They can choose languages they are already familiar with, such as C#, C++, or Rust, and compile their code to WebAssembly, opening up opportunities for code reuse and cross-platform development.

3. Code Security

WebAssembly code runs in a sandboxed environment, which enhances security by preventing direct access to browser internals. This isolation helps in mitigating security vulnerabilities and providing a safer user experience.

4. Cross-Platform Compatibility

WebAssembly is designed to work across different browsers and platforms. This compatibility ensures that applications built with WebAssembly will run consistently regardless of the user's chosen browser.

5. Enhanced Web App Capabilities

WebAssembly enables web applications to handle complex tasks and computations that were previously impractical in the browser. This opens doors for building advanced applications like video and image editing tools directly in the browser.

Building an E-commerce Application with C# and .NET Core using WebAssembly

Let's walk through an example of building a simple e-commerce application using C# and .NET Core with WebAssembly.

Setup: Create a new Blazor WebAssembly project using the .NET CLI.

dotnet new blazorwasm -n ECommerceApp
cd ECommerceApp

Define Models: Create C# classes to define the data models for your e-commerce application, such as Product and Cart.

public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    public decimal Price { get; set; }
}

public class CartItem
{
    public Product Product { get; set; }
    public int Quantity { get; set; }
}

Create Components: Build Blazor components to display products and manage the shopping cart.

@page "/"

<h3>Products</h3>
@foreach (var product in products)
{
    <div>
        <h4>@product.Name</h4>
        <p>@product.Price</p>
        <button @onclick="() => AddToCart(product)">Add to Cart</button>
    </div>
}

<h3>Shopping Cart</h3>
@foreach (var item in cart)
{
    <div>
        <h4>@item.Product.Name</h4>
        <p>Quantity: @item.Quantity</p>
    </div>
}

@code {
    List<Product> products = new List<Product>
    {
        new Product { Id = 1, Name = "Product 1", Price = 10.99m },
        new Product { Id = 2, Name = "Product 2", Price = 19.99m }
    };

    List<CartItem> cart = new List<CartItem>();

    void AddToCart(Product product)
    {
        var existingItem = cart.FirstOrDefault(item => item.Product.Id == product.Id);
        if (existingItem != null)
        {
            existingItem.Quantity++;
        }
        else
        {
            cart.Add(new CartItem { Product = product, Quantity = 1 });
        }
    }
}

Run the App: Build and run the application to see your e-commerce app in action.

dotnet build
dotnet run

Access the app in your browser, and you'll be able to view products and add them to the shopping cart.

Conclusion

WebAssembly is a game-changer for web application development. It introduces performance improvements, language diversity, and enhanced capabilities to web apps. The example of building an e-commerce application with C# and .NET Core using WebAssembly showcases how developers can leverage this technology to create efficient and feature-rich web applications. As WebAssembly continues to evolve and gain wider adoption, we can expect it to reshape the landscape of web development.

Introduction to Blazor

2023-08-27T15:43:00.001+05:30

Introduction to Blazor with C#

Blazor is a modern web framework developed by Microsoft that allows developers to

Build interactive web applications using C# and .NET instead of traditional web technologies like JavaScript.
Blazor enables you to create web applications using the same language and tools you use to build other .NET applications, such as desktop and mobile applications.
It provides a way to write client-side code that runs directly in the browser while leveraging the power of C# and the .NET runtime.

How Blazor Works

Blazor offers two hosting models: Blazor WebAssembly and Blazor Server.

Blazor WebAssembly

In the WebAssembly hosting model, the Blazor application is compiled into WebAssembly bytecode, which is a binary instruction format designed for safe and efficient execution in web browsers. This bytecode is downloaded by the browser and executed directly in a secure sandbox environment. The application communicates with the server only during initial download and for fetching data, making it possible to build fully client-side web applications.

Blazor Server

In the Blazor Server hosting model, the application's UI components and logic run on the server, while a lightweight JavaScript client handles user interactions and UI updates. Communication between the client and the server is achieved through SignalR, a real-time communication library. This model allows developers to build responsive and dynamic web applications while maintaining a strong connection to the server.

Advantages of Blazor

Single Language: Developers can use C# for both client-side and server-side development, which reduces the need to switch between different programming languages.
Code Sharing: Blazor allows code sharing between client and server components, promoting code reuse and consistent behavior.
Rich Ecosystem: Blazor benefits from the existing .NET ecosystem, including libraries, tools, and developer expertise.
Type Safety: Blazor leverages the static typing of C# to catch errors at compile time, reducing runtime issues.
Debugging: Developers can use familiar debugging tools to debug both client-side and server-side code in one go.
Performance: Blazor applications are optimized for performance, and WebAssembly execution can be highly efficient.

The Future of Blazor

Blazor has gained significant attention and adoption in the developer community due to its unique approach to web development. Microsoft continues to invest in Blazor, adding features, improving performance, and expanding its capabilities. As a result, the future of Blazor looks promising, and it is likely to become an even more viable option for building a wide range of web applications.

Hello World Example in Blazor

Here's a simple "Hello World" example using Blazor WebAssembly:

Create a new Blazor WebAssembly project using the .NET CLI:

dotnet new blazorwasm -n HelloWorldBlazor
cd HelloWorldBlazor

Open the Pages/Index.razor file and replace its contents with:

@page "/"
<h3>Hello Blazor World!</h3>
<p>This is a simple Blazor WebAssembly application.</p>

Build and run the application:

dotnet build
dotnet run

The application will start, and you can access it by navigating to the provided URL in your web browser. You should see the "Hello Blazor World!" message displayed on the page.

Conclusion

Blazor is a revolutionary web framework that enables developers to build web applications using C# and .NET technologies. With its various hosting models, code sharing capabilities, and performance optimizations, Blazor offers a compelling alternative to traditional JavaScript-based web development. As Microsoft continues to enhance and develop Blazor, it is poised to play a significant role in the future of web development.

Mitigation of SQL Injection attack in EF core

2023-08-25T10:43:00.002+05:30

Mitigating SQL Injection Attacks with Entity Framework Core

Introduction:

SQL injection is a serious security vulnerability that occurs when an attacker manipulates input data to execute unauthorized SQL queries. Entity Framework Core (EF Core) is an Object-Relational Mapping (ORM) framework that provides built-in safeguards against SQL injection attacks. This article explores how EF Core helps prevent SQL injection, discusses common attack vectors, and provides code samples to illustrate the concepts.

Understanding SQL Injection:

SQL injection occurs when untrusted user input is directly concatenated into SQL queries. Attackers exploit this vulnerability by injecting malicious SQL code, leading to data breaches, unauthorized access, and more.

How EF Core Helps Prevent SQL Injection:

Parameterized Queries: EF Core automatically generates parameterized queries. Instead of concatenating values directly into SQL statements, it binds input values as parameters. This prevents attackers from injecting malicious code.
LINQ-to-Entities: EF Core encourages the use of LINQ queries, where queries are expressed using language constructs. These queries are automatically translated into parameterized SQL queries by EF Core, further reducing the risk of injection.

Common Attack Vectors and Mitigation:

Simple SQL Injection:

Attack Vector:

string query = "SELECT * FROM Users WHERE Username = '" + userInput + "'";

Mitigation (EF Core Approach):

var user = await context.Users.FirstOrDefaultAsync(u => u.Username == userInput);

Union-Based Injection:

Attack Vector:

string query = "SELECT * FROM Products WHERE Id = " + userInput + " UNION ALL SELECT NULL, NULL, @@version--";

Mitigation (EF Core Approach):

var productId = Int32.Parse(userInput);
var products = await context.Products.FromSqlInterpolated($"SELECT * FROM Products WHERE Id = {productId}");

Time-Based Blind Injection:

Attack Vector:

string query = "SELECT * FROM Orders WHERE Status = 'Shipped' AND 1=(SELECT COUNT(*) FROM sysobjects WHERE name = 'tablename' AND user_name(1)=username)";

Mitigation (EF Core Approach):

var status = "Shipped";
var orders = await context.Orders.Where(o => o.Status == status).ToListAsync();

Sample Code

Using EF Core to Prevent Injection: Assuming a simple User entity and a DbContext, here's how you can use EF Core to prevent SQL injection:

public class User
{
    public int Id { get; set; }
    public string Username { get; set; }
    // Other properties...
}

public class AppDbContext : DbContext
{
    public DbSet<User> Users { get; set; }
    // Other DbSet properties...

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.UseSqlServer("your_connection_string_here");
    }
}

Conclusion:

Entity Framework Core provides strong safeguards against SQL injection attacks by utilizing parameterized queries and LINQ-to-Entities. Developers can rely on these features to mitigate the risks associated with direct input concatenation. By adopting EF Core best practices, developers can build more secure applications that protect against SQL injection vulnerabilities.

Securing application from vulnerable inputs through middleware in dotnet core

2023-08-25T10:33:00.006+05:30

Securing Applications with ASP.NET Core OWIN Middleware Against Payload Attacks

Introduction:

Cybersecurity is a crucial aspect of modern application development. One area of concern is preventing attacks that exploit vulnerabilities in request payloads. This article explores how to use ASP.NET Core OWIN middleware to enhance the security of your applications by addressing payload-based attacks.

Vulnerability:

Payload Attacks Payload attacks involve sending malicious data in the request payload to exploit vulnerabilities. These attacks can lead to various security breaches, including data exposure, injection attacks, and remote code execution.

Using OWIN Middleware:

ASP.NET Core provides the OWIN (Open Web Interface for .NET) middleware pipeline, which can be customized to intercept and process requests before they reach the application's core logic. This provides an opportunity to implement security measures.

Sample Code

Implementing Payload Inspection Middleware: Here's an example of implementing a simple OWIN middleware that inspects incoming request payloads for potential malicious content:

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using System.IO;
using System.Text;
using System.Threading.Tasks;

namespace PayloadSecurityMiddleware
{
    public class PayloadInspectionMiddleware
    {
        private readonly RequestDelegate _next;

        public PayloadInspectionMiddleware(RequestDelegate next)
        {
            _next = next;
        }

        public async Task Invoke(HttpContext context)
        {
            // Read the request payload
            string payload = await new StreamReader(context.Request.Body, Encoding.UTF8).ReadToEndAsync();

            // Perform payload inspection for potential threats
            if (HasMaliciousContent(payload))
            {
                context.Response.StatusCode = 400;
                await context.Response.WriteAsync("Malicious payload detected.");
                return;
            }

            await _next(context);
        }

        private bool HasMaliciousContent(string payload)
        {
            // Implement your payload inspection logic here
            // Return true if malicious content is detected
            return false;
        }
    }

    public static class PayloadInspectionMiddlewareExtensions
    {
        public static IApplicationBuilder UsePayloadInspectionMiddleware(this IApplicationBuilder builder)
        {
            return builder.UseMiddleware<PayloadInspectionMiddleware>();
        }
    }
}

In this example, the PayloadInspectionMiddleware reads the request payload and performs a basic inspection. You should replace the HasMaliciousContent method with actual logic to detect malicious content patterns.

Conclusion:

By leveraging ASP.NET Core OWIN middleware, developers can implement custom security measures to prevent payload-based attacks. This article covered the basics of using middleware for payload inspection, but further customization and integration with security libraries are recommended to create a comprehensive defense strategy against various payload attacks.

Download CSV file using JavaScript fetch API

2023-08-24T10:58:00.001+05:30

Downloading a CSV File from an API Using JavaScript Fetch API: A Step-by-Step Guide

Introduction: Downloading files from an API is a common task in web development. This article walks you through the process of downloading a CSV file from an API using the Fetch API in JavaScript. We'll cover the basics of making API requests and handling file downloads, complete with a sample code snippet.

Prerequisites: Ensure you have a basic understanding of JavaScript and web APIs. No additional libraries are required for this tutorial.

Step 1: Creating the HTML Structure: Start by creating a simple HTML structure that includes a button to initiate the file download.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>CSV File Download</title>
</head>
<body>
    <button id="downloadButton">Download CSV</button>
    <script src="script.js"></script>
</body>
</html>

Step 2: Writing JavaScript Code: Create a JavaScript file named script.js and write the code to download the CSV file using the Fetch API.

// Select the download button
const downloadButton = document.getElementById("downloadButton");

// Define the API URL for the CSV file
const apiUrl = "https://api.example.com/download/csv";

// Function to trigger file download
async function downloadCSV() {
    try {
        const response = await fetch(apiUrl);
        const blob = await response.blob();

        // Create a URL for the blob
        const blobUrl = URL.createObjectURL(blob);

        // Create a temporary anchor element
        const link = document.createElement("a");
        link.href = blobUrl;
        link.download = "data.csv";
        link.style.display = "none";

        // Append the anchor element to the body and click it
        document.body.appendChild(link);
        link.click();

        // Clean up
        document.body.removeChild(link);
        URL.revokeObjectURL(blobUrl);
    } catch (error) {
        console.error("Error downloading CSV:", error);
    }
}

// Attach click event listener to the button
downloadButton.addEventListener("click", downloadCSV);

Step 3: Explanation of the Code:

We select the download button element using document.getElementById.
Inside the downloadCSV function, we use the Fetch API to make a request to the API URL. We await the response and convert it to a Blob object, which represents binary data.
We create a URL for the Blob using URL.createObjectURL.
A temporary anchor (<a>) element is created and configured to download the CSV file. We set its href to the blob URL and download attribute to specify the filename.
The anchor element is appended to the document body and programmatically clicked to initiate the download.
Finally, the anchor element is removed, and the blob URL is revoked to free up resources.

Step 4: Testing the Code: Open the HTML file in a web browser. Clicking the "Download CSV" button will trigger the API request and download the CSV file to the user's device.

Conclusion: Using the Fetch API, you can easily download files from APIs in your web applications. This step-by-step guide demonstrated how to download a CSV file using the Fetch API, providing a seamless way to retrieve and save files from remote sources. Feel free to adapt this approach for different file types and use cases in your projects.

Data Migration Strategies for MySQL databases

2023-08-24T10:36:00.005+05:30

Data Migration Strategies for MySQL Databases: An Overview of few Approaches

Introduction: Data migration is a crucial aspect of database management, especially when transitioning from one MySQL database to another. It involves transferring data while ensuring its integrity, consistency, and availability. This article explores the challenges associated with data migration and provides detailed solutions along with sample code snippets.

Challenges in Data Migration:

Data Consistency: Ensuring that data remains consistent during the migration process is critical. Any discrepancies or errors can lead to incorrect results and data loss.
Downtime: Minimizing downtime is essential to maintain business continuity. Prolonged downtime can disrupt operations and affect user experience.
Schema Changes: When migrating to a new database version, the schema might change. Adapting the data to fit the new schema while preserving relationships can be complex.
Data Volume: Handling large data volumes requires efficient migration techniques to avoid performance bottlenecks and resource exhaustion.

Data Migration Strategies:

Dump and Restore: This strategy involves exporting the data from the source database into a file (using tools like mysqldump), transferring it to the target database, and then restoring it. While simple, it might not be suitable for large datasets due to longer downtime and performance impact.

Sample code for exporting data:
```
mysqldump -u username -p database_name > dump.sql
```

Replication: MySQL replication allows real-time data synchronization between the source and target databases. This approach minimizes downtime and ensures consistency.

Sample code for setting up replication:

-- On source server
GRANT REPLICATION SLAVE ON *.* TO 'replication_user'@'target_ip' IDENTIFIED BY 'password';
FLUSH PRIVILEGES;
FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

-- On target server
CHANGE MASTER TO MASTER_HOST='source_ip', MASTER_USER='replication_user', MASTER_PASSWORD='password', MASTER_LOG_FILE='bin_log_file', MASTER_LOG_POS=bin_log_position;
START SLAVE;

ETL (Extract, Transform, Load): ETL involves extracting data from the source, transforming it to fit the target schema, and loading it into the destination database. This is ideal for data transformations and filtering.

Sample code using Python and MySQL Connector:

import mysql.connector

source_db = mysql.connector.connect(host='source_host', user='username', password='password', database='source_db')
target_db = mysql.connector.connect(host='target_host', user='username', password='password', database='target_db')

source_cursor = source_db.cursor()
target_cursor = target_db.cursor()

source_cursor.execute('SELECT * FROM source_table')
data = source_cursor.fetchall()

for row in data:
    # Transform data if needed
    target_cursor.execute('INSERT INTO target_table VALUES (%s, %s, %s)', row)

target_db.commit()

Conclusion: Data migration is a critical phase in database management, requiring careful planning and execution. By understanding the challenges and implementing appropriate strategies like dump and restore, replication, or ETL, you can ensure a seamless transition while maintaining data integrity and minimizing downtime. Utilizing the sample code provided for each strategy will help you get started on your data migration journey.

Chrome for Testing: Browser Automation made easy

2023-08-24T09:30:00.001+05:30

Introduction

Google Chrome for Testing is a new Chrome flavor that specifically targets web app testing and automation use cases.

Browser and Driver Setup and Maintenance

Setting up a browser testing environment is a difficult task, and it needs more effort to keep the browser and the chrome driver up-to-date and it is a top web developer / automation engineer's pain point

The Auto-Update

One of Chrome’s most notable features is its ability to auto-update. This helps the users to keep their browsers up-to-date with the latest upgrades and be safe when they are online

As a developer / automation engineer, running a suite of end-to-end tests requires the below aspects to be taken care of

You want consistent, reproducible results across repeated test runs—but this may not happen if the browser executable or binary decides to update itself in between two runs.
You want to pin a specific browser version and check that version number into your source code repository, so that you can check out old commits and branches and re-run the tests against the browser binary from that point in time.

The process of a browser that auto-updates creates compatibitlity with webdriver and there needs to be a manual intervention to address this.

Versioned browser binaries

Google intentionally doesn’t make versioned Chrome downloads available, since users shouldn’t have to care about version numbers—they should always get updated to the latest version as soon as possible. This is great for users, but painful for developers needing to reproduce a bug report in an older Chrome version

Due to the manual intervention and addressing of the version issue, there are flaws.

First, these Chromium binaries are not reliably available across all platforms.
Second, they are built and published separately from the Chrome release process, making it impossible to map their versions back to real user-facing Chrome releases.
Third, Chromium is different from Chrome

Chrome for Testing

Designed to solve these problems, Chrome for Testing is a dedicated flavor of Chrome targeting the testing use case, without auto-update, integrated into the Chrome release process, made available for every Chrome release. A versioned binary that’s as close to regular Chrome as possible without negatively affecting the testing use case.

The difficulties we previously mentioned around finding a matching Chrome and ChromeDriver binary can be completely eliminated by integrating the ChromeDriver release process into the Chrome for Testing infrastructure. In addition to solving this user-facing pain point, this also aligns ChromeDriver releases with Chrome’s, and eliminates the manual ChromeDriver release process.

Conclusion

The release of chrome for testing is a big boon for the developers and the automation engineers that helps in fixing the chrome and driver compatibilty across upgrades. Please read through the below link to learn more and to use the latest commands from google team Chrome Blog

Happy & Peaceful Testing !!!

Git Bash Developer Productivity Tips

2023-08-23T10:44:00.008+05:30

Boosting Productivity with Custom Aliases in Git Bash

As a developer, efficiency and productivity are paramount. One of the lesser-known gems in Git Bash is the ability to create custom aliases, which are shortcuts for commonly used Git commands or even more complex workflows. These aliases not only save keystrokes but also streamline your workflow, making version control tasks smoother and quicker. In this article, we'll dive into the world of custom aliases, explore how to create them, and highlight their time-saving potential.

What are Custom Aliases?

Custom aliases in Git Bash are user-defined shortcuts for frequently used commands. These aliases can be as simple as abbreviations or even entire command sequences. They can encompass both Git-specific commands and regular shell commands.

Creating Custom Aliases

To create custom aliases, you need to modify your Git configuration. You can define aliases in two scopes:

local (for a specific repository)
global (for all repositories on your system)

Here's how you can set up custom aliases:

1. Global Aliases

To create a global alias, open your Git Bash and use the following command:

git config --global alias.alias_name 'original_command'

Replace alias_name with your desired alias and original_command with the Git command you want to abbreviate. For instance, to create an alias for git status, you would run:

git config --global alias.st 'status'

2. Local Aliases

Local aliases are specific to a particular repository. Inside the repository's directory, use the same git config command without the --global flag:

git config alias.alias_name 'original_command'

Time-Saving Examples

Now let's explore some real-world examples of how custom aliases can save a developer's time:

1. Abbreviating Common Commands

git config --global alias.ci 'commit -m'

With this alias, instead of typing git commit -m "your message", you can simply use git ci "your message". This minor change might seem inconsequential, but over numerous commits, the saved keystrokes add up.

2. Complex Workflows

git config --global alias.feature 'checkout -b feature/'

This alias allows you to create and switch to a new feature branch in one command:

git feature new-feature

3. Combining Commands

git config --global alias.ac '!git add -A && git commit'

This alias stages all changes and commits them in one step:

git ac -m "Updated feature"

4. Displaying a Colorful Log

git config --global alias.lg 'log --graph --oneline --all --decorate'

Now, the command git lg provides a compact, colorful overview of your repository's history.

Conclusion

Custom aliases in Git Bash are a developer's secret weapon for improving productivity. By creating shortcuts for frequent commands and complex workflows, you can streamline your version control tasks and save valuable time. Start by identifying the commands you use most often and create aliases to match. With a bit of setup, you'll be amazed at how much more efficient your Git workflow becomes.

Remember that while aliases are powerful tools, they should be used judiciously. Overloading your Git Bash environment with too many aliases might make it harder to remember them all. Choose aliases that genuinely enhance your workflow and align with your preferences.

So, go ahead and unleash the full potential of Git Bash custom aliases. Your fingers will thank you for the reduced typing, and your productivity will soar to new heights!

RIAK Database - A Distributed NoSQL Solution

2023-08-23T10:29:00.001+05:30

Exploring Riak Database: A Distributed NoSQL Solution

Introduction

In the world of modern technology, data storage and retrieval have become crucial aspects of software development. Traditional relational databases often struggle to handle the scale and distribution demands of today's applications. This is where Riak, a distributed NoSQL database, comes into play. In this article, we will delve into the problem that Riak database solves and explore how it addresses these challenges with real-world examples.

The Problem: Scalability and Fault Tolerance

Traditional relational databases are built around a fixed schema and a single server. As data grows, these databases might struggle to handle the load efficiently, leading to performance bottlenecks and scaling challenges. Moreover, relying on a single server can be risky – if that server fails, data might be lost or unavailable.

Riak's Solution: Distributed Architecture

Riak was designed to tackle these problems head-on. It employs a distributed architecture that allows data to be spread across multiple nodes or servers. This distribution not only enables horizontal scalability – adding more nodes as data grows – but also provides fault tolerance. In Riak, data is automatically replicated across nodes, ensuring that if one node fails, the data is still accessible from other nodes.

Data Model: Key-Value Store

Riak follows a key-value store data model. Each piece of data is associated with a unique key, and data retrieval is performed using these keys. This model simplifies data storage and retrieval and is well-suited for scenarios where flexible schema design is required.

Sample Scenario: E-Commerce Application

Imagine an e-commerce application using Riak to handle product information. Each product is identified by a unique product ID, which serves as the key. The product details, such as name, price, and description, are stored as values. Riak's distributed nature ensures that even during high traffic or server failures, the product information remains available and accessible.

CAP Theorem and Riak

The CAP theorem states that in a distributed system, it's challenging to achieve Consistency, Availability, and Partition tolerance simultaneously. Riak follows the AP (Availability and Partition tolerance) model. It ensures that even in the face of network partitions, the system remains available, sacrificing strong consistency.

Concurrency Control: Vector Clocks

Riak employs vector clocks to manage concurrent updates to the same piece of data across nodes. These clocks help determine the causal ordering of events, enabling Riak to resolve conflicts and reconcile multiple versions of the data.

Use Cases

Riak is particularly well-suited for scenarios where high availability and fault tolerance are critical. It's often used in applications like real-time analytics, content delivery, and session management.

Conclusion

Riak database addresses the challenges of scalability and fault tolerance that traditional relational databases face. Its distributed architecture, key-value data model, and use of vector clocks for concurrency control make it a powerful solution for applications requiring high availability and flexible data storage. By embracing Riak's design principles, developers can build robust, resilient, and scalable systems that meet the demands of modern technology.

Using Spring Cloud Config Server for Multi-tenancy configuration

2023-08-20T17:28:00.003+05:30

Let's walk through a basic example of how to set up and use Spring Cloud Config Server to manage configuration for multiple tenants in a Spring Boot Application.

Step 1: Create a Spring Cloud Config Server

Create a new Spring Boot project with the Spring Cloud Config Server dependency.

<!-- pom.xml -->
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-config-server</artifactId>
</dependency>

Configure the Spring Cloud Config Server:

// src/main/java/com/example/ConfigServerApplication.java
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.config.server.EnableConfigServer;

@SpringBootApplication
@EnableConfigServer
public class ConfigServerApplication {
    public static void main(String[] args) {
        SpringApplication.run(ConfigServerApplication.class, args);
    }
}

Configure the application properties to specify the Git repository where configuration files are stored:

# src/main/resources/application.yml
spring:
  cloud:
    config:
      server:
        git:
          uri: <URL to your Git repository>

Step 2: Create Configuration Files for Tenants

In your Git repository, organize configuration files based on tenants and profiles:

 config-repo/
|-- application-tenant1.properties
|-- application-tenant2.properties
|-- application.yml

Example content for application-tenant1.properties:

# application-tenant1.properties
greeting.message=Hello from Tenant 1!

Example content for application-tenant2.properties:

# application-tenant2.properties
greeting.message=Hello from Tenant 2!

Step 3: Create a Spring Boot Application

Create a new Spring Boot project for the multi-tenant application:


<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <!-- Other dependencies -->
</dependencies>

Configure the application to use the Spring Cloud Config Client:

# src/main/resources/bootstrap.yml 
spring:
  application:
    name: myapp
  cloud:
    config:
      uri: http://localhost:8888 # URL to your Config Server

Create a controller to retrieve the configuration property:

// src/main/java/com/example/MyAppController.java
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class MyAppController {
    @Value("${greeting.message}")
    private String greetingMessage;

    @GetMapping("/greeting")
    public String getGreetingMessage() {
        return greetingMessage;
    }
}

Step 4: Test the Application

Start the Spring Cloud Config Server application.
Start the multi-tenant Spring Boot application.
Access the greeting message for each tenant using the /greeting endpoint:
1. For Tenant 1: http://localhost:8080/greeting
2. For Tenant 2: http://localhost:8080/greeting

The Spring Cloud Config Server will provide the appropriate configuration properties based on the tenant-specific profile used in the application. This example demonstrates a basic setup of using Spring Cloud Config Server to manage configuration for multiple tenants in a Spring Boot application.

Dynamic Tenant identification - Simple approach

To dynamically load the configuration for different tenants based on a URL parameter or path segment, you can use Spring Cloud Config's native support for profiles and the application name to achieve this. Here's how you can modify the configuration to achieve dynamic loading of tenant-specific configurations:

Below given is a sample yaml configuration that can be used to dynamically identify the tenant

# src/main/resources/bootstrap.yml
spring:
  cloud:
    config:
      uri: http://localhost:8888
  application:
    name: myapp # The base application name
  profiles:
    active: @tenant # Placeholder for the active profile

Tenant Identification from URL

// src/main/java/com/example/MyAppController.java
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class MyAppController {
    @Value("${greeting.message}")
    private String greetingMessage;

    @GetMapping("/greeting/{tenant}")
    public String getGreetingMessage(@PathVariable String tenant) {
        // Set the active profile dynamically based on the tenant
        System.setProperty("spring.profiles.active", tenant);
        try {
            // Business logic using the tenant-specific profile
            return greetingMessage;
        } finally {
            // Clean up the active profile to avoid affecting subsequent requests
            System.clearProperty("spring.profiles.active");
        }
    }
}

Access Configuration via URL Path:

Now you can access the tenant-specific configuration by specifying the tenant as a path segment in the URL. For example:

For Tenant 1: http://localhost:8080/greeting/tenant1
For Tenant 2: http://localhost:8080/greeting/tenant2

Each URL will dynamically set the active profile based on the tenant, and the Spring Cloud Config Server will serve the corresponding tenant-specific configuration properties.

Using Apache Avro for Efficient Data Serialization

2023-08-20T16:19:00.002+05:30

Introduction

In the world of software development, efficient data serialization plays a crucial role in enabling smooth communication between different components, systems, and applications. One popular technology that addresses the challenges of data serialization is Apache Avro. Avro is a powerful and versatile framework that offers a compact and efficient way to serialize data, making it an ideal choice for various scenarios. In this article, we will explore the need for efficient data serialization, the problems it helps us solve, and how to use Apache Avro with Java, using a "Product" entity as an example.

The Need for Efficient Data Serialization

Modern software systems are often composed of multiple components running on different platforms and communicating over various protocols. This communication involves sending data between these components, which can be in different formats and structures. Data serialization is the process of converting complex data structures, such as objects, into a format that can be easily transmitted and reconstructed on the receiving end.

Efficient data serialization is crucial for several reasons:

Reduced Bandwidth Usage: Efficient serialization techniques minimize the amount of data that needs to be transmitted over the network, reducing bandwidth consumption and resulting in faster communication.
Optimized Storage: Compact serialized data requires less storage space, which is particularly important when dealing with large volumes of data.
Interoperability: Different programming languages and platforms might have different native data representations. A standardized serialization format ensures that data can be easily exchanged between systems regardless of the underlying technology.
Versioning: As software systems evolve, data structures may change. A well-designed serialization format should support backward and forward compatibility to avoid breaking existing systems when changes occur.

Problems Solved by Apache Avro

Apache Avro addresses several problems associated with data serialization:

Schema Evolution: Avro uses a schema-based approach to serialization, where the schema is included with the data. This enables seamless evolution of data structures over time without requiring modifications to the consumer code. Old and new versions of schemas can coexist, allowing data compatibility between different versions of an application.
Compact Binary Format: Avro's binary encoding is highly compact, resulting in reduced network traffic and storage requirements. This is particularly beneficial for applications dealing with large datasets.
Dynamic Typing: Avro allows dynamic typing, which means that data types are defined in the schema itself. This provides flexibility and makes it easier to work with data from different sources.
Code Generation: Avro provides code generation capabilities, allowing you to generate Java classes from Avro schemas. This reduces the effort required to serialize and deserialize data while also enhancing type safety.

Using Apache Avro with Java: The "Product" Entity Example

Let's explore how to use Apache Avro with a simple "Product" entity in Java. We will define an Avro schema for the "Product" entity, generate Java classes from the schema, and demonstrate serialization and deserialization.

Step 1: Define the Avro Schema

{
  "type": "record",
  "name": "Product",
  "fields": [
    {"name": "id", "type": "int"},
    {"name": "name", "type": "string"},
    {"name": "price", "type": "double"}
  ]
}

Step 2: Generate Java Classes

Use the Avro tool to generate Java classes from the schema:

$ java -jar avro-tools-1.10.2.jar compile schema product.avsc ./output

Step 3: Serialization and Deserialization

The sample code given below demonstrate the serialization and deserialization

import org.apache.avro.file.DataFileReader;
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DatumWriter;
import org.apache.avro.specific.*;
import java.io.File;
import java.io.IOException;

public class AvroExample {

    public static void main(String[] args) throws IOException {
        Product product = new Product();
        product.setId(1);
        product.setName("Sample Product");
        product.setPrice(29.99);

        // Serialize the object to a file
        DatumWriter<Product> productDatumWriter = new SpecificDatumWriter<>(Product.class);
        DataFileWriter<Product> dataFileWriter = new DataFileWriter<>(productDatumWriter);
        dataFileWriter.create(product.getSchema(), new File("product.avro"));
        dataFileWriter.append(product);
        dataFileWriter.close();

        // Deserialize the object from the file
        DatumReader<Product> productDatumReader = new SpecificDatumReader<>(Product.class);
        DataFileReader<Product> dataFileReader = new DataFileReader<>(new File("product.avro"), productDatumReader);
        Product deserializedProduct = dataFileReader.next();
        dataFileReader.close();

        System.out.println("Deserialized Product: " + deserializedProduct);
    }
}

In this example, we define an Avro schema for the "Product" entity, generate Java classes using the Avro tool, and then demonstrate serialization and deserialization using these classes.

Conclusion

Efficient data serialization is a critical aspect of modern software development, enabling smooth communication between different components and systems. Apache Avro addresses the challenges of data serialization by providing a schema-based, compact, and efficient approach. With its support for schema evolution, code generation, and dynamic typing, Avro is an excellent choice for various serialization scenarios. By following the steps outlined in this article and using the provided Java example, you can integrate Apache Avro into your projects to achieve efficient data serialization and transmission.

pages in next.js

2022-08-08T13:13:00.000+05:30

Introduction

Next.js contains a feature called as Pages that helps us to maintain a clear isolation of the page level components and their layouts.

This helps us in having page and routing implementations separate from the components, helping in isolation of the application layout / presentation with that of the functional components

This segregation helps us in having slim components and isolate the layout / page level actions. Also, since there are pages, the components that comprise a page atleast at the higher level can be easily found out in this system.

Problem Statement

We have been using react.js for building SPA (Single Page Applications) for a while and we experienced that some features like pages and routing are not very comfortable with react.js

Compared with other frameworks, react does not have option to fully perform the routing and having pages. There are times where we have to duplicate the layouts to all components that act like pages in the react application.

These are resulting in lots of code getting redundant and duplicated across various components in react.js application.

Solution

Pages are configured in a Next.js application and the directory structure looks like the one below

Application Root

> components

> pages

> aboutus

> contact

> products

> index.js (this normally is the entry point file in the page, this typically will invoke the list of products through its component)

> [productId].js (this filename contains the route parameter as a placeholder in the filename itself thereby visually expressing that this file needs a URL Query parameter. In our example, this productId is the id of the product and this page will have components that will render the details of a product.)

This file system organization helps us in the following,

1. Clear separation of what goes in a page

2. Visual indication in the file name itself of what a file is responsible for and also what the parameters the component requires

3. The pages are also react.js components, so that there is no extra effort to learn / implement the pages features

I will share sample application on features of Next.js in subsequent blog posts.

How to make table schema changes and restore data after appropriate transformation

2022-06-17T13:15:00.003+05:30

Introduction

In many cases we might have to encounter scenarios where we need to perform backup of data or perform a schema change like data type transformation etc.

This blog post illustrates the steps that can be used as a checklist to perform the operation at ease

Steps

The below sql query creates a new table with the same schema as given in the like field

create table <table_name>_old like <table_name>;

Now that we have the schema ready, we can copy the data for backup using the below command

insert into <table_name>_old select * from <table_name>;

Once the above command succeeds, we can drop the old table

drop table <table_name>;

Then we can create the new table with the same name as the one given in <table_name> so that we can allow applications to still use the same table name.

CREATE TABLE `<table_name>` (
  `id` bigint NOT NULL AUTO_INCREMENT,
...);

Now that the table with new schema is ready, we can migrate the data from the old table to the new table using below like query

insert into <table_name> select * from <table_name>_old;

Here, we are using a simple select * from query, but in real time there might be joins or other ways to arrive at the transformed data.

This way we can ensure that the table data is transformed to the new schema and is ready to be used.

Verify the newly migrated data using a select query like given below

select * from <table_name>;

Once all the records are looking good, we can drop the old table as it might be occupying lot of memory due to the redundant data, which can be done using the below command.

drop table <table_name>_old;

Hope this helps :)

using mysqldump command to backup the data from MySQL Database table

2022-06-16T16:25:00.002+05:30

Introduction

In this blog post, we will take a look at the mysqldump command that is shipped as part of the MySQL clients.

Data Backup

We normally deal with backup of data by exporting the data as JSON / CSV or as SQL Files. This is required so that any change that we are working on might have a impact, we can immediately use the backup to fix any unexpected database operation.

When working with JSON / CSV at times, we have to deal with the issues in the data formats. Further as SQL queries, we get the option to rewrite the query, insert them into a local database, perform the required transform or operations and then move to the server.

The MySQL Workbench provides the export option through which we can generate the SQL scripts, but there are issues like the target table names are empty and there is a new insert statement for each row in the table.

Example Command

mysqldump.exe 
--host=servername --port=3306 --default-character-set=utf8 
--user=(db_user_name) -p --protocol=tcp --column-statistics=FALSE --skip-triggers
--set-gtid-purged=OFF --no-create-info --quick --max_allowed_packet=512M --skip-lock-tables 
"db_name" "table_name" --where="column_name='value'" > script_name.sql

As we can see in the above command, we have provided the following arguments

--host -> to provide the server name

--user -> the database connectivity user name

--port -> default to 3306 or custom port if used

--default-character -> we are using utf8, it can be based on the char set defined in the table schema

-p -> to prompt for the server password

--protocol -> by default, we are using the tcp protocol

--skip-triggers -> to skip invocation of triggers

--set-git-purged -> to skip the global transaction identifiers

--no-create -> to not include the create table script as part of the generated sql files

--quick -> for handling large volume of data

--max_allowed_packet -> to set a max packet size, we are using 512 MB

--skip-lock-tables -> to avoid locking table during running the query, default is to lock the table

--where -> adds a filter condition to the generated query so that, we can split into chunks or focus on the required subset of data in the export

I have tried this and it works fine. Hope this helps for anyone in need.

http/3

2022-06-15T17:14:00.004+05:30

Introduction

HTTP/3 is the 3rd major version of the HTTP protocol that powers the internet. In comparison with the previous version of http which relied on TCP, http/3 relies on QUIC.

QUIC

Specification published at 6th June, 2022

Developed initially at Google in 2012, had its way to IETF and public by 2022. A decade for completion and standardization !!!

Are the semantics like requests, methods, status codes still the same like previous versions of http/2?

yes, they still remain the same

The underlying mechanisms are changed, like http/3 uses space congestion control over UDP (User Datagram protocol)

Head-Of-Line Blocking (HOL)

This is a performance problem where there is a queue of packets built due to the first packet that is yet to be consumed.

We have browsers that have limits on the number of parallel requests that it can send to a server, when they are used up, as we anticipate, a Queue is formed for the newer requests that start accumulating the newer requests till the former requests are completely processed.

We had HTTP/2 to address this issue by the process of multiplexing which eliminates this HOL at the application layer. However there is still a blocking HOL in the TCP layer.

This was a bottleneck that limits the performance. Hence, QUIC solves this problem by using multiplexing with UDP and thereby removing the HOL with TCP.

So, we get to leverage good performance.

0-RTT

For any server and client that have been connected in the past, there is no formal introduction - acknowledgements. More like we talking to our friends without any formal introduction / waiting for they to acknowledge our greetings etc, so we make requests responded quick through (QUIC)

Encryption by Default

The data at the application layer will always be encrypted. There is no separate TCP handshake and TLS handshake which were distinct events in the previous versions of HTTP.

In this case how are the encryptions and connections managed?

QUIC sets up the encrypted connections at the transport layer. Since this is happening as a single action, applications does not need to perform a TCP handshake and a TLS handshake.

Further, packet number and few other headers are encrypted by default giving a tough time for the hackers to understand what is going on.

Are there apps out in http/3?

yes, I have seen apps from Google, cloudflare using http3 for many of their apps. The below is the network capture in Google Chrome new tab which loads content from google

Browser Support

Most of the leading browsers support HTTP/3. Chrome (79 and onwards), Firefox (72 and onwards) etc

Web Server Support

Currently, the LiteSpeed Webserver supports HTTP/3. I did try it out for a Java App and realized that it was not easy as there were listeners and the team was not sure on the use of them. I feel that running an app on another webserver while using Litespeed as a proxy can be used to achieve support for HTTP/3.

I will try it out in the upcoming days and share my findings or steps for the same.

However, Kestrel, Nginx webservers do seem to have a good support for the HTTP/3. Also, IIS in Windows Server 2022 has support, looks like some registry keys are to be setup to enable HTTP3

We have just scratched the surface and as we delve into the details, things will get very interesting.

Split a very large file in to chunks

2022-06-13T13:36:00.000+05:30

Introduction

Many a times we are having to deal with the very large files, be it the logs from the server like catalina.out or any other huge sql files or csv or text files.

These are some artifacts that could have been generated in the server or might contain the data. There might not have been detailed logging systems that can do a RollingFile like appender that can get the files into smaller chunks for us to use easily. In some cases these could be large data which is hard to split into multiple chunks in an export mechanism.

Problem Statement

When we try to open them in an editor like Notepad / Notepad++ / VS Code, we normally run into memory issues and the application gets frozen.

At times, in case of windows, we might have to go to Task Manager , find the application and then End Task so that we can use the other apps or to allow that particular application to be usage

Solution

In this post, we explore on using git bash a tool. We normally use a git bash command line interface that can be used to perform the git operations. In this case we can use a useful command in git bash that can help us here. Open git bash from the windows or linux system.

Assuming that we have a large file to name as largefile.ext which we can think of as having more than 1 GB of size and that we wanted it to be split into smaller chunks of 300mb. The below command can be used from the command line

split largefile.ext -b 300m

Once run, we can use a dir incase of a windows OS or a ls (or) ls -al on a linux system that can help us view the list of files that were created in the process of splitting.

Remarks

I have been trying to literally open server logs by downloading them from server and given that they were very huge, I had to try with different editors and even if the file was opened, a search could take minutes or move the application (editor) into a frozen state which can take mins to unfreeze or might have to end task and restart.

This solution literally saved a lot of my time. Hope it does for the reader as well

Upgrade from http1.1 to http2 for Java spring boot applications hosted in tomcat

2022-06-13T13:22:00.002+05:30

In this post, we will list down the tasks to be done for enabling the HTTP 2.0 support in spring boot applications which are hosted in Apache tomcat webserver

Application Level Changes

Spring boot Application Configuration Changes

server.http2.enabled=true

In the spring boot application's application.properties file, we have to add the above line so that Spring boot can add the support for http2

Tomcat server configuration

In the tomcat web server, we should have SSL enabled before doing the below change.

To start with, we have to shutdown the tomcat server instance that is running

CD to the directory that has tomcat installed and cd to the bin directory and run the below command

sh shutdown.sh

We have add the UpgradeProtocol which adds the respective Http2Protocol handler classname to the connector pipeline that enables support for http2.0

<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />

The above UpgradeProtocol can be added to the connector as given below

<Connector port="443" protocol="org.apache.coyote.http11.Http11AprProtocol"
               maxThreads="150" SSLEnabled="true" ...>
        <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
</Connector>

The above command has omitted the SSL configuration that is present for brevity.

Once the above are done, we need to stop and start tomcat by running the below commands

1. CD to the directory that has the tomcat server's bin directory

2. Run the below command to start tomcat

sh startup.sh

Once done, navigate to the browser and open developer tools (F12 in chrome, or right click and choose "Inspect")

Navigate to the "Network" tab and view the requests that are being sent to the API Service. All of the requests should be sent using http2 identified by h2 in the protocol column.

If you are unable to find the protocol column, on the network tab header columns, right click and choose protocol

Jsonview in Spring boot

2022-05-29T17:10:00.002+05:30

Introduction

There are scenarios where we have a DTO or a model class that contains a collection of properties and there might be some endpoints that require a subset of them and some other endpoint that might need all of them.

Example: Consider a scenario where we need a list of Categories in a dropdown and we have an API that gets all the categories from the static masterdata and need to send to the UI in response to an API call.

Problem Statement

We might choose to use a `Map` data type so that we send out the key-value pairs to the database or to choose to build a custom DTO with only the Id, name of the categories.

In a longer run, if we opt to use multiple DTO in the source, sooner there will be a very large collection of DTO classes that represent subset of a static data.

Solution

Rather than ending up with a large number of such DTO classes, we can use a JSON View. This helps us have a single class and control which fields are part of which view.

Example: We can have a dropdown view in categories that will have an Id,Name alone. Similarly, there might be a grid view which might need some other subset of properties, which can be referred through a view.

This helps us a lot because we no longer have to worry about the multiple DTO, but we can be able to view in a single class definition, which properties are used by which parts of the application.

Below given is a sample application that illustrates the above concepts.

The Categories Class

public class Categories implements Serializable {
   
    @JsonView({CategoryViews.CategoryCollectionView.class, CategoryViews.CategoryExecutionView.class})

    private Long id = null;

    @NotNull
    @JsonView({CategoryViews.CategoryCollectionView.class, CategoryViews.CategoryExecutionView.class})

    private String name = null;

    @JsonView(CategoryViews.CategoryCollectionView.class)
    private Long createdBy = null;

    @JsonView(CategoryViews.CategoryCollectionView.class)
    private Date createdOn = new Date();

    @JsonView(CategoryViews.CategoryExecutionView.class)

    private boolean executable = false;

The CategoryView are defined like below

public interface CategoryViews{

    interface CategoryCollectionView{

    }

    interface CategoryExecutionView{
    }
}

The usage in an API controller

@GetMapping(value = "categories/forExecution", produces = {APPLICATION_JSON_VALUE})
@JsonView(CategoryViews.CategoryExecutionView.class)
public ResponseEntity<List<Categories>> getFilterExecutableCategories(... ) { ... }

Remarks

Here, we have used a view defined in the interface CategoryViews that can be used to define the multiple representation of the data.

We have 2 view namely CategoryCollectionView & CategoryExecutionView.

We have decorated our Categories class fields with the respective views based on the requirement.

These are seen in the code snippet given above.

Once the mappings are done, we can use the API controllers and decorate the endpoints with the respective JsonViews.

This helps spring boot to apply the appropriate serialization process to render the selected fields alone in the response rather than the whole set of properties in the entity.

This helps reduce the volume of data transferred over the wire from the API to the UI or mobile front-end. This also helps the consumer get a clear picture on what fields they are needing and get only those rather than send in full and clutter the code base to filter out the required propertiess.

Handling exceptions in the Executor service threads in Java

2022-05-21T18:15:00.006+05:30

Introduction

This is a continuation post on the exception handling strategies in the threads in Java.

For Introduction, please read this post

The second post is available here

This post addresses the problem statement "How to use the exception handlers in the threads spawned by the Executor Service in Java?"

Not all times, we will be using Thread classes to run our threads because we have to manage a lot of the underlying logic for managing threads. There is ExecutorService in Java which comes to the rescue for the above problem.

In the previous posts, we have discussed on how to handle the exceptions in plain threads. However, when using executor service, we do not create / manage threads, so how do we handle exception in this case.

We have a ThreadFactory as an argument which can be used to customize the way threads are created for use within the ExecutorService.

The below snippet of code leverages this feature to illustrate the exception handling, wherein we create a factory and internally we manage the thread and set its ExceptionHandler and allow the executor service to use this thread so that it can bring in threads with exception handlers customized.

class ExceptionHandlingThreadFactory extends NamedThreadFactory {
    @Override
    public Thread newThread(Runnable r) {
        ct++;
        Thread t = new Thread(r, format("%s%d", name, ct));
        t.setUncaughtExceptionHandler((t1, e) -> System.out.println("Handling the exception in factory for thread: " + t1.getName() + " and exception is: " + e.getMessage()));
        return t;
    }
}

The above code illustrates the user of the Threadfactory (in the above case the NamedThreadFactory is actually implementing ThreadFactory as given below)

public class NamedThreadFactory implements ThreadFactory {
    protected static int ct = 0;
    protected String name = "My_NamedThread_";

    @Override
    public Thread newThread(Runnable r) {
        ct++;
        return new Thread(r, format("%s%d", name, ct));
    }
}

Now, that we have the factories in place, how do we use them, below given snippet of code illustrates how it is done.

ExecutorService service = Executors.newCachedThreadPool(new ExceptionHandlingThreadFactory());
service.execute(new ExceptionLeakingTask());

Handling exceptions at thread level in Java

2022-05-21T17:52:00.005+05:30

Introduction

This is a continuation to the post on exception handling on threads. For those interested in the background, please visit this link

This post discusses on the topic "Can I use separate exception handlers for threads performing mission critical tasks vs non trivial tasks?"

Per-Thread Exception Handler

Let's us assume that we built a handler that wants to notify the support team that something has gone wrong in some special thread processing, like the one below (not really notifying anyone but helps correlate the context)


class SupportNotificationExceptionHandler implements Thread.UncaughtExceptionHandler {

    @Override
    public void uncaughtException(Thread t, Throwable e) {
        notifySupport(t, e);
    }

    private void notifySupport(Thread t, Throwable e) {
        System.out.println("Notified support team on: " + e.getMessage() + " raised from : " + t.getName());
    }
}

Let us now apply the above special handler to specific threads in the application. Here we are opting for support team to be notified for any exception in main thread and in one other specific thread.

Refer the snippet of code below,


public class ThreadSpecificHandler {

    public static void main(String[] args) {
        Thread.currentThread().setUncaughtExceptionHandler(new SupportNotificationExceptionHandler());

        Thread.setDefaultUncaughtExceptionHandler(new ThreadExceptionHandler("DEFAULT"));
        new Thread(new ExceptionLeakingTask()).start();
        Thread t2 = new Thread(new ExceptionLeakingTask());
        new Thread(new ExceptionLeakingTask()).start();
        new Thread(new ExceptionLeakingTask()).start();

        t2.setUncaughtExceptionHandler(new SupportNotificationExceptionHandler());
        t2.start();
        throw new ArithmeticException("whatever");
    }
}

As given in the snippet above, we are using the SupportNotificationExceptionHandler for the main thread and 2nd thread (t2).

Whenever an exception is getting raised in the main and t2, they are notified to the support team, remaining ones are all using the default handler to log to the console and continue.

This illustrates how we can build the handlers specific for the collection of threads based on the business critical threading tasks vs non trivial threading tasks

Below given is the snippet where we have a single handler to be used in many threads


public static void main(String[] args) {
    SupportNotificationExceptionHandler supportNotificationExceptionHandler = new SupportNotificationExceptionHandler();
    Thread.currentThread().setUncaughtExceptionHandler(supportNotificationExceptionHandler);

    Thread.setDefaultUncaughtExceptionHandler(new ThreadExceptionHandler("DEFAULT"));
    new Thread(new ExceptionLeakingTask()).start();
    Thread t2 = new Thread(new ExceptionLeakingTask());
    new Thread(new ExceptionLeakingTask()).start();
    new Thread(new ExceptionLeakingTask()).start();

    t2.setUncaughtExceptionHandler(supportNotificationExceptionHandler);
    t2.start();
    throw new ArithmeticException("whatever");
}

When using a spring framework like Spring boot, we can have the handlers autowired as singletons, thereby eliminating the new instance creation in the code base.

Handling Exceptions in Threads in Java

2022-05-21T17:37:00.005+05:30

Introduction

Exception handling is very critical to any application and any language that was used to develop. This helps the following stakeholders

Stakeholders

End-User:

A graceful message that indicates that something has gone wrong in the system due to some wrong input or due to some internal fault

Developer

A clean way to get to know which line of code caused the exception and the stack trace that can help us reach to the points impacted

Support Team

A helpful hint that they can try to verify and resolve if it is something within their reach or to escalate to the product team

SRE (Site Reliability Engineering)

An indication of how far the application is performing w.r.to being reliable to the end-users and how faults are handled or tolerated etc..

Problem Statement

When we use threads to get the job done for performing some intensive task (computation or Network intensive tasks), it becomes difficult to track down on the exceptions.

This post is one in a series of posts that will talk about the exception handling strategies w.r.to the threading and tasks in Java.

Let us consider the below given simple app that has multiple threads running tasks. Here, the task is to throw exception which is for demonstration, which can be correlated in realtime with any 3rd party service / external service invocation that might throw exceptions.

package com.ds.mt.exceptions.handling;

public class ExceptionLeaking {

    public static void main(String[] args) {

        try {
            new Thread(new ExceptionLeakingTask()).start();
            new Thread(new ExceptionLeakingTask()).start();
            new Thread(new ExceptionLeakingTask()).start();
            new Thread(new ExceptionLeakingTask()).start();
        } catch (Exception e) {
            //this catch block does not get executed as the exceptions are thrown inside threads (not the main thread)
            System.out.println("Executing the handler for the thread exceptions");
            e.printStackTrace();
        }

    }
}


class ExceptionLeakingTask implements Runnable {

    @Override
    public void run() {
        throw new RuntimeException("Hello World!!!");
    }
}

As mentioned in the comments above, the exceptions are raised in the task that is run inside a thread and the handler is in the main method that wraps all the threads.

However, in runtime, the exception handling catch block does not get executed as the exceptions are not thrown in the main thread, which causes the application to break irrespective of having try...catch.. blocks in place.

Solution

package com.ds.mt.exceptions.handling;

public class ExceptionLeaking {

    public static void main(String[] args) {

        Thread.setDefaultUncaughtExceptionHandler(new ThreadExceptionHandler("DEFAULT"));
//      Thread.setDefaultUncaughtExceptionHandler((t, e) -> System.out.println("Exception raised from : " + t.getName() + " and exception is: " + e.getMessage()));
       
         new Thread(new ExceptionLeakingTask()).start();
         new Thread(new ExceptionLeakingTask()).start();
         new Thread(new ExceptionLeakingTask()).start();
         new Thread(new ExceptionLeakingTask()).start();

    }
}

class ThreadExceptionHandler implements Thread.UncaughtExceptionHandler {
    private final String instanceId;

    public ThreadExceptionHandler(String instanceId) {
        this.instanceId = instanceId;
    }

    @Override
    public void uncaughtException(Thread t, Throwable e) {
        System.out.println("Exception raised from : " + t.getName() + " inside instance: " + instanceId + " and exception is: " + e.getMessage());
    }
}

class ExceptionLeakingTask implements Runnable {

    @Override
    public void run() {
        throw new RuntimeException("Hello World!!!");
    }
}

In the above sample code, we have setup a exception handler and wired it to the Thread class Exceptiohandler setter method so that when there is an exception in any of the thread's (all of the threads in the applicaiton), this handler gets invoked.

The handler also catches the exceptions that are thrown in the main method too... figuring out how... obviously its too a thread (A Thread with a name as "main").