czetsuyatech

How to Log Method Execution Time in Spring Boot Using AOP

2025-11-04T09:24:00.003+08:00

Overview

Monitoring method execution time is one of the easiest yet most effective ways to understand your application’s performance. In this guide, we’ll explore how to use Spring Boot with Aspect-Oriented Programming (AOP) to log method execution times cleanly—without cluttering your business logic. By leveraging the @Aspect annotation, we can intercept method calls, measure how long they take, and log this information for performance analysis. This approach is flexible, reusable, and perfect for tracking performance bottlenecks in large-scale applications. Let’s dive in and build a lightweight execution time logger using Spring AOP.

Aspect Annotation

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface TrackExecutionTime {
 //
}

Aspect Interceptor

@Aspect
@Component
@Slf4j
public class ExecutionTimeAspect {

  @Around("@annotation(com.dcx.nba.actions.annotations.TrackExecutionTime)")
  public Object logExecutionTime(ProceedingJoinPoint joinPoint) throws Throwable {

    long startTime = System.currentTimeMillis();

    Object result = joinPoint.proceed();

    long endTime = System.currentTimeMillis();
    long executionTime = endTime - startTime;

    log.info("{} executed in {}ms", joinPoint.getSignature().toShortString(), executionTime);

    return result;
  }
}

Annotation Usage

This method runs via scheduler and computes the total elapsed time during the execution

@Override
@TrackExecutionTime
@SchedulerLock(name = JOB_NAME)
@Scheduled(cron = "${app.schedulers.update-action-status.cron}")
public void scheduleUpdateStatus() {
}

Secure React and Spring Boot with Microsoft Entra SSO and Bearer Token Authentication

2025-07-13T19:31:00.007+08:00

Introduction

Implementing secure, seamless login across a modern full-stack application is essential—but it doesn’t have to be complicated. In this guide, we’ll walk through how to integrate Microsoft Entra (formerly Azure AD) for Single Sign-On (SSO) in a React frontend and Spring Boot backend. The Spring Boot application acts as a resource server, validating JWT bearer tokens issued by Entra. Beyond authentication, we’ll also demonstrate how to map Microsoft Entra groups to internal application roles, dynamically fetched from your own database. Whether you're building internal tools or enterprise-grade platforms, this setup gives your app robust security, fine-grained access control, and a smooth user experience across services.

Problem

You need SSO in your frontend to allow users to login to your system and authorize the call of backend endpoints by sending the bearer token. In the backend we will get the user group attached to the currently log user and fetch the corresponding internal permissions which we will add as roles to the JWT.

Our schema will look like:

Prerequisites for this Exercise

Microsoft Entrata account (https://entra.microsoft.com)
Familiar with React
Experience in Spring REST programming

Microsoft Entrata

If you haven't sign up yet visit Microsoft Entrata website and register a new account. After a successful registration you should be logged and redirect to the dashboard page where you can get the tenant id. Take a note because we will use it in the client and backend configuration later.

App Registration

Inside Entrata, under Manage / App Registrations click New Registrations. For this exercise since we will be doing the authentication in the frontend we need to register an app of type SPA. Don't forget to fill-in the redirect URI.

Take note of the Application (client) ID.

Expose an API

Now we need to add a scope that we will use during login. This will allow us to use the MS Graph version 2. Without a defined scope Entrata will use version 1 which will throw an error during JWT validation in the backend.

Update Token Version in the Manifest

In the left menu, find the Manifest.

In the JSON document, find the requestedAccessTokenVersion and set its value to 2.

Note that the change takes time to propagate.

Now, our Entrata app is ready for integration.

React SPA Client Application

This exercise will use a SPA project provided by Microsoft available at https://github.com/Azure-Samples/ms-identity-ciam-javascript-tutorial/tree/main/2-Authorization/1-call-api-react/SPA. It uses MSAL library to authenticate the users and access secured APIs by acquiring security tokens from Microsoft Entrata.

SPA Modifications

We need to do several updates on the project.

Open authConfig.js.

Replace the clientId with the value of Application (client) Id that we have generated when we register the app in Entrata.

Update the value of authority using the Directory / Tenant Id.

Then to enable Graph V2 when signing in, we need to request access to the scope which we defined earlier.

In the same file authConfig.js, at the end of the file.

The SPA is now ready to login via SSO using Microsoft Entrata.

Backend Spring Service

We will start with an empty Spring Boot project and add each piece.

Maven Dependencies

We need to have at least the following. This project will exchange Entrata group mapping it to an internal role map to permissions as shown in the diagram above. The permissions will be used as roles.

Convert Entrata Group to Internal Roles

In this step, we will be using our internal tables where roles and permissions are stored. The Entrata group's UUID is searched in the auth_role table and the role's permissions are added as authorities to the JWT token.

This converter is used as the AuthenticationConverter when setting up the oauth2ResourceServer in the securityFilterChain. Furthermore, we need to set the resourceserver's issuer-uri in the application yml file. This URL will be used when validating the token.

Testing

To test the integration we will be creating a new controller with PreAuthorize annotated endpoints validating the authority bound to the JWT token.

To enable the method level security we need to annotate a configuration class with @EnableMethodSecurity.

Development and Support

The Spring project used in this tutorial is available at https://github.com/czetsuyatech/spring-ms-entrata-oauth.

Unlock the full coding experience! As a GitHub Sponsor, you gain exclusive access to the code behind this article—start learning and building today!

I'm available for contracting services and support. You can reach me at: https://www.czetsuyatech.com/p/consultation-services.html.

Spring Data JPA: Creating Dynamic Search Filters with Specifications

2025-03-04T15:27:00.003+08:00

1. Introduction

In this article, I will share an implementation that utilizes a common base class to streamline the process of implementing search functionality in Spring Data JPA using Specifications. By creating a reusable base class, you can efficiently manage and extend your search logic across multiple entities. This approach reduces redundancy and promotes cleaner, more maintainable code while allowing for dynamic query creation based on varying search criteria. Whether you're dealing with simple or complex filtering requirements, this solution will simplify your data querying layer in Spring applications.

2. Supported Operations

The following operations are supported:

greater than
less than
greater than or equal
less than or equal
equal
isnull
like

These operation will be listed as enum for easier reference.

3. Base Model

We will use a user entity to demonstrate the filter.

package com.czetsuyatech.persistence.entities;

import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Table;
import lombok.Data;

@Table(name = "user_profile")
@Entity
@Data
public class UserEntity extends BaseEntity {

  @Column(name = "first_name")
  private String firstName;

  @Column(name = "last_name")
  private String lastName;

  @Column(name = "role")
  private String role;

  @Column(name = "age")
  private int age;
}

4. Class / Flow Diagram

As we can see above we will need a query builder that accepts UserSpecification and UserFields to generate the user entity specification that we need to feed to the UserRepository extending SliceJpaRepository, which is implemented by SimpleSliceJpaRepositoryImpl.

5. Repositories and Services

5.1 Anatomy of the user repository

@Repository
public interface UserRepository extends SliceJpaRepository<UserEntity, Long> {

}

@NoRepositoryBean
public interface SliceJpaRepository<ENTITY, ID extends Serializable> extends JpaRepository<ENTITY, ID>,
    JpaSpecificationExecutor<ENTITY>, JpaSpecificationExecutorWithProjection<ENTITY> {

  Slice<ENTITY> findAllSlice(@Nullable Specification<ENTITY> specification, Pageable pageable);
}

5.2 Query Builder

The query builder accepts the available filterable parameters of the user entity and the specification which define how we are going to implement the filter using criteria builder.

public enum UserSearchFields {

  FIRSTNAME {
    @Override
    public String toString() {
      return "firstName";
    }
  },
  LASTNAME {
    @Override
    public String toString() {
      return "lastName";
    }
  },
  ROLE {
    @Override
    public String toString() {
      return "role";
    }
  },
  AGE {
    @Override
    public String toString() {
      return "age";
    }
  }
}

@AllArgsConstructor
public class UserSpecification implements Specification<UserEntity> {

  private transient SearchCriteria criteria;

  @Override
  public Predicate toPredicate(Root<UserEntity> root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder) {

    UserSearchFields key = UserSearchFields.valueOf(StringUtils.upperCase(criteria.getKey()));
    RelationalOperators operator = RelationalOperators.getOperator(criteria.getOperation());
    Object value = criteria.getValue();
    Expression<?> expression = root.<String>get(key.toString());

    Predicate predicate = null;

    switch (operator) {
      case LIKE -> predicate = buildLike(key, value, expression, criteriaBuilder);
      case EQUAL -> predicate = buildEqual(key, value, expression, criteriaBuilder);
      case GREATER, GREATER_THAN_EQUAL -> predicate = buildGreater(key, value, expression, criteriaBuilder);
      case LESS, LESS_THAN_EQUAL -> predicate = buildLess(key, value, expression, criteriaBuilder);
      default -> throw new IllegalStateException("Unexpected value: " + operator);
    }

    return predicate;
  }

  private Predicate buildLess(UserSearchFields key, Object value, Expression<?> expression,
      CriteriaBuilder criteriaBuilder) {

    return criteriaBuilder.lessThanOrEqualTo((Expression<Integer>) expression, Integer.parseInt(value.toString()));
  }

  private Predicate buildGreater(UserSearchFields key, Object value, Expression<?> expression,
      CriteriaBuilder criteriaBuilder) {

    return criteriaBuilder.greaterThanOrEqualTo((Expression<Integer>) expression, Integer.parseInt(value.toString()));
  }

  private Predicate buildEqual(UserSearchFields key, Object value, Expression<?> expression,
      CriteriaBuilder criteriaBuilder) {

    return criteriaBuilder.equal(expression, value);
  }

  private Predicate buildLike(UserSearchFields key, Object value, Expression<?> expression,
      CriteriaBuilder criteriaBuilder) {

    return criteriaBuilder.like((Expression<String>) expression,
        value.toString() + SpecificationConstant.LIKE_WILDCARD);
  }
}

5.3 The user service that invokes the repository.

@Service
@RequiredArgsConstructor
public class UserServiceImpl implements UserService {

  private final UserRepository userRepository;
  private final UserMapper userMapper;

  @Transactional(readOnly = true)
  @Override
  public Slice<UserDTO> findUsers(String searchParams, Pageable pageable) {

    try {
      Specification<UserEntity> specResult = QueryBuilder.build(searchParams, UserSpecification.class,
          UserSearchFields.class);

      return userRepository.findAllSlice(specResult, pageable).map(userMapper::userToUserDTO);

    } catch (Exception e) {
      e.printStackTrace();
    }

    return new SliceImpl<>(Collections.emptyList());
  }
}

6. Controller

And this is how we are going to call the service from the controller.

@RequestMapping("/users")
@RestController
@RequiredArgsConstructor
@Slf4j
public class UserController {

  private final UserService userService;

  @GetMapping
  public ResponseEntity<Slice<UserDTO>> findUsers(@RequestParam(name = "search", required = false) String search,
      @PageableDefault Pageable pageable) {

    log.debug("Search: " + search);

    return ResponseEntity.ok(userService.findUsers(search, pageable));
  }
}

7. Postman Tests

8. Development and Support

Unlock the full coding experience! As a GitHub Sponsor, you gain exclusive access to the code behind this article—start learning and building today!

I'm available for contracting services and support. You can reach me at: https://www.czetsuyatech.com/p/consultation-services.html.

How to Handle Daylight Saving Time Conversion in Java

2024-12-07T08:27:00.002+08:00

Learn how to handle daylight saving time (DST) conversions in Java with clear, practical code examples. This guide focuses on using the right classes and methods to accurately convert dates and times across DST boundaries. Perfect for developers looking to simplify time zone management in their applications.

Code

Version 1

public class TimeZoneConversion {
    public static void main(String[] args) {
        // Example UTC date-time
        // String utcDateTimeStr = "2024-11-01T00:00:00";
		String utcDateTimeStr = "2024-03-24T00:00:00";
        DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss");
 
        // Parse UTC date-time
        LocalDateTime utcDateTime = LocalDateTime.parse(utcDateTimeStr, formatter);
 
        // Convert to UTC ZonedDateTime
        ZonedDateTime utcZonedDateTime = utcDateTime.atZone(ZoneId.of("UTC"));
 
        // Convert to Netherlands time zone
        ZonedDateTime netherlandsTime = utcZonedDateTime.withZoneSameInstant(ZoneId.of("Europe/Amsterdam"));
 
        // Format and print the result
        System.out.println("UTC Time: " + utcZonedDateTime);
        System.out.println("Netherlands Time: " + netherlandsTime.format(formatter));
    }
}

Version 2

public class TimeZoneConversion {
    public static void main(String[] args) {
		TimeZone tz = TimeZone.getTimeZone("Europe/Amsterdam");
		TimeZone.setDefault(tz);
		Calendar cal = Calendar.getInstance(tz, Locale.ITALIAN);
		DateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm", Locale.ITALIAN);
		Date dateBeforeDST = df.parse("2018-03-25 01:55");
		cal.setTime(dateBeforeDST);
		cal.add(Calendar.MINUTE, 10);

		System.out.println(cal.get(Calendar.ZONE_OFFSET));
		System.out.println(cal.get(Calendar.DST_OFFSET));
	}
}

How to Create a Jasper Report from an XML Datasource: A Step-by-Step Guide

2024-06-09T16:37:00.006+08:00

1. Introduction

JasperReports is an open-source Java reporting tool that allows developers to create sophisticated reports for display, printing, or exporting into a variety of formats such as PDF, HTML, CSV, and others. It's widely used in Java-based applications for generating pixel-perfect reports with rich content like charts, tables, and images. JasperReports provides a flexible and customizable framework for designing and generating reports from various data sources, including databases, XML files, and custom data sources. It's often integrated into Java web applications and enterprise systems for generating dynamic and professional-looking reports.

2. Use Case

In this article, we will take a look at how Opencell an open-source billing platform integrates Jasper reports to generate its invoicing documents.

The goal of this exercise is to:

Generate a financial document (invoice) using Jasper reports.
Use a custom font.
Use an XML document.

Using XML as a data source in JasperReports can be ideal for several reasons:

Flexibility: XML is a highly flexible data format that can represent structured data in a hierarchical manner. This makes it suitable for a wide range of data sources, including relational databases, web services, and custom data structures.
Platform-agnostic: XML is platform-independent, meaning it can be easily generated and consumed by different programming languages and technologies. This makes it a versatile choice for integrating with various systems and applications.
Ease of Integration: Many applications and systems already generate or consume XML data, so using XML as a data source can simplify integration with existing infrastructure.
Customizability: XML allows developers to define custom data structures and schemas tailored to their specific reporting needs. This enables precise control over the data consumed by JasperReports and the structure of the resulting reports.
Separation of Concerns: Using XML as a data source promotes a clean separation between data and presentation layers. This separation allows for easier maintenance, scalability, and reuse of report templates across different datasets.
Performance: When properly optimized, XML processing can offer good performance characteristics for report generation. Techniques such as indexing, caching, and streaming can be employed to efficiently handle large XML datasets.

Overall, XML serves as a versatile and effective data source for JasperReports, offering flexibility, interoperability, and ease of integration with diverse data sources and systems.

3. Coding

We'll develop a Java application to produce a PDF invoice by utilizing a Jasper template sourced from an XML data provider.

3.1 XML Datasource

Our XML document is generated from Opencell that contains customer and usage information.

  <invoice customerAccountCode="CA_2000000021" customerId="CUST_2000000021" id="101"
    invoiceCounter="00000027" number="INV00000027" templateName="invoice" type="INV">
  <header>
    <provider code="CT" description="CZETSUYATECH">
      <bankCoordinates>
        <ics>FRXXZZZ123456</ics>
        <iban>FR763000600001123456790189</iban>
        <bic>CTDEFRPPCCT</bic>
      </bankCoordinates>
    </provider>
    <customer brand="" category="CLIENT" code="CUST_2000000021" externalRef1="" externalRef2=""
        id="565" jobTitle="" registrationNo="" sellerCode="CZETSUYATECH" vatNo="">
      <address>
        <address1/>
        <address2/>
        <address3/>
        <city/>
        <postalCode/>
        <state/>
        <country/>
        <countryName/>
      </address>
      <contact/>
    </customer>
    <seller code="CZETSUYATECH" description="CZETSUYATECH">
      <address>
        <address1/>
        <address2/>
        <address3/>
        <city/>
        <postalCode/>
        <state/>
        <country/>
        <countryName/>
      </address>
      <contact/>
    </seller>
    <customerAccount accountTerminated="false" code="CA_2000000021" currency="EUR"
        description="Edward's Account" externalRef1="" externalRef2="" id="566" jobTitle=""
        language="English" vatNo="xxx">
      <contact email="czetsuya@gmail.com" fax="" mobile="" phone="(303) 829-5342"/>
      <paymentMethod type="CHECK"/>
      <name>
        <quality/>
        <name>Edward's Account</name>
      </name>
      <address>
        <address1>1234 Balamb Garden</address1>
        <address2/>
        <address3/>
        <city>Los Banos</city>
        <postalCode>4030</postalCode>
        <state/>
        <country>PH</country>
        <countryName>Philippines</countryName>
      </address>
    </customerAccount>
    <billingAccount billingCycleCode="MONTHLY_POST_PAID" code="BA_2000000021"
        description="BA_2000000021" endPeriod="31/05/2024" externalRef1="" externalRef2="" id="567"
        jobTitle="" startPeriod="09/05/2024">
      <billingCycle code="MONTHLY_POST_PAID" description="Monthly Invoice Cycle" id="6">
      </billingCycle>
      <email>czetsuya@gmail.com</email>
      <name>
        <quality/>
        <name>Edward's Account</name>
      </name>
      <address>
        <address1>1234 Balamb Garden</address1>
        <address2/>
        <address3/>
        <city>Los Banos</city>
        <postalCode>4030</postalCode>
        <state/>
        <country>PH</country>
        <countryName>Philippines</countryName>
      </address>
      <message/>
      <billTo>
        <![CDATA[Edward's Account<br/>1234 Balamb Garden<br/>Los Banos, Laguna 4030<br/>PH]]>
      </billTo>
      <fees/>
      <charges licensePlate="NLEDW_LE2021"
          urlImage="https://czetsuyatech.com/PHEDW_LE2021.latest.jpg">
        <charge amount="-161.000000000000" date="05/28/24 19:45" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
        <charge amount="161.000000000000" date="5/9/24 9:29 AM" description="GoPlus"
            transactionId="566170"/>
        <charge amount="161.000000000000" date="5/10/24 7:52 AM" description="GoPlus"
            transactionId="566171"/>
      </charges>
      <topups/>
    </billingAccount>
    <invoiceDate>13/05/2024</invoiceDate>
    <dueDate>13/06/2024</dueDate>
    <paymentMethod>CHECK</paymentMethod>
    <comment>
    </comment>
    <categories>
      <category code="CLASSIC_USAGE" label="Service Usage">
        <amountWithoutTax>322.000000000000</amountWithoutTax>
        <subCategories>
          <subCategory amountWithoutTax="322.000000000000" code="CLASSIC_USAGE"
              label="Service Usage" taxCode="TAX_00" taxPercent="0.000000000000"/>
        </subCategories>
      </category>
    </categories>
    <discounts/>
    <locale>en-US</locale>
  </header>
  <amount>
    <currency>EUR</currency>
    <amountWithoutTax>322.000000000000</amountWithoutTax>
    <amountWithTax>322.000000000000</amountWithTax>
    <netToPay>322.000000000000</netToPay>
    <taxes total="0.000000000000">
      <tax code="TAX_00" id="1">
        <name>0 Percent Tax</name>
        <percent>0.000000000000</percent>
        <amount>0.000000000000</amount>
        <amountHT>322.000000000000</amountHT>
      </tax>
    </taxes>
  </amount>
  <detail>
    <userAccounts>
      <userAccount code="UA_2000000021" description="UA_2000000021" id="568" jobTitle="">
        <customFields>
          <customField code="CUSTOMER_NUMBER" description="Customer Number">1190017814</customField>
          <customField code="ACCOUNT_NUMBER" description="Account Number">2000000021</customField>
        </customFields>
        <subscriptions>
          <subscription code="SUBS_2000000021_NLEDW_LE2021_1590017830"
              description="SUBS_2000000021_NLEDW_LE2021_1590017830" id="309" offerCode="GoPlus">
            <subscriptionDate>13/10/2023</subscriptionDate>
            <endAgreementDate/>
          </subscription>
        </subscriptions>
        <name>
          <quality/>
          <firstName>Ed</firstName>
          <name/>
        </name>
        <address>
          <address1/>
          <address2/>
          <address3/>
          <city/>
          <postalCode/>
          <state/>
          <country/>
          <countryName/>
        </address>
        <categories>
          <category code="CLASSIC_USAGE" label="Service Usage">
            <amountWithoutTax>322.000000000000</amountWithoutTax>
            <subCategories>
              <subCategory amountWithoutTax="322.000000000000" code="CLASSIC_USAGE"
                  label="Service Usage" taxCode="TAX_00" taxPercent="0.000000000000">
                <line code="GoPlus_CT"
                    param1="fad4c36b-215a-308d-a19d-b6074543d414:fad4c36b-215a-308d-a19d-b6074543d414"
                    param2="1" param3="566170">
                  <walletOperation code="GoPlus_CT" periodEndDate="" periodStartDate=""/>
                  <paramExtra>USAGE</paramExtra>
                  <pricePlan code="GoPlus_CT" description="Price Plan for GoPlus CT"/>
                  <label>
                    GoPlus||EUR|1.0|0.00|161.00|161.00|0E-12|161.000000000000|161.000000000000|USE_TRX_PRICE
                  </label>
                  <unitAmountWithoutTax>161.000000000000</unitAmountWithoutTax>
                  <amountWithoutTax>161.000000000000</amountWithoutTax>
                  <quantity>1.000000000000</quantity>
                  <usageDate>09/05/2024</usageDate>
                  <edr accessCode="AP_2000000021_NLEDW_LE2021" dateParam1="" dateParam2=""
                      dateParam3="" dateParam4="" dateParam5="" decimalParam1="161.000000000000"
                      decimalParam2="" decimalParam3="" decimalParam4="" decimalParam5=""
                      eventDate="2024-05-09T09:29:46" originBatch="API_192.168.67.123"
                      originRecord="opencell.admin_1715247534640"
                      parameter1="fad4c36b-215a-308d-a19d-b6074543d414:fad4c36b-215a-308d-a19d-b6074543d414"
                      parameter2="1" parameter3="566170" parameter4="CHARGE_GoPlus_CT"
                      parameter5="VTOLL" parameter6="NLEDW_LE2021" parameter7=""
                      parameter8="USE_TRX_PRICE" parameter9="" quantity="1.000000000000"
                      rejectReason="" status="RATED"
                      subscription="SUBS_2000000021_NLEDW_LE2021_1590017830"/>
                </line>
                <line code="GoPlus_CT"
                    param1="fad4c36b-215a-308d-a19d-b6074543d414:fad4c36b-215a-308d-a19d-b6074543d414"
                    param2="1" param3="566171">
                  <walletOperation code="GoPlus_CT" periodEndDate="" periodStartDate=""/>
                  <paramExtra>USAGE</paramExtra>
                  <pricePlan code="GoPlus_CT" description="Price Plan for GoPlus CT"/>
                  <label>
                    GoPlus||EUR|1.0|0.00|161.00|161.00|0E-12|161.000000000000|161.000000000000|USE_TRX_PRICE
                  </label>
                  <unitAmountWithoutTax>161.000000000000</unitAmountWithoutTax>
                  <amountWithoutTax>161.000000000000</amountWithoutTax>
                  <quantity>1.000000000000</quantity>
                  <usageDate>10/05/2024</usageDate>
                  <edr accessCode="AP_2000000021_NLEDW_LE2021" dateParam1="" dateParam2=""
                      dateParam3="" dateParam4="" dateParam5="" decimalParam1="161.000000000000"
                      decimalParam2="" decimalParam3="" decimalParam4="" decimalParam5=""
                      eventDate="2024-05-10T07:52:12" originBatch="API_192.168.67.60"
                      originRecord="opencell.admin_1715327638732"
                      parameter1="fad4c36b-215a-308d-a19d-b6074543d414:fad4c36b-215a-308d-a19d-b6074543d414"
                      parameter2="1" parameter3="566171" parameter4="CHARGE_GoPlus_CT"
                      parameter5="VTOLL" parameter6="NLEDW_LE2021" parameter7=""
                      parameter8="USE_TRX_PRICE" parameter9="" quantity="1.000000000000"
                      rejectReason="" status="RATED"
                      subscription="SUBS_2000000021_NLEDW_LE2021_1590017830"/>
                </line>
              </subCategory>
            </subCategories>
          </category>
        </categories>
      </userAccount>
      <userAccount description="-">
        <categories/>
      </userAccount>
    </userAccounts>
  </detail>
  <offers>
    <offer code="GoPlus" description="GoPlus" id="1">
      <customFields>
        <customField code="CF_SHARED" description="Offer Shared">INDIVIDUAL</customField>
      </customFields>
    </offer>
  </offers>
  <services>
    <service code="CT" description="CT" offerCode="GoPlus"/>
  </services>
  <priceplans>
    <priceplan code="GoPlus_CT" description="Price Plan for GoPlus CT">
    </priceplan>
  </priceplans>
  <orders/>
  <qrCode
      value="xxx"/>
  <invoiceQRCodeEmail
      value="xxx"/>
  <summary currentBalance="-644.00" outstandingBalance="0.00" previousBalance="0.00"
      totalAmountDue="322.00"/>
</invoice>

As we can see the root of our document is "invoice" .

3.2 Jasper Dependencies

<dependency>
  <groupId>net.sf.jasperreports</groupId>
  <artifactId>jasperreports</artifactId>
  <version>6.21.0</version>
</dependency>
<dependency>
  <groupId>jaxen</groupId>
  <artifactId>jaxen</artifactId>
  <version>1.2.0</version>
</dependency>
<dependency>
  <groupId>org.apache.groovy</groupId>
  <artifactId>groovy</artifactId>
  <version>4.0.21</version>
</dependency>
<dependency>
  <groupId>net.sf.jasperreports</groupId>
  <artifactId>jasperreports-fonts</artifactId>
  <version>6.20.0</version>
</dependency>

3.3 Java Code

package com.czetsuyatech;

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.StringWriter;
import java.net.URLClassLoader;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import net.sf.jasperreports.engine.DefaultJasperReportsContext;
import net.sf.jasperreports.engine.JRException;
import net.sf.jasperreports.engine.JRParameter;
import net.sf.jasperreports.engine.JRPropertiesUtil;
import net.sf.jasperreports.engine.JasperExportManager;
import net.sf.jasperreports.engine.JasperFillManager;
import net.sf.jasperreports.engine.JasperPrint;
import net.sf.jasperreports.engine.JasperReport;
import net.sf.jasperreports.engine.data.JRXmlDataSource;
import net.sf.jasperreports.engine.util.JRLoader;
import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.xml.sax.SAXException;

public class InvoiceReportGenerator {

  private final String jasperReportsPath;
  private ClassLoader cl;
  private String INVOICE_TAG_NAME = "invoice";

  public InvoiceReportGenerator(String jasperReportsPath, URLClassLoader cl) {

    this.jasperReportsPath = jasperReportsPath;
    this.cl = cl;
  }

  public void generate(String jasperPath, String xmlDataSource)
      throws IOException, ParserConfigurationException, SAXException, TransformerException, JRException {

    File jasperFile = new File(jasperReportsPath, jasperPath);
    String pdfFullFilename = jasperReportsPath + "/output.pdf";
    File invoiceXmlFile = new File(jasperReportsPath, xmlDataSource);
    Map<String, Object> parameters = getParameters();

    InputStream reportTemplate = new FileInputStream(jasperFile);
    Node invoiceNode = getInvoiceNode(invoiceXmlFile);

    JRXmlDataSource dataSource = new JRXmlDataSource(getJasperReportContext(invoiceNode), "/" + INVOICE_TAG_NAME);

    Map<String, JasperReport> jasperReportMap = new HashMap<>();

    String fileKey = jasperFile.getPath() + jasperFile.lastModified();
    JasperReport jasperReport = jasperReportMap.get(fileKey);
    if (jasperReport == null) {
      jasperReport = (JasperReport) JRLoader.loadObject(reportTemplate);
      jasperReportMap.put(fileKey, jasperReport);
    }

    DefaultJasperReportsContext context = DefaultJasperReportsContext.getInstance();
    JRPropertiesUtil.getInstance(context).setProperty("net.sf.jasperreports.xpath.executer.factory",
        "net.sf.jasperreports.engine.util.xml.JaxenXPathExecuterFactory");

    JasperPrint jasperPrint = JasperFillManager.fillReport(jasperReport, parameters, dataSource);

    JasperExportManager.exportReportToPdfFile(jasperPrint, pdfFullFilename);
  }

  private Node getInvoiceNode(File invoiceXmlFile)
      throws TransformerException, ParserConfigurationException, IOException, SAXException {

    DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
    DocumentBuilder db = dbf.newDocumentBuilder();
    dbf.setNamespaceAware(true);
    Document xmlDocument = db.parse(invoiceXmlFile);
    xmlDocument.getDocumentElement().normalize();
    Node invoiceNode = xmlDocument.getElementsByTagName(INVOICE_TAG_NAME).item(0);
    Transformer trans = TransformerFactory.newInstance().newTransformer();
    trans.setOutputProperty(OutputKeys.INDENT, "yes");
    StringWriter writer = new StringWriter();
    trans.transform(new DOMSource(xmlDocument), new StreamResult(writer));

    return invoiceNode;
  }

  private ByteArrayInputStream getJasperReportContext(Node invoiceNode) throws TransformerException {

    return new ByteArrayInputStream(getNodeXmlString(invoiceNode).getBytes(
        StandardCharsets.UTF_8));
  }

  private Map<String, Object> getParameters() {

    return new HashMap<>() {{
      put(JRParameter.REPORT_CLASS_LOADER, cl);
    }};
  }

  protected String getNodeXmlString(Node node) throws TransformerException {

    TransformerFactory transFactory = TransformerFactory.newInstance();
    Transformer transformer = transFactory.newTransformer();
    StringWriter buffer = new StringWriter();
    transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
    transformer.transform(new DOMSource(node), new StreamResult(buffer));

    return buffer.toString();
  }
}

4. Resources

The complete source code is available at GitHub: https://github.com/czetsuyatech/jasperreports-xml-datasource

5. Development and Support

Unlock the full coding experience! As a GitHub Sponsor, you gain exclusive access to the code behind this article—start learning and building today!

I'm available for contracting services and support. You can reach me at: https://www.czetsuyatech.com/p/consultation-services.html.

Hands-on Coding: Spring Boot Common Exceptions Handling

2024-04-20T18:03:00.017+08:00

Overview

What is an exception?

An exception is an unexpected event, behavior, or state during software execution. In Java, it is a subclass of java.lang.Throwable.

In this diagram where a user withdraws money from an account the system throws an exception when the amount is less than or equal to the account balance.

Exception Class Diagram

In this diagram, we can see two types of Throwable.

Exception - recoverable
Error - unrecoverable

Exception VS RuntimeException

Exception - we need to declare the thrown exception in the method signature

public String readFromInputStream(InputStream inputStream) throws IOException {
  StringBuilder resultStringBuilder = new StringBuilder();
  try (BufferedReader br = new BufferedReader(new InputStreamReader(inputStream))) {
    String line;
    // BufferedReader.readLine throws IOException
    while ((line = br.readLine()) != null) {
      resultStringBuilder.append(line).append("\n");
    }
  }
  return resultStringBuilder.toString();
}

RuntimeException - there is no need to add the exception in the method signature

public void rtException() {
  int arr[] = null; // null array
  System.out.println("NullPointerException: " + arr.length);
}

In the calling block, we can catch the exception that was thrown.

try {
  readFromInputStream(is);
  rtException()
  
} catch (IOException ioe) {
  //
} catch (NullPointerException npe) {
  //  
}

Pre-Spring Boot 3

Before Spring Boot 3 there are several approaches on how we handle the exceptions. This article is opinionated so I will just list them for reference.

@ExceptionHandler annotation inside a controller
Implement HandlerExceptionResolver or extend AbstractHandlerExceptionResolver
Extending ResponseStatusException
Zalando exception handling library

Microservice

Microservice is a software architecture that builds products as collections of small services. Often, these services are powered by Spring Boot, and they interact with each other via HTTP calls through REST endpoints. Before Spring Boot 3, developers often used Zalando to wrap and handle exceptions, but Spring patched this gap and now provides an elegant solution.

https://microservices.io/patterns/microservices.html

In this diagram, we can see that the API gateway calls other services. And internal services can also call each other. During these calls it’s possible to encounter exceptions. That’s why it’s important to have a common library that we can add as a dependency and customize the errors based on the domain.

Commons Web Exceptions

Commons Web Exceptions is a project I created in Spring Boot 3, it is built around:

RestControllerAdvice - for centralizing the exceptions
ResponseEntityExceptionHandler - for providing the basic exception information structure
ErrorResponseException - for initializing an exception

In this image, all the classes and enums inside the blue block are part of the commons web exceptions library that we can use and extend in our service.

NativeWebExceptionEnumCodes - is an enum where native exceptions such as BAD_REQUEST, and INVALID_FORMAT are defined. In here, we can also add the class name of an exception in case we want to provide a specific code. Eg. HTTP_REQUEST_METHOD_NOT_SUPPORTED_EXCEPTION.
AbstractWebExceptions - is a container for all the native and service-defined exceptions that we can register by extending this class.
WebBaseException - a model class that extends ErrorResponseException. It provides the basic structure of the exception.
AbstractWebExceptionHandler - which extends the ResponseEntityExceptionHandler class. It gives us default handlers for the most common exceptions such as HttpRequestMethodNotSupportedException, HttpMediaTypeNotSupportedException, and HttpMediaTypeNotAcceptableException. In this class, we override some methods that will allow us to provide custom decorations to our exceptions. For example, when handling the invalid method argument exception we can list the errors (eg notNull) in the custom property "ERRORS".

Usage

In this section, I will provide some examples of how we can use this library.

Initializing the Library

Before we can define our custom exception handlers, we should extend the necessary base classes first.

1. Define the service's exception codes as enum. For example, in one of the services I have in Hivemaster, a custom Keycloak project that provides a multi-tenant feature.

@Getter
public enum AppExceptionCodes {

  USER_CREATION_FAILED("A1001", "Use creation failed"),
  USER_EID_NOT_FOUND("A1002", "User EID not found"),
  USER_EMAIL_NOT_FOUND("A1003", "User email not found"),
  USER_PHONE_NOT_FOUND("A1004", "User phone not found"),
  ORGANIZATION_NOT_FOUND("A1005", "Organization not found");

  private String code;
  private String message;

  AppExceptionCodes(String code, String message) {
    this.code = code;
    this.message = message;
  }

  public static Map<String, String> getMapValues() {

    Map<String, String> map = new LinkedHashMap<>();
    for (AppExceptionCodes errCode : values()) {
      map.put(errCode.getCode(), errCode.getMessage());
    }

    return map;
  }
}

2. Register the exception codes by extending the class AbstractWebExceptionCodes, so that they can be accessed by the commons-exception project.

@Component
public class WebExceptions extends AbstractWebExceptions {

  @Value("${spring.application.name}")
  private String serviceName;

  public WebExceptions() {

    super(HttpStatus.OK);

    registerExceptionMap(AppExceptionCodes.getMapValues());
  }

  @Override
  public String getServiceName() {
    return serviceName;
  }
}

3. Extend the WebBaseException class. So that we can handle business exceptions specific to the service. This class extends ErrorResponseException and RuntimeException which should be extended by the service exception classes. This will allow us to override the decoration that happens on the base WebBaseException class.

public class WebException extends WebBaseException {

  public WebException(HttpStatusCode status, String code) {
    this(status, code, null);
  }

  public WebException(HttpStatusCode status, String code, String message) {
    super(status, code, message);
  }
}

4. Extend the base exception handler AbstractWebExceptionHandler, which provides custom error handling and decoration for exceptions like method argument, runtime, invalid format, etc.

@Slf4j
@RestControllerAdvice
@RequiredArgsConstructor
public class WebExceptionHandler extends AbstractWebExceptionHandler {

  private final WebExceptions webExceptions;

  @Override
  public String getServiceName() {
    return webExceptions.getServiceName();
  }
}

5. And finally, import the library into your project.

@EnableConfigurationProperties
@SpringBootApplication
@Import({WebExceptionHandlerConfig.class})
public class Application {}

Use Cases

Once all of these are done, we can now begin customizing our exceptions.

1. Handling native exceptions such as method invalid argument.

Here are the possible results that we may get:

Using custom assertion

{
    "type": "http://localhost:8080/errors/S404",
    "title": "Bad Request",
    "status": 400,
    "detail": "Validation failed for fields (object:userV1, field:emailOrPhoneOnly, message:AssertTrue)",
    "instance": "/api/native-exceptions/method-arguments",
    "code": "S404",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:41:21.660650900Z",
    "errors": [
        "object:userV1, field:emailOrPhoneOnly, message:AssertTrue"
    ]
}

Missing field

{
    "type": "http://localhost:8080/errors/S404",
    "title": "Bad Request",
    "status": 400,
    "detail": "Validation failed for fields (object:userV1, field:organization, message:NotNull)",
    "instance": "/api/native-exceptions/method-arguments",
    "code": "S404",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:43:25.418080500Z",
    "errors": [
        "object:userV1, field:organization, message:NotNull"
    ]
}

Invalid format

{
    "type": "http://localhost:8080/errors/S401",
    "title": "Bad Request",
    "status": 400,
    "detail": "Invalid format for (field: birthdate, value: xxx, type: Instant)",
    "instance": "/api/native-exceptions/method-arguments",
    "code": "S401",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:43:44.065740700Z",
    "errors": [
        "field:birthdate, value:xxx, type:Instant"
    ]
}

Invalid date

{
    "type": "http://localhost:8080/errors/S404",
    "title": "Bad Request",
    "status": 400,
    "detail": "Validation failed for fields (object:userV1, field:birthdate, message:Past)",
    "instance": "/api/native-exceptions/method-arguments",
    "code": "S404",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:44:28.829450600Z",
    "errors": [
        "object:userV1, field:birthdate, message:Past"
    ]
}

Missing resource

{
    "type": "http://localhost:8080/errors/S405",
    "title": "Not Found",
    "status": 404,
    "detail": "No static resource native-exceptions/resource-not-found.",
    "instance": "/api/native-exceptions/resource-not-found",
    "code": "S405",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:44:44.294238700Z",
    "errors": []
}

Unsupported method

{
    "type": "http://localhost:8080/errors/S402",
    "title": "Method Not Allowed",
    "status": 405,
    "detail": "Method 'PUT' is not supported.",
    "instance": "/api/native-exceptions/method-arguments",
    "code": "S402",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:45:00.240704900Z",
    "errors": []
}

Forbidden

{
    "type": "http://localhost:8080/errors/S501",
    "title": "Forbidden",
    "status": 403,
    "instance": "/api/native-exceptions/forbidden",
    "code": "S501",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:45:20.566494300Z",
    "errors": []
}

Exception defined in our service exception enum which could be business, application, or entity.

{
    "type": "http://localhost:8080/errors/A1001",
    "title": "Use creation failed",
    "status": 400,
    "instance": "/api/service-exceptions/users/exceptions",
    "code": "A1001",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:55:49.061434900Z",
    "errors": []
}

Custom exception

{
    "type": "http://localhost:8080/errors/B1001",
    "title": "Business exception",
    "status": 400,
    "detail": "Custom exception message",
    "instance": "/api/service-exceptions/users/custom-exceptions",
    "code": "B1001",
    "service": "commons-web-exception-client",
    "timestamp": "2024-04-20T09:56:30.612646200Z",
    "errors": []
}

Git Repository

Once again the repositories are available in GitHub.

Development and Support

Unlock the full coding experience! As a GitHub Sponsor, you gain exclusive access to the code behind this article—start learning and building today!

I'm available for contracting services and support. You can reach me at: https://www.czetsuyatech.com/p/consultation-services.html.

Hands on Coding: Spring Logging for Beginners

2024-04-06T17:36:00.010+08:00

1. Overview

Logging is a vital aspect of programming for both beginners and experts, often underestimated but crucial for understanding application behavior. Strategic placement of logging statements aids in debugging and comprehending program execution, especially in production environments.

This tutorial is hands-on so I'll just paste the reference for you: https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.logging

2. Hands-On Coding

Logging is available out-of-the-box from Spring, but we can customize its level, format, etc as defined in the document above.

I created a Spring Boot project to capture the behavior of the logging levels that you can use as a reference as you code.

2.1 Common Logging Interface

Let's introduce an interface that we will implement with different logging-level classes. The method logLevels will print all the logs applicable to a particular package.

package com.czetsuyatech.logger;

import org.slf4j.Logger;

public interface LoggerComponent {

  default void logLevels() {
    System.out.println(getClass().getPackageName() + ": " + "-".repeat(50));

    getLogger().info("Hello World");
    getLogger().debug("Hello World");
    getLogger().warn("Hello World");
    getLogger().trace("Hello World");
    getLogger().error("Hello World");
  }

  Logger getLogger();
}

2.2 Configure Package Names

We will define the following packages.

debug
error
info
trace
warn

For each package, we need to define a concrete class of interface LoggerComponent

For example, in the debug we will have something like:

@Component
@Slf4j
public class DebugLogger implements LoggerComponent {

  @Override
  public Logger getLogger() {
    return log;
  }
}

2.3 Spring Boot Application Class

We will inject the instances of LoggerComponent in a list so that we can iterate through each component. We will call the logLevels method to print the log.

@SpringBootApplication
@RequiredArgsConstructor
public class Application {

  private final List<LoggerComponent> loggers;

  public static void main(String[] args) {
    SpringApplication.run(Application.class, args);
  }

  @EventListener
  public void onStartup(ContextRefreshedEvent event) {

    loggers.stream().forEach(LoggerComponent::logLevels);
  }
}

2.4 Spring Configuration File

And finally, we will configure the log level per package.

logging:
  level:
    root: info
    com.czetsuyatech.logger.info: info
    com.czetsuyatech.logger.debug: debug
    com.czetsuyatech.logger.warn: warn
    com.czetsuyatech.logger.trace: trace
    com.czetsuyatech.logger.error: error

We should have the following logs in our console which should pretty much explain the logging behavior. For example, error level only prints errors, while debug prints info, warn, and error as well.

3. When to Use Each Logger

There is no standard on how and when to use the different logging levels. In my own experience, this is how I use them. Coupled with proper package structure, so I can control the logging level of a particular process.

Info

- Informational messages without state such as component startup and configuration settings.

Warn

- Informational state that can be ignored by the system.

- Example: An unknown status of an entity.

Debug

- Informational events with state that are useful for developers. I use this in public methods.

- Example: Controller endpoints and service methods.

Trace

- Informational events with state that are useful for developers. I use this in private methods.

- Example: Private methods in a service.

Error

- Event that causes the application to get into an error state. Recoverable.

- Example: Unable to communicate to an external service. Having a mechanism to retry sending the message.

Fatal

- Event that causes the application to get into an error state. Unrecoverable.

- Example: Database instance going down.

4. Code Repository

The code is available at https://github.com/czetsuya/lab-spring-logging.

Hands on Coding: Spring Metrics with Prometheus for Beginner

2024-04-06T10:53:00.007+08:00

1. Overview

Welcome to our hands-on guide where we'll delve into the world of monitoring Spring Boot applications using Prometheus and Grafana. In today's fast-paced digital landscape, ensuring the smooth operation and performance of our applications is paramount. With the powerful combination of Prometheus and Grafana, we can gather insightful metrics and visualize them in a meaningful way, allowing us to monitor and optimize our Spring Boot applications effectively.

In this guide, we'll walk through the process step-by-step, covering everything you need to know to set up basic monitoring for your Spring Boot 3 application with Docker.

2. Spring Boot 3 Application

The Spring Boot project contains the basic configuration needed for demonstration.

2.1 Dependencies

We need the actuator to expose the Spring metrics and use the micrometer registry to convert it to Prometheus.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

2.2 Configuration

In this section, we allow health and Prometheus endpoints from the actuator. By default, the actuator endpoint is/actuator, which lists all the enabled endpoints.

logging:
  level:
    root: info

server:
  port: 8080

spring:
  application:
    name: lab-spring-prometheus

management:
  endpoints:
    web:
      exposure:
        include: health, prometheus
  endpoint:
    health:
      show-details: always
  prometheus:
    metrics:
      export:
        enabled: true

3. Prometheus

In Prometheus configuration, we need to specify the metrics path and set some labels that we will need in our Grafana dashboard later.

Note: If you want to scrape data from a Spring Boot app that runs on your IDE, you must change the targets to host.docker.internal. Don't forget to update the extra_hosts in the docker-compose file as well.

scrape_configs:
  - job_name: 'lab-spring-prometheus'
    metrics_path: '/actuator/prometheus'
    scrape_interval: 5s
    static_configs:
      #      - targets: [ 'host.docker.internal:8080' ]
      - targets: [ 'spring-app:8080' ]
        labels:
          namespace: czetsuyatech
          application: 'lab-spring-prometheus'

4. Grafana

4.1 Spring Boot Statistics

If you want a Spring Boot statistics dashboard out of the box you may check the following plugin in Grafana's marketplace. Or simply, import them.

19004 - https://grafana.com/grafana/dashboards/19004-spring-boot-statistics/
11378 - https://grafana.com/grafana/dashboards/11378-justai-system-monitor/

For Grafana, we can start with a basic data source configuration. This points to the Prometheus URL we define in docker-compose. Note that you can also do this in the Grafana user interface.

Grafana's default username and password is "admin".

apiVersion: 1
datasources:
  - name: Lab Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true

5. Docker Configuration

5.1 Dockerfile

We are loading the Spring Boot jar to our docker container. Make sure to build the project first in your IDE, so that the jar is generated.

FROM eclipse-temurin:17-jdk-alpine
VOLUME /tmp
COPY ./target/*.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

5.2 Docker Compose File

This configuration runs our Spring boot app, Prometheus, and Grafana in docker with a common network. Thus, each service can access each other using the service name. If you want to access a non-docker component, let's say the Spring Boot app, you need to do the necessary configuration with the host.

version: '3.8'

networks:
  backend:

services:
  spring-app:
    build:
      dockerfile: docker/Dockerfile
      context: ../
    ports:
      - "8080:8080"
    networks:
      - backend

  prometheus:
    image: prom/prometheus:v2.51.1
    container_name: lab-prometheus
    restart: no
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    networks:
      - backend
#    extra_hosts:
#      - 'host.docker.internal:host-gateway'

  grafana:
    image: grafana/grafana
    container_name: lab-grafana
    restart: no
    ports:
      - "3000:3000"
    volumes:
      - ./grafana/datasources:/etc/grafana/provisioning/datasources
    networks:
      - backend

To run, simply go to docker folder of this project and execute: docker-compose up --build

6. GitHub Repository

The code is available in GitHub including the docker-compose file.

https://github.com/czetsuyatech/lab-spring-prometheus

7. Development and Support

Unlock the full coding experience! As a GitHub Sponsor, you gain exclusive access to the code behind this article—start learning and building today!

I'm available for contracting services and support. You can reach me at: https://www.czetsuyatech.com/p/consultation-services.html.

How to Transfer Money from Coins PH to Binance

2024-03-24T09:27:00.007+08:00

1. Introduction

Worried about the risk of falling victim to scams while engaging in peer-to-peer (P2P) transactions for buying and selling coins? While there are several strategies to mitigate this risk, it's important to note that transfer fees can often be costly. One platform I've found to have reasonable transfer fees is ARBITRUM. For this exercise, let's consider transferring a USDT coin from Coins PH to Binance.

For this exercise, you need a Binance and Coins PH accounts.

2. Preferring for Transfer - Binance

We need to get the USDT wallet address in preparation for the transfer.

2.1 From the menu, select Spot.

2.2 Find the USDT coin, click the "..." menu, and select "Deposit."

2.3 Select the ARBITRUM network and copy the deposit address.

3. Making the Transfer in Coins PH

3.1 Go to your account portfolio. Find USDT and select Send USDT.

3.2 Select the Arbitrum network to match what we selected from Binance.

3.3 Specify the amount that you want to send.

3.4 You will be presented with a list of checklists, once verified press "I Understand".

3.5 In the next screen, you will be asked to confirm your transaction details.

3.6 The next step would be for you to validate the transaction by entering the security code sent via email.

3.7 The transaction will be processed, and you should receive a successful message after a minute or two.

This process works the opposite way (Binance -> Coins PH).

4. Disclaimer

The transaction fee is subject to change without prior notice. Therefore, verifying the current rate beforehand and conducting a test transfer before proceeding with a large amount is essential.

Hands-On Coding: Exploring Hyperparameters for Programmers

2024-03-07T08:44:00.005+08:00

Introduction

In this article, we will explore different techniques for finding the optimal hyperparameter values from a given set of parameters in a grid. Particularly we will look at RandomizedSearchCV, GridSearchCV, and BayesSearchCV.

In this blog you will learn:

How to initialize the parameter grid.
How to find the optimal hyperparameters based on a given technique.
How to build a model (XGBClassifier) to use the hyperparameters.
How to score the performance of the model.

RandomizedSearchCV

param_grid = {
    "gamma": [0, 0.1, 0.2, 0.5, 1, 1.5, 2, 3, 6, 12, 20],
    "learning_rate": [0.01, 0.02, 0.03, 0.05, 0.1, 0.2, 0.3, 0.5, 0.7, 0.8],
    "max_depth": [1, 2, 3, 4, 5, 6, 8, 12],
    "n_estimators": [25, 50, 65, 80, 100, 115, 200]
}

grid_search = RandomizedSearchCV(estimator=classifier_0, param_distributions=param_grid, scoring=scoring)

GridSearchCV

param_grid = {
    "gamma": [0, 0.1, 0.2, 0.5, 1, 1.5, 2, 3, 6, 12, 20],
    "learning_rate": [0.01, 0.02, 0.03, 0.05, 0.1, 0.2, 0.3, 0.5, 0.7, 0.8],
    "max_depth": [2, 3, 4, 5, 6, 8, 12],
    "n_estimators": [25, 50, 65, 80, 100, 115, 200]
}

grid_search = GridSearchCV(estimator=classifier_0, param_grid=param_grid, scoring=scoring)

BayesSearchCV

param_bayes = {
    'gamma': Categorical(param_grid['gamma']),
    'learning_rate': Categorical(param_grid['learning_rate']),
    'max_depth': Categorical(param_grid['max_depth']),
    'n_estimators': Categorical(param_grid['n_estimators'])
}

grid_search = BayesSearchCV(estimator=classifier_0, search_spaces=param_bayes, scoring=scoring, n_jobs=-1, cv=10)

Finding the Best HyperParameters

best_model = grid_search.fit(X_train, y_train)
hyperparams = best_model.best_params_

Building and Scoring the Classifier using the HyperParameters

# Fitting the Model
ne = hyperparams['n_estimators']
lr = hyperparams['learning_rate']
md = hyperparams['max_depth']
gm = hyperparams['gamma']
print("Recommended Params >>", f"ne: {ne},", f"lr: {lr}", f"md: {md}", f"gm: {gm}")

# Build Classification Model
classifier_1 = XGBClassifier(
    base_score=0.5,
    colsample_bylevel=1,
    colsample_bynode=1,
    objective=objective,
    booster="gbtree",
    eval_metric=eval_metric_list,
    n_estimators=ne,
    learning_rate=lr,
    max_depth=md,
    gamma=gm,
    subsample=0.8,
    colsample_bytree=1,
    random_state=1
)

# Fit Model
eval_set = [(X_train, y_train)]
classifier_1.fit(
    X_train,
    y_train,
    eval_set=eval_set,
    verbose=False
)

# Get predictions for training data
train_yhat = classifier_1.predict(X_train)
print("Training Preds: \n", train_yhat[:5])

# Set K-Fold Cross Validation Levels
cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=1)

# Training Results
train_results = cross_val_score(classifier_1, X_train, y_train, scoring=scoring, cv=cv, n_jobs=1)

# Brief Review of Training Results
print("Average Accuracy K-Fold: ", round(train_results.mean(), 2))
print("Std Deviation K-Fold: ", round(train_results.std(), 2))
print("Precision Score 0: ", round(precision_score(y_train, train_yhat, average=None)[0], 3))
print("Precision Score 1: ", round(precision_score(y_train, train_yhat, average=None)[1], 3))

Performance

Machine: Laptop

Processor: AMD Ryzen 7

OS: Windows

DataFrame Shape: (7282, 17)

Technique	Time (s)	Avg Accuracy K-Fold	Std Deviation K-Fold	Precision Score: 0	Precision Score: 1
RandomizedSearchCV	17.02	0.54	0.07	0.576	0.601
BayesSearchCV	105.39	0.52	0.05	0.589	0.568
GridSearchCV	9413.60	0.53	0.06	0.605	0.623

Understanding How Scope Affects Values in Your Spring REST Controller

2024-03-02T11:29:00.002+08:00

Below we explore how a scope annotation affects an instance value in a Spring REST controller.

Each controller is annotated with scope.

@RestController
@Scope([SCOPE_VALUE])
public class XXXScopeController {}

How to Convert Vertically Stored Asset Data into Columnar Format for Cointegration Analysis

2024-02-24T17:05:00.002+08:00

Introduction

This piece of code fetches asset information from a table stored vertically.

Dependencies

Install the following package.

conda install pandas
conda install numpy as np
conda install mysql-connector-python
conda install sqlalchemy
conda install pymysql

Hands-on Coding

Connect to the database

def query_df(query):
    try: 
        engine_uri = f"mysql+pymysql://db_user:db_pass_123@localhost:3306/tradewise_pse"
        db_conn = create_engine(engine_uri)        
        df_result = pd.read_sql(query, db_conn)    
        return df_result
        
    except Exception as e:    
        print(str(e))

Fetching the Dataset

if not load_existing:
    sql_distinct_tickers = "select ticker from candlestick where event_time='2023-12-29' and ticker not like '^%%'"
    df_tickers = query_df(sql_distinct_tickers)
    
    df = pd.DataFrame(index=['event_time'])
    
    ### Get the candlesticks
    for ticker in df_tickers['ticker']:
        sql_ticker_col = "select event_time, close from candlestick where ticker='{0}'"
        df_temp = query_df(sql_ticker_col.format(ticker))
        df_temp.set_index('event_time', inplace=True)
        df_temp.rename(columns={'close': ticker}, inplace=True)        
        df = df.add(df_temp, fill_value=0)
        
    df.to_csv(file_name)

Load the dataset from file

df = pd.read_csv(file_name, index_col=0)
df.drop(index=df.index[-1],axis=0, inplace=True)

Drop NA

df.dropna(axis=1, inplace=True)

Print the dataset

print(f"Shape: {df.shape}")
print(f"Null values: {df.isnull().values.any()}")
df

Save to a file

df.to_csv(file_name)

This procedure is in preparation for cointegration testing.

Learn to Incorporate Rolling Hurst Values into Your DataFrame

2024-02-24T16:38:00.002+08:00

The hurst function.

def hurst(ts, min_lag=1, max_lag=7):
    lags = range(min_lag, max_lag)
    tau = [np.sqrt(np.std(np.subtract(ts[lag:], ts[:-lag]))) for lag in lags]
    poly = np.polyfit(np.log(lags), np.log(tau), 1)
    return poly[0]*2.0

Adding to our DataFrame

The hurst value is computed with the last 14 close values.

df['Hurst'] = df['close'].rolling(14).apply(hurst, raw=True)
df[10:20]

Implementing Glowroot: A Hands-On Tech Review for Application Monitoring Mastery

2024-02-01T15:48:00.006+08:00

1. Introduction

In the realms of information technology and systems management, Application Performance Management (APM) involves monitoring and overseeing the performance and availability of software applications. APM aims to identify and diagnose intricate application performance issues to uphold a predefined level of service.

Glowroot is an open-source APM that facilitates a quicker resolution of application performance issues by helping us pinpoint the root causes. It supports applications running from Java 6 onwards.

2. Key Features

Response Time Breakdown Charts: Visualizes the breakdown of response times for better analysis.
Response Time Percentile Charts: Offers percentile charts to understand response time distribution.
SQL Capture and Aggregation: Captures and aggregates SQL queries for in-depth analysis.
Service Call Capture and Aggregation: Gathers and aggregates data on service calls for comprehensive insights.
MBean Attribute Capture and Charts: Monitors and charts MBean attributes for performance evaluation.
Configurable Alerting: Allows users to configure alerts based on specific criteria.
Historical Rollup: Provides historical data rollup at different intervals (1m, 5m, 30m, 4h) with configurable retention settings.
Full Support for Async Requests: Supports asynchronous requests that span multiple threads.
Responsive UI with Mobile Support: User-friendly and responsive interface with mobile support for accessibility.
Optional Central Collector: Offers the flexibility of an optional central collector for centralized data management.
Supports Multiple Application Servers: Wildfly, JBoss EAP, Tomcat, TomEE, Jetty, Glassfish, Payara, WebLogic, WebSphere

3. Central Collector

The central collector collects runtime information from the registered services and offers a GUI for easy viewing. https://github.com/glowroot/glowroot/wiki/Central-Collector-Installation

3.1 Installation

You can follow the steps above for running the GlowRoot central collector either as a standalone or as a docker image. For this section, I'll share how it can be run locally.

Here's my docker-compose file for running glowroot-central with cassandra. Override username, password, and contactPoints in glowroot-central.properties.

  version: '3.8'

networks:
  tradewise-network:

services:
  cassandra:
    image: cassandra:latest
    container_name: cassandra
    restart: unless-stopped
    ports:
      - "9042:9042"
    networks:
      - tradewise-network

  glowroot-central:
    image: glowroot/glowroot-central:0.14.1
    container_name: glowroot-central
    restart: unless-stopped
    volumes:
      - ./glowroot-central.properties:/usr/share/glowroot-central/glowroot-central.properties
    depends_on:
      - cassandra
    ports:
      - "4000:4000"
      - "8181:8181"
    networks:
      - tradewise-network

4. Instrumentation

There are 4 ways in which we can integrate GlowRoot into our services https://glowroot.org/instrumentation.html. For this exercise, we will focus on using the agent API.

4.1 Editing the pom.xml File

In our Spring Boot project's pom.xml file, add the GlowRoot agent configuration.

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-resources-plugin</artifactId>
	<executions>
		<execution>
			<id>glowroot-plugins</id>
			<phase>validate</phase>
			<goals>
				<goal>copy-resources</goal>
			</goals>
			<configuration>
				<outputDirectory>target/glowroot-plugins/</outputDirectory>
				<resources>
					<resource>
						<directory>glowroot-plugins</directory>
						<filtering>false</filtering>
					</resource>
				</resources>
			</configuration>
		</execution>
	</executions>
</plugin>

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-dependency-plugin</artifactId>
	<version>${maven-dependency-plugin.version}</version>
	<executions>
		<execution>
			<id>copy-glowroot-jar</id>
			<phase>prepare-package</phase>
			<goals>
				<goal>copy</goal>
			</goals>
		</execution>
	</executions>
	<configuration>
		<artifactItems>
			<artifactItem>
				<groupId>org.glowroot</groupId>
				<artifactId>glowroot-agent</artifactId>
				<version>${glowroot-agent.version}</version>
				<type>jar</type>
				<overWrite>false</overWrite>
				<outputDirectory>${project.build.directory}</outputDirectory>
				<destFileName>glowroot.jar</destFileName>
			</artifactItem>
		</artifactItems>
	</configuration>
</plugin>

4.2 Preparing the Dockerfile

We need to add the GlowRoot files in the docker image.

# Base Image
FROM eclipse-temurin:17-jdk-alpine
LABEL author=CzetsuyaTech
LABEL maintainer=CzetsuyaTech

# Configuration
WORKDIR /

RUN addgroup --system czetsuyatech && \
    adduser --system czetsuyatech --ingroup czetsuyatech && \
    mkdir -p /glowroot /glowroot/tmp /glowroot/logs /glowroot/plugins && \
    echo '{ "web": { "bindAddress": "0.0.0.0" } }' > /glowroot/admin.json && \
    chown czetsuyatech:czetsuyatech -R /glowroot && \
    chmod -R 777 /glowroot

USER czetsuyatech
ADD --chown=czetsuyatech:czetsuyatech target/glowroot.jar /glowroot
ADD --chown=czetsuyatech:czetsuyatech target/glowroot-plugins /glowroot/plugins

# Service
ADD --chown=czetsuyatech:czetsuyatech target/*.jar app.jar

# Start
ENV JAVA_JAR "/app.jar"
ENV JAVA_OTHERS "-Xshare:off"
ENV JAVA_OPTS ${JAVA_OPTS}
ENV JAVA_MEM ${JAVA_MEM}

RUN echo "exec java $JAVA_MEM $JAVA_OPTS $JAVA_OTHERS -jar $JAVA_JAR"
ENTRYPOINT exec java $JAVA_MEM $JAVA_OPTS $JAVA_OTHERS -jar $JAVA_JAR

4.3 Running the Spring Boot Service

To enable instrumentation in our instance, we need to specify the javaagent property. And to send information to the central collector we need to specify the central collector and give our instance an agent id.

JAVA_OPTS=-javaagent:glowroot/glowroot.jar -Dglowroot.collector.address=localhost:8181 -server -Dglowroot.agent.id=TradewiseAI

5. GUI

5.1 Usage

5.2 Transactions

5.2.1 Web

To assess our REST Endpoints' performance, we access the "Web" section. In the provided example:

By choosing "Response Time," we can identify which HttpRequests and JDBC Queries are taking longer in the REST Endpoints.
Opting for "Slow Traces" allows us to pinpoint the specific endpoint that consumes more time.
Selecting "Queries" reveals insights into the queries made, indicating that using the count query is more resource-intensive compared to the select query. This information aids in optimizing and refining the performance of REST Endpoints.

5.2.2 Background

To analyze our background performance, we navigate to the "Background" section. In the given example:

Choosing "Response Time" allows us to identify which Job and Hibernate Queries are consuming more time in the background processes.
Opting for "Slow Traces" reveals that some calls take more time, providing insights into areas that may require attention.
Selecting "Queries" and filtering by the select query, we observe that it is called more frequently and takes longer, particularly when filtered by status. This information helps in understanding and addressing potential bottlenecks in the background processes.

5.2.3 Startup

To assess our startup performance, we can utilize the "Startup" section. In the provided example:

By choosing "Response Time," we can identify which startup and filter init processes consume more time.
Opting for "Slow Traces" reveals the duration it takes for the context to initialize fully.
Selecting "Queries" provides insights into the database queries. Some queries will be more resource intensive, with fewer calls but longer duration. This information aids in pinpointing specific areas for optimization within the startup processes.

5.3 Errors

5.3.1 Web

To visualize errors in our REST endpoints, we can navigate to the "Web" section. In the following example:

Choosing "Error Messages" reveals errors of type XXXException.
Opting for "Error Traces" provides details on the errors.
Clicking on a specific error allows us to view the detailed trace, aiding in the understanding and resolution of the issue.

5.4 JVM

In our services' JVM, we can monitor and analyze memory status. This allows us to gain insights into the memory usage patterns, allocations, and overall health of the Java Virtual Machine, aiding in the effective management and optimization of our services.

5.5 MBeanTree

The MBeanTree functionality in Glowroot proves invaluable in monitoring instance creation. For instance, to track thread-related metrics and identify potential Thread Leaks, users can navigate to the java.lang section, specifically under Threading. This allows for a detailed examination of thread-related information, aiding in the identification and resolution of potential thread-related issues, such as Thread Leaks.

6. Recommendation

In a typical product infrastructure, Glowroot serves as our APM tool in each microservice. All microservices are instrumented to gather and transmit data to Glowroot, enhancing our understanding of system behavior. We've chosen Glowroot based on various criteria, including its license-free nature, alignment with the Java ecosystem, and straightforward instrumentation for microservices, ensuring a simple ramp-up and configuration process.

Navigating the Code: A Guide to Environment Variables, Configuration, and Feature Flags

2023-12-14T08:00:00.003+08:00

Determining the optimal location for storing essential information crucial for an application's functionality necessitates thoughtful planning. This guide aims to assist you in making informed decisions about where to store crucial data. Here, the term "application" encompasses any software created with code and deployed on various platforms and runtimes, irrespective of the underlying software architecture.

A fundamental principle is to avoid hard-coding any values within your application. Instead, adopt the practice of retrieving all configuration settings from a distinct storage system, separate from the deployment environment. This approach allows for seamless modifications to configuration settings even after the application has been deployed, eliminating the need for redeployment. This separation ensures greater flexibility and facilitates the adjustment of critical parameters without disrupting the operational state of your deployed application.

When considering the storage location for key/value pairs, it's crucial to evaluate the volatility of each pair—how frequently and by whom it might undergo changes. Reflect on the dynamic nature of the data and the responsibilities associated with its modification. This thoughtful approach will guide you in choosing an appropriate storage solution that aligns with the specific needs and characteristics of each key/value pair. By understanding the frequency and ownership of potential changes, you can make informed decisions that optimize the efficiency and maintainability of your data storage strategy.

Environment Variables

Environment variables, characterized by their infrequent changes, typically occur only once per deployment. Common examples include sensitive information like usernames and passwords for databases, which are often stored securely in a Vault. Additionally, environment variables encompass essential details required for the system's initialization, such as environment specifics and the connection details for the application configuration store, which hosts additional settings. By utilizing environment variables for these foundational aspects, you ensure a secure and stable system setup while centralizing critical configuration information.

Application Configuration

Application configuration, in contrast to environment variables, operates independently of deployments. It encompasses supplementary settings for the system that have the potential to change between deployments at runtime. This flexibility allows for real-time adjustments to the application's behavior and features, accommodating evolving requirements without the need for redeployment. By leveraging application configuration for these dynamic settings, you establish a modular and adaptable structure, enhancing the responsiveness of your system to changing runtime conditions.

Runtime User Settings

Runtime user settings represent the most volatile category, as users can alter any key/value pair at any given moment. Therefore, the system must be designed to accommodate this high level of dynamism. Options for handling such volatility include implementing robust real-time validation mechanisms and ensuring the integrity and security of user-initiated changes. Additionally, the system should provide an intuitive user interface for managing these settings, enabling seamless customization while maintaining overall system stability and security.

Here are three options for handling configuration updates:

Re-read Configuration All the Time: Continuously monitor and re-read the configuration, ensuring that the system remains up-to-date with any changes. This approach provides real-time responsiveness to modifications but may impose a continuous processing overhead.
Re-read Configuration at Set Intervals (e.g., Every 15 Minutes): Implement a periodic schedule to re-read the configuration at predefined intervals, such as every 15 minutes. This approach balances responsiveness with reduced processing overhead, making it suitable for scenarios where near-real-time updates are acceptable.
Receive Push Notifications When New Application Configuration Is Available: Set up a push notification mechanism to alert the system when new application configuration becomes available. This approach minimizes the need for continuous or scheduled re-reading, optimizing efficiency by updating the system only when changes occur. However, it requires a reliable notification infrastructure. Choosing among these options depends on the specific requirements of your system, considering factors such as the criticality of real-time updates, resource constraints, and the overall responsiveness desired for configuration changes.

Navigating Software and Systems Architecture: A Curated Collection of Resources

2023-12-11T18:18:00.007+08:00

These books serve as valuable references for building software systems:

Martin Fowler: Patterns of Enterprise Application Architecture

Robert C. Martin: Clean Architecture: A Craftsman's Guide to Software Structure and Design

Martin Kleppmann: Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

Len Bass, Paul Clements, Rick Kazman: Software Architecture in Practice

Gregor Hope: Enterprise Integration Patterns

Erich Gamma: Design Patterns

Chris Richardson: Microservice Patterns

Mark Richards: Fundamentals of Software Architecture

Mark Masse: REST API Design Rulebook: Designing Consistent RESTful Web Service Interfaces and here

Matthew Skelton: Team Topologies

Svyatoslav Kotusev: The Practice of Enterprise Architecture: A Modern Approach to Business and IT Alignment

Websites

https://microservices.io - online version of Chris Richardsons' book
https://www.enterpriseintegrationpatterns.com/patterns/messaging - online version of Gregor Hopes book

Personalized Spring Boot Startups: Custom Banners and Build Information

2023-09-07T20:42:00.002+08:00

1. Overview

By default, when a Spring Boot application starts, it displays a banner. In this article, we will explore the process of creating a custom banner and utilizing it within Spring Boot applications.

2. Creating a Banner

Before we begin, it's essential to generate the custom banner, which will appear during the application startup process. You can choose to create the custom banner from scratch or leverage various tools designed for this purpose.

In this example, we are generating CzetsuyaTech's logo from https://devops.datenkollektiv.de/banner.txt/index.html.

Create a new file "banner.txt" inside the resources folder and copy and paste this content. This will print a colored banner with application information.

${AnsiColor.BLUE}  _____ ${AnsiColor.WHITE}         _                          ${AnsiColor.BLUE} _______       ${AnsiColor.WHITE}  _
${AnsiColor.BLUE} / ____|${AnsiColor.WHITE}        | |                         ${AnsiColor.BLUE}|__   __|      ${AnsiColor.WHITE} | |
${AnsiColor.BLUE}| |     ${AnsiColor.WHITE} _______| |_ ___ _   _ _   _  __ _  ${AnsiColor.BLUE}   | |${AnsiColor.WHITE}  ___  ___| |__
${AnsiColor.BLUE}| |     ${AnsiColor.WHITE}|_  / _ \ __/ __| | | | | | |/ _` | ${AnsiColor.BLUE}   | |${AnsiColor.WHITE} / _ \/ __| '_ \
${AnsiColor.BLUE}| |____ ${AnsiColor.WHITE} / /  __/ |_\__ \ |_| | |_| | (_| | ${AnsiColor.BLUE}   | |${AnsiColor.WHITE}|  __/ (__| | | |
${AnsiColor.BLUE} \_____/${AnsiColor.WHITE}/___\___|\__|___/\__,_|\__, |\__,_| ${AnsiColor.BLUE}   |_|${AnsiColor.WHITE} \___|\___|_| |_|
${AnsiColor.BLUE}        ${AnsiColor.WHITE}                        __/ |
${AnsiColor.BLUE}        ${AnsiColor.WHITE}                       |___/
${AnsiColor.BRIGHT_GREEN}#application.title# v#application.version#
Build: #build.time#
Powered by Spring Boot ${spring-boot.version}

3. Replacing the Variable Information in the Banner.

To replace the build information in the banner, we will use the replacer plugin.

<plugin>
	<groupId>com.google.code.maven-replacer-plugin</groupId>
	<artifactId>replacer</artifactId>
	<version>1.5.3</version>
	<executions>
	  <execution>
		<phase>prepare-package</phase>
		<goals>
		  <goal>replace</goal>
		</goals>
	  </execution>
	</executions>
	<configuration>
	  <file>target/classes/banner.txt</file>
	  <replacements>
		<replacement>
		  <token>#application.title#</token>
		  <value>${project.artifactId}</value>
		</replacement>
		<replacement>
		  <token>#application.version#</token>
		  <value>${project.version}</value>
		</replacement>
		<replacement>
		  <token>#build.time#</token>
		  <value>${maven.build.timestamp}</value>
		</replacement>
	  </replacements>
	</configuration>
</plugin>

4. Summary

In this short article, we have shown how you can set a customized banner for your Spring application.

Kickstarting Your Spring Cloud Journey with the 2022 Skeleton Project

2023-08-25T17:27:00.005+08:00

In this article, I will share a composition of a Spring Cloud 2022 skeleton project with Spring Boot 3.

1. Introduction

Spring Cloud 2022 is a long-awaited major release that builds on top of Spring Framework 6.x and Spring Boot 3.x. It needs at least Java 17 to run, which means it is fully compatible with Jakarta framework.

GIT repository is available below.

2. Spring Cloud Microservices

The following services demonstrate a basic microservice architecture.

2.1 config-service

This module depends on spring-cloud-config-server and connects with a git repo to serve the configuration settings. This is very useful if you want your configurations to be loaded from a central repository.

2.2 discovery-server

This module depends on spring-cloud-starter-netflix-eureka-server where other services can register and be looked up.

2.3 api-gateway

This module depends on spring-cloud-starter-gateway. It is the gateway for all incoming requests and makes the necessary load balance forwarding to the actual service implementation.

2.4 mail-services

This module depends on spring-cloud-starter-config to connect to the config server and spring-cloud-starter-netflix-eureka-client to register to the Eureka naming server.

It also depends on Webflux for providing a REST endpoint.

3. Running the Services

Make the folder /config-repo a git repo by going inside the folder and running the command below.

>git init

The services must be run in this order

config-server
discovery-server
mail-services
api-gateway

4. Testing the Services

mail-services

This service runs on port 8001.

The actual URL is http://localhost:8001/mails/profiles/1

api-gateway

This service runs on port 8001. It simply forwards the request to the mail-services.

The actual URL is http://localhost:8000/mails/profiles/1

5. Repository

https://github.com/czetsuya/spring-cloud-2022

6. Development and Support

Unlock the full coding experience! As a GitHub Sponsor, you gain exclusive access to the code behind this article—start learning and building today!

I'm available for contracting services and support. You can reach me at: https://www.czetsuyatech.com/p/consultation-services.html.

Java XML Validation: How to Validate an XML Document against an XSD

2023-06-28T16:49:00.005+08:00

Introducing our Java XML Validation Project! Master the art of validating XML effortlessly against XSD in a Java environment. Gain hands-on experience, ensuring the integrity and conformity of your data.

1. Introduction

Introducing our Java XML Validation Project! Gain hands-on experience in validating XML documents against XSD with ease. This project serves as a practical guide, equipping you with the necessary knowledge and tools to ensure the integrity and conformity of your XML data in a Java environment. Discover the simplicity of our step-by-step approach, which walks you through the entire validation process. Master the art of validating XML effortlessly and unlock a new level of confidence in your data. Join us on this journey as we empower you to harness the power of XML validation in your Java projects.

2. Generating Java Classes from XSD

For this exercise, we will be using an XSD from w3schools https://www.w3schools.com/xml/schema_example.asp.

To be able to generate the Java classes from XSD, we will be using Eclipse Enterprise Edition. IntelliJ can also do it with the Jakarta plugin, but Eclipse performs better, especially when dealing with XJB files.

IntelliJ

Eclipse JAXB Classes Generator

Generated Classes

3. Dependencies

<dependency>
	<groupId>com.fasterxml.jackson.dataformat</groupId>
	<artifactId>jackson-dataformat-xml</artifactId>
</dependency>

<dependency>
	<groupId>jakarta.activation</groupId>
	<artifactId>jakarta.activation-api</artifactId>
	<version>2.1.1</version>
</dependency>
<dependency>
	<groupId>jakarta.xml.bind</groupId>
	<artifactId>jakarta.xml.bind-api</artifactId>
	<version>${jakarta.version}</version>
</dependency>
<dependency>
	<groupId>org.glassfish.jaxb</groupId>
	<artifactId>jaxb-runtime</artifactId>
	<version>${jakarta.version}</version>
	<scope>runtime</scope>
</dependency>

4. Core Classes

To run the validations and throw the appropriate exception or name of the tag where a constraint is violated, we will need the following classes.

4.1 ResourceResolver

To import an external XSD, for example, if we will be using the XML digital signature.

<xs:import namespace="http://www.w3.org/2000/09/xmldsig#" schemaLocation="xmldsig-core-schema.xsd"/>

public class ResourceResolver implements LSResourceResolver {

  private static final Logger LOGGER = LoggerFactory.getLogger(ResourceResolver.class);

  public LSInput resolveResource(String type, String namespaceURI, String publicId, String systemId, String baseURI) {

    // note: in this sample, the XSD's are expected to be in the root of the classpath
    InputStream resourceAsStream = this.getClass().getClassLoader().getResourceAsStream("xsd/" + systemId);
    return new Input(publicId, systemId, resourceAsStream);
  }

  static class Input implements LSInput {

    private String publicId;

    private String systemId;

    public String getStringData() {

      String textDataFromFile;

      try {
        BufferedReader bufferReader = new BufferedReader(new InputStreamReader(inputStream));
        StringBuilder stringBuilder = new StringBuilder();
        String eachStringLine;

        while ((eachStringLine = bufferReader.readLine()) != null) {
          stringBuilder.append(eachStringLine).append("\n");
        }
        textDataFromFile = stringBuilder.toString();

        return textDataFromFile;

      } catch (IOException e) {
        LOGGER.error("Fail reading from inputStream={}", e.getMessage());
        return null;
      }
    }
...

4.2 SaxErrorHandler

For us to be able to know the tag where a constraint exception is thrown, we need to define a custom exception that accepts an XMLStreamReader object where we can get the localName property.

public class SaxErrorHandler implements ErrorHandler {

  private XMLStreamReader reader;

  public SaxErrorHandler(XMLStreamReader reader) {
    this.reader = reader;
  }

  @Override
  public void error(SAXParseException e) {
    warning(e);
  }

  @Override
  public void fatalError(SAXParseException e) {
    warning(e);
  }

  @Override
  public void warning(SAXParseException e) {

    throw new XmlValidationException(reader.getLocalName());
  }
}

4.3 Miscellaneous Classes

We will also be needing utility classes for:

Creating a JAXBElement
Convert JAXBElement to XMLStreamReader
Defining the correct marshaller object

These classes are all available in the project.

4.4 XmlSchemaValidator Interface

And finally, the interface that pieces together all components to do the validation.

  default void validateXMLSchema(String xsdPath, JAXBElement<?> xml) {

    Validator validator = null;
    try {
      SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
      factory.setProperty(XMLConstants.ACCESS_EXTERNAL_DTD, "");
      factory.setProperty(XMLConstants.ACCESS_EXTERNAL_SCHEMA, "");
      factory.setResourceResolver(new ResourceResolver());

      Schema schema = factory.newSchema(ResourceUtils.getFile(xsdPath));
      validator = schema.newValidator();
      XMLStreamReader xmlStreamReader = asStreamReader(xml);

      ErrorHandler errorHandler = new SaxErrorHandler(xmlStreamReader);
      validator.setErrorHandler(errorHandler);

      validator.validate(new StAXSource(xmlStreamReader));

    } catch (IOException | SAXException | JAXBException | XMLStreamException e) {

      var xmlEx = getXmlValidationExceptionIfPresent(e);
      throw new InvalidXmlException(xmlEx.getLocalName(), e);
    }
  }

5. Source Code / Git Repository

The complete source code for this project is available for my sponsors at https://github.com/czetsuyatech/java-xml-validation-by-xsd.

Sponsorship link: https://github.com/sponsors/czetsuya

Project Screenshot

Efficient HTTP Request Logging in Spring with Zalando: A Developer's Guide

2023-05-19T12:55:00.144+08:00

Logging internal HTTP requests using RestTemplate with Zalando library.

1. Overview

In some cases, it is critical that we log both internal and external HTTP requests. This is very useful for debugging communication between microservices. But this feature is already available with other Spring boot libraries. What I wanted to show in this exercise is the flexibility of logging or saving in a database of particular data.

2. Prerequisites

2.1 Dependencies

For this exercise, I will use the following dependencies:

Spring Boot 3
logbook-spring-boot-starter v3.x

2.2 The Spring Project

The project that we will work on will run on Spring Boot 3.x.

3. Hands On

3.1 Controller

We will start with our controller.

@RestController
@Slf4j
@RequiredArgsConstructor
public class LoggingController {

  private final RestTemplate restTemplate;

  @GetMapping("/logs/internal-requests")
  public void logBody() {

    log.debug("Logging internal request");

    restTemplate.getForEntity("http://localhost:8080/call-me", Computer.class);
  }

  @GetMapping("/call-me")
  public Computer callMe() {
    log.debug("calling internal url");

    return Computer.builder()
        .brand("Apple")
        .model("Macbook Pro")
        .cpu("M2")
        .ram(32)
        .build();
  }
}

This class exposes a REST endpoint /logs/internal-requests that will trigger an HTTP request to another endpoint /call-me. This call will be intercepted by the logbook.

Let's see how it works.

3.2 Web Configuration

We need to create the following beans for the logbook to intercept the HTTP requests.

@Configuration
@RequiredArgsConstructor
public class WebBeansConfig {

  private final Logbook logbook;

  @Bean
  public Logbook getLogbook() {
    return Logbook.builder()
        .strategy(new StatusAtLeastStrategy(200))
        .sink(new LogSink(new LogFormatter(), new LogWriter()))
        .build();
  }

  @Bean
  public RestTemplate restTemplate() {

    RestTemplate restTemplate = new RestTemplate();
    restTemplate.getInterceptors().add(getLogbookInterceptor());

    return restTemplate;
  }

  private LogbookClientHttpRequestInterceptor getLogbookInterceptor() {
    return new LogbookClientHttpRequestInterceptor(logbook);
  }
}

3.3 Logbook and Logging Classes

Here are the skeletons of the classes needed by the logbook.

After the logbook intercepts the request, it will cause this class to perform pre and post request data manipulation.

@RequiredArgsConstructor
public class LogSink implements Sink {

  private final LogFormatter formatter;
  private final LogWriter writer;

  @Override
  public void write(Precorrelation preCorrelation, HttpRequest request) throws IOException {

  }

  @Override
  public void write(Correlation correlation, HttpRequest request, HttpResponse response) throws IOException {

  }

  @Override
  public void writeBoth(Correlation correlation, HttpRequest request, HttpResponse response) throws IOException {
    writer.write(formatter.format(correlation.getId(), request, response));
  }
}

This class extracts the necessary values from the HTTP request and response object. We can also deduce whether the request is incoming or outgoing.

// Customizeable message model
public class LogFormatter {

  public Message format(String correlationId, HttpRequest request, HttpResponse response) throws IOException {

    return Message.builder()
        .correlationId(correlationId)
        .requestBody(request.getBodyAsString())
        .responseBody(response.getBodyAsString())

        .build();
  }
}

Finally, this class logs the message object in the console. We can also save the Message object in the database by injecting a mapper and converting the Message to an entity, then inject a repository that we can use to insert the entity in the database.

// Log writer
@Slf4j
public class LogWriter {

  // you can autowire a service here
  public void write(Message message) {
    log.debug("Http intercepted message={}", message);
  }
}

4. GitHub Repository

Sponsor this blog: https://github.com/sponsors/czetsuya
https://github.com/czetsuyatech/logging-http-zalando/

Unlocking Google Calendar API with Keycloak Via Token Exchange: A Developer's Guide

2023-05-19T12:53:00.494+08:00

1. Overview

Unlock the power of Keycloak for seamless authentication and authorization in your applications. This comprehensive guide is designed for software developers who want to integrate Keycloak into their systems to enable secure user login and token exchange with Google.

By following step-by-step instructions and best practices, you'll learn how to leverage Keycloak's robust features to facilitate seamless authentication and obtain a Google token. With this token, you can effortlessly access the Google Calendar API and unlock a wealth of functionality for your application. Enhance user experience and streamline calendar integration - start implementing Keycloak and Google integration today.

You need prior knowledge with:

Google
Keycloak
Spring Boot

2. Google

Before we can do the integration with Keycloak we need to create a project in Google Console, create a credential and obtain OAuth client id and secret.

If you haven't done so, you can create a Google console account at https://cloud.google.com/cloud-console.

2.1 Creating the Google Cloud Project

At the top left of your Google Cloud Console, click the project and click New Project.

I call this project "Lab".

2.2 Create OAuth2 Client

The next step is to create the OAuth2 client credential that will give us the client id and secret. This combination will be used when we set up the Google Identity provider in Keycloak.

With the Lab project selected, find API & Services / Credentials in the menu.

Click Create Credentials with the following inputs:

Type: OAuth client ID

Application type: Web application

Name: Lab

There are two important entries in this section:

Authorized Javascript origins

This should be the URL where the request for login will come from.

Authorized redirect URIs

This entry is a common cause of problems if not properly set up. This is where Google will redirect the user after a successful login.

Some common redirect URIs.

Amazon Cognito: https://lab.auth.ap-southeast-1.amazoncognito.com/oauth2/idpresponse

Postman: https://www.getpostman.com/oauth2/callback

Keycloak broker: http://localhost:8080/realms/czetsuyatech/broker/google/endpoint

Keycloak token: http://localhost:8080/realms/czetsuyatech/protocol/openid-connect/token

When you have a redirect issue, this is one of the first things you may want to check.

This is how my Lab client looks like.

On the same page, you should be able to see the client id and secret. Take note because we will need them later.

2.3 Setup OAuth Consent Screen

This section will ask for several application-related settings. You can enter values as you see fit, but normally I set the following.

Non-sensitive scopes:

auth-userinfo-email
auth-userinfo-profile
openid

Don't forget to add a test user. We will use it to login later.

3. Keycloak

For this exercise, we need to create a new Keycloak realm and client. We must enable authentication in the client.

3.1 Client

Enabling authentication will give us access to Keycloak's client and secret, which we will need later for testing. Take note of it.

We also need to download the Adapter config of our newly created client.

The downloaded config should look like this. Save it as we will use it in our Spring Boot application later.

{
  "realm": "czetsuyatech",
  "auth-server-url": "http://localhost:8080/",
  "ssl-required": "external",
  "resource": "web-front",
  "credentials": {
    "secret": "BxYWD11qCAW6ZybPTy6b9M8ej0thTxxx"
  },
  "confidential-port": 0
}

3.2 Identity Provider

Identity providers are third-party services that allow users to log in to Keycloak. In our example, we will do an authorization as well by exchanging the Keycloak token to Google to allow us to call Google API services such as Calendar and Gmail.

So login to Keycloak as admin, click Identity Providers on the left side, Click Add Provider, and select Google.

In the Client ID and secret fields, enter the values that we saved when we were configuring the Google client earlier.

Under Advance settings, change the value of Scopes to " email profile openid https://www.googleapis.com/auth/calendar https://www.googleapis.com/auth/calendar.events https://www.googleapis.com/auth/gmail.readonly", without double quotes. This will give us permission to call list endpoints from Calendar and Gmail APIs.

3.3 Permissions

In this section, we will give our client permission to exchange tokens with the newly created Google identity provider.

On the identity provider page, click the Permissions tab. Toggle Permissions enabled to ON. Under the Permission list, click token-exchange.

Set the Decision strategy to Unanimous. Click Save.

Next, we need to create the permission scope. At the top-left of the page, click Client details. It should redirect you to the Authorization tab. Open the Policies sub-tab. Create a new policy with the following values:

Type: Client

Name: google-token-exchange

Clients: web-front (the Keycloak client we created earlier)

Logic: Positive

Go back to your Google identity provider's permission page and set the Policies to the one we just created.

The setup should now be ready to exchange tokens.

4. Testing

To test the exchange of tokens between Keycloak and Google, I have created a Postman collection where we can:

Authenticate a user
Exchange Keycloak to Google token

Before we proceed with the testing, we need to run a Keycloak instance. I have prepared a docker-compose to do that.

We need to authenticate our user, so create a new POST request in Postman and use OAuth2 authorization. Set the URL to http://localhost:8080/realms/czetsuyatech/protocol/openid-connect/token.

Be sure to set the client id and secret to the ones we saved earlier when we created the Keycloak client.

For the truncated values:

Callback URL: https://www.getpostman.com/oauth2/callback

Auth URL: http://localhost:8080/realms/czetsuyatech/protocol/openid-connect/auth

Access Token URL: http://localhost:8080/realms/czetsuyatech/protocol/openid-connect/token

Click Get New Access Token. It should redirect us to a Google login page, use the test email address that you added earlier when creating the Google client.

The next page will show you the scopes that will be permitted for your user. We defined this when we created the Google identity provider earlier.

Click Allow.

Copy the access token and paste it on a bearerToken variable. The succeeding requests will have Authorization Type = Bearer and will use this value.

The Body of the request must have the following URL encoded parameters:

Click Send. It should return with the following response.

This is the token that we can use to access Google APIs.

Obviously, we need to automate the process if we wanted this feature in our application. Thus, I have created a Spring project to do it.

5. Spring Project

This project demonstrates the token-exchange feature available on Keycloak. With this, we can exchange Keycloak for Google's token, which we can use to call Google APIs.

Note: This project is only available for my sponsors https://github.com/sponsors/czetsuya.

5.1 Available Features

Get the bearer token from the request and store it in a Bean that can be autowired.
Exchange Keycloak token to Google, this will allow us to call Google APIs (Calendar, Gmail).
Use Spring exchange to provide abstraction in calling Google APIs.
Endpoints to list Calendar events and Gmail messages.

The heart of the application exchanges tokens between Keycloak and Google.

@Component
public class GoogleTokenExchange implements TokenExchange {

  @Override
  public String exchangeToken(String accessToken) {

    AuthzClient authzClient = AuthzClient.create();
    Configuration keycloakConfig = authzClient.getConfiguration();
    String baseUrl = authzClient.getServerConfiguration().getTokenEndpoint();

    RestTemplate restTemplate = new RestTemplate();

    HttpHeaders headers = new HttpHeaders();
    headers.add("Content-Type", MediaType.APPLICATION_FORM_URLENCODED.toString());
    headers.add("Accept", MediaType.APPLICATION_JSON.toString());

    MultiValueMap<String, String> requestBody = new LinkedMultiValueMap<>();
    requestBody.add("client_id", keycloakConfig.getResource());
    requestBody.add("client_secret", keycloakConfig.getCredentials().get("secret").toString());
    requestBody.add("grant_type", "urn:ietf:params:oauth:grant-type:token-exchange");
    requestBody.add("subject_token_type", "urn:ietf:params:oauth:token-type:access_token");
    requestBody.add("subject_token", accessToken);
    requestBody.add("requested_issuer", "google");

    HttpEntity formEntity = new HttpEntity<>(requestBody, headers);

    ResponseEntity<AccessTokenResponse> response = restTemplate.exchange(baseUrl, HttpMethod.POST, formEntity,
        AccessTokenResponse.class);

    return response.getBody().getToken();
  }
}

5.2 Testing

If you have subscribed as my sponsor, you will be given access to my repository and proceed with this testing.

The project provides two endpoints, to test calling list calendar events and Gmail messages from Google. Matching Postman requests are also available to make testing easier.

As we have done earlier, we need to get an access token from Keycloak and set it as bearerToken environment variable inside Postman.

Get Calendar Events Response

Get Gmail Messages

6. Source Code

All the resources used in this article and demo are available on the GitHub repository https://github.com/czetsuyatech/auth-exchange.

Spring project source code
Docker/compose file
Keycloak realm
Postman collection for testing

7. References

https://www.keycloak.org/docs/latest/securing_apps/#_token-exchange
https://www.keycloak.org/docs/latest/server_development/index.html#retrieving-external-idp-tokens
https://developers.google.com/calendar
https://developers.google.com/gmail/api/guides

Czetsuya Tech Sponsorship Benefits

2023-05-07T07:28:00.007+08:00

Bronze

Sponsorship badge
NO access to private repositories

Silver

You will have access to one of my private repositories
Does not include the repositories in the higher tiers

Gold

You will have access to a set of my selected private repositories

Pre-Quarkus Custom Keycloak

Does not include the repositories in the higher tiers.

Diamond

You will have access to my HiveMaster set of repositories that provides multi-tenancy IAM using Keycloak and Spring.

For contract services https://www.czetsuyatech.com/p/consultation-services.html.

Keep coding, and keep supporting!

Implementing X509 Certificate Validation in Java: A Step-by-Step Guide

2023-05-06T17:55:00.014+08:00

Unlock the power of secure communication with Java! Learn how to implement X509 certificate validation simply and straightforwardly, step-by-step.

Overview

An X.509 certificate is a digital certificate used to verify a particular entity's identity, such as a website or an individual. X.509 certificates are widely used in various applications, including secure communication protocols like HTTPS, SSL, and TLS.

An X.509 certificate contains several pieces of information, including the identity of the entity being verified, the public key that is associated with that entity, and various other pieces of metadata that provide additional information about the certificate itself.

Importance

One of the key benefits of X.509 certificates is that they allow entities to authenticate themselves to other parties securely and reliably. This can help to prevent various types of attacks, including man-in-the-middle attacks, where an attacker intercepts and modifies communication between two parties to steal sensitive information.

Overall, X.509 certificates play a critical role in modern security infrastructure and are essential to many different types of secure communication protocols.

Requirements

You must have the following installed on your computer.

- Git

- Maven

- IDE (IntellIJ / Visual Studio)

- Java 17+

Dependencies

This project uses the following dependencies.

- bcpkix-jdk15on - certificate revocation check

- spring-boot-starter-test (for testing)

- lombok (for logging)

- Certificates from https://www.digicert.com/kb/digicert-root-certificates.htm

Project Code

The certificate validation process.

To implement the validation, we will introduce a bean where we can define the:

Certificate that we are validating
The pem file that we can cross-check #1
Root certificate to check if the certificate is already revoked

Let's examine the code.

The CertificateBundle class is just a simple Java bean.

@Getter
@Setter
@Builder
public class CertificateBundle {

  // Certificate extracted from the message
  private X509Certificate certificate;

  // Certificate loaded from the application must match the certificate
  private X509Certificate confirmationCertificate;

  // RootCertificate of the certificate
  private X509Certificate rootCertificate;
}

To validate the certificate expiration, we need to extract the X509Certificate object and perform a time comparison against the value of getNotAfter(). Here, we're making sure that the current date is greater than the certificate's expiry date.

Date expiresOn = cert.getNotAfter();
Date now = new Date();

if (now.getTime() > expiresOn.getTime()) {
  throw new ExpiredCertificateException();
}

Comparing digital signature against a PEM file. In most fintech applications, there is a need to sign the message before sending it to another service. Upon receiving the message, the server needs to extract the public certificate embedded in it and compare it to a PEM file. This PEM file is a public key of the private key certificate used to sign the message.

boolean match = Arrays.equals(cert1.getPublicKey().getEncoded(),
	cert2.getPublicKey().getEncoded());

if (!match) {
  throw new MismatchCertificateException("Embedded certificate does not match the given PEM");
}

And finally, we do the revocation check. This process allows us to know whether a website's certificate is trustworthy or not. It uses the root certificate that we used for signing as the trust anchor.

try {
  List<X509Certificate> certs = Collections.singletonList(cb.getCertificate());
  CertificateFactory cf = CertificateFactory.getInstance("X509");
  CertPath cp = cf.generateCertPath(certs);

  // load the root CA cb
  X509Certificate rootCACert = cb.getRootCertificate();

  // init trusted certs
  TrustAnchor ta = new TrustAnchor(rootCACert, null);
  Set<TrustAnchor> trustedCerts = new HashSet<>();
  trustedCerts.add(ta);

  // init PKIX parameters
  PKIXParameters params = new PKIXParameters(trustedCerts);

  // load the CRL
  params.addCertStore(CertStore.getInstance("Collection",
	  new CollectionCertStoreParameters(getX509Crls(cf, getCrl(cb.getCertificate())))));

  // perform validation
  CertPathValidator cpv = CertPathValidator.getInstance("PKIX");
  PKIXCertPathValidatorResult cpvResult = (PKIXCertPathValidatorResult) cpv.validate(cp,
	  params);
  X509Certificate trustedCert = cpvResult.getTrustAnchor().getTrustedCert();

  if (trustedCert == null) {
	log.error("Trusted certificate not found");
	throw new CertificateException("Trusted certificate not found");

  } else {
	log.debug("Trusted CA DN = " + trustedCert.getSubjectX500Principal());
  }

} catch (InvalidAlgorithmParameterException | java.security.cert.CertificateException |
		 CRLException |
		 NoSuchAlgorithmException |
		 CertPathValidatorException |
		 IOException e) {
  throw new RevokedCertificateException(e);
}

Repository

https://github.com/czetsuyatech/java-x509certificate-validation - Available for my Silver sponsors: https://github.com/sponsors/czetsuya.

How to Use Lob Datatype in PostgreSQL

2023-03-31T11:35:00.002+08:00

1. Problem

Here's the typical code we use when creating a Lob column.

Column definition

@NotNull
@Lob
@Column(name = "message")
private String message;

But when we create our tables we would encounter the error below, because PostgreSQL doesn't support lob.

Caused by: org.hibernate.tool.schema.spi.SchemaManagementException: Schema-validation: wrong column type encountered in column [message] in table [comm_messages]; found [text (Types#VARCHAR)], but expecting [oid (Types#CLOB)]
    at org.hibernate.tool.schema.internal.AbstractSchemaValidator.validateColumnType(AbstractSchemaValidator.java:167)

2. Solution

To resolve this problem we can implement several solutions.

2.1 Set the column to TEXT in the table. With this approach, we need to manually edit the entity where the column is defined.

@NotNull
@Lob
@Column(name = "message", columnDefinition = "TEXT")
private String message;

2.2 If you are using Spring, you can set the non_contextual_creation to true in the application property file.

spring:
  jpa:
    properties:
      hibernate:
        jdbc:
          lob:
            non_contextual_creation: true

How to Use EntityListener for Effective Audit Trails in Spring

2023-03-19T20:30:00.006+08:00

1. Overview

An audit trail is important in persisting entities using JPA because it helps maintain data integrity and ensure compliance with regulatory requirements. An audit trail records a history of changes made to data over time, including who made the changes and when. By implementing an audit trail, you can track and verify any changes made to data, and identify any unauthorized or unexpected modifications. This can help detect and prevent data tampering, and provide transparency into data changes made by different users over time.

In the context of JPA, audit trails can be implemented using various techniques, including adding auditable columns to database tables or using interceptors to capture changes made to entities before they are persisted in the database. By including an audit trail in your JPA application, you can maintain a comprehensive history of changes made to data, which can be useful for various purposes such as auditing, compliance, and troubleshooting.

Overall, implementing an audit trail can provide greater transparency and accountability for data changes in your JPA application, which can help ensure data integrity and protect against unauthorized access or data tampering.

2. Hands-on Coding

Spring-data-jpa provides a convenient solution for implementing audit trails in your Java applications, and in this tutorial, we'll show you how to use it to quickly get started with auditing your classes.

2.1 Auditable Entity

An abstract class where we will store the audit information such as date and user. It should be annotated with AuditingEntityListener from spring-data-jpa. I'm using Lombok dependency for getter and setter here.


@Data
@SuperBuilder
@MappedSuperclass
@NoArgsConstructor
@EntityListeners(AuditingEntityListener.class)
public abstract class AuditableEntity extends BaseEntity {

  @Column(name = "created_at", nullable = false, updatable = false)
  @CreatedDate
  private LocalDateTime createdAt;

  @Column(name = "updated_at", nullable = false)
  @LastModifiedDate
  private LocalDateTime updatedAt;

  @Column(name = "created_by", nullable = false, updatable = false)
  @CreatedBy
  private String createdBy;

  @Column(name = "updated_by", nullable = false)
  @LastModifiedBy
  private String updatedBy;
}

2.2 Auditor Information

A class where we can get the currently logged user using SecurityContextHolder or a constant for system user.


public class AuditorAwareImpl implements AuditorAware<String> {

  @Override
  public Optional<String> getCurrentAuditor() {
    return Optional.of("SYSTEM");
  }
}

2.3 Jpa Configuration

We need to tell Spring context where to pull the auditor information as such we need to add this configuration.


@Configuration
@EnableJpaAuditing(auditorAwareRef = "auditorProvider")
@EnableTransactionManagement
public class JpaConfig {

  @Bean
  public AuditorAware<String> auditorProvider() {
    return new AuditorAwareImpl();
  }

}