Althaf's Journal

Is Spring a full replacement for J2EE?

2025-11-26T05:36:00.002+05:30

Spring is not 100% a replacement for Java EE — but it replaces almost all of Java EE’s painful parts and only relies on a few foundational Java EE/Jakarta EE APIs underneath.

Let’s break it down clearly.

Is Spring a full replacement for J2EE?

Practically YES (for most developers).
Technically NO (because Spring still uses a few underlying Java EE/Jakarta EE specs).

Spring was created specifically to replace the heavyweight parts of J2EE such as:

EJB (Enterprise JavaBeans)
J2EE container-managed transactions
J2EE security
J2EE messaging complexity
XML-heavy configuration
Vendor-specific application servers

Spring became the mainstream choice because it solves all those problems.

What parts of J2EE does Spring replace?

J2EE Component Spring Replacement

J2EE Component	Spring Replacement
EJB	Spring Beans, `@Service`, `@Transactional`
JSP/JSF/Servlet MVC complexity	Spring MVC (`@RestController`)
JTA (complex transactions)	Simplified Spring Tx + declarative transactions
JMS API complexity	Spring JMS abstractions
Security	Spring Security
JAX-RS	Spring Web (REST controllers)
Application Server (JBoss, WebLogic)	Embedded Tomcat/Jetty with Spring Boot

Spring Beans, @Service, @Transactional

JSP/JSF/Servlet MVC complexity

Spring MVC (@RestController)

JTA (complex transactions)

Simplified Spring Tx + declarative transactions

JMS API complexity

Spring JMS abstractions

Security

Spring Security

JAX-RS

Spring Web (REST controllers)

Application Server (JBoss, WebLogic)

Embedded Tomcat/Jetty with Spring Boot

Spring basically eliminated the need for:

EAR files
WAR deployment to app servers
EJB containers
JNDI hell
XML descriptors

What parts of J2EE/Jakarta EE does Spring still depend on?

This is important:

✔ Spring still uses Servlet API

Even Spring MVC runs inside the Java EE Servlet model:

javax.servlet.* (Java EE era)
jakarta.servlet.* (Jakarta EE era)

Servlet API provides:

Request/response handling
Filters
Dispatchers
Underlying HTTP model

Spring Boot bundles embedded Tomcat/Jetty which implement this API.

✔ Spring integrates with JPA API

Spring Data JPA uses:

jakarta.persistence.*
JPA is still a Java EE/Jakarta spec, implemented by Hibernate, EclipseLink, etc.

✔ Spring integrates with JTA (optional)

If distributed transactions are needed.

✔ Spring can use JMS API

Spring JMS wraps JMS (Java Message Service).

Important distinction

Spring does NOT require a Java EE Application Server.

Everything can run with:

Embedded Tomcat
Jetty
Undertow
Netty (for WebFlux)

This is why Spring revolutionized Java development.

Spring depends on parts of Java EE specs (like Servlet & JPA API), but it does not depend on Java EE runtime containers like:

JBoss
WebSphere
WebLogic
GlassFish

Instead, Spring Boot bundles only the needed pieces inside your JAR.

Analogy (very intuitive)

Java = The language
J2EE = A big enterprise machine with many heavy parts
Spring = A modern toolkit that replaces 90% of that machine with lightweight tools
Spring Boot = Spring + Auto-config + Built-in engine + Ready to drive

Spring uses only a few base components of the old machine (Servlet, JPA specs), but otherwise runs independently.

Final Answer

Spring is effectively a full replacement for J2EE application development.

It no longer depends on Java EE containers, but it still uses a few core Java EE (now Jakarta EE) APIs—such as Servlets, JPA, JMS—as building blocks.

So:
Spring replaces the J2EE programming model, not the underlying J2EE/Jakarta EE specifications.

What is WEB-INF folder in Java Enterprise Application ?

2025-02-09T21:34:00.001+05:30

The `WEB-INF` folder is a crucial directory in Java web applications, specifically in applications that follow the Java Servlet or Java EE (Jakarta EE) standards. It is a special folder located within the web application's root directory and is used to store configuration files, libraries, and other resources that are not directly accessible to clients (e.g., web browsers). Here's a detailed breakdown of its purpose and contents:

Purpose of the WEB-INF Folder

1. Security:

The contents of the `WEB-INF` folder are not publicly accessible by clients. This ensures that sensitive files (e.g., configuration files, Java classes) cannot be accessed directly via a URL.

2. Organization:

It provides a standardized structure for organizing web application resources, making it easier to manage and deploy applications.

3. Deployment:

The `WEB-INF` folder is required for proper deployment of Java web applications. It contains essential files that the server (e.g., Apache Tomcat, Jetty) uses to configure and run the application.

Contents of the WEB-INF Folder

The `WEB-INF` folder typically contains the following subdirectories and files:

1. `web.xml` (Deployment Descriptor):

This is the most important file in the `WEB-INF` folder. It is an XML file that defines the configuration of the web application, including:

Servlet mappings.
Filters and listeners.
Context parameters.
Security constraints.
Welcome files.

- In modern applications (e.g., using Spring Boot), `web.xml` is often optional because configurations can be done using annotations or Java-based configuration.

2. `classes` Directory:

This directory contains compiled Java classes (`.class` files) for the application. These classes include:

Servlets.
Utility classes.
Any other application-specific Java code.
The structure of this directory mirrors the package structure of your Java classes.

3. `lib` Directory:

This directory contains JAR files (Java libraries) that the application depends on. These libraries are automatically included in the application's classpath by the server.

4. Other Configuration Files:

Depending on the framework or tools you use, the `WEB-INF` folder may also contain additional configuration files, such as:

`spring-config.xml` (for Spring Framework applications).
`faces-config.xml` (for JavaServer Faces applications).
Custom configuration files for your application.

Structure of a Typical WEB-INF Folder

Here’s an example of what the `WEB-INF` folder might look like in a Java web application:

Understanding LLMs from Scratch Using Middle School Math

2024-12-05T05:19:00.000+05:30

A self-contained, full explanation to inner workings of an LLM

Rohit Patel

Published in

Towards Data Science

In this article, we talk about how Large Language Models (LLMs) work, from scratch — assuming only that you know how to add and multiply two numbers. The article is meant to be fully self-contained. We start by building a simple Generative AI on pen and paper, and then walk through everything we need to have a firm understanding of modern LLMs and the Transformer architecture. The article will strip out all the fancy language and jargon in ML and represent everything simply as they are: numbers. We will still call out what things are called to tether your thoughts when you read jargon-y content.

Going from addition/multiplication to the most advanced AI models today without assuming other knowledge or referring to other sources means we cover a LOT of ground. This is NOT a toy LLM explanation — a determined person can theoretically recreate a modern LLM from all the information here. I have cut out every word/line that was unnecessary and as such this article isn’t really meant to be browsed.

What will we cover?

A simple neural network
How are these models trained?
How does all this generate language?
What makes LLMs work so well?
Embeddings
Sub-word tokenizers
Self-attention
Softmax
Residual connections
Layer Normalization
Dropout
Multi-head attention
Positional embeddings
The GPT architecture
The transformer architecture

Let’s dive in.

The first thing to note is that neural networks can only take numbers as inputs and can only output numbers. No exceptions. The art is in figuring out how to feed your inputs as numbers, interpreting the output numbers in a way that achieves your goals. And finally, building neural nets that will take the inputs you provide and give you the outputs you want (given the interpretation you chose for these outputs). Let’s walk through how we get from adding and multiplying numbers to things like Llama 3.1.

A simple neural network:

Let’s work through a simple neural network that can classify an object:

Object data available: Dominant color (RGB) & Volume (in milli-liters)
Classify into: Leaf OR Flower

Here’s what the data for a leaf and a sunflower can look like:

Image by author

Let’s now build a neural net that does this classification. We need to decide on input/output interpretations. Our inputs are already numbers, so we can feed them directly into the network. Our outputs are two objects, leaf and flower which the neural network cannot output. Let’s look at a couple of schemes we can use here:

We can make the network output a single number. And if the number is positive we say it’s a leaf and if it is negative we say it’s a flower
OR, we can make the network output two numbers. We interpret the first one as a number for leaf and second one as the number for flower and we will say that the selection is whichever number is larger

Both schemes allow the network to output number(s) that we can interpret as leaf or flower. Let’s pick the second scheme here because it generalizes well to other things we will look at later. And here’s a neural network that does the classification using this scheme. Let’s work through it:

Image by author

Some jargon:

Neurons/nodes: The numbers in the circles

Weights: The colored numbers on the lines

Layers: A collection of neurons is called a layer. You could think of this network as having 3 layers: Input layer with 4 neurons, Middle layer with 3 neurons, and the Output layer with 2 neurons.

To calculate the prediction/output from this network (called a “forward pass”), you start from the left. We have the data available for the neurons in the Input layer. To move “forward” to the next layer, you multiply the number in the circle with the weight for the corresponding neuron pairing and you add them all up. We demonstrate blue and orange circle math above. Running the whole network we see that the first number in the output layer comes out higher so we interpret it as “network classified these (RGB,Vol) values as leaf”. A well trained network can take various inputs for (RGB,Vol) and correctly classify the object.

The model has no notion of what a leaf or a flower is, or what (RGB,Vol) are. It has a job of taking in exactly 4 numbers and giving out exactly 2 numbers. It is our interpretation that the 4 input numbers are (RGB,Vol) and it is also our decision to look at the output numbers and infer that if the first number is larger it’s a leaf and so on. And finally, it is also up to us to choose the right weights such that the model will take our input numbers and give us the right two numbers such that when we interpret them we get the interpretation we want.

An interesting side effect of this is that you can take the same network and instead of feeding RGB,Vol feed other 4 numbers like cloud cover, humidity etc.. and interpret the two numbers as “Sunny in an hour” or “Rainy in an hour” and then if you have the weights well calibrated you can get the exact same network to do two things at the same time — classify leaf/flower and predict rain in an hour! The network just gives you two numbers, whether you interpret it as classification or prediction or something else is entirely up to you.

Stuff left out for simplification (feel free to ignore without compromising comprehensibility):

Activation layer: A critical thing missing from this network is an “activation layer”. That’s a fancy word for saying that we take the number in each circle and apply a nonlinear function to it (RELU is a common function where you just take the number and set it to zero if it is negative, and leave it unchanged if it is positive). So basically in our case above, we would take the middle layer and replace the two numbers (-26.6 and -47.1) with zeros before we proceed further to the next layer. Of course, we would have to re-train the weights here to make the network useful again. Without the activation layer all the additions and multiplications in the network can be collapsed to a single layer. In our case, you could write the green circle as the sum of RGB directly with some weights and you would not need the middle layer. It would be something like (0.10 * -0.17 + 0.12 * 0.39–0.36 * 0.1) * R + (-0.29 * -0.17–0.05 * 0.39–0.21 * 0.1) * G …and so on. This is usually not possible if we have a nonlinearity there. This helps networks deal with more complex situations.
Bias: Networks will usually also contain another number associated with each node, this number is simply added to the product to calculate the value of the node and this number is called the “bias”. So if the bias for the top blue node was 0.25 then the value in the node would be: (32 * 0.10) + (107 * -0.29) + (56 * -0.07) + (11.2 * 0.46) + 0.25 = — 26.35. The word parameters is usually used to refer to all these numbers in the model that are not neurons/nodes.
Softmax: We don’t usually interpret the output layer directly as shown in our models. We convert the numbers into probabilities (i.e. make it so that all numbers are positive and add up to 1). If all the numbers in the output layer were already positive one way you could achieve this is by dividing each number by the sum of all numbers in the output layer. Though a “softmax” function is normally used which can handle both positive and negative numbers.

How are these models trained?

In the example above, we magically had the weights that allowed us to put data into the model and get a good output. But how are these weights determined? The process of setting these weights (or “parameters”) is called “training the model”, and we need some training data to train the model.

Let’s say we have some data where we have the inputs and we already know if each input corresponds to leaf or flower, this is our “training data” and since we have the leaf/flower label for each set of (R,G,B,Vol) numbers, this is “labeled data”.

Here’s how it works:

Start with a random numbers, i.e. set each parameter/weight to a random number
Now, we know that when we input the data corresponding to the leaf (R=32, G=107, B=56, Vol=11.2). Suppose we want a larger number for leaf in the output layer. Let’s say we want the number corresponding to leaf as 0.8 and the one corresponding to flower as 0.2 (as shown in example above, but these are illustrative numbers to demonstrate training, in reality we would not want 0.8 and 0.2. In reality these would be probabilities, which they are not here, and we would them to be 1 and 0)
We know the numbers we want in the output layer, and the numbers we are getting from the randomly selected parameters (which are different from what we want). So for all the neurons in the output layer, let’s take the difference between the number we want and the number we have. Then add up the differences. E.g., if the output layer is 0.6 and 0.4 in the two neurons, then we get: (0.8–0.6)=0.2 and (0.2–0.4)= -0.2 so we get a total of 0.4 (ignoring minus signs before adding). We can call this our “loss”. Ideally we want the loss to be close to zero, i.e. we want to “minimize the loss”.
Once we have the loss, we can slightly change each parameter to see if increasing or decreasing it will increase the loss. This is called the “gradient” of that parameter. Then we can move each of the parameters by a small amount in the direction where the loss goes down (opposite the direction of the gradient). Once we have moved all the parameters slightly, the loss should be lower
Keep repeating the process and you will reduce the loss, and eventually have a set of weights/parameters that are “trained”. This whole process is called “gradient descent”.

Couple of notes:

You often have multiple training examples, so when you change the weights slightly to minimize the loss for one example it might make the loss worse for another example. The way to deal with this is to define loss as average loss over all the examples and then take gradient over that average loss. This reduces the average loss over the entire training data set. Each such cycle is called an “epoch”. Then you can keep repeating the epochs thus finding weights that reduce average loss.
We don’t actually need to “move weights around” to calculate the gradient for each weight — we can just infer it from the formula (e.g. if the weight is 0.17 in the last step, and the value of neuron is positive, and we want a larger number in output we can see that increasing this number to 0.18 will help).

In practice, training deep networks is a hard and complex process because gradients can easily spiral out of control, going to zero or infinity during training (called “vanishing gradient” and “exploding gradient” problems). The simple definition of loss that we talked about here is perfectly valid, but rarely used as there are better functional forms that work well for specific purposes. With modern models containing billions of parameters, training a model requires massive compute resources which has its own problems (memory limitations, parallelization etc.)

How does all this help generate language?

Remember, neural nets take in some numbers, do some math based on the trained parameters, and give out some other numbers. Everything is about interpretation and training the parameters (i.e. setting them to some numbers). If we can interpret the two numbers as “leaf/flower” or “rain or sun in an hour”, we can also interpret them as “next character in a sentence”.

But there are more than 2 letters in English, and so we must expand the number of neurons in the output layer to, say, the 26 letters in the English language (let’s also throw in some symbols like space, period etc..). Each neuron can correspond to a character and we look at the (26 or so) neurons in the output layer and say that the character corresponding to the highest numbered neuron in the output layer is the output character. Now we have a network that can take some inputs and output a character.

What if we replace the input in our network with these characters: “Humpty Dumpt” and asked it to output a character and interpreted it as the “Network’s suggestion of the next character in the sequence that we just entered”. We can probably set the weights well enough for it to output “y” — thereby completing “Humpty Dumpty”. Except for one problem, how do we input these lists of characters in the network? Our network only accepts numbers!!

One simple solution is to assign a number to each character. Let’s say a=1, b=2 and so on. Now we can input “humpty dumpt” and train it to give us “y”. Our network looks something like this:

Image by author

Ok, so now we can predict one character ahead by providing the network a list of characters. We can use this fact to build a whole sentence. For example, once we have the “y” predicted, we can append that “y” to the list of characters we have and feed it to the network and ask it to predict the next character. And if well trained it should give us a space, and so on and so forth. By the end, we should be able to recursively generate “Humpty Dumpty sat on a wall”. We have Generative AI. Moreover, we now have a network capable of generating language! Now, nobody ever actually puts in randomly assigned numbers and we will see more sensible schemes down the line. If you cannot wait, feel free to check out the one-hot encoding section in the appendix.

Astute readers will note that we can’t actually input “Humpty Dumpty” into the network since the way the diagram is, it only has 12 neurons in the input layer one for each character in “humpty dumpt” (including the space). So how can we put in the “y” for the next pass. Putting a 13th neuron there would require us to modify the entire network, that’s not workable. The solution is simple, let’s kick the “h” out and send the 12 most recent characters. So we would be sending “umpty dumpty” and the network will predict a space. Then we would input “mpty dumpty “ and it will produce an s and so on. It looks something like this:

Image by author

We’re throwing away a lot of information in the last line by feeding the model only “ sat on the wal”. So what do the latest and greatest networks of today do? More or less exactly that. The length of inputs we can put into a network is fixed (determined by the size of the input layer). This is called “context length” — the context that is provided to the network to make future predictions. Modern networks can have very large context lengths (several thousand words) and that helps. There are some ways of inputting infinite length sequences but the performance of those methods, while impressive, has since been surpassed by other models with large (but fixed) context length.

One other thing careful readers will notice is that we have different interpretations for inputs and outputs for the same letters! For example, when inputting “h” we are simply denoting it with the number 8 but on the output layer we are not asking the model to output a single number (8 for “h”, 9 for “i” and so on..) instead we are are asking the model to output 26 numbers and then we see which one is the highest and then if the 8th number is highest we interpret the output as “h”. Why don’t we use the same, consistent, interpretation on both ends? We could, it’s just that in the case of language, freeing yourself to choose between different interpretations gives you a better chance of building better models. And it just so happens that the most effective currently known interpretations for the input and output are different. In-fact, the way we are inputting numbers in this model is not the best way to do it, we will look at better ways to do that shortly.

What makes large language models work so well?

Generating “Humpty Dumpty sat on a wall” character-by-character is a far cry from what modern LLMs can do. There are a number of differences and innovations that get us from the simple generative AI that we discussed above to the human-like bot. Let’s go through them:

Embeddings

Remember we said that the way that we are inputting characters into the model isn’t the best way to do it. We just arbitrarily selected a number for each character. What if there were better numbers we could assign that would make it possible for us to train better networks? How do we find these better numbers? Here’s a clever trick:

When we trained the models above, the way we did it was by moving around weights and seeing that gives us a smaller loss in the end. And then slowly and recursively changing the weights. At each turn we would:

Feed in the inputs
Calculate the output layer
Compare it to the output we ideally want and calculate the average loss
Adjust the weights and start again

In this process, the inputs are fixed. This made sense when inputs were (RGB, Vol). But the numbers we are putting in now for a,b,c etc.. are arbitrarily picked by us. What if at every iteration in addition to moving the weights around by a bit we also moved the input around and see if we can get a lower loss by using a different number to represent “a” and so on? We are definitely reducing the loss and making the model better (that’s the direction we moved a’s input in, by design). Basically, apply gradient descent not just to the weights but also the number representations for the inputs since they are arbitrarily picked numbers anyway. This is called an “embedding”. It is a mapping of inputs to numbers, and as you just saw, it needs to be trained. The process of training an embedding is much like that of training a parameter. One big advantage of this though is that once you train an embedding you can use it in another model if you wish. Keep in mind that you will consistently use the same embedding to represent a single token/character/word.

We talked about embeddings that are just one number per character. However, in reality embeddings have more than one number. That’s because it is hard to capture the richness of concept by a single number. If we look at our leaf and flower example, we have four numbers for each object (the size of the input layer). Each of these four numbers conveyed a property and the model was able to use all of them to effectively guess the object. If we had only one number, say the red channel of the color, it might have been a lot harder for the model. We’re trying to capture human language here — we’re going to need more than one number.

So instead of representing each character by a single number, maybe we can represent it by multiple numbers to capture the richness? Let’s assign a bunch of numbers to each character. Let’s call an ordered collection of numbers a “vector” (ordered as in each number has a position, and if we swap position of two numbers it gives us a different vector. This was the case with our leaf/flower data, if we swapped the R and G numbers for the leaf, we would get a different color, it would not be the same vector anymore). The length of a vector is simply how many numbers it contains. We’ll assign a vector to each character. Two questions arise:

If we have a vector assigned to each character instead of a number, how do we now feed “humpty dumpt” to the network? The answer is simple. Let’s say we assigned a vector of 10 numbers to each character. Then instead of the input layer having 12 neurons we would just put 120 neurons there since each of the 12 characters in “humpty dumpt” has 10 numbers to input. Now we just put the neurons next to each other and we are good to go
How do we find these vectors? Thankfully, we just learned how to train embedding numbers. Training an embedding vector is no different. You now have 120 inputs instead of 12 but all you are doing is moving them around to see how you can minimize loss. And then you take the first 10 of those and that’s the vector corresponding to “h” and so on.

All the embedding vectors must of course be the same length, otherwise we would not have a way of entering all the character combinations into the network. E.g. “humpty dumpt” and in the next iteration “umpty dumpty” — in both cases we are entering 12 characters in the network and if each of the 12 characters was not represented by vectors of length 10 we won’t be able to reliably feed them all into a 120-long input layer. Let’s visualize these embedding vectors:

Image by author

Let’s call an ordered collection of same-sized vectors a matrix. This matrix above is called an embedding matrix. You tell it a column number corresponding to your letter and looking at that column in the matrix will give you the vector that you are using to represent that letter. This can be applied more generally for embedding any arbitrary collection of things — you would just need to have as many columns in this matrix as the things you have.

Subword Tokenizers

So far, we have been working with characters as the basic building blocks of language. This has its limitations. The neural network weights have to do a lot of the heavy lifting where they must make sense of certain sequences of characters (i.e. words) appearing next to each other and then next to other words. What if we directly assigned embeddings to words and made the network predict the next word. The network doesn’t understand anything more than numbers anyway, so we can assign a 10-length vector to each of the words “humpty”, “dumpty”, “sat”, “on” etc.. and then we just feed it two words and it can give us the next word. “Token” is the term for a single unit that we embed and then feed to the model. Our models so far were using characters as tokens, now we are proposing to use entire words as a token (you can of course use entire sentences or phrases as tokens if you like).

Using word tokenization has one profound effect on our model. There are more than 180K words in the English language. Using our output interpretation scheme of having a neuron per possible output we need hundreds of thousands of neurons in the output layer insead of the 26 or so. With the size of the hidden layers needed to achieve meaningful results for modern networks, this issue becomes less pressing. What is however worth noting is that since we are treating each word separately, and we are starting with a random number embeddings for each — very similar words (e.g. “cat” and “cats”) will start with no relationship. You would expect that embeddings for the two words should be close to each other — which undoubtedly the model will learn. But, can we somehow use this obvious similarity to get a jumpstart and simplify matters?

Yes we can. The most common embedding scheme in language models today is something where you break words down into subwords and then embed them. In the cat example, we would break down cats into two tokens “cat” and ”s”. Now it is easier for the model to understand the concept of “s” followed by other familiar words and so on. This also reduces the number of tokens we need (sentencpiece is a common tokenizer with vocab size options in tens of thousands vs hundreds of thousands of words in english). A tokenizer is something that takes you input text (e.g. “Humpty Dumpt”) and splits it into the tokens and gives you the corresponding numbers that you need to look up the embedding vector for that token in the embedding matrix. For example, in case of “humpty dumpty” if we’re using character level tokenizer and we arranged our embedding matrix as in the picture above, then the tokenizer will first split humpty dumpt into characters [‘h’,’u’,…’t’] and then give you back the numbers [8,21,…20] because you need to look up the 8th column of the embedding matrix to get the embedding vector for ‘h’ (embedding vector is what you will feed into the model, not the number 8, unlike before). The arrangement of the columns in the matrix is completely irrelevant, we could assign any column to ‘h’ and as long as we look up the same vector every time we input ‘h’ we should be good. Tokenizers just give us an arbitrary (but fixed) number to make lookup easy. The main task we need them for really is splitting the sentence in tokens.

With embeddings and subword tokenization, a model could look something like this:

Image by author

The next few sections deal with more recent advances in language modeling, and the ones that made LLMs as powerful as they are today. However, to understand these there are a few basic math concepts you need to know. Here are the concepts:

Matrices and matrix multiplication
General concept of functions in mathematics
Raising numbers to powers (e.g. a3 = a*a*a)
Sample mean, variance, and standard deviation

I have added summaries of these concepts in the appendix.

Self Attention

So far we have seen only one simple neural network structure (called feedforward network), one which contains a number of layers and each layer is fully connected to the next (i.e., there is a line connecting any two neurons in consecutive layers), and it is only connected to the next layer (e.g. no lines between layer 1 and layer 3 etc..). However, as you can imagine there is nothing stopping us from removing or making other connections. Or even making more complex structures. Let’s explore a particularly important structure: self-attention.

If you look at the structure of human language, the next word that we want to predict will depend on all the words before. However, they may depend on some words before them to a greater degree than others. For example, if we are trying to predict the next word in “Damian had a secret child, a girl, and he had written in his will that all his belongings, along with the magical orb, will belong to ____”. This word here could be “her” or “him” and it depends specifically on a much earlier word in the sentence: girl/boy.

The good news is, our simple feedforward model connects to all the words in the context, and so it can learn the appropriate weights for important words, But here’s the problem, the weights connecting specific positions in our model through feed forward layers are fixed (for every position). If the important word was always in the same position, it would learn the weights appropriately and we would be fine. However, the relevant word to the next prediction could be anywhere in the system. We could paraphrase that sentence above and when guessing “her vs his”, one very important word for this prediction would be boy/girl no matter where it appeared in that sentence. So, we need weights that depend not only on the position but also on the content in that position. How do we achieve this?

Self attention does something like adding up the embedding vectors for each of the words, but instead of directly adding them up it applies some weights to each. So if the embedding vectors for humpty,dumpty, sat are x1, x2, x3 respectively, then it will multiply each one with a weight (a number) before adding them up. Something like output = 0.5 x1 + 0.25 x2 + 0.25 x3 where output is the self-attention output. If we write the weights as u1, u2, u3 such that output = u1x1+u2x2+u3x3 then how do we find these weights u1, u2, u3?

Ideally, we want these weights to be dependent on the vector we are adding — as we saw some may be more important than others. But important to whom? To the word we are about to predict. So we also want the weights to depend on the word we are about to predict. Now that’s an issue, we of course don’t know the word we are about to predict before we predict it. So, self attention uses the word immediately preceding the word we are about to predict, i.e., the last word in the sentence available (I don’t really know why this and why not something else, but a lot of things in deep learning are trial and error and I suspect this works well).

Great, so we want weights for these vectors, and we want each weight to depend on the word that we are aggregating and word immediately preceding the one we are going to predict. Basically, we want a function u1 = F(x1, x3) where x1 is the word we will weight and x3 is the last word in the sequence we have (assuming we have only 3 words). Now, a straightforward way of achieving this is to have a vector for x1 (let’s call it k1) and a separate vector for x3 (let’s call it q3) and then simply take their dot product. This will give us a number and it will depend on both x1 and x3. How do we get these vectors k1 and q3? We build a tiny single layer neural network to go from x1 to k1 (or x2 to k2, x3 to k3 and so on). And we build another network going from x3 to q3 etc… Using our matrix notation, we basically come up with weight matrices Wk and Wq such that k1 = Wkx1 and q1 =Wqx1 and so on. Now we can take a dot product of k1 and q3 to get a scalar, so u1 = F(x1,x3) = Wkx1 · Wqx3.

One additional thing that happens in self-attention is that we don’t directly take the weighted sum of the embedding vectors themselves. Instead, we take the weighted sum of some “value” of that embedding vector, which is obtained by another small single layer network. What this means is similar to k1 and q1, we also now have a v1 for the word x1 and we obtain it through a matrix Wv such that v1=Wvx1. This v1 is then aggregated. So it all looks something like this if we only have 3 words and we are trying to predict the fourth:

Self attention. Image by author

The plus sign represents a simple addition of the vectors, implying they have to have the same length. One last modification not shown here is that the scalars u1, u2, u3 etc.. won’t necessarily add up to 1. If we need them to be weights, we should make them add up. So we will apply a familiar trick here and use the softmax function.

This is self-attention. There is also cross-attention where you can have the q3 come from the last word, but the k’s and the v’s can come from another sentence altogether. This is for example valuable in translation tasks. Now we know what attention is.

This whole thing can now be put in a box and be called a “self attention block”. Basically, this self attention block takes in the embedding vectors and spits out a single output vector of any user-chosen length. This block has three parameters, Wk,Wq,Wv — it doesn’t need to be more complicated than that. There are many such blocks in the machine learning literature, and they are usually represented by boxes in diagrams with their name on it. Something like this:

Image by author

One of the things that you will notice with self-attention is that the position of things so far does not seem relevant. We are using the same W’s across the board and so switching Humpty and Dumpty won’t really make a difference here — all numbers will end up being the same. This means that while attention can figure out what to pay attention to, this won’t depend on word position. However, we do know that word positions are important in english and we can probably improve performance by giving the model some sense of a word’s position.

And so, when attention is used, we don’t often feed the embedding vectors directly to the self attention block. We will later see how “positional encoding” is added to embedding vectors before feeding to attention blocks.

Note for the pre-initiated: Those for whom this isn’t the first time reading about self-attention will note that we are not referencing any K and Q matrices, or applying masks etc.. That is because those things are implementation details arising out of how these models are commonly trained. A batch of data is fed and the model is simultaneously trained to predict dumpty from humpty, sat from humpty dumpty and so on. This is a matter of gaining efficiency and does not affect interpretation or even model outputs, and we have chosen to omit training efficiency hacks here.

Softmax

We talked briefly about softmax in the very first note. Here’s the problem softmax is trying to solve: In our output interpretation we have as many neurons as the options from which we want the network to select one. And we said that we are going to interpret the network’s choice as the highest value neuron. Then we said we are going to calculate loss as the difference between the value that network provides, and an ideal value we want. But what’s that ideal value we want? We set it to 0.8 in the leaf/flower example. But why 0.8? Why no 5, or 10, or 10 million? The higher the better for that training example. Ideally we want infinity there! Now that would make the problem intractable — all loss would be infinite and our plan of minimizing loss by moving around parameters (remember “gradient descent”) fails. How do we deal with this?

One simple thing we can do is cap the values we want. Let’s say between 0 and 1? This would make all loss finite, but now we have the issue of what happens when the network overshoots. Let’s say it outputs (5,1) for (leaf,flower) in one case, and (0,1) in another. The first case made the right choice but the loss is worse! Ok, so now we need a way to also convert the outputs of the last layer in (0,1) range so that it preserves the order. We could use any function (a “function” in mathematics is simply a mapping of one number to another — in goes one number, out comes another — it’s rule based in terms of what will be output for a given input) here to get the job done. One possible option is the logistic function (see graph below) which maps all numbers to numbers between (0,1) and preserves the order:

Image by author

Now, we have a number between 0 and 1 for each of the neurons in the last layer and we can calculate loss by setting the correct neuron to 1, others to 0 and taking the difference of that from what the network provides us. This will work, but can we do better?

Going back to our “Humpty dumpty” example, let’s say we are trying to generate dumpty character-by-character and our model makes a mistake when predicting “m” in dumpty. Instead of giving us the last layer with “m” as the highest value, it gives us “u” as the highest value but “m” is a close second.

Now we can continue with “duu” and try to predict next character and so on, but the model confidence will be low because there are not that many good continuations from “humpty duu..”. On the other hand, “m” was a close second, so we can also give “m” a shot, predict the next few characters, and see what happens? Maybe it gives us a better overall word?

So what we are talking about here is not just blindly selecting the max value, but trying a few. What’s a good way to do it? Well we have to assign a chance to each one — say we will pick the top one with 50%, second one with 25% and so on. That’s a good way to do it. But maybe we would want the chance to be dependent on the underlying model predictions. If the model predicts values for m and u to be really close to each other here (compared to other values) — then maybe a close 50–50 chance of exploring the two is a good idea?

So we need a nice rule that takes all these numbers and converts them into chances. That’s what softmax does. It is a generalization of the logistic function above but with additional features. If you give it 10 arbitrary numbers — it will give you 10 outputs, each between 0 and 1 and importantly, all 10 adding up to 1 so that we can interpret them as chance. You will find softmax as the last layer in nearly every language model.

Residual connections

We have slowly changed our visualization of networks as the sections progress. We are now using boxes/blocks to denote certain concepts. This notation is useful in denoting a particularly useful concept of residual connections. Let’s look at residual connection combined with a self-attention block:

A residual connection. Image by author

Note that we put “Input” and “Output” as boxes to make things simpler, but these are still basically just a collection of neurons/numbers same as shown above.

So what’s going on here? We are basically taking the output of self-attention block and before passing it to the next block, we are adding to it the original Input. First thing to note is that this would require that the dimensions of the self-attention block output must now be the same as that of the input. This is not a problem since as we noted the self-attention output is determined by the user. But why do this? We won’t get into all the details here but the key thing is that as networks get deeper (more layers between input and output) it gets increasingly harder to train them. Residual connections have been shown to help with these training challenges.

Layer Normalization

Layer normalization is a fairly simple layer that takes the data coming into the layer and normalizes it by subtracting the mean and dividing it by standard deviation (maybe a bit more, as we see below). For example, if we were to apply layer normalization immediately after the input, it would take all the neurons in the input layer and then it would calculate two statistics: their mean and their standard deviation. Let’s say the mean is M and the standard deviation is D then what layer norm is doing is taking each of these neurons and replacing it with (x-M)/D where x denotes any given neuron’s original value.

Now how does this help? It basically stabilizes the input vector and helps with training deep networks. One concern is that by normalizing inputs, are we removing some useful information from them that may be helpful in learning something valuable about our goal? To address this, the layer norm layer has a scale and a bias parameter. Basically, for each neuron you just multiply it with a scalar and then add a bias to it. These scalar and bias values are parameters that can be trained. This allows the network to learn some of the variation that may be valuable to the predictions. And since these are the only parameters, the LayerNorm block doesn’t have a lot of parameters to train. The whole thing looks something like this:

Layer Normalization. Image by author

The Scale and Bias are trainable parameters. You can see that layer norm is a relatively simple block where each number is only operated on pointwise (after the initial mean and std calculation). Reminds us of the activation layer (e.g. RELU) with the key difference being that here we have some trainable parameters (albeit lot fewer than other layers because of the simple pointwise operation).

Standard deviation is a statistical measure of how spread out the values are, e.g., if the values are all the same you would say the standard deviation is zero. If, in general, each value is really far from the mean of these very same values, then you will have a high standard deviation. The formula to calculate standard deviation for a set of numbers, a1, a2, a3…. (say N numbers) goes something like this: subtract the mean (of these numbers) from each of the numbers, then square the answer for each of N numbers. Add up all these numbers and then divide by N. Now take a square root of the answer.

Note for the pre-initiated: Experienced ML professionals will note that there is no discussion of batch norm here. In-fact, we haven’t even introduced the concept of batches in this article at all. For the most part, I believe batches are another training accelerant not related to the understanding of core concepts (except perhaps batch norm which we do not need here).

Dropout

Dropout is a simple but effective method to avoid model overfitting. Overfitting is a term for when you train the model on your training data, and it works well on that dataset but does not generalize well to the examples the model has not seen. Techniques that help us avoid overfitting are called “regularization techniques”, and dropout is one of them.

If you train a model, it might make errors on the data and/or overfit it in a particular way. If you train another model, it might do the same, but in a different way. What if you trained a number of these models and averaged the outputs? These are typically called “ensemble models” because they predict the outputs by combining outputs from an ensemble of models, and ensemble models generally perform better than any of the individual models.

In neural networks, you could do the same. You could build multiple (slightly different) models and then combine their outputs to get a better model. However, this can be computationally expensive. Dropout is a technique that doesn’t quite build ensemble models but does capture some of the essence of the concept.

The concept is simple, by inserting a dropout layer during training what you are doing is randomly deleting a certain percentage of the direct neuron connections between the layers that dropout is inserted. Considering our initial network and inserting a Dropout layer between the input and the middle layer with 50% dropout rate can look something like this:

Image by author

Now, this forces the network to train with a lot of redundancy. Essentially, you are training a number of different models at the same time — but they share weights.

Now for making inferences, we could follow the same approach as an ensemble model. We could make multiple predictions using dropouts and then combine them. However, since that is computationally intensive — and since our models share common weights — why don’t we just do a prediction using all the weights (so instead of using 50% of the weights at a time we use all at the same time). This should give us some approximation of what an ensemble will provide.

One issue though: the model trained with 50% of the weights will have very different numbers in the middle neurons than one using all the weights. What we want is more ensemble style averaging here. How do we do this? Well, a simple way is to simply take all the weights and multiply them by 0.5 since we are now using twice as many weights. This is what Droput does during inference. It will use the full network with all the weights and simply multiply the weights with (1- p) where p is the deletion probability. And this has been shown to work rather well as a regularization technique.

Multi-head Attention

This is the key block in the transformer architecture. We’ve already seen what an attention block is. Remember that the output of an attention block was determined by the user and it was the length of v’s. What a multi-attention head is basically you run several attention heads in parallel (they all take the same inputs). Then we take all their outputs and simply concatenate them. It looks something like this:

Multi-head attention. Image by author

Keep in mind the arrows going from v1 -> v1h1 are linear layers — there’s a matrix on each arrow that transforms. I just did not show them to avoid clutter.

What is going on here is that we are generating the same key, query and values for each of the heads. But then we are basically applying a linear transformation on top of that (separately to each k,q,v and separately for each head) before we use those k,q,v values. This extra layer did not exist in self attention.

A side note is that to me, this is a slightly surprising way of creating a multi-headed attention. For example, why not create separate Wk,Wq,Wv matrices for each of the heads rather than adding a new layer and sharing these weights. Let me know if you know — I really have no idea.

Positional encoding and embedding

We briefly talked about the motivation for using positional encoding in the self-attention section. What are these? While the picture shows positional encoding, using a positional embedding is more common than using an encoding. As such we talk about a common positional embedding here but the appendix also covers positional encoding used in the original paper. A positional embedding is no different than any other embedding except that instead of embedding the word vocabulary we will embed numbers 1, 2, 3 etc. So this embedding is a matrix of the same length as word embedding, and each column corresponds to a number. That’s really all there is to it.

The GPT architecture

Let’s talk about the GPT architecture. This is what is used in most GPT models (with variation across). If you have been following the article thus far, this should be fairly trivial to understand. Using the box notation, this is what the architecture looks like at high level:

The GPT Architecture. Image by author

At this point, other than the “GPT Transformer Block” all the other blocks have been discussed in great detail. The + sign here simply means that the two vectors are added together (which means the two embeddings must be the same size). Let’s look at this GPT Transformer Block:

And that’s pretty much it. It is called “transformer” here because it is derived from and is a type of transformer — which is an architecture we will look at in the next section. This doesn’t affect understanding as we’ve already covered all the building blocks shown here before. Let’s recap everything we’ve covered so far building up to this GPT architecture:

We saw how neural nets take numbers and output other numbers and have weights as parameters which can be trained
We can attach interpretations to these input/output numbers and give real world meaning to a neural network
We can chain neural networks to create bigger ones, and we can call each one a “block” and denote it with a box to make diagrams easier. Each block still does the same thing, take in a bunch of numbers and output other bunch of numbers
We learned a lot of different types of blocks that serve different purposes
GPT is just a special arrangement of these blocks that is shown above with an interpretation that we discussed in Part 1

Modifications have been made over time to this as companies have built up to powerful modern LLMs, but the basic remains the same.

Now, this GPT transformer is actually what is called a “decoder” in the original transformer paper that introduced the transformer architecture. Let’s take a look at that.

The transformer architecture

This is one of the key innovations driving rapid acceleration in the capabilities of language models recently. Transformers not only improved the prediction accuracy, they are also easier/more efficient than previous models (to train), allowing for larger model sizes. This is what the GPT architecture above is based on.

If you look at GPT architecture, you can see that it is great for generating the next word in the sequence. It fundamentally follows the same logic we discussed in Part 1. Start with a few words and then continue generating one at a time. But, what if you wanted to do translation. What if you had a sentence in german (e.g. “Wo wohnst du?” = “Where do you live?”) and you wanted to translate it to english. How would we train the model to do this?

Well, first thing we would need to do is figure out a way to input german words. Which means we have to expand our embedding to include both german and english. Now, I guess here is a simply way of inputting the information. Why don’t we just concatenate the german sentence at the beginning of whatever so far generated english is and feed it to the context. To make it easier for the model, we can add a separator. This would look something like this at each step:

Image by author

This will work, but it has room for improvement:

If the context length is fixed, sometimes the original sentence is lost
The model has a lot to learn here. Two languages simultaneously, but also to know that <SEP> is the separator token where it needs to start translating
You are processing the entire german sentence, with different offsets, for each word generation. This means there will be different internal representations of the same thing and the model should be able to work through it all for translation

Transformer was originally created for this task and consists of an “encoder” and a “decoder” — which are basically two separate blocks. One block simply takes the german sentence and gives out an intermediate representation (again, bunch of numbers, basically) — this is called the encoder.

The second block generates words (we’ve seen a lot of this so far). The only difference is that in addition to feeding it the words generated so far we also feed it the encoded german (from the encoder block) sentence. So as it is generating language, it’s context is basically all the words generated so far, plus the german. This block is called the decoder.

Each of these encoders and decoders consist of a few blocks, notably the attention block sandwiched between other layers. Let’s look at the illustration of a transformer from the paper “Attention is all you need” and try to understand it:

Image from Vaswani et al. (2017)

The vertical set of blocks on the left is called the “encoder” and the ones to the right is called the “decoder”. Let’s go over and understand anything that we have not already covered before:

Recap on how to read the diagram: Each of the boxes here is a block that takes in some inputs in the form of neurons, and spits out a set of neurons as output that can then either be processed by the next block or interpreted by us. The arrows show where the output of a block is going. As you can see, we will often take the output of one block and feed it in as input into multiple blocks. Let’s go through each thing here:

Feed forward: A feedforward network is one that does not contain cycles. Our original network in section 1 is a feed forward. In-fact, this block uses very much the same structure. It contains two linear layers, each followed by a RELU (see note on RELU in first section) and a dropout layer. Keep in mind that this feedforward neetwork applies to each position independently. What this means is that the information on position 0 has a feedforward network, and on position 1 has one and so on.. But the neurons from position x do not have a linkage to the feedforward network of position y. This is important because if we did not do this, it would allow the network to cheat during training time by looking forward.

Cross-attention: You will notice that the decoder has a multi-head attention with arrows coming from the encoder. What is going on here? Remember the value, key, query in self-attention and multi-head attention? They all came from the same sequence. The query was just from the last word of the sequence in-fact. So what if we kept the query but fetched the value and key from a completely different sequence altogether? That is what is happening here. The value and key come from the output of the encoder. Nothing has changed mathematically except where the inputs for key and value are coming from now.

Nx: The Nx here simply represents that this block is chain-repeated N times. So basically you are stacking the block back-to-back and passing the input from the previous block to the next one. This is a way to make the neural network deeper. Now, looking at the diagram there is room for confusion about how the encoder output is fed to the decoder. Let’s say N=5. Do we feed the output of each encoder layer to the corresponding decoder layer? No. Basically you run the encoder all the way through once and only once. Then you just take that representation and feed the same thing to every one of the 5 decoder layers.

Add & Norm block: This is basically the same as below (guess the authors were just trying to save space)

Image by author

Everything else has already been discussed. Now you have a complete explanation of the transformer architecture building up from simple sum and product operations and fully self contained! You know what every line, every sum, every box and word means in terms of how to build them from scratch. Theoretically, these notes contain what you need to code up the transformer from scratch. In-fact, if you are interested this repo does that for the GPT architecture above.

Appendix

Matrix Multiplication

We introduced vectors and matrices above in the context of embeddings. A matrix has two dimensions (number or rows and columns). A vector can also be thought of as a matrix where one of the dimensions equals one. Product of two matrices is defined as:

Image by author

Dots represent multiplication. Now let’s take a second look at the calculation of blue and organic neurons in the very first picture. If we write the weights as a matrix and the inputs as vectors, we can write the whole operation in the following way:

Image by author

If the weight matrix is called “W” and the inputs are called “x” then Wx is the result (the middle layer in this case). We can also transpose the two and write it as xW — this is a matter of preference.

Standard deviation

We use the concept of standard deviation in the Layer Normalization section. Standard deviation is a statistical measure of how spread out the values are (in a set of numbers), e.g., if the values are all the same you would say the standard deviation is zero. If, in general, each value is really far from the mean of these very same values, then you will have a high standard deviation. The formula to calculate standard deviation for a set of numbers, a1, a2, a3…. (say N numbers) goes something like this: subtract the mean (of these numbers) from each of the numbers, then square the answer for each of N numbers. Add up all these numbers and then divide by N. Now take a square root of the answer.

Positional Encoding

We talked about positional embedding above. A positional encoding is simply a vector of the same length as the word embedding vector, except it is not an embedding in the sense that it is not trained. We simply assign a unique vector to every position e.g. a different vector for position 1 and different one for position 2 and so on. A simple way of doing this is to make the vector for that position simply full of the position number. So the vector for position 1 would be [1,1,1…1] for 2 would be [2,2,2…2] and so on (remember length of each vector must match embedding length for addition to work). This is problematic because we can end up with large numbers in vectors which creates challenges during training. We can, of course, normalize these vectors by dividing every number by the max of position, so if there are 3 words total then position 1 is [.33,.33,..,.33] and 2 is [.67, .67, ..,.67] and so on. This has the problem now that we are constantly changing the encoding for position 1 (those numbers will be different when we feed 4 word sentence as input) and it creates challenges for the network to learn. So here, we want a scheme that allocates a unique vector to each position, and the numbers don’t explode. Basically if the context length is d (i.e., maximum number of tokens/words that we can feed into the network for predicting next token/word, see discussion in “how does it all generate language?” section) and if the length of the embedding vector is 10 (say), then we need a matrix with 10 rows and d columns where all the columns are unique and all the numbers lie between 0 and 1. Given that there are infinitely many numbers between zero and 1, and the matrix is finitely sized, this can be done in many ways.

The approach used in the “Attention is all you need” paper goes something like this:

Draw 10 sin curves each being si(p) = sin (p/10000(i/d)) (that’s 10k to power i/d)
Fill the encoding matrix with numbers such that (i,p)th number is si(p), e.g., for position 1 the 5th element of the encoding vector is s5(1)=sin (1/10000(5/d))

Why choose this method? By changing the power on 10k you are changing the amplitude of the sine function when viewed on the p-axis. And if you have 10 different sine functions with 10 different amplitudes, then it will be a long time before you get a repetition (i.e. all 10 values are the same) for changing values of p. And this helps give us unique values. Now, the actual paper uses both sine and cosine functions and the form of encoding is: si(p) = sin (p/10000(i/d)) if i is even and si(p) = cos(p/10000(i/d)) if i is odd.

Everything you need to know about Design Systems

2024-12-05T05:08:00.004+05:30

Audrey Hacq

Published in

UX Collective

→ Pour la version en Français, c’est par ici

Design Systems have become a key part of my everyday work… And from discussions around me, it seems that it is now the case for a lot of people ;)

After reading Alla Kholmatova’s book last year, I was lucky to attend the first European conference on the subject in March. All of this strengthened my conviction that, in the future, every brand and every product will use a Design System, simple or comprehensive, strict or loose, mono or cross platforms…

What is a Design System?

I keep hearing various definitions, so let me start by saying what a Design System isn’t: it is not a Sketch library, no more than a Style guide or a Pattern Library… Actually, it’s all of this and so much more!

A Design System is the single source of truth which groups all the elements that will allow the teams to design, realize and develop a product.

So a Design System is not a deliverable, but a set of deliverables. It will evolve constantly with the product, the tools and the new technologies.

As Jina Anne describes perfectly in this conference, the System is composed of tangibles and non-tangibles elements :

Tools for designers & developers, patterns, components, guidelines…
But also -and it’s often the most difficult thing to achieve- some abstract elements such as brand values, shared ways of working, mindset, shared beliefs…

Style Guide or Pattern Library: what’s the difference?

As it should be clear for you now, the Style guide and the Pattern Library are just some of the deliverables of a Design System.

A Style Guide -as indicated by its name- will focus on graphic styles (colours, fonts, illustrations…) and their usage.

A Pattern Library will integrate functional components and their usage.

Most of the current Design Systems contains both, just as Shopify for example which as one tab “Visuals” for its Style Guide and one tab “Components” for its Pattern Library.

Shopify — Polaris: Style guide & Pattern library

Why are we talking about that today?

The desire for factorizing design and components is not new. But we can see that the trend has accelerated during the past years. More and more companies are based on digital supports and do not even have Graphic guidelines for print anymore!

For a long time, digital was treated like the “side project”: we use to make Graphic guidelines for printed elements and then, add some extra-guidelines for the digital… Who has never received 300 pages of graphic guidelines with only 6 pages about digital? I have… And as a Designer, that’s not the kind of inputs I expect from a brand...

A designer who receives the Graphic guidelines of his client

We must now close the gap between print and digital around a shared language that will evolve with time. Design System is the direct descendant of the Graphic guidelines but it’s more mature and more integrated into the teams' workflow. We also now have the tools which allow us to build and share systems of components.

What’s inside?

The fundamental purpose of a Design System is to facilitate the work of the teams. So the first question we need to ask ourselves is not “What should I put inside my Design System?” but “Who is going to use it and how?”.

Once our target is defined and we have a first idea of what is already in place in the company (What is working or not? Which is the level of maturity of the teams about this subject? What are the existing tools?…), it will be easier to know where to start.

#1. Purpose and shared values

©Jahit Janberk

Where are we going? Why? And How?

Before starting anything, it’s essential to align teams around a clear set of shared goals. It will help to build a vision and making sure everyone looks in the same direction. These goals will evolve with time and it’s normal. We just have to make sure that changes are broadly communicated.

As for the values, they are like great ideals that will guide our choices, according to the brand objectives. We need to make sure that what we’re designing does not differ from these key values.

Beside these brand and product values, we could also define some team values that will bring each collaborator around a shared state of mind.

Posters the ASH team created for their core values

#2. Design Principles

Design Principles: A collection of Design Principles and methods

First thing to keep in mind: Design principles are so much more than just the visual aspect of a product…

Design principles are the guiding sentences that helps the teams to reach the purpose of the product thanks to the design.

They will help the teams to make meaningful Design decisions.

I will take the example of Medium. One of their Design principles is “Direction over choice”. Thanks to that principle, instead of designing a regular text editor with unlimited choices of colours and fonts, they opted for a simpler one. That allows the author to focus on the content of his article rather than on its visual aspect.

Design Principles must guide design choices

#3. Brand identity & language

Shopify brand identity

The identity should be defined in line with the strategy and the objectives of the brand. Alla Kholmatova speaks about the “perceptual patterns” that she lists as follow:

Colours
Fonts
Spaces
Shapes
Icons
Illustrations
Photographies
Animations
Voice and tone
Sounds

All of this forms the alphabet of the Brand.

It’s a good starting point but that’s surely not enough to speak a language! We have to go further to use this alphabet to compose some words and then associate these words to obtain sentences that make sense. These brand elements will need some rules which will be, in a way, the grammar and the conjugation of the System.

In our Design System, it’s materialized by guidelines, do’s and don’ts and “good examples” of use.

Examples of Identity Guidelines ©Backelite

This will ensure the design teams use the “right combinations” of this language, those who make the brand so unique and recognizable.

#4. Components & patterns

Components & Patterns are at the heart of the System. All the previously mentioned elements will help us to create them and deliver a consistent experience.

The components are our LEGO blocs. They’re used in Sketch by designers, and directly in the code by developers. Their functional behaviour must be specified.

As for the patterns, they are the building instructions that will allow us to use these components in a logical and consistent way, across all the products.

In the image below, we can clearly see that a component will be specified with technical and functional documentation, whereas a pattern will give recommendations about how to use it:

Pattern vs component ©Nathan Curtis

If you wonder which component you should develop first for your system, you can run a component prioritization workshop.

Best practices

Besides the documentation, that is often directly linked to the System, best practices will accompany teams in a larger and more transversal way.

The idea is to dig into the general best practices and extract only those who make sense, based on the product and the level of maturity of the teams.

Best practices will help the teams to form and develop their technicals skills.

“How to” in the BBC Design System

What kind of Design System?

We can almost say that there is one kind of Design System per team or product… To define our own, we have to ask ourselves the right questions:

How many people will use this system?
What is their profile and are they mature enough on the subject?
How many products must we align? On how many platforms? How many different technologies (Angular, React, others…)?
What degree of consistency do we want across our products?

All these answers will help us to define the most appropriate kind of Design System.

In her book, Alla Kholmatova gives us some inspiration:

Strict or loose?

On the left: Airbnb / On the right: Ted

A strict system will have a comprehensive and detailed documentation and will be fully synchronized between design and development. There will be a strict process for introducing a new pattern in the System. A strict system should be very broad in order to cover the majority of cases the teams may encounter.

A loose system will leave more space for experimentation. The System is here to provide a framework for the teams while preserving some freedom. Designers and developers are free to use it or not, regarding their particular needs for their product.

From my experience, we have to find the right balance between strictness and flexibility: a very strict system might repel designers and developers who will not want to use it. On the other side, can we still speak about Design System if this one is too loose?

Modular or integrated?

Modular vs integrated systems ©Alla Kholmatova

A modular system is made of interchangeable and reusable parts. It suits well for projects that have to scale quickly and that have to adapt to multiple user needs. The negative part is that it’s often more expensive to realize (because it can be difficult to make modules that can be independent while working well together). This kind of system will particularly fit large scale product as e-commerce, finance and governments websites. For a modular system, it will be very interesting to work with atomic design in mind.

An integrated system focus on one unique context. It’s also composed of parts, but these parts will not be interchangeable. This kind of system suits products that have very few repeating parts and that need a strong and often changing Art Direction (portfolios, showcases, marketing campaigns…).

Centralized or distributed?

The organization around Design Systems is really important for its evolution and scalability. In this article, Nathan Curtis details the different types of models. Here are two of them:

Which model for your Design System: centralized or distributed? ©Nathan Curtis

In a centralized model, one team is in charge of the System and makes it evolve. This team is here to facilitate the work of the other teams and has to be very close to them, to be sure that the System covers most of their needs.

In a distributed model, several people of several teams are in charge of the system. The adoption of the system is quicker because everyone feels involve but it also needs team leaders that will keep an overall vision of it.

In each case, I advise allowing everyone to participate and make suggestions to improve the system, in order to create a sense of membership.

Place your system on a scale

The idea will be thus to position our system on these three scales, in order to define the kind of system that we want:

Airbnb: a strict, modular and centralized system

Some examples

Without any originality, here are some of my favourites:

Material Design for its simplicity of navigation and the tools for designers & developers

Material Design System

Atlassian for its exhaustiveness

Atlassian Design System

Polaris from Shopify, because of its integration into the designer and developer’s workflow

Polaris by Shopify

IBM for its wonderful work on the Design language

IBM Design Language

Last words

A Design System is a full product, which is going to help the actors of a project to build other products.

As for every good product, it will have its own backlog and will have to build itself in an iterative way, keeping the users (designers, developers, PO…) at the heart of the approach.

The more integrated the system will be with the designers and developers workflows, the more effective it will be.

One really good example of this integration is the Sketch plugin of Polaris that displays the components and documentation of the system directly in Sketch, so the designers have everything they need during the conception phase:

Telescope: the sketch plugin, by Polaris

And all of this is only a beginning!

I’m convinced that the future will provide us with a lot of great technical evolutions to facilitate our lives and to allow us to concentrate on the experience of our products and our users ;)

And if you want to run some workshops around design systems, you can also read my articles about “How to prioritize your design system components?” and “How to measure your design system success?”.

Sources

Alla Kholmatova’s book on Design Systems
Every article of the fabulous Nathan Curtis
Conferences and writings by Jina Anne

You can also find other examples of very good Design Systems on this repo:

Design Systems Repo

This is not a comprehensive list of every design system that I have come across. Rather, it’s a collection of ones that…

designsystemsrepo.com

Thanks for reading!

What is the difference between JSF, Servlet and JSP?

2024-12-01T12:21:00.001+05:30

Servlets :
The Java Servlet API enables Java developers to write server-side code for delivering dynamic Web content. Like other proprietary Web server APIs, the Java Servlet API offered improved performance over CGI; however, it has some key additional advantages. Because servlets were coded in Java, they provides an object-oriented (OO) design approach and, more important, are able to run on any platform. Thus, the same code was portable to any host that supported Java. Servlets greatly contributed to the popularity of Java, as it became a widely used technology for server-side Web application development.
JSP :
JSP is built on top of servlets and provides a simpler, page-based solution to generating large amounts of dynamic HTML content for Web user interfaces. JavaServer Pages enables Web developers and designers to simply edit HTML pages with special tags for the dynamic, Java portions. JavaServer Pages works by having a special servlet known as a JSP container, which is installed on a Web server and handles all JSP page view requests. The JSP container translates a requested JSP into servlet code that is then compiled and immediately executed. Subsequent requests to the same page simply invoke the runtime servlet for the page. If a change is made to the JSP on the server, a request to view it triggers another translation, compilation, and restart of the runtime servlet.
JSF :
JavaServer Faces is a standard Java framework for building user interfaces for Web applications. Most important, it simplifies the development of the user interface, which is often one of the more difficult and tedious parts of Web application development.
Although it is possible to build user interfaces by using foundational Java Web technologies(such as Java servlets and JavaServer Pages) without a comprehensive framework designedfor enterprise Web application development, these core technologies can often lead to avariety of development and maintenance problems. More important, by the time the developers achieve a production-quality solution, the same set of problems solved by JSF will have been solved in a nonstandard manner. JavaServer Faces is designed to simplify the development of user interfaces for Java Web applications in the following ways:
• It provides a component-centric, client-independent development approach to building Web user interfaces, thus improving developer productivity and ease of use.
• It simplifies the access and management of application data from the Web user interface.
• It automatically manages the user interface state between multiple requests and multiple clients in a simple and unobtrusive manner.
• It supplies a development framework that is friendly to a diverse developer audience with different skill sets.
• It describes a standard set of architectural patterns for a web application.

[ Source : Complete reference:JSF ]

StateFlow and SharedFlow seem similar, but they are designed for different use cases

2024-10-05T10:10:00.004+05:30

StateFlow and SharedFlow seem similar, but they are designed for different use cases, and they should not be confused. Let's discuss the key differences and usages.

Let's start with how they work. SharedFlow resembles a broadcast channel—emitted values are delivered to all observers. It allows setting a replay parameter specifying how many past values should be emitted to new observers.

StateFlow behaves a lot like SharedFlow with reply = 1, but StateFlow must always have a value, so an initial value must be specified when a StateFlow is created. This value can be accessed or changed using value property.

But StateFlow was designed for a concrete use case: to represent an observable state. In Android, it is used to represent the state of our application, and views observe it and redraw whenever it changes. That is the key source of differences.

Redrawing view can be expensive, and state changes can be frequent. That is why two optimizations were introduced. First, updates are only emitted when they are different from the previous state. This behavior can be achieved on SharedFlow using distinctUntilChanged.

Second, StateFlow is conflated, meaning if observer is slower than value changes, it might lose some intermediate updates. That is appropriate for StateFlow, because we are not interested in drawing obsolete state.

StateFlow also have some tools for state update, like update function, that lets us safely update state, by creating a new one based on the current one.

That is why StateFlow should be used concretely to represent the observable state of our application. It should not be used as a "breadcast channel". For that we use SharedFlow.

Let's see an Android example: Things like a progress bar or data to display should be represented by StateFlow, but exceptions or messages to show to users as toasts should be represented as SharedFlow.

However, SharedFlow values are not completely safe to be used in view models, because updates made when there is no observer will not be observed (unless we use reply, but then the same values can be displayed again). So it is best to represent our state with StateFlow.

Now let's see a backend example. On a backend service, updates received from some websocket should be represented using SharedFlow, but an observable counter of application users can be represented using StateFlow.

To sum up, StateFlow and SharedFlow are similar, but they are designed for different use cases. StateFlow is for representing observable state, and SharedFlow is for broadcasting events. Use them accordingly.
Credits

https://www.linkedin.com/posts/marcin-moskala_stateflow-and-sharedflow-seem-similar-but-activity-7244273218692202496-yej-?utm_source=share&utm_medium=member_desktop

MVP : Event Bus pattern instead of Listener

2017-04-01T06:39:00.000+05:30

This question more towards paradigm. Why is that we don't use Event Bus instead of listeners in an MVP environment ? Typically, the "P" part has dependency injection of view and model references. Sure this has an advantage of showing explicit contract between the view and model via presenter, which is more readable.

However, wouldn't it be a cleaner approach to have presenter listen to events from views and events carry the view payload (eg: json representation). Same is the case with presenter talking back to the view. View will listen for events from presenter. Major advantage is, we don't have to write interfaces for each contract between view and presenter. If you look at the code you will see that presenter is getting exposed to view details like Text Fields, which i believe is increasing coupling between view and presenter. Say, if i'm replacing front end JavaFx instead of Vaadin, i will have to alter presenter as well.

This class is an example from a live project. Here we have different types of events ie i don't create event class for different cases. Eg: LoginViewEvent , DashBoardEvent etc which i believe is a maintenance pain.

public class UrayEvent {

    public static enum EventType {

        SESSION_SELECTED(1),
        DOCUMENT_SELECTED(2),
        DOCUMENT_EDIT_COMPLETE(3),
        DOCUMENT_EDIT_CANCELED(4),
        SHOW_SESSION_TABLES(5),
        SHOW_SESSION_DOCUMENTS(6),
        SHOW_SESSION_COLLABORATORS(7),
        USER_REQUESTED_REFRESH(8),
        AUTO_REFRESH(9),
        TABLE_SELECTED(10),
        DETACHED(11),
        SCHEDULER_NAVIGATION(12),
        JIRA_USER_SELECTED(13),
        DOCUMENT_SAVE_SUCCESS(14),
        DOCUMENT_SAVE_FAILURE(14);

        private final int value;

        private EventType(int value) {

            this.value = value;
        }

        public int getValue() {

            return value;
        }
    }

    public static class Event {

        private final EventType type;
        private final Object payload;

        public Event(EventType type, Object eventPayload) {

            this.type = type;
            this.payload = eventPayload;
        }

        public EventType getEventType() {

            return type;
        }

        public Object getEventPayload() {

            return payload;
        }
    }

}

Simple enough, the view send the event DOCUMENT_EDIT_COMPLETE .The presenter layer handles this event. I found this way, a better way to decouple views from presenter.

    @Subscribe
    public void handle(UrayEvent.Event event) {

        switch (event.getEventType()) {
            case DOCUMENT_EDIT_COMPLETE:
                  // event payload contains document model data
                  // like document id etc
                 saveDocument(event.getEventPayload);    
                break;
            default:
                break;
        }
    }

Advantage

Less boiler plate code,
For n-views we don't need n-interfaces
New event means adding event element to enum and updating respective subscribe methods handling this event.

Disadvantage

Memory leak if we forget to unregister from eventbus (faced it plenty of time)

Questions

1) This approach means, there would larger set enum elements as the application grow. Is this approach an anti pattern ?

2) As we saw it uses Event Bus extensively are there any drawbacks of using bus system instead of interface-listener pattern ?

Wanted your valuable suggestion on this regard. The main issue is, if i blindly apply this pattern extensively across the project i shouldn't regret doing so, what would be a possible pitfall in this approach.

Android Internals : just what application developers needs to know

2017-03-23T00:37:00.000+05:30

It is always good (not a must) to know the internals of a software platform, to be an efficient developer. After having exposed to the internals, many facts will make more sense to during the development, further you will get the clear picture of how each components connect and interact with each other.

Its been a while since the inception of the Android Mobile platform. Since then, there has been higher demand for android developers, not to mention how it made easy for existing Java developers to migrate. Statistics show, most mobile and embedded devices will default to Android soon or later.Learning this platform would be worth the time spend.

This article, will explain only necessary internal details just enough for user-space developer. Once complete we will have a clear picture of entire android stack starting from the source tree organisation, architecture component interaction, start up sequence, to how an application is launched.

1. Architecture

Fig 1 Conceptual Architecture

1.1 Kernel

A the bottom most level, the host operating system, is the Linux kernel.

Theoretically it could be any host kernel, practically, however it is a difficult task to achieve. Though, Linux kernel is being used, there were many kernel specific patches those were added to support various user level android policies and drivers.

Kernel patches (Details are skipped)

Wake Locks
Binder
Klogger
Anonymous Share Memory
Alarm Timers
Low Memory Killer
Network Security policy

Other than the patches above, one has to keep in mind, the device manufactures do add custom patches to support their hardware, what is mentioned above are core kernel patches for a android system to work.

Source Tree : kernel

1.2 Android Runtime

Consist of the Dalvik virtual machine, which is just like JVM on tbe desktop, that executes the android applications we run, Java runtime, that comes from the Apache Harmony project, system daemons, the init binary and various supporting binaries. Dalvik virtual machine executes dex files instead of class files. dex files are generated by post processing class files with dx utility.

Oracle Java = Java Language + class file + JVM + JRE libs

Android Java = Java Language + dex file + DVM + Apache Harmony Project

Most of the java.* , org.w3c.* junit.* org.json.* in the android API comes from here, including the sql jdbc driver

Source Tree: libcore , dalvik , system

1.3 Libraries

This is where, the 3rd party native libraries (C/C++) are hosted, they act as a support layer for low level access from the java libraries, it consist of google developed C library called Bionic, it is stripped down library from BSD standard C library. Bionic isn't compliance with POSIX, so native (non java) application and library porting from POSIX system is a hurdle. Though only few libraries have been mentioned in the diagram, it consist of more, starting from libusb to webRTC, kindly see the source tree.

The Application frame work access these libraries via JNI , one such example to site would be sqlite, we will see a source relation, check the links

Application Framework Sqlite Interface -> Sql-Lite-JNI -> Sql-Lite-Native-Library

So, in nutshell, these libraries are accessed by the application frame work via JNI.

Source Tree: external

1.4 Application Framework

Ah, finally we are at the top, the android.* APIs you see in the Android API reference sprouts from here, this is what we mostly use while developing an android applications. Framework don't just consist of developer apis, but, various services and interfaces to communicate with them. Services are not directly accessible via framework , however, via kernel supported mechanism called Binder. In short andorid.* APIs connected to service / system server via binder.

Source Tree: framework

1.5 Applications

There is were we have all the fun, we use the SDKs to access these framework APIs to develop the applications. Most of the application developers play here. There are other species of developers who hack the frameworks , android services and even the kernel subsystem to build custom roms, one such community is the CyanaogenMOD, we should say they are the first class android developers and us to be economy developers. ;-). Most android devices have set of default applications like dialers, mms, browsers you can find its source tree over here.

2. Android Hardware Abstraction Layer

Kernel does provide HAL , however Google engineers felt it wasn't enough, one reason could be GPL license that the kernel has. So, in order to make device manufacturers implement non GPLed drivers in the user space, this may be a favorable choice.

Libhardware provides interface for various devices and sensors, the device manufacturers can develop drivers in closed source environment, complying to these interfaces. Few of the interfaces are listed here.

Audio (libaudio.so)
Graphics (gralloc.so, copybit.so, libhgl.so)
Camera (libcamera.so)
GPS (libgps.so)
Sensors (libsensors.so)
Radio Interface (libril-vendor-version.so)

All that the manufacturer has to do is to put their library into /system/libs/hw/

Case Study : If you like to know how RIL is used for receiving calls, sending sms, receving sms etc take a look into this answer.

3. Android File System

Android though is Linux based doesn't strictly abide to the FHS , instead has a custom hierarchy

/system Immutable directory coming from the original build.

It contains native binaries and libraries, framework jar files,

configuration files, standard apps, etc.

/system/app All the pre-installed apps

/system/bin Binaries installed on the system (toolbox, vold, surfaceflinger)

/system/etc Configuration files

/system/fonts Fonts installed on the system

/system/framework Jar fi les for the framework

/system/lib Shared objects for the system libraries

/system/modules Kernel modules

/system/xbin External binaries

/data changing content of the system are put:

apps, data added by the user, data generated by all the apps at runtime, etc.

/etc Symbolic link to /system/etc

/dev Mount point for tmpfs filesystem

/proc Mount point for procfs filesystem which provide access to kernel data structures

/sbin Contains various daemons

/root As usual the root home directory

/mnt Mount point for external and internal filesystems

and some security related filessytem

/sys Mount point for sysfs filesystem which has information on

kernel's device objects

/sdcard Symbolic link to /mnt/sdcard

/cache Used by application to save data temporarily like Google Play Store download etc

/d Symbolic link to /sys/kernel/debug

/acct Mount point for the acct cgroup (control group),

which provides for user accounting.

/vendor Symbolic link to /system/vendor

4. Source Code Organisation Overview

bionic/ is where Android's standard C library is stored

bootable/ contains code samples regarding the boot of an Android device. In this folder,

you will find the protocol used by all Android bootloaders and a recovery image

build/ holds the core components of the build system

cts/ The Compatibility Test Suite

dalvik/ contains the source code of the Dalvik virtual machine

development/ holds the development tools, debug applications, API samples, etc

device/ contains the device-speci c components

external/ is one of the largest folders in the source code, it contains all the external

projects used in the Android code

frameworks/ holds the source code of the various parts of the framework

hardware/ contains all the hardware abstraction layers

libcore/ is the Java core library

ndk/ is the place where you will nd the Native Development Kit, which allows to build native

applications for Android

packages/ contains the standard Android applications

prebuilt/ holds all the prebuilt binaries, most notably the toolchains

sdk/ is where you will nd the Software Development Kit

system/ contains all the basic pieces of the Android system: init, shell, the volume manager, etc.

5. System Startup

Fig 2. start up sequence

Start up sequence till init is quite straight forward, this is similar usual x86 boot process, finally calling Init , however Android system differs starting from the init sequence. The init itself was written from scratch and not same as the one seen on Linux distribution.

Once the kernel start /init process, it reads the file name /init.rc and /init.(machine-name)

.rc , these rc script, first of all sets the environment variables, mounts various file system to its respective mount points, starts various daemons like rild (radio interface layer daemon), adb , vold, netd etc. Init become mother of all the process created subsequently. The next core job of init is to start up the Zygote, which is the mother of all the android applications.

Lets take a deeper view into Zygote start up,

Init runs the /system/bin/app_process and name it zygote.

service zygote /system/bin/app_process -Xzygote /system/bin --zygote --start-system-server

The entry above from ./init.rc that start the app_process to startup the zygote.

As seen in the source line 188 of app_main.cpp

188    if (zygote) {
189        runtime.start("com.android.internal.os.ZygoteInit",
190                startSystemServer ? "start-system-server" : "");
191    }

It start up the java implementation of Zygote , and 'start-system-server' flag is passed as the parameter. This is exactly what we had passed in the entry from the init.rc. The line 189 does many other tasks under the hood, the runtine.start() at line 839 start up up the Dalvik virtual machine, see line 443 of AndroidRuntime.cpp for more information.Ahead of zygote invocation, Dalvik runtime was setup and Dalvik virtual machine was initialized. ZygoteInit::main() get called at line 888 of AndroidRuntime.cpp , which start up the System Server this can be seen at line 544 of ZygoteInit.java, finally it runs runSelectLoopMode() and waits for future request to fork() new zygote instances which specialize into the android application.The System server is started in a two sequence process , first a libary is called to initialize the native services, which can be seen at line 1064 of SystemServer.java , then java based core services are run by ServerThread::run() which is seen at like 1067 of SystemServer.java. Check the source to find out the sequence of the services started, core services line the PowerManager, ActivityManager, Sensor Service etc and other non critical services are started this way. The main being the AcitivityManager, which handles the our Andorid Application.

501            /* Request to fork the system server process */
502            pid = Zygote.forkSystemServer(
503                    parsedArgs.uid, parsedArgs.gid,
504                    parsedArgs.gids,
505                    parsedArgs.debugFlags,
506                    null,
507                    parsedArgs.permittedCapabilities,
508                    parsedArgs.effectiveCapabilities);
509        } catch (IllegalArgumentException ex) {
510            throw new RuntimeException(ex);
511        }
512

544            if (argv[1].equals("start-system-server")) {
545                startSystemServer();
546            } else if (!argv[1].equals("")) {
547                throw new RuntimeException(argv[0] + USAGE_STRING);
548            }

                   Code Snippet Showing How System Server is started

Conclusion

Ah finally we have reached the end, though this was quick introduction into the android internals, it is my humble opinion that , information above would be sufficient enough to get you started with griping the Android APIs. This introduction will hopefully help you understand the relations and component interaction, which would make android application programming more exiting. Where go from here ? Now you can start reading Application Fundamentals from the android main resource, trust me, many things will make more sense now.

Reference

[1] Free Electrons Android System Development Training

[2] Android Internals Karim Yaghmour Slides

[3] Elinux Android

Git Build Number Generator

2017-02-17T09:30:00.000+05:30

This tool generate version or build information in a source file format from a pointing local git repository, during the pre-build phase. Currently only supports java code generation. However can be extended to a more generic plugin based system to support other programming languages.

Tool favors command line arguments instead of configuration files to read the parameters, this decision is to speed up the tool, as this tool may be run numbers of times in a time slice.

You can use this in your build script in pre build phase, use in eclipse without a build script is just a use case. Policy of when to run is left to your creativity.

Usage

gbn git-repo-directory version-file-directory package-name

git-repo-directory : Absolute path to local .git directory
version-file-directory : Absolute path to the directory, to place version source file
package-name : Java package name

eg: gbn /home/myhome/gbn/.git /home/myhome/gbn/src/com/alkber/gbn/version/

com.alkber.gbn.version

- VersionInfo.java -
package com.alkber.gbn.version;
/*
 * This is an auto generated file, modifications will not persist.
 * 2013/09/22 08:48:52
 */ 
public class VersionInfo {
                                                            
    public static final String buildNumber = "master.11.71740cc";
    public static final String branch = "master";
    public static final String commit = "11";
    public static final String version = "71740cc639811a5c4249f4344fe5e8586a7a494b";
    public static final String shortVersion = "71740cc";
                                                            
}
- VersionInfo.java -

It is highly advised to place .gitingore in the version-file-directory with .gitingnore content being 'VersionInfo.java'. As this file is updated frequently, it may cause git issues during commit and merge.Further to avoid git issues ignore the *class files.

Simplest Use Case

In eclipse, Goto Project->Properties->Builders , create a new builder

Location: [ path to java binary ]

eg: [ /home/myhome/bin/dk1.7.0_25/bin/java ]

Arguments: [ arguments to java binary]

eg: java -jar /home/myhome/bin/gbn.jar \ 
/home/myhome/git/Click2Limo/.git \ 
/home/myhome/git/Click2Limo/Click2Limo/src/com/frooday/click2limo/version/ \ 
com.frooday.click2limo.version

In the refresh tab of new builder setting specify the resource ie our version source file to be refreshed when ever tool is run.

In Build Options tab,

Enable-

Allocate Console
During Manual Build
During Auto Build
During Clean

Download Jar

Source

Java : convert milliseconds hours, minutes, seconds ,days

2017-02-15T15:13:00.000+05:30

An example using built in TimeUnit .

                            long uptime = System.currentTimeMillis();

                            long days = TimeUnit.MILLISECONDS
                                    .toDays(uptime);
                            uptime -= TimeUnit.DAYS.toMillis(days);

                            long hours = TimeUnit.MILLISECONDS
                                    .toHours(uptime);
                            uptime -= TimeUnit.HOURS.toMillis(hours);

                            long minutes = TimeUnit.MILLISECONDS
                                    .toMinutes(uptime);
                            uptime -= TimeUnit.MINUTES.toMillis(minutes);

                            long seconds = TimeUnit.MILLISECONDS
                                    .toSeconds(uptime);

`uptime:22hr 15min` 40sec 14 days

Android and Microsoft SQL (MS SQL SERVER 2008) marriage

2017-02-13T09:06:00.000+05:30

All you have to do is use the appropriate driver, being a opensource fan , i'd recommend using JTDS.

Instructions below assumes you are using

Android Developer Tools Build id: v22.0.1-685705,

Eclipse Version: 4.2.1.v20130118 Build id: M20130204-1200

JTDS 1.2.5

Though things will work out other versions higher than this and possibly lower as well , its just to ensure a known state where things worked out.

So, once the JTDS is jar is download, first step is to include jar to your build path

Next, it seems like a bug, go to order and Export tab and enable jtds

Since the database connection operation is networked related, all network operations should be in different thread away from the main UI thread, so use AsyncTask.

Working code is can be downloaded from here , read the code, it is well documented.

git@github.com:alkber/AndroidByExample.git

How to remember of relationships exponent, log and root ?

2017-02-12T21:02:00.000+05:30

There is a common relation between exponent, log and root, to know more about the intuition between these relation refer here. A simple trick to have all these relation derived is by just imagining this star.

This is meant for people with dyslexia, usually they get confused about these relation. Since we are really good with photo memory, this approach will help deduce these relations even in mind.

3 binary operators on the vertices of top triangle and 3 operands of bottom

^ for the exponent nth power
$\sqrt[n]{}$ for the nth root
${\log_n }$ log of nth base
b
x
y

Steps to find

Focus at the unknown variable of the context from the vertex of bottom triangle.
Follow the arrow at the the vertex of variable to the operator pointed by the arrow.
Apply the operation, while taking the operands in the clock wise order.

Finding x = b^y

Look at x , follow the arrow to the operator in the top triangle here ^

now apply the operation while taking the operands in the clock wise order

so b^y

Finding y = ${\log_b x}$

Look at y, follow the allow to the ${\log }$ now apply the operands in

clock wise order ${\log_b x}$

Finding b = $\sqrt[y]{x}$

left as an exercise to the reader.

How to use dokuwiki note plugin with blogger ?

2017-02-11T22:29:00.000+05:30

This is not exactly porting of docuwiki plugin to blogger, however bare metal approach which works neatly.

I love minimalist approach. If you have observed this blog, it has undergone various iteration to get into my type of theme. Fundamentally i'm inspired from the typeset and the black white dual tone from the research publications. It is so simple and lovely while conveying excellent ideas. Today i'm sharing how to setup blogger to have following note styles. I'm taking the inspiration from dokuwiki note plugin.

Below are the 4 types of pretty note types.

This is a sample of note important.

This is a sample of note warning.

This is a sample of note tip.

This is a sample of note classic.

Copy the entire CSS into Template > > Edit HTML

/* end plugin:configmanager */
.noteclassic, .noteimportant, .notewarning, .notetip {
  margin: 2em;
  margin-left: auto;
  margin-right: auto;
  width: 70% !important;
  min-height: 40px;
  clear: both;
  text-align: justify;
  vertical-align: middle;
  border-collapse: collapse;
  padding: 15px 20px 15px 80px;
  background-position: 20px 50%;
  background-repeat: no-repeat;
  -moz-border-radius: 20px;
  -khtml-border-radius: 20px;
  border-radius: 20px;
}
 
.noteclassic {
  /*border: 1px solid #99D;*/
  background-color: #eef;
  background-image: url(https://github.com/alkber/dokuwiki_note/blob/master/images/note.png?raw=true);
}
 
.noteimportant {
  /*border: 1px solid #ff0;*/
  background-color: #ffc;
  background-image: url(https://github.com/alkber/dokuwiki_note/blob/master/images/important.png?raw=true);
}
 
.notewarning {
  /*border: 1px solid #d99;*/
  background-color: #fdd;
  background-image: url(https://github.com/alkber/dokuwiki_note/blob/master/images/warning.png?raw=true);
}
 
.notetip {
  /*border: 1px solid #9d9;*/
  background-color: #dfd;
  background-image: url(https://github.com/alkber/dokuwiki_note/blob/master/images/tip.png?raw=true);
}

To use, when you are composing your blog post choose the HTML mode. Say you want to highlight something with a notetip type.

<div class="notetip">
This is a sample of note tip.
</div>

Found my L.K.G certificate ! yay

2017-02-10T18:23:00.000+05:30

It his really nostalgic to find our old things. I love the old texture and fragrance of the paper with golden fade. It bring back a lot of memories. We can take pride in the fact that we are the proud owner of paper that existed in 90s. I don't know if L.K.G is something known internationally. That expands to Little Kinder Garten.

UNIX (tm) Version History

2015-11-13T21:04:00.000+05:30

Well this about history of UNIX,main reason to write this is because of confusion about various versions and how they came to be.What ever given below are not my keystrokes rather from distributed sources like Wikipedia and others. What make this blog different , well i have compiled only necessary information for understanding version confusion.

---
Unix (officially trademarked as UNIX, sometimes also written as Unix with small caps) is a computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Douglas McIlroy, and Joe Ossanna.As of 2007, the owner of the trademark is The Open Group.Only systems fully compliant with and certified to the Single UNIX Specification are qualified to use the trademark; others are called "Unix system-like" or "Unix-like".Commercial start ups, the most notable of which are Solaris, HP-UX, AIX, and Mac OS X.

AT&T made Unix available to universities and commercial firms, as well as the United States government under licenses. The licenses included all source code including the machine-dependent parts of the kernel, which were written in PDP-11 assembly code. Copies of the annotated Unix kernel sources circulated widely in the late 1970s in the form of a much-copied book by John Lions of the University of New South Wales, the Lions' Commentary on UNIX 6th Edition, with Source Code, which led to considerable use of Unix as an educational example.

Version 5 and especially Version 6 led to a plethora of different Unix versions. Version 7 Unix, the last version of Research Unix to be released widely, was released in 1979. Versions 8, 9 and 10 were developed through the 1980s but were only released to a few universities.AT&T licensed UNIX System III, based largely on Version 7, for commercial use, the first version launching in 1982. This also included support for the VAX. AT&T continued to issue licenses for older Unix versions.To end the confusion between all its differing internal versions, AT&T combined them into UNIX System V Release 1.

Since the newer commercial UNIX licensing terms were not as favorable for academic use as the older versions of Unix, the Berkeley researchers continued to develop BSD Unix as an alternative to UNIX System III and V.the most important aspect of the BSD development effort was the addition of TCP/IP network code to the mainstream Unix kernel.

Other companies began to offer commercial versions of the UNIX System for their own mini-computers and workstations. Most of these new Unix flavors were developed from the System V base under a license from AT&T; however, others were based on BSD instead. One of the leading developers of BSD, Bill Joy, went on toco-found Sun Microsystems in 1982 and created SunOS for their workstation computers.

In 1991, a group of BSD developers (Donn Seeley, Mike Karels, Bill Jolitz, and Trent Hein) left the University of California to found Berkeley Software Design, Inc (BSDI). BSDI produced a fully functional commercial version of BSD Unix for the inexpensive and ubiquitous Intel platform, which started a wave of interest in the use of inexpensive hardware for production computing. Shortly after it was founded, Bill Jolitz left BSDI to pursue distribution of 386BSD, the free software ancestor of FreeBSD, OpenBSD, and NetBSD.A free derivative of BSD Unix, 386BSD, was also released in 1992 and led to the NetBSD and FreeBSD projects.

With the 1994 settlement of a lawsuit that UNIX Systems Laboratories brought against the University of California and Berkeley Software Design Inc. (USL v. BSDi), it was clarified that Berkeley had the right to distribute BSD Unix ¿ for free, if it so desired. Since then, BSD Unix has been developed in several different directions, including OpenBSD and DragonFly BSD.

Shortly after UNIX System V Release 4 was produced, AT&T sold all its rights to UNIX to Novell. (Dennis Ritchie likened this to the Biblical story of Esau selling his birthright for the proverbial "mess of pottage".[7]) Novell developed its own version, UnixWare, merging its NetWare with UNIX System V Release 4. Novell tried to use this to battle against Windows NT, but their core markets suffered considerably.

In 1993, Novell decided to transfer the UNIX trademark and certification rights to the X/Open Consortium. In 1996, X/Open merged with OSF, creating the Open Group. Various standards by the Open Group now define what is and what is not a "UNIX" operating system, notably the post-1998 Single UNIX Specification.

Beginning in the late 1980s, an open operating system standardization effort now known as POSIX provided a common baseline for all operating systems; IEEE based POSIX around the common structure of the major competing variants of the Unix system, publishing the first POSIX standard in 1988. In the early 1990s a separate but very similar effort was started by an industry consortium, the Common Open Software Environment (COSE) initiative, which eventually became the Single UNIX Specification administered by The Open Group. Starting in 1998 the Open Group and IEEE started the Austin Group, to provide a common definition of POSIX and the Single UNIX Specification.

In 1983, Richard Stallman announced the GNU project, an ambitious effort to create a free software Unix-like system; "free" in that everyone who received a copy would be free to use, study, modify, and redistribute it. The GNU project's own kernel development project, GNU Hurd, had not produced a working kernel, but in 1992 Linus Torvalds released the Linux kernel as free software under the GNU General Public License. In addition to their use in the Linux operating system, many GNU packages ¿ such as the GNU Compiler Collection (and the rest of the GNU toolchain), the GNU C library and the GNU core utilities ¿ have gone on to play central roles in other free Unix systems as well.

Linux distributions, comprising Linux and large collections of compatible software have become popular both with individual users and in business. Popular distributions include Red Hat Enterprise Linux, Fedora, SUSE Linux Enterprise, openSUSE, Debian GNU/Linux, Ubuntu, Mandriva Linux, Slackware Linux and Gentoo.

128 Bit bitmap operations

2015-03-10T14:13:00.000+05:30

I had to maintain a '128 bit' bitmap with

set
clear
is_set

operations , what came to my head was (128 / 8) = 16

ie if i declare

'unsigned char bitmap[16]'

Q. Why didn't i choose (128 / 32) = 4 ?

A. if the code be portable across other platforms, '8 bit' should be a logical choice , as most lower ended arch at most 8 bit.

Now to the code .

//Version 1
void bitmap8_set_bit(unsigned char *bitmap, unsigned int bit)
{
*bitmap |= (1 << bit);
}

void bitmap8_clr_bit(unsigned char *bitmap, unsigned int bit)
{
*bitmap &= (0xff & ~(1 << bit));
}

int bitmap8_is_bit_set(unsigned char *bitmap, unsigned int bit)
{
return ( ( (*bitmap) & (1 << bit) ) ? 1 : 0 );
}

//Version 2
void bitmap8_set_bit(unsigned char *bitmap, unsigned int bit)
{
bitmap[bit / 8] |= (1 << (bit % 8));
}

void bitmap8_clr_bit(unsigned char *bitmap, unsigned int bit)
{
bitmap[bit / 8] &= (0xff & ~(1 << (bit % 8)));
}

int bitmap8_is_bit_set(unsigned char *bitmap, unsigned int bit)
{
return ( (bitmap[bit / 8] & (1 << (bit % 8)) ) ? 1 : 0 );
}

int bitmap8_next_free_bit(unsigned char *bitmap)
{
register int i;

for (i = 0;i < 128; i++) {
if (0 == is_bit_set(bitmap, i))
return i;
}
return i;
}

Q. What is the big deal in the Version 2 ?

Declared 'unsigned char bitmap[16]' , i have to properly index the bitmap array to get the desired result, but the fundamental operation remains the same.

ie bitmap[bit / 8] -> would for bit = 16 would map to bitmap[2]

(bit % 8) -> (16 % 8) = 0 ,

which is exactly what we want.

SSH Login without password

2014-08-30T09:19:00.000+05:30


you@localhost$ ssh-keygen -t rsa -b 2048
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa): /home/username/.ssh/id_rsa.my_jee_server
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/username/.ssh/id_rsa.my_jee_server
Your public key has been saved in /home/username/.ssh/id_rsa.pub.my_jee_server.pub

Copy keys to server:

$ ssh-copy-id -i /home/username/.ssh/id_rsa.my_jee_server.pub you@server
you@server's password:

Try login

you@localhost$ ssh you@server
you@server:~$

Try copy

you@locahost$ scp ~/you_servlet.java you@server:/home/username

How to know if Linux swap is working

2013-07-12T12:25:00.000+05:30

After installation of Ubuntu 12.04 LTS, its seems like my swap was never, used, most of the time it remained inactive in my system monitor. In order to ensure, if swap is working, one trick is to write a C code that fills up the available RAM and ensure, if swap is being used.

The code below, allocates 10 MB per allocation, do monitor the swap.[1]

 #include <stdlib.h>  
 #include <stdio.h>  
 #include <string.h>  
 int main(int argc, char** argv) {  
   int max = -1, mb = 0;  
   char* buffer;  
   if(argc > 1)   
     max = atoi(argv[1]);  
   else {  
      printf("\nmeater <size in MB> eg: meater 2048\n");  
       exit(0);       
   }  
   printf("\n[INFO] attempting to allocate %d MB @ 10 MB\n", max);  
   while((buffer=malloc(10*1024*1024)) != NULL && mb != max/10) {  
     memset(buffer, 0, 10*1024*1024);  
     mb++;  
     printf("[INFO] allocated %d MB\n", mb*10);  
     sleep(1);  
   }  
   printf("[INFO] meater done eating, yummy..!\n");  
   return 0;  
 }

Let's get Solarized

2012-11-13T22:48:00.000+05:30

Ethan Schoonover

Solarized is a sixteen color palette (eight monotones, eight accent colors) designed for use with terminal and gui applications. It has several unique properties. I designed this colorscheme with both precise CIELAB lightness relationships and a refined set of hues based on fixed color wheel relationships. It has been tested extensively in real world use on color calibrated displays (as well as uncalibrated/intentionally miscalibrated displays) and in a variety of lighting conditions.

Now let start Solarizing

Gnome-Terminal
Google-Chrome
Eclipse

Dowload https://bitbucket.org/LukeHoersten/eclipse-solarized-color-theme
Install Eclipse Color Theme plugin from the market
Preferences > General > Appearance > Color Theme , apply the solarized xml file.

Vim
Pidgin

Download https://github.com/heichblatt/gtk2-theme-solarizeddark
Replace /home//gtkrc-2.0 with gtkrc

cp gtkrc ~/.purple && cd ~/.purple && mv gtkrc-2.0 gtkrc-2.0.bkp && mv ./gtkrc ./gtkrc-2.0

Local Data address Does not belong to any of this hosts local interfaces

2012-09-17T15:31:00.001+05:30

byte[] ip = new byte[4];
        ip[0] = (byte) 192;
        ip[1] = (byte) 168;
        ip[2] = (byte) 1;
        ip[3] = (byte) 8;
SessionAddress localSessionAddr = new SessionAddress(
                InetAddress.getByAddress(ip), LOCAL_RTP_PORT);

Throws: Local Data Address Does not belong to any of this hosts local interfaces

Solution and Cause

The problem is JMF seems to use InetAddress.getAllByName() which gives (in most cases) only single IP address as it may not be in the /etc/hosts. It can be sorted out by setting your interface ip (say eth0, wlan0) in /etc/hosts [1].

>  cat /etc/hosts> 127.0.0.1 localhost> 127.0.1.1 noor> 192.168.1.8 noor

Slackware and manual laptop screen off

2011-06-03T09:44:00.004+05:30

Sometime you may have large compilation (kernel?) / download, yea and you are bored to do anything else on your laptop, if so you may have puzzled about how to manually switchoff the laptop screen. heh, if you haven't i have ;-).

We make use of 'xset' and 'dpms' to handle this, i have assigned a keyboard shortcut for my KDE.I have also anticipated about the security so i use 'xlock' too.

xlock &  xset dpms 1200 1800 2100 ; sleep 2 ; xset dpms force off;

Assign the above command sequence to a keyboard short cut, your DE will have such a feature, KDE and xfce do.

Slackware 13.37 root and xapplications

2011-05-31T10:33:00.004+05:30

In slackware, we have few applications like pkgtool, sbopkg etc that has to run as root, however always using the console to punching in sudo is difficult, one way to over come is to execute them with a desktop link, however since the apps are executed in a auxiliary terminal emulator like xterm/konsole running in non-privileged user's xsession we have an issue, root can't connect to the xserver unless we allow. This might be the error you might see

sudo /usr/bin/firefox
No protocol specified
No protocol specified
Error: cannot open display: :0

Do the following to authorise unrestricted remote connection to x

xhost +

Consider the security implications of running xhost + read 'man 1 xhost'

Now you can enjoy running

sudo /usr/bin/konsole -e /usr/sbin/sbokpg

Slackware 13.37 and BroadComm [14e4:4727] wireless Marriage

2011-05-29T13:17:00.006+05:30

I have a Broadcom Wireless card, which was obvious enough not to work just like that.

root@noor:/home/althaf# lspci -vnn | grep 14e4
12:00.0 Network controller [0280]: Broadcom Corporation BCM4313
02.11b/g/n Wireless LAN Controller [14e4:4727] (rev 01)

Follow these procedures to make it work

Kernel configuration for (2.6.37 slackware 13.37 default)

Make sure you have enabled

Location: -> Device Drivers
-> Staging drivers
  -> Exclude Staging drivers from being built
      -> Broadcom IEEE802.11n WLAN drivers
          -> Broadcom IEEE802.11n driver style

Download the firmware

I was luck enough to have this card, well the broadcom guys had the mercy to open up their driver code, any way as usual it required firmware to be loaded.

[Firmware link]

We just need *bcm43xx-0.fw* and *bcm43xx_hdr-0.fw*, when downloaded make sure you have the same file name as given, the drivers have hard coded the path and firmware name in them, so it should be the same to work.

Copy them to /lib/firmware/brcm

Reboot

I highly recommend rebooting the system, though theoretically we could load the firmware on the go, however it didn't work for me.Once rebooted you will find wlan0 or similar interface.

References
[Brcm80211]

Slackware 13.37 hibernation

2011-05-04T02:30:00.004+05:30

Slackware doesn't provide a out of the box configuration to hibernate even though stock kernel is capable.

Few modifications to your lilo configuration could enable hibernation.

To do so edit your /etc/lilo.conf with the following, assuming that your are using the default configuration from slackware.

image = /boot/vmlinuz
root = /dev/sda1
label = Slackware(Res)  
#for resuming /dev/sda6 is the my swap
append = "resume=/dev/sda6" 
read-only

After saving the file re-run lilo to apply the changes.

Slackware - simplicity is fun

2009-04-11T20:09:00.000+05:30

Well as far my exposure to various Linux distros is concerned, Slackware resonated with my frequency. I felt intimidate connection between me a the philosophy of Slackware.

“ Slack, which generally stands for the sense of freedom, independence, and original thinking that comes when you achieve your personal goals ”.

So the meaning Slackware is obvious , till now revealed that its no easy for a newbie. Though my intention of using is to learn Linux from its raw form.It does provide an environment for learning Linux. Learning Linux means not the kernel alone but the utilities as well.

Choose Slackware

to learn Linux from scratch.
to tweak each and every settings.
to be a fan of simplicity.
if you have killer instinct to solve issues.

Slackware isn't a difficult to conquer once you have understood the UNIX philosophy. It is the best one I have come across for the purpose of learning Linux .Of course it is still powerful for other purposes.

For support /join IRC freenode ##slackware channel.

Althaf's Journal

Is Spring a full replacement for J2EE?

Is Spring a full replacement for J2EE?

What parts of J2EE does Spring replace?

What parts of J2EE/Jakarta EE does Spring still depend on?

✔ Spring still uses Servlet API

✔ Spring integrates with JPA API

✔ Spring integrates with JTA (optional)

✔ Spring can use JMS API

Important distinction

Spring does NOT require a Java EE Application Server.

Analogy (very intuitive)

Java = The languageJ2EE = A big enterprise machine with many heavy partsSpring = A modern toolkit that replaces 90% of that machine with lightweight toolsSpring Boot = Spring + Auto-config + Built-in engine + Ready to drive

Final Answer

What is WEB-INF folder in Java Enterprise Application ?

Understanding LLMs from Scratch Using Middle School Math

What will we cover?

A simple neural network:

How are these models trained?

How does all this help generate language?

What makes large language models work so well?

Embeddings

Subword Tokenizers

Self Attention

Softmax

Residual connections

Layer Normalization

Dropout

Multi-head Attention

Positional encoding and embedding

The GPT architecture

The transformer architecture

Appendix

Matrix Multiplication

Standard deviation

Positional Encoding

Everything you need to know about Design Systems

What is a Design System?

Style Guide or Pattern Library: what’s the difference?

Why are we talking about that today?

What’s inside?

#1. Purpose and shared values

#2. Design Principles

#3. Brand identity & language

#4. Components & patterns

Best practices

What kind of Design System?

Strict or loose?

Modular or integrated?

Centralized or distributed?

Place your system on a scale

Some examples

Last words

Design Systems Repo

This is not a comprehensive list of every design system that I have come across. Rather, it’s a collection of ones that…

What is the difference between JSF, Servlet and JSP?

StateFlow and SharedFlow seem similar, but they are designed for different use cases

MVP : Event Bus pattern instead of Listener

Android Internals : just what application developers needs to know

1. Architecture

Git Build Number Generator

Simplest Use Case

Java : convert milliseconds hours, minutes, seconds ,days

uptime:22hr 15min 40sec 14 days

Android and Microsoft SQL (MS SQL SERVER 2008) marriage

How to remember of relationships exponent, log and root ?

Steps to find

How to use dokuwiki note plugin with blogger ?

Found my L.K.G certificate ! yay

UNIX (tm) Version History

128 Bit bitmap operations

SSH Login without password

How to know if Linux swap is working

Let's get Solarized

Local Data address Does not belong to any of this hosts local interfaces

Slackware and manual laptop screen off

Slackware 13.37 root and xapplications

Slackware 13.37 and BroadComm [14e4:4727] wireless Marriage

Slackware 13.37 hibernation

Slackware - simplicity is fun

Java = The language
J2EE = A big enterprise machine with many heavy parts
Spring = A modern toolkit that replaces 90% of that machine with lightweight tools
Spring Boot = Spring + Auto-config + Built-in engine + Ready to drive

`uptime:22hr 15min` 40sec 14 days