Hiren's Technical Blog

NodeJs Read XML file and Parse Data

2022-10-02T16:45:00.003+05:30

Hello,

Few days back I worked on task to read XML file in NodeJs and parse the data and convert data to the JSON format. Here in this blog I am going explain how I did achieved this.

Following is our sample XML file.

<root>

<parent>

<firstchild>First Child Content 1</firstchild>

<secondchild>Second Child Content 1</secondchild>

</parent>

<parent>

<firstchild>First Child Content 2</firstchild>

<secondchild>Second Child Content 2</secondchild>

</parent>

</root>

Step 1 : Install necessary NPM packages

First install following packages in your NodeJs application.

npm install --save xmldom

mpm install --save hashmap

Here xmldom is the package we are going to use to parse the XML data and hasmap package we are going to use to convert and save data in JSON format.

Step 2 : Import necessary packages in script

const { readFile } = require('fs/promises');

const xmldom = require('xmldom');

const HashMap = require('hashmap');

We are going to use readFile from fs/promises. Also we will use xmldom to parse the string data. xmldom is a A JavaScript implementation of W3C DOM for Node.js, Rhino and the browser. Fully compatible with W3C DOM level2; and some compatible with level3. Supports DOMParser and XMLSerializer interface such as in browser.

Step 3 : Read the XML file

const readXMLFile = async()=> {

var parser = new xmldom.DOMParser();

const result = await readFile('./data.xml',"utf8");

const dom = parser.parseFromString(result, 'text/xml');

}

Here we are using async function to read the XML file as we want to wait till XML file is completely finished. Also we created a xmldom.DOMParser which we will use to parse the string data of the file. This parser will give you all the TAGs of the XML file just like we access standard HTML tags. With this we can use getElementsByTagName to get XML tags.

Step 4 : Convert XML data to hasmap

const readXMLFile = async()=> {

var parser = new xmldom.DOMParser();

const dataMap = new HashMap();

const result = await readFile('./data.xml',"utf8");

const dom = parser.parseFromString(result, 'text/xml');

var parentList = dom.getElementsByTagName("parent");

for(var i =0; i < parentList.length; i++) {

const parent = parentList[i];

const firstchild = parent.getElementsByTagName("firstchild")[0].textContent;

const secondchild = parent.getElementsByTagName("secondchild")[0].textContent;

const parentMap = new HashMap();

parentMap.set("firstchild", firstchild);

parentMap.set("secondchild", secondchild);

dataMap.set(i, parentMap);

}

In the above function we got list of parents by getElementsByTagName method and then we are iterating through it and accessing the child tags and getting it's text content. Then we are simply storing it in hasmap with unique key.

Hope this helps you.

Mock node-fetch with JEST

2022-07-03T11:25:00.001+05:30

Recently I tried my hands on Jest - the popular JavaScript Testing Library. In my application we were using node-fetch to make API calls. I used Jest to mock and test this API calls. Here in this blog I will explain on how to mock node-fetch with Jest.

Following is my function to call the GET API.

fetch_get.js

const fetch = require('node-fetch');

module.exports = async () => {

return await fetch('http://YOUR_GET_URL').then(res => res.json());

};

Now to test this create a test file with name test.fetch_get.js

Step 1: First import the function.

const fetch_get = require('../fetch_get');

Step 2 : Mock the node-fetch with Jest

const fetch = require('node-fetch');

jest.mock('node-fetch', ()=>jest.fn())

Step 3: Create Mock response

const mockedRes = {

"success": true,

"data": [

{

id: 1,

text: 'Test Data 1'

{

id: 1,

text: 'Test Data 2'

}

]

};

Step 4: Add Test

describe("Test get fetch", () => {

let data;

it('It should return the data', () => {

const response = Promise.resolve({

ok: true,

status: 200,

json: () => {

return mockedRes;

});

fetch.mockImplementation(()=> response)

data = await fetch_get();

expect(data).toEqual(mockedRes);

});

In above code we have created a rest and mocked the response to return our response when we call .json() method for response. Then we are simply comparing the response. This is a very simple test. Here in stead of comparing data, you may have other tests like checking length of data or checking specific data etc.

Step 5: Run Test

Run the test with npm test or npm run test and you will see the following result.

Test get fetch

✓ It should return the data (17 ms)

Test Suites: 1 passed, 1 total

Tests: 1 passed, 1 total

Snapshots: 0 total

Time: 0.313 s, estimated 1 s

Hope this helps you.

How to Execute Shell Script from Dockerfile

2022-01-29T16:32:00.001+05:30

Recently I faced a problem with Docker build. Before we build the image there was a requirement to cleanup something from MongoDB. For that we tried to create shell script and execute it from Dockerfile before we start the service. Here in this blog I will explain how you can execute the shell script from Dockerfile.

For that first create the shell script in the same context of Dockerfile. That means the file should be in the same folder or subfolder where Dockerfile is there.

We can not access file outside the Docker build context so file has to be in the same folder.

nano test.sh

You can put your content in this file for the tasks you want to do with this file.

Now first we will copy the file in docker build context. So that it can be executed. This step is mandatory or else it will not find file. Add following lines of code in your Dockerfile.

FROM alpine3.15

COPY test.sh ./test.sh

Next step we have to change access rights of the script so it can be executed.

RUN chmod +x ./test.sh

Now since we have file in the Docker build context and it has proper access rights we can execute the file.

RUN sh ./test.sh

That's it and now when to service starts it will execute the file and do all the actions which you have added in shell script.

Hope this helps you.

Kubernetes ImagePullBackOff error

2022-01-03T21:44:00.003+05:30

Recently I faced an issue while working Kubernetes cluster. I was trying to build docker image from local repository to test some changes in production before we merge the code. After building the docker image and recreating the container it was showing ImagePullBackOff error when I check the status with

kubectl get pods

In this blog I am going to mention the possible cause for this error and will explain the resolution which worked for me.

In a Kubernetes, there’s an agent on for node which is called kubelet. This is responsible for running containers in the nodes. For some reason if image can't be pulled the kublet will throw ImagePullBackOff error.

There are some possible reasons for this when kubectl is not able to pull image from mentioned registry.

Either image or tag is not available.

The name of the image or tag is wrong in deployment yaml file

kubectl does not have access to image registry.

In my case there was no such issues mentioned above. The image with tag was there. I have specified the correct name in deployment yaml file and because it was a local registry, there was no need of authentication.

So here is how I solved this issue. I set image pull policy to following value in my deployment yaml file.

imagePullPolicy: IfNotPresent

The reason it worked is, the kubectl was trying to find image in registry but it was not present because it was local registry so then it tried to find image in local repository and caches it.

This solution worked for me because I was using local registry. If you are facing such issue try setting above value to imagePullPolicy and it may work. Hope this helps you.

Wait for State Changes with Redux Saga

2021-07-31T15:14:00.002+05:30

Hello,

As we know Redux saga allows us to respond to actions outside Redux store. It's kind of middleware library to handle side effects with ease. This side effect could be anything like calling APIs, doing some IO operations or anything. While working with Saga sometime it happens that we also want to detect state changes and do some actions after it. In this blog I am going to explain how you can wait or detect Redux state changes with Saga.

Let's understand with an example. For example there is an API call to get access token and as soon as you have access toke available in state you want to call some other actions like background sync etc.

function* waitForLoginToken(selector) {

if (yield select(selector)) return;

while (true) {

yield take('*');

if (yield select(selector)) return;

}

export function* postSignInAction(payload) {

cost loginTokenSelector = state => state.auth.loginToken;

yield call(waitForLoginToken, loginTokenSelector);

yield put({ type: 'BACKGROUND_SYNC' });

}

function* mySaga(){

yield takeEvery('SIGN_IN', postSignInAction);

}

As you can see in above code, we take every sign in action and call postSignInAction which will wait for state change.

We defined state selector as state.auth.loginToken. Till the time loginToken is not set in state.auth it will wait as we are using while loop to wait for state change. Once we get it in state we call next action.

This way you can & detect state change with Redux Saga. Hope this helps you.

React-Draft-Wysiwyg - Find Text Option

2021-07-17T16:08:00.003+05:30

Hello,

Recently I was working on POC for rich text editor using React-Draft-Wysiwyg editor.

There we have a requirement to add find & replace dialog box which should find the matching text from all the blocks and height it with background color. For the POC we added find text functionality. Here in this blog I will explain how to do it.

For this I have created separate React component and add it as custom option to editor toolbar. Let's understand it step by step.

If you want to know how to add custom option you can find reference from here.

https://jpuri.github.io/react-draft-wysiwyg/#/docs

1) Create Modal with Text input and Button

Please note I have used grommet UI library to build the POC. So text input and button was imported from Grommet.

<TextInput

data-cy="find-text"

autoFocus

onChange={this.onChangeFindText}

value={this.state.findText}

<Button

id="find"

label="Find"

onClick={this.onFindClick}

2) Handle Find Click Event

onFindClick = () => {

const search = this.state.findText;

let contentState = convertToRaw(this.props.editorState.getCurrentContent());

contentState.blocks.forEach(block => {

const text = block.text;

const matches = [...text.matchAll(search)];

if(matches.length > 0) {

matches.forEach(match =>{

block.inlineStyleRanges.push({length: match[0].length,

offset: match.index,

style: "bgcolor-rgb(247,218,100)"

})

});

}

});

let newEditorState = {

editorState: EditorState.createWithContent(convertFromRaw({blocks: contentState.blocks,entityMap: {}})),

contentState: null

}

this.props.onChange(EditorState.createWithContent(convertFromRaw({blocks: contentState.blocks,entityMap: {}})));

}

In above code we are getting search text from the state and matching it with texts of all custom blocks. Where ever it matches we are setting inline styles in blocks with yellow color background. and then setting state of editor again with new blocks.

Hope this helps you.

Docker MongoDB terminates when it runs out of memory

2021-07-04T15:55:00.000+05:30

When you have multiple services running in docker container it's quite possible that you have an issues with certain services when your docker container runs out of memory. MongoDB is one such service.

On docker container when you have MongoDB running and when it starts storing huge data it starts consuming lots of memory and that's where you have an issue. MongoDB will crash after sometime where isn't much memory left.

The reason behind this is the IO model of MongoDB, it tries to keep as much data as possible in cache so read and write operations are much faster. But this creates an issue with docker as we have limited memory and lots of services are sharing that.

Starting from MongoDB 3.2 on words WiredTiger storage engine is the default one for MongoDB and it's recommended.

There are various advantages of WiredTiger storage engine. For example,

Document Level Concurrency
Snapshots and Checkpoints
Journal
Compression
Memory Use

One of most useful feature is Memory use.

With WiredTiger, MongoDB utilizes both the WiredTiger internal cache and the filesystem cache.

You can control it with --wiredTigerCacheSizeGB configuration.

The --wiredTigerCacheSizeGB limits the size of the WiredTiger internal cache. The operating system will use the available free memory for filesystem cache, which allows the compressed MongoDB data files to stay in memory. In addition, the operating system will use any free RAM to buffer file system blocks and file system cache.

With this setting you can enhance memory usage. MongoDB will not use excessive memory and with heavy data usage on docker container MongoDB will not crash on excessive memory usage.

Hope this helps you.

ReactJs Peer to Peer Communication

2021-07-04T15:15:00.001+05:30

Recently I evaluated peer to peer communication approach for one of my project so here I am going to share it and how you can use it in your project in case you want to implement peer to peer communication in your project.

We used library called PeerJS , it's simple peer to peer built on top of webRTC. For this first you have to create a server, which will act as only connection broker. No peer to peer data goes through this server. Let's just create a simple server.

Let's first install peer from npm.

npm install peer

Now let's create NodeJs script.

const { PeerServer } = require('peer');

const peerServer = PeerServer({ port: 9000, path: '/server' });

peerServer.on('connection', (client) => {

console.log(client);

});

That's is now you can run this script through terminal and it will run your server on 9000 port.

Now let's connect to server from our ReactJS component.

First lets install peerjs npm which is peer client.

npm install peerjs

We can connect to server in componentDidMount method and add some callbacks function.

import Peer from 'peerjs';

componentDidMount = () => {

this.peer = new Peer("USERNAME", {

host: 'localhost',

port: 9000,

path: '/server'

});

this.peer.on("error", err => {

console.log("error: ", err)

})

this.peer.on("open", id => {

console.log(id)

})

this.peer.on("connection", (con) => {

console.log("connection opened");

con.on("data", i => {

console.log(i)

});

})

}

In above code first function is the error callback function. Second once is when peer connection is opened. Third one is when you receive connection from some other peer and get some data.

Now let's take an example of how you can connect to other peer and send data.

const conn = this.peer.connect('REMOTE_PEER');

conn.on('open', () => {

conn.send('DATA');

});

In above code we are connecting to some remote peer and sending some data to it.

Please note here peer to peer data goes through ICE server which you can setup and assign when you create peer server or else it will use PeerCloud server by default. For the development purpose that's ok but for the production you should create your on TURN or STUN server.

Hope this helps you in setting up

Create Question And Answer NLP Model With Bert

2021-06-26T18:13:00.001+05:30

Hello,

Recently I worked on POC for chatbot where I evaluated Question Answering with Bert. Here in this blog we will see how you can create Question, Answering with Bert.

What is Bert?

According to team, who developed Bert

BERT stands for Bidirectional Encoder Representations from Transformers. It is designed to pre-train deep bidirectional representations from unlabelled text by jointly conditioning on both left and right context. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of NLP tasks.”

Bert is pertained on massive dataset and large corpus of unlabelled text. That's the hidden power of Bert as it uses knowledge gained from pre-training and apply it to dataset given.

For this POC we used HuggingFace's transformers. So first you have to install transformers. Using this model you can get advantage of pre trained data and then you can pass your reference text to it and this model will try to find answers from it.

pip install transformers

pip3 install transformers

Because this models are very big and it takes time to load and download. Let's first save it. Create model.py file and add following code.

from transformers import BertForQuestionAnswering

from transformers import BertTokenizer

BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad').save_pretrained('./trainedModel')

BertTokenizer.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad').save_pretrained('./trainedModel')

Now execute it with python command. It will create trainedModel directory and save model there with all required files. Now we can load this pertained saved model.

from transformers import BertForQuestionAnswering

from transformers import BertTokenizer

import torch

bertPreTrainedModel = BertForQuestionAnswering.from_pretrained('./trainedModel')

tokenizer = BertTokenizer.from_pretrained('./trainedModel')

encoded = tokenizer.encode('YOUR_QUESTION', 'YOUR_REFERENCE_TEXT')

tokens = tokenizer.convert_ids_to_tokens(encoded)

sepLocation = encoded.index(tokenizer.sep_token_id)

first_seg_len, second_seg_len = sepLocation + 1, len(encoded) - (sepLocation + 1)

seg_embedding = [0] * first_seg_len + [1] * second_seg_len

modelScores = bertPreTrainedModel(torch.tensor([encoded]), token_type_ids=torch.tensor([seg_embedding]))

ans_start_loc, ans_end_loc = torch.argmax(modelScores[0]), torch.argmax(modelScores[len(modelScores)-1])

result = ' '.join(tokens[ans_start_loc:ans_end_loc + 1])

result = result.replace(' ##', '')

Here you will get your answer in result. This way you can develop your model using BERT and transformers.

How to Create Custom Theme for UI using Grommet V2

2021-06-26T17:33:00.003+05:30

Hello,

In this blog we will understand how you can create custom theme for your using Grommet V2 in your React app.

Using Grommet v2, you can leverage the theme capabilities and define a customTheme as a JS object with your desired colors, margin padding etc. Let's see how we can do this.

First define your custom theme as JS object. As you can see we have defined some existing grommet color like background, brand. Also we have defined our own color like custom-1, custom-2 etc and then used it in button.

const customTheme = {

global: {

colors: {

background: "#fff",

brand: "#000",

"custom-1": "#aaaaaa",

"custom-2": "#bbbbbb",

"custom-3": "#444444"

}

button: {

default: {

background: "brand",

color: "white"

primary: {

background: "custom-1",

color: "white"

secondary: {

background: "custom-3",

color: "dark-2"

}

For more information on theme customisation and JSON structure you can visit Official Grommet site

Now you can apply this theme to your application as below.

import { Grommet, Box } from 'grommet';

const DemoGrommet = () => (

<Grommet theme={customTheme}>

<Box background="custom-1" >

</Box>

</Grommet>

);

export default DemoGrommet;

In case you want to keep default values of Grommet theme and want to override only few things with your theme, you can use deepMerge function provided by grommet v2 utils.

import { deepMerge } from 'grommet/utils';

import { generate } from 'grommet/themes/base';

const mergedTheme = deepMerge(generate(16), customTheme);

Here generate function generates default theme with 16 pixel font size and then it merge it with your custom theme and provide you updated theme.

How to Get Video Thumbnails with Javascript

2021-06-26T16:32:00.000+05:30

Hello,

In this blog we will see how you can generate thumbnail of video using JavaScript. Let's assume you have file upload in your HTML as following.

<input type="file" id="uploadVideo" accept="video/mp4" />

Now let's add an onChange listener for this.

document.querySelector("#uploadVideo").addEventListener('change', function() {

var inputFile = document.querySelector("#uploadVideo").files[0];

getVideoThumbnail(inputFile)

});

Now when you upload video it will call this function and you can access this video by using files property. Now we got the file. First step we will do is get blob URL of video so we can access metadata of video. Check below function.

Here first what we are doing is creating video element and assigning blob URL to it from input file using URL.createObjectURL method.

Then we have added event listener for meta data. This is necessary as we can't get video data till them time meta data is loaded.

Then we seek video to 0.0 second. You can add your own time here in case if you need. Then we have added seeked listener. By this time we have video data is available. Now we are using canvas to drawImage from video frame data.

Then using toDataURL method we get the base64Data. Here 0.60 is value of quality. If you don't want to compromise quality of image then you can pass 1 here.

function getVideoThumbnail(inputFile) {

try {

let video = document.createElement('video');

let blobSrc = URL.createObjectURL(inputFile);

video.setAttribute('src', blobSrc);

video.addEventListener('error', () => {

console.log("Error")

});

video.addEventListener('loadedmetadata', () => {

setTimeout(() => {

video.currentTime = 0.0;

}, 200);

video.addEventListener('seeked', () => {

let canvas = document.createElement('canvas');

video.pause();

canvas.width = 400

canvas.height = 400;

let ctx = canvas.getContext('2d');

ctx.drawImage(video, 0, 0, canvas.width, canvas.height);

var base64Data = ctx.canvas.toDataURL(

'image/png',

0.60

);

// SET this data in SRC of image tag or you can add your own logic.

});

video.load();

} catch (err) {

}

Hope this helps you.

How to solve proxy_fcgi:error AH01071: Got error 'Unable to open primary script

2021-03-19T18:29:00.004+05:30

Hello,

Recently I got stuck with strange bug on centOS server. After I updated document root for my primary domain which has Laravel App installed.

When we try to access app through domain, we got following error.

No input file specified.

I initially thought it's the permission issue on .htaccess issue. Tried changing permissions and adding some rewrite rules but it didn't work. When I checked apache error logs. I got following error.

proxy_fcgi:error AH01071: Got error 'Unable to open primary script

I had no idea how to solve this error. So tried few things like

Restart apache
Restart php-fpm
Restart VPS
Removed and added account few times.
Disabling php-fpm

But nothing worked. I spent almost entire night in solving this issue but no luck. But finally I was able to solve the issue by following steps. It was actually the issue with php fpm, as it was not able to find out root folder for the domain. So here is what you have to do.

First go to userdata directory

/var/cpanel/userdata/<USERNAME>

Here you will find one file.

yourdomain.com.php-fpm.yaml

Open this file with nano or vim editor and add following line at the end of it.

php_admin_value_doc_root: { name: 'php_admin_value[doc_root]', value: /home/<USERNAME>/public_html/<DOCUMENT_ROOT> }

Save the file and then execute following commands one by one.

/scripts/php_fpm_config --rebuild

/scripts/restartsrv_apache_php_fpm

/scripts/restartsrv_httpd

This will rebuild php fpm config and restart php fpm service for apache. Now in the same folder open

YOURDOMAIN.com

YOURDOMAIN.com_SSL

Change document root here as well and run following commands.

/scripts/rebuildhttpdconf

/scripts/restartsrv_httpd

This will rebuild apache config file and restart apache.

This is it and now if you visit your domain, you will not have the issue of file missing.

Hope this saves your time

Restore MySql Database From Just frm and idb File

2020-12-09T17:48:00.004+05:30

Hello,

Few days back, I encountered a situation where we needed to restore MySql Database but we only had frm and idb file. Didn't have .sql file or log file from MySql server which is very much needed when you want to restore. So here in this blog I am going to explain how you can still restore MySql Database from only frm and idb file.

First you will need a tool called dbsake which you can install with following command.

curl -s http://get.dbsake.net > dbsake

Once installed, run following command to check if it's installed properly or not.

chmod u+x dbsake

./dbsake --version

If it shows version and all, it's installed. Now first by using this tool we will get schema of table from frm file. Run following command.

./dbsake frmdump /PATH_TO_FRM/file.frm

It will give you create table schema like this.

CREATE TABLE `table_name` (

`column1` int(11) NOT NULL AUTO_INCREMENT,

`column2` varchar(255) DEFAULT NULL,

`column3` varchar(255) DEFAULT NULL,

PRIMARY KEY (`column1`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Copy it and run it in your sql editor. If you are migrating it from MySql 5 to 6 you may want to add ROW_FORMAT

CREATE TABLE `table_name` (

`column1` int(11) NOT NULL AUTO_INCREMENT,

`column2` varchar(255) DEFAULT NULL,

`column3` varchar(255) DEFAULT NULL,

PRIMARY KEY (`column1`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=compact;

Now it will create table. It's time to load data into it.

First we will delete the table space which is idb file created. Run following command.

ALTER TABLE table_name DISCARD TABLESPACE

Now copy the old idb file and replace the existing one in your MySql server.

Now attach table space again.

ALTER TABLE table_name IMPORT TABLESPACE

That's it and now if you browse the data in table, you can see all your old rows.

MySql Export / Import Large Database

2020-09-29T11:17:00.002+05:30

Hello,

In this blog I am going to explain a trick which I recently used for Export / Import large database.

Let me first explain the issue I faced. The database was over 250 MB. I was using MySql WorkBench to export in single self contained file. But it was breaking in between as some tables were really huge.

So here is what I did. First in MySql WorkBench, in stead of exporting it to self contained single file, export it to Dump project folder. Here it will create sql file for each table in the folder. So in my case it created approximately 180 sql files.

Now open terminal and go to that folder. First step here is we will concat all those files and make a single file.

Run the following command in terminal from folder where you have all the sql files.

cat *.sql > .single_file.sql

Above command will concat all the files into one file and create that file in same folder. Now we will use this file to import into our database.

/Applications/MAMP/Library/bin/mysql -u DB_USER -p DB < .single_file.sql

That's it. With this trick you can Import / Export large database. Please note that, for this you should have terminal access to your MySql server.

Connect to Web Socket From Your Android App using OkHttp

2020-09-07T11:41:00.002+05:30

Hello,

In this blog I am going to explain how you can connect with Web Socket from your Android App using OkHttp. This will be useful when you want to send some data real time from your Android app to Web app.

First add following line in your build.gradle file dependencies.

implementation 'com.squareup.okhttp3:okhttp:3.11.0'

Now let the gradle build and sync. Next we will make a class which extends WebSocketListener from OkHttp3 and a new class and name it MyWebSocketListener.java and add following code to it.

import okhttp3.Response;

import okhttp3.WebSocket;

import okhttp3.WebSocketListener;

import okio.ByteString;

public final class MyWebSocketListener extends WebSocketListener {

private static final int NORMAL_CLOSURE_STATUS = 5000;

public MyWebSocketListener() {

}

@Override

public void onOpen(WebSocket webSocket, Response response) {

webSocket.close(NORMAL_CLOSURE_STATUS, "Exit");

}

@Override

public void onMessage(WebSocket webSocket, String text) {

//String Meesage received

}

@Override

public void onMessage(WebSocket webSocket, ByteString bytes) {

//Meesage received in form of binary data

}

@Override

public void onClosing(WebSocket webSocket, int code, String reason) {

webSocket.close(NORMAL_CLOSURE_STATUS, null);

//Closing socket

}

@Override

public void onFailure(WebSocket webSocket, Throwable t, Response response) {

t.printStackTrace();

}

Now we have listener created. Let's create client and attach it to Web Socket and Listener.

val listener = MyWebSocketListener()

val webSocket = OkHttpClient.Builder().build().newWebSocket(request, listener)

That's it, now you can use this instance to send or receive message.

Create Secure Web Socket (WSS) with Ratchet PHP

2020-09-05T12:55:00.002+05:30

Hello,

Recently I was working on PHP project where we I was create secure web socket which is accessible with wss:// protocol. For this I struggled for couple of hours so here in blog I am going to explain how to do that so it can save your time.

Earlier version was not supporting WSS but later it introduced React socket server which allows SSL connection. So here are steps you need to follow.

First add all the required classed in your php file.

use Ratchet\Server\IoServer;

use Ratchet\Http\HttpServer;

use Ratchet\WebSocket\WsServer;

use MyApp\Socket;

In above example Socket is my class file which has all listeners. After this add auto load file.

require dirname( __FILE__ ) . '/vendor/autoload.php';

Next we will create our socket app.

$app = new \Ratchet\Http\HttpServer(

new \Ratchet\WebSocket\WsServer(

new \MyApp\Socket()

)

);

Now next step is to create React Secure server.

$loop = \React\EventLoop\Factory::create();

$webSock = new \React\Socket\Server('0.0.0.0:8080', $loop);

We are using 0.0.0.0 so this socket can be connected from anywhere.

Now lets create secure server by adding path to certificate and key

$webSock = new \React\Socket\SecureServer($webSock, $loop, [

'local_cert' => 'CRT_PATH',

'local_pk'=> 'KEY_PATH',

'allow_self_signed' => true,

'verify_peer' => false

]);

Now finally we will run our server.

$webSock = new \Ratchet\Server\IoServer($app, $webSock, $loop);

$webSock->run();

That's it. It will start your server to which you can connect with WSS protocol. To test you can use WebSocket ECHO test.

https://www.websocket.org/echo.html

Hope this helps you.

AWS PHP SDK - Create an Entry in Route53 - Laravel

2020-08-20T16:04:00.001+05:30

Hello,

In this blog I will explain you how to you can create an entry in Route53 hosted zone using AWS PHP SDK. Solution I mentioned here is particularly for Laravel app, but you can still use it in any of the PHP project.

This solution is pretty much useful when you want to make a dynamic entry in your hosted. For example a dynamic sub domain creation.

First of all your need install couple of packages using composer. Following are the packages.

"aws/aws-sdk-php": "^3.148",

"aws/aws-sdk-php-laravel": "~3.0",

You can either install it by adding it in composer.json file or you can install it using composer require.

Once the installation is done, follow the steps mentioned in below URL.

https://github.com/aws/aws-sdk-php-laravel

Once installation is done. You need to add following keys in your env file.

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

AWS_REGION (default = us-east-1)

Set the values in above keys from your AWS account. That's it your AWS SDK is ready. Now add following line your controller where you want to have logic for adding value in Route53

use Aws\Route53\Route53Client;

use Aws\Common\Credentials\Credentials;

Now next step is to create a client

$client = Route53Client::factory(array(

'credentials' => config('aws.credentials'),

'region' => config('aws.region'),

'version' => config('aws.version'),

));

After client is created we will use changeResourceRecordSets to create an entry.

$result = $client->changeResourceRecordSets(array(

// HostedZoneId is required

'HostedZoneId' => 'YOUR_HOSTED_ZONE_ID',

// ChangeBatch is required

'ChangeBatch' => array(

'Comment' => 'string',

// Changes is required

'Changes' => array(

array(

// Action is required

'Action' => 'CREATE',

// ResourceRecordSet is required

'ResourceRecordSet' => array(

// Name is required

'Name' => 'YOUR_VALUE',

// Type is required

'Type' => 'CNAME', //A, CANME

'TTL' => 600,

'ResourceRecords' => array(

array(

// Value is required

'Value' => 'YOUR_VALUE', //IP address or load balancer.

));

That's it and it will create an entry in Route53 hosted zone.

Android Build Error - The Crashlytics build ID is missing

2020-08-04T15:45:00.005+05:30

Hello,

Recently I faced a very strange issue in one of my android app. We were using Fabric IO for crash analytics earlier in the app and since now it's already merged with Firebase Analytics. we need to update the app.

I removed Firebase maven repository and removed all plugins from build gradle file and removed Fabric code from everywhere. And build was successful. After that I followed all the instructions on getting started page of Firebase and crash analytics and build was successful. And then problem started. Once we run the app, it crashes and shows following error in Logcat.

"The Crashlytics build ID is missing. This occurs when Crashlytics tooling is absent from your app's build configuration. Please review Crashlytics onboarding instructions and ensure you have a valid Crashlytics account"

Now it seems there is an issue with Firebase crash analytics account so I regenerated the google service json file add updated in project. But it didn't help.

Searched on Google, answers suggest to put back Fabric plugin again in gradle file. Which does not make any sense to me. Since Fabric is out dated, why do we need to add it again.

Finally after 2 days I was able to solve the problem. Here is the solution

Remove google service json file. Use Firebase assistant and connect with your account again. Select the project and it again generate google service json file. Keep link to fabric io repository in top level gradle file.

maven { url 'https://maven.fabric.io/public' }

I don't know how it worked but it works and I really don't know the reason of why it didn't work and why it started working after above solution. But it does solve the problem.

If anyone who is reading this blog, knows the reason please put in comment.

Angular 7 App ng build Works But ng build --prod Gives Error

2020-03-22T11:36:00.000+05:30

Hello,

Recently I was trying to generate production build of one of our older Angular app which was built on Angular 6 and later upgraded to Angular 7. When we run ng build it works fine but when we run ng build --prod it gives so many errors.

Here in this blog I am going to explain what are those errors and how I solved it.

Problem 1 : Strict Parameter checking for function.

There were few events and handler defined in the app where there were parameter mismatch. For example in html file we have following event.

<componentName (event)="eventHandler($event)"></componentName >

And in component TS file event handler was defined like this

eventHandler() {
}

As you can see from html file it was passing $event param but on handler param was not mentioned. So make sure that your function signature and function declaration matches.

Problem 2 : Duplicate Declaration of Components

By mistake we have duplicate declaration of components in both App module and and other child modules of the app. Make sure you either declare all your components to app module or if you are defining it into sub module then make sure you remove it from app module.

Problem 3 : Cannot read property 'moduleType' of undefined

When you face above issue please check your app module by mistake you may have defined following line twice in your app module.

platformBrowserDynamic().bootstrapModule(AppModule);

It's like you are trying to bootstrap your Angular app twice and it gives above error. So try to avoid it.

Problem 4 : Enable IVY

In Angular 7 by default IVY is disabled hence even if you generate production build you app size it bit large. To enable IVY add following line to your tsconfig.json file.

"angularCompilerOptions": {
"enableIvy": true
}

Hope this blog post helps you.

Laravel App Connect and Emit to Socket.io

2020-03-22T11:12:00.000+05:30

Hello,

Recently in one of the project I was working on we have socket.io server created using NodeJS and there were couple of Angular applications and android applications connects with it. However there was a requirement to connect with socket from Laravel app and emit the event.

Actually there was a form in Laravel app on submit of Form we have to notify the socket and socket further notify all the clients connected to it. In this blog I am going to mention how we solved this.

We all know Laravel by default comes with integration of Pusher we could have used it. But since Socket.io is free so we consider to use Socket.io.

Let's go step by step and understand how to connect with Socket.io

Step 1

Create new folder socket in your app directory of Laravel app.

Step 2

Copy following class in to app/Socket folder and rename it to SocketIO.php

https://github.com/psinetron/PHP_SocketIO_Client

There are many PHP socket IO client is available but we used above one. You can choose either one of your choice.

Step 3

Connect and emit to socket. In your controller file where you want to emit the event. Add following code.

$socketIO = new SocketIO();
$returnValue = $socketIO->send('YOUR_URL', YOUR_PORT, 'YOUR_EVENT',json_encode(YOUR_PHP_ARRAY))

For this add following line to top of your controller.

use App\socket\SocketIO;

That's it and now you can connect with socket and emit event to it.

Socket.io Not Working on Android 9 (API 28)

2020-03-22T10:56:00.000+05:30

Hi,

Recently in one of the project we faced a situation where we have used socket.io on backend and Android app connects to it. Socket.io was using http protocol for testing. It worked fine in older android versions but faced an issue in Android 9 and above where socket was connected but emit was not getting on server.

After couple of hours struggle finally found the problem. The problem was socket io was using http protocol and it's clearText HTTP request and it is banned in recent android version considering security reasons.

In earlier android version by default clearText is set as true but in later version default value is false. That means android components like HTTP and FTP and classes DownloadManager, and MediaPlayer can not use HTTP.

To over come this issue and allow socket to connect with HTTP protocol you have to allow clearText traffic on app level and for that you have to add following attribute in Application tag of your AndroidManifest.xml

<application
android:usesCleartextTraffic="true">
.....
</application>

Once you add this to your manifest file, your socket emit will start working and you can still use HTTP with your socket.

Please note that since it was testing app we used HTTP based socket. However in production you should always use HTTPs protocol.

Hope this helps you.

NodeJs MySQL Observer

2019-08-21T09:49:00.000+05:30

Hello,

In this blog we are going to learn how to use NodeJs to observe changes in MySql databases. This is useful when you want to track MySQL changes and based on that want to send some events to frontends or want to do any other actions.

For this first of all you have to enable binary logging in you database. Binary logging is very much useful for real time MySQL replication. In Amazon RDS, it's by default available and you can switch on it from configurations. For your local database if you are using MAMP, you can do following trick.

Create a file with name my.cnf and add following content to it.

[mysqld]
server-id = 1
default-storage-engine = InnoDB
log-bin=bin.log
log-bin-index=bin-log.index
max_binlog_size=100M
expire_logs_days = 10
binlog_format=row
socket=mysql.sock

Add this file to conf folder of your MAMP directory and restart MySQL server. This will enable binary logging in your database.

Now to observe this changes we use npm package called zongji . Install it with NPM.

Add following code to your NodeJs script.

var ZongJi = require('zongji');
var _underScore = require('underscore');

var zongji = new ZongJi({

user : 'YOUR_USERNAME',

password : "YOUR_PASSWORD",

database: 'YOUR_DATABASE',

socketPath : '/Applications/MAMP/tmp/mysql/mysql.sock'

});

Now add event on binlog.

zongji.on('binlog', function(evt) {

});

This event is triggered whenever there is a change in any of your database tables.

Inside this event you can have logic of checking new rows, updates rows, deleted rows.

zongji.on('binlog', function(evt) {

if (evt.getEventName() === 'writerows' || evt.getEventName() === 'updaterows' || evt.getEventName() === 'deleterows') {

var database = evt.tableMap[evt.tableId].parentSchema;

var table = evt.tableMap[evt.tableId].tableName;

var columns = evt.tableMap[evt.tableId].columns;

_underScore.each(evt.rows, function(row) {
});
}
});

At last start the process and pass the events you want to watch.
zongji.start({
includeEvents: ['tablemap', 'writerows', 'updaterows', 'deleterows']
});

ReactJs Material UI Table Infinite Scroll

2019-08-20T09:08:00.000+05:30

Hello,

Recently in one of my ReactJs project, I faced a challenge in implementing infinite scroll in Material UI table. In this blog I am going to mention trick I have used.

First of all I was really surprised to see that Material UI table does not have infinite scroll function out of the box. It's very much needed. Sometimes something can not be achieved with frameworks, can be achieved via basics of JavaScript. In this I have done something similar.

I tried adding on scroll events on Table, Table Body but it didn't work. I also tried adding refs to body and then bind the Scroll event but that also did not work. After struggling for couple of hours, I finally decided to it with Pure JavaScript.

Step 1 : Wrap material UI table inside the container with fixed height and set overflow = scroll to container.

import styled from "styled-components";

export const Table = props => (
<TableWrapper id={props.id}>
<MuiTable {...props}/>
</TableWrapper>

);

const TableWrapper = styled.div`
max-height: 500px;
overflow: scroll;

::-webkit-scrollbar {
width: 3px;
height: 3px;
}
`;

As you can see I created a wrapper of table and set max height to it. You can make it dynamic as well depending on window height.

Step 2: Import Table to your component

import {
Table

} from './components/Table';

return (
<>
<Table id={"tableID"}/>
</>
);

Step 3: Bind scroll event to wrapper

let me = this;
document.getElementById('tableID').onscroll = function(event){
if(this.scrollTop == (this.scrollHeight - this.clientHeight)){
//User reached bottom of table after scroll
//Logic to call web services to get next set of data
}
};

Hope this helps you.

Accessing Data From Redis Using NodeJs

2019-08-19T08:15:00.000+05:30

Hello,

When you are working with business applications, it's sometimes need to cache the data. At this point Redis can be very useful, it can be used as database or cache database. You can store any kind of data like strings, JSON objects etc. in Redis.

Problem we face while working with NodeJs and Redis, get data operation from Redis is Asynchronous operations so it gives you callback and your code execution will continue. This may create a problem when you want to handle it in Synchronous way. For example you may have loops inside that you are trying to access data from Redis.

In this blog I am going to explain how you can have Synchronous operations. In nutshell we got to promisify the redis module.

There is a library called bluebird, that can be used for this. Lets go step by step.

Step 1

Install bluebird and redis in your NodeJs app.

npm install bluebird
npm install redis

Step 2

Import it in NodeJs app.

var redis = require('redis');
var bluebird = require("bluebird");

Step 3

Promisify Redis.

bluebird.promisifyAll(redis.RedisClient.prototype);

bluebird.promisifyAll(redis.Multi.prototype);

Step 4

Connect to Redis client.

var client = redis.createClient();

client.on('connect', function() {

console.log('Redis client connected');

});

Step 5

Use Async version of get function to get data.

client.getAsync("MY_KEY").then(function(res) {

//Access Data

});

This is how you can have Sync operations with Redis.

ReactJs / Angular - Override Material UI Theme

2019-08-15T15:43:00.001+05:30

Hello,

While working with React or Angular Applications we normally uses Material UI themes and components and it's great choice because it has nice set of themes and colors and ready to use components. That makes our life easy.

One challenge we face is, in case if we want to change or override the theme. For example Material ui has dark blue theme. What if you want to change colors or dark blue theme with your own color. One option is you can declare the class with same name and give your own property. However the better solution is to use theme override. In this blog I am going to mention how to this.

Please note the method I am going to mention here is specific to React application.

First of all create theme.js file in your app and add following code to it.

import React from "react";
import { MuiThemeProvider, createMuiTheme } from "@material-ui/core/styles";
import CssBaseline from "@material-ui/core/CssBaseline";
import blueGrey from "@material-ui/core/colors/blueGrey";
import lightBlue from "@material-ui/core/colors/lightBlue";
import "typeface-roboto";

import { ThemeProvider as ScThemeProvider } from "styled-components";

export default props => (
<ScThemeProvider theme={muiTheme}>
<MuiThemeProvider theme={muiTheme}>
<CssBaseline>{props.children}</CssBaseline>
</MuiThemeProvider>
</ScThemeProvider>
);

const theme = {
overrides: {

palette: {

primary: {

main: '#MAIN_COLOR'

secondary: {

main: '#SECONDARY_COLOR'

background: {

default: '#BACKGROUND_COLOR'

}

type: "dark"

};

const muiTheme = createMuiTheme(theme);

Now here in overrides you can add code to override. For example if you want to change panel header. If you inspect html element you will find it has a class MuiPaper-root

Here is how you can override base CSS of it.

overrides: {

MuiPaper: {

root: {

background: '#YOUR_COLOR'

rounded: {

background: '#YOUR_COLOR'

}

Same way for any other components you can do it. Inspect the HTML structure, find out Class Name and override it.