tag:blogger.com,1999:blog-58933839819895241592026-10-22T15:15:50.377+05:30I write to discover what I know. (Flannery O'Connor)Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.comBlogger180125tag:blogger.com,1999:blog-5893383981989524159.post-23802189150722946282025-03-12T23:15:00.004+05:302025-03-12T23:15:55.043+05:30

Caching is a technique used to store frequently accessed data in a temporary storage layer to improve system performance and reduce latency. There are various caching strategies, each with different use cases, benefits, and trade-offs.Different caching strategies dictate how data is read from and written to the cache and the underlying data store (e.g., a database). Below, are the key caching

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-57639503134512613632025-03-11T18:07:00.001+05:302025-03-11T18:07:15.679+05:30

In distributed systems, consistency defines the guarantees about the visibility and ordering of updates across different nodes. The choice of consistency level impacts performance, availability, and reliability.  Understanding consistency levels helps designing the system effectively within given constraints. 1. Strong Consistency Definition: Strong consistency guarantees

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-31233661931493292082021-03-16T09:31:00.004+05:302021-03-16T09:31:58.506+05:30

Go standard library comes with "net/http" package which has excellent support for HTTP Client and Server.  In order to post JSON body during post request, we need to convert the data to byte array format and send it along with the request.You can convert the JSON to a byte array using "encoding/json" package. Then use the NewBuffer method to pass this byte array to the post method.

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-63893046267365773542021-03-15T07:22:00.000+05:302021-03-15T07:22:12.187+05:30

Go standard library comes with "net/http" package which has excellent support for  HTTP Client and Server.  In this example we will be using "http.Get" method to execute Http Get method. We will convert the response to String and print it. package main import ( "fmt" "io/ioutil" "net/http" ) func main(){ //Use get method to hit the rest API resp, err := http.Get("https://

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-5547317022123231402021-03-14T13:25:00.006+05:302021-03-14T13:25:59.341+05:30

Usually, environment variables are used to provide the configuration information for the program. You can set the environment outside of the program and you can assess these during execution. Go provides the simplest mechanism to access the environment variables. You can use 'OS' package to get or set the environment variables. package main import ( "fmt" "os" "strings" ) func

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-47754651365210420132021-03-02T11:06:00.001+05:302021-03-02T11:06:15.471+05:30

Writing data to file is the most common use case while writing a program.  Here we are going to see multiple ways of writing data to file.   Write using ioutil.WriteFile You can directly write data to file using ioutil.WriteFile" method. It will take care of creating file if the file does not exists. package main import ( "fmt" "io/ioutil" ) func main() { writeStringToFile("

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-51700499571777980322021-02-28T20:55:00.002+05:302021-03-01T07:37:15.734+05:30

In the last post, we have seen how we can read data from the file line by line.  Sometimes if the file size small then you can read the whole file in one go.  In this example, we will be reading the whole file in one go. We will be using 'ReadFile' from 'ioutil' lib to read the file content.  package main import ( "fmt" "io/ioutil" ) func main() { readFile("

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-60065246647117096982021-02-28T20:20:00.004+05:302021-03-02T09:02:41.311+05:30

Reading data from files is a common use case. Most of the time you need to read data line by line.There are multiple ways to read data from the file.  The simplest way is to open a file and use a scanner to read data line by line. The sample code below does the same.  package main import ( "bufio" "fmt" "os" ) func main() { readLineByLine("file_path") } func readLineByLine(

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-9530955787466859212021-02-27T19:15:00.003+05:302021-03-03T18:25:43.997+05:30

Sometimes you want to know the hostname and IP address of the machine you are running the program on.  We can get the hostname using the OS package.For IP address we can use the Net package. You can read all IP addresses using InterfaceAddrs, you need to filter out the loopback address to get real IP.  package main import ( "fmt" "net" "os" ) func main() { //Reading the hostname

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-11328620684683302032021-02-25T12:25:00.002+05:302021-03-01T09:18:08.330+05:30

It is easy to check if the file with the given path exists or not in Go lang. You can use 'Stat' method from 'os'.  Stat returns a FileInfo describing the named file. So if there is no error then you can safely assume the file exists.     import ( "fmt" "os" ) func main() { if _, err := os.Stat("file.path"); err == nil { fmt.Println("File with given path exists.")

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-45640275841433526802020-10-30T09:33:00.000+05:302020-10-30T11:22:52.105+05:30

This is an unusual problem you might face while consuming data from Kafka broker over SSL.  java.lang.IllegalStateException: Buffer overflow when available data size (16384) >= application buffer size (16384) at org.apache.kafka.common.network.SslTransportLayer.read(SslTransportLayer.java:592) at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:95) at

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-35273038389271441962020-10-30T09:24:00.004+05:302020-10-30T09:24:19.479+05:30

If Python requests.get fails with a timeout and the same URL works in the browser then you will need to simulate the browser request as is while invoking requests.get.  Usually, this will get resolved if you put the "user-agent" header, which tells the server which browser is requesting.  session = requests.Session() headers = { 'User-Agent': 'Mozilla/5.0 (Macintosh;

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-38869383148777450472020-10-28T15:14:00.007+05:302020-10-28T17:55:27.189+05:30

Here is the list of commonly used commands on Unix/Linux systems.1. Split File based on number of lines   If you want to split text file "abc.txt" containing 50 lines by each line use the below command csplit abc.txt 1 {49} 2. Find out Top 10 Memory intensive processes   If you want to figure out which are the top 10 memory consuming processes use the command below ps -eo

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-17201582835533312232020-10-28T11:05:00.003+05:302020-10-28T13:41:49.335+05:30

Apache Kafka comes with the kafka-*-perf-test tool, which can be used to benchmark the performance of the Kafka cluster. This can be used as the first level of benchmarking tool before you run your production load tests. Kafka Producer TestYou can use kafka-producer-perf-test to measure the Kafka write performance.  The below command demonstrates running the Kafka producer test for 100K

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-65706276001169375132019-05-08T22:18:00.002+05:302019-05-08T22:20:26.790+05:30

Springboot application may fail to start with error "Zip64 archives are not supported" with giving you stack trace as below. Caused by: java.lang.IllegalStateException: Zip64 archives are not supported at org.springframework.boot.loader.jar.CentralDirectoryEndRecord.getNumberOfRecords(CentralDirectoryEndRecord.java:124) at org.springframework.boot.loader.jar.JarFileEntries.visitStart(

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-11253972843225114702018-09-09T22:54:00.000+05:302018-09-09T22:54:13.984+05:30

Camera is the mostly used app on smart phones, and its frequently required app. Having quick shortcut to launch the camera is always helpful.  Samsung Galaxy Note 9 offers the always on display mode, where it allows to enable on screen home button.  Always on display also allows to set camera as a app to launch when double tap on home button.  To enable this go to Setting ->

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-86580203619155927292018-09-03T09:47:00.000+05:302018-09-03T09:47:02.027+05:30

If you like to save your battery more, you can turn off the always on display on your Samsung Galaxy Note 9.  To do so, go to Settings -> Display. Now on display settings scroll down to the end, you will find the always  on  Display  setting. Other option is go to Settings. Now using search bar search for "Always on". Here enable/disable always on display by using the

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-8823715085695935092018-09-02T14:51:00.000+05:302018-09-02T14:51:21.422+05:30

Kafka Connect is a tool for scalably and reliably streaming data between Apache Kafka and other systems. It can also be used in secured Kafka environment. In Kerberixed Kafka installation also you can use the Kafka Connect utilities.  You just have to provide the Java security config and Kerberos config as parameters to connect utils.  Below command shows how can you specify the

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-59862903827622227592018-01-04T20:07:00.000+05:302018-01-04T20:07:05.013+05:30

During execution of ansible playbook or ansible script on new remote host (which you have not connected via ssh so far) the script might fail with below error UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ----} To avoid this you can set the environment variable (ANSIBLE_HOST_KEY_CHECKING) to disable host check before ssh during ansible script execution.

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-9740131046140597582018-01-01T11:05:00.000+05:302018-01-01T11:05:40.321+05:30

HMaster fails to start with below error util.FSHDFSUtils: attempt=0 on file=hdfs://krbhdfs/apps/hbase/data/MasterProcWALs/state-00000000000000000001.log after 1ms java.io.IOException: Filesystem closed. There could be multiple causes of the error.  If you are willing to loose the data, remove manually the logs under "apps/hbase/MasterProcWALs" and  "apps/hbase/WALs" from HDFS. After

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-68483857974071046412017-12-30T08:43:00.000+05:302017-12-30T08:43:54.396+05:30

Region server might crash with below error INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=ip-100-122-218-159.us-east-1.ec2.aws.symcpe.net:2181,ip-******.us-east-1.ec2.aws.net:2181,ip-*****.us-east-1.ec2.aws.net:2181 sessionTimeout=180000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@253c1256 ERROR [main] regionserver.HRegionServerCommandLine: Region

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-66416009205973761772017-12-29T20:00:00.004+05:302017-12-29T20:00:50.660+05:30

HMaster throws error as below during startup FATAL [ip-:16000.activeMasterManager] master.HMaster: The coprocessor org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw java.lang.RuntimeException: java.io.FileNotFoundException: /etc/hbase/2.5.0.55-1/0/xasecure-audit.xml (No such file or directory) This is happening because the ranger co-processor is defined in xml config

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-37513577104350468082017-12-27T11:42:00.001+05:302017-12-27T11:42:07.667+05:30

Sometimes starting the namenode might throw below error. ERROR config.RangerConfiguration (RangerConfiguration.java:addResourceIfReadable(110)) - addResourceIfReadable(ranger-hdfs-security.xml): couldn't find resource file location INFO provider.AuditProviderFactory (AuditProviderFactory.java:(77)) - AuditProviderFactory: creating.. FATAL conf.Configuration (Configuration.java:loadResource(

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com2tag:blogger.com,1999:blog-5893383981989524159.post-58739387615634501882017-10-17T19:04:00.002+05:302017-10-17T19:04:34.985+05:30

Python is widely used programing language and pip is the tool need to install different python packages. Before you go ahead check if you have pip installed by using command below. $ pip --version pip 9.0.1 from /Library/Python/2.7/site-packages (python 2.7) You can install pip in multiple ways on CentOS. I will discuss here two ways to install pip.  Install PIP using yup Update the

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-25711444368081050492017-10-04T20:52:00.001+05:302017-10-04T20:52:24.869+05:30

Python is most commonly used development and scripting tool. You can setup Python on CentOS by multiple ways. Setup Python using Yum This is simplest method to install development tools on CentOS. Run below commands to setup the python using yum. [vagrant@localhost ~]$ sudo yum -y update [vagrant@localhost ~]$ sudo yum groupinstall -y development Or [vagrant@localhost ~]$ sudo yum

Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0