Learn Data Science using Python

How to CORRECTLY convert Darknet/Yolov4 weights to TFLite format?

noreply@blogger.com (Learn Data Science with PrateekG) — Fri, 7 Feb 2025 16:41:00 +0530

Greetings, computer vision practitioners👮! I hope you're all doing well in your projects. Recently, I received a new requirement from a client regarding an object detection solution that had been successfully deployed as a REST API. The solution was seamlessly integrated into both web and mobile applications. However, the client encountered network reliability issues, particularly in rural areas where poor connectivity occasionally caused the solution to fail on mobile devices💔.

As we know, network disruptions are a common challenge, but they become critical when they impact business operations. My client sought a robust solution to address this issue, ensuring uninterrupted functionality regardless of network conditions. In this blog post, we will explore how I tackled this challenge and implemented an effective solution💓.

The Solution: Enabling Offline Object Detection for Mobile Devices

Given that network connectivity issues are beyond our control, the most effective solution I proposed was to implement an offline model directly within the mobile application💥. Instead of relying on a cloud-based or on-premise Rest API, the object detection model would run natively on the device, allowing it to function seamlessly without an internet connection💣.

Sounds interesting💁? Absolutely! Since my client was using a Darknet/YOLOv4-based model, the next step was to convert it into a format optimized for offline deployment on mobile devices. For both Android and iOS, the most suitable format is TensorFlow Lite (TFLite), previously known as LiteRT. In the next section, we will dive into the process of converting the YOLOv4 model to TFLite and integrating it into the mobile application💪.

Converting Darknet/YOLOv4 Weights to TFLite Format:-

In this section, we will explore the step-by-step process of successfully converting Darknet/YOLOv4 weights to TFLite format😎. For demonstration purposes, I will be using a Kaggle Notebook, but you are free to use any environment of your choice for the conversion.

It is important to note that Darknet/YOLOv4 weights cannot be directly converted to TFLite (formerly LiteRT). The conversion process involves an intermediate step:

1. First, we convert the Darknet model to the ONNX format, which is an open standard for machine learning model interoperability.

2. Then, we convert the ONNX model to TFLite, making it optimized for mobile deployment.

To begin the conversion, we will clone the required GitHub repository (and if you find it useful, don’t forget to give it a star ⭐ on GitHub!). Let’s get started!-

Next, we will install the following required package for onnx conversion-

After installing the above package, we will run the following conversion command from the yolov4_pytorch directory-

Understanding the ONNX Conversion Command:-

In the command above, I have specified the paths for the Darknet/YOLOv4 configuration file, object names file, and weight file, along with an example image. This image is used to visualize the object detection results by drawing bounding boxes after converting the model to the ONNX format:

Customizing and Verifying the ONNX Conversion:-

You can replace the specified configuration, object names, and weight files with your own as needed. Upon executing the command, the conversion process begins, and the output image is saved with predicted bounding boxes using the ONNX model💫. This allows us to verify whether there are any discrepancies between the original Darknet/YOLOv4 model and the converted ONNX model.

At the end of the logs, you will see messages similar to the following:

Proceeding to TFLite Conversion:-

As seen in the logs above, the converted ONNX model successfully detects and predicts objects such as the bicycle, truck, and dog, confirming that the conversion process was accurate💘.

Now, we can proceed with converting the ONNX model to TFLite format. For this step, we will utilize a highly efficient GitHub repository specifically designed for ONNX-to-TFLite conversion. I highly recommend giving this repository a ⭐ on GitHub, as it provides a seamless and reliable way to convert almost any ONNX model into TFLite format. Let’s dive into the conversion process!

Next, we need to install also few supportive libraries one by one-

Next, we will run the following command to start the tflite conversion-

Executing the ONNX to TFLite Conversion:-

In the command above, we simply need to specify the path to the ONNX model. Once executed, the script will handle all the complex computations required for the conversion. Upon completion, it will generate Float16 and Float32 TFLite models, which will be stored inside the saved_model folder.

Model Size and Verifying Predictions:-

If you check the size of the Float16 TFLite model, it is approximately 11 MB, significantly smaller than the original YOLOv4-Tiny model, which is 23 MB💥. This reduction in size is crucial, as it ensures efficient deployment within a mobile application.

Next, we need to verify whether the converted TFLite model produces the same predictions as the original model. To do this, we can either write a Python inference script or collaborate with an Android developer to implement it directly within the mobile application👷.

Below, I am sharing my Python script for running inference with the converted model.

We can use the OpenCV library to load the tflite model directly as below-

Next, we will apply some pre-processing-

Post-Processing and Displaying TFLite Model Output:-

After applying essential post-processing techniques, such as Non-Maximum Suppression (NMS) to eliminate redundant bounding boxes, we can visualize the final output of the TFLite model on the given image, as shown below.

The output image will be like below where you can see the predicted objects-

We can now confirm that the converted TFLite model produces predictions identical to those of the original model👊. Additionally, since this TFLite model is in NHWC format, which is the required format for Android development, it can be seamlessly integrated into mobile applications. With the model ready, we can hand it over to the mobile developer, who can effortlessly import it into Android Studio for further app development and deployment👌.

That’s all for today, everyone! In this blog post, we successfully walked through the correct process of converting Darknet/YOLOv4 weights to TFLite format, enabling seamless integration into mobile applications as an offline model. This skill not only enhances your expertise but also strengthens your capabilities as a computer vision practitioner👲.

Now, it's your turn! Use my Kaggle Notebook to convert your custom-trained YOLOv4 model to TFLite format and deploy it within a mobile application to demonstrate your computer vision expertise😇.

In the next post, we will explore the YOLOv7 model conversion to TFLite. Until then, keep pushing forward, chase your dreams, and make every second count! 🚀 See you in the next one!

Generative AI: Retrieval Augmented Generation(RAG)

noreply@blogger.com (Learn Data Science with PrateekG) — Fri, 19 Jan 2024 17:08:00 +0530

Another blog post starts with you beautiful people👦. I hope you have explored my last blog post about 2x faster fine-tuning of Mistral 7b model on a custom dataset👈. In this blog post, we are going to learn an essential technique in Generative AI: Retrieval Augmented Generation (RAG).

What is RAG?

Retrieval Augmented Generation (RAG) is an innovative approach that melds generative models, like transformers, with a retrieval mechanism. By tapping into existing knowledge, RAG retrieves pertinent information from expansive external datasets or knowledge bases to enhance the generation process, thereby elevating the model's content relevance and factual accuracy💪. This versatility renders RAG particularly beneficial for tasks demanding the assimilation of external knowledge, such as question answering or content creation.

Upon receiving input, RAG actively searches for relevant documents from specified sources (e.g., Wikipedia, company knowledge base, etc.). It then seamlessly amalgamates this retrieved data with the input, offering a comprehensive output complete with references. This unique structure enables RAG to effortlessly integrate new and evolving information without the need to retrain the entire model from scratch💥.

RAG vs Fine-Tuning?

RAG augments the prompt with the external data, while fine-tuning incorporates the additional knowledge into the model itself. RAG requires less labeled data and resources than fine-tuning processes, making it less costly. Much of RAG expenses often go into setting up embedding and retrieval systems. In contrast, fine-tuning requires more labeled data, significant computational resources, and state-of-the-art hardware like high-performance GPUs or TPUs. As a result, the overall cost of fine-tuning is relatively higher than RAG💸.

RAG Architecture?

A standard RAG application comprises two primary elements:

A. Indexing: a data ingestion pipeline that sources and indexes data, typically conducted offline. The indexing sequence is as follows-

1. Load: First we need to load our data. This is done with DocumentLoaders.

2. Split: Text splitters break large Documents into smaller chunks. This is useful both for indexing data and for passing it into a model, since large chunks are harder to search over and won’t fit in a model’s finite context window.

3. Store: We need somewhere to store and index our splits so that they can later be searched. This is often done using a VectorStore and Embeddings model.

B. Retrieval and generation: the operational RAG chain, is responsible for receiving user queries during runtime, retrieving pertinent data from the index, and passing it to the model. The sequence is as follows-

1. Retrieve: Given a user input, relevant splits are retrieved from storage using a Retriever.

2. Generate: A ChatModel / LLM produces an answer using a prompt that includes the question and the retrieved data.

Let's build something with RAG!

Now we are ready to use RAG with a large language model-Mistral 7B that we also used in the last blog. As of date 18 January'24, the hot topic in India is the new temple of lord Ram in Ayodhya city. If we ask any question related to this to an LLM, they cannot reply with accurate answers since they are not trained with current news. To prove this point, we can load the pre-trained LLM model and ask a related question to it like below-

Load the 4-bit Mistral 7b model as we did in my last post-

And I asked a current question to the loaded pre-trained model as below-

And as I expected, the model is unable to give the answer-

I hope you have understood the problem statement.👀 Now we will use RAG to get the required information from an external source to teach our LLM about the current topic. As you have read above in the architecture section of the RAG, we will need to store that source somewhere. For this purpose, we are going to use FAISS but you can use any other vector database like CHROMA. Let's install the FAISS library using pip command in our colab notebook-

Also, install other required libraries like langchain, langchain-community, unsloth, transformers and sentence-transformers if you still need to install them.

The next step is to download the data from our source. In my case, it is 'https://srjbtkshetra.org/'. You can replace this with any other site of your requirement. To load all text from this webpage into a document format, we will use WebBaseLoader from the LangChain tool. Please note that LangChain supports various types of data loaders and you can refer this link for all the details-

The next step is to divide the loaded texts into smaller chunks that can fit into our model's context window. For example, GPT-3.5-turbo has a context window of 4,097 tokens and Mistral 7B has 8,000. So if we try to pass that context window then the behavior of LLMs will be unpredictable and suffer from severe performance degradation👽. For this purpose, we will use text splitter as below-

The next step is to encode these chunks of text into the embeddings and then perform the indexing of those embeddings to our vector database as below-

Here, for embeddings, we use the 'sentence-transformers/all-mpnet-base-v2' model from the Hugging Face hub but you can explore other sentence-transformers models as well from this link. Now we will construct a receiver to fetch the documents from the vector db as below-

Next, we will create and load our pipeline for the inference as you already read in last blog-

Next, we will create a prompt template for giving the instruction to the model, format the output as our need, and create a chain of RAG all of this like below-

Now our RAG is ready to ask any relative questions to our added source. For example, I asked about the dimensions of the newly built Lord Ram Temple and the model is now able to provide me accurate answers as below-

Another one-

How cool is this, right guys👏. In this post, we learned about the creation of PromptTemplate & Chain, usage of RunnablePassthrough, invocation of Retriever, integration of context Integration, and the LLM invocation. The usage of RAG with any other possible use case is endless💫. So don't wait. Make a copy of this colab notebook in your colab notebook and start playing with your own data and any source you want to update with the LLM. In the next post, we will further learn something useful use case of Gen AI, till then 👉 Go chase your dreams, have an awesome day, make every second count, and see you later in my next post.

Generative AI: Faster fine-tuning of a Mistral Model with less memory

noreply@blogger.com (Learn Data Science with PrateekG) — Mon, 15 Jan 2024 20:40:00 +0530

Another blog post starts with you beautiful people💥. I hope you have started your Generative AI learning from my last post and if you have not, then I recommend you read that one before proceeding to this blog. In this post, we will delve into the transformative power of generative AI, specifically exploring the cutting-edge Mistral 7B model fine-tuning🚀. This revolutionary technology has not only redefined the boundaries of artificial intelligence but also sparked a paradigm shift in how we approach data generation and creativity. Join me on a journey through the fascinating intersection of machine learning and creativity, where Mistral 7B stands as a beacon of innovation, pushing the boundaries of what's possible in the realm of generative AI👈.

Fine-tuning a large language model(LLM) refers to the process of training the model on a specific, smaller dataset to adapt it to a particular task or domain. Large language models, like ChatGPT, are pre-trained on massive datasets to learn general language patterns and context. However, fine-tuning allows users to tailor the model for more specific applications, such as sentiment analysis, question-answering, or domain-specific content generation👌.

But Fine-tuning an LLM has its own challenges👿 if you want to do it in your local system due to the following reasons-

1. Memory Requirements: Large language models often have a considerable number of parameters, resulting in high memory requirements. Fine-tuning involves loading both the pre-trained model and the dataset into memory, and low RAM can limit the size of the model and the dataset that can be efficiently processed.

2. Processing Power: Fine-tuning is a computationally intensive task. Low RAM computers may lack the necessary processing power to handle the optimization process efficiently. This can lead to slow training times and potential issues with model convergence.

3. Batch Size Constraints: During fine-tuning, models are typically trained in batches to improve computational efficiency. Low RAM can restrict the batch size, impacting the model's ability to generalize well and slowing down the convergence of the training process.

4. Data Loading Constraints: Loading and processing large datasets during fine-tuning can strain the available memory. Insufficient RAM may lead to data loading errors or inefficient use of available resources.

To address these challenges, we are going to use a cutting-edge fine-tuning approach:QLORA but with 2.2X faster & 62% less memory💁. Can you believe that? Yes, we are so let's start this in our colab notebook using T4 GPU only. To achieve this faster and less RAM-consuming fine-tuning approach we are going to use a library called unsloth👮. This fantastic library can be used to fine-tune other LLMs like Llama 2 7b as well so don't forget to give a star in github to the library👍. For our learning, we will stick to my favorite Mistral 7b. This library has a built-in 4-bit pre-quantized Mistral 7b model which means neither we need to download this model from elsewhere nor do we have to convert the original model to 4-bit👌. Another benefit of using this library is that it supports Huggingface's TRL, Trainer, Seq2SeqTrainer, or even Pytorch code!

First, we will install the unsloth library based on our GPU type since it is sensitive to GPU type, and then the Transformers library from the Hugging Face which provides APIs and tools to easily download and train state-of-the-art pre-trained models💪 -

Next, we download the pre-trained 4-bit Mistral 7B model with its tokenizer like the below-

See, how easily and with very few lines of code we can download the model💫. You will see the following output in the console while running the above code-

Before running the next line of code, we need to know that large-scale unsupervised language models (LMs) learn broad world knowledge and some reasoning skills, achieving precise control of their behavior is difficult😑 due to the completely unsupervised nature of their training. Existing methods for gaining such steerability collect human labels of the relative quality of model generations and fine-tune the unsupervised LM to align with these preferences, often with reinforcement learning from human feedback (RLHF).

However, RLHF is a complex and often unstable procedure😓, first fitting a reward model that reflects the human preferences, and then fine-tuning the large unsupervised LM using reinforcement learning to maximize this estimated reward without drifting too far from the original model. To address this big issue, we are going to use another cutting-edge algorithm called DPO 💪which can fine-tune LMs to align with human preferences as well as or better than existing methods. Also, fine-tuning with DPO exceeds PPO-based RLHF in the ability to control the sentiment of generations and matches or improves response quality in summarization and single-turn dialogue while being substantially simpler to implement and train.

Don't worry using the DPO algorithm is also very easy due to unsloth library💓. We will add LoRA adapters to update 1 to 10% of all parameters for patching the model like below-

Here, r is the dimension of the low-rank matrices, lora_alpha is the scaling factor for the low-rank matrices and lora_dropout is the dropout probability of the LoRA layers.

Next, we need a dataset that we can fine-tune. For this purpose, we will use cleaned 52k samples of alpaca-cleaned dataset from the original Alpaca Dataset released by Stanford. The Alpaca dataset is designed for instruction training pre-trained language models💨. But you can replace this with your own dataset. It looks like in the below format-

Since from our last blog post we know that the Mistral 7B model expects a specific format, we will convert this dataset to the specific format after loading it like below-

The downloading will look like below-

Next, we can start training the model on this formatted dataset using the Huggingface TRL's SFTTrainer. TRL is a full stack library to train transformer language models with Reinforcement Learning, from the Supervised Fine-tuning step (SFT), and Reward Modeling step (RM) to the Proximal Policy Optimization (PPO) step. The library is integrated with transformers-

Here, per_device_train_batch_size is the batch size per GPU/TPU core/CPU for training, gradient_accumulation_steps is the number of update steps to accumulate the gradients before performing a backward/update pass, fp16 is whether to use 16-bit (mixed) precision training (through NVIDIA apex) instead of 32-bit training.

Before starting the training, we can check the memory of the notebook as below-

Now, we will start the training using the following command-

For 1 epoch, it took approx 15:36 mins to complete the training and after the training completion when I checked the memory I was quite surprised to see the stats as below-

Now, we can test our fine-tuned model on a given prompt🙋. For example here I am asking my model to complete the Fibonacci series as below-

Here, in the prompt template, I have given one sample sequence of the Fibonacci series and then I asked it to complete the next sequence with the max token size of 128. You can increase this size to see more completion in the final output. I got the following output from my model-

Looks promising right!😇 We successfully fine-tuned a 4-bit quantized Mistral 7B model to a dataset. Next, to save this fine-tuned model, we can use Huggingface's push_to_hub for an online save or save_pretrained for a local save like the below-

Next, to load this saved model, we can use the PeftModel class from the Hugging Face as below-

That's it guys. In this blog post, we learned about the loading of a 4-bit quantized Mistral 7b model & fine-tuning of that model on a dataset. This process taught us some cutting-edge algorithms like QLORA & DPO. We learned about using a very powerful large language model on a simple T4 GPU-enabled colab notebook. Phew that was indeed quite a lot of learning today👷. But don't need to wait, do your own hands-on by copying my shared notebook in your colab environment and play with the parameters of the APIs I used in the notebook. In the next post, we will learn about the retrieval augmented generation in Gen AI, till then 👉 Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇

Generative AI with LangChain: Basics

noreply@blogger.com (Learn Data Science with PrateekG) — Thu, 11 Jan 2024 17:37:00 +0530

Wishing everyone a Happy New Year '24😇 I trust that you've found valuable insights in my previous blog posts. Embarking on a new learning adventure with this latest post, we'll delve into the realm of Generative AI applications using LangChain💪. This article will initially cover the basics of Language Models and LangChain. Subsequent posts will guide you through hands-on experiences with various Generative AI use cases using LangChain. Let's kick off by exploring the essential fundamentals💁

What is a Large Language Model (LLM)?

A large language model denotes a category of artificial intelligence (AI) models that undergo extensive training with extensive textual data to comprehend and produce language resembling human expression🙇. Such a large language model constitutes a scaled-up transformer model, often too extensive for execution on a single computer. Consequently, it is commonly deployed as a service accessible through an API or web interface. These models are typically constructed using deep learning methodologies and neural networks, with a primary emphasis on processing and generating natural language.

As per NVIDIA- "Large language models largely represent a class of deep learning architectures called transformer networks. A transformer model is a neural network that learns context and meaning by tracking relationships in sequential data, like the words in this sentence". The large language model component generates output (in this case, text) based on the prompt and input. These LLMs can further be fine-tuned to match the needs of specific conversational agents (e.g. if you are building a customer care chatbot). A few use cases of LLM are as below-

How Large an LLM can be?

As per Google- "Early language models could predict the probability of a single word but modern large language models can predict the probability of sentences, paragraphs, or even entire documents". The size and capability of language models have exploded over the last few years as computer memory, dataset size, and processing power increase, and more effective techniques for modeling longer text sequences are developed💥.

Here "Large" can refer either to the number of parameters in the model, or sometimes the number of words in the dataset. Here parameters are the weights the model learned during training, used to predict the next token in the sequence. You can refer following table to imagine the largeness of few available pre-trained LLMs-

Model_Name	Parameters	License	Task
ChatGPT	175 billion	Proprietary	Multi
microsoft/phi-2	2.7 billion	MIT	Text Generation
TinyLlama/TinyLlama-1.1B-Chat-v1.0	1.1B	Apache-2.0	Text Generation
mistralai/Mixtral-8x7B-Instruct-v0.1	46.7B	Apache-2.0	Text Generation
openai/whisper-large-v3	1 million hours of weakly labeled audio and 4 million hours of pseudolabeled audio	Apache-2.0	Speech Recognition
nvidia/parakeet-rnnt-1.1b	1.1B	cc-by-4.0	Speech Recognition
bert-base-uncased	110 million	Apache-2.0	Fill-Mask
bert-large-uncased	340 million	Apache-2.0	Fill-Mask

How Do Large Language Models Work?

Large language models are trained using unsupervised learning👈. With unsupervised learning, models can find previously unknown patterns in data using unlabelled datasets. This also eliminates the need for extensive data labeling, which is one of the biggest challenges in building AI models.

The ability of the foundation model to generate text for a wide variety of purposes without much instruction or training is called zero-shot learning👏. Different variations of this capability include one-shot or few-shot learning, wherein the foundation model is fed one or a few examples illustrating how a task can be accomplished to understand and better perform on select use cases.

What is Context Length and Token in LLM?

Context length is the number of tokens a language model can process at once. It is the maximum length of the input sequence. A Token is the model’s way of representing a word with a series of numbers. 100 words is around 130 tokens. If a model does not recognize a word, it breaks the word into multiple tokens. A model can only summarize an article no longer than the context length. In chat apps, the context length dictates how much of the previous conversation is remembered. Language models like ChatGPT or Llama 2-Chat do not remember anything. They are stateless. They know the past conversation because they were included in your current input before feeding it to the model.

What is LangChain?

Much like our familiarity with frameworks such as TensorFlow and PyTorch in the domains of image classification and object detection, LangChain serves as a framework tailored for building applications fueled by language models🙆. Specifically designed for working with Large Language Models (LLMs), LangChain simplifies the intricate process of developing extensive language models by presenting a systematic sequence of steps to generate text from a given input prompt. This toolkit manages various aspects, including prompt input, text generation, and manipulation of the generated output. Essentially, it facilitates the creation of conversational agents that leverage LLMs to generate coherent and natural language responses☝.

Components of LangChain Framework?

This framework consists of several parts-

1. LangChain Libraries: The Python and JavaScript libraries. Contains interfaces and integrations for a myriad of components, a basic run time for combining these components into chains and agents, and off-the-shelf implementations of chains and agents.

2. LangChain Templates: A collection of easily deployable reference architectures for a wide variety of tasks.

3. LangServe: A library for deploying LangChain chains as a REST API.

4. LangSmith: A developer platform that lets you debug, test, evaluate, and monitor chains built on any LLM framework and seamlessly integrates with LangChain. Currently, it is in the beta stage as of writing this post on 10 Jan'24.

How does LangChain work?

LangChain provides a pipeline of steps that generate text from input prompts. The pipeline comprises seven main components: input (prompt templates), the large language model, agents, utilities, document loaders, chains, indexes, and memory.

The input component specifies the input prompt or the initial input provided to the pipeline. This comes in the form of a template that defines the structure of the prompt, including the format and content.

LangChain enables access to a range of pre-trained LLMs (e.g., GPT-3.5) trained on large datasets. The large language model component generates output (in this case, text) based on the prompt and input. These LLMs can further be fine-tuned to match the needs of specific conversational agents (e.g. if you are building a customer care chatbot).

With the Document Loaders module, we can ingest documents (e.g., PDFs, powerpoints, etc) into the LLM for further analysis (typically question answering).

LangChain provides an extensive collection of common utilities (Utils) to use in our application, such as Python REPLs (LLM would generate code to calculate the answer, run that code to get the answer, and print it out), bash commands (e.g., to interface with the local system) or search engines, as well as a requests wrapper (to link a URL published post-2021 that ChatGPT doesn’t know about, for instance).

Agents are systems that use a language model to interact with other tools. They can be used to create chatbots or power the next generation of personal assistants.

Chains provide an end-to-end pipeline for utilizing language models. These chains seamlessly integrate models, prompts, memory, parsing output, and debugging capabilities, offering a user-friendly interface.

In LangChain, Memory refers to the mechanism that stores and manages the conversation history between a user and the AI. It helps maintain context and coherency throughout the interaction, enabling the AI to generate more relevant and accurate responses.

How to install LangChain?

We can install the LangChain library using the pip or conda command. The pip command is pip install langchain and conda command is conda install langchain -c conda-forge

For example,, in google colab notebook installation will look like the below-

Building something with LangChain?

LangChain enables building applications that connect external sources of data and computation to LLMs. We will start with a simple LLM chain, which just relies on information in the prompt template to respond using one of the open-sourced LLM mistralai/Mistral-7B-v0.1. Mistral 7B is a new 7.3 billion parameter language model that represents a major advance in large language model (LLM) capabilities👀. It has outperformed the 13 billion parameter Llama 2 model on all tasks and outperforms the 34 billion parameter Llama 1 on many benchmarks👊. You can refer following table showing the number of parameters and minimum RAM required to run the two available Mistalai LLMs:

These models are available in Hugging Face and Hugging Face has inbuilt APIs to use these models easily. Let's start with installing the required library from the hugging face-

Next, we will import the required libraries-

Next, we will download the pre-trained LLM from the Hugging Face space along with its tokenizer. Please note that tokenization is a crucial text-processing step for large language models👈. It splits text into smaller units called tokens that can be fed into the language model. Also to download the model you will need 16GB RAM at least. But free and Pro Google Colab accounts do not have enough CPU RAM to download the official model containing the weights and you will get Out of memory error😓 if you try to download the whole model at once-

To solve the out-of-memory error or to be able to download the model in a resource-constrained notebook, we can download the sharded version of the original model that somebody had already done for us😀. Before that install following these two libraries as well in colab and don't forget to restart the session-

Now we will load the shared version of the model and it's tokenizer like below-

Here, the BitsAndBytes library is used to create 4-bit quantization with NF4-type configuration to load our model in 4-bit precision. It will help us load the model faster and reduce the memory footprint💣.

Once you run the above-mentioned code, downloading will be started without any out of memory error like the below-

Once we download the model successfully👍, the next step is to test it with user given prompt and see how it answers that prompt query. Mistral expects the prompts in the following format-

So let's create a sample prompt by following the above format:

Now, we will pass out text to the tokenizer and then generate the answer to the user query from our model in the following way-

Here, we use the model's generate function to generate the text based on our query. The output of the above code will look like below-

As you can see, the answer is quite promising and accurate👌. We have a list of models with their source links and explanations. This shows the power of text generation and understanding of human language by a machine💥. The generative AI field is quite impressive, right?

That's it for today's guys! It's time for you to do some hands-on and play with the code using this link. Copy the notebook in your space and do some experiments with it to get familiar with LangChain and Large Language Models👲. In the next post, we will learn to fine-tune this Mistral model to our use case, till then 👉 Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇

How to train a custom dataset with YOLOv7 for instance segmentation?

noreply@blogger.com (Learn Data Science with PrateekG) — Fri, 16 Sep 2022 18:48:00 +0530

Another post starts with you beautiful people! It is overwhelming for me to see massive interest in my last three posts about the YOLOv7 series💓. Your response keeps me motivated to share my learning with you all 💝. If you have not checked my previous posts about YOLOv7, then I am sharing here links to read those once and then proceed with this post-

Till now we have learned about object detection with YOLOv7. In this post, we are going to learn how can we train a custom dataset for instance segmentation task with YOLOv7 👌. For your information instance segmentation is the task of detecting and delineating each distinct object of interest appearing in an image. For our hands-on we need a dataset having images and their annotations in polygon format and of course in YOLO format. So I have found and downloaded the American Sign Language dataset in the required format from this link.

This dataset has all letters A through Z in American Sign Language labeled with polygon labels. To our knowledge, the alphabet and its correspondence signs look like below-

For this post also I am using the google colab environment. Let's first install its required library and restart the colab notebook runtime once it is installed-

Next, we will download the code repo of the YOLOv7's segmentation branch since while writing this post author has not officially declared the main branch for the segmentation code. So I have downloaded the u7 branch and uploaded it to my gdrive so that I can access it in my colab notebook 😀. After uploading the branch code to drive, I unzipped it like as below-

The unzipped folder structure of the dataset will look like as below-

Next, we will prepare the required YAML files for our instance segmentation training. First, we will create our configuration YAML file where will define the path of the training and validation images, the number of the target classes, and the names of the target classes as below-

Here I have provided the train and validation images path, and the number of classes as 26 since our classes are from the alphabet A-Z. Next, we will create a second YAML file having the content of it's default yolov7-seg.yaml but with one difference- the value of 'nc' parameter from 80 to 26-

Next, we will create the hyperparameter YAML file having the same content as its official hyp.scratch-high.yaml as below-

Next, we will download the pre-trained weight file for the training as below-

Now we are ready to start our instance segmentation training with the below command-

Here, change the .yaml files paths and the weight file path as per yours. In my colab environment for 100 epochs it has taken 1 hour to complete. for better accuracy, you should train it for 300 epochs for this dataset. Once the training is completed, you will see the following output in the console-

After completion of the training, let's check the trained model on a test image as below-

You will see the following output in the console after successfully running the above command-

Let's open the processed image and see what it looks like-

See our trained model is able to do the instance segmentation and is able to predict the target class of a sign 💥. Let me share a few more output images-

Indeed we have a strong sign language detection model. Let's run the inference on a video-

The output in the console will look like the below after a successful run of the above command-

Let's open this video in our colab notebook-

And it will show the processed video file as below-

That's it for today's guys! We have learned another useful technique with YOLOv7. Here I am sharing my colab notebook for this tutorial. So no need to rest, just copy the notebook into your colab environment and try it with your own dataset. In my next post I will share something useful again till then 👉 Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇

Export your custom YOLOv7 model to TensorRT

noreply@blogger.com (Learn Data Science with PrateekG) — Thu, 8 Sep 2022 17:45:00 +0530

Another post starts with you beautiful people! Thanks for the great response to my last post where we successfully exported a custom-trained YOLOv7 model to ONNX format. In this post, we are going to move one step further to achieve high-performance inference using NVIDIA TensorRT. We will learn how can we export our YOLOv7 ONNX model to a TensorRT format.

NVIDIA TensorRT is an SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applications. This file format is widely used in case you need to deploy your model into edge devices but with high inference speed. So knowledge of exporting your YOLOv7 model to TensorRT format is very beneficial learning.

Please note the exportation of the model is environmentally sensitive. It is recommended to use the same environment where you have trained the original YOLOv7 model. For this post as well I am using the same Google Colab environment which I used for YOLOv7 training as well as for exporting to ONNX. As a first step, we will install the required dependencies as below-

Once the above-required libraries are installed successfully, as a second step we will download the original YOLOv7 code repo for exporting our custom trained YOLOv7 model to ONNX format. Please skip this step if you have already exported it using my last post.

Let's export the YOLOv7 weight file to ONNX format using the below command -

In the above command, replace the .pt file with your own YOLOv7 file and remove the zero after --device argument if you don't have GPU. Once successfully finished, this command will save a .onnx format file in the same location where your .pt file is. Now as a third step we will clone another code repo for TensorRT export as below-

After successfully cloning the repo, as a fourth step, we will use its export.py file to convert our ONNX file to TensorRT format as below-

Here again, replace the .onnx file with your own file and rename .trt file name as per your wish, the final exported file will be saved with that name. Once the above command runs successfully, you will see the following output at the end of the console-

So our onnx to trt exportation is done successfully ✌. Next as the fifth and last step, we need an inference script to load this .trt file and test it on a test image. Let's understand the inference script with code snippets-

First, we have imported the required library for the inference. Next, we will write a class with a few necessary functions. Inside this class, Our first function is for loading and initializing the TensorRT engine as below-

Our next four functions are for running the inference, transforming the input image into the required format, detecting the target object with confidence scores, printing the frame per second, and the bounding box coordinates as below-

Next, we will create a list of colors for use in the bounding box. A sample list will be looked like as below-

And our last function is for visualizing the bounding boxes around our target class as below-

Now we are ready to make an inference. Let's test an image with our TensorRT model-

Here in the first line pass your .trt file as engine_path and provide your test image in the inference() function. Next, I used matplotlib library to visualize the processed image and it looks like the below-

It looks perfect👌. I have got the result with 39.9 FPS which is very fast 💪 considering my original YOLOv7 custom model was based on its biggest size pre-trained weight which performance on MS COCO dataset was as below-

So if you train your dataset with its smallest version and then convert it to TensorRT, you may get 100+ FPS. Sounds interesting😮 Then why are you waiting? Start training your own YOLOv7 model, export it to the TensorRT engine, and compare the inference speed in CPU/GPU-based devices. Like always, you can find this post's colab notebook here. That's it for today guys. In my next post I will again share something useful till then 👉 Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇

Export your custom YOLOv7 PyTorch model to ONNX

noreply@blogger.com (Learn Data Science with PrateekG) — Fri, 2 Sep 2022 22:10:00 +0530

Another post starts with you beautiful people💓. In my previous post, we learned about training a custom dataset with an official Pytorch-based YOLOv7 object detector. If you have not seen that post, I recommend you check it once. The link is here. Once we achieve the best model, the next important step is to use that model. Sometimes you may need to use multiple ML models which you have trained on different-different ML frameworks like PyTorch, TensorFlow, Caffe, etc. In production or the real world, the trained model can be deployed as Rest API, or integrated with a web application without changing its form but what if you need to use that within your mobile device as an Android or iOS app as well as you want to use it with an embedded system like Nvidia Jetson 💣? Here comes the problem of interoperability. In this post, we are going to learn how can we export our custom YOLOv7 model to ONNX format.

ONNX is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers. For our export, I will use the same Google Colab platform where I had trained my custom yolov7 model. Let's first import some required dependencies-

Since Google Colab has already PyTorch installed so we don't need to install it. During this work, my Colab had the following version of Python and PyTorch-

Next, we will download the official code repo of YOLOv7 as below-

Once the code base is downloaded, let's check a test image with our trained YOLOv7 model so that in the last of this post we can compare it with the exported ONNX model inference. The command to do the inference is as below-

! python detect.py --weights best.pt --img-size 1280 --source test_image.jpg

Here, just replace best.pt with your custom YOLOv7 weight file and test_image.jpg with your test image. After successfully running of above command the resulting image will be saved in the path- runs/detect/exp/test_image.jpg. Let's check the outcome by opening the image below-

And it looks like as below-

So till now, everything looks fine. Let's move to our export. For this purpose, the official code repo that we had already downloaded has a script named export.py.

We will use the same to export the model into ONNX format as below-

You can check export.py to understand it's each argument. After successfully running of above command you will see the following output in the console-

Now, the next most important step is to do the inference with this exported model. Let's understand this with below code snippets one by one-

In the above code snippet, first, we have loaded the ONNX model, our test image, required libraries for inference, CPU or GPU selection, and the main class of the ONNX runtime as a session. In the next code snippet, we will create a function to do the following things with the given input image-

Resize and pad image while meeting stride-multiple constraints
Scale ratio
Compute padding
divide padding into 2 sides
Resize and find the top, bottom, left, and right coordinates
Add border and return the image with the scale ratio and sides

Next, we will create a list of our target object names- in my case, it is wheat, color scheme for the bounding boxes. Then we will apply the above-created function- letterbox() to the image, will convert the image to a NumPy array and then we will normalize it for further processing. Next, we will extract the output from the model as below-

Now, we are ready for making the inference. First, we will fetch the output of the ONNX inference session and then we will iterate it to get the image, bounding box coordinates, confidence score, and the class name, and then we will use this information to draw the bounding boxes around the detected class with its name as below-

That's it! Our inference script is ready for our ONNX file and once executed it had shown me the output image as below-

If you compare this image with the previous output image you will find although the confidence score of the detected class has been dropped yet the accuracy of the exported model is the same. It means our exported ONNX model is working as expected. One more point to observe in our inference script is that we have not imported the PyTorch library which we used to train the custom dataset with YOLOv7. That is the interoperability of the ONNX format. Now our production deployment team can choose any ML framework they are comfortable with and using this unified inference framework (ONNX), the exported model can be deployed to anywhere. You can find the complete Colab Notebook here.

So why wait now☝ Just copy the notebook, export your custom YOLOv7 model to ONNX format and deploy it to production💥. That's it for today guys. In my next post, I will share to export the model into TensorRT format till then👉 Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇

How to train a custom dataset and make inference with Official YOLOv7 on Kaggle?

noreply@blogger.com (Learn Data Science with PrateekG) — Thu, 4 Aug 2022 21:28:00 +0530

Another post starts with you beautiful people! I hope you have already learned the state-of-the-art object detection technique Darknet's YOLOv4 from my previous posts. If you are new to my blog then I recommend you to once go through this link to learn about end-to-end implementation of YOLOv4. In this post we are going to learn the successor of the official YOLOv4 : YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors 💥. Since the official implementation of YOLOv7 is still not done in the Darknet framework, in this post we are going to use the Pytorch framework to train a custom dataset and then make inferences using the same Pytorch framework. But don't worry if you are new to PyTorch 👲, we are not going to reinvent the wheel in fact as a practitioner our goal should be to learn how to use it.

For our learning, we will use a real-world dataset and the problem- detecting starfish in underwater images. This challenge was held on Kaggle. It is a perfect use case for our learning since there are many other tutorials that use public datasets. However, preparing a dataset is also an important job before applying the training. So first let's understand the given dataset format-

Goal: To predict the presence and position of crown-of-thorns starfish in sequences of underwater images taken at various times and locations around the Great Barrier Reef.

Dataset: train/ - Folder containing training set photos of the form video_{video_id}/{video_frame_number}.jpg. [train/test].csv - containes following metadata for the images: video_id, video_frame, sequence id, sequence_frame number, image_id, annotations in COCO format.

Dataset Preparation: Let's take a look at the dataset with our Python code:

Let's verify if all the images have the annotations or not. We will later remove the images from our training dataset which don't have the annotations otherwise it will affect our model accuracy-

So we have only 20.93% annotated images in the given dataset. Let's remove un-annotated images metadata from our dataframe-

Also, let's remove the unannotated images with the help of joblib's parallel function for speedup. Here we need to copy the data to Kaggle's output directory since the input directory doesn't have the write access-

So, we have now a total of 4919 annotated images to train/validate. Our next step is to visualize the images and see what annotations look like in these images. For this purpose, we don't need to write the code from scratch instead we will use bounding box utility github 👼. Don't forget to give a start to this GitHub repo 👍. Let's import the helpful bounding box creation functions from the repo as below-

Next, let's fetch the bounding box coordinates using the above function-

Let's check the newly created column in our dataframe-

Now we have bounding box coordinates as well in COCO format. Since the given dataset has images of fixed sizes" (1280. 720); it's better to set another two columns- width and height in our dataframe as below-

Next, following the best practice of training dataset preparation, we will apply cross-validation to the dataset as below-

As you can see the number of samples is not same in each fold, so for the training we should experiment with each fold but for this post, I will use the first fold-

Prepare YOLO format dataset: Since till now we have cleaned and finalized the training dataset; the next step is to prepare this dataset in YOLO required format. For this step, we will export the labels into a .txt file. This .txt file will contain one row per target class where each row is in the following format: target_class_name [x_center, y_center, width, height] . Here the coordinates must be in normalized form and target class numbers start with zero. All these required conversions can be done using the below code-

Let's check our converted bounding box coordinates in one of our cross folded dataframe-

Now we can visualize the training dataset images with their annotations using the matplotlib library as below-

And the sample annotated images look like the below-

After seeing the above few samples, we can imagine how complex a problem is this to solve with computer vision as compared to publically available datasets. Here the target class is very small and images are blurred. Let's see how YOLOv7 is going to detect these starfish 👀. But for now, let's create the .txt files required by YOLO-

Here 'nc' means a number of classes to detect that is 1 in our case, 'names' is the target class name that I defined as cots but you can give any name to it.

YOLOv7 training:

Till now we have prepared the dataset to start the training. For the custom training, YOLOv7 has provided the different sizes of pre-trained weights-

Here I am going to use the largest one: yolov7-e6e_training.pt. To use this pre-trained weight for our training, we also need to use its configuration file (yolov7-e6e.yaml) that we can get from this official config files link and the hyperparameter file (hyp.scratch.custom.yaml) that we can get from this custom hyperparameter file link. Now first we will create a new hyperparameter configuration file and then we will copy the content from hyp.scratch.custom.yaml file to this new file in Kaggle as below-

The same we will do to copy the content of the yolov7-e6e.yaml file to a new file in Kaggle-

Notice here in the above content, I have changed 'nc:80' to 'nc:1' since we have only one class to predict. Rest all parameters are the same. Next, we will clone the official YOLOv7 repo in Kaggle's output directory, and then we will install the dependencies as below-

Once all dependencies are installed, we will download the pre-trained weights as below-

Now, we are ready to start the training with the below command-

Here, a number of workers I have defined as 8 which you can change accordingly to the system of yours, device as zero means single GPU training, the default number of epochs are 300 but due to limited training time in Kaggle I have used 26 as epoch size, batch size I have used as 3 to avoid out of memory issue but it should be 16 and rest other parameters are the pre-trained weight file path and the config file paths. Once the training is completed you can find the best-trained weight file (best.pt) under /yolov7/runs/train/ path as below-

YOLOV7 Inference: Since our training is completed, it's time to make inferences on the test images and prepare a submission file as required by the competition. For making inferences first step is to load our trained model. For this purpose, we can use torch.hub.load() function as below-

In this function, the first parameter is the YOLOv7 repo path, the second parameter is 'custom' since we are using a custom trained model, the third parameter is the best weight file path, and the fourth parameter is given as 'local' since we are loading the model from our local file system and the rest other configurations we are setting to our loaded model are easily understandable.

After loading the model next step is to make the prediction. As required by this competition we need to predict the bounding box coordinates in COCO format and the confidence score of a given image in prediction and that can be achieved as below-

Here for bounding box conversion, I have used the helper code from this awsome repo 🙏. Now we can make the required submission as below-

Here, we have iterated the iter_test variable since test images were not given as a whole instead organizer shared the API. After iterating each test image we made predictions using our custom-trained YOLOv7 model and formatted the predictions in the required format. Let's see one test image with the prediction-

It looks good 👌 but how can we know the goodness of our model? That's why I used the Kaggle dataset so that I can submit the prediction to this challenge and see my private score. Same I did and I got the following score with my trained model-

Next, I checked for any other submission but with YOLOv5 and found the below score-

As we can see here, with default configurations of YOLOv7, my model has achieved a 0.602 private score as compared to 0.588 achieved by YOLOv5 which indicates that indeed the latest official release of YOLOv7 is faster and more accurate than other object detectors 💪. We can achieve a better result with hyperparameter tuning, different augmentation, etc.

For complete code reference please follow my training script here and the inference script here. So no need to rest guys 💣. Just forked my training notebook, do experiments, make submissions, and see how much can you score on the leaderboard 💨. This post and the shared notebook will be helpful in your journey to learn and use the state-of-the-art object detector. In my next post I will again bring you something amazing to learn till then 👉 Go chase your dreams, have an awesome day, make every second count, and see you later in my next post 😇

YOLObile- a new state of the art Real-Time Object Detection model for Mobile Devices

noreply@blogger.com (Learn Data Science with PrateekG) — Wed, 14 Oct 2020 16:36:00 +0530

Another post starts with you beautiful people!

Thanks for giving so many views on my previous post 👍.

I am glad to see my previous posts are helping people to use state of the art object detection and recognition deep learning model in their projects. If you are new to my blog, I recommend seeing once my previous posts, and you will not be disappointed if your goal is to learn applied computer vision free of cost. Continuing my journey of sharing knowledge in this post I am going to share with you a new state of the art framework for object detection on mobile devices- YOLObile 📱

There has been a trade-off between speed and the accuracy of object detections. For example, the state of the art, YOLOv4 model gives us a very accurate detection but its speed is slow if we want to use it on a mobile device. On the other hand, its lighter version YOLOv4-tiny works very fast on a mobile device but its accuracy reduces. For a detailed comparison of FPS vs mAP you can refer to the above diagram. Nowadays computer vision techniques are improving day by day due to hardware advancement and so using AI in mobile devices is a hot choice for every mobile app.

However, the current state-of-the-art object detection works are either accuracy-oriented using a large model but leading to high latency or speed-oriented using a lightweight model but sacrificing accuracy. To solve this problem respected authors [Yuxuan Cai, Hongjia Li, Geng Yuan, Wei Niu, Yanyu Li, Xulong Tang, Bin Ren, Yanzhi Wang] of this paper have introduced the YOLObile framework, a real-time object detection on mobile devices via compression-compilation co-design. 💥

As per the authors- A novel block-punched pruning scheme is proposed for any kernel size. To improve computational efficiency on mobile devices, a GPU-CPU collaborative scheme is adopted along with advanced compiler-assisted optimizations. Experimental results indicate that our pruning scheme achieves a 14x compression rate of YOLOv4 with 49.0 mAP. Under our YOLObile framework, we achieve 17 FPS inference speed using GPU on Samsung Galaxy S20. By incorporating our proposed GPU-CPU collaborative scheme, the inference speed is increased to 19.1 FPS and outperforms the original YOLOv4 by 5x speedup.

That's quite a solid approach right 💪. If I summarize their approach then I would say that their approach is basically unstructured weights pruning, with a bit of structure on top, where they group the weights in blocks of 4 channels and 8 filters (x32). This structure allows them to improve parallelism. One important point to know here is that unstructured sparsity is very difficult to parallelize compare to structured sparsity also it is hardware-specific.On the other hand, up to 93% of reduction in weight size without a real loss in accuracy is really tempting with this framework. 💝

I know after reading this all of you want to see the result of this new approach on any benchmark dataset💥 and the good news is that authors have recently released the weights (in PyTorch format) which you can download from this link

From the download link, you will get two weights files of 245 MB size: best8x-514.pt and best14x-49.pt. These two weights are the output of the 8x and 14x pruning models. According to the authors- these models are trained on a server with eight NVIDIA RTX 2080Ti GPUs. The training methods are implemented using PyTorch API. The details of both weights are as below:-

I tested the MAP on coco 2017 dataset (80 classes) with image-size of 320, batch-size of 8, best8x-514 weights in my Windows 10 laptop having 6GB of NVIDIA GTX 1060 and results are as below-

For evaluating this framework, the authors used an off-the-shelf Samsung Galaxy S20 smart-phone, which has a Qualcomm Snapdragon 865 Octa-core CPU and a Qualcomm Adreno 650 GPU. Each test ran on 50 different input frames (images), with the average speed results reported. The YOLObile is derived based onYOLOv4, with 320×320 input size, and train on the MS COCO dataset. They denote mAP as the Average Precision under IoU 0.5 threshold and AP@[.5:.95] as the Aver-age Precision under IoU from 0.5 to 0.95-

The above evaluation result is quite amazing right💥. A real-time object detection framework on mobile devices via compression-compilation co-design will definitely going to be a game-changer. I have uploaded the inference code in my github repository You can follow the same to test the new framework in your own system. Since this framework is quite new there are no open-source code is available for the training on a custom dataset but as soon as I found any one of the Darknet or TensorFlow implementation, I will share with you all. Till then Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇

How to use opencv-python with Darknet's YOLOv4?

noreply@blogger.com (Learn Data Science with PrateekG) — Fri, 14 Aug 2020 21:22:00 +0530

Another post starts with you beautiful people 😊

Thank you all for messaging me your doubts about Darknet's YOLOv4. I am very happy to see in a very short amount of time my lovely aspiring data scientists have learned a state of the art object detection and recognition technique. If you are new to my blog and to computer vision then please check my following blog posts one by one-

Since now we have learned to use YOLOv4 built on Darknet's framework. In this post, I am going to share with you how can you use your trained YOLOv4 model with another awesome computer vision and machine learning software library- OpenCV and of course with Python 🐍. Yes, the Python wrapper of OpenCV library has just released it's latest version with support of YOLOv4 which you can install in your system using below command-

pip install opencv-python --upgrade

For Google Colab and Kaggle, use the following command-

!pip install opencv-python --upgrade

For example in my Colab notebook, the above command with output is like below-

As you can see in the last line of the output, the version of opencv-python is 4.4.0.40 which is, in fact, the latest one so our installation is done successfully. Now I will show you how can we test a pre-trained YOLOv4 model trained on coco dataset with Python and OpenCV-

Let's understand the above code snippet💭-

First I have imported the OpenCV library as cv and cv2_imshow library specific to Colab for opening image in Colab notebook.

In the next line, I have created an OpenCV model from the YOLOv4 configuration and pre-trained weights file. If you don't have these files then you can download those from this link. Here OpenCV's dnn_DetectionModel() function is all that you need for this purpose 💓

After creating the model, I did some input image preprocessing for this model like resizing the image to a specific resolution, normalizing, and maintaining the RGB format.

In the next line, I have uploaded a new unseen image where I want to detect some objects.

Next, I am iterating the names of the objects. Since our pre-trained model is trained on 80 classes known as the coco dataset, we need to take the names one by one from the file.

In the next line, I am using detect() function to detect the objects on the input image. This function is equivalent to the Darknet's detection function. It gives the object name, calculates the confidence score, and gives the bounding boxes.

Next, inside the for loop, I have written logic to put object name, confidence score, and draw a bounding box around the detected object using OpenCV inbuilt functions.

In the end, to show the image with the detection I am using the cv2_imshow() function. If you are using your own system then instead of this function use the OpenCV's imshow() function.

Now we are ready to test our image. Once I run the above code, I get the following image with the detection-

See the result, it's amazing right 💥. The pre-trained YOLOv4 model with OpenCV and Python is successfully able to detect person, wine glass, dining table, bowl, bottle with a high confidence score. In the same way, you can test your custom trained YOLOv4 model. You just need to replace YOLOv4 specific configuration file, weights, and class name file with your own, and the remaining code will be the same as it is. Let me show you the example where I have used my custom YOLOv4 model which I trained for a Global Wheat Head Detection Challenge. Since at the time of the participation OpenCV support was not released for Darknet's YOLOv4 so I had to convert my YOLOv4 model to keras/TensorFlow model and made changes accordingly. But now I can use it very easily 👲-

Wow, that was so easy 💪. For this competition, I had to put so much effort to make Darknet work in the Kaggle platform. But now with opencv-python, it is very easy for me. If you are interested to see my complete solution to the challenge you can check this link ☝.

That's it guys for today! From this post, you have learned to use your Darknet's YOLOv4 model with OpenCV and Python. Now there is no solid reason to not use this state of the art object detection and recognition neural network model in your project. The links I have shared at the starting of this blog and my solution link of a real-world object detection competition are quite enough to master in this field. So don't stop yourself 💥, try the learning from this post on your own, explore the code with experiments and in my next blog post, I will again share with you something new and exciting. Till then Go chase your dreams, have an awesome day, make every second count, and see you later in my next post😇

Learn the fastest way to build data apps

noreply@blogger.com (Learn Data Science with PrateekG) — Sat, 11 Jul 2020 22:37:00 +0530

Another post starts with you beautiful people!
I hope you have enjoyed and learned something new from my previous three posts about machine learning model deployment. In one post we have learned How to deploy a model as FastAPI? In the second post, we have learned How to deploy a deep learning model as RestAPI? and in the third post, we have also learned How to scale your deep learning model API? If you are following my blog posts, you have seen how easily you have transit yourselves from aspiring to a mature data scientist. In this new post, I am going to share a new framework- Streamlit which will help you to easily create a beautiful app with Python only. I will show here how had I used the Streamlit framework to create an app for my YOLOv3 custom model.

What is Streamlit?
Streamlit’s open-source app framework is the easiest way for data scientists and machine learning engineers to create beautiful, performant apps in only a few hours! All in pure Python. All for free. With this framework, building an app with ML/DL/NLP model is as easy as writing Python scripts.

Installing Streamlit?
To install Streamlit, open your activated virtual env in Anaconda Prompt, or if you have not one then create it. Then run the following command to install the Streamlit framework: pip install --upgrade streamlit
In my case, I have created a virtual env 'stream' in Anaconda Prompt with Python 3.7 and it looks like below-

Once the installation is done we can verify it by running its inbuilt help app with below command-

As soon as you run the above command, it will ask you to enter your email. Fill up your email and press enter. After this, a welcome screen will appear as well as in the next few seconds the sample app will open in a new tab in your default browser. The welcome console will look like below-

And the default app in the browser will look like below-

Click on the 'Choose a demo' dropdown. You will see their sample demo options-

Now click on any of the options. I selected the last option and it looks like below after the selection-

How beautiful it is, right! This demo app demonstrates a Pandas Dataframe representing the gross agricultural production for some years. Now scroll down a little bit and you will see the code used to build this beautiful app. See the complete code is in Python. The dataset 'agri.csv.gz' is imported from AWS S3 bucket location like we usually do while importing the data in csv format and rest in few lines the UI logic is written as per Streamlit framework-

Now I will jump to the actual demo of my app. Here I will show you how had I created an app for my object detection and recognition neural networks model. This YOLOv3 based model I created to detect wine bottles and recognized their names from the image. With Pycharm IDE I have created a project, then I created an empty folder 'model' . There I copied my model weights and configuration files. Next, I created the app.py file where I will write my code of image uploading, image preprocessing, passing the image to the model, and showing results in Streamlit UI.

Here, I show you my project structure in Pycharm IDE-

I have saves some sample images inside the images folder. the model folder contains weights and configuration files of my object detection and recognition models I trained on YOLOv3. app.py is my main file which will act as Streamlit application and contains logic to display bounding boxes and brand names in the image. brand_recognition.py is another file which contains logic to recognize the brand names. Based on your machine learning or deep learning model you can maintain this structure as you want. Here the main thing is to see how app.py is using Streamlit related features. Let's understand app.py structure-

In above code snippet, after importing required libraries I have defined the app layout containing title, header, and a slider. The 'st' object of Streamlit has many useful functions for UI which you can easily check in Pycharm IDE as soon as you type dot(.) after st object like below screenshot-

For details inbuilt functions please refer this official link. Next, I have defined a function to read the image. For this purpose I used OpenCV. Since OpenCV loads the image in BGR format I converted the image in RGB format otherwise your image will show in blue color-

Above this function, I used a cache decorator. The Streamlit cache allows our app to execute quickly even when loading data from the web, manipulating large datasets, or performing expensive computations. Based on your need you can add/remove this. After this, I have defined a function that contains YOLOv3 related logic- loading the weights and configuration files, getting the output layer, identifying the best bounding boxes, and recognized the brand names to the detected bottles. Here you can call or write your logic of your ML/DL model-

Inside the above yolo_v3() function in the last, I used the following Streamlit functions to display my final output image-

Next, for showing some sample images options in the left sidebar of the app I used st.sidebar.selectbox() function and passed some options there. You can think of this as a dropdown value. In the end, I read the given image using my read_img() function and then called the yolo_v3() function to show the output-

That's it we have our simple app is ready to test. For running the Streamlit server run following command: streamlit run app.py

Your default browser window open as soon the above command runs and you will see following look like screen-

Looks amazing right! With very few Python-like syntaxes of Streamlit, we have easily created an app for our model. No need of UI or JavaScript knowledge to create an app now. As a Data Scientist with Python knowledge now you can give more focus on your model performance and in less time without any help you can create an app also. That's the power of the Streamlit framework.

Let's move ahead and what if you want to give the user an option of uploading the image instead of static images? For this, we need only a few changes in our app.py file. Using Streamlit's .file_uploader() function, we can provide image uploading option to the user. We need to make the following changes in our app.py file to achieve this functionality-

Here I am going to open the uploaded image using PIL's Image module that's why we don't need to again open the image using cv2.imread() function and also we don't need to convert the image in RGB format. Next change is the comment the left sidebar sections and put the image upload option of Streamlit-

That's it. Our app is now ready. Restart the Streamlit server and you will see upload image option in the app like below screen-

Click on the browse files link and upload an image to test your model. After uploading an image following look like screen will appear-

See, now our app is ready to test any new image user wants to check. With the Streamlit framework, we can now easily create an app with our ML/DL model. In this post, I shared you the basics of this amazing and powerful framework. The easiest way to master in using Streamlit is to try things out yourself. I know you have many ML/DL models to test and after reading this post you are very excited to create many apps for all of your models. So don't stop yourself! Use this post as reference, start creating your app, do more experiments with it's UI, and deploy your app in Heroku. Till then Go chase your dreams, have an awesome day, make every second count, and see you later in my next post.

My solution to HackerEarth's Identify the dance form challenge

noreply@blogger.com (Learn Data Science with PrateekG) — Sun, 5 Jul 2020 21:14:00 +0530

Another post starts with you beautiful people!
Today an interesting deep learning challenge is finished in HackerEarth and I got 91.17026 mAP score in the leader board. One drawback I see in HackerEarth is due to small dataset many participants manually prepare the submission files and show 100% score in the leader board. Many aspiring data scientists see this and become nervous. Even with getting score 75+, they become demotivated and leave their experiments in between the challenge. Also the winning approach is not disclosed after the challenge. With this post I will try to motivate my all aspiring data scientists and I will share my solution so that in their next challenge they can easily get 85+ score or even 92+ score :)

Problem statement
An event management company organized an evening of Indian classical dance performances to celebrate the rich, eloquent, and elegant art of dance. After the event, the company plans to create a micro site to promote and raise awareness among people about these dance forms. However, identifying them from images is a difficult task.

You are appointed as a Machine Learning Engineer for this project. Your task is to build a deep learning model that can help the company classify these images into eight categories of Indian classical dance-

Manipuri
Bharatanatyam
Odissi
Kathakali
Kathak
Sattriya
Kuchipudi
Mohiniyattam

My Solution

Due to small size of the dataset, I decided to frame this challenge as object detection and recognition problem and used AlexeyAB/darknet/YOLOv4 to train and test the images. If you are new to this framework, please follow my below two posts to get started-

For the dataset preparation, I manually annotated the dataset in required YOLO format. The complete annotation process took 1.5 hours. The complete training of the model took 18 hours in my system. If you don't have GPU enabled system, don't worry! Follow this amazing notebook to install, build and train your custom dataset with YOLOv4 in colab-

YOLOv4 in Colab

Once the training is completed, weights files are generated inside the backup folder of the darknet root directory. The file name is yolo-obj_final.weights and it's size is around 250 mb. Now to make prediction on your all test images run the following command-

./darknet detector test data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_final.weights -ext_output -dont_show -out data/dance_result.json < data/test.txt

The above command will generate a json file. This json file contains the detected object name, it's bounding box coordinates and confidence score. To prepare the final submission file I created a script json_csv_submission2.py. After submitting this submission file I got score 91.17026 in the leaderboard. I have uploaded my solutions files to the github and weight file in the kaggle since we cannot push file larger than 100 mb. You can fork or download the training files from below link-

my github repository And you can download the trained weights from this link.

In this post all required things are mentioned to solve an image related deep learning problem. Only one thing is expected from all of you is to get started. Until you practice yourself, you will not learn. I am quite sure that in your next image related challenge you will definitely going to use this object detection and recognition technique and spot a better rank. So push yourself, read my mentioned links, practice yourselves and in my next blog post I will share you another interesting learning. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How to deploy your ML model as Fast API?

noreply@blogger.com (Learn Data Science with PrateekG) — Mon, 29 Jun 2020 20:17:00 +0530

Another post starts with you beautiful people!
Thank you all for showing so much interests in my last posts about object detection and recognition using YOLOv4. I was very happy to see many aspiring data scientists have learnt from my past three posts about using YOLOv4. Today I am going to share you all a new skill to learn. Most of you have seen my post about deploying and consuming ML models as Flask API where we have learnt to deploy and consume a keras model with Flask API . In this post you are going to learn a new framework- FastAPI to deploy your model as Rest API. After completing this post you will have a new industry standard skill.

What is FastAPI?
FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python type hints. It is easy to learn, fast to code and ready for production. Yes, you heard it right! Flask is not meant to be used in production but with FastAPI you can use your model in production with minimal code. Along with this advantage FastAPI also have inbuilt OpenAPI (previously known as Swagger) support. With this OpenAPI support you don't need to use Postman or any other tool for testing your API. You will have a UI where you can easily test your endpoints.

How to install FastAPI?
We can install FastAPI using pip. For this installation open your Anaconda Prompt with admin rights, activate your virtual environment and run the following two commands one by one-

pip install fastapi
pip install uvicorn

Here uvicorn is a lightning-fast ASGI server and enables us to use FastAPI for all our asynchronous calls without any worry. Once you installed this framework, you can confirm the installation by following code snippet-

See the similarity with Flask API. Here in the same manner we have initialized the FastAPI as app variable. Save the above file and to run this file use the command: uvicorn main:app --reload where main is your Python file name and app is the variable we used for the initialization of FastAPI() function. Here --reload parameter is used to take any changes in Python file immediately. After making any changes you will not need to rerun the server. Once your server is up and running you will see following look like message in console-

Now open the url http://127.0.0.1:8000/items/5?q=somequery in a browser window. You will see a json response. Now open the url http://127.0.0.1:8000/docs and you will see screen like below-

See, this is the OpenAPI UI where you can easily test your API endpoints. Now you are aware of basic of FastAPI, we will move to the next step.

Serving my YOLO model as FastAPI?

In this post I am going to deploy my YOLO model as FastAPI. If you still have not tried YOLO then you can learn from previous post and trained your custom model. The API deployment process will be the same for any kind of ML model. Here I am using my custom YOLOv3 models which detects price tags in a given shelf image and then extract the price from those tags. Basically aim of these models are to extract the price of a brand and send in response to front end. So in this example input will be the complete path of an image and response will be extracted price.

Here is my code snippet of main Python file-

As you can see code is similar as we saw earlier. Here I have defined my get_price() function as asynchronous using 'async' keyword. Function is taking one input arguement- image which is the complete path of the image. I am reading the image using OpenCV. The HTTP method of this call is 'GET'. The complete logic for price tag detection and recognition is written in another Python file tag_detection.py which I have imported and called the required function. This tag_detection() function will return the extracted prices. You can replace this function with your own classification/regression or NLP model logic.

I have saved this file as price_extraction.py. So I ran following command to run the server: uvicorn price_extraction:app --reload and after server started I open the OpenAPI UI with command http://127.0.0.1:8000/docs-

Now to test the endpoint click on the GET or your HTTP method button. In my case it is looking like below-

Once you expand your HTTP method, you need to click on Try it out button to enable the input field. Based on your ML model input fields may be more than one. In my case it is one. After clicking on this button, enter the values of the input field and click on the Execute button. This button will trigger your endpoint and return the response.

In my case response screen looks like below-

And my input image was this-

Quite interesting right! With minimal code we are able to deploy and consume our model as Rest API. We also don't need to use any external tool for testing the endpoints. With OpenAPI inbuilt support no wonder companies like Microsoft, Netflix and Uber are using the FastAPI in production. Now you don't need to think too much; just take your existing ML model, wrap it in FastAPI code you learned today from this post and start experimenting with it because we learn by doing experiments in Data Science.

Once you are done with this go ahead and deploy your FastAPI in Heroku by following this link to showcase your model with others. In my next post I will share you my new learning till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How to convert your YOLOv4 weights to TensorFlow 2.2.0?

noreply@blogger.com (Learn Data Science with PrateekG) — Sat, 6 Jun 2020 11:02:00 +0530

Another post starts with you beautiful people!
Thank you all for your overwhelming response in my last two posts about the YOLOv4. It is quite clear that my beloved aspiring data scientists are very much curious to learn state of the art computer vision technique but they were not able to achieve that due to the lack of proper guidance. Now they have learnt exact steps to use a state of the art object detection and recognition technique from my last two posts. If you are new to my blog and want to use YOLOv4 in your project then please follow below two links-

In my last post we have trained our custom dataset to identify eight types of Indian classical dance forms. After the model training we have got the YOLOv4 specific weights file as 'yolo-obj_final.weights'. This YOLOv4 specific weight file cannot be used directly to either with OpenCV or with TensorFlow currently because in the latest release of YOLO a new activation function-Mish is introduced. With current version of OpenCV and TF using YOLOv4 directly is not possible. In this blog post I am going to share you my jupyter notebook code for converting YOLOv4 weights to TF2.2.0.

TensorFlow(TF) is an easy to use framework in production. If we have a TF model we can deploy it in a web application, mobile application or embedded devices. Since I am a Keras guy, I prefer to use Keras API with TensorFlow backend in my projects :) . I have taken the help from the older post of Jason Brownlee PhD and modified the code with YOLOv4 related changes. You can fork or download the notebook from my github repository You can gone through the notebook and if you want to update or add any thing to it, send me a PR in github :)

In the first cell of the notebook I have imported required libraries and also checking the TF and Keras version-

So we have the latest keras and TF version currently available in colab notebook. In next cell, I have defined the newly added activation function Mish as below-

And in third cell, this activation function is added to the convolution layer blocks-

In further cells, I have defined the YOLOv4 architecture as per keras & TF and then other helper functions like drawing bounding boxes, removing unwanted bounding boxes and showing the detecting in an image. Let me show you the screenshots where I have converted a pretrained YOLOv4 weights to a TF model file-

After running above cells you will have a .h5 format file which is infact a TensorFlow model file. Now we will verify that our converted model file is working same as YOLOv4 weights or not. To do so let's import this TensorFlow model file-

With this code our .h5 file is successfully loaded as keras model. Let's define a list of objects on which original YOLOv4 weights was trained and also a function to read those object labels-

Next step is to define a function to load and preprocess an image for keras-

Let's apply this preprocessing function to an image in this way-

Next, we will use the predict() function to make the prediction on our image-

As a last step we will draw the bounding boxes and predicted object on that bounding boxes in our picture. All required processing for this step is already done in the notebook.Once we run the following cell-

Along with predicted object with accuracy, it will give you following like image-

The above image and result confirms that our conversion code is working as expected. That's it for today guys! Go ahead- Fork or download the complete notebook from my github repo, try it yourself, explore it with your custom YOLOv4 weights file and see how can you make this notebook more useful. In my next blog post I will share my new learning with you all. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

Identify Eight types of Indian Classical Dance forms with YOLOv4

noreply@blogger.com (Learn Data Science with PrateekG) — Mon, 1 Jun 2020 18:30:00 +0530

Another post starts with you beautiful people!
Thank you all who had followed my last post about install and compile YOLOv4 in Windows10 and could able to successfully set up the Darknet in their machines. As I promised in last post and you asked for, in this post I am going to share you the steps required for training a custom object with YOLOv4. If you are seeing my blog first time, I recommend you to first follow my last post and then proceed further.
For this exercise I have choosen a dataset of eight Indian Classical Dance forms-

Manipuri from Manipur
Bharatanatyam from Tamil Nadu
Odissi from Orissa
Kathakali from Kerala
Kathak from Uttar Pradesh
Sattriya from Assam
Kuchipudi from Andhra Pradesh
Mohiniyattam from Kerala

You can download the dataset from this hackethon link. After downloading the dataset , you need to create 8 folders with class name and copy respective images from train folder to there. For this work I have written a simple Python script that creates eight folders with the images of dance forms. So you can also do the same. After completing this post you will be well aware of building a Neural Network model to classify eight categories of Indian classical dance. Our first step is to label the images in YOLO format. YOLO requires a specific type of labeling. The labelled format of an image should be in <object-class> <x_center> <y_center> <width> <height> format. Here <object-class> is integer object number from 0 to (classes-1). In our case it will be zero to seven. <x_center> <y_center> <width> <height> are float values relative to width and height of image which can be equal from [0.0 to 1.0]. This steps is the backbone of the YOLO and so it must be performed with precisely.

To label the image we will use labellmg annotation tool. Clone or Download the github code from this link in your machine. Once cloned/downloaded, open anaconda prompt with admin rights, go to the path of labellmg and install required pyqt library using command: conda install pyqt=5

After installing pyqt, run following command: pyrcc5 -o libs/resources.py resources.qrc

This will install annotation tool in your machine.

Now, create a text file named as classes.txt in labellmg root directory and put our target class names(dance forms) in this format-

Now our labellmg setup is done. We will start annotating the images later. first we will prepare some files and make some changes in our YOLO installed directory for the training of our custom dataset. For this step go to your YOLO installed directory(\vcpkg-master\installed\x64-windows\tools\darknet\cfg) and follow below steps-

1. Copy the yolov4-custom.cfg file, paste there and rename it to yolo-obj.cfg

2. Open yolo-obj.cfg file in notepad++ and make below changes-

change line batch to batch=64
change line subdivisions to subdivisions=64
change line max_batches to 16000(classes*2000)
change line steps to 80% and 90% of max_batches, f.e.12800, 14400
set network size width=416 height=416 or any value multiple of 32
change line classes=8
change [filters=255] to filters=39(classes + 5)x3 in the 3 [convolutional] before each [yolo] layer

3. Go to \vcpkg-master\installed\x64-windows\tools\darknet\data directory and create a file obj.names. Copy the content of the classes.txt file there

4. Create a file obj.data in the same directory with below content-

Here update the path according to your directory. For backup, create a folder with name backup in the root directory (\vcpkg-master\installed\x64-windows\tools\darknet). In this folder our custom trained weights files will be saved.
5. Create a folder named as obj inside the directory(\vcpkg-master\installed\x64-windows\tools\darknet\data)

Now your YOLO related changes are done. Next, we will start annotating the images. For this open the anaconda prompt,go to the labellmg directory path and run the following command-
python labelImg.py /dataset/images/bharatanatyam classes.txt

This command will open labellmg window. Here click on the 'Change Save Dir' icon and select the bharatanatyam folder path (/dataset/images/bharatanatyam)-

Above step will save our annotation txt file in the same location where our images are. So in this case of annotation we will have <img_name>.txt file inside the /dataset/images/bharatanatyam folder. Next click on the 'PascalVOC' icon-

Once you click on it, it will change to YOLO like in below screen-

Since we have already given the image folder path and classes.txt file while running the labellmg command, the first image from the image folder will be opened by default in labellmg window. Now click on the 'Create RectBox' icon and draw the annotation on the object you want to capture like below-

After drawing the bounding box, select the bharatnatyam option from the small window and click on ok then click on save button. That's it your first annotation file will be save in the location you have chosen while 'Change Save Dir' option. For next image just click on 'Next Image' icon and repeat the same till you get the last image. Once one folder is completed close this labellmg window, go to the anaconda prompt and run the command with change image directory path as below-
python labelImg.py /dataset/images/kathak classes.txt

It will again open a labellmg window with showing kathak's folder first image. Click on 'Change Save Dir' icon and select the /dataset/images/kathak and start annotating images like we did earlier. Repeat the same step for all target classes but don't forget to change the save dir path with every new class. Although this annotation step is time taking but requires your attention so do it carefully. I have done my annotations in 1.5 hours. Once done with all folders, got to each folder copy the .txt and .jpg files and paste in obj folder you have created in step 5. Do this for all folders. Now in obj folder you will have all image files and their respective txt files. Please make sure you vae not copied the classes.txt file in the obj folder. If yes then remove that file from obj folder.

Next, go to the directory \vcpkg-master\installed\x64-windows\tools\darknet\data and create a txt file named as train.txt. In this file we need to give the image name with file path. Here file path is
/vcpkg-master/installed/x64-windows/tools/darknet/data/obj so the whole file will be look like below-

You can write a Python script for this work. With this step we are ready to start the training of our dance form custom data.

Open windows powershell with admin rights and go to the root directory of the Darknet and run the command .\darknet.exe detector train data/obj.data cfg/yolo-obj.cfg yolov4.conv.137
If you have not downloaded the weights yolov4.conv.137, download from this link and put inside root directory of Darknet. Once you run the above command, Darknet will load the layers from this weights file and start the training on our custom data. It finds our data from the obj.data file where we have mentioned number of classes, training data path and location of saving the trained weights. This training will take quite enough time since we have given max_batches with 16000 and we have total 8 classes. In my machine it took 18 hours to complete the training so I advise you to disable the sleep property of your machine and start the training in night. After successfully completion of the training you will see following like screen-

This training command will also save loss vs iteration number chart in the root directory of the Darknet like below-

You can see in the starting of the training the average loss was very high but in the end of the training it is 0.6408. The final average loss can be from 0.05 (for a small model and easy dataset) to 3.0 (for a big model and a difficult dataset). So we have a very good model. Now we can test our model on unseen images. For this run the following command in the same powershell window-
.\darknet.exe detector test data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_final.weights /dataset/test/6.jpg -dont_show -out_filename

Here just change the test image path according to your system. Above command will predict the class on the given image with confidence in the powershell console as well as it will save the image with bounding box and class name in the root Darknet directory with name as 'predictions.jpg'-

And it's corresponding prediction image file is as below-

See our model is able to detect the dance form as Kathak for this unseen image which is correct. Let's check another unseen image using command .\darknet.exe detector test data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_final.weights /dataset/test/6.jpg -dont_show -out_filename

Looks like our model is working as expected. Please note for a very good YOLO model we should preferably have 2000 different images for each class or more, and we should train 2000*classes iterations or more. Due to my system configurations I have used image width and height as 416 but if you have got bigger avg loss then you should increase the network resolution to 608 or 832. Also if you don't want to detect any specific object then you should also include that object images with empty txt files in the obj folder.

That's it guys for today. If you have followed till the end, you have spent your valuable time in learning a state of the art computer vision technique. Here we are using the original Darknet framework, no any wrapper for our work. There are only very few persons who know using such a state of the art computer vision technique. Now you are one of them but to be a master on a technique you must practice and do experiments. So don't stop yourself just reading here. Do practical on your system, solve issues while doing hands on, train YOLOv4 on another custom dataset, deploy your custom model as Rest API and showcase in your github repository. In my next post I will share my new learning to you; till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How to install and compile YOLO v4 with GPU enable settings in Windows 10?

noreply@blogger.com (Learn Data Science with PrateekG) — Sat, 23 May 2020 22:28:00 +0530

Another post starts with you beautiful people!
Last year I had shared a post about installing and compiling Darknet YOLOv3 in your Windows machine and also how to detect an object using YOLOv3 with Keras. This year on April' 2020 the fourth generation of YOLO has arrived and since then I was curious to use this as soon as possible. Due to my project (built on YOLOv3 :)) work I could not find a chance to check this latest release. Today I got some relief and successfully able to install and compile YOLOv4 in my machine. In this post I am going to share a single shot way to do the same in your Windows 10 machine. If your machine does not have GPU then you can follow my previous post by just replacing YOLOv3 related files with YOLOv4 files. For GPU having Windows machine, follow my steps to avoid any issue while building the Darknet repository.

My machine has following configurations:
Windows 10 64 bit Intel Core i7
16 GB RAM
NVIDIA GeForce GTX 1660 Ti Version 445.87
CUDA 10.1, cuDNN 7.6
Visual Studio 2017

Before proceeding please make sure that you have installed CUDA, cuDNN as per your GPU drivers and Visual Studio 2017 is installed with C++ distributions. An invalid CUDA/cuDNN version will show unnecessary errors while installing. Once you check and assure all setup is up to date; disable your antivirus settings till the end of this process. For building the Darknet code I am here using Vcpkg instead of Darknet repo's build.ps1 file since with this build.ps1 file I was not able to build the code with GPU settings. Let's start the process-

1. Download Vcpkg from this link. It is a C++ library manager and will be used to install and compile the Darknet code.

2. Open Windows PowerShell with admin rights and go to the root directory of the Vcpkg folder you have downloaded from step 1 and run following two commands one by one:

.\bootstrap-vcpkg.bat
.\vcpkg integrate install

Please note if you are using Windows PowerShell first time, run Set-ExecutionPolicy RemoteSigned and press A when asked and then press Enter.

3. Next, run following command: .\vcpkg install darknet[full]:x64-windows

Above command will download the opencv with cuda, other required libraries and Darknet code. After downloading it will install the all libraries and then it will compile the Darknet code with GPU enable settings automatically. This process will take some time so have patience and wait to complete. Once this process is completed, you will see logs in your PowerShell and some green lines with done status. Make sure all green lines have done status. Once you see done status in all it means Darknet code is successfully compiled with GPU enable settings in your Windows machine. You can check it by going into the darknet directory: <Your_System_dir>\vcpkg-master\installed\x64-windows\tools\darknet

4. Download the yolov4.weights file from this link and put it into the darknet directory
5. Download a sample mp4 video file and put into the same darknet directory
6. In the same PowerShell window go to the path of your darknet directory and run the following command-
.\darknet.exe detector demo cfg/coco.data cfg/yolov4.cfg yolov4.weights Car_Racing.mp4 -dont_show -out_filename carRacing_result.mp4

Once you run the above command, in the starting of the running command you will see following like line-

You can see my CUDA, cuDNN and GPU count are showing in the console which confirms that my GPU is used while using the pre-trained YOLOv4 weights on a sample video file. Once this run is completed you can go to your darknet directory and found a output file carRacing_result.mp4 or your sample video output file. Open that file in a media player and you will see bounding boxes around the detected object names like person, car, truck on them. Inspite of fast moving objects in my video file, YOLOv4 is able to detect objects accurate. You can see my output video from this link . In my machine the average FPS with GPU was 6.7 for this video and with CPU it was 0.4. True power of using a real time object detector like YOLO is with GPU only :)

You can also test the detector with an image by running the same command but replacing video file with image file like below-
.\darknet.exe detector test cfg/coco.data cfg/yolov4.cfg yolov4.weights <IMAGE_FiLE> -dont_show -out_filename

Once you open the saved image (predictions.jpg), the image will show the detection result as below-

You can see pre-trained weights of YOLOv4 is able to detect person, cup, spoon, wine glass, fork, bowl, chair, backpack and knife in the above picture.

That's it for today. With this installation we have a real time object detector. Using it's pre-trained weights only we are able to detect many objects. I am using 2 custom YOLOv3 based models in my project and both are successfully running in production. Soon I will replace both models with the 4th generation YOLO. In my next post I will share you how I prepared my custom objects and trained them with YOLOv4. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

Building and deploying your ChatBot with Amazon Lex, AWS Lambda, Python and MongoDB

noreply@blogger.com (Learn Data Science with PrateekG) — Sat, 9 May 2020 21:49:00 +0530

Another post starts with you beautiful people!
Most of the businesses are adopting digital transformation to modernize customer communication and improve internal processes. By personalizing the user experience whether in a chatbot conversation, on a website or in email, you can make your user feel more valued and understood. Google DialogFlow and Amazon Lex are two pioneer vendors for building end to end personalized chatbot applications. In this post we are going to use Amazon Lex to build our chatbot and after the end of this post you will have your chatbot integrated with a web page and also your web page will be deployed on AWS cloud. This post is going to be long and very interesting so stay focus and keep reading the post till the end.

Step 1. Creating your account in AWS
To proceed with this post you must have an AWS account. If you don't have , just follow this link to create a free tier account there. While registration it may ask for your card details. No need to worry about any deduction. It's only for verification of your identity. Also we are going to use free tier services which is quite enough for our exercise so go ahead and make your account. As an extra security to avoid any charges after free tier expiration you must follow this link. After successful activation of your account you must login there by clicking on 'Sign In to the Console' button as showing here:

Once you login in your account, you need to update your region to 'US East (N. Virginia) us-east-1' as this region is required to deploy our chatbot using Amazon's CloudFormation. Your region should be look like as in below screen-

Step 2. Configuring Amazon Lex
Next, you need to open your Amazon Lex console. For this type 'Lex' or 'Amazon Lex' in the Find Services search box and select the Amazon Lex option as showing in below screen-

As soon you select the Amazon Lex option the page will be refreshed and you will see your Lex screen as below-

Click on the Get Started button and your lex console screen will be opened as below-

As you can see in above screen, Lex provides us some in built examples of personalized chatbots with Custom bot options. You can try any in built examples by selecting a Sample template. Before making our custom bot I recommend you to select OrderFlowers, check following options and click on create button -

Above selections are also applicable to Custom bot option. Based on your actual use case you can enable the Sentiment analysis option. If you want to enable both text and voice input to your bot the default IAM role is necessary. Once you click on the create button bot will open following look like console-

Since it is in built bot example, we have all set to test this bot functionality. you can test the bot by typing a question (I would like to order some flowers) mentioned in Sample utterances box in your test bot screen as below-

Go ahead follow the bot conversation flow and explore yourself with this example. Now I will move to the main goal of this bot- creating your custom bot and interact it with your MongoDB database to fetch some queries. This is an actual use case example where you have some product's or person's information stored in MongoDB database and to answer a question of a user you need to get from that db otherwise your bot is incomplete and could not able to fulfill the conversation.

Step 3. Setup A MongoDB Data
To move further you need some sample data in MongoDB. You can import any specific data for this exercise. If you are new to MongoDB don't worry I will tell you the steps. Any experienced person can skip this section and move to the step 4. To setup MongoDB, go to the Atlas Link and click on start free button. Please note MongoDB Atlas is a cloud based db setup so you don't need to do the db setup in your local machine. Complete the signup process and sign in your atlas account. We will use it's free tier (512 MB size) known as Shared Clusters to setup our db infrastructure as showing in below screen-

Click on Create a cluster button. In next screen select the aws as cloud provider & region, choose region as us-east-1 and click on create cluster button. After this step in next screen complete all options one by one as showing in below screen-

For example first create a database user there by clicking on 'Database Access' option and enter your username/password in below look like screen. Please must follow password guidelines link while creating your password otherwise you will face connection error later-

After creating your database user, next step is to add your ip to the Network Access option otherwise you will not be able to access this cloud cluster. Go to the Network Access and either add your ip to the list or select 'Allow Access From Anywhere' option as showing in below screen-

Once you whitelisted the ip, next step is to load some sample data to complete this exercise. Click on load sample option as showing in below screen-

Once the sample data is loaded last step is to connect to the cluster. We have three options to interact with our db. We will choose MongoDB Compass tool. Select the third option-

After selecting MongoDB Compass, in next screen choose your OS and click on Download Compass button. It will start the downloading of MongoDB Compass in your system-

Meanwhile copy the connection screen as showing in above screen and after downloading completed open the MongoDB Compass and paste there the connection string. Don't forget to update your password in this connection string. Once you entered the connection string there and connect you will see following look like screen with sample data-

You can choose any of the sample data. To know how to connect and fetch db query in MongoDB please follow this link before moving to the next step. I will use airbnb data for this exercise.

Step 4: Configuring Custom bot
Following the step 2, we will now create a custom bot. Select the Custom bot template and fill up the details of your bot as I have filled for my bot in below like screen-

In next screen, click on '+ Create Intent button' as showing in below screen-

Give a unique name to your intent and click on Add button in your next screen. Since in this exercise my goal is to show property details to the user. I have given my intent name as 'GetHouseDetails'. You will see following like screen after creating this intent-

Next, we will add some questions a user can ask in the Sample utterances section. Go ahead add some questions there as I have added like below-

Next, we will add a slot type to this intent. This slot type will appear once user a question you have just added in utterances. Click on + icon in Slot types and give it a name 'PropertyType'. Add some values in this slot type like- Villa, Apartment etc. and save the slot type to the intent. Once you save this slot type, it will be added automatically under the Slot tab as showing in below screen-

Next, fill up the Prompt screen with the bot response you want to provide and click on the Settings icon to add response card in this slot. Response card is a way in Lex to provide an option template to the user. You will see it once we complete this setup. After clicking on the Settings icon, a pop window will appear and there you need to fill the options as below-

Go ahead fill the options and save the settings.Similarly we will add 1 more slot type as 'BedroomCount' and will provide it's value as 1,2 and 3-

You can add slot types as much as you think it can help to make conversation better and complete. Now save the intent and click on build button.

Step 5: Creating AWS Lambda function in Python
Once we configure our bot, next step is to fetch the values from the database. In our case user will select the property type, number of bedrooms, cancellation policy and our bot will give him the total number of properties available with this criteria. This is just a sample use case for this exercise. You can make any query to db and fetch required details. Now to make lex and db interaction successful, we will create AWS lambda function in Python. This function requires some AWS specific format which you can read in this official link .

I generally create a python file in PyCharm and write the required code. Please note after creating the required Python file you need to create a folder in your local system and put that Python file there. Then you need to also import all required Python libraries in that folder and compress the content of that folder for uploading in AWS Lambda function console. For this example I have used pymongo library for db connection so I have installed this library in that folder. To do so once you have created an empty folder and pasted your Python file there, open anaconda prompt with that folder location run following command:
pip install --target <target_fodler_path> pymongo[tls,srv,gssapi]

Once above command run successfully, you can see some folders and files in your target folder. Now just select all files, compress them and a zip file will appear. This is the zip file you will upload in AWS Lambda console. Don't worry about writing the AWS lambda function from scratch. For your reference I am sharing my complete zip file which I have written by following best practices. You can download the complete zip file from this link .

Once you are ready with the zip file , in a new tab open Amazon Lambda console and click on Create function as showing in below screen-

In next screen fill up your function name, choose your Python version as 3.7 and click on Create function button-

Once your function is created, in next screen select option Upload a .zip file from the Code entry type dropdown as showing in below screen-

Click on Upload button and select your zip file and click on Save button. Once your zip file is uploaded there, next you need to change the Handler name. This handler name must starts with the same name as your Python file name. So change the name and click on save button. In my case my python file name is getpropertydetails.py so Handler name will be getpropertydetails.lambda_handler as showing in below screen

Step 6: Enabling AWS Lambda in Lex:
Once you have completed the step 5, go back to your Lex console and select the following two options with your lambda function name:

Lambda initialization and validation
Fulfillment

Click on Save Intent and then click on the Build button. Once build is successful, you can check your bot in Test bot window as below-

Quite interesting right! All hard work you have done till now, is paying you. Your bot is working as expected and you are now ready to integrate it with an actual web app.

Step 7: Integrate Lex bot with a web app and deploy in cloud:
You have created and tested your bot till step 6. Now you want to show it to your boss or client. You cannot show your test bot instead it should be shown either integrated with an app or a web page. Since I am not a mobile or app developer I will choose the web page integration. For this purpose AWS has it's own solution. With this in built solution not only you can integrate your bot with a working web page also you can deploy your bot in AWS cloud. So first publish your bot by clicking on Publish button in lex console, type an alias name and click on Publish button as showing in below screen-

Once your bot is published, you will see following screen-

If you are a mobile developer you can click on Download connection info button and if you want to integrate your bot with Facebook, Messenger, Slack etc click on Go to channels buttons otherwise close this window. For integrating with a web page, open AWS CloudFormation in a new tab and click on Create stack button-

Select the first option Template is ready. I am providing you my template which you can download from this link . Use the same template and click on Next-

In next screen give any name in Stack name field, enter your lex bot name in BotName field, enter bot's alias name in BotAlias field. Leave all fields as default and click on Next button. In next screen also click on Next button. Select both check boxes in next screen and click on Create stack button-

You will see following like screen-

Wait till it's status appears as CREATE_COMPLETED. Once you see this status go to the Outputs tab and you will see following look like screen-

That's it you have successfully deployed your chatbot web app in AWS cloud. Open the last link mentioned as WebAppUrl in a new tab and your chatbot deployed as web page will appear. Go ahead and ask some question to your property agent-

You have come so long! In this blog post not only you learned about Amazon Lex configuration but also some advanced tasks like creating AWS Lambda function, integration with MongoDB using Python and deploying your bot as a web page in AWS cloud. After completing this exercise not only you have completed an end to end real world project but also you have gained a rare skill sets. But don't stop here. Make your bot more mature by adding more slot types, db queries etc and help a customer to find out the best available Airbnb property for him/her. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How to achieve maximum parallel processing capabilities with XGBoost-1.0.0?

noreply@blogger.com (Learn Data Science with PrateekG) — Fri, 21 Feb 2020 16:36:00 +0530

Another post starts with you beautiful people!
XGBoost is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. Recently XGBoost is released with it's newer version 1.0.0 which has improvements like performance scaling for multi core CPUs, improved installation experience on Mac OSX, availability of distributed XGBoost on Kubernates etc. In this post we are going to explore it's multi processing capabilities on a real world ml problem Otto Group Product Classification Challenge. In the end of the post I will share my kaggle kernel link also so that you can explore my complete code.

Once you go to the challenge link in Kaggle and start your kernel, first you need to enable the Internet option in the notebook since current version of XGBoost installed in kernel notebook is 0.90. So for upgrading it run the following command in your anaconda prompt-
pip install --upgrade xgboost or !pip install --upgrade xgboost in your kaggle kernel. Screenshot of Anaconda prompt is as below-

And screen shot of my kernel is as below-

After running above command you will see following screen showing successful installation of the library-

Let's import the dataset-

This dataset describes the 95 details of 61,878 products grouped into 10 product categories (e.g. fashion, electronics, etc.). Input attributes are counts of different events of some kind. The goal is to make predictions for new products as an array of probabilities for each of the 10 categories and models are evaluated using multi-class logarithmic loss (also called cross entropy).

Please note that we get multi-threading support by default with XGBoost. But depending on our Python environment (e.g. Python 3) we may need to explicitly enable multi-threading support for XGBoost. We can confirm that XGBoost multi-threading support is working fine by building a
number of different XGBoost models, specifying the number of threads and timing how long it takes to build each model. The trend will both show us that multi-threading support is enabled and give us an indication of the effect it has when building models. Below is the code snippet showing how can you check this-

You can update the number of threads list based on your system configuration. Recommended way to set number of threads (nthread) is it should be equal to the number of physical CPU cores in your machine. After running the above code cell I am getting following result-

Also we can plot the above trend in following way-

From above plot we can see a nice trend in the decrease in execution time as the number of threads is increased. You can run the same code on a machine with a lot more cores and decrease the model training time. Now you know how to configure the number of threads with XGBoost in your machine. But there is one more important thing we can do as tuning. We always do cross validation to avoid overfiting in our model and this steps also time consuming. So is there any way to tune this process. Answer is absolutely yes! We can enable the multi-threading in both XGBoost as well as in cross validation.

The k-fold cross-validation also supports multi-threading. For example the n_jobs argument on the cross val score() function allows us to specify the number of parallel jobs to run. By default, this is set to 1, but can be set to -1 to use all of the CPU cores on our system. In following code snippet we will check three configurations with cross validation and XGBoost to achieve multi-threading and then compare the output of all three configurations-

In above code snippet you can see only configurable parameter required for multi-threading is nthread in XGBoost and is n_jobs in cross validation. For example only I am using 1 thread but you can change it to your no. of cores. After running above cell the best result is achieved by enabling multi-threading within XGBoost and not in cross-validation as you can see in below screen shot also-

So if you are going to to do cross validation with any library other than XGBoost 1.0.0, don't forget to enable the multi-treading feature in cross validation and if you are going to use XGBoost which you should be, don't forget to check no. of threads and enabling multi-threading within XGBoost. For the complete solution, you can find my kernel in following link: my kernel
Fork my kernel and start experimenting with it and if you like to learn more about XGBoost, follow the tutorials of this amazing guy: Jason Brownlee PhD.Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How can I become a TPU expert?

noreply@blogger.com (Learn Data Science with PrateekG) — Fri, 14 Feb 2020 18:00:00 +0530

Another post starts with you beautiful people!

I have two good news for all of you! First good news is that Tensorflow has released it's new version (TF 2.1) which is focused on TPUs and the most interesting thing about this release is that it now also supports Keras high level API. And second wonderful news is to help us get started Kaggle has launched a TPU Playground Challenge. This means there is no any way to stop you learning & using TPUs. In this post I am going to share you how to configure and use TPUs while solving a image classification problem.

What are TPUs?

You must have heard about TPU while using Google Colab. Now Kaggle also supports this hardware accelerator. TPUs or Tensor Processing Units are hardware accelerators specialized in deep learning tasks. They were created by Google and have been behind many cutting edge results in machine learning research. Kaggle Notebooks are configured with TPU v3-8s, which is a specialized hardware with 4 dual-core TPU chips for a total of 8 TPU cores. We can enable this accelerator in our Kaggle notebook by selecting 'TPU v3-8' option in Accelerator menu. We can use up to 30 hours per week of TPUs and up to 3 hours at a time in a single session.

How to enable TPU training in Tensorflow Keras?

Once we enable the TPU accelerator, next step is to enable the TPU training with Keras. You just need to first import the Tensorflow library and execute the following code to enable this-

Here TPUClusterResolver() locate the TPUs, TPUStrategy object contains the necessary distributed training code that will work on TPUs with their 8 compute cores. Finally, we use the TPUStrategy by instantiating our model in the scope of the strategy. This creates the model on the TPU. Model size is constrained by the TPU RAM only, not by the amount of memory available on the VM running our Python code. Model creation and model training use the usual Keras APIs.

TPUs and GCS?

TPUs read training data exclusively from Google Cloud Storage (GCS). Format of this training data in GCS is TFRecords. The TFRecord format is a container format frequently used in Tensorflow to group and shared data files for optimal training performance. Each file contains the id, label (the class of the sample, for training data) and img (the actual pixels in array form) information for many images. On Kaggle we can also use KaggleDatasets().get_gcs_path() to obtain the GCS path of a Kaggle dataset.

How to load a dataset from TFRecords file?

We can load a dataset from TFRecords files by using following code snippet:-

From multiple TFRecords we can also enable parallel streaming with below code snippet:

Above three code snippets are only required to use TPUs. Now I will show you how can we use TPUs while solving a 100 flower classification problem. I have tried different architectures with pretrained weights- VGG16, VGG19, ResNet, DenseNet etc and found best result (0.92313) with DenseNet201 with default learning rate. With a custom learning rate function I even reached a public score of 0.93205. In the end of the post I will share my Kaggle notebook link so that you can check it and also fork it to explore with any new model or settings. Let's start the steps I have followed to solve the problem-

1. Initiate and enable TPUs:

2. Load the Datasets: In this playground challenge, we're classifying 104 types of flowers based on their images drawn from five different public datasets and images are provided in TFRecord format. The dataset is already divided into train, validation and test sets. Each set contains 4 types of images having dimension of 192x192, 224x224, 331x331 and 512x512. We will load the datasets from GCS in following way:

Here I am using 224x224 dimension images, You can change it to any of the given dimension. After selecting the dataset we can use Tensorflow's io.gfile.glob() function to load the datasets. In above way you can load any TFRecord format files. Next step is to wrap your model initialization and compilation code in the TPU strategy scope. In my initial try I experimented with various pretrained weights of Keras API. I am here showing the best one.

3. Compile model with TPU: Model initialization, compiling and training step is same as you did while using keras API. Only different point here is to wrap our code inside TPU strategy scope. Below is code snippet of the same I followed:

4. Train the model: Training of the model is as same as we did with any of Keras API. We need to call .fit() function with required parameters. below is code snippet of the same:

Once my model training was completed. I calculated the confusion matrix and got below result:

When I submit my final submission file I got 0.93205 public score while the top score is 0.96652 which means there are many improvement still left. I am still trying. What about you? You don't need to type the above mentioned code manually. Just click on 'Copy and Edit' button in my notebook: my notebook url and start experimenting with new things till you get the maximum score. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How I 'Dockerized' my deep learning project?

noreply@blogger.com (Learn Data Science with PrateekG) — Sat, 28 Dec 2019 20:14:00 +0530

Another post starts with you beautiful people!
I hope you have enjoyed my last two posts about Yolo system and now you are well aware of using Yolo with kers api as well as Yolo with Darknet framework in your Windows machine. I know Linux is the most suitable environment for production ready configurations, but like me there are many Windows users who find sometime very difficult to setup an environment. In my previous post I have shared my Python setup requirements which you can use to setup Python computer vision environment in Windows. Continuing my knowledge sharing journey, today I am going to share you how can we setup and use Docker with our Python project.

What is Docker?
Generally in our Python project structure we use a requirements.txt file where we define Python library name as key and it's version as value. This file is used to install required Python libraries when we deploy our code to new environment ( for example from Windows to Linux environment). We just need to run 'pip install -r requirements.txt' command to install all dependencies in new environment and our application will be run in that new environment. But this process creates issue specially if you have installed Windows specific versions of different libraries and trying to use the same in Linux environment. In solving such dependency issues, we spent our valuable time. What if we can remove this issue? Is there any way of sharing and running our application in any environment without need of handling dependencies?

Yes, Docker is the solution of this dependency! Docker is a platform which uses the combination of the images and the containers concepts. Here we can think 'image' as a self contained package and 'container' as a running image on Docker platform. Currently Docker is a must know skill for building a cloud native microservices based applications. And future of all monolithic architecture based application is microservices. With Docker we can deliver our application with all required dependencies as an image. So as a Data Scientist we must also know atleast high level of this platform. Let's start todays learning-

How to install Docker?
We will use community version of the Docker since it is free. We can download the Docker Desktop from the official site-

Download the setup file. It will ask you to create an account so make it and then download the setup-

Before installing please make sure following two things-

Windows 10 64-bit: Pro, Enterprise, or Education (Build 15063 or later).
Hyper-V and Containers Windows features must be enabled.

Following is the way to do second option-

Open command prompt with admin rights and run the following command-

DISM /Online /Enable-Feature /All /FeatureName:Microsoft-Hyper-V

In my machine it looks like below once we run the above command-

When you type Y, system will be restarted. Now we are ready to install the Docker Desktop. Go to the downloaded file, run it as admin and follow the instruction. The setup process will look like below screens-

Click on Ok button, next screen will unpack the installation and then will ask you to logout of machine-

Once you click on Close and log out button an re login. You can see Docker desktop icon in your desktop and a small whale icon in Taskbar. If the setup has not asked you for re-login then you must restart your machine. Now when I click on the Desktop icon of Docker, it was showing me following access error-

To solve this issue copy the path till 'C:\ProgramData\docker' in address bar and press enter. A pop up will appear and ask for permanent access, select this button and you will be given the access. Now go back to the error showing console and click on Quit button. Again double click on the Desktop icon of Docker. This time a pop up will come and ask you for enabling the containers-

Click on Ok button and your system will be restarted. Once the system is restarted, you will see that Docker is running notification in taskbar notification. It means you have successfully install the Docker Desktop in your machine. Now whenever your system is restarted, Docker will be run automatically. But this default behaviour takes lots of RAM utilization, so it is recommended to disable this auto start feature and start the Docker when it is needed. We can disable this feature by following this step-
Right click on Docker Whale icon in Taskbar, Go to Settings, Click on General tab and uncheck the first option 'Start Docker Desktop when you log in' and restart the Docker-

Testing the installation
Once we completed the above steps, we can check if our installation is correct or not. To do so open command prompt and run following command- docker --version

There are various repositories with inbuilt Docker images are available for us to use. A simple example is 'Docker Hub' repository. We can use this and test our Hello World image by running following command- docker run hello-world

You can see in above console,a hello message is printed. It means hello-world image is downloaded from Docker Hub repository. Now we can print details of this downloaded image by running following command- docker image ls

Exploring an application
Next, we will see real power of the Docker by running a Ubuntu OS and a webserver- nginx. Sounds interesting, right! For this we need to switch our container from Windows to Linux. To do so, right click on Whale Docker icon in the Taskbar and select 'Switch to Linux containers.." option.

Click on Switch button. Close the existing command prompt console and open a new one and run the following command- docker run --interactive --tty ubuntu bash

See, the above root prompt. We are in the container. Let's check the hostname of the container by running typing 'hostname' and pressing enter-

'3b9f82bf5dea' is the id of the hostname. Exit the prompt by typing 'exit' and pressing enter. Now we will check containers available in this image by running following command- docker container ls --all

Now we will start the nginx webserver but to do so we must restart our Docker and open a new window of command prompt because if you do not do so and run directly the following command: docker run --detach --publish 80:80 --name mywebserver nginx , you will get following error-

docker: Error response from daemon: driver failed programming external connectivity on endpoint mywebserver

After restarting Docker, open a new console and run following command: docker run --detach --publish 8080:80 --name mywebserver nginx. Here mywebserver is my server name so you can use any name and also can give any port number instead of 8080.

Our server is up and running, we can find it's details by running following command-docker container ls

Open a browser window and type http://localhost:8080, press enter. You will see following screen-

You can stop your server by running following command:docker container stop <your_server_name>

Running a Python image
Let's run a Python image (for example python:latest) from Docker Hub repository. Open a command prompt and run following command:docker run -it --rm python:latest. This command will download the python:latest image from Docker Hub repository.

As you can see in console, command has downloaded latest version of Python- 3.8.1 on our Linux container. If you want to use a specific version then instead of python:latest use python:<version_number>. We have now our Python 3.8.1 running in Linux container.

Preparing my Deep Learning project specific Docker image
Now I will tell you how I have made my project as a Docker image. To do so we use concept of a Dockerfile. This file is simple text file having instructions and arguments. Docker reads this file and builds image automatically. According to official site syntax of a sample docker file looks like below format-

FROM ubuntu:18.04

COPY . /myapp

RUN make /myapp

CMD python /myapp/app.py

FROM creates a layer from the ubuntu:18.04 Docker image.
COPY adds files from your Docker client’s current directory.
RUN builds your application with make.
CMD specifies what command to run within the container.

Following the above syntax I have created a Dockerfile.txt for my project:

My project structure looks like below-

Before moving further please make sure you have created a requirements.txt file in your project structure. requirements.txt file for my project looks like below-

Now we will build our image. Go to the the path of your project and open it in a command prompt and run following command:docker build -t dsbyprateekg .
Here dsbyprateekg is my image name. One you run the above command, Docker will read your Dockerfile.txt and follow it's instruction.

Once the process is completed you will see following look like message in the end of the server console-

Now you can check if your newly created image exist or not by running following command:docker images

Here under the 'TAG' column you see 'latest'. We can change this value to a more specific name. For example I have changed 'latest' to 'prateek_gupta' by running following command:docker tag <IMAGE_ID> <IMAGE_NAME>:<TAG>

Next, we can test our container locally by running following command:docker run <IMAGE_NAME>:<TAG>

If all is well then you will see your Python file running successfully as in my case I have deployed my deep learning project as Flask Rest API so my flask server is up and running-

It means we have tested our Docker container locally and it can be run anywhere Docker is installed. In last step we can now push our repository to our account in Docker Hub. For this first we need to tag our image in require format: docker tag <IMAGE_NAME> <YOUR_DOCKERHUB_NAME>/<IMAGE_NAME> .
In my case my IMAGE_NAME is dsbyprateekg and YOUR_DOCKERHUB_NAME is my Docker account name. Then we need to login in Docker with our account by running following command:docker login --username <Your_Docker_Username>
It will ask you to enter your password, give the password and press enter. Now you can push your changes by running following command: docker push <Your_Docker_Username>/<IMAGE_NAME>

Once your Docker image is pushed, you can verify it also by going to your Docker account. Now you can easily share this to anyone. You just need to ask them to install Docker in their system, run following command:docker pull <IMAGE_NAME> to get the image and then just run the container. They don't need to do any code setup in their system.That's it for today. We have learnt about Docker, it's installation in our machine, switching from Windows to Linux container, successfully made a Docker image of a project and pushed our changes in Docker Hub. I have shared my project structure, Dockerfile screenshot so that you can get a reference from it and can easily apply it in your machine learning or deep learning based project. So try it yourself in your project and explore more. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How can I install and use Darknet framework in Windows?

noreply@blogger.com (Learn Data Science with PrateekG) — Thu, 5 Dec 2019 12:37:00 +0530

Another post starts with you beautiful people! I hope you have enjoyed my last post about using real time object detection system- Yolo with keras api. In that post I mentioned that Yolo is built on Darknet framework and this framework is written on C and cuda. That's why we used Python wrapper of Darknet framework instead of installing original framework. Many readers asked me about how can we install and use the original framework in our window machine. In this post I will try to show you the steps about this installation. Before following the steps I strongly recommend you to activate virtual env and install all libraries I have mentioned in my last post.

For this setup I have followed this original github repository- AlexeyAB/darknet . this repo is as same as original Darknet repo with additional Windows support. So don't forget to give a star to this repo as a token of our respect to the author. If you are reading my blog first time, then I recommend to read the Requirements section mentioned in this repo first.

Once your machine is ready with all requirements, next step is to install a compilation software- vcpkg which you can download from this link. Once you download the zip file, extract it, open command prompt with admin rights, navigate to the extracted vcpkg-master folder and run the following command-

Once the above command finishes, run the below command in same prompt-

Next, we need to add vcpkg root path in our environment variable. So open the system variable and add the location of the vcpkg-master folder under the name of VCPKG_ROOT like below-

Along with this new variable, add another variable VCPKG_DEFAULT_TRIPLET with following value-

Now we are ready for installing Darknet with this compilation software. Open Anaconda Poweshell Prompt with admin rights, navigate to the vcpkg-master folder by running following command:
cd $env:VCPKG_ROOT
After this run the following command:
.\vcpkg install pthreads opencv[ffmpeg]
Once you run above two commands, screen will look like below-

It will take some time to complete. Once this process is completed next step is to download the code from AlexeyAB/darknet. Once you downlaod the zip file from this repo, extract it and in the Anaconda Powershell Prompt navigate to the location of the darknet folder and run .\build.ps1 like below-

This command will install darknet repository once finishes. After this run following command:
Set-ExecutionPolicy -ExecutionPolicy Restricted and then when asked press A to exclude any restriction. Once this is done, you can navigate to the installed repository in your respective folder and see all installed files there. Now you need to copy the pretrained Yolo weights file in this location. You can download the weights from this link. After this we are ready to test the original YOLO system built on Darknet framework on our image.

To test any image you need to run following command in same Anaconda Powershell Prompt-
.\darknet.exe detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights -thresh 0.25

Once this command finishes , it will ask you to enter the path of your test image. enter the full path and press enter-

Once you enter the image path and press enter, your input image will be opened in default photo viewer having bounding boxes and label. Also in server console accuracy of the prediction will be shown with each detected object-

That's amazing right. In this post we have successfully installed Darknet framework in our Windows machine and in my last post we have successfully setup the Python wrapper of Darknet. Now you are familiar with both ways. If you are working with a client who has Windows infrastructure like mine then this post will help you to deliver state of the art object detection model built on Yolo system. I recommend you to give some time to learn this framework and get your hands dirty by practicing it on different image dataset many times. In my next post I will share another my learnings. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How to detect an object in real time using keras-yolo3?

noreply@blogger.com (Learn Data Science with PrateekG) — Mon, 25 Nov 2019 15:19:00 +0530

Another post starts with you beautiful people! In the past few months I was working on a complex object detection and recognition problem. My client is from a leading winery industry and they had an existing system built on vgg19 and keras-retinanet. This system was built to help client in their sales forecasting. The problem with current system was it was inaccurate- it did not detect most of the wine bottles and brands, it did not give the result in real time. You can imagine how this bad model can affect the business!

To solve the existing issues I tried a lot of things- changing hyperparameters, increasing the datasets, different keras applications but it was not giving me satisfactory result. May be I was not doing it right but I had put a lot of time and efforts with it. Then while doing R&D, I read this fantastic blog and I came to know about a state of the art and real time object detection system- YOLO. You Only Look Once or YOLO is a custom deep learning framework written on C and you can read more about it in it's official site.

Instead of writing the code from scratch I found two github repository as third party implementation of YOLO version 3-

You can follow any of the above link and run this code to see how it works. In this post I will share how I tested this in my system. For testing this model on unseen pictures you need to follow below steps-

A. Prepare your virtual environment- The first step before starting your object detection and recognition journey is to install all required libraries. I recommend to create a virtual environment and install all libraries there instead of installing in base location. This saves your base location in case of any corruption while installing. To create and activate the virtual environment, open anaconda prompt with admin rights and run following two commands one by one-

conda create --name myNewEnv python=3.7.3

activate myNewEnv

Here myNewEnv is the name of my virtual environment and 3.7.3 is my Python version. Replace it with your own name and version. Once you activate it, it's time to install required libraries. Before installing these libraries make sure you have Visual Studio 2017 with C++ extension is installed. If not then please install it and add C++ extension in it otherwise you will face unnecessary issues. Here I share list of mine which you can also use-

pip install keras==2.2.4

pip install tensorflow-gpu==1.14.0

pip install scikit-learn

conda install anaconda-client

conda install -c conda-forge/label/cf201901 opencv

pip install keras-retinanet

conda install shapely

conda install -c conda-forge imgaug

conda install -c conda-forge google-cloud-vision

conda install -c pjamesjoyce imutils

conda install -c anaconda flask

conda install -c conda-forge/label/cf201901 flask-restful

pip install tqdm

pip install boto3

pip install matplotlib

pip install seaborn

pip install xlrd

pip install pytesseract

pip install apscheduler

In above list you can see I have used variation of conda and pip. It is because while installing opencv and keras-retinanet in my windows machine I faced so much issues and I resolved those issues after so many efforts. So it is recommended to use above command as exact as I have mentioned. This is one time setup and most of the required image processing related libraries are mentioned in this list. Once all the installation is successful you can proceed with next step.

B. Download pre-trained weights- Second step is to download the pre-trained weights from This Link. It's a 235 mb file with name yolov3.weights.

C. Define keras model- Our next step is to define a keras model to match with the downloaded weights. It means our keras model should have right number of layers and right types of the layers to match with Yolo weights. This is the actual complex part but in the github repositories I have shared earlier, you can find the functions written already for this task. So just copy those but don't forget to give a Star to the original authors in github. Here is screen shot of the code snippet-

# Create block of layers

# Create the model

D. Load Model Weights- Our next step is to load the downloaded weights. But we cannot load and read that file directly in keras since downloaded weights are written in Darknet architecture and we are using keras architecture here. For this reading and parsing purpose we can use following class-

We can then easily use above functions and classes and then save result in keras format like below-

That's it. We have successfully completed the complex part. Now we can use this model like any other keras model we have used. Once you run the above code, you will see following like output in console-

Once the script is completed, in your current working directory 'model.h5' will be saved.

E. Test the model-

Like any other keras application, this model also requires input image in a defined shape. The YOLO system expects input shape of the image is in 416 x 416 pixels. You can use following code snippet to test the model-

Once you run the above code, it will display output in NumPy array format-

F. Decode the output- Currently from the output we cannot say anything because it is NumPy format, in order to understand this we need to decode it. Here decoding means in terms of bounding boxes around our object. In the github repository of experiencor there is a function called decode netout(), which takes each NumPy array from our output one by one and decode the bounding box and prediction-

Once we apply the above function it will return the bounding boxes. But these bounding boxes can be stretched back into the shape of original image. To fix this issue the experiencor script provides the correct_yolo_boxes() function-

Now we will get fixed bounding boxes but one issue with this is those bounding boxes may overlap. To fix this issue the experiencor script provides a do_nms() function that takes the list of bounding boxes and a threshold parameter-

Next, we need to assure to get only those bounding boxes which have strong presence of an object. For this we need to enumerate over all boxes and checking the class predictions. In this way we can then add class label. Following code snippet does the same-

You can test this function on a list. In our case list contains the name of various objects like below-

Our next step is to draw the bounding boxes around our detected object. That can be done using below function-

Once you summarize all above functions or run the script provided in github repo, you will see bounding boxes and name of the object detected in your image like I am getting in my input image-

For my image, the model with pre-trained weights is showing amazing results. It is correctly able to detect bottle in my image. Check yourself on different images with different objects and see how this amazing model works in real time. I have used the same repositories to do R&D for my work and after a lot of practice and trials I was able to successfully use this model with my custom dataset. For now try the above steps I have shown you, read the code many times, change it's configurable values and analyze the effect. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How to use Keras for Computer Vision- Part 3

noreply@blogger.com (Learn Data Science with PrateekG) — Wed, 17 Jul 2019 14:16:00 +0530

Another post starts with you beautiful people!
I hope you have enjoyed my last post about image data augmentation and now you are comfortable with increasing the size of your small training dataset. Today's post is going to be interesting because we are going to learn how to handle color images and examples of using convolution with keras. Let's starts our learning-

In our first post of this series we have seen that typically an image can be stored in 3-dimensional format- one is for height, one is for width and one is for channels. Colored images have three channels- red,blue,green components. A deep learning network requires that image data should be provided as a 3-D arrays. There are 2 ways to represent the image data as a 3-D array. First way is known as 'channel last' and second way is known as 'channel first'.

In 'channel last', last channel represents the color channels while in 'channel first', first channel represents the color channels. Please note some deep learning libraries prefer channel first ordering while some prefers last channel ordering. For example If we are using Tensorflow backend with keras then it prefers 'channels last' ordering. If you want to see the configuration file of keras with this setting you need to open keras.json file which resides in your home directory under .keras folder. In my system it is in following path: C:\Users\gupta\.keras and it's look like below-

You can modify above settings as per your need.

Let's see how can we use NumPy library to add a channel to a grayscale image?
Grayscale images are loaded as 2-D array so before modeling we need to add a channel dimension to that image. This means we are not adding any new data to the image, infact we are changing the array structure to have an extra axes with one dimension so that it can hold grayscale pixel values. NumPy has expand_dims() function for doing this task. Let's load a color image using Pillow library and see how can we use this function-

In above input cell you can see we have used 'axes' argument with expand_dims() function to define channel first and channel last techniques. Our loaded image has dimension of (375,500) which is a 2-D array. Once we added the channel first then it's dimension changed to a 3-D array (1,375,500) and when we added channel last then it became (375,500,1) dimension array.

Now we will move further and learn about Convolution in computer vision. A convolution is the simple application of a filter to an input that results in an activation. It is a common technique used in CV. Convolution neural networks do not learn a single filter; they learn multiple features in parallel for a given input. For color image filter must always have the same number of channels as the input, also known as depth. If an input image has 3 channels then a filter applied to that image must also have 3 channels. In this case, a 3 × 3 filter would be 3 × 3 × 3 or [3, 3, 3] for rows, columns, and depth.

Let's see a simple example of 1-D Convolution Layer: [0,0,0,1,1,0,0,0]. In this example we have eight elements.Now to use this input in keras we need to pass it as 3 dimension. We can represent this as [1,8,1] where first dimension represents the input sample, second dimension represents length of each sample and third dimension represents no of channels in each sample. So in keras we represent our 1-D convolution layer as below-

Next, we will define/create a model using keras' Sequential api as below-

In above input cell we have added our convolution 1-D layer with some required arguments. Here input_shape() argument represents input samples which is (8,1). kernel_size represents the shape of the filter which is 3 in our case and the filters argument represents the default filter of Conv1D layer which is 1 in our example. By default, the filters in a convolution layer are initialized with random weights. Let's apply this model to our input data using predict() function-

In above output you can see some negative values. This is because each filter in convolution layer has a bias input value which requires to manually specify a weight that will set to zero. We can achieve this by forcing the weights in our convolution layer to use a handcrafted filter. In our case it will be like [0,1,0]. So with this change our code will be like below-

Now you can see with our custom filter our model's prediction is correct!

In the same way you can use convolution 2-D layer. Here you need to use Conv2D() instead of Conv1D() layer. Also remember in this case input must be 4-dimensional where first dimension defines the samples, second dimension defines the rows, the third dimension defines the columns and the last dimension defines the channels.For in depth knowledge, I would recommend you to read following awesome blog-
https://machinelearningmastery.com/convolutional-layers-for-deep-learning-neural-networks/

That's it for today. We have learnt color channel ordering and basics of convolution in computer vision. Try above examples with 2-D layer and see what changes do you need. In my next post we will learn more about filter and kernel in convolution neural networks. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How to use Keras for Computer Vision- Part 2

noreply@blogger.com (Learn Data Science with PrateekG) — Sat, 13 Jul 2019 22:21:00 +0530

Another post starts with you beautiful people!
I hope you have enjoyed my last post and now you have become familiar with image loading, manipulation and saving in deep learning. If you have not read my last post then I recommend you to read the first part of this series. In this second part of my post we will learn about one of the most important technique used for a small image training dataset- Image Data Augmentation!

1. What is Image Data Augmentation?
Image Data Augmentation is a technique to expand the size of a training dataset. This technique can create modified versions of images which helps when we have a small dataset. Image shifting, flipping and zooming are some examples of this technique. Keras provides ImageDataGenerator class for this purpose.

2. How to use this technique?
We will learn different uses of this technique one by one by applying on an image-

A. How to shift an image?
Shifting an image means we are moving all pixels in one direction while keeping the image dimensions the same. ImageDataGenerator class has width_shift_range and height_shift_range arguements for shifting an image. Let's see how can we horizontal shift an image-

In above code cell I have loaded an image of ICC Cricket World Cup 2019 Trophy. I will horizontal shift this image using width_shift_range arguement. This arguement takes min and max range. In our example I am taking -150 and 150 as min and max pixels. After creating the iterator we are calling it nine times for augmentation. Once we run above cell we get following output-

In above output cell you can see our loaded image is augmented 9 times. The plot is showing randomly selected positive and negative horizontal shifts. You can try the same code but with height_shift_range arguement for vertical shifting. Remember in this case you need to pass the percentage of image to shift (for example height_shift_range=0.4) instead of [min,max] values.

B. How to flip an image?
Flipping an image means reversing the rows or column of pixels. The ImageDataGenerator class has horizontal_flip and vertical_flip arguements for flipping. Let's see how can we augment an image with horizontal flip-

The above code is quite same as we saw earliar except one change. Here we are passing horizontal_flip arguement which takes a boolean value- either True or False. After running the above cell we will get following plot-

Compare this plot with first one. What changes are you seeing?

C. How to rotate an image?
Image rotation is also part of augmentation technique.We can randomly rotate the image clockwise by just giving a number between 0 to 360 in rotation_range arguement. Let's see how can we do it-

Once we run above cell we see our loaded image is rotated as below-

D. How to augment an image by brightness?
We can augment an image by it's brightness using the brightness_range arguement. This arguement takes min and max range indicating percentage of darkness or brightness amount. Let's see how can we use this type of augmentation-

In above code cell I have given brightness_range arguement with 0.2 and 1.0 value. Here value less than 1.0 means darkening the image while larger than 1.0 means brighten the image whereas 1.0 means no affect on brightness. running above cell gives us following plot-

E. How to apply zoom augmentation?
Zoom augmentation randomly zooms an image. We can use zoom_range arguement which takes percentage of zoom as value. The zoom values less than 1.0 will zoom the image in and values larger than 1.0 will zoom the image out. Let's see how can we do that-

Running above code gives you following result having zoom in effect-

That's it for today. You can save the augmented images using the save_img() function as we did in our first post. In this way you can larger your small image training datatset. We have learnt 5 types of image augmentation. Apply each technique on your own image and experiment it with different different values of arguements. For more details of image data augmentation I recommend you to read following two awesome blogs-
1. https://www.pyimagesearch.com/2019/07/08/keras-imagedatagenerator-and-data-augmentation/
2. https://machinelearningmastery.com/how-to-configure-image-data-augmentation-when-training-deep-learning-neural-networks/
In next post of this series we will learn about the different colour channel ordering. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.

How to use Keras for Computer Vision- Part 1

noreply@blogger.com (Learn Data Science with PrateekG) — Tue, 9 Jul 2019 21:59:00 +0530

Another post starts with you beautiful people! Today we are going to enter into the Computer Vision (CV) field with this first post. With this and further coming posts, we will learn how to use Deep Learning for CV. We will cover following three key concepts with Python code in this post-
1. What is CV?
2. How to load and manipulate image using Keras?
3. How to scale your image using Keras?
Let's starts our learning-

1. What is CV?
Computer Vision or CV is a sub-field of Machine Learning and Artificial Intelligence, focused on the problem of helping computers to see. The goal of CV is to understand the content of digital images. CV has a wide variety of tasks-

Object Classification
Object Identification
Object Verification
Object Detection
Object Segmentation
Object Recognition

2. How to load and manipulate image using Keras?

A. Loading an image using Keras api is very easy. Keras provides load_img() function for loading an image. Before moving further make sure you have install keras. If not then run following command in Anaconda Prompt: conda install keras. The below Python code snippet demonstrates how to use this function-

In above output shell we can see my uploaded image format, it's mode and size. The image will not open in our notebook. It will be displayed using default application. In my case it was displayed by Photos app-

B. Manipulating an image is an integral part in deep learning and it starts with converting the image into numpy array format. Keras api provided img_to_array() and array_to_img() functions for this manipulation. Let's see how can we use both these two functions-

In above output cell, the uploaded image is converted into numpy array in float data type. The pixel values are converted into the format of (Height, Width, Channels). Finally the image is converted back to original PIL format.

C. Saving an image using Keras api is also easy. It provides save_img() function for this task. Following example will demonstrate how to load an image in grayscale format, converts it to numpy array and then save it to a new file name-

In above output, you can see our loaded image is converted into grayscale format and in numpy array. We have also saved and uploaded the new grayscale image.

3. How to scale your image using Keras?

Deep Learning models accept input images in scaled form. So we must scale our input images before training and evaluation of our deep learning model. Keras provides ImageDataGenerator class and API for just-in-time image scaling. To understand how to use this class and API, we will work on the same MNIST digit dataset which we have seen earliar in this post. For a quick revision the MNIST dataset, is an image classification problem comprised of 70,000 images of handwritten digits. The goal of the problem is to classify a given image of a handwritten digit as an integer from 0 to 9. So it is a multiclass image classification problem.

A. Load the dataset using keras

From above output we can see that there are 60000 images in training dataset and 10000 images in testing dataset. All images are 28 x 28 pixels. Pixel values are in between 0 to 255 and mean/std deviation of datasets are similar.

B. Scale the pixels using ImageDataGenerator

ImageDataGenerator class has three main scaling methods: Pixel Normalization, Pixel Centering, Pixel Standardization. Pixel Normalization technique scales pixel value to the range 0-1. Pixel Centering technique scales pixel values to have a zero mean. Pixel Standardization technique scales pixel values to have a zero mean and unit variance. Let's learn how to use each technique one by one-

1. Pixel Normalization: This can be achieved by setting the rescale argument to a ratio by which each pixel can be multiplied to achieve the desired range. In our case it would be 1.0/255.0.

In above output cell we can see that minimum and maximum pixels of training and testing datasets- 0 and 255. Since MNIST dataset has only black and white images, it has only one channel. If you are applying this to color images then minimum and maximum values will be calculated across all channels. In this example the ImageDataGenerator does not need to be fit on the training dataset as there is nothing that needs to be calculated, we have provided the scale factor directly.

2. Pixel Centering: In this technique we calculate the mean pixel value across the entire training dataset, then subtract it from each image.

In above output cell you are seeing the mean pixel values of training and test datasets. The ImageDataGenerator is fit on the training dataset and we have confirmed that this mean pixel value matches with our calculated mean pixel value in next line. You can see mean pixel value for the scaled dataset is close to zero.

3. Pixel Standardization: Standardization of images is achieved by subtracting the mean pixel value and dividing the result by the standard deviation of the pixel values. The mean and standard deviation statistics can be calculated on the training dataset.

In above output cell, you can see in the first line- the mean and standard deviation of pixel values in both datasets. The data generator is then configured for feature-wise standardization and the statistics are calculated on the training dataset, which is matching what we would expect when the statistics are calculated manually. The test is then repeated on the entire training dataset and we can confirm that the mean is very small value close to 0.0 and the standard deviation is a value very close to 1.0.

That is enough learning for today. After reading this post I recommend you to run above code examples in your notebook and understand each line. After this you can repeat the same techniques with any color image dataset. Working on a color dataset will teach you how to modify data scaling techniques. If you follow what I suggest then believe me you will become computer vision Ninja at the end of coming posts. In the next post we will learn the standard way to lay out our image data and image augmentation for modeling. Till then Go chase your dreams, have an awesome day, make every second count and see you later in my next post.