Siva R Vaka

Tensor Flow Docker Image

2017-10-30T20:57:00.002-07:00

TensorFlow Docker Installation & Setup

1. Get the Tensor Flow Docker image

:/> docker pull tensorflow/tensorflow

2. Start the instance

:/> docker run -it -p 8888:8888 --rm tensorflow/tensorflow

3. Once you access the url with port and token as shown in the above command

4. Login to the tensor container

NOTE: "tensor" below is name of the conatiner otherwise use the container id.

REF: https://www.tensorflow.org/versions/r0.12/get_started/os_setup#docker_installation

Removing Docker images and containers

2017-10-30T20:45:00.004-07:00

1. List & Remove images

docker images -a

docker rmi image-name1 image-name2

2. List & removeDangaling Images

docker images -f dangling=true

docker rmi $(docker images -f dangling=true -q)

3. Remove all images

docker rmi $(docker images -a -q)

4. List & Remove images by pattern

docker ps -a | grep "pattern"

docker images | grep "pattern" | awk '{print $1}' | xargs docker rm

5. List & Remove containers

docker ps -a

docker rm container1_id container2_id

docker container rm contianer-name

6. Remove container upon exit

docker run --rm image_name

7. List & Remove all exited containers

docker ps -a -f status=exited

docker rm $(docker ps -a -f status=exited -q)

8. Listing & Removing using Multiple filters

docker ps -a -f status=exited -f status=created

docker rm $(docker ps -a -f status=exited -f status=created -q)

9. List and Remove all containers by pattern

docker ps -a | grep "pattern”

docker ps -a | grep "pattern" | awk '{print $3}' | xargs docker rmi

10. Stop and remove all containers

docker ps -a

docker stop $(docker ps -a -q)

docker rm $(docker ps -a -q)

docker stop container-name

11. List & Removing Volumes

docker volume ls

docker volume rm volume_name volume_name

12. List and remove all dangling volumes

docker volume ls -f dangling=true

docker volume rm $(docker volume ls -f dangling=true -q)

13. Remove Volume and its container

docker rm -v container_name

Docker Swarm and Creating Cluster on Docker

2017-10-30T20:45:00.003-07:00

Initializing and Joining Swarm

Sudo docker swarm init

NOTE: if you have multiple network interfaces on the host (or on guest VM) , then you need to specify the "--advertise-addr" with specific ip

Worked fine after specifying the ip

From the second VM , run the following command to join as worker node in swarm

To list all swarm nodes connected to manager

NOTE: Only swarm managers execute Docker commands; workers are just for capacity.

For deploying applications (service)

To Change no of Replicas (nodes)

a. Simple modify the docker compose file

b. Re run the stack deploy command as shown

:/> docker stack deploy -c docker-compose.yml getstartedlab

To list the service which are running

:/> docker service ls

To list the nodes where the service is running

:/>docker service ps getstartedlab

To remove (uninstall) the service

:/> docker service rm getstartedlab

Leaving Swarm

Sudo docker swarm leave

NOTE: you may need to use the "--force" to leave the last manager from the swarm.

Docker Volumes

2017-10-30T20:45:00.002-07:00

Docker offers three different ways to mount data into a container from the Docker host: volumes, bind mounts, or tmpfs volumes. When in doubt, volumes are almost always the right choice

· Volumes are stored in a part of the host filesystem which is managed by Docker(/var/lib/docker/volumes/ on Linux). Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker.

· Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time.

· tmpfs mounts are stored in the host system’s memory only, and are never written to the host system’s filesystem

There are three main use cases for Docker data volumes:

1. To keep data around when a container is removed

2. To share data between the host filesystem and the Docker container

3. To share data with other Docker containers

Volume Commands

Create a volume:

$ docker volume create my-vol

List volumes:

$ docker volume ls

Inspect a volume:

$ docker volume inspect my-vol

Remove a volume:

$ docker volume rm my-vol

start the container using volume

$ docker run -d -it --name=nginxtest -v nginx-vol:/usr/share/nginx/html nginx:latest

Clean container with volume

$ docker container stop nginxtest

$ docker container rm nginxtest

$ docker volume rm nginx-vol

For creating readonly containers

$ docker run -d -it --name=nginxtest -v nginx-vol:/usr/share/nginx/html:ro nginx:latest

Binding Mounts (Sharing Data Between the Host and the Docker Container)

$ docker run -dit --name devtest -v "$(pwd)"/target:/app nginx:latest

$ docker run -d -v ~/nginxlogs:/var/log/nginx -p 5000:80 -i nginx:latest

-v ~/nginxlogs:/var/log/nginx — This will set up a volume that links the /var/log/nginx directory from inside the Nginx container to the ~/nginxlogs directory on the host machine. Docker uses a : to split the host's path from the container path, and the host path always comes first.

Installing Docker CE on Ubuntu

2017-10-30T20:45:00.001-07:00

Before you install Docker CE (first time) on a new host machine, you need to set up the Docker repository. Afterward, you can install and update Docker from the repository.

1. Update the apt package index:

$ sudo apt-get update

2. Install packages to allow apt to use a repository over HTTPS:

$ sudo apt-get install \

apt-transport-https \

ca-certificates \

curl \

software-properties-common

3. Add Docker’s official GPG key:

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

4. Verify that you now have the key with the fingerprint 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88, by searching for the last 8 characters of the fingerprint.

$ sudo apt-key fingerprint 0EBFCD88

pub 4096R/0EBFCD88 2017-02-22

Key fingerprint = 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88

uid Docker Release (CE deb) <docker@docker.com>

sub 4096R/F273FCD8 2017-02-22

5. Use the following command to set up the stable repository. You always need the stable repository, even if you want to install builds from the edge or testing repositories as well. To add the edge or testing repository, add the word edge or testing (or both) after the word stable in the commands below.

Note: The lsb_release -cs sub-command below returns the name of your Ubuntu distribution, such as xenial. Sometimes, in a distribution like Linux Mint, you might have to change $(lsb_release -cs) to your parent Ubuntu distribution. For example, if you are using Linux Mint Rafaela, you could use trusty.sudo

amd64:

$ sudo add-apt-repository \

"deb [arch=amd64] https://download.docker.com/linux/ubuntu \

$(lsb_release -cs) \

stable"

NOTE: It will sources url to "sources.list" under /etc/apt repository.

INSTALL DOCKER CE

1. Update the apt package index.

$ sudo apt-get update

2. Install the latest version of Docker CE, or go to the next step to install a specific version. Any existing installation of Docker is replaced.

$ sudo apt-get install docker-ce

or to install specific version use below command

$ sudo apt-get install docker-ce=<VERSION>

The Docker daemon starts automatically.

3. Verify that Docker CE is installed correctly by running the hello-world image.

$ sudo docker run hello-world

This command downloads a test image and runs it in a container. When the container runs, it prints an informational message and exits.

Ref:

https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-from-a-package

Virtual Machines Vs Docker Container

2017-10-30T20:45:00.000-07:00

What is docker?

A Docker container can be described as a wrap around a piece of software that contains everything needed in order to run the software. This is done in order to make sure that the app will run the same no matter what environment it runs in.

VirtualMachines Vs Docker Containers

VirtualBox and VMWare are virtualization apps that create virtual machines that are isolated at the hardware level.

Docker is a containerization app that isolates apps at software level.

irtual Machines	Docker Containers
Hardware level process isolation	OS level process isolation
VMs offer complete isolation of applications from host OS	Docker containers can share some resources with host OS
Each VM has separate OS	Each docker container can share OS resources
Boots in minutes	Boots in seconds
More resource usage	Less resource usage
Pre-configured VMs are hard to find and manage	Pre-built docker containers for home server apps already available
Customizing pre-configured VMs requires work	Building a custom setup with containers is easy
VMs are typically bigger in size as they contain whole OS underneath	Docker containers are small in size with only docker engine over the host OS
VMs can be easily moved to a new host OS	Containers are destroyed and recreated rather than moving (data volume is backed up)
Creating VMs take relatively long time	Dockers can be created in seconds
Virtualized Apps are harder to find and it takes more time to install and run them	Containerized apps such as SickBeard, Sonarr, CouchPotato etc. can be found and installed easily within minutes

Docker vs Linux LXC

2017-10-30T20:44:00.000-07:00

Linux cgroups, originally developed by Google, govern the isolation and usage of system resources, such as CPU and memory, for a group of processes.

Linux namespaces, originally developed by IBM, wrap a set of system resources and present them to a process to make it look like they are dedicated to that process.

The original Linux container technology is Linux Containers, commonly known as LXC. LXC is a Linux operating system level virtualization method for running multiple isolated Linux systems on a single host. Namespaces and cgroups make LXC possible.

Single vs. multiprocess. Docker restricts containers to run as a single process. If your application environment consists of X concurrent processes, Docker wants you to run X containers, each with a distinct process. By contrast, LXC containers have a conventional init process and can run multiple processes.

Stateless vs. stateful. Docker containers are designed to be stateless, more so than LXC. First, Docker does not support persistent storage. Docker gets around this by allowing you to mount host storage as a “Docker volume” from your containers. Because the volumes are mounted, they are not really part of the container environment.

Second, Docker containers consist of read-only layers. This means that, once the container image has been created, it does not change. During runtime, if the process in a container makes changes to its internal state, a “diff” is made between the internal state and the image from which the container was created. If you run the docker commit command, the diff becomes part of a new image—not the original image, but a new image, from which you can create new containers. Otherwise, if you delete the container, the diff disappears.

Docker Terminology - Basics

2017-10-30T20:43:00.001-07:00

An image is a lightweight, stand-alone, executable package that includes everything needed to run a piece of software, including the code, a runtime, libraries, environment variables, and config files.

A container is a runtime instance of an image—what the image becomes in memory when actually executed. It runs completely isolated from the host environment by default, only accessing host files and ports if configured to do so.

Containers run apps natively on the host machine’s kernel.

Docker daemon - The background service running on the host that manages building, running and distributing Docker containers.

Docker client - The command line tool that allows the user to interact with the Docker daemon.

Docker Store - A registry of Docker images, where you can find trusted and enterprise ready containers, plugins, and Docker editions. You'll be using this later in this tutorial.

An important distinction with regard to images is between base images and child images.

· Base images are images that have no parent images, usually images with an OS like ubuntu, alpine or debian.

· Child images are images that build on base images and add additional functionality.

Another key concept is the idea of official images and user images. (Both of which can be base images or child images.)

· Official images are Docker sanctioned images. Docker, Inc. sponsors a dedicated team that is responsible for reviewing and publishing all Official Repositories content. This team works in collaboration with upstream software maintainers, security experts, and the broader Docker community. These are not prefixed by an organization or user name. In the list of images above, the python, node, alpine and nginx images are official (base) images. To find out more about them, check out the Official Images Documentation.

· User images are images created and shared by users like you. They build on base images and add additional functionality. Typically these are formatted as user/image-name. The user value in the image name is your Docker Store user or organization name.

A registry is a collection of repositories, and a repository is a collection of images

Getting Started on AI (Artificial Intelligence) , ML (Machine Learning) & Data Science

2017-10-30T20:33:00.001-07:00

Angular 2 Proxy Configuration (CORS Headers Issue for local testing)

2017-01-01T12:31:00.002-08:00

As part of integrating the WebSphere Commerce REST services in Angular2 application, created bunch of WCS MOCK REST Services for unit testing using SOAP UI. It helps to test multiple scenarios (different data and fail scenarios ...etc).

You can use SOAPUI or other tools to mock the REST services during the development.

1. Create and Start mock services (or you may backend services running on different port)

While creating the MOCK Service, I have given following port and host address.

Start the mock services by clicking “run” button, then you can access mock services on http://localhost:4646

Problem : You will see the problem similar to below when you try to access these MOCK services from the Angular2 App because of cross domain (same origin) policy issues, We can mitigate this by adding Cross-Origin-Resource-Sharing (CORS) headers in server level , if you backend (api server) supports.

Error:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost:4646/wcs/resources/store/14321/cart/shipping_modes. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

Solution: As I am not using the any backend API server running in my local for adding CORS headers, thought to use the angular2 proxy configuration to bypass this errors.

1. Create proxy.config.json file under project root directory

2. Add following in proxy configuration file

{

"/localapi/*": {

"target": "http://localhost:4646",

"pathRewrite": {

"^/localapi": ""

"changeOrigin": true,

"secure": false,

"logLevel": "debug"

}

3. Then start the local angular dev server as shown below

a. Using NPM START

Add “—proxy-config proxy.config.json” in package.json as shown below.

b. Or Using NG SERVE

:/> ng serve –proxy-config proxy.config.json

4. Use the “/localapi” context while making HTTP call from the angular 2

http.get("/localapi/wcs/resources/store/14321/cart/shipping_modes", options)

5. Now when you access the application then browser makes call on 4200 port and get response from SOAP UI mock service that is running on 4646.

Debugging Angular 2 application in Visual Studio Code (Angular-Cli workspace)

2016-12-30T20:37:00.001-08:00

1. Go to debug view in the VSCode , and click on “gear” icon and select the “chrome”

2. It will create “.vscode” folder and “launch.json” configuration file under it

3. Start the Chrome in debug mode

Add “--remote-debugging-port=9222” while opening the chrome application (you can rename shortcut name also in “general” tab like “Google Chrome –Debug” then it will easy to find in start menu programs)

4. Click “google chrome debug“ shortcut above, it will open chrome in debug mode. You can validate if the chrome is running in the debug mode or not by accessing the http://localhost:9222/

(Note: close all chrome instances before opening debug mode)

5. Put breakpoint in the typescript file and launch the debug from debug view

6. Above default configuration works fine if you create angular 2 project without using the “Angular-CLI” or “webpack build”. (Just need to make sure you have proper port numbers)

i.e

9222 àis the Chrome debug port

8080 àlocal server port number (where your angular app runs)

7. But If you create the project using the Angular-Cli (which uses “WebPack” for application build), then control may not go to the our debug point at all .

Instead our debug point looks disabled with warning like “Breakpoint ignored because generated code not found (source map problem?). “ (Even though “SourceMap” configuration is set to “true” I was getting this problem.)

Problem

8. To avoid this problem Add following to launch.json

9. Following is full launch.json which worked for me (If you project is different path make sure to have proper path i.e. if you created project in D drive then “webpack:///D:*”).

{

"version": "0.2.0",

"configurations": [

{

"name": "Launch Chrome against localhost, with sourcemaps",

"type": "chrome",

"request": "launch",

"url": "http://localhost:4200",

"sourceMaps": true,

"sourceMapPathOverrides": {

"webpack:///C:*": "C:/*"

"webRoot": "${workspaceRoot}"

{

"name": "Attach to Chrome, with sourcemaps",

"type": "chrome",

"request": "attach",

"port": 9222,

"sourceMaps": true,

"sourceMapPathOverrides": {

"webpack:///C:*": "C:/*"

"webRoot": "${workspaceRoot}"

}

]

}

10. Now launch the debug configuration, control will go to our debug point

WebSphere Commerce Developer Enterprise(WCDE) v7- FEP6 to FEP8 upgrade - Part1

2016-02-21T22:56:00.005-08:00

We need to execute the following steps to upgrade WebSphere Commerce Developer Enterprise(WCDE) v7- FEP6 to FEP8.

Step 1: Update WAS Fixpack (min required is 7.0.0.29)

a. Install UpdateInstaller for WAS (latest WAS UpdateInstaller 7.0.0.39).

b. Install WAS Fix using UpdateInstaller (min required is 7.0.0.29 for FEP8)

Step 2: Update WCDE with Feature Pack.

a. Install the Commerce Server Fix pack (min required is 7.0.0.9)

b. Update the Feature Pack 8

Step 3: Installation Verification.

a. Using versionInfo.bat/sh

b. CheckEnablementStatus.bat/sh

c. Using DB Query

Step 4: Issues during installation & Fixes.

a. Missing MadisionMobile.sar and causes installation failures

References

1. Migrating WebSphere Commerce Developer

http://www-01.ibm.com/support/knowledgecenter/SSZLC2_7.0.0/com.ibm.commerce.install.doc/tasks/tmgwcdev.htm

2. WAS update installer

http://www-01.ibm.com/support/docview.wss?uid=swg24020212

3. WAS Fix pack

http://www-01.ibm.com/support/docview.wss?uid=swg24034966

4. FEP 8 Announcement

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=an&subtype=ca&appname=gpateam&supplier=897&letternum=ENUS214-379

WebSphere Commerce Developer Enterprise(WCDE) v7- FEP6 to FEP8 upgrade - Part2 (WAS Fixpack update)

2016-02-21T22:54:00.002-08:00

Download and install the WAS UpdateInstaller software

1. Downlowad the update installer from below url.

http://www-01.ibm.com/support/docview.wss?uid=swg24020212

NOTE: Have downloaded “7.0.0.39-WS-UPDI-WinIA32.zip” which is latest update installer.

2. Extract and click on the “install.exe”

3. It will ask location to install the updateinstaller, give path like

C:\Program Files (x86)\IBM\SDP\runtimes\base_v7\updi

i.e.

/WAS Installed Location/updateInstaller

NOTE: Tried to give installation directory name as “C:\Program Files (x86)\IBM\SDP\runtimes\ base_v7\UpdateInstaller”, but as path is too long I had to give

directory name as “updi”

Download and install the WebSphere Application Server Fix Pack

1. Download WAS fixpack version 7.0.0.29 (min WAS Fix required for WCS7 FEP8)

http://www-01.ibm.com/support/docview.wss?uid=swg24034966

2. Place the 7.0.0-WS-WAS-WinX32-FP0000029.pak under following directory

C:\Program Files (x86)\IBM\SDP\runtimes\base_v7\updi\maintenance

NOTE: “C:\Program Files (x86)\IBM\SDP\runtimes\base_v7” is my local WAS server location.

3. Click on the “update.exe” from the “C:\Program Files (x86)\IBM\SDP\runtimes\base_v7\updi” directory

4. Select the WAS production location

5. Select “install maintenance” option

6. It will ask for the fix pack location point to “C:\Program Files (x86)\IBM\SDP\runtimes\base_v7\updi\maintenance”

7. Click Next (by default it selects fixpack location)

8. Confirm and click next and “finish” after installation.

WebSphere Commerce Developer Enterprise(WCDE) v7- FEP6 to FEP8 upgrade - Part3

2016-02-21T22:54:00.000-08:00

Used installation manager to update following

a. Commerce Developer with Fixpack ( 7.0.0.9)

b. Commerce Developer with Feature Pack (FEP 8)

Add following repositories to Installation Manager

1. Download “7.0.0-WS-WCDeveloperEnterprise-FP009.zip” file and extract zip.

2. Download “WC_V7.0_-_FEP8_FOR_MP_ML” and extract.

3. Open Installation manager (Run as Admin)

4. Add following repository to Installation Manger (PreferencesàManage Repositories).

WC_V7.0_-_FEP8_FOR_MP_ML\developer\repository.config

7.0.0-WS-WCDeveloperEnterprise-FP009\repository.config

5. Select “update” in Installation Manager and select WebSphere Commerce and click next.

WebSphere Commerce Developer Enterprise(WCDE) v7- FEP6 to FEP8 upgrade - Part4 (Check WCS VersionInfo)

2016-02-21T22:53:00.001-08:00

Once the FEP 8 feature update is done , We can validate the version using any of following ways.

a. Using versionInfo.bat

/WCDE-install-direcotry/bin/versionInfo.bat

b. CheckEnablementStatus.bat

Will generate the enableFeature.log with all details.

CheckEnablementStatus:

[echo] component list file = W:\IBM\WCDE_E~1\properties\version\FEP8.componentlist

[forEachTask] featureName: foundation

CheckPrereqInDB:

[CheckFeaturePrerequisite] Feature foundation is enabled in EAR.

[CheckFeaturePrerequisite] Feature foundation is enabled in DB.

CheckFeature: