For Every System Admin

Vagrant Commands and Working with "Vagrantfile" !!!

2017-03-05T17:08:00.000-08:00

Let us see working with different commands of Vagrant. In this post, I wanted to explain about, how we can play with "Vagrantfile" to build an development environment.

For example, if we need 25 similar Ubuntu machines for our development team, we can simply do some modifications related to vm.config in our "Vagrantfile" and simply by giving one "vagrant up" command we can spin 25 machines within minutes only.

Other than regular vagrant halt/destroy/suspend/resume we are going to see about the VM provisioning which we can do through "Vagrantfile". We can even integrate our configuration management tools like Chef and Puppet with Vagrant to automate few things.

Let us start with some basic commands...

USER@USER-PC MINGW64 /f $ cd Workspace_local/vagrant/

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

default running (virtualbox)

The VM is running. To stop this VM, you can run `vagrant halt` to shut it down forcefully, or you can run `vagrant suspend` to simply suspend the virtual machine. In either case, to restart it again, simply run `vagrant up`.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

To view list of vagrant boxes which are downloaded and readily available....

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant box list

ubuntu/trusty64 (virtualbox, 20170222.0.1)

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

To halt our running vagrant....

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant halt default

==> default: Attempting graceful shutdown of VM...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

default poweroff (virtualbox)

The VM is powered off. To restart the VM, simply run `vagrant up`

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

To restart we can use the command reload or up,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant reload

==> default: Checking if box 'ubuntu/trusty64' is up to date...

==> default: Clearing any previously set forwarded ports...

==> default: Clearing any previously set network interfaces...

==> default: Preparing network interfaces based on configuration...

default: Adapter 1: nat

==> default: Forwarding ports...

default: 22 (guest) => 2222 (host) (adapter 1)

==> default: Booting VM...

==> default: Waiting for machine to boot. This may take a few minutes...

default: SSH address: 127.0.0.1:2222

default: SSH username: vagrant

default: SSH auth method: private key

==> default: Machine booted and ready!

==> default: Checking for guest additions in VM...

default: The guest additions on this VM do not match the installed version of

default: VirtualBox! In most cases this is fine, but in rare cases it can

default: prevent things such as shared folders from working properly. If you see

default: shared folder errors, please make sure the guest additions within the

default: virtual machine match the version of VirtualBox you have installed on

default: your host and reload your VM.

default:

default: Guest Additions Version: 4.3.36

default: VirtualBox Version: 5.0

==> default: Mounting shared folders...

default: /vagrant => F:/Workspace_local/vagrant

==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`

==> default: flag to force provisioning. Provisioners marked to run always will still run.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Let us check the current status,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

default running (virtualbox)

The VM is running. To stop this VM, you can run `vagrant halt` to shut it down forcefully, or you can run `vagrant suspend` to simply suspend the virtual machine. In either case, to restart it again, simply run `vagrant up`.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Previously what we did is Shutdown, we have a option to suspend our Vagrant instance...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant suspend

==> default: Saving VM state and suspending execution...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

default saved (virtualbox)

To resume this VM, simply run `vagrant up`.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

We can resume our vagrant instance from its current state,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant resume

==> default: Resuming suspended VM...

==> default: Booting VM...

==> default: Waiting for machine to boot. This may take a few minutes...

default: SSH address: 127.0.0.1:2222

default: SSH username: vagrant

default: SSH auth method: private key

==> default: Machine booted and ready!

==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`

==> default: flag to force provisioning. Provisioners marked to run always will still run.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Lets check the status again...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

default running (virtualbox)

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

We can completely destroy our instance, but before destroy we need to halt...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant halt

==> default: Attempting graceful shutdown of VM...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

default poweroff (virtualbox)

The VM is powered off. To restart the VM, simply run `vagrant up`

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Now we can simply destroy,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant destroy

default: Are you sure you want to destroy the 'default' VM? [y/N] y

==> default: Destroying VM and associated drives...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

default not created (virtualbox)

The environment has not yet been created. Run `vagrant up` to create the environment. If a machine is not created, only the default provider will be shown. So if a provider is not listed, then the machine is not created for that environment.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

If we delete our existing Vagrantfile, then...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

A Vagrant environment or target machine is required to run this command. Run `vagrant init` to create a new Vagrant environment. Or, get an ID of a target machine from `vagrant global-status` to run this command on. A final option is to change to a directory with a Vagrantfile and to try again.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Let us see how we can run a script at the time of "vagrant up" by including it in the "Vagrantfile".

Shell Script:

Below I am going to make some changes in default Vagrantfile and include a script which will run and install "Apache" during the start up of instance...

USER@USER-PC MINGW64 / $ cd /f/Workspace_local/vagrant

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vi Vagrantfile

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ cat Vagrantfile | grep vag1

config.vm.define "vag1" do |vag1|

vag1.vm.box = "ubuntu/trusty64"

vag1.vm.network "private_network",type:"dhcp"

vag1.vm.synced_folder "../vagrant", "/home/vagrant"

vag1.vm.box_url = "https://atlas.hashicorp.com/ubuntu/boxes/trusty64"

vag1.vm.network "forwarded_port", guest: 80, host: 8080

vag1.vm.provision "shell", path: "apache_script.sh"

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

If we check the status, now we can see...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

vag1 not created (virtualbox)

The environment has not yet been created. Run `vagrant up` to create the environment. If a machine is not created, only the default provider will be shown. So if a provider is not listed, then the machine is not created for that environment.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

As you see, I included a script "apache_script.sh" in my Vagrantfile. I am going to include few lines, so that my script is responsible for Apache installation in the Vagrant instance...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vi apache_script.sh

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ cat apache_script.sh

date

apt-get update

apt-get install -y apache2

echo " checking apache service"

service apache2 status

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

If we up this instance, it will straightaway creates a ubuntu instance and installs Apache in it.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant up

Bringing machine 'vag1' up with 'virtualbox' provider...

==> vag1: Importing base box 'ubuntu/trusty64'...

==> vag1: Matching MAC address for NAT networking...

==> vag1: Checking if box 'ubuntu/trusty64' is up to date...

************ OUTPUT TRUNCATED ********************

vag1: virtual machine match the version of VirtualBox you have installed on

vag1: your host and reload your VM.

vag1:

vag1: Guest Additions Version: 4.3.36

vag1: VirtualBox Version: 5.0

==> vag1: Configuring and enabling network interfaces...

==> vag1: Mounting shared folders...

vag1: /vagrant => F:/Workspace_local/vagrant

vag1: /home/vagrant => F:/Workspace_local/vagrant

==> vag1: Running provisioner: shell...

vag1: Running: C:/Users/USER/AppData/Local/Temp/vagrant-shell20170305-8664-103uhrn.sh

==> vag1: Get:1 http://security.ubuntu.com trusty-security InRelease [65.9 kB]

==> vag1: Ign http://archive.ubuntu.com trusty InRelease

************ OUTPUT TRUNCATED ********************

==> vag1: Ign http://archive.ubuntu.com trusty/restricted Translation-en_US

==> vag1: Ign http://archive.ubuntu.com trusty/universe Translation-en_US

==> vag1: Fetched 11.8 MB in 35s (336 kB/s)

==> vag1: Reading package lists...

==> vag1: Building dependency tree...

==> vag1: Reading state information...

==> vag1: The following extra packages will be installed:

************ OUTPUT TRUNCATED ********************

==> vag1: Preparing to unpack .../libaprutil1-dbd-sqlite3_1.5.3-1_amd64.deb ...

==> vag1: Unpacking libaprutil1-dbd-sqlite3:amd64 (1.5.3-1) ...

==> vag1: Selecting previously unselected package libaprutil1-ldap:amd64.

==> vag1: Preparing to unpack .../libaprutil1-ldap_1.5.3-1_amd64.deb ...

==> vag1: Unpacking libaprutil1-ldap:amd64 (1.5.3-1) ...

************ OUTPUT TRUNCATED ********************

==> vag1: *

==> vag1: Setting up ssl-cert (1.0.33) ...

==> vag1: Processing triggers for libc-bin (2.19-0ubuntu6.9) ...

==> vag1: Processing triggers for ureadahead (0.100.0-16) ...

==> vag1: Processing triggers for ufw (0.34~rc-0ubuntu2) ...

==> vag1: checking apache service

==> vag1: * apache2 is running

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Enter localhost:8080 in the browser url and check...

Lets login and check whether the home directory is in sync or not...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant ssh

vagrant@127.0.0.1's password:

Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-110-generic x86_64)

* Documentation: https://help.ubuntu.com/

System information as of Sun Mar 5 21:14:39 UTC 2017

System load: 0.75 Processes: 79

Usage of /: 3.6% of 39.34GB Users logged in: 0

Memory usage: 25% IP address for eth0: 10.0.2.15

Swap usage: 0%

************ OUTPUT TRUNCATED ********************

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by

applicable law.

vagrant@vagrant-ubuntu-trusty-64:~$

Since this is not the default one, it asks for the password of vagrant user...

vagrant@vagrant-ubuntu-trusty-64:~$ pwd

/home/vagrant

vagrant@vagrant-ubuntu-trusty-64:~$ ls

apache_script.sh Vagrantfile Vagrantfile_orig

vagrant@vagrant-ubuntu-trusty-64:~$

My script and Vagrantfile modifications done their job perfectly.

CHEF:

Let us try initializing another instance using chef by writing a simple recipe to build an Apache server.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vi Vagrantfile

Edited Vagrantfile, so that it will create a vagrant instance with name chef-vag and chef installed at the time of spin up.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ cat Vagrantfile | grep chefvag

config.vm.define "chef-vag" do |chefvag|

chefvag.vm.box = "ubuntu/trusty64"

chefvag.vm.network "private_network",type:"dhcp"

chefvag.vm.synced_folder "../vagrant", "/home/vagrant"

chefvag.vm.box_url = "https://atlas.hashicorp.com/ubuntu/boxes/trusty64"

chefvag.vm.network "forwarded_port", guest: 80, host: 1234

chefvag.vm.provision "chef_solo" do |chef|

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

To run the recipe we need to create the chef directory structure,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ mkdir -p cookbooks/myvag/recipes

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ cd cookbooks/myvag/recipes/

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/cookbooks/myvag/recipes $ vi default.rb

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/cookbooks/myvag/recipes $

Sample Chef recipe which helps for apache installation on my chef-vag instance,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/cookbooks/myvag/recipes $ cat default.rb

execute "apt-get update"

package "apache2"

execute "service apache2 status"

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/cookbooks/myvag/recipes $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/cookbooks/myvag/recipes $ cd ../../../

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Lets check the current status of my instance,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

chef-vag not created (virtualbox)

The environment has not yet been created. Run `vagrant up` to create the environment. If a machine is not created, only the default provider will be shown. So if a provider is not listed, then the machine is not created for that environment.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Command to check the status of all existing vagrant instances,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant global-status

id name provider state directory

-----------------------------------------------------------------------

92481d5 vag1 virtualbox running F:/Workspace_local/vagrant

The above shows information about all known Vagrant environments on this machine. This data is cached and may not be completely up-to-date. To interact with any of the machines, you can go to that directory and run Vagrant, or you can use the ID directly with Vagrant commands from any directory. For example: "vagrant destroy 1a2b3c4d"

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Let us try to start the chef-vag instance....

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant up

Bringing machine 'chef-vag' up with 'virtualbox' provider...

==> chef-vag: Checking if box 'ubuntu/trusty64' is up to date...

==> chef-vag: Fixed port collision for 22 => 2222. Now on port 2200.

==> chef-vag: Clearing any previously set network interfaces...

==> chef-vag: Preparing network interfaces based on configuration...

chef-vag: Adapter 1: nat

chef-vag: Adapter 2: hostonly

==> chef-vag: Forwarding ports...

chef-vag: 80 (guest) => 1234 (host) (adapter 1)

chef-vag: 22 (guest) => 2200 (host) (adapter 1)

==> chef-vag: Booting VM...

==> chef-vag: Waiting for machine to boot. This may take a few minutes...

chef-vag: SSH address: 127.0.0.1:2200

chef-vag: SSH username: vagrant

chef-vag: SSH auth method: private key

chef-vag:

************ OUTPUT TRUNCATED ********************

chef-vag: /home/vagrant => F:/Workspace_local/vagrant

chef-vag: /tmp/vagrant-chef/55cd743a2054d1c91454ad65be2b0aa4/cookbooks => F:/Workspace_local/vagrant/cookbooks

==> chef-vag: Running provisioner: chef_solo...

chef-vag: Installing Chef (latest)...

==> chef-vag: Generating chef JSON and uploading...

==> chef-vag: Running chef-solo...

==> chef-vag: stdin: is not a tty

==> chef-vag: [2017-03-05T23:01:59+00:00] INFO: Started chef-zero at chefzero://localhost:8889 with repository at /tmp/vagrant-chef/55cd743a2054d1c91454ad65be2b0aa4, /tmp/vagrant-chef

==> chef-vag: One version per cookbook

==> chef-vag: [2017-03-05T23:01:59+00:00] INFO: Forking chef instance to converge...

==> chef-vag: Starting Chef Client, version 12.19.36

==> chef-vag: [2017-03-05T23:01:59+00:00] INFO: *** Chef 12.19.36 ***

************ OUTPUT TRUNCATED ********************

==> chef-vag:

==> chef-vag: - myvag (0.0.0)

==> chef-vag: Installing Cookbook Gems:

==> chef-vag: Compiling Cookbooks...

==> chef-vag: Converging 3 resources

==> chef-vag: Recipe: myvag::default

==> chef-vag:

==> chef-vag: * execute[apt-get update] action run

==> chef-vag:

==> chef-vag: [execute] Hit http://security.ubuntu.com trusty-security InRelease

==> chef-vag: Ign http://archive.ubuntu.com trusty InRelease

==> chef-vag:

==> chef-vag: Hit http://archive.ubuntu.com trusty-updates InRelease

==> chef-vag: Hit http://security.ubuntu.com trusty-security/main Sources

************ OUTPUT TRUNCATED ********************

==> chef-vag:

==> chef-vag: Hit http://archive.ubuntu.com trusty-backports/universe Sources

==> chef-vag: Hit http://archive.ubuntu.com trusty-backports/multiverse Sources

==> chef-vag: Hit http://archive.ubuntu.com trusty-backports/main amd64 Packages

==> chef-vag: Hit http://archive.ubuntu.com trusty-backports/multiverse amd64 Packages

==> chef-vag:

==> chef-vag: Hit http://archive.ubuntu.com trusty-backports/main Translation-en

==> chef-vag:

************ OUTPUT TRUNCATED ********************

==> chef-vag: [2017-03-05T23:02:08+00:00] INFO: execute[apt-get update] ran successfully

==> chef-vag: - execute apt-get update

==> chef-vag: * apt_package[apache2] action install

==> chef-vag: [2017-03-05T23:02:18+00:00] INFO: apt_package[apache2] installed apache2 at 2.4.7-1ubuntu4.13

==> chef-vag:

==> chef-vag: - install version 2.4.7-1ubuntu4.13 of package apache2

==> chef-vag: * execute[service apache2 status] action run

==> chef-vag:

==> chef-vag: [execute] * apache2 is running

==> chef-vag: [2017-03-05T23:02:18+00:00] INFO: execute[service apache2 status] ran successfully

==> chef-vag: - execute service apache2 status

==> chef-vag: [2017-03-05T23:02:18+00:00] INFO: Chef Run complete in 16.691522998 seconds

==> chef-vag: Running handlers:

==> chef-vag: [2017-03-05T23:02:18+00:00] INFO: Running report handlers

==> chef-vag: Running handlers complete

==> chef-vag: [2017-03-05T23:02:18+00:00] INFO: Report handlers complete

==> chef-vag: Chef Client finished, 3/3 resources updated in 18 seconds

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

chef-vag running (virtualbox)

The VM is running. To stop this VM, you can run `vagrant halt` to shut it down forcefully, or you can run `vagrant suspend` to simply suspend the virtual machine. In either case, to restart it again, simply run `vagrant up`.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant global-status

id name provider state directory

-------------------------------------------------------------------------

92481d5 vag1 virtualbox running F:/Workspace_local/vagrant

bd57218 chef-vag virtualbox running F:/Workspace_local/vagrant

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

PUPPET:

Similar to Chef recipe, we can even write a Puppet manifest file and provision puppet in the Vagrantfile to automate things...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant$ vi Vagrantfile

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant$ cat Vagrantfile

# -*- mode: ruby -*-

# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure

# configures the configuration version (we support older styles for

# backwards compatibility). Please don't change it unless you know what

# you're doing.

Vagrant.configure("2") do |config|

config.vm.define "puppet-vag" do |puppetvag|

puppetvag.vm.hostname = "pupp-ci"

puppetvag.vm.box = "ubuntu/trusty64"

puppetvag.vm.box_url = "https://atlas.hashicorp.com/ubuntu/boxes/trusty64"

puppetvag.vm.network "forwarded_port", guest: 80, host: 1111

puppetvag.vm.provision "puppet" do |puppet|

puppet.manifests_path="manifests"

puppet.manifest_file="default.pp"

end

config.vm.define "puppet-vag-db" do |puppetvagdb|

puppetvagdb.vm.hostname = "pupp-db"

puppetvagdb.vm.box = "centos/7"

puppetvagdb.vm.box_url = "https://atlas.hashicorp.com/centos/boxes/7"

end

USER@USER-PC MINGW64 /f/Workspace_local/vagrant$

This time I am creating a instance with CentOS template....

USER@USER-PC MINGW64 /f/Workspace_local/vagrant$ vagrant status

Current machine states:

puppet-vag not created (virtualbox)

puppet-vag-db not created (virtualbox)

This environment represents multiple VMs. The VMs are all listed above with their current state. For more information about a specific VM, run `vagrant status NAME`.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant$

Like Chef, we need to create directory structure of Puppet to run the Puppet default.pp manifest file during initiation of instance...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant$ mkdir manifests

USER@USER-PC MINGW64 /f/Workspace_local/vagrant$ cd manifests/

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/manifests $ vi default.pp

Sample manifest file to automate apache installation,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/manifests $ cat default.pp

exec { "apt-get update":

command => "/usr/bin/apt-get update",

}

package { "apache2":

require => Exec["apt-get update"],

}

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/manifests $

Lets spin up the instances with puppet provisioning,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/manifests $ vagrant up

Bringing machine 'puppet-vag' up with 'virtualbox' provider...

Bringing machine 'puppet-vag-db' up with 'virtualbox' provider...

==> puppet-vag: Importing base box 'ubuntu/trusty64'...

**************** OUTPUT TRUNCATED ***************

==> puppet-vag: Booting VM...

==> puppet-vag: Waiting for machine to boot. This may take a few minutes...

puppet-vag: SSH address: 127.0.0.1:2201

puppet-vag: SSH username: vagrant

puppet-vag: SSH auth method: private key

puppet-vag:

puppet-vag: Vagrant insecure key detected. Vagrant will automatically replace

puppet-vag: this with a newly generated keypair for better security.

puppet-vag:

puppet-vag: Inserting generated public key within guest...

puppet-vag: Removing insecure key from the guest if it's present...

puppet-vag: Key inserted! Disconnecting and reconnecting using new SSH key...

==> puppet-vag: Machine booted and ready!

**************** OUTPUT TRUNCATED ***************

==> puppet-vag: Setting hostname...

==> puppet-vag: Mounting shared folders...

puppet-vag: /vagrant => F:/Workspace_local/vagrant

puppet-vag: /tmp/vagrant-puppet/manifests-a11d1078b1b1f2e3bdea27312f6ba513 => F:/Workspace_local/vagrant/manifests

==> puppet-vag: Running provisioner: puppet...

==> puppet-vag: Running Puppet with default.pp...

==> puppet-vag: stdin: is not a tty

==> puppet-vag: Notice: Compiled catalog for pupp-ci.hsd1.nj.comcast.net in environment production in 0.18 seconds

==> puppet-vag: Notice: /Stage[main]/Main/Exec[apt-get update]/returns: executed successfully

==> puppet-vag: Notice: /Stage[main]/Main/Package[apache2]/ensure: ensure changed 'purged' to 'present'

==> puppet-vag: Notice: Finished catalog run in 80.51 seconds

==> puppet-vag-db: Box 'centos/7' could not be found. Attempting to find and install...

puppet-vag-db: Box Provider: virtualbox

puppet-vag-db: Box Version: >= 0

==> puppet-vag-db: Loading metadata for box 'https://atlas.hashicorp.com/centos/boxes/7'

puppet-vag-db: URL: https://atlas.hashicorp.com/centos/boxes/7

==> puppet-vag-db: Adding box 'centos/7' (v1611.01) for provider: virtualbox

puppet-vag-db: Downloading: https://atlas.hashicorp.com/centos/boxes/7/versions/1611.01/providers/virtualbox.box

puppet-vag-db:

==> puppet-vag-db: Successfully added box 'centos/7' (v1611.01) for 'virtualbox'!

**************** OUTPUT TRUNCATED ***************

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/manifests $

Lets check the apache server from browser's url... (Always remember to change port number, when you are running more than 1 Apache server)

localhost:1111

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/manifests $ vagrant status

Current machine states:

puppet-vag running (virtualbox)

puppet-vag-db running (virtualbox)

This environment represents multiple VMs. The VMs are all listed above with their current state. For more information about a specific VM, run `vagrant status NAME`.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant/manifests $

One last command which I wanted to discuss is "vagrant box add"

We can create our own vagrant boxes which means, in our infrastructure if we have a instance and we need to replicate more from existing one, then we can package a box and using that packaged box we can spin new instances...

Let me explain with a scenario, we implemented different instances so far and my development environment is in need of more "vag1" instances.

Package it,

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant box list

centos/7 (virtualbox, 1611.01)

ubuntu/trusty64 (virtualbox, 20170222.0.1)

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant package vag1

This will package by compressing the instance, once the package is ready...

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant box add <package_name>

These are many other things to work on Vagrant, these are different main things which we should know atleast. Depending upon different scenarios and requirements we modify the Vagrantfile in different ways.

################################################################################

Installation of Vagrant in Windows, VMware and Virtualbox !!!

2017-03-05T10:08:00.003-08:00

Many of us are familiar with two leading virtualization providers VMware (Workstation) and Oracle (Virtualbox). Generally we create Virtual Machine (VMs), allocate resources to them and install OS. Vagrant is an open-source software, using which we can create virtual environment within seconds. It was started by Mitchelle Hashimoto.

We can use it to initiate a instance within Windows PC, VMware Workstation, Oracle Virtualbox and in AWS cloud environment. In real time, Vagrant is highly used for building development and test environments in Software Industry.

We can download the software from,

https://www.vagrantup.com/

Useful and related information like what and when, from:

https://en.wikipedia.org/wiki/Vagrant_(software)

Let me show how to use it in Windows, how we can initiate a simple instance to work with Unix environment in our Computer/Laptop.

Click on the Download....

Select the platform, to download the installer...

Vagrant_1.9.msi for Windows...

Vagrant_1.9.rpm for CentOS...

Installation procedure is normal .exe installer...

Now lets move on to our Command Line....

C:\Users\lokes>vagrant --version

Vagrant 1.9.1

C:\Users\lokes>

To initiate the Vagrant instance, we need the Vagrant box url....

Now click on the desired Vagrant box,

Use this url to initiate and download the Vagrant box template...

Go to desired directory location,

F:\Workspace_local>cd vagrant
F:\Workspace_local\vagrant>

F:\Workspace_local\vagrant>dir

Volume in drive F is ENTERTAINMENT
Volume Serial Number is 42C5-DACA
Directory of F:\Workspace_local\vagrant
03/04/2017 06:32 PM <DIR> .
03/04/2017 06:32 PM <DIR> ..
0 File(s) 0 bytes
2 Dir(s) 221,537,452,032 bytes free
F:\Workspace_local\vagrant>

F:\Workspace_local\vagrant>vagrant init ubuntu/trusty64 https://atlas.hashicorp.com/ubuntu/boxes/trusty64 box
A `Vagrantfile` has been placed in this directory. You are now ready to `vagrant up` your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on `vagrantup.com` for more information on using Vagrant.
F:\Workspace_local\vagrant>

Once initiated, we need to use "vagrant up". It initiates the Vagrant box with defaults as per the "Vagrantfile".

F:\Workspace_local\vagrant>vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'ubuntu/trusty64' could not be found. Attempting to find and install...
default: Box Provider: virtualbox
default: Box Version: >= 0
==> default: Loading metadata for box 'https://atlas.hashicorp.com/ubuntu/boxes/trusty64'
default: URL: https://atlas.hashicorp.com/ubuntu/boxes/trusty64
==> default: Adding box 'ubuntu/trusty64' (v20170222.0.1) for provider: virtualbox
default: Downloading: https://atlas.hashicorp.com/ubuntu/boxes/trusty64/versions/20170222.0.1/providers/virtualbox.box
default: Progress: 100% (Rate: 4633k/s, Estimated time remaining: --:--:--)
==> default: Successfully added box 'ubuntu/trusty64' (v20170222.0.1) for 'virtualbox'!
==> default: Importing base box 'ubuntu/trusty64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'ubuntu/trusty64' is up to date...
==> default: Setting the name of the VM: vagrant_default_1488670674498_57274
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
default: Adapter 1: nat
==> default: Forwarding ports...
default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
default:
default: Vagrant insecure key detected. Vagrant will automatically replace
default: this with a newly generated keypair for better security.
default:
default: Inserting generated public key within guest...
default: Removing insecure key from the guest if it's present...
default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
default: The guest additions on this VM do not match the installed version of
default: VirtualBox! In most cases this is fine, but in rare cases it can
default: prevent things such as shared folders from working properly. If you see
default: shared folder errors, please make sure the guest additions within the
default: virtual machine match the version of VirtualBox you have installed on
default: your host and reload your VM.
default:
default: Guest Additions Version: 4.3.36
default: VirtualBox Version: 5.0
==> default: Mounting shared folders...
default: /vagrant => F:/Workspace_local/vagrant

F:\Workspace_local\vagrant>

Command to check list of boxes available,

F:\Workspace_local\vagrant>vagrant box list

ubuntu/trusty64 (virtualbox, 20170222.0.1)

F:\Workspace_local\vagrant>

Command to login to our Vagrant instance,

F:\Workspace_local\vagrant>vagrant ssh

`ssh` executable not found in any directories in the %PATH% variable. Is an

SSH client installed? Try installing Cygwin, MinGW or Git, all of which

contain an SSH client. Or use your favorite SSH client with the following

authentication information shown below:

Host: 127.0.0.1

Port: 2222

Username: vagrant

Private key: F:/Workspace_local/vagrant/.vagrant/machines/default/virtualbox/private_key

F:\Workspace_local\vagrant>

I have Git bash already installed, we can even install openSSH for ssh service....

USER@USER-PC MINGW64 ~ $ cd F:

USER@USER-PC MINGW64 /f $ cd Workspace_local/vagrant/

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ git --version

git version 2.11.0.windows.3

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant status

Current machine states:

default running (virtualbox)

The VM is running. To stop this VM, you can run `vagrant halt` to

shut it down forcefully, or you can run `vagrant suspend` to simply

suspend the virtual machine. In either case, to restart it again,

simply run `vagrant up`.

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Now we can "ssh" to the vagrant box....

USER@USER-PC MINGW64 /f/Workspace_local/vagrant $ vagrant ssh

Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-110-generic x86_64)

* Documentation: https://help.ubuntu.com/

System information as of Sun Mar 5 16:26:29 UTC 2017

System load: 0.58 Processes: 82

Usage of /: 3.6% of 39.34GB Users logged in: 0

Memory usage: 27% IP address for eth0: 10.0.2.15

Swap usage: 0%

Graph this data and manage this system at:

https://landscape.canonical.com/

Get cloud support with Ubuntu Advantage Cloud Guest:

http://www.ubuntu.com/business/services/cloud

0 packages can be updated.

0 updates are security updates.

New release '16.04.2 LTS' available.

Run 'do-release-upgrade' to upgrade to it.

vagrant@vagrant-ubuntu-trusty-64:~$

vagrant@vagrant-ubuntu-trusty-64:~$ uname -a
Linux vagrant-ubuntu-trusty-64 3.13.0-110-generic #157-Ubuntu SMP Mon Feb 20 11:54:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
vagrant@vagrant-ubuntu-trusty-64:~$

Now we are inside our vagrant instance, we can run any Linux/Ubuntu commands,

vagrant@vagrant-ubuntu-trusty-64:~$ df -kh
Filesystem Size Used Avail Use% Mounted on
udev 241M 12K 241M 1% /dev
tmpfs 49M 352K 49M 1% /run
/dev/sda1 40G 1.5G 37G 4% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 245M 0 245M 0% /run/shm
none 100M 0 100M 0% /run/user
none 450G 251G 200G 56% /vagrant
vagrant@vagrant-ubuntu-trusty-64:~$

vagrant@vagrant-ubuntu-trusty-64:~$ifconfig
eth0 Link encap:Ethernet HWaddr 08:00:27:41:9a:fb
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe41:9afb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:510 errors:0 dropped:0 overruns:0 frame:0
TX packets:365 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:50359 (50.3 KB) TX bytes:43637 (43.6 KB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

vagrant@vagrant-ubuntu-trusty-64:~$ exit
logout
Connection to 127.0.0.1 closed.
USER@USER-PC MINGW64 /f/Workspace_local/vagrant $

Now let us see how to initiate vagrant in VMware and Virtualbox....

To install Vagrant in VMware or Virtualbox we need OVF (Open Virtualization Format) file of the downloaded template file of the Vagrant box.

It will be found in .vagrant.d directory....

VMware

Using this OVF template, we can initiate vagrant...

Select the OVF file,

Give the desired location for the VM,

Vagrant is booting up....

Login Credentials:
user: vagrant passwd: vagrant

We can run normal Ubuntu commands,

VirtualBox:

In Virtualbox also it is the same procedure to deploy the OVF in Oracle Virtualbox.
Click on "Import Appliance"...

Browse the OVF file location...

Login Credentials:
user: vagrant passwd: vagrant

We can run normal Ubuntu commands,

In my next post, I will discuss about working with Vagrant and different commands used in Administration of Vagrant boxes.

We can build development environment so easily with the help of Vagrant, we can automate the tasks by editing "Vagrantfile" as per the requirement of our infrastructure.

################################################################################

Project creation in Jenkins !!!

2017-02-05T18:30:00.001-08:00

In the previous post, we have setup the Jenkins Master/Slave environment. In this post, lets see the project creation and the automation of build process along with the Continuous Integration.

Most important tabs in Jenkins UI.

Click on the "create new jobs"....

Choose a name for your project,

We selected Free style, because it gives platform independent template for our project...
We can restrict where this project can run .... I selected my Windows_Slave....

We have 5 main categories while building a template for out project....
First of all, let us provide details related to our SCM..

Build Triggers are something which are responsible to build our projects in time intervals..

Build Periodically: Allows us to schedule timings (as per cron schedule) to trigger our build..

Build after other project: It restricts a build that it should be build only if a particular build is done..

Poll SCM: Our SCM will poll and (if any changes done to our SVN/Git) then triggers our build.

In Build environment, we can provide our command for the build process....

Different options for build step...

Different options for Post build actions....

Post build actions: These actions decide what to be performed once a build is created...

I chose to "Archive the artifacts"...

Once we are good with the Configuration part, we can save and apply....

Sample project which I built during my MAVEN post, I committed to my Git SCM...

Click on Build Now....

Console output:

It is failed, saying mvn is not recognized as internal or external command...

In the Configuration part, we already set the Home for Maven.

We need to add the MAVEN tool....

Now it is enabled in configuration.. choose the tool location...

Let's try building our project again....

This time it is a SUCCESS...

Even the Post-Build job is executed successfully, we can see the archived package file java_app.1.0.jar is generated...

As you can see, I failed 6 times. It is so important to maintain required plugins, tools and Home locations...

One more important thing which comes under Jenkins Administration:
Prepare for Shutdown...
In case of any maintenance activities we can create an alarm using this option.... shows a message and restricts any new builds till we cancel it.

################################################################################

Jenkins Master/Slave Environment !!!

2017-02-05T14:40:00.000-08:00

Jenkins is a Open source Java based program used to automate the Continuous Integration process of Project builds. In real time, every project will be having different phases like Source Code development, Build process, Continuous Integration and Continuous Delivery.

For more productivity, automation plays most important role. To automate build process, many Organizations make use of different tools like ANT, Maven... Similarly for automating Integration process, we use Jenkins tool. In this post, I am going to discuss about the setup of Jenkins Master/Slave environment and how to create/build projects.

Jenkins is developed by Kohsuke Kawaguchi, it is exact copy of Hudson. After acquisition of SUN by Oracle, Hudson rights moved to Oracle and then its not completely Open source. So, Kohsuke developed Jenkins from Hudson and released it as Open Source.

In Jenkins, to balance the load of jobs we setup master/slave environment. So that the jobs will be shared among the master and its slave nodes. It is better to leave Master node for Jenkins Administration and distribute the jobs among slave nodes.

Installation is so simple, just copy the war file to /home/ubu/apache-tomcat-7.0.75/webapps and once tomcat started, Jenkins will be deployed.

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75# cd webapps/

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps#

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps# ls
docs examples git git.war host-manager jenkins jenkins.war manager ROOT
root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps#

Open in the browser,

Copy the password from given location,

To create admin user,

To manage Jenkins nodes,

Choose a name and select permanent agent,

The red X mark indicates to launch the agent in our slave node...
There are different launch methods, but I prefer launching via web start....

Go to Manage Jenkins ---> Configure Global System...

Select "Random", TCP port for JNLP agents...

Now, Launch agent via Java Web start is enabled in launch method...

Click on Launch...

Click on Run, it will start the agent in Slave node...

Agent started....

Similarly we can create different slave nodes and we can launch the agent in different ways....

################################################################################

MAVEN Installation and Configuration !!!

2017-01-31T15:10:00.002-08:00

MAVEN is the most popular Build automation tool. In JAVA environment, packaging is a common thing. Whenever we develop some code, we will create packages. In real time, the source code may contain different packages, to build many packages we require some scripts to automate that build process.

Here comes the part of Build automation tools. There are many such tools, in this post we are going to discuss MAVEN. It can be used to automate the entire build process, pom.xml (POM -Project Object Model) is the main configuration file. Unlike ANT, Maven uses its own directory structure.

Maven follows different phases in its Default Life Cycle.

Validate : In this phase, Maven downloads all required packages and jars.

Compile : It compiles the source code.

Test cases : Runs test cases.

Run Tests : Runs tests (can be QA, UAT).

Package : Creates/generates package files.

Install : Responsible to perform installations.

Deploy : Actual deployment of jar files to Web Server.

Note: “Whenever a phase is called, maven automatically calls all its prior phases….”

Installation is so simple, just to unzip the file and set the environment variable for Maven_Home.

Set the Environment Variable:

To check the version,

C:\Users\lokes>mvn --version

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T11:41:47-05:00)

Maven home: F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\bin\..

Java version: 1.8.0_121, vendor: Oracle Corporation JAVA is a must.....

Java home: C:\Program Files (x86)\Java\jdk1.8.0_121\jre

Default locale: en_US, platform encoding: Cp1252

OS name: "windows 10", version: "10.0", arch: "x86", family: "dos"

C:\Users\lokes>

As we discussed, MAVEN follows own directory structure. To get this, we need to generate the archetype (Template kind of) for our Maven.

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9>mvn archetype:generate

[INFO] Scanning for projects...

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom (4 KB at 2.5 KB/sec)

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/22/maven-plugins-22.pom

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/22/maven-plugins-22.pom (13 KB at 82.7 KB/sec)

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/maven-

************ OUTPUT TRUNCATED ************

1760: remote -> tr.com.obss.sdlc.archetype:obss-archetype-java (This archetype provides a common skelton for the Java packages.)

1761: remote -> tr.com.obss.sdlc.archetype:obss-archetype-webapp (This archetype provides a skelton for the Java Web Application packages.)

1769: remote -> us.fatehi:schemacrawler-archetype-plugin-lint (-)

Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): 923:

Choose org.apache.maven.archetypes:maven-archetype-quickstart version:

1: 1.0-alpha-1

2: 1.0-alpha-2

3: 1.0-alpha-3

4: 1.0-alpha-4

5: 1.0

6: 1.1

Choose a number: 6: 5

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/archetypes/maven-archetype-quickstart/1.0/maven-archetype-quickstart-1.0.jar

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/archetypes/maven-archetype-quickstart/1.0/maven-archetype-quickstart-1.0.jar (5 KB at 28.2 KB/sec)

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/archetypes/maven-archetype-quickstart/1.0/maven-archetype-quickstart-1.0.pom

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/archetypes/maven-archetype-quickstart/1.0/maven-archetype-quickstart-1.0.pom (703 B at 10.2 KB/sec)

Define value for property 'groupId': : com.myjava

Define value for property 'artifactId': : java_app

Define value for property 'version': 1.0-SNAPSHOT: : 1.0

Define value for property 'package': com.myjava: : jar

Confirm properties configuration:

groupId: com.myjava

artifactId: java_app

version: 1.0

package: jar

Y: : Y

[INFO] ----------------------------------------------------------------------------

[INFO] Parameter: package, Value: jar

[INFO] Parameter: groupId, Value: com.myjava

[INFO] Parameter: artifactId, Value: java_app

[INFO] Parameter: packageName, Value: jar

[INFO] Parameter: version, Value: 1.0

[INFO] project created from Old (1.x) Archetype in dir: F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 02:24 min

[INFO] Finished at: 2017-02-02T20:25:08-05:00

[INFO] Final Memory: 11M/51M

[INFO] ------------------------------------------------------------------------

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9>

Lets see the directory structure,

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\Maven_Practice\apache-maven-3.3.9

02/02/2017 08:25 PM <DIR> .

02/02/2017 08:25 PM <DIR> ..

11/10/2015 11:44 AM <DIR> bin

11/10/2015 11:44 AM <DIR> boot

11/10/2015 11:38 AM <DIR> conf

02/02/2017 08:25 PM <DIR> java_app

11/10/2015 11:44 AM <DIR> lib

11/10/2015 11:44 AM 19,335 LICENSE

11/10/2015 11:44 AM 182 NOTICE

11/10/2015 11:38 AM 2,541 README.txt

3 File(s) 22,058 bytes

7 Dir(s) 226,675,310,592 bytes free

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9>

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9>cd java_app
F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app

02/02/2017 08:25 PM <DIR> .

02/02/2017 08:25 PM <DIR> ..

02/02/2017 08:25 PM 648 pom.xml

02/02/2017 08:25 PM <DIR> src

1 File(s) 648 bytes

3 Dir(s) 226,675,310,592 bytes free

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>cd src

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>cd src\main\java
F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\java>
F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\java>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\java

02/02/2017 08:25 PM <DIR> .

02/02/2017 08:25 PM <DIR> ..

02/02/2017 08:25 PM <DIR> jar

0 File(s) 0 bytes

3 Dir(s) 226,675,310,592 bytes free
F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\java>cd jar
F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\java\jar>

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\java\jar>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\java\jar

02/02/2017 08:25 PM <DIR> .

02/02/2017 08:25 PM <DIR> ..

02/02/2017 08:25 PM 179 App.java

1 File(s) 179 bytes

2 Dir(s) 226,675,310,592 bytes free
F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\java\jar>

Default pom.xml generated by archetype....

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

<modelVersion>4.0.0</modelVersion>

<groupId>com.myjava</groupId>

<artifactId>java_app</artifactId>

<packaging>jar</packaging>

<version>1.0</version>

<name>java_app</name>

<url>http://maven.apache.org</url>

<dependencies>

<dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

<version>3.8.1</version>

<scope>test</scope>

</dependency>

</dependencies>

</project>

Let's try to run and observe different phases of Maven Life Cycle....

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>mvn compile

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building java_app 1.0

[INFO] ------------------------------------------------------------------------

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom (8 KB at 4.3 KB/sec)

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/maven/2.0.6/maven-2.0.6.pom (9 KB at 90.2 KB/sec)

************** OUTPUT TRUNCATED **************

Downloaded: https://repo.maven.apache.org/maven2/junit/junit/3.8.2/junit-3.8.2.jar (118 KB at 62.6 KB/sec)

Downloaded: https://repo.maven.apache.org/maven2/com/google/collections/google-collections/1.0/google-collections-1.0.jar (625 KB at 327.7 KB/sec)

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 22.574 s

[INFO] Finished at: 2017-02-02T20:57:20-05:00

[INFO] Final Memory: 12M/29M

[INFO] ------------------------------------------------------------------------

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>

Let’s Check whether “target” directory created or not….

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app

02/02/2017 08:57 PM <DIR> .

02/02/2017 08:57 PM <DIR> ..

02/02/2017 08:25 PM 648 pom.xml

02/02/2017 08:25 PM <DIR> src

02/02/2017 08:57 PM <DIR> target

1 File(s) 648 bytes

4 Dir(s) 226,675,310,592 bytes free

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>cd target

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target>

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target

02/02/2017 08:57 PM <DIR> .

02/02/2017 08:57 PM <DIR> ..

02/02/2017 08:57 PM <DIR> classes

02/02/2017 08:57 PM <DIR> maven-status

0 File(s) 0 bytes

4 Dir(s) 226,675,310,592 bytes free

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target>

If you try compiling one more time, it won’t download anything. Since the download of required packages and jars is one time.

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9>cd ../

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>mvn compile

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building java_app 1.0

[INFO] ------------------------------------------------------------------------

[INFO]

[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ java_app ---

[INFO]

[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ java_app ---

[INFO] Nothing to compile - all classes are up to date

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 2.492 s

[INFO] Finished at: 2017-02-02T21:01:27-05:00

[INFO] Final Memory: 6M/16M

[INFO] ------------------------------------------------------------------------

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>

Now let’s try to run the test phase (So, it will call Validate, Compile and Test)

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>mvn test

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building java_app 1.0

[INFO] ------------------------------------------------------------------------

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.pom

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.pom (11 KB at 15.1 KB/sec)

[INFO]

[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ java_app ---

************ OUTPUT TRUNCATED ************

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/2.12.4/surefire-booter-2.12.4.pom

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/2.12.4/surefire-booter-2.12.4.pom (3 KB at 39.0 KB/sec)

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/2.12.4/surefire-api-2.12.4.pom

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running jar.AppTest

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 7.584 s

[INFO] Finished at: 2017-02-02T21:02:42-05:00

[INFO] Final Memory: 14M/35M

[INFO] ------------------------------------------------------------------------

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>

Let’s check the “package” phase…

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>mvn package

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building java_app 1.0

[INFO] ------------------------------------------------------------------------

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/2.4/maven-jar-plugin-2.4.pom

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/2.4/maven-jar-plugin-2.4.pom (6 KB at 6.3 KB/sec)

[INFO]

[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ java_app ---

[INFO] skip non existing resourceDirectory F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\src\main\resources

************ OUTPUT TRUNCATED ************

[INFO]

[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ java_app ---

[INFO] Surefire report directory: F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target\surefire-reports

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running jar.AppTest

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO]

[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ java_app ---

Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/2.5/maven-archiver-2.5.pom

Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/2.5/maven-archiver-2.5.pom (5 KB at 59.1 KB/sec)

************ OUTPUT TRUNCATED ************

Downloaded: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.0/plexus-utils-3.0.jar (221 KB at 550.6 KB/sec)

[INFO] Building jar: F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target\java_app-1.0.jar

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 8.880 s

[INFO] Finished at: 2017-02-02T21:04:33-05:00

[INFO] Final Memory: 9M/23M

[INFO] ------------------------------------------------------------------------

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>

We can see clearly, "package" phase called all its prior phases (Validate, Compile, Test cases and then Package) …

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app

02/02/2017 08:57 PM <DIR> .

02/02/2017 08:57 PM <DIR> ..

02/02/2017 08:25 PM 648 pom.xml

02/02/2017 08:25 PM <DIR> src

02/02/2017 09:04 PM <DIR> target

1 File(s) 648 bytes

4 Dir(s) 226,675,298,304 bytes free

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>

Final output of package phase should be, generating a jar file...

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app>cd target

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target

02/02/2017 09:04 PM <DIR> .

02/02/2017 09:04 PM <DIR> ..

02/02/2017 08:57 PM <DIR> classes

02/02/2017 09:04 PM 1,969 java_app-1.0.jar

02/02/2017 09:04 PM <DIR> maven-archiver

02/02/2017 08:57 PM <DIR> maven-status

02/02/2017 09:02 PM <DIR> surefire-reports

02/02/2017 09:02 PM <DIR> test-classes

1 File(s) 1,969 bytes

7 Dir(s) 226,675,298,304 bytes free

F:\Workspace_local\Maven_Practice\apache-maven-3.3.9\java_app\target>

Similar to all these phases, install and deploy phases will be. Deploy will be the final phase where our source code will be deployed to our Web Server.

In real time, this Build automation will be performed for many builds at a time and so all of them should be integrated continuously and will deployed continuously. To achieve automated Build process along with Continuous Integration and followed by Continuous Delivery we use tools like Jenkins, which will be covered in our next post.

################################################################################

.

2017-01-31T15:09:00.000-08:00

Git Commands and Administration !!!

2017-01-30T19:56:00.000-08:00

In my previous post, I discussed about the installation of Git SCM. In this post, let's see some working with Git. How a developer works, commands they use and the concepts they follow. Git is a distributed VCS, it has many advantages compared to SVN. I am going to explain why we are maintaining a local and remote repo.

Git uses its own directory structure,

1. master

main workspace... contains stable code.

2. branches

development branches, usually for every Release.

3. tags

Usually these are read only, contains files related to every build.

We have already seen basic commands like git add, status, --version, commit in the previous post. Lets go to some other regular commands.

Git log command: full log output

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git log

commit e4c2887da9aad9dedbd325b0926f53773076b057

Author: userh d <userh d>

Date: Sun Jan 29 22:32:57 2017 -0500

1st file

commit c5ef99904b07aabc9d31324c96cd4d62649370b1

Author: admin <admin@gitblit>

Date: Sun Jan 29 22:30:30 2017 -0500

Initial commit

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Git Short log command: one liner output

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git shortlog

userh d (1):

1st file

admin (1):

Initial commit

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Creating users, simple GUI task...

Clone the repo to aj workspace,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ cd ../aj_git/

user@userH-PC MINGW64 /f/Workspace_local/aj_git $

user@userH-PC MINGW64 /f/Workspace_local/aj_git $ git clone ssh://aj@10.0.0.229:29418/git_onp_repo.git

Cloning into 'git_onp_repo'...

Password authentication

Password:

remote: Counting objects: 6, done

remote: Finding sources: 100% (6/6)

remote: Getting sizes: 100% (4/4)

remote: Compressing objects: 100% (72/72)

remote: Total 6 (delta 0), reused 3 (delta 0)

Receiving objects: 100% (6/6), 435 bytes | 0 bytes/s, done.

Resolving deltas: 100% (1/1), done.

user@userH-PC MINGW64 /f/Workspace_local/aj_git $

user@userH-PC MINGW64 /f/Workspace_local/aj_git $ ls

git_onp_repo/

user@userH-PC MINGW64 /f/Workspace_local/aj_git $ cd git_onp_repo/

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ ls

first_file README.md

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ touch 123

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ ls

123 first_file README.md

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Untracked files:

(use "git add <file>..." to include in what will be committed)

123

nothing added to commit but untracked files present (use "git add" to track)

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

Lets Commit,

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git add 123

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Changes to be committed:

(use "git reset HEAD <file>..." to unstage)

new file: 123

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git commit -m "add 123 file"

[master 9799557] add 123 file

Committer: userh d <userh d>

1 file changed, 0 insertions(+), 0 deletions(-)

create mode 100644 123

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git status

On branch master

Your branch is ahead of 'origin/master' by 1 commit.

(use "git push" to publish your local commits)

nothing to commit, working tree clean

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

Let's push these changes to remote repo,

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git push

Password authentication

Password:

Counting objects: 2, done.

Delta compression using up to 4 threads.

Compressing objects: 100% (2/2), done.

Writing objects: 100% (2/2), 267 bytes | 0 bytes/s, done.

Total 2 (delta 0), reused 0 (delta 0)

To ssh://10.0.0.229:29418/git_onp_repo.git

! [remote rejected] master -> master (User "aj" does not have push permissions for "git_onp_repo.git"!)

error: failed to push some refs to 'ssh://aj@10.0.0.229:29418/git_onp_repo.git'

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

My aj user does not have permissions for this, so lets assign them....

Let's try now....

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git push

Password authentication

Password:

Counting objects: 2, done.

Delta compression using up to 4 threads.

Compressing objects: 100% (2/2), done.

Writing objects: 100% (2/2), 267 bytes | 0 bytes/s, done.

Total 2 (delta 0), reused 0 (delta 0)

remote: Updating references: 100% (1/1)

To ssh://10.0.0.229:29418/git_onp_repo.git

e4c2887..9799557 master -> master

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

These changes should not be there on any other user's repo, lets move onto our earlier repo....

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ cd ../../git_onp_repo/

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

first_file README.md

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

nothing to commit, working tree clean

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

We can pull/fetch the diff changes or the changes pushed by other developers onto remote repo to our local repos...

Difference between pull and fetch…

We can pull the remote repo changes to local using both pull and fetch, but when we used fetch they won’t be applied to “local workspace”. Whereas when we use “pull” they will get applied to local.

No Untracked files, so lets try pulling these diff changes between local and remote repo.

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git pull ssh://admin@10.0.0.229:29418/git_onp_repo.git

Password authentication

Password:

remote: Counting objects: 3, done

remote: Finding sources: 100% (2/2)

remote: Getting sizes: 100% (2/2)

remote: Total 2 (delta 0), reused 2 (delta 0)

Unpacking objects: 100% (2/2), done.

From ssh://10.0.0.229:29418/git_onp_repo

* branch HEAD -> FETCH_HEAD

Updating e4c2887..9799557

Fast-forward

123 | 0

1 file changed, 0 insertions(+), 0 deletions(-)

create mode 100644 123

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

123 first_file README.md We got them, 123 file is pulled….

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Command to see available remote repos...

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git remote -v

origin ssh://admin@10.0.0.229:29418/git_onp_repo.git (fetch)

origin ssh://admin@10.0.0.229:29418/git_onp_repo.git (push)

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Command to clean untracked files,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ touch 7 8 9

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

123 7 8 9 abc first_file README.md

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Untracked files:

(use "git add <file>..." to include in what will be committed)

nothing added to commit but untracked files present (use "git add" to track)

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git clean -f

Removing 7

Removing 8

Removing 9

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

nothing to commit, working tree clean

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

123 abc first_file README.md

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Commands to deal with staged files,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ touch 7 8 9

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git add .

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Changes to be committed:

(use "git reset HEAD <file>..." to unstage)

new file: 7

new file: 8

new file: 9

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

There may be situations where we are not supposed to commit these files, but in future we may need them. So, to store them without committing we do stashing.

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git stash

Saved working directory and index state WIP on master: ca8adff add abc

HEAD is now at ca8adff add abc

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git stash list

stash@{0}: WIP on master: ca8adff add abc

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

123 abc first_file README.md

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Now after few days, if we want those changes to be applied, (always be careful with the naming)

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git stash apply stash@{0}

On branch master

Your branch is up-to-date with 'origin/master'.

Changes to be committed:

(use "git reset HEAD <file>..." to unstage)

new file: 7

new file: 8

new file: 9

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

123 7 8 9 abc first_file README.md

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

We can commit these changes, if we are sure with the files....

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Changes to be committed:

(use "git reset HEAD <file>..." to unstage)

new file: 7

new file: 8

new file: 9

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

To view the stash list,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git stash list

stash@{0}: WIP on master: ca8adff add abc

To drop/delete a stash,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git stash drop stash@{0}

Dropped stash@{0}(7c48e681cf4a8bd25bfe3ce133f42d08aff01d50)

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git stash list

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git stash pop

No stash found. ---- If found, it will applied and deleted from the list

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

To create branches,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git branch b1

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git checkout b1

A 7

A 8

A 9

Switched to branch 'b1'

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git status

On branch b1

Changes to be committed:

(use "git reset HEAD <file>..." to unstage)

new file: 7

new file: 8

new file: 9

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ ls

123 7 8 9 abc first_file README.md

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $

To push this change to git server,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git push --set-upstream origin b1

Password authentication

Password:

Total 0 (delta 0), reused 0 (delta 0)

remote: Updating references: 100% (1/1)

Branch b1 set up to track remote branch b1 from origin.

To ssh://10.0.0.229:29418/git_onp_repo.git

* [new branch] b1 -> b1

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $

To view the Git directory structure,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git ls-remote

Password authentication

Password:

ca8adff0f969129fef85dc20fc6d185b67c7d5fb HEAD

ca8adff0f969129fef85dc20fc6d185b67c7d5fb refs/heads/b1

ca8adff0f969129fef85dc20fc6d185b67c7d5fb refs/heads/master

24c6ca01c3306070b5ace459827ca5e37353e2aa refs/meta/gitblit/reflog

From ssh://admin@10.0.0.229:29418/git_onp_repo.git

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $

Lets create/delete another branch.

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git branch b2

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git branch

* b1

master

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git push --set-upstream origin b2

Password authentication

Password:

Total 0 (delta 0), reused 0 (delta 0)

remote: Updating references: 100% (1/1)

Branch b2 set up to track remote branch b2 from origin.

To ssh://10.0.0.229:29418/git_onp_repo.git

* [new branch] b2 -> b2

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git branch -d b2

Deleted branch b2 (was ca8adff).

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git branch

* b1

master

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $

To push this change to remote repo,

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git push origin :b2

Password authentication

Password:

remote: Updating references: 100% (1/1)

To ssh://10.0.0.229:29418/git_onp_repo.git

- [deleted] b2

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git branch

* b1

master

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $

Commands to work with patchsets. Patchsets are helpful to share the data/files which are not pushed to main remote repo. Dev A can share his files to Dev B, without pushing them to actual git server using patchsets.

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (b1) $ git checkout master

Your branch is up-to-date with 'origin/master'.

Switched to branch 'master'

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ touch aa ab ac ad

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git add .

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Changes to be committed:

(use "git reset HEAD <file>..." to unstage)

new file: aa

new file: ab

new file: ac

new file: ad

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Let’s create a patch to share these changes to other developer “aj”…

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git format-patch origin/master --stdout >patch1.patch

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

123 aa ab abc ac ad first_file patch1.patch README.md

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

I am simply copying, since we are the user aj. In general we do sharing through “scp”, “mails”, “shared dirs.” etc…

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ cp patch1.patch ../aj_git/git_onp_repo/

user@userH-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

“aj” user in his workspace….

user@userH-PC MINGW64 ~$ cd F:

user@userH-PC MINGW64 /f $ cd Workspace_local/aj_git/git_onp_repo/

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ ls

123 abc first_file patch1.patch README.md

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Untracked files:

(use "git add <file>..." to include in what will be committed)

patch1.patch

nothing added to commit but untracked files present (use "git add" to track)

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

As of now, he does not have the changes done by admin…. Only untracked file is the patch file, which we copied just now…

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git apply patch1.patch

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Untracked files:

(use "git add <file>..." to include in what will be committed)

patch1.patch

nothing added to commit but untracked files present (use "git add" to track)

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $ ls

123 aa ab abc ac ad first_file patch1.patch patchset_1.patch README.md

user@userH-PC MINGW64 /f/Workspace_local/aj_git/git_onp_repo (master) $

If we are OK with the files, we can commit them to our local repo…

These are the few commands which are helpful while working with Git SCM. Still I didn’t discuss few concepts related to branch merging, rebase and cherry pick. They are simple and can be implemented easily according to the situation..

######################################################################################

Git for Source Code Management !!!

2017-01-29T21:26:00.000-08:00

For source code management, different organizations make use of different tools. In my previous post we discussed about one such SCM tool which is SVN. In this post, I am going to discuss about Git. Now a days Git is the most popular and commonly used SCM tool.

Unlike in SVN, Git makes use of two repos.

1. Local repo
This is local to developers workspace. His/her locally stored workspace.

2. Remote repo
This is actual repo maintained by Git server. Every time a developer works and commits to his local repo, if he/she is satisfied with the changes then he will push these changes to remote repo.

Why Organizations prefer Git than SVN:
Main reason is, Git is a Distributed Version Controlling System and can be maintained both on premises and on cloud.

Distributed because: Every developer has their local repos distributed, but the main commits can be done/shared to/from the remote repo (Git server).

ON CLOUD:

There are different online open source providers for hosting Git repo.

github, gitbasket and gitlab --- on cloud
stash, gitblit and gerrit --- on premises
collaborator and gitblit --- both

Open Github.com,

Click on Sign in and enter required details:

Choose according to wish, since for practice I am selecting public repository.

Fill in the details:

Click on New repository:

Give a name for your repo, If we include a README file, repo will be created along with it.

If you want a Private repo, then do.....

Find the Https URL:

Now all we need to do, is to install Gitblit client in our local machine.

https://git-scm.com/download/win File: Git-2.11.0.3-64-bit.exe

It is a normal .exe, installation process is next--next--next...
Use command prompt or git bash (Git bash is preferable for Linux users)

user@USER-PC MINGW64 ~$ pwd
   /c/Users/user
user@USER-PC MINGW64 ~$ git --version
   git version 2.11.0.windows.3

Go to desired and use git clone to checkout our repo:

user@USER-PC MINGW64 ~$ cd F:/
user@USER-PC MINGW64 /f $ cd Workspace_local/
user@USER-PC MINGW64 /f/Workspace_local $ ls
   antivir.txt hp_sr.txt subversion/
user@USER-PC MINGW64 /f/Workspace_local $ git clone https://github.com/oncloudrepo/git_repo.git
   Cloning into 'git_repo'...
   remote: Counting objects: 3, done.
   remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 3
   Unpacking objects: 100% (3/3), done.
user@USER-PC MINGW64 /f/Workspace_local $ ls
   antivir.txt git_repo/ hp_sr.txt subversion/

user@USER-PC MINGW64 /f/Workspace_local $ cd git_repo/
user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $ ls
   README.md
user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $

Copied a file to my workspace..

user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $ ls
   antivir.txt README.md

user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $ git status

   On branch master
   Your branch is up-to-date with 'origin/master'.
   Untracked files:
      (use "git add <file>..." to include in what will be committed)
          antivir.txt
   nothing added to commit but untracked files present (use "git add" to track)
user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $

A file which is created, but yet to be added ==== UnTracked file

user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $ git add .
user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $
user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $ git status
   On branch master
   Your branch is up-to-date with 'origin/master'.
   Changes to be committed:
        (use "git reset HEAD <file>..." to unstage)
              new file: antivir.txt
user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $

Before committing, only README file is in repo....

A file which is added, but yet to be committed ===== Staged File

user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $ git commit -m "anti file add"
   [master 90ba8e9] anti file add
    Committer: USER <USER >
   Your name and email address were configured automatically based
   on your username and hostname. Please check that they are accurate.
   You can suppress this message by setting them explicitly. Run the
   following command and follow the instructions in your editor to edit
   your configuration file:
          git config --global --edit
   After doing this, you may fix the identity used for this commit with:
          git commit --amend --reset-author
          1 file changed, 11 insertions(+)
          create mode 100644 antivir.txt
   user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $

A file which is committed to local repo, but yet to be pushed ===== Tracked File

user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $ git status
   On branch master
   Your branch is ahead of 'origin/master' by 1 commit.
       (use "git push" to publish your local commits)
   nothing to commit, working tree clean
user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $

So far file was committed to our local repo, now we need to push to remote repo to reflect these changes in Git server.

user@USER-PC MINGW64 /f/Workspace_local/git_repo (master) $ git push

   Counting objects: 3, done.
   Delta compression using up to 4 threads.
   Compressing objects: 100% (3/3), done.
   Writing objects: 100% (3/3), 399 bytes | 0 bytes/s, done.
   Total 3 (delta 0), reused 0 (delta 0)
   To https://github.com/oncloudrepo/git_repo.git
      1785bc4..90ba8e9 master -> master
user@USER-PC MINGW64 /f/Workspace_local/git_repo (master)

My commits are pushed to my On cloud git repo successfully......

ON PREMISES:

To setup Git server within our Organization premises, we need a Windows/Unix server. Download and copy the files to the server.

We access our Git server through browser, which means Gitblit is a Web application. To deploy a web application, we need Tomcat and to deploy war files we need JAVA.

So, I copied Tomcat and gitblit.war file to the server...

root@ubuntu-kvm:~# cd /home/ubu/

root@ubuntu-kvm:/home/ubu#

root@ubuntu-kvm:/home/ubu# ls

Desktop Downloads jenkins.war Pictures Templates

apache-tomcat-7.0.75.tar.gz Documents git.war Music Public Videos

root@ubuntu-kvm:/home/ubu#

gunzip it....

root@ubuntu-kvm:/home/ubu# tar -zxf apache-tomcat-7.0.75.tar.gz

root@ubuntu-kvm:/home/ubu# ls

apache-tomcat-7.0.75 Desktop Downloads jenkins.war Pictures Templates

apache-tomcat-7.0.75.tar.gz Documents git.war Music Public Videos

root@ubuntu-kvm:/home/ubu#

root@ubuntu-kvm:/home/ubu# cd apache-tomcat-7.0.75/

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75# ls

bin conf lib LICENSE logs NOTICE RELEASE-NOTES RUNNING.txt temp webapps work

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75#

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75# cd webapps/

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps# cp ../../git.war .

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps# ls

docs examples git.war host-manager manager ROOT

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps#

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps# cd ../bin/

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/bin# ls

bootstrap.jar commons-daemon-native.tar.gz digest.sh startup.bat tool-wrapper.sh

catalina.bat configtest.bat setclasspath.bat startup.sh version.bat

catalina.sh configtest.sh setclasspath.sh tomcat-juli.jar version.sh

catalina-tasks.xml daemon.sh shutdown.bat tomcat-native.tar.gz

commons-daemon.jar digest.bat shutdown.sh tool-wrapper.bat

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/bin#

To start the Tomcat server....

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/bin# ./startup.sh

Using CATALINA_BASE: /home/ubu/apache-tomcat-7.0.75

Using CATALINA_HOME: /home/ubu/apache-tomcat-7.0.75

Using CATALINA_TMPDIR: /home/ubu/apache-tomcat-7.0.75/temp

Using JRE_HOME: /usr

Using CLASSPATH: /home/ubu/apache-tomcat-7.0.75/bin/bootstrap.jar:/home/ubu/apache-tomcat-7.0.75/bin/tomcat-juli.jar

Tomcat started.

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/bin#

During the start of Tomcat server, war files will be deployed.... git folder was generated...

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/bin# cd ../webapps

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps# ls

docs examples git git.war host-manager manager ROOT

root@ubuntu-kvm:/home/ubu/apache-tomcat-7.0.75/webapps#

Enter URL along with "git" to access git.... Login username: admin and password: admin

Let us create new repo on premises through gitblit.... Choose name and type, then Create...

Use the URL which is generated for our repo to checkout the git repo.

user@USERS-PC MINGW64 /f/Workspace_local $ git clone ssh://admin@10.0.0.229:29418/git_onp_repo.git

Cloning into 'git_onp_repo'...

The authenticity of host '[10.0.0.229]:29418 ([10.0.0.229]:29418)' can't be established. RSA key fingerprint is SHA256:UqXvhPBG8s2CQAgJTc0IR5oqzAzJQv0mM7j+OgKcgng.0

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '[10.0.0.229]:29418' (RSA) to the list of known hosts.

Password authentication

Password:

remote: Counting objects: 3, done

remote: Finding sources: 100% (3/3)

remote: Getting sizes: 100% (2/2)

remote: Compressing objects: 100% (72/72)

remote: Total 3 (delta 0), reused 0 (delta 0)

Receiving objects: 100% (3/3), done.

user@USERS-PC MINGW64 /f/Workspace_local $

user@USERS-PC MINGW64 /f/Workspace_local $ ls

antivir.txt git_onp_repo/ git_repo/ hp_sr.txt subversion/

user@USERS-PC MINGW64 /f/Workspace_local $

user@USERS-PC MINGW64 /f/Workspace_local $ cd git_onp_repo/

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

README.md

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Create a new file ....

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ tee -a first_file

hi this is my first file thru git

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ ls

first_file README.md

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Check the status and add it...

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Untracked files:

(use "git add <file>..." to include in what will be committed)

first_file

nothing added to commit but untracked files present (use "git add" to track)

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git add .

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is up-to-date with 'origin/master'.

Changes to be committed:

(use "git reset HEAD <file>..." to unstage)

new file: first_file

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

Lets commit this..

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git commit -m "1st file"

[master e4c2887] 1st file

Committer: USERS <USERS >

Your name and email address were configured automatically based

on your username and hostname. Please check that they are accurate.

You can suppress this message by setting them explicitly. Run the

following command and follow the instructions in your editor to edit

your configuration file:

git config --global --edit

After doing this, you may fix the identity used for this commit with:

git commit --amend --reset-author

1 file changed, 0 insertions(+), 0 deletions(-)

create mode 100644 first_file

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git status

On branch master

Your branch is ahead of 'origin/master' by 1 commit.

(use "git push" to publish your local commits)

nothing to commit, working tree clean

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

To push these changes to remote repo...

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $ git push

Password authentication

Password:

Counting objects: 3, done.

Delta compression using up to 4 threads.

Compressing objects: 100% (2/2), done.

Writing objects: 100% (3/3), 267 bytes | 0 bytes/s, done.

Total 3 (delta 0), reused 0 (delta 0)

remote: Updating references: 100% (1/1)

To ssh://10.0.0.229:29418/git_onp_repo.git

c5ef999..e4c2887 master -> master

user@USERS-PC MINGW64 /f/Workspace_local/git_onp_repo (master) $

In this post we covered installing and deploying gitblit server. Committed few files to both local and remote repo, also both in on cloud and on premises git server...

################################################################################

SVN Commands and Administration !!!

2017-01-11T20:09:00.003-08:00

In my previous post we discussed the concept of SVN and its installation. In this post I am going to show the working with commands of SVN from Client and administration part at the server end.

SVN is a source code management tool which is used by a developer to push the code to the actual SVN server. Sometimes different users may work on the same directory and files, so to avoid conflicts related to data, svn uses "commit" which allows whether a file can be committed or any other user is still working on it.

If other users working on it, svn throws us a message asking whether to continue with the changes by A developer or to commit another developer B's work.

Let us get into the action by practicing few basic commands:

We have seen already, how to add a file and commit. Now lets see how to add a directory which contains different files.

F:\Workspace_local>svn st
svn: warning: W155007: 'F:\Workspace_local' is not a working copy

F:\Workspace_local>

Important rule never forget to perform work/ execute svn cmds in your Workspace.

F:\Workspace_local>cd subversion
F:\Workspace_local\subversion>
F:\Workspace_local\subversion>svn st --- just to know workspace is in sync with repo or not....
F:\Workspace_local\subversion>

Before adding a directory to my repo,

F:\Workspace_local\subversion>dir
Volume in drive F is ENTERTAINMENT
Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\subversion

01/09/2017 05:51 PM <DIR> .
01/09/2017 05:51 PM <DIR> ..
01/09/2017 05:50 PM 1,914 README.md
01/09/2017 05:53 PM 17 Welcome.txt
2 File(s) 1,931 bytes
2 Dir(s) 230,929,895,424 bytes free
F:\Workspace_local\subversion>

Adding a folder named "New folder", it contains some text files and a image file.

Now add it to our server repo,

F:\Workspace_local\subversion>dir
Volume in drive F is ENTERTAINMENT
Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\subversion

01/11/2017 07:17 PM <DIR> .
01/11/2017 07:17 PM <DIR> ..
01/11/2017 07:13 PM <DIR> New folder
01/09/2017 05:50 PM 1,914 README.md
01/09/2017 05:53 PM 17 Welcome.txt
2 File(s) 1,931 bytes
3 Dir(s) 230,929,895,424 bytes free
F:\Workspace_local\subversion> Above is, current status before commit and after copying..

F:\Workspace_local\subversion>svn st
? New folder
F:\Workspace_local\subversion> Status shows recent changes which are yet to sync local with repo..

It is time to add our directory...

F:\Workspace_local\subversion>svn add "New folder"
A New folder
A (bin) New folder\Capture1.PNG
A New folder\hp_sr.txt
A New folder\antivir.txt
A New folder\icici amts.txt
F:\Workspace_local\subversion>

Once added, we need to commit this to reflect in repo....

F:\Workspace_local\subversion>svn commit -m "adding newfolder to repo"
Adding New folder
Adding (bin) New folder\Capture1.PNG
Adding New folder\antivir.txt
Adding New folder\hp_sr.txt
Adding New folder\icici amts.txt
Transmitting file data ....
Committed revision 3.
F:\Workspace_local\subversion>

To delete a file from local and repo,

F:\Workspace_local\subversion>svn delete "New folder"\hp_sr.txt
D New folder\hp_sr.txt
F:\Workspace_local\subversion>cd "New folder"
F:\Workspace_local\subversion\New folder>
F:\Workspace_local\subversion\New folder>dir
Volume in drive F is ENTERTAINMENT
Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\subversion\New folder

01/11/2017 07:22 PM <DIR> .
01/11/2017 07:22 PM <DIR> ..
02/15/2016 01:13 PM 157 antivir.txt
01/11/2017 07:12 PM 170,057 Capture1.PNG
09/08/2016 10:37 AM 50 icici amts.txt
3 File(s) 170,264 bytes
2 Dir(s) 230,929,723,392 bytes free
F:\Workspace_local\subversion\New folder>

So far this is temporary. If we check in browser, this won't be reflected at server end.

Before committing and after deleting at local, still we can view the file and its contents...

F:\Workspace_local\subversion\New folder>svn commit -m "deleted hp_sr"
Deleting hp_sr.txt
Committed revision 4.
F:\Workspace_local\subversion\New folder>

After commit, it will be permanently deleted....

To check the logs, we have "svn log"command.

F:\Workspace_local\subversion>svn log -r3
------------------------------------------------------------------------
r3 | (no author) | 2017-01-11 19:18:45 -0500 (Wed, 11 Jan 2017) | 1 line

adding newfolder to repo
------------------------------------------------------------------------
F:\Workspace_local\subversion>

Similarly to view log for a range of revisions,

F:\Workspace_local\subversion>svn log -r3:4
------------------------------------------------------------------------
r3 | (no author) | 2017-01-11 19:18:45 -0500 (Wed, 11 Jan 2017) | 1 line

adding newfolder to repo
------------------------------------------------------------------------
r4 | (no author) | 2017-01-11 19:23:55 -0500 (Wed, 11 Jan 2017) | 1 line

deleted hp_sr
------------------------------------------------------------------------
F:\Workspace_local\subversion>

Whenever we feel like updating repo with older we have "update" command,
If we consider that our previous deletion is not a proper step and want changes to go back.

F:\Workspace_local\subversion\New folder>svn update -r3
Updating '.':
A hp_sr.txt
Updated to revision 3.
F:\Workspace_local\subversion\New folder>

F:\Workspace_local\subversion\New folder>dir
Volume in drive F is ENTERTAINMENT
Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local\subversion\New folder

01/11/2017 07:32 PM <DIR> .
01/11/2017 07:32 PM <DIR> ..
02/15/2016 01:13 PM 157 antivir.txt
01/11/2017 07:12 PM 170,057 Capture1.PNG
01/11/2017 07:32 PM 11 hp_sr.txt
09/08/2016 10:37 AM 50 icici amts.txt
4 File(s) 170,275 bytes
2 Dir(s) 230,929,723,392 bytes free
F:\Workspace_local\subversion\New folder>

There are two other main commands with special usage. When it comes to a developer, he has his workspace checked out with the repo. But what if a one time user want to push some data from his PC. Similarly what if someone wants to pull a single file.

Usually we can use checkout command to push entire directory, so to push a single file we have "svn import" command.

Note: To make use of "import" command, no need to run it from Workspace. I copied a file outside of my workspace "subversion".

F:\Workspace_local>svn st

svn: warning: W155007: 'F:\Workspace_local' is not a working copy

F:\Workspace_local>

F:\Workspace_local>svn import hp_sr.txt http://10.0.0.218:81/subversion/New\folder/hp_sr.txt -m "adding hp_sr thru import"

Adding hp_sr.txt

Committed revision 5.

F:\Workspace_local>

One import to New\folder... and below import to "subversion" (Workspace)...

F:\Workspace_local>svn import hp_sr.txt http://10.0.0.218:81/subversion/hp_sr.txt -m "adding hp_sr thru import"

Adding hp_sr.txt

Committed revision 6.

F:\Workspace_local>

F:\Workspace_local>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local

01/11/2017 07:39 PM <DIR> .

01/11/2017 07:39 PM <DIR> ..

01/11/2017 07:32 PM 11 hp_sr.txt

01/11/2017 07:17 PM <DIR> subversion

1 File(s) 11 bytes

3 Dir(s) 230,929,723,392 bytes free

F:\Workspace_local>

To fetch/pull a single file, we make use of "export" command... We can run it from outside of workspace.

F:\Workspace_local>svn export http://10.0.0.218:81/subversion/New%20folder/antivir.txt

A antivir.txt

Export complete.

F:\Workspace_local>

F:\Workspace_local>dir

Volume in drive F is ENTERTAINMENT

Volume Serial Number is 42C5-DACA

Directory of F:\Workspace_local

01/11/2017 07:48 PM <DIR> .

01/11/2017 07:48 PM <DIR> ..

01/11/2017 07:18 PM 157 antivir.txt

01/11/2017 07:32 PM 11 hp_sr.txt

01/11/2017 07:17 PM <DIR> subversion

2 File(s) 168 bytes

3 Dir(s) 230,929,723,392 bytes free

F:\Workspace_local>

Other than these, there are few other commands like "svn revert" to revert any operations before committing, and "svn diff" used to view difference between two files.

So far we completed svn commands practical from client side. Now let us see some SVN admin tasks.

Admin should perform tasks related to Authorization, backups of repo and restricting few tasks or applying conditions to perform a task.

Observe the output:

F:\Workspace_local\subversion>svn log -r2:6
------------------------------------------------------------------------
r2 | (no author) | 2017-01-09 18:01:13 -0500 (Mon, 09 Jan 2017) | 1 line

1st Commit
------------------------------------------------------------------------
r3 | (no author) | 2017-01-11 19:18:45 -0500 (Wed, 11 Jan 2017) | 1 line

adding newfolder to repo
------------------------------------------------------------------------
r4 | (no author) | 2017-01-11 19:23:55 -0500 (Wed, 11 Jan 2017) | 1 line

deleted hp_sr
------------------------------------------------------------------------
r5 | (no author) | 2017-01-11 19:44:45 -0500 (Wed, 11 Jan 2017) | 1 line

adding hp_sr thru import
------------------------------------------------------------------------
r6 | (no author) | 2017-01-11 19:46:34 -0500 (Wed, 11 Jan 2017) | 1 line

adding hp_sr thru import
------------------------------------------------------------------------

F:\Workspace_local\subversion>

We are working with anonymous user, we can restrict this by creating separate users and also by restricting their access to certain level (directories).

To create users,

root@ubuntu-new:/opt/subversion-1.9.5-0# cd apache2/
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2#
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2# htpasswd -c /opt/subversion-1.9.5-0/svn_users dev1
New password:
Re-type new password:
Adding password for user dev1
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2#

For second developer, used -m to modify the users file. If we use -c it will create the file again...

root@ubuntu-new:/opt/subversion-1.9.5-0/apache2# htpasswd -m /opt/subversion-1.9.5-0/svn_users dev2
New password:
Re-type new password:
Adding password for user dev2
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2#

root@ubuntu-new:/opt/subversion-1.9.5-0/apache2# cat /opt/subversion-1.9.5-0/svn_users
dev1:$apr1$hb7S2CtJ$GwVa5dEBDzY3EFS/K6yj71
dev2:$apr1$vhccSww7$y.W95.QX5ofsBosFPcxZH0
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2#

Let me show, if I use -c option:

root@ubuntu-new:/opt/subversion-1.9.5-0/apache2# htpasswd -c /opt/subversion-1.9.5-0/svn_users dev3
New password:
Re-type new password:
Adding password for user dev3
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2#

root@ubuntu-new:/opt/subversion-1.9.5-0/apache2# cat /opt/subversion-1.9.5-0/svn_users
dev3:$apr1$Fa3y4u5Q$INzi63RXzYiThQZ9hm8S.1
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2#

Create a new file with name "svn_access" (can give according to admin wish)

root@ubuntu-new:/opt/subversion-1.9.5-0/apache2# vi /opt/subversion-1.9.5-0/svn_access
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2#

root@ubuntu-new:/opt/subversion-1.9.5-0# cat svn_access
[subversion:/] ------ can mention desired directory
dev3=rw
[groups] ------ can assign a group to a user

root@ubuntu-new:/opt/subversion-1.9.5-0#

To enable this authentication in our apache server,

root@ubuntu-new:/opt/subversion-1.9.5-0/apache2/conf# vi httpd.conf
root@ubuntu-new:/opt/subversion-1.9.5-0/apache2/conf#

root@ubuntu-new:/opt/subversion-1.9.5-0/apache2/conf# cat httpd.conf | tail -12
Include "/opt/subversion-1.9.5-0/apache2/conf/ssi.conf"
Include "/opt/subversion-1.9.5-0/apache2/conf/bitnami/bitnami.conf"
<Location /subversion>
DAV svn
SVNPath "/opt/subversion-1.9.5-0/repository"

AuthType Basic
AuthName "Subversion Project1"
AuthUserFile /opt/subversion-1.9.5-0/svn_users
Require valid-user
AuthzSVNAccessFile /opt/subversion-1.9.5-0/svn_access
</Location>

root@ubuntu-new:/opt/subversion-1.9.5-0/apache2/conf#

Then restart the apache server,

root@ubuntu-new:/opt/subversion-1.9.5-0# ./ctlscript.sh restart
/opt/subversion-1.9.5-0/subversion/scripts/ctl.sh : subversion stopped
Syntax OK
/opt/subversion-1.9.5-0/apache2/scripts/ctl.sh : httpd stopped
Syntax OK
/opt/subversion-1.9.5-0/apache2/scripts/ctl.sh : httpd started at port 81
/opt/subversion-1.9.5-0/subversion/scripts/ctl.sh : subversion started at port 3690

root@ubuntu-new:/opt/subversion-1.9.5-0#

Now try to open in browser, this time server will pop-up the Authentication window....

Use of svn_access file, is to restrict the access permissions of a user...

root@ubuntu-new:/opt/subversion-1.9.5-0# vi svn_access

root@ubuntu-new:/opt/subversion-1.9.5-0#

root@ubuntu-new:/opt/subversion-1.9.5-0# cat svn_access

[subversion:/New folder]

dev3=rw

[groups]

root@ubuntu-new:/opt/subversion-1.9.5-0#

Provided access only to "New folder" for dev3 user...

Before changing access,

After providing new access, he can only view and edit "New folder"

Another admin task is to handle backup and restore of repo.

To take dump,

svnadmin dump /opt/subversion-1.9.5-0/repository/ > /file_loc

To load it,

svnadmin load /opt/subversion-1.9.5-0/repository/ < /file_loc

Similar to these, an admin can restrict commits with some conditions. Admin can restrict few types of files to get added to server.

root@ubuntu-new:/opt/subversion-1.9.5-0/repository# cd hooks
root@ubuntu-new:/opt/subversion-1.9.5-0/repository/hooks#

In this hooks directory, we have different templates where we can write our scripts to restrict any kind of task...

root@ubuntu-new:/opt/subversion-1.9.5-0/repository/hooks# ls
post-commit.tmpl post-revprop-change.tmpl pre-commit.tmpl pre-revprop-change.tmpl start-commit.tmpl
post-lock.tmpl post-unlock.tmpl pre-lock.tmpl pre-unlock.tmpl

root@ubuntu-new:/opt/subversion-1.9.5-0/repository/hooks#

For example, we can write a script inside "pre-commit.tmpl" which needs to be executed before commiting any type of file/dir...

An admin can perform/restrict different tasks, in this post I covered few of them....

################################################################################

SubVersion (SVN) for Source Code Management !!!

2017-01-09T18:57:00.000-08:00

In any Software project, there are different methodologies and tools used during the project. Coming to methodologies there are Agile and Waterfall. Similarly when it comes to tools there are many tools we use during different stages of project.

In this post I am going to discuss about such a tool which widely used for Source Code Management during a Software Project. In Production environment, for any project there will be different releases.

All these releases will be scheduled with some timelines. To avoid confusion related to source code, we need some tools to manage it. SubVersion (SVN) is tool many of the Software Companies make use for Code management.

Using SVN, we create a directory structure which helps to maintain clarity about the code related to every release. This proper structure make use of following branching strategies:

1. Sequential

Sequential is a strategy, in which we can work on one release after completion of the other. No work related to different releases can be carried out during single timeline.

R2 will be started after completion of R1. Similarly R3 will be started after R2. So there will be no confusion in moving the developers data to production. But in real time this strategy will not be a good one.

2. Parallel

Most of the Clients cannot wait to start working on R2 till the R1 completes. So then comes the Parallel branching strategy. In Parallel, we can work on more than one Release at single point of time.

This helps to overcome the problem of longer duration for every release as we face in Sequential branching. To maintain proper directory structure, after completion of coding for each release we update the production with delta changes.

There is a problem with this strategy. For initial releases the updation of production is something like we merge 90% of new data to 10% of old data. But when it comes to 15th release or more, to update 10% of new data we need to update it by merging with 90% of old data.

3. Merging

To avoid the problem with Parallel branching, the Merging strategy was evolved. I will make use of two locations Core and Prod. Prod is the data what actually our Production server contains and core is data at developer end.

After completion of every release the Core should equals with Prod. When we use this strategy, during the development of R15 data which we need to update to Prod is just R13, R14 and R15 (Since R13,R14 may be in development stage).

What I mean, when R1 is released, core=prod=R1+old

R2 is released, core=R1+old Prod=R2+R1+old to make Core=Prod

we copy R2 to core......

R8 , core=R1...R6/7+old and Prod=R8 + R1....R6/7 + old

So to make Core=Prod, copy back R8 to Core.

For R15, we need delta changes from R13, R14 and R15 only....

But in Parallel branching, we need to update Prod with R15,R14,R13 changes by merging with R1....R12 data.

Now lets see some practical part like installing SVN and by working with some commands usually used by developers at Client level....

I am using svn from bitnami which is parented by APACHE.

root@ubuntu-new:~# cd /home/ubu/

root@ubuntu-new:/home/ubu#

root@ubuntu-new:/home/ubu# ls
bitnami-subversion-1.9.5-0-linux-x64-installer.run Music
Desktop Pictures
docker-master.zip Public
Documents Setup-Subversion-1.8.17.msi
Downloads Templates
libapache2-mod-php5_5.3.10-1ubuntu3.25_amd64.deb Videos
root@ubuntu-new:/home/ubu#

Already downloaded and copied my installer file to my Ubuntu server...

root@ubuntu-new:/home/ubu# ./bitnami-subversion-1.9.5-0-linux-x64-installer.run
----------------------------------------------------------------------------
Welcome to the Bitnami Subversion Stack Setup Wizard.

----------------------------------------------------------------------------
Installation folder

Please, choose a folder to install Bitnami Subversion Stack

Select a folder [/opt/subversion-1.9.5-0]:

----------------------------------------------------------------------------
Web Server Port

Choose a port that is not currently in use, such as port 81.

Apache Web Server Port [81]:

----------------------------------------------------------------------------
Repository

Please configure your repository

Path to repository [/opt/subversion-1.9.5-0/repository]:

----------------------------------------------------------------------------
Setup is now ready to begin installing Bitnami Subversion Stack on your
computer.

Do you want to continue? [Y/n]: Y

----------------------------------------------------------------------------
Please wait while Setup installs Bitnami Subversion Stack on your computer.

Installing
0% ______________ 50% ______________ 100%
#########################################

----------------------------------------------------------------------------
Setup has finished installing Bitnami Subversion Stack on your computer.

Launch Bitnami Subversion Stack [Y/n]: Y

root@ubuntu-new:/home/ubu#

To Check the status of our subversion...

root@ubuntu-new:/home/ubu# cd

/opt/subversion-1.9.5-0
root@ubuntu-new:/opt/subversion-1.9.5-0# ls
apache2 config licenses README.txt sqlite uninstall.dat
changelog.txt ctlscript.sh manager-linux-x64.run repository subversion use_subversion
common img properties.ini scripts uninstall
root@ubuntu-new:/opt/subversion-1.9.5-0#

root@ubuntu-new:/opt/subversion-1.9.5-0# ./ctlscript.sh status
subversion already running
apache already running
root@ubuntu-new:/opt/subversion-1.9.5-0#

Through browser we can check.....

Now to work with SVN CLI, lets install the client in our Windows.

We need to checkout the SVN server using the URL in our desired location from our client...

Most important thing is, Workspace is the place where a developer can do all the coding. We can add and then commit the files which are inside our Workspace.

As you see, soon after checkout command, there is a subversion directory within which readme file was created. Our workspace is the subversion directory...

Above I tried executing svn commands outside the workspace, it didn't allow me. But when I tried changing directory to subversion then I am able to execute "svn info".

Let us try creating a file and adding it to our SVN server.

Above I created Welcome file with some text in it....

Soon after creating a file, we can check the "svn status" command....

Command Prompt output of our SVN Client machine....

F:\Workspace_local\subversion>

F:\Workspace_local\subversion>svn status

? Welcome.txt

F:\Workspace_local\subversion>

In the above output, ? represents this file is not yet reflected on our server....

First add it, then we need to commit....

Note: Commit always needs a message, we can give any message so that whenever we check the logs, we can remember what are the changes done with this commit....

F:\Workspace_local\subversion>svn add Welcome.txt

A Welcome.txt

F:\Workspace_local\subversion>svn commit Welcome.txt -m "1st Commit"

Adding Welcome.txt

Transmitting file data .

Committed revision 2.

F:\Workspace_local\subversion>

This time if we check status command, no new changes done at client side. All data is in sync with server...

F:\Workspace_local\subversion>svn status

F:\Workspace_local\subversion>

Screenshots from browser,

Revision number before commit of new file....

Revision number incremented after commit of new file....

We can view the content of the file....

################################################################################