A Technical Gush..Devops Diary

2017-03-17T08:19:00.000+05:30

Devops Optimization

Introduction
With the current trend of evolving technology stack and business needs, every organization has to keep their environment razor sharp for a cutting edge technology. No matter how many tools or processes are introduced on hourly basis, one has to keep optimizing their environments, tools & processes in order to sustain the pace of growth and business requirement especially on the Devops Arena.
This article articulates the dire need of optimization, the benefits and techniques to optimize Devops tools and processes
What is Optimization?
Optimization is a process to stem the preeminent value out of the services in order to produce greater business benefits. Going by the business dictionary definition, it’s an alternative with the most cost effective or highly achievable performance under given constraints.
This very basic principle of optimization is applicable for Devops & Software Configuration Management as it renders various services under its portfolio such as Version control, Build Management, Code Release & Deployment Management.

Optimization Genres - PPT
Optimization can be classified into three broad spectra namely People, Process & Technology. Any organization or a project is continuously working around these spectra to optimize their delivery. . Whether it’s a service based organization or product based organization everyone are eyeing to optimize PPT.

How SCM Optimization is conducive to Business value?
Poor or no SCM (ANSI/IEEE Std-828-1998, IEEE Standard for Software Configuration Management Plans, and ANSI/IEEE Std-1042-1987, IEEE Guide to Software Configuration Management) in place triggers repeated bugs that has been fixed, impossible to rebuild the code releases, with no accountability and traceability on data integrity. Thus the resulting surge in

delivery cost, decreased productivity & delay in Time-To-Market.

On the other hand, adoption of SCM tools in the Software development market is only 20% (according to Ovum Research). Certainly notwithstanding the fact that any organization has to foot a bill for maintaining SCM process and tools which is not leveraged to its fullest capability, sadly

Radical Drivers for Devops Optimization
The key drivers for optimizing Devops are Version Control, Build, Deployment , Release management & Infrastructure. The individual continuums (although four

peas in pod) have their own processes and technology optimization as Shown in the diagram. For instance, to augment the use of version control capabilities, source control over cloud is niche to look forward as the users can leverage the capabilities on the flight.

When talking about the Build management,
Setting up Dockers, Continuous integration (CI) & Continuous Delivery are already batting a thousand for seamless and successful product release. Release management office and continuous deployment are virtue for any given environment.
Standing tall, Application Lifecycle Management (ALM) is another rudimentary driver for optimizing SCM process and tools. It has the capability to integrate, coordinate and manage the different phases of software delivery process being Agile or traditional waterfall model.

Continuous Integration (CI) & Continuous Delivery (CD)
CI keeps the development team in sync and eradicates integration issues off bay. Jenkins, Hudson, Cruise Control & Team City are some of the tools that embrace the technology.
CD automates and significantly improves the fashion of software delivery leveraging Continuous Integration, automated testing and automated deployment thus supporting agile practice

Optimization through Cutting-edge Technologies

The Dockers, SCM over cloud and Devops are the front runners / trending technology stack in the market.

People Optimization
From the People optimization front, it varies from organizations to organizations. In general, the typical roles in SCM arena are Version control Tool Administrator, Build Engineer / Lead / Manager, Deployment Engineer / Lead/ Manager and similarly Release Engineer / Release Lead / Manager.
To effectively optimize the roles, cross skilling is solution front forward as these roles go hand in hand. Tool Administrator can play the pivotal role of a Release Manager provided he /She are cross skilled. Likewise, Deployment Engineer / Lead can render tool administration and ensure the release processes are adhered before executing the builds as required. Aligning people ideally with their best roles in processes and in achieving goals/task.

This would indeed pave way for achieving substantial return on investment (ROI)

Roadmap & Step Forward
The distinctive optimization framework would serve the roadmap for implementation
Step 1: Initiate & Identify Pain Areas
Aligning to business goals, one has to identify the pain areas hindering the development advancements proportional to business growth. Validate the pain areas to ascertain by meticulous data collection.
Step 2: Design / Develop / Validate a Prototype
Prototype is a traditional technical term, rather would like to use the term as solution. Develop a solution based on the data collection, pain areas and mitigation approach. Ex. Development environment becomes too complex when there are no defined branching and merging strategy. The simple solution on branching methodology will ensure seamless merging and delivery process. Similarly develop a solution for every aforementioned continuum.
Step 3: Pilot Implementation / Validation
Identify the least significant environment or application for pilot implementation to lower the risk of failure, improve the ability to estimate cost saving on the proposed solution, increase buy-in and most importantly receive candid feedbacks
Step 4 : Enterprise Roll out Step 5: Communication
Successful rollout communication is a rudimentary step to adhere as this will ensure all the stakeholders within organization are well informed to accord the business benefit derived out of optimization.

Optimization is an inevitable process in IT industry constantly piercing through development lifecycle. It is up to the organization to equip and adopt the right technology stack and environment with trending strategies to cater the customer requirement. This paper would help to assimilate what needs to be optimized, how to optimize leveraging the market trend and the right approach towards implementation.

Configuration Management Elements

2011-07-05T00:58:00.004+05:30

It has been so... so long that i updated my blog . Nevertheless, i realized now that, it is so. Thank God :)

Of late am getting stung with a glucose 'ed question...Yes it was the engine that dragged me to rejuvenate...!!

-------------------------------------------------------------------

"Spell out the Basic Elements of Configuration Management ??"

------------------------------------------------------------------

Initially, it wasn't influencing me to give a thought process. But later, it did.

When i tried to dig into this question , a few were gushing out. They are

(Click the Image for clear view)

- Version Control : A systematic process over tools to track the changes to a file / project / any configuration identification

Version Control Tools :
i) ClearCase
ii) Subversion
iii) TFS
iv) CVS etc...

- Build : Combining the sources into a usable artifact.

Build Tools :

i) Make

ii) ANT

iii) MS Build etc

- Release : Providing the usable artifact by a defined process

- Deployment : Installing / injecting the artifacts into a system or application.

The above written are very few tools that are used in real world....just wiped off a drop from the ocean of Configuration Management.

**Build tools and Build Management tools are two different entities and so as the others, which am not going pitch in at this point of time. My next posts will have a bouquet of Build Management , Release Management and Deployment Management process etc.**

Duplicating a Vob

2010-04-20T15:33:00.001+05:30

A sudden strike jolted on duplicating a Vob. Thought whether we can duplicate a Vob, does that work and of course how ?. This was an opportunity that paved me to explore the possibles for duplicating a Vob.

The following procedures were NOT recommended / suggested by IBM but a different approach that glued well.

Clearcase registry registers / accepts the Vob, based on the Vob ID. So thats the key for us, going to change / modify the existing Vob id for duplication.

Ok, but how ??.

- Copy the entire Vob Storage (.vbs) to another preferred location

- Edit the vob_oid and replica_uuid text files, do any similar modification on both files.

- Register the storage with normal cleartool register command

- Create a new Vob tag and mount the newly created Vob tag.

Will illustrate with an Example:

(Click the Image for Better View)

- In the above given image, the "/test/test_vob" is being duplicated.
- The Vob storage (test_vob.vbs) is copied to another location
- After copying, edit the vob_oid and replica_uuid files. In this case, the vod_oid file's content (57ba5732.f5ba11dd.82b9.00:50:56:c0:00:01) is being modified to (57ba5732.f5ba11dd.82b9.00:50:56:c0:00:02) and in replica_uuid file as well.
- Once the modification is done, new vobstorage location is registered using typical cleartool register command and then NEW Vob tag is created.

- Mount the Newly created Vob for accessing the same.

Now the Vob is Duplicated with another Vob tag but the contents of both Vobs are very much similar.

Config Spec Time Based

2010-03-11T18:36:00.001+05:30

I stamp the Config Spec Utility in Clearcase as a Masterpiece because, just feed him a few lines of instructions to get what EVER version of the file / directory you want.

Many handy functions makes him so, one such function is "created_since" rule in config spec.

Syntax : {created_since()}

It is a known fact that the Config Spec is used to pick the selected versions. So as the the function (created_since) , pickups the version which has been created after the time / date specified.

E.g.

(Click the Image for Clear View)

- Taking the above given example, could see 3 Files namely file_1, file_2 & file_3 and also their creation time stamp is evident for us.
- If you take a look at their versions, all of them belong to a branch(test_branch) latest version and of course it has to be since the config rule is set so.

Below given is the picture describing "created_since" rule,

- The latest versions of the files file_1 & file_2 under the branch(test_branch) is not visible because they have been created before the time which is slated in the rule . i.e before 01.11 PM, 11-March-2010.
- But, the since_created rule matches the file_3 because the version has been created after the time specified.

So this would be a very handy function when ever there is need to select time based versions.

To be very specific, this function might be very fruitful during trouble shooting of any issues, tracking down any changes based on time etc... :)

removing interesting versions in clearcase

2010-01-10T23:27:00.011+05:30

**************************************************
Wishing everyone a Very Happy and Prosperous New Year..!!
**************************************************

You all know that a version of an element can be removed using rmver command.

While removing a particular version in clearcase, interestingly, you might be invited with some error message stating that " Not removing interesting versions". It means that, you cannot remove a version when a label is attached to it.

Its not a warning ,a error,which is obvious that you aren't successful :(

So, to remove a version when label (attribute) is attached to it, we can use rmlabel command to remove the label first and then execute rmver command to remove the version.

In spite of the above two steps, spice it up in one step by using -xlabel option in rmver command to remove the versions which have labels attached to it.

command: #cleartool rmver -xlabel

Example:

(Click the Image for Clear View)

- In the given example, the file hello.c has a label (REL1.0) in its second version on main branch
-When trying to remove the second version using rmver command, the error message is prompted. (Not Removing the interesting versions)
- Using the -xlabel option in rmver command, the second version of hello.c file has been removed.

Hence when ever a file's version needs to be removed even if a label is attached, simply using the -xlabel option in rmver command :)

restore directories and files in clearcase.....

2009-12-16T23:44:00.003+05:30

sometimes we are tend to rmname (remove from version control) a element which doesn't look fair enough at that moment. At later stage do realize that those element are the exact ones which is actually a show stopper :(

obviously you have to park this thinking somewhere.."can i restore that element right away ? " .......and how do you feel when someone says...why not, restore that element with a ln (link) command.

Clearcase Command : cleartool ln

- a element that was rmnamed (removed ) from version control can be restored back on the next or subsequent version using cleartool ln command

Lets dig into this with an example :-

(Click the Image for clear view)

-In the above example, there are two directory elements named "dir" and "dir1" listed with respective version information.
-Parent directory is checked-out from /main/2 version for modification and dir1 element has been rmnamed (removed).
-Once after modification, parent directory is checked-in and created another version /main/3 ( in which the dir1 element doesn't exists)
How to restore the element now?

- Check-out the parent directory , use the cleartool ln command for linking dir1 element to the version /main/2 of parent directory which holds the element.

#cleartool ln .@@/main/2/dir1 dir1 --> Watch the syntax. linking the dir1 element in the current directory from the 2nd version.

tweak the clearcase view...

2009-12-15T00:40:00.004+05:30

My curiosity on tweaking a dynamic view's property finally handshakes with a crispy command - "chview"

- By default when a dynamic view is created, its is destined to have the read-write mode for that view.
-Of course, after all we create a view to check-out the elements to local view for development activity and its pretty much a fact.
-But using "chview" command , we can tweak the view in read-only mode. Which typically brings out the fact that, when a view is set in a read-only mode, you cannot check-out any element to this view.
-Effectively uses in scenario where, a view is required only to browse the code, not to perform any check-outs :).....!!!

How about providing this kind of views to managers :) ? killer isn't it

Command Explained with an example

chview

Ex:
(click the Image for Clearview)

- In the above given example, a view named "testview" has the read-write property in his pocket.
-By which, he can coolly check-out any element to have the read write copy in his workspace.

-Now the "testview" property has been changed from read-write mode to ready only mode using the chview command
- So, from now on this particular view cannot check-out any element, since it is in readonly mode.

**Apart from this property, the chview command can change the view's cache size **
** Can change the shareable do's option to non-sharable do's**
Ex

files with dot (.)checkedout extension

2009-12-07T23:51:00.007+05:30

Oops, do you get the filename with .checkedout extension and prompted with bag full of errors while check-out.

The prompted errors (unable to rename,,could not copy,,not selected by view etc) may tickle your brain to think of multiple issues.

Instead of that gulp, take a sip .....CHECK your CONFIG SPEC for the rule "element * CHECKEDOUT"

   - either the rule might not be there
   or
   - Spelling mistake in the "CHECKEDOUT" label. (funny in fact , though, it is)

By the moment you see / get a file with .checkedout extension while check-out, check your config spec for the CHECKEDOUT rule.

Example:

- Its quite evident that there is a spelling error in the first rule of the config spec.
-Having this config spec, the file "hello.c" is being checked-out and resulted with errors along with the file being copied as "hello.c.checkedout".
-To solve this issue, cancel the check-out of hello.c file using unco command and remove hello.c.checkedout which is a view private file.
- And of course, do correct the config spec with exact rule (element * CHECKEDOUT)

is it dim-witted ? relocate and mv

2009-12-02T01:20:00.002+05:30

Once I had a belief that the difference between clearcase relocate and mv command is nothing. But the fact is "NO", please find my digging below

A BIG difference

- relocate command is used to move the version control data from one vob to another vob (Across vobs)

But

- mv command is used to move the version control data within in a vob
**cleartool mv command is also highly recommended file renaming withing clearcase**

Example

relocate:
(Click the image for clear view )

- The relocate command in clearcase is used to move the version control data (Directories/files) across the vobs
-In the example, the "src" directory is being moved from one vob to the other vob (/test/demo_vob).
Condition:
- The present working directory (from where you are moving) should be checked-out and the destionation directory (another vob) should also be checked-out

mv:

(Click the Image for clearview)

- The picture shows the steps for using the clearcase mv command
-This is an recommended command to use for renaming a file in clearcase.

We always dont have time :) to enter the comments for check-in

2009-12-01T01:03:00.003+05:30

I bet you, almost 90 percent of the engineers who work on clearcase say this, "come on dude, i dont have time to enter the comment". Do you agree this ?

Even at times, i do skip from entering the comments for the events i perform in clearcase ,but take it from me, that is a sin i commit being a SCM person.

So Friends, always enter the comments for all the activities you perform,
- Let it be whether you check-out/check-in a file, create a branch type or label type.
DO ENTER the comments, which plays a active role in Traceability (One of key event in Software Configuration Management)

**That's why in many companies, they implement triggers policing you for entering comments. i.e you cant perform any events without entering comments**

Alright, i did a mistake,is there any way to enter the comments after checked-in or after event occurred. (Its never a wrong time to do the right thing).

YES, chevent command in clearcase would do that for you. cooooool, isn't it ?

#cleartool chevent

#cleartool chevent brtype:

Will walk you through an example:

(Click the Image for Clear View)

- The file hello.c have been created without entering the comments.
-By use of the chevent command, the comment is appended the file.

(Click the Image for Clear View)

- The chevent command is used to append the comment for the branchtype "test_branch" which was created without entering a comment.

Hide out a directory using config Spec

2009-11-29T03:34:00.002+05:30

Penning a Classic option to hide out the directory/files in a repository.(clearcase)

This option is very handy during a situation where certain directories needs to be masked from others accessing it. Truly NOT filling the bucket with this scenario alone , there are many scenarios where directory / file masking needs. In other ways, avoiding a directory / file to be selected in a repository.

*******************************
element * CHECKEDOUT
element -dir -none #This -none option is used to mask the directory#
element * /main/LATEST

*******************************

Would like to prove with an example:

(Click the Image for Clear view)

- There are four directory named cc,cvs,pvcs & svn in the repository
-By editing the config spec and by use of the "-none" option , the two directories namely cvs and pvcs are masked or the directories are not picked up.

Clearcase Metadata Data

2009-11-29T02:29:00.000+05:30

What are metadata ?

In my terms or i pocket as "a supporting information".

But the actual / widely considered definition is

- Metadata is set of constructs and annotations that attach to clearcase objects. It manages you to organize, manipulate and manage those objects.

- Enables you to find , list. sort and retrieve information

- Access Work faster and more directly

There are 6 Kinds of Metadata:

Elements
Branches
Labels
Attributes
Triggers
Hyperlinks

changing a branch name for specific files in clearcase

2009-11-28T01:26:00.002+05:30

Want to change the branch name for a particular / specified files , then you should NOT use the rename command, that is used to change the entire branch type and files associated to it.

Yes, have to pickup the "chtype" command for this...

chtype : Used to rename a branch for specific and also to change the type of an element.

This example reveals the steps to rename a branch name for the particular / specified file.

(Click the Image for Clear view )

- Have two files namely hello.c and hru.h. Both files have similar branch structure which is evident in the image.
- By use of this chtype command, the branch "branch_r2" is changed to "branch_r1" for the file hello.c
- Where as the file hru.h doesn't undergo any change, version tree of the same proves it :)

-Hope you had got the glimpse of the error while executing the command.
-I heard you :) , you are RIGHT !. The Vob should contain the branch type named "branch_r1", if not create and execute the command. Until then, you wont be able to rename the branch.

chtype: A example to change the type of an element

-By default clearcase has the capability to identify the file types for all the files and store in its database. (file,text_file, compressed_file, compressed_text_file)
-But you can always change the element type by using the chtype command.

Ex:

Difference between rmname and rename in clearcase

2009-11-27T00:13:00.000+05:30

rmname :

- rmname command in clearcase is used for removing the file from the version control. In other words, the file is removed from the particualr version of the directory and not from the Source.
- When a file is removed using rmname command and if the directory is checked-in, you could see the file in the previous version of the directory but not in the version where you have removed and checked-in.
Ex : (Click the image for Clear View)

The example given above depicts the same.

- A file name abc.txt is removed using rmname command.

-After removing from the version control, the directory is checked-in, quite obvious that it would create next version of the directory

- The cleartool lsvtree command shows the version tree of the present working directory and the difference between the older version and new version reveals that the file is removed in the new version.

- If you want to access the file, change the config spec to point older version of the directory and access the file.

rename:

- The rename command is simply used to rename the branch types and label types. When a branch type is renamed from branch A to branch B, the files/versions which have branch A will be renamed to branch B. This condition applies to the labels as well

Example

- The present working directory has branch named "branch_r1" and has two revisions under it.

-When the branch type is renamed (using rename command) to "branch_r2", the directories branch has been renamed too.
- Same for label types too :)

To conclude in a nutshell, the rmname command is used to remove the files from the version control, where as in rename command is used to rename the replica, vobstorage and the type objects, such as labels and branches.

Eye "find" in Unix

2009-11-24T12:28:00.001+05:30

Unix find command has many beautiful options, penning down one such beauty.
"prune" in find command

- The prune option will omit the directory /file which you dont want to display and rest of the output will be displayed

example:
#ls
test1 test2 test3 test4

-> you have four directories as named

#find .
./test1
./test2
./test3
./test4

-> if you want to omit directory named test3 and its content when you execute a find command, then use

#find . -name "test3" -prune -o -print -> this would print the below stated output
.
./test1
./test2
./test4

-> Using xargs utility in unix, you can use to multiple activities using the output

#find . -name "test3" -prune -o -print xargs rm -rf -> This is would remove all the folders except "test3" folder
:)

Zeroth Version - Clearcase

2009-11-23T19:16:00.000+05:30

In clearcase and other version control tools, zeroth version is always an empty. A files zeroth version will be always empty and the same condition applies for the directories too.

An example to create a empty file in clearcase.

-When you are intended to create a file in VOB, say under /vobs/mydirectory

-You need to checkout the directory first. REASON : You are going to make a change in the directory, which means the directory undergoes a change.Either removal of file / addition of file.

#cd /vobs/mydirectory

#cleartool co -nc .

#cleartool mkelem myfile.txt

#cleartool unco -rm myfile.txt

#cleartool ci -nc .

#cleartool lsvtree myfile.txt
myfile.txt@@/main
myfile.txt@@/main/0

The Zeroth version is always empty.