Simplest Enterprise Continuous Integration Solutions

Software Development: Practice Jenkins Pipeline as Code

2016-04-30T14:53:00.000-04:00

Practice Jenkins Pipeline as Code with Git flow, JFrog Artifactory, Azure Container Registry, GCP Container Registry.

RELEASE build (master, release branches)

PRIVATE build (feature, bugfix. hotfix branches)

Jenkinsfile (Jenkins Pipeline as Code) Development

Software Development: Git flow overview

2015-10-03T18:03:00.000-04:00

Permanent branches

master - the integration branch used for development. Feature branches are merged back into this branch
production - support preparation of a new production release (allow for minor bug fixes)

Temporary branches

features - used for specific feature work. Typically, this branches from and merges back into the development branch
releases - used for release tasks and long-term maintenance. Typically, this branches from the development branch and changes are merged back into the development branch
hotfixes - typically used to quickly fix the production branch

Salt REST API: Rest_cherrypy

2015-09-19T13:22:00.000-04:00

Install Salt-API

[root@salt-master-server ~]# uname -a
Linux salt-master-server 2.6.32-504.16.2.el6.x86_64 #1 SMP Tue Apr 21 08:37:59 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@salt-master-server ~]# yum install salt-api
Loaded plugins: security
Setting up Install Process
epel/metalink | 12 kB 00:00
epel | 4.3 kB 00:00
epel/primary_db | 5.7 MB 00:16
public_ol6_latest | 1.4 kB 00:00
public_ol6_latest/primary | 52 MB 00:12
public_ol6_latest 32176/32176
Resolving Dependencies
--> Running transaction check
---> Package salt-api.noarch 0:2015.5.3-3.el6 will be installed
--> Processing Dependency: salt-master = 2015.5.3-3.el6 for package: salt-api-2015.5.3-3.el6.noarch
http://bencane.com/2014/07/17/integrating-saltstack-with-other-services-via-salt-api/--> Processing Dependency: python-cherrypy for package: salt-api-2015.5.3-3.el6.noarch
--> Running transaction check
---> Package python-cherrypy.noarch 0:3.2.2-3.el6 will be installed
---> Package salt-master.noarch 0:2015.5.0-1.el6 will be updated
---> Package salt-master.noarch 0:2015.5.3-3.el6 will be an update
--> Processing Dependency: salt = 2015.5.3-3.el6 for package: salt-master-2015.5.3-3.el6.noarch
--> Running transaction check
---> Package salt.noarch 0:2015.5.0-1.el6 will be updated
--> Processing Dependency: salt = 2015.5.0-1.el6 for package: salt-minion-2015.5.0-1.el6.noarch
---> Package salt.noarch 0:2015.5.3-3.el6 will be an update
--> Running transaction check
---> Package salt-minion.noarch 0:2015.5.0-1.el6 will be updated
---> Package salt-minion.noarch 0:2015.5.3-3.el6 will be an update
--> Finished Dependency Resolution

Dependencies Resolved
=================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================
Installing:
salt-api noarch 2015.5.3-3.el6 epel 13 k
Installing for dependencies:
python-cherrypy noarch 3.2.2-3.el6 epel 465 k
Updating for dependencies:
salt noarch 2015.5.3-3.el6 epel 4.1 M
salt-master noarch 2015.5.3-3.el6 epel 965 k
salt-minion noarch 2015.5.3-3.el6 epel 26 k

Transaction Summary
=================================================================================================================================================
Install 2 Package(s)
Upgrade 3 Package(s)

Total download size: 5.5 M
Is this ok [y/N]: y
Is this ok [y/N]: y
Downloading Packages:
(1/5): python-cherrypy-3.2.2-3.el6.noarch.rpm | 465 kB 00:00
(2/5): salt-2015.5.3-3.el6.noarch.rpm | 4.1 MB 00:09
(3/5): salt-api-2015.5.3-3.el6.noarch.rpm | 13 kB 00:00
(4/5): salt-master-2015.5.3-3.el6.noarch.rpm | 965 kB 00:02
(5/5): salt-minion-2015.5.3-3.el6.noarch.rpm | 26 kB 00:00
-------------------------------------------------------------------------------------------------------------------------------------------------
Total 409 kB/s | 5.5 MB 00:13
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
Updating : salt-2015.5.3-3.el6.noarch 1/8
Updating : salt-master-2015.5.3-3.el6.noarch 2/8
warning: /etc/salt/master created as /etc/salt/master.rpmnew
Installing : python-cherrypy-3.2.2-3.el6.noarch 3/8
Installing : salt-api-2015.5.3-3.el6.noarch 4/8
Updating : salt-minion-2015.5.3-3.el6.noarch 5/8
warning: /etc/salt/minion created as /etc/salt/minion.rpmnew
Cleanup : salt-minion-2015.5.0-1.el6.noarch 6/8
Cleanup : salt-master-2015.5.0-1.el6.noarch 7/8
Cleanup : salt-2015.5.0-1.el6.noarch 8/8
Verifying : salt-master-2015.5.3-3.el6.noarch 1/8
Verifying : python-cherrypy-3.2.2-3.el6.noarch 2/8
Verifying : salt-2015.5.3-3.el6.noarch 3/8
Verifying : salt-api-2015.5.3-3.el6.noarch 4/8
Verifying : salt-minion-2015.5.3-3.el6.noarch 5/8
Verifying : salt-minion-2015.5.0-1.el6.noarch 6/8
Verifying : salt-2015.5.0-1.el6.noarch 7/8
Verifying : salt-master-2015.5.0-1.el6.noarch 8/8

Installed:
salt-api.noarch 0:2015.5.3-3.el6
Dependency Installed:
python-cherrypy.noarch 0:3.2.2-3.el6
Dependency Updated:
salt.noarch 0:2015.5.3-3.el6 salt-master.noarch 0:2015.5.3-3.el6 salt-minion.noarch 0:2015.5.3-3.el6

Complete!

Generate the key, sign the key and generate a certificate

[root@salt-master-server ~]# mkdir -p /etc/ssl/private

[root@salt-master-server ~]# openssl genrsa -out /etc/ssl/private/key.pem 4096

Generating RSA private key, 4096 bit long modulus

.........................++

.................................++

e is 65537 (0x10001)

[root@salt-master-server ~]# openssl req -new -x509 -key /etc/ssl/private/key.pem -out /etc/ssl/private/cert.pem -days 1826

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [XX]:<my-country-code>

State or Province Name (full name) []:<my-state-code>

Locality Name (eg, city) [Default City]:<my-city-code>

Organization Name (eg, company) [Default Company Ltd]:<my-org-name>

Organizational Unit Name (eg, section) []:<my-unit-name>

Common Name (eg, your name or your server's hostname) []:salt-master-server

Email Address []:

Basic salt-api configuration, Salt external authentication system

[root@salt-master-server salt]# diff -u master.orig master

--- master.orig 2015-09-09 15:37:28.733636701 -0400

+++ master 2015-09-09 15:44:14.764932660 -0400

@@ -264,6 +264,11 @@

# fred:

# - test.*

+external_auth:

+ pam:

+ bldmaster:

+ - .*

# Time (in seconds) for a newly generated token to live. Default: 12 hours

#token_expire: 43200

@@ -726,3 +731,9 @@

############################################

# Which returner(s) will be used for minion's result:

#return: mysql

+rest_cherrypy:

+ port: 8080

+ host: 10.3.23.160

+ ssl_crt: /etc/ssl/private/cert.pem

+ ssl_key: /etc/ssl/private/key.pem

Restart salt-master and salt-api services

[root@salt-master-server ~]# chmod 666 /var/log/salt/master

[root@salt-master-server ~]# service salt-master restart

Stopping salt-master daemon: [ OK ]

Starting salt-master daemon: [ OK ]

[root@salt-master-server ~]# service salt-api restart

Stopping salt-api daemon: [ OK ]

Starting salt-api daemon: [ OK ]

[root@salt-master-server ~]# chkconfig salt-master --list

salt-master 0:off 1:off 2:on 3:on 4:on 5:on 6:off

[root@salt-master-server ~]# chkconfig salt-api --list

salt-api 0:off 1:off 2:off 3:on 4:off 5:on 6:off

Test Salt external authentication

[bldmaster@salt-master-server ~]$ salt -a pam '*' test.ping

username: bldmaster

password:

salt-master-server:

True

salt-minion-host:

True

[bldmaster@salt-master-server ~]$ salt -T -a pam '*' test.ping

username: bldmaster

password:

salt-minion-host:

True

salt-master-server:

True

Rest_cherrypy

[root@salt-master-server ~]# curl -sSki -X POST https://salt-master-server/login -H 'Accept: application/x-yaml' -d username="bldmaster" -d password="my-password" -d eauth=pam -c ./salt-cookie.txt

HTTP/1.1 200 OK

Content-Length: 161

Access-Control-Expose-Headers: GET, POST

Vary: Accept-Encoding

Server: CherryPy/3.2.2

Allow: GET, HEAD, POST

Access-Control-Allow-Credentials: true

Date: Wed, 09 Sep 2015 20:06:40 GMT

Access-Control-Allow-Origin: *

X-Auth-Token: 6392989d7874abf2a90e89b9f850adca4bd3d53a

Content-Type: application/x-yaml

Set-Cookie: session_id=6392989d7874abf2a90e89b9f850adca4bd3d53a; expires=Thu, 10 Sep 2015 06:06:40 GMT; Path=/

return:

- eauth: pam

expire: 1441872400.749799

perms:

- .*

start: 1441829200.7497981

token: 6392989d7874abf2a90e89b9f850adca4bd3d53a

user: bldmaster

[root@salt-master-server ~]# cat ./salt-cookie.txt

# Netscape HTTP Cookie File

# http://curl.haxx.se/rfc/cookie_spec.html

# This file was generated by libcurl! Edit at your own risk.

10.3.23.160 FALSE / FALSE 1441865200 session_id 6392989d7874abf2a90e89b9f850adca4bd3d53a

Software Development: Practice Jenkins CI/CD pipelines

2015-04-19T18:51:00.000-04:00

Practice Jenkins CI/CD pipelines with git source control and saltstack deployment

Jenkins REST API: Practice within Continuous Integration

2015-01-17T14:00:00.000-05:00

Jenkins REST API launch a parameterized build

/usr/bin/curl -X POST http://<jenkins-user>:<jenkins-token>@<jenkins-server-to-build-job>/build --form 'json={"parameter":[{"name":"RELEASE_VERSION","value":"1.0.0"},{"name":"DEVELOPMENT_VERSION","value":"1.0.0-SNAPSHOT"},{"name":"JIRA_ISSUE","value":"<jira-key>"}]}'

Monitor Jenkins build status

http://<jenkins-server-to-build-job>/api/xml

http://<jenkins-server-to-build-job>/lastBuild/api/xml (Note: only valid while nextbuildnumber > 1)

Docker: Practice with Jenkins, Salt

2014-12-13T13:13:00.000-05:00

Automate generate Dockerfile within Jenkins pipeline job

# generate Dockerfile
/bin/rm -rf ${WORKSPACE}/docker
/bin/mkdir -p ${WORKSPACE}/docker

cd ${WORKSPACE}/docker
touch Dockerfile

cat <<EOF > Dockerfile
# set the base image to Centos
FROM centos:6.7

# File Author / Maintainer
MAINTAINER buildmaster <buildmaster@qxc.com>

RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

RUN yum install -y salt salt-minion

# udpate salt file_roots, and pillar_roots
RUN echo 'file_roots:' >> /etc/salt/minion
RUN echo ' base:' >> /etc/salt/minion
RUN echo ' - /srv/salt' >> /etc/salt/minion
RUN echo ' - /srv/formulas' >> /etc/salt/minion
RUN echo ' dev:' >> /etc/salt/minion
RUN echo ' - /srv/salt' >> /etc/salt/minion
RUN echo ' - /srv/formulas' >> /etc/salt/minion
RUN echo 'pillar_roots:' >> /etc/salt/minion
RUN echo ' base:' >> /etc/salt/minion
RUN echo ' - /srv/pillar' >> /etc/salt/minion
RUN echo ' dev:' >> /etc/salt/minion
RUN echo ' - /srv/pillar/qa' >> /etc/salt/minion

# use local file_client
RUN echo 'file_client: local' >> /etc/salt/minion

# define Salt role
RUN salt-call grains.setval roles "[${MY_ROLE}]"
EOF

Build docker image, consume with docker container

# build a docker image from Dockerfile
cd ${WORKSPACE}/docker

docker build -t ${DOCKER_IMAGE}.v${PIPELINE_VERSION} .

# add host directory as a data volume (which only works with at least centos6.7)
# invoke salt-call with docker container
docker run –v salt_scripts:/srv/rw -t ${DOCKER_IMAGE}.v${PIPELINE_VERSION} salt-call --local pillar.items | /usr/bin/tee ${OUTPUT}

# handle return code and clean up successful docker container
CHECK_SH=${WORKSPACE}/${PIPELINE_VERSION}.`date "+%Y%m%d%M%S"`.sh
/bin/echo "#!/bin.sh" > ${CHECK_SH}
/bin/echo "RET_CODE=0" >> ${CHECK_SH}
/bin/echo "/bin/grep -n -i error ${OUTPUT} > ${OUTPUT}.tmp" >> ${CHECK_SH}
/bin/echo "if [ ! -s ${OUTPUT}.tmp ]; then" >> ${CHECK_SH}
/bin/echo " docker rmi -f ${DOCKER_IMAGE}.v${PIPELINE_VERSION}" >> ${CHECK_SH}
/bin/echo "else" >> ${CHECK_SH}
/bin/echo " RET_CODE=1" >> ${CHECK_SH}
/bin/echo "fi" >> ${CHECK_SH}
/bin/echo "exit ${RET_CODE}" >> ${CHECK_SH}

/bin/sh ${CHECK_SH}

JIRA REST API: query JIRA issue summary

2014-10-11T12:57:00.000-04:00

Enable JIRA to accept remote API calls:

Enable Accept remote API calls (Administrator > General Configuration > Set Accept remote API calls to On)

Examples for http and https with summary field display

curl -s -u <jira-user>:<jira-password> -X GET -H "Content-Type: application/json" http://<jira-server>:9090/rest/api/2/search?jql=key=<jira-key>&fields=summary

curl -sk -u <jira-user>:<jira-password> -X GET -H "Content-Type: application/json" http://<jira-server>:8443/rest/api/2/search?jql=key=<jira-key>&fields=summary

Docker: image, repository and registry

2014-09-13T20:33:00.000-04:00

Docker Registry (version 1.0)

# OS version, docker engine version

[root@docker-registry-server ~]# uname -a
Linux docker-registry-server 2.6.32-504.16.2.el6.x86_64 #1 SMP Tue Apr 21 08:37:59 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@docker-registry-server ~]# docker -v
Docker version 1.6.2, build 7c8fca2
[root@docker-registry-server ~]# service docker status
docker (pid 1410) is running...

# docker registry IP address

[root@docker-registry-server ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:50:56:92:6F:78
inet addr:10.3.22.191 Bcast:10.3.23.255 Mask:255.255.252.0

# docker daemon process with argument

[root@docker-registry-server ~]# ps -ef | grep docker
root 3862 1 0 13:01 pts/2 00:00:00 /usr/bin/docker -d --insecure-registry 10.3.22.191:5000
root 3918 2963 0 13:01 pts/0 00:00:00 docker run -p 5000:5000 registry
root 3956 3862 0 13:01 pts/2 00:00:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 5000 -container-ip 172.17.0.1 -container-port 5000
root 3968 3862 0 13:01 ? 00:00:00 /usr/bin/python /usr/local/bin/gunicorn --access-logfile - --error-logfile - --max-requests 100 -k gevent --graceful-timeout 3600 -t 3600 -w 4 -b 0.0.0.0:5000 --reload docker_registry.wsgi:application
root 4029 3968 1 13:01 ? 00:00:00 /usr/bin/python /usr/local/bin/gunicorn --access-logfile - --error-logfile - --max-requests 100 -k gevent --graceful-timeout 3600 -t 3600 -w 4 -b 0.0.0.0:5000 --reload docker_registry.wsgi:application
root 4030 3968 1 13:01 ? 00:00:00 /usr/bin/python /usr/local/bin/gunicorn --access-logfile - --error-logfile - --max-requests 100 -k gevent --graceful-timeout 3600 -t 3600 -w 4 -b 0.0.0.0:5000 --reload docker_registry.wsgi:application
root 4031 3968 1 13:01 ? 00:00:00 /usr/bin/python /usr/local/bin/gunicorn --access-logfile - --error-logfile - --max-requests 100 -k gevent --graceful-timeout 3600 -t 3600 -w 4 -b 0.0.0.0:5000 --reload docker_registry.wsgi:application
root 4032 3968 1 13:01 ? 00:00:00 /usr/bin/python /usr/local/bin/gunicorn --access-logfile - --error-logfile - --max-requests 100 -k gevent --graceful-timeout 3600 -t 3600 -w 4 -b 0.0.0.0:5000 --reload docker_registry.wsgi:application
root 4050 2899 0 13:03 pts/2 00:00:00 grep docker

[root@docker-registry-server ~]# curl 10.3.22.191:5000
"\"docker-registry server\""[root@docker-registry-server ~]#

# docker build image myimage-app:v1 from Dockerfile

[root@docker-registry-server ~]# cd /tmp/myimage-app
[root@docker-registry-server myimage-app]# cat Dockerfile
# Set the base image to Centos
FROM centos

# File Author / Maintainer
MAINTAINER BuildMaster <buildmaster@qxc.com>

# install RPM package
RUN rpm -Uvh myiamge-app-package-1.0.0-20140524.noarch.rpm
[root@docker-registry-server myimage-app]# docker build -t myimage-app:v1 .
Sending build context to Docker daemon 2.048 kB
Sending build context to Docker daemon
Step 0 : FROM centos
---> fd44297e2ddb
Step 1 : MAINTAINER BuildMaster <buildmaster@qxc.com>
---> Using cache
---> 4eb21a2527e9
Step 2 : RUN rpm -Uvh myimage-app-package-1.0.0-20140524.noarch.rpm
---> Running in 701420d2d28a
Retrieving myimage-app-package-1.0.0-20140524.noarch.rpm
Preparing... ########################################
Updating / installing...
myimage-app-package-1.0.0-20140524########################################
---> d124c70feeee
Removing intermediate container 701420d2d28a
Successfully built d124c70feeee

[root@docker-registry-server ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
myimage-app v1 d124c70feeee 41 seconds ago 226.3 MB
registry latest 204704ce3137 11 days ago 413.8 MB
jenkins latest 1520f72eb8b6 3 weeks ago 662 MB
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# docker tag for private docker registry

[root@docker-registry-server ~]# docker tag myimage-app:v1 10.3.22.191:5000/myimage
[root@docker-registry-server ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
myimage-app v1 d124c70feeee 2 hours ago 226.3 MB
10.3.22.191:5000/myimage latest d124c70feeee 2 hours ago 226.3 MB
registry latest 204704ce3137 11 days ago 413.8 MB
jenkins latest 1520f72eb8b6 3 weeks ago 662 MB
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# docker push to private docker registry

[root@docker-registry-server ~]# docker push 10.3.22.191:5000/myimage
The push refers to a repository [10.3.22.191:5000/myimage] (len: 1)
Sending image list
Pushing repository 10.3.22.191:5000/myimage (1 tags)
6941bfcbbfca: Image successfully pushed
41459f052977: Image successfully pushed
fd44297e2ddb: Image successfully pushed
4eb21a2527e9: Image successfully pushed
d124c70feeee: Image successfully pushed
Pushing tag for rev [d124c70feeee] on {http://10.3.22.191:5000/v1/repositories/myimage/tags/latest}
[root@docker-registry-server ~]# curl http://10.3.22.191:5000/v1/repositories/myimage/tags/latest
"d124c70feeee098660422e98a515c0eabe20f9f93c6d28a593d294fcf302abb1"[root@docker-registry-server ~]#

Docker client (docker engine version could be previous version)

# OS version, docker engine version

[root@docker-client1 ~]# uname -a
Linux docker-client1 2.6.32-504.el6.x86_64 #1 SMP Tue Oct 14 01:47:47 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@docker-client1 ~]# docker -v
Docker version 1.5.0, build a8a31ef/1.5.0
[root@docker-client1 ~]# service docker status
docker (pid 27176) is running...

# docker client IP address

[root@docker-client1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:77:1A:BB
inet addr:10.3.23.142 Bcast:10.3.23.255 Mask:255.255.252.0

# docker daemon process with arguments

[root@docker-client1 ~]# ps -ef | grep docker
root 27176 1 0 23:18 pts/1 00:00:00 /usr/bin/docker -d --insecure-registry 10.3.22.191:5000
root 27326 27019 0 23:32 pts/1 00:00:00 grep docker

[root@docker-client1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# pull an image from remote private docker registry (version 1.0)

[root@docker-client1 ~]# docker pull 10.3.22.191:5000/myimage
Pulling repository 10.3.22.191:5000/myimage
d124c70feeee: Download complete
6941bfcbbfca: Download complete
41459f052977: Download complete
fd44297e2ddb: Download complete
4eb21a2527e9: Download complete
Status: Downloaded newer image for 10.3.22.191:5000/myimage:latest
[root@docker-client1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
10.3.22.191:5000/myimage latest d124c70feeee 8 hours ago 226.3 MB
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# check container contents

[root@docker-client1 ~]# docker run -t -i 10.3.22.191:5000/myimage
[root@1f5c3796e479 /]# ls -al /tmp
total 28
drwxrwxrwt 7 root root 4096 May 25 23:34 .
drwxr-xr-x 17 root root 4096 May 25 23:34 ..
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .ICE-unix
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .Test-unix
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .X11-unix
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .XIM-unix
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .font-unix
[root@1f5c3796e479 /]# rpm -qa | grep myimage
DEV-myimage-app-package-1.0.0.37-20150522.noarch
[root@1f5c3796e479 /]# exit
exit

Docker Registry (version 2.0)

# OS version, docker engine version

# docker daemon process and argument

[root@docker-registry-server ~]# ps -ef | grep docker
root 3862 1 7 13:01 pts/2 00:01:44 /usr/bin/docker -d --insecure-registry 10.3.22.191:5000
root 4217 2963 0 13:16 pts/0 00:00:00 docker run -p 5000:5000 registry:2.0
root 4625 3862 0 13:17 pts/2 00:00:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 5000 -container-ip 172.17.0.2 -container-port 5000
root 5111 2899 0 13:23 pts/2 00:00:00 grep docker

[root@docker-registry-server ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
10.3.22.191:5000/myimage latest d124c70feeee 2 hours ago 226.3 MB
myimage-app v1 d124c70feeee 2 hours ago 226.3 MB
registry 2.0 0ff65644861b 11 days ago 548.5 MB
registry latest 204704ce3137 11 days ago 413.8 MB
jenkins latest 1520f72eb8b6 3 weeks ago 662 MB
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# create a new image on a docker container

[root@docker-registry-server ~]# docker run -t -i myimage-app:v1
[root@4d0a41299372 /]# touch /tmp/aaa
[root@4d0a41299372 /]# exit
exit

# commit an image

[root@docker-registry-server ~]# docker commit -m "myimage:v2" 4d0a41299372 myimage:v2
1545e9605745217b70d0201b3768c8d364b7f5aa599525b9336d133bff2b6284
[root@docker-registry-server ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
myimage v2 1545e9605745 15 seconds ago 226.3 MB
myimage-app v1 d124c70feeee 2 hours ago 226.3 MB
10.3.22.191:5000/myimage latest d124c70feeee 2 hours ago 226.3 MB
registry 2.0 0ff65644861b 11 days ago 548.5 MB
registry latest 204704ce3137 11 days ago 413.8 MB
jenkins latest 1520f72eb8b6 3 weeks ago 662 MB
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# docker tag for private docker registry

[root@docker-registry-server ~]# docker tag -f myimage:v2 10.3.22.191:5000/myimage
[root@docker-registry-server ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
10.3.22.191:5000/myimage latest 1545e9605745 About a minute ago 226.3 MB
myimage v2 1545e9605745 About a minute ago 226.3 MB
myimage-app v1 d124c70feeee 2 hours ago 226.3 MB
registry 2.0 0ff65644861b 11 days ago 548.5 MB
registry latest 204704ce3137 11 days ago 413.8 MB
jenkins latest 1520f72eb8b6 3 weeks ago 662 MB
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# docker push to private docker registry

[root@docker-registry-server ~]# docker push 10.3.22.191:5000/myimage
The push refers to a repository [10.3.22.191:5000/myimage] (len: 1)
1545e9605745: Image already exists
d124c70feeee: Image successfully pushed
4eb21a2527e9: Image successfully pushed
fd44297e2ddb: Image successfully pushed
41459f052977: Image successfully pushed
6941bfcbbfca: Image successfully pushed
Digest: sha256:57d2e2ba8e4bef77ce22f922d126b2c4ace4bc882950609cfa75edd2a70d75af
[root@docker-registry-server ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
myimage v2 1545e9605745 3 minutes ago 226.3 MB
10.3.22.191:5000/myimage latest 1545e9605745 3 minutes ago 226.3 MB
myimage-app v1 d124c70feeee 2 hours ago 226.3 MB
registry 2.0 0ff65644861b 11 days ago 548.5 MB
registry latest 204704ce3137 11 days ago 413.8 MB
jenkins latest 1520f72eb8b6 3 weeks ago 662 MB
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

Docker client (docker 1.6 for Docker Registry 2.0)

# OS version, docker engine version

[root@docker-client2 ~]# uname -a
Linux docker-client2 2.6.32-504.16.2.el6.x86_64 #1 SMP Tue Apr 21 08:37:59 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@docker-client2 ~]# docker -v
Docker version 1.6.2, build 7c8fca2
[root@docker-client2 ~]# service docker status
docker (pid 3024) is running...

# docker client IP address

[root@docker-client2 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:21:45:93
inet addr:10.3.23.160 Bcast:10.3.23.255 Mask:255.255.252.0

# docker daemon with argument

[root@docker-client2 ~]# ps -ef | grep docker
root 3024 1 0 May24 ? 00:00:10 /usr/bin/docker -d --insecure-registry 10.3.22.191:5000
root 7548 7445 0 19:51 pts/1 00:00:00 grep docker

[root@docker-client2 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# docker pull an image from remote private docker registry (version 2.0) server

[root@docker-client2 ~]# docker pull 10.3.22.191:5000/myimage
latest: Pulling from 10.3.22.191:5000/myimage

4eb21a2527e9: Pull complete
d124c70feeee: Pull complete
1545e9605745: Already exists
6941bfcbbfca: Already exists
41459f052977: Already exists
fd44297e2ddb: Already exists
Digest: sha256:57d2e2ba8e4bef77ce22f922d126b2c4ace4bc882950609cfa75edd2a70d75af
Status: Downloaded newer image for 10.3.22.191:5000/myimage:latest
[root@docker-client2 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
10.3.22.191:5000/myimage latest 1545e9605745 6 hours ago 226.3 MB
centos latest fd44297e2ddb 4 weeks ago 215.7 MB

# check container content

[root@docker-client2 ~]# docker run -t -i 10.3.22.191:5000/myimage
[root@fc25630ffe07 /]# ls -al /tmp
total 28
drwxrwxrwt 7 root root 4096 May 25 23:53 .
drwxr-xr-x 17 root root 4096 May 25 23:53 ..
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .ICE-unix
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .Test-unix
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .X11-unix
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .XIM-unix
drwxrwxrwt 2 root root 4096 Apr 15 14:29 .font-unix
-rw-r--r-- 1 root root 0 May 25 17:19 aaa
[root@fc25630ffe07 /]# exit
exit

Enterprise Linux: Subversion Edge updates

2014-08-16T10:20:00.000-04:00

Subversion Edge comes with auto-update mechanism, which will alert the new release. Here are the Subversion Edge updates from 2.3.0 to 4.0.11.

Bamboo security: Bamboo 5 5.0 Tomcat with SSL

2014-05-03T18:30:00.000-04:00

Bamboo 5.5.0 runs on HTTP looks like

Configure Bamboo 5.5.0 runs over HTTPS

# Login as root on Bamboo Linux server.

[root@linux64-bamboo-server ~]# id

uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel)

# su to bamboo running user

[root@linux64-bamboo-server ~]# su - bamboo

# Change dir to bamboo install directory

[root@linux64-bamboo-server ~]$ cd /opt/atlassian-bamboo-5.5.0

# Generate a private key

[bamboo@linux64-bamboo-server atlassian-bamboo-5.5.0]$ $JAVA_HOME/bin/keytool -genkey -keyalg RSA -alias tomcat

Enter keystore password:

Re-enter new password:

What is your first and last name?

[Unknown]: linux64-bamboo-server.domain.com

What is the name of your organizational unit?

[Unknown]: <my unit>

What is the name of your organization?

[Unknown]: <my organization>

What is the name of your City or Locality?

[Unknown]: <my city>

What is the name of your State or Province?

[Unknown]: <my state>

What is the two-letter country code for this unit?

[Unknown]: <my country>

Is CN=linux64-bamboo-server.domain.com, OU=<my unit>, O=<my organization>., L=<my city>, ST=<my state>, C=<my country> correct?

[no]: y

Enter key password for <tomcat>

(RETURN if same as keystore password):

# Generate a CSR (Certificate Signing Request)

[bamboo@linux64-bamboo-server atlassian-bamboo-5.5.0]$ $JAVA_HOME/bin/keytool -certreq -keyalg RSA -alias tomcat -file certreq.csr

Enter keystore password:

# Submit certreq.csr to a certificate authority

Submit the generated certreq.csr to a Certificate Authority (for example, MY-ENTCASERVER) with “Web Server” Certificate template and download certificate chain (DER encoded or Base 64 encoded), save it as bamboo550_Base64.p7b.

# Import signed certificate into keystore

[bamboo@linux64-bamboo-server atlassian-bamboo-5.5.0]$ $JAVA_HOME/bin/keytool -import -alias tomcat -file bamboo550_Der.p7b –keystore ~/.keystore

# Modify server.xml as below accordingly

[root@mtl-nvc-emea08 atlassian-bamboo-5.5.0]# diff -u /opt/atlassian-bamboo-5.5.0/conf/server.xml.orig /opt/atlassian-bamboo-5.5.0/conf/server.xml

--- /opt/atlassian-bamboo-5.5.0/conf/server.xml.orig 2014-08-29 12:05:04.000000000 -0400

+++ /opt/atlassian-bamboo-5.5.0/conf/server.xml 2014-08-29 12:24:14.000000000 -0400

@@ -61,7 +61,6 @@

redirectPort="8443"

acceptCount="100"

disableUploadTimeout="true"/>

<!--

====================================================================================

@@ -127,5 +126,20 @@

pattern="%a %t "%m %U%q %H" %s %b %D "%{Referer}i" "%{User-Agent}i""/>

</Engine>

+<Connector port="8443"

+ maxHttpHeaderSize="8192"

+ SSLEnabled="true"

+ maxThreads="150"

+ minSpareThreads="25"

+ maxSpareThreads="75"

+ enableLookups="false"

+ disableUploadTimeout="true"

+ useBodyEncodingForURI="true"

+ acceptCount="100"

+ scheme="https"

+ secure="true"

+ clientAuth="false"

+ sslProtocol="TLS"

+ keystoreFile="/home/bamboo/.keystore" />

</Service>

</Server>

# Restart bamboo service

# Secutiry Bamboo 5.5.0 with Tomcat SSL runs over HTTPS looks like

# Bamboo signed own SSL certificate looks like

Enterprise Linux: Oracle Linux in-place upgrade

2014-01-04T13:04:00.000-05:00

Oracle Linux in-place upgrade from 6u3 to 6u5

# check installed OEL6.3 Red Hat compatible kernel

[root@oel6u3 ~]# uname -a

Linux oel6u3 2.6.32-279.el6.x86_64 #1 SMP Thu Jun 21 15:00:18 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

[root@oel6u3 ~]# cat /etc/Red Hat-release

Red Hat Enterprise Linux Server release 6.3 (Santiago)

# updated public yum repo to enable public_ol6_latest for OEL6.5

[root@oel6u5 ~]# cat /etc/yum.repos.d/public-yum-ol6.repo

[public_ol6_latest]

name=Oracle Linux $releasever Latest ($basearch)

baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL6/latest/$basearch/

gpgkey=http://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6

gpgcheck=1

enabled=1

# execute yum update

[root@oel6u5 ~]# yum clean all

[root@oel6u5 ~]# yum check-update

[root@oel6u5 ~]# yum update -y

# reboot OS after executed yum update to get the OEL6.5 Red Hat compatible kernel

[root@oel6u5 ~]# reboot

# upgraded to OEL6.5 Red Hat compatible kernel

[root@oel6u5 ~]# uname -a

Linux oel6u5 2.6.32-431.23.3.el6.x86_64 #1 SMP Tue Jul 29 09:20:28 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux

[root@oel6u5 ~]# cat /etc/Red Hat-release

Red Hat Enterprise Linux Server release 6.5 (Santiago)

Enterprise Linux: WebDav Subversion configuration with authentication and authorization

2013-09-14T14:58:00.000-04:00

An example about how to configure WebDav Subversion with authentication and authorization

Requirements:

1. /svn/acme/trunk has been configured with Read/Write access (authorization) for all of users under dev_eng LDAP group (authentication)
2. require to congiure an outsourcing development under /svn/acme/branches/outsourcing with only Read/Write access (authorization) for all of users under outsourcing_dev (authentication)

Configuration procedures:

1. create a new outsourcing_dev LDAP group (for authentication) with all of outsourcing developers
2. create a svn access file for outsourcing_dev Read/Write access (for authorization), /etc/httpd/conf.d/svn-access-control with contents as below
[groups]
outsourcing_dev = user1, user2

[acme_outsourcing:/branches/outsourcing]
@outsourcing_dev = rw
3. insert outsourcing_dev under /etc/httpd/conf.d/subversion.conf
# acme outsourcing repository
<Location "/svn/acme-outsourcing">
DAV svn
SVNPath /svn/repositories/acme
SVNReposName "Subversion Repository [acme-outsourcing]"
AuthzSVNAccessFile /etc/httpd/conf.d/svn-access-control

# Specify the type of authentication system to use.
AuthType Basic

# Specify the authorization realm for use in HTTP authentication.
AuthName "ACME Subversion Repository"

# Specify the authentication provider for this location.
AuthBasicProvider ldap

# Prevent other authentication modules from authenticating the user if this one fails.
AuthzLDAPAuthoritative on

# Specify the LDAP server, the base DN, the attribute to use in the search,
# as well as the extra search filter to use.
AuthLDAPURL "ldap://acme:389/OU=Corp Accounts,DC=acme,DC=com?sAMAccountName?sub?(objectClass=*)" NONE

# Specify the DN to bind with during the search phase.
AuthLDAPBindDN "CN=srv-svn-ldap,OU=No-logon-rights,OU=Generic,OU=Corp Accounts,DC=acme,DC=com"

# Specify the password to bind with during the search phase.
AuthLDAPBindPassword uCG4Q79hkG

# Require a valid user.
Require ldap-group CN=acme_outsourcing,OU=Distribution,OU=Corp Groups,DC=acme,DC=com
</Location>

4. restart Apache Web service