Arun's Blog

Upgrade, Restore Drupal 7

2013-01-08T14:55:00.000+03:00

Shell script to upgrade and restore Drupal 7 website
This script will take care of the necessary actions required for upgrading drupal to higher versions.

USAGE

Copy the script to your webserver.

Edit the script and change the variables to match with your setup

Give execute privilege to the owner of the script (chmod u+x upgrade-restore-drupal7.sh)

Execute the script ./upgrade-restore-drupal7.sh

UPGRADE

$ ./upgrade-restore-drupal7.sh 
 Please enter your choice:
 1. Update drupal
 2. Restore an old installation from backup
 3. Exit
1
Please enter the new drupal version (eg: 7.15) : 7.18
Downloading drupal-7.18
Downloaded the the drupal version drupal-7.18
Current site backup is created: /home/foo/backups/08-01-2013-0938
Database backup created: /home/foo/backups/08-01-2013-0938.sql
Site is in maintanence mode now
Removed all drupal core files from destination
Copied the new version contents
Drupal updated to drupal-7.18
Site is active again, but please update your database,

please visit http://<yourwebsite>/update.php

to finalize the process

RESTORE

$ ./upgrade-restore-drupal7.sh 
 Please enter your choice:
 1. Update drupal
 2. Restore an old installation from backup
 3. Exit
2
List of available backups
08-01-2013-0753
08-01-2013-0758
08-01-2013-0804
08-01-2013-0841
08-01-2013-0849
08-01-2013-0858
08-01-2013-0900
08-01-2013-0904
08-01-2013-0905
08-01-2013-0938
Please enter the backup file name to restore: (eg: 08-01-2013-0753): 
08-01-2013-0905
Site is offline now
Removed production files
Restored the filesystem backup 
Restored the database
Site is restored

View on github

Upgrading Linux Mint 13 (maya) to Linux Mint 14 (nadia).

2013-01-05T18:30:00.000+03:00

Take a backup of the current sources.list, preferably make a full backup of the system.

Edit the sources.list file,
replace the occurrences of maya with nadia and precise with quantal.

$ vi /etc/sources.list

:%s/maya/nadia/g
:%s/precise/quantal/g

Resulting file may look like.

deb http://packages.linuxmint.com/ nadia main upstream import
deb http://archive.ubuntu.com/ubuntu/ quantal main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ quantal-updates main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu/ quantal-security main restricted universe multiverse
deb http://archive.canonical.com/ubuntu/ quantal partner
deb http://packages.medibuntu.org/ quantal free non-free

Update the system

$ sudo apt-get update
$ sudo apt-get dist-upgrade

SAN and Tape backup with bacula

2012-12-31T17:47:00.000+03:00

Install and configure bacula for SAN and Tape backup

There is already an excellent document about bacula installation and configurations at bacula website. This article is one way of getting SAN and Tape backup working together with single bacula director installation. It assumes that you already have installed and mounted the SAN and configured the tape device.

This configuration aim at:

Incremental daily for 20 days

Differential weekly for 3 months

Monthly full for 6 months

Eject the tape to mailslot after the back and notify admin etc.

customise it based on your requirements.

The configurations are tested with HP MSL 2024 Tape library and MSA SAN array.

Bacula server setup

The configurare are done on Redhat Enterprise linux, likely similar for other Linux distros.

Create a user for backup

# useradd -d /home/backup backup

Install bacula server and create the database and database users : ref: http://www.bacula.org/5.2.x-manuals/en/main/main/Installing_Bacula.html for installation instructions.

Create the necessary directories:

# su - backup
$ mkdir -p /home/backup/bacula/var/lock/subsys
$ mkdir /home/backup/bacula/var/run/

Configure the director (bacula-dir.conf)

$ cat ~/bacula-dir.conf

# Define the director, common for SAN and Tape
Director { # define myself
Name = {hostname}-dir # use your hostname
DIRport = 9101 # where we listen for UA connections
QueryFile = "/home/backup/bacula/script/query.sql"
WorkingDirectory = "/home/backup/bacula/wdir"
PidDirectory = "/home/backup/bacula/var/run"
Maximum Concurrent Jobs = 3
Password = "{console_password} # Console password
Messages = Daemon
}

# List of files to be backed up to SAN
FileSet {
 Name = "File Set"
 Include {
 Options {
 signature = MD5
 }
 File = /
 }

 Exclude {
 File = /proc
 File = /tmp
 File = /.journal
 File = /.fsck
 }
}

# List of files to be backed up to tape
FileSet {
 Name = "tape Set"
 Include {
 Options {
 signature = MD5
 }
 File = /
 }

 Exclude {
 File = /proc
 File = /tmp
 File = /.journal
 File = /.fsck
 }
}

# Schedule for SAN backup
Schedule {
 Name = "WeeklyCycle"
 Run = Full 1st sun at 01:00
 Run = Differential 2nd-5th sun at 01:00
 Run = Incremental mon-sat at 01:00
}

# Schedule for tape backup
Schedule {
 Name = "TapeWeeklyFull"
 Run = Level=Full 1st sun at 03:00
}

# Definition of file storage (SAN)
Storage {
 Name = File
# Do not use "localhost" here
 Address = {FQDN} # N.B. Use a fully qualified name here
 SDPort = 9103
 Password = "{sdpassword}"
 Device = FileStorage
 Media Type = File
}

# Define storage (Tape)
Storage {
 Name = msl2024
 Address = {director-address}
 SDPort = 9103
 Password = "{director-password}"
 Device = MSL2024
 Media Type = LTO-4
 Autochanger = yes
 Maximum Concurrent Jobs = 3
}

# Generic catalog service
Catalog {
 Name = MyCatalog
 dbname = "dbname"; dbuser = "dbuser"; dbpassword = "dbpass"
}

# Tape catalog
Job {
 Name = "TapeBackupCatalog"
 JobDefs = "{dir-host-name}-tape"
 Level = Full
 FileSet="Catalog"
 Schedule = "CatalogAfterTapeBackup"
 RunBeforeJob = "/home/backup/bacula/script/make_catalog_backup.pl MyCatalog"
 RunAfterJob = "/home/backup/bacula/script/delete_catalog_backup"
 Write Bootstrap = "/home/backup/bacula/wdir/%n.bsr"
 Priority = 20 # run after main backup
}

# Default pool definition
Pool { 
 Name = Default
 Pool Type = Backup 
 Recycle = yes # Bacula can automatically recycle Volumes
 AutoPrune = yes # Prune expired volumes
 Volume Retention = 365 days # one year
}

# General Tape backup pool
Pool { 
 Name = TapePool
 Pool Type = Backup 
 Recycle = yes # Bacula can automatically recycle Volumes
 AutoPrune = yes # Prune expired volumes
 Volume Retention = 6 months # 6 months
 Recycle Oldest Volume = yes
 Storage = msl2024 
 Volume Use Duration = 4 days
}

## Do the following configurations for each client

# Job definition, define it for each bacula client, replace clientX_hostname, Fileset accordingly
# SAN
JobDefs {
Name = "{clientX_hostname}"
Type = Backup
Client = {clientX_hostname}-fd
FileSet = "File Set"
Schedule = "WeeklyCycle"
Storage = File
Messages = Standard
Pool = File
Full Backup Pool = Full-Pool-{clientX_hostname}
Incremental Backup Pool = Inc-Pool-{clientX_hostname}
Differential Backup Pool = Diff-Pool-{clientX_hostname}
Priority = 10
Write Bootstrap = "/home/backup/bacula/wdir/%c.bsr"
}

# Tape

JobDefs {

 Name = "{clientX_hostname}-tape"
 Type = Backup
 Client = {clientX_hostname}-tape-fd
 FileSet = "tape set"
 Schedule = "TapeWeeklyFull"
 Storage = msl2024
 Messages = Standard
 Pool = TapePool
 Full Backup Pool = TapePool
 Priority = 10
 Write Bootstrap = "/home/backup/bacula/wdir/%c.bsr"
}

# Define Job, replace clientX_hostname
# SAN
Job {
 Name = "{clientX_hostname}"
 JobDefs = "{clientX_hostname}"
}

# Tape
Job {
 Name = "{clientX_hostname}"
 JobDefs = "{clientX_hostname}-tape"
}

# Define restore job
# SAN
Job {
 Name = "RestoreFiles-{clientX_hostname}"
 Type = Restore
 Client={clientX_hostname}-fd
 FileSet="File Set" 
 Storage = File
 Pool = Default
 Messages = Standard
 Where = /home/backup/archive/bacula-restores
}

# Tape
Job {

 Name = "RestoreFiles-{clientX_hostname}-tape"
 Type = Restore
 Client={clientX_hostname}-tape-fd
 FileSet= "tape set"
 Storage = msl2024
 Pool = TapePool
 Messages = Standard
 Where = /home/backup/archive/bacula-restores
}

# Client (File Services) to backup
# SAN
Client { 
 Name = {clientX_hostname}-fd
 Address = {client_address}
 FDPort = 9102
 Catalog = MyCatalog
 Password = "{client_password}" # password for FileDaemon
 File Retention = 60 days # 60 days
 Job Retention = 6 months # six months
 AutoPrune = yes # Prune expired Jobs/Files
}

# Tape
Client {
 Name = {clientX_hostname}-tape-fd
 Address = {client_address}
 FDPort = 9202 # use different port
 Catalog = MyCatalog
 Password = "{client_password}" # password for FileDaemon
 File Retention = 6 months
 Job Retention = 6 months
 AutoPrune = yes
}

# Pool for each client
# SAN
Pool {
 Name = Full-Pool-{clientX_hostname}
 Pool Type = Backup
 Recycle = yes
 AutoPrune = yes
 Volume Retention = 6 months
 Maximum Volume Jobs = 1
 Label Format = Full-Pool-{clientX_hostname}-
 Maximum Volumes = 9
}

Pool { 
 Name = Inc-Pool-{clientX_hostname}
 Pool Type = Backup 
 Recycle = yes # automatically recycle Volumes
 AutoPrune = yes # Prune expired volumes
 Volume Retention = 20 days
 Maximum Volume Jobs = 6
 Label Format = Inc-Pool-{clientX_hostname}-
 Maximum Volumes = 7
}

Pool { 
 Name = Diff-Pool-{clientX_hostname}
 Pool Type = Backup
 Recycle = yes
 AutoPrune = yes
 Volume Retention = 40 days
 Maximum Volume Jobs = 1
 Label Format = Diff-Pool-{clientX_hostname}-
 Maximum Volumes = 10
}

# Tape, no extra definition required.

Make sure you label the tape and add it to the TapePool, if you tape drive has barcode device available, use

$ bconsole

* label barcode

then select the TapePool

If you have mailslot enabled you could configure the bacula to eject the tape to mailslot after backup finished and will notify.

$ cat /home/backup/bacula/script/delete_catalog_backup
# Unload the tape for storage
mtx -f /dev/sg1 unload 24 # replace 24 with your mailslot

# Send mail
/home/backup/bacula/script/mail.sh | mail -s "Tape backup done" admin@example.com

Configure storage daemon

Storage 
{ # definition of myself
 Name = {director_hostanme}-sd
 SDPort = 9103 # Director's port 
 WorkingDirectory = "/home/backup/bacula/wdir"
 Pid Directory = "/home/backup/bacula/var/run"
 Maximum Concurrent Jobs = 20
}

#
# List Directors who are permitted to contact Storage daemon
#
Director {
 Name = {director_hostname}-dir
 Password = "{director_password}"
}

# SAN
Device {
 Name = FileStorage
 Media Type = File
 Archive Device = /media/san/bacula/ # SAN volume
 LabelMedia = yes # lets Bacula label unlabeled media
 Random Access = yes
 AutomaticMount = yes # when device opened, read it
 RemovableMedia = no
 AlwaysOpen = no
}

# Tape
Autochanger {
 Name = MSL2024
 Device = lto4drive
 Changer Command = "/home/backup/bacula/script/mtx-changer %c %o %S %a %d"
 Changer Device = /dev/sg1 # change it based on your setup
}

Device {
 Name = lto4drive
 Drive Index = 0
 Media Type = LTO-4
 Archive Device = /dev/nst0
 AutomaticMount = no # when device opened, read it
 AlwaysOpen = no
 RemovableMedia = yes
 RandomAccess = no
 AutoChanger = yes
}

Client configuration

Install the bacula package on the client machines, except use --enable-client-only

Remove the director and storage daemon startup scripts

rm /etc/init.d/bacula-dir
rm /etc/init.d/bacula-sd

Create necessary directories

mkdir -p /home/backup/bacula/wdir /home/backup/bacula/var/run  /home/backup/bacula/var/lock/subsys/

Create bacula-filedeamon configuration for tape and san seperately

SAN (bacula-fd.conf)

FileDaemon { # this is me
 Name = {clientX_hostname}-fd
 FDport = 9102 # where we listen for the director
 WorkingDirectory = /home/backup/bacula/wdir
 Pid Directory = /home/backup/bacula/var/run
 Maximum Concurrent Jobs = 20
 }

Tape (bacula-fd-tape.conf)

FileDaemon { # this is me
 Name = {clientX_hostname}-tape-fd
 FDport = 9102 # different port than the san
 WorkingDirectory = /home/backup/bacula/wdir
 Pid Directory = /home/backup/bacula/var/run
 Maximum Concurrent Jobs = 20
 }

Edit the bacula-fd startup script and add the extra line to start the tape file daemon

daemon /home/backup/bacula/sbin/bacula-fd $2 ${FD_OPTIONS} -c /home/backup/bacula/etc/bacula-fd-tape.conf

./arun

Host group based access restriction - Nagios

2012-10-03T13:35:00.000+03:00

This is useful especially when you have different host groups belongs to different entities and you need to have access separation.

The basic idea is to use the same login user name in the contact groups. I assume that you have Apache htaccess authentication or LDAP authentication in place.

You may create new contact group of use the already existing one , just make sure your username and contact_name matches.

- Create a contact group
define contactgroup {

 contactgroup_name customer1
 alias Customer1 Servers
 members customer1
}

- Create the contact
define contact {
 contact_name customer1 #make sure this matches with the username
 alias Customer1 Contact
 service_notification_period 24x7
 host_notifications_enabled 0
 host_notification_period 24x7
 service_notification_options w,u,c,r
 host_notification_options d,u,r
 service_notification_commands notify-by-email
 host_notification_commands host-notify-by-email
 email customer1@example.com
}

- Use this contact group in host definition

define host {
 use generic-alerted-host
 host_name customer1-host
 address 8.8.8.8
 contact_groups customer1 # make sure this matches with the contactgroup_name
 max_check_attempts 3
}

Just restart nagios and try to login with the new user account. You may give more privileges to this user if required from cgi.cfg

./run

Delete mailman archieves

2012-04-09T22:07:00.000+03:00

If required take backup of the archives

$ cp -a /var/lib/mailman/archives/private/<listname>/* <backup_directory>

Remove the archives

$ rm -rf <listname>/*

Once it is removed recreate the html archive files

$ mailman/bin/arch <listname>

./arun

Detected bug in an extension! Hook FCKeditor_MediaWiki

2012-04-09T21:58:00.001+03:00

Detected bug in an extension! Hook FCKeditor_MediaWiki::onCustomEditor failed to return a value; should return true to continue hook processing or false to abort.

Backtrace:

#0 mediawiki/includes/Wiki.php(497): wfRunHooks('CustomEditor', Array)
 #1 mediawiki/includes/Wiki.php(63): MediaWiki->performAction(Object(OutputPage), Object(Article), 
Object(Title), Object(User), Object(WebRequest))
 #2 mediawiki/index.php(114): MediaWiki->initialize(Object(Title), Object(Article), Object(OutputPage), 
Object(User), Object(WebRequest))
 #3 {main}

Edit the following file to fix this issue:

"FCKeditor/FCKeditor.body.php"
 -- public function onCustomEditor(&$article, &$user) {
 ++ public function onCustomEditor($article, $user) {

reference: http://dev.ckeditor.com/ticket/3530
./arun

svn: Can't convert string from 'UTF-8' to native encoding:

2012-04-09T21:53:00.001+03:00

"svn: Can't convert string from 'UTF-8' to native encoding:"

This usually happens with special characters in the file name, which the client cannot understand.

Just set proper locale in the client to fix this issues,

$ export LC_CTYPE=en_US.UTF-8
// make sure the locale is properly set.
$ locale
LC_CTYPE=en_US.UTF-8

./arun

Fix categories and tags in wordpress custom post_type

2011-12-13T18:24:00.001+03:00

By default word press does not look in to custom post_types for categories and tags, even though the category names are visible you get a NOT FOUND page when you click on the category.

A work around found for this issue is :

Edit : functions.php

add_filter('pre_get_posts', 'query_post_type');
function query_post_type($query) {
if(is_category() || is_tag()) {
$post_type = get_query_var('post_type');
if($post_type)
$post_type = $post_type;
else
$post_type = array('post','custom_post_type_name','nav_menu_item'); // replace custom_post_type_name with your post_type, and keep nav_menu_item to display menu in category page.
$query->set('post_type',$post_type);
return $query;
}
}

Reference: http://wordpress.org/support/topic/custom-post-type-tagscategories-archive-page

Thanks to paranoid for guiding to the fix . ;)

./arun

Replace broken hard drive in software RAID1

2011-12-05T17:51:00.001+03:00

This scenario assumes that you have two hard disk with RAID1 setup and one of them is broken (say sdb).

To check the status of RAID:

$ cat /proc/mdstat

Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md2 : active raid1 sda3[1]
730202368 blocks [2/1] [U_]
md1 : active raid1 sda2[1]
264960 blocks [2/1] [U_]
md0 : active (auto-read-only) raid1 sda1[1]
2102464 blocks [2/1] [U_]

you will see [_U] or [U_] if there is a broken RAID.

If required remove the broken hardrive from RAID from all md devices.

# mdadm --manage /dev/md0 --fail /dev/sdb1

# mdadm --manage /dev/md1 --fail /dev/sdb2

# mdadm --manage /dev/md2 --fail /dev/sdb3

Shutdown the machine and replace the hard drive.

Once the server is booted, you will see the new device (either sda or sdb depends on what drive is broken)

# ls -l /dev/sd*

Now we need to replicate the partition schema on the new drive.

sfdisk -d /dev/sda | sfdisk /dev/sdb

// -d Dump the partitions of a device

We can add the partition to the RAID now, you could verify the partitions with fdisk -l.

# mdadm --manage /dev/md0 --add /dev/sdb1

# mdadm --manage /dev/md1 --add /dev/sdb2

# mdadm --manage /dev/md2 --add /dev/sdb3

It will start sync the data and will be ready once completed.

You may verify the mdstat

# cat /proc/mdstat
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md2 : active raid1 sda3[0] sdb3[1]
7302023 blocks [2/2] [UU]

md1 : active raid1 sda2[0] sdb2[1]
2649 blocks [2/2] [UU]

md0 : active (auto-read-only) raid1 sda1[0] sdb1[1]
21024 blocks [2/2] [UU]

./arun

Enable IPv6 on Direct Admin

2011-11-14T16:11:00.001+03:00

It was rather easy to get IPv6 working with DA if you have the IPv6 subnet allocated for your server.

Make sure that you have IPv6 enabled on your DA.

# grep ipv6 /usr/local/directadmin/conf/directadmin.conf
ipv6=1

Add the IPv6 Addresses to your direct admin, through IP Management (This will add IPv6 address to the interface).
Enter the IPv6 Address in IP field and keep the Netmask as 255.255.255.0

Add IPv6 address for your name servers,
go to DNS administration --> Select your name server domain --> Add AAAA records for your Name Servers. Make sure you have added the IPv6 addresses already to DA.

Check your name server is resolving/reachable via IPv6.

Now to add IPv6 Address to hosted domains, select the domain --> Modify user --> select the IPv6 address in "Add Additional IP". If the IPv6 address is not visible, make sure it is added to DA and to the reseller account you are editing the domain.

Add the IPv6 address(AAAA) to the corresponding domains DNS configuration.

Here we go, ping666666

./arun

Extending LVM disk space

2011-10-02T18:56:00.001+03:00

Add the new disk drive to the system, you need to reboot the machine and configure the hardware RAID if required.

Add the new disk to Volume group

For e.g.: if the disk is HP with Smart array

# pvcreate /dev/cciss/c0d1
# vgextend <volume_group_name> /dev/cciss/c0d1

Extend the Logical volume:
# lvextend -L<+mention_the_size> /dev/<volume_group>/<logical_volume>
eg: # lvextend -L+25G /dev/localhost/var
# lvextend -L+10G /dev/localhost/home

Resize the file system:
# resizefs <file_system>
eg:
# resizefs /dev/mapper/localhost-var
# resizefs /dev/mapper/localhost-home

./arun

Install and configure rsnapshot for central backup (without root privilege)

2011-08-22T22:28:00.001+03:00

Download and install RsnapShot
Download the latest package from: http://rsnapshot.org/downloads.html

# wget http://rsnapshot.org/downloads/rsnapshot-1.3.1-1.noarch.rpm
# rpm -Uvh rsnapshot-1.3.1-1.noarch.rpm

Configure public key authentication

- Enable public key authentication with remote hosts with normal user privilege

local# ssh-keygen -t rsa
local# scp id_rsa.pub ssh-remote-server:id_rsa.pub

remote# useradd -c "Backup user" -d /data/home/backup/ backup
remote# su - backup

remote# vi .ssh/authorized_keys

remote# chmod 600 .ssh/authorized_keys

remote# cat id_rsa.pub >> authorized_keys ; rm id_rsa.pub

Add the command allowed to execute in the authorized_keys

command="/home/backup/validate-rsync.sh"

Create the /home/backup/validate-rsync.sh script with following contents

#!/bin/sh
case "$SSH_ORIGINAL_COMMAND" in
*\&*)
    echo "Rejected 1"
    ;;
*\;*)
    echo "Rejected 2"
    ;;
    rsync*)
    $SSH_ORIGINAL_COMMAND
    ;;
*true*)
    echo $SSH_ORIGINAL_COMMAND
    ;;
*)
    echo "Rejected 3"
    ;;
esac

$ chmod 700 validate-rsync.sh

Create the rsync wrapper script

$ cat > /usr/local/bin/rsync_wrapper.sh

#!/bin/sh
/usr/bin/sudo /usr/bin/rsync "$@";

# chmod 755 /usr/local/bin/rsync_wrapper.sh

This steps will basically force the ssh connection to execute the rsync as sudo

Grant user to execute rsync as root

backup ALL=(root) NOPASSWD: /usr/bin/rsync

Configure Rsnapshot

master# cp /etc/rsnapshot.conf.default /etc/rsnapshot.conf

Configure path for cp, rsync, ssh, logger, du etc

set link_dest = 1

change rsync_long_args like

rsync_long_args --rsync-path=rsync_wrapper.sh --delete --numeric-ids --relative --delete-excluded

If you require daily backup for a week,

interval daily 7

More details are on the how to section for rsnapshot website
http://rsnapshot.org/howto/1.2/rsnapshot-HOWTO.en.html

Configure the hosts and file system to backup

backup backup@remotehost:/etc/ remotehost/

./arun

Upgrading php to 5.2 or 5.3 in Redhat EL 5

2011-08-14T19:22:00.001+03:00

Unfortunately RHEL 5 does not have php.5.2 package, which is required by most of the applications including latest wordpress and drupal.

First thought of compiling php from source, but hard to keep it uptodate. So decided to make the life easier with EPEL/IUS repositories.

Remove all existing php related packages:

# rpm -e php php-mysql php-cli php-pdo php-common

Download and install the EPEL/IUS RPMs

# wget http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/i386/epel-release-1-1.ius.el5.noarch.rpm

# wget http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/i386/ius-release-1.0-8.ius.el5.noarch.rpm

incase if the list not working just browse and find the rpm.

Install the RPMs

# rpm -Uvh *-release-*.rpm

Now you can install php 5.2 or 5.3 like:

# yum install php52 php52-mysql

./arun

svn over ssh tunnel

2011-06-20T01:57:00.001+03:00

It is very often required that you need to commit/update to the svn repository which is only indirectly accessible through a gateway (user can ssh to gateway and gateway can ssh to internal svn server)

Suppose you have a working copy (locally on your machine) setup with the real svn url (eg : svn+ssh://svn.example.com)

- Make ssh connection with local port forwarding to the gateway server

# sudo ssh gateway.example.com -L 22:svn.example.com:22

Change the repository url to localhost, since the local host connection forward to remote svn server through the gateway.

#cd <local_svn_path>

# svn switch --relocate svn+ssh://svn.example.com/trunk svn+ssh://localhost/trunk

Now you should be able to update, commit, etc to/from your repository.

You can switch it back to the original url when you have direct access to repository.

./arun

Install directadmin on Debian 6

2011-06-14T19:21:00.001+03:00

Install the basic debian server without any server components

Install the dependency packages:

# sudo apt-get install gcc g++ make flex bison openssl libssl-dev perl perl-base perl-modules libperl-dev libaio1 libaio-dev bind9

Purchase and register the server ip at directadmin.com

Once the license is activated, login to the server:

$ wget http://www.directadmin.com/setup.sh

$ chmod 755 setup.sh

# sudo ./setup.sh

// enter the client id, license id, FQDN etc, and follow the instructions.

Once the installation completed the direct admin interface will be accessible at https://< server_ip >: 2222/

./arun

Racktable, Apache+LDAP authentication

2011-06-14T18:59:00.001+03:00

Login to the Rack tables as admin:

Add the following line under configuration--> permission

allow {$tab_default}

* This is for read only account, assign extra permissions if required

Configure Apache + LDAP

< Directory /var/www/racktables > Options +Indexes FollowSymLinks MultiViews DirectoryIndex index.php AuthName "Rack Tables" AuthType Basic AuthBasicProvider ldap AuthzLDAPAuthoritative on AuthLDAPURL "ldaps://ldaphost.com/dc=company,dc=com?uid?sub?(objectClass=< depends_on_ldap >)"

# Bind if required AuthLDAPBindDN "uid=userid,ou=people,dc=company,dc=com" AuthLDAPBindPassword "xxxxxx" AuthLDAPGroupAttribute uniqueMember AuthLDAPGroupAttributeIsDN on require ldap-group cn=group_name,dc=company,dc=com require ldap-attribute cn=group-name-allowed < /Directory >

* Most of LDAP configs based on your setup

Configure Rack Tables:

Edit the inc/secret.php

Set :

$user_auth_src = 'httpd'; $require_local_account = FALSE;

NOTE: to get the logout working properly make sure the Apache AuthName matches with the one configured for Rack tables authentication.

Referene : http://sourceforge.net/apps/mediawiki/racktables/index.php?title=RackTablesAdminGuide

Reference

./arun

Rsnapshot Lchown

2011-06-06T13:28:00.001+03:00

# rsnapshot du localhost
require Lchown
Lchown module not found

Install the Lchown module:

# wget http://www.rsnapshot.org/downloads/extras/Lchown-1.00.tar.gz

# tar xvzf Lchown-1.00.tar.gz

# cd Lchown-1.00

# perl Makefile.PL
Checking if your kit is complete...
Looks good

# make install

# rsnapshot du localhost
require Lchown
Lchown module loaded successfully

You can also try installing the module from Perl CPAN

# perl -MCPAN -e 'install qw(Lchown)'

./arun

My Favourite MAC OS X applications

2011-06-05T00:29:00.001+03:00

Text Editor

vi

Text Edit

Smultron: http://www.peterborgapps.com/smultron/

Komodo Edit: http://www.activestate.com/komodo-edit

Voice/Video Calls

Skype : http://skype.com

Google talk (browser plugin): : http://www.google.com/chat/video

G+ hangout

Web Browser

Firefox: http://www.mozilla.com/en-US/firefox/fx/

Chrome: http://www.google.com/chrome

Twitter Client

A Plus: http://www.aplus-app.com/

Adium: http://adium.im/

Office Documents

Libre office: http://www.libreoffice.org/

Video Player

VLC: http://www.videolan.org/vlc/

Miro: http://www.getmiro.com/ ( an excellent stream and bit torrent client as well)

VOIP

Express Talk(free edition) : http://www.nch.com.au/talk/talkmaci.zip

Virtulization

VMware Fusion : http://www.vmware.com/products/fusion/overview.html

Sun Virtual Box (free): http://www.virtualbox.org/

Parellels: http://www.parallels.com/

PDF viewer

Adobe Reader: http://get.adobe.com/reader/

Audio Editing

Audacity: http://audacity.sourceforge.net/

Disk burning

Burn: http://burn-osx.sourceforge.net/

Disk Image backup

Carbon copy cloner: http://www.bombich.com/

VNC client

Chicken of the VNC: http://sourceforge.net/projects/cotvnc/

IPSEC VPN

Cisco Any connect

Subversion

Command line as always

Cornerstone, good if you require an interface: http://www.zennaware.com/cornerstone/index.php

Image Manipulation

GIMP: http://www.gimp.org/

Picasa: http://picasa.google.com/mac/

Xee: http://wakaba.c3.cx/s/apps/xee

Password Management

KeePassX: http://www.keepassx.org/

GPG Key Chain

MAC GPG: http://macgpg.sourceforge.net/

Terminal

iTerm: http://iterm.sourceforge.net/

The default terminal is really nice in Lion

Mail

Thunderbird: http://www.mozillamessaging.com/en-US/thunderbird/

Diagram

OmniGraffle: http://www.omnigroup.com/products/omnigraffle/

Packet Analysis

wireshark: http://www.wireshark.org/

./arun

Install Cpanel on FreeBSD 8.2

2011-05-29T00:51:00.001+03:00

- Install FreeBSD with proper network and file system configuration (Ref: http://docs.cpanel.net/twiki/bin/view/AllDocumentation/InstallationGuide/Quick-StartInstallationGuide)

- Install dependency packages:

# pkg_add -r wget

# pkg_add -r perl

# pkg_add -r rsync (required later for ports sync)

# pkg_add -r gmake

To Fix:

creating glibconfig.h
config.status: executing default commands
gmake: not found
child exited with value 127
Died at /usr/local/cpanel/bin/rrdtoolinstall line 109.

# pkg_add -r png // To fix the following error

To Fix : configure: error: requested PNG backend could not be enabled

- Create the following symlinks

# ln -s /usr/local/bin/wget /usr/bin/wget

# ln -s /lib/libcrypt.so.5 /lib/libcrypt.so.2 // To Fix: Shared object "libcrypt.so.2" not found,

# ln -s /lib/libc.so.7 /lib/libc.so.5 // To Fix: Shared object "libc.so.5" not found

ln -s /lib/libm.so.5 /lib/libm.so.3 // To Fix: "libm.so.3" not found

- Install Cpanel

cd /home

wget -N http://httpupdate.cpanel.net/latest

sh latest

- Once installation successful activate the license (make sure the ip is licensed - http://www.cpanel.net/apps/verify/) :

# /usr/local/cpanel/cpkeyclt

- Start Cpanel

- Touch the following file

# touch /etc/rc.d/init.d/function // To fix: Could not find functions file, your system may be broken

# /etc/init.d/cpanel start

Now you should be able to access Cpanel at https://< yourip >:2087/

Try to upgrade

Exim: /scripts/eximup --force (this will get the free bsd ports as well)
Cpanel : # /scripts/upcp

./arun

Drupal 7 issue with SQL Mode TRADITIONAL

2011-05-22T12:43:00.001+03:00

PDOException: SQLSTATE[42000]: Syntax error or access violation: 1231 Variable 'sql_mode' can't be set to the value of 'TRADITIONAL' in lock_may_be_available() (line 165 of /includes/lock.inc).

This was the case when I installed Drupal 7 with Cpanel/Fantastico, the drupal site was displaying the above error.

This issue is discussed at drupal issues . try to patch it as mentioned in the url.

But for me it works with the following change, just removed the TRADITIONAL mode, not sure it is the correct way to fix it. You can verify the sql modes at http://dev.mysql.com/doc/refman/5.0/en/server-sql-mode.html , Anyway now there is no errors in drupal site and I am able to login.

(includes/database/mysql/database.inc) Line: 65

New file
< $this->exec("SET sql_mode='ANSI,ONLY_FULL_GROUP_BY'");
---
Old file
> $this->exec("SET sql_mode='ANSI,TRADITIONAL'");

Also setting up the sql connection mode to SET SESSION sql_mode = "ANSI,TRADITIONAL"; is an option instead of above change.

./arun

Install and configure RSA web agent with Redhat EL5 and Apache

2011-05-15T17:21:00.001+03:00

Login to RSA interface:

- Create the apache server as agent host with type web agent
- Generate the config file (zip file containing sdconf.rec) from RSA interface, and download to your local machine

Login to the web server

- Download the RSA web agent installation files from RSA website.

# mkdir -p /var/ace

- Copy and extract (sdconf.rec) the downloaded config file
# chmod 755 sdconf.rec

- Create the sdopts.rec file with the ip address of the machine, if you have multiple ip address assigned to the server or if the RSA we agent is a virtual machine. Otherwise the authentication might break with following kind of errors
"100: Access denied. The RSA ACE/Server rejected the Passcode. Please try again." , "attempted to authenticate using authenticator “SecurID_Native”. The user belongs to security domain “SystemDomain”"

# echo "CLIENT_IP=" > sdopts.rec
# chown -R webuser:webuser /var/ace

- Now install the RSA web agent

# tar xvf WebAgent_70_Apache_RHEL5_32_379_08201024.tar
# cd CD/
# chmod u+x install
# ./install

- Enter the location of sdconf.rec

- Configure the apache virtual host

It was found that web agent breaks if apache starts multiple server thread, so would be better to limit the thread.

< IfModule prefork.c >
StartServers       1
MinSpareServers    1
MaxSpareServers    1
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
< /IfModule >

- Now start apache and you will be able to access the RSA web interface.

Once authenticated, the RSA server will create a node secret for the agent host and will be copied automatically to the web server.

This web interface is mainly useful for the token users to reset or enable to token assigned to him.

./arun

Changing RSA passwords

2011-05-04T19:11:00.001+03:00

Reset SSH Password:
-------------------
Log in to Operations console (Master):

Administration --> Networking --> Configure connectivity using SSH

Do the same procedure on Replica also (if any), it is manual process.

Reset Operations console password:
----------------------------------
Login to the master appliance with SSH

$ sudo -s // sudo as root
# su - rsaadmin // change user to rsaadmin
$ cd /usr/local/RSASecurity/RSAAuthenticationManager/ //navigate to RSA installation path
$ cd utils
$ ./rsautil manage-oc-administrator -a update // Execute

// Enter Super administrators name
// Enter Super administrators password
// Enter operation console user
// Set the new password

Do the same steps on replica
Error: This operation can be performed only on a primary instance.
But the password got changed

Reset Master password:
---------------------
Login to the master appliance with SSH

$ sudo -s // sudo as root
# su - rsaadmin // change user to rsaadmin
$ cd /usr/local/RSASecurity/RSAAuthenticationManager/ //navigate to RSA installation path
$ cd utils
$ ./rsautil manage-secrets --action change --new-password <enter_the_new_password>

Do the same procedure on the replica device.

Reset Security Console Password:
---------------------------------
Login to Security Console

Identity-->Users-->Manage Existing // Search for your admin username in internal datase

Change password for the admin user
This action will be automatically replicated to replica.

./arun

Configure IPv6 squid

2011-03-01T14:01:00.001+03:00

Squid 3.1+ support IPv6

- Compile and install 3.1.x version of squid.

# useradd -d /opt/squid squid

# wget http://www.squid-cache.org/Versions/v3/3.1/squid-3.1.11-20110228.tar.gz

# tar xvzf squid-3.1.11-20110228.tar.gz

# cd squid-3.1.11-20110228; ./configure --prefix=/opt/squid/

# make; make install

# /opt/squid/sbin/squid -z

Configure squid

edit /opt/squid/etc/squid.conf

cache_effective_user squid

acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

acl to_ipv6 dst ipv6
http_access allow to_ipv6 !all
tcp_outgoing_address 2001:: to_ipv6
tcp_outgoing_address 12.34.56.78 !to_ipv6

And add necessary acls to allow access for your network to squid.

Testing:

$ telnet <v6_address> 3128
Trying 2001::...
Connected to 2001::..
Escape character is '^]'.
get http://ipv6.google.com/
HTTP/1.0 200 OK

Check the access.log to make sure

1298965141.719 477 2001:: TCP_MISS/200 25918 GET http://ipv6.google.com/ - DIRECT/2a00:1450:8002::6a text/html

Performance issues with KVM - Redhat

2011-02-22T22:26:00.001+03:00

The general performance issue with KVM is due to DISK I/O

- by default the Redhat KVM guest are created with x86_64 architecuture, if you installed 32 bit operating system change this to i686.

<os>
<type arch='i686' machine='rhel5.6.0'>hvm</type>
<boot dev='hd'/>

- Make sure the hypervisor used is correct in the configuration , either qemu or kvm

<domain type='kvm'>

or

<domain type='kvm'>

- Use virtio drivers if the guest is paravirtulized (http://www.ibm.com/developerworks/linux/library/l-virtio/index.html?ca=dgr-lnxw97Viriodth-LX&S_TACT=105AGX59&S_CMP=grlnxw97 , http://publib.boulder.ibm.com/infocenter/lnxinfo/v3r0m0/index.jsp?topic=/liaat/liaatbpparavirt.htm)

./arun

yum update, IndexError: tuple index out of range

2011-02-07T17:45:00.001+03:00

If you happen to get this error while updating server with yum update
--
.
.
.
File "/usr/lib/python2.4/site-packages/M2Crypto/m2urllib2.py", line 82, in https_open
h.request(req.get_method(), req.get_selector(), req.data, headers)
File "/usr/lib/python2.4/httplib.py", line 813, in request
if v[0] != 32 or not self.auto_open:
IndexError: tuple index out of range
--
disable the location aware access from rhn.

Ref:
http://kbase.redhat.com/faq/docs/DOC-16881

./arun