Unix and Linux systems, clouds and scaling

Adding global DNS servers

2012-01-25T12:46:00.002+02:00

Sometimes you need to preconfigure your servers for dns resolution and you are not sure what local DNS addresses will be or just to use backup dns servers for resolution.
Here is small list of globally accessible DNS servers:

nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 210.80.60.1
nameserver 210.80.60.2
nameserver 208.67.222.222
nameserver 208.67.220.220

NTP servers for Israel

2012-01-24T10:43:00.000+02:00

For Israeli NTP servers you can use: il.pool.ntp.org

Since there are not enough servers in this zone, we recommend you buck up by use the Asia zone (asia.pool.ntp.org):

server 0.asia.pool.ntp.org
server 1.asia.pool.ntp.org
server 2.asia.pool.ntp.org
server 3.asia.pool.ntp.org

allowing public access to amazon s3 bucket

2011-10-24T19:54:00.003+02:00

You might need to give public access to bucket contents, for example if you use the bucket to load images for your site. Simple GUI option supplied by Amazon only gives access for a specific files but if you need a whole bucket, you have to use policy.

To edit policy, in GUI go to

'Bucket => Properties => Add Bucket Policy'

And add for example (just change yourbucketname with the name of your bucket:

{
   "Version": "2011-10-24",
   "Statement": [
       {
           "Sid": "AddPerm",
           "Effect": "Allow",
           "Principal": {
               "AWS": "*"
           },
           "Action": "s3:GetObject",
           "Resource": "arn:aws:s3:::yourbucketname/*"
       }
   ]
}

save it and policy is applied.

Upgrading PHP in CentOS 5

2011-05-23T16:23:00.002+03:00

CentOS 5.x comes with PHP 5.1, this version is outdated and you probably would want to upgrade it. I used Atomic repository.

You can add it by running following command:

wget -q -O - http://www.atomicorp.com/installers/atomic.sh | sh

Now you can update PHP:

yum update php

yum update php-mcrypt

(you might also need to update mysql)

Verify version of running PHP:

php -v

And that all modules working as order:

php -m

Restart Apache, you are all set.

"SQL Error: 13: Can't create/write to file" on other then /tmp directories (apparmor)

2011-05-22T19:47:00.005+03:00

If you get SQL Error: 13 after changing tmpdir parameter in you my.cnf first check permissions in a new directory. If permissions are correct you might be having an issue with AppArmor. AppArmor is a GNU security module for the Linux kernel and it might prevent MySQL from writing to the new directory.

Run: aa-status

To verify if it`s on. If so you will get something like:

profiles are loaded

profiles are in enforce mode

Edit this file:

/etc/apparmor.d/usr.sbin.mysqld

and append your new temp directory in the end.

####

/usr/sbin/mysqld {

...

/new_tmp/ r,

/new_tmp/* rw,

}

Reload AppArmor with:

/etc/init.d/apparmor reload

Thats it.

How to update an auto scale launch configuration in Amazon EC

2011-04-07T15:15:00.001+03:00

In order to update existing auto scale launch configuration in Amazon EC you need to create a new launch configuration using as-create-launch-config and then assign it to launch configuration using as-update-auto-scaling-group

Mounting S3 as a file system on Linux (Ubuntu)

2011-04-07T13:56:00.005+03:00

In order to mount S3 on Linux you need to install ssh file system (FUSE) libraries first.

Install: aptitude install build-essential libcurl4-openssl-dev libxml2-dev libfuse-dev comerr-dev libfuse2 libidn11-dev libkadm55 libkrb5-dev libldap2-dev libselinux1-dev libsepol1-dev pkg-config fuse-utils sshfs

Follow the instantiations in S3fs WIKI:

https://code.google.com/p/s3fs/wiki/InstallationNotes

I was using http download, not SVN.

Create a mount point for the new file system (ex. mkdir /mnt/s3)

Mount the file system:

ex.: s3fs tmpname -o use_cache=/tmp -o allow_other /mnt/s3

this mount comes with option of using a tmp as a cache for S3 content, you can clean that cache as you feel fit.

You can also add your bucket to /etc/fstab

s3fs#mybucket /mnt/s3 fuse allow_other,url=https://s3.amazonaws.com 0 0

Update for mounting S3 bucket on CentOS 5.5 and other old distros.
CentOS 5.5 comes with old FUSE version 2.7.4 as latest in it`s repository.
You need to manually compile version 2.8.4 befor you can use S3fs.

Remove old FUSE
yum remove fuse fuse* fuse-devel

Install needed libraries:
yum install gcc libstdc++-devel gcc-c++ curl curl* curl-devel libxml2 libxml2* libxml2-devel openssl-devel mailcap

Download new FUSE from the Sourceforge:
wget "https://downloads.sourceforge.net/project/fuse/fuse-2.X/2.8.4/fuse-2.8.4.tar.gz?r=&ts=1299709935&use_mirror=cdnetworks-us-1"

Untar and compile:

tar -xzvf fuse-2.8.4.tar.gz
cd fuse-2.8.4
./configure --prefix=/usr
make; make install

Configure FUSE in the system:
export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/
ldconfig
modprobe fuse

Confirm that 2.8.4 is the version of FUSE displayed :
pkg-config --modversion fuse

Proceed with regular S3FS installation.

Clearos, OpenVPN and mutiwan issue

2011-01-04T20:46:00.002+02:00

Just had some problems running OpenVPN in mutiwan mode on ClearOS 5.2 SP1.

Sometime connection got dropped down and user could not reconnect. Problem was solved then i disabled on of the multiwan NICs. What fixed the problem was setting OpenVPN to work in TCP mode instead of UDP. I guess packets were sent using wrong NIC on UDP based connections.

Edit /etc/openvpn/clients.conf
and set "proto tcp"

Setting DNS server on Amazon EC2

2010-07-06T00:57:00.003+03:00

Setting up bind9 DNS server on Amazon EC2 today tough me that Amazon`s built in service definition in security group only includes TCP. be sure to open UDP port 53 as well for your server to function properly.

Searching for a pattern string under a directory

2009-05-24T11:58:00.002+03:00

In order to search for a pattern string in all files under a specific directory you can use following command:
find $dir -exec grep $pattern {} \; -print
just replace $dir with the directory you want to start searching in and $patter with patter you are searching for.
for example:
find /etc -exec grep support {} \; -print

changing editor in ubuntu linux

2009-05-09T23:43:00.006+03:00

To permanently change editor in Linux do following:
edit your shell configuration file,
for example, bash configuration file is ~/.bashrc
To set editor as VI:
export VISUAL=vi
or to set nano:
export VISUAL=nano
to start it run:
source ~/.bashrc

Running manual NTP (network time protocol) In Ubuntu Linux

2009-04-12T12:05:00.000+03:00

If your Ubuntu is out of time sync you can run a manual comand to syncronize the clock. command is:
ntpdate
Running ntpdate with no options / switches will couse it to try to sync with default servers. If none are set you will get an error similar to:
ntpdate[15110]: no servers can be used, exiting
then run it with servers address.
for example:
ntpdate europe.pool.ntp.org north-america.pool.ntp.org
you might need to run sudo if you are not system root:
sudo ntpdate europe.pool.ntp.org north-america.pool.ntp.org
P.S. Server pool for Israel is:
il.pool.ntp.org

Fix (repair) MySQL replication in case of duplicate items

2009-04-06T14:34:00.000+03:00

If you as i am using MySQL replication mechanism, master-slave or master-master replication you sometimes loose replication due to duplicate entries.
You can fix it by simply skipping the record:
1. log into MySQL:
mysql -u root -p
2. in MySQL shell stop slave replication
mysql> STOP SLAVE;
3. skip the affected record
mysql> SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
4. start slave replication process
mysql> START SLAVE;
5. verify that replication is working:
mysql> SHOW SLAVE STATUS \G
6. exit MySQL shell:
mysql> quit;

MySQL and debian-sys-maint account

2008-11-26T22:08:00.001+02:00

If for some reason your debian-sys-maint account is unset in MySQL, during MySQL start you will have connect to server error. You might need to set this account in the MySQL server itself.
cat /etc/mysql/debian.cnf to get the password value from the file.
Then connect to the DB
mysql -u root -p
and execute:
GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY 'YOUR_PASSWORD' WITH GRANT OPTION

Changing time on CentOS

2008-11-10T14:44:00.000+02:00

just copy relevant timezone file from /usr/share/zoneinfo to /etc/localtime

like:
cp /usr/share/zoneinfo/Europe/Moscow /etc/localtime

unix backup with tar and mt

2008-07-15T20:37:00.000+03:00

Rewind tape drive:# mt -f /dev/st0 rewind
Backup directory /www and /home with tar command (z - compressed):


# tar -czf /dev/st0 /www /home

Find out what block you are at with mt command:


# mt -f /dev/st0 tell

Display list of files on tape drive:


# tar -tzf /dev/st0

Restore /www directory:


# cd /
# mt -f /dev/st0 rewind
# tar -xzf /dev/st0 www

Unload the tape:


# mt -f /dev/st0  offline

Display status information about the tape unit:


# mt -f /dev/st0   status

Erase the tape:
# mt -f /dev/st0 erase
You can go BACKWARD or FORWARD on tape with mt command itself:
(a) Go to end of data:


# mt -f /dev/nst0 eod

(b) Goto previous record:


# mt -f /dev/nst0  bsfm 1


# mt -f /dev/nst0  fsf 1

deleting message from postfix queue

2008-07-10T11:23:00.000+03:00

To list the queue execute:
mailq
You can find the problematic messages id and the execute:

find /var/spool/postfix -name "messegeID" | xargs rm

Keyboard shortcuts for Windows

2008-06-18T14:55:00.001+03:00

Yes i know, it`s a UNIX blog. but sometimes we use M$ as well.

http://support.microsoft.com/kb/126449

InPlace CMS

2007-10-14T16:48:00.000+02:00

New CMS release, InPlace, very simple to work with. It is a is a Python wrapper for XSLT. and of course freely distributed.

project`s homepage is here.

small flow diagram to make things simpler:

Why InPlace rocks? The documentation is already in the HTML form, no need for the generation step, no errors occur.

bmc configuration manager ( marimba ) cli runchannel return codes

2007-10-14T16:41:00.000+02:00

those are CLI return codes for marimba runchannel on UNIX.
The return codes are 0, for success, and -1 through -12, for failure, as described below.
0 - runchannel -help was requested (and provided). A channel was started and completed an operation successfully.
1 - In the tuner's installation directory, the properties.txt file was missing from the lib/tuner/ directory. There was an incorrect runchannel command-line argument. An exception occurred while trying to start the tuner. There were notifications related to channel exceptions, RPC Disconnects, or RPC timeouts.
2 - An unexpected exception was caught. An associated stack trace gives more details. If you see this code while publishing a signed channel, then make sure that the cert DN (certificate's distinguished name) is specified correctly.
3 - Command was canceled.
4 - I/O error. Log entries should give more details, but this error can mean that the command failed to get data channel properties (that is, a communication problem between the endpoint and transmitter exists). This error can also mean that the command failed to move internal directories in or out during a publish or install operation.
5 - Publish error. A publish or unpublish command failed in the publish process. The actual error is described in the logs.
6 - Cleanup error. Not currently used.
7 - Update error. An install or uninstall command failed in the installation process. The actual error is described in the logs.
8 - Concurrency error. This error can occur only when the application is used as a library. The same instance of a kernel was used to perform multiple publish or installation commands simultaneously.
9 - Exec error. An error occurred while trying to execute a pre- or post- installation command.
10 - Unsigned error. Not currently used. Attempting to install a channel that is not properly signed returns a -3 (canceled) code, but a log entry indicating that the channel was not signed correctly also exists.
11 - Unknown error. This error should not be returned. If this error is returned, the logs contain entries indicating the problem.
12 - No-such-channel error. An error occurred while getting the data channel list from a transmitter. When installing, the data channel is what the application publishes to the transmitter during a publish command.

how to restore system user in oracle and lock it back

2007-10-14T16:38:00.000+02:00

Following instructions will help you unlock the oracle db system user and lock it back when needed.
Those are the instructions:
1) login to a machine as user oracle
2) connect to sqlplus: sqlplus '/ as sysdba'
3) unlock user: alter user system account unlock;
4) change password: alter user system identified by manager;
5) verify connection: conn system/manager
6) restore password: ALTER USER system IDENTIFIED BY VALUES 'F2B917B76C7DFF0A';
7) lock user: alter user system account lock;
8) verify lock: conn system/manager
you should see the following message: ERROR: ORA-28000: the account is locked

how to locate oracle SID on linux / unix machine

2007-10-14T16:36:00.001+02:00

Log into a UNIX or Linux machine as root.
execute: ps -ef |grep -i pmon
you should see something like that:
oracle 27161 1 0 Mar07 ? 00:03:53 ora_pmon_ODS
then ODS is a SID of that oracle.

How to set or disable auto-logout on bash shell

2007-10-14T16:34:00.000+02:00

Following information is intended for bash shell only.
The system variable TMOUT can be set to specify the amount of time the user is inactive before the user is automatically logged out.
Set the following listed below in the /root/.bashrc file: TMOUT=XXX #
The system will logout the root account after one hour of inactivity. The value entered for the TMOUT= variable is in seconds. To represent 1 hour, we have to multiply as follows: 1 hour is 60 minutes. 60 minutes * 60 seconds = 3600 seconds.
After setting the parameter, the intended user will need to log out and then log in for the changes to take effect. To make modifications apply to all users on the system, specify the TMOUT variable in /etc/ profile.
To disable auto-logout simply comment out or delete the line with TMOUT value

small legato cluster commands guide

2007-10-14T16:24:00.000+02:00

Just a small legato cluster (on Linux in my case) commands reference guide.

to list resources in legato cluster:
$FT_DIR/bin/ftcli -cmd lrg (lists the legato resource groups)
$FT_DIR/bin/ftcli -cmd lip (lists ip numbers in legato cluster)
$FT_DIR/bin/ftcli -cmd ln (lists legato cluster nodes)
$FT_DIR/bin/ftcli -cmd lp (lists all the processes in legato cluster)

to start a legato cluster resource group:
$FT_DIR/bin/ftcli -cmd "bgo resource_group_name node"
to stop a legato cluster resourse group:
$FT_DIR/bin/ftcli -cmd "tgo resource_group_name"

to start a legato cluster monitored application:
$FT_DIR/bin/ftcli -cmd "start application node"
to stop a legato cluster monitored application:
$FT_DIR/bin/ftcli -cmd "stop application"

double loop script example

2006-12-03T18:49:00.000+02:00

#!/bin/bash

i=0
for ID1 in $(awk -F":" '{print $2}' /tmp/components) ; do
id1[$i]=$ID1
i=$(($i+1))
done
j=0 ; while [ $j -lt $i ] ; do cmvt_component_os=${id1[$j]}; echo $cmvt_component_os ; j=$(($j+1)) ; done