Sysadmin Notebook

DOCKER_OPTS in /etc/default/docker ignored on Ubuntu

2015-08-03T16:36:00.005+01:00

For some reason, in debian 8 and ubuntu 15.01 systemd is skipping the execution of /etc/default/docker.

How to fix?

Copy the original systemd service file to /etc/systemd/system/

sudo cp /lib/systemd/system/docker.service /etc/systemd/system/

Edit file /etc/systemd/system/docker.service

...
[Service]
ExecStart=/usr/bin/docker -d -H fd:// $DOCKER_OPTS
...
EnvironmentFile=-/etc/default/docker
...

Then execute:

sudo systemctl daemon-reload
sudo systemctl restart docker

Verify that /etc/default/docker is loaded

ps auxwww | grep docker
root 4989 0.8 0.1 265540 16608 ? Ssl 10:37 0:00 /usr/bin/docker -d -H fd:// --insecure-registry

That's it.

Enable Layer3 on a Cisco Switch

2015-07-27T21:56:00.002+01:00

This is what worked for me on a Cisco WS-C2960-24PS-L switch:

First you need to change the sdm prefer from dafult to lanbase-routing.
The lanbase-routing template supports IPv4 unicast routes for configuring static routing SVIs.
Static routing is supported only on switched virtual interfaces (SVIs) and not on physical interfaces. The switch does not support routing protocols.

conf t
sdm prefer lanbase-routing
end
wr
reload

Then you have to enable ip routing:

conf t
ip routing

Then you can add the static routes you like.

Configure CISCO Catalyst 2960 ports to monitoring/mirroring mode

2015-02-16T18:09:00.000+00:00

This will send all traffic that comes on the 0/1-0/9 ports to the Ga0/1 interface:

switch>enable
switch#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.

switch(config)#monitor session 1 source interface fastEthernet 0/1
switch(config)#monitor session 1 source interface fastEthernet 0/2
switch(config)#monitor session 1 source interface fastEthernet 0/3
switch(config)#monitor session 1 source interface fastEthernet 0/4
switch(config)#monitor session 1 source interface fastEthernet 0/5
switch(config)#monitor session 1 source interface fastEthernet 0/6
switch(config)#monitor session 1 source interface fastEthernet 0/7
switch(config)#monitor session 1 source interface fastEthernet 0/8

switch(config)#monitor session 1 destination interface gigabitEthernet 0/1

Show info:

switch#show monitor session 1
Session 1
---------
Type : Local Session
Source Ports :
Both: Fa0/1-8
Destination Ports : Gi0/1
Encapsulation : Native
Ingress : Disabled

And now we can capture this traffic:

# tcpdump -i eth0 -n

One line web server

2015-02-09T20:56:00.000+00:00

The following one line script will create a web server running on port 80 using nc (netcat):

while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; cat index.html; } | nc -l 8080; done

Purge Removed packages

2014-09-07T15:46:00.000+01:00

Packages marked as rc by dpkg mean that the configuration files are not yet removed. The following command will purge them:

dpkg --list |grep "^rc" | cut -d " " -f 3 | xargs -r sudo dpkg --purge

How to permanently delete ALL older kernels

2014-09-07T15:44:00.001+01:00

This script will remove ALL versions but two, the currently active and the most recent of the remaining installed versions:

#/bin/bash
keep=2
ls /boot/ | grep vmlinuz | sed 's@vmlinuz-@linux-image-@g' | grep -v $(uname -r) | sort -Vr | tail -n +$keep | while read I
do
aptitude purge -y $I
done
update-grub

you can specify how many kernels to keep by adjusting the keep variable, if you set it to 1, only the active kernel will be left installed.

Or you can do it in one line:

ls /boot/ | grep vmlinuz | sed 's@vmlinuz-@linux-image-@g' | grep -v $(uname -r) | sort -Vr | tail -n +2 | xargs -r sudo aptitude purge -y

that you can use in crontab.

GitLab update script

2014-08-30T16:10:00.000+01:00

I've recently installed GitLab and they provide easy to install deb and rpm packages but not a repository to help us keep our installation up to date. So I developed the following script that will check https://about.gitlab.com/downloads/archives/ for newer versions and install them when available:

#!/bin/bash
OS="ubuntu"
OS_Version="14.04"
OS_ARCHITECTURE="amd64"
# Ubuntu/Debian:
INSTALLED_VERSION=$(dpkg -s gitlab | grep -i version | cut -d" " -f2)
# CentOS:
#INSTALLED_VERSION=$(rpm -qa | grep omnibus)
# Uses sort -V to compare versions
LATEST=$(wget -q -O- https://about.gitlab.com/downloads/archives/ | grep -i "$OS" | grep -i "$OS_VERSION" | grep -i $OS_ARCHITECTURE | grep -Eo 'href=".*"' | cut -d'"' -f2 | sort -V | tail -n 1)
PACKAGE=${LATEST##*/}
LATEST_VERSION=$(echo $PACKAGE | cut -d_ -f2)
echo ""
echo " Current version: $INSTALLED_VERSION"
echo " Latest version: $LATEST_VERSION"
if [[ "$INSTALLED_VERSION" != "$LATEST_VERSION" && "$LATEST_VERSION" != "" ]]; then
echo " Update to $LATEST_VERSION available!"
echo -n " Do you wich to upgrade? [y/N]? "
read answer
case $answer in
y*)
# Backup branding:
cp /opt/gitlab/embedded/service/gitlab-rails/public/assets/*logo*.png /tmp/
wget $LATEST
# Stop unicorn and sidekiq so we can do database migrations
sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop sidekiq
# Create a database backup in case the upgrade fails
sudo gitlab-rake gitlab:backup:create
# Install the latest package
# Ubuntu/Debian:
sudo dpkg -i $PACKAGE
# CentOS:
#sudo rpm -Uvh $PACKAGE
# Restore branding:
sudo cp /tmp/*logo*.png /opt/gitlab/embedded/service/gitlab-rails/public/assets/
# Reconfigure GitLab (includes database migrations)
sudo gitlab-ctl reconfigure
# Restart all gitlab services
sudo gitlab-ctl restart
rm $PACKAGE
;;
*)
echo "No change"
;;
esac
else
echo " Nothing to do!"
fi
echo ""

I haven't tested this script on a CentOS machine so it might need some adjustments to work there.

Removing hosts from backuppc

2014-07-25T14:25:00.001+01:00

Simply remove the host from the web interface and rm -rf the pc/<host> directory, then wait for the next BackupPC_nightly run - it will remove all superfluous files from the pools.

The path for this directory usually is:

/var/lib/Backuppc/pc/<hostname>

If you want to force the clean-up process, you can remove your host like this:

1. Login to the Backuppc server
2. Remove the host in the Backuppc web-interface (under hosts)
3. remove it's directory /var/lib/Backuppc/pc/<hostname>:

rm -rf /var/lib/Backuppc/pc/<hostname>

4. Shutdown backuppc:

service backuppc stop

5. Change into backuppc:

su - backuppc

6. Run the nightly script:

/usr/share/BackupPC/bin/BackupPC_nightly 0 255

7. Go back to root:

exit

8. Start backuppc again:

service backuppc start

Reclaim free space from Time Machine sparsebundle

2014-07-25T11:36:00.000+01:00

You msut run these commands as root:

sudo su -

Make sure the mount point exists:

mkdir -p /Volumes/TM

Then mount the afp share:

mount_afp 'afp://user:password@afp_server_address/share_name' /Volumes/TM

Now use hdiutil to reclaim the available free space:

hdiutil compact /Volumes/TM/ComputerName.sparsebundle/

unmont the share:

umount /Volumes/TM/

If you get an error message saying:

hdiutil: compact failed - Resource temporarily unavailable

You must make sure you don't have the afp share mounted elsewhere, you can check your mounts with:

df -h

If the output contains a line with your afp server's address or with the string "Time Machine" you have to unmount them.

The following script will do all that for you:

SRV="afp_server_address"
SAVEIFS=$IFS
IFS=$'\n';
for v in $(df -h | grep -E "$SRV|Time\sMachine" | cut -d"%" -f3 | sed -e "s/ *\//\//"); do
umount "$v"
done
IFS=$SAVEIFS
mkdir -p /Volumes/TM
mount_afp "afp://user:password@$SRV/share" /Volumes/TM
hdiutil compact "/Volumes/TM/$(scutil --get ComputerName).sparsebundle"
umount /Volumes/TM/

Changing Time Machine Backup Interval

2014-07-24T22:02:00.000+01:00

You can use the following command to change the Time Machine backup interval:

sudo defaults write /System/Library/LaunchDaemons/com.apple.backupd-auto StartInterval -int 14400

The time interval is in seconds, so 43200 will start a backup every 12hrs.

Checkout my previous post to learn how to manually delete Time Machine backups.

Manage time machine backups

2014-07-24T19:08:00.001+01:00

Some times you get some errors saying that your time machine's disk is full and Time Machine could not complete the backup so you need to manually delete old backups.

tmutil provides methods of controlling and interacting with Time Machine, as well as examining and manipulating Time Machine backups. Common abilities include restoring data from backups, editing exclusions, and comparing backups.

tmutil latestbackup

Will output the path to the most recent backup and

tmutil listbackups

will list all existing backups, if you use the same backup disk for multiple machines, you can get just the backups from your machine with:

tmutil listbackups | grep "$(scutil --get ComputerName)"

The following command will delete the backups from a mac named old_mac_name:

sudo tmutil delete /Volumes/drive_name/Backups.backupdb/old_mac_name

If you want to be safe, you can pick one snapshot to delete first to be sure the command works as intended. This is nice since it could take hours to clean up some larger backup sets and you want to leave the Mac confident it's deleting the correct information store.

You can use the tmutil tool to delete backups one by one.

sudo tmutil delete /Volumes/drive_name/Backups.backupdb/mac_name/YYYY-MM-DD-hhmmss

Since tmutil was introduced with Lion, this will not work on earlier OS versions.

The tmutil delete command only removes the backup from the sparse bundle. It doesn’t actually free the disk space. To do that, you have to go a little deeper.

On your Mac is a mount point called /Volumes. You can examine the contents of this mount point with ls:

cd /Volumes
ls -1

Should output something like:

Macintosh HD
Recovery HD
Time Machine Backups
TimeMachine

These are the names of all the mounted disks (or things that look like disks) on your Mac. Notice two likely candidates for your actual TimeMachine volume. Yours may be named slightly differently, but the one you want is the one that actually shows files of type .sparsebundle . In my case, it is the volume TimeMachine:

sudo ls -l TimeMachine/

and you should see something similar to:

...
drwxr-x---@ 1 root wheel 264 Jul 25 08:21 sysadmin’s MacbookPro.sparsebundle
...

Notice that you don’t actually own the file. (Had I not used the sudo command with ls I could not have listed the contents of /Volumes/TimeMachine)

That .sparsebundle file for your Mac is where all your backup sets live. TimeMachine manages the contents of this file, but doesn’t do anything automatically to reduce its size. Luckily there is another tool for that, but you’ll have to be root to run it:

sudo su -
hdiutil compact /Volumes/TimeMachine/YourBackup.sparsebundle

Sample output:

Starting to compact…
Reclaiming free space…
...................................................
Finishing compaction…
Reclaimed 3.1 GB out of 304.1 GB possible.

That’s it! In this example I reclaimed 3.1GB of actual disk space on my TimeMachine volume.

The following bash script will remove the oldest backup and reclaim the free space:

COMPUTER_NAME=$(/usr/sbin/scutil --get ComputerName)
NBACKUPS=$(/usr/bin/tmutil listbackups | /usr/bin/grep "$COMPUTER_NAME" | /usr/bin/wc -l)
OLDEST_BACKUP=$(/usr/bin/tmutil listbackups | /usr/bin/grep "$COMPUTER_NAME" | /usr/bin/head -n1)
LATEST_BACKUP=$(/usr/bin/tmutil latestbackup)
echo Latest backup: $LATEST_BACKUP
if [[ -n "$LATEST_BACKUP" && "$LATEST_BACKUP" != "$OLDEST_BACKUP" ]]
then
echo "$NBACKUPS backups. Delete oldest: ${OLDEST_BACKUP##*/} [y/N]? \c"
read answer
case $answer in
   y*)
echo Running: /usr/bin/sudo /usr/bin/tmutil delete "$OLDEST_BACKUP"
/usr/bin/sudo time /usr/bin/tmutil delete "$OLDEST_BACKUP"
   echo "Do you wish to reclaim the free space now? [y/N]? \c"
   read answer
   case $answer in
y*)
mkdir -p /Volumes/TM
   mount_afp 'afp://user:pass@afp_server_address/share_name' /Volumes/TM
   hdiutil compact "/Volumes/TM/$(scutil --get ComputerName).sparsebundle"
   umount /Volumes/TM/
;;
*)
echo No change
;;
esac
   ;;
   *)
echo No change
   ;;
esac
else
echo "No backup available for deletion"
fi

In the script above, don't forget to change the afp URL (afp://user:pass@afp_server_address/share_name) to your own.

Sorry, Command-not-found Has Crashed

2014-07-23T21:41:00.001+01:00

When you try to execute a command that is not installed Ubuntu tries to hint you on the package that you should install but some times, especially after an upgrade, you get an error message saying:

Sorry, command-not-found has crashed! Please file a bug report at:
(...)

This solves the problem:

export LANGUAGE=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
locale-gen en_US.UTF-8
sudo dpkg-reconfigure locales

Ubuntu as Time Machine server

2014-07-23T21:36:00.003+01:00

This guide will help you to install and configure the netatalk servise on an Ubuntu server so it can function as a Time Machine backup server for your Mac OS machines.

First install the necessary packages:

sudo apt-get install netatalk avahi-daemon libnss-mdns

Open the netatalk default configuration file:

sudo vi /etc/default/netatalk

Modify the lines:

ATALKD_RUN=yes
PAPD_RUN=no
CNID_METAD_RUN=yes
AFPD_RUN=yes
TIMELORD_RUN=no
A2BOOT_RUN=no

Edit the atalkd.conf file:

sudo vi /etc/netatalk/atalkd.conf

add to the bottom

eth0

Edit the AppleVolumes.default file:

sudo vi /etc/netatalk/AppleVolumes.default

add to the bottom:

/backups/timemachine "Time Machine" allow:@admin cnidscheme:cdb volsizelimit:200000 options:usedots,upriv,tm

The example above also limits the size shown to OS X as 200 GB (the number is given in MiB, so it's 200,000 times 1024 in the real world)

Edit the afpd configuration file:

sudo vi /etc/netatalk/afpd.conf

add to the bottom:

- -transall -uamlist uams_dhx.so,uams_dhx2.so -nosavepassword -advertise_ssh -mimicmodel TimeCapsule6,106 -setuplog "default log_warn /var/log/afpd.log"

Create a configuration file for the avahi afpd discovery:

sudo vi /etc/avahi/services/afpd.service

and enter the following into it:

<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name replace-wildcards="yes">%h</name>
<service>
<type>_afpovertcp._tcp</type>
<port>548</port>
</service>
<service>
<type>_device-info._tcp</type>
<port>0</port>
<txt-record>model=MacPro</txt-record>
</service>
</service-group>

Restart the services:

sudo service netatalk restart
sudo service avahi-daemon restart

MySQL and Oracle command equivalents

2014-07-19T19:29:00.000+01:00

MySQL has specific commands, which provides the easy access to the information_schema database, to get the schema level details. But oracle does not provide such easy access to some of the schema level meta data.

Here are some MySQL specific commands/Syntaxes & equivalent Oracle techniques:

To get the list of databases

MySQL :

show databases

Oracle :

SELECT username FROM all_users ORDER BY username;

To get the current schema

MySQL :

select DATABASE();

Oracle :

SELECT sys_context('USERENV', 'CURRENT_SCHEMA') FROM dual;

To get the list of tables within the current database

MySQL :

use database_name;
show tables;

Oracle :

select * from user_tables;

Here schema is based on the connected username, so it is selected during the creation of the connection.
USER_TABLES will have a row for every table in your schema. If you are looking for the tables in your schema, this would be the correct query. If you are looking for the tables in some other schema, this is not the right table to use.

ALL_TABLES will have a row for every table you have access to regardless of schema. You would, presumably, want to qualify the query by specifying the name of the schema you are interested in, i.e.

SELECT table_name
FROM all_tables
WHERE owner = <<name of schema>>

Of course, that assumes that you have at least SELECT access on every table in that schema. If that is not the case, then you would need to use DBA_TABLES (which would require that the DBA grant you access to that table), i.e.

SELECT table_name
FROM dba_tables
WHERE owner = <<name of schema>>

To get the connected connection info

MySQL :

show processlist

Oracle :

SELECT sess.process, sess.status, sess.username, sess.schemaname, sql.sql_text FROM v$session sess, v$sql sql WHERE sql.sql_id(+) = sess.sql_id AND sess.type = 'USER';

To limit the selection

MySQL :

select * from user limit 10;

Oracle :

select * from table_name where ROWNUM <= 10;

To select rows which is somewhere middle

MySQL :

select username from user limit 10, 15;

Oracle :

select element_name from (select element_name, ROWNUM as row_number from table_name) as t1 where t1.row_number > 10 and t1.row_number <= 15;

Note: Here we have to use SubQuery rather than call it directly as "select element_name from table_name as t1 where ROWNUM > 10 and ROWNUM <= 15;". This cannot be done as these ROWNUMs are assigned once they are satisfied the given conditions, which follows the WHERE. Since condition "ROWNUM > 10 and ROWNUM <= 15" will never be satisfied from the start ROWNUMs will never be incremented. So we need to use the Subqueries to let the ROWNUMs assigned within the Subquery and later filter the required results from the outside query.

Describe table has a same syntax in both MySQL & Oracle.

desc table_name;

To view errors/warnings

MySQL :

show warings / show errors

Oracle :

select * from user_errors;/ show errors

MySQL has auto_increment Columns

MySQL :

create table table_name (element_id int AUTO_INCREMENT primary, element_name varchar(20));

Oracle :
i) Create table without the auto_increment keywords (because it does not exist in Oracle)

create table table_name (element_id int primary, element_name varchar(20));

ii) Create a sequence, which provides the incremented values

create sequence auto_incrementor;

iii) Create a trigger, which gets the next value from the sequence and updates it to the column to be auto_incremented

CREATE TRIGGER trig_incrementor BEFORE INSERT ON table_name
FOR EACH ROW
BEGIN
SELECT auto_incrementer.NEXTVAL into :new.element_id FROM dual;
END;

To get the table create script back

MySQL :

show create table table_name;

Oracle :
Make sure that the select_catalog_role is already available for the given user if not assign the role, as shown below.
grant select_catalog_role to [username];Increase the page size and maximum width for displaying the results so that complete table definition can be displayed in the sqlplus console.

set pagesize 999
set long 9000
select dbms_metadata.get_ddl('TABLE', 'TABLE_NAME', 'database_name') from dual;

To get the session variables

MySQL :

show variables; or show variables like 'inno%';

Oracle :

SELECT name, value FROM gv$parameter; or SELECT sys_context('USERENV', ) FROM dual;

Explain the execution plan of a sql statement

MySQL :

explain select * from table_name;

Oracle :

i) First execute the explain plan so that it will fill the plan_table (this table need to be created according to the standard plan_table format, if it does not exist already)

explain plan select * from table_name;

ii) Now the results of the explain plan will be populated in the plan_table, so we need to use a row connecting query to get a readable summary of the results.

select substr (lpad(' ', level-1) || operation || ' (' || options || ')',1,30 ) "Operation", object_name "Object" from plan_table start with id = 0 connect by prior id=parent_id;

MySQL and Postgres command equivalents

2014-07-17T19:16:00.000+01:00

Task: list existing databases, connect to one of the databases, then list existing tables and finally show the structure of one of the tables.

In Mysql:

show databases;
use database;
show tables;
describe table;
exit

In PostgreSQL:

\l
\c database
\dt
\d+ table
\q

Test imap using telnet

2014-07-16T22:44:00.002+01:00

For added security, you can encrypt your IMAP connection. This requires that your server supports SSL or TLS and that you have access to an SSL/TLS client program, for example OpenSSL, to use instead of telnet.

As the port-number normally is 993, an example OpenSSL command would be openssl s_client -connect imap.example.com:993 -quiet. (If you would like to see the public key of the server, as well as some other encryption-related information, omit -quiet.) The server should then start an IMAP session, displaying a greeting such as the * OK Dovecot ready example below.

telnet imap.example.com 143

#output: Trying 193.136.28.29...
#output: Connected to imap.example.com.
#output: Escape character is '^]'.
#output: * OK Dovecot ready.

a1 LOGIN MyUsername MyPassword
#output: a1 OK Logged in.

a2 LIST "" "*"
#output: * LIST (\HasNoChildren) "." "INBOX"
#output: a2 OK List completed.

a3 EXAMINE INBOX
#output: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
#output: * OK [PERMANENTFLAGS ()] Read-only mailbox.
#output: * 1 EXISTS
#output: * 1 RECENT
#output: * OK [UNSEEN 1] First unseen.
#output: * OK [UIDVALIDITY 1257842737] UIDs valid
#output: * OK [UIDNEXT 2] Predicted next UID
#output: a3 OK [READ-ONLY] Select completed.

a4 FETCH 1 BODY[]
#output: * 1 FETCH (BODY[] {405}
#output: Return-Path: sender@example.com
#output: Received: from client.example.com ([192.0.2.1])
#output: by mx1.example.com with ESMTP
#output: id <20040120203404.CCCC18555.mx1.example.com@client.example.com>
#output: for <recipient@example.com>; Tue, 20 Jan 2004 22:34:24 +0200
#output: From: sender@example.com
#output: Subject: Test message
#output: To: recipient@example.com
#output: Message-Id: <20040120203404.CCCC18555.mx1.example.com@client.example.com>
#output:
#output: This is a test message.
#output: )
#output: a4 OK Fetch completed.

a5 LOGOUT
#output: * BYE Logging out
#output: a5 OK Logout completed.

Source IP based reverse proxy

2014-07-16T22:34:00.002+01:00

If you want to proxy some source IP ranges/subnets to one server and onther subnets to go to a different server, you can do this using mod_rewrite for proxying. You will have to setup a rewrite condition based on the source IP and a rewrite rule with the [P] flag. Something like this should work:

RewriteCond %{REMOTE_ADDR} ^10\.2\.
RewriteRule ^/(.*) http://old-app/$1 [P]
ProxyPassReverse / http://serverA/

RewriteCond %{REMOTE_ADDR} ^10\.3\.
RewriteRule ^/(.*) http://new-app/$1 [P]
ProxyPassReverse / http://serverB/

One possible scenario for this if you want to migrate your users from server A to server B but you want to migrate one IP range at the time.

If you're using Apache 2.4 or newer you can also achieve this with the following configuration:

<If "-R '10.2.0.0/16'">
ProxyPassReverse / /http://serverA/
</If>
<ElseIf "-R '10.3.0.0/16'">
ProxyPassReverse / /http://serverB/
</ElseIf>
<Else>
ProxyPassReverse / /http://serverC/
</Else>

Preserving client ip with apache reverse proxy

2014-07-15T14:50:00.000+01:00

The first thing that I thought of was the “X-Forwarded-For” headers, which is an HTTP header inserted into the original HTTP GET request whose value is equal to the client’s public IP. Turns out apache reverse proxy inserts this header by default. So we somehow need to instruct the backend server itself to provide the application with the correct client IP.

If your backend server is a Tomcat server the solution cam be using the RemoteIP tomcat valve.

It’s quite simple to configure in that all that needs to be done is to modify tomcat server.xml to recognise original client IP rather than the proxy IP by adding the following to server.xml:

<Valve className="org.apache.catalina.valves.RemoteIpValve" internalProxies="127\.0\.0\.1" />

make sure to change 127.0.0.1 to the address of the apache reverse proxy.

The application could now recognise the original client IP.

The apache equivalent of the above method is using mod_rpaf for Apache 1.3 & 2.2.x and mod_remoteip for Apache 2.4 and 2.5.

These apache modules can be used to preserve both remote IP/HOST. Internally they use X-Forwarded-For header to detect a proxy in it’s list of known proxies and reset the headers accordingly. This works with any proxy server in the front end provided that the proxy server sets X-Forwarded-For header.

To use mod_rpaf, install and enable it in the backend server and add following directives in the module’s configuration:

RPAFenable On
RPAFsethostname On
RPAFproxy_ips 127.0.0.1

Remote IP is automatically preserved when RPAFenable On directive is used. RPAFsethostname On directive should be used to preserve host and RPAFproxy_ips is the list of known proxy ips.

Restart backend apache server and you are good to go.

For mod_remoteip, it’s a bit similar, the configuration should look something lke this:

RemoteIPHeader X-Real-IP
RemoteIPInternalProxy 1.2.3.4
RemoteIPInternalProxy 5.6.7.8

mod_remoteip however has a lot more configuration options.

When the proxy server is an Apache, ProxyPreserveHost directive in mod_proxy can be used to preserve the remote host not the remote ip. This is useful for situations where name based virtual hosting is used and the backend server needs to know the virtual name of host.

Open mod_proxy configuration file of your proxy server and enter directive, ProxyPreserveHost On, and restart your apache instance.

Finding external IP using the command line

2014-07-10T19:14:00.000+01:00

The easiest way is to use an external service via a commandline browser or download tool. Since wget is available by default in Ubuntu, we can use that.

To find your ip, use:

wget -qO- http://ipecho.net/plain ;

You can do the same using curl:

curl ipecho.net/plain ; echo

How to test a listening TCP/UDP port through nc

2014-07-09T14:13:00.001+01:00

Netcat (nc) can also be used for a lot of other purposes. It can also be used as a very fast basic port scanner, you can scan a port or a range.

To scan a range of UDP ports 1 to 1000:

nc -vzu destination_ip 1-1000

To scan a range of TCP ports 1 to 1000

nc -vz destination_ip 1-1000

Install Vsftpd on Ubuntu

2014-06-26T00:48:00.000+01:00

On my last post I've talked about enabling the userdir module on Apache, you can use vsftpd to give your users FTP access to their own pages, this is hoe you can install it:

aptitude -y install vsftpd

Then edit it's configuration file:

vi /etc/vsftpd.conf

And make the following changes:

# line 29: uncomment
write_enable=YES
# line 97,98: uncomment ( allow ascii mode transfer )
ascii_upload_enable=YES
ascii_download_enable=YES
# line 120: uncomment ( enable chroot )
chroot_local_user=YES
# line 121: uncomment ( enable chroot list )
chroot_list_enable=YES
# line 123: uncomment ( enable chroot list )
chroot_list_file=/etc/vsftpd.chroot_list
# line 129: uncomment
ls_recurse_enable=YES
# add at the last line
# specify root directory ( if don't specify, users' home directory equals FTP home directory)
#local_root=public_html
# turn off seccomp filter
seccomp_sandbox=NO

Edit the list of users that can access your server.

vi /etc/vsftpd.chroot_list

Add the users you allow to move over their home directory

Finally restart the FTP service:

service vsftpd restart

Enable userdir Apache module on Ubuntu

2014-06-26T00:10:00.001+01:00

First activate the module:

sudo a2enmod userdir

now edit the module's conf file:

sudo vi /etc/apache2/mods-enabled/userdir.conf

and change the line:

AllowOverride FileInfo AuthConfig Limit Indexes

AllowOverride All

By default PHP is explicitly turned off in user directories, to enable it edit the php module conf file:

sudo vi /etc/apache2/mods-enabled/php5.conf

and comment out the following lines:

#<IfModule mod_userdir.c>
# <Directory /home/*/public_html>
# php_admin_flag engine Off
# </Directory>
#</IfModule>

Now just restart your apache srerver and that's it:

sudo service apache2 restart

You can now create a public_html folder on every users homes with the following script:

#!/bin/bash
for I in /home/*; do
if [ ! -d $I/$FOLDER ]; then
mkdir -p $I/$FOLDER
U=$(basename $I)
chown $U $I/$FOLDER
chgrp $U $I/$FOLDER
fi
done # for

Now if you whant to go further and create dynamic vhost for each of your users you can change your default virtual host with something like this:

<VirtualHost *:80>
RewriteEngine on
RewriteMap lowercase int:tolower
# allow CGIs to work
RewriteCond %{REQUEST_URI} !^/cgi-bin/
# check the hostname is right so that the RewriteRule works
RewriteCond ${lowercase:%{SERVER_NAME}} ^[a-z-]+\.example\.com$
# concatenate the virtual host name onto the start of the URI
# the [C] means do the next rewrite on the result of this one
RewriteRule ^(.+) ${lowercase:%{SERVER_NAME}}$1 [C]
# now create the real file name
RewriteRule ^([a-z-]+)\.example\.com/(.*) /home/$1/public_html/$2
<Location / >
Order allow,deny
allow from all
</Location>
DocumentRoot /var/www/
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
# define the global CGI directory
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

This will allow you to use user.example.com to access your user's pages.

IPTables debugging

2014-06-17T12:05:00.000+01:00

The following command will only show rules that have the action set to DROP or REJECT and omit the rules that didn't had any matches:

watch -n1 "iptables -nvL | grep -i 'DROP\|REJECT\' | egrep -v '^\s*0\s*0'"

This one does the same but with some colour highlighting, it will only show rules with matches, the words DROP and REJECT will appear in red and the word ACCEPT will be in green:

watch --color -n1 "iptables -nvL | egrep -v '^\s*0\s*0' | sed 's/$DROP\|REJECT$/\x1b[49;31m\1\x1b[0m/g' | sed 's/$ACCEPT$/\x1b[49;32m\1\x1b[0m/g'"

Using the IP command

2014-06-16T19:43:00.000+01:00

The command /bin/ip has been around for some time now. But people continue using the older command /sbin/ifconfig. ifconfig won't go away quickly, but its newer version, ip, is more powerful and will eventually replace it.
So here are the basics of the new ip command.

Assign a IP Address to Specific Interface:

sudo ip addr add 192.168.50.5 dev eth1

Check an IP Address:

sudo ip addr show

Remove an IP Address:

sudo ip addr del 192.168.50.5/24 dev eth1

Enable Network Interface:

sudo ip link set eth1 up

Disable Network Interface:

sudo ip link set eth1 down

Check Route Table:

sudo ip route show

Add Static Route:

sudo ip route add 10.10.20.0/24 via 192.168.50.100 dev eth0

Remove Static Route:

sudo ip route del 10.10.20.0/24

Add Default Gateway:

sudo ip route add default via 192.168.50.100

Change Root DN Password on OpenLDAP

2014-06-15T14:59:00.001+01:00

First, we need to locate the credentials information of the administrator account in the correct database within the LDAP tree.

This can be done using the ldapsearch command:

ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -b cn=config olcRootDN=cn=admin,dc=example,dc=com dn olcRootDN olcRootPW

(replace the olcRootDN value with the correct value to match your configuration)

This command will return something like:

SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
dn: olcDatabase={1}hdb,cn=config
olcRootDN: cn=admin,dc=example,dc=com
olcRootPW: {SHA}ks1xBVfgRXavGCpkPefc9hRHL4X=

There are two interesting information we know now:

we need to modify the entry “dn: olcDatabase={1}hdb,cn=config“
the current password is hashed with SHA1 algorythm.
To generate our new password with the same algorythm we'll use the command slappasswd with the syntax:

slappasswd -h <the hashing scheme we want to use - for example {SHA}>

The system will then prompt you for the new password to use, twice, and will finally display the hashed value we’re interested in:

root@testbox:~# slappasswd -h {SHA}
New password:
Re-enter new password:
{SHA}W6ph5Mm7Ps6GglULbPgzG37mj0g=

Then we’ll proceed to modify the entry we’ve identified above using the command:

root@testbox:~# ldapmodify -Y EXTERNAL -H ldapi:///

The system will start the listening mode for modifying commands:

SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0

First, we enter the entry we want to modify:

dn: olcDatabase={1}hdb,cn=config

Second, we type in the parameter we want to modify:

replace: olcRootPW

Third, we type in the new password generated above (copy and paste is MUCH less error prone than manual typing at this point ;) )

olcRootPW: {SHA}W6ph5Mm7Ps6GglULbPgzG37mj0g=

Hit Enter another time to commit the modification and the following line will appear:

modifying entry "olcDatabase={1}hdb,cn=config"

After this, you can exit the listening mode with CTRL+C and restart the LDAP database service using:

service slapd stop
service slapd start

and login now with the new password set.