Linux Server Security Secrets and Administration

Setup more than one sshd daemon in one box

2009-06-13T14:48:00.003-03:00

You have one Linux server and want to set two ssh daemon, may be two different ssh servers each with a different version.

1) Download the openssh from http://www.openssh.org/portable.html

2) Configure and make the sources:
# ./configure --with-kerberos5 --with-tcp-wrappers --with-pam
# ./make

2.1) Common errors:

"configure: error: PAM headers not found"
Solution: yum install pam-devel

3) You can do a "make install" but i wouldn't do that if you have already an ssh daemon installed, from sources or any package manager like rpm or apt.

4) Copy the ssh daemon directory or link to it from /usr/local/sbin/
# ln -s ./openssh-new /usr/local/sbin/sshd-new/

5) Supposing that you already have a /etc/ssh/sshd_config file, then you have to create
another configuration file for the new ssh server, lets put it in another directory:
/usr/local/etc/ssh/sshd_config

And edit this sshd_config file and put there this line, or edit it if it already exists:
# This line specifies which port you want to use:
Port 2253

6) Create the ssh key files:
# ssh-keygen -t rsa -f /usr/local/etc/ssh/ssh_host_rsa_key
# ssh-keygen -t dsa -f /usr/local/etc/ssh/ssh_host_dsa_key

7) Make sure that in the startup script /etc/init.d/sshd2 you instruct the daemon where to find its key files, something like this:

# Some functions to make the below more readable
KEYGEN=/usr/local/sbin/openssh-5.2p1/ssh-keygen
SSHD=/usr/local/sbin/openssh-5.2p1/sshd
RSA1_KEY=/usr/local/etc/ssh/ssh_host_key
RSA_KEY=/usr/local/etc/ssh/ssh_host_rsa_key
DSA_KEY=/usr/local/etc/ssh/ssh_host_dsa_key
PID_FILE=/var/run/sshd2.pid
OPTIONS="-f /usr/local/etc/ssh/sshd_config"

8) Finally, start the new ssh server.
/etc/init.d/sshd2 start

Public key for is not installed

2009-06-12T11:00:00.004-03:00

The .rpm package has been signed with a private key to provide nonrepudiation and integrity, basically, its genuinety. The package should be verified with the public key.

You can use rpm to install the package:

# rpm --import http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5


# yum install ./mysqlclient10-3.23.58-9.2.c4.x86_64.rpm

Creating a VIP on Linux

2009-03-17T18:09:00.002-03:00

If you want to have two or more ip addresses on one physical interface, in Linux, there are many ways to do it, but to have a change permanent you have to do these steps on Redhat Linux and probably other distros:

Make a copy from the physical interface startup config file:
cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth0:0

Modify the /etc/sysconfig/network-scripts/ifcfg-eth0:0
DEVICE=eth0
BOOTPROTO=static
HWADDR=00:53:52:A2:43:43
IPADDR=192.168.100.30
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
MACADDR=00:53:52:A2:43:43

Finaly, do a :
# ifup eth0:0

/usr/lib64: file not recognized: Is a directory

2008-12-13T00:07:00.003-02:00

I have seen this error several times when compiling and the solution is always the same:

Edit the Makefile and replace:
LDFLAGS = -L /usr/lib64
for:
LDFLAGS = -L/usr/lib64
(remove the space between the -L and /usr/lib64)

Error with PHP and Mysql

2008-12-12T16:15:00.003-02:00

skipping incompatible /usr/lib/mysql/libmysqlclient.so when searching for -lmysqlclient

Seems that this errors come because of incompatibility between i386 and x64.

# cd /usr/lib/mysql
# ln -s ../../lib64/mysql/libmysqlclient.so

If this do not works, you can patch the Makefile to use the x64 version of the
libraries instead of the x32:

# sed -i -e 's@-L/usr/lib/mysql@-L/usr/lib64/mysql@g' Makefile

Error starting snmp on Centos.

2008-12-12T00:46:00.002-02:00

/usr/sbin/snmpd: symbol lookup error: /usr/sbin/snmpd: undefined symbol: smux_snmp_select_list_get_length

Resolution:
yum update net-snmp-libs

The Linux Knowledge Base and Tutorial

2008-08-25T18:27:00.000-03:00

Looking for an in-depth and easy-to-understand introduction to Linux? Then look no further!
We don't just show you how to execute a handful of commands and use a few utilities. The Linux Tutorial goes beyond the basics, providing you with the knowledge necessary to get the most out of your Linux system.

[...] Keep reading in: Linux Knowledge Base and Tutorial

100 Tips and Tools to Set Up Your Own Home LAMP Server

2008-08-22T09:45:00.003-03:00

LAMP (Linux, Apache, MySQL and PHP) servers are very popular for their ease of use and flexibility. They’re also easy to initiate, because the components are easy to aquire and there’s a lot of documentation available to help with getting started. Read on to find some of the best resources available for creating your own home LAMP server, from installation to maintenance.
[...]
Keep reading in:
100 Tips and Tools to Set Up Your Own Home LAMP Server

Recover /etc/fstab file

2008-08-19T16:41:00.002-03:00

If you have deleted the /etc/fstab file it is possible to recover it in the following way:

cat /proc/mounts > /etc/fstab
chmod 644 /etc/fstab

The "proc" filesystem, which is /proc/mounts, is used to handle sytem configuration parameters, it is a virtual filesystem.

Managing links in Linux

2008-08-18T19:46:00.003-03:00

Hard and Symbolic links are the two types of files that exist in a Unix Operating System to point to another file.

Hard Links: They are a pointer that is exactly as the same than the file it points to, no mather if it has a different name, any modification done to the pointer are also done to the target file. The hard links can only point to other files, but not directories (though directories are a special kind of files). And the other main difference with Symbolic Links is that hard links MUST reside in the same filesystem than the file they point to, because they have the same inode number.

Symbolic liks: The are a pointer to another file but they contain the name of the file they point to, it can span filesystems (they have a different inode number), and they can point to files or direcoties.

Creating Hard and Symbolic Links:
ln -fs <source> [<target>]

By default ln (without arguments) it creates hard links.
-f, --force : Remove existing destination files.
-s, --symbolic : Make symbolic links instead of hard links.

Using wget behind a proxy in Linux

2008-06-30T13:19:00.002-03:00

If you are behind a proxy and would like to download a file, you could do this by issuing this commands:

export http_proxy=proxy.anonymous.com:8080
export use_proxy=on
wget http://www.serversolaris.com/Crypt-SSLeay.tgz

A strong Unix Password

2008-06-30T12:19:00.003-03:00

The strong of a cryptographic solution is not recommended to be into the obscurity of the algorithm, the most usefull and most hard to vulnere solutions are public algorithms, which everybody can see, for example PKI, CAST, PGP, all them are public.
So.. the obvious hard point of a security system is a strong password, they part of the mechanism that no other knows. The password could be interpreted as something you know, you have, something that uniquely identifies you or a combination of these.

But taking in consideration just a password (something you know) i would like to share some best practices in Linux.

DON'T DO THIS

A password of less than 8 characters is easily breaked by brute force attack. You can set the PASS_MIN_LENGTH in /etc/login.defs file to force long enough passwords.

Don't use words that can be found in a dictionary or encyclopedy of any existing language, a good technique would be to input the password in "google" and see if something was found :-)

Don't use any personal detail (phone, ages, names, etc).

Any combination of these.

THESE METHODS ARE MORE EFFECTIVE:

Replace letters with numbers, for example "3" for "e", "4" for "A", "7" for "T".

Create a mnemonic from a phrase only you know, for example "i like linux and security" would be converted in "illas", add some numbers and your password would be secure, remember to use a larger than 8 digits password.

Mix uppercase and lowercase letters

Use special characters: "-", "!", ":", "@".

In Linux or Solaris it is possible to create secure passwords with the "mkpasswd" utility.

# mkpasswd -l 20
jnXbrScbzbtnwqg99hho

Forcing logrotation with logrotate

2008-06-24T23:08:00.005-03:00

In my previous post i explained how to setup a simple logrotation (http://serverlinux.blogspot.com/2006/02/logrotate-in-linux.html), now i would like to show a simple but usefull command that forces the logrotation, independent of the normal execution.

# logrotate -f /etc/logrotate.conf

For more information, see "man logrotate". LOGROTATE(8).

10 most important Unix Security issues

2008-06-23T21:57:00.007-03:00

A good place for security education is the SANS/FBI site (http://www.sans.org/top20/). These are the most important Unix related security issues:

1. Web Server. One of the places that an intruder is going to check first is for vulnerabilities in your Apache version and in you cgi-scripts.

2. Remote Procedure Calls. RPC Services should be down if they are not required, they allow a remote user to execute instructions in your computer; the intruder usualy gains root access this way.

3. SNMP (Simple Network Management Protocol). This protocol is known to have had its vulnerabilities and their password can be easily cracked and more easier captured from the network.

4. SSH (Secure Shell). SSH has been exploited before, if you do not need it then you can turn it off, or filter the source ip addresses with TCP Wrapper.

5. Remote Services (Trusted host). This was a setup in the machines based on the rely of other machines IP address, and leaved access without asking password. Their binaries are "rsh", "rcp", "rlogin" and "rexec". They exist and can be used also today, the attacked can do a party with your machine if they use a technique known as "ip spoofing".

6. FTP (File Transfer Protocol). Many vulnerabilities have been found in FTP, as exploits and protocol weaknesses, like clear text password transfer (resolved in SFTP).

7. LPD (Line Printer Daemon). This daemon is also remotely exploitable with help of an overflow and a shellcode, gaining root access if the server is running as root.

8. BIND/DNS (Dynamic Name Server). DNS Flooding, exploits and other attacks are available, if you are going to set up a DNS, use a firewall to filter any port that you do not want.

9. Sendmail. This mail transfer agent is known for its buffer overflows and remote exploits, though it has resolved its issues, always appears something new. It is recommended to use qmail.

10. Weak Password / No Passwords in the system. I do not need to explain this.

Many people that talk about security talk about a false sense of security that one can have in the cyberspace, i do not totaly agree with them, i see very often thay it is created a false sense of insecurity also. The items i have listed before create some sense of insecurity and alert; but do not worry, if you are going to run one of this critical services, just keep in mind:

* Use a well configured firewall (pay more attention to "well configured" than "firewall")
* Set up correctly an Intrusion Detection and Prevention System.
* Ask for help a security professional, here in Argentina we have very good ones :-)

Find out the php.ini location in your LAMP setup

2008-06-21T12:49:00.001-03:00

In the command line write:

echo "<?php phpinfo(); ?>" | php | grep php.ini

How to enable shadow passwords in Linux

2008-06-20T12:52:00.003-03:00

It is important that you use shadow passwords, because this puts another layer of security against password cracking and also enable the possibility to set policies about how often the password must be modified.

Using the Linux Shadow Suite
I am going to show you the "chage" command, it changes the number of days between password changes and the date of the last password change.

chage [-m mindays] [-M maxdays] [-d lastday] [-I inactive] [-E expiredate] [-W warndays] user

It is not recommended to modify the /etc/passwd or /etc/shadow file because this could result in the corruption if at the same time another process writes the files. To modify the /etc/passwd file, use the vipw command:

Vipw edits the password file after setting the appropriate locks, and does any necessary processing after the password file is unlocked. If the password file is already locked for editing by another user, vipw will ask you to try again later. The default editor for vipw is vi(1).

To list the users password expiration settins:

$ chage -l walter
Minimum: 0
Maximum: 99999
Warning: 6
Inactive: -1
Last Change: Apr 09, 2004
Password Expires: Never
Password Inactive: Never
Account Expires: Never

Suggestions:

* Set the minimum password lenght to eight characters.
* Force the users to change their passwords every four to six weeks.

Introduction to Iptables (Part 3)

2008-06-19T13:12:00.004-03:00

In this case i am going to explain some iptables features related with UDP. UDP (User Datagram Protocol) has the characteristic of being connectionless.

The packets format is this:
In this packet format can be seen that UDP has no flags like TCP. UDP cares only about the source and destination addresses.

In Iptables, udp is specified with the "-p udp" argument. Similar rules apply to udp than with TCP matching, negation and port ranges are allowed:

-sport (--source-port) <port>
--dport (--destination-port) <port>

This rule matches any UDP packet with source port of 161 (SNMP)
iptables -A INPUT -p udp --sport 161 -j ACCEPT

This rule logs all the packets with destination port with range from 161 to 180
iptables -A INPUT -p udp --dport 161:180 -j LOG

To learn more about UDP, see the RFS 768

Introduction to Iptables (Part 2)

2008-06-18T09:41:00.004-03:00

In my previous post i explained some general packet matching with Iptables, now i am going to explain the TCP packet matching generalities, where the commands displayed following, all match specific values from the TCP packet headers, for example the source and destination ports, tcp options, tcp flags (for example the syn, fin, etc).

You use the --protocol argument to match TCP packets, and optionally, the source port of the packet can be specified with "--sport (--source-port) ", the source port can be a numeric value or the name of the port, that should match the port number we want in the /etc/services file.

Here are two examples:
iptables -A INPUT -p tcp --sport 23 -j REJECT
iptables -A INPUT -p tcp --sport telnet -j REJECT

This rules do the same, they reject the inbound traffic from the TCP port 23 of the remote host). The usage of port names instead of the number, creates a little more cpu consume and could be told as "speed penalty" in large rulesets.

It is possible also to specify a range of ports in the rules, lower and upper port separated by a colon. Here i filter all the ports between 10 and 999:
iptables -A OUTPUT -p tcp --sport 10:999 -j REJECT

All TCP source ports except the 80 are accepted with this rule:
iptables -A INPUT -p tcp --sport ! 80 -j ACCEPT

The same can be done with a range of ports:
iptables -A INPUT -p tcp --sport ! 1024:40000 -j LOG

If the first port is numerically higher than the second, iptables swaps both numbers around automatically.

To specify a TCP destination port, the "-dport (--destination-port) [port]" is used, and its rules are the same than in TCP source port matching.

For example, to stop users in your private network to connect to IRC, supposing that IRC uses the ports between 6667 to 66670, you may want to add this rule:
iptables -A OUTPUT -p tcp --dport 6667:6670 -j REJECT

To match a specific flag in the TCP header, you have to make use of "--tcp-flags [mask] [flags]".

The [mask] argument is a list of flags separated by commas, which should be matched.
The [flags] argument is a list of flags that must be set, any flag listed in the [mask] argument, but not in the second, this means that the flag must be unset.

The possible flags are: SYN, ACK, FIN, RST, PSH and URG. ALL and NONE are also possible.

In this example, the SYN,ACK and FIN flags are the mask, and the SYN flag is the one that has to be set:
iptables -A FORWARD -p tcp --tcp-flags SYN,ACK,FIN SYN -j ACCEPT

The mask can also be inverted, meaning that the ACK and FIN should be set, but not the SYN:
iptables -A FORWARD -p tcp --tcp-flags ! SYN,ACK,FIN SYN -j ACCEPT

Matching a particular TCP flag:
This is accomplished with the "--syn" flag, it is usefull because the SYN flag is the TCP start sequence also known as the "3 way handshake", explained in this picture:

iptables -A FORWARD -p tcp --syn -j ACCEPT iptables -A FORWARD -p tcp ! --syn -j ACCEPT

Creating a complete LAMP solution in Centos Linux

2008-06-18T00:23:00.005-03:00

Download Apache: httpd-2.2.9.tgz
Download PHP: php-5.2.6.tgz

Uncompress and compile Apache:
./configure --with-pgsql --with-apxs2=/usr/local/apache/bin/apxs --with-xml --disable-dom --without-pear --with-pgsql --with-gd --with-zlib --enable-sockets

Uncompress and compile PHP:
./configure --with-pgsql --with-apxs2=/usr/local/apache/bin/apxs --with-xml --disable-dom --without-pear --with-pgsql --with-gd --with-zlib --enable-sockets --with-mysql

If you get this message:
If configure fails try --with-jpeg-dir==[dir]
configure: error: libpng.(a|so) not found.

Do the following:
Download libpng-1.2.29.tar.bz2 from http://www.libpng.org/pub/png/libpng.html
Uncompress and:
cd libpng-1.2.29
cp scripts/makefile.linux Makefile
make prefix=/usr && make install

Download http://www.zlib.net/zlib-1.2.3.tar.gz, uncompress it, then:
cd zlib-1.2.3
./configure --prefix=/usr && make && make install

To add jpeg support to PHP, download jpegsrc.v6b.tar.gz from ftp://ftp.uu.net/graphics/jpeg/
Uncompress it and: jpeg-6b]# ./configure --prefix=/usr && make && make install

Now configure PHP with jpeg and postgresql support:
./configure --with-pg --with-apxs2=/usr/local/apache/bin/apxs --disable-dom --without-pear --with-pgsql --with-gd --with-zlib --enable-sockets --with-jpeg-dir=../jpeg-6b/

Introduction to Iptables usage in Linux

2008-06-17T13:47:00.003-03:00

I am going to explain the generic matches, the ones that apply to all the IP packets. In general, the patch pattern looks like "-s (--src, --source).

For example:
iptables -A INPUT -s 10.10.10.5 -j DROP

The IP address could also be a hostname, in that case it would be resolved to an ip address before being added to the chain. The field of the IP address could also be a range of addresses using a netmask. This instruction is applied in the INPUT chain, but it could be used also in the OUTPUT chain if the machine has more than one ip address.

iptables -A INPUT -s 10.10.10.0/24 -j DROP

This instructions matches the first 24 bits of the address. This means, it matches addresses between 10.10.10.0 - 10.10.10.255.

iptables -A INPUT -s ! 10.10.10.5 -j DROP

The exclamation mark negates the ipaddress, this matches the packets where the source IP is no 10.10.10.5

The "-d (--dst --destination)" matches the destination address of the packets and is used generaly on the OUTPUT chain. The same rules than in the -s option apply (address ranges can be specified as hostnames, a single IP address or a range, and negation of the addresses). For example:

iptables -A OUTPUT -d ! 10.10.10.4 -j REJECT

The "-i (--in-interface) " specify on which network interface the rule should take effect, for example "eth0". This options could be used in the FORWARD, INPUT and PREROUTING chains. The network interface also accepts wildcards, for example, if you want to filter all the traffic from a privat eaddress such as 10.10.10.2 in all the interfaces:

iptables -A INPUT -s 10.10.10.2 -i eth* -j DROP

This would drop all packets arriving on eth0, eth1, eth2, etc.

A final "-p" option allows to work with a specific protocol, for example, if you want to drop all the UDP packets:

iptables -A INPUT -p udp -j DROP

The protocols that can be used are:
TCP, UDP, ICMP, ALL (this is for all the protocols).

Analysing logs in Linux

2008-06-16T18:11:00.002-03:00

Part of the security and sysadmins tasks is the log analysis and decision taking. There is plenty of information in http://www.linux.org/apps/all/Administration/Log_Analyzers.html.

The tools i recommend is called "Lire", this tool permits the creation of several reporting formats, including html, pdf, xml, between others. It also permits to analyze many log file formats, which include MySQL, Iptables, BIND, Apache, Qmail, Postfix, Syslog and more. Lire is GPL'ed Free Software (and Open Source), built around the idea of extendibility.

This tool is available from http://www.logreport.org/lire, it has been deveploped in Perl and i recommend you to install all the dependence modules with CPAN (type "perl -M CPAN -e shell" on the command line as root).

Logging on a remote server with syslog

2008-06-15T19:30:00.001-03:00

Hi, it is a good security feature to log in a remote host, because an attacker should have access to that host to delete the logs, and this adds another security layer to the architecture you build.

Linux logging facilities are managed mostly by the syslog Daemon. Syslog uses the configuration file /etc/syslog.conf to know where to log every system message.

If you want to log in a remote host, you would add this line to syslog.conf

auth.*; authpriv.* @192.168.100.7

It is required that the syslog daemon is started in the 192.168.100.7 host and it should also, be listening for messages thay come from the network. It is important that you know that this feature is by default "off". You have to start syslog with a "-r" argument to enable this.

To force syslog to re-read syslog.conf, send a SIGHUP signal to syslog.

Happy logging !

Loading modules in RHEL 3.0 and prior Linux versions

2008-06-14T10:37:00.004-03:00

In RHEL 3.0 when you install a new module you may be tempted to put them in the file modules.conf. This may not work, because the modules are not in the initrd boot image.

The solution is to create a new file called /etc/rc.modules which loads the modules, this file do not exist by default, but if you see the file /etc/rc.d/rc.sysinit there you are going to see that it checks for the existence of the file, and if it exists, it executes it:

# Load modules (for backward compatibility with VARs)
if [ -f /etc/rc.modules ]; then
/etc/rc.modules
fi

This is the procedure to create the file:

echo "modprobe qla2300" >> /etc/rc.modules
echo "modprobe qla2300_conf" >> /etc/rc.modules
chmod 700 /etc/rc.modules

Infinite loops with shell scripting

2008-03-31T17:38:00.004-03:00

This way one can create useful infinite loops:

while [ 1 ]; do echo "1"; done

for ((;;)); do echo $RANDOM; sleep 1; done

The $RANDOM variable has a random number.

How to use ssh through a proxy in Ubuntu

2008-02-21T15:55:00.003-02:00

This time i had to connect through an http proxy to other machines using ssh, this took me some investigation, mather that i like ;-)

First i downloaded the connect-proxy package, the source code is also available but i had problems while compiling it, so i downloaded it from here:
http://archive.ubuntu.com/ubuntu/pool/universe/c/connect-proxy/connect-proxy_1.95-3_i386.deb

I had to download it withe Firefox, because i was behind a proxy and do not had still the possibility to download other ways. wget probably accepts a proxy argument, but i used the fastest way.

Then, i installed it:
dpkg -i /home/walter/connect-proxy_1.95-3_i386.deb

Then, create a file ~/.ssh/config, whit this:
---
Host *
ProxyCommand connect-proxy -H proxy2.your.proxy.com:8080 %h %p
---

Finally, test the connection:
ssh -F .ssh/config mysite.argentina.com.ar -p 2210

And i did a tunnel to another application:
ssh -F .ssh/config -g -L 8000:192.168.2.1:2211 mysite.argentina.com.ar -p 2210

For more information about how to do a tunnel, see my previous post:
Secure Tunnel