Linux Tips, Troubleshooting, Package, Installation Tools

ipsec tunnel pfSense and Centos

2011-01-04T14:43:00.000+04:00

pfSense 1.2.3

--------

external ip: 1.1.1.1

internal ip: 172.20.1.20

internal network: 172.20.1.0/24

Centos 5.5

--------

external ip: 2.2.2.2

internal ip: 172.20.2.1

internal network: 172.20.2.0/24

pfSense config from a reset.

Firewall rule to allow all ipsec communication (all protocols).

pfSense ipsec config

--------------------

Mode: Tunnel

Interface: WAN (I'm not sure this should be WAN, but changing it to LAN makes no difference)

Local subnet: 172.20.1.0/24

Remote subnet: 172.20.2.0/24

Remote gateway: 2.2.2.2

Phase 1

Negotiation mode: agressive

My identifier: My IP adress

Encryption algorithm: 3DES

Hash algorithm: SHA1

DH key group: 2

Authentication method: Pre-shared key

Pre-Shared Key: secret

Phase 2

Protocol: ESP

Encryption algorithms: Rijndael (AES)

Hash algorithms: SHA1

PFS key group: 2

Centos ipsec config

-------------------

/etc/sysconfig/network-scripts/ifcfg-ipsec0

TYPE=IPSEC

ONBOOT=yes

IKE_METHOD=PSK

SRCGW=172.20.2.1

DSTGW=172.20.1.20

SRCNET=172.20.2.0/24

DSTNET=172.20.1.0/24

DST=1.1.1.1

/etc/sysconfig/network-scripts/keys-ipsec0

IKE_PSK=secret

/etc/racoon/racoon.conf

path include "/etc/racoon";

path pre_shared_key "/etc/racoon/psk.txt";

path certificate "/etc/racoon/certs";

sainfo anonymous

{

pfs_group 2;

lifetime time 1 hour ;

encryption_algorithm rijndael ;

authentication_algorithm hmac_sha1 ;

compression_algorithm deflate ;

}

include "/etc/racoon/1.1.1.1.conf";

/etc/racoon/1.1.1.1.conf

remote 1.1.1.1

{

exchange_mode aggressive, main;

my_identifier address;

proposal {

encryption_algorithm 3des;

hash_algorithm sha1;

authentication_method pre_shared_key;

dh_group 2;

}

}

Ipsec log of pfSense

--------------------

Nov 28 19:38:11 racoon: INFO: @(#)ipsec-tools 0.7.2 (http://ipsec-tools.sourceforge.net)

Nov 28 19:38:11 racoon: INFO: @(#)This product linked OpenSSL 0.9.8e 23 Feb 2007 (http://www.openssl.org/)

Nov 28 19:38:11 racoon: INFO: Reading configuration from "/var/etc/racoon.conf"

Nov 28 19:38:11 racoon: [Self]: INFO: 127.0.0.1[500] used as isakmp port (fd=14)

Nov 28 19:38:11 racoon: [Self]: INFO: 1.1.1.1[500] used as isakmp port (fd=15)

Nov 28 19:38:11 racoon: [Self]: INFO: 172.20.1.20[500] used as isakmp port (fd=16)

Nov 28 19:38:11 racoon: INFO: unsupported PF_KEY message REGISTER

Nov 28 19:38:11 racoon: ERROR: such policy already exists. anyway replace it: 172.20.1.0/24[0] 172.20.2.0/24[0] proto=any dir=out

Nov 28 19:38:11 racoon: ERROR: such policy already exists. anyway replace it: 172.20.2.0/24[0] 172.20.1.0/24[0] proto=any dir=in

Nov 28 19:38:11 racoon: [Self]: INFO: 127.0.0.1[500] used as isakmp port (fd=14)

Nov 28 19:38:11 racoon: [Self]: INFO: 1.1.1.1[500] used as isakmp port (fd=15)

Nov 28 19:38:11 racoon: [Self]: INFO: 172.20.1.20[500] used as isakmp port (fd=16)

Nov 28 19:41:11 racoon: [IPsec tunnel]: INFO: respond new phase 1 negotiation: 1.1.1.1[500]<=>2.2.2.2[500]

Nov 28 19:41:11 racoon: INFO: begin Aggressive mode.

Nov 28 19:41:11 racoon: INFO: received Vendor ID: DPD

Nov 28 19:41:11 racoon: NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.

Nov 28 19:41:11 racoon: [IPsec tunnel]: INFO: ISAKMP-SA established 1.1.1.1[500]-2.2.2.2[500] spi:36580e589a8cb1e8:f3cef1de14e3ec5e

Nov 28 19:41:12 racoon: [IPsec tunnel]: INFO: respond new phase 2 negotiation: 1.1.1.1[0]<=>2.2.2.2[0]

Nov 28 19:41:12 racoon: ERROR: not matched

Nov 28 19:41:12 racoon: ERROR: no suitable policy found.

Nov 28 19:41:12 racoon: ERROR: failed to pre-process packet.

Nov 28 19:41:22 racoon: [IPsec tunnel]: INFO: respond new phase 2 negotiation: 1.1.1.1[0]<=>2.2.2.2[0]

Nov 28 19:41:22 racoon: ERROR: not matched

Nov 28 19:41:22 racoon: ERROR: no suitable policy found.

Nov 28 19:41:22 racoon: ERROR: failed to pre-process packet.

Nov 28 19:41:32 racoon: [IPsec tunnel]: INFO: respond new phase 2 negotiation: 1.1.1.1[0]<=>2.2.2.2[0]

Nov 28 19:41:32 racoon: ERROR: not matched

Nov 28 19:41:32 racoon: ERROR: no suitable policy found.

Nov 28 19:41:32 racoon: ERROR: failed to pre-process packet.

/var/log/messages of Centos

----------------------------

Nov 28 19:40:34 racoon: INFO: unsupported PF_KEY message REGISTER

Nov 28 19:40:34 racoon: INFO: 127.0.0.1[500] used as isakmp port (fd=37)

Nov 28 19:40:34 racoon: INFO: 127.0.0.1[500] used for NAT-T

Nov 28 19:40:34 racoon: INFO: 2.2.2.2[500] used as isakmp port (fd=38)

Nov 28 19:40:34 racoon: INFO: 2.2.2.2[500] used for NAT-T

Nov 28 19:40:34 racoon: INFO: 172.20.2.1[500] used as isakmp port (fd=39)

Nov 28 19:40:34 racoon: INFO: 172.20.2.1[500] used for NAT-T

Nov 28 19:40:34 racoon: INFO: ::1[500] used as isakmp port (fd=40)

Nov 28 19:40:34 racoon: INFO: fe80::216:1aff:fed9:1fc5%eth0[500] used as isakmp port (fd=41)

Nov 28 19:40:44 racoon: INFO: unsupported PF_KEY message REGISTER

Nov 28 19:40:44 last message repeated 4 times

Nov 28 19:40:44 racoon: ERROR: such policy already exists. anyway replace it: 172.20.2.0/24[0] 172.20.1.0/24[0] proto=any dir=out

Nov 28 19:40:44 racoon: ERROR: such policy already exists. anyway replace it: 172.20.1.0/24[0] 172.20.2.0/24[0] proto=any dir=in

Nov 28 19:40:44 racoon: ERROR: such policy already exists. anyway replace it: 172.20.1.0/24[0] 172.20.2.0/24[0] proto=any dir=fwd

Nov 28 19:40:44 racoon: INFO: 127.0.0.1[500] used as isakmp port (fd=38)

Nov 28 19:40:44 racoon: INFO: 127.0.0.1[500] used for NAT-T

Nov 28 19:40:44 racoon: INFO: 2.2.2.2[500] used as isakmp port (fd=39)

Nov 28 19:40:44 racoon: INFO: 2.2.2.2[500] used for NAT-T

Nov 28 19:40:44 racoon: INFO: 172.20.2.1[500] used as isakmp port (fd=40)

Nov 28 19:40:44 racoon: INFO: 172.20.2.1[500] used for NAT-T

Nov 28 19:40:44 racoon: INFO: ::1[500] used as isakmp port (fd=41)

Nov 28 19:40:44 racoon: INFO: fe80::216:1aff:fed9:1fc5%eth0[500] used as isakmp port (fd=42)

Nov 28 19:41:09 racoon: INFO: IPsec-SA request for 1.1.1.1 queued due to no phase1 found.

Nov 28 19:41:09 racoon: INFO: initiate new phase 1 negotiation: 2.2.2.2[500]<=>1.1.1.1[500]

Nov 28 19:41:09 racoon: INFO: begin Aggressive mode.

Nov 28 19:41:09 racoon: INFO: received Vendor ID: DPD

Nov 28 19:41:09 racoon: NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.

Nov 28 19:41:09 racoon: INFO: ISAKMP-SA established 2.2.2.2[500]-1.1.1.1[500] spi:36580e589a8cb1e8:f3cef1de14e3ec5e

Nov 28 19:41:10 racoon: INFO: initiate new phase 2 negotiation: 2.2.2.2[500]<=>1.1.1.1[500]

Nov 28 19:41:10 racoon: ERROR: unknown notify message, no phase2 handle found.

Nov 28 19:41:30 last message repeated 2 times

Nov 28 19:41:40 racoon: INFO: IPsec-SA expired: AH/Tunnel 1.1.1.1[0]->2.2.2.2[0] spi=206806147(0xc539c83)

Nov 28 19:41:40 racoon: WARNING: the expire message is received but the handler has not been established.

Nov 28 19:41:40 racoon: ERROR: 1.1.1.1 give up to get IPsec-SA due to time up to wait.

Nov 28 19:41:40 racoon: INFO: IPsec-SA expired: ESP/Tunnel 1.1.1.1[0]->2.2.2.2[0] spi=72570967(0x4535857)

changed the config of the Centos computer

/etc/sysconfig/network-scripts/ifcfg-ipsec0

TYPE=IPSEC

ONBOOT=yes

IKE_METHOD=PSK

AH_PROTO=none

SRCGW=172.20.2.1

DSTGW=172.20.1.20

SRCNET=172.20.2.0/24

DSTNET=172.20.1.0/24

DST=1.1.1.1

Top 10 Open Source Firewall

2010-10-20T15:09:00.000+04:00

With perilous threats from crackers and script kiddes lurking in the network, IT administrators could do no better than placing a firewall protection. Firewall prevents unwanted access to departmental systems while preventing local systems from attacking systems on the other network. It ensures that the traffic entering and leaving the secured LAN is accessing the correct applications on the correct computers. We had already done with the top free Windows firewall. However, there is cool open source firewall to take advantage of. Open source firewall not only offers better customization options, but also reduces the cost of ownership. After a comprehensive search we assorted the top 10 open source firewall.

1. Endian Firewall

This is an open source firewall based on the IPCop Linux Firewall. It is one of the most widely used open source firewall with comprehensive features. It is almost an opensource Universal threat Management (UTM) device with a Statefull firewall, VPN, Webproxy, SIP proxy for VoIP, Web Security, Content Filtering, Mail gateway and Antivirus, Anti-spyware and Anti-phishing capabilities.

Some of the crucial features of Endian Firewall include

 Easy to configure and administer web interface
 Routing and NAT support
 Port forwarding
 NTP client and NTP Server support
 Reporting with system, network and traffic logging
 Remote Sysol server support
 IPSec VPN for site to site VPN and remote users support with Endian VPN client (Windows, MacOSX, Linux)using OpenVPN
 Mail security for SMTP and POP3 with Antivirus,Anti-Phishing, Anti-spyware
 Webproxy supporting HTTP,FTP and DNS bypass
 SPAM protection using pyzor Spam filter
 DNS bypass and Dynamic DNS support
 SIP proxy for VoIP
 Antivirus support provided by ClamAV
 Traffic Shapping on the Internet facing interface for QoS
 Traffic Monitoring using nTop
 DNS and DHCP server using dnsmasq
 Zone based Firewalling
 User Management with Windows Domain, LDAP, RADIUS and Samba support

Endian Firewall

2010-10-20T15:08:00.000+04:00

This is an open source firewall based on the IPCop Linux Firewall. It is one of the most widely used open source firewall with comprehensive features. It is almost an opensource Universal threat Management (UTM) device with a Statefull firewall, VPN, Webproxy, SIP proxy for VoIP, Web Security, Content Filtering, Mail gateway and Antivirus, Anti-spyware and Anti-phishing capabilities.

Some of the crucial features of Endian Firewall include

 Easy to configure and administer web interface

 Routing and NAT support

 Port forwarding

 NTP client and NTP Server support

 Reporting with system, network and traffic logging

 Remote Sysol server support

 IPSec VPN for site to site VPN and remote users support with Endian VPN client (Windows, MacOSX, Linux)using OpenVPN

 Mail security for SMTP and POP3 with Antivirus,Anti-Phishing, Anti-spyware

 Webproxy supporting HTTP,FTP and DNS bypass

 SPAM protection using pyzor Spam filter

 DNS bypass and Dynamic DNS support

 SIP proxy for VoIP

 Antivirus support provided by ClamAV

 Traffic Shapping on the Internet facing interface for QoS

 Traffic Monitoring using nTop

 DNS and DHCP server using dnsmasq

 Zone based Firewalling

 User Management with Windows Domain, LDAP, RADIUS and Samba support

Best open source firewall 2011

2010-10-20T15:07:00.000+04:00

Top 5 Open Source Firewall

1) IPTable

User, which allows a system administrator to configure the tables provided by the firewall of the Linux kernel (as units executed Netfilter different) rules and chains and stores. And is currently using the kernel modules and different programs for different protocols; iptables applies to IPv4, ip6tables to IPv6, arp tables for the rehabilitation of agriculture, as a special ebtables Ethernet frames

IPtables require elevated privileges to work must be carried out by the root user, but it failed to function. In most Linux systems, iptables is installed as /usr/directory/iptables and documented in the man page to him [2], which can be opened using `man iptables` when installed. Can also be found in the /sbin/iptables, but since iptables is not "dual core", but more like a service, is still the preferred place / usr / Guide

2) IPCop

The IPCop Firewall is a Linux firewall distribution. It is geared towards home and SOHO users. The IPCop web-interface is very user-friendly and makes usage easy.
The (old) information from http://www.ipcop.org can be found here: http://www.ipcop.org/index-pn.php Be aware that most of the information is old and obsolete.
One day there may be a flashy, web-2.0 style homepage, but until then the page you are now reading is the home of IPCop.

3) ShoreWall

Shorewall is a gateway/firewall configuration tool for GNU/Linux.

For a high level description of Shorewall, see the Introduction to Shorewall.

Uses Netfilter's connection tracking facilities for stateful packet filtering.
Can be used in a wide range of router/firewall/gateway applications.
Supports centralized firewall administration.
QuickStart Guides (HOWTOs)
Extensive documentation is available in both Docbook XML and HTML formats.
Flexible address management/routing support (and you can use all types in the same firewall):
Blacklisting of individual IP addresses and subnetworks is supported.
VPN Support.
Support for Traffic Control/Shaping.
Media Access Control (MAC) Address Verification.
Traffic Accounting.
Bridge/Firewall support
IPv6 Support
Works with a wide range of Virtualization Solutions

Additional Firewall Software

Following are additional firewalls mentioned by readers along with the total number of votes it received.

CheckPoint FireWall-1 5
pfsense 5
Firestarter 5
Netfilter 4
SmoothWall Express 3
Guarddog 3
ipchain 3
Endian 2
Susefirewall 1
Cisco ASA/PIX 1
ClearOS 1
APF 1
Firewall Builder 1
Auto firewall in Puppy Linux 1
Drawbridge 1
Monowall 1
Firehol 1
SuSEfirewall2 1
Plesk 1

Configuring the virtual path devices

2010-09-21T17:47:00.000+04:00

We recommend that you start with a "fresh" disk configuration, so it is a good idea to delete all previously configured FC adapters and their child (disk) devices.

On node1, we checked which disks are still defined:

{node1:root}/-> lspvhdisk0 0022be2ab1cd11ac rootvg activehdisk1 0022be2a3d02ead0 Nonehdisk2 0022be2a4cbbafd8 Nonehdisk3 none NoneThese are the internal SCSI disk drives

{node1:root}/-> lscfg
grep disk+ hdisk3 U1.9-P2/Z2-A8 16 Bit LVD SCSI Disk Drive (36400 MB)+ hdisk2 U1.9-P2/Z1-A8 16 Bit LVD SCSI Disk Drive (36400 MB)+ hdisk1 U1.9-P1/Z2-A8 16 Bit LVD SCSI Disk Drive (36400 MB)+ hdisk0 U1.9-P1/Z1-A8 16 Bit LVD SCSI Disk Drive (36400 MB)In order to include the ESS disks, run the configuration manager on each node:

{node1:root}/-> cfgmgr -vSince ESS was configured with two host paths for each node (node1a and node1b), this results in two hdisks on the nodes. Actually, those two logical hdisks represent the same physical disk, accessed via the two ESS configured paths.

When the SDD driver package is installed, a third disk entry, named vpath* (virtual path) is automatically created. To benefit from the SDD capabilities, when configuring volume groups, use this virtual path disk entry instead of the regular hdisk. This provides for load balancing and high availability.

In normal mode, storage traffic is balanced over the two FC adapters installed. In case one of the two FC adapters fails for any reason, data access continues over the surviving adapter.

Check the new disk configuration discovered by cfgmgr:

Example 3-13 Listing the virtual path devices

{node1:root}/-> lsdev -Cs dpovpath0 Available Data Path Optimizer Pseudo Device Driver <-+-vpath1 Available Data Path Optimizer Pseudo Device Driver
vpath2 Available Data Path Optimizer Pseudo Device Driver
Each vpath vpath3 Available Data Path Optimizer Pseudo Device Driver
represents onevpath4 Available Data Path Optimizer Pseudo Device Driver
physical LUN.vpath5 Available Data Path Optimizer Pseudo Device Driver
vpath6 Available Data Path Optimizer Pseudo Device Driver <-+-

Example 3-14 Listing the FC attached disks (LUNs)

{node1:root}/-> lsdev -Cs fcphdisk4 Available 2V-08-01 IBM FC 2105800 <--+-hdisk5 Available 2V-08-01 IBM FC 2105800
hdisk6 Available 2V-08-01 IBM FC 2105800
hdisk7 Available 2V-08-01 IBM FC 2105800
hdisk8 Available 2V-08-01 IBM FC 2105800
hdisk9 Available 2V-08-01 IBM FC 2105800
These are the FC disks; there are onlyhdisk10 Available 2V-08-01 IBM FC 2105800
seven physical LUNs, but they are hdisk11 Available 2v-08-01 IBM FC 2105800
doubled because each node is connectedhdisk12 Available 2v-08-01 IBM FC 2105800
to the storage using two FC adapters.hdisk13 Available 2v-08-01 IBM FC 2105800
hdisk14 Available 2v-08-01 IBM FC 2105800
hdisk15 Available 2v-08-01 IBM FC 2105800
hdisk16 Available 2v-08-01 IBM FC 2105800
hdisk17 Available 2v-08-01 IBM FC 2105800 <--+-

Important: depending on the number of hdisks defined on a node prior to the ESS disk setup, the numbering of the new hdisks and vpaths may vary from one node to another. The only information you should rely on is the PVID (Physical Volume IDentifier) of the disk. This identifier is written on the disk, and retrieved by each node. It is the second column of the lspv command output.

To check the physical volumes, execute the following command:

Example 3-15 List of vpath devices created by SDD (extract)

{node1:root}/-> lspvhdisk0 0022be2ab1cd11ac rootvg activehdisk1 0022be2a3d02ead0 None...hdisk9 0022be2a31fa6b48 None...hdisk16 0022be2a31fa6b48 Nonehdisk17 none Nonevpath0 none Nonevpath1 none Nonevpath2 none Nonevpath3 none Nonevpath4 none Nonevpath5 none Nonevpath6 none None

To make these disks available to all the cluster nodes, type:

{node1:root}/-> chdev -l vpath5 -a pv=yes{node1:root}/-> lspv
grep vpath5vpath5 0022be2a31fa6b48 NoneDisk Name PVID Volume Group

Note: To benefit from high availability and load balancing of SDD, only the vpath must be used for further LVM configurations (only vpaths should be used for volume groups).

.The SDD driver provides a set of commands to manage the virtual path devices:

Example 3-16 Datapath command arguments

datapath query adapter [n] datapath query device [n] datapath set adapter online/offline datapath set device path online/offline datapath set device /( ) policy rr/fo/lb/df datapath query adaptstats [n] datapath query devstats [n] datapath open device path

To see the correspondence between the hdisks and vpaths, use the following command:

Example 3-17 Datapath query device command

{node1:root}/-> datapath query device

Total Devices : 7

DEV#: 0 DEVICE NAME: vpath0 TYPE: 2105800 SERIAL: 30022513

POLICY: Optimized

==========================================================================

Path# Adapter/Hard Disk State Mode Select Errors

0 fscsi0/hdisk4 OPEN NORMAL 253908 0

1 fscsi1/hdisk11 OPEN NORMAL 255351 0

DEV#: 1 DEVICE NAME: vpath1 TYPE: 2105800 SERIAL: 30122513

POLICY: Optimized

==========================================================================

Path# Adapter/Hard Disk State Mode Select Errors

0 fscsi0/hdisk 5 OPEN NORMAL 1749 0

1 fscsi1/hdisk 12 OPEN NORMAL 1661 0

DEV#: 2 DEVICE NAME: vpath2 TYPE: 2105800 SERIAL: 30222513

POLICY: Optimized

==========================================================================

Path# Adapter/Hard Disk State Mode Select Errors

0 fscsi0/hdisk 6 OPEN NORMAL 1755 0

1 fscsi1/hdisk 13 OPEN NORMAL 1773 0

DEV#: 3 DEVICE NAME: vpath3 TYPE: 2105800 SERIAL: 30322513

POLICY: Optimized

==========================================================================

Path# Adapter/Hard Disk State Mode Select Errors

0 fscsi0/hdisk 7 OPEN NORMAL 263 0

1 fscsi1/hdisk1 4 OPEN NORMAL 278 0

DEV#: 4 DEVICE NAME: vpath4 TYPE: 2105800 SERIAL: 30422513

POLICY: Optimized

==========================================================================

Path# Adapter/Hard Disk State Mode Select Errors

0 fscsi0/hdisk 8 OPEN NORMAL 1696 0

1 fscsi1/hdisk 15 OPEN NORMAL 1670 0

DEV#: 5 DEVICE NAME: vpath5 TYPE: 2105800 SERIAL: 30522513

POLICY: Optimized

==========================================================================

Path# Adapter/Hard Disk State Mode Select Errors

0 fscsi0/hdisk 9 CLOSE NORMAL 1485 0

1 fscsi1/hdisk 16 CLOSE NORMAL 1455 0

DEV#: 6 DEVICE NAME: vpath6 TYPE: 2105800 SERIAL: 30622513

POLICY: Optimized

==========================================================================

Path# Adapter/Hard Disk State Mode Select Errors

0 fscsi0/hdisk 10 CLOSE NORMAL 0 0

1 fscsi1/hdisk 17 CLOSE NORMAL 0 0

ESS Subsystem Device Driver setup

2010-09-21T17:08:00.002+04:00

In a high availability environment, there is a special device driver, designed for ESS, named Subsystem Device Driver (SDD). This device driver allows for redundant links and load sharing for storage traffic when multiple fiber connections exist between nodes and the ESS storage subsystem.

SDD comes as an AIX installable fileset, named ibm2105.rte. This has to be installed on all cluster nodes, even if not all nodes in the cluster have more than one FC adapter.

In our configuration, since each node is connected to the ESS using two optical cables, each disk can be accessed via any of the two paths. When SDD is installed, a virtual path is created. This virtual path represents the same storage space, but is accessible via both fiber links.

There are two versions of the SDD driver for AIX:

ibmSdd_510.rte - This is suitable for non-HACMP configurations or for concurrent HACMP (HACMP/ESCRM).

ibmSdd_510nchamp.rte - This has to be used in nonconcurrent HACMP environments.

In our environment, we are using HACMP, but not with concurrent disk access (locking is provided by GPFS and Oracle), thus we installed ibmSdd_510nchamp.rte.

See Appendix A, Operating system fileset levels for more details on the fileset names and releases.

The latest SAN software can be downloaded from:

http://ssddom02.storage.ibm.com/techsup/webnav.nsf/support/2105

Dynamic tracking of Fibre Channel adapters

2010-09-21T17:07:00.000+04:00

AIX 5.2 provides support for Dynamic Tracking of Fibre Channel devices. In previous AIX releases, the user was required to unconfigure the FC storage device and adapter device instances before making changes on the SAN that might result in an N_Port ID (SCSI ID) change of any remote storage ports.

If Dynamic Tracking for FC devices is enabled, the FC adapter driver will detect when the Fibre Channel N_Port ID of a device changes, and will reroute traffic designated for that device to the new address, while the devices are still online. Examples of events that can cause an N_Port ID to change are moving a cable between a switch and storage device from one switch port to another, connecting two separate switches via an Inter-Switch Link (ISL), and possibly rebooting a switch.

Dynamic tracking of FC devices is controlled by a new fscsi device attribute, named dyntrk. The default setting for this attribute is no. Setting this attribute to yes enables dynamic tracking:

{node1:root}/-> chdev -l fscsi0 -a dyntrk=yes

Dynamic tracking logic is invoked when the adapter driver receives an indication from the switch that there has been a link event involving a remote storage device port.

Enable Fast I/O Failure for FC adapt

2010-09-21T17:05:00.000+04:00

AIX 5.2 supports the Fast I/O Failure feature for Fibre Channel devices in case of link events in a switched environment. If the FC adapter driver detects a link event (for example, a lost link between a storage device and a switch), the FC adapter driver waits a short period of time, about 15 seconds, to allow the SAN fabric to stabilize. At this point, if the FC adapter driver detects that the device is not on the SAN fabric, it begins failing all I/O requests at the adapter driver level. Any new I/O request, or future retries of the previously failed I/Os, are failed immediately by the adapter, until the adapter driver detects that the device has rejoined the SAN fabric.

Fast I/O Failure is controlled by a new fscsi device attribute, fc_err_recov. The default setting for this attribute is delayed_fail, which is the I/O failure behavior that has existed in previous versions of AIX. Setting this attribute to fast_fail enables Fast I/O Failure, as shown in Example 3-12.

Example 3-12 FC adapter fast_fail option

{node1:root}/_> chdev -l fscsi0 -a fc_err_recov=fast_fail

The fscsi0 device is used for storage data transfer and is a child device of fcs0 (the FC adapter):

+ fcs0 U0.1-P1-I4/Q1 FC Adapter* fscsi0 U0.1-P1-I4/Q1 FC SCSI I/O Controller Protocol DeviceFast fail logic is invoked when the adapter driver receives an indication from the switch that there has been a link event involving a remote storage device port via a Registered State Change Notification (RSCN) from the switch.

Fast I/O Failure may be desirable in situations where multipath software is being used. Setting fc_err_recov to fast_fail may decrease the I/O fail times due to link loss between the storage device and switch and allow faster failover to alternate paths.

In single-path configurations, especially in configurations with a single path to a paging device, the default delayed_fail setting is recommended.

Fast I/O Failure is only supported in a switched environment. It is not supported in arbitrated loop environments, including public loop.

We configured Fast I/O Failure since we were using the Subsystem Device Driver, two adapters per node and two paths to the storage subsystem.

The requirements for Fast I/O Failure support are the following:

FC 6227 adapter firmware - level 3.22A1 or greater

FC 6228 adapter firmware - level 3.82A1 or greater

FC 6239 adapter firmware - all firmware levels

Configuring RAC logical disks AIX

2010-09-21T17:03:00.000+04:00

There are two FC protocols, depending on whether an FC switch (fabric configuration) or an FC HUB (arbitrated loop) is used:

The nodes are connected directly to the ESS, excluding any network equipment. Choose Arbitrated Loop (al protocol). This is the default value.

The nodes are connected to a SAN, using a switch. Select Point-to-Point (pt2pt protocol).

Because our platform is using a switched SAN, the point-to-point protocol is selected on all nodes and ESS Fibre Channel adapters.

To view the actual value set on the FC adapter:

{node3:root}/-> lsattr -El fcs0
grep init_linkinit_link al INIT Link flags TrueTo change this value, use "smitty devices", select the FC Adapter menu, and change the field INIT Link flags. Or, you can use the command:

{node3:root}/-> chdev -l fcs0 -a init_link=pt2pt

If you are unsure about the way your nodes are connected to the disk subsystems, choose init_link=al. This setting first tries to detect a switch. If that fails, it changes to arbitrated loop. It is like an auto detect feature. The FC adapter will recognize the device it is connected to, that is, FC hub or switch.

Check whether all fiber cables are connected (nodes to FC switch, FC switch to ESS). This step assumes that the ESS storage configuration has been performed and checked.

For consistency, we recommend that you remove any previous definition of the FC adapters and their child devices, then run a fresh cfgmgr. On each node and for each FC adapter, issue:

{node3:root}/-> rmdev -Rdl fcs0

{node3:root}/-> cfgmgr

The HACMP clstat command does not work

2010-09-21T17:00:00.000+04:00

Clinfo daemon started?

The clstat command retrieves the information from the clinfo daemon. This daemon is not started automatically. It is an option in the SMIT HACMP startup menu (smitty clstart). If it is not running, stop HACMP on this node, and start it up again with clinfo activated.

To check the status of the HACMP processes: lssrc -a | grep -E "ES | svcs"

SNMP version in AIX 5L version 5.2

AIX 5L version 5.2 uses SNMP version 3 agents, whereas HACMP uses a SNMP version 1 configuration. Both clinfo and CSPOC require SNMP version 1. With the default SNMP agent, the clinfo daemon is not able to supply cluster information, thus the clinfo command will not work properly.

Run the script provided in Example 3-38 to switch to version 1 agents.

Example 3-38 Switch SNMP agents to version 1

#!/bin/kshLOG=/var/adm/chg_snmpd.logtouch $LOG/usr/es/sbin/cluster/utilities/clstop -N -g >> $LOG 2>&1sleep 5for i in muxatmd aixmibd snmpmibd hostmibd snmpddostopsrc -s $i >> $LOG 2>&1sleep 2doneln -sf /usr/sbin/snmpdv1 /usr/sbin/snmpdfor i in snmpd hostmibd snmpmibd aixmibd muxatmddostartsrc -s $i >> $LOG 2>&1sleep 2donesleep 10/usr/es/sbin/cluster/etc/rc.cluster -N -i >> $LOG 2>&1sleep 5

You can also use the following command to change the running SNMP configuration back to version 1:

/usr/sbin/snmpv3_ssw -1

For security reasons do not use the default "public" SNMP community. You can change the community name in the /etc/snmpd.conf file, then stop cluster services (incl. clinfo) and specify this new name to clinfo: chssys -s clinfo(ES) -a "-c NEW_COMMUNITY_NAME" , then restart HACMP.

Oracle RAC does not start

2010-09-21T16:56:00.003+04:00

HACMP started

Before starting Oracle9i RAC instances, the HACMP cluster must be up and running. Check the cluster state with the /usr/sbin/cluster/clstat command.

Hagsuser group

These tests must be performed on all nodes:

Check that the Oracle user is part of the hagsuser group. The name of this group is mandatory, and cannot be changed.

Check, and change if necessary, the permissions on the cldomain executable. This program must be executable by everybody (user, group, other):

# chmod a+x /usr/es/sbin/cluster/utilities/cldomain

Check, and change the group to hagsuser if necessary, for the svcsdsocket.oracle9irac socket file: (assuming that oracle9irac is the name of your cluster, returned by the cldomain command):

{node2:root}/-> chgrp hagsuser /var/ha/soc/grpsvcsdsocket.oracle9irac

Check, and change if necessary, the group permissions for the grpsvcsdsocket.oracle9irac socket:

{node2:root}/-> chmod g+w /var/ha/soc/grpsvcsdsocket.oracle9irac

The HAGS socket file needs to be writable by the Oracle user and the cldomain executable needs to be executable by Oracle. By configuring the group and permissions for the grpsvcsdsocket.oracle9irac file, the instance will be able to communicate with HAGS and the instance will mount.

See Example 3-39 for the valid permissions and group membership for the socket files used by Oracle9i RAC.

Example 3-39 Permissions on the /var/ha/soc directory

{node2:root}/var/ha/soc-> ls -ltotal 0
srw-rw-rw- 1 root haemrm 0 Jun 10 18:18 em.clsrv.oracle9irac
srw-rw---- 1 root haemrm 0 Jun 10 18:18 em.rmsrv.oracle9irac
drwxrwxrwx 2 root system 256 Jun 11 19:20 grpsvcs.clients.oracle9i
racsrw-rw-rw- 1 root hagsuser 0 Jun 10 18:18 grpsvcsdsocket.oracle9i
racdrwxrwx--- 2 root haemrm 256 Jun 10 18:18 haem
drwxr-xr-x 2 root system 256 May 13 17:54 hats
drwxr-xr-x 2 root system 256 Jun 10 18:18 topsvcs

UDP tuning

The udp_sendspace value must always be greater than Oracle9i RAC's db_block_size. Otherwise Oracle9i RAC will not start.

GPFS issues

GPFS requires a quorum of nodes to run. It also requires that both nodes holding the GPFS configuration data be up and running for any configuration changes to be performed.

In case of a system restart, GPFS will be started automatically and the GPFS file systems will be mounted.

To shut down and unmount the GPFS file systems on all nodes, issue:

# mmshutdown -a

To start GPFS and mount the GPFS file systems on all nodes, issue:

# mmstartup -a

ESS Specialist does not list the WWPN

2010-09-21T16:35:00.000+04:00

Each Fibre Channel adapter on the nodes has a unique World Wide Number (Identified in ESS Specialist as World Wide Port Number - WWPN).

When creating a new host with ESS Specialist, select the appropriate WWPN in a scroll list. If you don't see this WWPN, ensure the following:

The link is correct. You should see the red laser light emitted by the host adapter on the fiber connected to the Storage Area Network - SAN (FC switch).

The FC adapters on the nodes and the ESS have the proper link type value of point-to-point or arbitrated loop. If you are unsure about your network topology, leave the default, which is arbitrated loop.

The command cfgmgr -vsl fcs0 has been run on the nodes.

FC adapter microcode

2010-09-21T16:33:00.000+04:00

Always check your FC adapter microcode level and resolve any issues with your IBM service representative.

Example 3-11 Fibre Channel adapter microcode (on the nodes)

{node1:root}/-> lscfg -vl fcs0 fcs0 U1.9-P1-I1/Q1 FC Adapter

Part Number.................00P2995
EC Level....................A
Serial Number...............1A238006CD
Manufacturer................001A FRU Number.................. 00P2996
Network Address.............10000000C92F4E44 <--- World Wide Number
ROS Level and ID............02C03891
Device Specific.(Z0)........2002606D
Device Specific.(Z1)........00000000
Device Specific.(Z2)........00000000
Device Specific.(Z3)........02000909
Device Specific.(Z4)........FF401050
Device Specific.(Z5)........02C03891
Device Specific.(Z6)........06433891
Device Specific.(Z7)........07433891
Device Specific.(Z8)........20000000C92F4E44
Device Specific.(Z9)........CS3.82A1
Device Specific.(ZA)........C1D3.82A1
Device Specific.(ZB)........C2D3.82A1
Device Specific.(YL)........U1.9-P1-I1/Q1

To obtain the latest microcode level, check this link:

http://techsupport.services.ibm.com/server/mdownload/download.html#adapter

Oracle Transparent Network Failover Failback (TNFF)

2010-09-21T16:26:00.000+04:00

Oracle offers a high availability feature called Oracle Transparent Network Failover Failback (TNFF). In an IBM pSeries environment, this works closely with HACMP/ES.

Tip: The Oracle9i RAC network selection mechanism must be well understood before planning and configuring HACMP/ES.

Interconnect network selection

The Oracle cluster interconnect networks can be configured using either a "private" or a "public" network type defined in HCMP/ES. The Oracle IPC traffic is sent over one of the networks. Oracle RAC chooses the network interface according to the following rules:

1. HACMP "service" networks are required.

Oracle chooses only HACMP/ES networks configured as "service".

2. Oracle uses the HACMP cllsif utility to determine the network interface.

Oracle uses the HACMP utility cllsif to determine which network interface to use based on the rules described below. Oracle chooses the network to use according to the cllsif alphanumeric sort list.

3. Private networks are preferred over public networks.

If there are multiple private networks under HACMP, RAC chooses the interconnect communication by performance quality in the following order: SPswitch > FDDI > Ethernet

Note: Although networks can be labeled inside HACMP as public or private, this attribute has no functional meaning. It's only a way for Oracle to make network selections.

4. Maximum is three networks.

Oracle will choose up to three networks. Oracle recommends two private and one (or more) public networks for the cluster interconnect.

The Network failover and failback sequence

A typical configuration, using common network hardware, has two private interconnects and one public interconnect scenarios.

If the first HACMP/ES private service network goes down, Oracle TNFF fails over to the second private service network. If the second private service network also fails, TNFF continues to fail over to the public network.

If any of the two failed private interconnects is restored, the TNFF fails back to that private service interconnect. If both failed private interconnects are fixed, then TNFF restores the RAC communication to the first listed private service network, according to the cllsif command output alphanumeric sort order.

Typical Oracle RAC configurations

2010-09-21T15:50:00.000+04:00

All configurations are based on the following building blocks:

Hardware

Server nodes

Storage

Networking

Software

Operating system

Cluster software

Oracle RAC (application)

Application architecture

Oracle9i RAC on RAW devices is based on a shared disk architecture. Figure 2-1 shows a two-node cluster. The lower solid line is the primary Oracle interconnect, the middle dashed line is the secondary Oracle interconnect. For high availability, both these networks should be defined in the HACMP as "private".

HACMP/ESCRM provides Oracle9i RAC with the infrastructure for concurrent access to disks. Although HACMP provides concurrent access and a disk locking mechanism, this mechanism is not used. Oracle, instead, provides its own locking mechanism for concurrent data access, integrity, and consistency.

Volume groups are varied on all the nodes, thus ensuring short failover time. This type of concurrent access can only be provided for RAW logical volumes (devices). HACMP/ESCRM does not support concurrent file systems.

Oracle datafiles use the RAW devices located on the shared disk subsystem. In this configuration, an HACMP resource group has to be defined to handle the concurrent volume groups.

Oracle9i RAC installation and configuration (on GPFS)

In a GPFS file system environment you have two choices for installing Oracle code:

1. Install the code on each node (non-shared file system)

Create a JFS or JFS2 file system on each node and set oracle.dna ownership on these file systems. When installing Real Application Clusters, the Oracle Universal Installer (OUI) will copy the Oracle code from the node from which you are running the installer to the other nodes in the cluster. This results in one copy of the Oracle binaries on each node.

2. Install the Oracle software on a GPFS file system, thus creating only one copy of the Oracle binaries. All instances will share the same code. This option is more convenient because all configuration and SW maintenance can be performed from any node in the cluster.
Single point of control instance management

On each node, create the directory /var/opt/oracle and set the ownership to the user oracle. During Oracle9i RAC installation, a file called srvConfig.loc will be created in this directory.

This file is used for specifying the destination of the Oracle server manager utility (srvctl) common configuration file. This file must be on a shared file system (GPFS), and is used by the Oracle server manager for central management of the Oracle9i RAC instances.

Oracle code file system configuration

In our environment we chose a GPFS file system to store the Oracle code. For GPFS file system configuration, refer to 3.10.2, GPFS cluster configuration.

Check if the designated Oracle code file system is mounted. We used /oracle as mount point for this file system. All binaries, Oracle logs, and initialization (init) files will be on the GPFS file system. Be sure to allocate enough space on this file system (see Example 4-6), considering future SW upgrades and Oracle log files.

Example 4-6 File system allocation

{node1:root}/-> df -PkFilesystem 1024-blocks Used Available Capacity Mounted on/dev/hd4 65536 37936 27600 58% //dev/hd2 1376256 1354380 21876 99% /usr/dev/hd9var 131072 66216 64856 51% /var/dev/hd3 917504 49744 867760 6% /tmp/dev/hd1 65536 15132 50404 24% /home/proc - - - - /proc/dev/hd10opt 131072 57436 73636 44% /opt/dev/data 144424960 129734240 14690720 90% /data/dev/oracle 9748480 6607360 3141120 68% /oracle

Change the ownership and access permission for the Oracle directory:

chown -R oracle.dba /oracle
Oracle user environment

Set up the Oracle environment in the $HOME/.profile file of the user oracle. Depending on your configuration, you may choose the user oracle home directory on an internal disk (/home/oracle, in which case you have to propagate the same environment on all nodes), or on a GPFS file system, in the /oracle directory (/oracle/home in our environment).

Example 4-7 Setting environment variables in ~/.profile

>>> Previous lines are generic environment lines (MAILMESG, PS1 etc.,)<<<<# Oracle specific environment starts HEREHOST=`hostname -s`# This stanza selects the value of ORACLE_SID variable depending on the host the oracle# user logs in.case ${HOST} in node1) SID=1;; node2) SID=2;; node3) SID=3;; node4) SID=4;;esac# Variables needed during installation and normal operationexport ORACLE_SID=rac${SID}export DISPLAY=node1:0.0export TMPDIR=/oracle/tempexport TEMP=/oracle/tempexport ORACLE_BASE=/oracleexport ORACLE_HOME=/oracle/product/9.2.0export PATH=$ORACLE_HOME/bin:$PATH

The following variables are mandatory to perform an Oracle9i RAC installation:
ATH.

ORACLE_SID is the system identifier for an Oracle server instance. This variable uniquely identifies a database instance. For consistency, we chose RAC as instance name prefix (see Example 4-7 and Table 4-1).

Table 4-1

Host name Node name Thread ID SID
node1 node1 1 rac1
node2 node2 2 rac2
node3 node3 3 rac3
node4 node4 4 rac4

Instance name selection

ORACLE_HOME is the directory that contains the Oracle software (binaries, libraries etc.).
TMPDIR and TMP: during installation Oracle needs approx. 800 MB of temporary space. For maintaining the system /tmp directory under control, we allocated a separate temporary space in the /oracle/temp directory, and assigned the two variables to point to this directory.
ORACLE_BASE specifies the base directory for Oracle software.

Also, the DISPLAY variable is needed for nodes that do not have a graphical display.
the oracle user environment by logging in to the systems (as user oracle) and displaying the variables, as shown in Example 4-8.

Example 4-8 Testing the environment

{node1:oracle}/oracle/home-> echo $ORACLE_SIDrac1

Now start the Oracle9i Universal Installer (OUI) graphical interface tool.

Creating and validating the database

This section describes how to plan and create a database and the associated storage.
We created a Real Application Clusters Database using the dbca graphical user interface (GUI).

The database can also be created manually, using scripts.

Various Site-to-Site IPSec VPN: Cisco, Juniper, Checkpoint, Sonicwall, Zywall

2010-09-19T18:04:00.000+04:00

Introduction

Setting up site-to-site IPSec VPN connection in general involves two phases. Phase 1 is called ISAKMP SA (Security Association) establishment and Phase 2 is called IPSec SA establishment.

Phase 1
In general, Phase 1 deals with confirmation among sites that are about to establish secure connection across unsecure network. This process is to verify that each site is authorized to establish such connection. Following is further description.

Phase 1 is to establish the ISAKMP key matching with remote site. One popular technique of this ISAKMP key matching is to use preshared key. This key is basically a string (combination of alphabets, numbers, and characters) that both sites agree to use. The key is then stored (and encrypted) within each VPN device configuration.

Phase 1 in IPSec VPN connection establishment is also involving the remote VPN device IP address (peer). A popular technique is to specifically set the remote peer IP address (for security purposes); known as static configuration. With this specific static configuration, both preshared key and remote IP address are statically configured into the VPN device.

During the Phase 1 VPN tunnel establishment using the static configuration of both preshared key and remote IP address, the two VPN peer IP addresses (the local and the remote) must match. If the two VPN peer IP addresses match, then the next step is to match the preshared key between the two VPN devices.

This preshared key matching process is done within an encapsulated secure (encrypted) tunnel. The encapsulation type and method used is the encryption specified for the Phase 1. In other word, Phase 1 VPN tunnel establishment in this case involves matching process of three factors where all the three are statically configured into both VPN devices. If there is a change needed to the either one of the three, manual adjustment is needed.

The three factors are VPN peer IP addresses, preshared key, and encryption type and method. In this specific example, those three factors are the key of how Phase 1 process take place to verify security association establishment between sites that are about to setup secure connection over untrusted network.

Phase 2

Once Phase 1 is passed successfully, then the setup process moves to the Phase 2. In general, Phase 2 deals with traffic management of the actual data communication between sites. There will be mechanism to determine which data goes where, encrypted or not.

In Cisco security device, one mechanism factor is to use access list. An access list is used to specify or regulate which data (source and destination IP addresses or subnets) need to be encrypted or decrypted (going through the VPN tunnel).

Similar to the Phase 1, there is also specific remote VPN peer IP addresses and IPSec VPN tunnel type and method only for the Phase 2. All the access list, remote VPN peer IP addresses, and the Phase 2 IPSec VPN tunnel type and method are statically configured into both VPN devices. The actual data passing (that are encrypted before leaving local VPN device to go to the remote VPN device; and are decrypted when arriving at local VPN device from the remote VPN device) are encapsulated within the Phase 2 IPSec VPN tunnel.

In other word, the access list, VPN peer IP addresses, and IPSec VPN tunnel type and method are the key to establish the Phase 2. Once Phase 2 is established, the actual data between sites will be passing.

Between Phase 1 and Phase 2

Note that only the Phase 2 involves the IPSec protocol, either ESP (Protocol 50) or AH (Protocol 51). Both Phase 1 (ISAKMP) and Phase 2 (IPSec) use specific encryption type (i.e. AES, 3DES, DES) and hash (MD5 or SHA). Specifically for Phase 1, there is the Diffie-Hellman group type (Group 1, 2, or 5) and the ISAKMP SA (Security Association) timeout or lifetime.

Cisco Configuration Guide

An Introduction to IP Security (IPSec) Encryption

Virtual Private Networks with the Cisco PIX Firewall - Introduction and Implementation

Illustration

Let's review the following PIX IPSec VPN tunnel configuration

PIX Version 6.1(2) interface ethernet0 auto interface ethernet1 auto nameif ethernet0 outside security0 nameif ethernet1 inside security100 enable password 8Ry2YjIyt7RRXU24 encrypted passwd 2KFQnbNIdI.2KYOU encrypted hostname pix_1 fixup protocol ftp 21 fixup protocol http 80 fixup protocol h323 1720 fixup protocol rsh 514 fixup protocol smtp 25 fixup protocol sqlnet 1521 fixup protocol sip 5060 names ! !--- These are the access list that regulate the actual data passing within Phase 2 IPSec VPN tunnel ! !--- The IPSec VPN tunnel between PIX 1 and PIX 2: !--- The 10.1.1.0/24 is local subnet to this PIX 1 VPN device as the source subnet !--- The 10.2.2.0/24 is remote subnet reside at the PIX 2 remote VPN device as the destination subnet access-list 120 permit ip 10.1.1.0 255.255.255.0 10.2.2.0 255.255.255.0 ! !--- The IPSec VPN tunnel between PIX 1 and PIX 3: !--- The 10.1.1.0/24 is local subnet to this PIX 1 VPN device as the source subnet !--- The 10.3.3.0/24 is remote subnet reside at the PIX 3 remote VPN device as the destination subnet access-list 130 permit ip 10.1.1.0 255.255.255.0 10.3.3.0 255.255.255.0 ! !--- No NAT in place for traffic to other PIX Firewall private networks !--- This access list associates with the nat 0 (inside) command access-list 100 permit ip 10.1.1.0 255.255.255.0 10.2.2.0 255.255.255.0 access-list 100 permit ip 10.1.1.0 255.255.255.0 10.3.3.0 255.255.255.0 ! pager lines 24 logging on logging facility 20 logging queue 512 mtu outside 1500 mtu inside 1500 ! !--- The outside interface IP address will be used as the local VPN peer IP address ip address outside 172.18.124.153 255.255.255.0 ! ip address inside 10.1.1.1 255.255.255.0 ip audit info action alarm ip audit attack action alarm no failover failover timeout 0:00:00 failover poll 15 failover ip address outside 0.0.0.0 failover ip address inside 0.0.0.0 arp timeout 14400 ! !--- Do not perform NAT for traffic to other PIX Firewalls !--- This assumes that there is no overlapping subnet within your network nat (inside) 0 access-list 100 ! !--- Note that the following static route also cover IP reachability to the remote subnets route outside 0.0.0.0 0.0.0.0 172.18.124.1 1 ! timeout xlate 3:00:00 timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h323 0:05:00 sip 0:30:00 sip_media 0:02:00 timeout uauth 0:05:00 absolute aaa-server TACACS+ protocol tacacs+ aaa-server RADIUS protocol radius no snmp-server location no snmp-server contact snmp-server community public snmp-server enable traps floodguard enable ! !--- This is to permit data passing (regulated by the Phase 2 access list) between sites !--- without applying specific inbound access list into the local IPSec VPN tunnel interface sysopt connection permit-ipsec ! no sysopt route dnat ! !--- The Phase 2 IPSec VPN type and method used !--- In this illustration, Protocol 50 (ESP) with DES encryption and MD5 hash is used crypto ipsec transform-set myset esp-des esp-md5-hmac ! !--- The "crypto map" command specify the access list that regulate data passing between !--- local and remote VPN devices; !--- specify the remote VPN peer IP address; !--- and specify the Phase 2 IPSec VPN tunnel type and method used ! !--- IPsec configuration for the Phase 2 IPSec VPN tunnel to PIX 2: ! crypto map newmap 20 ipsec-isakmp crypto map newmap 20 match address 120 crypto map newmap 20 set peer 172.18.124.154 crypto map newmap 20 set transform-set myset ! !--- IPsec configuration for the Phase 2 IPSec VPN tunnel to PIX 3: ! crypto map newmap 30 ipsec-isakmp crypto map newmap 30 match address 130 crypto map newmap 30 set peer 172.18.124.157 crypto map newmap 30 set transform-set myset ! !--- This is to specify that outside interface is to be used as the local VPN peer interface crypto map newmap interface outside ! !--- This is to enable ISAKMP key exchanging on the outside interface as the local VPN peer interface isakmp enable outside ! !--- The "isakmp key" command specify the preshared key and remote VPN peer IP addresses !--- of each Phase 1 VPN tunnel isakmp key ******** address 172.18.124.154 netmask 255.255.255.255 no-xauth no-config-mode isakmp key ******** address 172.18.124.157 netmask 255.255.255.255 no-xauth no-config-mode ! !--- This is to use the local VPN peer IP address for Phase 1 VPN peer IP address matching process isakmp identity address ! !--- The Phase 1 VPN tunnel type and method used ! !--- The lowest number of ISAKMP policy will be used to see if it is working. !--- If the policy number does not work, then !--- proceed to the next bigger policy number to see if that works isakmp policy 10 authentication pre-share isakmp policy 10 encryption 3des isakmp policy 10 hash md5 isakmp policy 10 group 1 isakmp policy 10 lifetime 1000 ! isakmp policy 20 authentication pre-share isakmp policy 20 encryption des isakmp policy 20 hash md5 isakmp policy 20 group 1 isakmp policy 20 lifetime 1000 ! telnet timeout 5 ssh timeout 5 terminal width 80 Cryptochecksum:436c96500052d0276324b9ef33221b2d : end

IBM SAN Multipath installation on Red Hat Enterprise Linux 5.X

2010-08-01T16:13:00.000+04:00

This section describes the steps for installing Red Hat Enterprise Linux 5.2 on a multipath device.

1.Determine which multipath device your machine boots from. To do this, note down the LUN (Logical Unit) number of the bootable device that is made available by your Fibre Channel Adapter card during firmware boot.

In the test environment, the firmware displays the boot disk, showing lun number 0.

2.Start the installation by providing the keyword mpath in the kernel command line. In the test environment, linux mpath vnc was used.

The Partitioning screen displays a list of multipath devices as mapper/mpath*.

3. Determine the multipath device that corresponds to the bootable device for the host being installed. Go back to the console window if using VNC, or start an alternate console. In the test environment, the System x® is hooked up to an RSA. In the remote control console, we pressed Ctrl-Alt-F2 for an alternate console.

From the output, we found the device that corresponds to the bootable LUN.

In the test environment, System x is connected to an RSA. In the console, we ran the command:

multipath -ll

grep -E ':

mpath'The following output was displayed:

From the output of the command as shown above, dm-6, the fourth multipath device listed, corresponds to LUN 0. First, look for the numbers in x:x:x:x format. The devices with x:x:x:0 are the devices correspond to LUN 0. In this case, dm-6, with the two multipaths 0:0:0:0 and 0:0:1:0, is the bootable multipath device of the host. mapper/mpath4, which corresponds to dm-6, is then the bootable multipath device for installation.

4. Continue the install by going back to your VNC session, or switching back to the console window if installing through text mode. In the test environment, we pressed Ctrl-Alt-F1 in the remote control console.

5.Select the appropriate multipath device for installation. In the test environment, mapper/mpath4 is the only device checked.

6. Select Review and modify partitioning layout and then press Next.

7. Boot loader might not selected by default to install into the bootable multipath device. In the test environment, the boot loader defaulted to /dev/mapper/mpath0, as shown in the following screen. Note the first line of the screen:

The GRUB boot loader will be installed on /dev/mapper/mpath0.

8. If the installer did not select the correct multipath bootable device, change where the boot loader is installed by selecting Configure advanced boot loader options and press Next.

Repeat this step until the bootable mulitpath device is on top of the list.

11. Press OK.

12. The bootable device is now selected for boot loader install. As in the test environment, note that /dev/mapper/mpath4 is selected.

13.Continue and complete the installation.

Note: If you are a different IBM® supported multipath storage, additional steps are needed to finish the installation. See Additional multipath configuration information for IBM storage for more details.

14. Reboot the system.

15.Verify that the firmware boot sequence is properly configured to boot from the correct multipath device.

In the test environment, we pressed F1 in the BIOS screen (where the IBM eServer logo appears) to go to Configuration/Setup menu. Choose Start Options and then Startup Sequence Options. Change the option to have Hard Disk 0 as the first startup device to boot from the multipath device (mpath4).

16. Follow these steps to verify the installation was successful:

a.Run df and cat /proc/swaps to verify that the correct partitions are in a multipath device. In the test environment, the following partitions are displayed: root, /boot, and swap as /dev/mapper/mpath4p3, /dev/mapper/mpath4p1, and /dev/mapper/mpath4p2 respectively. These partitions are installed correctly to use the Device Mapper (DM) Multipath feature of Linux®.

b.Run multipath -ll to verify that the specific multipath device has as many paths as were configured. In the test environment, two paths are displayed, verifying that all the paths are properly configured.

c.If installed on LVM, run dmsetup ls and demsetup table to verify that the LVM volumes are created as linear devices on top of the chosen multipath device.

25 Linux tips for Windows switchers

2010-07-20T11:13:00.001+04:00

Don't see what all the Windows 7 fuss is about and thinking of jumping ship to Linux?
The experience of switching to Linux needn't be a traumatic one.
Here are 25 things you need to know that will make your transition to an open source OS easy.
The basics
1. What's a distribution?
Linux isn't sold as a single package like Windows or MacOS. There are lots of variations on the basic operating system put together by different people for different reasons.
Some might be hardware specific - designed to run on netbooks, for example - while some might be tailored towards particular uses, like general desktops, webservers or multimedia workstations.
Think of them as like the different versions of Vista, but with many more apps in each package. Each of these different bundles is called a 'distribution'.

The pain-free guide to switching Linux distros

2. How are they different?
The most obvious differences between distributions are the number and type of applications that come pre-installed. Under the hood, they may use a different technique for managing sound playback, too.
Not all distributions will use the same versions of core system files either, rather the ones they've tested and are most likely to work with the other programs in the package.
3. Which distribution should I choose?
Ubuntu is by far the most popular distribution for beginners, but most of the big names - OpenSuse, Fedora and so on - are just as approachable and have their strengths. Mandriva works well when syncing with Windows Mobile, for example.

50 handy Ubuntu time-saving tips

4. Understand desktop environments
The familiar windowed desktop of any operating system is simply a layer on top of the core code that makes it easier for you to interact with your computer. Because Linux is highly modular, this desktop environment is easily separated from the core operating system, and you can choose one from several options.
Most popular distributions will give you a choice of desktop environment, while smaller ones, like Mint, may only come in one flavour.
5. Which is best for Windows' switchers?
Gnome and KDE are the closest to Vista or OS X, and most people will find Gnome easier to pick up. Others, life XFCE or LXDE are designed to run quickly on low end systems.
oDon't worry too much, though. The great thing about Linux is you want to try a different desktop manager without reinstalling your OS, you can. In fact, you can have several different ones on the same machine to choose from at login.

Best Linux distros for power users, gamers, newbies and more

IN CONTROL: Whichever desktop environment you use, you have an incredible amount of control over the way it looks and behaves
Installation
6. Live CDs
Once you've decided which distribution you'd like to try, download the installation file (usually a .iso file) and burn it to CD or DVD. With many distributions, booting from that disc will give you the option to try a 'live version'. This lets you boot into a Linux desktop without altering your current hard drive.
7. Look elsewhere
Don't forget, though, that the Live version will have its limitations. Before you discard a distribution for looking dull, run a few searches to see if it can be spruced up. Remember, most distributions are designed for maximum hardware compatibility. Your PC can almost certain handle better desktop effects, for example.
8. Putting Linux on your PC
If you've never installed an operating system before, it can be quite daunting. Don't fear, though, it's a straightforward operating and the big distributions have excellent documentation for first timers, like this how-to at Ubuntu.
9. Keep the Windows open
The easiest way to install a distribution would be to wipe your PC clean and start over. You may want to keep your existing Windows installation, though, to run applications like games that don't work so well in Linux.
Most installation processes will guide you through setting up a dual boot system, with two or more operating systems on. You will need a separate hard drive or partition for each, though.

PACKAGE MANAGER: Synaptic is a popular package manager for Debian-based distributions, like Ubuntu

Finding your way around
10. Getting to grips with the file system
Navigating the C: drive in Windows is second nature to most people. Opening a file browser in Linux can be a shock. Instead of a few neatly named folders like Program Files, there are several obscure ones with names like "etc" and "opt" whose contents appear to be the same.
Linux programs spread bits of themselves around in each of these folders, and while it's logical to the trained eye, it takes experience to understand. Take the easy option - don't worry about it any more than you would the contents of C:\Windows.
11. Stay in your Home
All your important files, like documents, music and video, are kept in a separate partition called the Home partition. Each username you create has their own password protected area within Home, so you can set one up for everyone in your household.
12. Understanding Root
Just like Windows, there are two types of user accounts - administrator and normal. The administrator in Linux is called 'Root', and system files are locked for ordinary users to edit. If you ever try to edit a document and find it can't be saved, or a folder can't be copied, it's probably locked for Root access too.
13. Become a superuser
Anyone can temporarily be elevated to the position of Superuser, allowing them to perform operations restricted to root. This is done by opening a terminal and starting a command with the prefix "sudo" or "su", depending on your distribution. You'll then be prompted for the adminstrator password, which will allow the command to be completed.
14. Don't be afraid of the terminal
It's one of those words that scares off the non-geek, but a terminal is simply a program that lets you enter commands as text, rather than mouse clicks.
Because of the Linux file structure, it's simply quicker for many expert users to use the terminal rather than the file browser to do stuff. One day you might feel the same way, honest.
15. Open file manager as Root
If you prefer the familiarity of a graphical user interface, though, you can use the terminal to open your file manager with administrator access, to move locked folders or edit locked text files.
If you're using Gnome, just enter 'sudo nautilus' in the terminal and an empowered file manager will appear. Remember there's no restrictions to stop you irrevocably damaging your system, though.
16. Installing new programs
Unlike Windows or OS X, most Linux distributions come with a graphics editor, office suite, messenger software and so on pre-installed, and getting new programs is simple.
In your main menu you'll find a
"package manager", usually Synaptic or Yum, which lists all the files currently installed on your PC, along with ones you can download for free.
17. Learn about repositories
The package manager gets these software lists from repositories, which are FTP servers full of free software maintained by Linux advocates or organisations. You can add new ones in the Software Sources section of the start menu.
18. Don't always update
The package manager will cross reference software you have installed with the version in your listed repositories to see if you're running the latest build. You don't have to update programs when the update manager warns you, though. Remember, with Linux, you're in control.

EASY OFFICE: OpenOffice is a complete office suite which is also available for Windows. It looks a bit old fashioned, but will read and save Microsoft Office 2007 files

Troubleshooting

19. Fix display problems
If you're getting strange screen artefacts, like text and cursors vanishing in OpenOffice or your mail client, and are using an Nvidia or AMD graphics card, it's worth going to the manufacturers' website to see if newer Linux drivers are available.

20. Manually installing drivers
Unlike Windows, you may never need to install a device driver on your Linux system. Most hardware is supported straight from the kernel. Manually installing drivers in Linux can be tough, though, even when the process is well documented. It's a good idea to print off the readme file before you get started.

21. Drivers or programs don't work?
Upgrading core kernel files with "header" in their name will likely break any manually installed graphics drivers and some programs too. The fix is simple - just reinstall them.

22. Recovery mode
Most distributions have a recovery mode of some sort available at the first menu screen to help you sort out problems with an operating system that refuses to boot. Ubuntu scores highly in this regard, because it has an option to easily reset the graphics system, often a cause of problems.

23. Look before you ask
The helpfulness of the Ubuntu forums - and indeed most Linux forums - when it comes to helping newbies out is legendary. Don't just log in and start asking, though, run a few searches for the problems you're encountering first.

In the unlikely event a walkthrough doesn't already exist with a solution, you'll get a better idea of how to phrase your question so everyone understands you first time.

24. Keep notes
The web is full of really useful Linux walkthroughs that can give you PC superpowers just by cutting and pasting a few lines into a terminal. Sometimes, these walkthroughs won't work. It's a good idea to keep track of changes you've made in case you need to go back and fix stuff later.

25. Get Wine
There are some Windows programs you may find you simply can't do without. Spotify, Evernote, World of Warcraft, Left4Dead perhaps? Wine adds libraries to get most Windows apps running in Linux.

And finally...
Get creative with the look and feel of your desktop. You can tweak anything you want about the look, from the position of panels and menus to adding an OS X-like dock. There are loads of themes to get you started at opendesktop.org, just click on the artwork tab at the top.

PLENTY OF APPS: There are few types of app which Linux doesn't have a program capable of matching a Windows peer. Accounting programs are about the only thing that's thin on the ground

-------------------------------------------------------------------------------------------------------

VIM Faster Navigation

2010-07-20T10:54:00.000+04:00

Sometimes even the simplest of tasks like navigating through a single file can be optimized. Vim offers several methods of navigation within a file, which can adapt to the contents of the file and how it is organized. Some of these methods are obvious, while others are more complex.

Mostly, the files we are editing are well structured. If our files are text, then this structure can be in the form of paragraphs, sentences, and words, or at other times code with functions, blocks, and code lines.

Vim supports jumping around the file, according to the structure in the file, and has key bindings that make it easy to go to the exact place in the file where you want to go.

Moving within a text file

You are working on a normal text file and in the middle of a sentence you realize that you have forgotten to make the first letter in the paragraphs uppercase. You could of course, use the arrow-keys or the h/j/k/l navigation keys to move to the beginning of the paragraph to correct this. It would, however, be faster to just press the following in normal mode:

{

You are now placed at the beginning of the paragraph—or in the empty line just above it (if one exists). Now, you go into normal mode by pressing the Esc key and then use the movement command { to go to the beginning of the paragraph. The same can be applied if you want to go back to the end of the paragraph—you simply have to use the counterpart of {, namely:

}

Again, Vim helps you move faster and offers you a pair of commands to move to the beginning and end of the current sentence. The commands are as follows:

( : Move to the beginning of the sentence.

) : Move to the end of the sentence.

w : Move to the beginning of the next word.

b : Move to the beginning of the previous word.

e : Move to the end of the word.

These commands can be combined such that if you want to go to the end of the next word you simply press:

we

Moving in a Code file

Compared to text files, code does not have any paragraphs or sentences to navigate through. It does, however, frequently contain a lot of structures and blocks, each of which has a very specific contextual meaning within the code. An example could be the simple code block:

If( a == b) {

print "a and b are the same";

}

Here the line with print is within the context of the if block surrounding it.

Because Vim is the favorite editor of many programmers, it offers a lot of movement commands to use when you are working with code. Common for all of them is that the parts of the code you want to jump between need to have a contextual connection to each other.

A simple example could be a construction like the #if-#else-#endif construction from the C programming language. Here we have a beginning (#if), an end (#endif) and a midpoint (#else).

So if you are in the line with #if and press the following command:

%

you will go to the #else part. Pressing it again, will take you to the #endif, and yet another execution of the command will get you back to the #if.

Vim does not know all programming language constructs, but by default it knows most of the contextual constructs of the C programming language. Besides this, it knows the normal block construction methods from most programming languages—the use of parentheses and brackets (for example, '{' is block start and '}' is block end).

So for Vim to find the beginning of the function, it simply has to find the outermost bracket pair and then go to the

opening bracket.

function myExample() {

...many lines of code...

/* cursor is placed at the beginning of this line */

...many lines of code...

}

In the above example, the % command would take us to the closing bracket and pressing it again would take us to the opening bracket. But what if the cursor was actually placed inside another pair of brackets? In that case, the % command would only move the cursor to them, and not the beginning of the function.

Again Vim has some handy commands for you:

[[ and ][ : Move backwards/forward to the next section beginning (e.g., start of a function)

[] and ]] : Move backwards/forward to the next section end (e.g., end of a function)

Executing these commands multiple times in a row takes you to the beginning/end of the next/previous section, and therefore gives you a convenient way of cycling through the functions in a file.

[{ : Move to the beginning of the block

]} : Move to the end of the block

[/ : Move to the beginning of the comment block

]/ : Move to the end of the comment block

Boot up ubuntu faste

2010-07-20T10:51:00.000+04:00

Ubuntu reached the 10 second boot time with Lucid Lynx. what if your system isn’t quite reaching that point?

You can speed it up using profiling option. To do this follow these steps:

1.At your boot screen press “e” (for edit).

2.Use your arrow key and move down to the entry beginning with “kernel”.

3.Press “e” again.

4.Add “profile” (no quotes) at the end of this line.

5.Hit Enter.

6.Click “b” (for boot).

The firs time you boot with profile it will take a little longer. What this is doing is profiling your boot process so it knows what files are accessed and then sorts them according to how they are stored on your drive. Your next boot should be considerably faster.

How To Display / Delete IP Route Cache on Linux

2010-07-20T09:50:00.000+04:00

How to display the route cache

ip route show cache
A sample entry from the list is:

217.212.227.18 from 172.16.0.69 via 192.168.0.1 dev eth2 src 172.16.0.50

cache mtu 1500 advmss 1460 hoplimit 64 iif eth0
This tells us that all requests from 172.16.0.69 (my machine's IP) to 217.212.227.18 (Akamai machine) gets routed via eth2 interface using the gateway 192.168.0.1 (Tata Indicom Broadband).

How to delete the route cache

ip route flush cache

How To: Load Balancing & Failover With Dual/ Multi WAN / ADSL / Cable Connections on Linux

2010-07-20T09:46:00.000+04:00

Requirements for Load Balancing multiple ADSL / Cable Connections

1. Obviously you need to have multiple (A)DSL or Cable connections in the first place. Login as root for this job.
2. Find out the LAN / internal IP address of the modems. They may be same like 1921.168.1.1.

Check if the internal / LAN IP address of both (or multiple) modems are same. In that use the web / telnet interface of the modems to configure one of the modems to have a different internal IP address preferably in different networks like 192.168.0.1 or 192.168.2.1 etc. If you are using multiple modems then you should configure each of them to have different subnets. This is important because now you can easily access the different modems from their web interface and you don't have to bother connecting to a modem through a particular interface. It is also important because now you can easily configure the interfaces to be associated with different netmasks / sub-network.
3. Connect each modem to the computer using a different interface (eth0, eth1 etc.). You may be able to use the same interface but this guide doesn't cover that. In short you will make your life complicated using the same interface or even different virtual interface. My recommendation is that you should use one interface per modem. Don't scrimp on cheap ethernet adapters. This has the added benefit of redundancy should one adapter go bad down the road.
4. Configure the IP address of each interface to be in the same sub-network as the modem. For example my modems have IP addresses of 192.168.0.1 and 192.168.1.1. The corresponding addresses & netmasks of the interfaces are: 192.168.0.10 (netmask: 255.255.255.0) and 192.168.1.10 (netmask: 255.255.255.0).
5. Find out the following information before you proceed with the rest of the guide:

IP address of external interfaces (interfaces connected to your modems). This is not the gateway address.
Gateway IP address of each broadband connections. This is the first hop gateway, could be your DSL modem IP address if it has been configured as the gateway following the tip below.
Name, IP address & netmask of external interfaces like eth1, eth2 etc. My external interfaces are eth1 & eth2.
Relative weights you want to assign to each connection. My Tata connection is 4 times faster than BSNL connection. So I assign the weight of 4 to Tata and 1 to BSNL. You must use low positive integer values for weights. For same connection speeds weights of 1 & 1 are appropriate. The weights determine how the load is balanced across multiple connections. In my case Tata is 4 times as likely to be used as route for a particular site in comparison with BSNL.

Note: Refer to Netmask guide for details on netmasks.
Optional step

Check the tips on configuring (A)DSL modems. They are not required for using this guide. However they are beneficial in maximizing your benefits.

How to setup default load balancing for multiple ADSL / Cable connections

Unlike other guides on this topic I will use a real example - the configuration on our internal network. So to begin with here are the basic data for my network:

#IP address of external interfaces. This is not the gateway address.

IP1=192.168.1.10

IP2=192.168.0.10
#Gateway IP addresses. This is the first (hop) gateway, could be your router IP

#address if it has been configured as the gateway

GW1=192.168.1.1

GW2=192.168.0.1
# Relative weights of routes. Keep this to a low integer value. I am using 4

# for TATA connection because it is 4 times faster

W1=1

W2=4
# Broadband providers name; use your own names here.

NAME1=bsnl

NAME2=tata

You must change the example below to use your own IP addresses and other details. Even with that inconvenience a real example is much easier to understand than examples with complex notations. The example given below is copy-pasted from our intranet configuration. It works perfectly as advertised.
Note: In this step fail-over is not addressed. It is provided later with a script which runs on startup.
First you need to create two (or more) routes in the routing table ( /etc/iproute2/rt_tables ). Open the file and make changes similar to what is show below. I added the following for my two connections:





1 bsnl



2 tata

To add a default load balancing route for our outgoing traffic using our dual internet connections (ADSL broadband connections from BSNL & Tata Indicom) here are the lines I included in rc.local file:





ip route add 192.168.1.0/24 dev eth1 src 192.168.1.10 table bsnl



ip route add default via 192.168.1.1 table bsnl



ip route add 192.168.0.0/24 dev eth2 src 192.168.0.10 table tata



ip route add default via 192.168.0.1 table tata



ip rule add from 192.168.1.10 table bsnl



ip rule add from 192.168.0.10 table tata



ip route add default scope global nexthop via 192.168.1.1 dev eth1 weight 1 nexthop via 192.168.0.1 dev eth2 weight 4

Adding them to rc.local ensures that they are execute automatically on startup. You can also run them manually from the command line.
This completes the load balancing part. Let's now see how we can achieve fail-over so the routes are automatically changed when one or more connections are down and then changed again when one or more more connections come back up again. To do this magic I used a script.

How to setup fail-over over multiple load balanced ADSL / Cable connections

Please follow the steps below and preferably in the same order:

First download the script which checks for and provides fail-over over dual ADSL / Cable internet connections and save it to /usr/sbin directory (or any other directory which is mounted available while loading the OS).
Change the file permissions to 755:

chmod 755 /usr/sbin/gwping
Open the file (as root) in an editor like vi or gedit and edit the following parameters for your environment:

#IP Address or domain name to ping. The script relies on the domain being pingable and always available

TESTIP=www.yahoo.com
#Ping timeout in seconds

TIMEOUT=2
# External interfaces

EXTIF1=eth1

EXTIF2=eth2
#IP address of external interfaces. This is not the gateway address.

IP1=192.168.1.10

IP2=192.168.0.10
#Gateway IP addresses. This is the first (hop) gateway, could be your router IP

#address if it has been configured as the gateway

GW1=192.168.1.1

GW2=192.168.0.1
# Relative weights of routes. Keep this to a low integer value. I am using 4

# for TATA connection because it is 4 times faster

W1=1

W2=4
# Broadband providers name; use your own names here.

NAME1=BSNL

NAME2=TATA
#No of repeats of success or failure before changing status of connection

SUCCESSREPEATCOUNT=4

FAILUREREPEATCOUNT=1
Note: Four consecutive success indicates that the gateway is up and one (consecutive) failure indicates that the gateway went down for my environment. You may want to modify it to better match your environment.
Add the following line to the end of /etc/rc.local file:

nohup /usr/sbin/gwping &

In the end my /etc/rc.local file has the following lines added in total:

ip route add 192.168.1.0/24 dev eth1 src 192.168.1.10 table bsnl

ip route add default via 192.168.1.1 table bsnl

ip route add 192.168.0.0/24 dev eth2 src 192.168.0.10 table tata

ip route add default via 192.168.0.1 table tata

ip rule add from 192.168.1.10 table bsnl

ip rule add from 192.168.0.10 table tata

ip route add default scope global nexthop via 192.168.1.1 dev eth1 weight 1 nexthop via 192.168.0.1 dev eth2 weight 4

nohup /usr/sbin/gwping &

An astute reader may note that the default setup with dual load balanced routing (7th line) is really not required as the script is configured to force routing based on the current status the very first time. However it is there to ensure proper routing before the script forces the routing for the first time which is about 40 seconds in my setup (can you tell why it takes 40 second for the first time?).

Six Linux softphone's list

2010-05-11T17:28:00.001+04:00

VoIP has improved a lot since its first days, today a lot of multinational business are using it as a reliable way to keep stay in touch.

With more and more telecommuters, and business man working, and with cell phone roaming costs still high, VoIP is a real option for both big and small corporations.

And when you talk to a CFO about investing to save costs, you usually will see the checkered flag for your project.

Most of these projects will involve IP phones or ATAs, but you may use softphones too, which are very convenient for the traveler guy, here we will review some of the soft phones available for Linux.

Linphone

Linphone is a mature piece of software, it uses SIP, for voice and video over IP, it can work as a stand alone application, and you can call other sip enabled devices just entering its ips on the dial window of Linphone, or you can configure it to use an Asterisk PBX.

There are binary packages for the most common distros, so try to install it using you package manager.

Ekiga

Ekiga, is the softphone for Gnome Desktop, it supports both SIP and H.323, also has support for webcams if you kernel recognize yours.

As with Linphone you may use Ekiga standalone, or with a softswitch, you can also create an account using the Ekiga configuration druid, but over all, it is a great soft phone to be used with Asterisk.

Here is a screencast to learn how to configure it, to work with Asterisk.

Kiax

Kiax is an IAX softphone, so it is intended only for use with Asterisk, its configuration is really simple, but keep in mind that you need to create the username in the iax.conf file, instead of the sip.conf file to be able to use this softphone, it is really a good option if you are working with Asterisk.

Skype

Skype is a closed softphone, it means you may not use it with any gateway, it uses a proprietary protocol, and only works with the skype Network skype site, but it lets you add contacts, call them by free, or chat with them.

You may also call land lines and cell phones using a credit card or paypal to add credit to your account, the rates are really low compared with normal Telco rates, as there is versions for Windows and Mac, it is widely used, and maybe the most popular softphone, it also has the advantage that you do not need to configure your own PBX.

Gizmo

Gizmo is another network like Skype, but the good thing is that its softphone also support SIP configuration, and can be connected to an Asterisk PBX, while using its own network!, you may also configure is with your favorite IM network, MSN, Yahoo messenger, etc.
The configuration is really easy, and you add credit to call to outside numbers, using its network, or use you own PBX to make those calls, I really recommend the use of this software, which can be downloaded from gizmo.com you can also download versions for cell phones, if you are using Ubuntu it is in medibuntu repository, if are using Crunchbang Linux you just have to install it with:

sudo aptitude install gizmo5

Twinkle

Twinkle is another great SIP softphone, which can be easy used with Asterisk, or any other SIP gateway, it is easy to configure, supports STUN server, NAT traversal, Instant messaging, and as the others, can be used in a phone to phone configuration, or using a Gateway / SIP registrant.

Squid Access Lists

2010-05-11T16:14:00.000+04:00

Access Lists

There are a number of different access lists:

http_access: Allows HTTP clients (browsers) to access the HTTP port. This is the primary access control list.
http_reply_access: Allows HTTP clients (browsers) to receive the reply to their request. This further restricts permissions given by http_access, and is primarily intended to be used together with rep_mime_type acl for blocking different content types.
icp_access: Allows neighbor caches to query your cache with ICP.
miss_access: Allows certain clients to forward cache misses through your cache. This further restricts permissions given by http_access, and is primarily intended to be used for enforcing sibling relations by denying siblings from forwarding cache misses through your cache.
cache: Defines responses that should not be cached.
url_rewrite_access: Controls which requests are sent through the redirector pool.
ident_lookup_access: Controls which requests need an Ident lookup.
always_direct: Controls which requests should always be forwarded directly to origin servers.
never_direct: Controls which requests should never be forwarded directly to origin servers.
snmp_access: Controls SNMP client access to the cache.
broken_posts: Defines requests for which squid appends an extra CRLF after POST message bodies as required by some broken origin servers.
cache_peer_access: Controls which requests can be forwarded to a given neighbor (cache_peer).
htcp_access: Controls which remote machines are able to make HTCP requests.
htcp_clr_access: Controls which remote machines are able to make HTCP CLR requests.
request_header_access: Controls which request headers are removed when violating HTTP protocol.
reply_header_access: Controls which reply headers are removed from delivery to the client when violating HTTP protocol.
delay_access: Controls which requests are handled by what delay pool
icap_access: (replaced by adaptation_access in Squid-3.1) What requests may be sent to a particular ICAP server.
adaptation_access: What requests may be sent to a particular ICAP or eCAP filter service.
log_access: Controls which requests are logged. This is global and overrides specific file access lists appended to access_log directives.

Notes:

An access list rule consists of an allow or deny keyword, followed by a list of ACL element names.

An access list consists of one or more access list rules.

Access list rules are checked in the order they are written. List searching terminates as soon as one of the rules is a match.

If a rule has multiple ACL elements, it uses AND logic. In other words, all ACL elements of the rule must be a match in order for the rule to be a match. This means that it is possible to write a rule that can never be matched. For example, a port number can never be equal to both 80 AND 8000 at the same time.

To summarize the ACL logics can be described as:

http_access allow|deny acl AND acl AND ...
        OR
http_access allow|deny acl AND acl AND ...
        OR
...

If none of the rules are matched, then the default action is the opposite of the last rule in the list. Its a good idea to be explicit with the default action. The best way is to use the all ACL. For example:

http_access deny all

How do I allow my clients to use the cache?

Define an ACL that corresponds to your client's IP addresses. For example:

acl myclients src 172.16.5.0/24

Next, allow those clients in the http_access list:

http_access allow myclients

how do I configure Squid not to cache a specific server?

acl someserver dstdomain .someserver.com
cache deny someserver

How do I implement an ACL ban list?

As an example, we will assume that you would like to prevent users from accessing cooking recipes.

One way to implement this would be to deny access to any URLs that contain the words "cooking" or "recipe." You would use these configuration lines:

acl Cooking1 url_regex cooking
acl Recipe1 url_regex recipe
acl myclients src 172.16.5.0/24
http_access deny Cooking1
http_access deny Recipe1
http_access allow myclients
http_access deny all

The url_regex means to search the entire URL for the regular expression you specify. Note that these regular expressions are case-sensitive, so a url containing "Cooking" would not be denied.

Another way is to deny access to specific servers which are known to hold recipes. For example:

acl Cooking2 dstdomain www.gourmet-chef.com
http_access deny Cooking2
http_access allow all

The dstdomain means to search the hostname in the URL for the string "www.gourmet-chef.com." Note that when IP addresses are used in URLs (instead of domain names), Squid implements relaxed access controls. If the a domain name for the IP address has been saved in Squid's "FQDN cache", then Squid can compare the destination domain against the access controls. However, if the domain is not immediately available, Squid allows the request and makes a lookup for the IP address so that it may be available for future requests.

How do I block specific users or groups from accessing my cache?

Using Ident

You can use ident lookups to allow specific users access to your cache. This requires that an ident server process runs on the user's machine(s). In your squid.conf configuration file you would write something like this:

ident_lookup_access allow all
acl friends ident kim lisa frank joe
http_access allow friends
http_access deny all

Note that ident_lookup_access only permits/denies whether a machine is tested for its Ident. This does not directly alter access to the users request.

Is there a way to do ident lookups only for a certain host and compare the result with a userlist in squid.conf?

You can use the ident_lookup_access directive to control for which hosts Squid will issue ident lookup requests.

Additionally, if you use a ident ACL in squid.conf, then Squid will make sure an ident lookup is performed while evaluating the acl even if ident_lookup_access does not indicate ident lookups should be performed earlier.

However, Squid does not wait for the lookup to complete unless the ACL rules require it. Consider this configuration:

acl host1 src 10.0.0.1
acl host2 src 10.0.0.2
acl pals  ident kim lisa frank joe
http_access allow host1
http_access allow host2 pals

Requests coming from 10.0.0.1 will be allowed immediately because there are no user requirements for that host. However, requests from 10.0.0.2 will be allowed only after the ident lookup completes, and if the username is in the set kim, lisa, frank, or joe.

Using Proxy Authentication

Another option is to use proxy-authentication. In this scheme, you assign usernames and passwords to individuals. When they first use the proxy they are asked to authenticate themselves by entering their username and password.

In Squid this authentication is handled via external processes. For information on how to configure this, please see SquidFaq/ProxyAuthentication.

Do you have a CGI program which lets users change their own proxy passwords?

Pedro L Orso has adapted the Apache's htpasswd into a CGI program called chpasswd.cgi.

Common Mistakes

h3 id="And.2BAC8-Or_logic">And/Or logic

You've probably noticed (and been frustrated by) the fact that you cannot combine access controls with terms like "and" or "or." These operations are already built in to the access control scheme in a fundamental way which you must understand.

All elements of an acl entry are OR'ed together.
All elements of an access entry are AND'ed together (e.g. http_access and icp_access)

For example, the following access control configuration will never work:

acl ME src 10.0.0.1
acl YOU src 10.0.0.2
http_access allow ME YOU

In order for the request to be allowed, it must match the "ME" acl AND the "YOU" acl. This is impossible because any IP address could only match one or the other. This should instead be rewritten as:

acl ME src 10.0.0.1
acl YOU src 10.0.0.2
http_access allow ME
http_access allow YOU

Or, alternatively, this would also work:

acl US src 10.0.0.1 10.0.0.2
http_access allow US

allow/deny mixups

I have read through my squid.conf numerous times, spoken to my neighbors, read the FAQ and Squid Docs and cannot for the life of me work out why the following will not work.

I can successfully access cachemgr.cgi from our web server machine here, but I would like to use MRTG to monitor various aspects of our proxy. When I try to use squidclient or GET cache_object from the machine the proxy is running on, I always get access denied.

acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl server    src 1.2.3.4/255.255.255.255
acl ourhosts  src 1.2.0.0/255.255.0.0
http_access deny manager !localhost !server
http_access allow ourhosts
http_access deny all

The intent here is to allow cache manager requests from the localhost and server addresses, and deny all others. This policy has been expressed here:

http_access deny manager !localhost !server

The problem here is that for allowable requests, this access rule is not matched. For example, if the source IP address is localhost, then "!localhost" is false and the access rule is not matched, so Squid continues checking the other rules. Cache manager requests from the server address work because server is a subset of ourhosts and the second access rule will match and allow the request. Also note that this means any cache manager request from ourhosts would be allowed.

To implement the desired policy correctly, the access rules should be rewritten as

http_access allow manager localhost
http_access allow manager server
http_access deny manager
http_access allow ourhosts
http_access deny all

If you're using miss_access, then don't forget to also add a miss_access rule for the cache manager:

miss_access allow manager

You may be concerned that the having five access rules instead of three may have an impact on the cache performance. In our experience this is not the case. Squid is able to handle a moderate amount of access control checking without degrading overall performance. You may like to verify that for yourself, however.

Differences between ''src'' and ''srcdomain'' ACL types

For the srcdomain ACL type, Squid does a reverse lookup of the client's IP address and checks the result with the domains given on the acl line. With the src ACL type, Squid converts hostnames to IP addresses at startup and then only compares the client's IP address. The src ACL is preferred over srcdomain because it does not require address-to-name lookups for each request.

I set up my access controls, but they don't work! why?

If ACLs are giving you problems and you don't know why they aren't working, you can use this tip to debug them.

In squid.conf enable debugging for section 33 at level 2. For example:

debug_options ALL,1 33,2

Then restart or reconfigure squid.

From now on, your cache.log should contain a line for every request that explains if it was allowed, or denied, and which ACL was the last one that it matched.

If this does not give you sufficient information to nail down the problem you can also enable detailed debug information on ACL processing

debug_options ALL,1 33,2 28,9

Then restart or reconfigure squid as above.

From now on, your cache.log should contain detailed traces of all access list processing. Be warned that this can be quite some lines per request.

Proxy-authentication and neighbor caches

The problem

               [ Parents ]
               /         \
              /           \
       [ Proxy A ] --- [ Proxy B ]
           |
           |
          USER

Proxy A sends and ICP query to Proxy B about an object, Proxy B replies with an ICP_HIT. Proxy A forwards the HTTP request to Proxy B, but does not pass on the authentication details, therefore the HTTP GET from Proxy A fails.

Only ONE proxy cache in a chain is allowed to "use" the Proxy-Authentication request header. Once the header is used, it must not be passed on to other proxies.

Therefore, you must allow the neighbor caches to request from each other without proxy authentication. This is simply accomplished by listing the neighbor ACL's first in the list of http_access lines. For example:

acl proxy-A src 10.0.0.1
acl proxy-B src 10.0.0.2
acl user_passwords proxy_auth /tmp/user_passwds
http_access allow proxy-A
http_access allow proxy-B
http_access allow user_passwords
http_access deny all

Squid-2.5 allows two exceptions to this rule, by defining the appropriate cache_peer options:

cache_peer parent.foo.com parent login=PASS

This will forward the user's credentials as-is to the parent proxy which will be thus able to authenticate again.

This will only work with the Basic authentication scheme. If any other scheme is enabled, it will fail

cache_peer parent.foo.com parent login=*:somepassword

This will perform Basic authentication against the parent, sending the username of the current client connection and as password always somepassword. The parent will need to authorization against the child cache's IP address, as if there was no authentication forwarding, and it will need to perform client authentication for all usernames against somepassword via a specially-designed authentication helper. The purpose is to log the client cache's usernames into the parent's access.log. You can find an example semi-tested helper of that kind as parent_auth.pl .

Is there an easy way of banning all Destination addresses except one?

acl GOOD dst 10.0.0.1
http_access allow GOOD
http_access deny all

How can I block access to porn sites?

Often, the hardest part about using Squid to deny pornography is coming up with the list of sites that should be blocked. You may want to maintain such a list yourself, or get one from somewhere else (see below).

The ACL syntax for using such a list depends on its contents. If the list contains regular expressions, use this:

acl PornSites url_regex "/usr/local/squid/etc/pornlist"
http_access deny PornSites

On the other hand, if the list contains origin server hostnames, simply change url_regex to dstdomain in this example.

Does anyone have a ban list of porn sites and such?

The SquidGuard redirector folks have links to some lists.
Bill Stearns maintains the sa-blacklist of known spammers. By blocking the spammer web sites in squid, users can no longer use up bandwidth downloading spam images and html. Even more importantly, they can no longer send out requests for things like scripts and gifs that have a unique identifer attached, showing that they opened the email and making their addresses more valuable to the spammer.
The SleezeBall site has a list of patterns that you can download.

Squid doesn't match my subdomains

If you are using Squid-2.4 or later then keep in mind that dstdomain acls uses different syntax for exact host matches and entire domain matches. www.example.com matches the exact host www.example.com, while .example.com matches the entire domain example.com (including example.com alone)

There is also subtle issues if your dstdomain ACLs contains matches for both an exact host in a domain and the whole domain where both are in the same domain (i.e. both www.example.com and .example.com). Depending on how your data is ordered this may cause only the most specific of these (e.g. www.example.com) to be used.

Squid-2.4 and later will warn you when this kind of configuration is used. If your Squid does not warn you while reading the configuration file you do not have the problem described below. Also the configuration here uses the dstdomain syntax of Squid-2.1 or earlier.. (Squid-2.2 and later needs to have domains prefixed by a dot)

There is a subtle problem with domain-name based access controls when a single ACL element has an entry that is a subdomain of another entry. For example, consider this list:

acl FOO dstdomain boulder.co.us vail.co.us co.us

In the first place, the above list is simply wrong because the first two (boulder.co.us and vail.co.us) are unnecessary. Any domain name that matches one of the first two will also match the last one (co.us). Ok, but why does this happen?

The problem stems from the data structure used to index domain names in an access control list. Squid uses Splay trees for lists of domain names. As other tree-based data structures, the searching algorithm requires a comparison function that returns -1, 0, or +1 for any pair of keys (domain names). This is similar to the way that strcmp() works.

The problem is that it is wrong to say that co.us is greater-than, equal-to, or less-than boulder.co.us.

For example, if you said that co.us is LESS than fff.co.us, then the Splay tree searching algorithm might never discover co.us as a match for kkk.co.us.

similarly, if you said that co.us is GREATER than fff.co.us, then the Splay tree searching algorithm might never discover co.us as a match for bbb.co.us.

The bottom line is that you can't have one entry that is a subdomain of another. Squid will warn you if it detects this condition.

Why does Squid deny some port numbers?

It is dangerous to allow Squid to connect to certain port numbers. For example, it has been demonstrated that someone can use Squid as an SMTP (email) relay. As I'm sure you know, SMTP relays are one of the ways that spammers are able to flood our mailboxes. To prevent mail relaying, Squid denies requests when the URL port number is 25. Other ports should be blocked as well, as a precaution against other less common attacks.

There are two ways to filter by port number: either allow specific ports, or deny specific ports. By default, Squid does the first. This is the ACL entry that comes in the default squid.conf:

acl Safe_ports port 80 21 443 563 70 210 1025-65535
http_access deny !Safe_ports

The above configuration denies requests when the URL port number is not in the list. The list allows connections to the standard ports for HTTP, FTP, Gopher, SSL, WAIS, and all non-privileged ports.

Another approach is to deny dangerous ports. The dangerous port list should look something like:

acl Dangerous_ports port 7 9 19 22 23 25 53 109 110 119
http_access deny Dangerous_ports

...and probably many others.

Please consult the /etc/services file on your system for a list of known ports and protocols.

Does Squid support the use of a database such as mySQL for storing the ACL list?

Yes, Squid supports acl interaction with external data sources via the external_acl_type directive. Helpers for LDAP and NT Domain group membership is included in the distribution and it's very easy to write additional helpers to fit your environment.

How can I allow a single address to access a specific URL?

This example allows only the special_client to access the special_url. Any other client that tries to access the special_url is denied.

acl special_client src 10.1.2.3
acl special_url url_regex ^http://www.squid-cache.org/Doc/FAQ/$
http_access allow special_client special_url
http_access deny special_url

How can I allow some clients to use the cache at specific times?

Let's say you have two workstations that should only be allowed access to the Internet during working hours (8:30 - 17:30). You can use something like this:

acl FOO src 10.1.2.3 10.1.2.4
acl WORKING time MTWHF 08:30-17:30
http_access allow FOO WORKING
http_access deny FOO

How can I allow some users to use the cache at specific times?

acl USER1 proxy_auth Dick
acl USER2 proxy_auth Jane
acl DAY time 06:00-18:00
http_access allow USER1 DAY
http_access deny USER1
http_access allow USER2 !DAY
http_access deny USER2

Problems with IP ACL's that have complicated netmasks

The following ACL entry gives inconsistent or unexpected results:

acl restricted  src 10.0.0.128/255.0.0.128 10.85.0.0/16

The reason is that IP access lists are stored in "splay" tree data structures. These trees require the keys to be sortable. When you use a complicated, or non-standard, netmask (255.0.0.128), it confuses the function that compares two address/mask pairs.

The best way to fix this problem is to use separate ACL names for each ACL value. For example, change the above to:

acl restricted1 src 10.0.0.128/255.0.0.128
acl restricted2 src 10.85.0.0/16

Then, of course, you'll have to rewrite your http_access lines as well.

Can I set up ACL's based on MAC address rather than IP?

Yes, for some operating systes. Squid calls these "ARP ACLs" and they are supported on Linux, Solaris, and probably BSD variants.

MAC address is only available for clients that are on the same subnet. If the client is on a different subnet, then Squid can not find out its MAC address as the MAC is replaced by the router MAC when a packet is router.

To use ARP (MAC) access controls, you first need to compile in the optional code. Do this with the --enable-arp-acl configure option:

% ./configure --enable-arp-acl ...
% make clean
% make

If src/acl.c doesn't compile, then ARP ACLs are probably not supported on your system.

If everything compiles, then you can add some ARP ACL lines to your squid.conf:

acl M1 arp 01:02:03:04:05:06
acl M2 arp 11:12:13:14:15:16
http_access allow M1
http_access allow M2
http_access deny all

Can I limit the number of connections from a client?

Yes, use the maxconn ACL type in conjunction with http_access deny. For example:

acl losers src 1.2.3.0/24
acl 5CONN maxconn 5
http_access deny 5CONN losers

Given the above configuration, when a client whose source IP address is in the 1.2.3.0/24 subnet tries to establish 6 or more connections at once, Squid returns an error page. Unless you use the deny_info feature, the error message will just say "access denied."

The maxconn ACL requires the client_db feature. If you've disabled client_db (for example with client_db off) then maxconn ALCs will not work.

Note, the maxconn ACL type is kind of tricky because it uses less-than comparison. The ACL is a match when the number of established connections is greater than the value you specify. Because of that, you don't want to use the maxconn ACL with http_access allow.

Also note that you could use maxconn in conjunction with a user type (ident, proxy_auth), rather than an IP address type.

I'm trying to deny ''foo.com'', but it's not working.

In Squid-2.3 we changed the way that Squid matches subdomains. There is a difference between .foo.com and foo.com. The first matches any domain in foo.com, while the latter matches only "foo.com" exactly. So if you want to deny bar.foo.com, you should write

acl yuck dstdomain .foo.com
http_access deny yuck

I want to customize, or make my own error messages.

You can customize the existing error messages as described in Customizable Error Messages in SquidFaq/MiscFeatures. You can also create new error messages and use these in conjunction with the deny_info option.

For example, lets say you want your users to see a special message when they request something that matches your pornography list. First, create a file named ERR_NO_PORNO in the /usr/local/squid/etc/errors directory. That file might contain something like this:

Our company policy is to deny requests to known porno sites.  If you
feel you've received this message in error, please contact
the support staff (support@this.company.com, 555-1234).

Next, set up your access controls as follows:

acl porn url_regex "/usr/local/squid/etc/porno.txt"
deny_info ERR_NO_PORNO porn
http_access deny porn
(additional http_access lines ...)

I want to use local time zone in error messages.

Squid, by default, uses GMT as timestamp in all generated error messages. This to allow the cache to participate in a hierarchy of caches in different timezones without risking confusion about what the time is.

To change the timestamp in Squid generated error messages you must change the Squid signature. See Customizable Error Messages in MiscFeatures. The signature by defaults uses %T as timestamp, but if you like then you can use %t instead for a timestamp using local time zone.

I want to put ACL parameters in an external file.

by Adam Aube

Squid can read ACL parameters from an external file. To do this, first place the acl parameters, one per line, in a file. Then, on the ACL line in squid.conf, put the full path to the file in double quotes.

For example, instead of:

acl trusted_users proxy_auth john jane jim

you would have:

acl trusted_users proxy_auth "/usr/local/squid/etc/trusted_users.txt"

Inside trusted_users.txt, there is:

john
jane
jim

I want to authorize users depending on their MS Windows group memberships

There is an excellent resource over at http://workaround.org/squid-ldap on how to use LDAP-based group membership checking.

Also the LDAP or Active Directory config example here in the squid wiki might prove useful.

Maximum length of an acl name

By default the maximum length of an ACL name is 32-1 = 31 characters, but it can be changed by editing the source: in defines.h

#define ACL_NAME_SZ 32

Fast and Slow ACLs

Some ACL types require information which may not be already available to Squid. Checking them requires suspending work on the current request, querying some external source, and resuming work when the needed information becomes available. This is for example the case for DNS, authenticators or external authorization scripts. ACLs can thus be divided in FAST ACLs, which do not require going to external sources to be fulfilled, and SLOW ACLs, which do.

Fast ACLs include (as of squid 3.1.0.7):

all (built-in)
src
dstdomain
dstdom_regex
myip
arp
src_as
peername
time
url_regex
urlpath_regex
port
myport
myportname
proto
method
http_status {R}
browser
referer_regex
snmp_community
maxconn
max_user_ip
req_mime_type
req_header
rep_mime_type {R}
user_cert
ca_cert

Slow ACLs include:

dst
dst_as
srcdomain
srcdom_regex
ident
ident_regex
proxy_auth
proxy_auth_regex
external
ext_user
ext_user_regex

This list may be incomplete or out-of-date. See your squid.conf.documented file for details. ACL types marked with {R} are reply ACLs, see the dedicated FAQ chapter.

Squid caches the results of ACL lookups whenever possible, thus slow ACLs will not always need to go to the external data-source.

Knowing the behaviour of an ACL type is relevant because not all ACL matching directives support all kinds of ACLs. Some check-points will not suspend the request: they allow (or deny) immediately. If a SLOW acl has to be checked, and the results of the check are not cached, the corresponding ACL result will be as if it didn't match. In other words, such ACL types are in general not reliable in all access check clauses.

The following are SLOW access clauses:

These are instead FAST access clauses:

Thus the safest course of action is to only use fast ACLs in fast access clauses, and any kind of ACL in slow access clauses.

A possible workaround which can mitigate the effect of this characteristic consists in exploiting caching, by setting some "useless" ACL checks in slow clauses, so that subsequent fast clauses may have a cached result to evaluate against.

Back to the SquidFaq

apt-get, aptitude, autoremove and orphans

2010-05-11T15:15:00.000+04:00

Until some weeks ago I used to use only apt-get, and then I realized that aptitude is better in taking care of orphan packages.

I am no expert here, but this is the way I think they both work, at least on Debian Etch, I have heard that in Lenny both works more or less the same, I use Lenny, and yes they can do the same thing but in different ways.

Well when you install a package with apt-get, and that packages needs other packages due to dependencies, it will install all of them for you, which is great, but then when you want to remove the installed package, apt-get will let the automatically installed packages in your PC, while aptitude will rid them away. Aptitude marks all installed packages as "manual installed" and "automatically installed", the first are those you explicitly ask it to install and the latter are those which it decided to install by you, because they are needed, so when you ask aptitude to remove a package it will remove also the automatically installed ones, since Lenny you can use them both, just be sure to use autoremove with apt-get

Here are some examples with mplayer

ggarron@debian:~$ sudo apt-get install mplayer
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libamrnb3 libamrwb3 libasyncns0 libfaac0 libggi2 libgii1 libgii1-target-x libpulse0 libx264-57 libxvidcore4 mplayer-skin-blue
Suggested packages:
  libggi-target-emu libggi-target-monotext libggimisc2 pulseaudio mplayer-doc w32codecs
Recommended packages:
  libggi-target-x libggi-target
The following NEW packages will be installed:
  libamrnb3 libamrwb3 libasyncns0 libfaac0 libggi2 libgii1 libgii1-target-x libpulse0 libx264-57 libxvidcore4 mplayer mplayer-skin-blue
0 upgraded, 12 newly installed, 0 to remove and 100 not upgraded.
Need to get 0B/6785kB of archives.
After unpacking 15.7MB of additional disk space will be used.
Do you want to continue [Y/n]? Y
Selecting previously deselected package libasyncns0.
(Reading database ... 116538 files and directories currently installed.)
Unpacking libasyncns0 (from .../libasyncns0_0.3-1_i386.deb) ...
Selecting previously deselected package libfaac0.
Unpacking libfaac0 (from .../libfaac0_1.26-0.1_i386.deb) ...
Selecting previously deselected package libgii1-target-x.
Unpacking libgii1-target-x (from .../libgii1-target-x_1%3a1.0.1-3_i386.deb) ...
Selecting previously deselected package libgii1.
Unpacking libgii1 (from .../libgii1_1%3a1.0.1-3_i386.deb) ...
Selecting previously deselected package libggi2.
Unpacking libggi2 (from .../libggi2_1%3a2.2.1-5_i386.deb) ...
Selecting previously deselected package libpulse0.
Unpacking libpulse0 (from .../libpulse0_0.9.9-1_i386.deb) ...
Selecting previously deselected package libx264-57.
Unpacking libx264-57 (from .../libx264-57_0.svn20071224-0.0_i386.deb) ...
Selecting previously deselected package libxvidcore4.
Unpacking libxvidcore4 (from .../libxvidcore4_2%3a1.1.3-0.3_i386.deb) ...
Selecting previously deselected package mplayer-skin-blue.
Unpacking mplayer-skin-blue (from .../mplayer-skin-blue_1.6-2_all.deb) ...
Selecting previously deselected package libamrnb3.
Unpacking libamrnb3 (from .../libamrnb3_7.0.0.0-0.0_i386.deb) ...
Selecting previously deselected package libamrwb3.
Unpacking libamrwb3 (from .../libamrwb3_7.0.0.2-0.1_i386.deb) ...
Selecting previously deselected package mplayer.
Unpacking mplayer (from .../mplayer_1%3a1.0.rc2svn20071207-0.2_i386.deb) ...
Setting up libasyncns0 (0.3-1) ...
Setting up libfaac0 (1.26-0.1) ...
Setting up libgii1-target-x (1:1.0.1-3) ...
Setting up libgii1 (1:1.0.1-3) ...
Setting up libggi2 (1:2.2.1-5) ...
Setting up libpulse0 (0.9.9-1) ...
Setting up libx264-57 (0.svn20071224-0.0) ...
Setting up libxvidcore4 (2:1.1.3-0.3) ...
Setting up mplayer-skin-blue (1.6-2) ...
Setting up libamrnb3 (7.0.0.0-0.0) ...
Setting up libamrwb3 (7.0.0.2-0.1) ...
Setting up mplayer (1:1.0.rc2svn20071207-0.2) ...

It is now installed, lets uninstall it, first using apt-get

ggarron@debian:~$ sudo apt-get remove mplayer
[sudo] password for ggarron: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libxvidcore4 libamrnb3 libx264-57 libamrwb3 libggi2 libgii1 mplayer-skin-blue libgii1-target-x libasyncns0 libpulse0 libfaac0
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
  mplayer
0 upgraded, 0 newly installed, 1 to remove and 100 not upgraded.
Need to get 0B of archives.
After unpacking 10.8MB disk space will be freed.

As you can see not all the dependencies are going to be uninstalled, and apt-get itself recommends you to use autoremove
Let's try with it now:

ggarron@debian:~$ sudo apt-get autoremove mplayer
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libxvidcore4 libamrnb3 libx264-57 libamrwb3 libggi2 libgii1 mplayer-skin-blue libgii1-target-x libasyncns0 libpulse0 libfaac0
The following packages will be REMOVED:
  libamrnb3 libamrwb3 libasyncns0 libfaac0 libggi2 libgii1 libgii1-target-x libpulse0 libx264-57 libxvidcore4 mplayer mplayer-skin-blue
0 upgraded, 0 newly installed, 12 to remove and 100 not upgraded.
Need to get 0B of archives.
After unpacking 15.7MB disk space will be freed.

Now it is working better, try now aptitude

ggarron@debian:~$ sudo aptitude remove mplayer
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Reading extended state information      
Initializing package states... Done
Reading task descriptions... Done  
Building tag database... Done    
The following packages are unused and will be REMOVED:
  libamrnb3 libamrwb3 libasyncns0 libfaac0 libggi2 libgii1 libgii1-target-x libpulse0 libx264-57 libxvidcore4 mplayer-skin-blue 
The following packages have been automatically kept back:
  cpp-4.2 dpkg-dev evolution evolution-common evolution-data-server evolution-data-server-common g++-4.2 gcc-4.2 gij gnome-cards-data gnome-games gnome-games-data gnome-mount gtk2-engines-pixbuf 
  java-gcj-compat kdelibs4c2a libafterimage0 libaprutil1 libcupsimage2 libcupsys2 libcurl3 libcurl3-gnutls libexchange-storage1.2-3 libffi4 libgcj-bc libgconf2-4 libglib2.0-0 libgomp1 libgtk2.0-0 
  libgtk2.0-bin libgtk2.0-common libhsqldb-java libopal-2.2 libpq5 libpt-1.10.10 libqt4-core libqt4-gui libqt4-qt3support libqt4-sql libsane libsmbclient libsoup2.2-8 libstdc++6-4.2-dev libungif4g libxine1 
  libxine1-console libxine1-ffmpeg libxine1-gnome libxine1-misc-plugins libxine1-plugins libxine1-x liferea-xulrunner nautilus nautilus-cd-burner nautilus-data openoffice.org-base openoffice.org-calc 
  openoffice.org-common openoffice.org-core openoffice.org-draw openoffice.org-filter-binfilter openoffice.org-gtk openoffice.org-impress openoffice.org-math openoffice.org-officebean 
  openoffice.org-style-andromeda openoffice.org-style-tango openoffice.org-writer python-apt python-uno qt4-qtconfig vim-gnome vim-gui-common vim-runtime 
The following packages have been kept back:
  apt apt-utils aptitude cupsys epiphany-browser epiphany-extensions evolution-exchange evolution-plugins evolution-webcal gcc-4.2-base gnupg gthumb hardinfo imagemagick libgcc1 libnfsidmap2 libstdc++6 
  liferea netcat openoffice.org openoffice.org-evolution openoffice.org-gnome openoffice.org-help-en-us synaptic vim-common vim-tiny 
The following packages will be REMOVED:
  mplayer 
0 packages upgraded, 0 newly installed, 12 to remove and 100 not upgraded.
Need to get 0B of archives. After unpacking 15.7MB will be freed.
Do you want to continue? [Y/n/?] n

It works the same, so since Lenny it seems to be the same if you use apt-get or aptitude, but until Etch (I have not tested it) you better use aptitude as apt-get does not take care of the orphans.