Killer Configurations

Structuring the network services - A simple start

2010-03-30T10:29:00.001+05:30

As a network administrator, we keep our network growing and up all the time. But, some decisions which we have taken early, becomes a mess and creates many issues now. Yes the one of the issues well known for most of the administrators are the IP assigning.

One of the network started with 4 computers, later became 2 servers and 10 stations, later increased and became more than 50. The computers were not fixed in same place, they started moving between departments, a new one started to get added every week, while new laptop users jumped on and off. 3rd party providers came to office and started working for some time and they were, on and off.

The static IPs were split into zones and assigned but later the tracks were not right.. The PC entry log(at the gates) said we have more than 200 visitors every 3 months. Where our sub-net supports only 255 computer!!!!!!

Yes the answer is simple, we need a DHCP.... but we also needed proper network structure.
We need to know the network usage properly. We have wireless users and wired network users.
All wireless users are laptop users, while few laptops connect through the wired network.

The first level of separation is wired users and wireless users. Our internet gateway server has 2 Network cards which connects to internet and wired network, while the wired and network and wireless are mixed through the hubs.

Now a new NIC card is added in the gateway server and that is used to connect to the wireless hub. A DHCP is used to assign range of IPs for wired network (192.168.1.51 - 150). While the gateway server for wireless network is the same gateway server, but it uses different network address range in 192.168.2.x, A DHCP for this zone is enabled and the IP is leased from 192.168.2.51 to 192.168.2.100.

Now the network has few servers where they belong to wired networks... their IPs are added to static range from 192.168.1.2 to 192.168.1.50.

The gateway server now has a firewall / routing rule. Only few mac address of the wireless network are allowed to connect the wired network while the rest can only access internet.
While the wired network can access all the wireless network.

The new installed PCs now work with DHCP never need to care about the IP when new users come in. Security to a level is in place..... but not 100% will discuss more about this in upcoming posts.

Basics of Networking - Part 4 (Debugging Basics)

2010-02-19T19:31:00.006+05:30

The past posts on the blog were on the basics of networking....
Now we are about to see how can we debug is something is wrong in the setup.

The following posts will help to debug faster.

The default way to go through the debugging will be the following way.
This approach starts to analyze the problem from the PC where the problem is found.

ping 127.0.0.1
If fails, check do the network services are started.
ping assigned IP.
If fails, check the network cable is properly plugged-in
ping gateway
If fails, check the gateway is on and the IPs are in same subnet.
ping DNS / Name servers. (Only if routed / NAT is available)
If fails ping the same from gateway.
ping the gateway of the gateway from the gateway computer.
If fails, check do you have the broadband signals / link is up.
All works but still can't connect?
try tracepath / traceroute with a google.com or yahoo.com
Find at which level it fails.....

Most probably the debugging the issues are based on what problems we have...
Remember a blind issue of internet not working is OK to hear from others... but not when we are working in detail.
So here are some FAQs....

Could not ping Gateway but my network wires are properly plugged.
Gateway pings but could not resolve host names.
I use a proxy. My http connection works but not https and ftp.
SSH connections are not working after introduction of proxy.
I use proxy. DNS name resolving works in the browser but fails in the terminal.
Internet works through browser, but can't ping any IPs / Hosts in the internet.

1. Could not ping Gateway but my network wires are properly plugged.
This may be due to improper IP assigning. We need to make sure that the IP of the PC and the gateway are in same network. (i.e are they in same subnet...). Theoretically they should be ping able to fix this issue. (Please refer the post assigning IPs)

2. Gateway pings but could not resolve host names.
This is due to improper DNS configuration. Is it possible to ping the DNS servers? If yes we need to be sure, they are really DNS servers ;-). If not ping able, we need to know do the DNS servers IPs are in our range of IPs (Within our subnet) or not. If the IP is within our subnet we may need to verify the DNS server configuration to make it work right. If the IP is out of our network. We may need to ping the DNS server from the gateway PC, i.e sometimes the gateway of the gateway might have network issues not letting us to connect to Internet....

3. I use a proxy. My http connection works but not https and ftp.
The proxy server has different way to support different protocols. Some proxy servers use same port for all kinds of requests. If so the client setting should have same proxy setting for different kind of services. Some proxy servers block may not serve certain protocols, better check the proxy configuration to very the supported protocols.

4. SSH connections are not working after introduction of proxy.
If the client is PuTTy we can configure the proxy settings in the PuTTy. If the client is a linux terminal and we have the problem only for SSH. we need to use http proxy for SSH, tools like corkscrew with ProxyCommand in linux will help. The other workaround is to support NAT in the gateway so both proxy and NAT.

5. I use proxy. DNS name resolving works in the browser but fails in the terminal.
When the browser works with proxy, the name resolving happens in the proxy server while when we try in terminal we have the name resolving based on the DNS server settings in the IP / Network configuration, May be NAT is disabled in the network so we cannot resolve the DNS directly from the current PC.

6. Internet works through browser, but can't ping any IPs / Hosts in the internet.
This is similar to the previous question, enabling NAT will support pinging from any PC in network, The internet works in browser because of the Proxy settings.

The above are not the complete list of problems that might come... they will change according to the network and the usage of network services. The NAT / Proxy has its own advantages and disadvantages where the issues are because of them... Planning the network again falls on what kind of services we use and the debugging procedure remains the same, how big the network is.

Will keep you posted on some new network services and setting up a right infrastructure.

Basics of Networking - Part 3 (Internet through Proxy)

2010-02-04T23:08:00.004+05:30

Hurray.... My network is UP........
Hurray.... My Router shares the internet connection.......

Do I need a Proxy?
A good point that makes us to think. Do we need a proxy? When the router shares the internet. Why do we need a proxy?

If the ADSL modem is our router. We need to think about proxy based on our network size. Sharing about hard learning, we felt the router was extremely good to share internet connection acting as a gateway. But when the network size started to grow we faced frequent network connection drops.... Why?

The ADSL Router was not good enough to handle the too many requests from different machines. May be this is not the case for all the routers but our router did this to us (The router is a least version provided by the ISP, not designed for high traffic).

Whats up next?
Let the internet connection be bridged. Let the PC take up the load....
Let the PC take up the Proxy......

Yes. We are to the topic now.......... Let us know about proxies to get internet shared in the network.

What is a Proxy?
To keep it short. It is an application that acts a layer in between our application (browser) and the web server.

Let us understand the network now.
All the PCs are in same sub-net
PC - A - 192.168.1.1
Laptop - 192.168.1.2
PC - C - 192.168.1.3

Gateway for all should be 192.168.1.1 (PC A, should not have a gateway)

The PC A is running on Windows.........
It has two NIC (Network Interface Card). The first one connects to the ADSL router for Internet connection using the bridging option. The second one now connects to the local network with the IP 192.168.1.1

To make the internet connection simple, use AnalogX Proxy.
Download and install it. When we run it... We see it runs on a Port 6588.

Yes it listens on 6588 Port on 192.168.1.1
We need to say this in our browsers and other internet accessing application like GTalk, Skype, Yahoo Chat and more

Click here to know on how to configure your browser.

Do we need to go only with AnalogX?
No not at all.....
We have too many proxy software with very advanced operations.

Are you having a SOHO (Small Office / Home Office) Network?
Wanted more than a normal proxy?
Still wanted the NAT(Network Address Translation) Feature of the ADSL router with a PC as a gateway?
Wanted more features of Proxy, Firewall and Advanced gateway?

The answer would be, try IPCop-Linux........

When the network grows..... Want too many things to do for internet?
Keep watching..... We will see, how to load balance internet connectivity with multiple Internet connections and multiple proxy servers. There are more to come, for now will go with basics in the network.

Basics of Networking - Part 2 (Connecting Internet)

2010-01-30T14:10:00.006+05:30

For a long time network was an unknown thing, while I was using the network services without knowing how it works....

Have I understood it now? Well the answer is "partially". Yes still it is an unknown mystery for me.

But how could I write about something I don't know?
I would say. I write something that I have learned hard.... spent months and years and found a simple solution may I was in wrong direction, I had no right person behind me to teach. All you see in this blog is not learned from a course... but learned when needed, some through other sources, some through practical experience and what ever worked well after the learning is written but they are not always best.... ;-) You find a better way later or you may know it. If so please correct me when they are wrong.

Going to the topic... Let us start inter-networking (I mean, connect to internet).

What are we going to and not going to discuss in this connecting to internet

We are about to see how can we connect the entire network to internet and we are not going to discuss about single PC internet connection as that will be mostly explained by the ISP.

We assume that we are using a broadband connection to share among our network.

Always an broadband internet connection has to go through a router, also called as ADSL modems. These modems take care of two things

Digital signal transmission through the telephone lines.
Acts as a router and becomes our gateway.

The second point looks odd and we are not clear on what it is going to do. Let us make it clear.

Router is usually a device that is usually used to forward information between two networks, basically to connect networks of different subnet.

To access internet we need an IP address that is matching to the network of the provider (ISP). So the Router gets the IP from the ISP and on the other end it also has a local IP of our network.

Do this means it has two IPs?
Yes, it has two IPs, one end for the internet and other for our local network.
It acts as the gateway for the network. (Read - Basics of networking).
So all the internet requests navigate through this gateway and this gateway contacts the ISP to get our requests answered.

What else it can do?
This router also connects in an other mode called bridging. The bridging is a simple way of only acting as a modem and it translates the computer signals through the ISDN wire while the IP of the provider is directly assigned to the computer to which the router is connected. The bridging is possible if only one computer connects to the internet through the router.

The IP will be assigned dynamically or statically. They become active on boot or using PPPoE (Point to Point Protocol over Ethernet) dial-up.

How do we share the internet from the router?
IP Details

Modem/Router - 192.168.1.1
PC A - 192.168.1.2
Laptop B - 192.168.1.3
PC C - 192.168.1.4

All the above has same subnet and same gateway 192.168.1.1 which means the router is the gateway for all IPs.

All the PC needs DNS Server IP to identify the websites out of its network. The DNS IPs will be provided by the ISP or we can use Google's Public DNS.

Thus the ADSL router makes internet available for all the computers in the network.

While we will discuss on sharing the internet using a proxy in upcoming posts.

Basics of networking - Part 1 (Assiging IPs)

2010-01-30T12:41:00.009+05:30

Let us start networking.... ;-) Not social networking

Since the start of the blog, we have been to the topic and this time too we are to the topic.

We are about to connect more than two computers to form a network. This involves various process to make it happen. As this blog is more about configuration management, we expect the readers to know more than basics in the computers. To start with they should know to change IP addresses in the OS.

I assume we are not about to discuss about hardware issues here and the following are correct.

The network cables are properly crimped and they work.
The network switch or hub used to connect is working good.
The NIC (Network Interface Card) is installed properly and is working good.
The OS has necessary drivers and supports TCP / IP (IPv4)
The user has enough rights to change and play with Network Setting in his environment

When most of us know “what is an IP Address” and “how it is useful”. We forget to understand how it really connects to more computers than what we have near us.

Hmmmmmmmmmm.................. What are we going to learn about IP Address now?

Though most of us know what an IP Address is, am adding some simple explanations to go further.

IP Address is like a name to a computer, Which we use to identify the computer, but these are not names with alphabets but with numbers. They are 4 numbers each number separated by a “.” . Each number has a range from 0 – 255 (8 bit). Eg: 192.168.1.1

To make a machine work in network it needs an IP Address to identify in the crowd and this should be unique within the network.

As we decided to connect more than two computers in a network. We are going with the following example.

The IP Address are differentiated into classes A,B, C. Since we are more into action, I would recommend to read about it more detailed. We are having a sample IP Address 192.168.1.1, Let us use this for our network. Before using we need to ensure that they are connected to each other as in the above diagram.

A – 192.168.1.1
B – 192.168.1.2
C – 192.168.1.3

OK. Is this IP address enough to communicate? No we need to say a subnet to make this work.

Subnet............ What is it?

Subnet is a notatation or a number used to say how many computers do this IP Address can connect and what is the starting IP of this range and ending IP of this range. The subnets are also similar to IP but they have few calculations. I would recommend to try the application in http://www.subnet-calculator.com/ where it explains the change in subnets and the change in ranges for that.

So we choose subnet 255.255.255.248 as it has range of 192.168.1.1 – 192.168.1.6 (6 computers in the network)

What happens when an IP is out of this range? How can we access it?
Here comes a gateway for the network. Which always has the door(gate) to access the other network IP. The gate way will be always the first IP in the subnet range, this is not a rule but this is a best practice to identify the gateway in any network. 192.168.1.1 is the gateway here. Setting this up in all the machines(A, B and C) should make the network accessible within the A, B and C.

So, we should be able to ping 192.168.1.2 from Machine A and C and the rest of IPs from other machines(A,B and C). This confirms the network setup.

Lets Make this blog interesting

2009-12-10T12:17:00.002+05:30

This post is something related to the first post in this blog.

The blog for the five months have been random and was rolling across different topics, not consistent on any topic and now its time to make it organized and more interesting for reader, by bringing a series of topics closely related to each other, helps to learn from it and implement on a run.

Q:Whats gonna be in the series?
A:To be short the series will be for system / server / network administrators.

Q: Whats is the upcoming series?
A: Steps in setting up a small / medium office network.

Q: .....?
A: Hope your further questions will be answered in upcoming posts ;-)

Will make a reliable network with basic services...... Till then keep watching the blog

Adding our own Linux startup scripts

2009-11-30T21:10:00.002+05:30

Do we need to start something when Linux system starts?
Its not a service.... But I need to run this command when system starts....

Yes here is a small part which astonished me as I have not learnt this for years and missed it when I need....

Let us take a sample case: We might need to start a SVN daemon on the system.
#svnserve -d /srv/repositories

We need to run the above command on every start-up autiomatically. So we don't need to start this daemon manually.

Simple way is add this along with other startup scripts. Find which runlevel the system runs normally.

[root@sf03 ~]# runlevel
N 3
[root@sf03 ~]#

Our server runs in run-level 3 so lets take that as an example.
The server runs on Fedora Linux 10

The startup scripts for run-level 3 resides in the directory /etc/rc.d/rc3.d/
The scripts for run-level 5 will be at /etc/rc.d/5.d/

The directory contains shell scripts that runs on the ascending order on by one.

The last script that runs is S99local

Which has the content similar to this.
[root@sf03 ~]# cat /etc/rc.d/rc3.d/S99local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
[root@sf03 ~]#

Use the vi editor and add the startup command we need to add to this.

Example:

[root@sf03 ~]# cat /etc/rc.d/rc3.d/S99local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

# Start SVN Server at startup

svnserve -d /srv/repositories

[root@sf03 ~]#

Restart the server and check the script.

Keep the php-pear up-to-date

2009-11-30T10:00:00.000+05:30

PHP has evolved a lot and when we need add-on libraries, we opt for PEAR packages or PECL extentions to add more libraries that resolves our purpose.

Recently in one of our servers CEntOS 5.2, we were about to install phpUnits to run unit tests in it.

Unfortunately phpUnit was not installed on it.

The website gave the following options to install.



pear channel-discover pear.phpunit.de

andpear install phpunit/PHPUnit

But the installation failed........ Oops it was odd to understand why?

The real cause was the pear module has not been upgraded to latest version the the new standard packages were not installed with this.

It would be better to do

pear upgrade pear

before we start any pear installations. Keep the pear up-to-date to make it work with latest library packages.

Integrated Project Tracking Tools

2009-10-30T21:06:00.011+05:30

Project Management is an art in software development. When too many requirements, bugs and more falls into the project with parallel releases and more. Things become hectic to manage them separate with tools.

Commits goes in after code freeze
Commits in cruical areas
Bugs are added. Count jumps more than accepted limit.

When we have tools like SVN or CVS or any other version control system we could track the changes in it. We have tools like websvn, fisheye and more to see the SVN commits. Bugzilla and more to track the bugs

But things are tough when we need to use too many tools to look into to get our final data for management.

Integration of these tools is a good option and getting all the data at one place would be better solution.

Let us look into the existing tools to do those.
1. Trac - A cool python based lightweight Project management tool.
2. Redmine - A RoR based project management tool
3. Indefero - PHP based project management tool

Integrates well with Version Control system RSS/Atom Feeds are available, with multiple project support. Helps to track bugs and manage releases with features and sprints.

We have the above offline applications which can be downloaded and configured to integrate with our environment tools and serve our network.

We have providers like.
sourceforge.net - Open source projects support
code.google.com - Open source projects support
indefero.net - Open source and private projects support
kenai.com - Open source projects support
and more............

Each tool has its own pros and cons but all of these helps in having control of the project from planning to delivery and to post delivery support.

Web SVN Repository Browser

2009-10-22T16:26:00.016+05:30

The need for a Source Control System in a development environment often increases but they are not just the Versioning system they need to do more...

A developer using SVN has many options with his IDE to work with SVN like diff between revisions, comparing and browsing histories etc., but will the IDE fill the complete usage requirement of the SVN?

How about a config manager or a project manager looking into the code base to get some information, do they need to check out the code and use IDE?

Our previous USVN gave few options to browse through the code but it supports only the view for latest version. In the scenarios like this we often require more tools to do this.

Now we are about to explore the WebSVN a tool to browse the repository at different revisions, get a RSS feed intimation when a new checkin happens, also to tar and archive the repository from the branch we need.

Lets look into it.

WebSVN is provided by Tigris the famous SVN tool provider.

What do we need to install WebSVN?
1. PHP Hosted Server ( I prefer a Fedora Linux as it could install all dependencies)
2. PECL and PEAR support to install few modules required by PHP
3. SVN

Installing WebSVN
Login as root or use sudo to perform yum installation

#yum install websvn

It installs all dependencies with WebSVN.

Making WebSVN accessible for external world.

Make the installed directory of WebSVN a sub directory in the existing web server.
#ln -s /usr/share/websvn /var/www/html

Edit the config.php

add the following line before the LOOK AND FEEL Section in the config file.

$config->addRepository('NameToDisplay', 'URL (e.g. http://path/to/rep)', 'group', 'username', 'password');

For Example
$config->addRepository('HR App', 'http://svnserver.local/repository/hrm/', NULL, 'admin', 'admin');

Access the website matching the path we could see WebSVN working as below.

We can add more projects/repositories by adding similar config lines as explained above. The details of the project will look as shown below.

The RSS Feeds can be subscribed and the new check-in and repository changes can be accessed via RSS updates.

The WebSVN also alows to make tar and download repositories by changing more configuration in the config.php.

The WebSVN solves problems like version comparison, change notifications and more.

To conclude its a good utilility for SVN, with few drawbacks.
1. The repository addtion requires config file change - Better if we could do in front end.
2. Has no authentication system so if a repository is added every one who has access to the WebSVN can see all repositories.

Soon we will look into more tools similar to this.

Apache Proxy Security Issue

2009-09-18T18:21:00.011+05:30

Recently We were deploying a ROR (Ruby On Rails) application to be specific its Redmine. Since our web server had too many virtual host running on Apache we couldn't run the webrick web server directly on port 80. We decided to run it on Port 8000 and let apache virtual host for this redmine will be proxying to the port 8000

Whats the configutration?

-------------- Configurations Begins ---------------
<VirtualHost...... >

ProxyRequests On
ProxyVia On


....

       ProxyPass / http://localhost:8000/

ProxyPassReverse / http://localhost:8000/


.....

</VirtualHost.>

-------------- Configurations Ends -----------------

Later a month we observed our web server became too slow, We saw the response taking too much time. Looking at the performance Apache was consuming more memory and cpu load.

Just a top command explained the change in apache's behaviour

Looking into Apache's access log we saw too many web requests unrelated domains to the server were accessed. Finally we realised that the apache became a proxy server and now it is acting as a proxy to many people and they access their banned sites through the apache's proxy service.

The fix is to remove the following entries


ProxyRequests On
 ProxyVia On

The ROR application was still proxied because of the other entry in the VirtualHost. The application still worked and we stopped the open proxy behaviour.

Once the fix was done, we observed all the proxy request in the access logs were denied with 404 and thus the server is saved ;-)

SQUID Load balancing for web applications

2009-09-01T06:22:00.003+05:30

Squid Load Balancer Configurations

A short note on we have done to make the squid load balancer working
The servers are CEnt OS 5.x

Total machines 2 (Don't ask me why it is 2, This is what I had to test and play with in my lab)
Machine 1 - IPs 192.168.5.50 and 192.168.5.51 (2 LAN cards)
Machine 2 - IPs 192.168.100 and 192.168.5.101 (2 LAN cards)

The machine 1 and 2 Will have Apache application running on them which needs to be load balanced.

Problem: We need a seperate machine which should act as load balancer for both, but we don't have any other machine than these 2.
Solution: Machine 1 will act as load balancer and also as a application server(not adviceable) but can go ahead if we run out of resource as we have no other option.

DNS Names
DOMAIN Name for public access webapp.office.lan

webapp.office.lan - 192.168.5.50
server_1_a.office.lan - 192.168.5.50
server_1_b.office.lan - 192.168.5.51

server_2_a.office.lan - 192.168.5.100
server_2_b.office.lan - 192.168.5.101

The web application is configured in both the server with apache vhosts.

The default site is webapp.office.lan and squid listens to the queries on that.

So the first part is making the application work in

server_1_b.office.lan - 192.168.5.51
server_2_b.office.lan - 192.168.5.101

Configuring apache right to make this work properly.
Since Squid and Apache are going to run on same machine(1) and are to be used in same port number 80 we need to make apache listen only for requests on IP:192.168.5.51

To do the above
Change the httpd.conf
Modify the line
Listen 80
to
Listen 192.168.5.51:80

So apache listens only the IP: 192.168.5.51 on Port 80

Now configure the vhost of the web app accordingly in Apache on IP:192.168.5.51
Verify do the site works by accessing server_1_b.office.lan

Configuring App in server_2_b.office.lan - 192.168.5.101
Since the app is now configured on a different machine we don't need to change the Apache Listen property.
but once configured just check the site works with server_2_b.office.lan URL

Squid in Action
Installing squid in CentOS is as same as installing Apache with yum installer.

Configuring Squid
Add the following lines in /etc/squid/squid.conf

------------------------------ Lines to be added in squid.conf -------------------------------
#Make SQUID Listen on PORT 80
http_port 192.168.5.50:80 defaultsite=webapp.office.lan vhost

# Mapping 192.168.5.51 as server_1
cache_peer server_1_b.office.lan parent 80 0 no-query originserver name=server_1 login=PASS
cache_peer_domain server_1 server_1_b.office.lan login=PASS

# Mapping 192.168.5.101 as server_2
cache_peer server_2_b.office.lan parent 80 0 no-query originserver name=server_2 login=PASS
cache_peer_domain server_2 server_2_b.office.lan login=PASS

cache_peer server_1_b.office.lan parent 80 0 round-robin no-query originserver login=PASS
cache_peer server_2_b.office.lan parent 80 0 round-robin no-query originserver login=PASS

----------------------------- End of lines to be added in squid.conf ----------------------

Now the squid can be restarted.
The server will be listening to webapp.office.lan each request will be diverted to different server based on the round robin flow and if any one fails the other will server the request continuously we can add 'n' servers similar to this to make the count higher
Note all the vhost in the Apache should listen to domain name "webapp.office.lan".

Hope this gave a useful information on SQUID loadbalancing, This is not only for apache but can be any webserver serving similarly.

SQUID Load Balancing For HTTP-AUTH Applications

2009-08-19T08:33:00.003+05:30

Recently I was working with SQUID Load Balancing server for one of the PHP Based Web Application.

The app uses HTTP-AUTH for one of its protected directory. It uses Apache .htaccess with .htpasswd Unfortunately the login was completely failing in the live environment but not in test environment.

The difference found was the live environment had a SQUID Load balancing which was not in TEST (Something wrong should not be the case, both environments should resemble similar).

Then it was observed that the User name and password sent from the client is not reaching the real-application server. It is chopped at the SQUID.

Why SQUID is not passing the information?
SQUID has features to do proxy / load balancing with authentication, where SQUID assumes that the AUTH header is for SQUID and not for the web application so it never forwards the AUTH Header.

How to forward the AUTH Header?
Looking in to the squid.conf

cache_peer IP.ADDRESS parent 80 0 no-query originserver login=PASS

The last suffix login=PASS fixed the problem.

The login=PASS forwards the HTTP-AUTH credentials to the destination server.

SQUID Clearing cache in a load balancer / caching server

2009-08-15T08:01:00.004+05:30

Are you running a SQUID Caching server before your web server to boost up the performance?
If yes and you face issues when the content of the site changes. It might be due to the squid's cache still having the old content. The following steps will help to refresh the cache.

Why we need to clear the cache?
In most cases the content in the cache is out dated with the live data.

What is the normal way to clear the cache?
We can clear the cache by removing the files in the cache directory.

Where is the cache directory?
The cache directory changes from system to system based on the configuration file settings.
We can find the cache directory by looking for the cache_dir property in the /etc/squid/squid.conf file.
Steps to clear the cache:
1. Login as privileged user.
2. Shutdown squid.
Eg: Fedora / Redhat / CentOS
# service squid stop
3. Remove the cache files.
The directory is the path specified in cache_dir
#rm -rf /var/spool/squid/*
4. Start the squid again
# service squid start
5. We should be able to see a message cache created in cache_dir directory.

Make Dynamic VirtualHost in Apache

2009-08-14T01:23:00.002+05:30

Are you working in Apache? Are you configuring VritualHost often?
Here is a cool solution that avoids us configuring the VirtualHost directive in Apache often.
Any name based apache virtual host will be automatically mapped with some predefined directory path. Which reduces the time of configuring the apache vhosts.

The below example config change in apache will do the following

Configures all virtual host to the /srv/www directory
All VirtualHost by name should have a directory with the domain name. Ex: Vhost test.example.com will have a directory /srv/www/test.example.com
The Document root directory will be htdocs by default
The error log will be added to the specific domain file and all access logs are added to common file.

Example Configuration to add in httpd.conf
# this log format can be split per-virtual-host based on the first field
LogFormat "%V %h %l %u %t \"%r\" %s %b" vcommon
CustomLog logs/access_log vcommon
#ErrorLog logs/%0_error_log

# include the server name in the filenames used to satisfy requests
VirtualDocumentRoot /srv/www/%0/htdocs
VirtualScriptAlias /srv/www/%0/cgi-bin

Easy SVN Web Administration

2009-08-07T21:46:00.011+05:30

In the last month blog we were looking into "How to install and configure an SVN server".
The blog gives a basic SVN server configuration with command line and managing it through the command line. When we end up with more projects / users we need more repositories and managing the users authentication details becomes a nightmare. To simplify the user creation and repository management we can go for a web based SVN solution.

;-) Ohh don't think that we are going to do the Apache setup for each repository and more. We have a better solution.

Here comes a better SVN with USVN (User friendly SVN)

What do we need for this?
Requirements:

PHP 5 (5.1.2 <= ver)
apache2
mod_dav_svn enable
mod_rewrite enable
subversion
mod_svn enable
mod_authz_svn enable

Steps to setup.
1. Download USVN from http://www.usvn.info/download
2. Extract the zip to the root directory of apache web directory
3. Access the page via web to start the installation and proceed through the installation.

Sample configurations followed in Success Factory.
1. Apache Config

###########################################
# Vhost: svn.successfactory.local #
# Note: we need proper DNS setup to #
# make the URL work #
###########################################
<VirtualHost *:80>
DocumentRoot /srv/www/svn.successfactory.local/htdocs
ServerName svn.successfactory.local
<Directory />
AllowOverride All
</Directory>
<Location /repository/>
ErrorDocument 404 default
DAV svn
Require valid-user
SVNParentPath /srv/svn
SVNListParentPath off
AuthType Basic
AuthName "USVN"
AuthUserFile /srv/svn/htpasswd
AuthzSVNAccessFile /srv/svn/authz
</Location>
</VirtualHost>

The SVN repository resides in /srv/svn as explained in SVN installation post.
An SQLite DB is selected to maintain the SVN management informations.

After installation we can manage the SVN through web like
http://svn.successfactory.local

You will come across a login page as shown below.

We can manage new projects / users / groups through the simple web panel as shown below.

Hope this USVN brings a peace of mind in administring multiple SVN repositories.

Apache RewriteMap with RewriteLock

2009-08-04T21:07:00.004+05:30

Recently I was working with a image gallery site. The site was developed with PHP on Apache which stores images and renders it out.

The photos where stored in a similar path stated below.
/images/photo_id/photos_style/photo_id.jpg

Example:
/images/200/portrait/200.jpg

The requirement was not to show the original path in URL and it should resemble like the below
/<photo_id>/<photo_id>_<_style>.jpg

Example:
/200/200_portrait.jpg

The logic had more complexity than explained here which required a math calculation to get the complete path. To achieve the calculation a perl rewrite rule was introduced.

RewriteMap prg MapType:/path/to/rewrite_rule.pl

The perl script was something similar to below with more logic

#!/usr/bin/perl
$| = 1;
while () {
    # ...put here any transformations or lookups...
    print $_;
}

The script started working well by redirecting to original directory (internally) with output like
/images/200/portrait/200.jpg

But when the server got loaded heavily with more requests. The output scrambled like
/mages/200/portit/200.jpg
/ramages/200/portrait/200.jpg

etc...

Which was due to the perl script not in sync with Apache. The problem was solved when a RewriteLock was introduced. But still a surprise how this solved it immediately... ;-)

RewriteLock "/path/to/empty/lock/file"
in the global section of httpd.conf

Linux: Setting UP DNS Cache server

2009-07-29T00:04:00.005+05:30

1. The DNS servers in a network may be with huge traffic or it might have more downtime resulting in failure of resolving domain names.

2. May be a dialup machine has very slow internet connection where resolving a DNS query might take more time.

The solution for both the problems is to have a caching DNS server. Installing a dnsmasq and running it as a service on local host will resolve the issue.

Steps to Setup DNS Cache Server
(Following lines works good in Fedora / Redhat / CentOS)

Install dnsmasq
$ yum install dnsmasq

Make dnsmasq start on boot
$ chkconfig dnsmasq on

Start dnsmasq immediately
$ service dnsmasq start

Change the network setting to work through this cache server

Open the network settings

Add the Primary DNS as localhost by adding 127.0.0.1
Move the primary and secondary to secondary and tertiary.
Click File->Save

Restart the network
$ service network restart

Test the network DNS resolving speed after the first time access to the site. It will be much faster as it comes from local.

Install & Configure SVN Server

2009-07-25T14:00:00.002+05:30

There are many tutorials available to work with SVN and the best of all is the
svnbook.read-bean.com, This article is about making a quick SVN server with very few steps without much issues.

Server Environment: Redhat / CentOS / Fedora

Install SVN
# yum install subversion

Create SVN Directory

# mkdir -p /srv/svn

Start SVN Server
# svnserve -d /srv/svn

Create SVN Repository
# svnadmin create /srv/svn/myproject

Create Users in /srv/svn/myproject/conf/passwd (Add the following lines)

admin = adM!nPassw0rd
developer = password

Grant User Permissions in /srv/svn/myproject/conf/auth (Add the following lines)

[/]
admin = rw
developer = rw

Test the setup
Checkout from different macine or in same machine with different directory

192.168.1.1 is assumed IP of the SVN server

# svn co http://192.168.1.1/myproject/ myproject

Test SVN Commit

# cd myproject
# echo "Hello World" > test.txt
# svn add test.txt
# svn commit -m "Test Commit"

You can checkout the same project from any machine but now you will find the test.txt.

More details on how to backup and restore svn and structuring to use are explained step by step in TechysPage(SVN-Revision-Control)
It would be good to go through the article in TechysPage.

The recent POST on User friendly SVN will help you to configure SVN with Admin panel

My First Blog for Killer Configurations

2009-07-25T10:56:00.001+05:30

After years of this dream to share what I want to...................
After months of this dream to write a blog on what I want to...................

Finally it happened.

To be short to the point this is my first blog post. This blog is going to be more on Configuration Management.

The blog is named as Killer Configurations as configuring any server / application kills our time, we seek for answer days and night killing our self while the answer is always a simple configuration change.

This blog is for saving people by providing the tips / solution on configuration and to reduce the time of problem solving.

To be more in detail it will be on

Server Setup
Application configuration
Installing Tools
Optimizing the servers
Tricky problem solving in server configurations
Task Automations
Ease up server / service administration
etc........

The topic is not just this will continue more......................
Hope this blog solves the need of the reader ;-)

Wait for upcoming posts.