I've been a Dropbox user for a few years now, but have started using it actively only in the last 12 months or so. It's been an invaluable tool for me thanks to its integration with 1Password, the password tool of my choice. Dropbox also helps with lots of day-to-day tasks and thats why I decided it's time to share some of the tips.

Having used Dropbox extensively on Windows systems (XP on laptop and Win7 on desktops), I've recently moved on to using Dropbox with my Mac OSX desktop and Linux hosting.

So here are the top tips for using Dropbox with Unix – each one does wonders for me and so I hope you like them as well.

Important: If you're not a Dropbox user yet, please use this link to sign up – it means I'll get a small bonus (extra 500MB to my free account) for referring you.

Storing all the common apps and tools in Dropbox

Dropbox is really smart when it comes to uploading your files into the cloud storage and making it universally accessible across all the devices that you choose to pair with your Dropbox account.

One thing I particularly like using Dropbox for is storing the latest (or sometimes not the latest but verified to be fully working) versions of apps and tools I find handy to have on my desktops. In addition to having installers for all your favourite tools avaialble on each workstation, Dropbox account is also handy for simply storing all the necessary software in one location. When traveling, for example, I can open my Dropbox account and safely download the exact version of a particular tool that I need. It saved me a lot of time because I don't have to go to each website and search for that download link.

Syncronizing scripts and config files between hosting systems

I have a dedicated server and use it for running a number of Ubuntu VMs. I've created a separate Dropbox account for my hosting needs, and this means that I now have 2.5Gb of space available for my VMs to exchange files or store immediate backups. Because Dropbox takes care of synchronizing all the content (and it has a LAN sync feature meaning VMs transfer files directly to each other instead of uploading back to the Dropbox site), it's super easy and super fast to have a particular script updated and deployed to multiple systems.

I'm not quite there yet with actually running stuff like important automation or whole websites straight from Dropbox directory, but I use it for deploying scripts and configs all the time – once I get something working properly on one VM, I can then hope from one system to another and run the same set of commands against the files which are synchronized by Dropbox.

Transferring files to and from my hosting

This is a very recent addition to the things I do with Dropbox, but it's an incredibly useful one. Having setup a separate Dropbox account for hosting, I shared one of the folders with my personal Dropbox account, and this means that transferring any files to and from my hosting had gotten to be this much easier. By putting a file into a local directory on my desktop, I have it accessible accross all the VMs on my hosting within seconds.

Likewise, if I'm reading logs or working on updating a particular config file, I can always copy it into Dropbox directory and have it synced back to my desktop.

Prior to this setup I had to rely on scp (passwordless logins using passphrase), and although it was pretty convenient to use, Dropbox approach is much more robust. Because files appear to be local, you get to work with them and manage directories as you like. You don't have to remember the directory tree structure or follow any naming conventions – your files are the same across all the systesm and you don't have to remeber to always sync.

Keeping backups of DBs or websites in Dropbox

Since majority of my websites are publicly available blogs, I don't consider most of the backups to be a sensitive information. To be clear, I don't store my passwords (wp-config.php file or htpasswd files) in Dropbox account, but everything else gets copied into it as a first level backup. I also have been doing automatic backups to Amazon's S3 storage for about 5 years now, this means I can recover from most disasters quickly enough.

The reason Dropbox wins is because I don't have to pay for each minor transfer or for storing an extra gigabyte or two – and yes, every little helps even though Amazon's services are quite affordable. Another major reason I started doing backups to Dropbox is because it's a local directory – I don't have to use any extra tools to access all the backups in a simple directories/files structure. With Amazon's S3 it's also possible but setup is not as trivial.

Using Dropbox for controlling Unix systems remotely

With a few minutes and a really simple script, it's possible to setup your own mission control for all the VMs in your hosting.

For example, if you create a cronjob which looks for a particular file, you can control which DB server your systems will connect to or which directory you'll get the latest important log file copied into.

I'm also playing with services management based on the Dropbox account. If there's a file present, I keep a particular service running. As soon as the file is gone, my cronjob gracefully stops the service. A slightly more sophisticated approach involves storing services names and system names associations in a Dropbox synchronized file – this allows for more flexibility as I can specify which service I want to be running on which nodes.

Sure enough, this isn't the most straightforward way to manage your system, but such an approach can be used on the go from your iPhone. For example, I can restart a webserver by just touching a file from my iPhone, while previously I would have to find the nearest computer I trust, download SSH client, connect to the box and only then fix the problem.

Have I convinced you enough? Did you like any tips, or do you have some more perhaps? Let me know in the comments section!

pS: if you don't have a Dropbox account or perhaps if I persuaded you to create a separate one for your hosting – please use this link so that I get some extra Dropbox space for referring. Thanks!

To make yum command use proxy, your best best is to edit /etc/yum.conf and add your proxy server reference:

proxy=http://192.168.3.1:3128

You don't have to restart anything but it may be a good idea to do yum clean all and then yum check-update:

[root@testvm1 ~]# yum check-update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: be.mirror.eurid.eu
* epel: epel.uni-oldenburg.de
* extras: be.mirror.eurid.eu
* updates: centosa5-msync-dvd.centos.org
base                                         | 3.7 kB 00:00
base/primary_db                              | 4.5 MB 00:01
cr                                           | 3.0 kB 00:00
cr/primary_db                                | 1.2 kB 00:00
epel                                         | 3.4 kB 00:00
epel/primary_db                              | 3.7 MB 00:00
extras                                       | 3.5 kB 00:00
extras/primary_db                            | 6.3 kB 00:00
updates                                      | 3.5 kB 00:00
updates/primary_db                           | 1.8 MB 00:01

Not only that, but the behaviour was quite bizzare: upon my first attempt to connect the public key would get rejected and a regular password would be requested by the ssh session. But once I successfully logged in with my password, any subsequent ssh connections would happily authenticate by my public key and would let me in without a problem.

Those of you using home dir encrypiton in Ubuntu are probably smiling right now! But becase I have never consciously configured or used this feature, it took me a good few hours to troubleshoot the issue and come up with the fix.

Why public-key based SSH doesn't work with encrypted home directories

The answer is quite simple: before your server can decide whether you are providing a valid and trusted SSH key, it must read your public key stored in your homedir. But if your homedir is encrypted, this becomes a classical chicken-and-egg scenario – until you log in and therefore decrypt your homedir the server won't gain access to your public key. Only you wouldn't be needing the public key by then, would you?

Store your authorized SSH keys outside your encrypted home directory

If you happen to like your homedir encryption AND would like to use public/private key SSH authentication,  there is a way out: you need to store your authorized keys outside of your encrypted homedir.

The usual access restrictions and directory/file permissions still apply, so the only thing you're changing is moving your authorized keys outside of the encrypted homedir on your server. This way things will work exactly as you expect: you authenticate with your private key and this results in your automatically mounted and decrypted homedir.

Here are the steps to make this happen. You're going to need superuser privileges for my scenario because it caters for all the users on your Ubuntu server, not just one account that belongs to you (use sudo to become root).

Step 1: create a directory structure for your authorized keys.

First, the main directory, I created it under /var – seems quite a safe choice since this directory is unlikely to grow and is equally unlikely to get removed by accident.

# mkdir /var/openssh

Perfect! Now we need to create user-specific directories, just to keep this dir really tidy. My username is "greys", so here is the directory:

# mkdir /var/openssh/greys
# chown greys /var/openssh/greys

Step 2: copy existing authorized keys file into new location

(you must log in as your username for this, otherwise the homedir will stay encrypted)

$ cp /home/greys/.ssh/authorized_keys /var/openssh/greys

Step 3: update SSHd config with new location for authorized_keys file

You're going to do this as root once again:

# vi /etc/ssh/sshd_config

update the value of the AuthorizedKeysFile so that it looks like this:

AuthorizedKeysFile        /var/openssh/%u/authorized_keys

Step 4: Restart SSH service

# service ssh restart
ssh start/running, process 3708

That's it! Give it a try and let me know how it worked out.

Recommended books:

See also

• How To Enable Secure Shell service in Ubuntu

I have great plans for UnixTutorial in 2012, and would welcome any opportunity to share knowledge and experience with all of the readers and new visitors
of this blog.

Here's just a few of the things I plan to do:

• UnixTutorial members area – long time coming, this area of the website will finally make a proper debut in the next few months. I'll be announcing the next round of email subscriptions shortly, so don't miss out if you're still interested
• A series of UnixTutorial eBooks – eventually a balanced collection of free and paid material in PDF and Kindle formats (polls to decide which topis are in demand will follow shortly)
• Broader coverage of Unix topics – this year I expect to write a lot more about Mac OS and AIX systems
• New WordPress theme and quite likely a mobile copy of the website (let me know what devices you have, I own  iPhone and iPad so will do initial testing)
• Completion and expansion of the Basic Unix Commands and Advanced Unix Commands sections
• More Unix book reviews and recommendations
• Reviews of latest Unix-like OS releases
• Even more Questions and Answers

If you expect to see even more – now would be a really good time to let me know by leaving a comment. Thanks and stay tuned!

Not everyone is aware that dig command is very useful for entry-specific DNS research. This post just shows you a very simple example.

dig to confirm TTL for a DNS entry

When using dig, we're usually after a specific section of its output. Consider this simple query (your output may slightly vary):

srv1# dig www.google.com

This query targets a specific DNS record – namely the www. one, rather than a whole google.com domain.

; <<>> DiG 9.7.1-P2 <<>> www.google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4968
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 13, ADDITIONAL: 8

;; QUESTION SECTION:
;www.google.com.                        IN      A

;; ANSWER SECTION:
www.google.com.         541187  IN      CNAME   www.l.google.com.
www.l.google.com.       156     IN      A       209.85.148.104
www.l.google.com.       156     IN      A       209.85.148.105
www.l.google.com.       156     IN      A       209.85.148.106
www.l.google.com.       156     IN      A       209.85.148.147
www.l.google.com.       156     IN      A       209.85.148.99
www.l.google.com.       156     IN      A       209.85.148.103

;; AUTHORITY SECTION:
com.                    37414   IN      NS      d.gtld-servers.net.
com.                    37414   IN      NS      e.gtld-servers.net.
com.                    37414   IN      NS      g.gtld-servers.net.
com.                    37414   IN      NS      f.gtld-servers.net.
com.                    37414   IN      NS      a.gtld-servers.net.
com.                    37414   IN      NS      h.gtld-servers.net.
com.                    37414   IN      NS      b.gtld-servers.net.
com.                    37414   IN      NS      m.gtld-servers.net.
com.                    37414   IN      NS      c.gtld-servers.net.
com.                    37414   IN      NS      j.gtld-servers.net.
com.                    37414   IN      NS      i.gtld-servers.net.
com.                    37414   IN      NS      l.gtld-servers.net.
com.                    37414   IN      NS      k.gtld-servers.net.

;; ADDITIONAL SECTION:
a.gtld-servers.net.     22830   IN      A       192.5.6.30
a.gtld-servers.net.     23008   IN      AAAA    2001:503:a83e::2:30
c.gtld-servers.net.     581     IN      A       192.26.92.30
d.gtld-servers.net.     581     IN      A       192.31.80.30
e.gtld-servers.net.     581     IN      A       192.12.94.30
f.gtld-servers.net.     581     IN      A       192.35.51.30
g.gtld-servers.net.     23226   IN      A       192.42.93.30
h.gtld-servers.net.     581     IN      A       192.54.112.30

;; Query time: 1 msec
;; SERVER: 88.198.6.2#53(88.198.6.2)
;; WHEN: Fri Jul  1 03:50:38 2011
;; MSG SIZE  rcvd: 512

We're only interested in the ANSWER section:

;; ANSWER SECTION:
www.google.com.         541187  IN      CNAME   www.l.google.com.
www.l.google.com.       156     IN      A       209.85.148.104
www.l.google.com.       156     IN      A       209.85.148.105
www.l.google.com.       156     IN      A       209.85.148.106
www.l.google.com.       156     IN      A       209.85.148.147
www.l.google.com.       156     IN      A       209.85.148.99
www.l.google.com.       156     IN      A       209.85.148.103

As you can see from this example, the global www.google.com name is a CNAME entry with quite a high TTL, pointing to a number of www.l.google.com A entries with a much smaller TTL. In this particular example, the TTL for each www.l.google.com is 156 seconds, which is slightly less than 3 minutes.

Once that's done, you can use do-release-upgrade for a hassle free upgrade.

IMPORTANT: are you can see, I've used a really old Ubuntu server with 8.10, hence your procedure for upgrading more recent Ubuntu versions may be slightly different. For example, later upgrades will warn you if you're doing a release upgrade over ssh.

What do-release-upgrade is and when you should use it

do-release-script is a Python script which automates the process of updating multiple packages. It relies upon Ubuntu's core package management functionality.

Apart from downloading and installing updated versions of packages found on your system, this command attempts to take care of all the necessary Ubuntu-release related file changes.

Step 1: Run do-release-upgrade

Once you type the do-release-upgrade command name and press Enter, you should see how vital information about packages currently installed is being collected:

# do-release-upgrade
Checking for a new ubuntu release Done
Upgrade tool signature Done
Upgrade tool Done
downloading
extracting 'jaunty.tar.gz'
authenticate 'jaunty.tar.gz' against 'jaunty.tar.gz.gpg'
Reading cache
Checking package manager
Reading package lists: Done
Reading state information: Done
Updating repository information
Done http://archive.ubuntu.com jaunty Release.gpg
Done http://archive.ubuntu.com jaunty-updates Release.gpg
Done http://security.ubuntu.com jaunty-security Release.gpg
Done http://us.archive.ubuntu.com jaunty-backports Release.gpg
Done http://security.ubuntu.com jaunty-security Release
…
Checking package manager
Reading package lists: Done
jaunty-security/multiverse
Packages: 98  2
Reading state information: Done
Reading state information: Done
Reading state information: Done
Calculating the changes

2. Confirming what upgrading will do

This is your last change to change your mind. All the necessary information about your current Ubuntu release is collected, and now you're presented with the exact upgrade details: how many packages will be removed, how many new ones will be installed, how many will be upgraded. You also are given details about the required amount of data to be downloaded should you decide to proceed with the upgrade;

Do you want to start the upgrade?

1 package is going to be removed. 23 new packages are going to be installed. 420 packages are going to be upgraded.

You have to download a total of 248M. This download will take about 7 minutes with your connection.

Fetching and installing the upgrade can take several hours. Once the download has finished, the process cannot be cancelled.

Continue [yN]  Details [d]

Ready? Press y for yes!

3. Downloading all the packages

Just like with apt-get, you will now see the progress of downloading all the updated packages for your Ubuntu OS. At the bottom of the screen you will see the overall completeness of the download (22% in my example), the current download speed (598kB/s in my case) and the ETA:

Done http://archive.ubuntu.com jaunty-updates/main libbz2-1.0 1.0.5-1ubuntu1.1
Done http://archive.ubuntu.com jaunty/main libdb4.7 4.7.25-6ubuntu1
Done http://archive.ubuntu.com jaunty/main libncursesw5 5.7+20090207-1ubuntu1
Done http://archive.ubuntu.com jaunty-updates/main libssl-dev 0.9.8g-15ubuntu3.6
Done http://archive.ubuntu.com jaunty-updates/main libssl0.9.8 0.9.8g-15ubuntu3.6
Done http://archive.ubuntu.com jaunty/main python2.6 2.6.2-0ubuntu1
[23%] 598kB/s 5min17s

4. Upgrade

Once package are downloaded, they will get installed once by one, with package-specific questions asked for software like postfix or apache.

5. Reboot

To finalize the distro upgrade, you will need to do a reboot. Once completed, you should have a shine next release available.

Recommended books:

While Mac OS is unlike any Unix-like operating system I've managed so far, there are certainly some of similarities. I can honestly say that I'm enjoying the Mac Book Pro so far, and hope to discover most of the differences compared to my previous Unix-like desktop which is Ubuntu 9.10.

Mounting NFS on MAC OS X

One thing which I noticed immediately was that out of the box it was impossible to mount any NFS shares from my Ubuntu NAS server. Any attempt to mount a remote filesystem would give me an error like this:

mbp:~ root# mount nasbox:/try /mnt
mount_nfs: /mnt: Operation not permitted

This error was happening for a relatively simple NFS share on the Ubuntu box:

nasbox# cat /etc/exports
/try            (rw)

… so I started looking around and realized that the reason for this strange problem is quite simple.

Mac OS X uses non-standard port for outgoing NFS connections

That's right – apparently, Mac OS X uses a non-privileged port (2049) for TCP and UDP connections serving the NFS transport. What this means is that most likely attempts to mount remote filesystems will fail, because most NFS servers don't really like connections from insecure ports.

There are two ways to approach this problem:

1. Fix it on the client side (probably makes more sense)
2. Fix it on the NFS server side (if you manage both systems)

Using reserved NFS port number on Mac OS X

There's a mount option supported by the mount_nfs command, which allows you to force the NFS client connections to originate from a privileged port. This will magically make your attempts to mount remote filesystems successful. The option is called resvport.

First we double-check that default mounts still don't work:

mbp:~ root# mount nasbox:/try /mnt
mount_nfs: /mnt: Operation not permitted

… and now let's use the resvport option:

mbp:~ root# mount -o resvport nasbox:/try /mnt

… and make sure we're actually looking at a mounted filesystem:

mbp:~ root# df -h /mnt
Filesystem   Size   Used  Avail Capacity  Mounted on
nasbox:/try    61Gi   56Gi  2.6Gi    96%    /mnt

Allowing connections from non-privileged ports on NFS server

Like I said, if you manage both the Mac OS based client and the NFS server, perhaps it makes more sense to relax the default NFS server security and allow the connections from non-privileged ports.

Just to remind you about the validity of such a decision, the option to allow non-privileged connections is called insecure.

Here's how you use it:

nasbox# cat /etc/exports
/try            (rw,insecure)

After making this change to the /etc/exports file, you'll have to restart your NFS server. On my Ubuntu NAS box, it's done like this:

nasbox# /etc/init.d/nfs-kernel-server restart
* Stopping NFS kernel daemon
…done.
* Unexporting directories for NFS kernel daemon…
…done.
* Exporting directories for NFS kernel daemon…
…done.
* Starting NFS kernel daemon
…done.

We know are ready to attempt the default mount of the same filesystem on the Mac OS X client:

mbp:~ root# mount nasbox:/try /mnt

That's it! I won't promise any Mac OS posts just yet, but if there is enough interest – I'd love to do the research and to post all the discoveries on the Unix Tutorial pages.

If you're an existing member and have a topic which you think belongs to the Operating Systems Basics module, please leave a comment.

The purpose of the module is to give a very high level overview of how modern operating systems work and to also explain the importance of securely managing all the available resources (hence a few topics on files, users and privileges). Almost every single topic deserves a separate module, if not a course, on its own – but for the moment it will be really basic stuff.

Here's the list of topics so far:

• What OS is and why we need it
• Features of a modern OS
• Kernel and kernel modules
• Features of Unix OS
• MINDMAP: Features and components of an OS
• Everything is a file! Types of files in Unix
• A typical filesystem tree of a Unix-like OS
• Users, privileges and file ownership
• Executing a binary file in Unix

To all those of you asking for the date when this module will be posted: I will make the Operating Systems Basics module available to all the existing Unix Tutorial Members on February 1st, 2010.

See also:

• Unix Tutorial – members area
• What Unix Tutorial members area is about
• Unix Tutorial gets a Facebook page

See you all there, and feel free to share your suggestions – either here or on the wall of the Unix Tutorial page.

How to confirm if IPv6 is running on your system

IPv6 is implemented as a kernel module, so you can use the lsmod command to confirm if it's currently running on your Red Hat system:

$ lsmod | grep ip
ipv6                  410913  36

If lsmod doesn't return anything, it confirms that your system isn't running IPv6.

Prevent IPv6 from getting started by modprobe

As you probably know, modprobe command is used for probing modules upon system boot. Probing simply means a module is loaded and an attempt is made to start it up. With any luck, the module starts successfully and its functionality becomes available to the Linux kernel.

For the boot stage, modprobe uses a special config file which helps you control its behavior: /etc/modprobe.conf. For now, let's just concentrate on the IPv6 task at hand. To make sure modprobe doesn't load the actual module next time your OS reboots, add the following line to the top of the /etc/modprobe.conf file (yes, you're going to need root privileges for this):

install ipv6 /bin/true

WHY THIS WORKS: just to quickly explain the line above, here's why we're using the /bin/true command. install is rule of the modprobe config file which overrides the standard way of probing a module. Effectively, it tells modprobe to just run the specified command for starting a module. In the line above, we're telling modprobe to use the /bin/true command for probing the ipv6 module. And as you remember, /bin/true is a command which doesn't do anything but still returns a successful completion code – so in effect we're doing nothing instead of loading the ipv6 module, and we're looking good while doing this too.

Next, add the the following two lines to the same /etc/modprobe.conf file, and they will ensure that common aliases used for starting the IPv6 module are ignored by modprobe:

alias net-pf-10 off
alias ipv6 off

IMPORTANT: These change doesn't immediately disable IPv6 on your system. Being a pretty important module, IPv6 isn't easy to disable on a live system, and so the easiest is to always follow the changes above with a reboot.

Make sure your IPv6 firewall is disabled

The ip6tables service (/etc/init.d/ip6tables sciprt) is responsible for starting IPv6 firewall, and so you probably want to disable it:

# chkconfig ip6tables off

and then confirm that it won't be started next time you reboot or switch to any runlevel:

# chkconfig --list ip6tables
ip6tables       0:off   1:off   2:off   3:off   4:off   5:off   6:off

That's it, there really is nothing else to disabling IPv6. Let me know if you have any questions, otherwise I'll talk to you soon!