The other day I was copying the entire contents of one USB drive to another. To use this I used the dd command, as I needed the images to be identical. Once the copy started I started wondering about its progress, and was reminded that dd does not have any kind of progress bar or other status output. In this article I’ll outline how to output the status / progress of dd during a large copy.

A note about speed

Before I outline how to output the status of the dd command, I wanted to comment briefly on the speed of the dd reads/writes. The dd command allows you to specify the input file, output file, block size and count. I have found that not specifying the block size can have detrimental effects on the speed of your transfer. Currently I find the best performance specifying a 4k block size, the same as the default block size on ext file systems.

To specify this block size, use:

dd if=/source/file of=/dest/file bs=4k

Status of dd command

The quickest and easiest way I’ve found to check the status of the dd command is to run the following:

watch -n 10 killall -USR1 dd

There are, of course, other ways to find the result, but this is what I use. This command will report the status on any running dd process every 10 seconds. When the transfer is finished you can kill this reporting via ctrl-c.

Installing VirtualBox guest additions seems to vary between distributions. The other day I saw an article outlining how to install VirtualBox Guest Additions on Ubuntu, but those same steps did not apply to my Fedora 13 Virtual Machine. In this article I’ll outline the packages and steps required to install VirtualBox Guest Additions on a Fedora Guest.

Requirements

This tutorial assumes you already have VirtualBox installed and a Fedora Virtual Machine running.

Installation

To successfully install VirtualBox Guest Additions on a Fedora guest, you’ll need to install the following:

yum install kernel-headers kernel-devel gcc

Once these packages are installed you’ll be able to mount the VirtualBox Guest Additions image, and run the required script. This is done using the following steps:

1. In the VirtualBox menu, under Devices, select “Install Guest Additions”
2. Select the VBOXADDITIONS* image from the Places menu, which will mount the image.
3. In a Terminal, navigate to /media/VBOXADDITIONS*/ and run: sudo ./VboxLinuxAdditions-(x86,amd64).sh
4. Reboot the Fedora guest

Installing VirtualBox Guest Additions isn’t difficult, it is just different on most Linux distributions. Hopefully this helps those using Fedora quickly get it installed on their systems.

I was getting curious about the progress of the new GNOME-Shell (which will be part of GNOME 3.x), so I went to find how to install it in Arch Linux. It is, just like a number of things, available in the Arch User Repository (AUR), but I also found that there is an official package in the gnome-unstable repository. This post outlines how to add the gnome-unstable repository and install GNOME-Shell in Arch Linux

Repository

The first requirement for installing GNOME-Shell is the addition of the gnome-unstable repository. This can be done by adding the following to your /etc/pacman.conf:

[gnome-unstable]
Server = http://mirrors.kernel.org/archlinux/$repo/os/$arch/


You’ll notice I’m using mirrors.kernel.org, but you should be able to use your own preferred mirror.

Installation
Once you’ve added the repository, you can add the GNOME-Shell using the following command:

sudo pacman -Sy gnome-shell clutter

Usage

To launch GNOME-Shell (and replace your current environment for this session), run:

gnome-shell --replace

You should now be in GNOME Shell. Have fun playing with this, still in development, Desktop Environment.

Package : libthai
Vulnerability : integer overflow
Problem type : local (remote)
Debian-specific : no
CVE Id : CVE-2009-4012

Tim Starling discovered that libthai, a set of Thai language support routines, is vulnerable of integer/heap overflow. This vulnerability could allow an attacker to run arbitrary code by sending a very long string.

For the oldstable distribution (etch), this problem has been fixed in version 0.1.6-1+etch1.

For the stable distribution (lenny), this problem has been fixed in version 0.1.9-4+lenny1.

For the testing distribution (squeeze), and the unstable distribution (sid), this problem will be fixed soon.

We recommend that you upgrade your libthai package.

I have been using Thunderbird 3 since its final release, and overall I have enjoyed it. I never quite liked version 2, but version 3 appeals to me a bit more. I’ve got multiple accounts configured, and it has been working very well. The one issue that I recently discovered is the HUGE amount of disk-space that it can take up. In this post I’ll outline how to regain that lost space.

Thunderbird 3

If you have been using Thunderbird 3, take a look at how much space the .thunderbird folder is using. Open a terminal (Applications > Accessories > Terminal), and run the command:

cd .thunderbird/
du -sh

On my machine Thunderbird was using 923M of disk space! I was very surprised to find that amount–I didn’t even think I had that much email! This got me curious, because Thunderbird 2 never used nearly this much disk space! I made my way to Google to do some digging, and this is what I found out.

Thunderbird Profile

Your .thunderbird profile directory contains all of your settings, as well as your ImapMail headers and other information. If using the default Thunderbird settings, all email will also be synchronized to your local computer. This means that your email, including attachments, will be stored within this directory (ie; .thunderbird/*.profile/ImapMail/). I have this option turned off, and I’ll outline how to do so below.

I bring up the topic of the profile directory simply so that you are aware of the files contained there. Some of these files are those that you’ll end up deleting to remove some of the bloat that can come along with Thunderbird 3 settings. Generally, the files that take up the most space are:

• /ImapMail/
• global-messages-db.sqlite

We’ll have a look at each of these below.

ImapMail – Mail Synch and Cache

Assuming you are using IMAP with your mail accounts, you will have an ImapMail folder, caching email and headers for your Imap accounts. If you take a look into this directory you’ll find sub-folders for each of your accounts. If you have “Message Synchronization” activated (which, I believe, is the default), you should have a local cache of all your mail in this folder.

If you’d prefer to disable this feature to regain disk space, you can do the following:

1. Delete the contents of the ImapMail/mail.domain.tld/ folder (where mail.domain.tld is your mail server(s)).
2. Disable “Message Synchronization” in Edit > Account Settings > Synchronization and Storage
3. I would suggest leaving the remaining settings at their defaults. This post doesn’t cover them.

Global Search and Indexer

Once I deleted the previously synchronized mail from my ImapMail folder I still found that there was over 200M being used elsewhere. After some additional digging I found that the bulk of this was used by the global-messages-db.sqlite file. This file is used for the Global Search and Indexing features of Thunderbird 3. I did a little research and this sqlite database is used to index your messages, attachments, etc. The new additions in Thunderbird 3 allow you to search this database and quickly find files and messages.

I realized that I rarely use the search feature, so I disabled this setting and freed up a lot of additional space. I use Message Filters to sort my mail, so disabling Indexing and Searching doesn’t affect me. If you would like to disable Global Search and Indexer, and free up an additional 200M of disk-space, do the following:

1. Delete the global-messages-db.sqlite file, and regain the disk-space.
2. Uncheck “Enable Global Search and Indexer” in Edit > Preferences > Advanced > General

I would be interested in any additional feedback the readers have about improving Thunderbird 3. How to make the client more efficient or customizable.

I was reinstalling Arch Linux just the other day and I noticed that, while my Desktop looked natural enough, my web browser fonts did not. I did some quick reading and found some of the fonts that seem to clear up the issue. If your fonts look a bit fuzzy, you’ll likely want to keep reading.

Arch Linux Fonts

To get a better all-around font experience, including the desktop and the web, I installed the following packages:

• ttf-droid
• ttf-liberation
• ttf-dejavu
• ttf-ms-fonts

To install all of these, simply run:

sudo pacman -S ttf-droid ttf-liberation ttf-dejavu ttf-ms-fonts

You may need to restart your browser, or even logout and log back in, but you should notice a change once these fonts are installed.

I manage a number of different types of systems. I have an Arch Linux laptop, a Macbook installed with OS X, CentOS and RHEL machines at work, as well as Debian servers both at work and home. As if that weren’t enough I also maintain a few FreeBSD servers. Between all of these different Operating Systems and variants I find that certain files and commands aren’t always where I might expect them to be. For today’s article I’ve outlined different methods for finding files and binaries on Linux and UNIX systems.

Locate

The first tool I’ll share is the locate command. This command searches through an index, built-daily, of all files and folders on your machine. Locate should be available on all standard UNIX systems, and is generally a very fast way to search for files.

• Positives: fast
• Negatives: not very specific, index updated daily

If you need to re-build your index to include recently added files, you can run the command:

updatedb

Find

The find command is much, much more granular than the locate command but it can also be more complicated to use. Explaining all of its options would warrant a post of its own, so I’ll just outline a few basic uses.

• Positives: very, very granular search parameters
• Negatives: slower than other methods

To use the find command, refer to one of these examples:

find / -type f -iname "httpd.conf"
find . -type f -name ".bashrc"
find /etc/ -type f -name "hosts.*"
find /usr/local/ -type d -iname "www"


These examples are all very similar, with only slight variations, to give you an idea of some of the power of find. Again, find can do much more than this, but this is enough to get you started.

• The first command searches from the root directory for any files (-type f) for the case-insensitive string of “httpd.conf”.
• The second command searches the current working directory for any files with the case-sensitive name of “.bashrc”.
• The third command searches within the /etc/ directory (and subdirectories) for any files matching the name “hosts.*”
• The last command searches within the /usr/local/ directory for any directories (-type d) with the case-insensitive name of “www”.

As you can see, find can be very flexible and this only touches on the advanced search patterns it can define. I highly suggest you have a look at the find man page for more information!

Which

The which command searches for binary files within your PATH. For example, let’s say you need to know the full path to the wget command. You could use:

which wget

You should see something along the lines of:

/usr/bin/wget

Binaries may be stored in different places on different systems, which is again why it is important to know how to find them. I use which frequently within my scripting in an attempt to make scripts portable. Let me give you an example:

#!/bin/bash
if [ -x $(which wget) ]; then
$(which wget) http://example.com/file.txt
fi

This very simply script doesn’t make any assumptions about the availability or the location of the wget tool. It does a simple check to see if the binary, as defined by its full path, is executable and if so use it to download a file. If wget does not exist the script will do nothing, and if it does exist it will be sure to execute it by its full path.

This tool has been helpful when moving between operating systems and variants. It helps me ensure I know exactly where files are, and not make any assumptions.

Whereis

The whereis tool is similar to the locate tool in that it is not as granular as find, and it is also similar to the which tool in that it searches only a predefined PATH for files. I use whereis constantly on my FreeBSD systems to search for ports within the ports tree. An example:

whereis portmaster

This would give me the output (assuming it is installed), of:

portmaster: /usr/local/sbin/portmaster /usr/local/man/man8/portmaster.8.gz /usr/ports/ports-mgmt/portmaster

This searches the standard binary, manual page, and source directories. As you can see from the above output, I have a result from each. This tells me where the binary is in its full path, where the man page(s) is stored as well as within the ports tree. If I didn’t have it installed, the only result I would be given would be the ports tree entry. This would help me find the path, allowing me to install it.

Conclusion

All of these tools are standard UNIX binaries that you should find on any system. Each tool has its own strengths and weaknesses, and none of them will always do the job the right way. I find myself using each of these on a regular basis, each for its own strengths. I would invite you to start using these commands in your day-to-day as needed, and read more about them.

Today I ran into a problem at work that I thought would be worth sharing. It is one of those odd only-happens-once-in-a-blue-moon errors, so writing it down and putting it out there on the internet may help those lucky few who run into it.

The Situation

I had a report from a user that the system account he and his team share was unable to run cron jobs. My initial checklist of things to verify were:

• Verify the syntax of the cronjob(s) by viewing the crontab: crontab -u <username> -l
• Verify user was listed in /etc/cron.allow, or *not* listed in /etc/cron.deny.
• Check the /var/log/cron for informational messages.

The cron syntax looked fine–I didn’t see any errors. I also verified that they were listed in the cron.allow file. (Our systems implement a cron.allow policy, for security.). From the crontab man page:

If the cron.allow file exists, then you must be listed therein in order to be allowed to use this command. If the cron.allow file exists, then you must be listed therein in order to be allowed to use this command.

It was the third entry, the system log, that alerted me to the problem.

Jan 5 10:26:01 hostname crond[21536]: User account has expired
Jan 5 10:26:01 hostname crond[21536]: CRON (username) ERROR: failed to open PAM security session: Success
Jan 5 10:26:01 hostname crond[21536]: CRON (username) ERROR: cannot set security context

The key piece of information here is “User account has expired.” While the shared system account was still usable–it doesn’t require a password–it had technically expired which meant cron would restrict its jobs. Remember, 99% of the time the system log tells you exactly what the problem is. The key is reading!

[ad#Google Adsense]

The Solution

The solution was to unexpire the system account. To do so you can use the chage command or the passwd command. In this situation, because this is a shared system account that does not need to expire, I set it to never expire:

passwd -x -1 username

From the passwd man page:

This will set the maximum password lifetime,  in  days,  if  the user’s  account  supports password lifetimes.  Available to root only. This will set the maximum password lifetime,  in  days,  if  the user’s  account  supports password lifetimes.  Available to root only.

As you might guess, -1 sets an infinite value meaning it will never expire.

My wife and I are movie fans. For as long as I can remember one of our favorite things to do is sit down together and watch a movie. Then we had kids and our movies started becoming toys. It is really frustrating to pull a DVD off the shelf and find either that it is missing or that it has baby fingerprints all over it!

I finally found a very simple solution. RipIT – The Mac DVD Ripper. This utility lets you easily rip DVDs onto your hard drive for archiving. This way I can archive my DVDs to my hard drive, and store all of the originals in a box. This way I can still watch my movies (I watch them now from my Mac) and keep the originals safely out of reach of the kids.

Installation

To install the latest version of RipIT for Mac, visit the site or use this link: Download

Drag-and-drop the RipIT icon into your Applications folder, and start things up.

The free version allows you 10 DVD rips. After that it has a very affordable registration fee to unlock unlimited rips.

Use

I’ve included some screenshots, courtesy of the main RipIT site, to give you an idea of just how incredibly simple it is to archive a DVD onto your hard drive:

It really is as easy as 1, 2, 3!

I’m happy to present the AUR Package of the Week, and also the first package of the new year! Today I thought I would showcase the old and trusted browser, Firefox, with a slight twist from the package available in the main repositories.

Installation

The version of Firefox available in the main repositories is distribution-restricted based on the Mozilla license. You may notice that the version you install is lacking the Firefox branding and name. Again, this is due to a clause in the license. The license say that while Firefox is Open Source software, the name and the branding are not. Unless you’ve made an agreement with Mozilla, it is outside of the license to distribute the browser along with the name and logo. It is fine, however, to get the source from Mozilla and retain the logo and other branding.

With that said, to install a branded Firefox browser use the command:

yaourt -S firefox-branded

Compilation

As is the case with many other packages in the AUR, this one will require compilation. I’ve found Firefox to take a bit of time to compile, but its not nearly as bad as some other packages. Your mileage may vary.

Conclusion

Firefox has long been the flagship of the free software world. I think this is changing with the release of so many other new (WebKit) browsers, but Firefox still retains the market lead. I have found that, while I have become a fan of Chrome, that Firefox is always dependable as a fall back browser.