What does an amp do?
Well, the short answer is easy : it makes things louder. The problem is, that’s not all an amp will do. The better any audio equipment is, the less it does to the signal (other than increase volume) and conversely, the worse the equipment is the more it does to the signal (you’ll always see me say that your audio setup only ever sounds as good as the worst component).
The headphone jack that’s built into your portable player is basically designed to “do the job” – it will drive headphones, and in fact it will probably provide enough volume to almost any (except high end/inefficient/high impedance models) headphones, but the problem is that it will actually fiddle with the sound while doing so, resulting in less than optimal sound quality.
Some of what your average headphone jack does is reduce the dynamic range and add some distortion, which leads to things sounding less than ideal. A good headphone stage will provide nice clean power resulting in good volume and overall clarity, maintaining important things like dynamic range.

Now I’ll deviate off topic for a second to address another thing that a lot of people seem to think, or argue, and that’s “I can’t hear a difference”. Actually unless you have hearing damage it’s generally very easy to hear the difference between good and bad audio equipment. It gets more difficult when you’re comparing super high end stuff, but I’d be surprised at anyone who can’t hear the difference between stock ipod buds and something decent, and the same goes for headphones straight from a headphone out on an iPod vs a decent headphone amp.

Anyway, back on track.
If you’re going to buy decent headphones for your portable device, it’s well worth considering a portable headphone amp. Something like the Corda XXS is a fantastic place to start.

BUT WAIT! There’s no point adding an amp to an amp (the headphone out on portables has to have a small amp circuit so it can drive headphones), if you’re going to use a headphone amp you need to be running it from a lineout (if your source has one). If you’re an iPod user you’re probably in luck as you can just use a LOD adaptor such as the ZY Cable to bypass the headphone out on your ipod (iPod -> Zy Cable -> Portable amp -> Headphones -> Happy Ears).

The search for perfection is never over. Or as I once said (unintentionally), I’m working on being a perfectionist, but I’m not quite there yet.