Zen of Linux

Windows XP stuck in endless activation/login loop

tengo — Mon, 01 Mar 2010 15:10:19 +0000

Not everyone is using *nix OS on their computers. Recently, I had a problem with another machine that was stuck in an endless loop of the login screen, completely preventing a proper startup of the machine.

The story was that the machine got a few hardware problems after transport which lead to DRAM replacement and changes in the BIOS config. Booting again afterwards presented a “Three days left to register” message on subsequent boots.

Don’t ask why the activation period passed, but after it was over, the machine asked for product activation on every boot. The logon/login screen, the one with the user accounts on it, came up. A click on one of the accounts made a message window pop-up asking if the user wished to activate, Yes or No. Clicking on either one made the computer boot a bit, presenting the Desktop without icons, then the “Welcome”-on-blue screen and - back to the Accounts/Login Screen, starting the cycle over again. Booting into Safe-Mode didn’t help either as executing Windows Product Key Update Tool isn’t possible in Safe-Mode, basically all Activation activities do not work in Safe-Mode!

Many posts on the net suggest a repair with the Windows Install CD. I even tried deleting the regkey files “wpd.*” in C:\Windows\System32 - nothing worked. I was close to wiping XP with a fresh install. Then I found these posts post 1 post 2. Both told me that the Activation process was somehow damaged and needed simple repair.

The Solution:
What I did was this:

I booted XP into “Safe-Mode without Network drivers”. Then did Start > Run and re-registered some login-related dlls into regserver:
regsvr32.exe licwmi.dll
regsvr32.exe regwizc.dll
regsvr32.exe licdll.dll

Additionally, I don’t know if that made a difference (or if skipping this step would fix the problem without the need to re-activate), I set the registry to ask for activation again:

Start > Run, type “regedit”
Viewed the registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon
Added a DWORD value to this key named “Activation Required”, and
Set the value of this DWORD to 1 (dword:00000001).

After that XP booted again into a state beyond logon and asked me to register. I registered via Phone and tataaa!: Desktop with Icons, everything back to normal.

German translation: Dieser Artikel beschreibt, was die Lösung sein kann, wenn: Windows XP bootet bis zum Login Screen. Wenn man auf einen Benutzer Account klickt wird man darauf hingewiesen, dass Windows neu aktiviert werden muss. Egal ob man Ja oder Nein wählt, der Rechner lädt kurz, zeigt eventuell den Desktop ohne Icons, gefolgt von “Welcome/Willkommen” und kehr dann wieder zurück zum Login Screen ohne den Nutzer ins System zu lassen oder die Aktivierung anzubieten. Man ist effektiv ausgesperrt, in einer Endlosschleife aus Registrierungs-Aufforderungen und Win-Logon. Da eine erneute Aktivierung im Safe-Mode nicht möglich ist, kommt man so auch nicht weiter.

Copy DVD: Device or resource busy

tengo — Mon, 21 Dec 2009 11:04:44 +0000

While wanting to use DVD::Rip, Thoggen DVD Ripper and GnomeBaker to copy (or backup) a DVD today, I encountered the strange “Error trying to open: device or resource busy” on my Ubuntu box today.

I solved this by freeing the dvd drive from system access with:

sudo umount /dev/sr0

and everything worked. BTW: I recommend GnomeBaker for easy copies…

HTTP Live Streaming to the iPhone

tengo — Wed, 25 Nov 2009 18:31:48 +0000

Live http streaming (specs here) is essentially a number of short video segments (~10 seconds each) and a permanently updated index file (.m3u8) that tells the iPhone where to fetch the next segment. Each segment is x264 video in a mpeg ts container.

You can setup a working streaming chain using ffmpeg as input, a segmenter and a simple http webserver.

First you need to compile the open-source segmenter.c from assembla:

Download the segmenter.c and Makefile

If you get this error:

segmenter.c: In function 'add_output_stream':
segmenter.c:44: error: 'AVCodecContext' has no member named 'ticks_per_frame'
segmenter.c:46: error: 'AVCodecContext' has no member named 'ticks_per_frame'
segmenter.c:54: error: 'AVCodecContext' has no member named 'channel_layout'
segmenter.c:54: error: 'AVCodecContext' has no member named 'channel_layout'
segmenter.c: In function 'main':
segmenter.c:184: error: 'HUGE_VAL' undeclared (first use in this function)
segmenter.c:184: error: (Each undeclared identifier is reported only once
segmenter.c:184: error: for each function it appears in.)
segmenter.c:193: error: 'INT_MAX' undeclared (first use in this function)
segmenter.c:248: warning: implicit declaration of function 'avformat_alloc_context'
segmenter.c:248: warning: assignment makes pointer from integer without a cast
make: *** [all] Error 1

you need to modify the command-line passed to gcc via the Makefile. So edit Makefile and pass a path to ffmpeg sources to the compiler via the -I flag.
After you passed this one, you might get another error:

gcc -Wall -I//ffmpeg -g segmenter.c -o segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad
/usr/bin/ld: cannot find -lbz2
collect2: ld returned 1 exit status
make: *** [all] Error 1

in this case you need the libbz2-dev package. Install with apt-get install libbz2-dev

If you still get errors after that, your system libs appear to be a bit messed up and gcc picks up and old libav version from somewhere. A cure should be to build a recent version of ffmpeg with the –enable-shared flag set, so that new versions of the libav libraries end up being installed in /usr/local/lib.

If your freshly compiled ffmpeg (and subsequently your segmenter) tells you”ffmpeg: error while loading shared libraries: libavdevice.so.52: cannot open shared object file: No such file or directory then you need to add the shared libraries directory to your shell environment with “export LD_LIBRARY_PATH=/usr/local/lib“. Help on that here.
Another approach to permanently let the system know where these libs reside is to: as root - create a file /etc/ld.so.conf.d/ffmpeg.conf and put this line into it “/usr/local/lib“. Then run the command ldconfig -v to rebuild the library linker environment.
We’ve also discussed lib linking issues in my compiling ffmpeg post.

In order to stream proper videos to your audience, regarding video size, take these iPhone specs and some calculations into account:

480 x 320 iPhone v1 + v2 native Display resolution
640 x 360 common 16:9 video resolution # divide width like this (640 / 16) * 9 = to get height
480 x 270 iPhone 16:9 letterboxed, full width video
320 x 180 iPhone common setting for 16:9 low Quality streaming video

Now, you need to figure out for which bandwidth you’l like to transcode your source video. The iPhone can, based on hardware revision and telco provider, consume quite a number of transmission standards:

3G    iPhone v2, v1 standard cellular data rate, below 100kbps in sum
GPRS  iPhone v2, v1 55-171kbps in sum
EDGE  iPhone v2, v1 108up, 217down, 236kbps in sum
3G                  some tell me <384kbps for audio+video is okay for 3G
UMTS  iPhone v2     348kbps in sum
HSDPA iPhone v2     700-5700kbps in sum
WiFi  iPhone v2

Use this list with caution. As a general rule of thumb, these bandwidth figures might suffice. But I am no expert here (!) and haven’t checked them thoroughly. So correct me if I am wrong. A better comparison might be here. 192kbps video+audio streams should be playlable on most iPhones.

Encoding videos for segmentation with ffmpeg should be a no brainer. But it turned out it is trickier than thought. Some useful command lines are shared here, here and here. One particular thing I ran into was the error “dts < pcr ” which resulted in invalid segments and thus an invalid stream. Other users are getting this error as well. A bit of explanation is here. There are some related issues on the ffmpeg issue tracker, e.g. this.

Sadly, there is no desktop application that supports Apple’s pantos HTTP Live Streaming standard, at least as of now. This makes it a bit cumbersome for a developer to debug an iPhone stream, at least without an iPhone at hand.
Mplayer is the wonderweapon of media-players and is *nearly* capable of streaming iPhone streams/segments. Start it with:

mplayer -playlist http://example.com/path/to/playlist.m3u8

and it will play the first segment and choke on subsequent ones. But as mplayer can play any shit thrown at it, it will even play segments that the iPhone won’t play. So this is really for basic testing only.

Note:
The ioncannon posts got a working combination of ffmpeg and segementation together - using mp3 for audio. Still, I don’t know why my tests with aac as audio encoder (as the original specs require) were so unsuccessful.

Further reading:
draft-pantos-http-live-streaming-01 - HTTP Live Streaming
draft-pantos-http-live-streaming-02 - HTTP Live Streaming
HTTP Live Streaming - Wikipedia

Building vlc

tengo — Thu, 05 Nov 2009 12:45:19 +0000

Compiling a recent/up-to-date vlc from source, from the latest SVN can be a pain as vlc has LOTS of dependencies. It can be, but it must not. Below I’m goint to present a number of strategies to take the hurdles and get a (more of less) working version of a recent vlc.

I’d like you to note that it might be relevant which version of Linux you are running, for example vlc versions newer than 0.8.6 tend to be not compilable under Debian 4.0 Etch without customizations. That’s why Debian Etch has this ancient 0.8.6a version onboard. See and here. The main reason for this is that newer versions of vlc rely on libraries that in turn are not compilable on Etch as well.

Having said this, this post here is my experience compiling vlc on debian 4.0 etch and I got a very recent version of vlc compiled and working!

The UnixCompile vlc docs page should be an excellent guide. Before you read on, read it over at videolan.org!

Strategy 1: Use the build system

Yes, vlc relies on many libs. Knowing this, the vlc developers thankfully added a contribs system in the vlc dirs that makes adding missing libraries, or libriaries you wouldn’t like to install on your system, a breeze.
Just in case you don’t know, a compile relies on libs available from the environment, your system. To get through the build you can install these libs via apt-get, or tell the compiler to look for the libs in another than the standard location. This location is, eg., the vlc contribs dir. Using the build system will populate this path. It might be useful to have another look at How do I compile stuff or vlc Wiki UnixCompile or ConfigureLine.

Using the build system is, according to the UnixCompile docs, for a “distro that is really bad and doesn’t provide the libs”, as you normally would install required libs with apt-get build-dep vlc. As Debian Etch is a bit behind, I like to call using the contribs system chosing the “universal approach”…

What’s a “contribs system”? Basically it is a subdir in your vlc-sources dir - put all the lib sources you’ll need to build vlc here and configure will find them. Or let the “contribs system” do this for you:

> cd extras/contrib
> ./bootstrap
> make

For the guide below, I worked with this git snapshot version from the mirror repo.

Still, keep this quote from here in mind:

The contribs are a way to supplement bad OS/distros which don't
provide a simple way to get the libs and development files. It's full
of hack and patches and initially designed for MacOSX and win. So it's
possible that iconv is not correctly build on unix.

The fribidi error

If the build system approach exits with an error, as it did for me (the fribidi lib just won’t compile and broke the make), then you forgot to make sure automake and autoconf is installed (see this thread here). So do:

apt-get install automake autoconf

wipe your vlc build dir and start over. It should run through now. If you still experience fribidi errors, try this thread.

The out of registers error while building ffmpeg

The next error I got was: “error: can’t find a register in class ‘GENERAL_REGS’ while reloading ‘asm’” when it got to ffmpeg. First a look at ffmpeg compile problems might help.
The solutions seems to be enable our gcc to handle inline assembler code the way it should and provide enough registers. This seems related to the -O(”oh”) switch, optimized code and “-fomit-frame-pointer“.

Look at what the extras/contrib’s make passes to ffmpeg, it should be quite a long ./configure string. Copy it to clipboard, traverse manually into extra/contrib/src/ffmpeg and do: plus “-O3″ in the extra-cflags category. On my system it looked like:

./configure –prefix=//vlc/extras/contrib –extra-cflags=”-O3 -I//vlc/extras/contrib/include -DNDEBUG -isystem //vlc/extras/contrib/include -fomit-frame-pointer” –extra-ldflags=”-L//vlc/extras/contrib/lib “ –enable-libmp3lame –enable-libgsm –enable-pic –disable-debug –enable-gpl –enable-postproc –disable-yasm –disable-ffserver –disable-ffmpeg –disable-ffplay –disable-devices –disable-protocols –disable-network –disable-shared –enable-static && make && make install-libs install-headers

The next error I got was: “libavcodec/x86/h264dsp_mmx.c: In function ‘put_h264_qpel4_h_lowpass_3dnow’ … error: can’t find a register in class ‘GENERAL_REGS’ while reloading ‘asm’” when it got to ffmpeg. As it seems this much encountered error has to do with the Debian Etch version of gcc and the type of badly written code some gcc versions allow and some don’t. Subsequently leading to a lack of registers and other hickups, especially in combination with some code passages related to CPU optimizations, mmx and inline assembler code.

The solutions are manyfold. For some users, raising the number of available registers with the -O (”O as in Oscar”) flag solves it, e.g. -O. Others disable frame-pointer with -fomit-frame-pointer to free up another register. Others take PIC out of the mix by omiting the -DPIC -fPIC combo in their configure string. Omitting the flag –enable-pic does the same trick.
A clean solution for me was to disable the mmx optimizations, as my compile hung there, by adding the –disable-mmx flag (via) - although I do not know if this considerably slows down my resulting vlc further down the lane.

Here are the steps:

Look at what the extras/contrib’s make passes to ffmpeg, it should be a quite long ./configure string. Copy it to clipboard
~~traverse manually into extras/contrib/src/ffmpeg and do: plus “–disable-mmx”.~~
Go to extras/contrib/src and open Makefile. Look for the ffmpeg section, and the options Makefile passes to the ffmpeg compile. Should be around line 1040. And let it look like:
FFMPEGCONF += \
–disable-mmx \
–disable-debug \
–enable-gpl \
–enable-postproc \
–disable-ffserver \
–disable-ffmpeg \
–disable-ffplay \
–disable-devices \
–disable-protocols \
–disable-network
After that ffmpeg fully compiles. So cd up again to extras/contrib and continue with make.

The libtool dirac version mismatch error

After fixing the ffmpeg issue, a few minutes later compilation stopped again with:

libtool: Version mismatch error. This is libtool 2.2.6, but the
libtool: definition of this LT_INIT comes from an older release.
libtool: You should recreate aclocal.m4 with macros from libtool 2.2.6
libtool: and run autoconf again.
make[3]: *** [libdirac_byteio_la-accessunit_byteio.lo] Error 63
make[3]: Leaving directory `//vlc/extras/contrib/src/dirac/libdirac_byteio’
make[2]: *** [install-recursive] Error 1
make[2]: Leaving directory `//vlc/extras/contrib/src/dirac’
make[1]: *** [.dirac] Error 2
make[1]: Leaving directory `//vlc/extras/contrib/src’
make: *** [using-src] Error 2

A quick visit to a search engine brought me to this thread. So:

cd into extras/contrib/src/dirac
./bootstrap which worked its magic on aclocal.m4
cd ../..
make

The pangocairo interruption

The next stop was:
checking for PANGOCAIRO… configure: error: Package requirements (pangocairo >= 0.16) were not met:
No package ‘pangocairo’ found
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.
Alternatively, you may set the environment variables PANGOCAIRO_CFLAGS
and PANGOCAIRO_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
make[1]: *** [.tiger] Error 1
make[1]: Leaving directory `/root/vlc/extras/contrib/src’
make: *** [using-src] Error 2

This is a problem with the newly included libtiger and libkate libs and their dependencies.

~~Open extras/contrib/src/packages.mak in your editor of choice. Go to thes lines for libkate and libtiger and alter the version variables to require more recent ones, like this:~~

…
KATE_VERSION=0.3.6
KATE_URL=http://libkate.googlecode.com/files/libkate-$(KATE_VERSION).tar.gz
TIGER_VERSION=0.3.3
…
Then remove the libkate and libtiger build directories from src:
in extras/contrib/src do rm -r libtiger libkate
cd up into extras/contrib and do make

Install the pango + cairo combo:

First cairo: apt-get install libcairo2 libcairo2-dev
Then pango, which requires glib, most recent on Debian Etch is: apt-get install libglib2.0-dev
then on to pango. One newer release that works with this glib is 1.10, so:
- ~~wget http://ftp.gnome.org/pub/GNOME/sources/pango/1.10/pango-1.10.1.tar.gz~~
- ~~tar xvf pango-1.10.1.tar.gz~~
- ~~cd pango-1.10.1.tar.gz~~
- ~~./configure && make && make install~~
Then pango: apt-get install libpango1.0-0 libpango1.0-dev
and: no-success!! (Tell me what’s wrong!)

Finally giving up on this, I disabled libkate + libtiger from the contribs system by editing extras/contrib/distro.mak and commenting out the line with .kate .tiger. (Alternatively, if you haven’t had run ./bootstrap in the contribs before, edit extras/contrib/src/Distributions/.mak.) Then running make again, the contribs make exited with ‘nothing todo’. And so -

The contribs system was complete!

In order to finish the vlc build, do:

cd up into the vlc root directory, e.g. with cd ../..
run ./bootstrap which sets the extras/contrib dir as one of the sources for compilation, etc.and spits out files for configure
run configure, sadly with the –disable-tiger/kate flags: ./configure –disable-tiger –disable-kate

Final glitches

Still, some issues arose. With libiconv I got:

/usr/bin/ld: warning: libiconv.so.2, needed by /usr/local/lib/libvlccore.so.4, not found (try using -rpath or -rpath-link)
/usr/local/lib/libvlccore.so.4: undefined reference to `libiconv_open’
/usr/local/lib/libvlccore.so.4: undefined reference to `libiconv_close’
/usr/local/lib/libvlccore.so.4: undefined reference to `libiconv’

although the contribs system properly supplied it. I found it in extras/contrib/lib and thus added the flag –with-libiconv-prefix=//vlc/extras/contrib/lib to my ./configure command. Sometimes it is a bit of a collision between the system’s libiconv and the one in the crontribs. One guy suggested moving away libiconv from contribs - may help someone…

Which got me a bit further until: “text/unicode.c:56:3: error: #error No UTF8 charset conversion implemented on this platform!“. So, additionally adding the –disable-nls switch, which disables native language support (via), solved this one.

Anyway, after a make && make install you are done.

If you’re still stuck, you might want to turn to strategy 2:

Strategy 2: disable unneeded components that break compilation

While running the ./configure && make combination a number of errors will occur if your system - like my Debain 4.0 Etch - is not yet ready for Sid’s “unstable” or Lenny’s “testing” code. A first reflex might be to try to seperately compile these libs, but don’t waste your time - there’s a reason why these aren’t available, for example Qt4, bonjour/avahi, etc. They just don’t compile.

Anyway, on a server system you might not need the glx, skin2, etc. GUI elements of vlc. For example, I just needed the most up-to-date streaming functionalities of vlc and thus could easily neglect the user-interface bells and whistle.

Proceed as follows:

Download one of the newer source-code tars from the ftp archive: ftp index here
Or even get one of the nightly trunk or branch snapshots: ftp index here
Untar with tar xvf
cd into the dir and run ./configure
Systematically note where the configure breaks, then use a search engine to track down which library this error relates to, for example errors regarding avahi point to the bonjour lib. Sometime configure directly tells you what to disable. Or:
Look for the disable switch with ./configure –help | grep ‘’

Then disable this lib by, e.g., ./configure –disable-bonjour and note additonal errors

Try different versions of the vlc source code, work your way slowly up the version tree. For me, some versions compile with, some only without x264 enabled. Most versions will need disabled libpostproc (used for some transcoding?), glx (part of GUI), skins2 (part of GUI), remoteosd (used when using the remote on screen display), swscale (for transcoding involving rescale), fribidi (an open source multi-language text drawing driver, for subtitles), bonjour (device discovery, like portable mp3 players, related to the avahi daemon/client on Linux), libgcrypt (used to do authentication on remote logins via remote OSD), …

With some libs like libfaad etc. already installed on my system and this configure command:

./configure –disable-postproc –disable-glx –disable-skins2 –disable-remoteosd –disable-swscale –disable-fribidi –disable-bonjour –disable-libgcrypt –disable-xcb –disable-v4l2
… I was able tobuild vlc-1.1.0 Yellow Bastard on my Debian Etch / Debian 4.0 system - although it threw a slew of ugly warnings on compile time and later when executed - probably that’s why it’s called Yellow Bastard. A previous 0.9.9a Grishenko build ran much smoother…

Some would argue these no-gui vlcs are cripled, but they are sufficient for a (my) server usage.

Strategy 3: Building an older version (checkout a tag version of vlc from git)

Of course, our aim is to get a newer version than 0.8.9a on Etch, but even small victories might help here, as the Janus release is especially in streaming and transcoding - due to patents (Debian!) - in regards to encoders (mp3,wma) severly “broken”. A cure could be to compile a not-so-recent version. Get it, as shown above, as a git-snapshot from the ftp server or via git directly:

git clone git://git.videolan.org/vlc.git vlc-git

cd vlc-git

git checkout -b ''

In essence, you get the whole repository, cd into it, then revert it to an older version. A bit of explanation here. You can get a list of available tags via “git tag -l” after cd into the repository.

Although vlc relies on git as version control, for reference, here is the post on how to checkout older versions from svn.

Strategy 4: desperately search for readily build vlc packages around the web

Some users did the hard work for you and provide packages of their compile results. For example, this site gives advice on how to get the amd64 package of a 0.9.8a vlc. Anyway, this advice here provided as ‘untested’ since it didn’t help me.

ffmpeg - audio and video length difference issues

tengo — Thu, 29 Oct 2009 12:43:07 +0000

When you use ffmpeg to, for example, extract audio and video as seperate streams, or if you import it into Blender’s fabulous NLE, the Sequence Editor, you might encounter a different length of the audio and video part of the video. Call this audio video offset, audio video mismatch or simply audio video runtime/length difference (this sentence to allow users find this post here…).

Having a look at ffmpeg’s command-line output when -i identifying or converting such a video, you are likely to see something like: “Seems stream 1 codec frame rate differs from container frame rate: 59.99 (11998/200) -> 30.00 (30/1)” This is a good hint at that the video is somehow screwed up on the encoder side. And this is not uncommon, I saw it on .mp4 videos downloaded from YouTube etc. Also, it seems as if some containers are prone to “forgetting” about fps rates of their elements…

A look at ffmpeg’s docs tells us that there is (might be) a cure for this:

`-vsync parameter‘: Video sync method. Video will be stretched/squeezed to match the timestamps, it is done by duplicating and dropping frames. With -map you can select from which stream the timestamps should be taken. You can leave either video or audio unchanged and sync the remaining stream(s) to the unchanged one.
`-async samples_per_second‘: Audio sync method. “Stretches/squeezes” the audio stream to match the timestamps, the parameter is the maximum samples per second by which the audio is changed. -async 1 is a special case where only the start of the audio stream is corrected without any later correction

A scenario where these options are useless is, let’s say, where you extract the audio in one step, and the video in another, so ffmpeg can’t adjust the two against each other. In this case, I found out that relying on the extracted audio length is a good solution as it appears that ffmpeg seldomly (at least it never happened to me) speeds up/ slows down audio - it’s always played at the right rate, whereas the video often receives a speedup/slowdown. So you need to find a way to adjust the video’s length to the audio-length-reference. In my case I did this by adding a speed control effect in Blender’s SequenceEditor and setting the lenth of video to be == the length of audio.

A look around
(other users wrangling with this problem and their ideas/solutions):

‎Update
Today I needed to finally solve this issue and entered another round in tackling this problem. Doing video editing in Blender, I just could not get rid of these audio-video offsets and strips differed in length. A fresh search around the web brought me on another track: fps (framerate/frames per second)!

Some threads (here and here) described exactly what I was experiencing. So I took mplayer to tell me the specs of a few videos, videos where I ran into the problem and videos which Blender imported okay. And voila: all videos that resulted in out of sync audio/video had NTSC framerates of29.xxx. As my Blender project had a fps base of 25fps this naturally resulted in offset video. Audio never differs in length as the audio track doesn’t know of framerates.

So solutions are:

Setting playback to 30fps and a timebase of 1.001 to get the NTSC of 29.97fps.
Or outside of Blender, change the fps rate with, for example ffmpeg. A suitable command would be: ffmpeg -i -r 25 -sameq . Sadly, you can’t use the -acodec copy/-vcodec copy switches if you are changing fps, you need to re-encode…

FFserver Guide

tengo — Wed, 07 Oct 2009 16:10:35 +0000

This post is meant as a FAQ, an introductory guide and a general reference manual for streaming video with ffmpeg’s FFserver. It’s is mainly the stuff I found out while setting up ffserver myself, and fills in some bits the official docs leave out.

Be sure to use a very up-to-date version (and proper combination) of ffmpeg and ffserver. Refer to Compiling a recent ffmpeg for that. For Win32 / Win64 users: Yes, you can compile ffserver on Windows (using mingw) and No, I’ve got no precompiled binaries for you.

One note before you dive into it: I think FFserver is not really usable yet. Although spending hours getting it to work, my streams broke, stuttered or didn’t work at all. Until the developers get it right, look into vlc, which is similarly hard to get right but at least can produce usable streams. Comment on this post if your experience with or opinion towards FFserver differs.

Understanding the basic FFserver concept

The idea behind ffserver is that it acts as a deflector/repeater for stream data that it gets fed by an instance of ffmpeg. The feeder (ffmpeg) and the deflector/server (ffserver) can run on the same machine, or on two connected machines, based on your setup.

ffmpeg[ reads input video file -> transcodes -> sends to IP:Port ] —-> ffserver[ running on IP:Port -> receives data -> serves clients with received data]

ffmpeg, although seldomly used in classic transcoding scenarios, is able to transcode to multiple formats at once. This way, ffserver gets enabled to stream one source - transcoded into multiple formats by ffmpeg -as these multiple formats to clients.

ffmpeg and ffserver communicate via a “pseudo” file, in most cases named feed1.ffm, the default in the example config file. On connect, ffmpeg reads the parameters ffserver requests from ffserver’s feed1.ffm file, starts transcoding, and then relays resulting data to ffserver.

Configuring and starting ffserver itself

Q: How does it work?
A: The official documentation can be found here: ffserver Documentation.

Q: Where can I get a sample ffserver config file ffserver.conf?
A: At the ffmpeg.org official website: here. But as it seems, not all Streams sections really work. See mytests below.

Q: I get an okay stream by connecting with mplayer. A few seconds or minutes into the stream, it breaks and stops.
A: Check if your ffserver logs an “invalid stream index” error. I think this means that the input video is corrupt at this point. ffserver passes it througzh and makes mplayer hickup and drop the stream. A curing hack would be to try a different video input file and checking if it runs through.

Q: I get “Unknown AudioCodec: mp3″ error when starting ffserver.
A: You’ve got an mp3 stream configured in your ffservcer.conf but your ffmpeg/ffserver version has changed names for mp3. Try setting AudioCodec to “libmp3lame”.

Q: What is Stream 0 or Stream 1?
A: Streams are counted up beginning at 0 in your config file, mostly Stream 0 is the Audio stream part embedded in a Stream and Stream 1 the video portion.

Q: I get “Codecs do not match for stream 0″. Or stream 1, etc.
A: This means ahh.. somehow ffserver dislikes your chosen bitrate. See it in the source. And: ffserver seems to be a bit inconsistent in parsing options from ffserver.conf, also try a ffserver restart or rearranging the order of Audio and Video declarations in your section.

Q: I get “Codec bitrates do not match for stream 0″. Or stream 1, etc.
A: This means ahh.. somehow ffserver dislikes your chosen bitrate. See it in the source. And: ffserver seems to be a bit inconsistent in parsing options from ffserver.conf, also try a ffserver restart or rearranging the order of Audio and Video declarations in your section.

Q: What formats/codecs/containers can ffserver handle/stream?
A: Should be all formats ffmpeg can Encode to: see > ffmpeg -formats for that

Q: May I indent stuff in the ffserver.conf config file for better readability?
A: Seems so. I’ve done it and it works.

Q: Is streaming from a file still broken?
A: As it seems it is still broken. One of the developers admits it is at least not working very well. So far, I haven’t got it working with a revision 20182 ffserver. Anyway, up-to-date SVN versions seem to break/fix it from time to time. Hints indicate that the 0.5 ffmpeg release is stable in that regard.

Q: What’s the official ffserver name/spelling?
A: Seems to be FFserver.

FFmpeg issues when feeding to FFserver

Q: I get the error: http://localhost:8090/feed1.ffm: I/O error occurred. Usually that means that input file is truncated and/or corrupted.
A: This error in most cases means that ffmpegis completely unable to connect to the feed1.ffm file in the first place. 1. Check if ffserver is really running. 2. Check if you typed the correct IP:Port. 3. Check if you can connect with your browser to http://localhost:8090/stat.html (if you set up this special stream).

Q: I get the error message: http://localhost:8090/feed1.ffm: Unknown format.
A: Usually this means your ffserver.conf file has contradicting settings for “MaxBandWidth” and stream bandwidth: for example: input.avi (=1872kbps) > MaxBandWidth (=400kbps). via Try setting MaxBandWidth to a value that has enough headroom, like 10000 and test again.

Q: ffmpeg does not appear to be really doing something, and connecting to the resulting ffserver stream either caches a few % then breaks or doesn’t connect at all.
A: Check if your ffmpeg/transcoding machine is fast enough to handle the transcoding, try lowering the input video’s quality, reduce the transcoding work by leveling source format and destination stream format or migrate to a bigger CPU.

Q: I get “rc buffer underflow” error when ffmpeg starts encoding and feeding to ffserver.
A: If we can believe this post from 2005, this is no error and occurs on the beginning of encoding. Also might be related to this, also from 2005. My experience: ignore it.

Whishlist (things I think are missing in FFserver)

Even out hickups in incoming feeds/streams (which currently lead to clients loosing connection)
HTTP-Live-Streaming, see RFC (currently only Apple’s and Wowza’s server have an implmenetation)
Ogg-Video Streaming (.ogg audio streaming is supported, is .ogv?) Icecast does it, halfway…

Working sections

While trying to get things working, I came across a number of Stream-Settings which worked for my revision20182 ffserver. Some reside here for quick copy&past, in case you want to test them.

# works
#
#    Feed feed1.ffm
#    Format swf
#    VideoCodec flv
#    VideoFrameRate 15
#    VideoBufferSize 80000
#    VideoBitRate 100
#    VideoQMin 1
#    VideoQMax 5
#    VideoSize 352×288
#    PreRoll 0
#    Noaudio
#

# works, asf seems to be the most stable
#
#    Feed feed1.ffm
# Format asf
#    VideoFrameRate 15
#    VideoSize 720×480
#    VideoBitRate 256
#    VideoBufferSize 40
#    VideoGopSize 30
#    AudioBitRate 64
#    StartSendOnKey
#

# works
#
#    Feed feed1.ffm
#    Format asf
#    VideoFrameRate 15
#    VideoSize 720×480
#    VideoBitRate 728
#    VideoBufferSize 40
#    VideoGopSize 30
#    AudioBitRate 64
#    StartSendOnKey
#

# works
#
#    Feed feed1.ffm
#    Format mp2
#    AudioCodec libmp3lame
#    AudioBitRate 128
#    AudioChannels 1
#    AudioSampleRate 44100
#    NoVideo
#

# Real with audio and video at 64 kbits (from example.conf)
# works in mplayer and real-player
#
#    Feed feed1.ffm
#    Format rm
#    AudioBitRate 32
#    VideoBitRate 128
#    VideoFrameRate 25
#    VideoGopSize 25
#

# works in mplayer and real-player
#
#    Feed feed1.ffm
#    Format rm
#    VideoBitRate 128
#    VideoFrameRate 25
#    VideoGopSize 25
#    AudioBitRate 64
#

# Multipart JPEG (from example conf)
# works, but speeds your video up, setting FrameRate to 25 for example fixes this
#
#    Feed feed1.ffm
#    Format mpjpeg
#    VideoFrameRate 2
#    VideoIntraOnly
#    NoAudio
#    Strict -1
#

Compiling a recent ffmpeg

tengo — Tue, 06 Oct 2009 23:33:02 +0000

Before you start, it might be useful to again read the post How do I compile stuff.

Okay, here is how you build a recent binary of ffmpeg from source. Please be aware that configure switches and library neames tend to change over time and as this post ages might need adjustment.

We will need a very recent yasm (a rewrite of nasm, etc.), an assembler:
Download yasm from the official yasm homepage:

> wget http://www.tortall.net/projects/yasm/releases/yasm-0.8.0.tar.gz
> tar xvf yasm-0.8.0.tar.gz
> cd yasm-0.8.0
> ./configure
> make
> make install

cd up again and now we are going to build a recent x264:

> aptitude install git-core
> git clone git://git.videolan.org/x264.git
> cd x264
> ./configure –enable-pthread –enable-shared –enable-mp4-output –enable-pic
> make
> make install

On my version of Debian, I got an error on the make step: “/usr/bin/ld: cannot find -lgpac_static”. A thread discusion this issue can be found here. As I was unable to get libgpac-dev installed easily (lots of broken packages, unsatisfied dependencies, etc. - not sure if my system is screwed or x264 is too new for my Debian…) I had to find a different solution - which was to omit “–enable-pthread” on configure. After that make && make install worked ok.

After that, back to ffmpeg. ffmpeg has quite a few dependencies, so its good to start with some basic libs, everything else configure will tell you (you can search for the missing -dev package with apt-cache search lib):

> aptitude install libswscale-dev libavcodec-dev libavformat-dev libvorbis-dev libtheora-dev liblame-dev liblame0 libgsm1-dev libgsm-tools libgsm1 libfaad2-dev libfaac-dev libgpac-dev libxvidcore4-dev liba52-0.7.4 liba52-0.7.4-dev libdc1394-13-dev
> svn export svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg-trunk
Note 1: If I remember correctly, I checked out version/revision 20182.
Note 2: The now official svn url is: svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg-trunk
> cd ffmpeg-trunk
> ./configure –enable-gpl –enable-shared –enable-pthreads –enable-libdc1394 –enable-libfaac –enable-libfaad –enable-libgsm –enable-libmp3lame –enable-libtheora –enable-libvorbis –enable-libx264 –disable-debug –enable-nonfree
Think about adding: –enable-libxvid –enable-shared (the latter installs the libav* libs in /usr/local/lib
> make

On newer versions of ffmpeg “–enable-swscale” isn’t needed anymore, its permanently on. We need to set the the nonfree flag, as libfaac breaks the “free”. And as it seems, the omitted pthread flag on x264 has no relevance or at least doesn’t break the build of ffmpeg. (Comments welcome!).

Now: wait a minute!
As you may have noticed, we did not make install. The build binary is still residing in our ffmpeg-trunk dir, which is quite convenient, as we have a solid, older ffmpeg in /usr/bin and a turbocharged new version here. But there are some caveats - it doesn’t run out of the box. First we need to symlink some libs into place (otherwise you would get an error on loading shared libraries libavdevice.so.52, for example):
(Be sure to symlink the libs your build actually requests and not blindly copy/paste the lines below, versions may vary!)

> ln -s libswscale/libswscale.so.0 ./
> ln -s libavdevice/libavdevice.so.52 ./
> ln -s libavformat/libavformat.so.52 ./
> ln -s libavcodec/libavcodec.so.52 ./
> ln -s libavutil/libavutil.so.50 ./

And you need to add another library search path to bash, for example by calling our new ffmpeg like this:

> LD_LIBRARY_PATH=”/path/to/ffmpeg-trunk” /path/to/ffmpeg-trunk/ffmpeg

or by permanently export‘ing this LD_LIB_PATH with:

> export LD_LIBRARY_PATH=”/path/to/ffmpeg-trunk”

A third option, preferred by some users, is to configure with –prefix=/opt/ffmpeg. This leads to, after make install, ffmpeg ending up under /opt in /opt/ffmpeg and binaries at /opt/ffmpeg/bin which conveneintly finds libraries at /opt/ffmpeg/lib. A bit more elegant that tweaking with ldconfig or the above symlinks/export LD_LIBRARY_PATH.

A nice guide to get a all-feature ffmpeg is here.

Other helpful ffmpeg compiling HOW-TOs are (both slightly out of date):

TagFS, tracking progress in the field of semantic file systems

tengo — Sun, 04 Oct 2009 12:38:17 +0000

Having an eye on recent developments in the field of storage, data archival and file systems, I think one can say that one of the more interesting approaches towards saving data on disk is the idea to flatten folder hierarchies and in turn introduce a filesystem that is based on tags. Each file can be “saved” under different “categories” of metadata descriptions, so you never have to search for files in deep directory hierarchies again.

Why using directories to organize files and data is (or can be) a bad idea is nice illustrated on John’s blog code@work here. The basic idea being a number of files belonging to different tags:

My Test Doc, belonging to Document John’s Report, belonging to Document, Report
Yearly Report, belonging to Document, Report
My Recruiting Report, belonging to HR, Document, Report
Victor’s Resume, belonging to HR, Document, Resume
HR Evaluation, belonging to HR, Document, Report

Now each of these tags, like “Document”, in turn is its own directory, entering it will list you all the files belonging to the “Document” group. The concept further clarifies when reading questor’s tage on it. He also provied a Dokan based prototype of an implementation there. Other sources discuss the topic, one being an Ubuntu idea, but still, as far as I know, there are few projects beyond a proof-of-concept or early dev stage. Another thread discussing the factual implementation points out that “database oriented filesystems are the future”.

Undelete, unerase - file recovery on Linux

tengo — Fri, 29 May 2009 14:19:14 +0000

Recently, while using an USB memory stick formatted in FAT32, I accidentally deleted a few files. My first idea while sitting on a Windows machine was to seek for a simple utility from the DOS world to get the files back. I thought it would be a no-brainer, but after wading through tons of Google spam results, nasty Shareware and fake apps I was near giving up. The final stage was to try an older Version of Norton Utilites’ Unerase, but as I soon found out, it wasn’t able to “see” my USB device…

The next idea was to take the stick to the Unix/Linux/ Ubuntu world, as the free and open source movement produced quite a few very professional, versatile and useful tools in the domain of data handling and storage device management. (If you don’t believe me: software raid, ntfs drivers, the SystemRescueCD, … the list goes on).

So I fired up apt-get and browsed around:

recoverjpeg
foremost

Were the first apps I tried. recoverjpeg was useless for me, as I was looking for a zip file (it returned zero results!).

Next was the mighty foremost, so forensic and versatile that it didn’t work for me at all (note the cynicism). The other obstacle I soon discovered was that I was trying to recover files from a USB stick, which foremost and many other tools do not read in easily. Or at least these apps were no help in discovering where my USB-stick had been mounted by the system on a lower level (/dev/sdd by the way). /media/USBSTICK, the convenience path supplied by Ubuntu, as path on the command-line to foremost got rejected. And then this annoying “output directoryis not empty” error, each time I fired foremost without success it created empty folders in my output dir…

Further, on other blogs, people kept telling that I first need to copy the USB stick raw and then inspect it:

sudo dd if=/dev/sdb of=sdcard.img bs=1M

sudo - used to perform command as root user (i.e. have correct privileges)

dd - command to copy data a byte level

if=/dev/sdb - read data from /dev/sdb - you may need to change this depending on where your memory card it located.

of=sdcard.img - save this to the sdcard.img file in the current directory

bs=1M - Copy it in 1M chunks

Just too lazy to do that. My hope sank and I turned to some more apps. On some thread a guy recommended “fatback“, which I downloaded but could not get compiled.

Finally I found photrorec a helpful and working little app that saved the day for me! Initially developed for photo recovery from removable media (hence the name) the tool finds all types of files. Fire it up and it conveniently displays all mounted devices (no need to know about the structure of the /dev path…). Select the USB device here. The next screen asks if you know about the type of filesystem - don’t select “Windows/PC” or similar here, as it will limit your results (I don’t know why)! Select the raw/unformatted option (can’t quite remember the exact name, it’s the default. Proceed through the next screens and photorec will spit out all it can undelete from this device, although the names will be lost. And in contrast to foremost, photorec is smart enough to create ascending subdirs in your supplied output path, so you can run it multiple times without changing the output over and over. And it is not CLI only, it has a simple shell GUI!

Thumbs up for photorec!

DVD drive won’t play DVDs properly when no region is set

tengo — Wed, 29 Apr 2009 17:09:54 +0000

Recently, I installed a dual-boot OS on a brand-new laptop, absolutely fresh system, no OS, no formatted disc. I started with Microsoft XP and after the basic install was complete, I added vlc-0.9.9 to test DVD playback. Surprise: I did not work. Although vlc should be able to playback a DVD without any further decryption drivers, DVD software or costly add-ons, menus were a garbled mess of half-way decompressed video. The DVD was an original, nothing esoteric as a burned DVD or the like. Next I tried a few other DVDs with mixed results, some played a bit of the menu video and sound, some rejected playback altogether.

My first idea was a broken drive - but on a mint new PC? After reading a few forum posts on the net I actually thought about cleaning the lens, but thank god I didn’t (you will never get a lens clean as it was after manufacture-quality-control-unboxing…). Fiddling a bit around with Window’s right-click menu > Drive Properties, I noticed that on the “DVD-Region” tab no region was set.

A bit more research on the net brought me to this enlighting post on the Ubuntu forum, which seconds what I found out that same minute myself. Set a region for the drive and magically all problems are gone! As it seems, my OS-free system had been shipped with an undefined-region DVD drive. After the install, I oversaw this fact and the drive produced errors on the hardware-firmware level, before it reached vlc. The same is true for Linux systems - so the next time your brand new DVD drive produces errors, check the region-setting first!