The library includes support for RTSP, RDT, SDPP, ASMRP, and RMFF, and it is all in pure Python and Pythonically coded where possible (see sdpp.py). It requires Python 2.6 and the Twisted Python library. The library has only one known bug, and I can’t for the life of me find it. Perhaps another set of eyes will effect a solution.

To use the library to simply download an RTSP stream:
python rdt.py -u <URL> -f <FILENAME>

If you like this post, please comment. Commenting is fast, anonymous, and greatly appreciated.
Download the Twisted Python RDT-RTSP Library here

Update After doing some searching, I’ve found a FireFox plugin caled Video DownloadHelper which performs much the same function as the program I’ve written, but is more stable and integrated directly into FireFox. Its only downside is that it doesn’t automatically download the songs as you listen, you need to start downloading the ones you want. But it is more stable than my program and much more user friendly. Be sure to use Mp3tag (listed below) to quickly rename all the files you’ve downloaded.

Download and install the following pre-requisites:

1. Python 2.6.5 – http://www.python.org/ftp/python/2.6.5/python-2.6.5.msi
2. dpkt 1.7 – http://dpkt.googlecode.com/files/dpkt-1.7.win32.exe
3. pypcap 1.1 – pcap-1.1.win32-py2.6.exe

Save this Python program into a folder of its own: groovegrab.py

Run the program using python groovegrab.py

As you listen to songs on Grooveshark, the folder will populate with mp3 files. Each are named Track_X.mp3. But there’s a really neat, easy way to rename them. Download Mp3tag and in the program go to File > Change Directory and choose the directory where the mp3 files are. Select all the mp3 files in the main pane and go to Convert > Tag to Filename.

Enjoy your perfectly named mp3 files from a Grooveshark listening session.

Thanks to dugsong for dpkt and pypcap, and thanks to Andrew Brampton for his tutorial on dpkt/pypcap!

If you liked this post, please comment. Commeting is fast, anonymous, and greatly appreciated.

The simplest method is to just record whatever the computer plays. This can be done using Audacity, or anything which will record the audio the computer is currently playing. I found an application which uses this method to record audio from streaming websites. Streaming Audio Recorder records streaming audio by all audio coming from the computer. This is a very naive approach and suffers from quality degradation if the file was not originally going to be re-encoded, but it has the advantage of working in all cases.

Another approach is to capture the stream urls as they are played. Examples of this are WmRecorder, Jaksta, and Orbit Grab++. The stream is then requested a second time and the program records the response. This method downloads each stream twice, once from the original site, then again using the capture program. This works for many sites, but Grooveshark has a handshake challenge which breaks this method.

A final approach, and perhaps the best, is to capture the stream the first time. Unlike the previous approach, it downloads the stream only once. To accomplish this method on Grooveshark, I used WireShark.

Downloading the mp3 from Grooveshark:

1. In WireShark, go to Capture > Options. Under Capture Filter, enter in port 80 and start the capture.
2. Begin listening to the song on Grooveshark and many new lines should begin scrolling in WireShark. At the top of WireShark is a Filter: box. Enter in http.content_type == "audio/mpeg", press enter, and only one line should appear (more lines if more than one song was played).
3. Click the line once and if it comes up processing, hit Stop. Then in the middle pane click the Media Type line (again, Stop if necessary). Right-click the Media Type line and choose Export Selected Packet Bytes… Then save the file somewhere as an mp3 file, choosing All Files as the filetype.

The nice thing about this method is that no extra packets are sent to Grooveshark, so the method is undetectable.

If you liked this post, please comment. Commenting is fast, anonymous, and greatly appreciated.

If you like this post, please comment. Commenting is fast, anonymous, and greatly appreciated.

To install the script, save it as resize-match-dpi.scm into C:\Users\Odie\.gimp-2.6\scripts\. Open up GIMP and the script will appear under the Script-Fu menu at the top of an image window. If you are really adventurous, I’ve included a small wrapper allowing the script to be called from the console as follows:

"C:\Program Files\GIMP-2.0\bin\gimp-console-2.6" -i -b "(resize_match_dpi \"input.bmp\" \"output.bmp\" 1280 0)" -b "(gimp-quit 0)"

resize-match-dpi.scm:

; Released under the GNU GPL 3.0 or later
(define (resize-match-dpi image drawable newwidth newheight)
  (let* (
	 (oldwidth (car (gimp-image-width image)))
	 (oldheight (car (gimp-image-height image)))
	 (newdpi 0)
        )
    (if (= newheight 0)
        (set! newheight (round (* (/ oldheight oldwidth) newwidth)))
	()
    )
    (if (= newwidth 0)
	(set! newwidth (round (* (/ oldwidth oldheight) newheight)))
	()
    )
    (set! newdpi (/ (* newwidth (car (gimp-image-get-resolution image))) oldwidth))
    (gimp-image-undo-group-start image)
    (gimp-image-scale image newwidth newheight)
    (gimp-image-set-resolution image newdpi newdpi)
    (gimp-image-undo-group-end image)
))
 
(define (resize_match_dpi filename output newwidth newheight)
  (let* ((image    (car (gimp-file-load RUN-NONINTERACTIVE filename filename)))
         (drawable (car (gimp-image-get-active-layer image)))
        )
    (resize-match-dpi image drawable newwidth newheight)
    (gimp-file-save RUN-NONINTERACTIVE image drawable output output)
    (gimp-image-delete image)
))
 
(define (resize_match_dpi_multi pattern outdir newwidth newheight)
  (let* ((filelist (cadr (file-glob pattern 1))))
    (while (not (null? filelist))
	   (let* ((filename (car filelist))
		  (image    (car (gimp-file-load RUN-NONINTERACTIVE
						 filename filename)))
		  (drawable (car (gimp-image-get-active-layer image)))
		  (outf (string-append outdir "/" filename)))
	     (resize-match-dpi image drawable newwidth newheight)
	     (gimp-file-save RUN-NONINTERACTIVE image drawable outf outf)
	     (gimp-image-delete image))
	   (set! filelist (cdr filelist)))))
 
(script-fu-register "resize-match-dpi"
  "<Image>/Script-Fu/Resize Match DPI"
  "Resize and match DPI"
  "odie5533"
  "odie5533"
  "2010-05-16"
  "RGB*"
  SF-IMAGE "Image"  0
  SF-DRAWABLE "Drawable" 0
 
  SF-VALUE  "New width" "1280"
  SF-VALUE  "New height" "0"
)

Example script usage:

$ awk -f tinypic.awk The_Grass_Snake_-_Natrix_natrix.jpg
http://i39.tinypic.com/dlnb6r.jpg

Let me start by saying I have no idea why I wrote this in AWK. I’ve only used AWK once before for a very short one-liner. I find AWK to be better than Perl, but worse than Python. It does make for a light and portable language: the Windows AWK executable (available here) is only 345 KB, combine with Windows cURL (available here) which is 405 KB and you’re still under a meg for both. My bloated Python install is over 100 MB and I just installed Strawberry Perl which is 174 MB.

If you like this post, please comment. Commenting is fast, anonymous, and much appreciated.

Here’s the script. Save as tinypic.awk

BEGIN {
	file = ARGV[1];
	# parse file extension
	split(file, arr, ".");
	file_ext = arr[length(arr)];
 
	#2>&1 works on Win/Linux. Since the output is filtered anyway
	#it shouldn't bother anything
	while (("curl tinypic.com 2>&1" | getline $0) > 0)
	{
	        # parse necessary post variables from the HTML page
		if (match($0, "form action=\"http://s(.)", arr))
			server = arr[1];
		if(match($0, /id="uid" value="([^"]+)/, arr))
			uid = arr[1];
		if (match($0, /name="upk" value="([^"]+)/, arr))
			upk = arr[1];
	}
 
	# compile curl command to upload the file
	curl_cmd = sprintf("curl -L -F UPLOAD_IDENTIFIER=\"%s\" -F upk=\"%s\" -F domain_lang=\"en\" -F action=\"upload\" -F MAX_FILE_SIZE=\"500000000\" -F shareopt=\"true\" -F the_file=\"@%s\" -F file_type=\"image\" -F dimension=\"1600\" \"http://s%s.tinypic.com/upload.php\" -H \"Expect:\" -e \"http://tinypic.com\" 2>&1", uid, upk, file, server);
 
	# upload the picture and parse where the picture URL
	while ((curl_cmd | getline $0) > 0) {
		if (match($0, /name="pic" value="([^"]+)/, arr))
			img = arr[1];
		if (match($0, /name="ival" value="([^"]+)"/, arr))
			ival = arr[1];
	}
	printf("http://i%s.tinypic.com/%s%s", ival, img, "." file_ext);
}

For the sake of brevity, I will assume the Xbox already has XBMC installed and the ability to FTP files to the Xbox and execute xbe files.

Software used:
Xdsl iso, and use 7-zip to extract the iso
SSH/Putty
FileZilla or any FTP client
RealVNC

Part 1: Uploading Xdsl via FTP
Extract the files in the the Xdsl iso. Create an FTP session with the Xbox. Copy the KNOPPIX folder from the extracted iso to the Xbox’s E: drive. Open the KNOPPIX folder on the Xbox and copy minirt24.gz and linux24 to this folder. E:\KNOPPIX should now have three files, one of which is named KNOPPIX, and no further subdirectories. Also copy README-FTP\linuxboot.cfg to E:\linuxboot.cfg. Now make a folder in the E:\Apps directory called simply Linux and copy the default.xbe file to E:\Apps\Linux\default.xbe.

In total, four files are copied and the overall structure is:
E:\linuxboot.cfg
E:\Apps\Linux\default.xbe
E:\KNOPPIX\KNOPPIX
E:\KNOPPIX\linux24
E:\KNOPPIX\minirt24.gz

In XBMC run the Linux default.xbe. The Xromwell blue screen will appear. Select the hard drive and press A. It should boot directly into Xdsl.

Part 2: Installing Xdsl
The D-pad and left joystick move the mouse. Press Y to right-click the desktop and go to System > Daemons > ssh > start. Open putty/ssh and connect to the Xbox with username admin and password xbox. Then run the command xbox-loopbackinstall.

ssh root@192.168.1.101
root@192.168.1.101's password: xbox
xbox-loopbackinstall

Go through the menus and when it asks for the size of the loopback filesystem, change it from 200 to 600. Continue through selecting for the defaults. After it’s finished, reboot the Xbox:

reboot

Part 3: Configuring Xdsl for SSH and VNC
After the Xbox is done rebooting, go back in Xbmc and run Xromwell again and select the hard drive. This time a menu will appear. Choose XDSL Loopback, the second choice. Start the ssh daemon again (right-click the desktop and go to System > Daemons > ssh > start). Connect to the Xbox using Putty/ssh again. Users who are using ssh will need to remove the key from their system:

ssh-keygen -R 192.168.1.101

To set the ssh daemon to start when Xdsl boots, enter the following. If you are using Putty on Windows, copy it to the clipboard then right-click the Putty window to paste.

ln /etc/init.d/ssh /etc/rc5.d/S20ssh -sfn

Now on the Xbox go right-click the desktop and go to MyDSL > MyDSL Browser. In the left panel expand + net and scroll down and select vnc4x-1.4.6.dsl. Click the Install Selected button. Close out of the MyDSL Browser and right-click the desktop and go to MyDsl > vncserver > vnc4x0. Start up the RealVNC client and enter the Xbox’s IP and click connect.

Part 4: Installing BitTorrent
Now, using RealVNC, right-click the desktop XDSL Upgrades > Upgrade to GNU Utils, then XDSL Upgrades > Enable Apt. Right-click desktop, MyDSL > MyDSL Browser. Install net > bittorrent-cli.dsl.

Now in the ssh session enter:

ln -sfn /usr/bin/python2.2 /usr/bin/python

Mount the E: drive, then open a screen session and su to the regular user (dsl):

mount /dev/hda50
mkdir /mnt/hda50/torrents
apt-get install screen
screen
su dsl

BitTorrent can be run using the following command.

$ btdownloadcurses --saveas /mnt/hda50/torrents --minport 52130 --maxport 52140 CompletelyLegalLinuxDistribution.torrent

Note: VNC uses about 70% of the CPU. Only run it when you have to; use SSH.

To start more than one torrent, press [Ctrl]+[A], then [C] and type the command with the next torrent. [Ctrl]+[A],[P] and [Ctrl]+[A],[N] go back and forth between the two buffers.

That’s all for now. You can use SFTP to access the hard drive, SSH, VNC, and BitTorrent. Let me know in the comments if you enjoyed this tutorial. Comments are always appreciated.

Part 5: Serving files using Samba
Per a request in the comments, here’s how to have the Xbox serve files using Samba.

In Xdsl, right-click the desktop and go to MyDSL > MyDSL Browser. In the left pane, click the [+] net, and then find samba.dsl and click it. Click the Install Selected button at the top of the window.

Open up an SSH session to Xdsl. Enter the following commands, but you may need to change WORKGROUP to MSHOME. These commands backup your Samba configuration file, then change the workgroup, and finally add the E, F, and G partitions as shares. Copy the entire contents of the following, then right-click in Putty to paste them.

cp /opt/samba/smb.conf /opt/samba/smb.conf.bak
sed -i -e 's/^\tworkgroup = .*$/\tworkgroup = WORKGROUP/' /opt/samba/smb.conf
sed -i -e 's/;  encrypt passwords/encrypt passwords/' /opt/samba/smb.conf
printf "[E]\n   path = /mnt/hda50\n   public = yes\n   writable = yes\n\n[F]\n   path = /mnt/hda55\n   public = yes\n   writable = yes\n\n[G]\n   path = /mnt/hda56\n   public = yes\n   writable = yes\n\n" >> /opt/samba/smb.conf
printf "mount /dev/hda55\nmount /dev/hda56\nnmbd -D -s /opt/samba/smb.conf\nsmbd -D -s /opt/samba/smb.conf\n" >> /opt/bootlocal.sh

Now you need to mount the other partitions, restart Samba, and set a new Samba password for root. Just hit enter twice when it asks for a password to use a blank password.

mount /dev/hda55
mount /dev/hda56
nmbd -D -s /opt/samba/smb.conf
smbd -D -s /opt/samba/smb.conf
sudo smbpasswd -a root

Download the latest dump of Simple English Wikipedia here
Download Wiki Parser v0.1 here

Download microdc2 v0.15.6 for Windows/Cygwin here

microdc2

Here’s a boilerplate to quickly recreate packets using Twisted. It requests and saves a copy of Google.com. Simply replace with your favorite URL, or un-comment the two lines and comment the other getPage to send a POST request.

from twisted.web import client
from twisted.internet import reactor
 
agent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729)"
headers = {'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language':'en-us,en;q=0.5', 'Accept-Charset':'ISO-8859-1,utf-8;q=0.7'}
headers_post = headers.copy()
headers_post['Content-Type'] = 'application/x-www-form-urlencoded'
 
def success(data):
    print "Success!"
    f = open('save.html','w')
    f.write(data)
    f.close()
    reactor.stop()
 
def failure(fail):
    print fail.getErrorMessage()
    reactor.stop()
 
def request():
    url = "http://google.com/"
    d = client.getPage(url, headers=headers, agent=agent)
    #post_data = ""
    #d = client.getPage(url, postdata=post_data, method="POST", headers=headers_post, agent=agent)
    d.addCallback(success).addErrback(failure)
 
request()
reactor.run()

To make it continually request the same page, change the success method to this:

def success(data):
    print "Success!"
    f = open('save.html','w')
    f.write(data)
    f.close()
    reactor.callLater(0.5, request)

Hello,

Unfortunately but your data is damaged un-fixable

Thank you,
www.000webhost.com

Data damaged un-fixable! But I’m smart; I make backups. I found the file in my backup folder, extracted the SQL, then opened it to verify everything was fine. I only make backups every few months, but I had made one this month on the July 7th. When I do backups I backup all my websites and carefully label them and file them away on one of my hard drives, then I burn a copy of my entire backup folder to a DVD, just in case my backup hard drive dies. Sadly, this past month when I was making backups, I mislabeled one. Wouldn’t you know it had to be the one with data damaged un-fixable. The SQL file belonged to a different site, and my blog’s data really is gone.

Google cache has some of the old posts and I’ll work on moving the ones that matter which are really only two posts. This backup is from November 2008. Not much lost since I haven’t been active lately, but data loss is never fun.