Since then, I’ve applied to positions for an internship at school and finished my semester. Went through a bunch of interviews and finally accepted a position at ATI (i.e. AMD) which is awesome! I’m not really sure yet if I’m allowed to blog about the things I’ll be working on there but I’ll find out. Would be cool to keep track of the progress I make while there for the next 16 months. Along with that, I’ve bought myself a car to get to and from work – FREEDOM!

Elsewhere in my tech-space, I’ve begun familiarizing myself with the ASP.NET framework (in C#) and MsSQL as the project I’m working on at AMD seems to require such. I actually ended up coding a test project for an interview for another company using the framework/database and posted the project (more info below). It’s my first dive into ASP and far from the proper way of doing things, but I’ve already learned quite a bit since then so I think I’m on a good course.

I also won 2nd place in the International Student Design Competition (iSDC 2010) for my group project in CSC318! The conference is being held in Germany and our projects will be presented there soon Unfortunately, no one in our group will be able to attend but we’re going to try to get our poster printed out for display there. We’ll probably be listed on the iSDC website too once that’s done. Look for the Grocery Garden project by The Crew

Finally, I also got myself hardeepsingh.ca because well, why not? It just points to the same host as theootz.com though.

Back to list

Share

No related posts.

What You Need

You’ll need PHP, bash, CRON access and python to get this working. And given the bash/cron requirements, that also means it assumes you’re running a *nix based server. It’s probably possible to get the requirements down, but I just wanted to get something done quick and it’s what I came up with.

The Files

backup.sh

First up is the bash script (based on this one) that does most of the work. It will tar+gzip all your site files, export your database(s), and remove old backup files (+5 days old) when necessary. You should only really have to edit the top part of the script. Make sure you remember what you set your BACKUP_DIR and FILES_DIR to – you’ll need them in other scripts later on.

#!/bin/sh

# EDIT THIS PART ------------------
# where all these backup scripts are stored
BACKUP_DIR=/home/theootz/backups

# where you want to keep all the actual backup folders
FILES_DIR=/home/theootz/backups/files

# where all your html files reside
HTML_DIR=/home/theootz/public_html

# your MySQL database name, username and password
SQL_USER1=myusername1
SQL_PASS1=mypass1
SQL_DB1=mydb1

# a second database
# Uncomment these lines if you want to back up a second database.
# Also uncomment the line specified below!
# You can also add more databases and follow the same format.
#SQL_USER2=myusername2
#SQL_PASS2=mypass2
#SQL_DB2=mydb2
# STOP EDITING ------------------

THEDATE=`date +%Y-%m-%d_%H-%M-%S`

mkdir -p $BACKUP_DIR/files

mysqldump -u${SQL_USER1} -p${SQL_PASS1} ${SQL_DB1} > ${FILES_DIR}/${THEDATE}_db1backup.sql

# Uncomment the line below if have a second database to back up
#mysqldump -u${SQL_USER2} -p${SQL_PASS2} ${SQL_DB2} > ${FILES_DIR}/${THEDATE}_db2backup.sql

tar -cf ${FILES_DIR}/${THEDATE}_site.tar ${HTML_DIR}
gzip ${FILES_DIR}/${THEDATE}_site.tar

find ${FILES_DIR}/*site* -mtime +5 -exec rm {} \;
find ${FILES_DIR}/*db* -mtime +5 -exec rm {} \;

backup_upload.py

Next is the Python script to upload the files. It uses a PHP library that I’ll give more information on below.

import sys
import os

########## Make sure these values match the ones you set in your shell script!
BACKUP_DIR = "/home/theootz/backups"
FILES_DIR = "/home/theootz/backups/files"

# this needs to be changed to reflect your settings on your s3 account
S3_BUCKET = "theootz"
S3_FOLDER = "mysite/backups"
##########

j = os.path.join
fname = os.path.basename(sys.argv[1])
cmdstr = j(BACKUP_DIR, "s3.php") + " put " + \
    j(FILES_DIR, fname) + " " + \
    j(S3_BUCKET, j(S3_FOLDER, fname.split("_")[-1]))

os.system(cmdstr)

PHP S3

The python script uses a PHP library to do all the S3 uploading work. The original is available here.

I had to edit a few paths in the php library to get it to work, so if you want to download my version it’s available here:

Regardless of which one you pick, make sure you open and edit the s3.php file! You’ll need to put your S3 keys in there (near the top of the file). It will look something like this:

// S3_CONF file
$aws_key = null;
$aws_secret = null;

You’ll want to put your access and secret keys there as strings, as follows:

// S3_CONF file
$aws_key = "my access key here";
$aws_secret = "my secret key here";

How To Get It All Working?

Ok, so you’ve got the files. Now upload them to your web server to your BACKUP_DIR folder (whatever you choose it as, I picked /home/theootz/backups). Finally, take the files from the the PHP S3 library and put them in your BACKUP_DIR as well. This should leave you with the following files in a single folder:

• backup.sh
• backup_upload.py
• s3sync.php
• s3.php
• libs3.php
• libaws.php

I’m pretty sure most of the files from the S3 library aren’t needed but once again not something I bothered to check.

Now you’ll want to enable execution for all our scripts. You can do this by running the following commands while in your BACKUP_DIR folder:

chmod u+x backup.sh
chmod u+x backup_upload.py
chmod u+x s3.php

At this point, you can go ahead and run backup.sh and see if everything worked. If it did, then we can set up the cron jobs to have this all work automatically for us

You’ll want to add two entries to cron. The first entry will run our backup.sh on a daily basis for us. This will give us 5 days worth of backups at any one point. The second script will upload our latest backup to the S3 service, replacing whatever is already there.

1       1       *       *       *       /path/to/BACKUP_DIR/backup.sh
1       1       *       *       *       find /path/to/FILES_DIR/* -mtime 0 -exec python /path/to/BACKUP_DIR/backup_upload.py '{}' \;

Make sure you edit the paths to reflect the way you have it configured for your system! For example, my cron jobs look like this:

0       0       *       *       *       /home/theootz/backups/backup.sh
0       0       *       *       0       find /home/theootz/backups/files/* -mtime 0 -exec python /home/theootz/backups/backup_upload.py '{}' \;

Done!

And that’s it Your website along with databases and other content will now be backed up daily to the server itself, and uploaded to your S3 account weekly. If you wanna be super extra paranoid you can download the backups from the FILES_DIR yourself or use rsync or something similar to automate that as well. One idea is perhaps sending it to your e-mail – use that g-mail space for something right?

Hope that’s some help to someone

Share

No related posts.

With the release of the Windows Phone 7 Series (I really really hate that name so I’m gonna just use winmo7) SDK, I decided to give it a shot and see how hard (or even easy?) creating an application on the platform would be.

I should probably mention I have very little experience coding in SilverLight and only dabbled with it a while ago, and I’ve been coding in C# for a good half year or so now but once again not quite that much experience in the language. On top of that, most C# applications I’ve written use Windows Forms and not WPF. I’ve also decided that my first application will be a simple browser for Amazon S3 (considering how much I’ve been fiddling with it recently), of whose API I have zero experience with as well. So from what I can tell, I have quite a learning curve ahead of me

I’m not going to go over any problems I have over the IDE, APIs, language, etc… but I’m going to try to focus on the experience of creating an actual application for the platform.

Initial Setup

The download and install of the SDK was relatively painless. It did take quite a while though because I had the beta of Visual Studio 2010 still installed and the SDK requires either the RC or it will install Visual Studio 2010 Express on its own. I didn’t really feel like downloading and installing Visual Studio 2010 again, so I had to uninstall VS2010 along with the .Net 4 beta. After which, the SDK installer downloaded and installed the RC for me. It was kind of annoying how it didn’t really tell me why the beta simply HAD to be uninstalled and I had to search through the depths of their forums to figure it out, but I guess most shouldn’t have this problem.

One thing to remember if you’re using the Express version of VS2010 though, I don’t think it’s possible to have any VCS integration so you’ll have to do it outside the IDE.

My First App

Opening up VS2010, I’m presented with a few nice big buttons on pre-set winmo7 specific projects. You can create a Windows Phone App, a Windows Phone List App (not sure what this is exactly but it’s somehow specific to “list and navigation controls”) and a Windows Phone Library.

I figured I want an application and I’ll code in a simple library for working with S3 inside that.

So I went ahead and choose the first option and it creates a nice looking UI preview for me with the phone chrome and all so I can see exactly how it will look. Not bad. Running the program launches the emulator (which takes quite a while :/) and launches my app which currently does nothing. Good enough of a Hello World for me!

Also of interest, the emulator need only be loaded ONCE. It can then be used for multiple debug/run sessions, very nice considering how long it took to load up!

Designing the S3 Browser

So at this point I need to figure out how I want the program to look and behave. I’ve decided to use this desktop S3 Browser by NetSDK Software as a basis on functionality. At the bare minimum, I’m going to need a way for users to enter in their keys and save them (because typing them in every time on a mobile device will DEFINITELY be annoying). They then need to be able to view buckets, and then be able to explore the contents of each bucket. Perhaps even adding in the ability to upload, download, delete and rename files if possible. Though at this point, I’m not even sure if winmo7 has a concept of a user accessible file system or if it’s abstracted/sandboxed like the iPhone.

Login Screen Design

Screens will probably be something akin to:

(1) Login/Key Manager -> (2) Buckets -> (3) Browse

(To the left you can see my first stab at the login screen using the built in controls.)

I’m going to have screen (1) act as a login screen for now, no saving of multiple accounts. That can be added later. Screen (2) will simply list buckets, and then later allow creation/deletion of them as well possibly. And finally screen (3) will list all your files and other functionality to be added later.

The actual design of each screen may be somewhat trivial, but I want the look and feel of application to be as if its a part of the operating system. A quick google search shows that there ARE a set of guidelines so I’m going to have to read them over and figure it out. From what I can tell though, as long as I’m using the Controls that are given to me then most of it should follow the look/feel anyways.

Initial Thoughts

So far, it seems that Microsoft has gone out of their to make it very easy to develop for the platform – leveraging technologies that developers should already be familiar with: Visual Studio, .Net, Expression Blend, SilverLight, etc…

I suspect that the SDK will become better as time goes on. For example, being based on .Net it wouldn’t surprise me if they allow programmers to use languages other than C#. Also from what I’ve been reading, a lot of people have had success porting over their existing SilverLight applications relatively easily and quickly to the new platform.

What’s Next?

Well, I’ve gotta make a library of some sort to connect to S3. I’ve gotta figure out how to use WPF but that shouldn’t be too difficult. Then I’ve gotta get it all working in the emulator. I’ll post updates as I have them, for now my project is available below along with a repository (SVN where it also turns out I managed to misspell theootz as theooz :/) link – feel free to check it out.

Back to list

Share

No related posts.

There’s the new “Projects” section which is powered by a nice little back-end script to let me easily manage links and data to all the projects I’m working on, or have worked on.

I added a “LifeStream” – based on the same concept that facebook uses to aggregate many social sites into one ’stream’. Ironically, I disabled facebook from showing up on the stream because it was posting way too much. Though the upside is, it gave me an idea for getting my whole cross-posting idea to work that I’m going to explore later. Originally I was trying to use my blog to push data to all the other services I use, but I figured I could also use facebook to push to my blog and from my blog to the rest of the services. I’d have to work out how I could make my blog still push unique posts to facebook and not push those specific posts but I haven’t really looked at any code yet so I’ll figure it out then.

Next, I finally figured out how to set up my blog as an OpenID provider So instead of remembering that convoluted verisign address that I *ALWAYS* forget, I can just use my blog address. Nice and simple.

I’ve added a massive caching system and found another use for my Amazon S3 account! I decided to create a new s3 bucket and linked it to Amazon’s Cloud Front. All media, most javascript, css, etc… is now served from local amazon end points (@ cloud.theootz.com) instead of my hosting provider I haven’t tested to see if this has resulted in any real speed improvement, but at least at a glance it seems to help a lot.

And to end it off, I found a wordpress plug-in for managing resume’s that I’m looking at using for posting my CV up on the site. Haven’t fiddled with it much yet but it seems to hold promise. Also another plug-in to find and display ‘related posts’ at the bottom of each post I make. Gotta love wordpress plug-ins

Share

Related posts:

1. video+cross post test, second attempt

Click here to view the embedded video.

And of course an embed test.

Share

Related posts:

1. video+cross posting test

I managed to add effectively full (for the program’s purposes anyways) youtube support to the program now. So it’ll batch render the mp3s to videos, and then upload all of them to youtube. The user can choose a youtube description for each file or just use the default one, as well as tags, the youtube category (synch’d with their schema), tags, etc…

Next up is figuring out the wordpress API to make posts on there, changing album art and video art per file. I might also decide to change the way it uploads the file to youtube and do it manually instead of using their .net library so I can add a progress bar but that would come after I’ve finished the more necessary features.

Share

Related posts:

1. Da Stank Bank “Uploader”

Higher quality vimeo link
Youtube embed…cause I think this should get embedded on tumblr?

Share

Related posts:

1. video+cross post test, second attempt

Anyways, long story short – un-installing NOD32 and installing Microsoft Security Essentials (which is more than good enough anyways) seems to let me sync things. Jungle Disk is still giving odd errors but I’ll see if I can work through them.

Not ready to ditch Dropbox yet — but it does keep reminding me I’m getting close to running out of space :/

Share

Related posts:

1. Jungle Disk – great idea, but…

Problem with dropbox’s paid service however, was that I was using only 2gb. It seemed like overkill to upgrade to a 50gb account for $10/month. I then remembered from a while back when I was first looking into cloud backup/sync software of one called Jungle Disk. It sounded great – uses Amazon S2 (as does dropbox) but with an open source client that gave you far more controla. Only reason I went with dropbox was because it was free and suited my needs at the time. So, it was time to give Jungle Disk another shot.

The sign up, payment and install processes went without a hitch. I decided to go with the “Jungle Disk Desktop” version, and hey turned out that with the base monthly fee they decided to include 5gb storage too – great! Payment was processed instantly and away I went!

On installing the program, it presents you with a wizard to help you set up your first ‘drive’. I figured I’d just enable all options and go from there – whether that’s a good thing or not I don’t know. To begin with, I wanted to sync my “My Dropbox” folder and if all goes well, I’d start with other folders too maybe – who knows? Yea, never got that far though. I set it to my drop box folder and let it sync over night. Waking up in the morning and expecting it to be complete I instead see several errors complaining it wasn’t able to upload one file or another and failed. It didn’t just skip the file (which from what I can tell, it’s supposed to!) but kept trying that same file over and over again until it finally gave up. So I figured, yea sure first time with the software – maybe it’s conflicting with dropbox already running in the background? So I gave that a close, and tried again. It got to the same file and stopped. No errors this time though (I was only able to check because of the logs I turned on from curiosity). Made changes to files? NEVER updated. I tried viewing what was saved on my disk via the virtual network drive – it only got bits of 2 main folders from about a dozen. Horrible!

Essentially, I’ve spent the last two days fighting with a horrible piece of software. It’s been constantly crashing on me whenever I try to make changes to configuration. And I’ve been making config changes trying to get it to sync successfully, if only just once! But every time it starts syncing, it gets to the same files and goes into an endless loop. Click cancel on the upload for that file? And it crashes! One of the files it was hanging up on was insignificant, so I decided to delete it but sure enough it finds another file to hang on.

I got frustrated to the point where I’ve decided to delete the virtual disk and try starting again. See where that leads me. If it -still- fails then I’ll contact support and see if the money I’m paying is worth it at all. If not, then I guess I’ll find something else. I’ve recently learned of another service called Mozy that seems to offer unlimited backup for around the same price as Jungle Disk’s base price but unlimited hosting of anything always makes me weary – more research would definitely be required. I’ve also been toying around with using Microsoft Mesh but the software is still a bit sluggish and doesn’t always sync properly. It also has no revision history which I’ve found can be VERY helpful!

Anyone else have any experience with Jungle Disk or other online backup/sync solutions? Any recommendations?

Share

Related posts:

1. Jungle Disk – continued

The current process goes somewhat like this:

• Open premiere.
  • Add our background image.
  • Add the mp3 file in an audio track.
  • Change the background image’s length to match the audio track.
  • Render (or queue) the file to be made into a video (usually an .flv).
• Go to youtube.
  • Upload the video(s).
  • Enter all the information: title, tags, description, etc…
  • Wait for it to process and get the embed code.
• Go to our blog.
  • Write a new post.
  • Add the same title, same tags, etc…
  • Set it’s category (dub/dnb/etc…)
  • Add the album art, and our embedded video below.
  • Save and publish the post.

As you can see, this can be quite the lengthy process for each video. So I figured I should be able to create my own program to make this easier.

My intended work flow would try to simplify a lot of the processes. So far, I see it working something like this:

• Open program
  • If it’s the first time, put in your youtube+wordpress login info
• Add all the mp3 files to be converted
• Set the background image
• For each mp3 file, enter the main information (extracting from ID3 tags first if possible). This includes:
  • Artist name, song title, album art, tags (using the tags from wordpress), category (using the categories from wordpress), background image (defaulting to a single image to be used on all tracks but can be changed for individual songs if required.)
• Click a button to render+upload.
  • The program will then render all the files to videos, upload them to youtube and fill in all data, get the embed code and make a wordpress post with all data as well.

Seems a hell of a lot simpler to me.

I started work on this last night, and so far I’ve got a basic program I’ve written up in C#. Right now it has the functionality there to be able to convert batches of mp3s to video files (I’m using windows media 9 to convert to wmv files – can’t seem to figure out how to make it an FLV file.) It also already logs in to your youtube account and I can get data from there, but haven’t figured out the uploading yet.

All in all, I figure this would make a cool week-long-ish project for me to do during the break so we’ll see how it goes

Share

Related posts:

1. Da Stank Bank Uploader – progress!