Unless I'm Wrong

How to associate a file name extension with a darwin /linux/unix command-line executable on osx/macintosh.

2014-07-25T11:02:00.004-07:00

One of the things one can do with self-serving (a multi-platform command-line executable for making simple tools that employ the user's browser as the UI and javascript for the logic), is to bundle a bunch of resources into a file with the ".slfsrv" extension; for example "morningMenu.slfsrv". On my macintosh I could execute this from the command line "self-serving morningMenu.slfsrv" or on the Windows command line with "self-serving.exe morningMenu.slfsrv".

I wanted to associate the ".slfsrv" extension with the executable, so that one could run a .slfsrv program simply by double-clicking on the file in OSX's Finder or Windows' explorer. This was very easy in Windows but on the Mac it took me quite a while to figure out. It turns out you can't associate a file extension with a simple executable, it has to be a Mac App.

How to mac a Mac App from a darwin executable

To make a Mac App from a darwin executable

Launch the Automator application
Click "New Document"
For document type select Application
From the Actions library on the left select select "Run Shell Script" and drag that to the workplace on the right
Change the "Pass input" option to "as arguments"
Edit the shell script for what to be something like one of these:

/path/to/my/exec/myexecutable "$@"

or

for f in "$@"
do
/path/to/my/exec/myexecutable "$f"done
Select File/Save and create a name and location for your ".app" file. In my case I called this "SelfServing.app" and put it in my "~/bin" folder

You next want to associate a file extension with the .app you just created, as follows:

In the finder open a folder containing a file of that type. In my case I opened the folder containing the file morninMenu.slfsrv
Right click on the file and select "Get Info"
Under "Open With" select "Other..." and locate the .app file you created from Automator (you probably want to have all files with that extension opened with your app).

Using it

That's it. Now you can run the command-line program on all files with that executable simply by double clicking that type of file in Finder, or from the command-line typing "open myfile.myextension"

Sharing picasa photos, albums, and tags between Windows and Macintosh OSX computers

2014-06-12T20:12:00.000-07:00

So, you have all your pictures on your computer, you view and tag them with Picasa, and your Windows-using spouse wants to have the same photos and tags and albums and such on her Windows machine. What do you do?

First you share your picture folders between machines using BitTorrent Sync. That works great. Oh Joy!

Next, after you've spent hours on your Macintosh tagging people in all the photos, you want your spouse to see all those same tags on her Windows machine. So you play around with sharing the Picasa "contacts" folder, and you try sharing the "db3" folder, and you try the "store name tags in photo" Picasa options, and the experimental "write faces to XMP" and various combinations of the above. No matter what you try, at best it doesn't share data between machines, and at worst it corrupts your data.

So you stop syncing everything except the photo folders themselves. And you restore your "contacts" and "db3" folders (you did remember to back those up, didn't you), and you give up on you and your spouse ever having the benefit of being able to share your tagged people faces.

Sigh.

Thinking about Brownian Motion, Air Molecules, and Airplane Wings

2014-06-06T15:36:00.000-07:00

This entry has moved to my primary blog location.

How I use Google AdSense and Google Analytics with Meteor websites

2014-05-27T15:02:00.003-07:00

Here is how I managed to get Google AdSense and Google Analytics into my Meteor websites: airbnbconfessions and NoMorePasswordsJustEmail.

Google Analytics is easy

It's not difficult getting Google Analytics into your Meteor web app. Simply take the Google Analytics startup code and put it in client-side of your Meteor source.

Here, for example, is a file I have within the /client directory of my Meteor airbnbconfessions source code.

Meteor.startup(function() {
$(document).ready(function() {
// show google analytics if running on the live web site
if ( -1 != document.URL.indexOf("www.airbnbconfessions.com") )
{
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-YOUR-CODE-HERE', 'airbnbconfessions.com');
ga('send', 'pageview');
}
});
});

Note that I rely on jQuery and $(document).ready() but I assume Meteor.startup() probably would work as well.

Google Adsense is trickier

Two problems: 1) Meteor really wants to be in controls of rendering all your HTML, and 2) Google really wants the page to exists as some raw HTML before rendering the ads.

For (1), Meteor provides the /public directory. Anything in /public bypassed the Meteor rendering system and is just available to the web client like any other raw internet file.

For (2), the ads will be loaded within iframes, refereing to those /public files, from the main page.

Bringing Google Adsense and Meteor together

In NoMorePasswordsJustEmail, I want the ads to appear in the right for web browsers, but on the bottom for mobile browsers. So in my primary index.html file uses bootstrap classes in this code:

<div id="list-stuff">
<div class="row">

<div class="span9">

...page stuff...

</div>

<div class="span3 hidden-phone" style="text-align:center;">
<center>
{{> adsRight}}
</center>
</div>
</div>

<div class="row visible-phone" style="margin-left:-20px;margin-right:-20px;">{{! style makes up for 20 pixels added to body }}
<div class="span10" style="text-align:center;">
<center>
{{> adsBottom}}
</center>
</div>
</div>

</div>

Those adsright and adsbottom templates render iframes that will load the files that were in /public and look like this client/ads.html file.



<template name="adsRight">
<div id="adsRight">
<iframe src="/adright.html?v={{{unique}}}" seamless width="160" height="600" scrolling="no"
frameborder="0" marginheight="0" marginwidth="0"
style="margin:0;padding:0;border:none;width:160px;height:600px"></iframe>
</div>
</template>
<template name="adsBottom">
<div id="adsBottom">
<hr/>
<iframe src="/adbottom.html?v={{{unique}}}" seamless width="336" height="280" scrolling="no"
frameborder="0" marginheight="0" marginwidth="0"
style="margin:0;padding:0;border:none;width:336px;height:280px"></iframe>
</div>
</template>

That code refers to unique, just force Google to take a moment to check the text on the page each time, and provide a fresh ad as the text of the page changes with new reloads. client/ads.js provides that:

Template.adsRight.unique = function() {
return (new Date()).valueOf();
};

Template.adsBottom.unique = function() {
return (new Date()).valueOf();
};

Finally, the two files that like to the ads themselves.

public/adbottom.html:

<html>
<head>
<style type="text/css">
* {
margin: 0;
padding: 0;
}
</style>
</head>
<body>
<div>

<script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>

<ins class="adsbygoogle"
style="display:inline-block;width:336px;height:280px"
data-ad-client="ca-pub-YOUR-GOOGLE-CODE"
data-ad-slot="YOUR-GOOGLE-ADSLOT"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>

</div>
</body>
</html>

public/adright.html:

<html>
<head>
<style type="text/css">
* {
margin: 0;
padding: 0;
}
</style>
</head>
<body>
<div>

<script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>

<ins class="adsbygoogle"
style="display:inline-block;width:160px;height:600px"
data-ad-client="ca-pub-YOUR-GOOGLE-CODE"
data-ad-slot="YOUR-GOOGLE-ADSLOT"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>

</div>
</body>
</html>

That's all.

Good luck. My fingers are crossed for you.

Setting up Bitnami tomcat stack on VirtualBox on OSX 10.9 (Mavericks), with a shared folder

2013-12-11T07:53:00.001-08:00

I had such a hard time getting the Bitnami Tomcat stack to run on the latest VirtualBox (4.3.4), and especially with adding shared folders. I was all day long retrying over and over. Sheesh. These are the magic steps I took, recorded here in hopes that it can save someone else some time.

Creating the bitnami stack on OSX

Install (a couple of times until it works, dagnabit) a new version of virtual box from http://www.oracle.com/technetwork/server-storage/virtualbox/downloads/index.html

Download the bitnami tomcat vm image from http://bitnami.com/stack/tomcat (the folder contains bitnami-tomcatstack...vmdk)

When starting a new virtual box image, pay attention to settings here: http://wiki.bitnami.com/Virtual_Appliances_Quick_Start_Guide#Virtual_Box

type: Linux
version: Ubuntu (64 bit)
Use an existing virtual hard drive file (that bitnami-tomcatstack...vmdk file mentioned earlier)
Create ->
In settings, for Network, select Attached to: Bridged Adapter
Press the big Start arrow

Shortly a new linux console will be running, with mysql, apache, tomcat, whooopee!

On first login “bitnami” with password “bitnami”. Your first login will force you to select a new password.

For java tools, I added this to ~/.bashrc

JAVA_HOME=/opt/bitnami/java
export JAVA_HOME

To verify the success, look in your local mac browser for the tomcat server at the address presented on the VM console (in my case “You can access the application at http://10.0.0.6/ ***”)

Getting a shared folder that is ~/Desktop on both host and client (via Guest Additions):

1) download VBoxGuestAdditions_4.3.4.iso from here: http://download.virtualbox.org/virtualbox/4.3.4/

2) Then in virtual machine

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install dkms
$ sudo reboot

3) then (partially from http://wiki.bitnami.com/Virtual_Appliances_Quick_Start_Guide#How_to_install_VirtualBox_Guest_Additions_on_BitNami_Virtual_Appliances.3f)

$ uname -a ### to get kernel version, e.g. 3.2.0-53-virtual
$ sudo apt-get -y install dkms build-essential linux-headers-3.2.0-53-virtual
$ sudo reboot

4) use VBoxGuestAdditions.iso stuff from the vbox menu Devices -> CD/DVD Devices -> Choose a Virtual CD/DVD… as found at http://wiki.bitnami.com/Virtual_Appliances_Quick_Start_Guide#How_to_install_VirtualBox_Guest_Additions_on_BitNami_Virtual_Appliances.3f to select “VBoxGuestAdditions.iso” (wherever you downloaded that to in step 1)

$ sudo mkdir /mnt/cdrom
$ sudo mount /dev/cdrom /mnt/cdrom
$ cd /mnt/cdrom
$ sudo sh ./VBoxLinuxAdditions.run --nox11 # ignore “could not find x.org…” error

5) Recreate (or update or something) the VB guest addition modules:

$ sudo /etc/init.d/vboxadd setup
$ sudo reboot

6) In the VB menu Devices -> “Shared Folders Settings...” set up ~/Desktop to be shared (auto-mounted, permanent, and r/w) as “Desktop”

7) Mount that Desktop menu that was mounted 23r2

$ mkdir ~/Desktop
$ sudo mount -t vboxsf ~/Desktop Desktop

8) In theory I should have been able to make that mounting automatic, even after reboots, via an entry in /etc/fstab but I was never able to get that working. So in the end, to get the mounting happen each time I log in, I added this line to ~/.bashrc

sudo mount -t vboxsf ~/Desktop Desktop

Not that from the VM side, ~/Desktop is read-only for the bitnami user, and so “sudo” must be used for a bunch of the commands to read and copy files and such.

SSH to the bitnami machine

On the bitnami shell:

$ sudo mv /etc/init/ssh.conf.back /etc/init/ssh.conf
$ sudo start ssh

On my mac:

# ssh bitnami@10.0.0.6

That's what worked for me. Good Luck!

Stripes jsp / jstl forms won't handle unicode / utf-8 characters. Yikes. What's wrong?

2013-08-24T15:52:00.000-07:00

I don't know what's wrong in your case. But in my case stripes+tomcat+mysql case I had my database set up OK (from my.cnf)

[mysqld]#default-character-set=utf8#default-collation=utf8_general_cicharacter-set-server=utf8collation-server=utf8_general_ciinit-connect='SET NAMES utf8'character-set-filesystem = UTF8
[client]default-character-set=utf8

and all my forms looked great:

<s:form beanclass="${actionBean.getClass().name}" acceptcharset="utf-8">

but unicode values would not return forms in utf-8 format.

I tried all the solutions found at TechnologyNotes: UTF-8 Encoding fix (Tomcat, JSP, etc), but those didn't work. So I undid most of them.

What did work for me was found within this Stripes HowTo. I changed this:

<init-param> <param-name>LocalePicker.Locales</param-name>
<param-value>en:UTF-8, en_US:UTF-8</param-value>
</init-param>

to this:

<init-param>
  <param-name>LocalePicker.Locales</param-name>
  <param-value>en:UTF-8, en_US:UTF-8</param-value>
</init-param>

And unicode characters in forms worked swell. Whoopee.

M. Dung, the KFOG host of the 80's is found!

2012-07-27T14:41:00.001-07:00

I've you've sadly been missing M. Dung, the KFOG morning host from the mid 80's, like I was, be sad no more. He's still producing The Mutant Idiotshow every week. Here's more:

how to adjust the macintosh startup sound

2012-07-17T11:43:00.003-07:00

Mac startup sound is too loud? I've wasted most of the morning trying to figure out how to make my apple macbook pro ess loud when I boot it, so that it doesn't wake everyone in the house. Here's the tool that worked for me: StartupSound.prefPane

Building HelloSun, Step 6: Speech Bubbles

2012-06-11T14:23:00.000-07:00

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 5	T.O.C.

I'm not very happy with this scrolling "chat" part of the web page, which so far looks like this:

I'd much prefer each person's comment to appear in a speech bubble, as I've sketched on this post-it note:

prototyping with newfangled css layout and rounded corners

Borrowing liberally from Pure CSS speech bubbles, I tweaked this local html file:

<html>

<head>

<style type="text/css">




.bubble-content {

  position:relative;

  padding:6px;

  margin:4px 0 0 0;

  color:#000;

  background:#f3961c; /* for browsers
without gradient
support */

  /* css3 */

  background:-webkit-gradient(linear, 0 0, 0 100%,
from(#f9d835),
to(#f3961c));

  background:-moz-linear-gradient(#f9d835, #f3961c);

  background:-o-linear-gradient(#f9d835, #f3961c);

  background:linear-gradient(#f9d835, #f3961c);

  -webkit-border-radius:10px;

  -moz-border-radius:10px;

  border-radius:10px;

  -webkit-border-bottom-left-radius:0;

  -moz-border-bottom-left-radius:0;

  border-bottom-left-radius:0;

  font-family: Arial;

  font-size: 12px;

}



.bubble-citation {

  margin:0 0 8px 0;

}



.whom {

  margin: -8px 0 0 14px;

  color:#000;

  font-family: Arial;

  font-size: 12px;

}



.quote-tail {

  background:#f3961c;

  width:20px;

  height:16px;

  -webkit-border-radius:8px;

  -moz-border-radius:8px;

  border-radius:8px;

  margin:0;

  margin-top:-9px;

}



.quote-tail2 {

  background:#fff;

  width:14px;

  height:12px;

  -webkit-border-radius:6px;

  -moz-border-radius:6px;

  border-radius:6px;

  margin:0;

  margin-top:-12px;

  margin-left:8px;

}

</style>



</head>

<body>

  <div class="bubble-content">

    This is some bubble content I could just go on
and on and on

    and maybe I will because, really, you know, I
have an awful

    lot to say because I'm just very verbose.

  </div>

  <div class="bubble-citation">

    <div
class="quote-tail"></div>

    <div
class="quote-tail2"></div>

    <div
class="whom">Tommy</div>

  </div>



  <div class="bubble-content">

    This is some more bubble content.

  </div>

  <div class="bubble-citation">

    <div
class="quote-tail"></div>

    <div
class="quote-tail2"></div>

    <div
class="whom">Sally</div>

  </div>

</body>

</html>

In a modern browser the HTML above produces this page

That looks enough like the bubble quotes I drew on the post-it note that I'll use it. (I may come to regret relying so much on the latest CSS, when I get around to testing in different browsers--but since the primary clients I'm considering will be the most popular smartphones, and they all support this level of CSS, I won't worry about this issue yet.)

change the chat insertion code to use these newfangled css bubble quotes

By adding the css classes from the local html prototype file into the "chatty.css" file, and changing the "appendMsg()" function in "chatty.js to this:

function
appendMsg(msg) {

    $('#msgs').append(function() {

        var div,
html;

        div =
$('<div>');

        html =
'<div class="bubble-content">' +

       
       
    simple_html_escape(msg.message) +

       
       
'</div>\r\n' +

       
       
'<div class="quote-tail"></div>\r\n' +

       
       
    '<div
class="quote-tail2"></div>\r\n' +

       
       
    '<div class="whom">' +

       
       
       
simple_html_escape(msg.username) +

       
       
    '</div>\r\n' +

       
       
'</div>\r\n';

       
div.html(html);

        return
div;

    });

    $('#msgs')[0].scrollTop =
$('#msgs')[0].scrollHeight;

}

the server now displays the chats within speech bubbles.

let's look better on an iPhone

The primary client for HelloSun is expected to be mobile clients. A couple of small adjustments can be made to make this code look better on iphones. The primary change is adding a "meta" tag to the layout of all files in this line added to layout.jade:

 
meta(name="viewport",content="width=device-width; initial-scale=1;
maximum-scale=1")

check for blank inputs

The next little change for today's code is to prevent users from submitting if they haven't entered either a name or a message, because if those fields are blank it looks silly on the client displays. That's done with a simple function to remove space from the beginning and end of fields (the "trim()" method found at Javascript Trim LTrim and RTrim Functions), and a simple check in the "sendMsg()" function of chatty.js to verify that the fields are not blank.

always scroll for new messages

The final change for today's code is to automatically scroll to the bottom of the page when a new message comes. This ensures that users are always seeing the latest messages from other users. This line in "chatty.js" function "appendMsg()" replaces the old style to work with the new type of page. (It's not particularly elegant, but it works.)

window.scrollTo(0, document.body.scrollHeight);

the new look

With all of today's changes, the chat program now looks something like this:

...and that's good enough for today.

Getting the code

Today's code is labeled v0001.0005 and is available in these formats:

browse the code
all v0001.0004...v0001.0005 code changes
https://github.com/BrentNoorda/hellosun-tutorial/zipball/v0001.0005
tarball: https://github.com/BrentNoorda/hellosun-tutorial/tarball/v0001.0005
git:
# git clone git@github.com:BrentNoorda/hellosun-tutorial.git
# cd hellosun-tutorial
# git checkout v0001.0005

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 5	T.O.C.

Building HelloSun, Step 5: Clean Code is Happy Code

2012-06-09T14:23:00.001-07:00

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 4	T.O.C.	Step 6 ⇒

So far the HelloSun code has been very sloppy, and sloppy code is bug-prone code. So today we won't add any new functionality, but will just clean up some areas of sloppiness. These are all small changes, but they'll make me feel better (I'm a bit anal-retentive that way).

use a lint tool

lint tools look for obvious problems in source code. I don't see any valid reason not to use them. I'm using JSLint, but there are alternatives.

This early in coding, JSLint recommended a few minor changes (all of which I accepted, in one case using jQuery's "each" call to avoid using "hasOwnProperty"), but found only one real bug: (the variable "i" was not declared in "chatty.jade", so would have overwritten any global "i" and would also have been inefficient).

At the top of all the JavaScript files are now lines similar to the following, which are hints to the JSLint tool I'm using.

  /*jslint white:false plusplus:false nomen:false
*/

  /*globals $, io */

I won't be submitting any more javascript code before running lint on it.

use a minimized version of jQuery

jQuery is nearly a standard in client-side Javascript now, and is used in the node-simple-chat example code we plagiarized in Step 4. To save a little download time I've changed to jQuery download in chatty.jade from

  script(src="http://code.jquery.com/jquery-1.7.1.js",

       
 type="text/javascript")

  script(src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js",

       
 type="text/javascript")

move server-side javascript into its own "chatty.js" file

"chatty.jade" contained a few dozen lines of javascript code embedded in the html file. I prefer to move this javascript code to its own "javascript/chatty.js" file because it 1) let's me run lint against the code, 2) treats the javascript as its own resource, and 3) is easier for me to read.

don't alter Array.prototype

The node-simple-chat example code we plagiarized in Step 4 used a trick of altering JavaScript's "Array.prototype" simply to add an "inject" method used once. I find it very bothersome to be altering a built-in Javascript object (can we be sure it's writable? can we be sure no other user of the Javascript global is not using a similar function, or is not iterating over an expected list?).

So I changed "Array.prototype.inject" in "chatserver.js" to a simple "inject_message" function.

prevent users from injecting malicious html code

This is the only serious problem we're fixing today.

Whenever you're allowing user-generated as your source for output, you're providing a big hole for malicious user to cause problems. In our chat program so far, we take whatever text the user enters for their name or their message, and display this text directly in the browser of all users.

Imagine that a user enters the message "I like <font color="green"><i>green italics</i><font>." This would immediately show up in every users window as:

myame: I like green italics.

You might think this innocent use of html codes is kind of cool. But if a user weren't so innocent, they might be inserting code to links, or uncouth images, or even malicious script that would be executed within every user's browser.

This would allow any user of your service to deliver malicious functionality to every other user of your service. Yikes!

A very simple solution is just to escape a few special characters ("<", ">", and "&") so that if they're part of user input that we are injecting into html, they will not be seen as raw html characters. This is done with the "simple_html_escape" function added to chatty.js:

  function simple_html_escape(s)

  {

      return
s.replace(/&/g,'&amp;')

           
  .replace(/&lt;/g,'&lt;')

           
  .replace(/>/g,'&gt;');

  }

"simple_html_escape()" is then added to "function appendMsg()" wherever the user-generated strings are display in the output page.

Getting the code

Today's code is labeled v0001.0004 and is available in these formats:

browse the code
all v0001.0003...v0001.0004 code changes
https://github.com/BrentNoorda/hellosun-tutorial/zipball/v0001.0004
tarball: https://github.com/BrentNoorda/hellosun-tutorial/tarball/v0001.0004
git:
# git clone git@github.com:BrentNoorda/hellosun-tutorial.git
# cd hellosun-tutorial
# git checkout v0001.0004

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 4	T.O.C.	Step 6 ⇒

Building HelloSun, Step 4: Real-Time Chat

2012-06-08T11:11:00.000-07:00

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 3	T.O.C.	Step 5 ⇒

HelloSun is going to be a kind of chat app, so for the next step let's make a very very very basic real-time basic chat page. It seems that the first thing anyone does when doing a node server is a chat app, so there are a whole lot of examples to borrow from.

For the first chat page I’ll steal liberally from the example at https://github.com/jmg/node-simple-chat, making only minimal changes to make it work with Heroku and with the jade template language (for no other reason than I’m looking for excuses to learn more about Heroku and jade).

choosing socket.io for real-time(ish) push

A real-time chat app is one where as soon as somebody enters a message, everyone else on the chat sees it right away. In the traditional browser model, the browser client is not updated until the browser pulls data from the server (e.g. through a page refresh), but in this use it makes more sense to think of the browser pushing data to the client when it's available. Over the years many tricks have been developed to make it seem like the server can push data (many documented at Wikipedia Push Technology), and even more options are available as WebSockets become standardized.

The existing push technologies are a big bag of kludgy tricks that vary depending on the capabilities of the exact browser being used, and so I'm very glad that someone has bundled up a library of code to manage all of these differences once and for all, in a library called socket.io. The the socket.io javascript library provides a single simple interface to direct communication between the client and the server, hiding whatever different methods are used to pull off the trick. (Thank you very much, Guillermo Rauch!) Another nice feature of socket.io is that the API for the client and for the server are nearly identical (which is the "client" and which is the "server" doesn't matter any more).

socket.io has just two primary methods that we care the most about:

emit() sends a message to the other end of the connection
on() receives an event (usually a message from the other end of the connection)

Installing socket.io

socket.io is under rapid development, but I believe I've found one that works stably on Heroku. To use this version of socket.io add this dependency line to your "package.json" file:

"socket.io": "0.9.6"

then run “npm install -d" to cause the latest version of socket.io to be installed.

chatserver.js - server-side chat code

The server side of this chat server is in the new file "chatserver.js" (see Getting the code below, or view chatserver.js). "chatserver.js" is almost directly taken from the "chat.js" file at the node-simple-chat project we're plagiarizing. The primary change is in these lines:

  io.configure(function () { io.set("transports",
["xhr-polling"]);

  io.set("polling duration", 10); });

which are taken from Heroku's advice at Using Socket.IO with Node.js on Heroku to prevent socket.io from trying tricks that Heroku doesn't support.

This "chatserver.js" code, after connecting with the client, simply receives JSON strings labelled as "msg" (we could have picked any label, so long as it matches label the client sends), and appends that to the list of messages, and broadcasts the message to all connected clients. Pretty straightforward. (I'm not a fan of the "Array.prototype.inject" trick in this code, but plagiarizers don't have much right to complain.)

To load "chatserver.js", the file "app.js" (see Getting the code below, or view app.js) gets this line added at the end:

  var chatserver_io =
require('./chatserver').start(app);

chatty.jade - client-side chat code

The client side of the chat server is in the new file "views/chatty.jade" (see Getting the code below, or view views/chatty.jade) "chatty.jade" is almost directly taken from the "chat.html" file at the node-simple-chat project we're plagiarizing, only changed to support the jade templating library.

"chatty.jade" is straightforward. It collects input, sends it to the server (with socket.emit('msg',...)), and displays in the chat window any chat messages it received from the server (with socket.on('msg')).

Warning: including 'socket.io.js' in the web page contains magic

In my opinion, the creators of "socket.io" were just a tiny bit too clever in integrating socket.io with node.js, because the way it gets included seems to much like magic. In "chatty.jade" socket.io is included with this single line:

script(src="/socket.io/socket.io.js")

which jade turns into this line in the client html file:

  &lt;script
src="/socket.io/socket.io.js">&lt;/script>

But if you look on the server under the directory where you expect javascript files to be loaded, which is in "/public/javascripts", there is no "socket.io.js" file there. So how does "socket.io.js" get loaded in the client if the file doesn't exist on the server? That's where the magic comes in:

When socket.io is initialized on the server (require('socket.io').listen(app) in chatserver.js) the socket.io adds a lot of code, much of which alters the server socket code. Part of that alteration is to specifically manage a request for the file "socket.io/socket.io.js", at which point the server-side socket.io code puts together an appropriate socket.io.js file and delivers that to the client. Magic!

SUMMARY OF CHANGES

Here's a summary of all the changes made in this version (see Getting the code below for full source):

add chatserver.js (plagiarized) to use socket.io, and initialize from apps.js
create chatty.jade and chatty.css (plagiarized) to be a page for the chatty client, and make express render aware of it through changes to routes/index.js and app.js
change layout.js to support block in templates, and alter views/index.jade and apps.js to turn that on

Test and Deploy:

If you run this new code locally (# foreman start) and check http://localhost:3000/, or push this code to the live server (commit with git, then # heroku restart web.1), you should see a new chat window looking much like this:

Now you have a chat-window, live on the web, that anyone can join at any time (from any browser?). Whoopee!

Getting the code

Today's code is labeled v0001.0003 and is available in these formats:

browse the code
all v0001.0002...v0001.0003 code changes
zip: https://github.com/BrentNoorda/hellosun-tutorial/zipball/v0001.0003
tarball: https://github.com/BrentNoorda/hellosun-tutorial/tarball/v0001.0003
git:
# git clone git@github.com:BrentNoorda/hellosun-tutorial.git
# cd hellosun-tutorial
# git checkout v0001.0003

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 3	T.O.C.	Step 5 ⇒

Building HelloSun

2012-06-07T10:31:00.000-07:00

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.

The latest version of this application is always running at http://hellosun.herokuapp.com/ The code is also available on github.

Step 1: Planning - What's the project? What are the Goals?
Step 2: Hello Node & Heroku - Choosing a PaaS and Server Technology
Step 3: Being Express-y - Configuration the default Node framework: Express
Step 4: Real-Time Chat - Adding a real-time chat page with socket.io
Step 5: Clean Code is Happy Code - Minor code cleanup; no new functionality
Step 6: Speech Bubbles - Use speech bubbles, and css changes, to look like a better chat program, especially on a mobile phone
Step 7: ???

Building HelloSun, Step 3: Being Express-y

2012-06-07T10:22:00.001-07:00

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 2	T.O.C.	Step 4 ⇒

In Step 2: Hello Node & Heroku, the Heroku setup and default were a bit too basic. We're going to want to make something slightly more complicated than a simple "Hello World!" page. We'll eventually want multiple pages, with multiple HTML pages and related atrtibutes (CSS, Javasript, and images). We don't want to write any more code than necessary, so we'll want to use some existing framework where most of that grunt-work code has already been written. For this project we'll use Express.

Why choose Express?

I put very little thought into choosing Express as my framework. A version of Express was part of the default Heroku install, and a cursory review of discussions about the topic make it look like Express is currently the most active and popular. So I'll start there. If Express turns out to be terrible... well, there's no better way to find out that it sucks than to try it out for myself.

In other words, I'm using Express because everyone else is using it. (And, yes, if all my friends are jumping off a cliff then I'm jumping off with them.)

(Re)Installing Express

Even though the Heroku install used Express by default, it didn't use Express in a standard "Expressy" way. So we'll do some quick changes to convert the default Heroku install into something that looks normal for default Express users. (Let's try not to reinvent anything fancy yet, and stick to the defaults.)

From within our application's directory (which we've been calling "~/hellosunx444") let's generate a default express app as defined at http://expressjs.com/guide.html with this command:

# cd ~/hellosunx444
# node_modules/express/bin/express .

This command will add a lot of default directories that Express depends on (e.g. "views", "public"), will add some dependencies on Jade (the default Express template engine--we like starting with defaults), and a new "app.js" file.

This install will also have altered "package.json" in small ways, some of which we want to fix, so that "package.json" now looks like this:

  {

      "name": "hellosun"

    , "version": "0.0.1"

    , "private": true

    , "dependencies": {

        "express": "2.5.8"

      , "jade": ">= 0.0.1"

      , "pg": "0.5.4"

    }

  }

As we'll do after every change to "package.json", run this command so all our packages are updated:

# npm install -d

Adjusting default Express files to run on Heroku

A couple of changes to the Heroku files will get us back on track to have a standard Express layout be running on Heroku.

Express' default starting file is “app.js”, while the original heroku sample ran as “web.js”. You want Heroku to be using this “app.js” instead of the old “web.js” so edit the file “Procfile” to be

web: node app.js

We only want one change from the previous "web.js" file, and that is part that adjusts the port to be running on for the Heroku environment. When running locally we can specify the port, but in the heroku environment heroku will be selecting that port for us (it's background system will have some kind of gateway and port 80, but that will be redirecting to some port of its choosing). In "web.js" that line was:

var port = process.env.PORT || 3000;

In the “app.js” file created by Express the same can be done by replacing:

app.listen(3000);

with

app.listen(process.env.PORT || 3000);

If we test this code locally (# foreman start) and check http://localhost:3000/, or push this code to the live server (# heroku restart web.1), then check http://hellosunx444.herokuapp.com/ we see it’s running the new Express code.

Go ahead and delete the "web.js" file; it is no longer needed. (I don't like to keep files around that aren't being used--that's just confusing.)

Create the first HelloSun Express application

The default home file that Express created is in “routes/index.js” (more on how this resolves later). Change "title" to “HelloSun” to match our program.

After pushing to the server (# heroku restart web.1), we have an Express app named “HelloSun” that runs on Heroku. Yea!

Getting the code

Today's code is labeled v0001.0002 and is available in these formats:

browse the code
all v0001.0001...v0001.0002 code changes
zip: https://github.com/BrentNoorda/hellosun-tutorial/zipball/v0001.0002
tarball: https://github.com/BrentNoorda/hellosun-tutorial/tarball/v0001.0002
git:
# git clone git@github.com:BrentNoorda/hellosun-tutorial.git
# cd hellosun-tutorial
# git checkout v0001.0002

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 2	T.O.C.	Step 4 ⇒

Building HelloSun, Step 2: Hello Node & Heroku

2012-06-05T14:02:00.000-07:00

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 1	T.O.C.	Step 3 ⇒

In Step 1: Planning, part of the goal was to “use the latest & simplest techniques and tools, and zero upfront costs.”

The “simplest techniques” goal leads me to choose a Platform as a Service (PaaS) solution. With PaaS, the service provider handles most of the infrastructure involved in handling a web server. I simply must write the code that’s unique to my particular application. There is an Ever-Growing List of PaaS Companies to choose from.

The “zero upfront costs” goal narrows down the list to those that offer a free option. Many of them offer free getting-started options, either for a limited time or for a limited amount of CPU and bandwidth.

The “latest techniques and tools” goal makes me want to chose Node.js as the server technology. Node.js is a javascript-based server that, because it almost totally event-driven and non-blocking, leads to efficiency, meaning: this model should allow me to support the most clients possible before having to move to a paid service. I almost went with Ruby, simply as an excuse to learn a new language, but it’s not quite “the latest thing” anymore, I had trouble installing it locally (more on that later), and I hear some disappointing things about its throughput. (I also am persuaded to choose Node.js because of a personal long history behind Javascript, as well as recent fun creating the CrappyNet bad-networking RESTful stress tester.)

I chose Heroku as my PaaS for this project because it supports Node.js and has a nice free option: the equivalent of one server process (enough for Node.js to handle many clients) along with a small free database options (enough for a sunrise-limited window of chats, if that even needs to be part of a database), and because of these additional considerations:

Documentation – Heroku’s documentation seems concise and mostly correct.
Local Development and Testing – With the Heroku stack I can easily run the app on my local computer very close to how it will run on the server. Thing globally, develop and test locally.
Easy Deployment – After testing locally, deploying to Heroku is as easy as a git push.
Easy to Switch (no Heroku lock-in) – Because there is nothing special or proprietary to Heroku or the stack I’m using, I can easily stop using Heroku at any time and switch to many other PaaS or, if this service is doing spectacularly well, I can easily move to my own servers at any time.

Get git

git version control is required for working with Heroku. Introducing git is outside the scope of this post, but if you’re doing anything in software these days you need to become familiar with it, so you may as well start at the Official git website.

Install Node locally

Using the Node Installer is super simple. Just download from http://nodejs.org/#download. I do not recommend getting the latest version of Node from their github repository because those versions are too new to match what is running on Heroku.

After installing node I can run “node --version" at the command line and see that I have “v0.6.18”. This is close to what is on Heroku and hopefully will work fine.

Get a Heroku Account

Sign Up for an account at https://api.heroku.com/signup.

Install the Heroku Toolbelt Locally

The Heroku documentation is very straightforward at https://devcenter.heroku.com/. In particular, I followed the steps at Getting Started with Node.js on Heroku/Cedar.

In Local Workstation Setup you’ll install the Heroku Toolbelt, which is what will let you run all the necessary Heroku tools from the command line on your local computer.

Write and test your first App

Heroku’s Write Your App instructions worked almost perfectly for me, except that the version of “express” in their example did not match the default I found with the heroku install, and I had a new name for my app. So my version of package.json is:

  {
    "name": "hellosun", // your name, e.g. "hellosunx444"
    "version": "0.0.1",
    "dependencies": {
      "express": "2.5.8"
    },

"engines": {

"node": "0.6.x",
"npm": "1.1.x"

}
}

After following the next set of the instruction at at Getting Started with Node.js on Heroku/Cedar (create web.js, “npm install”, create .gitignore, create Procfile) you should be able to run the code locally with this command:

$ foreman start

Test this with a browser at http://localhost:5000/. If that shows “Hello World!” then you’ve got a local node server running the web.js code. Congratulations!

Deploy your first App to the World Wide Web

Heroku’s Getting Started/Deploy instructions worked fine for me. They are:

$ git init
$ git add .
$ git commit -m "init"
$ heroku create hellosunx444 --stack cedar
$ git push heroku master
$ heroku ps:scale web=1
$ heroku config:add NODE_ENV=production

Testing your first App to the World Wide Web

Browse to http://hellosunx444.herokuapp.com/ and if you now see “Hello World!” then your application is now deployed to heroku. Congratulations!

Changing code and restarting

In step 3 through step X, we’ll be changing this code a lot. So let’s quickly cover how to change the code to “Hello Squirrel!” and redeploy:

Edit web.js to change “Hello World!” to “Hello Squirrel!”
Test locally with “foreman start” and verify that http://localhost:5000/ displays “Hello Squirrel!”
Use git to push your change to web.js to the heroku git server (using your favorite git tools or commands, such as “$ git commit –a”, “$ git push”.
Restart the heroku server with your new code: “$ heroku restart web.1”
Verify that http://hellosunx444.herokuapp.com/ now shows “Hello Squirrel!”

Getting the code

Today’s code is labeled v0001.0001 and is available in these formats:

browse the code
zip: https://github.com/BrentNoorda/hellosun-tutorial/zipball/v0001.0001
tarball: https://github.com/BrentNoorda/hellosun-tutorial/tarball/v0001.0001
git:
# git clone git@github.com:BrentNoorda/hellosun-tutorial.git
# cd hellosun-tutorial
# git checkout v0001.0001

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
⇐ Step 1	T.O.C.	Step 3 ⇒

Building HelloSun, Step 1: Planning

2012-06-04T11:50:00.003-07:00

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
	T.O.C.	Step 2 ⇒

Background

Much has changed since I wrote my first computer software in the late 1970s. Back then, if you’d told people you wanted to spend a couple of weeks creating a program that would instantaneously connect everyone in the world in a live experience, and that you intended to do it single handedly and for zero cost, they would have called you nuts.

But now, in 2012, using the latest tools and languages, this should be a piece of cake. In this X-part series I intend to see if that’s true, and to document each step of the process.

Goal

Use the latest & simplest techniques and tools, and zero upfront costs, to create a social networking app. And to document how it’s done.

What is “HelloSun” (and why)?

First, I want to tackle something that contains most components of modern social software, but that is not too complex. “Chat” embodies a smallest-common-denominator component of social software, and does not introduce any radically new ideas.

Second, I want something with a limited storage need, because I haven’t yet figured out how to stick to my “zero cost” goal if very much storage is involved (even though storage is very very cheap, anything beyond a fixed size is not yet free).

Third, I want to explore the concept of the value of limitations. It seems that most applications or websites that catch on do so because they do so very little (at least initially), and that their virality is inversely proportional to their complexity. (Would Twitter have caught on if it allowed for long diatribes? Would Instagram have caught on if it didn’t have so few options? Would Facebook have ever reached the size it is now if it had started out as complicated-to-use as it is now?) So I want to create something that does as little as possible.

To achieve the above goals I’m planning the “HelloSun” application. For most of the day (23+ hours of each day) HelloSun will do absolutely nothing. But during a few minutes before and after sunrise (whenever sunrise is for each user’s location), it will be a simple chat app shared by anyone in the world who is also experiencing sunrise. Then, for the rest of each day, it goes back to doing nothing, and the mornings chats are all gone forever (poof).

HelloSun will be a brief chat window at the start of each day. That’s all!

Plan

I don’t know exactly what the plan is, since I haven’t started yet, but on day 1 here’s what I expect to do:

Choose a PaaS (Platform as a service) provider that offers a free getting-started plan (probably Heroku).
Choose a framework for writing server code (probably NodeJS because that seems to have all the latest buzz around it).
Write a very basic web page (i.e. “hello world”).
Write a basic chat server as a web page.
Tie the server into a local (small) database of users (maybe) and chat info.
Make the service use location information to only work at sunrise.
Make the web-based chat server look pretty on multiple browser sizes (from desktop to mobile).
Choose some framework for creating mobile apps (probably PhoneGap with Build).
Create App version of HelloSun and get into the app stores (for iOS? Android? Etc…?)
?
Profit

(Thanks to "The Underpants Gnomes" for the final phases of this business plan.)

HelloSun: A guide to creating a social networking empire, from scratch, for free, circa 2012.
	T.O.C.	Step 2 ⇒

Code to convert .mp3 or .wav audio file to iPhone .caf format

2010-06-10T19:08:00.001-07:00

iPhone likes to work with audio files compressed with ima4 and wrapped in the .caf format. On OS X the afconvert command will do this, but I needed something for linux. So I wrote some C code to do the conversion I needed: wav2caf.c

wav2caf takes a wav file as input, and outputs an ima4-compressed .caf file suitable for playing on the iphone. For source code, build, and install instructions, download here.

Caveats: This source code expect the wav file to be 22050 Hz and single channel. SOX will convert just about any audio file (e.g. mp3, m4a) to this format with a command such as "sox -r 22050 -c 1 output.wav"

There's not much to the source file. wav2caf.c looks a bit like aifc2wav, because that's what I started with, then reversed it, sort of. Good luck. If you make improvements let me know.

Help us move from Flash to HTML5 (part 1).

2010-05-06T21:22:00.000-07:00

We found out that about 1 million internet users (those using an iPad) are unable to use our http://RadioWeave.com/ social/streaming radio site because it doesn't support Flash.

So we've tried to convert everything to HTML5, to get the same or good-enough functionality out of HTML5. There's no problem with the graphic stuff (HTML and Javascript can move images around just fine, thank you very much), but now we're down to replacing Flash audio with HTML5 audio, and it's giving us trouble.

To make the very best audio experience (under even bad wifi connections), this is what we've been doing with the Flash Sound and SoundChannel classes:

buffer the downloading of MP3 files, a few ahead of time
measure download rates (so we can download hifi or lofi files accordingly)
play during download
measure how much has been downloaded to display that information to the user
allow user to pause/play/pause
allow the user to reposition where in the track to play

With HTML5's <audio> tag and "HTMLAudioElement" we can download an MP3 file and make it play (although even this tends to crash Safari on OSX about as much as our Flash code would if pushed). What we haven't figured out is how to know exactly when buffering has made progress, how much is buffered, how fast it is buffering, when playing starts, and how to restart play from any given position.

We keep hearing that Flash is dead and no longer needed because HTML5 is ready to replace Flash.

So, come on you Flash haters. A little help, please? Let us know (in the comments, so everyone can benefit) how to replace our Flash sound capabilities with HTML5.

Thanks.

Django without cookies but putting the sessionid in the URL

2009-02-26T16:21:00.000-08:00

Django sessions require cookies

Django is written to assume that its client is a browser, and that its browser supports cookies. The "How to use sessions" document makes it clear (here) that this is an intentional design decision that use should be well aware of, especially if you want to break those rules as described here. The reasons for that decision are clear and correct, but were not good for our uses.

The Django cookie requirement was not good for our project

But in our own project, a community-based online radio network called YiqYaq, which must support a wide range of client types, I found more and more reasons to want to use cookies. These reasons were:

Our clients were often not browsers, but different tools. We provide a simple url-based API to most functionality and forcing those clients to understand and support cookies too much work on the client's shoulders. I.E. we want to make our API as easy as possible so more people will want to use it.
A surprising number of early testers (friends who we asked to preview our site and services) where on browsers that did not support cookies for random URLs, usually because that was IT corporate policy at their work, but sometimes because well-meaning any-malware supported that policy.
We were moving do Django from a tomcat implementation, and missed the convenence of having jsession id inserted and used (almost) automatically whenever cookies were not supported.

So, for our project, we did the near-equivalent of tomcat's "jsessionid" for our web pages, and for our non-web clients expect them always to use jsessionid.

Our implementation of "jsessionid" Django version 0.96.2

Note: applies to Django version 0.96.2 -- I've no idea what other versions this works with -- I also have not idea whether this is the best way to accomplish what I wanted, but it's the way that works for me

I won't describe our utility functions for how we do the equivalent of tomcat's encodeURL, so that the sessionid is in the URL whenever cookies are not supported. That is a fairly trivial issue, and is bound to be different on each site. This article is about the primary problem, which is how to pass a string to the client to represent a session, how to get that string back from the client, and how to turn that string into a django session.

1) Create the session_key magic value

Once the authentication has happened, in standard django ways, we use this code to generate the "session_key" string

from django.contrib.sessions.models import Session from django.contrib.sessions.middleware import SessionWrapper ... ... ... request.session = SessionWrapper("") auth.login(request, authuser) obj = Session.objects.get_new_session_object() session_key = obj.session_key new_session = Session.objects.save(session_key, request.session._session, datetime.datetime.utcnow() + datetime.timedelta(seconds=(10 * 60 * 60))) # good for 10 hours

The session_key is then passed on to the client as the magic value to represent this session.

2) Create the session_key magic value

When the client contacts the server, and is not uing cookeis, it must then pass back the magic session_key value to the server as a parameter in its URL (or however else you want to pass it). Turning that magic string into a session so that all your other django code can use it normally is trivial:

request.session = SessionWrapper(sessionid)

what is less trivial is how to write data back to the session, in the same was as the cookie middleware would do it, so that changes to the session are preserved along with the magic cookie string. In other words, the following code must be applied before your server returns to the client or else all the changes it may have made to the cookies will be lost:

# sessions are written to work with browser cookies, but we don't want to work
# with browswer cookies - so fake it
# this code is copied from django.contrib.session.middleware - the difference is
# this code ignores cookies
# If request.session was modified, or if response.session was set, save
# those changes and set a session cookie.
try:
accessed = request.session.accessed
modified = request.session.modified
except AttributeError:
pass
else:
if modified:
print "MODIFIED!!!"
if request.session.session_key:
print "### USE OLD KEY ###"
session_key = request.session.session_key
else:
print "### CREATE NEW KEY ###"
obj = Session.objects.get_new_session_object()
session_key = obj.session_key

print "### AND SO ON ###"
new_session = Session.objects.save(session_key, request.session._session,
datetime.datetime.utcnow() + datetime.timedelta(seconds=(10 * 60 * 60)))
# good for 10 hours

Good luck

That's it. I hope it works well for you. It's been working well at YiqYaq so far. And if you have better ways to do this I hope you let me know.

Converting fom Screenflow .mov to Flash video .flv for our web site.

2009-02-24T18:11:00.000-08:00

Over the weekend I did my first screencast demo for our YiqYaq web-radio site, using Screenflow on Mac OS X. The result was a big .mov file that had a couple of problems 1) it was too darn big, and 2) I could find lots of browser clients that would not play that .mov format. Quick research showed that close to 100% of my audience could play flash videos.

So my problem became, how to turn the Screenflow .mov file to a .flv playing on our web site.

The first approach was to upload it to YouTube. YouTube accepted the video, and converted it smoothly, but the final results was squashed down 360 pixels tall, which is too small to effectively show our demo. (See the too-small youtube version here).

After lots of trial and error, I did finally get to an acceptable embedded flash video .flv file, that I can embed on our web site. Maybe there's a better approach, but these are the steps I took:

How I turned Screenflow into embedded Flash vide on our own web site (as seen here):

1) Record and edit with Screenflow

I recorded the interraction of my browser along with audio. The browser window happened to be set at 778x950 (widthxheight). Editted the result, and did all the other demo stuff.

2) Screenflow Export

Selecting Screenflow Export I chose these settings:

web high best quality
scale 74% to 576x702 (because 702 is short enough to fit on most computers yet tall enough to represent my screencast well, and also because 576 is divisible by 16--which turns out later to be important--and 702 is even; if these requirements had not been met I would have used a tiny bit of letterboxing to make sure they were matched)
enabled motion blur
change frame rate to 30 fps
change vide settings to optimize for streaming
change audio settings for best quality

3) Compress and prepare with Stomp

Stomp is software to compress video. It costs a few dollars ($29). For my purposes I didn't actually use it to compress so much as to reorder the audio/video so that the next phase (ffmpegX) doesn't have the jerky problems it gets from the way the Screenflow outputs audio in .mov files.

For Stomp I used these settings always chooseing best quality versus lowest compression (which actually resulted in a larger file that what I started with, but I think that is taken care of later):

Make my own preset
30 fps
multipass
best compressor quality
audio best

4) Convert to .flv with ffmpegX

ffmpegX is a minimal wrapper around a lot of open-source video software. It's $15 shareware. You can build all those video convert tools from the open source, but for me ffmpegX was worth it because, even though it was an effort to install, it was a whole lot less effort than installing all tools it's made from.

From ffmpegX open the Stomped file and output to FLV format. I used these options:

From options selected "High Quality" and "Two-pass encoding"
video size 576x702
framerate 15 (which is half the framerate 30 used in steps up to this point)
play around with the video bitrate field until you get a value that ffmpegX shows up as green--sometimes it is slow to show this so you need to switch on different tabs to force it to update--in my case I ended up with video bitrate 1400

5) AddFLV metadata with FLV MetaData Injector

The file that results from ffmpegX does not have the metadata (especially for height and width) that the next step (JW Player) is going to need to display the video correctly (without the right metadata it will try to change the video to fit standard sizes). To perform this step I downloaded the free windows tool "FLV MetaData Injector" and run the executable ("flvmdi.exe") from a parallels session. There are other FLV metadata injector source tools you can use and build, but this was the quickest way there that I found.

The result of this step is a .FLV file suitable for any flash player.

6) Create web page and embed on web page with JW FLV Media Player

Many users can play the FLV files directly, but they won't look very elegant. It's also not very difficult to find a lot of FLV players, or even to roll your own. But I was in a hurry and JW FLV Media Player did everything I wanted (at a cost--open source and free for non commercial but €39 for a commercial license on one web domain)

Longtail provides very clear instructions for how to use their tool. Remember to set the width and height to the video size of the .FLV file you've been making, and to add at least 20 pixels to the height if you're going to use their control bar (which is the default).

That's it. Like I said, it may not be the best approach but it's the one I stumbled into and it worked for me (as seen here).

Using Django in command-line scripts

2008-05-24T19:45:00.000-07:00

Over at this page is a reference to "Using Django in command-line scripts". Those tricks, which seem to load in enough of the django environment, weren't working for me in more complext db operations. I made a little change to the setup described there, to force a loading of more of the django environment, and so far all of the operations that work rom within a django web-server can now be called from the command line, at least for me.

In these cases, I'm running the standalone pythong scripts (e.g. foo.py) from 2 directories below where the "manage.py" and "settings.py" scripts exist.

Here's the code that starts all of the stand-alone python scripts running in that directory:


##################################### BEGIN DJANGO PREAMBLE ####################################################################
import os, sys
# Preamble so we can use Django's DB API
django_manager_dir = os.path.normpath(os.path.dirname(os.path.abspath(__file__)) + "/../.." ) + "/";
sys.path.append(django_manager_dir)
from django.core.management import execute_manager
import settings
execute_manager(settings, argv=["blah","validate"])
###################################### END DJANGO PREAMBLE #####################################################################

After that preamble I seem to be able to use any of the django db calls without a problem. Calling execute_manager(.."validate") seems a little silly, since it should be like a no-op, but obviously it does something important for setting up the environment.

FOLLOWUP: By using the suggestions at http://www.b-list.org/weblog/2007/sep/22/standalone-django-scripts/ there's a better approach. Replace the last three lines with these:

from django.core.management import setup_environ
import settings
setup_environ(settings)

iTunes won't recognize iPhone after Leopard upgrade

2008-05-16T12:31:00.001-07:00

I had this problem too. Reinstalling iTunes solved it, and no data was lost.

Installing Java and Tomcat (without apache) on base Linux system (Amazon EC2 Fedora Core 4 image)

2007-11-27T15:25:00.001-08:00

For YIQYAQ we're running Tomcat as the web server, on basic Linux Fedore Core 4 images at Amazon EC2. These are the steps taken to install Java and Tomcat on those systems and get it running. I'm not much of a Linux, Java, or Tomcat expert, so I can't say these are the best steps to take, only that they're what I did and they seem to work.

Get java from http://java.sun.com/javase/downloads/index.jsp, install JRE self extracting file jre-6u3-linux-i586.bin into the root of the linux system, then follow these command from the linux system

mkdir /java
mv /jre-6u3-linux-i586.bin /java/
cd /java
chmod 555 jre-6u3-linux-i586.bin
./jre-6u3-linux-i586.bin
rm jre-6u3-linux-i586.bin

Setup Java environment variables

cd ~
nano .bash_profile
add lines
JAVA_HOME=/java/jre1.6.0_03
export JAVA_HOME
JAVAHOME=/java/jre1.6.0_03
export JAVAHOME
lower down, change from
PATH=$PATH:$HOME/bin
to
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
exit nano, exit shell, and restart shell to get new environment set up

Download tomcat 6.0.14 from http://tomcat.apache.org/download-60.cgi#6.0.14 to the root of your linux system as apache-tomcat-6.0.14.tar.gz, and follow these commands to install it

mkdir /tomcat
cd /tomcat
mv /apache-tomcat-6.0.14.tar.gz .
gunzip -c apache-tomcat-6.0.14.tar.gz | tar xopf –
rm apache-tomcat-6.0.14.tar.gz

Setup environment variables for tomcat

cd ~
nano .bash_profile
add lines
CATALINA_HOME=/tomcat/apache-tomcat-6.0.14
export CATALINA_HOME
exit nano, exit shell, and restart shell to get new environment set up

Test that tomcat is working.

cd ~CATALINA_HOME/bin
./startup.sh
From browser, verify that http://your-web-domain-address:8080/ returns the default tomcat page

Change tomcat to run on port 80 instead of 8080

cd $CATALINA_HOME
cp conf/server.xml conf/server.xml.original
nano conf/server.xml
replace all 8080 with 80, then exit and save
bin/shutdown.sh
bin/startup.sh
From browser, verify that http://your-web-domain-address/ returns the default tomcat page

Setup manager for Tomcat

nano $CATALINA_HOME/conf/tomcat-users.xml
add this user
<tomcat-users>
<role rolename="standard"/>
<role rolename="manager"/>
<user username="managername" password="managerpwd" roles="standard,manager"/>
</tomcat-users>
replace managername and managerpwd with your real, super-duper top-secret values
restart tomcat
$CATALINA_HOME/bin/shutdown.sh
$CATALINA_HOME/bin/shutdown.sh

That's it. All done.

Installing ImageMagick with JPEG and PNG Support

2007-11-27T14:46:00.000-08:00

ImageMagick is a set of command-line tools for manipulating image files. By default it supports GIF but not the other command interweb formats of JPG (JPEG) and PNG. These are the steps I followed to install ImageMagick on an Amazon EC2 Fedora core-4 Linux image.

This process is also described at Installing ImageMagick with JPEG, TIFF and PNG Support, but I had some problems with a couple of the commands there, it was using files older than the current, and it included TIFF support which I don't expect to be using.

The first step was to get the required source files for each of the components required. These are the files I retrieved, and the web addresses were I retrieved them. (Note, the exact version numbers change over time.) Each of the .tar.gz files I downloaded and copied to the /usr/local/src directory of my EC2 linux system.

ImageMagick.tar.gz from http://imagemagick.org/script/install-source.php#unix
libpng-1.2.23.tar.gz from http://libpng.org/pub/png/libpng.html
jpegsrc.v6b.tar.gz from http://www.ijg.org/files/
zlib-1.2.3.tar.gz from http://www.zlib.net/

from my linux image I then performed these commands (and in this order) to unpack, compile, and install the components and ImageMagick.

cd /usr/local/src
gunzip -c zlib-1.2.3.tar.gz | tar xvf -
rm zlib-1.2.3.tar.gz
cd zlib-1.2.3
./configure
make
make install
cd /usr/local/src
gunzip -c libpng-1.2.23.tar.gz | tar xvf -
rm libpng-1.2.23.tar.gz
cd libpng-1.2.23
./configure
make
make install
cd /usr/local/src
gunzip -c jpegsrc.v6b.tar.gz | tar xvf -
rm jpegsrc.v6b.tar.gz
cd jpeg-6b
./configure --enable-shared
make
make install
[may need to create man directories]
cd /usr/local/src
gunzip -c ImageMagick.tar.gz | tar xvf -
rm ImageMagick.tar.gz
cd ImageMagick-6.3.7
./configure
[verify that jpeg and png are on]
make
make install

Now run a simple test to see that ImageMagick is working

cd /
identify –list Format
[verify that jpeg and png are on]
convert logo: logo.gif
convert logo: logo.jpg
convert logo: logo.png
[verify with separate viewer that this images are all OK]

That's it. The ImageMagick command are now ready.

Installing GCC on Amazon EC2 fedora core 4 base

2007-11-27T14:38:00.000-08:00

I'm using the ec2-public-images/fedora-core4-base.manifest.xml image on Amazon's EC2 system. (I don't know how this differs from the develop-image.manifest.xml, but wanted to start with something as basic as possible).

This image comes without a GCC compiler. There's almost nothing you can make out of Linux without a compiler.

Adding a compiler is a piece of cake with this command

yum install gcc

That's it. super easy. You can find more about yum (such as yum update) here.

Portable MP3 in car sounds great via Cassette Adapter

2007-10-01T20:27:00.000-07:00

We recently purchased a new adapter for playing our ipod / iphone in the car: the Belkin Mobile Cassette Adapter. It sounds fantastic, especially compared to the old FM adapter we used to use. Those FM mp3 & ipod adapters require finding an empty band, squashing music, sound yucky, and cost a lot more. This cassette adapter was only about $15 at Fry's. Any quality FM adapter costs a whole lot more and doesn't sound nearly as good.

This cassette adapter worked right off with our old ipod mini. For the iPhone I used a razor blade to trim away the last eighth-inch or so of rubber off the end of the plug, so it fits into the recessed iphone ear jack, and it works great.