dandean.com

What should this thing be?

2013-08-31T00:00:00-07:00

What should this thing be?

Aug 31, 2013

What‘s the point of this website? Why do I bother keeping and nominally maintaining it?

There are a few reason, all of which range from completely unrealistic to somewhat reasonable. The first and possibly most unreasonable is that I dislike the idea of my content – the web-facing record of my daily thoughts and life – existing soley within an array of proprietary walled gardens:

I genuinely enjoy what these services do, and how they allow me to interact with my friends, family and acquaintances without much effort. The chances that these same people are going to come find “my content” here on my website is so close to zero that I might as well not even consider it an option.

But STILL, jesus fucking christ, I hate that these services essentially own my content, and have it within their means to control who gets to see it, and if it continues to exist into the future. The only reason they choose to maintain a record of my life on their servers is to figure out a way to squeeze a few pennies out of it. Sigh…

So, keeping this in mind – that very few will ever see content posted here, but that it is at least within my full control – what exactly can I do with this space? The ideal format for what is posted here should be in the format of a tumblog:

Images and videos with little or no text
Short thoughts and observations that capture a snapshot of a fleeting moment
Audio snippets captured with my phone’s microphone (with still image?)
Longer form posts, similar to this one, about my life or about specific projects that I’m working on

Is low-friction convesion and publishing possible?

How to update and post to this without while creating as few barriers as possible? I should be able to, for the most part, post to the other services I use and have them automatically get cross-posted to this website. How could that work? Probably the easiest way would be to mark posts on other services in a way that could tell an automated script to pull them down and generate the files necessary to push here.

Content from specific services could be targeted, converted and posted:

Post an image to Flickr with, tagged with “keep”
- Original-size images are pulled down and uploaded to a folder on my website
- Posts are created with the image date and linked to the flickr post
- Image is given any associated caption from the flickr post
Post a tweet that I want to keep
- I DO NOT want to include a fucking hash tag the designates the need for re-posting…
- Other options… a specific ASCII or Unicode character could be included as a marker. Something pretty but unobtrusive:
  - ⥿
  - ⥁
  - ↯
  - °
  - ★
  - ⚐
  - ☉
  - ☇
  - ⚀
  - ◎
  - ◉
  - ⦿
  - ◆
  - ⊙
  - ⨕
  - ⋔
  - ⫓
  - ⩶
  - ⩩
  - ⩪
- The most practical option, though, would be something which is accessible from the phone’s keyboard, but rarely used.
Post a Facebook image or video: like flickr, pull down the file and upload it to this server. Tagging looks like it sucks, so would probably be limited to the above-mentioned ASCII/Unicode character marker in the photo description or title.
Facebook status update: Turn it into a regular text post. Would need to mark it with a special character.
Soundcloud: ???
Pinterest pins… is there really a need here? One possible use case is for a recipe. Save a draft with a link to the original recipe. I could then create a fleshed out version consisting of how I ended up modifying the recipe, photos of the result, and thoughts about how it turned out.

If any post contains geodata, a map should be embedded on the page.

The automatic posting processs

IFTTT periodically checks the various services for new posts
If not IFTTT, a web application could pull individual services, triggered by cron.
All new content is posted to service-specific endpoints on my domain:
- http://asdfasdfadf.dandean.com/flickr
- http://asdfasdfadf.dandean.com/twitter
- etc…
Specific endpoints process content and transform it into a format suitable for this website
New resources are saved and pages are built
Textual markers are removed from content
Where possible, original resources get a link back to this site’s version in the format of a comment or reply
A text message is sent to me with a link to the new post.
The endpoints create the new files, kick of the Jekyll build process, and upload the files (if on a different server) or move them in to place (if on the same server).

Building it

This is the shitty (but really FUN) part. It takes time to build these things. The most realistic thing to do would be to implement one service at a time, starting with the most used. The endpoints themselves could exist as a single application on my own server, or live on a single and tiny application on a cloud provider like EC2 or Nodejitsu.

Separate projects:

Triggered build and deployments of the website
Discovery: service-specific discovery modules
Conversion: service-specific content conversion modules
Transport: sends converted post data to service-specific endpoints
Jekyll Post Builder: builds jekyll posts using service-specific data

Now, we get to see if I actually do this.

JSON Transformer

2013-04-18T00:00:00-07:00

JSON Transformer Is Rad

Apr 18, 2013

The Problem:

Servers and clients often speak in different JSON variants. The two most frequent use either snake_case or camelCase keys.

It can be frustrating to worked with a mixed bag JSON formats. json-transformer helps with this issue by applying a string transformation to the raw JSON string to enforce either snake_case or camelCase formatting.

A Solution:

With json-transformer, JSON can be transformed before deserialization on the way into your application, and after serialization on the way out. This allows you to work with a consistent JSON format within your application regardless of the preferences of the services your application depends on.

Sample Data

var snake_case_data = JSON.stringify({
  "property_one": "foo",
  "property_two": "bar",
});
// "{"property_one":"foo","property_two":"bar"}"

var camelCaseData = JSON.stringify({
  "propertyOne": "biz",
  "propertyTwo": "baz",
});
// "{"propertyOne":"biz","propertyTwo":"baz"}"

Convert `snake_case` to `camelCase`:

var transformer = require('json-transformer');
transformer.underscoreToCamel(snake_case_data);
// {"propertyOne":"foo","propertyTwo":"bar"}

Convert `camelCase` to `snake_case`:

transformer.camelToUnderscore(camelCaseData);
// "{"property_one":"biz","property_two":"baz"}"

Tubbs - Database Agnostic Model Layer for Node.js

2011-12-29T00:00:00-08:00

More in-depth documentation can be found in README on GitHub.

Tubbs is a feature-rich and database-agnostic model layer for you Node.js applications.

Some of Tubbs’ excellent features are:

Rails ActiveModel style property validators, like formatOf, inclusionOf, etc…
Hackable custom async validation
Model extensibility
ES5 getter/setter properties with configurable default values (none of that model.set(...) garbage)
Non-serialized virtual properties
Can be used with any database as long as an adaptor is provided
Current built-in support for Riak and in-memory databases

Example: define a User model module:

var Tubbs = require('tubbs');
var Validate = Tubbs.Validate;
var Regex = require('regex');

var User = module.exports = Tubbs.create({
  // Persist our data with Riak
  dataStore: new Tubbs.RiakStore({ bucket: 'users' }),
  
  // Use the `username` property as the primary key for operations like `find`.
  // Defaults to the `id` property.
  primaryKey: 'username',

  fields: {
    username: undefined,
    password: undefined,
    first: undefined,
    last: undefined,
    email: undefined
  },

  // Instances will have a `name` property, but it will not be serialized as
  // JSON when saved to the database.
  virtual: {
    name: function() {
      return ((this.first || '') + ' ' + (this.last || '')).trim();
    }
  },

  // Add some validation...
  validation: [
    Validate.required('username'),
    Validate.formatOf('username', {
      with: Regex.username
    }),

    Validate.required('username'),
    Validate.formatOf('password', {
      with: Regex.passwordHash
    }),

    Validate.formatOf('email', {
      with: Regex.email
    })
  ]
});

Example: the defined User model can pull data from our database:

var user = User.find('dandean');
user.email = 'whatever@stuff.com';
user.save(function(e, result) {
  if (e) {
    // Could not save, respond with the error messages
    res.send({
      error: e.message,
      errors: user.errors
    }, 400);
  } else {
    // User found, send the user bac to the client.
    res.send({ user: user }, 200);
  }
});

Tubbs can do a whole lot more, and can make your database code pretty concise and easy to work with. Take a look at the repository for more examples and documentation.

fspkg - File System Packager

2011-12-28T00:00:00-08:00

`fspkg` - File System Packager

Dec 28, 2011

Take a look at the fspkg repository on GitHub for more detailed documentation.

Example - package a CommonJS module from a directory of mustache.js templates:

var pkgr = new require('fspkg').Builder();
var result = pkgr.build('./views');
require('fs').writeFileSync('./views.js');

This would create the file “views.js”, and would look something like this:

module.exports = {
  'auth/login.mustache': '<input type="text" name="username" value="" placeholder="Your Username">\n<input type="password" name="password" value="" placeholder="Your Password">\n<button type="submit">Login</button>\n',
  'auth/register.mustache': '<input type="email" name="email" value="" placeholder="Email Address">\n<input type="text" name="username" value="" placeholder="Choose Username">\n<input type="password" name="password" value="" placeholder="Password">\n<input type="password" name="passwordConfirmation" value="" placeholder="Password Confirmation">\n<p>By registering you agree to Awesome\'s <a href="">terms &amp; conditions</a>.</p>\n<button type="submit"><span></span><span></span>Login</button>\n',
  'header.mustache': '\n  <div>\n    <h1><span>Awesome</span> &nbsp;&rarr;&nbsp; <span></span></h1>\n    <nav>\n      <a href="/library/work">Add Work</a>\n      <a href="/library">Library</a>\n      <a href="/profile">Profile</a>\n      <a href="/logout">Logout</a>\n    </nav>\n  </div>\n\n',
  'index.mustache': '',
  'library/collections.mustache': '<header>\n  <h1>Collections</h1>\n  <button type="button">New Collection</button>\n</header>\n<form>\n  <input type="input" name="new-collection" id="new-collection" placeholder="Enter a name for your new collection...">\n</form>\n<ul></ul>\n',
  'library/work.mustache': '<section id="work-media-well"></section>\n\n<div id="work-data">\n  <section id="work-actions">\n    <button type="button" id="save-work">Save</button>\n\n    <button type="button" class="grouped-left">Basic Information</button>\n    <button type="button" class="grouped">Editions</button>\n    <button type="button" class="grouped">Media</button>\n    <button type="button" class="grouped-right" tabindex="-1"> </button>\n\n    <button type="button" id="delete-work">Delete</button>\n  </section>\n\n  <section id="work-info">\n    <input type="text" name="title" placeholder="Title" value=""><span>,</span>\n    <input type="text" name="year" placeholder="Year" value="">\n    <br>\n\n    <input type="text" name="medium" placeholder="Medium" value="">\n\n    <fieldset>\n      <input type="text" name="width" placeholder="Width" value="">\n      x\n      <input type="text" name="height" placeholder="Height" value="">\n      x\n      <input type="text" name="depth" placeholder="Depth" value="">\n    </fieldset>\n\n    <textarea name="text" placeholder="About this piece"></textarea>\n  </section>\n  \n  <section id="work-assets">\n    <h2>Media</h2>\n    <ul></ul>\n  </section>\n</div>\n'
}

Since it’s a CommonJS module, you load it up on the client-side and use it as simply as this:

var views = require('./views.js');
var html = Mustache.to_html(views['auth/login.mustache'], data);

Example command line usage:

fspkg is also a command-line application, so you can use it from any of your programs:

fspkg -s ./views.js ./views

Installation

npm install fspkg

If you want to use the command line utility, add the -g flag:

npm install -g fspkg

Installing Node, NPM and Express on Mac OSX

2011-04-01T00:00:00-07:00

Installing Node, NPM and Express on OSX from Scratch

Apr 01, 2011

Prerequisites

You need to have Git installed. By far, the easiest way to do this is with Homebrew. Follow these Homebrew Installation Instructions.

Now that you’ve got Homebrew installed, use it to install Git from within your Terminal:

$ brew install git

Installing Node

If you don’t have Git installed, use Homebrew to install that first.

Here’s a basic installation guide for Node.

Getting Node’s source code

If you don’t already have a place to store external source code, put it somewhere usefuls, like a “src” directory in your home folder:

$ mkdir ~/src

Now that we’ve got a place to put it, let’s download Node:

$ cd ~/src
$ git clone https://github.com/joyent/node.git
$ cd node

Building Node

You’ll want the latest stable version. Stable version are use even numbers, while unstable dev versions use odd. At the time of this writing, the latest stable release is 0.4.7, so let’s checkout that tag before building:

$ git checkout v0.4.7   # you'll get a note about a detached HEAD state.
$ export JOBS=2         # optional, sets number of parallel commands.
$ mkdir ~/local
$ ./configure --prefix=$HOME/local/node
$ make
$ make install

Telling your system where to find Node

We’ve got to add the node bin folder to your path so that you can easily use node from the command line. Open up your .bash_profile and add Node’s path to your PATH.

$ nano ~/.bash_profile

If you already have a line that starts with export PATH=, just add Node’s path to the beginning. For example this:

export PATH=/usr/local/bin:$PATH

would become:

export PATH=$HOME/local/node/bin:/usr/local/bin:$PATH

If you don’t already have a PATH export declared, just add this to the file and save:

export PATH=$HOME/local/node/bin:$PATH

Testing your Node installation

To make sure this all worked as expected, open up a new Terminal window and run:

$ node -v
> v0.4.7

Success!

Installing NPM

Now, I’m guessing you’d like to install some Node programs, like Express or something like that. EASY! NPM is incredibly easy to install:

$ curl http://npmjs.org/install.sh | sh

This should print out a bunch of stuff, ending in It worked. There may also be a single npm WARN Not installed in blah blah blah... line – don’t worry about it.

Testing your NPM installation

Again, just open a new Terminal window at run:

$ npm -v
> 1.0.1rc4

Another success!

Bash Completion

Once incredible helpful feature of NPM is that implements Bash completion, which makes using NPM quite a bit easier at times. To enable this you’ve got to source NPM’s completion file into your .bash_profile. Open .bash_profile up again, and add this to it:

source ~/local/node/lib/node_modules/npm/lib/utils/completion.sh

Installing Express

Now that we’ve got NPM installed, we can use that to install all the Node.js goodness we can get our grubby little hands on, including Express!

NPM 1.0 is more more opinionated than earlier releases – instead of installing modules globally by default, modules are installed at the project-level. Command-line tools, on the other hand, can be installed globally.

The `express` Command Line Tool

Express is both a project module, and a command line tool, so we’re going to install it in both places. Let’s start by installing it globally:

$ npm install -g express

Now that we’ve got our express command-line tool, let’s use that to create an Express application:

$ mkdir ~/Sites/bitchin-express-app
$ cd ~/Sites/bitchin-express-app
$ express -s -t jade

Done!

Dependency Management

Next we need to specify our application’s dependencies. NPM gives us a clean and maintainable way to do this with the “package.json” file. Create this by running npm init from with the root of your project, and just follow the prompts:

$ npm init

When you’re done with that, add your dependency declarations to the file. Express and Jade are both dependencies for me. This is what my final “package.json” looks like:

{
  "name": "bitchin-express-app",
  "description": "My Bitchin' Express App",
  "version": "0.0.0",
  "repository": {
    "url": ""
  },
  "author": "Dan Dean <me@dandean.com> (http://dandean.com)",
  "directories": {
    "lib": "."
  },
  "scripts": {
    "test": "expresso"
  },
  "engines": {
    "node": ">= 0.4.7 < 0.5.0"
  },
  "dependencies": {
    "express": ">= 2.0.0 < 3.0.0",
    "jade": "*"
  }
}

Dependency management is one of NPM’s excellent features. We can automatically install all of our application dependencies by simply asking NPM to take care of it. From within your project folder run:

$ npm install

You should notice that your project now has a “node_modules” folder. Inside that you’ll see an “express” folder, which contains the Express framework module for use just with your new application.

Running Your Application

We’ve got to create a couple more little files before we’re ready to go: our “layout” and “index” files.

$ touch views/layout.jade views/index.jade

You’ll notice that back when we created our application we passed the -t jade argument to express. This configured our app to use the Jade templating engine.

Open up “views/layout.jade” and paste this into it:

!!!
html
  head
    title Hell's Yeah.
    link(rel="stylesheet", href="/stylesheets/style.css")
  body!= body

And paste this into “views/index.jade”:

h1 This is a Bitchin' Web Applications!
p Am I right?

Now all we have to do is start our fucking rad, Bitchin’ application! From the root of your project run this, the open the http://localhost:3000 in a browser:

$ node app.js
> Express server listening on port 3000

In Conclusion

Hopefully this introduction is enough to get you up and running. There’s a whole lot going on in the community. Keep up to date by following the Node blog. If you want to explore the many modules available for use in your applications, poke around the NPM Module Registry.

Installing Node.js on a Dreamhost Shared Server

2010-04-09T00:00:00-07:00

Installing Node.js on a Dreamhost Shared Server

Apr 09, 2010

UPDATE: It’s worth noting that Dreamhost does not allow long-running processes, so this will not work for most production deployment needs.

First, get the latest Node.js source onto your server. I keep my git downloads at ~/downloads/git.

If it doesn’t already exist, create your download directories and cd into it:

$ mkdir ~/downloads/git
$ cd ~/downloads/git

Then, pull down the latest Node.js source code from GitHub:

$ git clone git://github.com/ry/node.git

Since we’re on a Dreamhost shared server, we have to specify a personal directory for the node binaries to install to. This is because Dreamhost won’t allow us (for good reason) to install whatever we want into location which is shared by every other user on your server.

So, lets create our OWN /usr/local/bin directory:

$ mkdir ~/usr/local/bin

Now that we’ve got that, let’s get into the node Git repo and configure node to use our new custom bin directory:

$ ./configure --prefix=~/usr/local

If everything went well, make should handle the rest!

$ make
> ... a whole bunch of compilation output ...
$ make install
> ... more output ...
> Done!

Now, you need to do one more thing to make node visible to your own scripts: add the new node binaries to your PATH! To do this you need to add the following line to two files in your home directory: .bashrc and .bash_profile.

export PATH=$PATH:/home/[YOUR USER NAME]/usr/local/bin

You’re almost done! Just save both of those files, close your connection, reconnect, and check if you can run node! You should get something like this:

$ node --version
> v0.1.33-221-gff56d63

Hot shit! Now go build some killer Node.js apps!

dandean.com

What should this thing be?

What should this thing be?

Is low-friction convesion and publishing possible?

The automatic posting processs

Building it

JSON Transformer

JSON Transformer Is Rad

The Problem:

A Solution:

Sample Data

Convert snake_case to camelCase:

Convert camelCase to snake_case:

Tubbs - Database Agnostic Model Layer for Node.js

Tubbs is a feature-rich and database-agnostic model layer for you Node.js applications.

Example: define a User model module:

Example: the defined User model can pull data from our database:

fspkg - File System Packager

fspkg - File System Packager

Example - package a CommonJS module from a directory of mustache.js templates:

Example command line usage:

Installation

Installing Node, NPM and Express on Mac OSX

Installing Node, NPM and Express on OSX from Scratch

Prerequisites

Installing Node

Getting Node’s source code

Building Node

Telling your system where to find Node

Testing your Node installation

Installing NPM

Testing your NPM installation

Bash Completion

Installing Express

The express Command Line Tool

Dependency Management

Running Your Application

In Conclusion

Installing Node.js on a Dreamhost Shared Server

Installing Node.js on a Dreamhost Shared Server

Convert `snake_case` to `camelCase`:

Convert `camelCase` to `snake_case`:

`fspkg` - File System Packager

The `express` Command Line Tool