The YakShave

Get list of URLs that Google Search has indexed

2015-09-16T12:34:00.002-07:00

Here is a quick and dirty way to get the list of google results for a domain.

1. Do a google search for site:example.com
2. Open your browser's console
3. copy paste:

var elems = document.getElementsByTagName('cite')
for(var i=0; i<elems.length; i++) {
console.log(elems[i].innerText)
}

Mounting S3 on an Ubuntu EC2 intance

2012-08-07T11:28:00.001-07:00

Compile S3FS

sudo apt-get install build-essential libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support

curl -O http://s3fs.googlecode.com/files/s3fs-1.61.tar.gz

tar -xzvf s3fs-1.61.tar.gz

cd s3fs-1.61/

configure

make

sudo make install

setup security

vim /etc/passwd-s3fs

accessKeyId:secretAccessKey

sudo chmod 640 /etc/passwd-s3fs

mount it

Everything a Web Developer needs for OSX

2012-08-04T00:15:00.000-07:00

Setting Up New Computer.

Within the last month received a new MacBook Pro, and had to re-install another. So I decided to list the software I installed.

Browsers

Chrome www.google.com/chrome/
Firefox www.mozilla.org/firefox

FireBug
LastPass lastpass.com/

Command line Tools

Git git-scm.com/
php 5.4 php-osx.liip.ch/
nmap nmap.org/
homebrew mxcl.github.com/homebrew/
Xcode (3.22 for SnowLeopard | 4.4 for Lion)

IDEs

NetBeans netbeans.org/

OSX Java

PhpStorm www.jetbrains.com/phpstorm/
Sublime Text www.sublimetext.com/
GitX(L) gitx.laullon.com/
SourceTree www.sourcetreeapp.com/

DataBase

Other Apps

Minuteur www.phg-home.com/index_mac.html
ScreenFlow www.telestream.net/screen-flow/

Tweaks

Terminal

Change theme to home brew
Add command key
When the shell exists: Close if the shell exited cleanly

Keyboard ShortCuts Prefs

All controls for tab key

Shell Tweaks

edit ~/.profile
function parse_git_branch_and_add_brackets {

git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* $.*$/\ \[\1\]/'

}

PATH="/usr/local/php5/bin:/usr/local/mysql/bin:$PATH"

PS1='\[\e[01;32m\]\u@\h:\[\e[01;34m\]\w\[\e[0;31m\]$(parse_git_branch_and_add_brackets)\[\e[0m\]$ '

alias ls='ls -G'

OpenSSL - Create your own Certificate of Authority and sign your SSL Certs

2012-01-12T23:49:00.000-08:00

Creating your own CA for Signing Dev Certs

Here are some handy commands to create a CA and sign your own cert for development.

The commands are fully scriptable: all key passwords have been bypassed with the -nodes flag, and the interactive openssl prompts not necessary with in the the -subj flag.

tldr; Short version for the Impatient:

# Create Certificate of Authority
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Development/CN=Dev Certificate of Authority"
openssl req -subj "$SUBJ" -nodes -new -x509 -extensions v3_ca -keyout devCA.key -out devCA.crt -days 365

# Create Key and CSR
HOST="appname.local"
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Dev Web Sites/CN=$HOST"
openssl req -subj "$SUBJ" -newkey rsa:2048 -nodes -keyout $HOST.key -out $HOST.csr -days 365

# Create the Cert by signing the CSR with our CA
openssl x509 -req -CA devCA.crt -CAkey devCA.key -CAcreateserial -in $HOST.csr -out $HOST.crt

Add the appname.key and appname.crt to your apache/nginx/IIS config, and add the devCA.crt to your browser.

Long Version.

When developing webservices that will be using SSL in production, I like to use SSL in development.

The old way

The traditional way to do this is create a self-signed cert. Where you sign your Certificate Signing Request (CSR) with the same key that created it. (Thus "self-signed".)

HOST="appname.local"
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Dev/CN=$HOST"
openssl req -subj "$SUBJ" -x509 -newkey rsa:2048 -nodes -keyout $FILENAME.key -out $FILENAME.crt -days 3650  ## look mom! One line.

Then add it to the apache.conf

<VirtualHost 172.16.1.1:443>
   ServerName appname.local:443
   DocumentRoot "/Users/lance/Sites/AppName/src/webroot"

   SSLEngine on
   SSLCertificateFile "/Users/lance/Sites/AppName/certs/appname.local.crt"
   SSLCertificateKeyFile "/Users/lance/Sites/AppName/certs/appname.local.key"
</VirtualHost>

This works great, but now with every new ssl cert I generate I have to add a trust exception in my browser.

A better way

To avoid multiple trust exceptions of you self-signed certs, is to first generate a "Certificate of Authority" (CA), add that CA to your browser, and then sign all of your certs with that CA.

# Create CA
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Development/CN=Dev Certificate of Authority"
openssl req -subj "$SUBJ" -new -x509 -nodes -extensions v3_ca -keyout devCA.key -out devCA.crt -days 3650

# Create Key and CSR (same as above)
HOST="appname.local"
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Dev Web Sites/CN=$HOST"
openssl req -subj "$SUBJ" -newkey rsa:2048 -nodes -keyout $HOST.key -out $HOST.csr -days 365

# Create the Cert by signing the CSR with our CA
openssl x509 -req -CA devCA.crt -CAkey devCA.key -CAcreateserial -in $HOST.csr -out $HOST.crt

# cleanup CSR
rm $HOST.csr

Thoughts

These commands are the shortest way I've found to create keys, CSRs, and signing. Other guides often use three steps (1 genrsa, 2 key export w/o pass, 3 req ) to generate the CSR, whereas it is possible to do it with 1 step.

Hint: By using the -subj flag we can bypass openssl's interactive prompts

Hint: I use Virtual IPs for each https web service I need:
OsX: $ ifconfig lo0 alias 172.16.1.1
Linux: $ ifconfig lo:0 172.16.1.1
windows: http://support.microsoft.com/kb/236869

http://gagravarr.org/writing/openssl-certs/ca.shtml
http://www.g-loaded.eu/2005/11/10/be-your-own-ca/

Fixing my Bash Prompt on OSX

2011-04-06T08:50:00.000-07:00

I have been having a problem with the bash prompt. When I use the up and down arrows to scroll throw the history buffer, the output of the previous commands would get garbled and not line up. After messing around for 20 minutes I found a solution.

My old prompt:


PS1='\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[0;31m\]$(parse_git_branch_and_add_brackets)\[\033[00m\]\$ '

My new prompt:


PS1='\[\e[01;32m\]\u@\h:\[\e[01;34m\]\w\[\e[0;31m\]$(parse_git_branch_and_add_brackets)\[\e[0m\]$ '

Something about changing the \033 to \e works. Why? I don't know, if you do please comment.

BASH Prompt in OSX with git branch

2011-03-07T13:16:00.000-08:00

I found others doing the same.

Here is the content of my ~/.profile.


function parse_git_branch_and_add_brackets {
git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\ \[\1\]/'
}

PS1="\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[0;31m\]\$(parse_git_branch_and_add_brackets)\[\033[00m\]\\$ "

Quickly rename files to lower case

2011-02-19T11:01:00.001-08:00

Here is a simple bash oneliner to rename all folders/files in a directory to lower case

for old in * ; do new=`echo $old | tr "[[:upper:]]" "[[:lower:]"`; mv $old $new;  done;

Getting PHP's pcntl working on Snow Lepard

2011-02-14T10:32:00.000-08:00

(make sure you've installed Apple's developer tools)

1. Check your version you have installed and download the PHP source code.

$ php -v
PHP 5.3.3 (cli) (built: Aug 22 2010 19:41:55) 
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies
    with Xdebug v2.1.0, Copyright (c) 2002-2010, by Derick Rethans

$ curl -O http://us.php.net/distributions/php-5.3.3.tar.gz

2. unpack, phpize, configure, make, make install

$ tar -xzvf php-5.3.3.tar.gz
$ cd php-5.3.3/ext/pcntl/
$ phpize
$ ./configure
$ make
$ sudo make install

3. Add "extension=pcntl.so" to your /etc/php.ini

4. Confirm

$ php -i | grep pcntl
pcntl
pcntl support => enabled

Done

New Year's Resolutions: New Passwords for 2011

2010-12-31T20:20:00.000-08:00

I'll admit it. I've been bad. I'm a password reuse addict. I just love the way the muscle memory makes it so easy to type. And yes, I have a couple variations on the theme, but I've known for a long time that I would have to get clean. The recent gawker compromise has given me the push I need.

A couple of years ago I've started using LastPass. LastPass offers many nifty security features, but I use it b/c I move between computers, and it fills my forms in for me. (lazy.) Well no more! Time to leverage LastPass to it's potential.

Tonight I'm going clean. No more password reuse for 2011.

Windows Re-install

2007-11-15T09:42:00.000-08:00

Whenever I have a fresh install of XP (twice this week) I have a set of software I need to install.

Firefox

Smart Bookmarks Bar
Firebug

WOL

2007-10-14T23:31:00.000-07:00

http://ubuntuforums.org/showthread.php?t=234588

Recover / Reset Linux Password

2007-08-06T09:57:00.000-07:00

Found these concise steps for reseting a password at: http://ubuntuforums.org/showthread.php?t=3609

Just thought I'd write it down for myself, instead of having google it up again.

Q: I am a complete moron and forgot my password. How can I get back into my system?
(Hey, YOU forgot it, I think that gives me the right to do some degrading before letting you back in!)

A:
1. Turn off your computer.
2. Tap it 3 times
3. Say your favorite magic word.
---- If that doesn't fix your problem, do the steps below... #-o
4. Turn your computer on.
5. Press ESC at the grub prompt.
6. Press e for edit.
7. Highlight the line that begins kernel ........., press e
8. Go to the very end of the line, add rw init=/bin/bash
9. press enter, then press b to boot your system.
10. Your system will boot up to a passwordless root shell.
CAUTION: This is a FULL ROOT SHELL! You can damage your system if not careful!
11. Type in passwd . Set your password.
12. Type in reboot.
NOTE: steps 1-3 may help at this point.
13. Bow down to me....

Initial Postgres Setup for Ubuntu

2007-06-26T12:13:00.000-07:00

After installing postgresql, here are the first few things I usually do. I'm usually running Ubuntu, but these would be necessary for any postgres install.

Create database users
$ sudo su - postgres # become the postgres user $ createuser -P lance # I'm usually a super user $ exit # exit out of the postgres user $ createuser account1 ... # Create the standard accounts
Setup conf files
$ sudo vi /etc/postgresql/8.2/main/pg_hba.conf $ sudo vi /etc/postgresql/8.2/main/postgresql.conf
Move Data to separate partition (/data)
$ sudo /etc/init.d/postgresql-8.2 stop $ sudo mv /var/lib/postgresql/8.2 /data/postgresql/8.2 $ sudo ln -fs /data/postgresql/8.2/main /etc/postgresql/8.2/main/pgdata $ sudo /etc/init.d/postgresql-8.2 start
Setup admin pack
$ sudo -u postgres psql < /usr/share/postgresql/8.2/contrib/adminpack.sql

plr for postgres 8.2 on Ubuntu Feisty using Gutsy deb-src

2007-06-26T10:25:00.000-07:00

The only reason I have not upgraded to postgresql 8.2 is lack of a plr package.

After yesterday's dissection of the plr install for postgres 8.1, I decided to create a 8.2 install.

I noticed plr 8.2 is already available for Gutsy (which is still in development at the time of this writing).
http://packages.ubuntu.com/gutsy/libs/postgresql-8.2-plr

So I tried the Gutsy .deb (binary), but it would not install b/c of a unmet gcc dependency. So I tried building it from the deb sources, but plr.c would not compile as is. After hacking around I was able to get it to build.

Using Gutsy deb-src to build a Feisty .deb (binary) for postgesql 8.2 plr

$ wget http://mirrors.easynews.com/linux/ubuntu/pool/universe/p/plr/plr_8.2.0.1-1.diff.gz
$ wget http://mirrors.easynews.com/linux/ubuntu/pool/universe/p/plr/plr_8.2.0.1-1.dsc
$ wget http://mirrors.easynews.com/linux/ubuntu/pool/universe/p/plr/plr_8.2.0.1.orig.tar.gz
$ dpkg-source -x plr_8.2.0.1-1.dsc
$ cd plr-8.2.0.1
$ vi +185 plr.h

Remove the ", R_NilValue" from the end of the line. Line should read:
#define R_PARSEVECTOR(a_, b_, c_) R_ParseVector(a_, b_, (ParseStatus *) c_)

$ dpkg-buildpackage -rfakeroot -b
$ cd ..
$ sudo dpkg --install postgresql-8.2-plr_8.2.0.1-1_amd64.deb

Ubuntu: postgresql-8.1-plr for Feisty missing plr.so and plr.sql

2007-06-25T13:19:00.000-07:00

I am trying to get plr installed. However after running apt-get install postgresql-8.1-plr I get the dreaded:
ERROR: could not access file "$libdir/plr": No such file or directory
I double check the package list for plr

$ dpkg-query --listfiles postgresql-8.1-plr
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/postgresql-8.1-plr
/usr/share/doc/postgresql-8.1-plr/index.html
/usr/share/doc/postgresql-8.1-plr/plr-aggregate-funcs.html
/usr/share/doc/postgresql-8.1-plr/plr-data.html
/usr/share/doc/postgresql-8.1-plr/plr-func-naming.html
/usr/share/doc/postgresql-8.1-plr/plr-funcs.html
/usr/share/doc/postgresql-8.1-plr/plr-global-data.html
/usr/share/doc/postgresql-8.1-plr/plr-install.html
/usr/share/doc/postgresql-8.1-plr/plr-license.html
/usr/share/doc/postgresql-8.1-plr/plr-module-funcs.html
/usr/share/doc/postgresql-8.1-plr/plr-overview.html
/usr/share/doc/postgresql-8.1-plr/plr-pgsql-support-funcs.html
/usr/share/doc/postgresql-8.1-plr/plr-spi-rsupport-funcs.html
/usr/share/doc/postgresql-8.1-plr/plr-trigger-func.html
/usr/share/doc/postgresql-8.1-plr/stylesheet.css
/usr/share/doc/postgresql-8.1-plr/copyright
/usr/share/doc/postgresql-8.1-plr/changelog.Debian.gz

Hmm.. where's the .so???

I check around on the ubuntu packages and confirm that other releases have the the .so file:
http://packages.ubuntu.com/dapper/libs/postgresql-8.1-plr
http://packages.ubuntu.com/edgy/libs/postgresql-8.1-plr

So I file a bug and start looking into a way to fix it.

I search for how to build a package: and come up the official guide.

$ mkdir -p ~src/plr/feisty
$ cd ~src/plr/feisty
$ apt-get source postgresql-8.1-plr
$ sudo apt-get build-dep postgresql-8.1-plr
$ cd plr-0.6.2
$ dpkg-buildpackage -rfakeroot -uc -b 
$ cd ..

The resulting postgresql-8.1-plr_0.6.2-4ubuntu1_amd64.deb still is missing the files. $ dpkg --contents postgresql-8.1-plr_0.6.2-4ubuntu1_amd64.deb

After a lot of poking at the problem I notice that the binary files are created in a directory called install-8.1. However in that dir, are directories for 8.2.

After tracking the problem down, I created a diff file:
diff -u debian/rules /home/lance/rules.patched > /home/lance/rules.patch
And then I uploaded to launchpad.

--- debian/rules        2007-06-25 15:37:43.000000000 -0700
+++ /home/lance/rules.patched   2007-06-25 15:37:23.707021486 -0700
@@ -47,7 +47,7 @@
        dh_testroot

        # 8.1
-       $(MAKE) -C build-8.1/plr -f Makefile.pgxs R_HOME=/usr/lib/R install DESTDIR=$(HERE)/install-8.1 PGXS=/usr/lib/postgresql/8.1/lib/pgxs/src/makefiles/pgxs.mk
+       $(MAKE) -C build-8.1/plr -f Makefile.pgxs R_HOME=/usr/lib/R install DESTDIR=$(HERE)/install-8.1 PGXS=/usr/lib/postgresql/8.1/lib/pgxs/src/makefiles/pgxs.mk datadir=/usr/share/postgresql/8.1 docdir=/usr/share/doc/postgresql-doc-8.1 pkglibdir=/usr/lib/postgresql/8.1/lib

        # install files
        dh_installdocs -a

Download the patch file to your home directory as rules.patch, then do:

$ mkdir /tmp/plr
$ cd /tmp/plr
$ apt-get source postgresql-8.1-plr
$ sudo apt-get build-dep postgresql-8.1-plr
$ cd plr-0.6.2
$ patch -p0 -i ~/rules.patch
$ dpkg-buildpackage -rfakeroot -uc -b
$ cd ..
$ sudo dpkg --install postgresql-8.1-plr_0.6.2-4ubuntu1_amd64.deb

Transfering multiple files with netcat

2007-06-25T10:45:00.000-07:00

Today I needed to backup some files from one server to another. The servers sit next to one another on the network. Usually I use scp or rsync to move files around, but today I felt like trying something different.

First I considered using FTP. However, I do not like running FTP servers and didn't feel like configuring the daemon.

Then I thought of netcat. A quick Google search got me this page: http://www.oreillynet.com/pub/h/1058

However I want to transfer multiple files (all in the same directory). So I decided to use tar, and instead of creating a file, I would pipe it's output through netcat.

On the Receiving Computer (Server):

nc -v -w 30 -p 5600 -l | tar -x

And on the Sending Computer (Client):

tar -c * | netcat -v -w 2 10.0.0.2 5600;

The files transfered fast (2 min 20 seconds for 1.6 Gigs). I ran a md5sum on the client and server to confirm, and everything was perfect.

Guides - To Read

2007-06-24T09:33:00.003-07:00

WEB

http://www.smashingmagazine.com/2007/05/15/best-of-april-2007/

Guides and Tools

2007-06-24T09:30:00.000-07:00

List of Guides and tools I've found on the Net.

(This is like a bookmarks.)

Web Site Administration:

Pingdom's full page test looks very interesting:
http://tools.pingdom.com/fpt/

Breaking Stuff:

A little guide/intro to aircrack
http://www.grape-info.com/doc/win2000srv/security/aircrack-2.3.html
Windows Assembly Tutorials. Great for taking stuff apart.
http://win32assembly.online.fr/tutorials.html

WEBertainment

2007-06-24T09:22:00.000-07:00

It is the results from every MythBusters show.
http://mythbustersresults.com/

First Post

2007-06-24T09:12:00.000-07:00

Another blog. After years of poking at blogging software (Drupal, different Nukes, MT, GeekLog, etc), I've decided not to invest anymore time is investigating software. Instead, my limited blogging time, I should invest in content.

And while I've always know content to be king, I've never practiced what I preach. Instead I usually get caught up in the solving the puzzle of installing and running different pieces of software. And while that is a worth while endeavor, I no longer have the extra amount for those pursuits.

I've started using Google's services for some of my mail hosting, and have been pleased. So, it's about time I tried out blogger.com/Google's blogging software.

So here I come Brave new world of blogging. This blog is primarily a tool for myself, but I hope anything I contribute will be useful for others.