Stazz's Stuff

Would you pay for Facebook?

2009-12-20T22:23:00.002-05:00

It shouldn’t come as much of a surprise that we spend the majority of our waking lives trading resources, participating in several economies. The simplest example of this is the financial transactions we make. We trade a limited resource, money, for other limited resources, good and services. Everyday at work, we trade a very limited resource, time, for money. In addition to fiscal transactions, we constantly trade on social markets. We do what we can to do nice things for others (writing a note to let someone know when we appreciate something, fixing friends’ computers, etc.) in exchange for people being nice to us in return. The nicer you are to people, the more social capital you accrue and the more other people will want to be nice to you. In this case, we are trading on intangibles: we’re trading time/effort for good will, and maybe eventually, time/effort in return. But perhaps the most fascinating exchange we trade on – whether we realize it or not – is with your Internet usage, where we frequently trade a limited resource, information about ourselves, in exchange for services like Google and Facebook.

Google trades us something we want, the ability to give order to the vast expanse of the Web, in exchange for a little bit of ourselves, the knowledge that we seek added to a database that makes Google smarter by the second. As Google becomes smarter (as it learns more about us), we trade on our privacy. Make no mistake, Google and Facebook are not free: you’re paying, just not with money. So, I ask this: would you pay some nominal monthly fee to use Google or Facebook rather than surrender your privacy with each query?

Targeted advertisement is the bread and butter of sites like Facebook and Google. It’s not a secret that Facebook ads are presented to us based on information that we provide in our profiles. For example, if you tell Facebook that you’re over 21 years old and list your zip code, you may start to get ads for bars in your area. If you list drumming as one of your activities, you may start getting ads for the newest, coolest snare drum. If you get an email to your GMail account that references baseball, you may start getting ads for Phillies tickets. If you search the Web for “Hawaii,” you may have ads for travel agents flanking your search results. These instances are examples of what some think of as the constant watchful gaze of “Little Brother.” Whereas Orwell’s “Big Brother” refers to government-to-citizen surveillance, this younger sibling refers to business-to-consumer surveillance – capturing information about customers for use in generating revenue.

But what if you’re uncomfortable with all of your information being sold to advertisers? After all, most of the info that we post to Facebook is password-protected, intended for friends and family. We consider our search queries to be private (even though they are not), a map of our thought processes and the information that we seek. What if we don’t want this information harvested and used in commerce? Well, then someone would have to pay for all of these “free” services that we consume. Someone would need to compensate Facebook and Google for their server resources and staff and perpetual innovation. That someone would still be us, except if we didn’t pay with the slow erosion of our privacy, we would have to pay in good old cash.
So, the question becomes, would you do it? Would you pay $5/month for Facebook and $10/month for Google in exchange for preserving some degree of privacy? You probably pay for Netflix now (am I being presumptuous? I assume that most people have Netflix). Conversely, would you take your Netflix for free if all of the movies you watched were cataloged and used to sell you DVDs? I suspect that in the future, we will have to make a choice.

Teaching Traceability - When RO is RW

2009-12-02T21:14:00.002-05:00

I introduced the students in my class to Google Analytics. The purpose of this exercise was not to demonstrate how to properly "monetize" (oh, man, do I hate that word) a site, but rather to demonstrate the breadth of information harvested, even when we just visit a site.

Analytics, using a small bit of Javascript code embedded in the website a user visits, gathers information such as the city/state/country of each visitor, the search terms they typed into their search engine to get to your site, the network they came from (e.g. which university's Internet connection they are using, Comcast, etc.), the amount of time they spent on the site, the way they got to the site (Facebook link, link from another web page, etc.). In other words, it harvests a lot of information about a site's visitors, potentially personally-identifiable information. The interesting part: most sites run this code, and you don't need to post any information about yourself to have information taken.

For me, being a datahead, I like to think in terms of read-only (RO) and read-write (RW) when it comes to information. In his book Remix, Lawrence Lessig speaks about RO culture vs. RW culture. He uses it to juxtapose the old paradigm of cultural consumption (listening to music, watching movies, reading text) - RO - with the modern process of collaboratively creating culture by using technology to create music, videos, and text that build upon other works (remixes) - RW. I know that this is technically inaccurate, but I often think of the old Web, Web 1.0, as being RO. Sure, anyone could post to it with a text editor (or Netscape Communicator) and an FTP client, but you had to know some HTML have have access to an animated GIF of a guy digging. :) That is, content was created by a small, select group of nerds. I think of the modern, Web 2.0 culture of being RW - it's trivial to write information to the web by filling out a form (blogs, comments, Facebook), no knowledge of the workings of HTML required. Therefore, it's more open, more egalitarian; anyone can contribute without having to be a tech guru.

That said, we often think of posting information to web to come from RW activities - updating Facebook/Twitter, posting to a blog, tagging a picture, etc. So, I find it both interesting and a little scary the amount of information we "post" to a site just by engaging in an RO activity, browsing web pages, classic Web 0.1. In a standard session of browsing the Web, you leave this information on every site that you visit, possibly more information than you intentionally post to your Facebook site - your current location, the terms you search for, etc. At least on Facebook, you consciously choose which information to post.

So, I guess the moral of the story is that everything you do on the web is RW. We constantly leave a breadcrumb trail as we visit sites, usually without knowing we're doing so. I'll show you how to leave less of a trail tomorrow.

Until then, I'm off to check my Analytics to see how you got to my site...

Higher Education "Opportunity" Act

2009-11-30T22:13:00.005-05:00

One of the most egregious manifestations of RIAA corporate lobbying has to be the new burdens placed on universities to enforce copyright law. You see, last year, the Department of Education created the Higher Education "Opportunity" Act, amendments to the 1965 Higher Education Act, which sets the guidelines for a university to receive federal financial aid for its students (correct me if I got this wrong). Somehow, skillfully woven into the document, among requirements for federal aid (the same aid that allows most students to attend college), some new, unexpected requirements came forth from the aether. They require that the university:

A) "has institutional policies and sanctions related to copyright infringement and civil and criminal liabilities students may face for unauthorized distribution of copyrighted materials which includes unauthorized peer-to-peer file sharing and the prohibited use of the institution’s information technology system for those activities"

B)" will, to the extent practicable, offer alternatives to illegal downloading or peer-to-peer distribution of intellectual property, as determined by the institution in consultation with the chief technology officer or other designated officer of the institution "

C) "[will regularly provide] a description of
institutional policies and sanctions related to copyright infringement and civil and criminal liabilities students may face for unauthorized distribution of copyrighted materials which includes unauthorized peer-to-peer file sharing and the prohibited use of the institution’s information technology system for those activities"

D) will aggressively serve DMCA cease-and-desist warnings to students

OK, let's try to set ire and pathos aside for a minute and analyze this like rational human beings.

(A) places a burden on the university's IT department to deploy some tech solution to hunt down those "stealing" from the RIAA and the MPAA, thereby incurring significant cost and become vigilantes. Now some lawyers interpret that (D) can satisfy the requirement in (A). Serving DMCA warnings is nothing new for most universities, but, I can assure you from firsthand experience, the cost is not trival - many large universities can use more than an FTE on this (so figure almost 6 figures per annum, counting benefits). Also, what other industries are required to deploy these tech-based deterrents?

(B) effectively requires that universities, in addition to expending the cost of (A) and (D), look into solutions where we can pay the RIAA and MPAA to borrow their content (note the intentional and accurate use of "borrow"; no one would own it but them). Going this route would have one of two effects: sustaining even more of a financial burden or passing it along to students (as is the case in many new pooled models).

(C) requires us to teach a moral lesson. It reminds me of the debate where Jack Valenti, former head of the MPAA, asked Stanford University, after a student admitted to downloading music, "What are you teaching these kids? What kind of moral platform will sustain this young man in his later life?" (Lessig, Remix). And, what better way to teach morals than side by side with the creators of the DMCA? Seriously, though, give me another example of a case where a university is required to regularly attempt to "teach" students about not committing a crime.

So, let's say you think these clauses are garbage. The cost of noncompliance? The university losing all federal financial aid for its students. That is, taking away access to education for students who need assistance in paying for it.

Give me one good reason why these clauses belong in an act relating to federal funding of education - an act that parades under the moniker of "opportunity."

Tor - Internet Proxy for Preserving Anonymity

2009-11-29T18:13:00.002-05:00

In my previous post I promised to explore some ways that we can maintain anonymity on the Web, even in light of the numerous ways personal data is mined everyday. The key piece of personally-identifiable information that we scatter across the web (frequently without any thought) is our IP addresses. Almost every time you visit a website, your IP address is logged - a digital record that you have been to that site, that you have requested to view its content. It's trivial, in most cases, to find out who your Internet service provider (Comcast, Verizon, Time Warner, Qwest, etc.) is with this IP address. Someone who wants to bring a tort case against you or a governmental agency interested in you for any reason can subpoena your ISP for your identity - recall the link in my last post about National Security Letters. Therefore, if you want to maintain some semblance of privacy as you browse the Web, you cannot be leaving y0ur IP address at every site you visit.

One solution to obscuring your IP is using a web proxy, like Tor. When you use a proxy, all of your Web traffic passes through the proxy - that is, the proxy is the only server you connect to. Then, the proxy connects to any site you wish to visit on your behalf. Thus, if you were using a proxy and if all visitors to this blog were subpoenaed, your IP would not be there - just the IP of the proxy, who likely keeps no logs, and thus could not comply with a subpoena.

Tor is a good, free proxy service. The technology was originally developed by the U.S. Navy, and it is more complex than your average proxy. Tor bounces your Internet traffic to multiple relays all around the globe before connecting to your intended destination. You can learn about the Tor Network and the values it stands for here. To get started using Tor, you'll have to install a Tor client. I use a Mac, so I installed Vidalia.

Once you've got a Tor client installed, you can use it to connect to the Tor network. This is only half of what you need: you'll need to tell any programs - Web, torrent, IM, etc. - that you want to anonymize to use Tor. For Firefox, it's easy. Just install the Tor add-on. The nice thing about about it is that you can enable and disable Tor on the fly - you wouldn't want to keep Tor on all the time because it is very slow.

I just installed Tor recently to see what it's about. I notice that it has a few downsides:
1) it's very slow
2) many web pages detect your location (using your IP) and render the page differently depending on where you are coming from. While using Tor, web sites will detect your location as the location of your Tor proxy. Therefore, I noticed that several web pages were displaying themselves to me in German. (This goes to show how much information about you is used to determine your browsing experience.)
3) I read that some employers may not appreciate your use of Tor at work.

Anyway, for me, at least, the hassle of using Tor outweighs any benefits it serves. I like the ideology of Tor more than I actually find it useful to me. :)

Of course, I would be remiss not to note that there are certainly downsides to the proxy methodology: like many good ideas, it can be abused by those who seek to perform nefarious actions under the cloak of anonymity.

The Value of Anonymity

2009-11-28T14:53:00.004-05:00

This semester, I have been teaching a class titled "Privacy in a Networked World." The class explores the evolving expectation of privacy with regard to the Internet, in addition to areas where the right to privacy and freedom of speech come into conflict. On a related note, one of my research interests lately has been the enforcement of copyright/intellectual property laws vis-à-vis digital content - DMCA, etc. These two areas of study recently intersected for me as I was doing research on how copyright-enforcement entities frequently flirt with violating the privacy of their "suspects."

I began reading about Bit Torrent (a protocol for sharing files, sometimes copyrighted), with which, believe it or not, I have no firsthand experience. In the process, I read a post at Lifehacker about preserving anonymity online using Internet proxy servers. Apparently, the RIAA, MPAA, and other copyright-enforcement cartels actively monitor torrent users to locate their next victims. Given this, it's reasonable to assume that, if these entities track our activities online (thereby violating our privacy), there are others (governmental, commercial [think targeted ads], etc.) doing the same. So, the question becomes, can we preserve privacy online or do we give it up and accept the fact that "privacy is dead" and "get over it?"

First, in order to ward off any "if you've got nothing to hide..." arguments, let's take a step back and consider the value of privacy and anonymity online. Consider journalists in countries with staunch commitments to censorship - China, Iran, etc. These countries frequently intimidate and incarcerate (or worse) their political dissidents, usually for nothing more than what we would consider exercising basic, American First-Amendment rights. If disclosing the actual events that led to the Tiananmen Square incident could earn a Chinese journalist almost a decade in prison, there needs to be some kind of mechanism to ensure anonymity, and therefore freedom of information dissemination, online.

It’s certainly true that there are numerous tales of anonymity protecting free speech, like those of the Chinese journalist and the obvious example of Twitter’s role in protests of the Iranian elections. However, it’s important to also consider the right to privacy itself and what it is meant to prevent: a person being judged out of context. For example, if one has an interest in the Manhattan Project, he should be able to pursue his curiosity without his search queries for “atomic bomb” being construed to mean that he is attempting to build one. One mustn’t do much work to find cases of clandestine subpoenas from the Department of Justice seeking the identities of all visitors to a particular website.

In my opinion, while I see privacy online as a waning right in this era of DMCA and the Patriot Act, there are some actions we can take to prevent privacy violations from occurring – in addition to supporting the ACLU and Electronic Frontier Foundation in fighting injustices. Over the next couple of posts, I will outline some ways we can preserve anonymity and limit the privacy violations that have become commonplace.

Why my Blackberry Tour is awesome...

2009-07-29T19:38:00.002-04:00

I can recall discrete points sprinkled over the timeline of my existence where I knew that something that I encountered would be life-changing. Examples: when a grade school teacher told me about a device called a "modem," the first time I opened yahoo.com in Netscape Navigator 1.0, the first time I heard of a DVD player, when I first met Noell, etc. (Noell, you should be honored to be listed among these other items :) )One less dramatic instance of this phenomenon was when I upgraded to my new Blackberry Tour.

Before upgrading, I had two iterations of the Palm Treo (650 and 700), which I loved. It was tough to make the move away from the Treo, but when my contract was up, I decided to give Blackberry a try. The verdict: why did I wait so long?

The Tour is awesome. And I will tell you why:

1) Form-factor - the Treo was one of the best phones I have ever used from a perspective of form-factor. I could walk around and use it with one hand and it had a tactile keyboard. The Blackberry is *almost* as good. The keyboard is great (nice textured keys) and I can use it with one hand, but I often get the urge to tap on the screen to select something - the Tour does not have a touch-screen.

2) Applications - I will do a separate blog post about some of the apps that I like, but here are the highlights:
- Google Talk
- AIM
- Google Maps
- UberTwitter
- Facebook
- Bloomberg
- Pandora

3) Message aggregation - All alerts (IM's, email from different accounts, Facebook alerts, etc) are visible in my messages screen. This weirded me out at first, but I have grown to love going to one place to see all alerts.

4) Battery life - I can get about 36 hours of good use out of a charge

5) The browser murders the Blazer web browser that haunted the Treo

Things I don't like:

1) BIS email - I hate that I can't set the interval of mail checking. It kills me that my email shows up around every 10 minutes or so. This will hurt my aspiration to keep up an email response time similar to my friends Al and Ira (around 2 min, on average [but it's only that high if you average in sleeping time]).

2) That it doesn't have or that I can't find speakerphone. I need that for conference calls.

3) That I cannot find a Yelp app for it

4) That the trackball thing can be a little unresponsive sometimes

5)And the biggest one, that Verizon attempts to charge monthly fees to use the device's built-in GPS and visual voicemail.

Overall, it is a great phone and I will certainly be posting more about it. I *highly* recommend that Tour if you're looking for a new phone.

Oh, man, I cannot tell you how much I love being able to Twitter anything that I find interesting as soon as I think of it (I live in perpetual fear that others will be deprived of my constant stream of clever thoughts). And, how much I love being able to IM while walking between buildings. And, how much I love listening to Pandora at the gym while answering email. And,...

--Chris

Priceline and BetterBidding

2009-07-10T12:41:00.002-04:00

The Web, and the information it renders accessible, have a near-infinite amount of applications, the greatest of which is facilitating cheapness - er, frugality. With just an Internet connection and a little savvy, your run-of-the-mill cheapskate can grow into the uber-miser (the miser meister?). Today, I am going to give a quick lesson on one of my favorite methods to save a bundle.

I am big fan of traveling. There is not much I would rather do than pack up and go on an impromptu trip with the wife. Unfortunately, one of my least favorite things is paying for said vacation. Enter priceline.com. Priceline has been around since the time of the Stegosaurus, so you have probably heard of it. In addition to more traditional methods of booking hotels, flights, etc., they also have their signature model. Let's take a look at this model with regard to hotels.

In the Priceline model, you select the city you would like to go to, the dates when you want to go, the section of the city you would like to stay in, and the level of hotel (up to 5 stars) that you would like to stay at. Then, you enter the price you would be willing to pay for a room meeting the aforementioned parameters and enter your credit card info. If Priceline accepts your bid, your card is automatically charged and the hotel where you will be staying is revealed. If your bid is not accepted, you have to wait 24 hours to bid again or change your search parameters. This is to prevent you from starting your bid at $1 and incrementing by $1 until you find the sweet spot.

Pretty simple. But, how does one know that he or she is getting the best deal? It KILLS me to think that I got a good deal and to later find out that someone else got a better one - kills me.
OK, so we need some help.

Enter BetterBidding.com. BetterBidding is a simple discussion board where you can search for a city and find see what others were able to bid successfully for hotel rooms. It has three real uses: 1) to attempt to predetermine what the mystery hotel is, 2) to find out what others were able to successfully (and unsuccessfully) bid for a room, and 3) to brag about the deal that you got so that others can attempt to replicate it.

Give it a try. Go to betterbidding.com and search for some city. Check out the posts and you will get a better idea of what I mean.

Now that we have a rough idea of the range of successful bids and the possible hotels we could be dealt, we are in a better position to make a bid on a room. I usually try to adjust for day of week, season, and these days, for the sagging economy.

Recently, I was able to get the Sheraton Centre in downtown Toronto for $50USD/night (usually over $200) and the Hyatt in Buffalo for $40USD/night. Not bad.

In conclusion, I believe that some guy said that "variety is the spice of life." Rubbish. Competition is the essence of life and what better way to compete than to save a bunch of money and get to brag about it all in one fell swoop? I highly recommend that you attempt this method if you're going on a trip and you don't care at which specific hotel you stay. Obviously, this wouldn't be preferable for traveling to a conference where you want to stay at the hotel where the conference is being held.

Good luck, and let me know if you are able to score any good deals!

--Chris

NOTE: If you know of other innovative ways to cheap out, I would love to hear about them - really, please contact me ASAP. I could be wasting money as we speak!

Borrowing CD's

2009-07-06T10:32:00.002-04:00

Good morning, everyone. I now fully awake and blogging from the balcony again. I saw an interesting ad on the way to the shore on Saturday. There was a little CD store with a sign offering "CD test-drives." You could, for $4, "borrow" a CD for 15 days and then either purchase it or return it.

Now, here is why this is interesting. I believe physical media to be dead, or at least on its deathbed awaiting its inevitable fate. I hate the idea of having put a physical disc into a media player to listen to music or watch a DVD. I find it to be the definition of primitive. So, what does one do if one runs a business predicated on an antiquated technology? Let people borrow CD's, take them home, and rip them to digital form for less than the cost of buying the album on iTunes.

Sure, this isn't what is explicitly advertised - but I'm pretty sure the intent is painfully obvious. Right, it's not legal - and I'm not condoning or endorsing it with this post. I am just saying that I understand the thinking behind it, it's clever, and it's interesting.

If we compare this model to other methods of obtaining music, it seems to do pretty well when graphed on the risk-to-reward Cartesian plane, mainly because there is about zero chance you will receive a pre-litigation letter from the RIAA.

Anyway, just thought you might find this interesting. (Again, I'm not endorsing this model so don't comment about how I am a jerk - well, to clarify, you can call me a jerk, but not for this reason.)

--Chris

Smcfancontrol

2009-07-05T08:26:00.002-04:00

I have a first-generation Macbook Pro (2006) and it has always run pretty hot. The underside gets pretty warm and the thin strip of casing between the keys and the display could cook a strip of bacon.

This hasn't been a huge inconvenience for me until recently, when I had to start running Second Life on my laptop for a work project. The problem with SL is that it is extremely graphics intensive, so the laptop reaches temperatures comparable to a solar flare. One of my friends told me about smcfancontrol, an application for the Mac that lets you control its fan speed. So, you can actually increase the fan speed to cool your computer faster, if need be.

Now, I am not usually a fan of overriding hardware controls, but in this case there definitely seems to be something wrong with how the fans are tuned. I usually only override fan speed when I'm running SL and the temp gets up around 70C.

One of the cooler things, I think, about SMC is that it displays the current internal temperature of your Mac, along with the current fan speed, in the menu bar. So, you can actually see what's going on inside your Mac.

Anyway, if you have a Mac that tends to run hot, this app might be of use. However, I make no claims of knowing the effect of using such a tool on your warranty. :)

I'm back (again)

2009-07-02T13:31:00.002-04:00

I figured, given that it's been almost a year since I have updated my blog (!), that I should update it before some sort of Blogger inactivity script shuts down my account. I'm getting ready to teach a course on how technology affects privacy and reputation ("Privacy in a Networked World").

While working on my syllabus, I came across a great quote in the book that I am using ("The Future of Reputation" by Daniel J. Solove). This quote is from Perry Barlow's Declaration of the Independence of Cyberspace. I love it:

"Governments of the Industrial World, you weary giants of flesh and steel, I come from Cyberspace, the new home of the Mind. On behalf of the future, I ask you of the past to leave me alone. You are not welcome among us. You have no sovereignty where we gather.

We have no elected government, nor are we likely to have one, so I address you with no greater authority than that with which liberty itself always speaks. "

Here's the full text: http://homes.eff.org/~barlow/Declaration-Final.html

I think that is one of the more powerful pieces of writing I have seen in a while. I can't wait for the class to read it.

Anyway, good to see you again, and I will really try to keep the blog updated.

--Chris

Retrieving Your Lost Office Product Key

2008-07-23T07:35:00.002-04:00

Someone asked me recently what to do if you have a copy of Office installed on an old, ready-to-be-decommissioned computer and want to move the software to your new box. Problem: you have lost the installation product key.

Interesting question. This has never really been an issue for me at work because we have a volume license (and therefore, all copies use the same key), but I could see how this could be a problem for a home user. So, I did a little research...

Let me start out by saying that if you own an "OEM" copy of Office (one that came with your computer), the license is tied to the hardware - you cannot install it on another machine. But, if you bought a boxed copy of Office, you are entitled to install it on another machine, provided you remove it from the old one.

OK, so we're established that you're allowed to move the license to a new box. Now what? We've lost the product key and won't be able to install it. You have two avenues from this point:

1) Contact MS and ask them for a new product key

2) Use a software package to retrieve the original product key from your old computer

The former case is pretty self-explanatory. For the latter case, you could check out Magical Jelly Bean (yes, it's really called that). MJB is an open-source package that will retrieve your original key for you.

Note: I assume that using such software does not violate any software laws, but I have no idea as to the degree of legality of this program. Just a friendly disclaimer... :)

Anyway, hope you find this info useful.

--Chris

P.S. Thanks to the other Chris for telling me about MJB.

Link Evaluator Plug-In

2008-07-21T21:43:00.002-04:00

I had an email forwarded to me recently pointing out an add-on for Firefox called Link Evaluator. I installed it and was really pleased with it. It allows you to do a quick spot-check of all of the links on any given page to check for broken links. You just navigate to a page, right-click on the body, and select "Evaluate Links." You'll then see every link on the page turn either green (good) or red (broken). That's it. Simple concept, useful tool.

Obviously, this wouldn't optimal for large-scale link checking, but for a few pages here and there, it is a nice addition to the toolbox.

--Chris

Google Alerts

2008-07-19T14:51:00.003-04:00

I think I may be a little behind the early-adopter curve on this, but I just started using Google Alerts. Alerts basically monitors a given Google query and emails you when there are new pages about that topic. For example, if you want to know as soon as Google indexes a new chili recipe, you could set up an alert for "chili recipe" and you would get an email whenever Google finds a new one. Super easy.

You can even set the frequency with which you receive alerts - daily, weekly, real-time. And, you can choose to receive your emails in HTML or plain-text.

I'm ashamed to admit that I can be a little techno-narcissistic and *blush* tend to Google my own name regularly to see if anyone is referencing my blog or has written anything interesting about me. Google Alerts automates this for me! OK, enough about that...

Anyway, if you're doing research on some topic, this could be a great tool to learn about the latest sources for information on said topic.

--Chris

Vi Assistant - Hilarious

2008-07-19T14:39:00.002-04:00

If you've ever used the good ol' vi text editor in Unix, you will certainly appreciate this:

http://www.linuxhaxor.net/2008/06/24/vi-assistant/

Thanks to Chris L. for sending.

--Chris

Finding all file types in a directory tree

2008-07-17T22:19:00.002-04:00

So, this is going to be a pretty nerdy post - please avert your eyes if you're not into getting uber-geeky.

Today, I was working on a web migration plan and I needed to know all of the distinct file extensions that existed in our web directory. To do get the answer to my question, I executed a pretty sweet command that I learned from Vince, Master of the Unixverse.

find /www/my_folder | awk -F. '{ print $NF }' | sort | uniq -c

Here's a quick analysis of this command:

"find /www/my_folder" - lists all of the files, recursively, in /www/my_folder and all of its subdirectories

"awk -F. '{ print $NF }' " - get the file extension from each file (everything after the last dot in each file path)

"sort" - sort all of the resultant extensions alphabetically

"uniq -c" - list each file extension only once and print a count of how many times it occurs

Yes, this is a pretty oblique, archaic post, but you may need it someday!

--Chris

Proofpoint Spam Filtering

2008-07-17T21:42:00.003-04:00

One of the most amazing products I have seen in a long time is the Proofpoint spam filterieng appliance. I've been one of the first to use it before we roll it out to our user base, and I have to say that the result has been amazing. I literally get zero spam now that I am using the appliance to filter my mail.

It's actually a pretty novel concept as far as spam filtering goes. All mail passes through the Proofpoint appliance, which lives upstream of the mail server. The appliance grabs any messages it considers to be spam and passes the rest along to your inbox. Then, at some set interval (e.g. daily), you get a "spam digest," an email message delivered to your inbox listing all of the messages (subject line, sender, etc.) that the appliance caught. Next to each message listed in the digest, there are links which allow you to take an action on that specific message. For example, you can click a link to "release" a message, which will tell Proofpoint to let go of the message and deliver it to your inbox. You can safelist the sender of that message so that sender's messages will never be marked as spam again.

In addition to the digest, you can log into the web interface anytime and see the messages that Proofpoint has caught. From the web UI, you can perform any of the actions you can from the digest.

This model is a really nice way of dealing with spam, but the best part of this product is really its accuracy. I have experienced very few false positives and have been very satisfied with the product overall. I can't wait to roll this out to our users; they are going to love it.

--Chris

Text Editors with FTP Integration

2008-07-15T07:34:00.003-04:00

Even though it may be hard to believe in this world of blogs and wikis, sometimes one wants to create one's own web site from *gasp*....hand-written HTML! Yes, it still happens - and for the nerdiest among us, it provides a sense of great joy to be able to forge a web page using nothing but the simplest of tools, a text editor and an FTP client. But, I've discovered a new luxury that I believe provides simplicity, yet retains the coveted nerd cachet, the text editor with built-in FTP capabilities!

So, I remember in the old days (like 4 months ago) when I used to have to open my FTP client, download an HTML page from the server, edit it on my computer, and the re-FTP it back up to the server. In fact, I would usually edit my pages right on the server using "pico" so as to avoid this whole process. That us until, I discovered TextWrangler. TextWrangler is just for Mac, but in the process of helping my wife create the web site for her new dance studio, I came across PSPad for Windows.

UPDATE: I should have pointed out that PSPad does not support SFTP - TextWrangler does. Another text editor for Windows, JEdit, does provide SFTP. Thanks to Warren for bringing this up.

Both applications FTP clients built-in so that you can open files stored on a server (running FTP) just as easily as you would a file that resides on your local machine. For example, you could browse to the server, open an HTML file, edit it, and just click save. It will save back up to the server and you are done!

This technology has not only saved me a bunch of time, but has made working on web pages easier by providing a nice, syntax-colored environment within which to work. So, I would highly recommend installing one of these on your computer if you do any web work (or even write server scripts).

--Chris

Bit.ly - Like TinyURL, but Better

2008-07-09T21:17:00.002-04:00

I came across Bit.ly on Lifehacker today and probably would have passed right over it if it weren't for the fact that it helps to solve a current problem we are seeing. It was recently brought to my attention that Spamhaus has started blocking some emails that contain links to the blogspot.com domain because some spammers have setup shop on blogspot and sent out URL's linking to their sites. (Editorial on blocking the blogspot domain: Ridiculous. Here's an idea, let's block the gmail.com domain because some spam has originated from it.) Anyway, sorry for the tangential rant here.

The point is that we had to figure out a way for someone whose blog resides in the blogspot namespace to be able to send out emails referencing his scholarly blog posts. My first thought was TinyURL. It would provide a good wrapper for him to get his links through. But, then I learned about Bit.ly.

It functions exactly like TinyURL does, but it provides stats and tracking for your links. So, if I create a bit.ly link to one of my pages and send it out to a bunch of people, I can actually see how many people clicked the link! Sweet. I am a major stats junkie, so this is right up my alley.

--Chris

Searching your command line history (ctrl+r)

2008-07-09T20:57:00.002-04:00

Recently, Warren showed me a trick that has saved me a ton of time. I frequently use Unix/Linux/Mac Terminal and there are some commands that I execute frequently - ssh connections to various servers, etc. I usually just hit the up arrow on my keyboard until the command that I want appears. Sometimes this is completely nonsensical because I find myself pressing the up key more times then there are characters in the command I want to execute!

But, Warren recently told me about using the Ctrl+R key combination to search my command history. It does a "reverse search" of the commands you have recently executed. So, for example, let's say you have recently executed the command "ssh username@superlonghostname.com." If, at the prompt, you hit ctrl+r and start typing "ssh...," it will auto-complete the rest of the command and you can just hit enter to execute it.

Anyway, if you use the command prompt a lot, I highly recommend this shortcut.

--Chris

Union Square Green Market

2008-07-07T21:02:00.002-04:00

I know this post is a little off topic for this blog, but I have to tell you what a great experience I had at the outdoor farmer's market at Union Square in NY this weekend. I was up in NYC for the holiday weekend to hang out and we visited the farmer's market at Union Square (17th & Broadway) en route to the Chelsea Market (home of the Food Network).

To make a long story short, it was incredible. All of the produce is locally farm grown and beautiful. And, to add to that, it is all very reasonably priced, some of it even cheaper than supermarket fare. I got a huge bunch of leafy, flowing broccoli rabe ($2), a bunch of carrots so fresh they still had dirt on them ($2), baby fennel ($3), etc.

I've been on a pretentious cooking spree all week. I enjoy telling my wife that for dinner we're having "sauteed broccoli rabe over fettuccine finished with a shallot butter." :)

The market also had eggs from real pasture chickens (i.e. old MacDonald's chickens rather than McDonald's chickens), organic milk, cheeses, meats, and seafood (one guy was selling diver scallops the size of coffee saucers for $15/lb!).

Anyway, if you find yourself in NYC anytime soon, I *highly* recommend you check out this market.

--Chris

Addendum to Choosing Your Seat On a Plane - Southwest Extension

2008-07-04T08:33:00.002-04:00

So, I've gotten some positive feedback on me previous post about choosing your seat on a plane, and I would like to present an extension to this theorem - The Southwest Postulate.

For those of you who do not know, Southwest Airlines has a boarding schema different than other airlines. With SW, you are not assigned a specific seat on the plane, only a board class - A through C. For example, if you have an A-class ticket, you are in the first cohort to board the plane and may choose any open seat. B-class boards second and can choose from any of the remaining seats, and so forth.

You can see from this model that our original theorem does not apply. But, never fear; I think there is a solution.

1. It's pretty important that you secure the A-class ticket. This is not required, per se, but will lead to a significant advantage. To the best of my knowledge the classes of ticket are handed out chronologically based on when one checks in for the flight. You can check in as early as 24 hours before the flight. Sitting in front of you computer hitting the refresh button in your browser 24 hours and 5 seconds before your flight is recommended. :) Just kidding, but not really, though - I do it.

2. Once you have secured an A-class ticket and it is your turn to board the plane, I would suggest doing the opposite of the original theorem - select a seat in the middle of an unoccupied aisle! Assuming that the plane is not full, no one will really want to climb over you to the window seat and someone may take the aisle seat because it is convenient.

3. Once the plane is in flight, move over to the window seat, creating the much-sought-after one-seat buffer. Sweet!

I don't think the row you choose matters too much, but wouldn't recommend the very front of the plane; I think there is a high demand for the ability to get off the plane first, so this area might prove to be an exception to our theorem.

Anyway, let me know if you are ever able to put this one into practice!

--Chris

P.S. Thanks to Richard for helping me to devise this addendum!

Windows XP - Beginning of the End

2008-07-01T22:01:00.002-04:00

As of yesterday, 6/30/08, you can no longer walk into a store and purchase a fresh copy of Windows XP. There was a lot of speculation that MS would grant XP an last-minute reprieve, but that, unfortunately, was not to be. Although, as a post in Lifehacker shows, you can still get your hands on XP in a couple of ways. While I do think that the much-maligned Vista is not as bad as it is made out to be, I think it's unfortunate that MS is bringing an end to XP.

I just wanted to take this, my 100th post, to take the opportunity to present my homage to XP. I think that XP is one of the finest operating systems ever created. I always had good experiences with it and never had any significant problems using it. In this nerd's humble opinion, it was leaps and bounds better than its predecessor Windows 2000 - and certainly better than it's immediate successor. XP took 2k's 'plug-and-pray' and refined it into a fine plug-and-play setup. I always joke that you can plug your toaster in XP and it will say, "Windows has detected new hardware, a toaster. How would you like your toast?" (Sorry, geek humor.)

Anyway, I won't drone on. If you are in need of a fresh OS install on your Windows box, I would recommend using one of the tips in the Lifehacker post to get your hands on a copy of XP before January, when it will be completely discontinued.

Goodbye, old friend. We hope that Window 7 will be as good as you were!

--Chris

Seat Guru - Getting the best seat on the plane

2008-06-30T07:43:00.003-04:00

OK, I know that I have been obsessing over minute qualitative enhancements to airline travel lately, and I promise to try to think about possibly making this my last post on the topic.

I recently found out about Seat Guru, a site that helps you choose the best seat in any given model of aircraft. All you have to do is select your airline and make/model of plane and you will be presented with a seating chart for that specific aircraft. You can then hover over color-coded seats to find out why or why not said seat is a good choice.

Here's an example. We can see the seats that have limited leg room due to the location of the bulkhead, the seats that cannot fully recline, etc. Now all they need is a way to predict where the crying baby and little kid who kicks your seat will be seated!

This site is really cool. Thanks to Warren for telling me about it!

--Chris

Choosing Your Seat on a Plane

2008-06-28T21:25:00.003-04:00

Sorry to go with back-to-back plane-related posts, but I figured I would get all of them out of the way today. In my previous post, I spoke about the optimal way to fill an aircraft, a dilemma for the airline industry. Now, I'd like to present a problem more relevant to the individual traveler. Consider this:

- You are traveling alone (let's say on a business trip) in coach, or any other flight class that deals in rows of three seats per aisle side.

- When buying your ticket, you are presented with a choice of any seat you want. We will assume that you are the first person to get to select your seat - that is, you can have your choice of *any* seat in coach.

- You want to maximize the probability of ending up with an empty seat next to you. We'll assume the flight is near, but not at, capacity.

Which seat do you choose?

I won't present a conclusion here (because I don't have a great one), but will go over some factors that I use in choosing a seat.

Let's say N = the average number of travelers in a party. I will assume that, during times with the greatest number of business travelers, N = 1 -- people traveling alone. I would guess that on weekends and trips to more recreational destinations, N > 1. Just for simplicity's sake, let's assume N = 1 or 3\N (pronounced "3 divides N"; it basically means you can divide N by 3 and not have a remainder - 3, 6, 9, etc.). N > 0, of course.

In the cases where 3/N, the row cancels out, so we can throw out those cases, leaving us with N=1.

OK, I think it's safe to say that the middle seat in any given row is a losing proposition. You have twice the chance of ending up with a person sitting next to you, on either or both sides. Of course, I guess you could have the greatest payout, as well - having the entire row to yourself - but that seems unlikely. So, we'll consider the only logical choices to be window or aisle. Now, which one, and which row?

I think that the front rows have the highest probability of filling up. People like being the first to exit the plane. That is nice and all, but not the object of our game. So, since people have an incentive to choose a seat toward the front of the plane, we will go with the counter-option, the back of the plane.

I'm not sure about this, but I think that some people may choose the seats in the very back of the plan in order to be close to the bathroom, especially for long flights. So, I don't think we should choose the very last rows. So, if R = the number of rows on the plane and r = our row choice, I think we should go with something like:

(R - 3) <= r <= (R-6)

Finally, as for which seat to choose, aisle or window, I don't think it matters. If you choose, the window, someone will probably choose the corresponding aisle seat for your row (and vice versa), making the middle seat a very unappealing option for another player to select.

So, I guess this assumes that all players of the game will avoid choosing a seat adjacent to another player, if possible. We will assume, no, hope, that no one chooses an adjacent seat on purpose with the motive of being the incessant talker!

I used this algorithm myself recently and it worked. On a plane that was ~95% full, I ended up with an empty seat between me and the guy sitting in the window seat (Sweet!). I had seat 29C out of, I think, 35 rows.

So, to summarize, I would guess the answer to this problem is selecting a window or aisle seat where (R - 3) <= r <= (R-6).

Of course, there are other "quality-of-flight" factors that mess up a perfectly good plan, like this:

- the crying baby factor
- the little kid kicking the back of your seat factor
- the aforementioned incessant talker factor

We've made a lot of assumptions here, so if you have a more abstracted theorem, I would certainly be interested in hearing it!

--Chris

P.S. Yes, I am a major nerd.

Airplane Boarding Problem

2008-06-28T20:44:00.002-04:00

Inconvenience is the impetus for innovation. Despite being unnecessarily alliterative, I believe the preceding sentence to be true most of the time. One of the most inconvenient situations, I think, is the process of boarding a plane. Therefore, it's no wonder that significant research has gone into developing the optimal algorithm for boarding an aircraft.

Recently, on a trip back home from Seattle, I spent some time thinking about this when I noticed that I had a "zone number" printed on my ticket. The airline was boarding the aircraft serially, by zone. I was sitting in an aisle seat toward the back of the plane and had zone 5 (out of 7). This seemed to make sense, given that I had heard to optimal way to fill an aircraft is by filling all of the window seats first, then the middle, and finally the aisles. The concept here is that by getting people out of the aisle and into their seats quickly, you can board the plane faster - makes sense. The old process of filling the plane from the back forward was inefficient to say the least.

I recently read an article about a nuclear physicist that has patented his optimization algorithm for boarding planes. Here is an excerpt from the article:

"Then he ran various boarding options on his computer using the algorithm and found that one option was easily the best. This requires that each passenger is given a specific queuing position; that they board window seats first; and that they do so with an empty row between themselves and the next passenger. Thus passengers would board, if entering at the back of a 40-row aircraft, by filling up seat 40A, then 38A, then 36A and so on to the front; then they fill 39F, 37F, 35F, to the front; then 39A; 37A, 35A to the front; and so on, filling first window seats, then middle seats and finally aisle seats. If boarding from the front, they would begin at 1A, then move to 3A. The point is that people loading luggage do not block progress."

This appears to be an extension of the algorithm I described earlier.

Now, all of this makes sense in theory, yes? Well, in a vacuum, this would be incomparably faster than any process that we could implement in practice.

So, let's go ahead and add some corporate business logic to the mix. Consider the boarding process that U.S. Airways uses, the Reverse Pyramid. The algorithm is described in a Wired article here. Sounds pretty good, right? Now, let's take a look at the corporate implementation of the process:

Pre-board call - Customers needing assistance and families traveling with children
Zone1 – Dividend Miles Chairman, Platinum and Gold members, Star Alliance Gold™ members and all passengers in Zone 1
Zone 2 – Dividend Miles Silver members, US Airways Signature® Visa and World MasterCard® holders and United Premier® members and all passengers in Zone 2
Zones 3-7 – all other passengers following reverse pyramid system

Why does this have to contain all of these exceptions for certain classes of fliers and those who have the company's credit card? Don't answer that question - it is rhetorical. While, granted, there are probably a relatively small number of people on most flights that have these varying levels of status and credit cards, it seems like it could be enough to dilute the efficacy of the optimization algorithm the airline uses.

I should say that I have nothing against U.S. Air. I know that all airlines do stuff like this, but I wish they wouldn't let corporate partnerships and the like mess up a good efficiency algorithm.

Anyway, I always enjoy the application of mathematics and logic to solve everyday problems, so I thought I would post this.

--Chris