Nano2Sol by Marc Boucher

Apple 12″ PowerBook G4 Meet Yellow Dog Linux

2009-08-09T20:12:13Z

I hate it when a perfectly good computer just sits around doing nothing. In this case it's my old Apple 12" PowerBook G4 with the following specs: Processor: 867MHz, Memory: 640MB SDRAM, HardDrive: 40GB Ultra ATA.

Why would I replace the beautiful Apple OS with Linux?

There are two reason. The first is that as soon as Apple decided to go Intel with their processor's they dropped support for the PowerPC version of the OS like a hot potato. I was none two pleased. The second reason is that I can do more with linux running on these hardware specs than I could running Apple's now unsupported OS.

Why Yellow Dog Linux?

It really wasn't a hard decision picking Yellow Dog. My research showed that this distro had the most up to date support for PowerPC processor based computers. Ubuntu also supports the PowerPC but not nearly as well as Yellow Dog.

Getting and Installing Yellow Dog

Installing Yellow Dog was actually quite easy thanks to clear instructions on its web site and an install process that is smooth. First you need to get the latest distribution for the PowerPC. For me that was downloading the YDL 6.2 single ISO from one of it's mirrors.The download is a 3.8GB file which took me about 40 minutes. I have a pretty fast connection. Once you've got the ISO you need to burn to a DVD. I did this on my Mac desktop. You can follow these basic instructions.

Once you've got the ISO burned to a DVD you can download the YDL v6.2 for Apple instructions for a quick and easy install. I went with a single boot with a total wipe of my previous Apple OS.

What Yellow Dog Looks Like When Installed

From left to right you see: The main screen running the Yellow Dog E17 desktop, Terminal and OpenOffice apps open and lastly FireFox with a screenshot of the new SpaceRef web site under development.

What's Next?

Well the laptop will be used to test various code I write and may be used on any expeditions I go on.

I also have an Apple Power Mac G5 which I'm looking to turn into a test server platform so I'll probably put Yellow Dog on it after I've backed up the data. But that project is for another weekend.

New Media Hearings - CRTC Should Once Again Do Nothing

2009-02-18T06:59:02Z

Ten years ago I testified at the Canadian Radio-television Telecommunications Commission's (CRTC) New Media hearings in Ottawa and argued that they should not regulate Canadian Internet content. I stand by that decision and they did not regulate Canadian Internet content at that time.

Today the CRTC held its first hearing in 10 years to consider a new proposal that would see a $100-million fund created to support Canadian content on the Internet. That fund would be created by a levy on the Internet Service Providers who of course would then pass it on to the consumer. What really irks me about this discussion other than an increase in my monthly internet service provider bill, is that the content this fund would create is Canadian online video programming from Canadian television production companies. Don't we already subsidize Canadian television production companies? Why can't they just take their existing content and broadcast it on YouTube? Hey, they already do! And if more people are watching video online then in front of their TV's then why not shift some of their exsiting budget towards more online video? Creating a fund that will pay for doing something the television production industry should already be doing is ridiculous.

If the CRTC insists on meddling with New Media then why not create an online technology development fund that companies can draw on to create more high tech products made in Canada and marketed globally. This would create new jobs in the high tech industry helping position Canada as a leader in whatever sectors these companies enter. The fund should not be passed on to the consumer through Internet Service Providers though. It should come in the form of a new tax credit or other mechanism.

If the CRTC deems to impose this levy (read tax) then I will support the companies that fight this though legal channels.

- Follow the CRTC New Media Hearings Day 2 on the live Globe and Mail blog.

Analysis - Twitter's Business Model & Sector Sentiment

2009-02-02T17:00:14Z

I joined Twitter on May 8, 2007 on the advice of my business partner at @SpaceRef @KeithCowing. Not too long after that we participated in a NASA conference at NASA Ames on Participatory Exploration where @Biz (Stone) explained to the audience what Twitter was and it's participatory nature. Thinking on it now it's kind of ironic that Biz was giving us the talk as we were using Jaiku, a competitor then, for the conference. Jaiku is now pretty much history after Google bought it and recently decided to stop working on it and make it open source.

After listening to Biz that day I still wasn't convinced that Twitter was useful and would survive. In fact I sent an email to my contacts on LinkedIn asking what they thought of Twitter. I got back a variety of answers including; What's Twitter? To how essential some people thought Twitter would become to their business. So I decided to keep using Twitter and subsequently created over 30 new Twitter accounts for various entities I was involved in including @GeneRef, @OnOrbit, @SpaceMeme, @HMP, @Hyperix etc. I'm glad I created all those accounts because now I'm starting to get some real tangible benefit from being on Twitter and being a part of the conversation. If you don't know what Twitter is and need a quick simple primer than this recent article in the Vancouver Sun is excellent.

But what's in it for Twitter? What's the business model? (2) I don't think Twitter's not going to make money through advertising. The community could revolt and move to another platform. And if they wanted to place advertising on the service they would have, should have done so already. Twitter's not going to make money charging a fee to access the service. That would just make their audience flee and kill the exponential growth they are now experiencing. So how is Twitter going to make money? In my humble opinion the basis of how they are going to make money all started when they made their application programming interface (API) available.

On Friday, September 22, 2006 Twitter announced to the world they had an API and would you be so kind to use it and the developers came flocking. In fact I've lost track of how many applications have been developed using the Twitter API but the Twitter Fan Wiki lists many of them. But the point is the developers came, and in developing a variety of applications they helped define what Twitter is and how it can be used. And they defined Twitter better than Twitter itself was able to through the applications they created. And in so doing I think they provided insight to Twitter as to what could be their business model. And it's all about the data. The data from people's Tweets, location, clickthroughs etc. Twitter is a gold mine of data and as the service grows the available data to mine is getting better.

As a simple business example I use TweetDeck as my desktop client to post to Twitter and more importantly to follow areas of the conversation that are relevant to my businesses. I do this in two ways. I follow a lot of people, 580, at last count, but I can't follow a continuous single stream of the conversation that these people engage in. So I use TweetDeck's ability to create groups to aggregate people I follow based on the business sector they're in. This takes care of the people I follow, but what about the rest of the conversation on Twitter. The millions of people I don't follow and may not be aware of? I use TweetDeck's search function to create a stream of conversation by topic. For instance I'm interested in search engines like Google, Yahoo, Ask and all the others so I've created a search group for the keyword phrase "Search Engines". I've also done this for the BioTech sector. I think by now you should get the idea of the value of both the Twitter service and TweetDeck, the client built on Twitter's API, bring to my business. But what about Twitter's business model?

As I've said it's all about the data and I've shown an example of how Twitter data can be mined to follow the conversation for business purposes. But now let's look at the big picture. Twitter like many other online businesses such as Google is collecting a vast amount of data. And not only from the main service as not too long ago they bought a service called Summize which offered a Twitter search engine.

The data Twitter is collecting is very valuable. Analyzed and aggregated in a myriad of sectors it is useful to many businesses. That data can be sold or used to create follow on products. And therein lies the business model in my view for Twitter. In creating a service that forces users to be brief in expressing themselves they've created millions of conversation on the fly that are both timely and measure user sentiment. By data mining the conversation and sentiment they've created their business model.

Author's Note: My latest startup is called Hyperix and has been building a vertical search platform for the last two years to create vertical search engines in a variety of niches. We've also started a business intelligence unit that will provide products based on the data we are currently collecting and analyzing. Our web crawler HyperixScoop has been crawling Twitter creating a database of users for future use. You can follow me on Twitter @marckboucher and contact me in a variety of ways.

What Alternative Space News Sources Including Blogs, Social Networking Sites do you Use?

2009-01-02T00:42:36Z

There are many good space news sources available, they include SpaceRef, Space.com, NASA Watch, The Space Review, Spaceflight Now etc. I'm compiling a list of all space news sources and assigning them an authority ranking based on several criteria including traffic, update frequency, citations etc. and your input on the following question would be very valuable.

My question for the community is what alternative or non-mainstream sources do you find useful? Post a comment or email me (marc.boucher < AT > hyperix.com) with your top sources and alternatives.

My Reaction to the Liberal NDP Coalition and Prime Minister Harper’s Conservative Government

2008-12-04T06:47:27Z

I am not thrilled with the idea of a Liberal NDP coalition however since it is allowed within the Canadian constitution and is democratic I think the coalition should be given the opportunity to govern. While the Conservatives did win a minority mandate to govern during the recent election it was with only 37% of the popular vote, hardly a resounding mandate. In fact the Liberal NDP coalition had 44% of the popular vote. It is true though that the Liberals were pounded on election night and that Stephane Dion does not have the confidence of the people to lead.

Prime Minister Stephen Harper has miscalculated in a big way. His partisan politics with the economic update of last week was ridiculous. He made the mess he's in and it appears he will say and do anything to stay in power. There have been suggestions he should resign and that's not a bad idea. I don't think a majority of the public has confidence in him anymore. If he does ask the Governor General to prorogue parliament and succeeds in doing so I think he will be furthering his problems and in the next election he will pay for his miscalculation, especially if the Liberals have a stronger leader.

Now while I'm not a Stephane Dion fan either perhaps he will step up and lead well during his opportunity should he get it. However he would only be Prime Minister until the Liberals select a new leader in May. Hopefully that will be Michael Ignatieff. And should Ignatieff become Prime Minister I hope he will wait long enough before calling an election so that people can see what kind of leader he could be.

Politics in Canada has never been as heated as it is now, at least in my recent memory. And I, like most Canadians, am looking for stability and strong leadership in these challenging economic times and I don't see our Government stepping up to meet these challenges at this time.

Recharging My Apple Power Mac G5 with Linux

2008-12-02T03:24:36Z

For some time now I've been wondering what to do with my Apple Power Mac G5. At the moment it's basically an iTunes server for my music and video. However that seems like a waste of a good powerful computer. So I'll be upgrading the computer and installing Linux, replacing Mac OS X, but I have to say I'm very disappointed in Apple. Here's why.

When Apple decided to switch to Intel processors in the June of 2005 I was happy with the move. However I never expected Apple would within two years release an upgrade to OS X, being Leopard, that would not be supported by my PowerPC G5. So here I have a computer that is just over 4 years old and I'm told that future versions of the operating system that Apple produces won't be supported on my computer. Wow, talk about sticking it to the consumer. And it's not like I bought a low end computer. Apple computers have always been more expensive than their PC counterparts. And this was just the beginning in a series of moves by Apple that disappoints me. In their aggressive efforts to get the iPhone they sacrificed quality in my opinion. I have two iPhones, the original and a 3G version. Both run well most of the time but are susceptible to locking up and are hardly rugged. My previous Blackberry Pearl was rock solid and never crashed even when banged up a bit.

With Linux getting better with every year and running on more hardware at a lower cost than Apple computers it makes me wonder if I'll ever by another Apple. I've already made the switch with respect to servers for my company, we build our own boxes now. I like Apple products, don't get me wrong, but with higher prices than other providers and an attitude that doesn't put the customer first, I have to question their long term future. Windows 7 is coming and Microsoft may have learned from it's mistakes, Ubuntu and other Linux distros are making great strides, especially everywhere outside of North America, so Apple better adjust to the market or its market share will shrink again.

Do it Yourself Storage Area Network (SAN)

2008-11-20T21:41:56Z

If your IT department is being squeezed by the downturn in the economy but still needs to add let's say a storage area network (SAN) then why not do it yourself (DIY). Sure you could contact your favorite hardware vendor like Dell, HP, IBM etc., but when you compare the costs of an off the shelf product compared to the DIY project you'll realize big savings, in our case around 50%.

My company currently has a need for SAN. Based on our needs we decided to go with a 24 TB SATA in 4U form. Total cost for parts, taxes and shipping $9,500 Canadian. Comparable vendor systems start at around $30,000.

We generally buy Supermicro boards and chassis as they're good quality at a good price. Our operating system (OS) of choice is FreeBSD. Here's the basic specifications we used.

A 24 disk storage array consisting of the following hardware:

SUPERMICRO MBD-X7DWN+ Dual LGA 771 Intel 5400 Enhanced Extended ATX Server Motherboard

SUPERMICRO SC846TQ-R900B storage chassis

2 x Intel Xeon E5410 Quad Core 2.33GHZ LGA771 12MB 1333FSB Harpertown Retail Processor

2 x 4GB RAM (see all compatible Crucial DIMMs for this board, and see notes below about RAM requirements)

Areca ARC-1680ix-24 24-port RAID controller

Areca ARC-6120 Battery Backup Module

1 x 2GB cache memory module for Areca RAID card

24 1TB SATA II drives (current candidate is Western Digital Green Power 1TB, model Caviar GP WD10EACS)

1 x Western Digital Caviar Black 500GB boot drive (this resides on the motherboard controller)

1 x 4GB or 8GB compact flash card as maintenance boot device (resides in compact flash slot of motherboard)

1 x compact flash adapter (it might be worth it to get several as failsafe bootables)

NOTES:

Sufficient memory needs to be onboard for fsck of 20+TB, and large filesystems can consume a large amount of RAM or not work at all without it. 1GB RAM per 1TB storage is a common rule, so running 8GB is a lower limit, not an upper limit.

We also need reliable GPT support for this size of an array, which is not there yet on many OSs.

X7DWN+ is not listed on Supermicro's BSD compatibility matrix, but people report OK use under Free BSD 7.0 REL

A useful discussion about compatible generic memory for the Areca controller

Areca manuals, benchmarks, firmwares etc.

Drive compatibility with the RAID controller needs to be established. For compatible Western Digital drives this requires adjusting the TLER values in each drive's bios prior to using it in the RAID array. Note the DOS utility rewrites the BIOS of all attached drives, so this should only be done an a system dedicated to this task. Other non-RAID certified drives may need similar adjustments.

Western Digital reports the 1TB GP as compatible with the Areca.

As of 12 November 2008, only a handful of 1TB drives are certified by Areca and several common ones are explicitly NOT compatible. In the Western Digital line only the WD RE2 GP RAID Edition 1TB is certifed, with SAS controller firmware V1.45 80915 or later. Unfortunately such RAID-certified drives are nearly double the cost of non-RAID variants.

Western Digital Green Power (WD GP) drives are only 5400 RPM but reports suggest that differences in read/write performance between hard drive brands becomes less important as more drives are added to the array.

A test report of an Areca 1680-type SAS controller

This 2GB memory module reported to be compatible with a 1680 controller. Worth getting.

Without the battery backup the Areca cards turn off write-caching completely to help prevent data loss.

OTHER NOTE:

use http://www.shopbot.ca/ to find the lowest price on gear above for price matching purposes. Some items in the SAN parts list are already linked to shopbot.

The Cloud and Hyperix Search

2008-11-17T16:35:16Z

A lot has been written about cloud computing in the last year and each day seems to bring news of a new player in the cloud arena. So what does the cloud have to offer search engine companies like Hyperix? Well that depends on how deep our pockets are. After all, we need a lot of bandwidth, processing power and data storage to run any real search engine. And as we don't have deep pockets, nor an angel or venture firm backing us we've had to be find creative solutions and innovate where possible.

Up to this point we've been focusing solely on the technology we're using that will differentiate ourselves from any other vertical search platform entities out there. We've got our own small web crawling cluster setup which we've used for some time to test different web crawlers, collect and parse data and measure a variety web crawler values which determine how many CPU cycles, RAM, bandwidth, and storage is necessary to create the vertical search indexes we want. We've also been focusing on the quality of the data we're crawling, the algorithm which ranks the pages crawled, the parsing engines, and the results pages.

Having determined a baseline for our costs to crawl on our own we're now comparing that with web crawling using Amazon's Elastic Compute Cloud (Amazon EC2). After we've compared the two we'll decide which to use as we move forward with our production web crawls. We would prefer to use our own hardware but the cost can be prohibitive and ultimately you would think that at some point it would make financial sense to run the crawls on your own hardware, but until we actually test the crawl on Amazon EC2 we won't know the true costs. And while we could just crunch numbers in Amazon's calculator, anyone whose ever done crawling knows that there are many variables that determine how long a crawl will take, the RAM it will use and how many CPU's and nodes are required to successfully achieve an efficient crawl.

Aside from web crawling there's the search side of the equation. There are some search engines which use Amazon's web services to not only crawl for data, but also to serve up their searches. We've determined that Amazon's services as offered don't offer us a cost effective solution for our search needs. This primarily has to do with our search indexes. When users will search our vertical search niches they'll be querying our indexes which are held in memory. We've come up with an innovative solution that a) dramatically reduces our memory costs b) is faster than current index searching and c) is cheaper for us to run on our own hardware. The innovation which is theoretical at this point is going to be tested out for the first time later this year, but we are confident it will work.

Cloud computing for us at this time, using Amazon's EC2, may be useful for web crawling but not for our search servers.

My Personal ColabSpace Wiki

2008-11-08T04:38:20Z

Since we've now launched ColabSpace for the space community I would be remiss in not using the service myself.

So here's the link to my personal wiki space on ColabSpace. It provides a little background on myself along with some projects I'm working on. As well I'll occasionally post some source code from various project for anyone to reuse.

ColabSpace Launches - A Collaborative Service for the Space Community

2008-11-08T04:28:46Z

For some time now SpaceRef has wanted to create an online collaborative service for the space community. It took some time though to try and find the right platform that would allow us to provide the community with the appropriate tools so they could in turn create rich interactive collaborate services.

I'm happy to say that we've now launched our new service called ColabSpace under our brand OnOrbit.

The goal of the service is to provide the tools that will allow users to effectively collaborate on a project increasing productivity so that any given project moves ahead in a timely manner to conclusion.

For the platform we decided to use Deki, a social enterprise collaborative platform from MindTouch. Why did we use Deki? For starters it was because of its ease of use. If you want a simple wiki then it will do that. At first glance it looks just like another wiki, but under the hood it's a robust platform, where only your imagination is the limit. If you want to create some really cool collaborative mashup services then Deki provides a wide array of extensions allowing you to plug in to services like Flickr, Twitter, YouTube, Mantis for developers, Chat, Salesforce, LinkedIn, Microsoft SQL Server, MySQL and Google services such as Maps, Calendar, Apps including Docs, Spreadsheets and nearly a hundred more services.

The platform also comes with it's own scripting language called DekiScript which allows you even more flexibility in modifying your service and cretaing your own extensions. As well if you've already got a MediaWiki, Confluence, PBWiki or other wiki then odds are MindTouch has the conversion tool that will allow you to import it into ColabSpace.

At this time SpaceRef is seeking groups, organizations and even companies who want to access our service. For the small groups or non-profit organization we offer a free community site where you can build your services. For those who need a little more such as their own branding including use of their own domain name, we offer an enterprise option at a very reasonable monthly fee of $79.00 which includes full hosting with your own IP.

We're going to contribute to the community by providing some of the data we've collected from over 1500 companies, organizations, universities and space agencies over the last nine years to help build out a commercial space wiki. We won't be able to do it alone so we're appealing to the community to help us.

We're already using the service ourselves for the Space Elevator Reference were we've launched an Open Wiki and are working towards launching a Developers Wiki.

We are also currently in discussions with a few organizations to see what interest they have in using the free service and expect to have some of them using ColabSpace soon.

SpaceRef was started in 1999 to provide something back to the community from it's founders who have had a life long passion for space. Subsequently and because of our popularity we've turned SpaceRef into a business, however at our roots has always been a willingness to provide something back to the community. ColabSpace Community is yet another example of what we're offering back to the community.

I hope you'll join us as we build out ColabSpace and bring another great service to the space community.

Brightcove to Shut Down Free Accounts

2008-11-05T06:36:26Z

In July I compared Brightcove to YouTube and said I would provide feedback after a few months. Well yesterday I received an email from Brightcove informing me that our free OnOrbit corporate account which was based on advetising revenue sharing was being discontinued as of December 17th and I had the option to become a paid customer.

Now getting back to the my previous post where I was comparing traffic each service generated all I can say is that YouTube has continued to blow Brightcove out of the water. In fact not too long ago I posted a video to YouTube that got picked up by Wired and the next thing we knew it had 249,026 views and we saw our subscribers jump dramatically. So now that we have to make the decision on Brightcove I think YouTube has already made it for us. So in the future you'll see all our videos on our YouTube channel. And now that you can view them in their pristine hi-res version, so much the better.

Haven’t Posted in a While Here’s Why

2008-11-05T06:23:36Z

There are many reasons why I haven't updated my blog in some time. It's not because I wanted to neglect it, it's because of the following reasons.

I don't usually blog about my personal life but for the people who know me it's no secret that my fiancée is ill and has had a very rough summer into the fall. Combine that with two projects I've been working on, when I can, adds up to little time to blog.

My fiancée Sylvia's health has improved just a little, in other words she is stable enough that Ive been able to focus a little more on work.

The two projects I've been working on are ColabSpace for OnOrbit/SpaceRef and Hyperix a vertical search engine platform. I also have responsibilities at the Mars Institute which adds to my workload.

I'll elaborate more on this projects in my next post, but suffice it to say, I'm back.

It's Cool to be Cuil but is Cuil Cool?

2008-07-28T17:27:48Z

It's cool to be Cuil today. Cuil Inc. launched their new search alternative to Google today. Cuil pronounced Cool has received lot's of press today and it helps when it's in the right places. And if it we're not for the fact that the principals have a history of producing value add to existing search products like Google search, then this roll out would be hardly noticed.

But the fact that they have a track record, worked at Google and are boasting that they have an index bigger than Google, is newsworthy. Cuil is led by Anna Patterson a former engineer at Google. Along with her husband Tom Costello, a search expert in his right, Cuil aims to take on Google. No small feat.

But having a bigger index doesn't mean you're better. And only time will tell if they have what it takes to carve out a piece of the big search pie. They claim to be able to search across 120 billion web pages compared to an estimated 40 billion Google has. Google officially does not reveal how many pages it indexes but others sources suggest that they keep an index of around 60 billion pages. As well Google says that not all of the pages it crawls are indexed because many are duplicates. Working in this industry I can concur that there is a lot of duplicate content out there.

For Cuil to take some market share away from Google it will take more than the boasting of a bigger index. Reality is, with enough hardware and money a startup can build an index that is big, even huge as Cuil has. The test of whether Cuil can succeed will be if the public and business users find more relevant search results through Cuil. Being as big or fast as Google is not enough. You have to be able to change people's search preferences. And that's not easy.

What is noteworthy is that Cuil says they've developed a faster, better way to index pages and just as important use less hardware. Less hardware is important as the cost to index, store and serve up results can be prohibitive. The ongoing downward costs of hard drives, CPU's etc. helps. However even though RAM prices have come down, the price of RAM still is one of the most expensive aspect of creating a searchable index.

In my initial tests of Cuil I was both pleased with the results and disappointed. Some common searches resulted in no results. I'll attribute that to first day bugs. But I also found that sources like Wikipedia were heavily weighted, sometimes in favor of the actually site that I was looking for.

It's public day 1 for Cuil and they have people's attention. Let's see if they can keep it and build some momentum. In the meantime I'll give them a try and report back with my thoughts in the near future.

Comparing Brightcove vs YouTube Usage

2008-07-16T18:01:16Z

A little over a month ago I started a little experiment. I wanted to compare usage between our SpaceRef Brightcove video channel and our newly created SpaceRef YouTube channel. Up to that point SpaceRef had been using Brightcove exclusively as our video service provider.

The numbers should not have surprised me as YouTube has a tremendous lead over everyone else. But still I was. I uploaded a small selection of current videos to our SpaceRef YouTube channel that were also on our Brightcove channel and did absolutely no promotion for them. None. I didn't link to them from SpaceRef or any of the sites on the SpaceRef network. What was the result? For the 1 month test run our YouTube channel had 29% more individual video views. That's pretty impressive.

There are some difference though between the two services. I find that the Brightcove channel has a little better video quality. As well Brightcove has for some time allowed you to monetize your channel and it's only recently that YouTube has integrated AdSense.

For the time being we're going use both video channels and compare them as we go. I'll report back in a few months on our progress.

Odd Google AdSense and Analytics Behaviour the Last Few Days

2008-05-17T17:15:45Z

I get daily reports on our Google AdSense channels as well Google Analytics for the SpaceRef network of sites. The last few days have seen some strange disconnect between the two. Google Analytics is reporting a surge in traffic, in particular to a couple of old stories.

Today it's reporting that a particular story which is 4 1/2 years old has generated 21,000 pageviews. The referring traffic indicates it's organic from Google and for a particular keyword phrase, "first laser", which is coming in 14th and on page two of Google results for the phrase. Yet when I look at Google AdSense the channel data suggest that traffic does not exist.

So which is right and which is wrong? I'm leaning to a problem in reporting from Google Analytics. It's only recently that for some of my larger sites I've switched over to Google Analytics and I'm wondering if I did the right thing?

I wonder how many other people are experiecing the same problem.