Chaotic clamoring

Links for 2009-11-11 [del.icio.us]

2009-11-12T00:00:00-08:00

DSPLink Integration | Timesys Embedded Linux
Process still seems pretty manual right now and I hope that they plan to simplify it in the future. Getting access to the DSP on the OMAP-L138 will give you many times more MIPS than you'd have on the ARM9 alone.

Links for 2009-11-09 [del.icio.us]

2009-11-10T00:00:00-08:00

OSGeo Events
'FOSS4G is heralded as the international "gathering of tribes" of open source geospatial communities, where developers and users show off their latest software and projects. The theme for 2009 is "User Driven", highlighting the power of Open Source to solve business problems.'
Physical Computing » Urban Defender
An interesting game that uses a BeagleBoard.
vjaquez-beagleboard - Gitorious
"Marmita is an OpenEmbedded based distribution for the BeagleBoard, oriented to hardware accelerated multimedia (and maybe Maemo5)."

Links for 2009-11-06 [del.icio.us]

2009-11-07T00:00:00-08:00

ElectronicosOnline.com Magazine - Explican funcionamiento de la Beagle Board de TI
Interview with TI exec in Mexico about the Beagle Board there.
Lego Mindstorms для программиста / LEGO / Хабрахабр
Russian blog about Lego Mindstorms robots with lots of comments about the BeagleBoard.
Introduction: a Typical Embedded System
A nice intro on embedded Linux systems. Once this becomes public, it will be a nice reference article for those new to the BeagleBoard.
Economy Size Geek - A Pico-Sized Platform with Potential
"When the package arrived, and I pulled out the box, my first reaction was wow! This thing is tiny!"

Links for 2009-11-05 [del.icio.us]

2009-11-06T00:00:00-08:00

KanManDet.Dk » Beagleboard.org

Links for 2009-11-04 [del.icio.us]

2009-11-05T00:00:00-08:00

WXHMD - A Wireless Head-Mounted Display with embedded Linux
based on the Gumstix Overo.
Demy - The Digital Recipe Reader

Links for 2009-11-02 [del.icio.us]

2009-11-03T00:00:00-08:00

Mulled wine - Wikipedia, the free encyclopedia
YouTube - libfreespace on the Beagleboard
Interesting 3D remote control used with the BeagleBoard.
Beagle Board

Links for 2009-10-30 [del.icio.us]

2009-10-31T00:00:00-07:00

ZachHoeken.com: Danger Shield v1.0

Google kills Blogger Web Comments

2008-07-19T11:00:00.004-05:00

Often I find websites where people are just being stupid and need to be told so. In those cases, I don't think relying on the ignorant host of the site to provide a comment page to let me tell him how much of an idiot he is being will really work.

Then there are those cases where I'm wondering about what other people think who are interested in this same site that doesn't allow direct comments to be posted, or where I don't trust the host to not pull down negative comments.

So, what are my options?

Well, I used to make a lot of use of the Blogger Web Comments for Firefox. This was a pretty handy tool that would fetch comments using Google's Blog Search. Since I've recently upgraded to Firefox 3, I thought it was a good time to go look for an update to the plug-in and to see if I could get that functionality back.

Unfortunately, the plug-in is no longer available. This isn't the first time I ran into a brick wall with Blogger Web Comments for Firefox, but it seems they've decided to drop it, rather than fix it.

Hopefully others will still see the promise in this sort of functionality and provide something, but in the short term, I'll be stuck performing copy-paste operations and executing 3-5 clicks to get similar output manually from del.icio.us, Google blog search, and Technorati.

I'll be visiting those search options regularly to see if someone picks up on this feature.

Coining a phrase, the Contextual Web

2008-03-30T12:46:00.005-05:00

I was getting started writing up a "master paper" to serve as a guideline for submissions to several conferences this year, including Lug Radio Live USA. In this paper, I planned to coin a phrase, "The Contextual Web". I figured, if I plan to coin a phrase, I should at least ask Google if anyone has tried to do that before me.

It turns out that someone has, they did it recently, and the synopsis looks eerily like the one I had written in some drafts. I'm not trying to claim that anyone stole my idea, or that I even had it significantly earlier than anyone else. To the contrary, I'm trying to claim that this idea is just that obvious. Here's a clip from the page I found when I did a Google search for "the contextual web":

The next generation of the web isn't going to be on your desktop, it may not even be on your mobile device. Context is going to be increasingly important and Nick will take you through the process of designing and architecting for context as well as regardless of the context.

Well, Nick Finck, you've got my attention. A few more searches with Nick's name in the search box return some additional gems:

There are four Elements of Context – the User, the Task, the Environment, and the Technology. Who is your user and what obstacles are they facing; what task are they trying to complete; what is the environment in which they are working; and what kind of computer or device are they using? Designing interactive experiences is not limited to the web on your computer or phone – consider gas pumps, fridges, or devices like Microsoft Surface.

This definitely puts my ego into perspective. Nick, I'm supporting the Beagle board just for you. :)

Adding a URL to 'gitweb'

2008-03-21T23:22:00.001-05:00

It is as simple as creating a 'cloneurl' file in the git repository directory, just like you can add a 'description' file.

This took about 7 minutes of exploring the CGI code of gitweb to find, which took another 2 minutes to find. I spend about 20 exploring the web based on some links I was given that were 'supposed' to explain this, because this was the big feature that was missing from my gitweb installation. Ugh!

Come on Linux folks, are you just trying to make easy things difficult?

Example: http://www.beagleboard.org/gitweb/?p=beagleboard.org.git as sourced by http://www.beagleboard.org/beagleboard.org.git.

Making the connection between Gears, GreaseMonkey, JXTA, and OpenID

2008-03-20T09:59:00.002-05:00

A while back, I wrote-up a "Collaborative GreaseMonkey" patent disclosure. It was a defensive measure to make sure no one else patented the idea and prevented the rest of us from using it. The disclosure never made it past our patent committee, and I think that is fine, since it is at least documented as prior art in some way. The code never got to the point where it was worth sharing, but I do plan to revive it at some point.

I'm seeing that more and more people are starting to get ideas that are more and more similar to what I had in mind. Today, I read about someone dreaming up thoughts on using Google Gears to perform OpenID and OAuth. I like the thought pattern.

Gears, GreaseMonkey, OpenID, and P2PSockets (JXTA) have the potential to re-invent the web and to establish a real web operating system. Gears enables the JavaScript written into web pages to become part of a real, persistent application with persistent data storage and threads. GreaseMonkey provides a solution to edit existing web applications with user-controled, local customizations and to create applications fully local, without needing to learn how to write a web server application. OpenID gives a single solution for authenticating yourself across those web applications. P2PSockets allows the applications and data you host locally to be discovered on the web without needing to own a web server.

The result is an application building environment that is an incremental step from simple HTML+JavaScript editing and allows everyone to invent their own web, rather than just rely on the web that the social networking sites control today.

The success of this web is, of course, controlled by the economy it creates. An a-la-carte business model, like the one provided by Amazon's web services, is a great way to ensure that the bandwidth and data storage necessary for the locally-hosted services to scale.

Thoughts?

Open source on TI devices

2008-03-07T10:32:00.002-06:00

I happen to like this article, TI targets Linux and open source with new OMAP chips, but I certainly have gotten the message "more patches, less powerpoints". We'll see over the next few months...

TI-Open-Source-Workshop-TIDEVCON08

2008-03-05T22:30:00.001-06:00

TI-Open-Source-Workshop-Jason-Kridner-TIDEVCON08
Originally uploaded by shutter_nut

I've mostly given up on trying to be relatively anonymous on this blog. I figure that people who know me already know how to find this site, but I'm starting to try to take on some relatively public responsibilities related to open source software and the newly "announced" BeagleBoard.

Actually, the BeagleBoard isn't officially "announced". The reason is that there really isn't a community or set of applications around it yet to make it something worth announcing. Instead, it is just an open project looking for some of the right folks to help make it happen.

I have a lot of confidence that the BeagleBoard will be a very real and active community project. Just let me know if and how you'd like to get involved.

For those who have read my blog posts in the past, rest assured that the BeagleBoard is quite intertwined with my vision for collaboration. My hope is that it will yield a nice starting point for building collaboration software that could be integrated into just about any form-factor and innovative human interface.

Heading to LugRadio Live

2008-02-06T09:07:00.000-06:00

You almost can't call it a business trip, but I will be filing an expense report...

Go to LugRadio Live USA 2008, 12-13 April, San Francisco! Watch this, then spread the word!

Could Pandora open up Linux games?

2008-01-29T18:21:00.001-06:00

They say the Open Pandora (P&|A) handheld gaming device compares in power to a Nintendo GameCube, and will offer full-speed Playstation and N64 emulation. How does the GameCube compare to other systems/CPUs?

Of course, what I think is really interesting about this device is it being a clam shell (to protect the screen), having real gaming controls, and being fully open for hacking. I expect a lot of nice software will come out of this device existing.

read more | digg story

Hello? Jabber was designed for cloud computing

2008-01-24T22:00:00.000-06:00

I just read Marshall Kirkpatrick's Read/Write Web post Could Instant Messaging (XMPP) Power the Future of Online Communication?. Despite his apparent bemusement with the "the rise of XMPP (called Jabber in IM) for powering communication services hosted in the cloud" this really shouldn't be much of a surprise. In one of my favorite books of all time, Peer-to-Peer, Jeremie Miller, inventor of Jabber, explained this to the world in 2001. Jabber was envisioned from its beginnings in 1998 to not just handle person-to-person conversations, but also person-to-application and application-to-application conversations.

I also recently read about using Jabber with my OLPC XO-1, which opened up a whole new world. All of a sudden, instead of just finding other XO's on my LAN, my screen was full of people to chat and collaborate with. Over Jabber, not just instant messages are shared from the XO, but every application can be shared and becomes a gathering place. You can take a look at how Jabber is used with the XO on the OLPC wiki.

Marshall goes on in his analysis to bring us back down to Earth regarding Jabber/XMPP relative to HTTP and he is right. HTTP rules today and I don't think there is any one killer reason to change that. If nothing else, however, Jabber/XMPP has a really nice specification on how to use HTTP more efficiently to get notifications without polling. Jabber/XMPP specifies this for the purpose of overcoming firewalls, but the result is that Jabber/XMPP can really be seen as simply some really cool stuff to do on top of HTTP.

Where is the Jazelle-RCT open source solution?

2008-01-23T11:03:00.002-06:00

Ugh. There is too much noise around open source virtual machines, including almost 16,000 projects on SourceForge.

The PhoneME, Android, or other open source JavaVM projects must be looking to support ARM's Jazelle-RCT technology, right? I know there are some interesting commercial efforts, but if anyone is aware of an on-going open source project, I'd want to hear about it.

Bug Labs device was cooler than I expected

2008-01-10T20:37:00.000-06:00

At CES this week, I managed to stop by the Bug Labs demo, which ended up winning an award for the best emerging technology. I've been hearing about this device for months from co-workers and I'd explored the website, but seeing the live demo was more impressive than I expected.

The little Lego-like embedded electronics development kit was quite flexible. As a challenge, in 8 minutes, they created a new application of a motion-triggered camera that would upload photos to a server. I was quite impressed.

They use Eclipse to create an easy-to-use development front-end and PhoneME to run Java applications on the device. The device is running both X11 with Athena Widgets (AWT) and Qt/Embedded. This isn't quite as nice as the GTK stuff running on the N810, but it shouldn't take them any time to get there. The demonstrator had no trouble throwing together a new program in Java and sending it down to the device over USB, despite being harassed by one of his co-workers about the missing award they had just won.

Apparently, someone decided to take their newly won prize. Hopefully, it was just on loan to one of the many television interviewers showering attention down on them.

Criticism of the OLPC XO-1 concept

2007-12-13T09:09:00.000-06:00

In John C. Dvorak's PC Magazine article "One Laptop per Child Doesn't Change the World", he writes:

Does anyone but me see the OLPC XO-1 as an insulting "let them eat cake" sort of message to the world's poor?

I like Dvorak and I often follow him on the TWiT podcast and CrankyGeeks on TiVo, but he polarizes issues in ways that sometimes aren't that useful, except for bringing attention to an issue. Hopefully the audience is paying enough attention to think for themselves, but that has proven repeatedly not to be the case.

In my view, if some service like "Mechanical Turk" pays living wages for these folks, then it was worth it. On average and over a lifetime, each of these students should be able to earn more than the cost/value of the computer.

Yes, the literacy rates and language barriers are an issue in making the computers useful at all. There would be, however, huge motivation to focus on literacy and additional languages, if some people are able to earn money with these machines.

So, Dvorak has given all of us XO enthusiasts a mission: enable students to make money using these machines by providing services like Mechanical Turk in the languages of the students and figure out how they can collect the resulting goods.

OK, I admit, this isn't a perfect idea. I've heard concerns that these laptops will be stolen if a market emerges for them and having them be a source of money would certainly make them valuable. This is also, to a degree, advocating some sort of child labor, which is a reality, despite the many objections we have in the developed world.

Better ideas?

Blogger Beta Ships OpenID

2007-12-01T08:34:00.001-06:00

Google has added new login options to Blogger, including OpenID. This is an important additional baby step towards a web with a single sign-on that allows you to have better control of your identity information.

read more | digg story

N810 demo video from Nokia

2007-11-20T17:13:00.001-06:00

Nothing too special about the web page, but the video gives lots of good angles and pictures of accessories. Sure would be nice if they used that GPS and a database to help find some WiFi hotspots. Too bad it doesn't have WiMax or EV-DO.

read more | digg story

Accessibility is the killer mashup/webos application

2007-10-01T13:03:00.000-05:00

After watching Douglas Crockford (of Yahoo and JavaScript fame) plea for Google, Microsoft, and others to participate in a mashup summit and reading some of the feedback around the web, I realized the critical application use-case is still missing.

Someone in the Q&A brought up a good example of a mashup implemented today in an undesirable method due to security reasons: Facebook accessing your GMail/MSN/... contacts to request more members. Contact sharing between applications is an excellent use-case for mashups, but I don't see it as a driving application. Certainly it gets to the heart of Crockford's talk: security is an excellent application for the Google Gears WorkerPools. If you are like me, you'll still be left thinking about how everyday web consumers will be motivated to download Gears, instead of walking down the questionable path of simply giving applications like Facebook access to all of your potentially private information.

Mashups are the most interesting innovation in software development in decades. Unfortunately, the browser's security model did not anticipate this development, so mashups are not safe if there is any confidential information in the page. Since virtually every page has at least some confidential information in it, this is a big problem. Google Gears may lead to the solution.

Security is important and is critical to the growth of new mashup applications and I'll be happy if that alone brings us worker threads and off-line support, but I think the killer mashup is the one that makes all of this great data exposed through APIs and structured web pages and makes it accessible in new ways.

Mark, of diveintoaccessibility and diveintogreasemonkey fame, who I admire for his vision of accessibility wrote in his blog post "if wishes were iPhones":

I don’t understand this continuing obsession with buying things that you need to break before they do what you want.

And with this thought I am reminded that the killer mashup/webos application is the one that takes all of those immensely useful web services out there and makes them measurably usable. And by usable, I mean giving the user control.

Mobile 2.0 Conference

2007-09-10T01:04:00.000-05:00

I'm thinking about trying to attend the Mobile 2.0 Conference on October 15th in San Francisco. Anyone have comments about this event? If there were a handful of people interested in chatting about how to enable creation of scalable web services generated from mobile/embedded devices, I'd make a point of going.

No server, no satisfaction

2007-09-06T09:17:00.000-05:00

There's just no satisfaction for the casual home user who wants to collaborate with friends. Even when dealing with a problem that has been solved many times over, it is really difficult without a server of your own and a fair amount of programming. The problem I'm talking about is planning events on a group calendar.

There is a group of somewhat over 25 people near where I live that frequently gets together to play outdoor roller hockey. We play in a parking lot in one of the area parks or offices. We have a mailing list on Yahoo, but most people are just copied on a repeatedly used e-mail thread. On that thread, the subject line is typically changed to match the proposed day and time. Every week, we all bombard each other with e-mails to make sure that enough of us are coming out to play. This has actually worked fairly well, but there have been some significant exceptions.

Sometimes we don't meet our threshold of 6 players and additional e-mails go out to entice people to sign-up to play. Calls are made. Threats are discussed. People who previously agreed to go attend might decline since they don't want to risk trying to play hockey with only 3 people. Chaos ensues.

One of they guys who used to come out regularly created a really simple sign-up page on a website. The site accepted a name, e-mail address, and phone number to sign-up for a given game. The name is listed beside the entry for that game. The e-mail was used to send out the "game-on" or "need-more-players" notification a few hours before the prospective game. Phone numbers were included to speed up the communication.

The application worked quite well and was simple-minded. Entering the same sign-up information twice would result in being removed from the list. The phone number and e-mail information had to match, providing a tiny amount of security from folks simply removing everyone from the list. No verification of the entry was done, but you can imagine a simple verification code being provided via SMS to the mobile phone number if we ever started to have problems with that. Life was good.

He stopped playing and his site stopped working. We were back to using e-mail. A few folks reminisced about the good 'ol days when we had our own web server. I own the domain name, so I decided to bring back the sign-up sheet. Where should I host it?

I don't really like the idea of pointing people to my home computer, so that's not my first choice. I don't really like the idea of paying for a full-featured (LAMP and/or Ruby enabled, ie. scripting and a database) hosting service just for this hobby. This is just calendar data! Why should I need a web server to do something that Yahoo and Google provide for free?

Our mailing list is on Yahoo, so I looked first at their group calendar. It was a disastrously complex to use and didn't provide any of the custom features we had with the much simpler web app. Similar problems were had with Google's calendar and Evite. The most fundamental issue with all of these calendaring solutions: they required account creation and login to utilize.

I tried to see if I could do something with static hosting, but it seems even Amazon's simple queuing service doesn't seem to work without having a dynamic host. At this point I gave up, but I'll get back to this application yet.

Still don't get the whole WebOS thing?

2007-09-03T08:39:00.000-05:00

I've gotten a bit smarter about explaining why there will be a sort of emerging web operating system to the people who inquire. For example, I've started calling it a "web services kit", instead of an operating system. Today's tech savvy minds can accept the idea of yet-another-SDK, whereas the idea of a web operating system is either tainted by the webtops or seen as inconceivable and unnecessary delusions to compete with Windows, Linux, or OSX.

What I haven't leveraged enough is the great summary of web service APIs provided by ProgrammableWeb. From their simple scorecard, you can get a quick overview of the categories of popular services and some of the key players. Ask yourself what sustainable advantage do any of these players have within their service space. Don't get fooled, it isn't an easy question. Keep in mind that standard service definitions are coming into existence for most of these services, such as XMPP for chatting and OpenID for identity. Take up the exercise to look across these service APIs, look for winners, and look for emerging standards.

Comfortable? Now, realize that it is only a matter of time before there are standards-based implementations of all of these services. Sure, it might take a while, but it'll happen.

If you are quick, you might be sighing and thinking to yourself, "what about the data?". I'm glad you asked, because that is really the point. These services are all about controlling access to data and looking for ways to monetize it. You might stumble over the idea that on-line office applications involve an incredibly complex pile-o-code, but then you'll remember that you already have 2-3 other viable choices of office applications to which you already had access. Over the long-term, it is all about the data.

Still don't feel like you're any closer to accepting the idea of a web operating system? That's okay, as long as you recognize the benefit of something that provides you with the capability to control and monetize access to data and some sort of well-understood integration layer back into your application. You'll come around when you start thinking about who you want to own your data.

BarCampHouston

2007-08-25T07:05:00.000-05:00

I'm heading up to BarCampHouston today. I don't know what to expect and I haven't had any free time whatsoever, so I won't put on much of a demo or presentation. I'll be happy to share any info on P2P collaboration and my Maemo SDK EC2 image, if anyone is interested. I'm also looking forward to simply discussing toolkits for building web services (web operating systems).

By the way, I've moved my EC2 script over to http://sdk-ami.garage.maemo.org.

Boot your N800 Maemo SDK today with Amazon's EC2

2007-08-18T01:01:00.000-05:00

I really appreciate that someone has created VMWare and QEMU images for running the Maemo SDK. Unfortunately, my machines, both Mac and PC, are typically too busy with other stuff to allow me to quickly fire-up a virtual machine image that will chew up all my computing resources. Instead, booting up a machine from Amazon for about $0.15/hour or so is affordable enough for my N800 development. No more downloading a 1.5+GB image; EC2 users can instead just share an image and a boot script and be up-and-running at a known-good starting point.

Of course, the first step was to get an EC2 (and S3) account. I waited almost a month for my EC2 account. At some point I'll figure some way to let other people just rent a machine from me to make it easy, but that'll require a bit of thought and management. For now, head on over to Amazon, request an account, and they'll get to you eventually. There isn't any monthly fee or hidden costs; you just pay for the time, bandwidth, and storage you use. As long as you copy your work off somewhere else, such as by using Subversion hosted by code.google.com, you can shutdown without having any recurring fees.

Once you have an EC2 and S3 account, you'll want to download the EC2 command-line tools and your access identifiers.

The next step was to choose the Linux image I wanted to use as my starting point. Personally, I'm a Gentoo fan because I think Linux has an excessive number of binary-compatible dependencies on the C library and Gentoo solves that by recompiling every new application, instead of needing to update your C library to match the binary-compatibility requirements of all your applications. Of course, that makes application installation slow and Maemo itself uses the Debian package model, so Debian or Ubuntu make the most sense. However, Amazon supplies some nice reference images on Fedora Core 4 that might simplify my life around issues like ssh login security when the root account password can't be secret. Nothing is easy, so I found an Ubuntu image that has reasonable documentation on how it was created such that someone could redo this all with a better supported AMI in the future.

At this point, the issues started to pile up and I decided my best hope was to document my steps in scripts so that I can reproduce them with a better starting image and make corrections that people point out to me. I decided to the Google Subversion server I mentioned earlier to host my script at http://code.google.com/p/maemo-sdk-image/.

I used a Mac to run my script, but I plan to eventually make it run on the N800 itself or on a Windows PC. Right now, the script uses bash, which isn't natively on either the N800 or Windows. Also, the Unix-style version of the EC2 command-line tools also utilizes bash. I think the solution for both is likely to install bash.

The access identifier information ended up placed in a subdirectory called 'secrets' under where I ran the script. These secrets end up getting copied temporarily to the EC2 images for the purpose of bundling them up. I exclude that directory from the bundle.

I say 'images', because I end up working with four different EC2 images in the script. The first one is the base Fedora Core image that Amazon makes public. The second one is a bare-bones Ubuntu Feisty image that can be boot on EC2. The third one is patched to be self-bundling. The fourth one actually contains the Maemo SDK. The third and fourth could easily be combined, but I am still inching along.

In theory, you can run all of the steps using 3 separate calls to the script:

./build_maemo_api build-feisty
./build_maemo_api patch-feisty
./build_maemo_api install-maemo

I think the first two should work reasonably well to create a self-bundling Ubuntu image, but I haven't run them exactly like that to test them out yet.

The third step certainly won't work. One issue is that the Nokia binaries require you to agree to a license, so you'll need to do that part manually. This also only gets you to version 3.1 of the SDK, so you'll need to update that as well. There are also several steps missing before the image is really usable, such as setting up the X server and VNC server to allow you to view the emulated N800 screen remotely.

Please feel free to post your comments here or on the wiki on how to improve the script. I won't hesitate to utilize your inputs on the script hosted on the Subversion server.

Working on an Amazon EC2 AMI for the Maemo SDK (scratchbox on Ubuntu)

2007-07-27T12:50:00.000-05:00

I'll get into why I want to create this Amazon EC2 AMI thing later, but I thought I'd get information out there on a problem I'm having. Tve did a nice write-up on RightScale on bundling-up an ubuntu EC2 instance. This is a really helpful write-up, but I get a shell script error and a hang when I try to bundle my Ubuntu image.

root@domU-...:~# ec2-bundle-vol -d /mnt -k ~root/pk-....pem -c ~root/cert-....pem -u ...
Copying / into the image file /mnt/image...
Excluding:
/sys
/var/lock
/dev/shm
/proc
/dev/pts
/proc/sys/fs/binfmt_misc
/var/run
/dev
/dev
/media
/mnt
/proc
/sys
/mnt/image
/mnt/img-mnt
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.003492 seconds, 300 MB/s
mke2fs 1.39 (29-May-2006)
warning: 256 blocks unused.

Bundling image file...
sh: Syntax error: Bad substitution

The script keeps running. I don't know Ruby well, but the script seems to be stuck in a shell call to 'openssl'. This seems to occur in bundle.rb line 51. It looks like the 'tar' call on line 57 that is meant to feed the pipe being read by the running 'openssl' died, but I don't see where the "Bad substitution" might be.

Tve didn't have a place to make a comment on the blog entry. Time to debug...

Quick update on when I canceled the process (need to think if this confirms or denies what I was thinking):

sh: Syntax error: Bad substitution
sh: cannot open /tmp/bundleimage-pipe1: Interrupted system call
error executing tar -chS -C /mnt image | tee /tmp/bundleimage-pipe1 |gzip | openssl enc -e -aes-128-cbc -K ... -iv ... > /mnt/image.tar.gz.enc; for i in ${PIPESTATUS[@]}; do [ $i == 0 ] || exit $i; done, exit status code 2
ec2-bundle-vol failed

...need to take a break and get back to my real work.

Update #2: The problem turned out to be assumption in Amazon's Ruby script that 'bash' would be the shell executed by default. The answer was within a script found on the Amazon developer's forum. I've created my first image and I'll be hosting my script using Google's open source code repository when it is done at http://code.google.com/p/maemo-sdk-image/.

Judgment vs. Jealousy: Jadon on Twitter

2007-04-27T02:07:00.000-05:00

After hearing so much talk about Twitter on the blogs, podcasts, and vlogs I read, hear, and watch, I decided I had to know what the fuss was about: http://twitter.com/Jadon. How do I manage to find time for this sort of waste?

Google AJAX Feed API

2007-04-19T01:01:00.000-05:00

I read Udi Dahan's post on how Google's Ajax API Simplifies Safe Mashups. It didn't take 5 minutes to take Google's "Hello World" AJAX Feed API example and embed into a web page of my own. This is fun stuff allowing me to finally be able to manipulate feeds across sites without injecting any server-side code.

What should be noted is that this is relying on Google services and must follow their terms. It means that they monitor all the content this API brings into my page while other search engines cannot. (Crawlers do not typically execute the JavaScript, so that's why they wouldn't see the results of the API calls.)

Certainly I'm not the only one disturbed by this, right? Where is the counter-movement to give the every-blogger ownership of his own services in just as simple a manner? Is there a better answer than Amazon's web services? (For those that don't know, Amazon provides hosting solutions that are API-driven, highly customizable, scalable, and billed-by-use.)

Google's solution is so simply by comparison that I am struggling to remember why this even matters to me. Something in my gut just keeps telling me it is wrong to rely on services where I can't understand the business model.

Also distracting from this actually-quite-cool service from Google is the fact that Yahoo! Pipes already offered this feature and many more. Additionally, I believe that Yahoo! doesn't require you to register for an API key against your URL, though there is a need to register to create a pipe (feed) if you aren't using one someone else already created.

P.S. If you are listening, sorry I'm not keeping up and haven't even uploaded the rest of my notes on my P2P collaboration presentation. I have lot's of activity at work these days keeping my creative energy going without having to resort to my blog rantings. :-) But, this was such a quick post.

Technorati Tags: google, yahoo, ajax, amazon

A CIO that "gets" open source and collaboration tools

2007-03-17T21:03:00.000-05:00

I've been really busy lately and will get back to the P2P Collaboration tool post soon. I've really been enjoying my new job and have been spending a bit too much time at it. :)

I was trying to catch up on a bit of my reading and I ran across a fantastic 50 minute video on open source in the enterprise when reading Stu's blog that I had to stop and share it really quickly. It is really worth the time. Be patient. If your job is related to information technology, you must hear this and consider it.

CIO JP Rangaswami breaks down the economic justification for using open source in the enterprise and many of the reasons we need to tear down the walled gardens. There are some specifics he gives for banks, but most technology companies will have very similar issues working with other companies. Any company taking an early stand for open source collaboration tools can gain the benefits of better recruiting as he explains. He also explains how the various tools are consolidating to standards and a small number of fundamental operations.

When I listen, it is more and more justification for the coming WebOS.

Again, the link is http://www.infoq.com/presentations/jp-rangaswami-open-source.

Peer-to-peer Collaboration Tools

2007-02-08T00:01:00.000-06:00

I gave this presentation at an internal company conference last week. Large corporations suffer from different collaboration issues than the open source world, but we also have much in common. My hope is to show folks at my company some tricks that we can learn from the open source world. Open source development is almost certain to be globally distributed and using on-line tools for almost all of the communication.

[Introduction]

It is rare that I get to speak about a topic for which I have such a great interest and I know will have such a great impact. The scale of the impact is on par with the emergence of e-mail as a technology. In fact, I don’t believe it will be very long before peer-to-peer-supported blog-like technology replaces e-mail as the primary communication mechanism over the Internet and corporate networks. Perhaps 3-5 years.

The title of my presentation is "overview of peer-to-peer collaboration technologies" and is sub-titled "managing communications with a global workforce". I chose this subtitle to emphasize that these technologies will help you address some of the many challenges of working in global teams. Communication is a fundamental problem of business that is complicated by globalization. I hope to show you here what is happening now to solve the communication requirements.

Purpose of this presentation
The purpose of this presentation is to illustrate that the emergence of peer-to-peer-based collaboration software is inevitable, show why [our company] should embrace peer-to-peer software architectures, and give an overview of some current peer-to-peer, metadata, and collaboration technologies. Some peer-to-peer collaboration architectures, such as Microsoft Office Groove 2007, will quickly and significantly alter the client-server architecture used for most existing content management systems, such as SharePoint, TWiki, and [our proprietary systems].

These peer-to-peer tools will allow the elimination of many IT dependencies. IT will obviously have significant roles to play, but those roles will change. With the emergence of these tools, team productivity can start almost immediately. Additional productivity and stability can then follow with more formalized IT involvement. For example, someone needs to create bridges between all the products and protocols. The various tools need to be combined into a single product with a consistent look and feel. The costs of deploying a collaboration solution can be optimized. Additional points of access and different accessibility features can be provided. My suggestion is that checkpoints requiring resource allocation, however, can be eliminated until there is a business benefit for IT involvement.

This isn't as much a choice I'm suggesting we make as it is the recognition that existing forces will drive acceptance of peer-to-peer related technology. On-the-go collaboration is a critical requirement of global business communication tools. Information locked into what can be called "walled gardens" or server silos cannot benefit all of the necessary people at the necessary times. This is certainly true in a global workforce of partnerships, third-parties, outsourcing, and ODMs. While business communications regulations, such as Sarbanes-Oxley and other E-Discovery requirements, will make the process more complex; they make it no less inevitable. The required technologies will survive by natural selection.

Tasks, e-mails, notes, and calendar items are mostly the same: they are a bit of collateral communication along with some specific metadata to allow the tool to know how to advise the user. There should be a tight association between the collateral, or content, or microcontent, and the metadata. These digital communications must be preserved in context. New metadata types, such as test case and product requirement associations, must also be supported. Making special tools to handle these bits of metadata is a formula for disaster.

The madness of creating new centralized walled gardens of highly specialized data repositories must be replaced with a vision of building new interoperable data definitions, widely accessible visualization tools, and improvements to existing communications infrastructure.

In this presentation, I will cover:

how I got interested in this problem,
why we need collaboration tools,
why collaboration will go peer-to-peer,
what a peer-to-peer collaboration tool might look like,
why should [our company] continue to evaluate peer-to-peer collaboration technologies, and
how you can get involved.

How did I get interested in this problem?
[Our company] entered the portable media player market in 1999. This market largely grew out of the popular usage of peer-to-peer file sharing applications, such as the old Napster. These peer-to-peer file sharing applications provided a source for content that established distributors weren't yet willing to provide. Learning about the architecture of these systems convinced me that they were useful for far more productive arenas than the sharing of pirated music.

The most critical aspect of making both peer-to-peer file sharing networks and portable media players work for people is the management of metadata. Metadata is the information associated to the content, such as the artist, title, genre, format, or rating. With all of the possible content available for download and playback, user satisfaction is driven by the ability to find desirable content quickly and easily using metadata. Apple's combination of the iPod scroll-wheel and the iTunes Music Store, along with a bit of slick advertising, gave people answers to where to get to music they wanted quickly. It led them to great success, without necessarily relying on peer-to-peer file swapping. Still, file swapping networks are still the quickest and easiest way to get to some content and they remain in wide use.

Failed attempts to defeat the networks convinced me that the technology will continue to be widely adopted, despite organized attempts to the contrary. The file swapping networks that are still in existence survived because they use peer-to-peer architectures. In a peer-to-peer application, there isn't a central server that can be shutdown to disable the network. Peers directly share metadata with each other, providing a path for sharing content. To build a peer-to-peer network, you only need peers speaking the same protocols and a willingness to participate.

The thing to remember here is that intelligent managing of metadata is how content is found, no matter what the platform.

Another experience I had was when this team went global. My job quickly shifted from a focus on media player technology to a focus on information management. I spent my time worrying about version control, status reports, bug tracking, requirements management, and portfolio management. The communication was less-than-efficient and the results were less-than-desired. This had nothing to do with the talent of the team, but had much to do with the communication mechanisms and processes that were in place. An 11 hour time-zone difference is literally one world away.

I saw that my experience in peer-to-peer technologies and metadata management could be applied to solve my new information management problems. Just like getting to media content quickly and easily helps media consumers, the product developers need quick and easy access to information.

Why do we need collaboration tools?
Communication in a global team cannot be handled entirely face-to-face. No matter how many worldwide face-to-face meetings you hold, when you break up and get back to work, there is always something left unsaid. Having frequent teleconferences helps close communication gaps, but they impact the work-flow of team members and never provide the depth of conversation required to establish a cohesive team.

While there is no substitute for strong individual communication skills, collaboration tools provide a path for competing with otherwise more convenient information sources and distractions. On-line tools provide opportunities for collaboration between people where distance, time, cultural, language, experience, and ability barriers exist.

You can see this in the success of open source projects like FireFox, which is a web browser that competes with Microsoft's Internet Explorer. Contributions to open source projects come from people all over the world in many different situations. To make open source projects work, the participants make extensive use of on-line collaboration tools.

Also note, the communication in these tools isn't one-way. The messages being communicated aren't dictates from a single expert telling everyone how to solve the problems of the whole. Requirements and solutions come from every member of the team and beyond.

Why will collaboration go peer-to-peer?
First, what do I mean by peer-to-peer? Back in 2001, Daniel Bricklin gave a speech at the O'Reilly P2P Conference on "The Cornucopia of the Commons". That speech was later printed in a great O'Reilly book. He quotes a 1968 essay on "The Tragedy of the Commons" summarizing a commonly expressed problem:

Therein is the tragedy. Each man is locked into a system that compels him to increase his herd without limit--in a world that is limited. Ruin is the destination toward which all men rush, each pursuing his own best interest in a society that believes in the freedom of the commons. Freedom in a commons brings ruin to all.

That isn't such a pleasant or welcome idea, but we can all see some degree of reality to that view.

Dan goes on to offer another view in the light of successful peer-to-peer architectures and their failings:

In the case of certain ingeniously planned services, we find a contrasting cornucopia of the commons: use brings overflowing abundance. Peer-to-peer architectures and technologies may have their benefits, but I think the historical lesson is clear: concentrate on what you can get from users, and use whatever protocol can maximize their voluntary contributions. That seems to be where the greatest promise lies for the new kinds of collaborative environments.

That is the thinking that launched the Web 2.0 explosion with businesses like Flickr, MySpace, and YouTube. Similarly, when I'm referring to peer-to-peer technologies, I'm talking about creating protocols that maximize the contributions from the edges. Those contributions generate a wealth of information and content highly valued by users.

The elimination of a required centralized server is one common technology applied in peer-to-peer architectures. The benefits that approach provides to IT may be a bit counter-intuitive. I've done a small amount of examination of four benefit areas to collaboration tools by elimination of a required centralized server: security, reliability, interactivity, and efficiency.

The elimination of a centralized server is sometimes required for security purposes. We occasionally enter contracts with other corporations that give very explicit rules about who can have certain data on their hard drives. A centralized server would violate that policy, whereas it may still be acceptable to share the data with specific peers. Ultimately, security is best provided if information managers control the access to data, rather than IT.

There is also the occasional need to utilize modern tools that aren't yet available on our internal servers. Some of those tools are available on external servers, but it isn't secure to place [our company's] private data on those servers. A tool that does not rely on a centralized server could be more easily deployed. This doesn't eliminate the need for security audits on the tool, but it does eliminate the overhead of evaluating the impact of that tool on other tools running on a common server.

Despite on-going improvements to server reliability, there doesn't ever seem to be an end to the reasons why a server must occasionally be shutdown or relocated. With an architecture that doesn't rely on a single centralized server, the reliability is certain to be increased. An application network that requires a high degree of reliability should still make use of high-reliability servers, but it makes sense to architect those networks to not rely on them exclusively when possible.

Efficiency benefits from eliminating requirements on a central server come from the ability to rapidly deploy a new peer-to-peer network. A business could deploy a new network right away to gain the benefits of collaboration without waiting for a high-reliability server to come on-line. Such a server could then later be deployed to increase the reliability of the network without creating downtime. Further, an information manager within the business can be directly responsible for adding and removing users, machines, and services on the network.

Efficiency benefits also come from more general aspects of peer-to-peer architecture. By allowing users to organize their own data, they can access it more efficiently. In most cases, there isn't one ideal solution to data organization. A peer-to-peer architecture allows everyone to organize their own data and allow others to benefit from that organization.

By interactivity, I mean both the possibility of working with other tools and the way the tool works for its users. Interoperability and connectivity to other tools doesn't necessarily require the elimination of a central server, but tools that don't attempt to create a centralized repository for information are more likely to support the standards required for interoperability. By creating protocols for use in a peer-to-peer network, some of the requirements for interacting with other tools are necessarily met.

One particular interactivity benefit of eliminating requirements on a central server is the possibility of providing "on-the-go" or disconnected collaboration. You experience this today with Outlook when you are on a plane. You can read your e-mail that is cached locally, create your responses off-line, and synchronize your mail once you are connected to the network again.

E-mail must be the most common on-line tool for collaboration. It is simple, universal, "on-the-go", search-able, you know who is on both ends, you get notification, and for the most part it just works.

Being such an effective collaboration tool, e-mail has many peer-to-peer architecture characteristics. Foremost, it is distributed and resilient. It relies on DNS, the domain name service that is used to look-up the address of servers on the Internet. DNS is distributed and resilient, allowing for server failures at many points in the system. Further, SMTP, the protocol used to forward e-mail messages, can be run on just about any computer. You could almost decide to run SMTP on your own desktop machine, but you'd quickly find that getting the DNS record to point to your machine is a bit of a hassle when you don't leave your machine running all of the time. This doesn't matter that much, though, since [our company] provides you with a high-reliability server to collect your e-mail and clients that cache the e-mail for off-line use.

So, what is wrong with e-mail as a collaboration tool? First of all, the information is trapped into little personal "silos" that no one else on your team can search or access. Certainly you don't want to share all of the information provided to you by e-mail, but some of it you do. Some of it you just want other people to know you have, but not necessarily give it to them without your approval. You can forward e-mail to individuals or mailing lists. You can archive e-mail sent to mailing lists on a website. You can even create shared mailboxes, though those are a bit complex for many people to handle. In the end, you are left with a large number of small silos of data that can't be organized as part of a larger body of information.

E-mail is not secure. Sure, there are tools for encrypting e-mail, but they are practically only ever used on the most sensitive data. E-mail encryption tools are simply too difficult to use and you cannot yet create encrypted messages for the vast majority of your e-mail address book and expect the recipients to be able to perform the decryption.

Monitoring e-mail for sensitive data is virtually impossible. E-mail is sent from all levels of the organization all across the world, without any approval of information managers. When inappropriate e-mail is detected, there is no realistic mechanism for confirming retraction of that data from recipients. Outlook has a "recall" feature, but it often fails and cannot be confirmed outside of our organization. E-mail must be the most important and most dangerous of all the collaboration tools available today.

Perhaps the worst aspect of e-mail is the lack of efficiency. Information coming in e-mails cannot be easily categorized. Creation of e-mail filters often makes the situation worse by creating yet more silos of data. Fields that would allow for some categorization, such as priority, action required flags, and deadlines, are typically never used and are often misused. Misuse is sometimes the result of spam, which is a problem that cannot easily be avoided. By one study, 94% of e-mail last month was spam.

What about TWiki or SharePoint? These tools are often called "content management systems". They can be quite effective in improving communication within a global team, but they have their own issues. Perhaps the best summary comes from a success story taken from the TWiki website from a company that deployed TWiki to improve support to field engineers:

People in the field were used to using email for communicating with the factory. Email is a one to one communication, a mailing list a one to many. The problem with email is that useful information does not reach everybody, email is not easy to search and email gets lost over time. Collaborating the Wiki way solves these problems, however changing habits is a difficult issue that needed to be coached.

Initially we also had a chicken-and-egg problem, i.e. voices like "why should I use this collaboration tool, the content is so limited". The solution was to assign a support engineer who monitored the mailing lists and entered useful information into TWiki.

Successful deployment took over 6 month[s], [which was] longer [than] expected. But now everybody is used to browse, search, collaborate and document the Wiki way.

The result was that customer satisfaction with the field support improved. The effort was a real success.

This sort of dedicated information management may not be something we can easily commit in our environment. If deployment took 6 months, how are we supposed to keep up with frequent release cycles? How do we convince managers to commit resources to TWiki when current searches for me today often return my own weekly reports, rather than something of valuable interest? With so little attention, even the top-level structure of the TWiki sites today can't even keep up with our organizational structure. I believe in TWiki, but we can only overcome this "chicken-and-egg" problem in each team by strongly evangelizing its use during the painful learning stages.

SharePoint has similar problems, but is a bit of a different beast. SharePoint is particularly well-suited for collaborating on Microsoft Office documents. It uses a standard protocol called WebDAV that allows for folder views in Windows Explorer. Most importantly, [our company] supports a mechanism for accessing SharePoint sites to customers and partners from outside the firewall.

The biggest problem with SharePoint, beyond the problems it shares with TWiki, is its complexity. The user permissions tables are extremely convoluted. Editing the content of any one page requires extensive knowledge of the overall system, rather than simply clicking an "edit" button and changing some text. I’m not saying that TWiki markup is trivial, but it doesn’t require learning specialized tools or extensive training. The help system is built right into the tool. Also, the SharePoint version control system is somewhat less than reliable because it allows overwriting and elimination of old document revisions.

We are still learning about how best to use TWiki and SharePoint on our projects and the best standard practices are not obvious with either tool. Neither provides great search solutions for the data you need on their own, especially if the data is in mixed formats. Instead of providing a complete knowledge picture, the combined usage of e-mail, TWiki, and SharePoint creates islands of information that must each be explored separately.

[Description of an internal collaboration tool]

Efforts like this should continue, but it is best to break up the platform into interoperable components. Consider interaction with a tool such as Microsoft Live Clipboard. Live Clipboard is a mechanism for users to initiate sharing of data between websites without requiring development of web services scripts or other complicated programming. By supporting such a feature in all of our collaboration tools, the islands of information can be bridged.

The same guy at Microsoft who dreamed up Live Clipboard, Ray Ozzie, has also brought us one of the more compelling peer-to-peer collaboration tools already available, Microsoft Office Groove 2007. Recently we made use of Groove on one of our projects. The project spanned two partner companies and two contractors. Some folks on the team were able to start collaborating on the very first day by simply installing the tool. It took a few more days for others to overcome some minor installation headaches that were likely related to the tool being in beta. The product will be released with the 2007 version of Microsoft Office.

Groove works across firewalls, provides account management, secures communications, provides synchronization for off-line usage, includes instant messaging with some voice capability, and has some limited integration with Office applications and SharePoint. When I talk about the integration being limited, however, it needs some emphasis. All of your important e-mail, calendar, and contact information isn't easily shared in Groove with something as simple as a single click on a category. To get that information into Groove, a user must jump through many hoops. A major concern for the team was lack of support for maintaining old versions of documents. Groove did a great job of ensuring everyone had a copy of the latest version of a document, but SharePoint was required to maintain historical copies.

Groove also lacks a client for any platform besides Windows and the client can be a bit slow because it consumes a large amount of memory at times. There is no way to see the data in Groove by simply logging into a web page. You can synchronize a SharePoint with Groove, but it is a tool separate from the other tools in Groove which all seem to act as more islands unto themselves.

Ultimately, usage of Groove suffers for many of the same reasons as the web-based content management tools. Some folks wouldn't use it regularly, instead using familiar tools such as e-mail. It never became part of the team's everyday work flow, partially because other tools were required to author rich documents and manage code. The client tool was seen as painful to start-up or leave running. Ultimately, nothing was pushing users to actively communicate using Groove.

Certainly there are some dangers with this going unchecked, primarily related to it being difficult for IT to log file exchanges. Exchanges over SSL secured websites or, to a lesser degree, encrypted e-mails offer similar challenges. Ultimately, there are always ways for employees to circumvent security and, in some cases, the risk of not progressing business is worse than the risk of compromising security.

What might a peer-to-peer collaboration tool look like?
Groove offers a good starting point for describing the peer-to-peer collaboration tools of the future, but it is not alone in its class. What I'd like now is to describe for you some of the building blocks for creating a tool like Groove and some of the building blocks that could be used to make a better tool. I won't draw you a complete picture of the ideal peer-to-peer collaboration tool, but I hope to point you in that direction.

I'll finish typing this up when I get back from vacation next week. I need to scrub and upload the pictures...