O'Reilly FYI Blog

Nikola Tesla--The Man Who Lit Up the World - An Excerpt from The Geek Atlas

2009-05-28T19:09:21Z

Just in time for summer vacation planning comes an exciting new travelers' guide from O'Reilly, The Geek Atlas: 128 Places Where Science & Technology Come Alive ($29.99), by John Graham-Cumming. Learn about Nikola Tesla in this excerpt from The Geek Atlas now. The excerpt includes a discussion of AC versus DC and information about the Tesla Museum in Belgrade, Serbia.

Excerpted from The Geek Atlas
Thomas Edison is the name most people associate with electric light, but the man responsible for providing the electricity to lights and every other electrical device used today is Nikola Tesla (Figure 30-1). Tesla was born to Serbian parents in what is now Croatia, and lived in Hungary and France before moving to the U.S. in 1885. It was there that Tesla became the greatest electrical experimenter since Michael Faraday (see Chapter 75) and laid the foundations for modern electricity distribution.

When Tesla moved to the U.S., he went to work in Edison's laboratory, where he redesigned Edison's DC generators and motors. He left after a disagreement about his pay, and after Edison had rebuffed Tesla's attempts to interest him in AC electricity generation.

Later, Tesla and Edison battled bitterly and publicly over AC and DC electrical power. Tesla had joined up with entrepreneur George Westinghouse to build AC power stations, while Edison was pushing DC power. Edison tried to show that AC was dangerous, and to prove it he carried out executions of dogs, cats, horses, cattle, and even an elephant using AC power.

Ultimately, Tesla was proved right: AC power is easier to generate (the generators are simpler, cheaper, and more reliable), it can be transmitted much further (DC power was limited to short distances and necessitated power stations close to consumers), and its voltage can be converted using a simple transformer.

But Tesla's inventions were not limited to AC power. Along with Marconi, he shares the honor of inventing radio (see Chapter 62), and he worked on wireless transmission of electricity, remote controls, vertical take-off and landing aircraft, directed-energy weaponry, robotics, spark plugs, and more. In all, he was awarded over 300 patents.

Nevertheless, Tesla died destitute, in room 3327 of the New Yorker Hotel in New York City. Two thousand people attended his funeral.

After his death, Tesla's nephew and heir (who was also the Yugoslav ambassador), Sava Kosanovic, arranged for Tesla's personal effects to be removed from the U.S. and returned to Yugoslavia. Today the Tesla Museum in Belgrade houses his complete collection of books, writing, and objects, as well as his cremated ashes on display in a golden sphere.

The museum explains many of Tesla's inventions, including AC power, and is the definitive place to understand Tesla's life and work.

Information about the museum and details of Tesla's life are available at http://www.tesla-museum.org/. If Serbia is too far away to visit, there's also a memorial to Tesla on the Canadian side of Niagara Falls, where Tesla's hydroelectric power plant was situated. See http://www.teslasociety.com/victoria.htm.

AC Versus DC

Direct current, or DC, is simple: it's the type of electricity that batteries supply. In a DC circuit, electricity flows in one direction only--for example, from the positive terminal of a battery through a circuit to the negative terminal. Alternating current, or AC, changes direction cyclically, typically in the form of a sine wave (Figure 30-2).

Figure 30-2. AC voltage varying with time

AC varies in voltage from a positive maximum to a negative minimum over time. To generate AC power, a current can be induced in a pair of coils using a rotating magnet. The current varies as the magnet rotates. Since the magnet does not touch the coils, AC generators are reliable and simple. DC generators, on the other hand, require a more complex mechanism, with rotating brushes touching metal connectors that are used to change the direction of the current to keep it positive.

It's simple to change the voltage of AC using a transformer (Figure 30-3). A basic transformer consists of a pair of coils, separated either by air or, more commonly, by some ferromagnetic material such as a bar of iron. Because the AC voltage varies over time, it creates a changing magnetic field around the coil it is connected to. This magnetic field induces an AC voltage in the other coil. The ratio of the number of windings of cable in the two coils determines the change in voltage (and current) across the transformer.

Figure 30-3. An AC transformer

But AC's biggest advantage is in power transmission. Because AC's voltage can be increased or decreased using transformers, it's possible to choose the most appropriate voltage for a given situation (see Figure 30-4); that is, power transmission can use a very high voltage that is then reduced by a transformer before entering a home. Generators in power plants can produce power at a lower voltage than the transmission line, with the voltage being increased before transmission by another transformer.

Figure 30-4. Power transmission using AC

Because the voltage can be changed so easily, AC is able to take advantage of the fact that power loss in a cable is proportional to the square of the current. By increasing the voltage (perhaps to hundreds of thousands of volts), with a corresponding decrease in current, power can be transmitted over great distances. It can then be transformed to a lower voltage (and higher current) for delivery.

AC can also be converted to DC, using a simple device called a rectifier. Small electrical appliances (like cell phones) usually operate on DC, and power adapters both convert the AC supply to a low voltage and turn it into DC.

If you enjoyed this excerpt, purchase The Geek Atlas now. and start planning your summer vacation!

DIY Projects from the Greater Bay Area

2009-05-27T17:39:00Z

Congratulations to Enrique Ruiz -- winner of our DIY Projects contest! He and his son should have a great weekend at Maker Faire! See you all there.

***

We've been receiving all sorts of photos of your DIY projects for the past few days in response to the contest to win 2 weekend passes to Maker Faire, May 30 & 31, in San Mateo, CA. Below are some of the entries. We'll be accepting entries through 12 pm PST tomorrow, May 28th, so keep sending those pictures in for your chance to win!

From Enrique Ruiz:

Here's a picture for an Arduino Controlled Bicycle Pump that I could not integrate in time for my son's Iron Man Suit with Tech for automatic repulsor burst. :-) (Also referenced by Maker Blog--here. My son loved that!)

From Kat:

After teaching myself to crochet, I found this awesome purse project, that included crochet skills AND basic sewing skills. I think it turned out great and is way cuter than something you can buy at a store.

From Kent Barnes:

This was a Moleskine Hack that I made by drawing a Maker Notebook in photoshop and applying the printed cover over a Moleskine. I think there should be a Maker Mini Notebook this size so I made one!

From Reese Baird:

This is my 120 dB Personal La Cucaracha System. It is heavier than it looks.

From Roland Kippenhan:

This is a picture of my usb drive in a car. The car still rolls, and the 16GB drive is quite functional.

From Sarah Pollock:

This is a cat bowl water monitor that my husband built after visiting last years Maker Faire. He bought a copy of Making Things Talk, an Arduino starter kit, and a BoArduino, and set to work to make this little gadget. When the water gets low, it emails us letting us know, and emails every few hours until the bowl is refilled. Now our kitties don't have to die of thirst :-)

For more information about this project, see http://blog.andrew.net.au/2009/01/04#catbowl_monitor

From Erin Kennedy:

MANOI is a 17-DOF humanoid robot. It can play hockey, and even has a hockey stick with light sensors to sense when there is a puck/ball in front of it. This usually is followed by a shooting action to shoot the puck, and depending on a random boolean it is either a goal or a miss. If it's a goal, MANOI starts to dance around. MANOI is equipped with a Sanguino, SSC-32, and it can be controlled via a Wii nunchuck.

From Eric Lechner:

I built a kayak, a Pygmy Osprey Standard. Unlike people who are diligent, it took me a long time, and I learned all sorts of problems with working with epoxy, and some of the solutions. It's built from a kit, using a technique called "stitch and glue". The kit came with all the materials, including pre-cut mahogany marine plywood for the shell. The edges of the frame are stitched together with wire stitches, then glued. The stitches are removed, and the boat is fiberglassed inside and out. I finished construction in late 1999, painting and varnishing it in 2001, and have successfully taken it out in the ocean without sinking.

From Josh Kopel:

Flotsam is a sculpture for the upcoming Dorkbot Seattle art show "People Doing Strange Things With Electricity 4." It will eventually have 8 "spines." It is intended to be reminiscent of a digital sea urchin and be made from recycled and natural materials (as much as possible).

From Adrian Wong:

This is a laser projection platform built from the guts of a laser pointer, a PC fan, and a laser-cut acrylic platform. Two players can play the classic game of pong on any surface, from an empty blackboard to the side of a building. The design details and schematics are available from almostsquare.com.

Want to participate? Email a picture of your DIY (do-it-yourself) project to press@oreilly.com and we'll enter you into a random drawing for 2 adult weekend tickets for this year's Maker Faire, held May 30-31, at the San Mateo County Fairgrounds.

County of San Mateo Declares May 30th & 31st Maker's Weekend - Send Us a Photo for a Chance to Win 2 Weekend Passes

2009-05-22T23:58:27Z

As this year's Maker Faire gets ready to launch, San Mateo County Supervisor Carole Groom has declared May 30th & 31st "Maker's Weekend".

"I'm delighted that Maker Faire has chosen to return to San Mateo County and the Event Center. This will be a great weekend for all the creative folks of Northern California and those who visit," said Groom. "America has such a thirst right now for creative self-expression and there's an incredible need for out-of-the-box innovation. Maker Faire embodies the best of both these worlds in a way that's smart and fun. We couldn't be happier to welcome Maker Faire to San Mateo and to declare May 30th & 31st, Maker's Weekend."

This two-day, family-friendly event invites everyone to MAKE, create, learn, invent, CRAFT, recycle, think, play, and be inspired by celebrating arts, crafts, engineering, food, music, science and technology.

This year's theme is Re-Make America, inspired by President Obama's all to "begin again the work of remaking America," and will celebrate those whom he declared are "the risk takers, the doers, and the makers of things."

Interested in attending Maker Faire for free? Email a picture of your DIY (do-it-yourself) project to press@oreilly.com and we'll enter you into a random drawing for 2 adult weekend tickets for this year's Maker Faire, held May 30-31, at the San Mateo County Fairgrounds.

For more information about Maker Faire, see makerfaire.com.

Reporting for ARRA Transparency - Reporting for the American Recovery and Reinvestment Act

2009-05-11T19:48:26Z

On Tuesday, May 12, Dux Raymond Sy, author of SharePoint for Project Management, is presenting a webcast on Leveraging SharePoint for the American Recovery and Reinvestment Act (ARRA) Transparency. We had a chance to ask Dux why the ARRA is so important for today's business. Read on for his comments.

Why is this topic so important right now?

The American Recovery and Reinvestment Act of 2009 (ARRA) is an unprecedented economic stimulus package that is intended to provide a stimulus to the U.S. economy in the wake of the economic downturn. The measures are nominally worth $787 billion. Recipients of the stimulus package must be able to report on key areas such as: how the fund is being spent, status and the number of jobs stimulus-funded projects have created. In order to provide transparency, technology plays a critical role in ensuring that relevant information is available. SharePoint is well suited to address these reporting needs as it is a flexible platform that is already available in various organizations and can provide the reporting requirements of ARRA.

What kind of businesses are going to be affected by the ARRA and how will it primarily affect them?

Businesses in the Construction, Healthcare, Education, Telecommunications, and Energy sector will benefit from ARRA since these are the areas of focus.

What happens if these businesses don't have this kind of transparency in place?

The aggressive reporting requirements currently in place at the Agency and Prime recipient levels will be expanded down to the contractor and sub-contractor level in the coming months. The ability for all levels from the agency down to meet weekly reporting guidelines and regulatory oversight will create an expensive burden on each level with legal consequences.

Will this sort of reporting be required from now on, or is it a one-time, short-lived thing?

While it is not currently stated, once we have achieved this level of transparency with regard to government spending it will be difficult to rollback in the future. Citizens will witness the capability to provide accurate reporting on the spending of their taxes and will wish for government to continue and expand for all activities.

Can even beginning SharePoint users put SharePoint to use for this sort of reporting?

With the addition of some key web parts, concepts, and processes, it is possible for any level of government to do this. However, the basics are creating the processes, controls, and discipline to manage and report project activities. Doing this within an environment where external contractors are so heavily leveraged will be challenging and game changing.

According to Dux, SharePoint is precisely the kind of reliable and scalable reporting technology platform that's necessary to fulfill the ARRA reporting requirements. To learn more, be sure to register for Dux's webcast on Tuesday, May 12. It's a free, live event taking place at 10 am PT.

Talking to Tim O'Reilly about the Architecture of Participation - An Excerpt from Beautiful Teams

2009-05-11T19:37:05Z

In this excerpt from Beautiful Teams: Inspiring and Cautionary Tales from Veteran Team Leaders, editors Andrew Stellman and Jennifer Greene interview Tim O'Reilly, founder and CEO, about leadership.

Chapter One, Leadership

Tim O'Reilly started the company that became O'Reilly Media, Inc., which published this book (and our others, as well). We've always personally admired him and the company that he founded, not just for his impact on the publishing industry, but also for his impact on the larger world of software and software development. We've worked with people from almost every part of O'Reilly for years, and we recognize a great team when we see it. We wanted to talk to Tim to hear his ideas about how he built his team and continues to bring out the best in them.

Jenny: One thing we noticed when we started putting together this book was that everyone seems to have a slightly different definition of "team." And it turned out that some of the stories we included talk about that question explicitly: whether a team is always people who are together temporarily to build some specific project, or whether a team can be a group of people who've never met.

Andrew: We kind of pulled a fast one and just left it open to interpretation. So how do you define a team?

Tim: My own experiences are around running a company. Yes, there are team experiences in that. But most of the reflections I have are around the broader question of how you exert leadership. Let me start at the top, with a few thoughts about leadership and management, which are part of the whole team thing. I'm not quite sure where the boundaries are.

There are two quotes I want to start you off with. One is from Harold Geneen, who was the guy who started ITT, which was really the first modern conglomerate. And he said, "The skill of management is achieving your objectives through the efforts of others." So, that's kind of an interesting perspective. And the question is of the different styles of doing that, where some of them are very directive. This is the classic "manager"--the idea of somebody who figures out what needs to be done, and who needs to do it, and builds the teams with the roles, and so on.

While I completely subscribe to the concept, because the skill of management is indeed achieving your objectives through the effort of others, I have always worked with the framing of another quote, which is actually about writing. It's from Edwin Schlossberg, who wrote a magazine article I read early in my career, and it's probably one of the seminal things that took root in my brain. He said, "The skill of writing is to create a context in which other people can think."

Jenny: So, can you think of how you apply that idea--"creating a context in which other people can think"--to software teams?

Tim: I'll make an observation here, and it relates to something I call the architecture of participation. In 1998, we did a book called Open Sources, and we did interviews for some of the people who produced their essays on open source. And I don't think it made it into the final book, but there's something that Linus Torvalds said in an interview that stuck with me: "I couldn't have done what I did with Linux for Windows, even if I'd had the Windows source code. It just wasn't architected that way." And it really set off a chain of thinking in my mind about the architecture of open source projects, and how they're designed to allow that sort of free-form improvisation. Because there are rules that are laid down.

Andrew: It's funny to hear Linus talk about not being able to do something--it's a good reminder that SourceForge.net is littered with open source projects that never went anywhere.

Tim: I think one of the reasons why certain projects fail is because they're mixing and matching from the wrong systems. We have to have a system that has a fundamental characteristic that there are small pieces that people can work on independently. I think this is why, for example, people have said that they'll do books as Wikis, and it hasn't really taken off. Why not? Because a book is a fairly large, complex thing with a single narrative thread. Wikipedia is a set of pages, and the atomic unit of content is something that a single individual can make a plausible promise at, and other people can update and tweak. And the whole is the sum of many, many such small parts. I think, for example, that there are certain types of works that lend themselves to that kind of collaborative activity--being more free-form precisely because they're designed in such a way that the pieces fit together.

Unix is designed a little more like a set of LEGOs, where the design principles are that you have these "innies" and "outies" and they snap together. Thinking about pipes and filters and all of those kinds of things--that people can write completely independent utilities with the knowledge that they just fit. Most programs read standard in and wrote standard out, and there were a few simple rules.

Similarly, every atomic piece has a manpage. I've always thought, if I were given a choice between Windows with open source code under, say, the GPL, versus Windows with a manpage for every little bit of code, what you could swap out and what you could change. Yes, you'd have the license issue, but the license wouldn't be sufficient. On the other hand, the manpages would be sufficient, although you'd be violating a few people's copyrights.

Andrew: So, what you're saying is that a key to running a good project--open source or otherwise--is to break it down into pieces that people can understand?

Tim: What I'm saying is that there's a framework in which you organize people's creativity. And there's a whole bunch of principles there. Let me give you a concrete story. Back in 1998, when I organized what came to be called the Open Source Summit, I had this plausible story about what was happening in the industry. There were other people talking about it, notably Eric Raymond, who'd written The Cathedral and the Bazaar. I'd done the Perl conference the year before. We were all thinking about a bunch of things in parallel. Netscape had just open sourced their browser, so there was a lot of ferment. And I noticed something: the iconic story that was being put out by Eric Raymond and others left out the whole BSD world. It was all about Linux; it was all about GPL'd software; it was all about the ideals of free software.

Meanwhile, there was this whole other tradition that I looked at. And I looked at it and said, "Wow, that tradition has actually had more impact." So, for example, I'd go around and say to people, "Tell me the top five programs on the Internet." They'd scratch their heads. And I'd say, "Number one: BIND, the Berkeley Internet Name Daemon. Every one of you has a website that's depending on that program that's maintained by this long-haired programmer in Redwood City. Number two, either sendmail or Apache. Seventy-five percent of Net email is routed by sendmail." You went down the list, and it was all Berkeley software. People weren't seeing that story.

Jenny: I remember that at the time, there was a lot of controversy around choosing an open source license, GPL versus BSD. It sounds like you found a way to rise above that and bring everyone together. How did that happen?

Tim: I brought together all the GPL people and all the Berkeley people, and I said, "We have to find out what we have in common. We have to tell a story. We have a press conference at five o'clock. I don't know what we're going to tell them, but we're going to tell 'em something at five." I had a plausible idea that there was something really interesting going on here. But I didn't know that in the course of that day, wrestling through the issues, what that group of 20 people would do. Eric came in and said, "Well, we had this meeting a couple weeks ago, and Christine Peterson proposed this new name, 'Open Source'." Michael Tiemann of Cygnus says, "Well, we've been thinking about this, too. You know; the problem of the name 'free software'." And Linus Torvalds said, "I didn't realize that 'free' had two meanings in English." And Michael Tiemann says, "Well, we've been using the term 'sourceware'."

So we had a vote. And we said, "We're all going to agree on one name, and we're all going to start using it." We voted on "sourceware" versus "open source," and "open source" won. We had the press conference at five, and the rest is history.

Andrew: That's interesting, since Jenny and I have written about how setting artificial deadlines can have negative consequences for teams. It seems like it really helped you guys.

Tim: A lot of what happened was that we brought together a very short-term group of people. I had a sense that there was something there, but didn't quite know what form it would take or ought to take. But I knew I wanted to make something happen. So one of the team principles is that creating an artificial deadline--"We've got a press conference at five o'clock, guys"--can be a pretty powerful tool. Now, a lot of companies and a lot of people really misuse that. I've seen situations where project managers lie to their teams about deadlines, trying to create an artificial sense of urgency. That violates trust.

But when you do it right, identifying that sense of urgency can be really good. It's a little bit like what Alexander Pope said about writing poetry in rhyming couplets: that the narrow aperture makes creativity shoot out like water from a fountain.

In terms of teams, the other thing that I would just really observe is the power of having people with complementary strengths. Right now at O'Reilly, I have a very strong chief operating officer, who makes a huge difference. We're kind of two halves of a coin. I'm the big-scale storytelling, and I spend much less time on internal minutiae of the business, which frees me up to do a lot more with that. She's all over the day-to-day running of the business, and doing a better job of it than when I was trying to do both jobs. Understanding complementary strengths is really critical.

Andrew: I've got a question. A while back, Jenny and I wrote a piece for ONLamp called "What Corporate Projects Should Learn from Open Source." And subsequently, we've done a talk called "What Makes Open Source Work." And what we focused on is the practices they use, because that's been our "thing" in the past. You've been talking a lot about the team, the personalities, complementary strengths, how to work with the people. But one of the things we've found is that if you look at the most successful open source projects that we all know and love--like Apache, Linux, Emacs--you find a lot of great practices. The developers, the programmers, the people on the projects voluntarily adopt practices that, if they faced in the office, they'd find stifling, even though they're the same practices. Practices like adopting a very strict build and release process, continuous integration, test-driven development.

Jenny: Lots and lots of code reviews. We've actually been tasked with putting those same practices in place in corporate environments and gotten lots of pushback from the very same people. So why do you think that is?

Tim: Well, let me put it this way. Anyone who's ridden a horse knows that the secret of success is to have the horse think that it's doing what it wants. So, I think that when people feel like somebody else is telling them what to do, there's going to be resistance. If they think it's what they themselves want, then they sign right up.

Lao Tzu said this 2,500 years ago: when the best leader leads, the people say we did it ourselves.

When some people create something really wonderful with an aesthetic vision, it seems really obvious. Now that we have the iPod, how could you imagine not having touch-screen devices? I remember the first time I picked up the Kindle. I started stroking the screen and nothing happened, and it was like, "What's wrong?" Now the Kindle has its own pieces of truth in it, so to speak--like the EVDO connectivity, where you go, "Yes, that's how it's supposed to work."

Andrew: I think it's interesting that you're talking about how a product's supposed to work. Most of the discussions I've had about open source are about things like the GPL versus the BSD license versus Creative Commons, and I think people tend to get a little bogged down in those details. But you're talking about open source software as a unique way of realizing a vision, and I've never actually thought of it that way.

Tim: Well, you have to realize that people always get hung up on licensing as critical to open source. And while it is certainly important, I think as you guys have suggested here, the practices are much, much more important. When I think about what we've done in our business, first of all books have always been user-contributed. Most of our products are created by people who don't work for us. We have to put out a plausible idea of something we want to get done. We have to find someone who says, "Yeah, that's a good idea, I want that, too." We give them coaching, we bring in a host of other people to review their work inside the company and outside. We have to impose a management system, although it's often very loose. There are a lot of different ways to do it, which is one of the reasons I love Larry Wall's Perl slogan, "There's more than one way to do it." Because I don't think there's any one answer.

Andrew: Especially not in Perl.

Tim: Yes. And take the book Programming Perl. I could spend six months remaking this book to be the kind of book I'd write. Or, I could say, "Wow, it's really good just the way it is." Even though it's not the kind of book I'd have done myself. So I blessed it and sent it through.

I remember this one book we published. It's really unfortunate because the editor was really fixated on making it into a different kind of book, so she spent two years working with the authors to somehow teach them, force them, cajole them into doing something other than what they were originally imagining. She just couldn't let go. So I said, "There's a book here, but it's a different one than you're imagining." What these guys had done was written a whole bunch of disconnected pieces. I said, "All you have to do now is tell the story of the book so that this makes sense."

Jenny: I've had a lot of people on my teams tell me that Programming Perl is one of the best programming books ever written. You had an expectation of what the book was going to be, but what you were given turned out to be different--and better.

Tim: That's sort of another piece to all of this. How do you find what's true in what you're given? I think this is almost the root of intelligence that draws everything else. There are different kinds of intelligence. One kind is essentially algorithmic and manipulative: you're given all of this data, and you're good at manipulating it. You see people who seem like they're really bright in some sense, but they're really dumb in other ways. Like they don't see things that are obvious, they don't have any common sense. Then you see somebody else, and they're not that good at symbol manipulation, can't spell, and can't do math. But they're geniuses at looking at a situation and understanding what's really going on. And the smartest people have both of those qualities. They can look at the world fresh; they can look at something and say, "Wow, I see what this can be."

So, going back to that other book, the editor had all of this training, all of these ideas about how a book should look, and what should be happening here. And that got in the way of her being able to see the material with fresh eyes and understand that the process could be better by just helping the authors to do what they wanted anyway.

Jenny: Do you think that an editor--or any team leader, really--should try not to have a strong hand in pushing the authors (or the team) toward a specific goal?

Tim: Most of my experience is very much being a leader, not necessarily being a team member. A great deal of it comes from directing people, and I try to direct them in such a way that I have as little to do as possible. A lot of that requires seeing people's strengths, seeing a situation and saying, "Here's what we can make of it."

It's a kind of pattern recognition, which is going back to how I think about the process of editing. It's a little bit like what Michelangelo used to say about making a statue; that it's about finding the image that's hidden in the stone. I think editing a book is like that. Leading a project is like that. When I started telling the story about Web 2.0, it was looking at a bunch of data and uncovering the statue that was hidden in the stone. Same thing when I told the story about open source. I think that leading your team is like that also. How do you get a group of people to achieve their potential? By seeing who they are, and what they can accomplish.

Andrew: Do you think it's possible to have a great team that doesn't have a great leader? That has more of a collective leadership?

Tim: Yes, it is possible. But here's the thing. Take Apache, because I think Apache is the great example of that. Tim Berners-Lee laid down the blueprint. He said, "I've created this idea for this hypertext server, this hypertext client." And the genius of Apache was in embracing the constraints. I still remember back in the mid-'90s, this moment where Netscape had added this, Microsoft had added that, and everyone was saying, "Apache seems to be standing still. They aren't adding all these features. They aren't keeping up!" And the guys at Apache said, "Yup. What we do is a hypertext server, and we have this nice extension mechanism where people who want to do something else can add it on."

And that goes back to that architecture of participation. They didn't build this big, conglomerate, complex application. They kept to a pure vision. The vision did actually come from a visionary leader; it just wasn't part of Apache. Apache came from a group of people who were abandoned by the NCSA server team when they all went to found Netscape. And there were a bunch of customers, so they said, "We have to maintain this, and keep it going." What was wonderful about that kind of team was that they accepted the constraints that were laid down by the design of the system. They didn't try to show off their ego or their creativity.

I think a lot of the work done by the IETF (the Internet Engineering Task Force) in the early years did the same thing. There were some wonderful principles laid down, and people really honored them. If you read some of John Postel's stuff in the TCP RFC about the robustness principle, it sounds like something out of the Bible, for Christ's sake! "Be conservative in what you do; be liberal in what you accept from others." Literally, that's what it says.

The point is that if you have the system architected right, you have a better chance of success for teams. You don't want teams that are dependent on a single vision or leader, because if you lose your leader, the whole team goes "pop."

Andrew: Speaking of things going "pop," can you think of anything from your past where things didn't go so well? Maybe a disaster or two that maybe you learned something important from?

Tim: There are probably quite a few disasters over the years, some of them which were turned into successes after the fact.

These aren't disasters, but they are failures--they're choices, going back to an aspect of leadership. Things can go wrong. Look at what happened here with Yahoo! and Microsoft. Everybody's out there saying that they can't make up their minds whether Jerry Yang or Steve Ballmer is the bigger loser, because there was a badly mismanaged process without a clear vision of what was the right thing to do. But we might end up looking back and saying, "Wow, Jerry was brilliant. He kept Yahoo! independent, and then he had a strategy that pulled his hat out of the fire." Can you imagine? Just like you look back at the early history of the PC, at the deal that Bill Gates had cut with IBM. What if IBM had been the sharp dealer? We'd have a very different history. So we don't really know.

Jenny: Right, what we think of as a set-in-stone success now might not even have seemed like a wise choice at the time. You make a series of choices, and the chips fall where they may. Is that how you see your own successes and failures?

Tim: I see in those kinds of moments the impact of choices. At O'Reilly, we did the first web portal. We did GNN, the Global Network Navigator, the pre-Yahoo!, the original portal of Internet sites. I was focused and I did not want to give up control of my company, so we sold GNN to AOL. I did reason correctly that unless I wanted to give up control of my company and take on investors, I wouldn't be able to keep up with the growth of the Internet. I'd read this wonderful book called Marketing High Technology by William Davidow (Free Press), an early venture capitalist. He said that it's simple math. Dominating a market means being more than half the market, and growing faster than the market as a whole. I looked at the Internet and said, "We can't do that, a private company that's completely self-funded. With the way that the Internet's exploding, there's no way we won't get marginalized. We have to either take in money or sell." So I sold. It was a big inexperience premium, so to speak. Jerry and David took in venture money, and went on to create a multibillion-dollar company despite the recent travails. I think, from a purely financial point of view, they did much better than we did. That being said, I was clear what my personal goals were: to keep an independent company where I could do what I wanted, and I've been doing it ever since.

I guess the point I'd make is that failure and success are relative to what you're trying to accomplish. If we'd been a venture-backed start-up, my decision would have been a disastrous failure. As a self-funded entrepreneur who was making my own choices, it was my choice to make or not. What I'm trying to get at here is that it's really important not to second-guess a "failure," because a failure may be a choice.

Andrew: But don't you need to have some sort of final accounting? In the end, a project either succeeds or it fails, right?

Tim: We have this idea that's very binary, that it's failure or success. We're trying to choose from a set of alternative futures. There's no best, there's just choice.

I think we've lost in many cases, both in business and in the design of software--the design of any work product, really--the importance of the aesthetic. Wallace Stevens wrote a book of essays called The Necessary Angel (Vintage). In it, one of the points he made was that we have the idea that choice matters. He had a poem called "Notes towards a Supreme Fiction." He had the idea that perhaps God was a fiction that we can all believe in. The goal in religion or even in science is to create an aesthetic vision that we can each believe in. You're trying to enroll people.

Andrew: I can hear a reader who works in a corporate environment building a database application reading that and thinking, "There's no aesthetic in what I'm doing."

Tim: And I would say, "Go look at Steve Jobs." That'd be my answer--he's a testament to the power of the aesthetic. He made a set of choices. For many, they feel that Apple screwed up because they didn't go with the dominant paradigm. They followed their own way, and their own aesthetic vision. Each time, he's been able to come back again and again because he has a compelling vision that he's been able to sell to people. People talk about the "Steve Jobs reality distortion field"--that's what it's all about. He can create a compelling vision that other people sign up for.

I remember one of my key employees laughing at me, saying, "I'd have a meeting with you, and I'd go away, and I've said 'yes' to all these things you've said. Then I go away and think, 'I don't actually believe all that stuff!'" I was able to persuade her. And that goes back to what may be an interesting observation. I remember when my company got beyond about fifty people. There was this moment when I really had to change how I worked, because I really had this experience in the early years of the company where I held people in my "reality distortion field," so to speak--everybody in the company at the same time. It was a small enough group and we were all working closely together.

I remember when I was in high school I used to sneak out at night in my father's car. So he wouldn't be able to hear, I'd have to push it out of the driveway and down the block before I started the engine. So I'd be pushing the car, and there'd be this feeling that you're pushing on this thing, and you're pushing and pushing, and gradually it starts to accelerate. I had that feeling with the people in the company--"Wow, this thing is just too heavy." So I had to let go. Then, I would talk to a few people at a time. You'd set people free to go do their thing, and then you'd check in.

There's a science fiction book I read early in my career that was very influential, a book called Rissa Kerguelen by F. M. Busby (Berkeley). Not that many people have read it. One of the key concepts that many people have played with over the years, particularly in the '50s and '60s (this book was written in the '70s), was the idea of time dilation. As you get close to the speed of light, one of the Einsteinian paradoxes is that the inertial frame is different and time goes much more slowly for the person traveling at high speed. Many science fiction stories would involve people going off and coming back, and everybody's very old. So this book, Rissa Kerguelen, had three parts, and one of them was called "The Long View." It was about how you do planning when you're about to go on an interstellar voyage, and you're going to show up 15 years later by planetary time. The idea is that you have to set something in motion, and then meet up with it. And I think there's something very powerful in that image, the fact that the things we were talking about earlier--the architecture of various systems--that's a way you set something in motion. And you can rendezvous with it, and find if it's developed in the way you expected. That was how I began to think about what I had to do in the company. I had to set things in motion and then go meet up with them.

Jenny: How did your team initially react when you started pulling away?

Tim: There are definitely cycles where people feel their own oats. They want to be the leader, and they want to take things in a different direction. And I think one thing about being a good leader is to know when somebody has the chops to do that. I'm a Celtics fan, and I remember there was a great story from the Larry Bird era. It was crunch time at the end of a basketball game--this was when K. C. Jones was coach. They'd come to the huddle, and Larry says, "Just give me the ball and everybody get out of the way." And K. C. says, "Larry, I'm the coach, so shut up! OK, everybody? Let's get Larry the ball, and everybody get out of the way."

So, sometimes people move ahead, and your job as coach or leader is to say, "Yeah, they're right, give it to them." I've had that relationship with Dale Dougherty a lot. Dale is actually responsible for a big part of O'Reilly's success. A great many things that are attributed to me, he played a major role in. He was the guy who originally got us into the World Wide Web. He was the guy who came up with the name "Web 2.0." He's currently the publisher of MAKE Magazine. It kind of feels like a dance in which he goes off and does his own thing, and then it comes back together. Sometimes it feels like a sibling relationship, where we're struggling over who's driving the bus. Other times we're really in harmony. You want people who will argue with you. You want people who have their own vision.

When you have a vision of something that is true--and I don't know what true means, if it's absolute truth or just aesthetic truth. When things just work, when you hear that perfect chord in music, when you see the line in a drawing or the curve of stone in a statue, and you go, "Beauty is truth, and truth beauty, that is all ye know on earth, and all ye need to know"--you know, the Keats line--that's how things come together. For me, the essence of getting people to work together is to have an aesthetic vision that you can get them to sign up for. Where you build a shared vision of the truth that you're building, where you've expressed an ideal. Because then you set people free to pursue that ideal on their own.

If you enjoyed this excerpt, buy Beautiful Teams here.

Women in Tech Rock! Join our Mother's Day Celebration - Give a Tech Book & Get 40% Off

2009-05-08T15:43:28Z

Twitterer @HilaryP is a developer, a technical evangelist, and the pregnant mommy of a toddler. And though she likes girly things like yoga and beauty products as much as the next gal she was also looking for a Mother's Day discount that appealed to her geeky desires. Tim O'Reilly, founder and CEO of O'Reilly Media agreed. And that's why we're now celebrating Mother's Day with a 40% off discount on tech books on oreilly.com. Use code MDAYT.

We're also posting photos of Moms and Grandmas with their favorite tech books here. Please send us your fav pics to press@oreilly.com.

Here's Bobbi Fox's offering: "(Actually, I'm also a technical grand-mom!) I've been in the software development biz since 1974, and have accumulated many O'Reilly books along the way. Attached is a picture of me with about half of my O'Reilly collection. Happy Mother's Day!"

And a great pic from Brynna Donn.

In the pic, Moya Watson and Leanne Waldal with daughter Lucy.

Susan is a professor of media communication & technology and Garen just graduated wth honors and is going on to Columbia U. for advanced studies in the fall. We are all very proud-- and exhausted. Thanks for the discount I am looking for a good book on Joomla and MySQL. This is a nice present!

And from the @HilaryP who got this celebration going!

Kathy Brade is mom to two kids and a long-time software developer. Happy Mother's Day to all of the other technical moms out there!

Priscilla Oppenheimer is a nerd and so is jer mother! She's 85. "She learned programming in her 50s about the same time as I was learning it in my 20s. O'Reilly books played an important part in our success!"

And Make managing editor Shawn Connally penned a tribute to her mom's craftiness.

Kimberly Kohler shares this with us: "This is me demonstrating my veteran geek-mom multitasking skills. The baby in the picture is actually my nephew at 3 days old, but the ability to juggle laptop and sleeping baby was learned years ago with my own two kids (and one more on the way!)

"Storytime at my house," writes Bonnie MacKellar, a software engineer, with her 3 kids Malachi, Naomi, and Elias. "Hopefully the technical bug will rub off on all of them!"

From Sarah Mei: that's me and my 5mo son. Not pictured: 4yo daughter (but that's her artwork in the background). Thanks for the mother's day discount!

Laura "@Pistachio" Fitton (founder, www.oneforty.com & co-author, Twitter for Dummies) has been cramming on the Twitter API book. Ahem, and has two daughters "S & Z" ages 3 and 2

"My name is Radhika Srivatsa, have been into software development since 1989 and am now with the Sony Playstation group. Here I am with some books of mine and my daughter Bhavani who loves to get into software Engineering like her Mom. I look forward to getting new books from O'Reilly and this is only about 1/4th of my collection!! I love your Learning and Programming series of books. The books make me very self-sufficient in understanding any new concepts - thank you!! Happy Mother's Day!!"

From Elizabeth Cortell: I am a second-generation geek mom (computing is matrilineal in my family) and the creator of the Mod Perl Lunchbox., presented at OSCON 2005.

From Gloria W: "My children grew up watching me write software. They understand and appreciate my obsession, even though they don't share it."

My name's Sharon Stern. I build ontologies and do semantic modeling. I have a two-year-old son and a fifteen-year-old daughter. Here's a photo of me with my son, Henry, and one of my favorite O'Reilly books. I told my daughter that, while I'll always cherish the heavy glass statue of a bird she gave me last year, I'd really like a copy of Practical RDF this year. Here's hoping! Thanks so much for doing this. It's great to see all the photos of moms who tech, and the discount's great, too."

From Donna Cornwell: "These are a few of my O'Reilly books that I have at home. My three kids like programming too--the ten year old is learning html, the 15 year old is working on LabView and the 18 year old has been pilfering my Python books! I have been doing less and less programming over the years but expect to get back into it as the kids grow up and leave home."

Rebeca and baby Sarah 10 years ago :)

From Cathy: "My daughter often climbs on my lap when I'm reading. It will be fun to be able to link my friends to this O'Reilly project when this pic is posted."

From Shanna Leonard: "This picture shows me in my favorite off-hours pursuit. (Along with playing Mafia Wars, that is.) After a long Mothers' Day spent following kids around from the pool, to the basketball court, and to their Jazz concert, there's nothing better than relaxing with a glass of wine and firefox set to my safari bookshelf."

Josh Clark on iWork '09: The Missing Manual

2009-05-06T19:15:56Z

I recently had a chance to catch up with Josh Clark, author of our newly released iWork '09: The Missing Manual. He had a lot to say about the new version of iWork, as well as a few tips & tricks to share to make the user experience even better.

Mary Rotman: What was your goal in writing this book, Josh?

Josh Clark: My goal is to help people use iWork to have a better, more creative work life. I've always believed that a great working environment makes for great work. Many of us take care to arrange our desks and offices to reflect that, but we often don't think much about our software environment--the screens where our eyes and minds spend much of the day. For decades, word processors and spreadsheets have been bland, even frustrating, work environments. Bland gets the job done, sure, but it doesn't inspire. iWork '09 is a real departure, providing a bright, creative workspace worthy of your work and ideas.

I wrote iWork '09: The Missing Manual to help readers understand not only how to use iWork's various features but also how it can help them be at once more productive and more creative. With Pages, Keynote, and Numbers, Apple gives us a fresh take on the word processor, presentation software, and spreadsheet. While familiar, all three programs are remarkably different from traditional productivity software like Microsoft Office. iWork puts an unprecedented emphasis on the design and polish of your final document, making it almost effortless to create results that look not only professional, but actually stunning. This isn't about glitz, it's about the quality of your working life: When even your spreadsheets sparkle, it's hard not to feel energized about your work.

MR: What will people walk away with as result of reading this book?

JC: The whole focus of the book, and of iWork itself, is to create gorgeous documents. After you put down this book, you'll have the know-how to create documents, newsletters, spreadsheet reports, and slideshow presentations that rival the pros. Never fear: If you're somebody who was out shopping for plaid pants the day they handed out design sensibility, the book shows you how to lean on iWork's superb collection of Apple-designed templates to turn out dazzling documents.

With nearly 900 pages, of course, the book isn't just about surface matters of design. You get all the nitty-gritty details of how to use iWork to capture, organize, and share your ideas and information. You'll learn how to craft perfect prose with Pages' typo-busting power tools. You'll learn how to build budgets, grade books, and invoices with Numbers' formulas and functions. And you'll learn to share your files with iWork's collaboration tools, including the new iWork.com.

Beyond just the mechanical aspects of using iWork, the book gives you practical aesthetic advice about document layout--tried-and-true principles of graphic design that the pros use to weave elegant presentations. In that vein, some of my favorite parts of the book depart from the typical, run-of-the-mill manual. There's a section about using grid frameworks to create sturdy and beautiful layouts in Pages. There's a whole chapter on planning and presenting Keynote-powered talks that won't turn into snoozefests. And another section walks you through responsible data presentation, helping you use Numbers' stunning chart features without losing your message in the razzle dazzle.

Throughout, you'll find carefully constructed sample documents showing how to put iWork's tools to best effect and, hopefully, provide a hint of inspiration for your own work.

MR: Who is your intended audience? Could someone who knows nothing about iWork pick this up & walk away with a working knowledge of the programs?

JC: This book really has two audiences. It's appropriate for first-timers who want to learn to use a word processor, spreadsheet, or presentation software. But it's also a great choice for individuals, educators, and small businesses looking for a vibrant alternative to Microsoft Office. Even advanced spreadsheet jockeys, for example, will learn how Numbers can help juggle numbers with greater sophistication--and with more style, too.

MR: What's the difference between iWork and Microsoft Office? If someone was trying to choose between the two, what would you suggest?

JC: The big headline here is that, in iWork '09, Microsoft Office finally has some real competition on its hands. While iWork can't beat Office in a feature-counting match, most folks won't even notice its "missing" features. Instead of Office's laundry list of esoteric options, iWork gives you an elegant, refined workspace that makes it genuinely easy to churn out designed documents that would be a headache (often impossible) to create with Microsoft Office. The changes introduced by iWork '09 focus mainly on polish and power features, an indicator that this software suite has already reached a certain level of maturity. The feature gap with Microsoft Office continues to shrink.

In the end, though, comparing iWork with Microsoft office is apples and oranges. At heart, iWork has a very different feel and approach. Word, Excel, and PowerPoint offer a deep feature list, but that depth also contributes to a heavy interface and, often, a frustrating hunt for features that should be easy to find. By contrast, Pages, Numbers, and Keynote have a lighter, more flexible interface that's simply a pleasure to use. Along with superior attention to document design, this creative workspace is an overall advantage that many will appreciate more than any absent features.

Whether or not you personally use Microsoft Office, of course, it's crucial to be able to exchange files with people who do. The iWork programs can read and create Word, Excel, and PowerPoint files, and overall they do a good job at it. There are occasional hitches, though, and this book details where you'll run into problems with your imports and exports so that you can work around potential pitfalls.

MR: What sets this book apart from others on iWork '09?

JC: I doubt that any other software manual has ever featured quite so many superheroes. Most of the book's sample documents follow the further adventures of a company called Up & Away, Megaville's leading superhero outfitter, where no task is impossible and every cape fits perfectly. The book uses Pages to design a real-estate catalog for secret fortresses; Numbers goes to work to organize a sidekick academy; and Keynote explores the highs and lows of superhero uniforms. You, of course, are the hero of this particular story. Well before the end of this book you will have discovered your superpower: an unwavering ability to create spectacular documents. How can you resist?

Josh also shared a few of his favorite tips & tricks from the new book.

Get off the email merry-go-round with iWork.com
New in iWork '09, iWork.com is Apple's Web-based service for sharing Pages, Numbers, and Keynote documents online with invited guests. Think of it as an alternative to the email merry-go-round with which we're all too familiar: You email a document to a group for feedback, and they respond by emailing their comments to you, or even to the entire group, attaching their own edited document for good measure. A flurry of electronic mail ensues, and you're stuck trying to sort out who changed what and where. iWork.com sidesteps the mail storm by providing a centralized place for document review. (iWork.com is currently available only as a free "beta" preview; Apple says it will eventually become a paid service.)

To share your file at iWork.com, open the document in Pages, Numbers, or Keynote, and click the toolbar's iWork.com button, or choose Share>Share via iWork.com. If you've done this before, you may already be signed into iWork.com; if not, enter your Apple ID and password. Type the names or email addresses of the people to invite, click Share, and the file zooms out to iWork.com. Your guests receive emails with a link to the page where they can read and hold forth on your document.

Although you can't edit files online, you and your guests can browse and comment on them right in the browser. The iWork.com web site is remarkably good at recreating the exact look of your document, no download required. No matter how complex your page-layout confection might become, iWork.com always displays a pixel-perfect replica within the web site itself. It doesn't matter what software your team may (or may not) have installed--Mac or Windows, iWork or Microsoft Office, or none of the above. As long as your reviewers have a web browser and an Internet connection, they can see your document just as you designed it, with all the fonts, photos, charts, and flaming logos you could muster. You can also download the file from iWork.com in any of several formats. A Pages document, for example, may be downloaded as a Microsoft Word, PDF, or Pages file.

Broadcast your slideshow on YouTube If iWork.com documents are aimed at small groups of friends and coworkers, YouTube movies are just the opposite--videos aimed at getting the attention of the whole world. You can add your Keynote slideshow to YouTube's boisterous collection of online movies--an easy way to share your slideshow and allow others to embed it in their own blog or web site, too. Your YouTube slideshow movie can flip through your slides at preset intervals, or even better, you can record your own narration (from your live presentation, for example) and have the slides change on cue--the next best thing to actually being there. To record your presentation and send the slideshow to YouTube:

Choose File>Record Slideshow. Start giving your presentation, speaking clearly to record your narration, and advancing through each slide as if you normally would. When you're done, press the Escape (Esc) key to stop the slideshow and save your recording.

Choose Share>Send To>YouTube, and Keynote shows you a form to provide your YouTube account information and details about the slideshow.

Click Next, and Keynote asks you a few questions about how you to format your slideshow movie. Choose "Recorded" and turn on the "Include the slideshow recording" checkbox.

Click Next, and Keynote treats you to a screenful of legalese from YouTube. Click Publish to agree to the web site's legal terms and conditions, and Keynote churns out the movie and uploads it to YouTube.

Build charts by the Numbers
Both Pages and Keynote let you build eye-popping charts from scratch, but they rely on the simple but clunky Chart Data Editor to do so. That's fine for simple data sets, but it's not ideal for larger collections of numbers, and the Chart Data Editor doesn't let you add formulas for automatic calculations as you can in Numbers tables. Save yourself time and frustration by creating your charts in Numbers and then pasting them into Pages or Keynote. They don't call it Numbers for nothin': The iWork spreadsheet is a natural at juggling data. Let Pages and Keynote focus on your words and slide design, while Numbers manages your digits. And here's the cool part: Behind the scenes, iWork '09 links your Pages or Keynote chart to the original Numbers spreadsheet; after you edit the data in Numbers, you can tell Pages or Keynote to fetch the revised data and update your chart with the click of a button. Here's how it works:

Create your chart in Numbers, and save the Numbers spreadsheet.

Copy the chart from Numbers, and paste it into Pages or Keynote. (Choose "Edit>Paste and Match Style" to make the pasted chart match your document's standard styles.)

When you select a chart pasted from Numbers, iWork indicates that it's linked to a Numbers spreadsheet by showing an arrow pointer immediately to the right of the chart. Click the Refresh button to reload the data from the Numbers document. Or click the link to open the Numbers spreadsheet and edit the source data.

Keep your paste clean
Did you catch that pointer in step 2 of the last tip? The "Paste and Match Style" command in all iWork applications (and many other Mac apps) lets you paste text and objects into your document with the same style of the surrounding text. Normally when you paste text, it hangs onto its original text style--sometimes with unhappy results. For example, if you copy a few words out of an italicized caption and paste them into a large, bold headline, the result is a headline with a few small italic words. Yuck. To sidestep this problem, all three iWork programs offer a style-conscious pasting option: Choose "Edit>Paste and Match Style" (or press Command-Option-Shift-V), and the pasted words assume the style of the paragraph where they land. In the example just mentioned, the small italic words automatically take on the big, bold headline style when pasted into the headline.

Use full-screen editing for big-screen collaboration
The new full-screen editing feature in Pages '09 draws a virtual curtain behind your document, banishing everything from your screen except for the document text. Even the menu bar, toolbar, and scroll bars disappear, leaving nothing but you and your words--along with a word and page count at the bottom of the screen. It's a great feature that helps you focus by making your document the one and only thing on your screen, hushing the siren calls of email, Web, and all the other goodies vying for your attention. But it turns out that it's also a handy way to review a document in meetings, too. Fill the entire screen with your prose-- big enough for several people huddled around your Mac to see, or for an entire room when you're projecting onto a screen.

Here's how:

Click the Full Screen button on the toolbar or choose View>Enter Full Screen.

Push your cursor to the top of the screen to reveal the menu bar.

Choose Fit Width from the View pop-up at the top right corner to make your document fill the entire screen.

Work your Address Book
Pages and Numbers are social butterflies, flitting easily through your Mac's Address Book application to make it easy to incorporate contact info into your documents. The mail merge feature in Pages makes it easy to automatically address letters, for example. When you open a template for a letter or invoice, Pages automatically enters your return address. Adding the recipient's address is as easy as dropping her Address Book card into the page. Similarly, if you have a form letter to send to lots of people at once, you can automate your mass mailing by dragging the card for an Address Book group into the window (or, new in iWork '09, by pointing Pages to a Numbers spreadsheet with all the contact info--or any kind of info, for that matter). Pages creates or prints a fresh version of the file for each person in the group, using their own individual contact info. Nice.

You can likewise load up a Numbers spreadsheet with contact info by dropping an Address Book card for either an individual or a group into a blank area of the sheet canvas. Numbers creates a brand new table with contacts listed in rows and address info in columns. The table doesn't look like much at first, showing only four columns of data: Last Name, First Name, Phone, and Email. Under the surface, though, the new table contains complete contact info for all of your contacts--Numbers is simply holding them back as hidden columns. To view the hidden columns, select the table, and choose "Table>Unhide All Columns". Numbers reveals the full cache of contact info--65 columns of address info in all. Alas, now you've probably got more info than you wanted: Go ahead and hide or delete any columns you don't want.

Make the iWork connection
iWork doesn't just help you maintain connections with your contacts; it helps you connect layout objects, too. New in iWork '09, connection lines let you draw lines in a variety of different strokes to connect two objects in your layout--great for diagrams, flow charts, or anywhere it's useful to suggest a visual link between two objects. Connection lines are available in all three iWork apps; select two objects and choose Insert→Connection Line, and an unadorned, straight line appears between the two objects. Click the connection line to select it, and iWork reveals three circular controls: a blue one on either end, and a white one in the center. Drag the white control to move the line, turning it into a bendy line; the line always passes through the white control, curving as necessary to keep the two objects connected. You can style your connection lines with the full range of iWork stroke styles and add arrow endpoints for emphasis.

Keep your cool by freezing header rows
As soon as you start working with more than a casual sprinkling of data, it doesn't take long before your tables extend well beyond the height or width of a single screen. But since the header row and header column are tied firmly to the top and left of the table, scrolling away from the first row or column means that your data labels slip out of view, too. And right on cue, you immediately forget what the heck all those columns of numbers represent. Happily, Numbers can put that forgetful scenario on ice by freezing your header rows or columns. When you turn on this feature for a table, then scroll its header row or column off the screen, Numbers freezes it in its tracks, stopping it at the edge of the visible sheet canvas to float above the rest of the table below. This keeps your data labels in view even when the "real" header is hundreds of rows offscreen. To freeze your header row(s), select the table, and then choose Freeze Header Rows from the Table menu or from the Header Row button in the Format Bar or Table Inspector.

For more information on iWork '09: The Missing Manual, see:
http://oreilly.com/catalog/9780596157586.

Sarah Milstein and The Twitter Book featured on The Agenda - Canada's Top Current Affairs Show Talks about Twitter and Its Impact on Journalism

2009-05-05T19:30:52Z

Hosted by award-winning journalist Steve Paiken, The Agenda explored Twitter and it's impact on how we get and share the news in last night's show. Twitter expert and coauthor of The Twitter Book, Sarah Milstein, joined the Toronto based show from a studio in San Francisco. Jay Rosen, professor with the Arthur L. Carter Journalism Institute, Amber MacArthur, new media journalist and web strategist, Mathew Ingram, the communities editor at The Globe and Mail, and David Cohn, founder of Spot.us--a nonprofit project to pioneer community funded reporting--also joined the discussion. Take a look.

Want to get The Twitter Book? The ebook is available now.

And if The Agenda inspired you to learn more about using Twitter effectively, don't miss the Twitter Boot Camp in New York City on June 15. Presented by O'Reilly Media, Twitter Boot Camp brings together an exciting roster of leading Twitter experts--including O'Reilly Media founder and CEO Tim O'Reilly, Edelman Digital SVP Steve Rubel, Zappos CEO Tony Hsieh, and more--in an exciting and affordable one-day conference.

"Twitter Boot Camp is focused on the business use of Twitter," explains Tim O'Reilly. "A lot of companies don't understand how to use Twitter effectively. Is it for customer service? Is it for marketing? How do you actually engage? How do you do it well? So we got together a set of speakers who will help business users understand effective twittering. Some of that is a little bit of a missionary effort because I don't want them to change the nature of the medium. I want them to learn how to make their efforts personal."

Adds O'Reilly: "I think any business user of Twitter will get a lot of learning from the practitioners who are on the cutting edge of what I think is going to be one of the most significant new tools that we've seen for business in a long time."

Even better, when you register for the Twitter Boot Camp and 140 Characters Conference you'll save. Hope to see you there.

What Can You Do with XMPP?

2009-05-04T21:39:20Z

The following is an excerpt from the book XMPP: The Definitive Guide by Peter Saint-Andre, Kevin Smith, and Remko Tronçon. It has been adapted for the Web.

"XMPP is the secret weapon of the most groundbreaking websites, as well as the infrastructure behind hte fastest growing instant messaging systems. Not only is this sorely needed bible written by a dream-team of XMPP experts, the facts are interesting, the jokes are amusing, and quite honestly, I really wish I'd written it."
-Dave Cridland, XMPP Lead, Isode Ltd.

What Can You Do with XMPP?

The Extensible Messaging and Presence Protocol (XMPP) is an open technology for real-time communication, using the Extensible Markup Language (XML) as the base format for exchanging information. In essence, XMPP provides a way to send small pieces of XML from one entity to another in close to real time.

XMPP is used in a wide range of applications, and it may be right for your application, too. To envision the possibilities, it's helpful to break the XMPP universe down at a high level into services and applications. The services are defined in two primary specifications published by the Internet Engineering Task Force (IETF) at http://ietf.org/ (the "RFC" series), and in dozens of extension specifications published by the XMPP Standards Foundation at http://xmpp.org/ (the "XEP" series); the applications are software programs and deployment scenarios that are of common interest to individuals and organizations, although the core services enable you to build many other application types as well.

RFC Revisions
As of this writing, RFC3920 and RFC3921 are under active revision to incorporate errata, clarify ambiguities, improve their readability, define additional error codes, etc. These documents, called rfc3920bis and rfc3921bis in the terminology of the IETF, provide the most accurate definition of XMPP and might have been published as replacement RFCs (with new numbers) once you read this book. For the latest versions of the revised specifications, visit http://xmpp.org.

Services

In this context, a service is a feature or function that can be used by any given application. XMPP implementations typically provide the following core services:

Channel encryption

This service, defined in RFC3920 and explained in Chapter 12 of this book, provides encryption of the connection between a client and a server, or between two servers. Although channel encryption is not necessarily exciting, it is an important building block for constructing secure applications.

Authentication

This service, also defined in RFC3920 and explained in Chapter 12 of this book, is another part of the foundation for secure application development. In this case, the authentication service ensures that entities attempting to communicate over the network are first authenticated by a server, which acts as a kind of gatekeeper for network access.

Presence

This service, defined in RFC3921 and explained in Chapter 3 of this book, enables you to find out about the network availability of other entities. At the most basic level, a presence service answers the question, "Is the entity online and available for communication, or offline and not available?" Presence data can also include more detailed information (such as whether a person is in a meeting). Typically, the sharing of presence information is based on an explicit presence subscription between two entities in order to protect the privacy of user information.

Contact lists

This service, also defined in RFC3921 and explained in Chapter 3 of this book, enables you to store a contact list, or roster, on an XMPP server. The most common use for this service is an instant messaging "friend list," but any entity that has an account on a server can use the service to maintain a list of known or trusted entities (e.g., it can be used by bots).

One-to-one messaging

This service, defined in RFC3920 and explained in Chapter 4 of this book, enables you to send messages to another entity. The classic use of one-to-one messaging is personal IM, but messages can be arbitrary XML, and any two entities on a network can exchange messages--they could be bots, servers, components, devices, XMPP-enabled web services, or any other XMPP entity.

Multi-party messaging

This service, defined in XEP-0045 and explained in Chapter 7 of this book, enables you to join a virtual chat room for the exchange of messages between multiple participants, similar to Internet Relay Chat (IRC). The messages can be plain text, or can contain XML extensions for more advanced functionality, such as room configuration, in-room voting, and various session control messages.

Notifications

This service, defined in XEP-0060 and explained in Chapter 8 of this book, enables you to generate a notification and have it delivered to multiple subscribers. This service is similar to multi-party messaging, but it is optimized for one-to-many delivery with explicit subscriptions to specific channels or topics (called "nodes").

Service discovery

This service, defined in XEP-0030 and explained in Chapter 5 of this book, enables you to find out which features are supported by another entity, as well as any additional entities that are associated with it (e.g., rooms hosted at a chat room service).

Capabilities advertisement

This service, defined in XEP-0115 and explained in Chapter 5 of this book, is an extension to the presence service that provides a shorthand notation for service discovery data so that you can easily cache the features that are supported by other entities on the network.

Structured data forms

This service, defined in XEP-0004 and explained in Chapter 6 of this book, enables you to exchange structured but flexible forms with other entities, similar to HTML forms. It is often used for configuration and other tasks where you need to gather ad-hoc information from other entities.

Workflow management

This service, defined in XEP-0050 and explained in Chapter 11 of this book, enables you to engage in a structured workflow interaction with another entity, with support for typical workflow actions, such as moving to the next stage of a business process or executing a command. It is often used in conjunction with data forms.

Peer-to-peer media sessions

This service, defined in XEP-0166 and explained in Chapter 9 of this book, enables you to negotiate and manage a media session with another entity. Such a session can be used for the purpose of voice chat, video chat, file transfer, and other real-time interactions.

These are some of the core services available to you (or your application) as a participant in an XMPP network. The XMPP developer community has defined additional features in various XMPP extensions, but here we focus on the services that we think you will find most useful in building real-time applications.

Applications

Given that you have a dozen core services at your disposal, what can you build? Here are a few possibilities:

Instant messaging

The classic instantmessaging systems that most people are familiar with combine three of the core services: presence, contact lists, and one-to-one messaging. Such systems can and often do include more services and features, but if you have these three services, you can build a bare-bones IM application.

Groupchat

The multi-party messaging service enables you to build groupchat systems similar to IRC. Often, groupchat systems are used for more specific applications, such as real-time trading systems in the financial industry, situation rooms for first responders and military personnel, and virtual classrooms.

Gaming

Combined with custom extensions, both one-to-one messaging and multi-party messaging enable you to build simple gaming systems. For example, the Chesspark service (http://www.chesspark.com/) is built entirely using XMPP. Other game developers are using XMPP to add presence and contact list features to existing multi-party games.

Systems control

The combination of one-to-one messaging and data forms makes it possible to deploy lightweight systems for control of and interaction with remote systems. Deployed applications in this domain include network management, scientific telemetry, and robotic control.

Geolocation

The XMPP notification service is payload-agnostic. One defined payload format is geolocation, which enables you to build fascinating location-based applications, such as vehicle tracking.

Middleware and cloud computing

A number of companies and research groups are actively working on XMPP-based systems for computation services, lightweight middleware, and management of cloud computing infrastructures. While the use of XMPP may be surprising here because such applications have traditionally relied on heavyweight messaging technologies, we have seen XMPP begin to nibble away at the lower end of this market. It appears that companies that already have an XMPP infrastructure in place figure they might as well use it for non-IM use cases. These systems often use the workflow extensions we explore in Chapters Chapter 6 and Chapter 11 for structured message exchange. Specific applications include bioinformatics.

Data syndication

Popular social networking applications are increasingly using the XMPP notification service to solve a particular problem they have: constant polling for updated information. Existing HTTP-based deployments have been found not to scale, because quite often a particular feed has not changed since the last time it was polled. By contrast, the XMPP notification service sends out an update only when a feed has changed, saving a significant amount of bandwidth and server resources that otherwise would be wasted on polling.

Voice over IP (VoIP)

The Google Talk application that launched in August 2005 first popularized the use of XMPP for voice chat. Since then, the XMPP extensions for media session services (called Jingle) have been formalized through the XSF, and have been implemented and deployed by the likes of Nokia and the One Laptop Per Child project. The same extensions can also be used to negotiate a wide range of media session types, including video, file transfer, whiteboarding, and collaborative editing.

Identity services

Given the existence of stable identifiers (JabberIDs) and a robust authentication service, it is possible to use XMPP in building identity and authorization services such as OpenID and OAuth.

Other application examples include data transfer, live chat integrated into websites, mobile device communications, and presence-enabled directories. We will mention relevant applications throughout this book to illustrate the most popular and interesting uses of XMPP.

Although we highlight many applications of XMPP, unfortunately we can't cover all of them. Not only do we lack the space and time, but the list keeps growing every day. Moreover, the most cutting-edge uses of XMPP are not standardized yet, which makes them too much of a moving target to describe in a book. Examples of ongoing work at the time of this writing include collaborative document editing, whiteboarding, calendar integration, file sharing, and personal media networks. If you want to learn more about these topics, we suggest that you get involved with the XMPP community (see Chapter 13) as we define new ways of using XMPP.

What does the future hold for XMPP technologies? Although we don't know for sure, the trends seem clear: deployment of XMPP systems at more organizations and service providers, XMPP interfaces to more web applications, use of XMPP features to solve more business problems, and continued growth in the XMPP developer community. It's an exciting time to be working on XMPP technologies, and we invite you to join the conversation!

Brief History

Jabber/XMPP technologies were invented by Jeremie Miller in 1998. Jeremie was tired of running four different clients for the closed IM services of the day, so in true open source fashion, he decided to scratch an itch, releasing an open source server called jabberd on January 4, 1999. Before long, a community of developers jumped in to help, writing open source clients for Linux, Macintosh, and Windows; add-on components that worked with the server; and code libraries for languages such as Perl and Java. During 1999 and early 2000, the community collaboratively worked out the details of the wire protocols we now call XMPP, culminating in the release of jabberd 1.0 in May 2000.

As the community grew larger and various companies became interested in building their own Jabber-compatible (but not necessarily open source) software, the loose collaboration evident in 1999 and 2000 became unsustainable. As a result, the community (spearheaded by a company called Jabber, Inc., acquired by Cisco in late 2008) formed the Jabber Software Foundation in August 2001. Ever since, this nonprofit membership organization, renamed the XMPP Standards Foundation in early 2007, has openly documented the protocols used in the developer community, and has defined a large number of extensions to the core protocols.

After several years of implementation and deployment experience, members of the developer community decided to seek a wider review of the core protocols by formalizing them within the IETF, which has standardized most of the core technologies for the Internet (including TCP/IP, HTTP, SMTP, POP, IMAP, and SSL/TLS). Given that most good protocols seem to be three- or four-letter acronyms ending with the letter "P," the relevant IETF working group labeled its topic the Extensible Messaging and Presence Protocol (XMPP). After less than two years of intensive work (mostly focused on tightening communications security), the IETF published the core XMPP specifications in its Request for Comments (RFC) series as RFC3920 and RFC3921 in October 2004.

Publication of these RFCs has resulted in widespread adoption of XMPP technologies. In August 2005, the Google Talk IM and Voice over Internet Protocol (VoIP) service was launched on a basis of XMPP. Thousands more services have followed. Prominent and emerging software companies use XMPP in their products, including the likes of Apple, Cisco, IBM, Nokia, and Sun. Countless businesses, universities, and government agencies have deployed XMPP-based instant messaging systems for their users. Many game developers and social networking applications are building XMPP into their services, and a number of organizations have used XMPP as the "secret sauce" behind some of their most innovative features.

Open Source and Open Standards

Although XMPP was originally developed in the Jabber open source community, the protocol itself is not an open source project like Apache, but rather an open standard like HTTP. As a result, XMPP is an open technology that is not tied to any single software project or company. The XMPP specifications define open protocols that are used for communication among network entities. Much as HTTP and HTML define the protocols and data formats that power the World Wide Web, XMPP defines the protocols and data formats that power real-time interactions over the Internet. The protocols are as free as the air, which means they can be implemented in code that is licensed as free software, open source software, shareware, freeware, commercial products, or in any other way. This open standards model is different from the open source or free-software model for software code, wherein the code is often licensed so that modifications must be contributed back to the developers.

That said, XMPP emerged from an open source developer community, specifically the community that formed around the open source jabberd server that Jeremie Miller released. Thus there are many open source implementations of XMPP, which can be downloaded for free by end users, system administrators, and developers alike. Much of this software is focused on instant messaging, as befits a technology that started as an open alternative to closed IM silos that did not interoperate. There are open source clients for just about every operating system and device; as a result, millions of end users communicate using XMPP-based services. There are open source servers that can be deployed at companies, schools, and service providers; as a result, tens of thousands of XMPP services inter-connect every day. There are open source libraries for all the major programming languages, which can be used to write bots, components, and other real-time applications; as a result, there are thousands of active developers in the XMPP community. Much of this software is linked to from http://xmpp.org/, and we provide an overview of some of the most popular codebases in Appendix C.

Extensibility

The original Jabber developers were focused on building an instant messaging system. However, the extensible nature of XML has made XMPP attractive to application developers who need a reliable infrastructure for rapidly exchanging structured data, not just IM features. As a result, XMPP has been used to build a wide range of applications, including content syndication, alerts and notifications, lightweight middleware and web services, whiteboarding, multimedia session negotiation, intelligent workflows, geolocation, online gaming, social networking, and more.

Over the years, the developer community defined a large number of extensions to the core protocols. These extensions are developed through an open, collaborative standards process and published in the XSF's XMPP Extension Protocol (XEP) series at http://xmpp.org/. As you'll discover, the core protocols and various extensions provide a long "runway" for just about any feature you might need in developing real-time applications. But if you find that a feature is missing from the XMPP protocol stack, it is easy enough to extend the protocol for your own purpose, and (optionally) work with the community in standardizing these new features, as we discuss in Chapter 13.

Summary

In this excerpt from the book, we looked at the core services XMPP provides and sampled the kinds of applications you can build with those services. In the next sections, you'll get acquainted with the basic workings of XMPP, and then dive into each of the core XMPP services in detail. If you like what you've read here, get a copy of XMPP: The Definitive Guide

Favorite Linux Book of All Time: Linux in a Nutshell

2009-05-01T14:33:00Z

The Linux Journal's annual Readers' Choice Awards take the pulse of the Linux community, allowing readers to choose their favorites in a number of categories, and write-ins also are accepted. This year, more than 6,000 individuals voted for their favorite Linux solutions—one of the biggest turnouts in in Linux Journal history. And Linux in a Nutshell by Ellen Siever, Aaron Weber, Stephen Figgins, Robert Love, Arnold Robbins won the award for Favorite Linux Book of All Time.

It's quite an honor. We extend our thanks to all the readers who voted for us and our congratulations to the authors and editor Andy Oram. We consider the book a classic, and we're glad to know that you do, too! To see how useful the book is, check out the Linux Command Line Directory. Click on any of the 687 commands to get a description and list of available options.

Celebrate Earth Day by Sharing Your Green Living Tips

2009-04-22T18:44:03Z

We're celebrating Earth Day in our offices today by running a contest on Twitter (@oreillymedia). We're asking people how they're living green, and we've been getting a lot of great responses! Here are a few:

schneby: Walking on the trail, picking up trash with my wife while my dog chases geese.

jdpiehl #EarthDay Used zero gas by working at home and didn't turn on a single light all day (only sun light). And enjoyed Earth!

cpuspeed: not using a computer today... oops damn it

rumblestrut: I don't celebrate #EarthDay; but it's awareness changed how my family lives. We recycle now, use CF lighting, and drive less.

zstepek: I planned to celebrate by cycling rather than driving, which didn't work out. I did spend some time outside enjoying nature.

seanrea: Celebrating #EarthDay by bottling homebrew into old bottles rather than sending them off to be downcycled.

Dotsworld: Tore myself away from my work when I saw how excited my son was to plant his free sappling. Planted it together.

corewarrior: I'm celebrating #Earthday by working in my yard w/o electrical/gas power tools or mowers. Today I reduce my carbon footprint!

katiero: drinking OCSW wines. Ocsw.org

technoplastique: Switching out 15 50 watt halogen bulbs for LED bulbs that use 2 watts each and last 10 years! Yay!

hanekomu: I'm wearing a green t-shirt (from YAPC::EU 2008) and my last name is Grünauer, which roughly translates to "green meadow".

Aru: I'm spreading the word around my country about #EarthDay and not driving a car today to college :3.

mrael: I'm living green today by riding a recumbent trike: http://tinyurl.com/c9dcg5

thinkgeek: We celebrated #earthday by measuring our Tron cabinet's kWh/year, running a contest, & carbon offsetting it w/ @Terrapass .

qhartman: I'm being green by converting my landscaping around my house to permaculture and food production.

agbiotec: I celebrate earth day by wearing my organic cotton, fair-trade cap, and thinking ways to minimize my environmental footprint!

oamyoamy: Living green with patchwork coffee sleeves that let me say no to the extra cardboard sleeve with my latte! http://tinyurl.com/dyq9ms

mel21clc: Took the bus and Metro to work today, then brought my own utensils to eat out (they use plastic ones).

mindboggld: Turning off lights for 5 minutes tonight, didn't use hairdryer, and using old clothes for fabric used in projects.

JustJingle: I posted a fun Green post today and we are committing to be more diligent in all that we already try to do at home to be Green!

neimad: I drive my Prius every day!... but today I am walking to lunch and homebrewing beer. All natural Mother Earthy goodness!

offmanhattan: i'm green every day! http://offmanhattan.com is a travel blog for places outside Manhattan that you can get to without a car.

designkitten: Ride bicycle to subway station every day. Today rode bicycle entire 15 miles to work for the first time. #earthday

labelreader: Every day: walk to work, compost, recycle, pick up litter, mow with http://is.gd/tVME, and turn my PCs off. #earthday

bobholt: Ordering used boxes for my upcoming move.

caramiaculpa: Don't own a car, I use public transit. I learned to ride a bike last year to reduce footprint. And my veggie garden is great!

rhbaby: living green today by being the fat guy on the bike turning blue. Also sweaty guy on train.

ChemBob: Trying to win funding to do environmental remediation on several residential blocks in Detroit.

jamespyles: Living green today by not using microwave and picking up trash at university/greenbelt during my walk for starters.

oreeon: biking everywhere I go and planting the obligatory tree, well two in my front yard. Beautify & health 4 me & the planet

Join in on the fun and tell us how you're living green today, and every day, for a chance to win!

What's new in Windows Server 2008 R2 Active Directory?

2009-04-21T22:54:21Z

Bring your Active Directory questions and learn what's new in Active Directory for Windows Server 2008 R2. In this free webcast taking place Friday, April 24, Active Directory MVPs and authors Brian Desmond (Active Directory, 4th Edition) and Laura E. Hunter (Active Directory Cookbook, 3rd Edition) discuss exciting new features in R2 for AD including the AD Recycle Bin, AD PowerShell, the Active Directory Administrative Center, Managed Service Accounts, and more. They'll reserve half of the event time to answer questions about the presentation and Active Directory in general, so this is your chance to get the answers you need. Register now!

What Brings Out Your Inner Geek?

2009-04-14T16:38:24Z

In the trailer for Colin Moock's new training videos, Lost ActionScript 3.0 Weekend, Moock touches on the ease of finding errors, mentioning that this feature of ActionScript brings out his inner geek.

Now we're interested in hearing what brings out your inner geek. Is it programming? Is it a certain online task that speeds up the tedium of your day? Is it finding the perfect way to present your resume on your website? Whatever it is -- we'd like to know!

We'll be running a contest on Twitter later today to see what brings out your inner geek, and choosing a random response (@oreillymedia) to win a copy of Moock's Lost ActionScript 3.0 Weekend. Keep an eye on @oreillymedia for more details. Best of luck!

*UPDATE* The contest is now over -- congrats to @interslice -- today's contest winner! Thanks to all who shared what brings out their inner geek. Keep your eyes peeled for more contests in the future!

Cloud Computing - an Excerpt from Cloud Application Architectures

2009-04-09T19:14:52Z

The following is an excerpt from Cloud Application Architectures, the latest book by George Reese. It has been adapted for the Web.

Cloud Computing

The hallmark of any buzzword is its ability to convey the appearance of meaning without conveying actual meaning. To many people, the term cloud computing has the feel of a buzzword.

It's used in many discordant contexts, often referencing apparently distinct things. In one conversation, people are talking about Google Gmail; in the next, they are talking about Amazon Elastic Compute Cloud (at least it has "cloud" in its name!).

But cloud computing is not a buzzword any more than the term the Web is. Cloud computing is the evolution of a variety of technologies that have come together to alter an organization's approach to building out an IT infrastructure. Like the Web a little over a decade ago, there is nothing fundamentally new in any of the technologies that make up cloud computing. Many of the technologies that made up the Web existed for decades when Netscape came along and made them accessible; similarly, most of the technologies that make up cloud computing have been around for ages. It just took Amazon to make them all accessible to the masses.

The purpose of this book is to empower developers of transactional web applications to leverage cloud infrastructure in the deployment of their applications. This book therefore focuses on the cloud as it relates to clouds such as Amazon EC2, more so than Google Gmail. Nevertheless, we should start things off by setting a common framework for the discussion of cloud computing.

The Cloud

The cloud is not simply the latest fashionable term for the Internet. Though the Internet is a necessary foundation for the cloud, the cloud is something more than the Internet. The cloud is where you go to use technology when you need it, for as long as you need it, and not a minute more. You do not install anything on your desktop, and you do not pay for the technology when you are not using it.

The cloud can be both software and infrastructure. It can be an application you access through the Web or a server that you provision exactly when you need it. Whether a service is software or hardware, the following is a simple test to determine whether that service is a cloud service:

If you can walk into any library or Internet cafe and sit down at any computer without preference for operating system or browser and access a service, that service is cloud-based.

I have defined three criteria I use in discussions on whether a particular service is a cloud service:

The service is accessible via a web browser (nonproprietary) or web services API.

Zero capital expenditure is necessary to get started.

You pay only for what you use as you use it.

I don't expect those three criteria to end the discussion, but they provide a solid basis for discussion and reflect how I view cloud services in this book.

If you don't like my boiled-down cloud computing definition, James Governor has an excellent blog entry on 15 Ways to Tell It's Not Cloud Computing.

Software

As I mentioned earlier, cloud services break down into software services and infrastructure services. In terms of maturity, software in the cloud is much more evolved than hardware in the cloud.

Software as a Service (SaaS) is basically a term that refers to software in the cloud. Although not all SaaS systems are cloud systems, most of them are.

SaaS is a web-based software deployment model that makes the software available entirely through a web browser. As a user of SaaS software, you don't care where the software is hosted, what kind of operating system it uses, or whether it is written in PHP, Java, or .NET. And, above all else, you don't have to install a single piece of software anywhere.

Gmail, for example, is nothing more than an email program you use in a browser. It provides the same functionality as Apple Mail or Outlook, but without the fat client. Even if your domain does not receive email through Gmail, you can still use Gmail to access your mail.

SalesForce.com is another variant on SaaS. SalesForce.com is an enterprise customer relationship management (CRM) system that enables sales people to track their prospects and leads, see where those individuals sit in the organization's sales process, and manage the workflow of sales from first contact through completion of a sale and beyond. As with Gmail, you don't need any software to access SalesForce.com: point your web browser to the SalesForce.com website, sign up for an account, and get started.

SaaS systems have a few defining characteristics:

Availability via a web browser

SaaS software never requires the installation of software on your laptop or desktop. You access it through a web browser using open standards or a ubiquitous browser plug-in. Cloud computing and proprietary desktop software simply don't mix.

On-demand availability

You should not have to go through a sales process to gain access to SaaS-based software. Once you have access, you should be able to go back into the software any time, from anywhere.

Payment terms based on usage

SaaS does not need any infrastructure investment or fancy setup, so you should not have to pay any massive setup fees. You should simply pay for the parts of the service you use as you use them. When you no longer need those services, you simply stop paying.

Minimal IT demands

If you don't have any servers to buy or any network to build out, why do you need an IT infrastructure? While SaaS systems may require some minimal technical knowledge for their configuration (such as DNS management for Google Apps), this knowledge lays within the realm of the power user and not the seasoned IT administrator.

One feature of some SaaS deployments that I have intentionally omitted is multitenancy. A number of SaaS vendors boast about their multitenancy capabilities--some even imply that multitenancy is a requirement of any SaaS system.

A multitenant application is server-based software that supports the deployment of multiple clients in a single software instance. This capability has obvious advantages for the SaaS vendor that, in some form, trickle down to the end user:

Support for more clients on fewer hardware components

Quicker and simpler rollouts of application updates and security patches

Architecture that is generally more sound

The ultimate benefit to the end user comes indirectly in the form of lower service fees, quicker access to new functionality, and (sometimes) quicker protection against security holes. However, because a core principle of cloud computing is a lack of concern for the underlying architecture of the applications you are using, the importance of multitenancy is diminished when looking at things from that perspective.

As we discuss in the next section, virtualization technologies essentially render the architectural advantages of multitenancy moot.

Hardware

In general, hardware in the cloud is conceptually harder for people to accept than software in the cloud. Hardware is something you can touch: you own it; you don't license it. If your server catches on fire, that disaster matters to you. It's hard for many people to imagine giving up the ability to touch and own their hardware.

With hardware in the cloud, you request a new "server" when you need it. It is ready as quickly as 10 minutes after your request. When you are done with it, you release it and it disappears back into the cloud. You have no idea what physical server your cloud-based server is running, and you probably don't even know its specific geographic location.

The Barrier of Old Expectations

The hardest part for me as a vendor of cloud-based computing services is answering the question, "Where are our servers?" The real answer is, inevitably, "I don't know--somewhere on the East Coast of the U.S. or Western Europe," which makes some customers very uncomfortable. This lack of knowledge of your servers' location, however, provides an interesting physical security benefit, as it becomes nearly impossible for a motivated attacker to use a physical attack vector to compromise your systems.

The advantages of a cloud infrastructure

Think about all of the things you have to worry about when you own and operate your own servers:

Running out of capacity?

Capacity planning is always important. When you own your own hardware, however, you have two problems that the cloud simplifies for you: what happens when you are wrong (either overoptimistic or pessimistic), and what happens if you don't have the expansion capital when the time comes to buy new hardware. When you manage your own infrastructure, you have to cough up a lot of cash for every new Storage Area Network (SAN) or every new server you buy. You also have a significant lead time from the moment you decide to make a purchase to getting it through the procurement process, to taking delivery, and finally to having the system racked, installed, and tested.

What happens when there is a problem?

Sure, any good server has redundancies in place to survive typical hardware problems. Even if you have an extra hard drive on hand when one of the drives in your RAID array fails, someone has to remove the old drive from the server, manage the RMA,[1] and put the new drive into the server. That takes time and skill, and it all needs to happen in a timely fashion to prevent a complete failure of the server.

[1] Return merchandise authorization. When you need to return a defective part, you generally have to go through some vendor process for returning that part and obtaining a replacement.

What happens when there is a disaster?

If an entire server goes down, unless you are in a high-availability infrastructure, you have a disaster on your hands and your team needs to rush to address the situation. Hopefully, you have solid backups in place and a strong disaster recovery plan to get things operational ASAP. This process is almost certainly manual.
Don't need that server anymore?
Perhaps your capacity needs are not what they used to be, or perhaps the time has come to decommission a fully depreciated server. What do you do with that old server? Even if you give it away, someone has to take the time to do something with that server. And if the server is not fully depreciated, you are incurring company expenses against a machine that is not doing anything for your business.

What about real estate and electricity?

When you run your own infrastructure (or even if you have a rack at an ISP), you may be paying for real estate and electricity that are largely unused. That's a very ungreen thing, and it is a huge waste of money.

None of these issues are concerns with a proper cloud infrastructure:

You add capacity into a cloud infrastructure the minute you need it, and not a moment sooner. You don't have any capital expense associated with the allocation, so you don't have to worry about the timing of capacity needs with budget needs. Finally, you can be up and running with new capacity in minutes, and thus look good even when you get caught with your pants down.

You don't worry about any of the underlying hardware, ever. You may never even know if the physical server you have been running on fails completely. And, with the right tools, you can automatically recover from the most significant disasters while your team is asleep.

When you no longer need the same capacity or you need to move to a different virtual hardware configuration, you simply deprovision your server. You do not need to dispose of the asset or worry about its environmental impact.

You don't have to pay for a lot of real estate and electricity you never use. Because you are using a fractional portion of a much beefier piece of hardware than you need, you are maximizing the efficiency of the physical space required to support your computing needs. Furthermore, you are not paying for an entire rack of servers with mostly idle CPU cycles consuming electricity.

Hardware virtualization

Hardware virtualization is the enabling technology behind many of the cloud infrastructure vendors offerings, including Amazon Web Services (AWS).[2] If you own a Mac and run Windows or Linux inside Parallels or Fusion, you are using a similar virtualization technology to those that support cloud computing. Through virtualization, an IT admin can partition a single physical server into any number of virtual servers running their own operating systems in their allocated memory, CPU, and disk footprints. Some virtualization technologies even enable you to move one running instance of a virtual server from one physical server to another. From the perspective of any user or application on the virtual server, no indication exists to suggest the server is not a real, physical server.

[2] Other approaches to cloud infrastructure exist, including physical hardware on-demand through companies such as AppNexus and NewClouds. In addition, providers such as GoGrid (summarized in Appendix B) offer hybrid solutions.

A number of virtualization technologies on the market take different approaches to the problem of virtualization. The Amazon solution is an extension of the popular open source virtualization system called Xen. Xen provides a hypervisor layer on which one or more guest operating systems operate. The hypervisor creates a hardware abstraction that enables the operating systems to share the resources of the physical server without being able to directly access those resources or their use by another guest operating system.

A common knock against virtualization--especially for those who have experienced it in desktop software--is that virtualized systems take a significant performance penalty. This attack on virtualization generally is not relevant in the cloud world for a few reasons:

The degraded performance of your cloud vendor's hardware is probably better than the optimal performance of your commodity server.

Enterprise virtualization technologies such as Xen and VMware use paravirtualization as well as the hardware-assisted virtualization capabilities of a variety of CPU manufacturers to achieve near-native performance.

Cloud storage

Abstracting your hardware in the cloud is not simply about replacing servers with virtualization. It's also about replacing your physical storage systems.

Cloud storage enables you to "throw" data into the cloud and without worrying about how it is stored or backing it up. When you need it again, you simply reach into the cloud and grab it. You don't know how it is stored, where it is stored, or what has happened to all the pieces of hardware between the time you put it in the cloud and the time you retrieved it.

As with the other elements of cloud computing, there are a number of approaches to cloud storage on the market. In general, they involve breaking your data into small chunks and storing that data across multiple servers with fancy checksums so that the data can be retrieved rapidly—no matter what has happened in the meantime to the storage devices that comprise the cloud.

I have seen a number of people as they get started with the cloud attempt to leverage cloud storage as if it were some kind of network storage device. Operationally, cloud storage and traditional network storage serve very different purposes. Cloud storage tends to be much slower with a higher degree of structure, which often renders it impractical for runtime storage for an application, regardless of whether that application is running in the cloud or somewhere else.

Cloud storage is not, generally speaking, appropriate for the operational needs of transactional cloud-based software. Later, we discuss in more detail the role of cloud storage in transaction application management. For now, think of cloud storage as a tape backup system in which you never have to manage any tapes.

Note: Amazon recently introduced a new offering called Amazon CloudFront, which leverages Amazon S3 as a content distribution network. The idea behind Amazon CloudFront is to replicate your cloud content to the edges of the network. While Amazon S3 cloud storage may not be appropriate for the operational needs of many transactional web applications, CloudFront will likely prove to be a critical component to the fast, worldwide distribution of static content.

Cloud Application Architectures

We could spend a lot of precious paper discussing Software as a Service or virtualization technologies (did you know that you can mix and match at least five kinds of virtualization?), but the focus of this book is how you write an application so that it can best take advantage of the cloud.

Grid Computing

Grid computing is the easiest application architecture to migrate into the cloud. A grid computing application is processor-intensive software that breaks up its processing into small chunks that can then be processed in isolation.

If you have used SETI@home, you have participated in grid computing. SETI (the Search for Extra-Terrestrial Intelligence) has radio telescopes that are constantly listening to activity in space. They collect volumes of data that subsequently need to be processed to search for a nonnatural signal that might represent attempts at communication by another civilization. It would take so long for one computer to process all of that data that we might as well wait until we can travel to the stars. But many computers using only their spare CPU cycles can tackle the problem extraordinarily quickly.

These computers running SETI@home--perhaps including your desktop--form the grid. When they have extra cycles, they query the SETI servers for data sets. They process the data sets and submit the results back to SETI. Your results are double-checked against processing by other participants, and interesting results are further checked.[3]

[3] For more information on SETI@home and the SETI project, pick up a copy of O'Reilly's Beyond Contact.

Back in 1999, SETI elected to use the spare cycles of regular consumers' desktop computers for its data processing. Commercial and government systems used to network a number of supercomputers together to perform the same calculations. More recently, server farms were created for grid computing tasks such as video rendering. Both supercomputers and server farms are very expensive, capital-intensive approaches to the problem of grid computing.

The cloud makes it cheap and easy to build a grid computing application. When you have data that needs to be processed, you simply bring up a server to process that data. Afterward, that server can either shut down or pull another data set to process.

Figure 1-1 illustrates the process flow of a grid computing application. First, a server or server cluster receives data that requires processing. It then submits that job to a message queue (1). Other servers--often called workers (or, in the case of SETI@home, other desktops)--watch the message queue (2) and wait for new data sets to appear. When a data set appears, the first computer to see it processes it and then sends the results back into the message queue (3). The two components can operate independently of each other, and one can even be running when no computer is running the other.

Figure 1-1. The grid application architecture separates the core application from its data processing nodes

Cloud computing comes to the rescue here because you do not need to own any servers when you have no data to process. You can then scale the number of servers to support the number of data sets that are coming into your application. In other words, instead of having idle computers process data as it comes in, you have servers turn themselves on as the rate of incoming data increases, and turn themselves off as the data rate decreases.

Because grid computing is currently limited to a small market (scientific, financial, and other large-scale data crunchers), this book doesn't focus on its particular needs. However, many of the principles in this book are still applicable.

Transactional Computing

Transactional computing makes up the bulk of business software and is the focus of this book. A transaction system is one in which one or more pieces of incoming data are processed together as a single transaction and establish relationships with other data already in the system. The core of a transactional system is generally a relational database that manages the relations among all of the data that make up the system.

Figure 1-2 shows the logical layout of a high-availability transactional system. Under this kind of architecture, an application server typically models the data stored in the database and presents it through a web-based user interface that enables a person to interact with the data. Most of the websites and web applications that you use every day are some form of transactional system. For high availability, all of these components may form a cluster, and the presentation/business logic tier can hide behind a load balancer.

Figure 1-2. A transactional application separates an application into presentation, business logic, and data storage

Deploying a transactional system in the cloud is a little more complex and less obvious than deploying a grid system. Whereas nodes in a grid system are designed to be short-lived, nodes in a transactional system must be long-lived.

A key challenge for any system requiring long-lived nodes in a cloud infrastructure is the basic fact that the mean time between failures (MTBF) of a virtual server is necessarily less than that for the underlying hardware. An admittedly gross oversimplification of the problem shows that if you have two physical servers with a three-year MTBF, you will be less likely to experience an outage across the entire system than you would be with a single physical server running two virtual nodes. The number of physical nodes basically governs the MTBF, and since there are fewer physical nodes, there is a higher MTBF for any given node in your cloud-based transactional system.

The cloud, however, provides a number of avenues that not only help mitigate the lower failure rate of individual nodes, but also potentially increase the overall MTBF for your transactional system. In this book, we cover the tricks that will enable you to achieve levels of availability that otherwise might not be possible under your budget while still maintaining transactional integrity of your cloud applications.

The Value of Cloud Computing

How far can you take all of this?

If you can deploy all of your custom-built software systems on cloud hardware and leverage SaaS systems for your packaged software, you might be able to achieve an all-cloud IT infrastructure. Table 1-1 lists the components of the typical small- or medium-sized business.

Table 1-1. The old IT infrastructure versus the cloud

The potential impact of the cloud is significant. For some organizations—particularly small- to medium-sized businesses—it makes it possible to never again purchase a server or own any software licenses. In other words, all of these worries diminish greatly or disappear altogether:

Am I current on all my software licenses? SaaS systems and software with cloud-friendly licensing simply charge your credit card for what you use.

When do I schedule my next software upgrade? SaaS vendors perform the upgrades for you; you rarely even know what version you are using.

What do I do when a piece of hardware fails at 3 a.m.? Cloud infrastructure management tools are capable of automating even the most traumatic disaster recovery policies.

How do I manage my technology assets? When you are in the cloud, you have fewer technology assets (computers, printers, etc.) to manage and track.

What do I do with my old hardware? You don't own the hardware, so you don't have to dispose of it.

How do I manage the depreciation of my IT assets? Your costs are based on usage and thus don't involve depreciable expenses.

When can I afford to add capacity to my infrastructure? In the cloud, you can add capacity discretely as the business needs it.

SaaS vendors (whom I've included as part of cloud computing) can run all their services in a hardware cloud provided by another vendor, and therefore offer a robust cloud infrastructure to their customers without owning their own hardware. In fact, my own business runs that way.

Options for an IT Infrastructure

The cloud competes against two approaches to IT:

Internal IT infrastructure and support

Outsourcing to managed services

If you own the boxes, you have an internally managed IT infrastructure—even if they are sitting in a rack in someone else's data center. For you, the key potential benefit of cloud computing (certainly financially) is the lack of capital investment required to leverage it.

Internal IT infrastructure and support is one in which you own the boxes and pay people—whether staff or contract employees—to maintain those boxes. When a box fails, you incur that cost, and you have no replacement absent a cold spare that you own.

Managed services outsourcing has similar benefits to the cloud in that you pay a fixed fee for someone else to own your servers and make sure they stay up. If a server goes down, it is the managed services company who has to worry about replacing it immediately (or within whatever terms have been defined in your service-level agreement). They provide the expertise to make sure the servers are fixed with the proper operating system patches and manage the network infrastructure in which the servers operate.

Table 1-2 provides a comparison between internal IT, managed services, and cloud-based IT with respect to various facets of IT infrastructure development.

Table 1-2. A comparison of IT infrastructure options

[4] From a financial perspective, the difference between coughing up cash today and borrowing it from a bank is inconsequential. Either way, spending $40K costs you money. If you borrow it, you pay interest. If you take it out of your bank account, you lose the opportunity to do something else with it (cost of capital).

The one obvious fact that should jump out of this chart is that building an IT infrastructure from scratch no longer makes any sense. The only companies that should have an internal IT are organizations with a significant preexisting investment in internal IT or with regulatory requirements that prevent data storage in third-party environments.

Everyone else should be using a managed services provider or the cloud.

The Economics

Perhaps the biggest benefit of cloud computing over building out your own IT infrastructure has nothing to do with technology—it's financial. The "pay for what you use" model of cloud computing is significantly cheaper for a company than the "pay for everything up front" model of internal IT.

Capital costs

The primary financial problem with an internally based IT infrastructure is the capital cost. A capital cost is cash you pay for assets prior to their entering into operations. If you buy a server, that purchase is a capital cost because you pay for it all up front, and then you realize its benefits (in other words, you use it) over the course of 2-3 years.

Let's look at the example of a $5,000 computer that costs $2,000 to set up. The $5,000 is a capital cost and the $2,000 is a one-time expense. From an accounting perspective, the $5,000 cost is just a "funny money" transaction, in that $5,000 is moved from one asset account (your bank account) into another asset account (your fixed assets account). The $2,000, on the other hand, is a real expense that offsets your profits.

The server is what is called a depreciable asset. As it is used, the server is depreciated in accordance with how much it has been used. In other words, the server's value to the company is reduced each month it is in use until it is worth nothing and removed from service. Each reduction in value is considered an expense that offsets the company's profits.

Finance managers hate capital costs for a variety of reasons. In fact, they hate any expenses that are not tied directly to the current operation of the company. The core rationale for this dislike is that you are losing cash today for a benefit that you will receive slowly over time (technically, over the course of the depreciation of the server). Any business owner or executive wants to focus the organization's cash on things that benefit them today. This concern is most acute with the small- and medium-sized business that may not have an easy time walking into the bank and asking for a loan.

The key problem with this delayed realization of value is that money costs money. A company will often fund their operational costs through revenues and pay for capital expenses through loans. If you can grow the company faster than the cost of money, you win. If you cannot grow that rapidly or—worse—you cannot get access to credit, the capital expenses become a significant drain on the organization.

Cost comparison

Managed services infrastructures and the cloud are so attractive to companies because they largely eliminate capital investment and other up-front costs. The cloud has the added advantage of tying your costs to exactly what you are using, meaning that you can often connect IT costs to revenue instead of treating them as overhead.

Table 1-3 compares the costs of setting up an infrastructure to support a single "moderately high availability" transactional web application with a load balancer, two application servers, and two database servers. I took typical costs at the time of writing, October 2008.

Table 1-3. Comparing the cost of different IT infrastructures

Table 1-3 makes the following assumptions:

The use of fairly standard 1u server systems, such as a Dell 2950 and the high-end Amazon instances.

The use of a hardware load balancer in the internal IT and managed services configuration and a software load balancer in the cloud.

No significant data storage or bandwidth needs (different bandwidth or storage needs can have a significant impact on this calculation).

The low end of the cost spectrum for each of the options (in particular, some managed services providers will charge up to three times the costs listed in the table for the same infrastructure).

Net costs denominated in today's dollars (in other words, don't worry about inflation).

A cost of capital of 10% (cost of capital is what you could have done with all of the up-front cash instead of sinking it into a server and setup fees--basically the money's interest rate plus opportunity costs).

The use of third-party cloud management tools such as enStratus or RightScale, incorporated into the cloud costs.

Staff costs representing a fraction of an individual (this isolated infrastructure does not demand a full-time employee under any model).

Perhaps the most controversial element of this analysis is what might appear to be an "apples versus oranges" comparison on the load balancer costs. The reality is that this architecture doesn't really require a hardware load balancer except for extremely high-volume websites. So you likely could get away with a software load balancer in all three options.

A software load balancer, however, is very problematic in both the internal IT and managed services infrastructures for a couple of reasons:

A normal server is much more likely to fail than a hardware load balancer. Because it is much harder to replace a server in the internal IT and managed services scenarios, the loss of that software load balancer is simply unacceptable in those two scenarios, whereas it would go unnoticed in the cloud scenario.

If you are investing in actual hardware, you may want a load balancer that will grow with your IT needs. A hardware load balancer is much more capable of doing that than a software load balancer. In the cloud, however, you can cheaply add dedicated software load balancers, so it becomes a nonissue.

In addition, some cloud providers (GoGrid, for example) include free hardware load balancing, which makes the entire software versus hardware discussion moot. Furthermore, Amazon is scheduled to offer its own load-balancing solution at some point in 2009. Nevertheless, if you don't buy into my rationale for comparing the hardware load balancers against the software load balancers, here is the comparison using all software load balancers: $134K for internal IT, $92K for managed services, and $106K for a cloud environment.

The bottom line

If we exclude sunk costs, the right managed services option and cloud computing are always financially more attractive than managing your own IT. Across all financial metrics—capital requirements, total cost of ownership, complexity of costs—internal IT is always the odd man out.

As your infrastructure becomes more complex, determining whether a managed services infrastructure, a mixed infrastructure, or a cloud infrastructure makes more economic sense becomes significantly more complex.

If you have an application that you know has to be available 24/7/365, and even 1 minute of downtime in a year is entirely unacceptable, you almost certainly want to opt for a managed services environment and not concern yourself too much with the cost differences (they may even favor the managed services provider in that scenario).

On the other hand, if you want to get high-availability on the cheap, and 99.995% is good enough, you can't beat the cloud.

Urquhart on Barriers to Exit

In November 2008, James Urquhart and I engaged in a Twitter discussion relating to the total cost of ownership of cloud computing (James is a market manager for the Data Center 3.0 strategy at Cisco Systems and member of the CNET blog network). What we realized is that I was looking at the problem from the perspective of starting with a clean slate; James was looking at the problem from the reality of massive existing investments in IT. What follows is a summary of our discussion that James has kindly put together for this book.

While it is easy to get enthusiastic about the economics of the cloud in "green-field" comparisons, most modern medium-to-large enterprises have made a significant investment in IT infrastructure that must be factored into the cost of moving to the cloud.

These organizations already own the racks, cooling, and power infrastructure to support new applications, and will not incur those capital costs anew. Therefore, the cost of installing and operating additional servers will be significantly less than in the examples.

In this case, these investments often tip the balance, and it becomes much cheaper to use existing infrastructure (though with some automation) to deliver relatively stable capacity loads. This existing investment in infrastructure therefore acts almost as a "barrier-to-exit" for such enterprises considering a move to the cloud.

Of course, there are certain classes of applications that even a large enterprise will find more cost effective to run in the cloud. These include:

Applications with widely varying loads, for which peak capacity is hard to predict, and for which purchasing for peak load would be inefficient most of the time anyway.

Applications with occasional or periodic load spikes, such as tax processing applications or retail systems hit hard prior to the holidays. The cloud can provide excess capacity in this case, through a technique called "cloudbursting."

New applications of the type described in this book that would require additional data center space or infrastructure investment, such as new cooling or power systems.

It seems to me highly ironic—and perhaps somewhat unique—that certain aspects of the cloud computing market will be blazed not by organizations with multiple data centers and thousands upon thousands of servers, but by the small business that used to own a few servers in a server hotel somewhere that finally shut them down and turned to Amazon. How cool is that?

If you like what you've read here and would like to read more, get the book: Cloud Application Architectures by George Reese.

Guess the Book, Episode Two - Back on the Streets of Sebastopol

2009-04-06T18:28:52Z

In this second episode of Guess the Book, we hit the streets of Sebastopol, CA (world headquarters of O'Reilly Media), armed with a copy of Programming ASP.NET 3.5. Our goal is to see if any of the locals can identify its contents simply by looking at the cover. The search leads O'Reilly's Sara Peyton and her trusty videographer to the offices of Sonoma West Times & News where one willing volunteer attempts to decipher the book's cover. Special thanks to George Snyder for being such a good sport and Round Table Pizza of Sebastopol for providing shelter from the rain and noise.

Now it's your turn to help out—which book should we use for our next episode and why?