I’ll try my hand at the dangerous game of internet financial prognostication, here. I have no clue what’s going to happen on the market, but I’d just like to point out something that I haven’t seen anybody really talk about yet. For all the talk about comparing this present market to the 90% bear market of the early 30s, there is one huge, obvious difference: information moves a lot faster today, and we have much more extensive “instrumentation” on the worlds economies. Not only does the flow of information mean that markets will adjust quicker, but the liquidity provided by ubiquitous electronic trading and derivatives means that market swings will occur on a much shorter time scale. Perhaps most importantly, we have much more universal access to economic data than ever before. This means that price information and the beginning of deflation have become apparent much quicker, and to a wider audience, than they did during the great depression.

The upshot of all of this, if I’m right (which is a big if), is that we may never see the kind of months long rallies that occured during the bear market that started in 1929. Maybe they’ll just last a few days, or maybe a couple of weeks. A corrolary of this is that volatility will be much higher than in the aftermath of the 1929 crash. This may be one explanation for why we’ve seen historic levels on the CBOE volatility index (VIX). Usually the VIX peaks around 40 or 50, but these days we’re seeing sustained levels around 70.

So, while I’m going out on a limb, if you’re sitting on cash and tempted to try to time the market and ride the rally we’re currently in, you might want to rethink it. In this day of electronic data and trading, I think the aftermath of the financial crisis will be an extended period of volatility that will eclipse anything the market has seen in its entire history. This crisis may have put the final nail in the coffin in the foolish academic theory of “efficient markets,” but that doesn’t mean the markets can’t be inscrutible and chaotic for an extended period.

The central cause of the current recession (or maybe depression) was the overextension of credit, especially in the housing sector. So, how do you get out of that kind of jam? Foreclosures and time. Those that can’t afford their homes will have to lose them, and the banks will get back as much as they can, allowing their balance sheets to be repaired up to a point. Those that can afford their houses will just have to stop taking on more debt, and begin the process of paying off the principle to the point where the loan is rightside up. This will eventually lead to healthy bank balance sheets, and thus the resumption of normal (but hopefully not excessive) lending that will get industry moving again.

If you really wanted to screw with the country, what would be the worst thing you could do? Halt foreclosures and/or slow down the repayment of debt. And yet that’s exactly what Arnold “I am here from the future to destroy your economy” Schwarzenegger has just proposed. He has asked that state lawmakers impose a 90 day moratorium on foreclosures. That sounds really good, but another way of saying it is that it’s a three month state imposed time window wherein banks will not be able to repair their balance sheets. Given that California is ground zero for the housing bubble, this is pretty significant, and in my amateur opinion is tantamount to extending the recession by at least three months. It’s also a slippery slope, and given that it’s highly unlikely a farm hand living in a $750k house (yes, true anecdote) will be able to suddenly improve his cash flow in 90 days in the middle of a recession, I can’t imagine the moratorium will do anything besides delay the day of reckoning.

It would also be the height of selfishness for Californians to use the power of their state government to clog up the nation’s financial system in order to avoid having to deal with the consequences of their feckless consumerism. Where California goes, so goes the country, and that includes really bad ideas. I expect other states, such as Nevada, to follow suit with similar populist legislation. Hopefully the Feds will clamp down before this gets out of hand, because otherwise I can see economically illiterate do-gooder state legislators launching the mother of all depressions by bringing down our entire financial system through a morass of legislation designed to prevent banks from clearing bad credit.

Why is foreclosure considered such an anathema, anyway, at least compared to the prospect of having our entire financial system collapse? People will be released of their debt burden, with the only consequence for their foolishness being that for the next seven years they’ll have to rent an apartment at a much cheaper monthly payment than they had before. I can’t imagine that’s such a bad thing, especially when the alternative is for the credit crunch to deepen, an outcome that would have potentially dire consequences for everybody. Given that the Fed has been only marginally successful in freeing credit markets, the last thing we need is for states to push the other way by obstructing the neccesary process of clearing out bad debt through default.

Mr. T understands that massive deleveraging will lead to an extended depression.

I envision Bernanke in a headlock in one arm, screaming “Ok, ok, we over inflated!” with Paulson in the other, choking “Let go, can’t breathe! Ack. Fine! We lied about TARP! Ok?”

However, straightforward logic will tell you that gold should not neccesarily go up in price during deflation. Yes, it’s money, but so are dollars! And since gold is always priced in units of somebody’s money, why should it always go up? Sure, it will increase in value, relative to other assets, during deflation, but without other factors to consider there is no reason to expect it will increase in price. After all, the Yen is money, so shouldn’t it go up during deflation? Obviously, every currency cannot go up relative to every other currency, and so, too, the price of gold will not automatically go up during deflation.

So the real question is this: is gold an attractive currency relative to the dollar? I’d argue it depends on the stuation. If you expect the deflation to be caused by lower demand for goods despite an increase in the money supply, you might consider gold a more stable value. But there are inherent disadvantages to gold, such that all things being equal, cash wins: You can’t pay debts with gold, and none of your future expenses are priced in units of ounces of the stuff. There’s no way to make gold disappear, but in our fractional reserve system, fiat currency can vanish during a period of debt destruction. If I were confident of such a monetary deflation persisting I’d personally want to keep my wealth in the ever scarcer fiat currency, not a form of money that’s dug out of the ground and hoarded by metric tons in central bank vaults around the world.

I certainly agree that it’s possible for gold to go up during deflation, and that if ever there were a situation where it should, it’s perhaps now. All I’m saying is that the idea that gold will always go up during deflation is as nonsensical and meaningless as saying the Euro will always go up during deflation. It depends on the situation. I think the best argument for holding gold during a period of deflation is simply the expectation that the government will soon do everything in its considerable power to reinflate the currency.

(Disclaimer: Lest anyone think I’m the typical antigold zealot sniping at the bugs, I’m actually long a significant portion of my meager grad student portfolio in GG and DGP.)

…This decision was made easier by the fact that the Big Three’s balance sheets have made even sympathetic Washington spenders worry about throwing money at a bankruptcy. Democrats decided it would be better to direct the funds in a way that allowed them to later deny fault.

The plan? Make it the Bush administration’s responsibility to give Detroit cash — namely by claiming after the event that the $700 billion rescue package for financial institutions was in fact a rescue package for auto makers. This was attempted with several hilarious “colloquys” — pre-scripted dialogues between members that were quietly inserted into the Congressional Record after the vote, all aimed at rewriting the “intent” of the law. Say, this one, from Oct. 1:

Michigan Sen. Carl Levin: “As Treasury implements this new program, it is clear to me from reading the definition of financial institution that auto financing companies would be among the many financial institutions that would be eligible sellers to the government. Do you agree?”

Connecticut Sen. Chris Dodd: “Yes, for purposes of this act, I agree that financial institution may encompass auto financing companies.”

Fun. Meanwhile, Democrats passed $25 billion in aid for Detroit, though under the careful guise of “green” funds to help it meet new fuel-efficiency standards.

Damn. I’m becoming fairly convinced that in the long run history will look upon the likes of Dodd and Pelosi and Barney Frank and Bush in pretty much all the same light: fools and crooks, all. These are the guys that sat by term after term letting the federal defecit grow and grow while they bought votes with the money. They let the government enable our massive dependency on debt through the Federal Reserve; we talk about the finance “industry” as a private enterprise that got out of hand, but fractional reserve banking doesn’t happen without the government as a neccesary and willing partner. The buck stops (literally) at our government.

These corrupt traitors did much to help get us into this mess, and now we are trusting them with trillions (when all is said and done) to get us out? Having Barney Frank and Chris Dodd in charge of congressional finance committes during a crisis is like opening the cockpit door of a plummeting 747 to find, well, Barney Frank and Chris Dodd. Those two lawyers have about as much training in finance as they do in flying heavy transport aircraft. And their incompetence would be tolerable were it not for their corruption and allegience to the finance industry. It’s like the fox guarding the hen house, if the fox were a dull weasel.

It is only when things are bleak that people are sufficiently motivated to change. We saw that last Tuesday, as red states turned blue with anger over the direction of the country. More importantly, I think we’re seeing the beginning of something even more productive than the electing of a figurehead for change. The magnitude of the crisis seems to be finally waking people up to the fact that our issues are not just due to one feckless moron who was elected president, but are systemic and of a magnitude that defy simplistic partisan finger pointing.

As depressing as this crisis is, the resulting national depression is–at last–our first rational act as a population in a long time. We should be depressed, but at least take comfort in that it is the first step of the recovery. I’m sorry to succumb to the overused addict metaphors in describing our country’s borrowing and economic discipline, but I do think that one absolutely valid similarity is that sometimes you just need to hit bottom in order to see the truth and take responsibility for yourself.

Well, America, as we sit weeping in $7 trillion of our own vomit, with the Chinese holding back our hair and telling us “Keep borrowing, you’ll feel better,” I think it’s safe to say: Congratulations, fellow citizens, we’ve hit rock bottom. Now, it’s tempting to try to escape back into the bottle (and congress is trying desperately to help us with that) but sometimes when the mind fails the body has a wisdom of its own. We just can’t keep any more credit down, no matter what we try, even the really good stuff from the Fed’s acronymic stash. So, we do what any good problem drinker does who is trying desperately to stretch the metaphor, and drunk dial an ex-girlfriend at 2 am. Except, as these things go, all our friends are junkies, too, and they’re all worse off than we are. All we get is a message from their Swiss roomate that they’re in rehab in Reykjavik and can’t hang around with us any more. We’re on our own.

So there’s nothing to do but buck up and change our ways, and the first place to start is with more cliches. “The night is always darkest before the dawn.” I’m not sure who first uttered that phrase, or what terrible situation prompted them to try to cheer somebody up with it, but it should come with a disclaimer: Dawn not guaranteed in the case of nuclear winter. Rock bottom is really only a call you can make in retrospect. It could, after all, just be the beginning of the dying process. We can use the troubles of the day as a motivation to finally clean up our collective act, or we can use it as a springboard for a spiral into degeneracy and looting. We can tighten our belts and sacrifice, or we can pass off our problems to future generations in the form of further debt, unsustainable bubble economics, or confiscatory taxes on decreasing pockets of wealth. We can chant “Yes we can” or we can quietly do.

What other revelations has the clarity of desperation brought? For one, that while we may have always been capitalists, and always will be, we’ve recently become merciless and stupid capitalists. Unfortunately, capitalism, even at its best, says nothing more than that markets find equilibrium subject to their constraints. We’ve forgotten that the constraints are in our control. If we consider it fair game to freely trade with countries that essentially allow slavery, then our free markets will eventually price labor at slave wages. Capitalism is a ruthless optimizer; it is brutally efficient at pricing but not so good at valuing. I think we may finally see that.

What gives me hope that we’ll choose the right path is that, as hokey as this might sound, Americans are often at their best when the going is toughest. We saw it briefly after 9/11, before our government went insane and declared martial law; we saw it, believe it or not, during Katrina, as Americans donated $2.6 billion of their own money while FEMA squandered their taxes, and over a million volunteers later moved in to supplement an incompetent federal response. There’s not a single country in the world whose citizens are more voluntarily generous than Americans. The common theme is that we are at our best when we depend on ourselves, not politicians. A culture of letting the government take care of us is slowly supplanting a culture of taking care of each other and ourselves; I hope part of hitting rock bottom is that we quit letting politicians be our enabler, and start taking care of ourselves. As Obama says, I am my brother’s keeper. 

I’d like to see GM bailed out by Americans buying their shitty cars, not by Federal loans. I’d like to see inner city poverty decreased not by welfare but because enterprising Americans start factories in Detroit to employ the unemployed, and other Americans buy the overpriced goods from those factories because they say “Made in the USA.” I’d like to see us start putting some effort into paying attention to the actions and ethics of those we do business and bank with, and not foolishly expect some government drone to be effective at doing so in our stead. I’d like to see Americans able to afford homes because we bear the temporary pain of letting their prices fall, not because congress bails out a few homeowners in 2009 at the expense of homeowners in the years 2010-2040. I hope we make healthcare truly more affordable by suing doctors less and using resources more wisely (coverage for Viagra?) not by falling for the false and unsustainable affordability of simply having somebody else pay for it.

You can criticize me for being naive in trusting citizen action over government fiat, but I think the greater naivete is thinking we can solve our problems without it.

Come on, Massachusetts. Grow a pair and just legalize pot. The war on drugs is doing far more damage to our society than drugs ever could. You’d have to be taking a lot of them not to see that.

But as you might expect, I’m also a lot less excited about this than other people who wanted to see McCain lose. The cult of personality that has sprung up around Obama among young people worries me, and I’m not sure it bodes well. For a group of people who hold religious zealots in such contempt, when Obama supporters gather it makes Baptist revivals look rational and sober. The students at Harvard were beside themselves last night, running around screaming and chanting his name. (One undergrad dislocated her knee during the ensuing revelry.) It was a bit scary, seeing Cambridge turn into Caracas during a Chavez rally.

That’s not the kind of country we are supposed to be. We don’t worship our leaders in America, we tolerate them. We don’t chant the name of our next president, we start making fun of him on SNL before he’s even in office. We don’t expect miracles of change to come through leaders, we expect progress to come from us and for our policians to get out of the way.

Obama will come to power, and inevitably not that much will really change. Of that I’m certain. It’s always that way. We have a campaign, and with all the focus-grouped advertising and inspiring speeches, we forget that a president can’t magically create jobs, or make oil cheaper, or reduce the cost of healthcare. The presidency is a small rudder on a huge boat. You can hold up a million signs that say Change, but when you put them in the water, they don’t do much to move a ship like America. The government can shift money around, and can turn insurance premiums into taxes, but it can’t change the great forces at work in the world. It can’t stop the fact that Chinese workers are 100 times cheaper than American labor, it can’t make the banks start lending again, and it can’t make MRI’s suddenly affordable. It can’t magically fix underperforming schools, or rejuvenate inner cities beset by crime.

I’m not suggesting nothing will get done. Policy certainly matters, and in the short term the government can be greatly effective in shifting burdens around. But I am saying that whatever will happen, it most certainly is not worth getting this excited about it. If America is to prevail over its problems, it will require ugly solutions, hard work, brutal sacrifice, a bit of luck, and a lot of time. It won’t come from jawboning about change and patting ourselves on the back for finally electing a guy who can put together a sentence.

I hope that I’m wrong, and that last night’s orgy of self-congratulation was just the relief of a long eight years finally being over. I hope that there hasn’t been a shift in the national cultural skepticism about politicians and the role of government. In my mind, nothing will bring about our failure quicker than the idea that our success lies in the hands of our politicians.

To stimulate lending, the bailout plan will attempt to recapitalize banks. The method of recapitalization is best described as robbing Taxpayer Pete to pay Wall Street Paul. In essence, money is taken from the poor (via taxes, printing, and weakening of the dollar) and given to the wealthy so the wealthy supposedly will have enough money to lend back (at interest) to those who have just been robbed.

All this talk about Strategy, Implementation, Recruitment, Procurement, operations, compliance, and other details masks the essence of the plan. And even though “A program as large and complex as this would normally take months — or even years — to establish“, the Secretary for Financial Stability is going to ramrod something through as quickly as possible.

Unfortunately, no matter what seat of the pants strategies they come up with, I can guarantee in advance that the unforeseen consequences of whatever decisions they make, simply will not be any good. Besides, it is axiomatic that plans to rob Peter to pay Paul, can never really work in the first place, regardless of how much time is spent crafting them.

This is not to say that it won’t work overall, as this plan at least finally addresses the fact that what banks need is capital, not access to credit. In fact, this is the first article of Mish’ that wasn’t outright beligerent in calling Paulson and the Fed complete fools, so I guess that is as close to an endorsement as you’ll get. But it’s just sad that the masters of the universe screw up royally, and poor people will disproportionately pay the price. It’s also sad that this further concentrates Chinese power over the American economy as they will be undoubtedly financing a lot of this.

感谢你，我荣幸的主人！

So, this bailout of Wall Street is going to be disproportionately paid for by the working poor. Good job congress.

Like most Americans, I’m sure you were glued to the news about the “financial crisis” and the government’s “fix” for the problem. You probably heard that the government was going to come in and save the mortgage industry by bailing out failing corporations. And, you probably heard both Barack Obama and John McCain say that this bailout was necessary, and good for the economy.

What you probably didn’t hear was that it was going to cost up to $1 trillion dollars, and you were responsible for this money.

“American taxpayers will come up with the money,” says the New York Times.

Such a number is hard to conceptualize if you’re not the government. So, in order to show you just how much $1 trillion dollars really is, we’ve put together these figures:

One trillion dollars ($1,000,000,000,000) is enough money:

• To buy everybody living in Los Angeles at least one Lamborghini Gallardo. 
• To buy 88,052, 394′ custom mega yachts; enough to stretch around ¼ of the world. 
• To buy everyone living in Belize and Malta a Manhattan apartment.
• To get half of the Democratic Party into a fundraiser for Barack Obama at the $28,500 admission price. 
• To give one out of every two men in the United States a Men’s Presidential Rolex watch.
• To buy every woman in the United States a Tiffany Diamond Starfish Pendant.
• To get two Mitsubishi 73″ HDTVs for every household in America.
• To buy four copies of The Office: Season Four on DVD, to every person on earth.
• To send everybody in America on an all-inclusive vacation to Tahiti (and some people can stay a few extra days).

AND…

$1 trillion is enough money for everyone in Buffalo, NY to buy their own 65-acre island in Panama.

This is how much the government is going to cost you (roughly $3,278 for every man, woman and child in the United States).

Barack Obama is for it. John McCain is for it.

But, Bob Barr and Libertarian Party are against it.

“No one voted to pour taxpayer funds into Wall Street,” says Bob Barr, the Libertarian Party’s presidential nominee, “and no one voted for the government to take over an insurance company. If the Federal Reserve can spend as much money as it desires to bail out any company that it desires, is there anything that it cannot do with taxpayer funds?

“She justifies what we do every day,” said Beth Tweddle, who works in sales and carried a sign she drew herself, saying “We [heart] Pit Bull Palin.” Tweddle was already a McCain supporter, she said, “but Sarah just energizes us and got us out here because she does what we do, she lives like we do.”

We don’t live in an age of looking up to authority anymore. We don’t cotton to the idea that there are people who are our betters. In this time of “American Idol,” bedroom bloggers and the belief that experience, knowledge and education don’t necessarily mean a whole lot, Palin is a symbol, a statement that anyone can make it if he or she really tries.

Karla Rupp, a real estate agent, went to see Palin on behalf of her three children, especially the one who has multiple disabilities and is in a nursing home. Val Lewis couldn’t stay away — “that’s how empowering it is to have Sarah up there. I have four children; she has five. And we get it done.”

…

“She’s just as flawed as we are,” Tweddle said. “It’s not the fact that she’s a woman but the way she does it all. And let me tell you: There’re more American parents with unwed pregnant teenaged children than American parents with Harvard grads. She’s real.”

For hours, I walked through the crowd talking to people, mostly women. Again and again, I heard variations on this idea: “She’s more like us than Obama, McCain or any of the others,” as Rupp put it. “She knows what we go through.”

…

Most people I spoke to readily conceded that Palin lacks experience with or knowledge of many important national and foreign issues. But, as Allison McGarvey, a teacher who lives in Stafford County, said, Palin is “a courageous woman, and what she doesn’t know, she can learn quickly. Let’s face it, no president knows all the issues. Anyway, I don’t see how a candidate can pick one stand and just stick to it. The world situation changes every day. It’s their moral and ethical background that’s important.”

…

Like many at the rally, Victoria Robinson-Worst sees Palin’s lack of experience as an asset. “I know people who have experience who are totally incompetent,” said Robinson-Worst, who lives in Loudoun County, designs wedding flowers and raises two children. “And I know people who have no experience who step in and get it right. I mean, women can do amazing things.”

Yup. They sure can.

Sure enough, I now get five bars of 3G signal in my office, where I used to get one or two! However, when I try to make a 3G call, the audio is warbled (is if packets are being constantly dropped) and it eventually either drops the call entirely, or switches back to standard GSM. All while showing five bars of 3G signal until it dies! Five bars of lies and deceit.

Update: Using my wife’s 2.0.2 iPhone 3G, I can confirm that Apple has, in fact, done little to nothing to improve the 3G reception beyond the psychological bromide of increasing the number of bars displayed. Holding the phones side by side, she gets one bar and I get five. However, download times for web pages are virtually identical (i.e. slow).

When I woke up this morning, my first thought was “You know, I feel way too good for a guy who’s only gotten four hours of sleep.” I then clicked the home button on my iPhone to check what time it was. I was not happy, not happy at all, to find that it was thirty minutes past the start of class and two hours after the alarm was supposed to go off.

As if to mock me, the alarm then finally starts to go off!

I checked online, and this is happening to other people, too. Apparently, if anything else triggers a notification (like a text message or calendar notification) the alarm will not fire until those notifications are seen. I’m not sure if this is what happened to me (I was pretty groggy) but it fits the observations of others that when an alarm is missed, it suddenly goes off when you wake up the iPhone.

So, we can add one more bug to the list of reception problems, crashing apps, crashing browser, etc. in the Apple public beta test that is the iPhone 3G.

OMGWTF?!?

I think the best way to appreciate how efficient F# is, especially for numerical analysis, is to show an implementation of a short program. The following is a bare-bones application which computes the basins of attraction for a Newton fixed point iteration which finds the roots of a polynomial in the complex plane. This computation is threaded across all available processors, and is a stand alone (albeit absolutely minimal) Windows application. The entire program is less than 50 lines, a testament to F# with respect to numerics and the .NET framework with respect to the GUI.

Briefly, Newton’s method is an iterative way to solve for the zeros of nonlinear functions. You start with an initial guess, and based on the local slope of the function, you make a refined guess for the root by following the slope all the way to zero. Unless the function really is a line this guess will be wrong, of course, but hopefully a little more accurate than where you started. If you start close enough to a root, repeated applications of the approximation will end up converging to the correct answer to arbitrary precision. If you start far away from one, however, you could end up at a root far away from your starting point, or you might never converge. It turns out that the map of where you end up as a function of where you start is a fractal, and a rather beautiful one for many equations. Below is a screen shot of the final program. Visible in the task manager is the nearly full usage of both cores of the processor during the render. (Click for a bigger version.)

Screenshot showing the nearly full use of both cores during the render.

The program below will iterate through up to 32 steps of Newton’s method on an arbitrary polynomial, starting at a grid of points in the complex plane. It shades each point based on the location of the final root found, with the hue determined by which root is converged to and with the brightness determined by how many steps were required. I’ll go through most of the functions, explaining each and pointing out how functional programming techniques are used. I do not claim this program is a highly efficient implementation! The main point was to illustrate several aspects of F#. It is however, pretty fast and makes use of multiple threads. In fact, the ease with which this program was parallelized is one of the main points I am trying to illustrate, and is made possible by the immutability of data in F# (and the Flying Frog Numerics library, a review of the which will be the subject of my final post on F#).

Below I’ve listed segments of code followed by brief explanations of each.

// program to plot newton basins for a polynomial in the complex plane
#light

// Load libraries
#r @"C:Program FilesFlyingFrogflyingfrog.numerics.dll"

// open some namespaces
open System
open Drawing
open Windows.Forms
open Math
open FlyingFrog

This is just boring namespace and library loading code. The #light syntax allows us to use abbreviated notation for many F# constructs, and seems to be the common usage. For example, it lets you forget about most end of line indicators. It kind of makes me wonder why they don’t just make #light the default and force you to use #heavy when you don’t want it, though I suppose the way language development works, that ship has sailed into the waters of backwards compatibility. The cost for the light syntax is that whitespace becomes significant, and you must indent your code correctly for the compiler to know what’s up. On the whole, this saves typing and makes for more readable code so I’ll stick with the #light syntax. Frankly, I’ve never actually seen anybody not use the light syntax.

// tail recursive auxilary function for polynomial evaluation
let rec polyval_aux (ps:float list) (x:Complex) (xpow:Complex) (accum:Complex) =
  match ps with
  | [] -> accum
  | c::psrest -> polyval_aux psrest x (x * xpow) (accum + (c $* xpow))

// evaluate polynomial with coefficients given by ps at x
// the highest order coefficient is last
let polyval ps x =
  polyval_aux ps x (Complex.Create (1.,0.)) (Complex.Create (0.,0.))

This is the recursive polynomial evaluation function that I spoke about in the previous post. The first function does all the work, and the second is just a wrapper that provides for a simpler call. (The use of recursion and pattern matching makes for a very simple imlementation, but for very large polynomials there are much more efficient ways to implement this that minimize the number of multiply operations.) The pattern matching facility of F# not only allows for a easy way to think about the control code (the two cases being those with empty lists and those with at least one entry) but it also simultaneously creates local variables (the first element “c” and the remaining elements in “psrest”) to be used within the second case. This is perhaps one of the most elegant aspects of F#: if you need to bother to specify a certain pattern of elements as a special case, then it’s likely you’ll want to use the specific instances of those elements. You thus specify your local variables at the same time that you specify your control code. Very elegent, useful and eminently readable. Pattern matching is by far my favorite aspect of F#.

let polyderiv (ps:float list) =
  match ps with
  | [] -> []
  | lowcoef::pshigh ->
    List.mapi (fun k pcoef -> float (k+1) * pcoef) pshigh

Since Newton’s method needs the derivative of a function, it will be handy to have a function which takes a polynomial and computes its derivative (which is just another polynomial). As with the previous functions, this takes a polynomial expressed as a list of coefficients and outputs another list of coefficients. (This function is also explained in better detail in the previous post.) The main part of this function illustrates the use of the higher order function List.mapi, which iterates over a list, applying a function to each element (presuming a nonempty list). The function here is defined anonymously, using the fun keyword. Another nice aspect of F#’s pattern matching is that the compiler ensures you cover all corner cases. In this case, the compiler will consider it a syntax error if you don’t include the [] case.

let maxcount = 32  // how many iterations before we give up on convergence

let rec fixed_point_count_aux f k (x : Complex) =
  let f_x = f x  // evaluate function
  if ((f_x = x) || (k > (maxcount - 1))) then
    (x,k)
  else
    fixed_point_count_aux f (k+1) f_x

let fixed_point_count f (x : Complex) =
  fixed_point_count_aux f 1 x

This set of functions simply iterates a function on a complex number, feeding the output back into the input, until either a fixed point is reached (to machine precisions) or maxcount iterations are completed. Additionally, a counter is kept of the total iterations. Naturally, the iteration is handled by a recursive function which starts by computing the given function on the current complex number. It then checks if the output of the function differs from the input or if the number of iterations (k) has exceeded maxcount. If so, it returns a “tuple” consisting of the final value and the iteration count. Otherwise, it calls itself with the updated complex number and the incremented iteration count. The second function is a wrapper for the first, which simply handles calling the auxiliary function with an initial iteration count of one.

Notice that we can define the fixed point iteration code completely generally and without knowledge of the fact that we’re going to be using it with Newton iterations. (In fact, if this weren’t such a simple implementation, it could be made into a library function, complete hiding all the details.) All we need to do now is supply the function “f” to be used, which we do below. Notice how much encapsulation is possible with functional programming without having to use a single object.

let newton_iter f df (x : Complex) =
  // compute derivative at x
  let (dfx : Complex) = df x
  // if the derivate is finite, compute step
  if (dfx.Magnitude = 0.) then x else x - (f x)/(df x)

let newton_fixed_point_fun p = newton_iter (polyval p) (polyval (polyderiv p))

The first function above performs a general Newton iteration given a function “f” whose root we seek, given a function “df” which computes its derivate, and a starting guess “x” given as a complex number. For convenience, we also define a function which takes a polynomial and uses a curried version of newton_iter to return a function which takes a complex number and performs a Newton iteration for that polynomial. If we repeatedly apply this function to an approximate guess for a root, it will converge to the exact root. The newton_fixed_point_fun routine takes care of generating the derivative function needed, by using our polyderiv function.

// compute display color from fixed point count result
let result_to_color (x : Complex, k : int) =
  let mag = float (maxcount - k) / (float maxcount)
  let angle = x.Phase
  let compg = int (255.*mag*(Math.Sin(angle)/2. + 0.5))
  let compb = int (255.*mag*(Math.Sin(angle + 6.28/3.)/2. + 0.5))
  let compr = int (255.*mag*(Math.Sin(angle + 2.*6.28/3.)/2. + 0.5))
  Color.FromArgb(compr, compg, compb)

Now we’ve got some utility functions to define. We need some way to display the results of each Newton solution at each point in the complex plane. The above function takes the output of fixed_point_count, which is a tuple consisting of the final point reached and the number of iterations required, and returns a Windows Color object. The details of this function aren’t very interesting, but the idea is that if a root is found from a given starting point, the pixel corresponding to that starting point will be colored based on which root it is, and with a brightness proportional to how quickly Newton’s method converged to it. If Newton’s method never converges, it will be black.

let linspace (low, high) n =
  let index_to_real k =
    low + (high - low)*(float k)/(float (n - 1))
  Array.map index_to_real [| 0 .. (n-1) |]

Another utility function. The way the computation will proceed is that each pixel in the output window will represent a value in the complex plane. We thus need a way to go from the pixel index to the number it refers to. This is done by linspace, which takes a low and high value, and the number of pixels n, and creates an array of n floats spanning from low to high. This kind of thing would usually have to be done with for loops in an imperative language like C. However, in F# it is done with the Array.map higher-order function, which applies a given function to an array. In my opinion, this is a much more conceptually satisfying way to do it, and one with far less potential for programmer errors.

This example also illustrates a very powerful aspect of functional programming: subfunctions. The index_to_real function which is passed to Array.map is defined inside linspace, and thus it can use all the local variables (and arguments) in linspace. This makes for much clearer usage in the map function.

// main computational function to evaluate single line of fractal in parallel
let compute_line f realparts imagpart =
  let cvals = Array.map (fun realpart -> Complex.Create(realpart,imagpart)) realparts
  Parallel.Array.map f cvals

The function compute_line is the heart of the program. It runs the Newton fixed point iterations for one single raster line of the fractal, distributing the computation over all available cores. It takes three arguments: the function “f,” a list of floats which represent the real part of the starting values at each pixel, and the single float which is the imaginary part of the starting values. (Since we’re doing a single horizontal line, the imaginary part stays the same.) The function “f” will be a curried version of the fixed_point_count function that we will define later, and it will return the aforementioned tuple which summarizes the results of the Newton iterations.

The first line of the function creates a list of complex values, representing the starting points of the Newton iterations. Then we simply call “f” on each complex value. Pretty simple, huh? What’s really cool is that by using Parallel.Array.map, a function in the Flying Frog Numerics Library, the computation is automatically threaded, distributing the computation over all available cores or processors. How can this be so simple? Why don’t we need to worry about thread safety? The reason this is possible is that every single variable used in the computation of the fractal is immutable. There’s no way for one thread to have side effects on the others. Now, in the case of this example program, the thread safety would be pretty trivial even in C#. However, the point here is that no matter how complex the algorithm, if you use immutable data structures in F# you will always be assured of thread safety. By simply typing the nine characters ‘Parallel.’ (and paying Flying Frog for the library, of course) you can turn a serial algorithm into a parallel one. This is a simple example, but there is a lot more coming along these lines. Microsoft, for all they get wrong, is really doing a good job staying on the ball with the seachange embodied by the move to many-core processors.

// Create a bitmap the size of the rectangle and fill it using
// the fixed_point_count function "f" which generates a tuple from a complex
let format = Imaging.PixelFormat.Format24bppRgb
let make_bitmap_of f r_range i_range (r : Rectangle) =
  let bitmap = new Bitmap(r.Width, r.Height, format)
  let realparts = linspace r_range r.Width
  let imagparts = linspace i_range r.Height
  // aux function to compute line and write to bitmap
  let do_line ki imagpart =
    // run computation over raster line
    let results = compute_line f realparts imagpart
    // turn results into colors
    let colors = Array.map result_to_color results
    // set bitmap pixels to colors
    Array.iteri (fun kr color -> bitmap.SetPixel(kr,ki,color)) colors
  // iterate over the imaginary parts, computing each corresponding line
  Array.iteri do_line imagparts
  bitmap

The function make_bitmap_of comprises the main GUI routine, and creates a bitmap given a function “f”, the real and imaginary ranges over which to run the computation (expressed as tuples) and a rectangle representing the desired size of the bitmap. Nothing really interesting happens here, though. After some helper variable initialization and the creation of the blank Bitmap object, an auxiliary function do_line is defined that is designed to be used with Array.iteri. The function do_line will be run over a list of the imaginary parts representing each line of the fractal bitmap. It thus takes an index value and a float representing the imaginary part. It then runs compute_line and puts the results into a variable. Then, result_to_color is applied to each of the result tuples to generate a list of Color objects, which are then used to set each pixel color in the bitmap object.

// Resize the form by replacing and rerendering the bitmap
let resize f r_range i_range (b : Bitmap ref) (w : #Form) _ =
  b := make_bitmap_of f r_range i_range w.ClientRectangle;
  w.Invalidate()

// Paint the form by drawing the bitmap
let paint (b : Bitmap ref) (v : #Form) (e : PaintEventArgs) =
  let r = e.ClipRectangle in
  e.Graphics.DrawImage(!b, r, r, GraphicsUnit.Pixel)

// Create a form that visualizes a bitmap that is rendered by the
// function "f"
let plot_newton f r_range i_range n =
  let form = new Form(Text="Newton Basins", Visible=true) in
  form.Size <- new Size(n, n);
  let r = form.ClientRectangle
  let bitmap = ref (new Bitmap(r.Width, r.Height, format))
  form.Paint.Add(paint bitmap form);
  form.ResizeEnd.Add(resize f r_range i_range bitmap form);
  form.KeyDown.Add(fun e -> if e.KeyCode = Keys.Escape then form.Close());
  form

This is just a bunch of boring .NET GUI code, which I’ll just leave as is for the sake of brevity. All I want to point out here is how relatively little of it is needed. This is not just a credit to .NET, but also to F# for its facile creation of callbacks. This code handles creating the window and setting callback functions to handing updating the fractal upon the window being resized, and to allow the user to quit by typing the escape key. Not much of a GUI, for sure, but it gets the job done. I think the .NET “form” object is the only mutable data structure in the entire program. (This snippet was essentially stolen wholesale from Jon Harrop.)

// *** run example ***
// create interesting polynomial
let ps = [-1.0;0.01;-0.01;0.0;0.0;1.0]
// create core function to run Newton's method
let newton_test = fixed_point_count (newton_fixed_point_fun ps)
// range to be used for both real and imagary parts to define plane
let range = (-2.25,2.25)
// create Windows form
let form = plot_newton newton_test range range 256

#if COMPILED
do Application.Run(form)
#endif

Finally, the core application code. This is the stuff that would be inside the main() function in a C program, except that F# lets you just start writing code in a random file and figures (quite rightly) that if you bothered to write some code, you probably want it to run after you compile it. This generates the following fractal (click to see at high resolution):

Newton basin fractal generated for a polynomial with five roots arrayed in a circle around the origin.

I hope this walk-through was at least somewhat useful in explaining some of the idioms of functional programming, and in particular those of F#. If anything requires better explanation, please send an e-mail or write a comment below. (Note: if you want to run this without the use of the FlyingFrog Numerics library, just delete the open command pertaining to it, and delete the “Parallel.” namespace prefix in the compute_line function.)

F#, however, is the first new language in a long time that I’ve felt is worth taking the time to learn. Developed by Microsoft research in England, F# is a functional language with roots in ML. Unlike many academic functional languages, however, F# is a pragmatic mix of imperative object oriented programming and functional constructs. Instead of forcing a paradigm on you, it simply makes several available. It can also leverage the full .NET framework, and is thus just as capable of implementing a GUI as it is an abstract recursive algorithm. It produces code that’s about as fast as C#, and it should only get faster with improvements in its CLR compiler and in .NET’s JIT compiler. A novel aspect of F# (in the context of .NET languages) is that it can be used as an interactive scripting language from within Visual Studio, allowing for interactive data visualization.

After working with it for a month or so, I find it to be a highly productive and expressive language, especially for numerical algorithms. It manages to be concise and high-level while maintaining coherence and readability. Well-written functions in F# are easier to follow and debug, relative to C++ or C#, and there are fewer opportunities for bugs to begin with. More than just cute syntax, it’s a language that actually encourages you to think differently. Programs implemented well in F# are a closer representation of the underlying concept and thinking, and are less obscured with with mundane details. Perhaps the best argument of all is empirical: the test program I wrote to compute fractal Newton basins worked the first time I compiled it, something that virtually never happens to me with C or C++.

In this post, I’ll provide a rough overview of functional programming and F#. There are much better introductions out there, but I nonetheless wanted to write this as a way to help myself learn about functional programming. I also figure F# is new enough that many of the people who read this blog might have not yet heard of it, and any small additional exposure this great language gets is worth it. This is not meant to provide a tutorial that will bring you up to speed on F#, but is intended to give you enough of an idea about it that you can decide whether or not you want to learn more.

In a follow up article, I’ll step through a short program in F# which uses some of the example functions I define below to implement a parallel computation of some pretty fractals generated by running Newton’s method for polynomial roots in the complex plane. In a third post, I’ll talk about some of the libraries available to speed up F# development. Because of the functional nature of F#, libraries written for F# can do some pretty impressive things. For example, the fractal generator detailed in the next post uses a third-party library function that handles all the details of multithreaded task parallelization through the surprisingly simple use of higher-order functions. In fact, parallelizing the program required typing exactly nine characters.

If you’d like to experiment with F#, you will first need some version of Visual Studio. I’m not sure if one of the free Express versions will work. (Someone please let me know if they have the answer to this.) The F# compiler is freely available from Microsoft Research, and integrates nicely with VS 2005. It is also reported to run under Mono on Mac OS X and Linux, though I haven’t verified this myself.

Functions as variables

A functional language is characterized by several key ideas. The core notion is that functions are fundamental objects, on par with variables. They can thus be passed to, and returned by, other functions. This enables some very efficient and conceptually satisfying ways of expressing algorithms, as we’ll see shortly. A functional language does nothing C or C# can’t do (sometimes in the hype of a new language, it’s easy to forget the old Church-Turing thesis) it’s just that it can often do so much more simply and in a way that is often closer to the way we think.

One oddity in F#, and your first hint that you’re dealing with a different kind of language, is the way function calls are defined. In a sense, an F# function really only ever takes one argument. To get a function to take multiple arguments, you are essentially writing a function which takes a single argument and returns another function which takes the second argument (and so on). For example, a built-in function which adds two integers ( + ), is listed in F# as having type (int -> int -> int). The interpretation of this odd notation is that it’s a function which takes an integer and returns a function of type (int -> int), which is itself a function which takes an integer and returns an integer. Perhaps a simpler way to think about this type is with the bracketed notation(int -> (int -> int)). When you write

z = ( + ) x y

what is happening, at least conceptually, is that the compiler greedily evaluates ( + ) x, which returns a function, and then it applies that function to y. (Don’t worry, F# also has infix syntax for binary operators so you can also just write the familiar z = x + y.)

One consequence of this is that you can do things like define a new function which increments an integer by 2 in terms of the ( + ) function with the following syntax

let incrtwo = ( + ) 2;;
let shouldbethree = incrtwo 1;;

This technique of making functions with multiple inputs from a chain of higher-order functions which each take single arguments is known as function currying. Granted, it’s pretty pointless in this static example, but consider the following interactive F# fragment:

> let makeincrementor s = ( + ) s;;
val makeincrementor : int -> (int -> int)
> let incrbyfive = makeincrementor 5;;
val incrbyfive : int -> int
> incrbyfive 6;;
val it : int = 11

The first line defines a function which takes an integer and returns a new function which increments its input by a run-time defined value s. The second applies this to define a new function which increments an integer by 5. So, function currying can be used to generate new functions dynamically. One exciting aspect of F# being a .NET language is that, in theory, this allows for a nice clean syntax for automatically generating specialized compiled functions on the fly during run time. (If anybody knows whether or not this already actually happens, please leave a comment with the answer.)

All functions in F# are inherently curried functions. Fortunately, there is syntax to write such functions without having to keep track of this fact. You can simply write

let myplus x y = x + y

and you automatically have a curried function of type (int -> int -> int). The fact that it will assume you want integer types all around is part of the automatic type inference of F#, which you will find to be both a blessing and a curse in practice.

The use of functions as arguments to other functions allows for tremendous abstraction, and certain “design patterns” which are no more than conceptual models in imperitive languages can actually be implemented as actual library routines in functional languages. One very powerful example of so called higher-order functions is the F# List.mapi function. It takes two arguments: a function and a list. The function is called repeatedly with two arguments: an integer representing the index into the list and the corresponding element of the list, creating a new list by looping over all elements of the initial list. A loop with an incremented local variable is one of the most common idioms in all of programming. How many times have you typed in the same for loop construct in C or C#? And how many times have you accidentally typed in the termination condition incorrectly, or missed a corner case?

As an example of how this is used in F#, below is a function which computes the derivative of a polynomial, passed in as a list of coefficients, ps, in ascending order:

let polyderiv (ps:float list) =
match ps with
| [] -> []
| lowcoef::pshigh ->
List.mapi (fun k pcoef -> float (k+1) * pcoef) pshigh

Pattern Matching

This example also uses another common idiom in functional programming: pattern matching. Program branch and control can often be handled very elegantly by matching an expression against a list of potential patterns. In the example function above, there are two possible situations. If the list is empty, we return an empty list. Otherwise, the list matches against lowcoef::pshigh, which is syntax for “take the first element as the local variable lowcoef and the remaining elements (if any) as pshigh.” This illustrates another powerful aspect of pattern matching: one simulaneously branches control while assigning relevent local variables to be used in the corresponding branch. Not only is this pattern matching approach more powerful and conceptually intuitive than if/then or switch statements, it also has the added side benefit of doing the work of local variable creation. After all, if you are bothering to treat a given pattern as a special case, you probably need to use at least one of the elements that match that pattern.

The last line uses the List.mapi higher order function applies an anonymous function to each coefficient (except the first), which simply multiplies the coefficient by its cardinal order in the list. Anonymous functions are to the function type what numerals are to the integer type: Just as you can write 3 + 7 without having to make two variables to hold each value, likewise you can just write a function without having to assign it a label. This makes for very efficient use of higher-order functions like List.iter or List.mapi, where a simple function to be given to them as argument can be defined on the spot. A curried function is also commonly used as an anonymous function.

Immutable Data

Another key idea, and one that is very odd at first, is the notion of immutable data structures. In most cases, when you declare a “variable” in F# you are really just creating a label for a value, and nothing more. You can reassign a new value to that label, but it’s just pure syntax. This seems like a huge impediment, but in practice it’s surprisingly not so, especially in the typical idioms of functional programming. If this doesn’t make sense yet, don’t worry, it will when you see more example code. For now, just consider immutable data structures to be like variables which can never be written to by pointer. Immutable data allows the compiler to do some unique optimizations and makes for efficient use of memory; if you create a new list that is the concatenation of two other lists, for example, no new memory needs to be created as the new list is simply composed of references to the immutable lists composing it. It also can make concurrent programming much more straightforward: provided you can express your algorithm in terms of immutable data structures, there’s no need to worry about locks as each thread can’t possibly have side effects that would affect the others. Having code that is automatically thread-safe makes programming concurrent algorithms much more straightforward. As many-core processors loom on the horizon, F# seems a rather timely language.

Recursions

The last functional programming aspect I’ll talk about is recursion. Mathematical induction is a natural way of thinking about many algorithms, and thus recursion is therefore a natural way to implement them that is concise and easily verified as correct. The typical example is computing the Fibonacci sequence: if you know the last two Fibonacci numbers, you just add them to get the next one. A recursive algorithm to compute the nth number would call itself with a request for the (n-1)st and (n-2)nd number and then add them together, with control code in the function for handling the base cases of n = 0 and n = 1. As a concrete example, and in keeping with the polynomial theme, here is a rather ugly function which uses recursion to evaluate a polynomial at a given value:

let rec polyval_aux (ps : float list) x xpow accum =
match ps with
| [] -> accum
| coef::psrest -> polyval_aux psrest x (x * xpow) (accum + (coef * xpow))

The rec keyword signifies to the compiler that this function calls itself, so it had better not try to evaluate the right hand side to greedily. This function certainly isn’t the most computationally efficient way to do this (though it is rather memory efficient) but it’s a simple example of a recursive function, one that implements two more common idioms: tail recursion and continuations. Tail recursion is the idea that if the complete final value of the function is simply the function called again, the stack only has to keep track of one function call at a time. It’s probably not a big deal here, but in cases where the function is recursively called a great number of times, it can matter.

You can also see one of the consequences of immutability: we have to keep passing the state of the computation. The arguments of the function are: a list ps of the polynomial coefficients, the value “x” at which to evaluate them, followed by two utility arguments which keep track of the state of the computation in the form of the current, a simple form of continuation. The values of “x” raised to increasing powers are passed on in “xpow,” and the running sum of the polynomial is kept in accum. Pattern matching is used as before, except now we use the head of the list, computing its contribution to the polynomial before passing along the remaining terms to the function again. When nothing is left, the accumulated sum is simply returned. The extra arguments are ugly, of course, so we use this as an auxilary function, to be called from the following wrapper function:

let polyval ps x =
polyval_aux ps x (Complex.Create (1.,0.)) (Complex.Create (0.,0.))

I think the above implementation of polynomial handling routines has two advantages over a similar set of functions in C. First, it’s really hard to screw up with stupid mistakes, like misallocation of arrays or bad referencing. Second, the intent of the code is more apparent, as the details of loops and counters have been hidden in the library functions. (Though I understand the intent may not be entirely transparent if you’re not used to the F# yet.)

If you’d like to learn more about F#, a couple good places to start are the Flying Frog consultancy and Microsoft Research. In the next post, I’ll present a stand alone Windows program that computes Newton basin fractals in 50 lines. Following that, I’ll finish by writing about some of the nascent resources that are starting to crop up for F#, in particular the excellent graphics and numerical libraries under development at Flying Frog.

walking from blu on Vimeo.

1. Purchase an ’R’ rated movie from the iTunes music store.
2. Pause movie during particularly noisy sex scene, preferably one involving farm animals.
3. Turn off phone.
4. Go to important meeting with your entire staff, including boss.
5. Turn off ringer using mute switch and put phone to sleep.
6. Think you’re safe.
7. Innocently pull out headphones from jack, momentarily shorting out badly designed mechanism in jack used to sense the remote play switch.
8. Remain blissfully unaware that you’ve just set a chain reaction in motion which will destroy your career.
9. Sit in stunned horror a few seconds later as sounds of impure love eminate loudly from your shirtpocket in the middle of your boss’ presentation, despite the silent switch being on.
10. Run back to office in tears to polish up resume.
11. (Optional.) Write Steve Jobs an angry letter about how good design includes more than just polished metal.

(No, this didn’t happen to me. At least not exactly. It wasn’t a meeting, it was a seminar. And my boss wasn’t there, thankfully. And instead of farm animals, it was far worse: it was emo music.)

iPhone maps

Last Friday (through means which I’m too embarrassed to publicly discuss) I got an iPhone 3G. Here are my initial impressions after using it for a couple of days.

Screen. Visually, it’s near perfect. Bright and with sufficient field of view. The only problem I have with it is that sometimes my finger just won’t slide over its surface. I don’t know if I’m just some sort of physical mutant, or if I’ve got weird finger chemistry, but sometimes it’s just unusable as a touch screen as my finger sticks to it, especially if the screen has just been cleaned. It seems that the screen was designed to work best when covered by a thin layer of grime from you fingers. I just don’t get why everybody loves this so much, but I guess I’m just weird. This problem was solved by getting a matte protective anti-glare film for the screen. The display isn’t as sharp, but its finally a pleasure to use the touch screen. I recommend you play with the iPhone in the store for a while and actually use it (try scrolling and dragging) to make sure you like dragging your finger across polished glass a million times a day.

3G network. This is a bit of a disappointment. As many many other people have found, the 3G reception is poor, and a huge battery killer. With 3G turned on I can’t get through a full day without having to recharge. Where I live in Cambridge the signal is so bad that calls will degrade and occasionally drop. It fluctuates between zero and three bars. Cambridge ain’t exactly the middle of nowhere, however, and I expect AT&T’s network is better than the iPhone’s performance would indicate. From what I’ve heard, this is a problem with the iPhone; an AT&T 3G phone from another manufacterer will often have five bars sitting right next to an iPhone with one.

I will say this, though: when it works, it’s quite fast, and I usually see speeds of around 1 Mb/s. Fortunately, you can disable 3G from the phone settings, but unfortunately there’s no way I can disable the $10 a month extra I’m paying for it. From all my previous experiences with Apple, I knew I was asking for trouble by buying the first batch of anything. Apple brutally punishes early adopters like no other company.

If there’s an excuse for this poor performance, it’s the near engineering miracle Apple had to pull off to get everything to work. In one tiny package, the iPhone contains GPS, multi-band 2.5G, multi-band 3G, WiFi, and Bluetooth radios. That’s a lot of RF going on in one place, and they all have to share antennas. I’m kind of amazed it works at all, frankly.

Data integration. For now, the iPhone only integrates natively with iCal (on a Mac), Outlook and Exchange. Fortunately, if you use Google calendar, there is a wonderful solution available from www.nuevasync.com. The folks at NuevaSync have essentially built an Exchange server that can pull your contacts and calendar from various online services (Google and Plaxo, for now) and make it available from the industry standard Exchange protocol. A brilliant idea, and a very timely one given the release of the iPhone 2.0 software which allows for Exchange integration and push. When I (or my wife) edit a calendar item online, it instantly appears on the iPhone.

App Store. As everybody predicted, there is a plethora of putative social networking revolutions, with trendy names like beepo and blue lemmingster, etc. But there are some surprisingly good apps available, and it seems the best ones are free. Some highlights: Bloomberg has an app gives you access to beautiful stock charts and a live news feed. AOL Radio provides dozens of live streaming radio channels across several genres, and it works over the cell network.

As Apple opens up the API more and more, I think the biggest impact of the iPhone will be as a new development platform for connected mobile applications. The most powerful applications of the phone are those which use the wireless broadband to connect to remote information and computing resources. It’s very satisfying, for example, to be out walking around outside and yet have access to the terabytes of satellite photos in Google Maps. On a more frivolous level, there is Shazamm, a program that will tell you the name and artist of virtually any song based on a 15 second sample played into the phone. One of the most interesting examples of the mobile-to-cloud computing paradigm is Jott, an app which will transcribe dictated notes. It records and compresses your voice at the iPhone, and then sends it to India where it is transcribed by a person and then sent back to your phone as text.

I’m looking forward to seeing what other kinds of applications are enabled by having a computer in your pocket with an always-on broadband internet connection. The iPhone isn’t so much a phone in this context as it is a rich mobile interface to remote supercomputers (and underpaid Indians).

What’s missing. Cut and paste, for one. Video conferencing. Live mobile TV, such as AT&T makes available on some of their other cell phones. Flash support in the browser. The ability to read PDFs anywhere but within an e-mail attachment or from a webpage. Push Gmail.

What’s just plain bad. The third from worst design flaw I’ve seen is that turning off the sound doesn’t actually turn off all sounds. Music can still play through the speaker in certain cases, and not all apps respect the mute setting. The second worst is that when you unplug the headphones, the phone sometimes turns on whatever music you were last playing in the iPod software (it fools the phone into thinking you momentarily hit the play button on the headphone remote). The absolute number one design flaw is that the second and third worst flaws can combine, so that you can end up blaring music in the middle of a lecture just by pulling out your headphones. I presume Apple will fix this in an update.

The 2.0 software feels like an early beta. The interface is often unresponsive, taking a few seconds to do something as simple as open a field for editing (the contacts program, of all things, is the worst offender). The browser crashes fairly regularly, and I’ve even hung the phone one or twice trying to watch a video podcast. It’s the first phone I’ve ever owned that crashes more than my laptop…

The battery life is rather poor. I haven’t gotten the phone to make it through the day yet, though a lot of that maybe that I leave the WiFi on. Regardless, I have no idea how Apple can claim five hours of 3G usage. Maybe that’s if you’re standing on the top of a cell tower.

Summary. Were it not for some of the aforementioned issues, it would be a truly remarkable piece of technology, especially in terms of the wonderful interface. It’s more enjoyable to use than a computer for most small tasks like checking e-mail. While I really love the iPhone in many ways, I have to admit that it’s clearly not worth the money when you factor in the plan. Of course, I’m kind of loath to return it at this point. It’s a kind of irrational psychology that is probably responsible for most of Apple’s revenue, I suspect: In the end, it’s just cool, and it would be hard to go back to klunky, even if klunky gets the jobs done for half the price. Look-and-feel counts for more than anybody (certainly I) would like to admit. Steve Jobs is a genius for being so cynical as to truly plumb the limits of this. So, I’m gonna stick with my heavy, big, overpriced, crashy, no-battery-life, embarass-me-in-meetings iPhone. Because it’s just so goddam fun to scroll around with a flick of your finger!