Good Math, Bad Math

The Balance of Screening Tests

Thu, 19 Nov 2009 15:51:28 -0500

As you've no doubt heard by now, there's been a new recommendation issues which proposes changing the breast-cancer screening protocol for women under 50, by eliminating mammograms for women who don't have significant risk factos. While Orac has done a terrific job of covering this here and here, I wanted to throw in a couple of notes and a personal perspective.

Read the rest of this post... | Read the comments on this post...

Shameful Innumeracy in the New York Times

Tue, 17 Nov 2009 13:08:14 -0500

I've been writing this blog for a long time - nearly four years. You'd think that after all of the bad math I've written about, I must have reached the point where I wouldn't be surprised at the sheer innumeracy of most people - even most supposedly educated people. But alas for me, I'm a hopeless idealist. I just never quite manage to absorb how clueless the average person is.

Today in the New York Times, there's an editorial which talks about the difficulties faced by the children of immigrants. In the course of their argument, they describe what they claim is the difference between the academic performance of native-born versus immigrant children:

Whereas native-born children's language skills follow a bell curve, immigrants' children were crowded in the lower ranks: More than three-quarters of the sample scored below the 85th percentile in English proficiency.

Scoring in the 85th percentile on a test means that you did better on that test than 85 percent of the people who took it. So for the population as a whole, 85% of the people who took it scored below the 85th percentile - by definition. So, if the immigrant population were perfectly matched with the population as a whole, then you'd expect more than 3/4s the score below the 85th percentile.

As they reported it, the most reasonable conclusion would be that on the whole, immigrant children do better than native-born children! The population of test takers consists of native-born children and immigrant children. (There's no third option - if you're going to school here, either you were born here, or you weren't.) If 3/4s of immigrant children are scoring 85th percentile or below, then that means that more than 85% of the non-immigrant children are scoring below 85th percentile.

I have no idea where they're getting their data. Nor do I have any idea of what they thought they were saying. But what they actually said is a mind-boggling stupid thing, and I can't imagine how anyone who had the most cursory understanding of what it actually meant would miss the fact that the statistic doesn't in any way, shape, or form support the statement it's attached to.

The people who write the editorials for the New York Times don't even know what percentiles mean. It's appalling. It's worse that appalling - it's an absolute disgrace.

Read the comments on this post...

Types in Haskell: Types are Propositions, Programs are Proofs

Tue, 17 Nov 2009 09:27:00 -0500

(This is a revised repost of an earlier part of my Haskell tutorial.)

Haskell is a strongly typed language. In fact, the type system in Haskell is both stricter and more expressive than any type system I've seen for any non-functional language. The moment we get beyond writing trivial integer-based functions, the type system inevitably becomes visible, so we need to take the time now to talk about it a little bit, in order to understand how it works.

Read the rest of this post... | Read the comments on this post...

Dembski Stoops Even Lower: Legal Threats to Silence a Critic

Mon, 16 Nov 2009 11:22:53 -0500

For those who have slightly better memory of recent events than an average gerbil, you'll surely remember that not too long ago, the Intelligent Design folks, with the help of Ben Stein, put together a whole movie about how evilutionists are all a bunch of evil fascists, out to silence the poor, hard-working IDers.

You'll also remember that Bill Dembski has been talking up the fact that he's got two peer reviewed papers allegedly about intelligent design. So, you'd think that after complaining about being locked out of the debate, now that he has some actual papers to talk about, he'd be eager to, well, talk about them!

Yeah, right. As it turns out, debate is the last thing that Bill wants. When someone took a good look at one of his papers, and posted a critique, Bill's response was the threaten to sue them for copyright violation. Knowing how utterly trustworthy the Disco gang is, I've got a screen-capture of the post with the threat below the fold, in case they try to change history by deleting it.

Read the rest of this post... | Read the comments on this post...

Writing Basic Functions in Haskell (edited repost)

Fri, 13 Nov 2009 15:47:35 -0500

(This is a heavily edited repost of the first article in my original Haskell tutorial.)

(I've attempted o write this as a literate haskell program. What that means is that if you just cut-and-paste the text of this post from your browser into a file whose name ends with ".lhs", you should be able to run it through a Haskell compiler: only lines that start with ">" are treated as code. The nice thing about this is that this blog post is itself a compilable, loadable Haskell source file - so I've compiled and tested all of the code in here in exactly this context.)

Read the rest of this post... | Read the comments on this post...

The Go I Forgot: Concurrency and Go-Routines

Fri, 13 Nov 2009 09:19:35 -0500

A couple of people pointed out that in my wednesday post about Go, I completely left out the concurrency stuff! That's what I get for rushing the post - I managed to leave out one of the most interesting subjects! Go provides very strong support for communicating processes.

I haven't done a lot of hacking with the concurrency stuff yet - so my impressions of it are still very preliminary. But my early impressions are very good.

Read the rest of this post... | Read the comments on this post...

Google's New Language: Go

Wed, 11 Nov 2009 16:08:33 -0500

I've been being peppered with questions about Go, the new programming language just released as open-source by Google. Yes, I know about it. And yes, I've used it. And yes, I've got some strong opinions about it.

Go is an interesting language. I think that there are many fantastic things about it. I also think that there are some really dreadful things about it.

A warning before I go on: this post is definitely a bit of a rush job. I wanted to get something out before my mailbox explodes :-). I'll probably try to do a couple of more polished posts about Go later. But this should give you a first taste.

Read the rest of this post... | Read the comments on this post...

Philosophizing about Programming; or "Why I'm learning to love functional programming"

Tue, 10 Nov 2009 13:46:03 -0500

Way back, about three years ago, I started writing a Haskell tutorial as a series of posts on this blog. After getting to monads, I moved on to other things. But based on some recent philosophizing, I think I'm going to come back to it. I'll start by explaining why, and then over the next few days, I'll re-run revised versions of old tutorial posts, and then start new material dealing with the more advanced topics that I didn't get to before.

To start with, why am I coming back to Haskell? What changed since the last time I wrote about it?

Read the rest of this post... | Read the comments on this post...

Berlinski - still pompous, still wrong.

Mon, 09 Nov 2009 22:05:48 -0500

An anonymous tipster sent me a note to let me know that on one of the Disco Institute's sites, my old pal David Berlinski has been arguing that all sorts of famous mathematicians were really anti-evolution.

I've written about Berlinski before. In my opinion, he's one of the most pointlessly arrogant pompous jackasses I've ever been unfortunate enough to deal with. He practically redefines the phrase "full of himself".

This latest spewing of him is quite typical. It is mostly content free - it consists of a whole lot of name-dropping, giving Berlinski a chance to talk about all of the wonderfully brilliant people he's close personal friends with. And, quite naturally, his close personal friends have told him all sorts of things about what other famous mathematicians really thought about evolution.

Read the rest of this post... | Read the comments on this post...

Friday Random Ten, 11/06

Fri, 06 Nov 2009 14:58:50 -0500

Porcupine Tree, "Kneel and Disconnect": New Porcupine Tree! It's always great to get new stuff from these guys. It's good, but it's not up to the quality of their last two albums. (But given that their last two were utterly amazing, that's not much of a criticism.)
Mind Games, "Royalty in Jeopardy": Some prog that I recently found via eMusic. They've got a sound that I describe as being sort of like a mix between Yes and Marillion. They're very good - I wouldn't put them in the top ranks of neo-prog, but they're not at the bottom either.
Riverside, "Cybernetic Pillow": Now, these guys, I would definitely put in the top ranks of neo-prog. Riverside is a Polish prog-rock band, formed by members of a couple of other heavy metal bands. They're absolutely brilliant. This track is off their album "Rapid Eye Movement", which I'd recommend as a first Riverside album.
Marillion, "Hard as Love (acoustic)": This is the version of "Hard as Love"" from their recent acoustic album. HaL was one of their louder, poppier, catchier tunes - a Marillion rocker. To call this just an acoustic mix doesn't do it justice. They took the basic bones of the song, and completely rebuilt it. It's an amazing change. The acoustic version swaps the bridge and the chorus, completely changing the fell of the structure, and turning it into something that's almost a ballad. Amazing, and much better than the original version of the song.
Thinking Plague, "This Weird Wind": Thinking Plague is a group that I have a hard time describing. To me, they sound like a very out-there post-rock group with classical influences, but I've been told that they call themselves a "Rock in Opposition" band. What they are is a distinctly peculiar ensemble. They've got vocals, but they use the singers voice like it's just another instrument in the mix - it's not leading the song in any way, it's just part of the music. The music itself is frequently atonal, with a very peculiar sound. The guitarist sounds very much like one of Robert Fripp's GuitarCraft students - but when I mentioned that in the past, he showed up in the comments saying "Who's Robert Fripp?" I love Thinking Plague, but I have a hard time recommending them - they're so strange that most people won't like them. If you're a big fan of both neo-progressive rock and 20th century classical, then definitely give them a listen.
EQ, "Closer": IQ is back! IQ is a progressive band that got started around the same time as Marillion. Also like Marillion, they started off sounding like a Peter Gabriel-era Genesis rip-off, but they've evolved their own very distinct sound over the years. They're absolutely fantastic - I'd put them up in the top of neo-progressive bands with Marillion and the Flower Kings. And they just released a new album, which is absolutely fantastic.
Sonic Youth, "Rain King (live)": Very typical Sonic Youth - strange tonality. Loud. Tons of hidden complexity. Brilliant. And performed live! No studio tricks here.
Kayo Dot, "The Useless Ladder": Another very hard-to-describe band. Roughly, they're what you get when a progressive metal band decides to start writing 21st century classical chamber music. Very, very highly recommended.
Red Sparrowes, "And By Our Own Hand Did Every Last Bird Lie Silent In Their Puddles, The Air Barren Of Songs As The Clouds Drifted Away. For Killing Their Greatest Enemy, The Locusts Noisily Thanked Us And Turned Their Jaws Toward Our Crops, Swallowing Our Greed Whole": It took me longer to type the title of that than it did to listen to it. Red Sparrowes is a really excellent post-rock band. But frankly, this track just annoys be because of the damn title.
Rachel's, "A French Gallease": A beautiful track by my favorite of the classically-leaning post-rock ensembles.

Read the comments on this post...

Orbits, Periodic Orbits, and Dense Orbits - Oh My!

Thu, 05 Nov 2009 12:16:18 -0500

Another one of the fundamental properties of a chaotic system is dense periodic orbits. It's a bit of an odd one: a chaotic system doesn't have to have periodic orbits at all. But if it does, then they have to be dense.

The dense periodic orbit rule is, in many ways, very similar to the sensitivity to initial conditions. But personally, I find it rather more interesting a way of describing key concept. The idea is, when you've got a dense periodic orbit, it's an odd thing. It's a repeating system, which will cycle through the same behavior, over and over again. But when you look at a state of the system, you can't tell which fixed path it's on. In fact, miniscule differences in the position, differences so small that you can't measure them, can put you onto dramatically different paths. There's the similarity with the initial conditions rule: you've got the same basic idea of tiny changes producing dramatic results.

Read the rest of this post... | Read the comments on this post...

Free Energy From Air? Sorry, no.

Mon, 02 Nov 2009 14:22:49 -0500

After the fiasco that was my flame against the downwind faster than the wind vehicle, you might think that I'd be afraid of touching on more air-powered perpetual motion. You'd be wrong :-). I'm not afraid to make a fool of myself if I stand a chance of learning something in the process - and in this case, it's so obviously bogus that even if I was afraid, the sheer stupidity here would be more than enough to paper over my anxieties. Take a look at this - the good part comes towards the end.

Read the rest of this post... | Read the comments on this post...

The Hallmarks of Crackpottery, Part 1: Two Comments

Wed, 28 Oct 2009 11:25:32 -0500

Another chaos theory post is in progress. But while I was working on it, a couple of comments arrived on some old posts. In general, I'd reply on those posts if I thought it was worth it. But the two comments are interesting not because they actually lend anything to the discussion to which they are attached, but because they are perfect demonstrations of two of the most common forms of crackpottery - what I call the "Education? I don't need no stinkin' education" school, and the "I'm so smart that I don't even need to read your arguments" school.

Read the rest of this post... | Read the comments on this post...

Chaos and Initial Conditions

Mon, 26 Oct 2009 22:07:15 -0500

One thing that I wanted to do when writing about Chaos is take a bit of time to really home in on each of the basic properties of chaos, and take a more detailed look at what they mean.

To refresh your memory, for a dynamical system to be chaotic, it needs to have three basic properties:

Sensitivity to initial conditions,
Dense periodic orbits, and
topological mixing

The phrase "sensitivity to initial conditions" is actually a fairly poor description of what we really want to say about chaotic systems. Lots of things are sensitive to initial conditions, but are definitely not chaotic.

Before I get into it, I want to explain why I'm obsessing over this condition. It is, in many ways, the least important condition of chaos! But here I am obsessing over it.

As I said in the first post in the series, it's the most widely known property of chaos. But I hate the way that it's usually described. It's just wrong. What chaos means by sensitivity to initial conditions is really quite different from the more general concept of sensitivity to initial conditions.

Read the rest of this post... | Read the comments on this post...

Back to Chaos: Bifurcation and Predictable Unpredictability

Tue, 20 Oct 2009 10:20:07 -0500

So I'm trying to ease back into the chaos theory posts. I thought that one good way of doing that was to take a look at one of the class chaos examples, which demonstrates just how simple a chaotic system can be. It really doesn't take much at all to push a system from being nice and smoothly predictable to being completely crazy.

This example comes from mathematical biology, and it generates a graph commonly known as the logistical map. The question behind the graph is, how can I predict what the stable population of a particular species will be over time?

Read the rest of this post... | Read the comments on this post...