July 22nd, 2008
Vulnerability disclosure gone awry: Understanding the DNS debacle
On July 7, the day before the release of the patch for the now infamous DNS design flaw, hacker Dan Kaminsky (with the help of Black Hat conference organizers) invited reporters to a press conference to “discuss the massive multivendor patch being released this Tuesday.”
“A synchronized release of this magnitude has not happened before,” read the invitation sent to the Black Hat conference press list.
By the time the patch was released, Kaminsky had briefed influential bloggers, recorded podcasts, scored a Wall Street Journal hit, celebrated an appearance on the front page of the BBC and won respect from his peers for coordinating such a massive cross-vendor patching effort.
It was a patching initiative that required six months of secrecy when countless security folks had to be kept in the loop. Potential patching hiccups had to be sorted out, important advisories/mitigations had to be prepared, DNS forwarding instructions had to be ready. A near impossible task, executed to perfection.
But, as Kaminsky admitted up front — and would soon find out — he made a major mistake of ignoring his peers in the hacker community, an intensely curious group prone to jealousies and stealing each other’s thunder.
In the days following the release of the patch, Kaminsky declined to provide technical details, insisting that affected vendors and end users needed at least 30 days to properly test and deploy the fix. Funny enough, the self-imposed 30-day deadline would end at the Black Hat conference where, at 11:15 a.m., Kaminsky would take to the stage and bask in the glory of his discovery.
Thomas Ptacek (right), principal of Matasano Security, was the first to call BS on the secrecy. Kaminsky immediately arranged a private conference call to spill the beans. Dino Dai Zovi, another researcher with hacker cred, was included. After the call, both Ptacek and Dai Zovi confirmed this was something super-serious that required immediate attention.
It was not enough. Monitoring the security mailing lists (Daily Dave, Full Disclosure, etc.), you could sense the backlash growing. Kaminsky’s request for a moratorium on public speculation — he even promised a Black Hat co-appearance for those who figured out the bug but maintained secrecy — did not sit will with everyone, including Ptacek.
Brand-name researchers started to grumble about the “cabal” approach to disclosure, openly venting that non-speculation and non-disclosure even after patch release were tantamount to being irresponsible.
Paul Vixie, of BIND fame, joined Kaminsky in pleading for the embargo but it was clear that public speculation would eventually emerge. It was only a matter of time before someone smart figured out how to forge and poison DNS lookups.
Halvar Flake (right), a reverse engineering guru who was among those arguing for public disclosure, published a guess/hypothesis that (almost) nailed the bug.
Ptacek’s Matasano followed up with a de-facto confirmation that filled in the missing pieces (the blog entry has since been pulled but the deed was done), forcing Kaminsky to acknowledge that his Black Hat thunder was stolen. Ptacek has since apologised but there are so many ruffled feathers, it’s hard to imagine things being the same in the land of trust/coordination/disclosure.
There’s a long list of researchers who argue that Kaminsky’s embargo was nothing but hype for the Black Hat conference. Kaminsky admits to being a media hacker and his pre-patch press conference and appearance on subsequent Black Hat marketing webcasts have done little to quell those concerns.
However, throughout this episode, I always got the sense that Kaminsky was genuine about wanting to give people adequate time to test and deploy the patch before things got ugly. Kaminsky has earned the right to be trusted on the severity of DNS-related issues so it’s sad that this debacle occured on his watch.
A lot of it was his own doing but, in the final analysis, maybe he deserved better.
There’s a lesson in here somewhere for those who try to figure out the politics and drama surrounding vulnerability disclosure.
For daily updates on Ryan's activities, follow him on Twitter.
