LinkedIn Not Liable

Earlier this week, a U.S. District Court in Northern California dismissed a class action lawsuit accusing LinkedIn of failing to deliver the level of security the plaintiffs say the social networking site’s privacy policy promised. A June 2012 data breach resulted in more than 6 million LinkedIn passwords being posted online. A few weeks later, a woman from Illinois and a woman from Virginia filed the suit—after learning that LinkedIn had encrypted the passwords with an outdated algorithm. Judge Edward Davila noted that the suit should not proceed to trial for several reasons. The plaintiffs, he said, wrongfully assumed that by paying for the site’s premium upgrade, they were entitled to a higher level of encryption for their data than users of the free version. Davila pointed out that, although the accusers admittedly never read the site’s privacy policy, it read,

“…we cannot ensure or warrant the security of any information you transmit to LinkedIn. There is no guarantee that information may not be accessed, disclosed, altered, or destroyed by breach of any of our physical, technical, or managerial safeguards. It is your responsibility to protect the security of your login information.”

The judge also failed to see how the posting of the passwords had, as the plaintiffs claimed, caused any economic harm or put them at future risk of identity theft.

Google’s Ups and Downs

It seems that the one-year anniversary of Google Play is not turning out to be the auspicious occasion Google had likely imagined. On Wednesday, the KrebsonSecurity.com blog reported that a new botkit is being used to trick Android users into downloading fraudulent banking apps capable of intercepting multifactor authentication messages from banks. The apps then send text messages with the purloined login credentials to the phony apps’ creators. That news appeared in the context of data that Google itself released on the Android developer blog showing that Android users can’t help but be plagued by malware. Google admitted that, based on data gleaned from mobile devices that accessed its app store during the two-week period that ended on Monday, only 16 percent of Android users have bothered to update their operating systems to the newest, safest versions. More than 40 percent of people with Android mobile devices still run a two-year old version known as Gingerbread. Kaspersky Lab, which keeps track of attempted malware installations on Android, reported that as of the end of 2012, Gingerbread was the most commonly targeted version of Google’s OS. (A SecurityLedger.com article notes that Apple, by contrast, has no such migration problems with its gadgets; 98 percent of all iPhone and iPad users run one or the other of the latest two iterations of iOS.)

The news isn't all bad about Google, though. The search-and-now-just-about-everything-else company did something this week for which it should be lauded. It struck a blow against the U.S. government surveillance program that has expanded rapidly since the passage of special laws that allow agencies such as the FBI to much more easily demand information from Internet service providers, credit bureaus, banks, and businesses like Google—all without a warrant. The demands for information, called National Security Letters (NSLs), come with a built-in gag order barring the companies receiving them form even mentioning that they’ve received them. But on Tuesday, Google became the first company to give a hint of the extent to which the FBI uses this authority. It published a document giving ballpark figures for the number of accounts for which it turned over information in a given year. For instance, it reported that in 2010 it divulged information on “2000–2999” customers; in 2009, 2011, and 2012, the range was “1000–1999.”

Although the U.S. Congress requires the FBI to disclose the number of times it issues NSLs (it sent out more than 16 000 in 2011), Google didn’t report exact numbers. “This is to address concerns raised by the FBI, Justice Department and other agencies that releasing exact numbers might reveal information about investigations,” Richard Salgado, a Google legal director, wrote in a blog post. But at least the existence of the NSLs and the potential for abuse is out in the open. The FBI continues to have this power to say information about you is “relevant” to an investigation and get unquestioned access to records—even after a 2007 Justice Department inquiry revealed that after the September 2001 terrorist attacks, the FBI regularly ran afoul of the relaxed rules regarding the acquisition of evidence.

There seems to be a slow but steady backlash growing among healthcare providers against the U.S. government’s $30 billion initiative to get all its citizens an electronic health record, initially set to happen by 2014 but now looking at 2020 or beyond. The backlash isn’t so much about the need for, or eventual benefits of, electronic health records but more about the perceived (and real) difficulties caused by the government's incentive program and a growing realization of the actual financial and operational costs involved in rolling out, using, and paying for EHR systems.

The backlash began to publicly surface last September when the U.S. government accused healthcare providers of “upcoding,” i.e., claiming with a single click on a field in a electronic health record to have provided a medical service or procedure when it wasn’t really performed. Kathleen Sebelius, the current HHS Secretary, and Eric Holder, the Attorney General, sent a letter to five major hospital trade associations (pdf) warning them that electronic health records were not to be used to “game the system” and “possibly” obtain “illegal payments” from Medicare. The letter said that Medicare billing is being scrutinized for fraud, and implied that those using EHRs to bill Medicare will be scrutinized even more carefully.

Healthcare providers were outraged by accusations in the letter, and said that the reason for the increased billing was that EHRs facilitated billing for services they used to provide to the government without charging for them.

About the same time, professors Stephen Soumerai from Harvard Medical School and Ross Koppel from the University of Pennsylvania wrote an article for the Wall Street Journal contending that EHRs don’t save money as claimed. They wrote that, “…. the most rigorous studies to date contradict the widely broadcast claims that the national investment in health IT—some $1 trillion will be spent, by our estimate—will pay off in reducing medical costs. Those studies that do claim savings rarely include the full cost of installation, training and maintenance—a large chunk of that trillion dollars—for the nation's nearly 6000 hospitals and more than 600 000 physicians. But by the time these health-care providers find out that the promised cost savings are an illusion, it will be too late. Having spent hundreds of millions on the technology, they won't be able to afford to throw it out like a defective toaster.”

The professors went on to say that, “We fully share the hope that health IT will achieve the promised cost and quality benefits. As applied researchers and evaluators, we actively work to realize both goals. But this will require an accurate appraisal of the technology's successes and failures, not a mixture of cheerleading and financial pressure by government agencies based on unsubstantiated promises.”

The professors’ conclusions soon came under attack by EHR vendors, but the article seemed to have struck a nerve with many EHR adopters.

Next, in November, the U.S. Department of Health and Human Services inspector general (IG) released a report  that stated in part that the U.S. electronic health record incentive program administered by the Centers for Medicare & Medicaid Services (CMS) was “vulnerable” to fraud. The IG said that CMS “has not implemented strong prepayment safeguards” to keep healthcare providers from falsely claiming that they are meeting the required meaningful use standard [i.e., capture, use, and share data], and that CMS’s “ability to safeguard incentive payments postpayment [i.e., conduct audits] is also limited.”

CMS agreed with the IG that it needed to start verifying that healthcare providers are indeed meeting the meaningful use criteria, but disagreed that it should do more than the minimal cross-checking needed to determine whether healthcare providers are being truthful or not when submitting their claims for incentive payments. Healthcare providers also expressed their unhappiness about having to offer more proof that there were indeed meeting the meaningful use standards.

The backlash gained momentum when RAND published a new EHR study in January of this year that basically repudiated a key RAND EHR study from 2005. The 2005 study, paid for by several large EHR vendors, claimed that the U.S. could save at least $81 billion per year in health care costs, as well as drive down the rate of healthcare spending, through the widespread use of EHR systems. The study was a major point behind selling Congress on the U.S. EHR initiative.

For EHR vendors, the 2005 study was money well spent. However, the latest RAND study now admits that it was overly optimistic—or, more to the point, hopelessly unrealistic—as its critics at the time said. RAND’s latest report has studiously avoided putting any numbers on how much EHRs reduce (or increase) costs.

Also in January, the American Medical Association (AMA) sent a letter [pdf] to the Office of the National Coordinator for Health Information Technology (ONC) that it needs to slow down and rethink the EHR “meaningful use” criteria that healthcare providers have to meet in order to get reimbursed for their EHR investments. The AMA wrote that while it “shares the administration’s goal of widespread EHR adoption and use, …  we again stress our continuing concern that the meaningful use program is moving forward without a comprehensive evaluation of previous stages to resolve existing problem. A full evaluation of past stages and more flexible program requirements will help physicians in different specialties and practice arrangements successfully adopt and use EHRs.”

The basic AMA complaint is that the ONC is rushing the adoption of EHR technology at the expensive of its effective use in realistic medical settings.

Then in February, a survey of over 17 000 EHR adopters found that some 17 percent are already considering changing their EHR vendor because their EHR systems fail to meet their basic needs. The opinion survey, conducted by Black Book Rankings, indicates that 2013 might be the “year of the great EHR vendor switch,” a story at Healthcare IT News reports. As was predicted, the U.S. government’s EHR incentive program created a “gold rush” mentality where new EHR vendors popped out of the woodwork offering highly immature products along with extremely poor customer support, and healthcare practitioners bought them nevertheless, so as to not lose out on ONC EHR incentive payments.

Even those incentive payments to healthcare providers may not be enough to make EHR adoption worthwhile. Just a few days ago, a study published in the March issue of the journal Health Affairs and reported in HealthLeaders Media found that, “The average physician would lose $43 743 over five years after adopting EHRs and only 27 percent of physicians would profit through the transition away from paper records without federal financial aid. And even when the $44 000 in meaningful use incentives are added to the pot, only 41 percent of physicians would be in the black.”

The study also states, “The largest difference between practices with a positive return on investment and those with a negative return was the extent to which they used their EHRs to increase revenue, primarily by seeing more patients per day or by improved billing that resulted in fewer rejected claims and more accurate coding.” However, as we noted above, using EHRs to increase revenue might be greeted with an audit for fraud by the U.S.  Government.

"EHR: money loser, or federal government audit magnet?" is not exactly a good marketing slogan.

Another contributor to the perception that EHR conversion is a money losing proposition is the fact that for the military, it is.The U.S. Departments of Veterans Affairs and Defense told Congress that the cost of integrating their two EHR systems has climbed from between $4 billion and $6 billion to some $12 billion, and so they were calling a halt to the effort, which was scheduled to be completed by 2017, until they figured out what to do next. Congress, which had been vigorously pushing for the integration since 2008, was highly not amused, especially since over $1 billion has already been spent with little to show for it.  

When VA and DoD record systems will interoperate seamlessly is anyone’s guess, as is the cost to make it happen. As is the date when doctors, hospitals, and other health care providers—everyone but the EHR vendors themselves—will start seeing an adequate return on their investment in terms of time or money saved.

Photo: Cuneyt Hizal/iStockphoto

It’s been a fairly quiet week in regard to IT glitches of any major significance. That said, there were still a sufficient number of snarls, snafus and errors to interfere with work as well as generally upset, annoy and outrage a lot of people. We start off this week's review with an issue affecting NASA’s $2.5 billion Mars rover mission.

NASA Curiosity Goes into Safe Mode Due to Memory Issue

Responding to a problem it detected Wednesday morning with the data coming from the Mars rover Curiosity, NASA announced on Thursday that it had “switched the rover to a redundant onboard computer in response to a memory issue on the computer that had been active.”

NASA said that it will shift the rover from its current “safe mode” operation to full operational status over the next few days as well as troubleshoot what is causing the “glitch in flash memory linked to the other, now-inactive, computer.”

The NASA press release stated that on Wednesday the rover communicated "at all scheduled communication windows…but it did not send recorded data, only current status information. The status information revealed that the computer had not switched to the usual daily ‘sleep’ mode when planned. Diagnostic work in a testing simulation at JPL indicates the situation involved corrupted memory at an A-side memory location used for addressing memory files.”

A detailed story at CNET quoted Curiosity Project Manager Richard Cook as telling CBS News that, “We were in a state where the software was partially working and partially not, and we wanted to switch from that state to a pristine version of the software running on a pristine set of hardware.”

The project team thinks that space radiation, while a remote possibility, may in fact be to blame, CNET said. Again quoting Cook:

“In general, there are lots of layers of protection, the memory is self correcting and the software is supposed to be tolerant to it…But what we are theorizing happened is that we got what's called a double bit error, where you get an uncorrectable memory error in a particularly sensitive place, which is where the directory for the whole memory was sitting…So you essentially lost knowledge of where everything was. Again, software is supposed to be tolerant of that...But it looks like there was potentially a problem where software kind of got into a confused state where parts of the software were working fine but other parts of software were kind of waiting on the memory to do something...and the hardware was confused as to where things were.”

Cook indicated that, in essence, a reboot of the inactive computer should clear things up, but that the team will do a lot of analysis before that happens to make sure that there isn’t anything more troublesome lurking about.

The rover problem no doubt annoyed many NASA scientists given that Curiosity had, only a few days earlier, drilled into the Mars surface to gather for analysis the “first sample ever collected from the interior of a rock on another planet.”

Amazon Bug Wipes Out iOS Users’ Kindle Libraries

Not as significant as the Curiosity issue but still annoying to its Apple product users was the Amazon update error related to its Kindle iOS app.

On Wednesday morning, Amazon warned Apple iOS users not to download its latest Kindle app because of “a known issue” which turned out to be something capable of deleting Kindle libraries from their devices, CNET reported.  Reading the comments at the Amazon Kindle forum seemed to indicate that not everyone was affected by the bug, however.

According to CNET, “After downloading the initial update [version 3.6.1], existing Kindle users were logged out of their accounts, and everything they had downloaded was deleted from their devices. They also lost bookmarks and other settings, according to angry comments on iTunes. Users then had to log back in to Kindle and redownload their books from the cloud. Some complained that they had to delete the app entirely and download it again.”

Amazon fixed the problem later in the day with version 3.6.2. How may users got whacked is not known, or at least Amazon isn’t saying.  Interestingly, on Amazon’s Kindle app for iPad, iPhone & iPod touch page, it says the latest app version is 3.5.

T-Shirt Maker Blames Computer for Violent Phrases Targeting Women

Another company has found that it needed some updating to its website as well. Worcester, Mass., T-shirt maker Solid Gold Bomb was in fully apology mode last week when its t-shirts appeared for sale on Amazon with a range of phrases such as “Keep Calm and Punch Her” being one of the “least hateful” ones, a story at the Daily Mail reported on Thursday and again with more vigor on Friday. Soon after the t-shirts appeared for sale, a flood of outrage appeared on social networks against both Solid Gold Bomb and Amazon.

Solid Gold Bomb founder Michael Fowler tried somewhat unconvincingly to explain that the problem resided with a poorly thought out and careless computer algorithm he created that allowed certain offensive combinations of words to appear on the t-shirts. Fowler didn’t say exactly why neither he nor anyone else in his company thought the words “Keep Calm and …” when combined with “murder,” “knife” and “rape” would be generally acceptable—especially when they could further be combined with the words “her” or most other personal pronouns, for that matter.

Fowler says that the company is correcting the problem, and that, “Rest assured, we do not condone the offense nor do we have any desire to promote it. Ultimately, it comes down to my error and I should singly accept repsonsibility (sic) for the mistake. Again, my sincere apologies for the unintended outcome.”

Maybe Solid Gold Bomb could invest in a spell checker as well.

EHR Problem at Canberra Hospital Forces Emergency Patients Elsewhere

There was a brief news item in the Canberra Times about a “glitch” in Canberra Hospital’s electronic health record system Wednesday afternoon that “forced their emergency department to direct patients with ‘non-urgent’ issues elsewhere.” The story went on to say that critical emergency patients were still being seen, but the emergency department was “reverting to paper systems while IT experts fix the problem.”

The problem was reportedly fixed by Thursday.

An eerily similar problem hit Bellevue Hospital in New York City a few weeks ago, according to the Wall Street Journal .  In this case, ambulances containing anyone other than psychiatric patients were diverted away from the hospital’s emergency room because of a “computer glitch.”

I was under the impression that quick and seamless rollover procedures to begin using paper medical records was standard operating procedure in event that an EHR system went offline, but it seems that this switch-over isn't so easy for emergency rooms.

BB&T Bank Customers Get Scare

Diverted attention could have been used to describe BB&T customers this weekend.  With all the stories about the dangers of identity thieves being able to drain personal bank accounts, many BB&T bank customers wondered if that had happened to them when they tried to access their accounts and saw that they showed zero balances.

According to WRCB TV in Chattanooga, Tennessee, an “internal computer issue” that affected ATM, mobile, and online systems incorrectly indicated to customers that they had zero bank balances and therefore could not access or move their money. I bet more than one customer experienced a quick panic attack on when that happened.

While BB&T, which is headquartered in Winston-Salem, North Carolina, has not said how many customers were affected (it has some 6.4 million customers), there were enough calls to its 1-800 customer service lines to overload its phone system. 

BB&T says everything is okay now, and assures its customers that it “will reverse any fees or charges that may have occurred as a result of this issue.”

Image: JPL-Caltech/MSSS/NASAJPL-Caltech/MSSS

Stuxnet’s Development Program Was a Long Thought-Out Process

On Tuesday, researchers from Symantec’s Security Response team released a report offering proof that the Stuxnet worm that targeted industrial facilities in Iran—most especially the Natanz uranium enrichment facility suspected to be part an Iranian effort to produce nuclear weapons— is two years older than previously thought. The 18-page report reveals that development of the malware dates back to 2005, although it first appeared in the wild in 2007. It wasn’t identified until July 2010. What explains the two-year lead time? An extended refinement process was probably part of what made Stuxnet and its precursor, Flame, so sophisticated. The exploits these bits of malware pulled off without attracting attention were "nothing short of amazing," Mikko H. Hypponen, chief research officer for F-Secure, a security firm in Helsinki, Finland, told IEEE Spectrum. Furthermore, says Hypponen, "You need a supercomputer and loads of scientists to do this." Symantec acknowledges that Stuxnet, which was designed to “take snapshots of the normal running state of the system, replay normal operating values during an attack so that the operators are unaware that the system is not operating normally... [and] prevent modification to the [compromised system] in case the operator tries to change any settings during the course of an attack cycle” is among the most complicated coding ever seen.

For more on how Stuxnet really worked and on the efforts to track it down, see "The Real Story of Stuxnet" in this month's issue of IEEE Spectrum.

Advanced Malware Escapes Sandbox with Help from Twitter

New malware designed to steal sensitive information exploits a patched sandbox-bypass vulnerability in Adobe Reader. The malicious code, dubbed MiniDuke by the researchers at Kaspersky Lab and CrySyS Lab, who discovered it and released a report about it this week, has attacked the systems of government agencies in 23 countries, mostly in Europe. Among its novel features are the use of steganography to hide the code it uses to create, then slip in and out of backdoors in the compromised systems; the ability to assess whether a computer is in use; and the ability to determine what detection capability the machine has. MiniDuke can also reach out to Twitter accounts created by the attackers to access tweets seeded with information pointing to command and control servers offering continually updated commands and encrypted backdoors. MiniDuke successfully bypassed the sandbox protection in Adobe Reader despite a patch meant to cover the vulnerability added on 20 February.

The Kaspersy and CrySyS researchers report that the malware is introduced via social engineering. A PDF claiming to contain information about Ukraine’s foreign policy and NATO membership plans and one purporting to provide information about a human rights seminar are laced with the infection.

“This is a unique and very strange attack. The many different targets hit in separate countries, together with the high profile appearance of the decoy documents and the weird backdoor functionality indicate an unusual threat actor,” says the Kaspersky and CrySyS report.

Cryptography No Longer an Effective Security Measure?

"We need to think about security in a post-cryptography world," says Adi Shamir, a luminary in the world of public-key cryptography. That comment was part of his remarks at the Cryptographers' Panel session at the RSA Conference on Tuesday. Shamir, who helped design the original RSA algorithm, noted that because advanced persistent threats (APTs) have penetrated even the most secure computer systems, “We should rethink how we protect ourselves.” He reasons that, “It's very hard to use cryptography effectively if you assume an APT is watching everything on a system.”

Internet Service Disruptions for Copyright Scofflaws

On Monday, leading U.S. Internet service providers announced that a program under which they will disrupt Internet access for repeat online copyright offenders has begun. The “Copyright Alert System” for which the nation’s major record labels and Hollywood studios strongly lobbied, features “mitigation measures” (.pdf) that kick in after four documented instances of unauthorized use or distribution of copyrighted material. These measures include slowing the user’s Internet download speed and redirecting their browser to an “educational” landing page about infringement. Though the Digital Millennium Copyright Act calls for ISPs to cancel the accounts of repeat copyright offenders, the newly created Center for Copyright Information, which is in charge of the Copyright Alert System, insists that it will not wield that weapon.

U.S. High Court Dismisses Government Cybersnooping Case

On Tuesday, the U.S. Supreme Court dismissed a legal challenge to the federal government’s warrantless electronic communications surveillance program. The 5-4 decision (.pdf) supported the government’s claim that wiretapping laws cannot be challenged in court. But its main conclusion was that the American Civil Liberties Union, journalists and human-rights groups that sought to end the warrantless snooping made permissible by the FISA Amendments Act, also known as §1881, had no right to sue. The majority’s rationale: “[The groups] have no actual knowledge of the Government’s §1881a targeting practices. Instead, [they] merely speculate and make assumptions about whether their communications with their foreign contacts will be acquired under §1881a.” So, in other words, because the plaintiffs couldn’t present black-and-white evidence that their calls and e-mails to people outside the country had been intercepted, they couldn’t demand that the government quit doing it. The High Court was not moved by the groups’ claims that the 2008 legislation has chilled their speech and violated their privacy rights under the Fourth Amendment.

Observers note that if the Supremes hadn’t dismissed the challenge, the government would have likely batted it away by invoking the state secrets privilege, claiming that the suit threatened to expose national security secrets.

There was a great story over at Ars Technica this week regarding a recently published special audit report (pdf) by West Virginia’s Legislative Auditor regarding the state’s purchase three years ago of 1164 Cisco model 3945 routers at a price of US $24 million using federal stimulus funds (a tip of the hat to a Risk Factor reader for bringing this to our attention in a comment to a recent post).  The auditor concluded that not only did the purchase bypass the state’s competitive purchasing rules for IT equipment; the state bought far more capability than it would ever need now or in the foreseeable future, and at non-competitive prices to boot. 

The audit report, for example, gives as an example the “city of Clay in Clay County [which] received 7 total routers to serve a population of 491. Five of these routers are located within .44 miles of the each other.” The cost of those seven servers—each of which can support 200 simultaneous users—was around $20 000 apiece.

The auditor noted that over $6.6 million was spent on Cisco model 3945 router features that weren’t necessary to begin with. Furthermore, if the state had actually purchased the correctly sized routers, it could have saved at least another $8 million or so. I say at least, because that number is based on router prices quoted in a non-competitive bidding environment—holding a competition that included other router manufacturers (Alcatel-Lucent, Brocade, HP, Juniper, et al.) would have likely saved even more money. For each $5 million saved on routers, the state could have purchased 104 additional miles of needed broadband fiber, the auditor noted.

I name those manufacturers specifically because the West Virginia audit report points to “California State University, the largest four-year university in America, [which] used a competitive bidding purchase to purchase an eight-year refreshing of its 23-campus 10G network. The Director of Cyber Infrastructure of California State University provided documentation showing that Alcatel-Lucent won the project with a bid of $22 million. Cisco’s bid was $122.8 million. The other bids were Brocade at $24 million, Juniper at $31.6 million, and HP at $41 million. Furthermore in May of 2011, Purdue University bid out replacement components for its Hansen Computer Cluster. Cisco won the Purdue University competitive bid process by offering a 76 percent discount off the cost of its products.”

Why did this wasteful fiasco happen? The audit report basically says no one really knows for certain—or at least is willing to 'fess up to being the party who screwed up: stuff just sort of happened.  The best that can be determined was that those receiving the federal stimulus funds wanted to spend as much of them as fast as possible, need be damned. Or in the auditor’s words, “Those making the decisions on how to spend the money did not consult individuals with technical knowledge on the best methods to utilize the funds.”

The audit report tried to place some of the blame on Cisco for selling routers that the state obviously didn’t need. Cisco, which admits its routers are expensive, says the blame doesn't rest with them: It merely sold equipment to West Virginia in the quantities and type the state asked for.

What happened with the routers does not bode well for West Virginia’s $98 million system modernization effort, which has been under fire for government project risk mismanagement.

If you have a chance, read over the Ars Technica article as well as the auditor’s report. There is also an article at WCHS Radio 58 concerning members of the U.S. House Subcommittee on Communications and Technology trying to find out why as part of the Cisco purchase a $22 500 router was bought for a West Virginia library with a single computer.

Head of the U.S. National Telecommunications & Information Administration, Lawrence Strickling, staunchly defended the purchase saying, “I think it's not at all clear from those reports that what West Virginia did was unreasonable in terms of its choice of a single platform Cisco router at the time they made it.” Strickling says that those making the purchase were smart to purchase equipment that would allow for future growth.

Strickling didn’t say when in the future he expected the population being served by the Marmet Public Library, where the lone router now sits and is open only three days a week, would grow to a level where the library would need to serve 220 simultaneous users. With the local population currently of about 1500, my guess it will be long after that Cisco router is sent to the recycling center.

After reading through the audit report and other articles, IT professionals will be highly amused, and American taxpayers, whose federal stimulus money was wantonly wasted, will be highly outraged. I was both.

 Photo: West Virginia Legislative Auditor

This past week has seen a hodgepodge of IT-related uff das, glitches and snarls. However, we are going to start this week off with millions of human errors avoided by IT.

Computerized Provider Order Entry Systems Avoid an Estimated 17.4 Million Medication Errors Per Year

Last week, the Journal of the American Medical Informatics Association (JAMIA) published a study that estimated the reduction in medication errors in U.S. hospitals that could reasonably be attributed to their computerized provider order entry (CPOE) systems.  The study’s authors said that they “conducted a systematic literature review and applied random-effects meta-analytic techniques” to develop a “pooled estimate” of the effects of CPOEs on medication errors.

They then took this estimate and combined it “with data from the 2006 American Society of Health-System Pharmacists Annual Survey, the 2007 American Hospital Association Annual Survey, and the latter's 2008 Electronic Health Record Adoption Database supplement to estimate the percentage and absolute reduction in medication errors attributable to CPOE.”

Working through the data, the authors concluded that a CPOE system decreases the likelihood of error by about 48 percent . "Given this effect size," say the authors, "and the degree of CPOE adoption and use in hospitals in 2008, we estimate a 12.5% reduction in medication errors, or ∼17.4 million medication errors averted in the USA in 1 year.”

The study authors are careful to note that it is unclear whether this reduction in medication error actually “translates into reduced harm for patients,” although the research tends to lead one towards that conclusion.

The number of medication errors avoided because of CPOEs is expected to rise as more hospitals install them. Only about 20 percent of U.S. hospitals had deployed CPOE systems as of the middle of 2012.

Microsoft’s Azure Goes Off-line Because Of Embarrassing Oversight

Last Friday afternoon, Microsoft shot itself in the foot when it let an SSL (secure sockets layer) certificate expire, taking down its Azure cloud services, the AP reported. A Forbes story said the outage eventually lasted over 12 hours. And according to a story at ComputerWorld, Microsoft also admitted that the issue caused problems with its Xbox Music and Video Store services.

ComputerWorld quoted Microsoft as saying, “Beginning Friday, February 22 at 12:44 PM PST, Storage experienced a worldwide outage impacting HTTPS operations (SSL traffic) due to an expired certificate… We apologize for any inconvenience this causes our customers.”

Well, Microsoft was at least forthcoming about what the problem was, despite how embarrassing the admission may have been. As you may remember from last week, IBM’s New Zealand Virtual Server Services suffered an outage that lasted from early Monday morning into Wednesday. IBM, however, still hasn’t explained what caused the outage or even bothered with a pro forma apology to its customers for the inconvenience caused by the outage.

Software Update Takes Out Communications to International Space Station

On Tuesday morning, a routine software update to the International Space Station’s flight systems caused the ISS to lose communication with NASA’s Houston Mission Control. According to a story at the Houston Chronicle, “Houston flight controllers were updating software onboard the station's flight computers at 8:45 a.m. when one of the station's data relay systems malfunctioned. Although a backup computer took over critical station functions from the primary computer, a NASA press alert noted that the station was not able to communicate with NASA's Tracking and Data Relay Satellites.”

The crew was able to restore communications via a back-up computer by around 11:30 a.m. CST—about the time when the ISS flew over Russia. I suspect the crew was secretly happy for the brief comms blackout while it lasted.

Glitches Hit Parking Systems in Pittsburgh and Vancouver

What software glitches giveth, they also taketh away. The Pittsburgh Post-Gazette reported that a programming error in Pittsburgh’s 550 new parking meter pay stations inadvertently gave motorists free parking last Monday at 3500 parking spaces. Though Monday was a U.S. federal holiday (Presidents Day), and many cities don’t charge for parking on federal holidays, Pittsburgh had intended to get revenue from the new meters. But the programming glitch (or oversight) caused all the pay stations to display messages indicating that parking was free because of the holiday.

Pittsburgh’s parking authority corrected the error by late Monday morning. But by then, it was too late to figure out when motorists had parked in the spots. So the parking authority decided to forego the normal fees for the rest of the day at the spaces covered by the pay stations. However, motorists still had to pay at the 4000 or so metered parking spots throughout the city that were not affected by the glitch.

Pittsburgh’s parking authority didn’t know how much money it failed to collect, but it didn’t think it lost too much because traffic was light—because of the holiday.

Motorists in Vancouver weren’t so lucky.  According to the Vancouver Sun, about 1000 motorists who parked at Diamond-run parking lots last Wednesday and Thursday and paid by credit or debit card were charged multiple times because of a “glitch” in the payment system. Diamond told the Sun that the problem occurred after “Elavon [the company that processes payments for Diamond] updated its processing system.”

Diamond apologized for the inconvenience.

Facebook: No One is Over 100

Our final glitch concerns a programming misstep by the good (young) folks at Facebook. The social media site won’t accept a registrant’s age if it is greater than 99. According to an AP story last week, Marguerite Joseph, a Michigan grandmother who is 104 and an avid Facebook user, has been unable to list her correct age on her Facebook page.  Every time her granddaughter, who takes care of the page for Mrs. Joseph, puts in her birth date of 1908, Facebook changes it to 1928. So Mrs. Joseph has had to settle on being 99 years of age for the past two years.

Facebook initially had no comment about the story, but soon admitted to the AP in another story that there is a “glitch in the system” with accepting pre-1910 birth dates. Facebook is now working on a fix. It has also apologized to Mrs. Joseph.

Photo: Rafe Swan/Getty Images

The Golden State's Department of Motor Vehicles (DMV) must think it has checked into an IT version of Hotel California, where once a DMV modernization project is started, it can never ever finish it.

Last week, on behalf of DMV's management, California’s CIO informed state legislators that it had decided to cancel at the end of January the remainder of its US $208 million, 6-year IT modernization project with Hewlett-Packard, which was supposed to be completed in May of this year. As reported in the LA Times, after spending some $134 million ($50 million on HP) and having “significant concerns with the lack of progress,” the DMV decided to call it quits and do a rethink of the program’s direction. HP had apparently saw the handwriting on the wall. Its contract ended last November, and HP refused to hire key staff until the contract was renegotiated.

The DMV IT modernization program was started in 2006 in the wake of a previous DMV project failure (called Info/California) that blew through $44 million between its start in 1987 and cancellation in 1994. That “hopeless failure,” as it was then described, was supposed to be a 5-year, $28 million effort; when it was terminated seven years in, the project’s cost to complete had skyrocketed to an estimated $201 million with an uncertain finish date. A 1994 LA Times story reported that an assessment found the DMV had limited experience in computer technology, grossly underestimated the project’s scope and size, and lacked consistent and sustained management. The project's failure also sparked a full legislative probe.

The current DMV debacle, along with this month’s termination of the MyCalPay’s project, has spurred calls for yet another probe. Legislators could save a lot of time and money by just cutting and pasting from the the earlier project's investigation. I'm sure they'll find a lot of the same inexperience, underestimating, and inconsistent management.

Not all was lost in the current effort: at least a new system for issuing California drivers’ licenses was rolled out. However, the critical vehicle registration portion of the DMV system, with its decades-old “dangerously antiquated technology” (pdf), will have to stay in use while a new go-forward plan is developed.

In looking over the latest project status reports, I couldn’t help but notice how California Technology Agency can be so exact with the project’s total estimated cost ($208 103 287.00), yet be so inexact about the DMV’s IT modernization status. According to the CTA’s dashboard, the project’s status was “yellow” (meaning that the project is slipping) right up until the day it was cancelled.  As far as I can tell it never reached “red,” meaning “a project that is in need of immediate intervention.” Coincidentally, the US Air Force’s $1 billion Expeditionary Combat Support System (ECSS) project humiliation was also rated by the US Department of Defense’s CIO as “yellow” right up to the day it was terminated. Maybe we should alert the fashion press that yellow is the new red.

This is the second DMV-related fiasco in the past six months for HP courtesy of EDS, which it bought in 2008. As I mentioned in December, HP and the state of Vermont agreed to a “no-fault” divorce over a botched six-year, $18.5 million DMV IT system modernization effort. HP agreed to refund Vermont the $8.37 million it was paid, and Vermont returned to HP all the physical and virtual rights to the software and documents created by HP. Vermont citizens will have to put up with their 36-year old DMV IT system for a while longer as well as the state tries to figure out what to do next.

EDS won the Vermont DMV contract in 2006, and that no doubt helped it in its bid to win the California DMV contract in 2007. However, I don’t think either state would now agree much with HP’s claim that as far as motor vehicle administration solutions, it has “The right technology and the right skills.”

I don't know the record length of time and number of attempts for trying to replace an IT legacy system, but for the moment, California's DMV's 26 years and two attempts and still counting has to be close. Anyone have one better?

Photo: Thomas Winz/Getty Images

This week’s IT snafus and snarls have a definite international flavor to them. The first story takes us to the U.K., and a story of some “crossed lines.”

O2 Customers Complain About Eavesdropping on Calls

Last Tuesday, the Register ran a story about some Birmingham, England-area customers of U.K. mobile provider O2 being able to listen in on calls apparently originating in Scotland. According to the Register, customers started to complain about the “crossed lines” the previous week, but the weekend was nearly over before O2 was even able to confirm that this eavesdropping was indeed happening. Still, said O2 to the Register on Monday, it was “unable to replicate the problem despite having received ‘a handful’ of complaints.’”

Then a story in the London Telegraph said that the problem had spread beyond Birmingham to Scotland, Wales, and Liverpool, and potentially involved anyone using the O2 network in the affected areas.

On Thursday, a Daily Mail story reported that O2 had traced the problem to a network cable and card. The Mail quoted an O2 spokesperson as saying that, “We had a problem with a network card responsible for transferring call traffic in the Birmingham area which resulted in a handful of customers experiencing crossed lines during phone conversations...Our engineers identified that a cable linked to the card was not working correctly and fixed the problem at 6.15pm on Tuesday. We have been monitoring the situation closely with no further reported issues. We apologise for any inconvenience caused to our customers.”

During the eavesdropping interlude, U.K. financial expert Martin Lewis warned O2 and other wireless customers to be careful what they said, especially concerning their financial and personal affairs.  But according to the Register, this same problem has been intermittently reported by O2 customers since 2010, and Martin's opinion is probably good advice given that the U.K. security services want to snoop on all phone calls being made.

Birmingham UK Bus Passengers Free to Ride

Along with getting to listen in on other people’s phone conversations, Birmingham, England, bus passengers also got free rides on National Express West Midlands bus routes last Wednesday morning. The Birmingham Mail said that an error involving a software update affected the buses’ onboard ticket machines, preventing passengers from purchasing Daysaver and return tickets. As a result, bus drivers allowed riders to travel free during Wednesday’s morning rush.

However, the machines were still capable of issuing single (more expensive) journey tickets and bus passengers riding later in the day were required to purchase them if they did not already possess a Travelcard. This did not make those particular passengers happy. The software error seemed to have been fixed by Thursday.

Disk Failure Hits India’s ID Number Registration

Last March, IEEE Spectrum’s associate editor Josh Romero wrote an in-depth story on India’s attempt to issue national identity numbers to its 1.2 billion citizens. In the story, Romero wrote that, “the project is called Aadhaar, which means 'foundation' or 'support,' because it’s meant to be a fundamental technology platform that will enable dozens of new public and private services to be created.”

The IT project, considered to be one of the largest in the world, has numerous challenges. As Romero stated, “It’s easy to list major challenges: How exactly do you collect biometrics from every single person in the world’s second most populous country, especially those living at the margins? How do you keep bad data from getting into the database in a country rife with corruption? And how can you build the entire system around online authentication in a country where fewer than one in 20 people have access to the Internet?”

Well, another challenge related to the last was recently added on the list. The Hindustan Times reported last week that there has been “an error in disks in which enrollment data was stored and provided to unique identification authority for de-duplication—the technical process before Aadhaar number is created.” As a result, tens of thousands of Indians who enrolled for their ID number six months ago or longer will need to re-enroll.

Making matters a bit more complicated, the Times reports, is that “the government has remained silent on the technical failure and the only way to know about it was by logging on to the UIDAI [Unique Identification Authority of India] website.” As Romero notes, this might cause a problem for the majority of people.

Exactly how many people are affected by the disk problem is not known.

IBM’s New Zealand Virtual Server Services Go Out

Moving next to New Zealand, IBM’s New Zealand SmartCloud Virtual Server Services (VSS) were offline all day Monday and into today. The New Zealand Herald reported that IBM’s NZ$80 million data center at Highbrook Park South Auckland went offline due to an unexplained technical issue at about 3am local time Monday and according to ComputerWorld, was still not fixed by the end of the day Tuesday.

IBM opened the data center in May 2011 and promised “industry-leading IBM service-level agreements of 99.9% uptime that you can count on.” This is probably ringing a bit hollow to the numerous New Zealand companies, schools, and universities that the Herald reported as being dependent on IBM’s cloud.

AT&T Software Error Caused Problems in Nevada

Back in the U.S., television station KTVN reported that a software error disrupted AT&T landline, cell, and Internet service for its customers in the Reno, Sparks, and Carson City, Nevada, areas from last Monday afternoon into Wednesday evening. AT&T declined to say how many business and residential customers were affected by the error. Some emergency 911 calling capability was reportedly impaired, KTVN reported.

A story from the Reno-Gazette Journal also indicated that there was an unrelated AT&T hardware issue in the Carson City region that compounded the outage problem.

Al Jazeera America Confirms Sarah Palin Not Going to Do Commentary

Our final glitch of the week involves the Washington Post, which had to publish a “correction” to columnist Suzi Parker's story last week about Sarah Palin’s intriguing decision after leaving Fox News to join Al Jazeera America as a host and commentator.

The only trouble was that Palin had made no such decision, as Al Jazeera quickly confirmed.

What Parker did, according to a story at Politico, was to read but not recognize as satire a story about Palin leaving Fox News to join Al Jazeera that the Daily Currant (which advertises itself as "The Global Satirical Newspaper of Record") ran in early February. Needless to say, Parker got hammered for her mistake, and the Daily Currant got a lot of traffic.

The Daily Currant “reported” on Monday that Palin had decided not to join Al Jazerra after all, but instead had decided to accept a position at the Kennedy School of Government at Harvard where she will teach four courses over the next three years, including:

• John Locke and the State of Exception: Extrajudicial Executive Action In the Age of Terror
• The Evolutionary Psychology of The Welfare State
• Pascal, Chateaubriand and The Modern U.S. Evangelical Movement
• The Geopolitics of Arctic Hydrocarbon Resource Development

According to a Palin spokesperson, the Currant said, “The governor was ‘thrilled’ to be working at Harvard, and hoped to bring a little ‘Wasilla main street’ to the Ivy Towers of America's most venerated university.”

We’ll see if anyone rises to the bait this time.

Photo:iStockphoto

In the post-Stuxnet world, the prospect of undeclared cyberwar has been dragged out of the shadows to the front pages. With that in mind, yesterday the U.S. National Institute of Standards and Technology (NIST) kicked off an effort to establish a set of best practices for protecting the networks and computers that run the country’s critical infrastructure. The Cybersecurity Framework was initiated at the behest of President Barack Obama, who issued an executive order calling for a common core of standards and procedures aimed at keeping power plants and financial, transportation, and communication systems from falling prey to any of a wide range of cybersecurity threats.

The first step, says NIST, will be a formal Request for Information from infrastructure owners and operators, plus federal agencies, local government authorities, and other standards-setting organizations. NIST says it wants to know what has been effective in terms of keeping the wolves at bay. To that end, it will hold a series of workshops over the next few months where it will gather more input. The agency says that when the framework is completed in about a year, it should give organizations “a menu of management, operational, and technical security controls, including policies and processes” that will make them reasonably sure that their efforts represent an effective use of their time and resources. 

Oddly, though, the press release announcing the development of the Cybersecurity Framework makes no mention that the final public version of a report titled, "Security and Privacy Controls for Federal Information Systems and Organizations" was released on 5 February and that the public comment period continues through 1 March.

Image: Linda Bucklin/iStockphoto

Ah, I love the smell of napalmed IT projects in the morning!

Not, though, when they are government IT projects and the wafting odor is from taxpayer monies going up in smoke.  And unfortunately, for past few weeks, the stench of burning government IT projects has been especially pungent.

We start off in California, where after burning through some $50 million, California State Controller John Chiang announced last Friday he had decided to terminate the state’s US $89.7 million contract “with SAP as the system integrator for the MyCalPAYS system, the largest payroll modernization effort in the nation.” The planned 5-phase effort mercifully never made it past the first pilot phase.

Furthermore, Chiang said that the Secretary of the California Technology Agency (CTA)  has “suspended further work until the CTA and SCO [State Controller’s Office] together conduct an independent assessment of SAP’s system to determine whether any of SAP’s work can be used in the SCO’s go-forward plan to address the State’s business needs.”

You may remember that Chiang sent SAP a letter last October warning that the project was “foundering and is in danger of collapsing,” and gave SAP one last chance in the form of a demand for urgent get-well efforts from the company. Chiang claimed that there were errors in one out of every three tasks performed by SAP's system, and that there hadn’t been a single pay cycle without material payroll errors occurring.

In Friday’s announcement, Chiang threw in the towel. He said that while he had hoped “for a successful cure to SAP’s failure to deliver an accurate, stable, reliable payroll system, SAP has not demonstrated an ability to do so.” This was especially disheartening, Chiang implied, given that the SAP effort covered only 1300 SCO employees who had “fairly simple payroll requirements.”  There was no way the SAP system could be trusted to support the payroll requirements of the state's "240 000 employees, operating out of 160 different departments, under 21 different bargaining units."

SAP said in response to the news of its contract termination that it was “extremely disappointed in the actions. SAP stands behind our software and actions.... SAP also believes we have satisfied all contractual obligations in this project.”

All of this, of course, suggests that when the napalm smoke clears, a date in court will be in the offing. Chiang as much as said so in the announcement: “The SCO will pursue every contractual and legal option available to hold SAP accountable for its failed performance and to protect the interests of the State and its taxpayers. This includes contractually required mediation and, if necessary, litigation.”

An SCO spokesperson called the project’s performance “frightening,” but what must be really frightening to California taxpayers is the continued inability of the state to manage the acquisition of its IT projects. So far, nearly $254 million has been spent so far in two unsuccessful attempts to get a state government payroll system in place, the LA Times reports. If SAP fights instead of settles, it would at least be a public service, exposing the depth of California’s IT project risk mismanagement.

The upshot is that California will continue to use its decades-old Cobol-based payroll system until it figures out what to do next. And to help it figure that out, the SCO has—in the best tradition of government—set up an IT Procurement Task Force. Whenever in doubt, form a committee.

I hope the Task Force members have strong stomachs; the stench of IT project failure coming out of California is of the mephitis variety.

UN’s Umoja Project Failure of Management

If you see another cloud of smoke on the horizon, it's not Vatican, announcing the new pope. It's too early for that, and the cloud is way too large. It's the UN’s troubled enterprise resource project dubbed Umoja (or “unity” in Swahili) that is supposed “to equip the organization with twenty-first century techniques, tools, training and technology.” Fox News reported on Monday that it's going to burn though a lot more money before it is completed, assuming it ever is.

The project’s cost has climbed from an original US $286 million in 2008 to $316 million in 2009 and now looks like it will reach at least $348 million by 2015, “with three years still to go after that,” Fox reports. The completion dates have climbed with the dollars: The project was supposed to be delivered in 2012, but the current forecast go live date has slipped from 2015 to now December 2018. The likelihood of achieving that date is probably close to zero.

According to the Fox News story, even after promises in 2009 to follow accepted IT project management practices in the light of project problems that had already surfaced back then, UN auditors have found the project does not have “systems in place that could link the budget to milestones and deliverables.” In fact, the project management team told what had to be startled auditors that “linking the budget milestones and deliverables was not a requirement under the United Nations system accounting standards.”

The auditors must have felt that they had to go back and reread what their job description was after that revelation.

Next, the auditors found that the Umoja project team has not yet planned on how it will move legacy system data into the new system; auditors estimate this effort alone could tack at least another $110 million onto the project. Nor has the project team started to fulfill another promise made back in 2009 of ensuring “effective and ongoing education and training in both the new system but also all the new processes and standards that are going to be required.” The approach that the project team appears to have decided on, the auditors discovered, is to roll out the new system and hope its every feature and function is self-evident to the user.

I wonder what the Swahili word is for unmitigated IT project conflagration.

US Army Business Systems Implementations Worry DoD OIG

There's an IT disaster cloud forming over the Pentagon as well, and it may eventually dwarf the others. The Department of Defense Office of the Inspector General published a report last week that shows grave problems in the Army Office of Business Transformation (OBT).

In a review of the OBT business systems information technology strategy that is supposed to guide the implementation of Army IT systems worth some $10.1 billion over their life-cycle, the OIG found that the OBT’s overall strategy “did not include specific ERP [enterprise resource management] implementation milestones and performance measures for accomplishing the Strategy’s goals, [for example] including a plan for using ERP capabilities, or clearly define the Army Enterprise Systems Integration Program’s ERP integration role or milestones.” Sound familiar? It's as if OBT had decided to follow the UN's IT project management best practices.

The reason for the lack of specific ERP implementation milestones and performance measures, the OIG report stated, was that “OBT officials [were too] focused on near-term milestones.” OBT officials must not have been focusing too hard, however, given the OIG also found that “although OBT officials included 25 implementation tasks in the Strategy, with due dates of May 2011 and August 2011, the Army did not complete 16 of these tasks as of March 2012. This occurred because OBT officials did not adequately monitor the development and completion of the implementation tasks.”

Army leadership promised the OIG that it is its “intention” to start practicing IT project management 101, but things might change because “Army business system strategies and processes are changing rapidly and frequently due to both internal and external influences.”

In other words, the OBT’s intentions might soon be OBE (overtaken by events).

USAF ECSS Billion Dollar Debacle: We Don’t Believe in Accountability

Last Saturday, NBC Nightly News ran another one of its “Fleecing of America” segments on US governmental waste. This episode (video) involved the burnt-out wreck of the USAF Expeditionary Combat Support System, one of my favorite IT project debacles—it has managed to scorch through $1 billion with absolutely nothing to show for it. While the NBC News segment didn’t have much new to say, it did highlight the facts that: (a) the primary contractor CSC is still insisting that “it provided the Air Force with capabilities and assets to deliver the system of the future and that taxpayers got their money's worth” for the billion dollars spent, and that (b) no one will be fired or even punished over the debacle.

When Sen. John McCain was asked by NBC reporter Lisa Myers, “Should people be fired over this?”  McCain answered, “Sure. Sure they should be. Will they be? No.”

In fact, Brig. Gen. Kathryn J. Johnson (who took over the project in May 2012) made it very clear that firing or even demoting anyone for blowing a billion dollars of taxpayers’ money was out of the question. In an interview with the Federal Times last November, when she was “asked if anyone has been fired or demoted for the program’s failure, Johnson said no. ‘We didn’t feel it was necessary to do that,’ she added.”

Maybe NBC Nightly News might want to interview Gen. Johnson and ask why the $%#$ not? And, at what point should some heads roll? Two billion? Ten billion? Or is managerial incompetence just part of the job description?

The Wall Street Journal ran a story last week on the USAF’s replacement for ECSS, which is supposed to be implemented by 2017. Not surprisingly, the Gen. Johnson refuses to state how much this replacement system will cost, but she does claim that it will cost less than ECSS.

My question: Is that cost in comparison to the original $668 million cost of ECSS or the nearly $8 billion  that it would have taken to complete ECSS to its original specifications?

Marin County Settles Lawsuit and Ensures It Can’t Talk About Its Own Incompetence

Did you think we were done talking about failed California government IT projects? California and SAP—yes, SAP again!—have an apparently inexhaustible supply of them. You may recall one from 2009, involving Marin County’s MERIT (Marin Enterprise Resource Integrated Technology) system. The county had originally thought in 2005 that the MERIT system would cost it US $12 million, but  the county ended up spending nearly $30 million over four years in a vain attempt to get the system to meet the county's original specifications before finally giving up.

In 2010, the county sued consulting company Deloitte Consulting LLP for $30 million along with unspecified punitive damages over what it claims was a botched SAP ERP implementation. It also named SAP and former county auditor Ernest Culver in another $90 million lawsuit. In that suit, Marin County claimed (pdf) that the project vendors had improperly influenced Culver to act favorably towards them by "approving Deloitte’s deficient work on the project, approving payments, and causing Marin County to enter into new contracts with Deloitte and SAP Public Services, Inc."

Well, after spending $5 million on its lawsuit in which the judge threw out many of the Marin County’s claims, the county and Deloitte reached a settlement last month in which the county will receive $3.9 million from Deloitte, and nothing from SAP and or Culver.  

According to the Marin Independent Journal, which has been following the lawsuits since the beginning, the terms of the settlement are “strictly confidential" and can't be discussed in any way, shape or form. The only thing the county would say in relation to the settlement was that it "reduced the total fees paid to Deloitte Consulting and spares the taxpayers the continued expense of litigation. It does not constitute an admission of wrongdoing by either party.”

When the Journal asked the Marin County Counsel about whether the “taxpayers got a raw deal [by] spending $5 million to get a $3.9 million settlement,” the Counsel would only say, “I can't say anything about what you just said.”

How very convenient. The confidentiality agreement allows Marin County managers to keep quiet on how they also colossally mismanaged the project.

As far as I can tell, no one has been fired or demoted over in this IT project debacle either. In fact, according to another Journal story, many of the same folks are working on the plans for the next attempt. But not to worry, County Administrator Mathew Hymel told the Journal, “When we fall short, our goal is to learn from it and do better next time.”

Of course, what those lessons might be can't be discussed—they're confidential, you know.

UK Government CIOs: ERP Systems Don’t Deliver

If you're starting to think that along with the specific incompetencies of all these IT organizations, there might be a fundamental problem with enterprise resource projects in general, you'll be happy to find some confirmation of that coming out of the U.K. An article in Information Week in late January that reported that a survey of 100 U. K. public CIOs and other senior IT and financial managers indicated that, “63 percent of [the] respondents said their ERP system hasn't met their expectations in at least one area. It also suggests that more than 33 percent of respondents have spent more than they had expected on their ERP implementation, 20 percent are disappointed by how their platform met their needs out of the box, and only 20 percent would be prepared to go through a similar ERP project a second time.”

The survey, Information Week stated, was carried out by an independent market researcher but paid for by a local U.K. ERP supplier Advanced Business Solutions.

Those 20 percent who indicated that they would be prepared to go through a similar ERP project a second time are either very good IT project managers, or masochists. Our advice in either event: Try to stay upwind of the stench and smoke from the failed projects started by those project arsonists playing with matches all around you.

 

Photo: iStockphoto

This week’s IT hiccups and snafus are a varied lot. We’ll start off with the University of Wisconsin’s ongoing payroll and benefits system saga.

$1.1 Million Lost Because of Glitches in UW Payroll System Glitches – More May Follow

The Wisconsin State Journal reported last week that “glitches” with the University of Wisconsin’s controversial payroll and benefits system had resulted in US $1.1 million in improper payments which the university may likely end up having to absorb. In addition, the Journal reported, University President Kevin Reilly warned that further examination of the payroll system “by system staff, an independent analyst and the state auditor are ‘likely to find more issues.’”

This news has not gone over well with Wisconsin state legislators, who were already upset when an audit by the Legislative Audit Bureau released late last month indicated that problems with the UW payroll system had resulted in $33 million in improper payments being made over the past two years. Another Journal article reported that while some $20 million of those $33 million in overpayments have been recovered, much of the remaining $13 million may well have to be written off.

When the $33 million in overpayments was first reported, UW's Reilly put out a statement that said in part, “I am deeply troubled by these mistakes…. We will identify exactly why and how these significant errors occurred, we will validate that steps we have already taken are working, we will take any additional steps that need to be taken, and we will make absolutely sure that similar errors do not happen again.”

Announcing more overpayment losses less than a month later and begrudgingly admitting that more are likely to come isn’t exactly a great way to create confidence that you have the situation under control—especially if the system in question already has a long and somewhat sordid history to begin with.

You see, the current $78 million payroll system, which went live in April 2011, was UW’s second attempt at trying to replace a 30-something-year-old legacy payroll system. UW originally tried to replace its legacy system in the early 2000s at a then estimated cost of $19.7 million and a go-live date of January 2005. However, that project ended up being canceled by Reilly in July 2006 after $28.5 million had been spent and its estimated final cost-to-complete had reached some $62 million.

The current payroll development efforts began in 2007, but it did not take too long for this effort to run into trouble as well. System planning costs, for example, quickly jumped from an estimated $1.6 million to $12 million. This did not amuse state legislators, who were seeing a host of troubled government IT projects state-wide.  By 2010, UW said that the project was back on track, within budget and on schedule. UW officials promised that the new payroll system, which UW said would be used for decades, would provide a “high level of reliability” and “accuracy.”

Right now, Wisconsin state legislators are acting more like they are from Missouri (the "Show Me" state), wanting UW to demonstrate that its promises of a reliable and accurate payroll system can be trusted. Given what has been happening, that may take a while.

Facebook Error Hides Prominent Web Sites from its Users

“More than 50 million pages and 10 million apps are now part of the Facebook platform,” Inside Facebook reported earlier this month. The totals come from a Facebook Security and Exchange Commission 10-K (pdf) filing the story said. The extent of Facebook’s reach was on display last Thursday when, for about 30 minutes, Internet users found out that they could not log into many of their favorite websites using their Facebook login credentials because of an error with Facebook Connect.

Facebook “explained” the error in a note stating that, “For a short period of time, there was a bug that redirected people logging in with Facebook from third party sites to Facebook.com. The issue was quickly resolved, and Login with Facebook is now working as usual.”

While this glitch would usually be minor news, what made it interesting – or maybe scary – was the number of web sites affected by the glitch. InfoWorld listed just some of them, including: “ABC, BuzzFeed, Capital.fm, CNN, DailyMail, ESPN, Etsy, Fox News, Gawker, Geico, HBO, Hollywood.com, The Huffington Post, Hulu, InfoWorld, NFL, OKCupid, People, Pinterest, Reddit, Slate, Smallworlds, SwagBucks, The Sydney-Melbourne Herald, TED, The Los Angeles Times, The New Zealand Herald, The Washington Post, Vulture, Weather.com, WikiAnswers, WordPress, XOJane, Yahoo, and YugaTech.”

As the InfoWorld story noted, “Website owners should be concerned about this, knowing that their site traffic and associated revenue could be cut off at any moment due to a Facebook glitch that's entirely beyond the site owners' control to fix.”

Best Buy Makes Good on BioShock Order Glitch

Last week, the gaming website Sidequesting reported that Best Buy accidentally cancelled thousands of preorders of the special Limited Edition of BioShock Infinite for some unexplained reason. At first, Best Buy offered a $10 gift card to those affected. But it soon figured out that that particular olive branch would not likely placate a lot of very disappointed and angry gamers.

So instead of taking a beating in the blogosphere it could ill afford, Best Buy decided to honor the pre- orders with a free copy of the game. Good move on Best Buy’s part.

Canadian Government Pours Salt into Some Data Breach Victims’ Wounds

In January, Human Resources and Skills Development Canada (HRSDC) informed some 538 000 people who had taken out loans through the Canada Student Loans Program between 2000 and 2006 that a hard-drive containing their personal information, including names, Social Insurance Numbers, dates of birth, contact information and loan balances had gone missing.

According to a story in the Winnipeg Free Press , “The hard drive was discovered missing from an office in Gatineau, Que, on Nov. 5 by an employee who had stored it in a filing cabinet. Management was not informed until Nov. 22. A detailed analysis of the files on the hard drive was completed Dec. 6. The Office of the Privacy Commissioner was notified on Dec. 14. HRSDC Minister Diane Finley called in the RCMP on Jan. 7. The news was made public Jan. 11.”

The loss of the hard drive has sparked multiple lawsuits against the HRSDC.

Unfortunately, even the agency's attempt to send letters to victims informing them of the data breach, was fraught with error. HRSCD said that a “technical issue” occurred which sent some of its notifications of the breach to the incorrect person. CTV News says that the government blamed the letters' misdirection to a printing error.

The HRSDC has played down the likelihood of anyone’s identify being at risk from the lost hard drive, but it has offered free credit monitoring to those whose information is on it. It also says the misdirected letters only contained names and addresses, and no other personal information, so it wasn't a big deal.

HRSDC told the press it will be sending “pre-paid envelopes to those who received letters intended for others so they can be returned.”

Somehow I doubt that gesture will do much in the way of restoring HRSDC’s perceived reputation for managerial incompetence.

Photo: iStockphoto

Minnesota Government Employee Wrongfully Accessed Driver’s License Data

It’s hard enough to keep your personal information out of the hands of cybercriminals bent on using it to steal from you or fraudulently acquire things in your name. But it seems like there’s no hope when organizations you trust with your personal details—like the Minnesota Department of Public Safety—mishandle them. That was likely the case for roughly 5000 state residents who found out this week that a former state employee has been charged with illegally accessing the records associated with their driver’s licenses. The data thief, who was once the state's Department of Natural Resources Enforcement Division's administrative manager, was authorized to look at a resident's records when they related to his office’s official business. But between 2008 and last October, he used his credentials to query the state Driver and Vehicle Services database more than 19 000 times. He looked up the names of politicians, judges, county and city attorneys, police officers, news reporters, family members and other state employees. Most of his downloads were of women whose pictures appeared in the database.

According to a Kaspersky Lab Threatpost article, four people who have been notified that their records were wrongfully accessed are suing the alleged perpetrator and other state employees. “They said the data breaches caused severe emotional stress and physical harm and were the result of ‘lax policies and lax enforcement’ that allowed an unsupervised, unmonitored Hunt to continually access records for years,” says the Threatpost article.

Government Agencies, Military Among Users of Vulnerable Industrial Control System

What do the FBI, the Drug Enforcement Agency, the U.S. Marshals Service, the IRS, the U.S. Passport Office, the British Army, and Boeing, have in common? They are just a few of the thousands of organizations whose facilities depend on an industrial control system with a security hole that could allow attackers to remotely control critical building functions such as electronic door locks, lighting systems, elevators, electricity and boiler systems, video surveillance cameras, alarms. The vulnerability in the Tridium Niagara AX Framework was reported on 5 February at the Kaspersky Security Analyst Summit.

Billy Rios and Terry McCorkle, security researchers with Cylance, demonstrated a zero-day attack that yields access to the system’s config.bog file, which holds login credentials and other data for operator work stations, and controls the systems that are managed by them. The exploit, say Rios and McCorkle, takes advantage of a vulnerability that gave them root on the system’s platform. “The platform is written in Java, which is really, really good from an exploitation standpoint,” Rios told Wired. “Once we can own the platform, a lot of the other stuff is very, very straightforward [to attack],” said Rios.

Rios and McCorkle reported that a search turned up roughly 21 000 Tridium systems that were accessible over the Internet.

In a written statement, Tridium revealed that the researchers notified it about the vulnerability in December; it has been working on a patch, which it says it expects to release by 13 February. In an attempt to downplay the vulnerability, the statement noted that, “The vast majority of Niagara AX systems are behind firewalls and VPNs—as we recommend—but clearly, as Rios and McCorkle have shown, there are many systems potentially at risk.” That’s a change of tune from Tridium’s stance just last year, when it told the Washington Post that its systems benefited from security through obscurity.

Tried-and-True Thieving Techniques Taken Up Again

Cyberthieves have developed sophisticated malware that can infiltrate a victim’s computer, allowing a thief to tap into online banking sessions initiated by customers in real time. Such malicious code is capable of conducting fraudulent transactions right under the victim’s nose and covering its tracks by updating the account balance and transaction history display in the victim’s browser. But because banks have developed countermeasures including software that detects anomalies in customers’ online access, some crooks are eschewing session hijacking and going back to the old and familiar: stealing login credentials for subsequent access from a separate computer. This shift was confirmed by researchers at security firm Trusteer, who reported this week that they noticed changes in the Tinba and Tilon financial Trojan programs. According to a 7 February blog post by Amit Klein, Trusteer's chief technology officer, the Trojans divert a customer attempting to access his or her bank’s website to a fake version. The rest is history, says Klein:

“Once the customer enters their login credentials into the fake page the malware presents an error message claiming that the online banking service is currently unavailable. In the meantime, the malware sends the stolen login credentials to the fraudster who then uses a completely different machine to log into the bank as the customer and executes fraudulent transactions.”

Now banks have to be on the lookout for both the new and (relatively) old-school techniques.

Adobe Releases Emergency Security Update

On 7 February, Adobe released a patch for its Flash Player meant to stop hackers from using two zero-day vulnerabilities to take over Windows PCs and Macs. Adobe was already planning to release a Flash Player update on 12 February, but because the software maker was “aware of reports that CVE-2013-0633 is being exploited in the wild in targeted attacks designed to trick the user into opening a Microsoft Word document delivered as an email attachment which contains malicious Flash content," it released the fixes as soon as they were ready. The other vulnerability was being used for so-called drive-by attacks that victimize computer users who navigate to a malicious website hosting an exploit.

Last April Fool’s Day, the BBC reported that the UK government was planning to introduce legislation that would allow the monitoring of all the “calls, emails, texts and website visits of everyone in the UK” by the Government Communications Headquarters ( GCHQ) intelligence agency. The information would be monitored in real-time and then stored for two years before being erased. The government needed the monitoring capability, it said, to be able to “investigate serious crime and terrorism and to protect the public.”

The government also promised that the legislation would “ensure that the use of communications data is compatible with the government's approach to civil liberties.”

It's good to see the tradition of doublethink is alive and well in the UK.

Almost immediately, members of even the government’s own party said that this legislation was a massive overreach and threatened civil liberties. Telecommunication and Internet providers weren’t too happy either, saying that the program was going to be expensive and a nightmare to implement.

A pre-legislative parliamentary scrutiny committee was set up to look into the feasibility of the proposed legislation, now being dubbed the “snoopers charter.” By late autumn, word was that the committee did not like what it saw and was preparing to say so in a report in early December. The UK Home Secretary, Theresa May, was aggressively pushing the legislation and on 3 December, upon hearing of the committee’s unflattering appraisal of it, launched a preemptive strike on the committee’s findings. She told the Sun newspaper that the legislation had to be passed, otherwise “we could see people dying” and “criminals going free” including “pedophiles who will not be identified.” She also warned of a reduction in “our ability to deal with this serious organized crime.”

May concluded, “Anybody who is against this bill is putting politics before people’s lives.”

However, the committee was unimpressed by May’s "you are either with us or against us" attack.  On 10 December, the Guardian published a story detailing the committee's determination that the legislation was unworkable as written, that it “tramples on the privacy of British citizens,” and further that the estimated cost of the effort of some £1.8 billion over 10 years was “fanciful and misleading.” Nick Clegg, the leader of the government’s Liberal Democrat coalition party, told May, “We cannot proceed with this bill and we have to go back to the drawing board."

So politics and common sense won out, at least for a little while.  There were warning signs that this wouldn't last, however. While May stated that she was “open-minded” about changing the legislation, the Guardian reported that she “remained determined to introduce it before the session ends next spring and get it on the statute book before the next election.”

This week May's snooping desires got a boost as the London Telegraph reported that the cross-party parliamentary Intelligence and Security Committee (ISC) has come out in support of the "snoopers charter," though it also warned that the “the government must do more to convince public of the need for them.”  Hmm, sounds like it time to beat the “it’s all for the sake of the children” drum a bit louder, or maybe, to say, a la Orwell, that the charter is needed as an “act of self-defense against a homicidal maniac.”

According to the Telegraph, the Director General of MI5, Jonathan Evans, said that without the legislation, “it was increasingly difficult to be confident that targets were being fully watched” because of rapid changes in communication technology. And in a related story at the Guardian, the Home Office claims that the charter is urgently needed as “there is already a 25 percent ‘capability gap’ between the tracking data that the security services need to access and their ability to do so.”

Evans did admit to the ISC, though, that the Home Office’s 25 percent figure depended upon some “pretty heroic assumptions,” the Guardian reported. In other words, it was most likely a number that made for a good news sound bite, but that the capability gap has little credibility indeed.

A story at the Daily Mail reports that the UK's intelligence service says it isn't interested in unfettered access to the content of every communication, and that its fetters would still be court orders, which it would continue to obtain. It just wants information on “who sends a message, where and how it is sent, and who receives it.”

Of course, with people's identities closely bound with their cellphones, and with all the GPS and other information that cellphones throw off these days, this metadata is often more important than the information content itself, much of which, by the way, can probably be inferred pretty quickly with advanced data analytics. And if the messages are passing though the communication channels being monitored by the U.S. National Security Agency, the contents can probably be provided to GCHQ without a UK court order request even being filed.

The Daily Mail article also points out that GCHQ isn’t worried whether the messages are encrypted, either. Apparently, it has “options” to deal with it.

How this all plays out, time will only tell. But the idea of a democratic government that maintains its belief in its citizens' right to privacy also claiming in the same breath it also has a right to snoop on all forms of electronic communication reminds me of another Orwell quote: “We have now sunk to a depth at which restatement of the obvious is the first duty of intelligent men.”

Image: iStockphoto

There was a real potpourri of IT-related glitches, snarls, and snafus to choose from last week. We start off with the lingering after-effects of the grounding of the USS Guardian on a Philippine reef—which we first noted a few weeks ago.

U.S. Navy Decides to Scrap Minesweeper Stuck on Ecologically Sensitive Philippine Reef

On 17 January, the U.S. Navy minesweeper USS Guardian ran hard aground on a reef within the protected Tubbataha Reefs Natural Park in Philippine waters where it remains stuck. A preliminary assessment indicates that the ship was following a National Geospatial-Intelligence Agency-supplied Coastal Digital Nautical Chart (DNC) that “misplaced the location of a reef by about eight nautical miles.” The reef is located in a UNESCO World Heritage restricted zone, and any damage caused to the reef is heavily fined.

The Navy had hoped that it could wrestle the USS Guardian free without too much damage to the reef or the ship, but those hopes were dashed when the 23-year-old wooden-hulled ship started taking on water.  As a result, the Navy decided that the best option was to dismantle the ship and remove it as three separate sections. A floating crane from Singapore is being brought in to help with the ship’s removal.

An interesting story last week at the website Maritime Accident Casebook indicates that the navigational snafu has been attributed to human error at the National Geospatial-Intelligence Agency (NGA). According to the story, the NGA decided to update its navigational charts in 2008 using  LANDSAT-derived imagery because of the age and uncertainty of information shown on the nautical charts in that area of the Pacific (some dating back to 1940 and 1942, an earlier MA Casebook article says). Some of the old charts even indicated the presence of “phantom islands.”

Quoting an NGA spokesperson, “One of these images included incorrect information about the location of the section of ocean that includes the Tubbataha Reef. At the time, no other source information existed to validate that imagery data. As a result, the reef was incorrectly placed in the DNC.”

Then, in 2011, the NGA became aware of the error, and corrected all the charts except one: that being the one for the area around the Tubbataha Reef. According to the NGA, this was a result of “a failure to follow established procedure.”

In the wake of the incident, the NGA has reexamined charts covering “more than 116 million square nautical miles of ocean” and found only one other error of a “magnitude similar to the misplacement of the Tubbataha Reef.” That one corresponded to an area off the coast of Chile. Mariners have been warned of the discrepancy.

The Navy expects that it will take about a month to remove the USS Guardian. The fine to be levied is unknown, but it is likely to be substantial. The political price may be substantial as well.

Technical Issues Hit Amazon, Bank of America, PayPal  and Twitter

A cluster of IT glitches last week hit some well-known companies. First, on Monday, there were reports that PayPal customers ended up being charged multiple times for their transactions over a period of about three hours. PayPal has strongly denied The Register's claims the problem lasted 15 hours. A story at FierceCIO says that the multiple-charge problem was the result of instant payment notifications that were delayed in being sent back to customers. The delay caused many customers to think their PayPal transactions didn’t go through, causing them to make one or more additional payments. PayPal says that, “All customers will be refunded for duplicate transactions as soon as possible.”

Then on Thursday, Amazon suffered a 49-minute outage that made its homepage inaccessible, although it said that its other pages were fine. Amazon has been closed-mouthed about what caused the outage, other than to say it wasn’t hacked nor was it a problem with its cloud. It has been estimated that the cost to the company will be around $5 million in lost revenue.

Also on Thursday, Twitter said in a message to its users that there were “intermittent issues affecting Web and mobile users, globally, between approximately 7:00am and 9:50am PST.” The message  went on to say that, “We apologize to users who were affected by this, and we’re working to ensure that similar issues do not occur.” The message did not say what those issues were.

In an apparently unrelated matter, Twitter then announced Friday that it had “discovered one live [security] attack and were able to shut it down in process moments later. However, our investigation has thus far indicated that the attackers may have had access to limited user information—usernames, email addresses, session tokens and encrypted/salted versions of passwords—for approximately 250,000 users.”

As a precaution, Twitter has “reset passwords and revoked session tokens for these accounts.” Affected users will be receiving an email asking them to reset their passwords; if you get one, just be careful it isn’t phish.

Also on Friday, Bank of America said that its electronic banking operations and telephone call centers were inaccessible. The Washington Post reported that the problem was caused by unexplained “technical issues” rather than a cyber attack. And according to a story at the BBC which coincidentally was published on Friday, no one should be surprised by similar outages at other banks this year because of the ever increasing complexity of banking software.

Another Week, Another Stock Market Gaffe

This week’s stock market gaffe happened Friday on India's National Stock Exchange. In this case, an error in the software being used by the brokerage Religare Capital Markets Ltd. caused TaTa Motors' stock price to fall by 10 percent. Religare was quoted by Bloomberg News as saying, “Due to some technical issue in the software, unintended transactions got executed.”

Bloomberg said that the error will likely cost the brokerage some 100 million rupees (around US $1.8 million).

Last year you may recall there was another trading glitch that caused the National Stock Exchange (NSE) Nifty index to plunge over 800 points in a few minutes, wiping out some $58 billion in value from the fourth largest market in Asia.

Saving London’s Iconic Black Cabs – At Least for Now

Finally, last October, I noted that Manganese Bronze, the maker of the iconic London black taxi, announced that it was going into administration—the U.K. version of U.S. bankruptcy law's Chapter 11. The reason was an accounting error that went unseen for over two years when the company switched to new accounting software. The result: the company understated by £3.9 million its historical losses. Given the poor economic health of the company and the intense competition in London’s taxi market, Manganese Bronze stock took a nosedive when the accounting error became public. It looked like only a matter of time before the company, which was then worth roughly £5 million, would go belly up.

Fortunately, last week, Chinese car manufacturer company Zhejiang Geely, which already owned 20 percent of Manganese Bronze, decided to buy the rest of the company and its assets for £11.04 million “through a newly established British subsidiary, Geely UK,” the London Telegraph reported. The new owners say they are “confident” the business will be profitable within three years.

I hope so. London wouldn’t really be the same without those black taxis.

 

Photo: Naval Aircrewman 3rd Class Geoffrey Trudell/U.S.Navy

Hackers Break Into News Outlets’ Computers to Peek at Reporters’ Notes

On 30 January, the New York Times reported on its site that it was the victim of a sophisticated campaign of cyberattacks aimed, it suspects, at uncovering the names of sources who provided information about the business dealings of Chinese Prime Minister Wen Jiabao and his family. (In fact, we’re learning that the Times was only the latest publication to have its systems raided, but more on that later.) According to the NYT article, Chinese hackers—who tried to cover their tracks by infecting and remotely controlling computers at U.S. colleges then using those compromised machines to send the malicious code—started snooping around the Times’ internal networks as early as 13 September. This after word got out that journalists at the daily’s Shanghai bureau were conducting research into how Wen had amassed a fortune worth billions. According to a researcher at Mandiant, the computer security company the paper hired to exorcise the malicious code:

“[The hackers] set up at least three back doors into users’ machines that they used as a digital base camp. From there they snooped around The Times’s systems for at least two weeks before they identified the domain controller that contains user names and hashed, or scrambled, passwords for every Times employee.”

Mandiant discovered that the hackers used the passwords to access the computers of 53 Times employees. But Times Executive Editor Jill Abramson, who was quoted for the story, says, “Computer security experts found no evidence that sensitive e-mails or files from the reporting of our articles about the Wen family were accessed, downloaded or copied.” The Times was also quick to offer reassurance that no customer data was stolen. But what the hackers did in fact take is still an open question.

Even after the article about Wen was published on 25 October, the hackers continued snooping. The Times articlereferences a December intelligence report prepared by Mandiant. The security firm had uncovered evidence that the “Chinese hackers had [from as far back as 2008] stolen e-mails, contacts and files from more than 30 journalists and executives at Western news organizations, and had maintained a ‘short list’ of journalists whose accounts they repeatedly attack.”

That assessment was confirmed on 31 January, when the Wall Street Journal admitted that hackers trying to monitor the newspaper's coverage of China, hacked into its systems. Bloomberg says it was targeted after publishing an article last June about Xi Jinping, China’s then vice president and current general secretary of the country’s Communist Party. But Bloomberg says that although its computer systems came under attack, they were never breached.

Thousands of Networked Gadgets Double as Gaping Security Holes

Computer World is reporting that faulty implementation of the Universal Plug and Play (UPnP) protocol standard has turned millions of network-enabled devices such as routers, printers, media servers, and even smart TVs into gateways through which hackers can get inside firewalls. On 29 January, security researchers from Rapid7 released a research paper in which they noted that more than 20 percent of the 80 million unique IP addresses they pinged exposed the UPnP Simple Object Access Protocol service to the Internet. This allows one networked device to discover another and remotely turn on the other gadget’s data sharing, media streaming, media playback control and other services. The Computer World article explains that:

“In one common scenario a file-sharing application running on a computer can tell a router via UPnP to open a specific port and map it to the computer's local network address in order to open its file-sharing service to Internet users.

Many had UPnP implemented through a library called the Portable UPnP SDK. Unfortunately, as the Rapid7 researchers discovered, UPnP SDK contains eight remotely exploitable vulnerabilities. Two of them can be used to inject code remotely.

The upshot: More than 23 million networked devices exhibited this vulnerability during the test. Rapid7 told Computer World that a patch has been released, but the firm’s chief security offer predicted in a 29 January blog post that “it will take a long time before each of the application and device vendors incorporate this patch into their products.”

The slow-to-update problem, says Rapid 7, also affects users of a UPnP library called MiniUPnP, which can be exploited for denial of service and remote code execution attacks. New versions released in 2008 and 2009 don’t contain those security holes. But according to Rapid7, 14 percent of the Internet-exposed UPnP devices it pinged were still using MiniUPnP 1.0 and were thus still vulnerable. Though Rapid7 has released a free tool called ScanNow for Universal Plug and Play, and a module that detects vulnerable UPnP services running inside a network, many vulnerable devices will remain unpatched.

“Many PC users don't even update PC software that they frequently use and are familiar with,” Thomas Kristensen, chief security officer at vulnerability research and management firm Secunia told Computer World. “The task of finding the Web interface of a vulnerable networked device, obtaining the firmware update and going through the whole update process will likely be too intimidating for many users,” he said.

Want to Use a Plug-in on Firefox? Ask For It

Mozilla announced this week that it would automatically disable all plug-ins in Firefox except the latest version of Adobe's Flash Player. In order for any plug-in to run, the user will have to manually override the block. This feature, which Mozilla calls “click-to-play,” used to bar only plug-ins that the Firefox browser judged to be unsafe or too far out of date. The move comes on the heels of numerous reports of hackers taking advantage of bugs in plug-ins, particularly the Java browser plug-in. The makers of other browsers such as Chrome and Opera include the click-to-play feature. But Mozilla is the first to turn it on by default. The others require the user to enable it.

Yahoo Mail Hijacking Case Solved

Security researchers at Australia-based BitDefender say they have gotten to the bottom of how some Yahoo Mail accounts have been hijacked over the past month. It seems that a link that is supposed to take them to an MSNBC News site, connects them with a domain registered in the Ukraine. Javascript that finds the user's contacts and sends spam under his or her name is placed on those pages so that its almost impossible not to click on it.

Bill Shocker Malware Spreading Like Wildfire in China

It was revealed this week that a new piece of malware dubbed “Bill Shocker” has infected at least 600 000 mobile devices in China. The malicious code, which targets several of the most popular mobile apps in China, including Tencent QQ Messenger and Sohu News, sends spam to the users’ contact lists—often costing mobile device users a lot of money by going beyond the number of messages included in the unsuspecting users’ messaging plans. In a 30 January blog post, Beijing- and Dallas-based NQ Mobile said that the malware can update itself and "automatically expand to other apps, multiplying the potentially disastrous effects.”

Photo: Jleon/Wikipedia