oliverf

Google Verified Malware Free

noreply@blogger.com (Oliver Fisher) — Tue, 06 Apr 2010 22:36:00 +0000

I don't officially speak for Google, but as far as I know there is no such thing as a "Google Verified Malware Free" site (and you'd think that I would know). So it is surprising to see an official-looking logo on Evony's website. Or, maybe it isn't surprising, since is has been suggested that Evony is "the most despised game on the web" and has a murky history including suing those who talk about it.

Nexus One in Canada

noreply@blogger.com (Oliver Fisher) — Tue, 05 Jan 2010 21:49:00 +0000

I've been lucky enough to be using a Google Nexus One for the past few weeks (thanks to my employer). The setup steps are basically the same as I posted for the G1 - as are the limitations.

First, you'll need a Nexus One. Apparently Google's online store realizes you're in Canada and won't sell you one - but the resourceful will get one. I'm sure Google will eventually release the Nexus One to Canada - probably when the limitations below are resolved.

As with the original G1, the Nexus One will work on the Rogers/Fido network, but not at 3G speeds. You'll only get EDGE speed. For most things this is perfectly fine - but watching a YouTube video is painful. If you can live with that limitation, read on.

You'll also need a Rogers (or Fido) SIM card. Make sure you have a data plan (not a Blackberry plan). But, do NOT insert the SIM yet. Doing the initial setup over wifi will be easier - the Android setup doesn't let you configure APN info during setup (which is silly) and your SIM card may not set it correctly.

So, make sure you're in a wifi area. Put the battery in, but do not put in the SIM card. Turn on the phone and go through the setup. You'll be able to configure the wifi connection and setup the phone with your Google account.

Once the phone is setup, turn it off and put the SIM card in. Turn it back on. You should have phone access and data may work depending on the SIM card (turn off the wifi to check). If data does not work, you'll need to configure the APN info. See my previous post for details.

Enjoy! It's a great phone (but, yes, I'm sure it's not perfect... yet).

Update: Corrected the mention of Bell/Telus. Because of the way they've built their 3G networks (as an overlay on their CDMA network) they don't have fallback capabilities that the Nexus One will support.

Google Online Security Blog

noreply@blogger.com (Oliver Fisher) — Fri, 30 Oct 2009 12:30:00 +0000

If you're a reader of this blog, you're probably already a reader of the Google Online Security blog. I just posted there: http://googleonlinesecurity.blogspot.com/2009/10/do-machines-dream-of-electric-malware.html

Sadly, I'm told by a user in China that all of blogspot.com has been blocked by the Great Firewall of China so no one there can (easily) see it. Instead, they'll get the error below. Of course, they also won't see this post...

Toronto Internation Film Festival

noreply@blogger.com (Oliver Fisher) — Fri, 11 Sep 2009 22:08:00 +0000

Once again this year, Kristi and I are off to the Toronto International Film Festival (TIFF). We won't be hanging with any of the stars, but we will be watching a lot of films. Here's our schedule.

We've got tickets to 17 films right now and the 1st and 2nd choice calendars are things we'll be aiming to get tickets to. So far I'm most excited about:

Accident (Yi Ngoi) - hopefully an awesome Hong Kong crime drama

Whip It! - Drew Barrymore's directorial debut with Ellen Page staring as a misfit who turns to roller derby. This can't possibly be bad!

A Serious Man - the Coen brothers go back to Minnesota in 1967 and hilarity ensues. We don't have tickets yet, but hopefully we'll snag some at the last minute.

Diagnosing the Diagnostics: Domain Summary

noreply@blogger.com (Oliver Fisher) — Tue, 21 Jul 2009 16:18:00 +0000

This is the second part of a series about Google's Safebrowsing Diagnostics page.

The first section of the diagnostics page (What is the current listing status for my site?) displays a summary of the status of the domain. It indicates whether the domain is currently listed in Google's Safebrowsing malware list. It may also list the number of times in the recent past that the domain has been added to and removed from the malware list.

Both these simple facts can be somewhat surprising...

A user or webmaster may see a malware warning in their browser when visiting a page. But the diagnostic page for that site may state that the domain is not currently on the malware list. This is usually a Cross-Site Warning. The domain is almost certainly infected with malware because users' browsers blocked requests to other malicious domains.

Webmasters may also find that their site has been added to the malware list numerous times. This may be an indication that the webmaster is cleaning up the infection on the server without closing all vulnerabilities. When Google's scanners process the malware review, the site is temporarily clean, so it is removed from the malware list. But because the server is still vulnerable, malicious content may be re-injected within hours. Google's malware scanners will quickly detect this and put the site back on the malware list.

Finally, for very new infections, the diagnostics page may be temporarily out of date. Google immediately flags sites found to be malicious and sends email to the webmasters. An alert webmaster who quickly notices that their site has been flagged may find that the diagnostics page lists their site as clean or even completely unscanned. The diagnostics page will update shortly - usually within a couple of hours.

The next post will deal with the most important section of Google's Safebrowsing Diagnostics page: What happened when Google visited this site?

Diagnosing the Diagnostics series:

Background
Domain Summary
More to come...

Diagnosing the Diagnostics: Background

noreply@blogger.com (Oliver Fisher) — Thu, 16 Jul 2009 21:58:00 +0000

This is the first part of a series about Google's Safebrowsing Diagnostics page.

Google's Safebrowsing Diagnostics pages are a valuable source of information for webmasters. Unfortunately the information can sometimes be difficult to understand. Hopefully the next few posts will help clarify - I've been struggling with them for the past couple of months since they're fairly complex. Bear with me and jump in with any questions you may have.

Before we get to the diagnostics pages, we'll need to understand how malware works. Malware distributors have created sophisticated infrastructures. Typically, they use three types of servers and domains:

Distribution Servers: Malware distributors usually have a set of servers that they control and use as a base of operations. The servers host malware tools and exploits and are usually on a network where they are unlikely to be shut down. They often don't have domain names associated with them, only IP addresses.
Compromised Domain: Malware authors compromise legitimate domains and insert malicious scripts. The owner of the website usually doesn't realize this has happened. Most of the entries in Google's Safebrowsing malware list are these sorts of servers that have unintentionally become dangerous.
Intermediary Domains: Between the distribution servers and the compromised domains there are often one or more intermediary domains. Malware authors establish domains to obfuscate their malicious code. For instance, they may use a common domain name with a slight misspelling, like gooqleanalytics (notice the Q), hoping that webmasters won't spot the typo.

The structure of Google's Safebrowsing Diagnostics pages mirrors these types of servers and domains. Most flagged domains will only be one type of server eg. compromised domains usually aren't also distribution servers.

With that background, you may already be able to understand the diagnostics pages a bit better. In future posts we'll dive into the details of each section of the diagnostics page.

Diagnosing the Diagnostics series:

Background
Domain Summary
More to come...

Suspicious and Really Suspicious

noreply@blogger.com (Oliver Fisher) — Thu, 29 Jan 2009 16:41:00 +0000

Google's Safebrowsing Diagnostic page lists "the last time suspicious content was found on this site". But what does "suspicious" mean?

Google's automated malware scanners have been highly accurate with an astonishingly low false-positive rate. Part of that success has been because their definition of "suspicious" actually means "has nasty malware". If the scanners aren't really sure that a site has malware, they won't add it to the malware list. And that's the definition of "suspicious" ("has nasty malware") that Google's Safebrowsing Diagnostic page uses - content bad enough to get a site added to the malware list.

When the scanners do a review of a site to check if it should be removed from the malware list, they use a more stringent definition of "suspicious". If there's any suspicious activity at all then the site will not be removed from the malware list. Often sites have been infected with malware in multiple ways and the scanners need to be sure that it has been thoroughly cleaned up.

Those different definitions of "suspicious" may cause confusion when looking at Google's Safebrowsing Diagnostic page for a site that has been reviewed. The review may have found "suspicious" content that was not "suspicious" enough to have added the site to the malware list - but it is "suspicious" enough to prevent it being removed from the list. Google's Safebrowsing Diagnostic page won't list the date of that review scan.

If you're looking for the status of a malware review, log into Google's Webmaster Tools - the same place you reqested the malware review. It will show whether the review succeeded and will list urls that were still found to be "suspicious".

Cross-site Warnings

noreply@blogger.com (Oliver Fisher) — Fri, 23 Jan 2009 22:00:00 +0000

Several browsers use data from Google's malware list to protect users. Firefox 3, Chrome and Safari all check sites that users are visiting against Google's list and warn users if they are about to visit a dangerous site. There are some small differences in implementation across browsers that can cause confusion.

All three browsers check the address of the top-level page a user is navigating to. That protects most users in most cases. But, a web page can include content from another web page and if the included content is malicious then users may be exposed. Chrome (and Safari*) check every request against Google's malware list. This means those browsers will protect users even if malicious content from a flagged page is embedded on a non-flagged page.

Although that approach provides better protection for users, it may be confusing for webmasters if content on their site comes from another site. Some users (those with Chrome or Safari) will get warnings even though the webmaster's site is not blacklisted. Because the webmaster's site isn't blacklisted, they won't be able to request a malware review via Google's Webmaster Tools. Fortunately, this situation usually doesn't exist for very long. Google's scanners have already identified the embedded content as malicious but they haven't yet flagged the webmaster's site that includes the dangerous content. As they continue to crawl the internet, the scanners will quickly flag the webmaster's site.

If you're a webmaster in this situation, you'll need to examine all the content you're including from other websites. Look carefully at the warning page that browsers display since it usually includes the name of the domain that caused the problem.

* I can't say for certain exactly how Safari behaves because I haven't seen the code. But based on observation, Safari seems to have adopted the approach of checking every request.

Updated: FF3.5 checks every request against the blacklist and helps better protect users. FF3.5, Chrome and Safari all behave the same now.

Kick Them All Out!

noreply@blogger.com (Oliver Fisher) — Mon, 03 Nov 2008 23:08:00 +0000

On the Google Webmaster Central blog, Ben asked why sites with malware aren't removed from search results.

Users often use Google to search for sites they already know about. Urls are hard to remember but users can remember a couple of search terms and use those instead - like "barak obama" or "john mccain". If a user typed either of those queries, they'd expect the candidate's official site to rank near the top. But malware can strike any website: big or small; Democrat or Republican; corporation, government or private citizen. If Google removed the candidate's website from search results, there would be a lot of confused users. (1)

By leaving the website in the search results and placing a warning on it, users can find what they are looking for but know that visiting the website may be dangerous.

(1) I am not suggesting that either candidate's website has ever had malware. It's just an example.

Update: Yes, yes, I misspelled "Barack". Fortunately, Google still finds the right site: http://www.google.com/search?q=barak+obama

Information

noreply@blogger.com (Oliver Fisher) — Sun, 02 Nov 2008 15:00:00 +0000

Webmasters often ask why Google can't give more information about malware on websites. A couple of factors come into play.

Google's automated scanners only see the end result of the infection. They have no way of knowing how or exactly when the malicious content was added to a website. There are many ways it could have happened, from sql injection to stolen passwords. The automated scanners just know the bad content is there now.

To some extent, the success of Google's automated scanners relies on the secrecy of the methods used. The team has published malware papers (like Ghost in the Browser [pdf]) that include overviews of the system. But we're uncomfortable publishing any more detail because malware authors can read that information too. We've observed malware trying to hide from our automated systems and a large part of my job is tweaking the system to adapt to new types of malware. Staying on top of what malware is doing isn't easy and providing exact descriptions of the scanners and what they detect would make it impossible.

Earlier this year Google released the Safe Browsing diagnostic page which list general information about what Google's automated scanners found on a website. Some webmasters have found that information helpful in cleaning up their site. Hopefully we can find other ways to share information with webmasters without compromising the success of the scanning system. If you've got specific ideas, please let me know!

Bowling

noreply@blogger.com (Oliver Fisher) — Sun, 26 Oct 2008 20:58:00 +0000

On Friday we went bowling for a friend's birthday. I hadn't been bowling in about 15 years. Several things were surprising. First, Google had the address wrong - it's 1205 Wellington, not 1025 - I've filed a bug. Second, the shoes were both comfortable and stylish - they had an embroidered "Rental" on the side - I almost stole them.

But the automatic scoring system fascinated me most. It detected which pins were down and calculated scores - players didn't need to do anything. Each lane had their own computer screen. You could manually override incorrect score - of course, none of us knew how to score anyway. Here's a blurry photo.

It's an old system - probably programmed by one or two people 20 years ago. It must be cool for them to know that their software is still chugging along. It was a good system - as players were added the height of the rows changed. There were awesome multi-coloured ASCII art animations when someone got a strike, spare or gutter-ball.

The system was from Mendes, a now defunct Quebec company. They built the software and the pin mechanisms. YouTube has a very cool video of the pin mechanism.

For the observant, yes, that's 5-pin bowling - a truly Canadian game.

Google Blogs

noreply@blogger.com (Oliver Fisher) — Fri, 24 Oct 2008 22:01:00 +0000

I've just published a post about malware on Google's Online Security Blog and cross-posted to Google's Webmaster Central Blog. Avid readers (all 5 of you) will recognize the content as being adapted from a few earlier posts on this blog: "This site may harm your computer", Advance Warning, and How long does a review take?

A new point in that post is about Google's new notifications to webmasters with hackable servers. The best type of malware warning is one you get before you've been infected! Hopefully webmasters will find those notification useful and the team building them will be able to expand the program.

Android G1 Phone in Canada on Rogers

noreply@blogger.com (Oliver Fisher) — Wed, 22 Oct 2008 16:07:00 +0000

Update: Also see my recent Nexus One in Canada post.

Getting the G1 working in Canada on Rogers' GSM network isn't hard, but you won't be able to use a true 3G connection - you'll only get an EDGE connection. For me, that's good enough and I love the phone. I've never owned an iPhone so I'll leave the iPhone vs G1 debate to others.

You'll need an unlocked phone. Apparently T-Mobile will unlock any phone for any customer who has been with T-Mobile at least 3 months. Of course, someone will write an unlocking program so new customers don't have to wait 3 months. You may also be able to buy an unlocked G1 from non-T-Mobile sources.

Put a Rogers SIM card into the G1 phone. You should have an iPhone/SmartPhone data plan with Rogers - the Blackberry plan won't work (at least not with these instructions). When I called to switch my plan I said that I had an iPhone - no point in confusing the operator. With the SIM in place you should have voice connectivity right away.

To get data working, go to Settings | Wireless controls | Mobile networks | Access Point Names. You'll see a long list of T-Mobile networks. Press the menu button, select New APN, then modify these values:

Name: Rogers
APN: internet.com
Username: wapuser1
Password: wap
MCC: 302 - should be preset correctly
MNC: 72 - should be preset correctly

Don't forget to press menu and Save. If you're in an EDGE coverage area, an EDGE icon should appear in the status bar next to the signal strength indicator. Mine appeared within a couple of seconds.

Browsing, email, video and SMS works, but I haven't been able to get MMS working. It may be that my Rogers plan isn't configured for MMS. I think the below settings should work - if you find they do work or what to change to make them work, please let me know. Create a New APN and set these values:

Name: Rogers MMS
APN: media.com
Username: media
Password: mda01
Server: 172.25.0.107 - or maybe this should be * - I've tried both
MMSC: http://mms.gprs.rogers.com
APN type: mms

EDGE is as good as it gets in Canada with the G1 hardware. The G1 has quad-band (850/900/1800/1900Mhz) GSM/GPRS/EDGE, but only dual band (1700/2100Mhz) UMTS/HSPA (3G). Rogers operates their network on 850/1900Mhz, so the G1 hardware can only use EDGE. Bell and Telus are building "4G" networks but they'll also operate on 850/1900Mhz. So, I don't recommend buying a G1 for use in Canada if you're a heavy data user. My unlocked handset came from my employer and replaced my EDGE Blackberry, so I'm very happy. Hopefully there will be other Android hardware released soon that will be compatible with 850/1900Mhz 3G in Canada.

I discovered a good Rogers developer document [pdf]. It's targeted at developers building applications for the Rogers network, but lists everything needed to get non-Rogers hardware connected. The first few pages are also a good overview of GSM/GPRS/EDGE, UMTS/HSPA, 3G, etc.

Update: To clarify, an EDGE connection is usually slower than a 3G connection. I say "usually" because it depends on a ton of network factors.

Update: If you have a G1 with a T-Mobile plan, you can bring it to Canada and it will just work, including data. You'll be roaming on Rogers network and you'll be charged whatever T-Mobile's roaming rates are, but no additional configuration is needed.

Update: Make sure to check out the comments for lots of other settings that work, including on some other Canadian carriers.

How long does a review take?

noreply@blogger.com (Oliver Fisher) — Fri, 17 Oct 2008 12:20:00 +0000

Webmasters are eager to have a Google malware label removed from their site and often ask how long a review of the site will take. Giving an exact answer is a bit difficult. Both the original scanning and the review process are fully automated. The systems analyze large portions of the internet, which is big place. And, there are other webmasters also requesting reviews. In short, there are a lot of variables that factor into how long any particular review will take. If you're lucky, the label will be removed within a few hours. At its longest, the process should take a day or so.

You can request a review via Google's Webmaster Tools and you can see the status of the review there. If you think the review is taking too long, make sure to check the status. Finding all the malware on a site is difficult and the automated scanners are far more accurate than humans. The scanners may have found something you've missed and the review may have failed. If your site has a malware label, Google's Webmaster Tools will also list some sample urls that have problems. This is not a full list of all of the problem urls (because that's often very, very long), but it should get you started.

Finally, don't confuse a malware review with a request for reconsideration. If Google's automated scanners find malware on your website, the site will usually not be removed from search results. There is also a different process that removes spammy websites from Google search results. If that's happened and you disagree with Google, you should submit a reconsideration request. But if your site has a malware label, that won't do much good - for malware you need to file a malware review from the Overview page:

If you're still having problems, I recommend stopbadware.org and their forums. There's tons of information that can help you clean things up and even some very helpful volunteers who will answer questions (sometimes I'm one of them). The Google Webmaster Central blog also has a quick security checklist for webmasters.

Advance Warning

noreply@blogger.com (Oliver Fisher) — Thu, 16 Oct 2008 13:14:00 +0000

I often hear webmasters asking Google for advance warning before a malware label is put on their website. When the label is applied, Google usually sends email the website owners and then posts a warning in Google's Webmaster Tools. But no warning is given ahead of time, before the label is applied. I can understand why that would be helpful - the webmaster may be able to quickly clean up the site without any drop in web traffic.

But, look at the situation from the user's point of view. As a user, I'd be pretty annoyed if Google sent me to a site it knew was dangerous and I had to wipe my computer or had my banking information stolen because of malware. Even a short delay would expose some users to that risk and it doesn't seem justified. I know it's frustrating for a webmaster to see Google traffic drop when a malware label is applied. But, ultimately, the webmaster is responsible for all the content on their website. And one frustrated webmaster seems better than hundreds or thousands of frustrated users with malware infections.

"This site may harm your computer"

noreply@blogger.com (Oliver Fisher) — Wed, 15 Oct 2008 12:30:00 +0000

You may have seen those words in Google search results - but what do they mean? If you click the search result link you get another warning page instead of the website you were expecting. But if the web page was your grandmother's baking blog, you're still confused. Surely your grandmother hasn't been secretly honing her l33t computer hacking skills at night school. Google must have made a mistake and your grandmother's web page is just fine...

I work with the team that helps put the warning in Google's search results so let me try to explain. The good news is that your grandmother is still kind and loves turtles. She isn't trying to start a botnet or steal credit card numbers. The bad news is that her website or the server that it runs on has a security vulnerability, probably from some out-of-date software. That vulnerability has been exploited and malicious code has been added to your grandmother's website. It's most likely an invisible script or iframe that pulls content from another website which tries to attack any computer that views the page. If the attack succeeds viruses, spyware, key loggers, botnets and other nasty stuff will get installed. At best you'll waste hours cleaning up you computer; at worst you'll lose all your data and have money stolen from your bank account.

If you see the warning on a site in Google's search results, it's best to pay attention to it. Google has automatic scanners that are constantly looking for these sorts of web pages. I help build the scanners and can say that they're astonishingly accurate. There is almost certainly something wrong with the website even if it is run by someone you trust, like your grandmother.

Servers are just like your home computer and need constant updating. There are lots of tools that make building a website really easy, but each one adds some risk of being exploited. Even if you're diligent and keep all your website components updated, your web host may not be. They control your website's server and may not have installed the most recent OS patches. And it's not just innocent grandmothers that this happens to. There have been warnings on banks, sports teams, and corporate and government websites.

If your website has been struck by malware, there are some resources to help you clean it up. stopbadware.org has some great information and their forums have a number of helpful and knowledgeable volunteers who may be able to help. You can also use Google SafeBrowsing diagnotics page for your site (http://www.google.com/safebrowsing/diagnostic?site=<insert-site-name-here>) to see specific information about what Google's automatic scanner have found. Once you've cleaned up your website, use Google's Webmaster Tools to request a review. The automatic systems will rescan your website and the warning will be removed if the malware is gone.

Best Breakfast and Brunch in Ottawa

noreply@blogger.com (Oliver Fisher) — Tue, 14 Oct 2008 17:31:00 +0000

Benny's Bistro - in the back of Le Boulanger Francais, which has the best croissants in Canada. I have a couple of friends who picked up a dozen to take back to Toronto with them. They ate them all on the four hour drive. That's a lot of butter.

Le Boulanger Francais also has a new location in the Glebe. Now we just need that bridge across the canal.

Who am I?

noreply@blogger.com (Oliver Fisher) — Mon, 13 Oct 2008 20:18:00 +0000

I wish I knew...

The answer most relevant to this blog is that my name is Oliver Fisher and I'm a Googler (a full-time employee of Google). I spend most of my time working on Google's Anti-Malware efforts. But this is a personal blog and has nothing to do with my employer. I'll probably talk mostly about things I know - like malware or maybe Canadian politics - but everything will be my opinion not Google's. My point of view may or may not agree with my employer's.

That was a slow and boring start to blog, but an important thing to say. Hopefully we'll all find future posts more interesting.