nathan buggia

Where's Nathan?

nbuggia — Sun, 07 Dec 2008 18:22:46 -0700

It has been quite a while since I've last updated this blog. It doesn't mean that I have stopped writing, just that I've been writing other places. For SEO, I will mostly continue to post articles on the Live Search Webmaster blog, for developer related content (and O'Reilly Found conference information) will continue to post on Jane and Robot, and finally for financial insight I'll be writing on Meridian Pacific Investments blog.

I've been really busy lately with Microsoft and the O'Reilly conference, but I still have a few additional side projects that I'll be introducing in the next couple months. Stay tuned. Here's a few of the articles I've published recently:

URL Referrer Tracking - overview of the various implementations available for tracking where your traffic is coming from in a search-friendly way.
SMX East: AJAX, CSS and Web 2.0 - a strategic look at implementation options for building search-friendly Web 2.0 applications.
SMX East: Unraveling URLs and Demystifying Domains - top 7 issues we at Live Search see with URLs and how to avoid them
SMX East: Webmaster Guidelines - overview of the search engine guidelines for indexing and background information on why they exist.

I will continue to update this blog as time permits, but I don't write very much that isn't finance or SEO related these days. Hopefully as I dig deeper into some of my other projects I'll have something useful to write.

Web 2.0 Expo: Advanced SEO for Developers

nbuggia — Wed, 17 Sep 2008 19:19:00 -0700

I would like to thank everyone for taking the time to check out my session at Web 2.0 Expo here in New York City. I've posted my presentation and answers to all the online questions on the Webmaster Center Blog at Live Search.

Import a CSV File into SQL Server

nbuggia — Wed, 03 Sep 2008 10:49:00 -0700

Metrics aggregation and reporting have always seemed to be a part of every job I've had. Over the years I've developed a system that allows me to slice and dice just about anything using Excel, SQL and a little bit of code. I used to rely heavily on the Data Transformation Services in SQL 2000 Enterprise Console, and haven't really found a good replacement (read: free replacement) until today. I just came across this little snippet of SQL that does the trick very well, here's what you do:

1. Create a new table in your database

Create a new table in your database, making sure each column data type is compatible with the corresponding column in your CSV file.

2. Properly format your input CSV file

What every data you want to suck in should be in a standard CSV file format as such. Save the file in a conspicuous location like c:\.

3. Run this script

Finally, execute the following script on your SQL Server. It should locate the CSV file, and import all the rows. Note, if it encounters an error on any single row, it will simply exclude that row in the resulting table. That could be a bit of a problem if you've got a lot of data.

This script seems to work in SQL Server 2005 and 2008. For more information, check out MSDN's reference material, there seem to be a lot more bells and whistles than I'm using here in this simple example. http://msdn.microsoft.com/en-us/library/ms188365.aspx

What Apple and Maslow Know About Your Customers

nbuggia — Sun, 20 Jul 2008 12:23:00 -0700

Yes, Microsoft is being trounced. But no, the Apple "marketing machine" not the cause, only a symptom. The real problem is that Apple has a deeper understanding of customers than Microsoft does, and are more efficient at turning that understanding into great products.

Microsoft's first issue that they still think a product’s worth is the sum of it’s discrete features, while Apple sees a product’s worth as the ability to solve a customer’s problem soup-to-nuts. Microsoft will include a feature even if it doesn’t really work (you know, MSFT gets it right on v3), but Apple will only include technologies that work well together out of the box. Think of iPod+iTunes vs. MSFT’s WinCE + “Plays for sure” program.

The second issue is that anyone can deliver the functionality these days (think Linux). Customers want more, they’ve gone up to the next level on Maslow’s fundamental needs:

Level 1 = functionality must work
Level 2 = it works and makes me feel good
Level 3 = works, I feel good and it is beautiful

Which level are your products in? And what about your competitors? Are you going to be an Apple or a Microsoft?

TechEd Developer 2008: Advanced SEO for Web Development

nbuggia — Thu, 05 Jun 2008 22:05:00 -0700

I would like to thank everyone for attending my session at this year's TechEd Developer conference in Orlando, FL. Below I've included a link to my slides, please feel free to contact me if you have any questions. This is very similar to the Mix08 Advanced SEO Presentation I gave a few months ago at Mix08, except I expanded the "Diagnosing SEO issues within your site" section. For a deeper dive on the subject, check out the 3 hour SEO for Web Development workshop Vanessa Fox and I presented on this topic at Web 2.0.

I also recommend you check out the Live Search Webmaster Center for information about how Live Search is crawling your site, news and discussion forums. Have I done enough shameless plugging yet?

Advanced-SEO-Web-Development-TechEd-2008.pptx (5.31 mb)

| View | Upload your own

Seattle (free) Networking Events for Developers

nbuggia — Fri, 09 May 2008 07:22:00 -0700

Are you in the local seattle area and interested in the technical side of search engine optimization? E.g. Implementation/ operational best practices, design patterns, site reviews, etc? Then you should come check out one of the upcoming events being hosted by Jane & Robot in May. Each event will have a couple 15 minute talks by a local expert, time for Q&A, and then a couple in depth site reviews. (not to mention free beer and snacks!)

Tuesday, May 13th @ 6pm

Solo Bar, 200 Roy Street, Seattle
Sponsored by Microsoft, so they’ll be providing lots of swag in addition to food and drinks. Vanessa Fox will be talking about how a search engine works, and someone from Microsoft will talk about ASP.Net and Silverlight best practices. Then we'll do the Q&A, site reviews and have general networking time.

Thursday, May 29th @ 6pm

Google Seattle office, 651 N 34th St. Seattle
Sponsored by Google, so expect a lot of primary-colored furniture. For this session we'll focus a bit more on diagnosing issues with your site, and look out for a few interesting guest speakers.

Startup Metrics 101 - Dave McClure's Talk at Web 2.0

nbuggia — Tue, 22 Apr 2008 08:24:00 -0700

Or better known as the "AARRR!" method for running a startup (Acquisition, Activation, Retention, Referral, Revenue). It turns out that this methodology is just as useful for established businesses as startups.

| View | Upload your own

An SEO Conference Just for Developers (Seattle)

nbuggia — Mon, 07 Apr 2008 11:32:00 -0700

Did you know that search engines drive nearly 30% of all page views to web sites? And that's just an average, the number can be much higher for smaller sites, and ecommerce sites. That's a lot of customers, possibly even more than the number of firefox users coming to your site. That begs the question - do you spend as much time testing your site for search as you do for FireFox?

Over the past year I've done several talks for web developers on SEO, covering how search engines work and some best practices, and I have been really surprised how low the awareness has been. Everyone seems interested, but there tend to be a lot of misconceptions about SEO (e.g. "I want to build my site for people not robots", or "SEO? No, I'm not a spammer"). Well, to help put those misconceptions aside, I've been working with Vanessa Fox at the SMX conference series to create a 1-day event on SEO for web developers. Here's the agenda:

Search Friendly Development
Highlights the most important elements to consider for search engine optimization (SEO) when building a web application infrastructure and provides tactical details about how to implement those elements. Topics include:

Developing a crawlable infrastructure

Considerations when developing rich internet applications (using technologies such as Flash, Silverlight, and AJAX)

URL rewriting, redirection, canonicalization, and visitor tracking

Platform Considerations for the Microsoft Stack and LAMP Stack
Practical tips, tricks, and workarounds for search-friendly architecture.

Microsoft Stack (IIS, ASP.Net, Silverlight)

LAMP Stack (Apache, PHP, Ruby, Flash/Flex)

CMS Platforms (e.g. BlogEngine.net, AxCMS, Wordpress, Movable Type, Drupal, Joomla)

Diagnosing Web Site Architecture Issues
Provides a checklist and workflow for diagnosing your web sites for SEO obstacles using freely available diagnostic tools.

Expert Technical Review of Your Website
This session will bring together experts who will use all of the information and tactics learned throughout the day and apply them to detailed site reviews of the code and infrastructure of sites submitted in advance by the audience.

Sign up here if you're interested in becoming a speaker at this event! Otherwise, if you're located in Seattle, you can register here.

We'll also be doing several free events leading up to this, stay tuned for more information on one of those as well.

Learn More About Your Customers

nbuggia — Mon, 10 Mar 2008 17:05:00 -0700

Marketing is all about building a deep understanding of your customers and using it to inform your product strategy, go-to-market strategy and advertising mix. A couple months ago, adCenter released a nearly-free tool that gives you access to much of the data Microsoft has stored internally about your web site, your customers and what they are looking for online. In this article I'll show you how to use this to further your own marketing efforts.

Download adCenter Keyword Research Excel Add-in (requires an adCenter account, which costs $5). To use the tool, download & install the add-in, open Excel 2007, and input your adCenter username & password in the Options section. This will get you 2,000 queries per day.
60-Trial of Excel 2007 (sorry, it isn't backwards compatible)

What do search engines think your page is about?

The first thing to look at are the Keyword Extraction and Keyword Suggestion features of the toolbar. Combined, these features help you understand what a search engine sees when it looks at your page. If that isn't exactly what you think will resonate with you customers, then you can use the keyword expansion tool to look for different types of synonyms that might resonate more effectively with your customers.

A classic example of this is with Microsoft's MSDN portal for developers. Most customers refer to "Visual Basic" as VB and that's what they type into a search engine. But Microsoft's branding guidelines state that we should stress the full name of the product. The end results is that customers have a difficult time finding authoritative content when searching for "VB".

The keyword suggestion tool has 3 modes of finding synonyms: Campaign Association - choose synonyms based on what other keywords advertisers are buying, Contained - similar words from content on the web and Similarity - based on similar words in our taxonomy.

To use the tool on your site, select a page on your web site, enter the URL in cell A1, then click on the Keyword Extraction button. This shows you the list of keywords that a search engine may find interesting on your web site. For this example, I've used Barack Obama's list of campaign issues.

The key question to ask yourself with these results is "Are these the words my customers will search for?". If not, use the keyword suggestion tool to see if you can find some better alternatives and include those in your page text.

Who are your customers?

Once you've taken a look at some of your pages and you feel like you've got a good sense of what keywords represent your site or your content, you can use the demographic tools to find out more information about the folks search for those terms.

The tool provides information on Gender, Age, and Geography. To use this, simply select one or more keywords in excel and click the appropriate button (Geography or Demographics). Tip: highlight the data and select Bar Chart from the Insert menu.

How many people are searching?

You can also use the tool to determine how frequently people are searching for your terms, and what other terms in a particular category are spiking over a specific period of time. The tool will also extrapolate the trend data several months into the future using a regression technique.

To get the trend data, simply select your keywords, and click on the Monthly Traffic button (note you can see daily or monthly traffic). The following is the monthly traffic for some of the current presidential candidates, with the most recent 3 months extrapolated. Sometimes analysts use this to infer trends within audiences or markets, I would be careful not to read too much into this and bet your whole business on it. But it can be an extremely useful when used in combination with other information about your customers or industry. (or just to show pretty charts to your boss)

One word of caution on the buzz index, the data is not filtered in the same way Google Trends is, so you may see some noise, like people searching for things like "politic". Some editorial staff use this data as one source in defining the topics they should write about to ensure they are producing the stories people are looking for.

The "buzz index" is associated within a particular category, not a specific keyword. To create a report, click on the Buzz Search button, and select a category you're interested in. Note that the numbers correspond to the actual number of times those keywords were searched for in Live Search & MSN Search.

So as you continue to increase your knowledge of your customer and market, you should spend the $5 and download the adCenter add-in for Excel. It provides a treasure trove of information about your business and your customers that should take you a little bit further in understanding them and thus building better products and marketing campaigns.

Mix08 Presentation - Advanced SEO for Developers

nbuggia — Wed, 05 Mar 2008 12:46:00 -0700

For the second year in a row, I was fortunate enough to be invited to give a talk on Advanced SEO for Developers at Mix08. I've uploaded my presentation below for all the folks that have requested it since the session. I also recommend that everyone check out our webmaster center for tools, news, seo information and answers to your questions.

| View | Upload your own

Are we in for a recession?

nbuggia — Thu, 08 Nov 2007 12:00:00 -0700

Google sure seems to think so...

UK Mix07 Presentations

nbuggia — Fri, 14 Sep 2007 17:17:00 -0700

I had the good fortune to be asked to come to the UK Mix07 event this year to do a session on Search Engine Optimization 101 for ASP.Net Developers. Due to a mixup (e.g. I can't read), I had also prepared another presentation on Using the Live Search Web Service to Create a Customer Site Search. Check them out and ping me with any questions:

Custom Site-Search Engine Using the Bing API

nbuggia — Thu, 13 Sep 2007 17:22:00 -0700

For Mix07 in London this week I created a sample site search to illustrate creating your own site search engine using the Live Search API. I've posted the sample code here, along with some notes on implementation (including a couple features I didn't have time to finish).

Here's a quick list of the features covered:

Spelling corrections
Web search results
Image search results
OpenSearch 1.1 (Putting your search in the browser searchbox)
Live Search Macros

Setting up your environment

Before we get to the code you'll need to provision yourself an Application ID to gain access to the web services, and then you'll also need to add a web reference to your visual studio project that references our WSDL: http://soap.search.msn.com/webservices.asmx?wsdl. Visual Studio will read the WSDL and use that to automatically generate several classes for rich objects that you can use instead of directly interacting with the SOAP API. Note, using these classes does not create any dependency on Visual Studio or your project AKA they only make your life easier.

Once you've added the reference, you can include the library in your project with use com.msn.search.soap

Define your custom search query

The first step is to use the live search query language to define the scope of your search. If you're building a site search there really is only one keyword you'll need, the "site:" operator. It scopes the search to just the domain/subdomain you specific, and you can join multiple together using the OR operator. Here's the query that we'll use in this application:

(site:msdn2.microsoft.com OR site:asp.net OR site:codeproject.com OR site:blogs.msdn.com)

Tip: including a lot of sites can dramatically slow down the query, so chose your sites carefully and consider omitting domains to simplify things (e.g. do this "site:asp.net" instead of this: "site:ajax.asp.net OR www.asp.net")

Executing the query

With each request, you can query any or all of the supported data sources: web, image, spelling, phonebook (directory) and news. The query must be the same for all sources, however, there are a few parameters you can adjust within each data source you request data from. This allows you to build a rich SERP with a single request to Live Search, and a single mark against your quota for the day.

   1:  // used to index the query request and the query results

   2:  protected int SR_SPELLING = 0;

   3:  protected int SR_IMAGE = 1;

   4:  protected int SR_WEB = 2;

5:

   6:  protected SearchResponse runQuery(string query)

   7:  {

   8:      MSNSearchService s = new MSNSearchService();

   9:      SearchRequest searchRequest = new SearchRequest();

10:

  11:      SourceRequest[] sr = new SourceRequest[3];

12:

  13:      // SPELLING SUGGESTION QUERY

  14:      sr[SR_SPELLING] = new SourceRequest();

  15:      sr[SR_SPELLING].Source = SourceType.Spelling;

  16:      sr[SR_SPELLING].ResultFields = ResultFieldMask.Title;

17:

  18:      // IMAGE SEARCH QUERY

  19:      sr[SR_IMAGE] = new SourceRequest();

  20:      sr[SR_IMAGE].Source = SourceType.Image;

  21:      sr[SR_IMAGE].ResultFields = ResultFieldMask.Description

  22:          | ResultFieldMask.Image | ResultFieldMask.Title;

  23:      sr[SR_IMAGE].Count = 5;

24:

  25:      // WEB SEARCH QUERY

  26:      sr[SR_WEB] = new SourceRequest();

  27:      sr[SR_WEB].Source = SourceType.Web;

  28:      sr[SR_WEB].ResultFields = ResultFieldMask.Description

  29:          | ResultFieldMask.DisplayUrl | ResultFieldMask.Title

  30:          | ResultFieldMask.Url;

  31:      sr[SR_WEB].Count = 10;

32:

  33:      searchRequest.Query = "futbol";

  34:      searchRequest.Requests = sr;

  35:      // You can provision your application id here: http://search.live.com/developer

  36:      searchRequest.AppID = "Your_ApplicationID_Goes_Here";

  37:      searchRequest.CultureInfo = "en-US";

38:

  39:      // this call executes the search

  40:      SearchResponse searchResponse = s.Search(searchRequest);

41:

  42:      return searchResponse;

  43:  }

Essentially you create a collection of SourceRequest objects, each one corresponding to a different data source within the Live Search Web service. You then create a SearchRequest object, give it the SourceRequests you configured, plus some information about the actual search itself. Finally, we grab the SearchResponse object, which will contain the search results for each data source from the Web service.

Parsing the query results

Because of the rich objects created by visual studio from the WSDL, parsing the search results is relatively trivial. I have removed the code that renders the data and just left the code required to get the values you probably care about out of SearchResponse object, but I encourage you to explore the object and see what other information is in there.

Let's get the results from the spelling suggestions first because it is the easiest. Note that you will only see a result here if Live Search thinks your query term may have been misspelled. Otherwise, it won't have any results.

   1:  // spelling suggestion

   2:  void renderSpellingSuggestion(SearchResponse searchResponse)

   3:  {

   4:      string spellingSuggestion = "";

5:

   6:      // Note that SR_SPELLING is simply an int defined to be 0 in this example, based on RunQuery()

   7:      //    method above.

   8:      if (!String.IsNullOrEmpty(searchResponse.Responses[SR_SPELLING].Results[0].Title))

   9:          spellingSuggestion = searchResponse.Responses[SR_SPELLING].Results[0].Title;

  10:  }

Next we'll parse the web results:

   1:  // web results

   2:  void renderWebResults(SearchResponse searchResponse)

   3:  {

   4:      // Note that SR_WEB is simply an int defined to be 2 in this example, based on RunQuery()

   5:      //    method above.

   6:      foreach (Result sourceResult in searchResponse.Responses[SR_WEB].Results)

   7:      {

   8:          string title, description, url;

9:

  10:          title = (!String.IsNullOrEmpty(sourceResult.Title)) ? sourceResult.Title : "";

  11:          description = (!String.IsNullOrEmpty(sourceResult.Description)) ? sourceResult.Description : "";

  12:          url = (!String.IsNullOrEmpty(sourceResult.Url)) ? sourceResult.Url : "";

  13:      }

  14:  }

And finally the image results, this is extra long because there are lots of interesting fields:

   1:  // image results

   2:  void renderImageResults(SearchResponse searchResponse)

   3:  {

   4:      // Note that SR_IMAGE is simply an int defined to be 1 in this example, based on RunQuery()

   5:      //    method above.

   6:      foreach (Result sourceResult in searchResponse.Responses[SR_IMAGE].Results)

   7:      {

   8:          string title, description, imageUrl, imagePageUrl, thumbnailUrl, thumbnailPageUrl;

   9:          int imageHeight, imageWidth, imageFileSize;

  10:          int thumbnailHeight, thumbnailWidth, thumbnailFileSize;

11:

  12:          title = (!String.IsNullOrEmpty(sourceResult.Title)) ? sourceResult.Title : "";

  13:          description = (!String.IsNullOrEmpty(sourceResult.Description)) ? sourceResult.Description : "";

14:

  15:          imagePageUrl = (!String.IsNullOrEmpty(sourceResult.Url)) ? sourceResult.Url : "";

  16:          imageUrl = (!String.IsNullOrEmpty(sourceResult.Image.ImageURL)) ? sourceResult.Image.ImageURL : "";

  17:          imageHeight = sourceResult.Image.ImageHeight;

  18:          imageWidth = sourceResult.Image.ImageWidth;

  19:          imageFileSize = sourceResult.Image.ImageFileSize;

20:

  21:          thumbnailPageUrl = (!String.IsNullOrEmpty(sourceResult.Url)) ? sourceResult.Url : "";

  22:          thumbnailUrl = (!String.IsNullOrEmpty(sourceResult.Image.ThumbnailURL)) ? sourceResult.Image.ThumbnailURL : "";

  23:          thumbnailHeight = sourceResult.Image.ThumbnailHeight;

  24:          thumbnailWidth = sourceResult.Image.ThumbnailWidth;

  25:          thumbnailFileSize = sourceResult.Image.ThumbnailFileSize;

  26:      }

  27:  }

Advanced Features

For more information on the advanced features like adding your search engine to the browser's search box, and deeper integration with Live.com, check out the power point presentation.

Files and resources:

Mortgage Planning Tool

nbuggia — Mon, 01 Jan 2007 13:48:00 -0700

Here's an easy modeling tool for use with standard 30 year, fixed-rate mortgages. You can use it to do "what if" analysis for different real estate properties you are considering, or evaluating the impact of making additional payments.

Download File: Mortgage Schedule Tool

Google Deprecates SOAP API in Favor of AJAX

nbuggia — Tue, 19 Dec 2006 13:19:00 -0700

A couple days ago Google quietly deprecated their SOAP API in favor of their newer, more fully featured AJAX API. Despite the rhetoric on the web, this is not a referendum on SOAP. Here's why I think they made the change:

They have a better understanding of their developers - Google originally released their API back in 2002, before they really had a deep understanding of any segmentation within the developer audience. Since then we've learned that there are many different types of developers on the web, from people who wrestle with HTML to JavaScript/DHTML/CSS hackers, to PHP/Ruby/PERL devs, to the ASP.Net/J2EE engineers. What Google has done isrecognize the specific needs of each group and released products optimized for each of them. For example the Google Custom Search Engine is built for the first group, and the AJAX API is built for the second two. The SOAP API is the righttechnology for the last group, but this group has been very slow to adopt web services like this. I predict that if Google believes there is an opportunity here, they will release an updated API specifically for this audience.
They want increased control over their services - With the SOAP API Google was giving developers direct access to their data and their service, relying on the Terms of Use, the developer's honesty and their legal staff to control how the service was used. The AJAX API and the Custom Search Engine use technology to restrict the scenarios the developer can implement and help enforce the Terms of Use.
They want to increase their business value - After 4 years of letting developers hack around with the SOAP API, Google now has a better understanding of how they can use it to drive business value, and it is all about site search and custom search. The advertising revenue pays for the program, the click-through data improves their search results, the advertising clicks increases the bidding cost of AdWords, and the branding keep consumers equating Google with search. And all this value lets them spend money to build better tools and services for developers, a win/win.

This is another example of how Google continues to out-innovate Microsoft, Yahoo and other top web companies. Nearly every one of their developer offerings has a clear business value for both Google and the developer, while their competition is still in the "we'll just put it out there an see what happens" phase.