Yahoo Pipes Podcast by Fast Wonder

Keyword CSV Files and Searching: 2 Minute Yahoo Pipes Demo

dawn@fastwonder.com (Dawn Foster) — Fri, 01 May 2009 19:08:46 +0000

It’s been a while since I’ve posted a 2 minute Yahoo Pipes video, but I had an opportunity to do a guest post on the Yahoo Developer Network blog (yay!), and I thought that a 2 minute screencast would be the best way to demonstrate my pipe.

This 2 minute demo takes a series of keywords from a CSV file and uses Yahoo Pipes to run each keyword through various searches. This technique can be very powerful when you search multiple services, but I’ve simplified the process to make it easier to understand the fundamentals of the technique by running the list of keywords only through Twitter search.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (22 MB) to watch. The quality will be much better than the flash version above.

Here’s a quote about the technical details of the pipe from the Yahoo Developer Network post:

Technical Details and More Information:

The Demo Pipe. A copy of the Keyword CSV demo pipe - click “View Source” to see the modules.

Fetch CSV Module. Enter the URL of the CSV file (make sure that it already exists in a publicly accessible location) along with information about column names and separation characters as needed.

Loop module with URL Builder. Loops through each element in the CSV file and builds a search URL formatted for RSS output. In this case, we used Twitter search.

Loop module with Fetch Feed. Loops through each URL built in the previous step and fetches the feed associated with the item.

Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.

Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training course on May 7th.

Related Fast Wonder Blog posts

Sharing is good

Accepting Google Docs Input: 2 Minute Yahoo Pipes Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 23 Feb 2009 20:30:57 +0000

This video shows how to use data from Google Docs as an input into Yahoo Pipes. In a previous video, I showed how to use a list of items in a CSV file. If you haven’t already watched the CSV input demo, you should start with it, since this pipe builds on the previous one. A CSV file is a great choice when you want to fetch a bunch of feeds and change them frequently without having to update your pipe, but a Google Doc is a great choice for when you want to easily update inputs to the pipe without needing to update a file on a webserver.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (20 MB) to watch. The quality will be much better than the flash version above.

More Details

The Demo Pipe. A copy of the Using Google Docs Inputs in Yahoo Pipes Demo click “View Source” to see the modules.
Fetch CSV Module. Enter the URL of a Google Spreadsheet that has already been published as a web page and append &output=csv to the end of the URL. Here is the example we used in the video.
Loop module with Fetch Feed. Loops through each element in the CSV file and fetches the feed associated with the item.
Filter Module. Filter by a couple of keywords.
Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Sharing is good

Using Location Data: 2 Minute Yahoo Pipes Video Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 09 Feb 2009 12:22:21 +0000

Yahoo Pipes can make it easier to work with location data (latitude / longitude) across multiple RSS feeds, even when the original feeds have different ways of specifying the location data. In this demo, I’ll show you how to use the Location Extracter module to normalize the data and then filter on lat / long data to get results only within a certain area.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (24 MB) or clicking the full screen toggle icon in the player to watch. The quality will be much better than the flash version embedded above.

More Details

The Demo Pipe. A copy of the Using Location Data: 2 Minute Yahoo Pipes Demo pipe click “View Source” to see the modules.
Fetch Feed. In this example, we will use a feed from Shizzow containing information about places and a feed from the USGS with earthquake data. Both store their lat / long data in different ways within the RSS Feed.
Location extractor. This module is atypical, since it takes no parameters and has no user configurable elements. It simply looks for lat / long data and saves what it finds as y:location.lat and y:location.lon.
Sort. Sort by date (descending) to make sure that the output from your various source feeds appears in sorted order.
Filter. We then filter the source feeds to include only the entries located in the western part of the United States (25 < latitude < 50 and -130 < longitude < -115).
Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Sharing is good

YQL (Yahoo Query Language): 2 Minute Yahoo Pipes Video Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 02 Feb 2009 01:41:00 +0000

After learning that Yahoo Pipes introduced a new module called YQL (Yahoo Query Language) last week, I knew that I had to find a way to work it into one of my 2 minute videos. YQL uses an SQL-like syntax for more powerful and flexible inputs into Yahoo Pipes; however, working it into a 2 minute demo was quite challenging, since queries can be complicated. I cheated a little by glossing over the query language and focusing on how to use YQL effectively within Pipes. Honestly, I’m never going to teach someone to use an SQL syntax without extensive training, so this 2 minute demo is really geared toward people with database experience or programming backgrounds. I also did a more complicated version of this pipe with a more extensive explanation that you might be interested in reading to get more details about using YQL within Pipes.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (18 MB) or clicking the full screen toggle icon in the player to watch. The quality will be much better than the flash version embedded above.

More Details

The Demo Pipe. A copy of the YQL: 2 Minute Yahoo Pipes Demo pipe click “View Source” to see the modules.
YQL. The query:
select * from flickr.photos.info(20) where photo_id in (select id from flickr.photos.search(20) where tags = “igniteportland”).
Selects 20 photos from Flickr with the tag “igniteportland”. flickr.photos.search is used to find the images, but it doesn’t provide much information about the images, so I ran it through flickr.photos.info to pull in more data. If you omit the (20), YQL returns 10 results by default, and you can set this number to something appropriate for your needs.
Rename. Maps item.urls.url.content into the more standard item.link, which is recognized by RSS readers and the Yahoo Pipes output for clickable links back to the original content.
Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Sharing is good

Filter By Date: A 2 Minute Yahoo Pipes Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 26 Jan 2009 12:16:55 +0000

This 2 minute Yahoo Pipes demos shows how to filter by date using the filter module combined with the date builder module to find all of the posts in the past 2 days from our feeds. If you haven’t already watched the 2 Minute Yahoo Pipes Introductory Demo, I strongly suggest that you watch it. This demo builds on the basic structure developed in the introductory demo with the same feeds from ReadWriteWeb and GigaOM that we used in the original demo.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (15 MB) or clicking the full screen toggle icon in the player to watch. The quality will be much better than the flash version embedded above.

More Details

The Demo Pipe. A copy of the Filter By Date Pipe click “View Source” to see the modules.
Fetch Feed. Fetches the elements from 2 feeds: ReadWriteWeb and GigaOM.
Date Builder Module. Defines the date parameters for filters. In this case, I’ve filtered on ‘2 days ago’
Filter Module. Filter by blocking any item where date (item.y:published.utime) is greater than 2 days ago.
Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Sharing is good

Pipes within Pipes: A 2 Minute Yahoo Pipes Video Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 19 Jan 2009 12:18:18 +0000

We’ve talked about many different uses for Yahoo Pipes in previous 2 minute Yahoo Pipes demos, but you can also use most existing Yahoo Pipes within other pipes. In this demo, we’ll use a couple of pipes I’ve previously created as the input into a new Yahoo Pipe, and we’ll introduce the union operator as a way to pull several inputs together.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (19 MB) or clicking the full screen toggle icon in the player to watch. The quality will be much better than the flash version embedded above.

More Details

The Demo Pipe. A copy of the Pipes within Pipes demo pipe click “View Source” to see the modules.
Union Module. Pull several inputs together into a single output.
Unique Module. Use this to prevent duplicate results when multiple search queries might return the same results by filtering on item.link.
Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Sharing is good

Using PostRank: 2 Minute Yahoo Pipes Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 12 Jan 2009 15:35:25 +0000

Yahoo Pipes can also be used with other services, like PostRank, for example. PostRank, originally called AideRSS, is a service that rates the posts on any blog by popularity using comments, links, bookmarks, tweets, and other measurements to give each post a rank from 1 - 10. Luckily for us, PostRank conveniently stores the rank right in the RSS feed allowing us to access it from Yahoo Pipes. In our last Yahoo Pipes demo, I showed you how to Modify RSS Feeds to work better for your purposes. I recommend watching the 2 minute demo about modifying RSS feeds if you haven’t already, since we will use those concepts again in this pipe.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (25 MB) or clicking the full screen toggle icon in the player to watch. The quality will be much better than the flash version embedded above.

More Details

The Demo Pipe. A copy of the Using PostRank demo pipe - click “View Source” to see the modules.
PostRank. Using the 2 feeds that we’ve been using for most of these demos, ReadWriteWeb and GigaOM, we analyze each of them using PostRank, filter by the best posts, and copy the RSS feed for the output.
Fetch Feed. Fetches the elements from 2 PostRank feeds for ReadWriteWeb and GigaOM from the previous step.
Sort Module. Sort by PostRank in descending order to keep the posts with the highest rank at the top.
Loop module with String Builder Module. Loops through each element in the feed and builds a string with item.postrank:postrank: item.title. This string is assigned back into item.title.
Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Sharing is good

Modifying RSS Feeds: 2 Minute Yahoo Pipes Video Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 05 Jan 2009 11:12:26 +0000

While many people use Yahoo Pipes to filter RSS feeds, Pipes can also be used to modify RSS feeds to work better for your purposes. In this example, we will add the author name to the beginning of the title to make it easier to see the author without opening the item in your rss reader. I recommend watching the Introduction to Yahoo Pipes: 2 minute Yahoo Pipes Video Demo if you haven’t already, since we’re going use the basic pipe developed in that demo using fetch feed and sort modules, but without the filter module.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

More Details

The Demo Pipe. A copy of the Modifying RSS Feeds demo pipe click “View Source” to see the modules.
Fetch Feed. Fetches the elements from 2 feeds: ReadWriteWeb and GigaOM.
Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
Loop module with String Builder Module. Loops through each element in the feed and builds a string with item.author: item.title. This string is assigned back into item.title.
Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Sharing is good

CSV File Inputs: 2 Minute Yahoo Pipes Video Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 29 Dec 2008 21:54:36 +0000

This video shows how to get input from a list of items in a CSV file, and it introduces the loop module. A CSV file is a great choice when you want to fetch a bunch of feeds and change them frequently without having to update your pipe. If you haven’t already watched the 2 Minute Yahoo Pipes Introductory Demo, I strongly suggest that you watch it. This demo builds on the basic structure developed in the introductory demo. The CSV file I’m using also contains the same feeds from ReadWriteWeb and GigaOM that we used in the original demo plus a few others.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (22 MB) to watch. The quality will be much better than the flash version above.

More Details

The Demo Pipe. A copy of the CSV File Input in Yahoo Pipes Demo click “View Source” to see the modules.
Fetch CSV Module. Enter the URL of the CSV file (make sure that it already exists in a publicly accessible location) along with information about column names and separation characters as needed.
Loop module with Fetch Feed. Loops through each element in the CSV file and fetches the feed associated with the item.
Filter Module. Filter by a couple of keywords.
Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Sharing is good

Using The Debugger: 2 Minute Yahoo Pipes Video Demo

dawn@fastwonder.com (Dawn Foster) — Mon, 15 Dec 2008 17:04:16 +0000

The debugger serves a couple of functions in Yahoo Pipes. The first one is pretty obvious - debugging. It allows you to see exactly what is happening in each component of your Yahoo Pipe to help with the debugging process when something isn’t working quite right.

The debugger also serves a less obvious purpose. You can use it to learn more about the elements of the rss feeds that you are using as input. For most blogs, the rss feeds are fairly standard, but other services store all kinds of useful information in their rss feeds in addition to standard elements like link, title, and date. The debugger can help you find this additional information in the feed.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

Using The Debugger: 2 Minute Yahoo Pipes Video Demo

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (19 MB) to watch. The quality will be much better than the flash version above.

More Details

The Demo Pipe. A copy of the Yahoo Pipes Debugger Demo Pipe. Click “View Source” to see the modules.
Fetch Feed Module. Contains 1 feed from Shizzow, which contains a wealth of location information within the feed.
Filter Module. Filters posts with a certain keyword out of the feed to demonstrate the debugger.
Pipe Output. The final module in every Yahoo Pipe.
The demo also spends time using the debugger to explore various elements of the feed.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Previous videos in the series:

Sharing is good