<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" media="screen" href="/~d/styles/rss2full.xsl"?><?xml-stylesheet type="text/css" media="screen" href="http://feeds.feedburner.com/~d/styles/itemcontent.css"?><rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0" version="2.0">

<channel>
	<title>Nagesh Susarla's Blog</title>
	
	<link>http://nagiworld.net</link>
	<description>Random thoughts on the state of my world</description>
	<pubDate>Thu, 04 Mar 2010 08:06:19 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.6.2</generator>
	<language>en</language>
			<atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="self" type="application/rss+xml" href="http://feeds.feedburner.com/nageshsblog" /><feedburner:info uri="nageshsblog" /><atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="hub" href="http://pubsubhubbub.appspot.com/" /><item>
		<title>Keeping secrets safe with YQL Storage</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/9Wmf6pHCouQ/keeping-secrets-safe-with-yql-storage</link>
		<comments>http://nagiworld.net/2010/03/keeping-secrets-safe-with-yql-storage#comments</comments>
		<pubDate>Thu, 04 Mar 2010 08:06:19 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[tools]]></category>

		<category><![CDATA[yql]]></category>

		<category><![CDATA[yql storage]]></category>

		<guid isPermaLink="false">http://nagiworld.net/?p=190</guid>
		<description><![CDATA[
Just FYI, this is not a discussion on self-help to keep you from divulging your deep and dark secrets in real life. It&#8217;s about YQL and the various web secrets such as API keys, Consumer Keys and other key-secrets for personal use which you&#8217;d rather not give out to anyone visiting your web applications.
With that [...]]]></description>
			<content:encoded><![CDATA[<div><a title="DSCN1583 by cliff1066™, on Flickr" href="http://www.flickr.com/photos/nostri-imago/2844842106/"><img class="aligncenter" src="http://farm4.static.flickr.com/3071/2844842106_0bd9ac5672.jpg" alt="DSCN1583" width="500" height="375" /></a></div>
<p>Just FYI, this is not a discussion on self-help to keep you from divulging your deep and dark secrets in real life. It&#8217;s about <strong>YQL</strong> and the various web secrets such as API keys, Consumer Keys and other key-secrets for personal use which you&#8217;d rather not give out to anyone visiting your web applications.</p>
<p>With that disclaimer, let me jump right into the scenario that I&#8217;m talking about. There are quite a few YQL Open Data Tables such as <a href="https://developer.yahoo.com/yql/console/?q=desc%20netflix.catalog&amp;env=store://datatables.org/alltableswithkeys">Netflix Catalog</a>, <a href="http://developer.yahoo.com/yql/console/?q=desc%20nyt.article.search&amp;env=store://datatables.org/alltableswithkeys">NY Times Article search</a>, <a href="http://developer.yahoo.com/yql/console/?q=desc%20amazon.ecs&amp;env=store://datatables.org/alltableswithkeys">Amazon Product Advertising table</a> etc which require a user to register and enter an API key and/or Secret into the query. Issuing a query in <a title="YQL" href="http://developer.yahoo.com/yql/console" target="_blank">YQL Console</a> for testing purposes seems great. You can see the results and ensure that the query is working as you desire before adding it into your webpage/webapp.</p>
<p>This YQL query can then be run, either by issuing a HTTP GET request call from your website or by directly embedding it in your webpage as part of the Javascript. In the former case, since you own the website, there is no need to hide the Consumer-Key and Secret. In the later case, where the query is directly embedded in the client-side JS code running in the browser, you run the risk of exposing your ConsumerKey and Secret which can then to be used to run various other queries on your behalf. In this post, I show how one can hide the secrets from the user and also lock down the secrets to a single table.<br />
<span id="more-190"></span><br />
<strong>Brief Introduction to &#8216;<em>SET</em>&#8216;</strong></p>
<p>&#8216;SET&#8217; statements in YQL let you decouple the constant keys from the varying keys in the <em>where</em> clause of the YQL statement. For example if I wanted only the PDF search results for a given search term, I would issue:</p>
<pre class="brush: sql;">select * from search.web where query='yql' and type='pdf'</pre>
<p><a href="http://bit.ly/aibcJP" target="_blank">Try in console</a></p>
<p>Since the <em>type=&#8217;pdf&#8217;</em> expression will be a constant key in all of my queries, I can decouple it by issuing the following query</p>
<pre class="brush: sql;">SET type='pdf' on search.web;
select * from search.web where query='yql'</pre>
<p><a title="Try in Console" href="http://bit.ly/bbSXIf" target="_blank">Try in console</a></p>
<p>In essence, the YQL compiler applies all the &#8216;SET&#8217; clauses for a particular table to each compatible branch of the YQL statement and then executes the query.</p>
<p>Here is another example which shows the Amazon product listing api wrapped in YQL that requires your AWSAccessKeyId and the secret to be part of the query. To test this query I would use the entire query:</p>
<pre class="brush: sql;">use 'http://nagiworld.net/yqldefs/yql/amazon.ecs.xml' as amazon.prodlist;
set AWSAccessKeyId='YOUR_KEY' on amazon.prodlist;
set secret='YOUR_SECRET' on amazon.prodlist;
set AssociateTag='YOUR_ASSOCIATE_TAG' on amazon.prodlist;

select * from amazon.prodlist where Title = 'panasonic' and SearchIndex="Electronics" and ResponseGroup='Images,ItemAttributes,SalesRank' and ItemLinks.ItemLink.Description='All Offers'</pre>
<p><strong>Storing your secrets</strong></p>
<p>YQL provides a way to store data in <a href="http://developer.yahoo.net/blog/archives/2009/06/sherpa.html">Yahoo&#8217;s own Sherpa</a>. For a deep dive into storing data with YQL you can checkout the <a href="http://developer.yahoo.com/yql/guide/yql-cloud-chapter.html" target="_blank">Hosted Storage docs</a></p>
<p>Building on the &#8216;USE&#8217; and &#8216;SET&#8217; statements described earlier, we can easily decouple the query into two components. The first component has all the &#8216;SET&#8217; statements whereas the second component has your query. Now to hide all the secret bits in YQL, all I have to do is insert the &#8216;SET&#8217; statements into <em>yahoo.storage</em> with the following query</p>
<pre class="brush: sql;">insert into yql.storage.admin (value) values ("use 'http://nagiworld.net/yqldefs/yql/amazon.ecs.xml' as amazon.prodlist;
set AWSAccessKeyId='YOUR_KEY' on amazon.prodlist;
set secret='YOUR_SECRET' on amazon.prodlist;
set AssociateTag='YOUR_ASSOCIATE_TAG' on amazon.prodlist;")</pre>
<p>The response contains the various keys such as execute, select and update.</p>
<pre class="brush: xml;">&lt;results&gt;
&lt;inserted&gt;
&lt;execute&gt;store://h0L6f1ecl32llKITPZEbOg &lt;/execute&gt;
&lt;select&gt;store://xGnPqWdslS..sqvoOYObn &lt;/select&gt;
&lt;update&gt;store://tiJkslt0pE..QEecsQyQn5 &lt;/update&gt;
&lt;/inserted&gt;
&lt;/results&gt;</pre>
<p>Each element described in the response has specific properties.</p>
<div>
<ul>
<li>The execute key can only be used to execute a query. This implies that exposing an execute key does not expose the data stored within the entry.</li>
<li>The select key should be kept private since it can be used on the yql.storage table to view the contents of this particular entry.</li>
<li>The update key lets you update the contents that are seen via the select key.</li>
</ul>
</div>
<p>Given that we have the execute key, we can now provide the execute key to the query parameter called &#8216;env&#8217;. This tells the YQL Servers to pull the data from the store and prepend it to your query before actually running it. So in essence YQL does a read through execution of the query.</p>
<p>http://developer.yahoo.com/yql/console?env=store://h0L6f1ecl32llKITPZEbOg</p>
<p>Once you pass in the env, you can issue the following query:</p>
<pre class="brush: xml;">select * from amazon.prodlist where Title = 'panasonic' and SearchIndex="Electronics" and ResponseGroup='Images,ItemAttributes,SalesRank' and ItemLinks.ItemLink.Description='All Offers'</pre>
<p><a href="http://bit.ly/by8Qex" target="_blank">Try it in Console</a></p>
<p>As you can be seen, the above query does not expose any of the your secrets specifically the amazon id, secret and your associate tag. None of the diagnostics or the debug information leak your URLs which at this point could contain the various keys that you&#8217;d like to keep secret.</p>
<p>Selecting an execute key does not show the secrets you put into the query</p>
<pre class="brush: sql;">select * from yql.storage where name='store://h0L6f1ecl32llKITPZEbOg'</pre>
<p><a href="http://bit.ly/c1WEGo" target="_blank">Try in Console</a></p>
<p>As expected this returns the following and thus Mission Accomplished! Secrets safe</p>
<pre class="brush: xml;">&lt;results&gt;
&lt;error&gt;No select permission or doesn't exist&lt;/error&gt;
&lt;/results&gt;</pre>
<p><em>Picture licensed from </em><a rel="cc:attributionURL" href="http://www.flickr.com/photos/nostri-imago/"><em>nostri-imago</em></a><em> under Creative Commons</em></p>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/9Wmf6pHCouQ" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2010/03/keeping-secrets-safe-with-yql-storage/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2010/03/keeping-secrets-safe-with-yql-storage</feedburner:origLink></item>
		<item>
		<title>Shorten URL’s with Bitly using YQL</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/Yze1eGIBTPY/shorten-urls-with-bitly-using-yql</link>
		<comments>http://nagiworld.net/2009/06/shorten-urls-with-bitly-using-yql#comments</comments>
		<pubDate>Tue, 02 Jun 2009 06:36:34 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[tools]]></category>

		<category><![CDATA[yql]]></category>

		<guid isPermaLink="false">http://nagiworld.net/?p=122</guid>
		<description><![CDATA[Bitly is a great URL shortener which also provides a very neat API. If you&#8217;re as API driven as me and love to automate the tasks around you, you&#8217;ll be happy to note that there is a new open data table that can shorten URL&#8217;s using YQL.

The API is neat but why remember the query [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://bit.ly">Bitly</a> is a great URL shortener which also provides a <a href="http://code.google.com/p/bitly-api/wiki/ApiDocumentation">very neat API</a>. If you&#8217;re as API driven as me and love to automate the tasks around you, you&#8217;ll be happy to note that there is a new open data table that can shorten URL&#8217;s using YQL.<br />
<span id="more-122"></span><br />
The API is neat but why remember the query parameters of a service when YQL can do the work for you?</p>
<pre class="brush: sql;">use 'http://nagiworld.net/yqldefs/bit.ly.shorten.xml';
select * from bit.ly.shorten where login='nag...' and apiKey='....'
           and longUrl='http://cnn.com/'</pre>
<p>This looks pretty easy but seems like a lot of work for a single URL to be shortened. Isn&#8217;t it? You can always go to the <a href="http://bit.ly">bit.ly</a> site and do the same in a single click. right? Yes! Infact the reason for this table is to shorten multiple URLs in a single statement using a YQL sub-select i.e.</p>
<pre class="brush: sql;">select * from bit.ly.shorten where login='nag...' and apiKey='...'
       and longUrl in ('http://cnn.com', 'http://yahoo.com')</pre>
<p>The output of the query looks like the following. Only the <em>results</em> element is shown here for brevity. Given this data, I can project the necessary fields and get either the shortened url or other elements.</p>
<pre class="brush: xml;">
    &lt;results&gt;
        &lt;bitly&gt;
            &lt;errorCode&gt;0&lt;/errorCode&gt;
            &lt;errorMessage/&gt;
            &lt;results&gt;
                &lt;nodeKeyVal&gt;
                    &lt;userHash&gt;xzV1m&lt;/userHash&gt;
                    &lt;shortKeywordUrl/&gt;
                    &lt;hash&gt;31IqMl&lt;/hash&gt;
                    &lt;nodeKey&gt;&lt;![CDATA[http://cnn.com/]]&gt;&lt;/nodeKey&gt;
                    &lt;shortUrl&gt;http://bit.ly/xzV1m&lt;/shortUrl&gt;
                &lt;/nodeKeyVal&gt;
            &lt;/results&gt;
            &lt;statusCode&gt;OK&lt;/statusCode&gt;
        &lt;/bitly&gt;
        &lt;bitly&gt;
            &lt;errorCode&gt;0&lt;/errorCode&gt;
            &lt;errorMessage/&gt;
            &lt;results&gt;
                &lt;nodeKeyVal&gt;
                    &lt;userHash&gt;h4H0n&lt;/userHash&gt;
                    &lt;shortKeywordUrl/&gt;
                    &lt;hash&gt;2deaFR&lt;/hash&gt;
                    &lt;nodeKey&gt;&lt;![CDATA[http://yahoo.com]]&gt;&lt;/nodeKey&gt;
                    &lt;shortUrl&gt;http://bit.ly/h4H0n&lt;/shortUrl&gt;
                &lt;/nodeKeyVal&gt;
            &lt;/results&gt;
            &lt;statusCode&gt;OK&lt;/statusCode&gt;
        &lt;/bitly&gt;
    &lt;/results&gt;
</pre>
<p>Better yet, take an entire html page, rss/atom feed or even a CSV file as an input containing all the URL&#8217;s to be shortened and zip it through YQL in a single statement. Now, Insn&#8217;t that something cool!</p>
<pre class="brush: sql;">select * from bit.ly.shorten where login = 'nag..' and apiKey='...'
    and longUrl in (select link from rss where url='.....')</pre>
<p><strong>Note:</strong> Once you register with <a href="http://bit.ly">bit.ly</a>, you can find the apiKey by going to the <a href="http://bit.ly/account">Account</a> section.</p>
<p>This table is already checked into the <a href="http://github.com/spullara/yql-tables/tree/master">YQL Open Table github repository</a>. There are many other open data tables of use ranging from bit.ly to amazon to iplocation contributed by the community.</p>
<p>Lastly, if you&#8217;d like to contribute your awesome open data table you can checkout <a href="http://developer.yahoo.net/blog/archives/2009/05/yql_open_tables.html">Fork me! YQL Open Tables!</a>. This article gives an excellent overview of how to get your table into github.</p>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/Yze1eGIBTPY" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2009/06/shorten-urls-with-bitly-using-yql/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2009/06/shorten-urls-with-bitly-using-yql</feedburner:origLink></item>
		<item>
		<title>YQL Execute Screencast &amp; Tutorial</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/wMHrjDozqZ4/yql-execute-screencast-tutorial</link>
		<comments>http://nagiworld.net/2009/05/yql-execute-screencast-tutorial#comments</comments>
		<pubDate>Sun, 03 May 2009 22:14:12 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[tools]]></category>

		<category><![CDATA[yql]]></category>

		<category><![CDATA[tutorial]]></category>

		<category><![CDATA[yql execute]]></category>

		<guid isPermaLink="false">http://nagiworld.net/?p=129</guid>
		<description><![CDATA[On 29th April &#8216;09 we released YQL Execute out into the open. You can find the actual release blog on yqlblog.net. We also released the following screencast which gives an overview of YQL Execute.  

Here&#8217;s the direct link to the YQL Execute Screencast on YDN. In the screencast, Sam Pullara gives an introduction to [...]]]></description>
			<content:encoded><![CDATA[<p>On 29th April &#8216;09 we released YQL Execute out into the open. You can find the actual release blog on <a href="http://www.yqlblog.net">yqlblog.net</a>. We also released the following screencast which gives an overview of YQL Execute.  </p>
<p><object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" width="400" height="300" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0"><param name="src" value="http://d.yimg.com/cosmos.bcst.yahoo.com/up/fop/embedflv/swf/fop.swf?shareEnable=1&amp;id=13227337&amp;autoStart=0&amp;infoEnable=0&amp;shareEnable=0&amp;prepanelEnable=1&amp;carouselEnable=0&amp;postpanelEnable=1" /><embed type="application/x-shockwave-flash" width="400" height="300" src="http://d.yimg.com/cosmos.bcst.yahoo.com/up/fop/embedflv/swf/fop.swf?shareEnable=1&amp;id=13227337&amp;autoStart=0&amp;infoEnable=0&amp;shareEnable=0&amp;prepanelEnable=1&amp;carouselEnable=0&amp;postpanelEnable=1"></embed></object></p>
<p>Here&#8217;s the direct link to the <a href="http://developer.yahoo.net/blogs/theater/archives/2009/04/yql_execute_screencast.html">YQL Execute Screencast on YDN</a>. In the screencast, <a href="http://www.javarants.com">Sam Pullara</a> gives an introduction to YQL Execute and I follow it up with a demo to show the power of this new feature. </p>
<p>For folks who prefer to read and follow the tutorial instead, here is the transcript of the entire demo along with the YQL queries as well as the Open Data Tables used to create the examples.</p>
<p><span id="more-129"></span><br />
Let&#8217;s get started with a little review on Open Data Tables to set the stage.</p>
<h4><strong>Dynamic Tables Recap</strong></h4>
<p>YQL already provides dynamic tables which let you access data from many formats across the internet such as  rss, atom, raw xml, json, csv or even html.</p>
<p>You can select xml directly from the yahoo local webservice.</p>
<pre class="brush: sql;">select * from xml where url="http://local.yahooapis.com/LocalSearchService/V3/localSearch?appid=YahooDemo&#038;query=pizza&#038;zip=94306&#038;results=2"</pre>
<p><a href="http://bit.ly/pTwkD">Try in YQL Console!</a></p>
<p>Behind the scenes, I read up the documentation on Local search and formed this query. Although this query returns the data from the remote service, you have the burden of constructing the URLs required. It also doesn&#8217;t give me the full power of YQL where I can use nested binary operators using the various query or path parameters. In essence you&#8217;d have to write all the code to collect the parameters and conditionally create the URL.</p>
<p>This issue can be easily resolved by using an <a href="http://developer.yahoo.com/yql/guide/yql-opentables-chapter.html">open data table</a>.</p>
<h4><strong>Open Data Tables</strong></h4>
<p>With an Open Data Table, we expose a way to bind YQL to your webservice with a single XML file. Let&#8217;s create an open data table for the local service</p>
<pre class="brush: xml;">
&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
&lt;table xmlns=&quot;http://query.yahooapis.com/v1/schema/table.xsd&quot;&gt;
    &lt;meta&gt;
        &lt;author&gt;Nagesh Susarla&lt;/author&gt;
        &lt;description&gt;Yahoo! Local Data table&lt;/description&gt;
        &lt;documentationURL&gt;http://developer.yahoo.com/search/local/V3/localSearch.html
        &lt;/documentationURL&gt;
        &lt;sampleQuery&gt;select * from {table} where zip=&#039;94085&#039; and query=&#039;pizza&#039;&lt;/sampleQuery&gt;
    &lt;/meta&gt;
    &lt;bindings&gt;
        &lt;select itemPath=&quot;ResultSet.Result&quot; produces=&quot;XML&quot;&gt;
            &lt;urls&gt;
                &lt;url&gt;http://local.yahooapis.com/LocalSearchService/V3/localSearch&lt;/url&gt;
            &lt;/urls&gt;
            &lt;paging model=&quot;offset&quot;&gt;
                &lt;start id=&quot;start&quot; default=&quot;1&quot;/&gt;
                &lt;pagesize id=&quot;results&quot; max=&quot;20&quot;/&gt;
            &lt;/paging&gt;
            &lt;inputs&gt;
                &lt;key id=&quot;zip&quot; type=&quot;xs:string&quot; paramType=&quot;query&quot; required=&quot;true&quot;/&gt;
                &lt;key id=&quot;query&quot; type=&quot;xs:string&quot; paramType=&quot;query&quot; required=&quot;true&quot;/&gt;
                &lt;key id=&quot;appid&quot; type=&quot;xs:string&quot; const=&quot;true&quot; private=&quot;true&quot; paramType=&quot;query&quot;
                     default=&quot;YahooDemo&quot;/&gt;
            &lt;/inputs&gt;
        &lt;/select&gt;
    &lt;/bindings&gt;
&lt;/table&gt;
</pre>
<p>The open data table contains a root &#8220;table&#8221; element which contains a meta tag. The meta element describes metadata such as author, documentation url and a sample query for the table. The next element &#8220;bindings&#8221; contains one or more select blocks. This element describes how YQL should map to the webservice. Each select has the base URL followed by the various input keys that it supports. (Before writing the table I read up the documentation and listed the various query parameters here)</p>
<p>Once I have the open data table ready, I simply upload it to my website and access it in yql console by performig the following query.</p>
<pre class="brush: sql;">
use 'http://nagiworld.net/yqldefs/mylocal.search.xml' as mylocal;
select * from mylocal where query = 'pizza' and zip = 94085
</pre>
<p><a href="http://bit.ly/12mDEP">Try in YQL Console!</a></p>
<p>I can even abitrarily nest the binary operators and YQL will do the work of creating the URLs and fetching the data.</p>
<pre class="brush:sql;">
use 'http://nagiworld.net/yqldefs/mylocal.search.xml' as mylocal;
select * from mylocal where query = 'pizza' and (zip = 94085 OR zip=94536)
</pre>
<p><a href="http://bit.ly/Gsa9k">Try in YQL Console!</a></p>
<p>This is an easy way to bring the power of YQL to disparate data sources.<br />
With YQL Open Tables, you can take almost any publicly available Web service and make it available as a table for others to use.</p>
<p>Now, what if you wanted to augment or enhance the results of a webservice?</p>
<h4><strong>YQL Execute</strong></h4>
<p>YQL Execute, our newest feature, lets you enhance or shape the resulting data to suit your needs. You can use &#8220;YQL Execute&#8221; to mash together or join data from multiple services and return the resulting data.</p>
<p>Let&#8217;s say I got an idea to build the next killer app. The idea is called &#8220;Search Rank&#8221; and its requirements are as follows:</p>
<ul>
<strong>Inputs:</strong>	</p>
<li>Give me a topic to search for &amp; such as &#8220;pizza&#8221;</li>
<li>Give me link to rank such as &#8220;pizzahut&#8221;</li>
</ul>
<p><strong>Output:</strong> The App should return you the search rank of the link as seen by Yahoo Boss.</p>
<p>Let&#8217;s see what <strong>search.web</strong> already provides today. For this lets go to console and try the following query.</p>
<pre class="brush: sql;">
select * from search.web where query = 'pizza'
</pre>
<p><a href="http://bit.ly/zspQF">Try in YQL Console!</a></p>
<p>results &#8230;</p>
<pre class="brush: xml; highlight:[9]">
&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
&lt;query...&gt;
    &lt;diagnostics/&gt;
    &lt;results&gt;
        &lt;result xmlns=&quot;http://www.inktomi.com/&quot;&gt;
            &lt;abstract&gt;&lt;![CDATA[Official site of Papa John&#039;s &lt;b&gt;pizza&lt;/b&gt; delivery and carry-out chain. Includes store locator, nutritional info, and franchise and employment opportunities.]]&gt;&lt;/abstract&gt;
            &lt;clickurl&gt;http://.......JWbks-/SIG=10u9f6tba/**http%3A//www.papajohns.com/&lt;/clickurl&gt;
            &lt;date&gt;2009/05/02&lt;/date&gt;
            &lt;dispurl&gt;&lt;![CDATA[www.&lt;b&gt;papajohns.com&lt;/b&gt;]]&gt;&lt;/dispurl&gt;
            &lt;size&gt;2237&lt;/size&gt;
            &lt;title&gt;Papa John&#039;s&lt;/title&gt;
            &lt;url&gt;http://www.papajohns.com/&lt;/url&gt;
        &lt;/result&gt;
	 &lt;/results&gt;
&lt;/query&gt;
</pre>
<p>You can see that the search results include a dispurl - which is the display URL. The only thing missing is the ranking. So lets write an Open data table to add a rank to each of these results.</p>
<pre class="brush: xml; highlight:[15,16,17,18,19,20,21,22,23,24,25,26,27]">
&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
&lt;table xmlns=&quot;http://query.yahooapis.com/v1/schema/table.xsd&quot;&gt;
    &lt;meta&gt;
        &lt;author&gt;Nagesh Susarla&lt;/author&gt;
        &lt;documentationURL&gt;My Search Rank&lt;/documentationURL&gt;
    &lt;/meta&gt;
    &lt;bindings&gt;
        &lt;select itemPath=&quot;results.result&quot; produces=&quot;XML&quot;&gt;
            &lt;urls&gt;
                &lt;url/&gt;
            &lt;/urls&gt;
            &lt;inputs&gt;
                &lt;key id=&quot;query&quot; type=&quot;xs:string&quot; paramType=&quot;variable&quot; required=&quot;true&quot;/&gt;
            &lt;/inputs&gt;
            &lt;execute&gt;&lt;![CDATA[
                    var search = y.query(&#039;select * from search.web(100) where query=@query&#039;,
                     {query:query}).results;
                    var rank = 0;
                    //y.log(&quot;initing rank&quot;);
                    default xml namespace = &#039;http://www.inktomi.com/&#039;;
                    for each (var result in search.result) {
                        rank++;
                        //y.log(&quot;rank for &quot; + rank);
                        result.rank += &lt;rank&gt;{rank}&lt;/rank&gt;;
                    }
                    response.object = search;
            ]]&gt;&lt;/execute&gt;
        &lt;/select&gt;
    &lt;/bindings&gt;
&lt;/table&gt;
</pre>
<p>The only additional element in this open data table is the &lt;execute&gt; block. This is where you add the code to enhance or reshape or mashup your data.</p>
<p>Here we use an object called &#8220;y&#8221; which is implicitly provided by the engine. Among the various methods that it supports is the &#8220;query&#8221; method which lets me run a YQL query right from inside my execute.</p>
<p>Here, I&#8217;m simply wrapping the search api, I issue a search.web call and get the results. Since search already orders the results according to their popularity, all I need to do is add a &lt;rank&gt; element to each result. Also note that the rank is just the index of the unmodified search result. So I loop over each result and use E4X to add a new element to it. Finally I upload this to my website and issue the following query to try it in the console.</p>
<pre class="brush:sql;">
use 'http://nagiworld.net/yqldefs/searchrank.xml';
select * from searchrank where query='pizza' and dispurl like '%pizzahut%'
</pre>
<p><a href="http://bit.ly/10KSXn">Try in YQL Console!</a></p>
<p>Result from console:</p>
<pre class="brush:xml; highlight:[13];">
&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
&lt;query&gt;
    &lt;diagnostics/&gt;
    &lt;results&gt;
        &lt;result xmlns=&quot;http://www.inktomi.com/&quot;&gt;
            &lt;abstract&gt;&lt;![CDATA[Official site for Pizz Hut provides online ordering for dine-in and delivery, a &lt;b&gt;Pizza&lt;/b&gt; Hut store finder, coupons, and menu.]]&gt;&lt;/abstract&gt;
            &lt;clickurl&gt;http://..._ylc=X...**http%3A//www.pizzahut.com/&lt;/clickurl&gt;
            &lt;date&gt;2009/05/02&lt;/date&gt;
            &lt;dispurl&gt;&lt;![CDATA[www.&lt;b&gt;pizzahut.com&lt;/b&gt;]]&gt;&lt;/dispurl&gt;
            &lt;size&gt;14113&lt;/size&gt;
            &lt;title&gt;&lt;![CDATA[&lt;b&gt;Pizza&lt;/b&gt; Hut]]&gt;&lt;/title&gt;
            &lt;url&gt;http://www.pizzahut.com/&lt;/url&gt;
            &lt;rank&gt;4&lt;/rank&gt;
        &lt;/result&gt;
    &lt;/results&gt;
&lt;/query&gt;
</pre>
<p>Voila! and you have the ranking of the particular element. Now I only have to wrap this in a nice UI and I have a killer app which does searchranking using YQL.</p>
<p>Here is the final product using some YUI goodness to wrap the YQL calls: <a href="http://nagiworld.net/yql/searchrank.html">SearchRank</a></p>
<p><strong>Useful links on YQL Execute:</strong></p>
<ul>
<li><a href="http://developer.yahoo.net/blog/archives/2009/04/yql_execute.html">Select * from Internet and Execute With YQL</a></li>
<li><a href="http://developer.yahoo.com/yql/guide/yql-execute-chapter.html">YQL Execute Docs</a></li>
<li><a href="http://www.datatables.org">DataTables.org</a>: Community site for data tables and a pointer to the best examples on YQL Execute</li>
<li><a href="http://github.com/spullara/yql-tables/tree/master">github repository</a>: Open Data Table repository</li>
</ul>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/wMHrjDozqZ4" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2009/05/yql-execute-screencast-tutorial/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2009/05/yql-execute-screencast-tutorial</feedburner:origLink></item>
		<item>
		<title>YQL @ Open Hack Day Bangalore</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/g2co0ZEsVZs/yql-open-hack-day-bangalore</link>
		<comments>http://nagiworld.net/2009/03/yql-open-hack-day-bangalore#comments</comments>
		<pubDate>Sun, 01 Mar 2009 07:56:21 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[yql]]></category>

		<category><![CDATA[openhackday]]></category>

		<guid isPermaLink="false">http://nagiworld.net/?p=108</guid>
		<description><![CDATA[I recently had the opportunity to give a tech talk on YQL at OpenHackday &#8216;09 in Bangalore.
Open Hack Day was an awesome event. It was very well organized and provided the ideal medium for hackers to put forth their ideas. Among the hackers, there was a lot of energy, enthusiasm and the will to do [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://nagiworld.net/blog/wp-content/uploads/2009/02/openhack2009web.jpg"><img class="alignleft size-medium wp-image-112" title="openhack2009web" src="http://nagiworld.net/blog/wp-content/uploads/2009/02/openhack2009web-300x143.jpg" alt="" width="300" height="143" /></a>I recently had the opportunity to give a tech talk on YQL at <a href="http://openhack2009.pbwiki.com/">OpenHackday &#8216;09</a> in Bangalore.</p>
<p>Open Hack Day was an awesome event. It was very well organized and provided the ideal medium for hackers to put forth their ideas. <span id="more-108"></span>Among the hackers, there was a lot of energy, enthusiasm and the will to do something innovative. I had the pleasure of meeting with a lot of the hackers and also helping answer their challenging questions on YQL and OAuth in anyway I could.</p>
<p>Being a developer of YQL, it was obviously very fulfilling to see YQL being used extensively by the hackers. After the electrifying weekend at hackday, I&#8217;m back at work building some cool YQL features.</p>
<p>Read more about hackday from the <a href="http://developer.yahoo.net/blog/archives/2009/02/openhack_bangalore.html">blog post on YDN by Chris Heilmann</a> aptly named &#8220;We came, we hacked, we picked winners - Open Hack Day Bangalore was a blast&#8221; <img src='http://nagiworld.net/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> </p>
<p>Here are the slides from my YQL talk at OpenHackDay &#8216;09</p>
<div id="__ss_1070743" style="width: 425px; text-align: left;"><a style="font:14px Helvetica,Arial,Sans-serif;display:block;margin:12px 0 3px 0;text-decoration:underline;" title="Yql Openhackday 2009" href="http://www.slideshare.net/nageshs/yql-openhackday-2009?type=presentation">Yql Openhackday 2009</a><object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" width="425" height="355" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0"><param name="allowFullScreen" value="true" /><param name="allowScriptAccess" value="always" /><param name="src" value="http://static.slideshare.net/swf/ssplayer2.swf?doc=yqlopenhackday2009-090225191655-phpapp02&amp;stripped_title=yql-openhackday-2009" /><embed type="application/x-shockwave-flash" width="425" height="355" src="http://static.slideshare.net/swf/ssplayer2.swf?doc=yqlopenhackday2009-090225191655-phpapp02&amp;stripped_title=yql-openhackday-2009" allowscriptaccess="always" allowfullscreen="true"></embed></object></p>
<div style="font-size: 11px; font-family: tahoma,arial; height: 26px; padding-top: 2px;">(tags: <a style="text-decoration:underline;" href="http://slideshare.net/tag/yql">yql</a> <a style="text-decoration:underline;" href="http://slideshare.net/tag/openhackday">openhackday</a>)</div>
</div>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/g2co0ZEsVZs" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2009/03/yql-open-hack-day-bangalore/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2009/03/yql-open-hack-day-bangalore</feedburner:origLink></item>
		<item>
		<title>YQL Social Queries (FAQ)</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/7nTuHapARws/yql-social-queries-faq</link>
		<comments>http://nagiworld.net/2008/12/yql-social-queries-faq#comments</comments>
		<pubDate>Wed, 31 Dec 2008 06:19:36 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[yql]]></category>

		<category><![CDATA[yql socialtables yql-faq]]></category>

		<guid isPermaLink="false">http://nagiworld.net/?p=49</guid>
		<description><![CDATA[

YQL Queries FAQ:
YQL provides tables which let you access the social profiles, connections, contacts as well as updates of a Yahoo User. The actual apis around which YQL builds can be found at http://developer.yahoo.com/social/. A compilation of questions and answers on the various social tables provided by YQL are listed below. More specifically, this post [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.flickr.com/photos/12337310@N03/2094426907/"><img class="aligncenter" title="Haleakala" src="http://farm3.static.flickr.com/2229/2094426907_cb2ac76db4.jpg?v=0" alt="" width="586" height="333" /></a></p>
<p style="text-align: center;">
<p><a href="http://developer.yahoo.com/yql/console">YQL</a> Queries FAQ:</p>
<p>YQL provides tables which let you access the social profiles, connections, contacts as well as updates of a Yahoo User. The actual apis around which YQL builds can be found at <a href="http://developer.yahoo.com/social/">http://developer.yahoo.com/social/</a>. A compilation of questions and answers on the various social tables provided by YQL are listed below. More specifically, this post deals with questions about how to get the necessary data (guids) which can then be used for querying the social tables. <span id="more-49"></span></p>
<ul>
<li>Whats with a GUID  being used in many Yahoo webservices?</li>
</ul>
<p style="padding-left: 30px;">A guid is a unique identifier which is NOT the YAHOO ID. Even though your yahoo id is unique, the guid is different and it looks pretty cryptic - example HEMOFFCCOH3MJ5HYPYQQQPI. I do agree that many users have very cryptic yahoo ids; what with all their birthdays and their birth city zipcodes and their lucky numbers right in the id, but the guid is just more uniformly cryptic and unique <img src='http://nagiworld.net/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> )</p>
<ul>
<li>How do I get my guid? Do i have to remember and type it in for each table that requires it?</li>
</ul>
<p style="padding-left: 30px;">YQL has already done this for you with the identifier <strong>me. </strong>It can be used anywhere your <strong>guid</strong> would appear. Just remember to drop the quotes and directly use the identifier me.</p>
<p style="padding-left: 60px;"><a title="open yql console" href="http://developer.yahoo.com/yql/console/?q=select * from social.profile where guid = me" target="_blank">select * from social.profile where guid = me</a></p>
<ul>
<li>How do I get a guid from a yahoo id?</li>
</ul>
<p style="padding-left: 30px;">YQL provides the <strong>yahoo.identity</strong> table which has a one-way mapping between a yahoo-id (yid) and a <strong>guid</strong>. Currently there is no way to get a yahoo id from a guid. This is prevent any possibility of spam that could have arisen from your Yahoo Id being exposed by such a service.</p>
<p style="padding-left: 30px;">Given a yahoo id (yid), you can get back the guid by running the following query:</p>
<p style="padding-left: 60px;"><a title="open yql console" href="http://developer.yahoo.com/yql/console?q=select guid from yahoo.identity where yid='tom'" target="_blank">select guid from yahoo.identity where yid = &#8216;tom&#8217;</a></p>
<ul>
<li>Is there a way to query the <strong>social.profile</strong> table with a <strong>yahoo id </strong>instead of a <strong>guid</strong>?</li>
</ul>
<p style="padding-left: 30px;">Yes! You can use a &#8220;IN&#8221; subselect query to find a the social profile given only a yahoo id</p>
<p style="padding-left: 60px;"><a title="open yql console" href="http://developer.yahoo.com/yql/console/?q=select * from social.profile where guid in (select guid from yahoo.identity where yid='tom')" target="_blank">select * from social.profile where guid in (select guid from yahoo.identity where yid=&#8217;tom&#8217;)</a></p>
<ul>
<li>How do I list out all the profiles of my connections?</li>
</ul>
<p style="padding-left: 30px;">As listed in one of YQL&#8217;s examples you can run the following query which first gets all the guids of your connections (friends) from <strong>social.connections</strong> and then fetches the profiles of each of the guids.</p>
<p style="padding-left: 60px;"><a title="open yql console" href="http://developer.yahoo.com/yql/console/?q=select * from social.profile where guid in (select guid from social.connections where owner_guid = me)" target="_blank">select * from social.profile where guid in (select guid from social.connections where owner_guid = me)</a></p>
<ul>
<li>Only 10 of my connections show up when querying <strong>social.connections </strong>but I&#8217;d like to see all of them.</li>
</ul>
<p style="padding-left: 30px;">To see all your connections, one needs to use an <strong>unbounded query</strong> listed in the documentation <a href="http://developer.yahoo.com/yql/guide/limit-offset-statements.html">here</a>. The following query returns all the connections. By specifying the (0), you&#8217;re telling YQL to do all the necessary paging.</p>
<p style="padding-left: 60px;"><a title="open yql console" href="http://developer.yahoo.com/yql/console/?q=select guid from social.connections(0) where owner_guid = me" target="_blank">select * from social.connections(0) where owner_guid = me</a></p>
<p>Similarly you can use the <strong>social.contacts </strong>table<strong> </strong>to fetch a guid&#8217;s contacts.</p>
<p><strong>Note: </strong>To view a guid&#8217;s profile or connections, you need explicit access from the guid. This is accomplished via 3 legged oauth. Unless the guid is your connection, you may not be able to see their profile in the yql console.</p>
<p>If you have more specific questions, let me know in the comments and I&#8217;ll do my best to answer them.</p>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/7nTuHapARws" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2008/12/yql-social-queries-faq/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2008/12/yql-social-queries-faq</feedburner:origLink></item>
		<item>
		<title>OAuth-ify this: 2 Legged OAuth service for YQL</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/9qMq3AkeGsg/oauth-ify-this-2-legged-oauth-service-for-yql</link>
		<comments>http://nagiworld.net/2008/11/oauth-ify-this-2-legged-oauth-service-for-yql#comments</comments>
		<pubDate>Sun, 09 Nov 2008 11:29:35 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[oauth]]></category>

		<category><![CDATA[yql]]></category>

		<category><![CDATA[appengine]]></category>

		<category><![CDATA[oauthproxy]]></category>

		<guid isPermaLink="false">http://nagiworld.net/?p=54</guid>
		<description><![CDATA[Introducing OAuth-ify/OAuthProxy, a service that performs two legged OAuth calls to backend webservices. As of today it only supports YQL.
Motivation:
With two-legged OAuth, there are only two parties involved i.e. the consumer of the api and the service provider. It doesn&#8217;t deal with user credentials or other private data that come into the picture with three-Legged [...]]]></description>
			<content:encoded><![CDATA[<p>Introducing <a href="http://oauthify.appspot.com">OAuth-ify/OAuthProxy</a>, a service that performs two legged OAuth calls to backend webservices. As of today it only supports YQL.</p>
<p><strong>Motivation:</strong></p>
<p>With two-legged OAuth, there are only two parties involved i.e. the consumer of the api and the service provider. It doesn&#8217;t deal with user credentials or other private data that come into the picture with three-Legged OAuth. YQL is one of the services which uses OAuth exclusively for all webservice access. (apart from the console)<br />
<span id="more-54"></span><br />
Two-Legged OAuth requires a user to sign the request in a spec mandated way and this implies that using a service such as Yahoo! Pipes to fetch non private data from YQL wasnt possible.</p>
<p><strong>Requirements:</strong></p>
<ol>
<li>The service should ask for everything it needs and respond with the result of the backend service. Simply put, the service must act as a proxy. Take in the Consumer Key, Consumer Secret and the YQL query, do the necessary signing and return the response.</li>
<li>What if I do not want to expose my consumer key/secret for the fear of it being compromised? This brings up a requirement that there should be a way to mask the Key and Secret. This is accomplished today by explicit registration. The user is asked to register their Consumer Key and Secret with the service and in return they get an appId which is URI friendly.</li>
<li>AppId should be disposable. If an AppId is compromised, it should be possible to regenerate a new one which inturn uses the same underlying key/secret pair. (Note that I still want the underlying Consumer Key and Secret to be the same for many track-ability reasons and relationship with the service provider)</li>
</ol>
<p><strong>Implementation:</strong></p>
<p>I deployed on Google App Engine (for lack of a platform that runs Java on the cloud!) and picked up a bit of python along the way. I  must say that GAE&#8217;s been pretty easy to pick up and use. Finally the service exposes two main endpoints</p>
<ul>
<li>Executes a query given the following parameters (http://oauthify.appspot.com/yqlQuery)
<ul>
<li>Scenario 1: <strong>http://oauthify.appspot.com/yqlQuery?ckey=aghvYXV0aGlmeXIOCxIIVXNlcklhZm8YBgw</strong><strong>&amp;csecret=PQ&amp;q=show%20tables </strong>
<ul>
<li>ckey - Consumer Key (required)</li>
<li>csecret - Consumer Secret (required)</li>
<li>q - YQL Query (defaults to show tables)</li>
<li>format - xml or json (passed to the backend) (defaults to XML)</li>
</ul>
</li>
</ul>
<ul>
<li>Scenario 2: <strong>http://oauthify.appspot.com/yqlQuery?appId=</strong><strong>aghvYXV0aGlmeXIOCxIIVXNlcklhZm8YBgw</strong><strong>&amp;q=show%20tables </strong>
<ul>
<li>appId - App Id provided by the service. (Also see the next endpoint)</li>
<li>q - YQL Query</li>
<li>format - (same as above)</li>
<li>Note that the prequel to this Scenario is that the user gets the appId by registration.</li>
</ul>
</li>
</ul>
</li>
</ul>
<ul>
<li>Register the Consumer Key and Secret to get an appId (http://oauthify.appspot.com/register)
<ul>
<li>Requires a user to Login (using a google account). This is to protect the user&#8217;s Key and Secret.</li>
<li>Provides a way to store CK and CS</li>
<li>Provides a way to see all previously stored Keys and to regenerate AppIds as needed.</li>
</ul>
</li>
</ul>
<p><strong>Finally:</strong></p>
<p>Was the Goal of running 2 Legged YQL in Pipes accomplished? <em>Yes!</em> check out <a href="http://pipes.yahoo.com/nageshs/yqlquery"><span id="permaEliTxvCs3RGtsl0NBhNMsA1580911237">http://pipes.yahoo.com/nageshs/yqlquery</span></a> which uses a PrivateString fields to use CS and CK from Scenario 1 above and last but not the least <a href="http://pipes.yahoo.com/nageshs/yqlquerymodule">http://pipes.yahoo.com/nageshs/yqlquerymodule</a> which uses an AppId from the web service. Here is a screen shot of the Pipe. I&#8217;m not in the least worried about sharing my AppId, since OAuthify will let me regenerate it any time I like <img src='http://nagiworld.net/blog/wp-includes/images/smilies/icon_wink.gif' alt=';-)' class='wp-smiley' /> </p>
<p><a href="http://nagiworld.net/blog/wp-content/uploads/2008/11/yqlpipemodule.jpg"><img class="alignnone size-full wp-image-60" title="yqlpipemodule" src="http://nagiworld.net/blog/wp-content/uploads/2008/11/yqlpipemodule.jpg" alt="" width="500" height="301" /></a></p>
<p>Enjoy &amp; Curl your YQL queries today!</p>
<p><strong>Update 1:</strong> Thanks to <a href="http://javarants.com">Sam</a>, OAuthify is now accessible via the rightly named domain <a href="http://www.oauthproxy.com">http://OAuthProxy.com</a> <a href="http://www.oauthproxy.com"></a></p>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/9qMq3AkeGsg" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2008/11/oauth-ify-this-2-legged-oauth-service-for-yql/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2008/11/oauth-ify-this-2-legged-oauth-service-for-yql</feedburner:origLink></item>
		<item>
		<title>YQL: A query language for the Web</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/i8AE_Iq2pfs/yql-a-query-language-for-the-web</link>
		<comments>http://nagiworld.net/2008/10/yql-a-query-language-for-the-web#comments</comments>
		<pubDate>Tue, 28 Oct 2008 20:56:43 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[Uncategorized]]></category>

		<category><![CDATA[yql]]></category>

		<guid isPermaLink="false">http://nagiworld.net/?p=43</guid>
		<description><![CDATA[After months in hiding, I can finally talk about the stuff I&#8217;ve been working on.
I was involved in creating YQL and was responsible for leading the query engine implementation. I am very pleased to see it being released to the public today.  Check out Jonathan&#8217;s talk which gives a great overview of YQL.  [...]]]></description>
			<content:encoded><![CDATA[<p>After months in hiding, I can finally talk about the stuff I&#8217;ve been working on.</p>
<p>I was involved in creating <a href="http://developer.yahoo.com/yql">YQL</a> and was responsible for leading the query engine implementation. I am very pleased to see it being released to the public today.  Check out <a href="http://jonathantrevor.net/?p=15">Jonathan&#8217;s talk</a> which gives a great overview of YQL.  (Jonathan&#8217;s posted about the YQL Launch <a href="http://jonathantrevor.net/?p=33">here</a>)</p>
<p>YQL adds the simplicity and self describable nature of SQL to the Web.  In addition to this, you can also join disparate webservices with common keys. For example Yahoo&#8217;s &#8220;guid&#8221; can be used to join many common webservices which are keyed on &#8220;guid&#8221;s.<br />
<span id="more-43"></span><br />
The basic goals that we had in mind while designing YQL are as follows:</p>
<ul>
<li>Ease of use - SQLish syntax lowers the barrier to entry since its very popular with developers everywhere.</li>
<li>Self describing - desc &lt;table&gt;, show tables (the beauty of SQL lies in its simplicity and self describing nature and we wanted to piggy back on these fundamentals)</li>
<li>Data Access Options -  With YQL you can access both Yahoo&#8217;s webservices as well the Web&#8217;s data (with tables such as RSS, ATOM, FEED and XML). We also support external webservices to stress on the &#8216;O&#8217; in YOS (Open)</li>
<li>Extensible - (Still in the works but we intend to make it easy to extend the list of supported tables)</li>
<li>Best Effort - In the Web world we realise that disparate services have different strengths and weaknesses. Keeping this in mind, YQL employs a &#8220;best effort&#8221; query fulfillment strategy with warning/error messages as necessary (in the diagnostics section of the response)</li>
</ul>
<p>Please try out the console at &#8220;<a href="http://developer.yahoo.com/yql/console">http://developer.yahoo.com/yql/console&#8221;</a> and provide your feedback <a href="http://developer.yahoo.net/forum/index.php?showforum=41">here</a>. As with all 1.0 stuff, please bear in mind we still have a lot of work to do and wrinkles to iron out.</p>
<p>Kudos to the rest of the YQL team - <a href="http://www.b7j0c.org/">Brad</a>, Josh, <a href="http://paul.donnelly.org/">Paul</a>, <a href="http://javarants.com/">Sam</a> and <a href="http://jonathantrevor.net/">Jonathan</a>.</p>
<p>Happy Diwali</p>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/i8AE_Iq2pfs" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2008/10/yql-a-query-language-for-the-web/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2008/10/yql-a-query-language-for-the-web</feedburner:origLink></item>
		<item>
		<title>Duplicate Package Finder</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/Ob01kIbB9n0/duplicate-package-finder</link>
		<comments>http://nagiworld.net/2007/10/duplicate-package-finder#comments</comments>
		<pubDate>Wed, 03 Oct 2007 07:33:00 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[classloader]]></category>

		<category><![CDATA[java]]></category>

		<guid isPermaLink="false">tag:blogger.com,1999:blog-28643943.post-8649480429910742839</guid>
		<description><![CDATA[It&#8217;s pretty common to encounter a situation where you want the following question answered.
&#8220;Find me the java packages in classpath &#8216;X&#8217; which are already present in &#8216;Y&#8217;&#8221;
where X = Classpath of an application;
and Y = System Classpath or some other classpath
I wrote a small command line utility which answers this question and groks the list [...]]]></description>
			<content:encoded><![CDATA[<p><span style="color: #000000;">It&#8217;s pretty common to encounter a situation where you want the following question answered.</span></p>
<p><span style="color: #000000;">&#8220;Find me the java packages in classpath &#8216;X&#8217; which are already present in &#8216;Y&#8217;&#8221;</span><br />
<span style="color: #000000;">where X = Classpath of an application;</span><br />
<span style="color: #000000;">and Y = System Classpath or some other classpath</span></p>
<p><span style="color: #000000;">I wrote a small command line utility which answers this question and groks the list of duplicate patterns. For WebLogic Server users,  this tool finds the duplicate packages and also emits some xml which can be pasted into  &#8216;META-INF/weblogic-application.xml&#8217;  to ensure that the version in the application always wins.</span><br />
<span id="more-15"></span><br />
<span style="color: #000000;">Here&#8217;s the usage:</span></p>
<pre style="color: #000000;"><span style="font-family: courier new;">~/code$ java -jar ./utils/dist/dupPkgfinder.jar
</span><span style="font-size: 100%; color: #333333;"><span style="font-family: courier new;">Usage: net.nagiworld.utils.DupPackageFinder</span><span style="font-family: courier new;">
      -appclasspath [String] Application Classpath
     </span><span style="font-family: courier new;"> -parentclasspath [String] Classpath to compare against
     </span><span style="font-family: courier new;"> -advice [flag] Emit Advice for use with WebLogic Server</span><span style="font-family: courier new;">
      -verbose [flag] Show me what's going on</span></span></pre>
<p style="color: #000000;">Example: Lets say I have a webapp which uses a specific version of log4j.jar. Since WebLogic already packages a version of log4j.jar in its System Classpath, using a different version requires an explicit descriptor element.</p>
<p>Given that the jar is in WEB-INF/lib, I ask the tool to help me out:</p>
<pre style="color: #000000;"><span style="font-size: 100%; font-family: courier new; color: #333333;">~/code/utils/test$ java -jar ../dist/dupPkgfinder.jar
-parentclasspath $CLASSPATH -appclasspath './WEB-INF/lib/*'
-advice

Duplicate Packages:
org/apache/log4j
org/apache/log4j/chainsaw
org/apache/log4j/config
org/apache/log4j/helpers
org/apache/log4j/jdbc
org/apache/log4j/jmx
org/apache/log4j/lf5
org/apache/log4j/lf5/util
org/apache/log4j/lf5/viewer
org/apache/log4j/lf5/viewer/categoryexplorer
org/apache/log4j/lf5/viewer/configure
org/apache/log4j/net
org/apache/log4j/nt
org/apache/log4j/or
org/apache/log4j/or/jms
org/apache/log4j/or/sax
org/apache/log4j/spi
org/apache/log4j/varia
org/apache/log4j/xml

Recommendation: Add the following snippet to 'META-INF/weblogic-application.xml'.

&lt;prefer-application-packages&gt;
&lt;package-name&gt;org.apache.log4j.*&lt;/package-name&gt;
&lt;/prefer-application-packages&gt;
</span></pre>
<p><span style="color: #000000;">If you&#8217;re familiar with FilteringClassLoader, then this tool is an excellent aid </span><span style="color: #000000;">to creating the filter patterns that should be added to the weblogic-application.xml </span><span style="color: #000000;">descriptor. See  http://edocs.bea.com/wls/docs92/programming/classloading.html </span><span style="color: #000000;">for more details on FilteringClassLoader.</span></p>
<p><span style="font-weight: bold; color: #000000;">NOTE:</span><span style="color: #000000;"> Duplicate packages in your application does not necessarily imply </span><span style="color: #000000;">that FilteringClassLoader be used. Often times unused jars also end up in WEB-INF/lib</span></p>
<p><span style="color: #000000;">This tool is a thin wrapper our DepHandler from </span><a style="color: #000000;" href="http://code.google.com/p/jarjar/">jarjar</a><span style="color: #000000;">. In addition to that it also depends </span><span style="color: #000000;">on </span><a style="color: #000000;" href="http://code.google.com/p/cli-parser/">cli-parser</a><span style="color: #000000;"> which is used for command line parsing. (using cool annotations!). </span><span style="color: #000000;">The tool  supports wildcards thanks to the jarjarlinks support for wildcards.</span></p>
<p style="color: #000000;">
<p style="color: #000000;">You can download the tool here - <a href="http://nagiworld.net/code/utils/dupPkgFinder.jar">dupPkgFinder.jar</a></p>
<p style="color: #000000;">Enjoy! Please do share your comments and any other cool uses/improvements for the tool <img src='http://nagiworld.net/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> </p>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/Ob01kIbB9n0" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2007/10/duplicate-package-finder/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2007/10/duplicate-package-finder</feedburner:origLink></item>
		<item>
		<title>JavaPolis ‘06 Talk online</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/8xJv9a15qEs/javapolis-06-talk-online</link>
		<comments>http://nagiworld.net/2007/07/javapolis-06-talk-online#comments</comments>
		<pubDate>Mon, 30 Jul 2007 19:55:00 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[java]]></category>

		<category><![CDATA[javapolis]]></category>

		<guid isPermaLink="false">tag:blogger.com,1999:blog-28643943.post-8831609875257106405</guid>
		<description><![CDATA[
My JavaPolis &#8216;06 talk on &#8220;Java EE Enhancements for Real World deployments&#8221; is now online here. I&#8217;ve uploaded a PDF version of the presentation which can be found here. The talk describes key enhancements in WebLogic Server such as Application Libraries, Side-by-Side deployment and deployment plans  which aid in real world deployments. Do let [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://parleys.com/display/PARLEYS/Java+EE+Enhancements+for+Real+World+deployments?showComments=true" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px;" src="http://www.nagiworld.net/pics/javapolis_nagesh.jpg" border="0" alt="" /></a><br />
My JavaPolis &#8216;06 talk on &#8220;Java EE Enhancements for Real World deployments&#8221; is now online <a href="http://parleys.com/display/PARLEYS/Java+EE+Enhancements+for+Real+World+deployments?showComments=true">here</a>. I&#8217;ve uploaded a PDF version of the presentation which can be found <a href="http://nagiworld.net/code/javapolis/JEE-Enhancements.pdf">here</a>. The talk describes key enhancements in WebLogic Server such as Application Libraries, Side-by-Side deployment and deployment plans  which aid in real world deployments. Do let me know if you have any comments or questions.</p>
<p><a href="http://javapolis.com/confluence/display/JP06/Home">JavaPolis &#8216;06</a> was held in Antwerp, Belgium from Dec 11th through 15th. It had all the elements of a perfect developer conference - Beer, geeks, frites and more geeks. Our bea booth had Lafe on tap which made it a lot easier to talk about very deep and serious topics <img src='http://nagiworld.net/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /><br />
Thanks to the JavaPolis organizers for making the talk available online. It&#8217;s easily one of the best java conferences I&#8217;ve ever been too.  Unconventional and original! is how I&#8217;d summarize the conference in two words.</p>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/8xJv9a15qEs" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2007/07/javapolis-06-talk-online/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2007/07/javapolis-06-talk-online</feedburner:origLink></item>
		<item>
		<title>Inbox Zero</title>
		<link>http://feedproxy.google.com/~r/nageshsblog/~3/ViW8g7G8ItQ/inbox-zero</link>
		<comments>http://nagiworld.net/2007/07/inbox-zero#comments</comments>
		<pubDate>Sun, 29 Jul 2007 20:14:00 +0000</pubDate>
		<dc:creator>nagesh</dc:creator>
		
		<category><![CDATA[Uncategorized]]></category>

		<category><![CDATA[lifehacking]]></category>

		<guid isPermaLink="false">tag:blogger.com,1999:blog-28643943.post-8007486273344137603</guid>
		<description><![CDATA[I came across this amazing talk by Merlin Mann. A must for every person who intends to be productive with Email.

]]></description>
			<content:encoded><![CDATA[<p>I came across this amazing <a href="http://www.43folders.com/2007/07/25/merlins-inbox-zero-talk/">talk by Merlin Mann</a>. A must for every person who intends to be productive with Email.</p>
<p><object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" width="100" height="100" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0"><param name="id" value="VideoPlayback" /><param name="src" value="http://video.google.com/googleplayer.swf?docId=973149761529535925&amp;hl=en" /><embed id="VideoPlayback" type="application/x-shockwave-flash" width="100" height="100" src="http://video.google.com/googleplayer.swf?docId=973149761529535925&amp;hl=en"></embed></object></p>
<img src="http://feeds.feedburner.com/~r/nageshsblog/~4/ViW8g7G8ItQ" height="1" width="1"/>]]></content:encoded>
			<wfw:commentRss>http://nagiworld.net/2007/07/inbox-zero/feed</wfw:commentRss>
		<feedburner:origLink>http://nagiworld.net/2007/07/inbox-zero</feedburner:origLink></item>
	</channel>
</rss>
