blog.stavi.sh

blog.stavi.sh http://blog.stavi.sh Most recent posts at blog.stavi.sh posterous.com Wed, 08 Dec 2010 01:27:00 -0800 Bicycle Commuters Overtake Train Commuters In The District Of Columbia! http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/BKytlKEEMzI/bicycles-overtake-train-commuters-in-the-dist http://blog.stavi.sh/bicycles-overtake-train-commuters-in-the-dist <p> <div>Recently, I've been looking at datasets from The Wasthington Post's Transportation & Development team. Very interesting. From the years 2004 to 2009, it shows big increases in 2 wheeled transportation (motorcycle, bicycle), in fact, the total number of bicycle commuters was more than train commuters (see below). Solidarity my two wheeled, friends. Solidarity. :)</div> <p /> <div><iframe scrolling="no" src="http://washingtonpost.socrata.com/w/tc59-58zm/xhqs-a7fj?cur=uHL_cnbOwmG&from=root" frameborder="0" height="646px" width="760px"><a href="http://washingtonpost.socrata.com/dataset/Commuting-By-Year-Timeline/tc59-58zm" title="Commuting By Year Timeline" target="_blank">Commuting By Year Timeline</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> </p> <p><a href="http://blog.stavi.sh/bicycles-overtake-train-commuters-in-the-dist">Permalink</a> | <a href="http://blog.stavi.sh/bicycles-overtake-train-commuters-in-the-dist#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/BKytlKEEMzI" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/bicycles-overtake-train-commuters-in-the-dist Fri, 01 Oct 2010 08:10:00 -0700 Synchronous microcontroller communication interfaces. Everything you wanted to know, but were afraid to ask. http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/UauTMfWOwis/synchronous-microcontroller-communication-int http://blog.stavi.sh/synchronous-microcontroller-communication-int <p> <p><span style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px;">Synchronous Interfaces (suitable for peripherals on the same board or <1 m):<br />I2C (Inter Integrated Circuit, I squared C)<br />SPI (Serial Peripheral Interface Bus )<br />U(S)ART (Universal Synchronous & Asynchronous Receiver Transmitter)<br />(aka Serial Communication Interface (SCI))<br />(S) can be synchronous<p /> I2C<br />* true bus, requires only two pins for multiple devices (address space)<br />* 100kbits/sec, 400kbits/sec, 1Mbits/sec<br />* half-duplex<br />* high noise sensitivity, lower data integrity<br />* slave ack / rx confirmation<p />SPI<br />* not a true bus, N devices require 3+N pins<br />* 10Mbits/sec<br />* full–duplex capability<br />* any message sizes, suited for longer data streams (I2C limited to 8 bit word)<p />USART<br />* standard serial connection (also over USB)<br />* few hundred bits per second (bps) up to 1.5Mbps<br />* error checking with parity bit</span></p> </p> <p><a href="http://blog.stavi.sh/synchronous-microcontroller-communication-int">Permalink</a> | <a href="http://blog.stavi.sh/synchronous-microcontroller-communication-int#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/UauTMfWOwis" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/synchronous-microcontroller-communication-int Thu, 30 Sep 2010 13:54:00 -0700 Unit Testing Arduino With Python http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/FxAggWOoIko/unit-testing-arduino-with-python http://blog.stavi.sh/unit-testing-arduino-with-python <p> <p><span style="font-family: helvetica, arial, freesans, clean, sans-serif; line-height: 20px;">Uses python to dynamically load Arduino hex images for unit testing. Supports windows and osx via configuration file. To find out where your hex images are placed by the Arduino IDE, hit the shift key before you hit the build (play) button. You can hit the shift for upload as well to find out where your avrdude (command line upload utility) is located.  Alternatively, you can look at the included config files and use your install location (using Arduino 0020).</span></p> <p><a href="http://github.com/toddstavish/Python-Arduino-Unit-Testing">http://github.com/toddstavish/Python-Arduino-Unit-Testing</a></p> </p> <p><a href="http://blog.stavi.sh/unit-testing-arduino-with-python">Permalink</a> | <a href="http://blog.stavi.sh/unit-testing-arduino-with-python#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/FxAggWOoIko" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/unit-testing-arduino-with-python Tue, 31 Aug 2010 12:51:00 -0700 Algerian Cities Spreadsheet, Latitude \ Longitude, And Map http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/_YjrAM5bZCI/algerian-cities-spreadsheet-latitude-longitud http://blog.stavi.sh/algerian-cities-spreadsheet-latitude-longitud <p> <p>Spreadsheet: <br /><a href="http://www.stavi.sh/downloads/algerian-cities-lat-long.xls">http://www.stavi.sh/downloads/algerian-cities-lat-long.xls</a> <p /> Google Docs: <br /><a href="https://spreadsheets0.google.com/ccc?key=tGdADtMhFEl1a10ruijx2Kw&hl=en">https://spreadsheets0.google.com/ccc?key=tGdADtMhFEl1a10ruijx2Kw&hl=en</a> <p /> Socrata:</p> <div><iframe scrolling="no" src="http://www.socrata.com/w/2ewa-umtn/y34g-bnf3?cur=uBhky0emYdt&from=root" frameborder="0" height="646px" width="760px"><a href="http://www.socrata.com/dataset/Algerian-Cities-Latitude-And-Longitude/2ewa-umtn" title="Algerian Cities Latitude And Longitude" target="_blank">Algerian Cities Latitude And Longitude</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> <div><iframe scrolling="no" src="http://www.socrata.com/w/yyyp-dz72/y34g-bnf3?cur=-3qnyifPw9P&from=root" frameborder="0" height="646px" width="760px"><a href="http://www.socrata.com/dataset/Cities-of-Algeria/yyyp-dz72" title="Cities of Algeria" target="_blank">Cities of Algeria</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> </p> <p><a href="http://blog.stavi.sh/algerian-cities-spreadsheet-latitude-longitud">Permalink</a> | <a href="http://blog.stavi.sh/algerian-cities-spreadsheet-latitude-longitud#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/_YjrAM5bZCI" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/algerian-cities-spreadsheet-latitude-longitud Tue, 31 Aug 2010 07:56:00 -0700 Afghani Cities Spreadsheet, Map, Latitude And Longitude http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/ig3SpyCddPw/cities-within-afghanistan-latitude-and-longit http://blog.stavi.sh/cities-within-afghanistan-latitude-and-longit <p> <p>Spreadsheet: <br /><a href="http://www.stavi.sh/downloads/Cities_Within_Afghanistan_Latitude_Longitude.xls">http://www.stavi.sh/downloads/Cities_Within_Afghanistan_Latitude_Longitude.xls</a> <p /> Google Docs: <br /><a href="https://spreadsheets.google.com/ccc?key=0At92oU3FPZ4QdHhfMFJ0a2lUcXNuN0J2bHRRUk01akE&hl=en">https://spreadsheets.google.com/ccc?key=0At92oU3FPZ4QdHhfMFJ0a2lUcXNuN0J2bHRR...</a> <p /> Socrata:</p> <div><iframe scrolling="no" src="http://www.socrata.com/w/vixk-q2bg/y34g-bnf3?cur=tXxwWi1cBoR&from=root" frameborder="0" height="646px" width="760px"><a href="http://www.socrata.com/dataset/Cities-Within-Afghanistan-Latitude-Longitude/vixk-q2bg" title="Cities Within Afghanistan Latitude Longitude" target="_blank">Cities Within Afghanistan Latitude Longitude</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> <div><iframe scrolling="no" src="http://www.socrata.com/w/n2yg-m6yk/y34g-bnf3?cur=YhZjHlhz1k0&from=root" frameborder="0" height="646px" width="760px"><a href="http://www.socrata.com/dataset/Cities-In-Afghanistan/n2yg-m6yk" title="Cities In Afghanistan" target="_blank">Cities In Afghanistan</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> </p> <p><a href="http://blog.stavi.sh/cities-within-afghanistan-latitude-and-longit">Permalink</a> | <a href="http://blog.stavi.sh/cities-within-afghanistan-latitude-and-longit#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/ig3SpyCddPw" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/cities-within-afghanistan-latitude-and-longit Thu, 26 Aug 2010 14:25:00 -0700 Real-time Relationship Analytics From Large-scale Graph Processing http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/W5PAZk5rdxg/real-time-relationship-analytics-from-large-s http://blog.stavi.sh/real-time-relationship-analytics-from-large-s <p> <p>Cassandra excels at storing large, active, decentralized datasets. Additionally, Cassandra’s rich data model allows efficient use for many applications beyond simple <a href="http://en.wikipedia.org/wiki/Associative_array">associative arrays</a>. One interesting application is the processing of large-scale <a href="http://en.wikipedia.org/wiki/Graph_theory">graph structures</a>.</p> <p>I have devised a <a href="http://github.com/toddstavish/Cassandra-Graph-Extract">graph application layer</a> to extract and process social network analysis data from Cassandra, using <a href="http://www.infinitegraph.com/">InfiniteGraph</a>. The technical benefits of the social-graph-extract application layer and its use of graph-oriented processing have been <a href="http://blog.stavi.sh/polyglot-persistence-integrating-low-latency">articulated</a>.</p> <p>Social network analysis is one application of a more general category, <a href="http://www.dbms2.com/2009/08/21/social-network-analysis-aka-relationship-analytics/">relationship analytics</a>, as defined by <a href="http://www.monash.com/curtbio.html">Curt Monash</a>. The relationship analytics problem domain maps well to the unique features of the Cassandra-InfiniteGraph hybrid system:</p> <ul> <li>dedicated vertex/edge API</li> <li>data can be clustered according to vertex/edge proximity</li> <li>disk-based/memory-centric access</li> <li>peer-to-peer communication from InfiniteGraph node to Cassandra node</li> <li>bidirectional updates between raw Cassandra data and Infinitegraph analytics</li> <li>parallel streaming and caching from InfintiteGraph</li> <li>modeling flexibility to support a variety of sources</li> <li>redundancy and high-availability</li> <li>precision and speed for graph analytics</li> <li>finding extremely long paths, all paths, unknown paths, or paths of nontrivial or indeterminate length</li> </ul> <p>Current business problems that can utilize these features:</p> <ul> <li>analyzing high-frequency trading</li> <li>discovering high degrees of mutual interconnection in social networks</li> <li>data mining subtle retail correlations</li> <li>product recommendation engines</li> <li>determining terrorist or criminal behavior inferred from known relationships</li> <li>finding a pattern of relationships for fraud detection</li> <li>investigating the directed relationships between proteins and genes</li> <li>checking which entity has the shortest average connection to a group of others for cyber security (botnet controller)</li> </ul> <p><a href="http://github.com/toddstavish/Cassandra-Graph-Extract"><strong>The working codebase for this Cassandra / InfiniteGraph integration can be retrieved from GitHub.</strong></a> Forking of the main project is welcome (including downstream updates). If you have any questions or suggestions, please contact <a href="http://twitter.com/toddstavish">@toddstavish</a>.</p> </p> <p><a href="http://blog.stavi.sh/real-time-relationship-analytics-from-large-s">Permalink</a> | <a href="http://blog.stavi.sh/real-time-relationship-analytics-from-large-s#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/W5PAZk5rdxg" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/real-time-relationship-analytics-from-large-s Wed, 18 Aug 2010 06:01:00 -0700 Python code that writes a multibyte value (long/float) over serial to Arduino http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/480etBo8Grg/python-code-that-writes-a-multibyte-value-lon http://blog.stavi.sh/python-code-that-writes-a-multibyte-value-lon <p> <p><div class="data type-python"> <table class="lines" cellspacing="0" cellpadding="0"> <tr> <td> <pre class="line_numbers"><span rel="#L1" id="L1">1</span> <span rel="#L2" id="L2">2</span> <span rel="#L3" id="L3">3</span> <span rel="#L4" id="L4">4</span> <span rel="#L5" id="L5">5</span> <span rel="#L6" id="L6">6</span> <span rel="#L7" id="L7">7</span> <span rel="#L8" id="L8">8</span> <span rel="#L9" id="L9">9</span> <span rel="#L10" id="L10">10</span> <span rel="#L11" id="L11">11</span> <span rel="#L12" id="L12">12</span> <span rel="#L13" id="L13">13</span> <span rel="#L14" id="L14">14</span> <span rel="#L15" id="L15">15</span> <span rel="#L16" id="L16">16</span> <span rel="#L17" id="L17">17</span> <span rel="#L18" id="L18">18</span> <span rel="#L19" id="L19">19</span> <span rel="#L20" id="L20">20</span> <span rel="#L21" id="L21">21</span> <span rel="#L22" id="L22">22</span> <span rel="#L23" id="L23">23</span> <span rel="#L24" id="L24">24</span> <span rel="#L25" id="L25">25</span> <span rel="#L26" id="L26">26</span> <span rel="#L27" id="L27">27</span> <span rel="#L28" id="L28">28</span> <span rel="#L29" id="L29">29</span> <span rel="#L30" id="L30">30</span> <span rel="#L31" id="L31">31</span> </pre> </td> <td width="100%"> <div class="highlight"><pre /><div class="line" id="LC1"><span class="n">Python</span><span class="p">:</span></div><div class="line" id="LC2"><br /></div><div class="line" id="LC3"><span class="k">def</span> <span class="nf">packIntegerAsULong</span><span class="p">(</span><span class="n">value</span><span class="p">):</span></div><div class="line" id="LC4">    <span class="sd">"""Packs a python 4 byte unsigned integer to an arduino unsigned long"""</span></div><div class="line" id="LC5">    <span class="k">return</span> <span class="n">struct</span><span class="o">.</span><span class="n">pack</span><span class="p">(</span><span class="s">'I'</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span> <span class="c">#should check bounds</span></div><div class="line" id="LC6"><br /></div><div class="line" id="LC7"><span class="c"># To see what it looks like on python side</span></div><div class="line" id="LC8"><span class="n">val</span> <span class="o">=</span> <span class="mi">15000</span></div><div class="line" id="LC9"><span class="k">print</span> <span class="n">binascii</span><span class="o">.</span><span class="n">hexlify</span><span class="p">(</span><span class="n">port</span><span class="o">.</span><span class="n">packIntegerAsULong</span><span class="p">(</span><span class="n">val</span><span class="p">))</span></div><div class="line" id="LC10"><br /></div><div class="line" id="LC11"><span class="c"># send and receive via pyserial</span></div><div class="line" id="LC12"><span class="n">ser</span> <span class="o">=</span> <span class="n">serial</span><span class="o">.</span><span class="n">Serial</span><span class="p">(</span><span class="n">serialport</span><span class="p">,</span> <span class="n">bps</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span></div><div class="line" id="LC13"><span class="n">ser</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">packIntegerAsULong</span><span class="p">(</span><span class="n">val</span><span class="p">))</span></div><div class="line" id="LC14"><span class="n">line</span> <span class="o">=</span> <span class="n">ser</span><span class="o">.</span><span class="n">readLine</span><span class="p">()</span></div><div class="line" id="LC15"><span class="k">print</span> <span class="n">line</span></div><div class="line" id="LC16"><br /></div><div class="line" id="LC17"><span class="n">Arduino</span> <span class="p">(</span><span class="n">put</span> <span class="n">this</span> <span class="ow">in</span> <span class="n">your</span> <span class="n">PDE</span><span class="p">):</span></div><div class="line" id="LC18"><br /></div><div class="line" id="LC19"><span class="n">unsigned</span> <span class="nb">long</span> <span class="n">readULongFromBytes</span><span class="p">()</span> <span class="p">{</span></div><div class="line" id="LC20">  <span class="n">union</span> <span class="n">u_tag</span> <span class="p">{</span></div><div class="line" id="LC21">    <span class="n">byte</span> <span class="n">b</span><span class="p">[</span><span class="mi">4</span><span class="p">];</span></div><div class="line" id="LC22">    <span class="n">unsigned</span> <span class="nb">long</span> <span class="n">ulval</span><span class="p">;</span></div><div class="line" id="LC23">  <span class="p">}</span> <span class="n">u</span><span class="p">;</span></div><div class="line" id="LC24">  <span class="n">u</span><span class="o">.</span><span class="n">b</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">Serial</span><span class="o">.</span><span class="n">read</span><span class="p">();</span></div><div class="line" id="LC25">  <span class="n">u</span><span class="o">.</span><span class="n">b</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">Serial</span><span class="o">.</span><span class="n">read</span><span class="p">();</span></div><div class="line" id="LC26">  <span class="n">u</span><span class="o">.</span><span class="n">b</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="n">Serial</span><span class="o">.</span><span class="n">read</span><span class="p">();</span></div><div class="line" id="LC27">  <span class="n">u</span><span class="o">.</span><span class="n">b</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="o">=</span> <span class="n">Serial</span><span class="o">.</span><span class="n">read</span><span class="p">();</span></div><div class="line" id="LC28">  <span class="k">return</span> <span class="n">u</span><span class="o">.</span><span class="n">ulval</span><span class="p">;</span></div><div class="line" id="LC29"><span class="p">}</span></div><div class="line" id="LC30"><span class="n">unsigned</span> <span class="nb">long</span> <span class="n">val</span> <span class="o">=</span> <span class="n">readULongFromBytes</span><span class="p">();</span></div><div class="line" id="LC31"><span class="n">Serial</span><span class="o">.</span><span class="k">print</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">DEC</span><span class="p">);</span> <span class="o">//</span> <span class="n">send</span> <span class="n">to</span> <span class="n">python</span> <span class="n">to</span> <span class="n">check</span></div></pre></div> </td> </tr> </table> </div></p> </p> <p><a href="http://blog.stavi.sh/python-code-that-writes-a-multibyte-value-lon">Permalink</a> | <a href="http://blog.stavi.sh/python-code-that-writes-a-multibyte-value-lon#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/480etBo8Grg" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/python-code-that-writes-a-multibyte-value-lon Wed, 11 Aug 2010 08:42:00 -0700 Visualizing Public Policy Polling Survey Data Of Delaware Voters (August 7-8, 2010) http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/CkM4f4JDJSw/analyzing-and-visualizing-public-policy-polli http://blog.stavi.sh/analyzing-and-visualizing-public-policy-polli <p> <p>The Daily Kos <a href="http://www.dailykos.com/storyonly/2010/8/10/11161/4340/280/891639" style="text-decoration: none;">posted</a> commentary and analysis of a recent Public Policy Polling <a href="http://www.dailykos.com/images/user/3/DelawareResults.pdf" style="text-decoration: none;">survey</a> of Delaware voters. The <a href="http://www.dailykos.com/images/user/3/DelawareRawData2.csv" style="text-decoration: none;">raw data</a> was also posted. I loaded the raw data into <a href="http://www.socrata.com" style="text-decoration: none;">Socrata’s</a> social data platform to do some quick visualizations and corroborate the analysis.</p> <p>After loading the raw data CSV, the first thing that I did was add <a href="http://en.wikipedia.org/wiki/Metadata" style="text-decoration: none;">meta-data</a> into the column labels in Socrata. To do this, I cut and pasted the text descriptions of the questions from the survey PDF to the original raw data column label. You can see the meta information when you hover-over the columns in the <a href="http://socrata.com/d/j7zu-5pf9" style="text-decoration: none;">tabular view</a> or in the <a href="http://www.stavi.sh/downloads/PPPDE-meta.jpg" style="text-decoration: none;">dataset summary</a>; which also includes usage statistics, attribution to <a href="http://www.publicpolicypolling.com/" style="text-decoration: none;">PPP</a>, and the orignal PDF and CSV (embedded as attachments).</p> <p>The benefit of adding meta-data is that anyone that looks at the raw data can now understand the values of the columns quickly. Additionally, in Socrata, the meta-information is decoupled from the actual data, so the data values can change without losing the semantic labeling. Generally, the meta information and data can be combined as part of the ingest process, the manual step was only required for this data-set because of the nature of the existing information. Note, my preferred choice for ingesting is via automated workflow using the web services <a href="http://dev.socrata.com/" style="text-decoration: none;">API</a>.</p> <p><a href="http://twitter.com/markos" style="text-decoration: none;">Markos’</a> commentary focused on questions 8 and 9 respectively:</p> <ul> <li>Q8: If the candidates for US Senate this fall were Republican Mike Castle and Democrat Chris Coons, who would you vote for? </li> <li>Q9: If the candidates for US Senate this fall were Republican Christine O’Donnell and Democrat Chris Coons, who would you vote for? </li> </ul> <p>To verify the voting result percentages from the survey raw data, I grouped the Question 8 results by vote calculating the sum and percentage (this can all be done in the Socrata User Interface). I could not find the integer vote choice definitions, therefore I had to infer the values from the summary information in the PDF report. Excluding rounding errors, the results are the same, as can be seen in the Social Data Player embedded below:</p> <div><iframe scrolling="no" src="http://www.socrata.com/w/supb-9uvn/y34g-bnf3?cur=bfBOM8TZOdt&from=root" frameborder="0" height="230px" width="500px"><a href="http://www.socrata.com/Government/PPP-Survey-If-the-DE-candidates-for-US-Senate-this/supb-9uvn" title="PPP Survey: If the DE candidates for US Senate this fall were Republican Mike Castle and Democrat Chris Coons, who would you vote for?" target="_blank">PPP Survey: If the DE candidates for US Senate this fall were Republican Mike Castle and Democrat Chris Coons, who would you vote for?</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> <p>For visualization choices, I plotted Coons vs Castle on total voter favorability:</p> <div><iframe scrolling="no" src="http://www.socrata.com/w/4cv4-kwx2/y34g-bnf3?cur=UPIezVtAdU5&from=root" frameborder="0" height="425px" width="500px"><a href="http://www.socrata.com/dataset/Coons-D-vs-Castle-R-Favorability-Chart/4cv4-kwx2" title="Coons (D) vs Castle (R) Favorability Chart" target="_blank">Coons (D) vs Castle (R) Favorability Chart</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> <p>Chis Coons charted against favorability by party:</p> <div><iframe scrolling="no" src="http://www.socrata.com/w/v9y5-9wrt/y34g-bnf3?cur=I6jxaV0QfqJ&from=root" frameborder="0" height="425px" width="500px"><a href="http://www.socrata.com/dataset/Chris-Coons-D-Favorability-By-Party/v9y5-9wrt" title="Chris Coons (D) Favorability By Party" target="_blank">Chris Coons (D) Favorability By Party</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> <p /> <p>Disclaimer: I have no affiliation or preference for any candidate in any State of Delaware political party. My goal for this write-up was to demonstrate how to visualize and share political data. If you have any questions on how to do any of these visualizations, please submit via <a href="http://twitter.com/toddstavish" style="text-decoration: none;">@toddstavish</a>.</p> <p> </p> </p> <p><a href="http://blog.stavi.sh/analyzing-and-visualizing-public-policy-polli">Permalink</a> | <a href="http://blog.stavi.sh/analyzing-and-visualizing-public-policy-polli#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/CkM4f4JDJSw" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/analyzing-and-visualizing-public-policy-polli Sat, 07 Aug 2010 15:41:00 -0700 Python Arduino Serial Port Text Communication (Send from PDE; Receive via PySerial) http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/Zt5EoaovfYI/python-arduino-serial-port-text-communication http://blog.stavi.sh/python-arduino-serial-port-text-communication <p> <p>Test code to make sure Python and Arduino are working properly. The PDE sends a string out of the serial port. The Python code uses pyserial to receive it and then prints the message to standard out. Pick it up at GitHub via “git clone git://github.com/toddstavish/Python-Arduino-Serial-Text-Send-Receive.git” Any questions or suggestions, you can find me <a href="http://twitter.com/toddstavish" style="color: #660000;">here</a>. </p> </p> <p><a href="http://blog.stavi.sh/python-arduino-serial-port-text-communication">Permalink</a> | <a href="http://blog.stavi.sh/python-arduino-serial-port-text-communication#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/Zt5EoaovfYI" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/python-arduino-serial-port-text-communication Wed, 04 Aug 2010 12:29:00 -0700 Country Spreadsheet List With Latitude, Longitude, ISO 3166 Codes, and Flag Images http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/KE53ZNSchd0/country-list-iso-3166-codes-latitude-longitud http://blog.stavi.sh/country-list-iso-3166-codes-latitude-longitud <p> <p>Spreadsheet to help create <a href="http://www.socrata.com/dataset/State-Heat-Map/eptn-wnzg">heatmaps</a> in Socrata or Google <a href="http://code.google.com/apis/visualization/documentation/gallery/geomap.html">geomap</a> visualizations.</p> <p><strong>Spreadsheet download:</strong></p> <p><a href="http://stavi.sh/downloads/Countries.xls" target="_blank">Spreadsheet with flag images (Countries.xls)</a></p> <p><strong>Google Docs:</strong></p> <p><a href="http://spreadsheets.google.com/ccc?key=0At92oU3FPZ4QdEIwVV8tZGJHRmV6VXV3LWpfQThiUnc&hl=en" target="_blank">Country List Latitude Longitude ISO 3166 Codes</a></p> <p><strong>Socrata:</strong></p> <p><a href="http://www.socrata.com/dataset/Country-List-ISO-3166-Codes-Latitude-Longitude/mnkm-8ram" target="_blank">Country List ISO 3166 Codes Latitude Longitude</a></p> <p><strong>Scribd:</strong></p> <p><a href="http://www.scribd.com/doc/35371527/Country-List-Latitude-Longitude-Flags-ISO-3166-Codes" target="_blank">Country List Latitude Longitude Flags ISO 3166 Codes</a></p> <p><strong>Previews:</strong></p> <div><iframe scrolling="no" src="http://www.socrata.com/w/mnkm-8ram/y34g-bnf3?cur=rDrJsGpNRlF&from=root" frameborder="0" height="646px" width="760px">&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;a href="http://www.socrata.com/dataset/Country-List-ISO-3166-Codes-Latitude-Longitude/mnkm-8ram" _mce_href="http://www.socrata.com/dataset/Country-List-ISO-3166-Codes-Latitude-Longitude/mnkm-8ram" title="Country List ISO 3166 Codes Latitude Longitude" target="_blank"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;Country List ISO 3166 Codes Latitude Longitude&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/a&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;</iframe></div> <p><a href="http://www.scribd.com/doc/35371527/Country-List-Latitude-Longitude-Flags-ISO-3166-Codes" title="View Country List Latitude Longitude Flags ISO 3166 Codes on Scribd" style="margin: 12px auto 6px auto; font-family: Helvetica,Arial,Sans-serif; font-style: normal; font-variant: normal; font-weight: normal; font-size: 14px; line-height: normal; display: block; text-decoration: underline;">Country List Latitude Longitude Flags ISO 3166 Codes </a> <object data="http://d1.scribdassets.com/ScribdViewer.swf" type="application/x-shockwave-flash" height="500" width="500"> <param name="movie" value="http://d1.scribdassets.com/ScribdViewer.swf" /> <param name="wmode" value="opaque" /> <param name="bgcolor" value="#ffffff" /> <param name="allowFullScreen" value="true" /> <param name="allowScriptAccess" value="always" /> <param name="FlashVars" value="document_id=35371527&access_key=key-1z7xcoyeca61iydjocwg&page=1&viewMode=list" /> <embed src="http://d1.scribdassets.com/ScribdViewer.swf?document_id=35371527&access_key=key-1z7xcoyeca61iydjocwg&page=1&viewMode=list" type="application/x-shockwave-flash" wmode="opaque" height="500" width="500"></embed> </object> </p> </p> <p><a href="http://blog.stavi.sh/country-list-iso-3166-codes-latitude-longitud">Permalink</a> | <a href="http://blog.stavi.sh/country-list-iso-3166-codes-latitude-longitud#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/KE53ZNSchd0" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/country-list-iso-3166-codes-latitude-longitud Wed, 28 Jul 2010 14:35:00 -0700 Polyglot Persistence: Integrating Low-Latency NoSQL Systems (Cassandra and InfiniteGraph) http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/P2t4_twX30I/polyglot-persistence-integrating-low-latency http://blog.stavi.sh/polyglot-persistence-integrating-low-latency <p> <p>Cassandra’s <a href="http://architects.dzone.com/news/cassandra-adds-hadoop">integration</a> to Hadoop’s MapReduce is a successful example of two overlapping NoSQL data-stores sharing common features. However, there are some features that are challenging to support across different NoSQL architectures, for instance the expression of the datamodel. Similar problems have been solved before in the programming world with the concept of <a href="http://polyglotprogramming.com/">polyglot programming</a>. The database analog, <a href="http://www.infoq.com/news/2009/07/leberknight-polyglot-persistence">polyglot persistence</a>, is now gaining widespread acceptance as well. In an effort to learn more about NoSQL and polyglot storage patterns, I began to wonder if it would be possible to extract data from Cassandra for analysis in a graph database. In other-words, implement my own polyglot persistence application by fusing <a href="http://www.infinitegraph.com/">InfiniteGraph</a> and <a href="http://cassandra.apache.org/">Cassandra</a>. In order to understand what each data-store can give to the other, I want to try to define their commonalities. InfiniteGraph and Cassandra share similarities in their underlying architecture. Both data-stores offer:</p> <ol> <li>distributed server nodes (for scalability and availability)</li> <li>based on proven underlying technologies <a href="http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html">Dynamo</a> and <a href="http://www.objectivity.com/">Objectivity</a></li> <li>configurability for both disk and memory driven deployment</li> <li>tuning features for durability, consistency, read / write performance</li> <li>avoidance of JOINs</li> <li>low-latency solutions</li> </ol> <p>The direct benefit of these common features are linear scalability, massive write performance, and low operational cost. There are some notable differences between InfiniteGraph and Cassandra though. Presently, Cassandra has more choices for asynchronous operations and the availability options are more mature. Both are considered to be on the rich end of the NoSQL datamodel spectrum (simple key-value stores on the other end). My interpretation is that InfiniteGraph’s model and programming interface are more intuitive to the general object oriented programmer. Especially, if your use-cases are <a href="http://bit.ly/bOCy4Q">graph oriented</a>: Friend Of A Friend (FOAF) analysis, recommendation engines, fraud detection, correlating IP addresses among log files, clickstream or ad-servering metrics, etc. </p> <p>Caveat: I have tried hard to understand each database; the fact is that both of these data-stores are quite comprehensive and summarizing their capabilities is not trivial. Additionally, both data-stores are under active development. I expect InfiniteGraph’s asynchronous features will expand and Cassandra’s rich client interface will improve as Avro becomes available. </p> <p>In general merging of the two datastores was straight-forward and useful. After extracting the ‘friends’ columns out of Cassandra, I used InfiniteGraph to perform path analysis to find a common friend amongst friends. A feature of the hybrid system that was particularly useful was the ability to to learn from the graph and update the original Cassandra store. For instance, learning a shorter path between friends in InfiniteGraph and then updating the columns in Cassandra. For a friend suggestion application, the shortest might be the critical one. However, InfiniteGraph can hop vertexes and edges quickly. Therefore, calculating the longer paths was possible too. Alternative longer path are useful for certain applications like tracking money laundering or other law enforcement needs. In general, I found that I could easily try multi-path analysis, update Cassandra, then run my original queries in Cassandra over the new data. As a potential followup post, I plan to try to use my new hybrid NoSQL database as a caching system. Using InfiniteGraph’s streaming capabilities to augment Cassandra’s <a href="http://wiki.apache.org/cassandra/CassandraLimitations">limitation</a>. </p> <p> The working prototype can be found in GitHub via “git clone <a href="mailto:git@github.com">git@github.com</a>:toddstavish/Cassandra-Graph-Extract.git.” If you have any questions or suggestions, you can find me <a href="http://twitter.com/toddstavish">here</a>. I look forward to trying this <a href="http://github.com/toddstavish/Cassandra-Graph-Extract">code base</a> against a larger dataset with multinode deployments on both sides of the InfiniteGraph / Cassandra equation. </p> </p> <p><a href="http://blog.stavi.sh/polyglot-persistence-integrating-low-latency">Permalink</a> | <a href="http://blog.stavi.sh/polyglot-persistence-integrating-low-latency#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/P2t4_twX30I" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/polyglot-persistence-integrating-low-latency Thu, 22 Jul 2010 06:53:00 -0700 Building a Cassandra Application with Maven http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/9cL9ak4G5QE/building-a-cassandra-application-with-maven http://blog.stavi.sh/building-a-cassandra-application-with-maven <p> <p>Here are the steps to get the <a href="http://en.wikipedia.org/wiki/NoSQL">NoSQL</a> data store, <a href="http://cassandra.apache.org/">Cassandra</a>, working with a <a href="http://maven.apache.org/index.html">Maven</a> project management environment. I used <a href="http://twitter.com/sodesocompany/">Ronald Mathies'</a> sample Cassandra <a href="http://www.sodeso.nl/?p=251">application</a> for the code base.</p> <ol> <li>mvn archetype:create -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId=nl.sodeso.blog -DartifactId=Authors</li> <li>Use this <a href="http://gist.github.com/484659">POM</a></li> <li>Copy Author.java to src/main/java/nl/sodeso/blog</li> <li>mvn compile</li> <li>/opt/apache-cassandra-0.6.3/bin/cassandra -f</li> <li>mvn exec:exec</li> </ol> <p>If you are new to Maven, it might be easier to get the entirety from GitHub via “git clone <a href="mailto:git@github.com">git@github.com</a>:toddstavish/Cassandra-Maven-Setup.git.” If you have any questions or suggestions, you can find me <a href="http://twitter.com/toddstavish">here</a>.</p> </p> <p><a href="http://blog.stavi.sh/building-a-cassandra-application-with-maven">Permalink</a> | <a href="http://blog.stavi.sh/building-a-cassandra-application-with-maven#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/9cL9ak4G5QE" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/building-a-cassandra-application-with-maven Fri, 16 Jul 2010 17:05:36 -0700 Socrata on the EPA homepage http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/eB2SWyAA-mU/socrata-on-the-epa-homepage http://blog.stavi.sh/socrata-on-the-epa-homepage <p> <div class='p_embed p_image_embed'> <a href="http://posterous.com/getfile/files.posterous.com/toddstavish/HxRNGW67k8Ptx7LYZPc8NtXSw0afEfjQCsRGvhP4XeatMxjUikHECVzXtvkj/US_Environmental_Protection_Ag.png"><img alt="Us_environmental_protection_ag" height="369" src="http://posterous.com/getfile/files.posterous.com/toddstavish/Q5Tt5ehxdDyMpN76DeglkOQjxIVTov5qh4Z52IlXJlBgVoGBz33JuVlPOzl4/US_Environmental_Protection_Ag.png.scaled.500.jpg" width="500" /></a> </div> </p> <p><a href="http://blog.stavi.sh/socrata-on-the-epa-homepage">Permalink</a> | <a href="http://blog.stavi.sh/socrata-on-the-epa-homepage#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/eB2SWyAA-mU" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/socrata-on-the-epa-homepage Fri, 16 Jul 2010 09:56:00 -0700 Building an InfiniteGraph Application with Maven http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/Z0ZCY9RK7u0/infinitegraph-maven-on-osx http://blog.stavi.sh/infinitegraph-maven-on-osx <p> <p>Here are the steps to get the graph data store, <a href="http://www.infinitegraph.com/">InfiniteGraph</a>, working with a <a href="http://maven.apache.org/">Maven</a> project management environment. The <a href="http://blog.stavi.sh/infinitegraph-first-look">codebase</a> is a small social network application.</p> <ol> <li>mvn archetype:create -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId=com.infinitegraph.sample -DartifactId=GraphAPISample</li> <li>Use this <a href="http://gist.github.com/484436">POM w/ filesystem dependency</a></li> <li>Copy GraphAPISample.java to src/main/java/com/infinitegraph/sample</li> <li>Add GraphAPISample to package com.infinitegraph.sample</li> <li>Create a resources directory for your properties file with ‘mkdir src/main/resources’</li> <li>Copy SampleGraphProperties.properties to the resources directory. Note, this will move to the target directory during the compile step, so it can be picked up at runtime without path qualification. Therefore, take out the path in your code, for example: GraphFactory.open(“SampleGraph”, “SampleGraphProperties.properties”);</li> <li>mvn compile</li> <li>mvn exec:exec</li> </ol> <p>Make sure you have the <a href="http://blog.infinitegraph.com/2010/07/16/concurrency-control-lockserver-ls/">distributed lockserver</a> running in advance and source the <a href="http://stavi.sh/demos/infinitegraph/mvn/IGConfig">IGConfg script</a>. If you are new to Maven, it might be easier to download <a href="http://stavi.sh/demos/infinitegraph/mvn/GraphAPISample.zip">my whole setup</a>. Or get it from GitHub via “git clone <a href="mailto:git@github.com">git@github.com</a>:toddstavish/InfiniteGraph-Maven-Example-Setup.git.” If you have any suggestions or questions, you can reach me <a href="http://twitter.com/toddstavish">here</a>.</p> </p> <p><a href="http://blog.stavi.sh/infinitegraph-maven-on-osx">Permalink</a> | <a href="http://blog.stavi.sh/infinitegraph-maven-on-osx#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/Z0ZCY9RK7u0" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/infinitegraph-maven-on-osx Wed, 14 Jul 2010 07:05:00 -0700 InfiniteGraph: Memory Model And Graph Partitioning http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/EpLP2sryuwY/infinitegraph-memory-model-and-graph-partitio http://blog.stavi.sh/infinitegraph-memory-model-and-graph-partitio <p> <p>A few questions have arisen on the <a href="http://bit.ly/9XvzXQ">partitioning</a> / <a href="http://bit.ly/dg44nr">sharding</a> / <a href="http://en.wikipedia.org/wiki/Software_distribution">distribution</a> features of the <a href="http://www.infinitegraph.com/">InfiniteGraph</a> <a href="http://bit.ly/aigcbj">demo code</a>. I thought I’d take a moment to help explain it better.</p> <p>First, <a href="http://www.linkedin.com/in/arvindmg">Arvind</a> posted a good description on InfiniteGraph’s <a href="http://bit.ly/9YP0w6">underlying plumbing</a>. InfiniteGraph is based on <a href="http://www.objectivity.com/">Objectivity</a>. Kind of like other <a href="http://en.wikipedia.org/wiki/Low_latency">low latency</a> database architectures such as <a href="http://hbase.apache.org/">HBase</a> on <a href="http://hadoop.apache.org/">Hadoop</a> or <a href="http://cassandra.apache.org/">Cassandra</a> on <a href="http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html">Dynamo</a>.</p> <p>Underneath an InfiniteGraph database instance is an Objectivity federated database. Objectivity has deep experience building <a href="http://www.objectivity.com/pages/downloads/whitepaper/html/slac.html">petascale database architectures</a>. Logically, a federated database, looks like a single database instance. Databases can be distributed accross servers or disks with all of the logical to physical mapping done by the database engine. Each database is a peer so there is no shared query engine or centralized server. Threads have their own database cache, so database requests stream directly from servers to threads without contention. The federated database model allows for edges to transparently span databases and subgraphs are delivered in pages to optimize I/O and network traffic.</p> <p>InfiniteGraph abstracts this idea even further programmatically. There is no thought of distribution in the application code. The placement managers take care of it all. The plan is that InfiniteGraph will provide a number of common placement algorithms. You will have plugin classes that are chosen at runtime from the configuration properties. The default multidatabase placement manager that I used in my example, basically streams the graph in sequentially per thread. The database the default multidatabase placement chooses depends on the capacity of storage remaining, or if there is an <a href="http://blog.infinitegraph.com/2010/07/16/concurrency-control-lockserver-ls/">open lock</a>.</p> <p>InfinteGraph’s partitioning is not a magic bullet. Partitioning will open doors scalability-wise for graph persistence and optimize a difficult problem. However, there are <a href="http://en.wikipedia.org/wiki/Trade-off">tradeoffs</a> that need to be made moving from an embedded graph instance to a distributed one, for instance, colocation or clustering advantages. If you have any suggestions or questions, you can reach me <a href="http://twitter.com/toddstavish">here</a>.</p> </p> <p><a href="http://blog.stavi.sh/infinitegraph-memory-model-and-graph-partitio">Permalink</a> | <a href="http://blog.stavi.sh/infinitegraph-memory-model-and-graph-partitio#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/EpLP2sryuwY" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/infinitegraph-memory-model-and-graph-partitio Tue, 13 Jul 2010 05:17:00 -0700 Social Graph Persistence In A Java Graph Database http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/jTMYZi9DA5E/social-graph-persistence-in-a-java-graph-data http://blog.stavi.sh/social-graph-persistence-in-a-java-graph-data <p> <p>Graph databases are excellent for storing and analyzing social network information. However, I could not find graphDB sample code that generates more than a dozen or so vertices and edges. In this example, I will demonstrate how to construct and store a synthetic social graph of 9,100,260 (vertices, edges, and properties). The synthetic graph has 151,671 people nodes which is a hard limit based on the number of artificial names that I generated. However, you could increase the edge connections or edge properties to expand the graph further, both are configurable variables.</p> <p>To generate the artificial names, I downloaded <a href="http://bit.ly/db9cGb">Census name lists</a> and combined the male and female first names with the last names list. You can download the completed list <a href="http://stavi.sh/downloads/names.txt">here</a>.</p> <p>The persistent graph is distributed across multiple database instances (aka graph partitioning or graph sharding) to provide scalability. To build the sharded graph, I downloaded and installed the <a href="http://www.infinitegraph.com/">InfiniteGraph Java Graph Database</a>. The next step is to set placement strategies that allow for multidatabase placement (set in the <a href="http://gist.github.com/472032">properties file</a> for the graph). Graph placement is configured at runtime with customization possible by implementing your own placement classes. I just used the <a href="http://post.ly/n9fQ">default multidatabase placement</a>.</p> <p>Here are the exact properties that need to be set for multidatabase placement:</p> <p><div class="data type-text"> <table class="lines" cellspacing="0" cellpadding="0"> <tr> <td> <pre class="line_numbers"><span rel="#L1" id="L1">1</span> <span rel="#L2" id="L2">2</span> <span rel="#L3" id="L3">3</span> </pre> </td> <td width="100%"> <div class="highlight"><pre /><div class="line" id="LC1"># Placement Strategy </div><div class="line" id="LC2">InfiniteGraph.Placement.ImplClass=com.objy.graph.impl.plugins.mdp.MultipleDatabasePlacement</div><div class="line" id="LC3">InfiniteGraph.Placement.MultipleDatabasePlacement.ConfigString=;1,;1</div></pre></div> </td> </tr> </table> </div></p> <p>The code loops through the name list file generating people vertices. A second loop goes through the people nodes connecting them together using a random algorithm. The social network context is one person pays another with different kinds of transactions: cash, wire, etc. The relationship edges between people are <em>paysTo</em> with a polymorphic collection of <em>TransactionType</em>. The collection along the edge stores instances of the specific payment type between the sender and recipient.</p> <p>To summarize what this sample shows:</p> <ul> <li><p>Creating a large synthetic social graph</p></li> <li><p>Edges with polymorphic collections</p></li> <li><p>Graph sharding over multiple databases</p></li> </ul> <p>Code:</p> <p><div class="data type-java"> <table class="lines" cellspacing="0" cellpadding="0"> <tr> <td> <pre class="line_numbers"><span rel="#L1" id="L1">1</span> <span rel="#L2" id="L2">2</span> <span rel="#L3" id="L3">3</span> <span rel="#L4" id="L4">4</span> <span rel="#L5" id="L5">5</span> <span rel="#L6" id="L6">6</span> <span rel="#L7" id="L7">7</span> <span rel="#L8" id="L8">8</span> <span rel="#L9" id="L9">9</span> <span rel="#L10" id="L10">10</span> <span rel="#L11" id="L11">11</span> <span rel="#L12" id="L12">12</span> <span rel="#L13" id="L13">13</span> <span rel="#L14" id="L14">14</span> <span rel="#L15" id="L15">15</span> <span rel="#L16" id="L16">16</span> <span rel="#L17" id="L17">17</span> <span rel="#L18" id="L18">18</span> <span rel="#L19" id="L19">19</span> <span rel="#L20" id="L20">20</span> <span rel="#L21" id="L21">21</span> <span rel="#L22" id="L22">22</span> <span rel="#L23" id="L23">23</span> <span rel="#L24" id="L24">24</span> <span rel="#L25" id="L25">25</span> <span rel="#L26" id="L26">26</span> <span rel="#L27" id="L27">27</span> <span rel="#L28" id="L28">28</span> <span rel="#L29" id="L29">29</span> <span rel="#L30" id="L30">30</span> <span rel="#L31" id="L31">31</span> <span rel="#L32" id="L32">32</span> <span rel="#L33" id="L33">33</span> <span rel="#L34" id="L34">34</span> <span rel="#L35" id="L35">35</span> <span rel="#L36" id="L36">36</span> <span rel="#L37" id="L37">37</span> <span rel="#L38" id="L38">38</span> <span rel="#L39" id="L39">39</span> <span rel="#L40" id="L40">40</span> <span rel="#L41" id="L41">41</span> <span rel="#L42" id="L42">42</span> <span rel="#L43" id="L43">43</span> <span rel="#L44" id="L44">44</span> <span rel="#L45" id="L45">45</span> <span rel="#L46" id="L46">46</span> <span rel="#L47" id="L47">47</span> <span rel="#L48" id="L48">48</span> <span rel="#L49" id="L49">49</span> <span rel="#L50" id="L50">50</span> <span rel="#L51" id="L51">51</span> <span rel="#L52" id="L52">52</span> <span rel="#L53" id="L53">53</span> <span rel="#L54" id="L54">54</span> <span rel="#L55" id="L55">55</span> <span rel="#L56" id="L56">56</span> <span rel="#L57" id="L57">57</span> <span rel="#L58" id="L58">58</span> <span rel="#L59" id="L59">59</span> <span rel="#L60" id="L60">60</span> <span rel="#L61" id="L61">61</span> <span rel="#L62" id="L62">62</span> <span rel="#L63" id="L63">63</span> <span rel="#L64" id="L64">64</span> <span rel="#L65" id="L65">65</span> <span rel="#L66" id="L66">66</span> <span rel="#L67" id="L67">67</span> <span rel="#L68" id="L68">68</span> <span rel="#L69" id="L69">69</span> <span rel="#L70" id="L70">70</span> <span rel="#L71" id="L71">71</span> <span rel="#L72" id="L72">72</span> <span rel="#L73" id="L73">73</span> <span rel="#L74" id="L74">74</span> <span rel="#L75" id="L75">75</span> <span rel="#L76" id="L76">76</span> <span rel="#L77" id="L77">77</span> <span rel="#L78" id="L78">78</span> <span rel="#L79" id="L79">79</span> <span rel="#L80" id="L80">80</span> <span rel="#L81" id="L81">81</span> <span rel="#L82" id="L82">82</span> <span rel="#L83" id="L83">83</span> <span rel="#L84" id="L84">84</span> <span rel="#L85" id="L85">85</span> <span rel="#L86" id="L86">86</span> <span rel="#L87" id="L87">87</span> <span rel="#L88" id="L88">88</span> <span rel="#L89" id="L89">89</span> <span rel="#L90" id="L90">90</span> <span rel="#L91" id="L91">91</span> <span rel="#L92" id="L92">92</span> <span rel="#L93" id="L93">93</span> <span rel="#L94" id="L94">94</span> <span rel="#L95" id="L95">95</span> <span rel="#L96" id="L96">96</span> <span rel="#L97" id="L97">97</span> <span rel="#L98" id="L98">98</span> <span rel="#L99" id="L99">99</span> <span rel="#L100" id="L100">100</span> <span rel="#L101" id="L101">101</span> <span rel="#L102" id="L102">102</span> <span rel="#L103" id="L103">103</span> <span rel="#L104" id="L104">104</span> <span rel="#L105" id="L105">105</span> <span rel="#L106" id="L106">106</span> <span rel="#L107" id="L107">107</span> <span rel="#L108" id="L108">108</span> <span rel="#L109" id="L109">109</span> <span rel="#L110" id="L110">110</span> <span rel="#L111" id="L111">111</span> <span rel="#L112" id="L112">112</span> <span rel="#L113" id="L113">113</span> <span rel="#L114" id="L114">114</span> <span rel="#L115" id="L115">115</span> <span rel="#L116" id="L116">116</span> <span rel="#L117" id="L117">117</span> <span rel="#L118" id="L118">118</span> <span rel="#L119" id="L119">119</span> <span rel="#L120" id="L120">120</span> <span rel="#L121" id="L121">121</span> <span rel="#L122" id="L122">122</span> <span rel="#L123" id="L123">123</span> <span rel="#L124" id="L124">124</span> <span rel="#L125" id="L125">125</span> <span rel="#L126" id="L126">126</span> <span rel="#L127" id="L127">127</span> <span rel="#L128" id="L128">128</span> <span rel="#L129" id="L129">129</span> <span rel="#L130" id="L130">130</span> <span rel="#L131" id="L131">131</span> <span rel="#L132" id="L132">132</span> <span rel="#L133" id="L133">133</span> <span rel="#L134" id="L134">134</span> <span rel="#L135" id="L135">135</span> <span rel="#L136" id="L136">136</span> <span rel="#L137" id="L137">137</span> <span rel="#L138" id="L138">138</span> <span rel="#L139" id="L139">139</span> <span rel="#L140" id="L140">140</span> <span rel="#L141" id="L141">141</span> <span rel="#L142" id="L142">142</span> <span rel="#L143" id="L143">143</span> <span rel="#L144" id="L144">144</span> <span rel="#L145" id="L145">145</span> <span rel="#L146" id="L146">146</span> <span rel="#L147" id="L147">147</span> <span rel="#L148" id="L148">148</span> </pre> </td> <td width="100%"> <div class="highlight"><pre /><div class="line" id="LC1"><span class="kn">import</span> <span class="nn">java.io.IOException</span><span class="o">;</span></div><div class="line" id="LC2"><span class="kn">import</span> <span class="nn">java.util.ArrayList</span><span class="o">;</span></div><div class="line" id="LC3"><span class="kn">import</span> <span class="nn">java.util.Iterator</span><span class="o">;</span></div><div class="line" id="LC4"><span class="kn">import</span> <span class="nn">java.util.Random</span><span class="o">;</span></div><div class="line" id="LC5"><br /></div><div class="line" id="LC6"><span class="kn">import</span> <span class="nn">com.objy.db.app.ooObj</span><span class="o">;</span></div><div class="line" id="LC7"><span class="kn">import</span> <span class="nn">com.objy.db.util.ooMap</span><span class="o">;</span></div><div class="line" id="LC8"><span class="kn">import</span> <span class="nn">com.objy.graph.AccessMode</span><span class="o">;</span></div><div class="line" id="LC9"><span class="kn">import</span> <span class="nn">com.objy.graph.BaseEdge</span><span class="o">;</span></div><div class="line" id="LC10"><span class="kn">import</span> <span class="nn">com.objy.graph.BaseVertex</span><span class="o">;</span></div><div class="line" id="LC11"><span class="kn">import</span> <span class="nn">com.objy.graph.EdgeKind</span><span class="o">;</span></div><div class="line" id="LC12"><span class="kn">import</span> <span class="nn">com.objy.graph.GraphDatabase</span><span class="o">;</span></div><div class="line" id="LC13"><span class="kn">import</span> <span class="nn">com.objy.graph.GraphFactory</span><span class="o">;</span></div><div class="line" id="LC14"><span class="kn">import</span> <span class="nn">com.objy.graph.Transaction</span><span class="o">;</span></div><div class="line" id="LC15"><br /></div><div class="line" id="LC16"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">ShardedSocialGraph</span></div><div class="line" id="LC17"><span class="o">{</span> </div><div class="line" id="LC18"><br /></div><div class="line" id="LC19"> <span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="n">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span></div><div class="line" id="LC20"> <span class="o">{</span></div><div class="line" id="LC21"> <span class="kt">int</span> <span class="n">numberOfPeople</span> <span class="o">=</span> <span class="mi">50000</span><span class="o">;</span> <span class="c1">// Max: 151,671</span></div><div class="line" id="LC22"> <span class="kt">int</span> <span class="n">numberOfRecipients</span> <span class="o">=</span> <span class="mi">10</span><span class="o">;</span></div><div class="line" id="LC23"> <span class="n">String</span> <span class="n">graphName</span> <span class="o">=</span> <span class="s">"ShardedGraph"</span><span class="o">;</span></div><div class="line" id="LC24"> <span class="n">String</span> <span class="n">props</span> <span class="o">=</span> <span class="s">"graph.properties"</span><span class="o">;</span></div><div class="line" id="LC25"> <span class="n">Transaction</span> <span class="n">tx</span> <span class="o">=</span> <span class="kc">null</span><span class="o">;</span></div><div class="line" id="LC26">     <span class="n">GraphDatabase</span> <span class="n">graphDB</span> <span class="o">=</span> <span class="kc">null</span><span class="o">;</span></div><div class="line" id="LC27"><br /></div><div class="line" id="LC28"> <span class="k">try</span></div><div class="line" id="LC29"> <span class="o">{</span></div><div class="line" id="LC30">         <span class="c1">// Create graph database</span></div><div class="line" id="LC31">         <span class="n">GraphFactory</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">graphName</span><span class="o">,</span> <span class="n">props</span><span class="o">);</span></div><div class="line" id="LC32">         <span class="n">graphDB</span> <span class="o">=</span> <span class="n">GraphFactory</span><span class="o">.</span><span class="na">open</span><span class="o">(</span><span class="n">graphName</span><span class="o">,</span> <span class="n">props</span><span class="o">);</span></div><div class="line" id="LC33">         </div><div class="line" id="LC34"> <span class="c1">// Generate people</span></div><div class="line" id="LC35"> <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Starting people ingest."</span><span class="o">);</span> </div><div class="line" id="LC36">     <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">names</span> <span class="o">=</span> <span class="n">FileLoader</span><span class="o">.</span><span class="na">load</span><span class="o">(</span><span class="s">"names.txt"</span><span class="o">,</span> <span class="n">numberOfPeople</span><span class="o">);</span> </div><div class="line" id="LC37">     <span class="n">Iterator</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">namesItr</span> <span class="o">=</span> <span class="n">names</span><span class="o">.</span><span class="na">iterator</span><span class="o">();</span></div><div class="line" id="LC38">     <span class="n">ArrayList</span><span class="o"><</span><span class="n">Long</span><span class="o">></span> <span class="n">people</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">Long</span><span class="o">>();</span></div><div class="line" id="LC39">     <span class="n">tx</span> <span class="o">=</span> <span class="n">graphDB</span><span class="o">.</span><span class="na">beginTransaction</span><span class="o">(</span><span class="n">AccessMode</span><span class="o">.</span><span class="na">READ_WRITE</span><span class="o">);</span></div><div class="line" id="LC40">     <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">peopleCnt</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">peopleCnt</span> <span class="o"><</span> <span class="n">numberOfPeople</span><span class="o">;</span> <span class="n">peopleCnt</span><span class="o">++)</span></div><div class="line" id="LC41">     <span class="o">{</span> </div><div class="line" id="LC42">     <span class="n">String</span> <span class="n">name</span><span class="o">=</span> <span class="n">namesItr</span><span class="o">.</span><span class="na">next</span><span class="o">();</span></div><div class="line" id="LC43">                        <span class="n">Person</span> <span class="n">person</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Person</span><span class="o">(</span><span class="n">name</span><span class="o">);</span></div><div class="line" id="LC44">                        <span class="n">graphDB</span><span class="o">.</span><span class="na">addVertex</span><span class="o">(</span><span class="n">person</span><span class="o">);</span></div><div class="line" id="LC45">                        <span class="n">people</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">person</span><span class="o">.</span><span class="na">getId</span><span class="o">());</span></div><div class="line" id="LC46">     <span class="o">}</span></div><div class="line" id="LC47"> <span class="n">tx</span><span class="o">.</span><span class="na">commit</span><span class="o">();</span></div><div class="line" id="LC48"> <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Ingested "</span> <span class="o">+</span> <span class="n">numberOfPeople</span> <span class="o">+</span> <span class="s">" people."</span><span class="o">);</span> </div><div class="line" id="LC49"><br /></div><div class="line" id="LC50">                  <span class="c1">// Generate synthetic social graph (random associations)</span></div><div class="line" id="LC51">     <span class="n">Random</span> <span class="n">random</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Random</span><span class="o">();</span></div><div class="line" id="LC52">     <span class="n">Iterator</span><span class="o"><</span><span class="n">Long</span><span class="o">></span> <span class="n">peopleItr</span> <span class="o">=</span> <span class="n">people</span><span class="o">.</span><span class="na">iterator</span><span class="o">();</span></div><div class="line" id="LC53"> <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Generating synthetic social graph."</span><span class="o">);</span></div><div class="line" id="LC54"> <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Connecting "</span> <span class="o">+</span> <span class="n">numberOfPeople</span> <span class="o">+</span> <span class="s">" senders with "</span> <span class="o">+</span> <span class="n">numberOfRecipients</span> <span class="o">+</span> <span class="s">" recipients."</span><span class="o">);</span></div><div class="line" id="LC55"> <span class="n">tx</span> <span class="o">=</span> <span class="n">graphDB</span><span class="o">.</span><span class="na">beginTransaction</span><span class="o">(</span><span class="n">AccessMode</span><span class="o">.</span><span class="na">READ_WRITE</span><span class="o">);</span></div><div class="line" id="LC56">     <span class="k">while</span> <span class="o">(</span><span class="n">peopleItr</span><span class="o">.</span><span class="na">hasNext</span><span class="o">())</span></div><div class="line" id="LC57">     <span class="o">{</span></div><div class="line" id="LC58"> <span class="n">Person</span> <span class="n">person1</span> <span class="o">=</span> <span class="o">(</span><span class="n">Person</span><span class="o">)</span><span class="n">graphDB</span><span class="o">.</span><span class="na">getVertex</span><span class="o">(</span><span class="n">people</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">random</span><span class="o">.</span><span class="na">nextInt</span><span class="o">(</span><span class="n">numberOfPeople</span><span class="o">)));</span></div><div class="line" id="LC59">     <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">numberOfRecipients</span><span class="o">;</span> <span class="n">i</span><span class="o">++)</span></div><div class="line" id="LC60">     <span class="o">{</span></div><div class="line" id="LC61">     <span class="n">Person</span> <span class="n">person2</span> <span class="o">=</span> <span class="o">(</span><span class="n">Person</span><span class="o">)</span><span class="n">graphDB</span><span class="o">.</span><span class="na">getVertex</span><span class="o">(</span><span class="n">people</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">random</span><span class="o">.</span><span class="na">nextInt</span><span class="o">(</span><span class="n">numberOfPeople</span><span class="o">)));</span></div><div class="line" id="LC62">     <span class="n">paysTo</span> <span class="n">pays</span> <span class="o">=</span> <span class="k">new</span> <span class="n">paysTo</span><span class="o">();</span></div><div class="line" id="LC63">     <span class="n">person1</span><span class="o">.</span><span class="na">addEdge</span><span class="o">(</span><span class="n">pays</span><span class="o">,</span> <span class="n">person2</span><span class="o">,</span> <span class="n">EdgeKind</span><span class="o">.</span><span class="na">BIDIRECTIONAL</span><span class="o">);</span></div><div class="line" id="LC64">     <span class="n">pays</span><span class="o">.</span><span class="na">addTransaction</span><span class="o">(</span><span class="k">new</span> <span class="n">Wire</span><span class="o">(),</span> <span class="s">"trx1"</span><span class="o">);</span></div><div class="line" id="LC65">     <span class="n">pays</span><span class="o">.</span><span class="na">addTransaction</span><span class="o">(</span><span class="k">new</span> <span class="n">Paypal</span><span class="o">(),</span> <span class="s">"trx2"</span><span class="o">);</span></div><div class="line" id="LC66">     <span class="n">pays</span><span class="o">.</span><span class="na">addTransaction</span><span class="o">(</span><span class="k">new</span> <span class="n">Cash</span><span class="o">(),</span> <span class="s">"trx3"</span><span class="o">);</span></div><div class="line" id="LC67">     <span class="o">}</span></div><div class="line" id="LC68">     <span class="n">peopleItr</span><span class="o">.</span><span class="na">next</span><span class="o">();</span></div><div class="line" id="LC69">     <span class="o">}</span></div><div class="line" id="LC70">            </div><div class="line" id="LC71">                   <span class="c1">// Commit</span></div><div class="line" id="LC72">                   <span class="n">tx</span><span class="o">.</span><span class="na">commit</span><span class="o">();</span></div><div class="line" id="LC73"> <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Completed ingest."</span><span class="o">);</span></div><div class="line" id="LC74"> <span class="o">}</span></div><div class="line" id="LC75"> <span class="k">catch</span><span class="o">(</span><span class="n">Exception</span> <span class="n">e1</span><span class="o">)</span></div><div class="line" id="LC76"> <span class="o">{</span></div><div class="line" id="LC77"> <span class="n">e1</span><span class="o">.</span><span class="na">printStackTrace</span><span class="o">();</span></div><div class="line" id="LC78"> <span class="k">if</span> <span class="o">(</span><span class="n">tx</span> <span class="o">!=</span> <span class="kc">null</span><span class="o">)</span></div><div class="line" id="LC79"> <span class="n">tx</span><span class="o">.</span><span class="na">complete</span><span class="o">();</span></div><div class="line" id="LC80"> <span class="k">if</span> <span class="o">(</span><span class="n">graphDB</span> <span class="o">!=</span> <span class="kc">null</span><span class="o">)</span></div><div class="line" id="LC81"> <span class="n">graphDB</span><span class="o">.</span><span class="na">close</span><span class="o">();</span></div><div class="line" id="LC82"> <span class="o">}</span></div><div class="line" id="LC83"> <span class="o">}</span></div><div class="line" id="LC84"><span class="o">}</span></div><div class="line" id="LC85"><br /></div><div class="line" id="LC86"><span class="kd">class</span> <span class="nc">Person</span> <span class="kd">extends</span> <span class="n">BaseVertex</span></div><div class="line" id="LC87"><span class="o">{</span></div><div class="line" id="LC88">    <span class="kd">private</span> <span class="n">String</span> <span class="n">name</span><span class="o">;</span></div><div class="line" id="LC89"><br /></div><div class="line" id="LC90">    <span class="kd">public</span> <span class="nf">Person</span><span class="o">(</span><span class="n">String</span> <span class="n">name</span><span class="o">)</span></div><div class="line" id="LC91">    <span class="o">{</span></div><div class="line" id="LC92">        <span class="n">setName</span><span class="o">(</span><span class="n">name</span><span class="o">);</span></div><div class="line" id="LC93">    <span class="o">}</span></div><div class="line" id="LC94"><br /></div><div class="line" id="LC95">    <span class="kd">public</span> <span class="n">String</span> <span class="nf">getName</span><span class="o">()</span> </div><div class="line" id="LC96">    <span class="o">{</span></div><div class="line" id="LC97">        <span class="n">fetch</span><span class="o">();</span></div><div class="line" id="LC98">        <span class="k">return</span> <span class="k">this</span><span class="o">.</span><span class="na">name</span><span class="o">;</span></div><div class="line" id="LC99">    <span class="o">}</span></div><div class="line" id="LC100"><br /></div><div class="line" id="LC101">    <span class="kd">public</span> <span class="kt">void</span> <span class="nf">setName</span><span class="o">(</span><span class="n">String</span> <span class="n">name</span><span class="o">)</span> </div><div class="line" id="LC102">    <span class="o">{</span> </div><div class="line" id="LC103">        <span class="n">markModified</span><span class="o">();</span></div><div class="line" id="LC104">        <span class="k">this</span><span class="o">.</span><span class="na">name</span> <span class="o">=</span> <span class="n">name</span><span class="o">;</span> </div><div class="line" id="LC105">    <span class="o">}</span> </div><div class="line" id="LC106"><br /></div><div class="line" id="LC107">    <span class="nd">@Override</span></div><div class="line" id="LC108">    <span class="kd">public</span> <span class="n">String</span> <span class="nf">toString</span><span class="o">()</span></div><div class="line" id="LC109">    <span class="o">{</span></div><div class="line" id="LC110">        <span class="k">return</span> <span class="k">this</span><span class="o">.</span><span class="na">name</span><span class="o">;</span></div><div class="line" id="LC111">    <span class="o">}</span></div><div class="line" id="LC112"><span class="o">};</span></div><div class="line" id="LC113"><br /></div><div class="line" id="LC114"><span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">FinancialEdge</span> <span class="kd">extends</span> <span class="n">BaseEdge</span></div><div class="line" id="LC115"><span class="o">{</span></div><div class="line" id="LC116"> <span class="n">ooMap</span> <span class="n">transactions</span><span class="o">;</span></div><div class="line" id="LC117"><br /></div><div class="line" id="LC118"> <span class="kd">abstract</span> <span class="kt">void</span> <span class="nf">addTransaction</span><span class="o">(</span><span class="n">TransactionType</span> <span class="n">trxn</span><span class="o">,</span> <span class="n">String</span> <span class="n">trxnName</span><span class="o">);</span></div><div class="line" id="LC119"><span class="o">};</span></div><div class="line" id="LC120"><br /></div><div class="line" id="LC121"><span class="kd">class</span> <span class="nc">paysTo</span> <span class="kd">extends</span> <span class="n">FinancialEdge</span></div><div class="line" id="LC122"><span class="o">{</span></div><div class="line" id="LC123"> <span class="kd">public</span> <span class="nf">paysTo</span><span class="o">()</span></div><div class="line" id="LC124"> <span class="o">{</span></div><div class="line" id="LC125">     <span class="n">transactions</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ooMap</span><span class="o">();</span></div><div class="line" id="LC126"> <span class="o">}</span></div><div class="line" id="LC127"><br /></div><div class="line" id="LC128"> <span class="kt">void</span> <span class="nf">addTransaction</span><span class="o">(</span><span class="n">TransactionType</span> <span class="n">trxn</span><span class="o">,</span> <span class="n">String</span> <span class="n">trxnName</span><span class="o">)</span></div><div class="line" id="LC129"> <span class="o">{</span></div><div class="line" id="LC130">        <span class="n">markModified</span><span class="o">();</span></div><div class="line" id="LC131">        <span class="k">if</span> <span class="o">(!</span><span class="n">transactions</span><span class="o">.</span><span class="na">isPersistent</span><span class="o">())</span></div><div class="line" id="LC132">         <span class="k">this</span><span class="o">.</span><span class="na">cluster</span><span class="o">(</span><span class="n">transactions</span><span class="o">);</span></div><div class="line" id="LC133">     <span class="n">transactions</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">trxn</span><span class="o">,</span> <span class="n">trxnName</span><span class="o">);</span></div><div class="line" id="LC134"> <span class="o">}</span></div><div class="line" id="LC135"><span class="o">};</span></div><div class="line" id="LC136"><br /></div><div class="line" id="LC137"><span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">TransactionType</span> <span class="kd">extends</span> <span class="n">ooObj</span></div><div class="line" id="LC138"><span class="o">{};</span></div><div class="line" id="LC139"><br /></div><div class="line" id="LC140"><span class="kd">class</span> <span class="nc">Wire</span> <span class="kd">extends</span> <span class="n">TransactionType</span></div><div class="line" id="LC141"><span class="o">{};</span></div><div class="line" id="LC142"><br /></div><div class="line" id="LC143"><span class="kd">class</span> <span class="nc">Paypal</span> <span class="kd">extends</span> <span class="n">TransactionType</span></div><div class="line" id="LC144"><span class="o">{};</span></div><div class="line" id="LC145"><br /></div><div class="line" id="LC146"><span class="kd">class</span> <span class="nc">Cash</span> <span class="kd">extends</span> <span class="n">TransactionType</span></div><div class="line" id="LC147"><span class="o">{};</span></div><div class="line" id="LC148"><br /></div></pre></div> </td> </tr> </table> </div></p> </p> <p><a href="http://blog.stavi.sh/social-graph-persistence-in-a-java-graph-data">Permalink</a> | <a href="http://blog.stavi.sh/social-graph-persistence-in-a-java-graph-data#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/jTMYZi9DA5E" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/social-graph-persistence-in-a-java-graph-data Sun, 11 Jul 2010 19:53:00 -0700 Graph Sharding In InfiniteGraph Database http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/NJ1O5jOvFpY/java-graph-sharding-in-infinitegraph-database http://blog.stavi.sh/java-graph-sharding-in-infinitegraph-database <p> <p>I have been experimenting with <a href="http://www.infinitegraph.com/">InfiniteGraph</a> quite a bit lately. One of my goals was to use InfiniteGraph’s distribution capabilities (aka graph partitioning or graph sharding). To shard a graph in InfiniteGraph, you need to set a placement strategy that enables multidatabase placement (set in the <a href="http://gist.github.com/472032">properties file</a> for the graph).</p> <p>Here are the specific properties that need to be set for multidatabase placement:</p> <p><div class="data type-text"> <table class="lines" cellspacing="0" cellpadding="0"> <tr> <td> <pre class="line_numbers"><span rel="#L1" id="L1">1</span> <span rel="#L2" id="L2">2</span> <span rel="#L3" id="L3">3</span> </pre> </td> <td width="100%"> <div class="highlight"><pre /><div class="line" id="LC1"># Placement Strategy </div><div class="line" id="LC2">InfiniteGraph.Placement.ImplClass=com.objy.graph.impl.plugins.mdp.MultipleDatabasePlacement</div><div class="line" id="LC3">InfiniteGraph.Placement.MultipleDatabasePlacement.ConfigString=;1,;1</div></pre></div> </td> </tr> </table> </div></p> <p>Runtime configuration means that graph distribution is also decided at runtime. Eventually you’ll be able to develop your own graph placement classes to shard anyway you want. InfiniteGraph will also be providing a selection for common sharding patterns.</p> <p>Another feature I wanted to try was InfiniteGraph’s thread model. The thread model is very flexible. Basically, each thread has its own database cache and locking, which means threads can operate on different parts of the graph simultaneously. There is no shared query engine or centralized cache. The architecture was designed for horizontal scaling by adding more database shards or more threads. Caches are pooled for efficient memory use. Threads can also pass their cache to another thread for pipelining.</p> <p>In the read case, threads can operate on the same parts of the graph without conflict. For updates, other threads are locked out until the update completes. If there is conflict, the thread can be configured to wait for the lock to release or move on. InfiniteGraph also detects deadlocks and race conditions. Eventually, InfiniteGraph will provide a work queue that will take care of all of the conflict resolution for you. The objective is to store the graph as it is ingested in any form and then work out the conflicts for you later.</p> <p>As I was coding this sample application, I thought I would try to create an edge that holds a collection. I wanted to have one edge between two vertices that stored several similar typed properties (decided at runtime). The example I came up with is if one person pays another with different kinds of transactions, for instance using paypal, giving cash, or a wire transfer. I wanted to have a single edge of type paysTo with a polymorphic collection of TransactionType’s that store instances of the specific payment type.</p> <p>To experiment, I created a synthetic dataset from US Census information. I used a list of the most popular surnames and combined them with common first names. I ended up with a <a href="http://stavi.sh/downloads/names.txt">unique list of names</a> that is 151,671 entries long. This is great for creating synthetic social networks. I connect the people in my fake social network randomly.</p> <p>My sample application does have a tendency to create deadlocks (which InfiniteGraph detects). I need to think of a better way of ingesting my social network to avoid this issue. It is really an artifact of the simple code that I wrote and not a reflection of InfiniteGraph capabilities or shortcomings. If anyone can think of a better approach or a more realistic use-case, please let me know. I’d be happy to try it a different way. I also want to thank <a href="http://au.linkedin.com/in/darrenjohnwood">Darren Wood</a>, the lead architect of InfiniteGraph for answering my questions. I am looking forward to future releases and learning how to use InfiniteGraph better.</p> <p>To summarize what this sample shows:</p> <ul> <li><p>Creating a synthetic social graph</p></li> <li><p>Edges with polymorphic collections</p></li> <li><p>Graph sharding over multiple databases</p></li> <li><p>Multithreaded graph ingest</p></li> </ul> <p>Here’s the code if you are interested.</p> <p><div class="data type-text"> <table class="lines" cellspacing="0" cellpadding="0"> <tr> <td> <pre class="line_numbers"><span rel="#L1" id="L1">1</span> <span rel="#L2" id="L2">2</span> <span rel="#L3" id="L3">3</span> <span rel="#L4" id="L4">4</span> <span rel="#L5" id="L5">5</span> <span rel="#L6" id="L6">6</span> <span rel="#L7" id="L7">7</span> <span rel="#L8" id="L8">8</span> <span rel="#L9" id="L9">9</span> <span rel="#L10" id="L10">10</span> <span rel="#L11" id="L11">11</span> <span rel="#L12" id="L12">12</span> <span rel="#L13" id="L13">13</span> <span rel="#L14" id="L14">14</span> <span rel="#L15" id="L15">15</span> <span rel="#L16" id="L16">16</span> <span rel="#L17" id="L17">17</span> <span rel="#L18" id="L18">18</span> <span rel="#L19" id="L19">19</span> <span rel="#L20" id="L20">20</span> <span rel="#L21" id="L21">21</span> <span rel="#L22" id="L22">22</span> <span rel="#L23" id="L23">23</span> <span rel="#L24" id="L24">24</span> <span rel="#L25" id="L25">25</span> <span rel="#L26" id="L26">26</span> <span rel="#L27" id="L27">27</span> <span rel="#L28" id="L28">28</span> <span rel="#L29" id="L29">29</span> <span rel="#L30" id="L30">30</span> <span rel="#L31" id="L31">31</span> <span rel="#L32" id="L32">32</span> <span rel="#L33" id="L33">33</span> <span rel="#L34" id="L34">34</span> <span rel="#L35" id="L35">35</span> <span rel="#L36" id="L36">36</span> <span rel="#L37" id="L37">37</span> <span rel="#L38" id="L38">38</span> <span rel="#L39" id="L39">39</span> <span rel="#L40" id="L40">40</span> <span rel="#L41" id="L41">41</span> <span rel="#L42" id="L42">42</span> <span rel="#L43" id="L43">43</span> <span rel="#L44" id="L44">44</span> <span rel="#L45" id="L45">45</span> <span rel="#L46" id="L46">46</span> <span rel="#L47" id="L47">47</span> <span rel="#L48" id="L48">48</span> <span rel="#L49" id="L49">49</span> <span rel="#L50" id="L50">50</span> <span rel="#L51" id="L51">51</span> <span rel="#L52" id="L52">52</span> <span rel="#L53" id="L53">53</span> <span rel="#L54" id="L54">54</span> <span rel="#L55" id="L55">55</span> <span rel="#L56" id="L56">56</span> <span rel="#L57" id="L57">57</span> <span rel="#L58" id="L58">58</span> <span rel="#L59" id="L59">59</span> <span rel="#L60" id="L60">60</span> <span rel="#L61" id="L61">61</span> <span rel="#L62" id="L62">62</span> <span rel="#L63" id="L63">63</span> <span rel="#L64" id="L64">64</span> <span rel="#L65" id="L65">65</span> <span rel="#L66" id="L66">66</span> <span rel="#L67" id="L67">67</span> <span rel="#L68" id="L68">68</span> <span rel="#L69" id="L69">69</span> <span rel="#L70" id="L70">70</span> <span rel="#L71" id="L71">71</span> <span rel="#L72" id="L72">72</span> <span rel="#L73" id="L73">73</span> <span rel="#L74" id="L74">74</span> <span rel="#L75" id="L75">75</span> <span rel="#L76" id="L76">76</span> <span rel="#L77" id="L77">77</span> <span rel="#L78" id="L78">78</span> <span rel="#L79" id="L79">79</span> <span rel="#L80" id="L80">80</span> <span rel="#L81" id="L81">81</span> <span rel="#L82" id="L82">82</span> <span rel="#L83" id="L83">83</span> <span rel="#L84" id="L84">84</span> <span rel="#L85" id="L85">85</span> <span rel="#L86" id="L86">86</span> <span rel="#L87" id="L87">87</span> <span rel="#L88" id="L88">88</span> <span rel="#L89" id="L89">89</span> <span rel="#L90" id="L90">90</span> <span rel="#L91" id="L91">91</span> <span rel="#L92" id="L92">92</span> <span rel="#L93" id="L93">93</span> <span rel="#L94" id="L94">94</span> <span rel="#L95" id="L95">95</span> <span rel="#L96" id="L96">96</span> <span rel="#L97" id="L97">97</span> <span rel="#L98" id="L98">98</span> <span rel="#L99" id="L99">99</span> <span rel="#L100" id="L100">100</span> <span rel="#L101" id="L101">101</span> <span rel="#L102" id="L102">102</span> <span rel="#L103" id="L103">103</span> <span rel="#L104" id="L104">104</span> <span rel="#L105" id="L105">105</span> <span rel="#L106" id="L106">106</span> <span rel="#L107" id="L107">107</span> <span rel="#L108" id="L108">108</span> <span rel="#L109" id="L109">109</span> <span rel="#L110" id="L110">110</span> <span rel="#L111" id="L111">111</span> <span rel="#L112" id="L112">112</span> <span rel="#L113" id="L113">113</span> <span rel="#L114" id="L114">114</span> <span rel="#L115" id="L115">115</span> <span rel="#L116" id="L116">116</span> <span rel="#L117" id="L117">117</span> <span rel="#L118" id="L118">118</span> <span rel="#L119" id="L119">119</span> <span rel="#L120" id="L120">120</span> <span rel="#L121" id="L121">121</span> <span rel="#L122" id="L122">122</span> <span rel="#L123" id="L123">123</span> <span rel="#L124" id="L124">124</span> <span rel="#L125" id="L125">125</span> <span rel="#L126" id="L126">126</span> <span rel="#L127" id="L127">127</span> <span rel="#L128" id="L128">128</span> <span rel="#L129" id="L129">129</span> <span rel="#L130" id="L130">130</span> <span rel="#L131" id="L131">131</span> <span rel="#L132" id="L132">132</span> <span rel="#L133" id="L133">133</span> <span rel="#L134" id="L134">134</span> <span rel="#L135" id="L135">135</span> </pre> </td> <td width="100%"> <div class="highlight"><pre /><div class="line" id="LC1">import java.util.ArrayList;</div><div class="line" id="LC2">import java.util.Iterator;</div><div class="line" id="LC3">import java.util.Random;</div><div class="line" id="LC4"><br /></div><div class="line" id="LC5">import com.objy.db.app.ooObj;</div><div class="line" id="LC6">import com.objy.db.util.ooMap;</div><div class="line" id="LC7">import com.objy.graph.AccessMode;</div><div class="line" id="LC8">import com.objy.graph.BaseEdge;</div><div class="line" id="LC9">import com.objy.graph.BaseVertex;</div><div class="line" id="LC10">import com.objy.graph.EdgeKind;</div><div class="line" id="LC11">import com.objy.graph.GraphDatabase;</div><div class="line" id="LC12">import com.objy.graph.Transaction;</div><div class="line" id="LC13"><br /></div><div class="line" id="LC14">class IngestThread implements Runnable</div><div class="line" id="LC15">{</div><div class="line" id="LC16">    private int numberOfRecipients = 0;</div><div class="line" id="LC17">    private GraphDatabase graphDB = null;</div><div class="line" id="LC18">    private ArrayList<Long> people = null;</div><div class="line" id="LC19"><br /></div><div class="line" id="LC20">    public IngestThread(GraphDatabase graph, ArrayList<Long> peopleList, int numOfRecipients)</div><div class="line" id="LC21">    {</div><div class="line" id="LC22"> graphDB = graph;</div><div class="line" id="LC23"> numberOfRecipients = numOfRecipients;</div><div class="line" id="LC24"> people = peopleList;</div><div class="line" id="LC25">    }</div><div class="line" id="LC26"><br /></div><div class="line" id="LC27">    @Override</div><div class="line" id="LC28">    public void run() </div><div class="line" id="LC29">    {</div><div class="line" id="LC30"> Transaction tx = null;</div><div class="line" id="LC31"> try</div><div class="line" id="LC32"> {</div><div class="line" id="LC33">            // Start a transaction</div><div class="line" id="LC34">            tx = graphDB.beginTransaction(AccessMode.READ_WRITE);</div><div class="line" id="LC35">            </div><div class="line" id="LC36">            // Generate synthetic social graph (random associations)</div><div class="line" id="LC37">            Random random = new Random();</div><div class="line" id="LC38">            int numberOfPeople = people.size();</div><div class="line" id="LC39">            Iterator<Long> peopleItr = people.iterator();</div><div class="line" id="LC40">            System.out.println("Generating synthetic social graph in thread [" + Thread.currentThread().getId() + "]");</div><div class="line" id="LC41">            System.out.println("Connecting " + numberOfPeople + " senders with " + numberOfRecipients + " recipients."); </div><div class="line" id="LC42">     while (peopleItr.hasNext())</div><div class="line" id="LC43">     {</div><div class="line" id="LC44">                Person person1 = (Person)graphDB.getVertex(people.get(random.nextInt(numberOfPeople)));</div><div class="line" id="LC45">     for (int i = 0; i < numberOfRecipients; i++)</div><div class="line" id="LC46">     {</div><div class="line" id="LC47">     Person person2 = (Person)graphDB.getVertex(people.get(random.nextInt(numberOfPeople)));</div><div class="line" id="LC48">     paysTo pays = new paysTo();</div><div class="line" id="LC49">     person1.addEdge(pays, person2, EdgeKind.BIDIRECTIONAL);</div><div class="line" id="LC50">     pays.addTransaction(new Wire(), "trx1");</div><div class="line" id="LC51">     pays.addTransaction(new Paypal(), "trx2");</div><div class="line" id="LC52">     pays.addTransaction(new Cash(), "trx3");</div><div class="line" id="LC53">     }</div><div class="line" id="LC54">     peopleItr.next();</div><div class="line" id="LC55">     }</div><div class="line" id="LC56">            </div><div class="line" id="LC57">            // Commit</div><div class="line" id="LC58">            tx.commit();</div><div class="line" id="LC59"> System.out.println("Thread [" + Thread.currentThread().getId() + "] completed ingest.");</div><div class="line" id="LC60"> }</div><div class="line" id="LC61"> catch(Exception e1)</div><div class="line" id="LC62"> {</div><div class="line" id="LC63"> System.out.println("[" + Thread.currentThread().getId() + "] Ingest Exception...");</div><div class="line" id="LC64"> e1.printStackTrace();</div><div class="line" id="LC65">            if (tx != null)</div><div class="line" id="LC66"> tx.complete();</div><div class="line" id="LC67"> }</div><div class="line" id="LC68">    }</div><div class="line" id="LC69">}</div><div class="line" id="LC70"><br /></div><div class="line" id="LC71">class Person extends BaseVertex</div><div class="line" id="LC72">{</div><div class="line" id="LC73">    private String name;</div><div class="line" id="LC74"><br /></div><div class="line" id="LC75">    public Person(String name)</div><div class="line" id="LC76">    {</div><div class="line" id="LC77">        setName(name);</div><div class="line" id="LC78">    }</div><div class="line" id="LC79"><br /></div><div class="line" id="LC80">    public String getName() </div><div class="line" id="LC81">    {</div><div class="line" id="LC82">        fetch();</div><div class="line" id="LC83">        return this.name;</div><div class="line" id="LC84">    }</div><div class="line" id="LC85"><br /></div><div class="line" id="LC86">    public void setName(String name) </div><div class="line" id="LC87">    { </div><div class="line" id="LC88">        markModified();</div><div class="line" id="LC89">        this.name = name; </div><div class="line" id="LC90">    } </div><div class="line" id="LC91"><br /></div><div class="line" id="LC92">    @Override</div><div class="line" id="LC93">    public String toString()</div><div class="line" id="LC94">    {</div><div class="line" id="LC95">        return this.name;</div><div class="line" id="LC96">    }</div><div class="line" id="LC97">};</div><div class="line" id="LC98"><br /></div><div class="line" id="LC99">abstract class FinancialEdge extends BaseEdge</div><div class="line" id="LC100">{</div><div class="line" id="LC101"> ooMap transactions;</div><div class="line" id="LC102"><br /></div><div class="line" id="LC103"> abstract void addTransaction(TransactionType trxn, String trxnName);</div><div class="line" id="LC104">};</div><div class="line" id="LC105"><br /></div><div class="line" id="LC106">class paysTo extends FinancialEdge</div><div class="line" id="LC107">{</div><div class="line" id="LC108"> public paysTo()</div><div class="line" id="LC109"> {</div><div class="line" id="LC110">     transactions = new ooMap();</div><div class="line" id="LC111"> }</div><div class="line" id="LC112"><br /></div><div class="line" id="LC113"> void addTransaction(TransactionType trxn, String trxnName)</div><div class="line" id="LC114"> {</div><div class="line" id="LC115">            markModified();</div><div class="line" id="LC116">            if (!transactions.isPersistent())</div><div class="line" id="LC117">         this.cluster(transactions);</div><div class="line" id="LC118">     transactions.add(trxn, trxnName);</div><div class="line" id="LC119"> }</div><div class="line" id="LC120">};</div><div class="line" id="LC121"><br /></div><div class="line" id="LC122">abstract class TransactionType extends ooObj</div><div class="line" id="LC123">{};</div><div class="line" id="LC124"><br /></div><div class="line" id="LC125">class Wire extends TransactionType</div><div class="line" id="LC126">{};</div><div class="line" id="LC127"><br /></div><div class="line" id="LC128">class Paypal extends TransactionType</div><div class="line" id="LC129">{};</div><div class="line" id="LC130"><br /></div><div class="line" id="LC131">class Cash extends TransactionType</div><div class="line" id="LC132">{};</div><div class="line" id="LC133"><br /></div><div class="line" id="LC134"><br /></div><div class="line" id="LC135"><br /></div></pre></div> </td> </tr> </table> </div></p> </p> <p><a href="http://blog.stavi.sh/java-graph-sharding-in-infinitegraph-database">Permalink</a> | <a href="http://blog.stavi.sh/java-graph-sharding-in-infinitegraph-database#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/NJ1O5jOvFpY" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/java-graph-sharding-in-infinitegraph-database Thu, 08 Jul 2010 11:27:00 -0700 New Socrata Social Data Player Template And Infograph For The DOE http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/xcUYR71jAuE/new-socrata-social-data-player-template-for-t http://blog.stavi.sh/new-socrata-social-data-player-template-for-t <p> <div> </div> <div>Here's the design for the <a href="http://www.socrata.com/solution" target="_blank">social data player</a> template that *hopefully* will be used on the <a href="http://www.energy.gov/recovery/energyefficiency.htm" target="_blank">DOE project</a> that I have been working on. I like how it came out. I tied the green and blue in the logo to the colors for the button and the title bar. Check out the <a href="http://socrata.com/d/yefd-dygx" target="_blank">map infograph</a> too. It's pretty slick. The heat / intensity shows how much funding each state received and selecting a state will click-through to a dedicated landing page with a data table breaking-out all of the recovery funding for that state. The DOE New Media Team is extremely good and fun to work with, BTW. Much of the guidance and credit goes to them.</div> <div> </div> <div> </div> <div> <iframe scrolling="no" src="http://www.socrata.com/w/stri-mmy6/2tyd-7i7p?cur=C2Ge5M3og5E" frameborder="0" height="425px" width="500px"><a href="http://www.socrata.com/dataset/DOE-Recovery-Pillars-Aggregate/stri-mmy6" title="DOE Recovery Pillars Aggregate" target="_blank">DOE Recovery Pillars Aggregate</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> <p> </p> </div> <div> <iframe scrolling="no" src="http://www.socrata.com/w/yefd-dygx/y34g-bnf3?cur=zNzgpW4qpx0" frameborder="0" height="425px" width="500px"><a href="http://www.socrata.com/dataset/Smart-Grid-Funding-Map/yefd-dygx" title="Smart Grid Funding Map" target="_blank">Smart Grid Funding Map</a></iframe> <p><a href="http://www.socrata.com/" target="_blank">Powered by Socrata</a></p> </div> </p> <p><a href="http://blog.stavi.sh/new-socrata-social-data-player-template-for-t">Permalink</a> | <a href="http://blog.stavi.sh/new-socrata-social-data-player-template-for-t#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/xcUYR71jAuE" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/new-socrata-social-data-player-template-for-t Wed, 07 Jul 2010 07:16:00 -0700 InfiniteGraph: Referential Integrity http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/gYtP0nGIEVs/infinitegraph-referential-integrity http://blog.stavi.sh/infinitegraph-referential-integrity <p> <p>Based on my <a href="http://post.ly/m733">post yesterday</a>, the guys over at <a href="http://infogrid.org/">InfoGrid</a> brought up some <a href="http://bit.ly/bnlHe0">good points</a> on <a href="http://www.infinitegraph.com/">InfiniteGraph’s</a> edge implementation. I wrote some code to see if you could create a dangling reference case. I couldn’t find a way to do it.</p> <p><div class="data type-text"> <table class="lines" cellspacing="0" cellpadding="0"> <tr> <td> <pre class="line_numbers"><span rel="#L1" id="L1">1</span> <span rel="#L2" id="L2">2</span> <span rel="#L3" id="L3">3</span> <span rel="#L4" id="L4">4</span> <span rel="#L5" id="L5">5</span> <span rel="#L6" id="L6">6</span> <span rel="#L7" id="L7">7</span> <span rel="#L8" id="L8">8</span> <span rel="#L9" id="L9">9</span> <span rel="#L10" id="L10">10</span> </pre> </td> <td width="100%"> <div class="highlight"><pre /><div class="line" id="LC1">// Garbage in, garbage out? </div><div class="line" id="LC2">/* Try to create a dangling reference */</div><div class="line" id="LC3">xkcd.addEdge(tagT4, null, EdgeKind.BIDIRECTIONAL); // Throws null exception</div><div class="line" id="LC4">/* Try to add a transient vertex */</div><div class="line" id="LC5">Resource transy = new Resource("http://ibm.com/"); // Throws a must be persistent</div><div class="line" id="LC6">xkcd.addEdge(tagT4, transy, EdgeKind.BIDIRECTIONAL); // element exception</div><div class="line" id="LC7">/* Try to add a circular reference */</div><div class="line" id="LC8">Resource boomerang = new Resource("http://cisco.com/"); // Throws exception</div><div class="line" id="LC9">graphDB.addVertex(boomerang);</div><div class="line" id="LC10">boomerang.addEdge(tagT4, boomerang, EdgeKind.BIDIRECTIONAL); </div></pre></div> </td> </tr> </table> </div></p> <p>It looks like, when you create an Edge, it is transient. To add it to the database you must call one of the addEdge variants which makes the edge persistent and <em>requires</em> you provide source and destination vertices. There are no dangling references, so referential integrity is kept.</p> </p> <p><a href="http://blog.stavi.sh/infinitegraph-referential-integrity">Permalink</a> | <a href="http://blog.stavi.sh/infinitegraph-referential-integrity#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/gYtP0nGIEVs" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/infinitegraph-referential-integrity Tue, 06 Jul 2010 11:36:00 -0700 Get A Taste Of InfiniteGraph: HelloWorld For Graph Databases http://feedproxy.google.com/~r/ToddStavishsPosterous/~3/i5UIw25pXv8/get-a-taste-of-infinitegraph-hello-world-for http://blog.stavi.sh/get-a-taste-of-infinitegraph-hello-world-for <p> <p>I agree with <a href="http://www.linkedin.com/in/alexandrup">Mr. Popescu</a>. The best way to validate that you’ve got the basics right about a system is to use some <a href="http://post.ly/lby1">basic code</a>. With this idea in mind, I ported his <a href="http://bit.ly/9C4lpP">tagging app</a> to <a href="http://www.infinitegraph.com/">InfiniteGraph</a>, a new graph database. Some things I learned in this effort about InfiniteGraph:</p> <ul> <li>non-primitive vertex attribute types are supported (check out the timestamp in the Resource vertex)</li> <li>efficient path evaluation with qualifiers and navigation result handlers</li> <li>named vertex indexing is simple but powerful</li> </ul> <p>I am looking forward to trying InfiniteGraph in a <a href="http://bit.ly/c8V50b">distributed deployment</a> for my <a href="http://post.ly/mzup">next post</a>.</p> <p>The tagging app console output looks like this:</p> <blockquote><p>Listing all tags and resources:</p> <p>Tag: good</p> <p> Resource: <a href="http://xkcd.com/">http://xkcd.com/</a></p> <p> Resource: <a href="http://cnn.com/">http://cnn.com/</a></p> <p>Tag: funny</p> <p> Resource: <a href="http://xkcd.com/">http://xkcd.com/</a></p> <p> Resource: <a href="http://theonion.com/">http://theonion.com/</a></p> <p>xkcd is tagged with:</p> <p> Tag: funny</p> <p> Tag: good</p> <p>Listing all paths to tagged sites:</p> <p>Found matching path : goodhttp://xkcd.com/</p> <p>Found matching path : goodhttp://cnn.com/</p> <p>Found matching path : goodhttp://xkcd.com/funnyhttp://xkcd.com/</p> <p>Found matching path : goodhttp://xkcd.com/funnyhttp://theonion.com/</p> <p>Found matching path : funnyhttp://xkcd.com/</p> <p>Found matching path : funnyhttp://theonion.com/</p> <p>Found matching path : funnyhttp://xkcd.com/goodhttp://xkcd.com/</p> <p>Found matching path : funnyhttp://xkcd.com/goodhttp://cnn.com/</p></blockquote> <p>Here’s the code if you are interested.</p> <p><div class="data type-text"> <table class="lines" cellspacing="0" cellpadding="0"> <tr> <td> <pre class="line_numbers"><span rel="#L1" id="L1">1</span> <span rel="#L2" id="L2">2</span> <span rel="#L3" id="L3">3</span> <span rel="#L4" id="L4">4</span> <span rel="#L5" id="L5">5</span> <span rel="#L6" id="L6">6</span> <span rel="#L7" id="L7">7</span> <span rel="#L8" id="L8">8</span> <span rel="#L9" id="L9">9</span> <span rel="#L10" id="L10">10</span> <span rel="#L11" id="L11">11</span> <span rel="#L12" id="L12">12</span> <span rel="#L13" id="L13">13</span> <span rel="#L14" id="L14">14</span> <span rel="#L15" id="L15">15</span> <span rel="#L16" id="L16">16</span> <span rel="#L17" id="L17">17</span> <span rel="#L18" id="L18">18</span> <span rel="#L19" id="L19">19</span> <span rel="#L20" id="L20">20</span> <span rel="#L21" id="L21">21</span> <span rel="#L22" id="L22">22</span> <span rel="#L23" id="L23">23</span> <span rel="#L24" id="L24">24</span> <span rel="#L25" id="L25">25</span> <span rel="#L26" id="L26">26</span> <span rel="#L27" id="L27">27</span> <span rel="#L28" id="L28">28</span> <span rel="#L29" id="L29">29</span> <span rel="#L30" id="L30">30</span> <span rel="#L31" id="L31">31</span> <span rel="#L32" id="L32">32</span> <span rel="#L33" id="L33">33</span> <span rel="#L34" id="L34">34</span> <span rel="#L35" id="L35">35</span> <span rel="#L36" id="L36">36</span> <span rel="#L37" id="L37">37</span> <span rel="#L38" id="L38">38</span> <span rel="#L39" id="L39">39</span> <span rel="#L40" id="L40">40</span> <span rel="#L41" id="L41">41</span> <span rel="#L42" id="L42">42</span> <span rel="#L43" id="L43">43</span> <span rel="#L44" id="L44">44</span> <span rel="#L45" id="L45">45</span> <span rel="#L46" id="L46">46</span> <span rel="#L47" id="L47">47</span> <span rel="#L48" id="L48">48</span> <span rel="#L49" id="L49">49</span> <span rel="#L50" id="L50">50</span> <span rel="#L51" id="L51">51</span> <span rel="#L52" id="L52">52</span> <span rel="#L53" id="L53">53</span> <span rel="#L54" id="L54">54</span> <span rel="#L55" id="L55">55</span> <span rel="#L56" id="L56">56</span> <span rel="#L57" id="L57">57</span> <span rel="#L58" id="L58">58</span> <span rel="#L59" id="L59">59</span> <span rel="#L60" id="L60">60</span> <span rel="#L61" id="L61">61</span> <span rel="#L62" id="L62">62</span> <span rel="#L63" id="L63">63</span> <span rel="#L64" id="L64">64</span> <span rel="#L65" id="L65">65</span> <span rel="#L66" id="L66">66</span> <span rel="#L67" id="L67">67</span> <span rel="#L68" id="L68">68</span> <span rel="#L69" id="L69">69</span> <span rel="#L70" id="L70">70</span> <span rel="#L71" id="L71">71</span> <span rel="#L72" id="L72">72</span> <span rel="#L73" id="L73">73</span> <span rel="#L74" id="L74">74</span> <span rel="#L75" id="L75">75</span> <span rel="#L76" id="L76">76</span> <span rel="#L77" id="L77">77</span> <span rel="#L78" id="L78">78</span> <span rel="#L79" id="L79">79</span> <span rel="#L80" id="L80">80</span> <span rel="#L81" id="L81">81</span> <span rel="#L82" id="L82">82</span> <span rel="#L83" id="L83">83</span> <span rel="#L84" id="L84">84</span> <span rel="#L85" id="L85">85</span> <span rel="#L86" id="L86">86</span> <span rel="#L87" id="L87">87</span> <span rel="#L88" id="L88">88</span> <span rel="#L89" id="L89">89</span> <span rel="#L90" id="L90">90</span> <span rel="#L91" id="L91">91</span> <span rel="#L92" id="L92">92</span> <span rel="#L93" id="L93">93</span> <span rel="#L94" id="L94">94</span> <span rel="#L95" id="L95">95</span> <span rel="#L96" id="L96">96</span> <span rel="#L97" id="L97">97</span> <span rel="#L98" id="L98">98</span> <span rel="#L99" id="L99">99</span> <span rel="#L100" id="L100">100</span> <span rel="#L101" id="L101">101</span> <span rel="#L102" id="L102">102</span> <span rel="#L103" id="L103">103</span> <span rel="#L104" id="L104">104</span> <span rel="#L105" id="L105">105</span> <span rel="#L106" id="L106">106</span> <span rel="#L107" id="L107">107</span> <span rel="#L108" id="L108">108</span> <span rel="#L109" id="L109">109</span> <span rel="#L110" id="L110">110</span> <span rel="#L111" id="L111">111</span> <span rel="#L112" id="L112">112</span> <span rel="#L113" id="L113">113</span> <span rel="#L114" id="L114">114</span> <span rel="#L115" id="L115">115</span> <span rel="#L116" id="L116">116</span> <span rel="#L117" id="L117">117</span> <span rel="#L118" id="L118">118</span> <span rel="#L119" id="L119">119</span> <span rel="#L120" id="L120">120</span> <span rel="#L121" id="L121">121</span> <span rel="#L122" id="L122">122</span> <span rel="#L123" id="L123">123</span> <span rel="#L124" id="L124">124</span> <span rel="#L125" id="L125">125</span> <span rel="#L126" id="L126">126</span> <span rel="#L127" id="L127">127</span> <span rel="#L128" id="L128">128</span> <span rel="#L129" id="L129">129</span> <span rel="#L130" id="L130">130</span> <span rel="#L131" id="L131">131</span> <span rel="#L132" id="L132">132</span> <span rel="#L133" id="L133">133</span> <span rel="#L134" id="L134">134</span> <span rel="#L135" id="L135">135</span> <span rel="#L136" id="L136">136</span> <span rel="#L137" id="L137">137</span> <span rel="#L138" id="L138">138</span> <span rel="#L139" id="L139">139</span> <span rel="#L140" id="L140">140</span> <span rel="#L141" id="L141">141</span> <span rel="#L142" id="L142">142</span> <span rel="#L143" id="L143">143</span> <span rel="#L144" id="L144">144</span> <span rel="#L145" id="L145">145</span> <span rel="#L146" id="L146">146</span> <span rel="#L147" id="L147">147</span> <span rel="#L148" id="L148">148</span> <span rel="#L149" id="L149">149</span> <span rel="#L150" id="L150">150</span> <span rel="#L151" id="L151">151</span> <span rel="#L152" id="L152">152</span> <span rel="#L153" id="L153">153</span> <span rel="#L154" id="L154">154</span> <span rel="#L155" id="L155">155</span> <span rel="#L156" id="L156">156</span> <span rel="#L157" id="L157">157</span> <span rel="#L158" id="L158">158</span> <span rel="#L159" id="L159">159</span> <span rel="#L160" id="L160">160</span> <span rel="#L161" id="L161">161</span> <span rel="#L162" id="L162">162</span> <span rel="#L163" id="L163">163</span> <span rel="#L164" id="L164">164</span> <span rel="#L165" id="L165">165</span> <span rel="#L166" id="L166">166</span> <span rel="#L167" id="L167">167</span> <span rel="#L168" id="L168">168</span> <span rel="#L169" id="L169">169</span> <span rel="#L170" id="L170">170</span> <span rel="#L171" id="L171">171</span> <span rel="#L172" id="L172">172</span> <span rel="#L173" id="L173">173</span> <span rel="#L174" id="L174">174</span> <span rel="#L175" id="L175">175</span> <span rel="#L176" id="L176">176</span> <span rel="#L177" id="L177">177</span> <span rel="#L178" id="L178">178</span> <span rel="#L179" id="L179">179</span> <span rel="#L180" id="L180">180</span> <span rel="#L181" id="L181">181</span> <span rel="#L182" id="L182">182</span> <span rel="#L183" id="L183">183</span> <span rel="#L184" id="L184">184</span> <span rel="#L185" id="L185">185</span> <span rel="#L186" id="L186">186</span> <span rel="#L187" id="L187">187</span> <span rel="#L188" id="L188">188</span> <span rel="#L189" id="L189">189</span> <span rel="#L190" id="L190">190</span> <span rel="#L191" id="L191">191</span> <span rel="#L192" id="L192">192</span> <span rel="#L193" id="L193">193</span> <span rel="#L194" id="L194">194</span> <span rel="#L195" id="L195">195</span> <span rel="#L196" id="L196">196</span> <span rel="#L197" id="L197">197</span> <span rel="#L198" id="L198">198</span> <span rel="#L199" id="L199">199</span> <span rel="#L200" id="L200">200</span> <span rel="#L201" id="L201">201</span> <span rel="#L202" id="L202">202</span> <span rel="#L203" id="L203">203</span> <span rel="#L204" id="L204">204</span> <span rel="#L205" id="L205">205</span> <span rel="#L206" id="L206">206</span> <span rel="#L207" id="L207">207</span> <span rel="#L208" id="L208">208</span> <span rel="#L209" id="L209">209</span> <span rel="#L210" id="L210">210</span> <span rel="#L211" id="L211">211</span> <span rel="#L212" id="L212">212</span> <span rel="#L213" id="L213">213</span> <span rel="#L214" id="L214">214</span> <span rel="#L215" id="L215">215</span> <span rel="#L216" id="L216">216</span> <span rel="#L217" id="L217">217</span> <span rel="#L218" id="L218">218</span> <span rel="#L219" id="L219">219</span> <span rel="#L220" id="L220">220</span> <span rel="#L221" id="L221">221</span> <span rel="#L222" id="L222">222</span> <span rel="#L223" id="L223">223</span> <span rel="#L224" id="L224">224</span> <span rel="#L225" id="L225">225</span> <span rel="#L226" id="L226">226</span> <span rel="#L227" id="L227">227</span> <span rel="#L228" id="L228">228</span> <span rel="#L229" id="L229">229</span> <span rel="#L230" id="L230">230</span> <span rel="#L231" id="L231">231</span> <span rel="#L232" id="L232">232</span> <span rel="#L233" id="L233">233</span> <span rel="#L234" id="L234">234</span> <span rel="#L235" id="L235">235</span> <span rel="#L236" id="L236">236</span> <span rel="#L237" id="L237">237</span> <span rel="#L238" id="L238">238</span> <span rel="#L239" id="L239">239</span> <span rel="#L240" id="L240">240</span> <span rel="#L241" id="L241">241</span> <span rel="#L242" id="L242">242</span> <span rel="#L243" id="L243">243</span> </pre> </td> <td width="100%"> <div class="highlight"><pre /><div class="line" id="LC1">import com.objy.graph.*;</div><div class="line" id="LC2">import com.objy.graph.navigation.*;</div><div class="line" id="LC3"><br /></div><div class="line" id="LC4">import java.sql.Timestamp;</div><div class="line" id="LC5"><br /></div><div class="line" id="LC6">public class TagExample</div><div class="line" id="LC7">{</div><div class="line" id="LC8"><br /></div><div class="line" id="LC9">  public static void main(String[] args)</div><div class="line" id="LC10">  { </div><div class="line" id="LC11">    Transaction tx = null;</div><div class="line" id="LC12">    GraphDatabase graphDB = null;</div><div class="line" id="LC13">    try</div><div class="line" id="LC14">    {</div><div class="line" id="LC15">      // Create graph database</div><div class="line" id="LC16">      String graphName = "TagGraph";</div><div class="line" id="LC17">      String props = "graph.properties";</div><div class="line" id="LC18">      GraphFactory.create(graphName, props);</div><div class="line" id="LC19">      graphDB = GraphFactory.open(graphName, props);</div><div class="line" id="LC20"><br /></div><div class="line" id="LC21">      // Start transaction</div><div class="line" id="LC22">      tx = graphDB.beginTransaction(AccessMode.READ_WRITE);</div><div class="line" id="LC23">      </div><div class="line" id="LC24">      // Create resources and tags</div><div class="line" id="LC25">      Resource cnn = new Resource("http://cnn.com/");</div><div class="line" id="LC26">      Resource onion = new Resource("http://theonion.com/");</div><div class="line" id="LC27">      Resource xkcd = new Resource("http://xkcd.com/");</div><div class="line" id="LC28">      Tag good = new Tag("good");</div><div class="line" id="LC29">      Tag funny = new Tag("funny");</div><div class="line" id="LC30">      </div><div class="line" id="LC31">      // Timestamp the tags</div><div class="line" id="LC32">      java.util.Date date = new java.util.Date();</div><div class="line" id="LC33">      Timestamp time = new Timestamp(date.getTime());</div><div class="line" id="LC34">      TaggedWith tagT1 = new TaggedWith(time);</div><div class="line" id="LC35">      TaggedWith tagT2 = new TaggedWith(time);</div><div class="line" id="LC36">      TaggedWith tagT3 = new TaggedWith(time);</div><div class="line" id="LC37">      TaggedWith tagT4 = new TaggedWith(time);</div><div class="line" id="LC38">      </div><div class="line" id="LC39">      // Add vertices to graph</div><div class="line" id="LC40">      graphDB.addVertex(cnn);</div><div class="line" id="LC41">      graphDB.addVertex(onion);</div><div class="line" id="LC42">      graphDB.addVertex(xkcd);</div><div class="line" id="LC43">      graphDB.addVertex(good);</div><div class="line" id="LC44">      graphDB.addVertex(funny);</div><div class="line" id="LC45">      </div><div class="line" id="LC46">      // Connect the vertices: Resources to tags</div><div class="line" id="LC47">      cnn.addEdge(tagT1, good, EdgeKind.BIDIRECTIONAL);</div><div class="line" id="LC48">      xkcd.addEdge(tagT2, good, EdgeKind.BIDIRECTIONAL);</div><div class="line" id="LC49">      onion.addEdge(tagT3, funny, EdgeKind.BIDIRECTIONAL);</div><div class="line" id="LC50">      xkcd.addEdge(tagT4, funny, EdgeKind.BIDIRECTIONAL);</div><div class="line" id="LC51"><br /></div><div class="line" id="LC52">      // Name the vertex for search capability</div><div class="line" id="LC53">      graphDB.nameVertex("xkcd", xkcd);</div><div class="line" id="LC54">      </div><div class="line" id="LC55">      // Commit the graph additions, downgrading to read</div><div class="line" id="LC56">      tx.checkpoint(true);</div><div class="line" id="LC57">      </div><div class="line" id="LC58">      // Do some querying</div><div class="line" id="LC59">      System.out.println("Listing all tags and resources:");</div><div class="line" id="LC60">      System.out.println("Tag: " + good.toString());</div><div class="line" id="LC61">      for (EdgeHandle edge: good.getEdges())</div><div class="line" id="LC62">     System.out.println(" Resource: " + edge.getEdge().getOrigin().toString());</div><div class="line" id="LC63">      System.out.println("Tag: " + funny.toString());</div><div class="line" id="LC64">      for (EdgeHandle edge: funny.getEdges())</div><div class="line" id="LC65">     System.out.println(" Resource: " + edge.getEdge().getOrigin().toString());</div><div class="line" id="LC66">      </div><div class="line" id="LC67">      // Find xkcd's tags</div><div class="line" id="LC68">      // Note: using named vertex retrieval instead of existing reference</div><div class="line" id="LC69">      System.out.println("xkcd is tagged with:");</div><div class="line" id="LC70">      Resource resource = (Resource)graphDB.getNamedVertex("xkcd");</div><div class="line" id="LC71">      for (EdgeHandle edge: resource.getEdges())</div><div class="line" id="LC72">       System.out.println(" Tag: " + edge.getEdge().getTarget().toString());</div><div class="line" id="LC73">      </div><div class="line" id="LC74">      // List all paths to tagged sites</div><div class="line" id="LC75">      System.out.println("Listing all paths to tagged sites:");</div><div class="line" id="LC76">      PrintPathResultsHandler resultPrinter = new PrintPathResultsHandler();</div><div class="line" id="LC77">      isResource qualifier = new isResource();</div><div class="line" id="LC78">      Navigator navigator;</div><div class="line" id="LC79">      navigator = good.navigate(Guide.SIMPLE_BREADTH_FIRST, Qualifier.ANY, qualifier, resultPrinter);</div><div class="line" id="LC80">      navigator.start();</div><div class="line" id="LC81">      navigator.stop();</div><div class="line" id="LC82">      navigator = funny.navigate(Guide.SIMPLE_BREADTH_FIRST, Qualifier.ANY, qualifier, resultPrinter);</div><div class="line" id="LC83">      navigator.start();</div><div class="line" id="LC84">      navigator.stop();</div><div class="line" id="LC85">      </div><div class="line" id="LC86">      // Commit transaction</div><div class="line" id="LC87">      tx.commit();</div><div class="line" id="LC88">      </div><div class="line" id="LC89">      //Delete the Database from the System</div><div class="line" id="LC90">      System.out.println();</div><div class="line" id="LC91">      System.out.println("Tag Database removed");</div><div class="line" id="LC92">      GraphFactory.delete(graphName, props);</div><div class="line" id="LC93">      </div><div class="line" id="LC94">    }</div><div class="line" id="LC95">    catch (ConfigurationException cE)</div><div class="line" id="LC96">    {</div><div class="line" id="LC97">      System.out.println(" Configuration Exception was thrown .. ");</div><div class="line" id="LC98">      System.out.println(cE.getMessage());</div><div class="line" id="LC99">    }</div><div class="line" id="LC100">    finally</div><div class="line" id="LC101">    {</div><div class="line" id="LC102">      // If the transaction was not committed, complete</div><div class="line" id="LC103">      // will roll it back</div><div class="line" id="LC104">      if (tx != null)</div><div class="line" id="LC105">        tx.complete();</div><div class="line" id="LC106">      if (graphDB != null)</div><div class="line" id="LC107">        graphDB.close();</div><div class="line" id="LC108">    }</div><div class="line" id="LC109">  }</div><div class="line" id="LC110">}</div><div class="line" id="LC111"><br /></div><div class="line" id="LC112">/*</div><div class="line" id="LC113">* The persistent class definition for the Resource Vertex type</div><div class="line" id="LC114">*/</div><div class="line" id="LC115">class Resource extends BaseVertex</div><div class="line" id="LC116">{</div><div class="line" id="LC117">  private String uri;</div><div class="line" id="LC118"><br /></div><div class="line" id="LC119">  public Resource(String uri)</div><div class="line" id="LC120">  {</div><div class="line" id="LC121">    setURI(uri);</div><div class="line" id="LC122">  }</div><div class="line" id="LC123"><br /></div><div class="line" id="LC124">  public String getURI() </div><div class="line" id="LC125">  {</div><div class="line" id="LC126">    fetch();</div><div class="line" id="LC127">    return this.uri;</div><div class="line" id="LC128">  }</div><div class="line" id="LC129"><br /></div><div class="line" id="LC130">  public void setURI(String uri) </div><div class="line" id="LC131">  { </div><div class="line" id="LC132">    markModified();</div><div class="line" id="LC133">    this.uri = uri; </div><div class="line" id="LC134">  } </div><div class="line" id="LC135"><br /></div><div class="line" id="LC136">  @Override</div><div class="line" id="LC137">  public String toString()</div><div class="line" id="LC138">  {</div><div class="line" id="LC139">    return this.uri;</div><div class="line" id="LC140">  }</div><div class="line" id="LC141"><br /></div><div class="line" id="LC142">};</div><div class="line" id="LC143"><br /></div><div class="line" id="LC144">/*</div><div class="line" id="LC145">* The persistent class definition for the Tag Vertex type</div><div class="line" id="LC146">*/</div><div class="line" id="LC147">class Tag extends BaseVertex</div><div class="line" id="LC148">{</div><div class="line" id="LC149">  private String label;</div><div class="line" id="LC150"><br /></div><div class="line" id="LC151">  public Tag(String label)</div><div class="line" id="LC152">  {</div><div class="line" id="LC153">    setLabel(label);</div><div class="line" id="LC154">  }</div><div class="line" id="LC155"><br /></div><div class="line" id="LC156">  public String getLabel() </div><div class="line" id="LC157">  {</div><div class="line" id="LC158">    fetch();</div><div class="line" id="LC159">    return this.label;</div><div class="line" id="LC160">  }</div><div class="line" id="LC161"><br /></div><div class="line" id="LC162">  public void setLabel(String label) </div><div class="line" id="LC163">  { </div><div class="line" id="LC164">    markModified();</div><div class="line" id="LC165">    this.label = label; </div><div class="line" id="LC166">  } </div><div class="line" id="LC167"><br /></div><div class="line" id="LC168">  @Override</div><div class="line" id="LC169">  public String toString()</div><div class="line" id="LC170">  {</div><div class="line" id="LC171">    return this.label;</div><div class="line" id="LC172">  }</div><div class="line" id="LC173"><br /></div><div class="line" id="LC174">};</div><div class="line" id="LC175"><br /></div><div class="line" id="LC176">/*</div><div class="line" id="LC177">* The persistent class definition for the TaggedWith Edge type</div><div class="line" id="LC178">*/</div><div class="line" id="LC179">class TaggedWith extends BaseEdge</div><div class="line" id="LC180">{</div><div class="line" id="LC181">  private Timestamp stamp;</div><div class="line" id="LC182">  </div><div class="line" id="LC183">  public TaggedWith(Timestamp stamp)</div><div class="line" id="LC184">  {</div><div class="line" id="LC185">    setStamp(stamp);</div><div class="line" id="LC186">  }</div><div class="line" id="LC187"><br /></div><div class="line" id="LC188">  public Timestamp getstamp() </div><div class="line" id="LC189">  {</div><div class="line" id="LC190">    fetch();</div><div class="line" id="LC191">    return this.stamp;</div><div class="line" id="LC192">  }</div><div class="line" id="LC193"><br /></div><div class="line" id="LC194">  public void setStamp(Timestamp stamp) </div><div class="line" id="LC195">  { </div><div class="line" id="LC196">    markModified();</div><div class="line" id="LC197">    this.stamp = stamp; </div><div class="line" id="LC198">  } </div><div class="line" id="LC199"><br /></div><div class="line" id="LC200">  public String getStamp()</div><div class="line" id="LC201">  {</div><div class="line" id="LC202">    return "Created on " + stamp;</div><div class="line" id="LC203">  }</div><div class="line" id="LC204">  </div><div class="line" id="LC205">};</div><div class="line" id="LC206"><br /></div><div class="line" id="LC207">/*</div><div class="line" id="LC208"> * For every result passing through all the qualifiers in the navigation,</div><div class="line" id="LC209"> * this will print out all the possible paths.</div><div class="line" id="LC210"> */</div><div class="line" id="LC211">class PrintPathResultsHandler implements NavigationResultHandler</div><div class="line" id="LC212">{</div><div class="line" id="LC213">  public void handleResultPath(Path result, Navigator navigator)</div><div class="line" id="LC214">  {</div><div class="line" id="LC215">    System.out.print("Found matching path : ");</div><div class="line" id="LC216">    System.out.print(result.get(0).getVertex().toString());</div><div class="line" id="LC217"><br /></div><div class="line" id="LC218">    // For h in p</div><div class="line" id="LC219">    for(Hop h : result)</div><div class="line" id="LC220">    {</div><div class="line" id="LC221">      if(h.hasEdge())</div><div class="line" id="LC222">      {</div><div class="line" id="LC223">        System.out.print("<->" + h.getVertex().toString());</div><div class="line" id="LC224">      } </div><div class="line" id="LC225">    }</div><div class="line" id="LC226">    System.out.println();</div><div class="line" id="LC227">  }</div><div class="line" id="LC228">};</div><div class="line" id="LC229"><br /></div><div class="line" id="LC230">/*</div><div class="line" id="LC231"> * This qualifier only return true if the end of the path is a resource</div><div class="line" id="LC232"> */</div><div class="line" id="LC233">class isResource implements Qualifier</div><div class="line" id="LC234">{</div><div class="line" id="LC235">  public boolean qualify(Path currentPath)</div><div class="line" id="LC236">  {</div><div class="line" id="LC237">    //this is a qualifying path if the last vertex of this path is a Resource</div><div class="line" id="LC238">    if (currentPath.getFinalHop().getVertex().hasProperty("uri"))</div><div class="line" id="LC239">      return true;</div><div class="line" id="LC240">    else</div><div class="line" id="LC241">      return false;</div><div class="line" id="LC242">  }</div><div class="line" id="LC243">};</div></pre></div> </td> </tr> </table> </div></p> </p> <p><a href="http://blog.stavi.sh/get-a-taste-of-infinitegraph-hello-world-for">Permalink</a> | <a href="http://blog.stavi.sh/get-a-taste-of-infinitegraph-hello-world-for#comment">Leave a comment  »</a> </p><img src="http://feeds.feedburner.com/~r/ToddStavishsPosterous/~4/i5UIw25pXv8" height="1" width="1"/> http://files.posterous.com/user_profile_pics/622304/iphone_110.jpeg http://posterous.com/users/5Al6VO24fgyt Todd Stavish toddstavish Todd Stavish http://blog.stavi.sh/get-a-taste-of-infinitegraph-hello-world-for