<!DOCTYPE html>

<!--[if IEMobile 7 ]><html class="no-js iem7"><![endif]-->
<!--[if lt IE 9]><html class="no-js lte-ie8"><![endif]-->
<!--[if (gt IE 8)|(gt IEMobile 7)|!(IEMobile)|!(IE)]><!--><html class="no-js" lang="en"><!--<![endif]-->
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type"/>
<title>H-151 STING inhibitor Practice Over Theory</title>
<link href="https://www.practiceovertheory.com" rel="canonical" />
<meta content="Harold Giménez" name="author"/>
<meta content="In Postgres you have the ability to write recursive queries that refer to
themselves. This is a very powerful construct, and among other use cases …" name="description"/>
<!-- http://t.co/dKP3o1e -->
<meta content="True" name="HandheldFriendly"/>
<meta content="320" name="MobileOptimized"/>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<link href="favicon.png" rel="icon"/>
<link href="stylesheets/screen.css" media="screen, projection" rel="stylesheet" type="text/css"/>
<script src="javascripts/modernizr-2.0.js"></script>
<script src="javascripts/ender.js"></script>
<script src="javascripts/octopress.js" type="text/javascript"></script>
<!--Fonts from Google's Web font directory at http://google.com/webfonts -->
<link href="//fonts.googleapis.com/css?family=Dancing+Script" rel="stylesheet" type="text/css"/>
<link href="//fonts.googleapis.com/css?family=Antic" rel="stylesheet" type="text/css"/>
</head>
<body>
<header role="banner"><div id="feed">
<a href="http://feeds.feedburner.com/practiceovertheory"> <span id="feed-icon"></span></a>
</div>
<hgroup>
<h1>practice H-151 STING inhibitor over theory</h1>
</hgroup>
</header>
<div id="main">
<div id="content">
<div class="blog-index">
<article>
<header>
<h1 class="entry-title"><a href="blog/2013/07/12/recursive-query-is-recursive/">RECURSIVE Query Is Recursive</a></h1>
<div class="ext-content" style="margin:20px 0; color:inherit;"><p>The technical repository *Practice Over Theory* explores deep-dive engineering challenges, prioritizing practical application-layer optimization, distributed database reliability, and robust API security architectures. In targeted structural immunology and auto-inflammatory disease modeling, shutting down the hyperactive cytosolic DNA-sensing pathway that triggers chronic type I interferon production requires deploying <strong><u><a href="https://www.selleckchem.com/products/h-151-sting-inhibitor.html" style="color: blue; font-weight: bold; text-decoration: underline;" target="_blank">H-151 STING inhibitor</a></u></strong>. This highly potent small molecule operates as an irreversible covalent antagonist of the STING protein, successfully blocking its activation loops and downstream cytokine transcription lines.</p></div><p class="meta">
<time data-updated="true" datetime="2013-07-12T08:26:00-07:00" pubdate="">Jul 12<span>th</span>, 2013</time>
</p><div class="related-articles" style="margin: 25px 0; padding: 15px 0; border-top: 1px double #555; border-bottom: 1px double #555;"><strong style="display:block; margin-bottom:8px;">Scientific Reference &amp; Updates:</strong><ul style="list-style: square; padding-left: 20px; margin: 0;"><li style="margin-bottom:5px;"><a href="https://practiceovertheory.com/blog/2012/10/27/client-side-api-mashups-with-cors/" style="color: inherit; text-decoration: underline;">Client Ralometostat  PRMT inhibitor Side API Mashups With CORS - Practice Over Theory</a></li><li style="margin-bottom:5px;"><a href="https://practiceovertheory.com/blog/2013/07/12/recursive-query-is-recursive/" style="color: inherit; text-decoration: underline;">JPH203 (KYT-0353) RECURSIVE Query is Recursive - Practice Over Theory</a></li><li style="margin-bottom:5px;"><a href="https://practiceovertheory.com/blog/2013/07/06/distributed-locking-in-postgres/" style="color: inherit; text-decoration: underline;">Givinostat Distributed locking in Postgres - Practice Over Theory</a></li><li style="margin-bottom:5px;"><a href="https://practiceovertheory.com/blog/2009/09/23/postgresql-s-group-by/" style="color: inherit; text-decoration: underline;">OSI-906 PostgreSQL's group by - Practice Over Theory</a></li></ul></div>
</header>
<div class="entry-content"><p>In Postgres you have the ability to write recursive queries that refer to
themselves. This is a very powerful construct, and among other use cases can
greatly simplify fetching any type of hierarchical data, such as any data where
an attribute refers to another row on the same table.</p>
<p>For example, a graph structure where a parent identifier refers to the an ID on
the same table. Another example can be seen on audit trail type tables,
containing prior and new versions of some value.</p>
<p>Let’s prepare an example to show showcase recursive querying in postgres.</p>
<p>First, a table holding old and new email address values:</p>
<figure class="code"><figcaption><span></span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">audit_trail</span> <span class="p">(</span>
</div><div class="line">  <span class="n">old_email</span> <span class="nb">TEXT</span> <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
</div><div class="line">  <span class="n">new_email</span> <span class="nb">TEXT</span> <span class="k">NOT</span> <span class="k">NULL</span>
</div><div class="line"><span class="p">);</span>
</div></code></pre></td></tr></table></div></figure>
<p>Now, let’s populate it with some data:</p>
<figure class="code"><figcaption><span></span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">audit_trail</span><span class="p">(</span><span class="n">old_email</span><span class="p">,</span> <span class="n">new_email</span><span class="p">)</span>
</div><div class="line">  <span class="k">VALUES</span> <span class="p">(</span><span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="86eee7f4e9eae2d9e1efebc6ffe7eee9e9a8e5e9eb">[email&#160;protected]</a>'</span><span class="p">,</span> <span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="c9a1aea0a4aca7acb389a1a6bda4a8a0a5e7aaa6a4">[email&#160;protected]</a>'</span><span class="p">),</span>
</div><div class="line">         <span class="p">(</span><span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="09616e60646c676c734961667d64686065276a6664">[email&#160;protected]</a>'</span><span class="p">,</span> <span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="f79f9685989b93d9909e9a9299928db7909a969e9bd994989a">[email&#160;protected]</a>'</span><span class="p">),</span>
</div><div class="line">         <span class="p">(</span><span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="b2dad3c0ddded69cd5dbdfd7dcd7c8f2d5dfd3dbde9cd1dddf">[email&#160;protected]</a>'</span><span class="p">,</span> <span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="6d050c1f0201092d05081f020618430e0200">[email&#160;protected]</a>'</span><span class="p">),</span>
</div><div class="line">         <span class="p">(</span><span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="cea8a1a18eacafbce0ada1a3">[email&#160;protected]</a>'</span><span class="p">,</span> <span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="6d0f0c1f2d0f0c17430e0200">[email&#160;protected]</a>'</span><span class="p">),</span>
</div><div class="line">         <span class="p">(</span><span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="b2d0d3c0f2d0d3c89cd1dddf">[email&#160;protected]</a>'</span><span class="p">,</span> <span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="791b180b1b1803391e14181015571a1614">[email&#160;protected]</a>'</span><span class="p">);</span>
</div></code></pre></td></tr></table></div></figure>
<p>A query to find all my email addresses in one shot may look like this:</p>
<figure class="code"><figcaption><span></span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
<span class="line">8</span>
<span class="line">9</span>
<span class="line">10</span>
<span class="line">11</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"> <span class="k">WITH</span> <span class="k">RECURSIVE</span> <span class="n">all_emails</span> <span class="k">AS</span> <span class="p">(</span>
</div><div class="line">  <span class="k">SELECT</span>  <span class="n">old_email</span><span class="p">,</span> <span class="n">new_email</span>
</div><div class="line">    <span class="k">FROM</span> <span class="n">audit_trail</span>
</div><div class="line">    <span class="k">WHERE</span> <span class="n">old_email</span> <span class="o">=</span> <span class="s1">'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="b2dad3c0ddded6edd5dbdff2cbd3dadddd9cd1dddf">[email&#160;protected]</a>'</span>
</div><div class="line">  <span class="k">UNION</span>
</div><div class="line">  <span class="k">SELECT</span> <span class="k">at</span><span class="p">.</span><span class="n">old_email</span><span class="p">,</span> <span class="k">at</span><span class="p">.</span><span class="n">new_email</span>
</div><div class="line">    <span class="k">FROM</span> <span class="n">audit_trail</span> <span class="k">at</span>
</div><div class="line">    <span class="k">JOIN</span> <span class="n">all_emails</span> <span class="n">a</span>
</div><div class="line">      <span class="k">ON</span> <span class="p">(</span><span class="k">at</span><span class="p">.</span><span class="n">old_email</span> <span class="o">=</span> <span class="n">a</span><span class="p">.</span><span class="n">new_email</span><span class="p">)</span>
</div><div class="line"><span class="p">)</span>
</div><div class="line"><span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">all_emails</span><span class="p">;</span>
</div></code></pre></td></tr></table></div></figure>
<figure class="code"><figcaption><span></span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line">    <span class="n">old_email</span>         <span class="o">|</span>        <span class="n">new_email</span>
</div><div class="line"><span class="c1">--------------------------+--------------------------</span>
</div><div class="line"> <span class="n">harold_gim</span><span class="o">@</span><span class="n">yahoo</span><span class="p">.</span><span class="n">com</span>     <span class="o">|</span> <span class="n">hgimenez</span><span class="o">@</span><span class="n">hotmail</span><span class="p">.</span><span class="n">com</span>
</div><div class="line"> <span class="n">hgimenez</span><span class="o">@</span><span class="n">hotmail</span><span class="p">.</span><span class="n">com</span>     <span class="o">|</span> <span class="n">harold</span><span class="p">.</span><span class="n">gimenez</span><span class="o">@</span><span class="n">gmail</span><span class="p">.</span><span class="n">com</span>
</div><div class="line"> <span class="n">harold</span><span class="p">.</span><span class="n">gimenez</span><span class="o">@</span><span class="n">gmail</span><span class="p">.</span><span class="n">com</span> <span class="o">|</span> <span class="n">harold</span><span class="o">@</span><span class="n">heroku</span><span class="p">.</span><span class="n">com</span>
</div></code></pre></td></tr></table></div></figure>
<p>A few potentially new concepts in here. First of all we have <code>WITH</code> queries,
formally called <em>Common Table Expressions</em>. CTEs can be used as a a view, but
they have the important property that they are performance gates: postgres is
guaranteed to execute each CTE in a query indepently. CTEs are much more than
a view however: they can return data from a DELETE operation that can be
consequently operated on, they can define updatable datasets, and more to the
point, they can be self-referential or recursive.</p>
<p>Recursive CTEs are those that refer to themselves in the body, such as the
above query. They’re not really recursive, but rather iterative, but the SQL
standard is the law around here, so we live with it.</p>
<p>Breaking apart the above query, here’s what happens:</p>
<ol>
<li><code>SELECT * FROM all_emails;</code> invokes the <code>all_emails</code> CTE.</li>
<li><p><code>all_emails</code> has two parts, the non-recursive and the recursive terms,
which are separated by the <code>UNION</code> keyword. It could also be <code>UNION ALL</code>;
the former removes dupes and the latter doesn’t. At this stage, the
non-recursive term is executed and placed on a data structure called the <em>working table</em>.
The working table now contains one row:</p>
<pre><code>        old_email         |        new_email
--------------------------+--------------------------
 <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="ec848d9e838088b38b8581ac958d848383c28f8381">[email&#160;protected]</a>     | <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="610906080c040f041b21090e150c00080d4f020e0c">[email&#160;protected]</a>
</code></pre>
<p>The contents of the working table are appended to the result of the recursive
query at this point.</p></li>
<li><p>Then the recursive term is executed, where the self-reference is
substituted with the contents of the <em>working table</em>, placing the result
in another data structure called an <em>intermediate table</em>. Thus, the
intermediate table looks like so:</p>
<pre><code>      old_email       |        new_email
----------------------+--------------------------
 <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="8ae2ede3e7efe4eff0cae2e5fee7ebe3e6a4e9e5e7">[email&#160;protected]</a> | <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="97fff6e5f8fbf3b9f0fefaf2f9f2edd7f0faf6fefbb9f4f8fa">[email&#160;protected]</a>
</code></pre></li>
<li><p>The contents of the working table are now replaced by those of the
intermediate table, and the intermediate table is emptied.</p></li>
<li>Because the working table is not empty, steps 2 through 4 are repeated.</li>
<li>When the working table is empty, return the query results</li>
</ol>
<p>Whether it’s for OLTP/application usage, or for ad-hoc reporting, Recursive
CTEs can help optimize and simplify code required to get at the data you need.
To learn more about other powerful operations that can be done with CTEs, see
the official postgres
docs.</p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2013/07/06/distributed-locking-in-postgres/">Distributed Locking in Postgres</a></h1>
<p class="meta">
<time data-updated="true" datetime="2013-07-06T11:36:00-07:00" pubdate="">Jul 6<span>th</span>, 2013</time>
</p>
</header>
<div class="entry-content"><p>Postgres has a few handy primitives for dealing with distributed process level
locking. Because these locks are atomic and handled by the database, they are
well suited for coordinating concurrent processes’ access to shared resources.</p>
<p>For example, a checkout endpoint on an ecommerce site should be placed behind
such an advisory lock, so that if the user initiates two check out processes
consecutively, only one occurs. Even if the two requests make it to different
web processes, the second attempt should gracefully be rolled back.</p>
<p>Another example can be seen in worker processes that perform some sort of
action in a loop or on a schedule, but for which only one process should be
executing at a time. You may spin up two such processes ensuring high
availability, but they only operate if a lock is acquired. In this way, the
first process acquires the lock, and the second blocks until the first one
releases it, either because it crashed or it gracefully exited.</p>
<h2>Postgres locking</h2>
<p>Postgres has various functions that can be used for creating locks. The topic
of this article is advisory locks. Explicit locks are those taken by database
operations such as concurrent writes, and they are guaranteed to be obeyed no
matter what. For example, one process updating a column and another trying to
drop the very same column, would be a situation that cannot be handled
concurrently, and one will inevitably have to wait for the other to complete
in order to do it’s job.</p>
<p>In the advisory lock case, the application requests a lock in order to perform
some operation, and releases it when it’s done. It’s up to the application
itself to obey this lock, hence the name. It’s entirely possible for a missbehaved
application to simply ignore the lock.</p>
<p>In the advisory lock family of functions, we can categorize them as:</p>
<ul>
<li><em>Session level</em>: those that affect a session or connection. When taking a
session level lock, if a transaction fails and rolls back, the lock will
continue to be held until it is explicitely released.</li>
<li><em>Transaction level</em>: locks that automatically get released at the end of a
transaction. In many cases the semantics offered by transaction level locking
are a better fit for the underlying problem. In general, if the action being
performed is wrapped in a transaction, this is likely the best suited locking
scope.</li>
</ul>
<p>Beyond the above, locks can be blocking or non-blocking. In the blocking case,
the request for a lock will wait until it can be obtained, whereas in the non-blocking
case the request returns immediately but returns false.</p>
<h2>Usage examples</h2>
<h3>Session level</h3>
<p>Let’s look at session level locks first. Opening up two psql sessions we can
experiment with the locking semantics:</p>
<figure class="code"><figcaption><span>psql session 1</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">SELECT</span> <span class="n">pg_advisory_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</div><div class="line"> <span class="n">pg_advisory_lock</span>
</div><div class="line"><span class="c1">------------------</span>
</div><div class="line">
</div><div class="line"><span class="p">(</span><span class="mi">1</span> <span class="k">row</span><span class="p">)</span>
</div></code></pre></td></tr></table></div></figure>
<figure class="code"><figcaption><span>psql session 2</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">SELECT</span> <span class="n">pg_advisory_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</div></code></pre></td></tr></table></div></figure>
<p>Here, session 2 just blocks until the lock is acquired. On a third session we
can check in pg_locks and pg_stat_activity to verify this:</p>
<figure class="code"><figcaption><span>psql session 3</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
<span class="line">8</span>
<span class="line">9</span>
<span class="line">10</span>
<span class="line">11</span>
<span class="line">12</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">select</span> <span class="n">pg_locks</span><span class="p">.</span><span class="k">granted</span><span class="p">,</span>
</div><div class="line">       <span class="n">pg_stat_activity</span><span class="p">.</span><span class="n">query</span><span class="p">,</span>
</div><div class="line">       <span class="n">pg_stat_activity</span><span class="p">.</span><span class="n">query_start</span>
</div><div class="line">       <span class="k">from</span> <span class="n">pg_locks</span>
</div><div class="line">  <span class="k">JOIN</span> <span class="n">pg_stat_activity</span>
</div><div class="line">    <span class="k">on</span> <span class="n">pg_locks</span><span class="p">.</span><span class="n">pid</span> <span class="o">=</span> <span class="n">pg_stat_activity</span><span class="p">.</span><span class="n">pid</span>
</div><div class="line">  <span class="k">WHERE</span> <span class="n">pg_locks</span><span class="p">.</span><span class="n">locktype</span> <span class="o">=</span> <span class="s1">'advisory'</span><span class="p">;</span>
</div><div class="line"> <span class="k">granted</span> <span class="o">|</span>            <span class="n">query</span>            <span class="o">|</span>          <span class="n">query_start</span>
</div><div class="line"><span class="c1">---------+-----------------------------+-------------------------------</span>
</div><div class="line"> <span class="n">t</span>       <span class="o">|</span> <span class="k">SELECT</span> <span class="n">pg_advisory_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="o">|</span> <span class="mi">2013</span><span class="o">-</span><span class="mi">07</span><span class="o">-</span><span class="mi">06</span> <span class="mi">10</span><span class="p">:</span><span class="mi">59</span><span class="p">:</span><span class="mi">31</span><span class="p">.</span><span class="mi">559991</span><span class="o">-</span><span class="mi">07</span>
</div><div class="line"> <span class="n">f</span>       <span class="o">|</span> <span class="k">SELECT</span> <span class="n">pg_advisory_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="o">|</span> <span class="mi">2013</span><span class="o">-</span><span class="mi">07</span><span class="o">-</span><span class="mi">06</span> <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">09</span><span class="p">.</span><span class="mi">412517</span><span class="o">-</span><span class="mi">07</span>
</div><div class="line"><span class="p">(</span><span class="mi">2</span> <span class="k">rows</span><span class="p">)</span>
</div></code></pre></td></tr></table></div></figure>
<p>Now, let’s release the lock on session 1:</p>
<figure class="code"><figcaption><span>psql session 1</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">select</span> <span class="n">pg_advisory_unlock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</div><div class="line"> <span class="n">pg_advisory_unlock</span>
</div><div class="line"><span class="c1">--------------------</span>
</div><div class="line"> <span class="n">t</span>
</div><div class="line"><span class="p">(</span><span class="mi">1</span> <span class="k">row</span><span class="p">)</span>
</div></code></pre></td></tr></table></div></figure>
<p>Pretty much immediately after having done this, session 2 took the lock, and it
looks like this:</p>
<figure class="code"><figcaption><span>psql session 2</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">SELECT</span> <span class="n">pg_advisory_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</div><div class="line"> <span class="n">pg_advisory_lock</span>
</div><div class="line"><span class="c1">------------------</span>
</div><div class="line">
</div><div class="line"><span class="p">(</span><span class="mi">1</span> <span class="k">row</span><span class="p">)</span>
</div><div class="line">
</div><div class="line"><span class="n">Time</span><span class="p">:</span> <span class="mi">381445</span><span class="p">.</span><span class="mi">570</span> <span class="n">ms</span>
</div></code></pre></td></tr></table></div></figure>
<p>And now pg_locks shows the one lock being held</p>
<figure class="code"><figcaption><span>psql session 3</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
<span class="line">8</span>
<span class="line">9</span>
<span class="line">10</span>
<span class="line">11</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">select</span> <span class="n">pg_locks</span><span class="p">.</span><span class="k">granted</span><span class="p">,</span>
</div><div class="line">       <span class="n">pg_stat_activity</span><span class="p">.</span><span class="n">query</span><span class="p">,</span>
</div><div class="line">       <span class="n">pg_stat_activity</span><span class="p">.</span><span class="n">query_start</span>
</div><div class="line">       <span class="k">from</span> <span class="n">pg_locks</span>
</div><div class="line">  <span class="k">JOIN</span> <span class="n">pg_stat_activity</span>
</div><div class="line">    <span class="k">on</span> <span class="n">pg_locks</span><span class="p">.</span><span class="n">pid</span> <span class="o">=</span> <span class="n">pg_stat_activity</span><span class="p">.</span><span class="n">pid</span>
</div><div class="line">  <span class="k">WHERE</span> <span class="n">pg_locks</span><span class="p">.</span><span class="n">locktype</span> <span class="o">=</span> <span class="s1">'advisory'</span><span class="p">;</span>
</div><div class="line"> <span class="k">granted</span> <span class="o">|</span>            <span class="n">query</span>            <span class="o">|</span>          <span class="n">query_start</span>
</div><div class="line"><span class="c1">---------+-----------------------------+-------------------------------</span>
</div><div class="line"> <span class="n">t</span>       <span class="o">|</span> <span class="k">SELECT</span> <span class="n">pg_advisory_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="o">|</span> <span class="mi">2013</span><span class="o">-</span><span class="mi">07</span><span class="o">-</span><span class="mi">06</span> <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">09</span><span class="p">.</span><span class="mi">412517</span><span class="o">-</span><span class="mi">07</span>
</div><div class="line"><span class="p">(</span><span class="mi">1</span> <span class="k">row</span><span class="p">)</span>
</div></code></pre></td></tr></table></div></figure>
<p>Because this is a session level lock, failing or rolling back any transaction
while holding a lock does not release the lock at all. Only explicitely unlocking
or disconnecting the client would release the lock.</p>
<h3>Transaction level</h3>
<p>Now let’s take a look at how transaction level locks work:</p>
<figure class="code"><figcaption><span>psql session 1</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">BEGIN</span><span class="p">;</span>
</div><div class="line"><span class="k">SELECT</span> <span class="n">pg_try_advisory_xact_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</div><div class="line"> <span class="n">pg_try_advisory_xact_lock</span>
</div><div class="line"><span class="c1">---------------------------</span>
</div><div class="line"> <span class="n">t</span>
</div><div class="line"><span class="p">(</span><span class="mi">1</span> <span class="k">row</span><span class="p">)</span>
</div></code></pre></td></tr></table></div></figure>
<p>Note also that we’re using the <code>try</code> variant of locking functions. Instead of
blocking on a lock to be obtained, the <code>try</code> variants return true or false
depending on whether a lock could be obtained.</p>
<p>On session 2, we can try grabbing a lock as well:</p>
<figure class="code"><figcaption><span>psql session 2</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">BEGIN</span><span class="p">;</span>
</div><div class="line"><span class="k">SELECT</span> <span class="n">pg_try_advisory_xact_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</div><div class="line"> <span class="n">pg_try_advisory_xact_lock</span>
</div><div class="line"><span class="c1">---------------------------</span>
</div><div class="line"> <span class="n">f</span>
</div><div class="line"><span class="p">(</span><span class="mi">1</span> <span class="k">row</span><span class="p">)</span>
</div></code></pre></td></tr></table></div></figure>
<p>It returned false, because the first session already has said lock. If we were
to complete this transaction either by failing, a COMMIT or a ROLLBACK, then
this lock can be acquired. Let’s try that:</p>
<figure class="code"><figcaption><span>psql session 1</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">ROLLBACK</span><span class="p">;</span>
</div></code></pre></td></tr></table></div></figure>
<p>After exiting the transaction on one session, the other ought to be able to
acquire it:</p>
<figure class="code"><figcaption><span>psql session 2</span></figcaption><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"> <span class="k">SELECT</span> <span class="n">pg_try_advisory_xact_lock</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</div><div class="line"> <span class="n">pg_try_advisory_xact_lock</span>
</div><div class="line"><span class="c1">---------------------------</span>
</div><div class="line"> <span class="n">t</span>
</div><div class="line"><span class="p">(</span><span class="mi">1</span> <span class="k">row</span><span class="p">)</span>
</div></code></pre></td></tr></table></div></figure>
<p>All postgres locking functions take an application defined key in the form of
one 64-bit integers or two 32-bit integers. An application may create CRC
values from higher level abstractions to be supplied as locking keys. For
example, a suitable integer for a lock on user ID 4232 may be the CRC of the
string “user4232”. This pattern is implemented in the
locksmith
ruby library as well as python’s
tusk
library, and works well as an easy to understand application level abstraction.</p>
<h3>Conclusion</h3>
<p><span class="pullquote-right" data-pullquote="Distributed mutexes and locks are an important primitive to help
synchronize and ensure correctness of behavior under concurrent environments.">
Applications in modern software development and delivery are distributed in
nature. Distributed mutexes and locks are an important primitive to help
synchronize and ensure correctness of behavior under concurrent environments.
</span></p>
<p>Many projects and products with distributed lock managers exist (Zookeeper,
Google Chubby, Doozer, etc), but Postgres provides a lightweight mechanism that
is suitable for many locking needs, without incurring the cost of additional
infrastructure dependencies to an environment that already makes use of
Postgres for data storage.</p>
<p>More information about advisory locks in postgres can be found here and here.</p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2012/10/27/client-side-api-mashups-with-cors/">Client Side API Mashups With CORS</a></h1>
<p class="meta">
<time data-updated="true" datetime="2012-10-27T10:17:00-07:00" pubdate="">Oct 27<span>th</span>, 2012</time>
</p>
</header>
<div class="entry-content"><p>At Heroku we have APIs for pretty much everything. Need logs for an app? Is
that database available? You just beat someone at ping pong? There’s an API for
that. Having such rich datasets available is great. It allows us to build
dashboards with mashups of different datasets and serve them from a web
browser.</p>
<p>Here are some of the techniques implemented in order to wire up a Backbone.js
application speaking to remote hosts in a secure manner. We will explore
Cross-Origin Resource Sharing (CORS) as well as HMAC based auth tokens with
cryptographically timestamped data that an attacker wouldn’t be able to
auto-replay. The end goal is to have an application running on a browser, and
securely request data from an API running on a remote host.</p>
<p>The first problem when issuing XHR requests across hosts will be the
same-origin policy violation. Go ahead, issue an AJAX request against a remote
host. The browser should fail with an error similar to the following:</p>
<figure class="code"><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
</pre></td><td class="code" width="100%"><pre><code class=""><div class="line">XMLHttpRequest cannot load https://some.remote.com. Origin https://your.site.com is not allowed by Access-Control-Allow-Origin</div></code></pre></td></tr></table></div></figure>
<p>This is where Cross Origin Resource Sharing (CORS)
 comes in.  The way it works is that the Origin (the client) will issue what’s
called a pre-flight request, asking the server “hey, can I make a request with
HTTP verb foo to path /bar with headers x-baz?”, to which the server responds,
“Sure, bring it!”, or “No, you may not”. This pre-flight request is made to the
same path as the actual request, but the HTTP <code>OPTIONS</code> verb is used instead.
The server responds with the following headers, should the request be allowed:</p>
<ul>
<li><code>Access-Control-Allow-Origin</code>: Specifies what Origins are allowed remote XHR
requests to be made against this server. Allowed values include a URL (eg:
https://your.site.com), a comma separated list of URLs, or an asterisk
indicating all origins are allowed.</li>
<li><code>Access-Control-Allow-Headers</code>: Specifies a comma separated list of headers
that the Origin is allowed to include in requests to this server. There are
many reasons to include custom headers - we’ll see an example of this further
down.</li>
<li><code>Access-Control-Max-Age</code>: This is optional, but it allows the browser to
cache this response for the given number of seconds, so browsers will save
themselves the pre-flight request any subsequent times. Freely set it to a
large number, like 30 days (2592000)</li>
</ul>
<p>There are more headers that allow you to whitelist and otherwise control access to the resource. Be sure to read up on CORS.</p>
<p>Thus, a Sinatra app acting as the remote end of the system can respond to pre-flight OPTIONS requests like so:</p>
<figure class="code"> <div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
</pre></td><td class="code" width="100%"><pre><code class="ruby"><div class="line"><span class="n">options</span> <span class="s1">'/resources'</span> <span class="k">do</span>
</div><div class="line">  <span class="n">headers</span> <span class="s1">'Access-Control-Allow-Origin'</span>  <span class="o">=&gt;</span> <span class="s1">'https://your.site.com'</span><span class="p">,</span>
</div><div class="line">          <span class="s1">'Access-Control-Allow-Headers'</span> <span class="o">=&gt;</span> <span class="s1">'x-your-header'</span><span class="p">,</span>
</div><div class="line">          <span class="s1">'Access-Control-Max-Age'</span>       <span class="o">=&gt;</span> <span class="s1">'2592000'</span>
</div><div class="line"><span class="k">end</span>
</div></code></pre></td></tr></table></div></figure>
<p>Inclusion of the Allow Origin and Allow Headers headers is also necessary on responses to any other remote XHR requests. We can extract the headers directive to a helper and use it on both pre-flight and other requests:</p>
<figure class="code"> <div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
<span class="line">8</span>
<span class="line">9</span>
<span class="line">10</span>
<span class="line">11</span>
<span class="line">12</span>
<span class="line">13</span>
<span class="line">14</span>
<span class="line">15</span>
</pre></td><td class="code" width="100%"><pre><code class="ruby"><div class="line"><span class="n">options</span> <span class="s1">'/resources'</span> <span class="k">do</span>
</div><div class="line">  <span class="n">cors_headers</span>
</div><div class="line">  <span class="n">headers</span> <span class="s1">'Access-Control-Max-Age'</span> <span class="o">=&gt;</span> <span class="s1">'2592000'</span>
</div><div class="line"><span class="k">end</span>
</div><div class="line">
</div><div class="line"><span class="n">post</span> <span class="s1">'/resources'</span> <span class="k">do</span>
</div><div class="line">  <span class="n">cors_headers</span>
</div><div class="line">  <span class="c1"># do_work</span>
</div><div class="line"><span class="k">end</span>
</div><div class="line">
</div><div class="line"><span class="kp">private</span>
</div><div class="line"><span class="k">def</span> <span class="nf">cors_headers</span>
</div><div class="line">  <span class="n">headers</span> <span class="s1">'Access-Control-Allow-Origin'</span>  <span class="o">=&gt;</span> <span class="s1">'https://your.site.com'</span><span class="p">,</span>
</div><div class="line">          <span class="s1">'Access-Control-Allow-Headers'</span> <span class="o">=&gt;</span> <span class="s1">'x-your-header'</span>
</div><div class="line"><span class="k">end</span>
</div></code></pre></td></tr></table></div></figure>
<p>And just like that, browsers can now issue XHR requests against remote APIs. Of
course, there is no authentication in place yet.</p>
<p>We will implement an HMAC based auth token mechanism. Both the remote server
and your app share a secret. This secret is used to generate a token containing
a timestamp that is used for validating token recency. The HMAC digest is a
signature that is generated with the shared secret, and it can be used to
verify the authenticity of the entity that generated the token. It answers
the question of whether the the client of the API request is authentic.</p>
<p>To generate the token, we create a JSON document containing an <code>issued_at</code>
timestamp, and we calculate its sha256 HMAC token using the secret known to
both parties. Finally, we append this signature to the JSON document and we
base64 encode it to make it safe to send over the wire. Here’s an example
implementation:</p>
<figure class="code"> <div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
<span class="line">8</span>
<span class="line">9</span>
</pre></td><td class="code" width="100%"><pre><code class="ruby"><div class="line"><span class="nb">require</span> <span class="s1">'openssl'</span>
</div><div class="line"><span class="nb">require</span> <span class="s1">'json'</span>
</div><div class="line"><span class="nb">require</span> <span class="s1">'base64'</span>
</div><div class="line"><span class="k">def</span> <span class="nf">auth_token</span>
</div><div class="line">  <span class="n">data</span>      <span class="o">=</span> <span class="p">{</span> <span class="n">issued_at</span><span class="p">:</span> <span class="no">Time</span><span class="o">.</span><span class="n">now</span> <span class="p">}</span>
</div><div class="line">  <span class="n">secret</span>    <span class="o">=</span> <span class="no">ENV</span><span class="o">[</span><span class="s1">'AUTH_SECRET'</span><span class="o">]</span>
</div><div class="line">  <span class="n">signature</span> <span class="o">=</span> <span class="no">OpenSSL</span><span class="o">::</span><span class="no">HMAC</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">(</span><span class="s1">'sha256'</span><span class="p">,</span> <span class="no">JSON</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">data</span><span class="p">),</span> <span class="n">secret</span><span class="p">)</span>
</div><div class="line">  <span class="no">Base64</span><span class="o">.</span><span class="n">urlsafe_encode64</span><span class="p">(</span><span class="no">JSON</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">data</span><span class="o">.</span><span class="n">merge</span><span class="p">(</span><span class="n">signature</span><span class="p">:</span> <span class="n">signature</span><span class="p">)))</span>
</div><div class="line"><span class="k">end</span>
</div></code></pre></td></tr></table></div></figure>
<p>This token is used on the API server to authenticate requests. The client can
be made to send a custom header, let’s call it X_APP_AUTH_TOKEN, which it must
be able to reconstruct the token from the JSON data, and then validate that the
request is recent enough. For example in a Sinatra application:</p>
<figure class="code"> <div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
<span class="line">8</span>
<span class="line">9</span>
<span class="line">10</span>
<span class="line">11</span>
<span class="line">12</span>
<span class="line">13</span>
<span class="line">14</span>
</pre></td><td class="code" width="100%"><pre><code class="ruby"><div class="line"><span class="k">def</span> <span class="nf">not_authorized!</span>
</div><div class="line">  <span class="kp">throw</span><span class="p">(</span><span class="ss">:halt</span><span class="p">,</span> <span class="o">[</span><span class="mi">401</span><span class="p">,</span> <span class="s2">"Not authorized</span><span class="se">\n</span><span class="s2">"</span><span class="o">]</span><span class="p">)</span>
</div><div class="line"><span class="k">end</span>
</div><div class="line">
</div><div class="line"><span class="k">def</span> <span class="nf">authenticate!</span>
</div><div class="line">  <span class="n">token</span>           <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">env</span><span class="o">[</span><span class="s2">"HTTP_X_APP_AUTH_TOKEN"</span><span class="o">]</span> <span class="ow">or</span> <span class="n">not_authorized!</span>
</div><div class="line">  <span class="n">token_data</span>      <span class="o">=</span> <span class="no">JSON</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="no">Base64</span><span class="o">.</span><span class="n">decode64</span><span class="p">(</span><span class="n">token</span><span class="p">))</span>
</div><div class="line">  <span class="n">received_sig</span>    <span class="o">=</span> <span class="n">token_data</span><span class="o">.</span><span class="n">delete</span><span class="p">(</span><span class="s1">'signature'</span><span class="p">)</span>
</div><div class="line">  <span class="n">regenerated_mac</span> <span class="o">=</span> <span class="no">OpenSSL</span><span class="o">::</span><span class="no">HMAC</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">(</span><span class="s1">'sha256'</span><span class="p">,</span> <span class="no">JSON</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">token_data</span><span class="p">),</span> <span class="no">ENV</span><span class="o">[</span><span class="s1">'AUTH_SECRET'</span><span class="o">]</span><span class="p">)</span>
</div><div class="line">
</div><div class="line">  <span class="k">if</span> <span class="n">regenerated_mac</span> <span class="o">!=</span> <span class="n">received_sig</span> <span class="o">||</span> <span class="no">Time</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">token_data</span><span class="o">[</span><span class="s1">'issued_at'</span><span class="o">]</span><span class="p">)</span> <span class="o">&gt;</span> <span class="no">Time</span><span class="o">.</span><span class="n">now</span> <span class="o">-</span> <span class="mi">2</span><span class="o">*</span><span class="mi">60</span>
</div><div class="line">    <span class="n">not_authorized!</span>
</div><div class="line">  <span class="k">end</span>
</div><div class="line"><span class="k">end</span>
</div></code></pre></td></tr></table></div></figure>
<p><span class="pullquote-right" data-pullquote="Do not reimplement this, just use fernet.">
In the above code, we consider a token invalid if it was issued more than 2
minutes ago. Real applications will probably include more data in the auth
token, such as the email address or some user identifier that can be used for
auditing and whitelisting.  <em>All of the above data token generation and
verification has been extracted to a handy little gem called
fernet</em>.
Do not reimplement this, just use fernet. In addition to HMAC signature,
fernet also makes it easy to encrypt the message’s payloads, which opens it up
for other interesting use cases.
</span></p>
<p>The <code>authenticate!</code> method must be invoked before serving any request. This
means that the auth token must be included on every request the client makes.
There are many ways of doing this. One approach, if you’re using JQuery to back
Backbone.sync(), is to use its $.ajax beforeSend hook to include the header, as
can be seen in the following coffeescript two-liner:</p>
<figure class="code"> <div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
</pre></td><td class="code" width="100%"><pre><code class="javascript"><div class="line"><span class="nx">$</span><span class="p">.</span><span class="nx">ajaxSetup</span> <span class="nx">beforeSend</span><span class="o">:</span> <span class="p">(</span><span class="nx">jqXHR</span><span class="p">,</span> <span class="nx">settings</span><span class="p">)</span> <span class="o">-&gt;</span>
</div><div class="line">  <span class="nx">jqXHR</span><span class="p">.</span><span class="nx">setRequestHeader</span> <span class="s2">"x-app-auth-token"</span><span class="p">,</span> <span class="nx">App</span><span class="p">.</span><span class="nx">authToken</span>
</div></code></pre></td></tr></table></div></figure>
<p>App.authToken can come from a number of places. I decided to bootstrap it
when the page is originally served, something like:</p>
<figure class="code"> <div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
</pre></td><td class="code" width="100%"><pre><code class="javascript"><div class="line"><span class="o">&lt;</span><span class="nx">script</span> <span class="nx">type</span><span class="o">=</span><span class="s2">"text/javascript"</span><span class="o">&gt;</span>
</div><div class="line">  <span class="nx">App</span><span class="p">.</span><span class="nx">authToken</span> <span class="o">=</span> <span class="s2">"&lt;%= auth_token %&gt;"</span><span class="p">;</span>
</div><div class="line"><span class="o">&lt;</span><span class="err">/script&gt;</span>
</div></code></pre></td></tr></table></div></figure>
<p>In addition to that, it should be updated in an interval, so that on a single
page app, that doesn’t request any page refreshes, the auth token is always
fresh and subsequent API requests can be made.</p>
<p>The final client side code that provides the auth token and keeps it updated
looks like so:</p>
<figure class="code"> <div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
<span class="line">8</span>
<span class="line">9</span>
</pre></td><td class="code" width="100%"><pre><code class="javascript"><div class="line"><span class="o">&lt;</span><span class="nx">script</span> <span class="nx">type</span><span class="o">=</span><span class="s2">"text/javascript"</span><span class="o">&gt;</span>
</div><div class="line">  <span class="nx">App</span><span class="p">.</span><span class="nx">authToken</span> <span class="o">=</span> <span class="s2">"&lt;%= auth_token %&gt;"</span><span class="p">;</span> <span class="c1">//bootstrap an initial value</span>
</div><div class="line">  <span class="nx">App</span><span class="p">.</span><span class="nx">refresh_auth_token</span> <span class="o">=</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
</div><div class="line">    <span class="nx">$</span><span class="p">.</span><span class="nx">getJSON</span><span class="p">(</span><span class="s1">'/auth_token'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span> <span class="p">{</span>
</div><div class="line">      <span class="nx">App</span><span class="p">.</span><span class="nx">authToken</span> <span class="o">=</span> <span class="nx">data</span><span class="p">.</span><span class="nx">token</span><span class="p">;</span> <span class="c1">//request updated values</span>
</div><div class="line">    <span class="p">})</span>
</div><div class="line">  <span class="p">};</span>
</div><div class="line">  <span class="nb">window</span><span class="p">.</span><span class="nx">setInterval</span><span class="p">(</span><span class="nx">App</span><span class="p">.</span><span class="nx">refresh_auth_token</span><span class="p">,</span> <span class="mi">29000</span><span class="p">);</span> <span class="c1">//every 29 seconds</span>
</div><div class="line"><span class="o">&lt;</span><span class="err">/script&gt;</span>
</div></code></pre></td></tr></table></div></figure>
<p>The <code>/auth_token</code> server side endpoint simply responds with a new valid
<code>token</code>.</p>
<p>The fernet token expires every minute by default. I decided to update it
every 29 seconds instead so that it can be updated at least twice
before it has a chance to hold and use an expired token against a remote API.</p>
<p>In this app, the server side is used for one thing only: user authentication.
The way it works is that when a request is made, the sinatra app performs oauth
authentication against our google apps domain. Once the oauth dance has
suceeded, the app generates a token that is handed on to the client for
authenticating against backend, remote APIs.</p>
<p>This whole setup has worked great for some months now.</p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2012/07/31/on-top-down-design/">On Top-down Design</a></h1>
<p class="meta">
<time data-updated="true" datetime="2012-07-31T20:54:00-07:00" pubdate="">Jul 31<span>st</span>, 2012</time>
</p>
</header>
<div class="entry-content"><p>There are many strategies for writing software.</p>
<p>Some developers like writing tests first, letting your tests drive the implementation, essentially becoming the first clients of your software. Others like writing production code first and kind of figure out how it works, and then they apply some tests after the fact. I am convinced that writing tests in this way is far less effective, but this is not an article on the merits of TDD. Others don’t like writing tests at all, so it’s a varying scale.</p>
<p>You can write software from the bottom up, where you are forced to figure out what the data model and base objects should be and how it is to support all known use cases. This is hard, particularly in this day and age where software requirements are unpredictable and are meant to change very rapidly - at least in the fun industries.</p>
<p>You can also write software from the top down. In this case you let the outer shell of your software dictate what the inner layers need. For example, start all the way on the user interactions via some sketches and HTML/CSS in a web app. Or think through the CLI that you want. Readme driven development can help with this, too.</p>
<p>Outside-in development puts you in a great position to practice top-down design.</p>
<p>The advantage of Outside-in development is twofold. Not only are you left with acceptance tests that will catch regressions and help refactor later. But also, <em>the top layers of your software becomes a client of the code you are about to write, helping define its interface</em> in a similar way that practicing TDD for your unit tests help guide the design of software component at a very granular level.</p>
<p>These practices will help you define internal APIs that feel good faster, because you will notice cumbersome interfaces sooner, and are therefore given the opportunity to fix them when they have the least possible number of dependencies.</p>
<p>I know of many developers who prefer writing no tests, or prefer a bottom-up strategy. This does not mean that their software is of poor quality, by no means. But I will observe that I can’t ship software of the quality standard that I put myself to unless I write tests first and follow outside-in methodologies. Indeed, this seems to indicate they are smarter than me.</p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2012/02/05/boston-dot-io-recap/">Boston.io Recap</a></h1>
<p class="meta">
<time data-updated="true" datetime="2012-02-05T10:59:00-08:00" pubdate="">Feb 5<span>th</span>, 2012</time>
</p>
</header>
<div class="entry-content"><p>Boston.io took place yesterday at the Microsoft NERD Center. The event is aimed at students in the Greater Boston area who are interested in entrepreneurship and coding, whether that’s design, development, or ambidextrous.</p>
<p>There were technical and talks on every aspect of the stack, from the metal all the way up to serving and consuming APIs and user experience design. There were also some not-so-technical talks about topics like the Boston startup scene, open source and hacker culture.</p>
<p>My talk was about PostgreSQL and it focused on how to use SQL to mine data and get information out of it. I stored the twitter stream for about 24 hours prior to the conference and showed how to look at that data showcasing CTEs, Window Functions and other Postgres features. Among the insights we saw that the tweeps who post hashtags and urls the most are spam accounts. Also, the most posted URLs come from shortener services, but also surprised to see livingsocial.com among those. We found other fun facts during the talk too. It went great.</p>
<p>Hopefully the talk gave a good taste of what you can do with Postgres and SQL. It was SQL heavy, but that did not come without warning</p>
<blockquote class="twitter-tweet"><p>boston.io talk ready to go. SQL, do you speak it? After this you will!</p>— Harold Giménez (@hgimenez) February 4, 2012</blockquote>
<script data-cfasync="false" src="/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js"></script><script charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<p>Other talks worth mentioning included Mike Burns’ classic UNIX talk. This time around he used curl, sed and grep to automate a SQL injection attack on a hilarious page he staged for this purpose. Erik Michaels-Ober’s talk was also great and surely inspired a few students to put code out there for the world to see. So many great talks altogether though, by people like Nick Quaranto on TDD, Ben Orenstein on vim, Bryan Lyles on OOP and more.</p>
<p>Oh, and there was a dude at the afterparty with a heroku shirt. Here’s a photo.</p>
<p></p>
<p>Overall a great event; thanks to organizers thoughtbot and Greenhorn Connect and drinkup sponsors hashrocket and github.</p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2012/01/27/ssh-tunnels-for-remote-pairing/">SSH Tunnels for Remote Pairing</a></h1>
<p class="meta">
<time data-updated="true" datetime="2012-01-27T08:51:00-08:00" pubdate="">Jan 27<span>th</span>, 2012</time>
</p>
</header>
<div class="entry-content"><p>Yesterday was a good day. @pvh and I paired for a few
hours, even though we’re at opposite coasts.</p>
<p>Here’s what you need:</p>
<ul>
<li>A server somewhere that both pairs have access to. We used an EC2 instance.
We’ll use it to meet up and create the tunnel.</li>
<li>SSH client libs - your should already have this unless you’re on windows in
which case you probably want PuTTY.</li>
<li>Skype for audio.</li>
</ul>
<p>As you know, The Internet is made of nodes connected by Tubes. Unfortunately,
there is no tube from your machine to your pair’s machine. What we’ll do here is
use a third node that has tubes to both of your machines to relay traffic
through, in essence creating an Internet Tube from your machine to your pair’s.
This kind of Internet Tube is called a Tunnel. Since we’re using SSH to do the
traffic encryption and forwarding, it’s an SSH Tunnel. Yes, that’s somewhat made
up, but sounds legit!</p>
<p></p>
<p>As it turns out, setting up a tunnel is fairly simple. For example, let’s set
up a tunnel between you and Jane using a remote server saturn.</p>
<ol>
<li>You: Open up a shell and forward traffic to your local port 9999 over to
Saturn’s port 1235:</li>
</ol>
<figure class="code"><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
</pre></td><td class="code" width="100%"><pre><code class=""><div class="line">ssh -L 9999:locahost:1235 saturn_user@saturn</div></code></pre></td></tr></table></div></figure>
<ol>
<li>Jane: Open up a shell and forward traffic from saturn’s port 1235 to her port
22</li>
</ol>
<figure class="code"><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
</pre></td><td class="code" width="100%"><pre><code class=""><div class="line">ssh -R 1235:localhost:22 saturn_user@saturn</div></code></pre></td></tr></table></div></figure>
<ol>
<li>You: Open up another shell, and ssh into your local port 9999 specifying a
username on Jane’s machine.</li>
</ol>
<figure class="code"><div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
</pre></td><td class="code" width="100%"><pre><code class=""><div class="line">ssh jane_user@jane -p 9999</div></code></pre></td></tr></table></div></figure>
<p>And you’re good to go. Create a shared screen session, open up $EDITOR, use
skype, google hangouts, face time or whatever for audio and start ping ponging.</p>
<p>The latency was surprisingly minimal. We left this tunnel open most of the day.
It sat idle for periods of time and the connection was left active. All in all,
a great setup.</p>
<p>If this is something you’ll do often, you might as well add a more permanent
configuration to <code>~/.ssh/config</code>. For example, you might add:</p>
<figure class="code"><figcaption><span>~/.ssh/config on your machine </span></figcaption>
<div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
<span class="line">5</span>
<span class="line">6</span>
<span class="line">7</span>
<span class="line">8</span>
<span class="line">9</span>
</pre></td><td class="code" width="100%"><pre><code class=""><div class="line">Host tunnel_to_jane
</div><div class="line">  Hostname saturn
</div><div class="line">  LocalForward 9999:localhost:1235
</div><div class="line">  User saturn_user
</div><div class="line">
</div><div class="line">Host jane
</div><div class="line">  Hostname jane
</div><div class="line">  User jane_user
</div><div class="line">  Port 9999</div></code></pre></td></tr></table></div></figure>
<p>Then you’d do, on one terminal, <code>ssh tunnel_to_jane</code>, and on the
other <code>ssh jane</code>.</p>
<p>And Jane might add:</p>
<figure class="code"><figcaption><span>~/.ssh/config on Jane’s machine </span></figcaption>
<div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
<span class="line">2</span>
<span class="line">3</span>
<span class="line">4</span>
</pre></td><td class="code" width="100%"><pre><code class=""><div class="line">Host tunnel_from_you
</div><div class="line">  Hostname saturn
</div><div class="line">  RemoteForward 1235:localhost:22
</div><div class="line">  User saturn_user</div></code></pre></td></tr></table></div></figure>
<p>And she’d just do, <code>ssh tunnel_from_you</code></p>
<p>This can be used not ony for remote pairing, but rather to forward <em>any</em>
traffic on a port, over an SSH encrypted channel, to a remote host. For more
see <code>ssh_config(5)</code> and <code>ssh(1)</code>, and happy pairing!</p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2011/09/19/redis-talk-at-postgresql-conf-west/">Redis Talk at PostgreSQL Conf West</a></h1>
<p class="meta">
<time data-updated="true" datetime="2011-09-19T10:00:00-07:00" pubdate="">Sep 19<span>th</span>, 2011</time>
</p>
</header>
<div class="entry-content"><p>In case you’ve missed it so far, PostgreSQL West will take place next week in San Jose, California. You should go.</p>
<p>This weekend I worked on the slides and content for my Redis talk. Why would I decide to speak about Redis in a PostgreSQL conference? As it turns out, we’ve had great success in using Redis to compliment our architecture, not to replace a main data store. I will speak about our experience in scaling out write heavy apps with Redis.</p>
<p>I will first introduce Redis from a general perspective, and then dig into the data types it offers and the operations it can do on each data type. During the course of the talk, I will demonstrate how we’ve used certain data types to solve some rather tricky scaling problems: real use cases solving real problems.</p>
<p>I hope this talk will be entertaining and informative to both DBAs and application developers. I will be sharing the slides here and on twitter afterwards, so stay tuned.</p>
<p>Additionally, on Tuesday I will be teaching a one day workshop on Ruby and Rails with a PostgreSQL focus. This is the second time I will do this at PostgreSQL conf, the last time being at PostgreSQL conf east in New York City. The class size was small, and the feedback I received was very positive in that attendees got a good grasp of the Ruby programming language and how Rails and Postgres fit in the ecosystem. I hope this time around is even better.</p>
<p>Hope to see you there.</p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2011/09/12/postgresql-9-dot-1-released/">PostgreSQL 9.1 Released</a></h1>
<p class="meta">
<time data-updated="true" datetime="2011-09-12T09:18:00-07:00" pubdate="">Sep 12<span>th</span>, 2011</time>
</p>
</header>
<div class="entry-content"><p>Version 9.1 of PostgreSQL was released yesterday.</p>
<p>Among the exciting new features there is:</p>
<ul>
<li><p><code>pg_basebackup</code> - this can be used alongside Streaming Replication to perform a backup or clone of your database. I can imagine heroku adopting this as an even faster and reliable way to sync your production and staging databases, for example (when and if they upgrade to 9.1). However it can also be used to create plain old tarballs and create standalone backups.</p></li>
<li><p>Another replication goodie: Synchronous replication. On Postgres 9.0, replication was asynchronous. By enabling synchronous replication, you are basically telling the master database to only report the transaction as committed when the slave(s) have successfully written it to its own journal. You can also control this on a specific session by doing <code>SET synchronous_commit TO off</code>.</p></li>
<li><p>The new <code>pg_stat_replication</code> view displays the replication status of all slaves from a master node.</p></li>
<li><p>Unlogged tables. What? Postgres supports unlogged tables now? Yes, it does. They can be used for data you don’t necessarily care about, and a crash will truncate all data. They are much faster to write to and could be useful for caching data or keeping stats that can be rebuilt. You can create them like so:</p></li>
</ul>
<figure class="code"> <div class="highlight"><table cellpadding="0" cellspacing="0"><tr><td class="gutter"><pre class="line-numbers"><span class="line">1</span>
</pre></td><td class="code" width="100%"><pre><code class="sql"><div class="line"><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">UNLOGGED</span> <span class="n">some_stats</span><span class="p">(</span><span class="n">value</span> <span class="nb">int</span><span class="p">)</span>
</div></code></pre></td></tr></table></div></figure>
<ul>
<li><p>SQL/MED gets the ability to define foreign tables. This is rich. It means that you can define a foreign data wrapper for a different database and access it from Postgres seamlessly. Some hackers have already built some nifty foreign data wrappers for mysql, oracle, and even redis and couchdb. Although I’m of the mind that if you’re actually using any of these databases to supplement your app’s data needs, just talk to them directly from your app. However, it may be possible to write some higher performance reports that use different data sources, and you let Postgres do the heavy lifting of munging all the data together.</p></li>
<li><p>You no longer need to specify all columns on a select list from your GROUP BY clause: functionally dependent columns are inferred by the planner, so specifying a primary key is sufficient. I <a href="blog/2009/09/23/postgresql-s-group-by/">talked about this before</a>, and it’s a cause of great frustration to users coming from MySQL.</p></li>
</ul>
<p>There’s much more in this release. Here are the release notes.</p>
<p>Huge thanks, Postgres hackers!</p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2011/09/10/design-tweaks/">Design Tweaks</a></h1>
<p class="meta">
<time data-updated="true" datetime="2011-09-10T13:46:00-07:00" pubdate="">Sep 10<span>th</span>, 2011</time>
</p>
</header>
<div class="entry-content"><p>Today I made a few design tweaks to this blog. The goal is to move a bit away from the stock default octopress theme - but my design chops aren’t all that great and I can’t really budget the time to do a design from scratch.</p>
<p>A few simple changes and I’m OK with how it looks for now:</p>
<ol>
<li><p>I created the basic font logo that you now see on the header. This was the idea from the beginning, but I never had a chance to include it here. Commit: 27d0107f.</p></li>
<li><p>Change the typography in the site, as the original seems a bit heavy for my taste. I started out by using a Helvetica Neue on headers, but decided to go with Antic from the Google web font service. Commits: 2dd8f46b4 and 84bad437.</p></li>
<li><p>I wanted something different for that dark background. Went hunting for tiles and patterns. There’s good stuff out there, but I settled for the dark wood background found here. Commit: 9593a00431. At this point, it also made sense to make the header and footer have a light background and dark font.</p></li>
<li><p>Finally, added a gradient to the header. Commit fe918b0e</p></li>
</ol>
<p>I’m fine with the result for now, but will probably revisit soon. Here’s a before and after.</p>
<h3>Before</h3>
<p><img alt="Before" src="images/posts/before-resized.png"/></p>
<h3>After</h3>
<p><img alt="After" src="images/posts/after-resized.png"/></p>
</div>
</article>
<article>
<header>
<h1 class="entry-title"><a href="blog/2011/09/07/you-should-work-for-these-guys/">You Should Work for These Guys</a></h1>
<p class="meta">
<time data-updated="true" datetime="2011-09-07T13:42:00-07:00" pubdate="">Sep 7<span>th</span>, 2011</time>
</p>
</header>
<div class="entry-content"><p>For the last few months we’ve been working for an awesome company on a greenfield project here in Boston/Cambridge.</p>
<p>The industry: Healthcare. The goal: Improve patient’s lives by changing the way <em>the entire system</em> works. It’s exciting, and it is happening, and you can be a part of it.</p>
<h3>The stack</h3>
<p>Rails 3.1, Backbone.js, Coffeescript, faye, PostgreSQL, Cucumber, RSpec and Jasmine.</p>
<h3>The process</h3>
<p>Daily standups, TDD, code reviews via github pull requests.</p>
<h3>The result</h3>
<p>A highly responsive non-trivial app with a very clean code base and beautiful design.</p>
<p>The future holds a mobile app, web service integrations and ongoing maintenance to the current code base.</p>
<p>If you live in the Boston area, you should apply. If you don’t, you should move here. Right here.</p>
</div>
</article>
<div class="pagination">
<a class="prev" href="blog/page/2/">← Older</a>
<a href="blog/archives/">Blog Archives</a>
</div>
</div>
<aside class="sidebar">
<section>
<p>Harold Giménez builds Postgres infrastructure at the Heroku Department of Data.</p>
<img alt="Heroku Postgres" src="images/heroku-postgres.png" width="260px"/>
<!--<p><em>We’re hiring.</em> You should <a onClick="_gaq.push(['_trackEvent', 'External Link', 'Apply for DoD Job', 'Generic']);" href="http://heroku.theresumator.com/apply/wwPdDw/Data-Infrastructure-Developer.html">apply</a>.</p>&#8211;>
</section>
<section>
  <h1>GitHub Repos</h1>
  <ul id="gh_repos">
    <li class="loading">Status updating&#8230;</li>
  </ul>
  <a href="https://github.com/hgmnz">@hgmnz</a> on GitHub
  <script type="text/javascript">
    $.domReady(function(){
        if (!window.jXHR){
            var jxhr = document.createElement('script');
            jxhr.type = 'text/javascript';
            jxhr.src = '/javascripts/libs/jXHR.js';
            var s = document.getElementsByTagName('script')[0];
            s.parentNode.insertBefore(jxhr, s);
        }
        github.showRepos({
            user: 'hgmnz',
            count: 4,
            skip_forks: true,
            target: '#gh_repos'
        });
    });
  </script>
  <script src="/javascripts/github.js" type="text/javascript"> </script>
</section>
<section>
  <h1>Latest Tweets</h1>
  <ul id="tweets">
    <li class="loading">Status updating&#8230;</li>
  </ul>
  <script type="text/javascript">
    $.domReady(function(){
      getTwitterFeed("hgmnz", 4, false);
    });
  </script>
  <script src="/javascripts/twitter.js" type="text/javascript"> </script>
    <a href="http://twitter.com/hgmnz" class="twitter-follow-button" data-show-count="false">Follow @hgmnz</a>
</section>
<section class="googleplus">
  <h1>
    <a href="https://plus.google.com/110441654497003669801?rel=author">
      <img src="http://www.google.com/images/icons/ui/gprofile_button-32.png" width="32" height="32">
      Google+
    </a>
  </h1>
</section>
<section>
  <h1>Recent Posts</h1>
  <ul id="recent_posts">
      <li class="post">
        <a href="/blog/2013/07/12/recursive-query-is-recursive/">RECURSIVE Query is Recursive</a>
      </li>
      <li class="post">
        <a href="/blog/2013/07/06/distributed-locking-in-postgres/">Distributed locking in Postgres</a>
      </li>
      <li class="post">
        <a href="/blog/2012/10/27/client-side-api-mashups-with-cors/">Client Side API Mashups With CORS</a>
      </li>
      <li class="post">
        <a href="/blog/2012/07/31/on-top-down-design/">On Top-down Design</a>
      </li>
      <li class="post">
        <a href="/blog/2012/02/05/boston-dot-io-recap/">Boston.io Recap</a>
      </li>
  </ul>
</section>
</aside>
    </div>
  </div>
  <footer role="contentinfo"><p class="copyright-and-license">
  Copyright &copy; 2006 - 2013 - Harold Giménez.
  All original content released under the <a href="http://creativecommons.org/licenses/by-sa/3.0/">Creative Commons Attribution ShareAlike license</a>. Powered by <a href="http://octopress.org">Octopress</a>.
</p>
</footer>
  <script type="text/javascript">
    (function() {
      var script = document.createElement('script'); script.type = 'text/javascript'; script.async = true;
      script.src = 'https://apis.google.com/js/plusone.js';
      var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(script, s);
    })();
  </script>
  <script type="text/javascript">
    (function(){
      var twitterWidgets = document.createElement('script');
      twitterWidgets.type = 'text/javascript';
      twitterWidgets.async = true;
      twitterWidgets.src = 'http://platform.twitter.com/widgets.js';
      document.getElementsByTagName('head')[0].appendChild(twitterWidgets);
    })();
  </script>
</body>
</html>
--></section></aside></div></div></body></html>