<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:thr="http://purl.org/syndication/thread/1.0">
    <title>Cozi Tech Blog</title>
    <link rel="self" type="application/atom+xml" href="http://blogs.cozi.com/tech/atom.xml" />
    <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/" />
    <id>tag:typepad.com,2003:weblog-1652090</id>
    <updated>2013-11-13T21:57:40-08:00</updated>
    
    <generator uri="http://www.typepad.com/">TypePad</generator>
    <entry>
        <title>Turn off Windows Defender on your builds</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2013/11/turn-off-windows-defender-on-your-builds.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2013/11/turn-off-windows-defender-on-your-builds.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-6a00d8341ca8a653ef019b010e97f5970b</id>
        <published>2013-11-13T21:57:40-08:00</published>
        <updated>2013-11-13T21:58:12-08:00</updated>
        <summary>I spent some time last weekend profiling a Python application on Windows, trying to find out why it was so much slower than on Mac or Linux. The application is an in-house build tool which reads a number of config...</summary>
        <author>
            <name>George V. Reilly</name>
        </author>
        <category scheme="http://www.sixapart.com/ns/types#category" term="Python" />
        
        
<content type="xhtml" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
<div xmlns="http://www.w3.org/1999/xhtml"><p>I spent some time last weekend profiling a Python application on Windows, trying to find out why it was so much slower than on Mac or Linux. The application is an in-house build tool which reads a number of config files, then writes some output files.</p>
<p>Using the <a class="reference external" href="http://www.vrplumber.com/programming/runsnakerun/">RunSnakeRun</a> Python profile viewer on Windows, two things immediately leapt out at me: we were running <tt class="docutils literal">os.stat</tt> a lot and <tt class="docutils literal">file.close</tt> was really expensive.</p>
<p>A quick test convinced me that we were <a class="reference external" href="http://docs.python.org/2/library/stat.html">stat-ing</a> the same files over and over. It was a combination of explicit checks and implicit code, like <tt class="docutils literal">os.walk</tt> calling <tt class="docutils literal">os.path.isdir</tt>. I wrote a little cache that memoizes the results, which brought the cost of the <tt class="docutils literal">os.stats</tt> down from 1.5 seconds to 0.6 seconds.</p>
<p>Figuring out why closing files was so expensive was harder. I was writing 77 files, totaling just over 1MB, and it was taking 3.5 seconds. It turned out that it wasn&#39;t the UTF-8 codec or newline translation. It was simply that <em>closing</em> those files took far longer than it should have.</p>
<p>I decided to try a different profiler, hoping to learn more. I downloaded the <a class="reference external" href="http://msdn.microsoft.com/en-us/library/windows/hardware/hh448170.aspx">Windows Performance Toolkit</a>. I recorded a couple of traces of my application running, then I looked at them in the Windows Performance Analyzer, whereupon I saw that in each case, the CPU spike of my app was followed by a CPU spike in MsMpEng.exe.</p>
<p><a class="reference external" href="http://helpdeskgeek.com/windows-vista-tips/what-is-msmpeng-exe/">What&#39;s MsMpEng.exe</a>? It&#39;s Microsoft&#39;s antimalware engine, at the heart of Windows Defender. I added my build tree to the list of excluded locations, and my runtime <em>halved</em>. The 3.5 seconds of file closing dropped to 60 milliseconds, a 98% reduction.</p>
<p>The moral of this story is: don&#39;t let your virus checker run on your builds.</p>
<p><img alt="Windows Defender: Exclude directory for scanning" src="http://www.georgevreilly.com/blog/content/binary/WindowsDefender-FileExclusion.png" /></p></div>
</content>


    </entry>
    <entry>
        <title>Setting up Git in a headless Windows environment</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2011/09/setting-up-git-in-a-headless-windows-environment.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2011/09/setting-up-git-in-a-headless-windows-environment.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-6a00d8341ca8a653ef015391ef4027970b</id>
        <published>2011-09-28T15:04:20-07:00</published>
        <updated>2011-09-28T15:04:20-07:00</updated>
        <summary>The documentation for setting up Git to work well in a headless Windows environments is surprisingly sparse and the process is extremely frustrating (in my experience). Hopefully this will help! Setup Git It is advisable to run services (like a...</summary>
        <author>
            <name>Thomas Van Doren</name>
        </author>
        <category scheme="http://www.sixapart.com/ns/types#category" term="Git" />
        
        
<content type="xhtml" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
<div xmlns="http://www.w3.org/1999/xhtml"><p>The documentation for setting up Git to work well in a headless Windows environments is surprisingly sparse and the process is extremely frustrating (in my experience). Hopefully this will help!</p>
<h2>Setup Git</h2>
<div>It is advisable to run services (like a build service) as a pre-defined user, as opposed to the SYSTEM user. However, if Git really must be used as the SYSTEM user, the following will help emulate that environment.</div>
<ul>
<li>To run commands as the SYSTEM user, you can use psexec.exe from <a href="http://technet.microsoft.com/en-us/sysinternals/bb545027" target="_blank" title="SysInternals Index">SysInternals</a>.            
<ul>
<li>From an Administrator cmd.exe prompt, psexec -i -s cmd.exe will open a new shell as the SYSTEM user.</li>
</ul>
</li>
</ul>
<h2>General Advice when Setting Up Git</h2>
<ul>
<li>Define a HOME env var equal to %USERPROFILE%.</li>
<li>Create <em>passphrase-less</em> rsa keys and put them in %HOME%/.ssh. These keys should be setup on whatever server hosts the Git repos. In GitHub, for example, you would need to add the public keys to your account.</li>
<li>Do an initial ssh git@github.com to add GitHub to the known_hosts.</li>
<li>Get rid of any GIT_SSH env vars if using the default OpenSSL ssh client for auth (as opposed to plink.exe, etc). GIT_SSH=c:\…\plink.exe may exist if you have previously used PuTTY/Pageant/TortoiseGit/etc to access Git repos.</li>
<li>ssh git@github.com (or wherever your repo is) is very useful for debugging. One to three -v flags (i.e. ssh -vv git@github.com) may be added to help debug the connection process.</li>
<li>Set the %HOME%/.ssh/config to specify which authentication to use:</li>
</ul>
<pre>Host github.com
    User git
    Hostname github.com
    PreferredAuthentications publickey
</pre>
<ul>
<li>If you see the following error message and your files do have the correct perms (0600), then you are suffering from a <a href="http://code.google.com/p/msysgit/issues/detail?id=261#c46" rel="nofollow" target="_blank" title="(external link)">bug in the msysgit ssh</a> executable. Unix permissions (0644) don&#39;t map to NTFS ACLs. Msys just fakes the behavior of chmod, but it can&#39;t fake a chmod to a restrictive enough permissions set. Steps to fix are below.</li>
</ul>
<pre>@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for &#39;/path/to/key&#39; are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /path/to/key
</pre>
<div>
<ul>
<li>Assuming cygwin is installed at c:\cygwin and msysgit is installed at c:\Program Files\Git, this will replace the ssh executable in msysgit with the one from cygwin, which recognizes file perms:</li>
</ul>
<pre>@rem From an Administrator cmd.exe
@rem This works for 32bit Windows. Adjust accordingly for 64bit.
c:
ren &quot;C:\Program Files\Git\bin\ssh.exe&quot; &quot;C:\Program Files\Git\bin\ssh.bak.exe&quot;
copy &quot;C:\cygwin\bin\ssh.exe&quot; &quot;C:\Program Files\Git\bin\ssh.exe&quot;
copy &quot;C:\cygwin\bin\cyg*.dll&quot; &quot;C:\Program Files\Git\bin\&quot;</pre>
</div>
<div>
<h2>Some Sources</h2>
<ul>
<li><a href="https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin#GitPlugin-" target="_blank">Jenkins, GIT Plugin and Windows</a></li>
<li><a href="http://help.github.com/win-set-up-git/" target="_blank">GitHub&#39;s Windows Git Setup</a></li>
<li><a href="http://www.eng.cam.ac.uk/help/jpmg/ssh/authorized_keys_howto.html" target="_blank">University of Cambridge – ssh authorized_keys HOWTO</a></li>
</ul>
</div>
<p>*** This is an excerpt from <a href="http://blog.thomasvandoren.com/archives/95" target="_blank" title="Jenkins Windows Slave and Git">Jenkins Windows Slave and Git</a>, originally published on <a href="http://blog.thomasvandoren.com/" target="_blank" title="Thomas Van Doren&#39;s blog">Thomas Van Doren&#39;s blog</a>.</p>
<p><em>Appreciated feedback from: <a href="http://weblogs.asp.net/george_v_reilly/" target="_blank">George Reilly</a></em></p></div>
</content>


    </entry>
    <entry>
        <title>Git fsck Is Your Friend, or &quot;How I recovered lost commits in Git&quot;</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2011/07/git-fsck-is-your-friend-or-how-i-recovered-lost-commits-in-git.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2011/07/git-fsck-is-your-friend-or-how-i-recovered-lost-commits-in-git.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-6a00d8341ca8a653ef01538fab87d5970b</id>
        <published>2011-07-05T12:56:48-07:00</published>
        <updated>2011-07-05T12:59:30-07:00</updated>
        <summary>The following is a little tale of mine about how to uncover lost work in Git. If you just want the answer, skip to the end. I realized that some of my code in the Git tree was not working...</summary>
        <author>
            <name>Tristan Eastburn</name>
        </author>
        <category scheme="http://www.sixapart.com/ns/types#category" term="Git" />
        
        
<content type="html" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
&lt;div xmlns=&quot;http://www.w3.org/1999/xhtml&quot;&gt;&lt;div class=&quot;ts-tech&quot;&gt;
&lt;p&gt;The following is a little tale of mine about how to uncover lost work in &lt;a href=&quot;http://en.wikipedia.org/wiki/Git_(software)&quot; target=&quot;_blank&quot;&gt;Git&lt;/a&gt;. If you just want the answer, skip to the end.&lt;/p&gt;
&lt;p&gt;I realized that some of my code in the Git tree was not working as expected, so I went into the code to look at it and... &lt;strong&gt;WHERE ARE MY CHANGES&lt;/strong&gt;?!?! The code looks as if I never even touched it! I could have sworn I had committed and pushed my changes long ago, but apparently I had not. Now going back to look for them, the local branch I was working on no longer exists. Whether I deleted the branch or not, I don’t remember, but in any case, it&#39;s not there. &lt;strong&gt;FREAK OUT&lt;/strong&gt; ensues. I frantically start combing &lt;code class=&quot;inline&quot;&gt;git log&lt;/code&gt;, GitHub, gitk and gitg for any evidence of my old changes. Nothing. More freaking out. The Git Gnomes are out to get me!!! Luckily for me, something called The Internet exists and I can search for things on it. I find &lt;a title=&quot;How to Recover Lost Git Branches&quot; href=&quot;http://techblog.ironfroggy.com/2007/12/how-to-recover-lost-git-branches.html&quot; target=&quot;_blank&quot;&gt;a page&lt;/a&gt; that tells me &lt;code class=&quot;inline&quot;&gt;git reflog&lt;/code&gt; can save me.&lt;/p&gt;
&lt;br /&gt; &lt;code&gt; git reflog show master&lt;br /&gt; git reflog show my_old_branch&lt;br /&gt; git help reflog&lt;br /&gt; git reflog --all&lt;br /&gt; &lt;/code&gt; &lt;br /&gt;
&lt;p&gt;Good try! But to no avail. None of the messages show my commits or any existence of the hideous, horrible ol&#39; my_old_branch. Those damned gnomes have run off with my codes! More searching of Los Internetes. Then I come across &lt;a title=&quot;Permanent Link to The illustrated guide to recovering lost commits with Git&quot; href=&quot;http://programblings.com/2008/06/07/the-illustrated-guide-to-recovering-lost-commits-with-git/&quot; target=&quot;_blank&quot;&gt;The illustrated guide to recovering lost commits with Git&lt;/a&gt;. After a decent amount of relatively useless reading not unlike the first part of this little tale, I come to the Recovering a Lost Commit section. &quot;&lt;code class=&quot;inline&quot;&gt;git fsck&lt;/code&gt; is your savior, Tristan.&quot; It says.&lt;/p&gt;
&lt;br /&gt; &lt;code&gt; git fsck --lost-found&lt;br /&gt; &lt;/code&gt; &lt;code class=&quot;output&quot;&gt; dangling commit 2846188e27ed330bc2925509d926835b783382cc&lt;br /&gt; dangling blob 9654b4c62f6b4e129b1f066efb0349f37fd35ac2&lt;br /&gt; dangling commit 5c7e8ca2f9ebdbcbc2685170bab9210748015a36&lt;br /&gt; ...&lt;br /&gt; dangling commit 7ccd8f8d6bfed71248f43091fd6ffac0afa571e0&lt;br /&gt; dangling commit 88d907d7b03936c9dd843673d67d70cab50e427b&lt;br /&gt; dangling blob 99f73f63b515398a0702bc2975ff3c0fb347084a&lt;br /&gt; &lt;/code&gt; &lt;br /&gt;
&lt;p&gt;I get some 38 lines of old commits and blobs that seem to have come out of nowhere. Doing a &lt;code class=&quot;inline&quot;&gt;git show&lt;/code&gt; on each one appears to be getting me nowhere... until!&lt;/p&gt;
&lt;br /&gt; &lt;code&gt; git show fe475203edfd01b7007809c82310077b0fe87750 &lt;/code&gt; &lt;br /&gt;
&lt;p&gt;Yes! My changes! And not just that, as I go on I see each of the 8 commits I made to good ol&#39; my_old_branch. The evil Git Gnomes have been vanquished and I have conquered the world of Gitarnia™! After a quick &lt;code class=&quot;inline&quot;&gt;git merge&lt;/code&gt; my changes are in place! An hour lost, a year of my life bequeathed back unto me.&lt;/p&gt;
&lt;p&gt;In conclusion, &lt;code class=&quot;inline&quot;&gt;git fsck --lost-found&lt;/code&gt; is your friend, treat her well!&lt;/p&gt;
&lt;/div&gt;
&lt;p&gt;&amp;nbsp;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: &lt;code&gt;git fsck --lost-found&lt;/code&gt; may not work if you have run &lt;code&gt;git gc&lt;/code&gt; recently, as it removes dangling commits.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Sources&lt;/strong&gt;&lt;br /&gt; &lt;a href=&quot;http://techblog.ironfroggy.com/2007/12/how-to-recover-lost-git-branches.html&quot; target=&quot;_blank&quot;&gt;http://techblog.ironfroggy.com/2007/12/how-to-recover-lost-git-branches.html&lt;/a&gt;&lt;br /&gt; &lt;a href=&quot;http://programblings.com/2008/06/07/the-illustrated-guide-to-recovering-lost-commits-with-git/&quot; target=&quot;_blank&quot;&gt;http://programblings.com/2008/06/07/the-illustrated-guide-to-recovering-lost-commits-with-git/&lt;/a&gt;&lt;/p&gt;&lt;/div&gt;
</content>


    </entry>
    <entry>
        <title>SerializationException: the constructor was not found</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2011/04/serializationexception-the-constructor-was-not-found.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2011/04/serializationexception-the-constructor-was-not-found.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-6a00d8341ca8a653ef01538e117e5c970b</id>
        <published>2011-04-22T15:19:36-07:00</published>
        <updated>2011-04-22T15:19:36-07:00</updated>
        <summary>I spent some time earlier this week trying to fix a SerializationException in some .NET code. The fix ultimately turned out to be quite simple, but it took a while for me to understand what the exception message was actually...</summary>
        <author>
            <name>George V. Reilly</name>
        </author>
        <category scheme="http://www.sixapart.com/ns/types#category" term=".NET and C# Development" />
        
        
<content type="xhtml" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
<div xmlns="http://www.w3.org/1999/xhtml"><img alt="Serialization" src="http://blogs.cozi.com/.a/6a00d8341ca8a653ef01538e117a13970b-800wi" />
<p>I spent some time earlier this week trying
to fix a <tt class="docutils literal">SerializationException</tt> in some .NET code.
The fix ultimately turned out to be quite simple,
but it took a while for me to understand what the exception message
was actually telling me.</p>
<p>We were seeing occasional <tt class="docutils literal">SerializationException</tt>s in our logs,
with callstacks like this:</p>
<pre class="literal-block">
System.Runtime.Serialization.SerializationException: The constructor
to deserialize an object of type 'Cozi.WebClient.AccountTagsMap'
was not found.
  at System.Runtime.Serialization.ObjectManager.GetConstructor(
        Type t, Type[] ctorParams)
  at System.Runtime.Serialization.ObjectManager.CompleteISerializableObject(
        Object obj, SerializationInfo info, StreamingContext context)
  --- End of inner exception stack trace ---
  at System.Runtime.Serialization.ObjectManager.CompleteISerializableObject(
        Object obj, SerializationInfo info, StreamingContext context)
  at System.Runtime.Serialization.ObjectManager.FixupSpecialObject(ObjectHolder holder)
  at System.Runtime.Serialization.ObjectManager.DoFixups()
  at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(
        HeaderHandler handler,__BinaryParser serParser, Boolean fCheck,
        Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
  at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(
        Stream serializationStream, HeaderHandler handler, Boolean fCheck,
        Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
  at System.Web.Util.AltSerialization.ReadValueFromStream(BinaryReader reader)
  at System.Web.SessionState.SessionStateItemCollection.ReadValueFromStreamWithAssert()
  at System.Web.SessionState.SessionStateItemCollection.DeserializeItem(
        String name, Boolean check)
  at System.Web.SessionState.SessionStateItemCollection.DeserializeAllItems()
  at System.Web.SessionState.SessionStateItemCollection.get_Keys()
  at System.Web.SessionState.HttpSessionStateContainer.get_Keys()
  at System.Web.SessionState.HttpSessionState.get_Keys()
  ...
</pre>
<p>where <tt class="docutils literal">AccountTagsMap</tt> looked like this:</p>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span style="color: #7D9029">[Serializable]</span>
<span style="color: #008000; font-weight: bold">public</span> <span style="color: #008000; font-weight: bold">class</span> <span style="color: #0000FF; font-weight: bold">AccountTagsMap</span> : Dictionary&lt;<span style="color: #B00040">string</span>, DateTime&gt;
{
    <span style="color: #408080; font-style: italic">// Various irrelevant methods, but no additional data</span>
    <span style="color: #408080; font-style: italic">// and no constructors</span>
}
</pre></div>
<p>It needed to be marked <tt class="docutils literal">[Serializable]</tt> as it was sometimes being stashed in
<tt class="docutils literal">HttpContext.Current.Session</tt>.</p>
<p>Understandably, I assumed from the error message that simply adding
a default constructor, <tt class="docutils literal">AccountTagsMap()</tt>, would fix the problem.</p>
<p>It took me some time, with the help of <a class="reference external" href="http://en.wikipedia.org/wiki/.NET_Reflector">Reflector</a>,
to realize that I also needed to add a <a class="reference external" href="http://msdn.microsoft.com/en-us/magazine/cc301767.aspx">serialization constructor</a>:</p>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span style="color: #7D9029">[Serializable]</span>
<span style="color: #008000; font-weight: bold">public</span> <span style="color: #008000; font-weight: bold">class</span> <span style="color: #0000FF; font-weight: bold">AccountTagsMap</span> : Dictionary&lt;<span style="color: #B00040">string</span>, DateTime&gt;
{
    <span style="color: #008000; font-weight: bold">public</span> <span style="color: #0000FF">AccountTagsMap</span>() : <span style="color: #008000; font-weight: bold">base</span>() { }
    <span style="color: #008000; font-weight: bold">protected</span> <span style="color: #0000FF">AccountTagsMap</span>(
        SerializationInfo info, StreamingContext context)
    : <span style="color: #008000; font-weight: bold">base</span>(info, context) {}
    <span style="color: #408080; font-style: italic">// ...</span>
}
</pre></div>
<p>This isn't a full implementation of serialization.
That's handled by the base class, <tt class="docutils literal">Dictionary&lt;K,V&gt;</tt>.
We wrongly assumed that inheriting from <tt class="docutils literal">Dictionary</tt> was all we needed to do.</p>
<p>Here's some test code that demonstrates the problem.
Comment out the serialization constructor to see the tests fail.</p>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span style="color: #008000; font-weight: bold">using</span> <span style="color: #0000FF; font-weight: bold">System</span>;
<span style="color: #008000; font-weight: bold">using</span> <span style="color: #0000FF; font-weight: bold">System.Collections.Generic</span>;
<span style="color: #008000; font-weight: bold">using</span> <span style="color: #0000FF; font-weight: bold">System.Reflection</span>;
<span style="color: #008000; font-weight: bold">using</span> <span style="color: #0000FF; font-weight: bold">System.Runtime.Serialization</span>;
<span style="color: #008000; font-weight: bold">using</span> <span style="color: #0000FF; font-weight: bold">System.Runtime.Serialization.Formatters.Binary</span>;
<span style="color: #008000; font-weight: bold">using</span> <span style="color: #0000FF; font-weight: bold">System.IO</span>;
<span style="color: #008000; font-weight: bold">using</span> <span style="color: #0000FF; font-weight: bold">NUnit.Framework</span>;

<span style="color: #008000; font-weight: bold">namespace</span> <span style="color: #0000FF; font-weight: bold">Cozi.Test</span>
{
<span style="color: #7D9029">    [TestFixture]</span>
    <span style="color: #008000; font-weight: bold">public</span> <span style="color: #008000; font-weight: bold">class</span> <span style="color: #0000FF; font-weight: bold">MapSerializationTests</span>
    {
<span style="color: #7D9029">        [Test]</span>
        <span style="color: #008000; font-weight: bold">public</span> <span style="color: #008000; font-weight: bold">void</span> <span style="color: #0000FF">SerializationCtor</span>()
        {
            Assert.IsNotNull(
                SerializationTester.GetConstructor(
                    <span style="color: #008000; font-weight: bold">typeof</span>(AccountTagsMap)));
        }

<span style="color: #7D9029">        [Test]</span>
        <span style="color: #008000; font-weight: bold">public</span> <span style="color: #008000; font-weight: bold">void</span> <span style="color: #0000FF">SerializationRoundtrip_AccountTagsMap</span>()
        {
            AccountTagsMap map1 = <span style="color: #008000; font-weight: bold">new</span> AccountTagsMap();
            map1[<span style="color: #BA2121">&quot;foo&quot;</span>] = DateTime.Now;
            map1[<span style="color: #BA2121">&quot;bar&quot;</span>] = DateTime.Now;
            AccountTagsMap map2 = SerializationTester.RoundTrip(map1);
            Assert.AreEqual(map1[<span style="color: #BA2121">&quot;foo&quot;</span>], map2[<span style="color: #BA2121">&quot;foo&quot;</span>]);
            Assert.AreEqual(map1[<span style="color: #BA2121">&quot;bar&quot;</span>], map2[<span style="color: #BA2121">&quot;bar&quot;</span>]);
        }
    }

    <span style="color: #008000; font-weight: bold">public</span> <span style="color: #008000; font-weight: bold">static</span> <span style="color: #008000; font-weight: bold">class</span> <span style="color: #0000FF; font-weight: bold">SerializationTester</span>
    {
    <span style="color: #408080; font-style: italic">// http://msdn.microsoft.com/en-us/library/ms182342(v=vs.80).aspx</span>
        <span style="color: #008000; font-weight: bold">public</span> <span style="color: #008000; font-weight: bold">static</span> T RoundTrip&lt;T&gt;(T <span style="color: #008000; font-weight: bold">value</span>)
        {
            <span style="color: #008000; font-weight: bold">if</span> (<span style="color: #008000; font-weight: bold">value</span> == <span style="color: #008000; font-weight: bold">null</span>)
                <span style="color: #008000; font-weight: bold">throw</span> <span style="color: #008000; font-weight: bold">new</span> <span style="color: #0000FF">ArgumentNullException</span>(<span style="color: #BA2121">&quot;value&quot;</span>);
            <span style="color: #008000; font-weight: bold">using</span> (MemoryStream stream = <span style="color: #008000; font-weight: bold">new</span> MemoryStream())
            {
                BinaryFormatter formatter = <span style="color: #008000; font-weight: bold">new</span> BinaryFormatter();
                formatter.Serialize(stream, <span style="color: #008000; font-weight: bold">value</span>);
                stream.Seek(<span style="color: #666666">0</span>, SeekOrigin.Begin);
                <span style="color: #008000; font-weight: bold">return</span> (T)formatter.Deserialize(stream);
            }
        }

        <span style="color: #408080; font-style: italic">// Reverse-engineered from System.Runtime.Serialization</span>
        <span style="color: #408080; font-style: italic">// .ObjectManager.GetConstructor with Reflector</span>
        <span style="color: #008000; font-weight: bold">public</span> <span style="color: #008000; font-weight: bold">static</span> ConstructorInfo <span style="color: #0000FF">GetConstructor</span>(Type t)
        {
            ConstructorInfo info = t.GetConstructor(
                BindingFlags.NonPublic | BindingFlags.Public
                    | BindingFlags.Instance,
                <span style="color: #008000; font-weight: bold">null</span>,
                CallingConventions.Any,
                <span style="color: #008000; font-weight: bold">new</span> Type[] { <span style="color: #008000; font-weight: bold">typeof</span>(SerializationInfo),
                             <span style="color: #008000; font-weight: bold">typeof</span>(StreamingContext) },
                <span style="color: #008000; font-weight: bold">null</span>) <span style="color: #008000; font-weight: bold">as</span> ConstructorInfo;
            <span style="color: #008000; font-weight: bold">if</span> (info == <span style="color: #008000; font-weight: bold">null</span>)
            {
                <span style="color: #008000; font-weight: bold">throw</span> <span style="color: #008000; font-weight: bold">new</span> <span style="color: #0000FF">SerializationException</span>(
                    String.Format(
                        <span style="color: #BA2121">&quot;Serialization_ConstructorNotFound: {0}&quot;</span>,
                        <span style="color: #008000; font-weight: bold">new</span> <span style="color: #B00040">object</span>[] { t.FullName }));
            }
            <span style="color: #008000; font-weight: bold">return</span> info;
        }
    }
}
</pre></div>
<p>I hope this saves you some time.</p>
</div>
</content>


    </entry>
    <entry>
        <title>Security 101 for Developers</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2011/04/security-101-for-developers.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2011/04/security-101-for-developers.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-6a00d8341ca8a653ef014e607ce57d970c</id>
        <published>2011-04-08T18:27:52-07:00</published>
        <updated>2011-04-08T18:29:22-07:00</updated>
        <summary>A few weeks ago, Andrew Abrahamowicz and I gave an introductory presentation on Secure Programming for Developers to the Cozi Dev Team. As it may be of interest to other developers, I uploaded it to SlideShare. Other Resources: 24 Deadly...</summary>
        <author>
            <name>George V. Reilly</name>
        </author>
        
        
<content type="xhtml" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
<div xmlns="http://www.w3.org/1999/xhtml"><p><a class="reference external image-reference" href="http://www.slideshare.net/george_v_reilly/security-101-7567002"><img alt="Why should we care about Security?" src="http://blogs.cozi.com/.a/6a00d8341ca8a653ef014e607cbcb7970c-800wi" /></a></p>
<p>A few weeks ago, Andrew Abrahamowicz and I gave an introductory presentation on Secure Programming for Developers to the Cozi Dev Team.</p>
<p>As it may be of interest to other developers, I uploaded it to <a class="reference external" href="http://www.slideshare.net/george_v_reilly/security-101-7567002">SlideShare</a>.</p>
<p>Other Resources:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.amazon.com/Deadly-Sins-Software-Security-Programming/dp/0071626751">24 Deadly Sins of Software Security: Programming Flaws and How to Fix Them</a></li>
<li><a class="reference external" href="http://www.microsoft.com/learning/en/us/book.aspx?ID=5957&amp;locale=en-us">Writing Secure Code, Second Edition</a></li>
</ul></div>
</content>


    </entry>
    <entry>
        <title>Generating UUIDs in JavaScript</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2010/04/generating-uuids-in-javascript.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2010/04/generating-uuids-in-javascript.html" thr:count="3" thr:updated="2010-09-15T20:07:26-07:00" />
        <id>tag:typepad.com,2003:post-6a00d8341ca8a653ef0133ed17ce09970b</id>
        <published>2010-04-30T18:38:57-07:00</published>
        <updated>2010-08-03T12:17:52-07:00</updated>
        <summary>A UUID is a universally unique 128-bit number, which can be generated without recourse to a central registry. UUIDs are used in many places, including database keys. Version 1 UUIDs were generated from a combination of a network MAC address...</summary>
        <author>
            <name>George V. Reilly</name>
        </author>
        <category scheme="http://www.sixapart.com/ns/types#category" term="Web Development" />
        
        
<content type="xhtml" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
<div xmlns="http://www.w3.org/1999/xhtml"><p><a class="reference external image-reference" href="http://en.wikipedia.org/wiki/Universally_Unique_Identifier"><img alt="UUID format" src="http://upload.wikimedia.org/wikipedia/commons/thumb/9/96/UUID_Format.svg/567px-UUID_Format.svg.png" /></a>
</p><p>A <a class="reference external" href="http://en.wikipedia.org/wiki/Universally_Unique_Identifier">UUID</a> is a universally unique 128-bit number,
which can be generated without recourse to a central registry.
UUIDs are used in many places, including database keys.</p>
<p>Version&#0160;1 UUIDs were generated from a combination of
a network MAC address and a timestamp.
Due to privacy concerns, most UUIDS are now <a class="reference external" href="http://www.faqs.org/rfcs/rfc4122.html">RFC 4122</a> Version&#0160;4 UUIDs,
generated from random data.</p>
<p>Most languages provide facilities to generate UUIDs,
often built on top of the operating system&#39;s entropy pool of random data.
JavaScript has a mediocre library and provides no way to generate UUIDs.</p>
<p>Robert Kieffer supplies one <a class="reference external" href="http://www.broofa.com/2008/09/javascript-uuid-function/">implementation at Broofa.com</a>.
It uses <a class="reference external" href="https://developer-stage.mozilla.org/en/Core_JavaScript_1.5_Reference/Global_Objects/Math/random">Math.random</a>, JavaScript&#39;s built-in pseudo-random number generator.</p>
<p>I looked around and found a much better <a class="reference external" href="http://davidbau.com/archives/2010/01/30/random_seeds_coded_hints_and_quintillions.html">seedable random number generator</a>
for JavaScript by David Bau.
The only downside is that if you don&#39;t specify an explicit random seed,
its default method of gathering entropy is a recursive traversal
of the <tt class="docutils literal">window</tt> object.
This traversal takes several hundred milliseconds on Firefox,
though it&#39;s much faster on other modern browsers.
Better, then, to supply seed data randomly generated on the server,
such as <tt class="docutils literal"><span class="pre">Math.seedrandom(&#39;&lt;%=</span> Guid.NewGuid <span class="pre">%&gt;&#39;)</span></tt>.</p>
<p>Here&#39;s another implementation of UUID generation.
If you&#39;re using Bau&#39;s seedrandom.js, it overrides <tt class="docutils literal">Math.random</tt>;
otherwise, you&#39;re using JavaScript&#39;s built-in RNG.</p>
<p class="highlight " style="background: #f8f8f8"><pre style="line-height: 125%"><span style="color: #008000; font-weight: bold">var</span> UUID <span style="color: #666666">=</span> {
 <span style="color: #408080; font-style: italic">// Return a randomly generated v4 UUID, per RFC 4122</span>
 uuid4<span style="color: #666666">:</span> <span style="color: #008000; font-weight: bold">function</span>()
 {
  <span style="color: #008000; font-weight: bold">return</span> <span style="color: #008000; font-weight: bold">this</span>._uuid(
    <span style="color: #008000; font-weight: bold">this</span>.randomInt(), <span style="color: #008000; font-weight: bold">this</span>.randomInt(),
    <span style="color: #008000; font-weight: bold">this</span>.randomInt(), <span style="color: #008000; font-weight: bold">this</span>.randomInt(), <span style="color: #666666">4</span>);
 },

 <span style="color: #408080; font-style: italic">// Create a versioned UUID from w1..w4, 32-bit non-negative ints</span>
 _uuid<span style="color: #666666">:</span> <span style="color: #008000; font-weight: bold">function</span>(w1, w2, w3, w4, version)
 {
  <span style="color: #008000; font-weight: bold">var</span> uuid <span style="color: #666666">=</span> <span style="color: #008000; font-weight: bold">new</span> <span style="color: #008000">Array</span>(<span style="color: #666666">36</span>);
  <span style="color: #008000; font-weight: bold">var</span> data <span style="color: #666666">=</span> [
   (w1 <span style="color: #666666">&amp;</span> <span style="color: #666666">0xFFFFFFFF</span>),
   (w2 <span style="color: #666666">&amp;</span> <span style="color: #666666">0xFFFF0FFF</span>) <span style="color: #666666">|</span> ((version <span style="color: #666666">||</span> <span style="color: #666666">4</span>) <span style="color: #666666">&lt;&lt;</span> <span style="color: #666666">12</span>), <span style="color: #408080; font-style: italic">// version (1-5)</span>
   (w3 <span style="color: #666666">&amp;</span> <span style="color: #666666">0x3FFFFFFF</span>) <span style="color: #666666">|</span> <span style="color: #666666">0x80000000</span>,    <span style="color: #408080; font-style: italic">// rfc 4122 variant</span>
   (w4 <span style="color: #666666">&amp;</span> <span style="color: #666666">0xFFFFFFFF</span>)
  ];
  <span style="color: #008000; font-weight: bold">for</span> (<span style="color: #008000; font-weight: bold">var</span> i <span style="color: #666666">=</span> <span style="color: #666666">0</span>, k <span style="color: #666666">=</span> <span style="color: #666666">0</span>; i <span style="color: #666666">&lt;</span> <span style="color: #666666">4</span>; i<span style="color: #666666">++</span>)
  {
   <span style="color: #008000; font-weight: bold">var</span> rnd <span style="color: #666666">=</span> data[i];
   <span style="color: #008000; font-weight: bold">for</span> (<span style="color: #008000; font-weight: bold">var</span> j <span style="color: #666666">=</span> <span style="color: #666666">0</span>; j <span style="color: #666666">&lt;</span> <span style="color: #666666">8</span>; j<span style="color: #666666">++</span>)
   {
    <span style="color: #008000; font-weight: bold">if</span> (k <span style="color: #666666">==</span> <span style="color: #666666">8</span> <span style="color: #666666">||</span> k <span style="color: #666666">==</span> <span style="color: #666666">13</span> <span style="color: #666666">||</span> k <span style="color: #666666">==</span> <span style="color: #666666">18</span> <span style="color: #666666">||</span> k <span style="color: #666666">==</span> <span style="color: #666666">23</span>) {
     uuid[k<span style="color: #666666">++</span>] <span style="color: #666666">=</span> <span style="color: #BA2121">&#39;-&#39;</span>;
    }
    <span style="color: #008000; font-weight: bold">var</span> r <span style="color: #666666">=</span> (rnd <span style="color: #666666">&gt;&gt;&gt;</span> <span style="color: #666666">28</span>) <span style="color: #666666">&amp;</span> <span style="color: #666666">0xf</span>; <span style="color: #408080; font-style: italic">// Take the high-order nybble</span>
    rnd <span style="color: #666666">=</span> (rnd <span style="color: #666666">&amp;</span> <span style="color: #666666">0x0FFFFFFF</span>) <span style="color: #666666">&lt;&lt;</span> <span style="color: #666666">4</span>;
    uuid[k<span style="color: #666666">++</span>] <span style="color: #666666">=</span> <span style="color: #008000; font-weight: bold">this</span>.hex.charAt(r);
   }
  }
  <span style="color: #008000; font-weight: bold">return</span> uuid.join(<span style="color: #BA2121">&#39;&#39;</span>);
 },

 hex<span style="color: #666666">:</span> <span style="color: #BA2121">&#39;0123456789abcdef&#39;</span>,

 <span style="color: #408080; font-style: italic">// Return a random integer in [0, 2^32).</span>
 randomInt<span style="color: #666666">:</span> <span style="color: #008000; font-weight: bold">function</span>()
 {
  <span style="color: #008000; font-weight: bold">return</span> <span style="color: #008000">Math</span>.floor(<span style="color: #666666">0x100000000</span> <span style="color: #666666">*</span> <span style="color: #008000">Math</span>.random());
 }
};
</pre></p>
<p>This should do a <a class="reference external" href="http://en.wikipedia.org/wiki/Universally_Unique_Identifier#Random_UUID_probability_of_duplicates">good-enough</a> job of generating UUIDs.
If these UUIDs are being sent to your servers,
remember that you must never trust data from a client.
If you&#39;re expecting unique UUIDs,
an attacker might get past your defenses by repeatedly sending you
an already used UUID.</p></div>
</content>


    </entry>
    <entry>
        <title>SQLAlchemy Sharding</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2010/04/sqlalchemy-sharding.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2010/04/sqlalchemy-sharding.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-6a00d8341ca8a653ef0133ec6ad511970b</id>
        <published>2010-04-02T17:45:18-07:00</published>
        <updated>2010-04-05T12:58:55-07:00</updated>
        <summary>Some time ago, we sharded our production database, splitting accounts onto a couple of partitions. We&#39;re using Consistent Hashing on account_ids to determine in which partition a particular account is located. Our older services are written in C#; our newer,...</summary>
        <author>
            <name>George V. Reilly</name>
        </author>
        
        
<content type="html" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
&lt;div xmlns=&quot;http://www.w3.org/1999/xhtml&quot;&gt;&lt;img alt=&quot;SQLAlchemy&quot; class=&quot;right-float&quot; src=&quot;http://www.sqlalchemy.org/_img/sqla-logo6.gif&quot; /&gt;
&lt;p&gt;Some time ago, we &lt;a class=&quot;reference external&quot; href=&quot;http://en.wikipedia.org/wiki/Shard_(database_architecture)&quot;&gt;sharded&lt;/a&gt; our production database,
splitting accounts onto a couple of partitions.
We&#39;re using &lt;a class=&quot;reference external&quot; href=&quot;http://en.wikipedia.org/wiki/Consistent_hashing&quot;&gt;Consistent Hashing&lt;/a&gt; on &lt;tt class=&quot;docutils literal&quot;&gt;account_id&lt;/tt&gt;s to determine
in which partition a particular account is located.&lt;/p&gt;
&lt;p&gt;Our older services are written in C#; our newer, in Python.
We use the &lt;a class=&quot;reference external&quot; href=&quot;http://www.sqlalchemy.org/&quot;&gt;SQLAlchemy&lt;/a&gt; object-relational mapper in the Python codebase,
and we based our sharding code on the supplied &lt;a class=&quot;reference external&quot; href=&quot;http://www.sqlalchemy.org/docs/examples.html#examples-sharding&quot;&gt;example&lt;/a&gt; for SQLAlchemy&amp;nbsp;0.5.&lt;/p&gt;
&lt;p&gt;Recently, it became apparent that some database queries that should have been sharded, weren&#39;t.
These queries were querying &lt;em&gt;all&lt;/em&gt; partitions.
That&#39;s not too costly when there are only two partitions, but it will not scale.&lt;/p&gt;
&lt;p&gt;SQLAlchemy&#39;s &lt;tt class=&quot;docutils literal&quot;&gt;ShardedQuery&lt;/tt&gt; calls our &lt;tt class=&quot;docutils literal&quot;&gt;query_chooser&lt;/tt&gt; for each query,
to compute a set of shard ids on which to execute the query.
To restrict a query to just one of our shards,
the query must include an explicit &lt;tt class=&quot;docutils literal&quot;&gt;account_id&lt;/tt&gt;,
so that we can compute the shard id.&lt;/p&gt;
&lt;p&gt;There were several reasons why these queries didn&#39;t shard properly.&lt;/p&gt;
&lt;ul class=&quot;simple&quot;&gt;
&lt;li&gt;&lt;tt class=&quot;docutils literal&quot;&gt;ForeignKey&lt;/tt&gt; relationships were not fully declared in some tables.
SQLAlchemy was smart enough to create queries that worked,
but did not have enough information to infer that the &lt;tt class=&quot;docutils literal&quot;&gt;account_id&lt;/tt&gt;
should be included in the query.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&quot;highlight&quot; style=&quot;background: #f8f8f8&quot;&gt;&lt;pre style=&quot;line-height: 125%&quot;&gt;&lt;span style=&quot;color: #008000; font-weight: bold&quot;&gt;class&lt;/span&gt; &lt;span style=&quot;color: #0000FF; font-weight: bold&quot;&gt;CalendarItem&lt;/span&gt;(Base):
  __tablename__ &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; &lt;span style=&quot;color: #BA2121&quot;&gt;&amp;#39;calendar_items&amp;#39;&lt;/span&gt;
  calendar_item_id &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; Column(UUID, nullable&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;False&lt;/span&gt;, primary_key&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;True&lt;/span&gt;)
  calendar_id &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; Column(UUID, ForeignKey(Calendar&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;calendar_id),
                       nullable&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;False&lt;/span&gt;, index&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;True&lt;/span&gt;)
  account_id  &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; Column(UUID, ForeignKey(Account&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;account_id),
                       nullable&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;False&lt;/span&gt;)
  &lt;span style=&quot;color: #408080; font-style: italic&quot;&gt;# ...&lt;/span&gt;

&lt;span style=&quot;color: #008000; font-weight: bold&quot;&gt;class&lt;/span&gt; &lt;span style=&quot;color: #0000FF; font-weight: bold&quot;&gt;CalendarItemRecurrence&lt;/span&gt;(Base):
  __tablename__ &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; &lt;span style=&quot;color: #BA2121&quot;&gt;&amp;#39;calendar_item_recurrences&amp;#39;&lt;/span&gt;
  calendar_item_recurrence_id &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; Column(UUID, nullable&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;False&lt;/span&gt;,
                                       primary_key&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;True&lt;/span&gt;)
  account_id &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; Column(UUID, ForeignKey(CalendarItem&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;account_id),
                      nullable&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;False&lt;/span&gt;, index&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;True&lt;/span&gt;)
  calendar_item_id &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; Column(UUID,
                       ForeignKey(CalendarItem&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;calendar_item_id),
                       nullable&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;False&lt;/span&gt;, index&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;True&lt;/span&gt;)
  &lt;span style=&quot;color: #408080; font-style: italic&quot;&gt;# ...&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;
&lt;ul class=&quot;simple&quot;&gt;
&lt;li&gt;&lt;tt class=&quot;docutils literal&quot;&gt;ForeignKey&lt;/tt&gt;s need to be directly to the foreign table,
not to the transitive closure.
In this case, &lt;tt class=&quot;docutils literal&quot;&gt;CalendarItemRecurrence.account_id&lt;/tt&gt; is a FK to
&lt;tt class=&quot;docutils literal&quot;&gt;CalendarItem.account_id&lt;/tt&gt;, not to &lt;tt class=&quot;docutils literal&quot;&gt;Account.account_id&lt;/tt&gt;.&lt;/li&gt;
&lt;li&gt;Some inter-table &lt;a class=&quot;reference external&quot; href=&quot;http://www.sqlalchemy.org/docs/05/ormtutorial.html#building-a-relation&quot;&gt;relations&lt;/a&gt; needed explicit &lt;tt class=&quot;docutils literal&quot;&gt;primaryjoin&lt;/tt&gt;s&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&quot;highlight&quot; style=&quot;background: #f8f8f8&quot;&gt;&lt;pre style=&quot;line-height: 125%&quot;&gt;CalendarItem&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;recurrence_dates &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; relation(
  CalendarItemRecurrenceDate,
  lazy&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;True&lt;/span&gt;,
  primaryjoin&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;and_(
      CalendarItemRecurrenceDate&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;account_id
            &lt;span style=&quot;color: #666666&quot;&gt;==&lt;/span&gt; CalendarItem&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;account_id,
      CalendarItemRecurrenceDate&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;calendar_item_id
            &lt;span style=&quot;color: #666666&quot;&gt;==&lt;/span&gt; CalendarItem&lt;span style=&quot;color: #666666&quot;&gt;.&lt;/span&gt;calendar_item_id
      ),
  uselist&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;True&lt;/span&gt;,
  cascade&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #BA2121&quot;&gt;&amp;#39;all, delete-orphan&amp;#39;&lt;/span&gt;,
  passive_updates&lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #008000&quot;&gt;False&lt;/span&gt;,
  )
&lt;/pre&gt;&lt;/div&gt;
&lt;ul class=&quot;simple&quot;&gt;
&lt;li&gt;Some explicit queries in our code should have filtered by &lt;tt class=&quot;docutils literal&quot;&gt;account_id&lt;/tt&gt;, but didn&#39;t.&lt;/li&gt;
&lt;li&gt;The provided &lt;a class=&quot;reference external&quot; href=&quot;http://www.sqlalchemy.org/docs/examples.html#examples-sharding&quot;&gt;example&lt;/a&gt; assumes that constants occur only on the right-hand side
of binary expressions. We found that in lazy-load relations,
constants could occur on the left-hand side.&lt;/li&gt;
&lt;li&gt;Some SQLAlchemy-generated queries had &lt;tt class=&quot;docutils literal&quot;&gt;_BindParamClause&lt;/tt&gt;s instead of constants&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&quot;highlight&quot; style=&quot;background: #f8f8f8&quot;&gt;&lt;pre style=&quot;line-height: 125%&quot;&gt;&lt;span style=&quot;color: #008000; font-weight: bold&quot;&gt;SELECT&lt;/span&gt; shopping_list_items.&lt;span style=&quot;color: #008000; font-weight: bold&quot;&gt;version&lt;/span&gt; &lt;span style=&quot;color: #008000; font-weight: bold&quot;&gt;AS&lt;/span&gt; shopping_list_items_version
&lt;span style=&quot;color: #008000; font-weight: bold&quot;&gt;FROM&lt;/span&gt; shopping_list_items
&lt;span style=&quot;color: #008000; font-weight: bold&quot;&gt;WHERE&lt;/span&gt; shopping_list_items.account_id &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; :param_1
  &lt;span style=&quot;color: #008000; font-weight: bold&quot;&gt;AND&lt;/span&gt; shopping_list_items.shopping_list_item_id &lt;span style=&quot;color: #666666&quot;&gt;=&lt;/span&gt; :param_2
&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;I posted to the &lt;a class=&quot;reference external&quot; href=&quot;http://groups.google.com/group/sqlalchemy/browse_thread/thread/99be523e5146dde0&quot;&gt;SQLAlchemy discussion list&lt;/a&gt; about the last two issues.
After a couple of iterations,
&lt;a class=&quot;reference external&quot; href=&quot;http://techspot.zzzeek.org/&quot;&gt;Michael Bayer&lt;/a&gt; came up with a much &lt;a class=&quot;reference external&quot; href=&quot;http://hg.sqlalchemy.org/sqlalchemy/rev/0ddd638f1d90&quot;&gt;cleaner solution&lt;/a&gt; for traversing a query
to choose ids, which has been checked into the 0.6&amp;nbsp;trunk.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Update: 2010/4/5&lt;/strong&gt;: We discovered our non-sharded queries
by adding some logging to the end of &lt;tt class=&quot;docutils literal&quot;&gt;query_chooser&lt;/tt&gt;,
in the case where &lt;tt class=&quot;docutils literal&quot;&gt;len(ids) == 0&lt;/tt&gt;.&lt;/p&gt;
&lt;/div&gt;
</content>


    </entry>
    <entry>
        <title>Engineering Jobs at Cozi</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2010/03/e.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2010/03/e.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-6a00d8341ca8a653ef01310fe6b288970c</id>
        <published>2010-03-26T22:55:42-07:00</published>
        <updated>2010-03-26T22:55:42-07:00</updated>
        <summary>Cozi is hiring. We have positions in Web Development, Software Engineering, and System Engineering at our headquarters in Seattle. Full details at the Careers Page.</summary>
        <author>
            <name>George V. Reilly</name>
        </author>
        
        
<content type="xhtml" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
<div xmlns="http://www.w3.org/1999/xhtml"><p>Cozi is hiring. We have positions in Web Development, Software Engineering, and System Engineering at our headquarters in Seattle.</p><br /><div>Full details at the <a href="http://www.cozi.com/Careers-Jobs-Engineering.htm">Careers Page</a>.</div></div>
</content>


    </entry>
    <entry>
        <title>Iframes: thinking outside the box</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2009/04/iframe.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2009/04/iframe.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-65256921</id>
        <published>2009-04-09T20:01:44-07:00</published>
        <updated>2009-04-09T20:01:44-07:00</updated>
        <summary>Iframes have their uses, but they are not easy to deal with. I added some text advertisements to our product this week. The standard technique for including advertising is to use an iframe. This works well for banner ads which...</summary>
        <author>
            <name>George V. Reilly</name>
        </author>
        <category scheme="http://www.sixapart.com/ns/types#category" term="Web Development" />
        
        
<content type="xhtml" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
<div xmlns="http://www.w3.org/1999/xhtml">&#0160;<p><a href="http://blogs.cozi.com/.a/6a00d8341ca8a653ef01156f18dd71970c-pi" style="display: block;"><img alt="Iframe-firebug-rag-ps" border="0" class="at-xid-6a00d8341ca8a653ef01156f18dd71970c " src="http://blogs.cozi.com/.a/6a00d8341ca8a653ef01156f18dd71970c-800wi" style="margin: 0px 0px 5px 5px;" title="Iframe-firebug-rag-ps" /></a> 
</p><p>Iframes have their uses, but they are not easy to deal with.</p>
<p>I added some text advertisements to our product this week.
The standard technique for including advertising is to use an <em>iframe</em>.
This works well for banner ads which come in <a class="reference external" href="http://iab.com/pages.php?page=6">well-known sizes</a>.</p>
<p>I immediately ran into a problem with text ads in an iframe:
there&#39;s no easy way to apply CSS to the contents of the iframe.
Styles do not cascade through the iframe barrier.
Normally, this is what you want,
a self-contained unit on the page.
It&#39;s fine for a banner ad, which requires no styling,
but Times Roman text is jarring in a page of Arial.</p>
<p>It&#39;s difficult, perhaps outright impossible,
to inject styles into an iframe coming from another domain.</p>
<p>Another problem is knowing how big to make the iframe.
They don&#39;t autosize and the ad text could be one or more lines long.</p>
<p>I needed another way.</p>
<p>Making an Ajax call to fetch just the raw data that I cared about
(title, body copy, link) was the obvious answer.
A little wad of JSON would be much easier to deal with than
trying to style an iframe.
Unfortunately, the XMLHTTPRequest object cannot make cross-domain calls.
But I read up on <a class="reference external" href="http://www.west-wind.com/Weblog/posts/107136.aspx">JSONP</a> last week,
so I knew that I could inject a <tt class="docutils literal"><span class="pre">script</span></tt> tag into my HTML DOM
and set the <tt class="docutils literal"><span class="pre">src</span></tt> attribute to the adserver.</p>
<p>jQuery makes this easy:
<a class="reference external" href="http://docs.jquery.com/Ajax/jQuery.getScript">jQuery.getScript</a> injects the script tag
and removes it after the script has loaded.</p>
<p>We uploaded a custom template to the adserver:</p>
<div class="highlight"><pre>setAdText({<br />  <span style="color: #ba2121;">&quot;adTitle&quot;</span><span style="color: #666666;">:</span> <span style="color: #ba2121;">&quot;%%TITLE%%&quot;</span><span style="color: #666666;">,</span>
  <span style="color: #ba2121;">&quot;bodyCopy&quot;</span><span style="color: #666666;">:</span> <span style="color: #ba2121;">&quot;%%BODYCOPY%%&quot;</span><span style="color: #666666;">,</span>
  <span style="color: #ba2121;">&quot;clickUrl&quot;</span><span style="color: #666666;">:</span> <span style="color: #ba2121;">&quot;%%CLICKURL%%&quot;</span>
});
</pre></div>
<p>I put this call to invoke the adserver&#39;s template in my page:</p>
<div class="highlight"><pre>$.getScript(adServerUrl <span style="color: #666666;">+</span> cache_busting_random_token());<br /></pre></div>
<p>And the <tt class="docutils literal"><span class="pre">setAdText</span></tt> handler in my HTML page looks like this:</p>
<div class="highlight"><pre><span style="color: #008000; font-weight: bold;">function</span> setAdText(data)<br />{<br />  console.log(<span style="color: #ba2121;">&quot;setAdText: adTitle=[%s], bodyCopy=[%s], clickUrl=%s.&quot;</span>
    data.adTitle<span style="color: #666666;">,</span> data.bodyCopy<span style="color: #666666;">,</span> data.clickUrl);<br />  <span style="color: #408080; font-style: italic;">// add the ad to the DOM</span>
}
</pre></div>
<p>Problem solved.</p></div>
</content>


    </entry>
    <entry>
        <title>Augmenting Python&#39;s strftime</title>
        <link rel="alternate" type="text/html" href="http://blogs.cozi.com/tech/2009/03/augmenting-pythons-strftime.html" />
        <link rel="replies" type="text/html" href="http://blogs.cozi.com/tech/2009/03/augmenting-pythons-strftime.html" thr:count="0" />
        <id>tag:typepad.com,2003:post-64872065</id>
        <published>2009-03-30T23:07:59-07:00</published>
        <updated>2009-03-31T15:18:51-07:00</updated>
        <summary>The strftime function is the prescribed way to format dates and times in Python (and other languages). It has limitations, such as forcing a leading zero on days of the month, 01-31, and on 12-hour clock hours, 01-12. Edit I...</summary>
        <author>
            <name>George V. Reilly</name>
        </author>
        <category scheme="http://www.sixapart.com/ns/types#category" term="Python" />
        
        
<content type="xhtml" xml:lang="en-US" xml:base="http://blogs.cozi.com/tech/">
<div xmlns="http://www.w3.org/1999/xhtml"><img alt="Big Ben" class="right-float" src="http://www.nullislove.com/wp-content/uploads/2007/05/big_ben.jpg" />
<p>The <a class="reference external" href="http://docs.python.org/library/time.html">strftime</a> function is the prescribed way to format dates and times
in Python (and other languages).
It has limitations, such as forcing a leading zero on days of the month,
<tt class="docutils literal"><span class="pre">01-31</span></tt>, and on 12-hour clock hours, <tt class="docutils literal"><span class="pre">01-12</span></tt>.</p>
<p><strong>Edit</strong> I noticed that we were repeatedly writing expressions like these</p>
<div class="highlight"><pre>d<span style="color: #666666">.</span>strftime(<span style="color: #BA2121">&#39;%A, %B &#39;</span>) <span style="color: #666666">+</span> <span style="color: #008000">str</span>(d<span style="color: #666666">.</span>day)
t<span style="color: #666666">.</span>strftime(<span style="color: #BA2121">&quot;%I:%M&quot;</span>)<span style="color: #666666">.</span>lstrip(<span style="color: #BA2121">&#39;0&#39;</span>) <span style="color: #666666">+</span> (<span style="color: #BA2121">&#39;a&#39;</span> <span style="color: #008000; font-weight: bold">if</span> t<span style="color: #666666">.</span>hour <span style="color: #666666">&lt;</span> <span style="color: #666666">12</span> <span style="color: #008000; font-weight: bold">else</span> <span style="color: #BA2121">&#39;p&#39;</span>)
</pre></div>
<p>and realized that there had to be a better way.</p>
<p>Here's a straightforward way to augment the directives:
preprocess the format string, replacing new directives with their values,
then let the underlying <tt class="docutils literal"><span class="pre">strftime</span></tt> implementation take care of the rest.</p>
<div class="highlight"><pre><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">re</span>

_re_aux_format <span style="color: #666666">=</span> re<span style="color: #666666">.</span>compile(<span style="color: #BA2121">&quot;%([DiP])&quot;</span>)

<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">strftime_aux</span>(d, format):
    <span style="color: #BA2121; font-style: italic">&quot;&quot;&quot;</span>
<span style="color: #BA2121; font-style: italic">    Augmented strftime that handles additional directives.</span>

<span style="color: #BA2121; font-style: italic">    %D  Day of the month as a decimal number [1,31] (no leading zero)</span>
<span style="color: #BA2121; font-style: italic">    %i  Hour (12-hour clock) as a decimal number [1,12] (no leading zero)</span>
<span style="color: #BA2121; font-style: italic">    %P  &#39;a&#39; for AM, &#39;p&#39; for PM</span>

<span style="color: #BA2121; font-style: italic">    &gt;&gt;&gt; import datetime</span>
<span style="color: #BA2121; font-style: italic">    &gt;&gt;&gt; d = datetime.datetime(2009, 4, 1, 9+12, 37)</span>
<span style="color: #BA2121; font-style: italic">    &gt;&gt;&gt; strftime_aux(d, &#39;%A, %B %d, %I:%M %p&#39;)</span>
<span style="color: #BA2121; font-style: italic">    &#39;Wednesday, April 01, 09:37 PM&#39;</span>
<span style="color: #BA2121; font-style: italic">    &gt;&gt;&gt; strftime_aux(d, &#39;%A, %B %D, %i:%M%P&#39;)</span>
<span style="color: #BA2121; font-style: italic">    &#39;Wednesday, April 1, 9:37p&#39;</span>
<span style="color: #BA2121; font-style: italic">    &quot;&quot;&quot;</span>

    <span style="color: #408080; font-style: italic"># Precompute the values of the augmented directives</span>
    directive_map <span style="color: #666666">=</span> {
        <span style="color: #BA2121">&#39;D&#39;</span>: <span style="color: #008000">str</span>(d<span style="color: #666666">.</span>day),
        <span style="color: #BA2121">&#39;i&#39;</span>: <span style="color: #BA2121">&#39;12&#39;</span> <span style="color: #008000; font-weight: bold">if</span> d<span style="color: #666666">.</span>hour <span style="color: #AA22FF; font-weight: bold">in</span> (<span style="color: #666666">0</span>, <span style="color: #666666">12</span>) <span style="color: #008000; font-weight: bold">else</span> <span style="color: #008000">str</span>(d<span style="color: #666666">.</span>hour <span style="color: #666666">%</span> <span style="color: #666666">12</span>),
        <span style="color: #BA2121">&#39;P&#39;</span>: <span style="color: #BA2121">&#39;a&#39;</span> <span style="color: #008000; font-weight: bold">if</span> d<span style="color: #666666">.</span>hour <span style="color: #666666">&lt;</span> <span style="color: #666666">12</span> <span style="color: #008000; font-weight: bold">else</span> <span style="color: #BA2121">&#39;p&#39;</span>,
    }
    <span style="color: #408080; font-style: italic"># Substitute those values into the format string</span>
    new_format <span style="color: #666666">=</span> _re_aux_format<span style="color: #666666">.</span>sub(
        <span style="color: #008000; font-weight: bold">lambda</span> match: directive_map<span style="color: #666666">.</span>get( match<span style="color: #666666">.</span>group(<span style="color: #666666">1</span>), <span style="color: #BA2121">&#39;&#39;</span>),
        format)
    <span style="color: #408080; font-style: italic"># Let the stock implementation of strftime handle everything else</span>
    <span style="color: #008000; font-weight: bold">return</span> d<span style="color: #666666">.</span>strftime(new_format)

<span style="color: #008000; font-weight: bold">if</span> __name__ <span style="color: #666666">==</span> <span style="color: #BA2121">&quot;__main__&quot;</span>:
    <span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">doctest</span>
    doctest<span style="color: #666666">.</span>testmod()
</pre></div>
</div>
</content>


    </entry>
 
</feed>

<!-- ph=1 -->
