<?xml version="1.0" encoding="UTF-8"?>
<!--Generated by Squarespace V5 Site Server v5.13.594-SNAPSHOT-1 (http://www.squarespace.com) on Sat, 18 Apr 2026 15:23:16 GMT--><rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0"><channel><title>The Experiment-Driven Life Blog</title><link>http://www.matthewcornell.org/blog/</link><description></description><lastBuildDate>Thu, 01 Oct 2015 00:57:48 +0000</lastBuildDate><copyright></copyright><language>en-US</language><generator>Squarespace V5 Site Server v5.13.594-SNAPSHOT-1 (http://www.squarespace.com)</generator><item><title>Notes From a Conversation With Political Activist Josh Silver (Represent.us)</title><dc:creator>Matthew Cornell</dc:creator><pubDate>Wed, 30 Sep 2015 13:44:24 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/9/30/notes-from-a-conversation-with-political-activist-josh-silve.html</link><guid isPermaLink="false">434480:8122567:35490556</guid><description><![CDATA[<p><span class="full-image-float-right ssNonEditable"><span><img style="width: 250px;" src="http://matthewcornell.squarespace.com/storage/post-images/represent-us.png?__SQUARESPACE_CACHEVERSION=1443621188688" alt="" /></span></span>Since   leaving my research programmer position at UMass (funding was drying   up), I've been reaching out to leaders to explore possible contributions   I could make in the <a href="https://en.wikipedia.org/wiki/Progressivism">progressive</a> or educational spaces <a href="http://markable.in/editor/#areas">1</a>.   Here I'd like to briefly share a few notes from my conversation with   one of them, Josh Silver. NB: Any errors I make are my own - please send   corrections!)</p>
<p>And a quick pitch: If you know someone who's hiring in those areas then please <a href="http://www.matthewcornell.org/contact">give me a ring</a>. (I'm looking for the <a href="https://en.wikipedia.org/wiki/Venn_diagram">intersection</a> of progressive work, software tool development, and solid funding.)</p>
<h1 id="josh-silver-representus">Josh Silver (Represent.us)</h1>
<p>I had the good fortune to meet <a href="https://en.wikipedia.org/wiki/Josh_Silver_%28nonprofit_director%29">Josh  Silver</a>,  who has made contributions in media reform and campaign finance reform  (think <a href="https://en.wikipedia.org/wiki/Citizens_United_v._FEC">Citizens United</a>). He currently runs <a href="https://represent.us/">Represent.Us</a>, and, before that, <a href="http://www.freepress.net/">Free Press</a>.   Josh summarized his perspective on what he sees as the two main   challenges to policy change (an informed public, and government   representatives who actually represent people, not big money or   corporations) and gave me a precis on how money is injected into   politics (direct to campaigns and <a href="http://www.opensecrets.org/pacs/superpacs.php">super PACs</a>, which can be tracked, and indirect via <a href="https://en.wikipedia.org/wiki/501%28c%29_organization">501c4 and 501c6 organizations</a>, which is not directly tracked - more <a href="http://www.dailykos.com/story/2013/05/16/1208999/-501-c-3-s-501-c-4-s-and-the-rest-A-primer">here</a>). Josh said there's a lot of activity in data and politics, for example:</p>
<ul>
<li><a href="http://sunlightfoundation.com/">Sunlight Foundation</a></li>
<li><a href="http://www.opensecrets.org/">OpenSecrets.org: Money in Politics -- See Who's Giving &amp; Who's Getting</a></li>
<li><a href="http://maplight.org/">U.S. Congress Campaign Contributions and Voting Database | MapLight - Money and Politics</a></li>
<li><a href="http://followthemoney.org/">Follow The Money</a></li>
<li>And of course the awesome <a href="https://twitter.com/nytgraphics">@nytgraphics</a> work.</li>
</ul>
<p>He also showed me the <a href="http://allaregreen.us/">Greenhouse browser extension</a>, which rewrites web pages via some information extraction to pop up campaign finance information for people on the page.</p>
<p>We talked a little about the fallibility of human nature and how <a href="http://www.motherjones.com/politics/2011/03/denial-science-chris-mooney">emotion and stories often trump facts</a> (<em>It's   a "basic human survival skill," explains political scientist Arthur   Lupia of the University of Michigan. We push threatening information   away; we pull friendly information close. We apply fight-or-flight   reflexes not only to predators, but to data itself.</em>). (I tend to arrogantly think I'm immune, but I'm not - see <a href="http://www.amazon.com/gp/product/0374533555/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=0374533555&amp;linkCode=as2&amp;tag=masidbl-20&amp;linkId=FFC4B5LYYPCQUMW3">Thinking, Fast and Slow</a> and <a href="http://www.amazon.com/gp/product/1592407366/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=1592407366&amp;linkCode=as2&amp;tag=masidbl-20&amp;linkId=HZMVNGSUBTRMEKXR">You  Are Not So Smart</a> - Amazon links)</p>
<p>Regarding funding for software development, Josh suggested either   contacting one of the above organizations, or propose funding for a   grant from someone like the <a href="http://www.knightfoundation.org/">Knight Foundation</a>.</p>
<p>I finished up by talking a bit with one of his research assistants   about possible tools that could help folks like him and reporters   collect campaign contribution information more easily. My takeaway was   that there's no good, central aggregator; information is disparate and   spans multiple sites.</p>
<p>Great stuff! Stay tuned...</p>
<h1 id="wzxhzdk0wzxhzdk1-areas-of-interest"><a name="areas"></a> Areas of Interest</h1>
<ul>
<li>Skepticism, critical thinking</li>
<li>Journalism, media reform</li>
<li>Education and science</li>
<li>Fact checking</li>
<li>Environment, climate change</li>
<li><a href="https://en.wikipedia.org/wiki/Freethought">Freethought</a></li>
<li>Politics, digital government, campaign finance reform</li>
<li>Citizenship, participation, digital government</li>
</ul>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35490556.xml</wfw:commentRss></item><item><title>Brief report: Boston Python User Group: Favorite Libraries Meetup</title><dc:creator>Matthew Cornell</dc:creator><pubDate>Fri, 25 Sep 2015 20:14:39 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/9/25/brief-report-boston-python-user-group-favorite-libraries-mee.html</link><guid isPermaLink="false">434480:8122567:35484584</guid><description><![CDATA[<div id="markdown-preview">
<p><span class="full-image-float-left ssNonEditable"><span><img title="Python logo" src="https://pypi.python.org/static/images/python-logo.png" alt="python logo" /></span></span></p>
<p>Last night I attended a <a href="http://www.meetup.com/bostonpython/events/223626917/">favorite          libraries</a> Python Meetup organized by <a href="http://www.meetup.com/bostonpython/">The          Boston Python User Group</a> (<a href="https://twitter.com/bostonpython">@bostonpython</a>)          and hosted by <a href="https://www.akamai.com/">Akamai</a>. Here I'll          give a brief summary of the libraries covered, and a newcomer's          perspective on the event. The presentations overall were great -          focused, not too long, with code examples and demos. And of course the          theme was a winner - who doesn't love learning about a tasty library          that may come in handy? Fun demos make it even better. I'm looking          forward to attending more of them.</p>
<h2>Neil Tenenholtz: <a href="https://pythonhosted.org/mrjob/">mrjob</a></h2>
<p><a href="https://pythonhosted.org/mrjob/">mrjob</a> is a library for          writing <a href="https://en.wikipedia.org/wiki/MapReduce">MapReduce</a> jobs (Hadoop tutorial          [here](https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.htm          l), original Google paper <a href="http://research.google.com/archive/mapreduce.html">here</a>).          Neil did a live demo showing how little code is needed to express          paralellizable tasks, at the simplest, just a <code>mapper()</code> and <code>reducer()</code> function. He demo'd wordcounta and showed          how to run it on different clusters, including simulated, local, and          Hadoop or <a href="http://aws.amazon.com/elasticmapreduce/">Amazon          Elastic MapReduce</a> installations. At one of the UMass labs I worked          in, we used Hadoop in a number of ways, including batch data          processing and cleanup (i.e., <a href="https://en.wikipedia.org/wiki/Extract,_transform,_load">ETL</a>)          for the <a href="https://en.wikipedia.org/wiki/Machine_learning">machine          learning</a> algorithms, and parallelized SQL via <a href="http://www.cloudera.com/content/cloudera/en/products-and-services/cdh/impala.html">Cloudera          Impala</a>. Yummy stuff!</p>
<h2>Lindsay Raymond: <a href="https://github.com/Suor/funcy">Funcy</a></h2>
<p><a href="https://github.com/Suor/funcy">Funcy</a> adds to Python some          functional tools "inspired by <a href="http://clojure.org/">clojure</a>,          <a href="http://underscorejs.org/">Underscore.js</a>, and [his] own          abstractions." (They reminded me a bit of the <a href="http://www.scala-lang.org/">Scala</a> idioms I saw in my brief exposure to it for a <a href="https://spark.apache.org/graphx/">Spark          GraphX</a> project I did to implement fast graph path searches. Ask me          sometime about my take on Scala as compared to Python ;-) Lindsay gave          some examples, and it was clear that there's a lot more to the library          than can be shown during a short demo, but she touched on concepts          like how readable elegantly composed functions are, and how testing is          eased by breaking computation down into stand-alone pieces. (As an <a href="https://en.wikipedia.org/wiki/Extreme_programming">XP</a> fan, I really appreciated this point.) While I've done less functional          programming than perhaps I'd like, funcy got me interested in learning          more.</p>
<h2>Scott Sanderson: <a href="http://click.pocoo.org/5/">Click</a></h2>
<p><span class="full-image-float-right ssNonEditable"><span><img title="click logo" width="50%" height="50%" src="http://click.pocoo.org/5/_static/click.png" alt="click logo" /></span></span></p>
<p><a href="http://click.pocoo.org/5/">Click</a> is a package that          simplifies writing command-line interfaces. Scott demonstrated (with          humor :-) the main features, showing the <a href="https://wiki.python.org/moin/PythonDecorators">decorator</a>-based          style it uses. Oftentimes I end up writing command line tools for          applications that don't need a UI or for ones that can be scripted,          and I'll definitely have a look at Click the next time the need comes          up. You can read the author's motivation for writing an alternative to          the inbuilt <a href="https://docs.python.org/3.4/library/argparse.html">argparse</a> module <a href="http://click.pocoo.org/5/why/">here</a>, and learn how          it compares to others at <a href="https://realpython.com/blog/python/comparing-python-command-line-parsing-libraries-argparse-docopt-click/">Comparing          Python Command-Line Parsing Libraries - Argparse, Docopt, and Click</a>.</p>
<h2>Amandalynne Paullada: <a href="http://www.nltk.org/">NLTK</a></h2>
<p><a href="http://www.nltk.org/">Natural Language Toolkit</a> is a new          favorite of mine, and I was excited to see Amandalynne's presentation.          The library is rich in features, including word and sentence <a href="https://en.wikipedia.org/wiki/Text_segmentation">segmentation</a>,          <a href="https://en.wikipedia.org/wiki/Part-of-speech_tagging">part-of-speech          tagging</a>, <a href="https://en.wikipedia.org/wiki/Statistical_classification">classifiers</a>,          <a href="https://en.wikipedia.org/wiki/Information_extraction">information          extraction</a>, and more. Amandalynne demo'd a clever nickname          generator (e.g., https://en.wikipedia.org/wiki/Text_segmentation)          which, as you'd expect, was a hit. Even (or especially) when it          surprised. I'm currently evaluating it for a <a href="http://www.matthewcornell.org/blog/2015/8/29/proposal-an-online-skeptical-toolbox.html">skeptical          toolbox</a> idea I have, using it for <a href="https://en.wikipedia.org/wiki/Named-entity_recognition">named-entity          recognition</a>, for example. (I learned a little about this area in          my last contract; it's surprisingly difficult.) There's a very well          done NLTK <a href="http://www.nltk.org/book/">book</a>, which I came          away from reading thinking that it would be a great introduction to          Python for newbies. This is because learning the language is set in          the context of text processing, which is inherently cool. Good stuff.</p>
<h2>Ned Jackson Lovely: <a href="http://pythonhosted.org/itsdangerous/">itsdangerous</a> and <a href="https://pypi.python.org/pypi/bcrypt/2.0.0">bcrypt</a></h2>
<p><span class="full-image-float-left ssNonEditable"><span><img title="itsdangerous logo" width="50%" height="50%" src="http://pythonhosted.org/itsdangerous/_static/itsdangerous.png" alt="itsdangerous logo" /></span></span></p>
<p><a href="http://pythonhosted.org/itsdangerous/">itsdangerous</a> is a          library for doing cryptographic signing, and can be applied to cases          like secure "amnesia" ("forgotten password") emails or saving          protected cookies. Using the module this way can, in some cases,          eliminate the need to store hashed data in server-side tables. <a href="https://pypi.python.org/pypi/bcrypt/2.0.0">bcrypt</a> is a tool for password hashing. Not having done work in this area, I          can't say much about these libraries, but I fully believed Ned's point          about DIY cryptography being hard to get right, and about getting          professionals involved for anything important, such as protecting          credit card information.</p>
</div>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35484584.xml</wfw:commentRss></item><item><title>An Arduino Relaxation Tool Using Two Cell Phone Motors</title><dc:creator>Matthew Cornell</dc:creator><pubDate>Tue, 08 Sep 2015 20:16:47 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/9/8/an-arduino-relaxation-tool-using-two-cell-phone-motors.html</link><guid isPermaLink="false">434480:8122567:35467365</guid><description><![CDATA[<p><span class="full-image-float-right ssNonEditable"><span><img style="width: 100px;" src="http://www.matthewcornell.org/storage/post-images/Arduino_logo_pantone.png?__SQUARESPACE_CACHEVERSION=1441744612777" alt="" /></span></span><span class="full-image-float-left ssNonEditable"><span><img style="width: 200px;" src="http://www.matthewcornell.org/storage/post-images/assembled-board.jpg?__SQUARESPACE_CACHEVERSION=1441744599417" alt="" /></span></span>Here's a fun little project I did a while back, which I thougth I'd share. The idea was to have two small motors (like the ones in cell phones) that you hold in each hand, one motor per hand. The motors alternately pulse back and forth between the left and right hands, with two potentiometers controlling how long each motor stays on, and the delay between switching. I thought it might be relaxing (and fund to build), and it actually is! It's also kind of a treat at parties (at least the geeky ones I'm invited to) because you get to see how differently people set the pots to get their own relaxation level. I like mine relatively slow, but maybe it could be cranked up to work as a stimulator.</p>
<p>Anyway, the project is on GitHub at <a href="https://github.com/matthewcornell/arduino-relaxation-tool">http://github.com/matthewcornell/arduino-relaxation-tool</a>. Below I've included the readme along with some pics. Enjoy!</p>
<h2>Circuit</h2>
<p><span class="full-image-float-right ssNonEditable"><span><img style="width: 200px;" src="http://www.matthewcornell.org/storage/post-images/circuit-diagram.jpg?__SQUARESPACE_CACHEVERSION=1441744772998" alt="" /></span></span>This is basically a mashup of two standard Arduino projects - reading analog values from a pot, and controlling a piezo motor. The circuits are duplicated for left and right sides. The circuit diagram is <a href="https://github.com/matthewcornell/arduino-relaxation-tool/blob/master/circuit-diagram.jpg">here</a>, and a picture of the assembled board is <a href="https://github.com/matthewcornell/arduino-relaxation-tool/blob/master/assembled-board.jpg">here</a>.</p>
<p>Here is an example of the individual projects:</p>
<ul>
<li>Reading a pot: <a href="https://www.arduino.cc/en/Tutorial/Potentiometer">Reading a Potentiometer (analog input)</a></li>
<li>Driving a piezo motor: <a href="https://books.google.com/books?id=raHyKejOBF4C&amp;pg=PA276&amp;lpg=PA276&amp;dq=figure+8-8+connecting+a+vibration+motor&amp;source=bl&amp;ots=gmm0_Rgz7K&amp;sig=LKvfqXUdqfFy6ZoNYdsy1oS8n7M&amp;hl=en&amp;sa=X&amp;ved=0CB8Q6AEwAGoVChMI38mBqpnmxwIVwj4-Ch15Hgby#v=onepage&amp;q=figure%208-8%20connecting%20a%20vibration%20motor&amp;f=false">Figure 8-8 of the Arduino Cookbook</a> (Amazon link <a href="http://www.amazon.com/gp/product/1449313876/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=1449313876&amp;linkCode=as2&amp;tag=masidbl-20&amp;linkId=2ND72CEWBWF4INHS">here</a>).</li>
</ul>
<h2>Parts</h2>
<ul>
<li>2 x 1K Ohm resistors</li>
<li>2 x 2N2222 transistors</li>
<li>2 x 1N4001 diodes</li>
<li>2 x 0.1uF capacitors</li>
<li>2 x piezo motors, e.g., <a href="http://www.ebay.com/itm/10X-Micro-Vibrating-Motor-Cell-Phone-Beam-Bristlebot-Robot-6x10mm-Arduino-PI-USA-/161237537955?hash=item258a8194a3">Micro Vibrating Motor Cell Phone Beam Bristlebot Robot 6x10mm</a></li>
<li>2 x 33 Ohm resistors</li>
<li>2 x 10K Ohm circuit board pots</li>
</ul>
<h2>Assembly</h2>
<p><span class="full-image-float-left ssNonEditable"><span><img style="width: 200px;" src="http://www.matthewcornell.org/storage/post-images/motor-closeup.JPG?__SQUARESPACE_CACHEVERSION=1441744718681" alt="" /></span></span>I used a basic Arduino breadboard - it was pretty straighforward. I didn't have any 33 Ohm resistors so I had to use 3 x 10 Ohm ones in series. For the motors I simply applied some heat shrink tubing for something to hold onto. I looked into more durable cases (I tried ping pong balls), but this worked OK for the prototype. See the pic <a href="https://github.com/matthewcornell/arduino-relaxation-tool/blob/master/motor-closeup.JPG">here</a>.</p>
<h2>Code</h2>
<p>The code manages four pins - two analog in ones for reading the pots,  and two digital out ones for controlling the motors. The fifth pin is  the built in LED digital out one that I pulse for effect. The pots are  read by <code>adjustPeriodAndDurationBasedOnPots()</code> and saved into two variables: <code>vibePeriod</code> (how often the motor vibrates in ms: a range ~ [100 ms, 2000 ms]) and <code>vibeDuration</code> (how long the motor vibrates, in the same time range). The last variable is <code>isCycleLedOn</code>, which alternates between 0 and 1 with each cycle. After <a href="https://www.arduino.cc/en/Reference/Setup">setup()</a> initializes the pins, The standard Arduino <a href="https://www.arduino.cc/en/Reference/Loop">loop()</a> function just reads the pots (which sets the two control variables), cycles the first motor, and repeats for the second one. <code>cycleMotor()</code> turns on the motor, cycles the LED, waits the appropriate duration, then repeats to turn the motor off. Finally, <code>adjustPeriodAndDurationBasedOnPots()</code> reads the analog value from each pot (a value between 0 and 1023) and  scales it to a range between ~100ms to 2000ms, which seemed to work  pretty well. (Any faster was irritating, and any slower would have been  boring :-)</p>
<p>&nbsp;</p>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35467365.xml</wfw:commentRss></item><item><title>A simple Fabric.js-based web app for annotating SVG files</title><dc:creator>Matthew Cornell</dc:creator><pubDate>Mon, 31 Aug 2015 14:45:37 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/8/31/a-simple-fabricjs-based-web-app-for-annotating-svg-files.html</link><guid isPermaLink="false">434480:8122567:35458857</guid><description><![CDATA[<p><span class="full-image-float-left ssNonEditable"><span><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/3/34/Javascript_icon.svg/150px-Javascript_icon.svg.png?__SQUARESPACE_CACHEVERSION=1441033072445" alt="" /></span></span>In the check-it-out category, I want to share a web app prototype I wrote recently for a client. It was my first experience with writing non-trivial JavaScript (<a href="http://www.peepweather.com/">peepweather.com</a> uses a little - its source is on GitHub at <a href="https://github.com/matthewcornell/peepweather">github.com/matthewcornell/peepweather</a>) and was a lot of fun to write. The app had no polish, as you can see from the screen shots, because all they wanted was for me to quickly write a solid starting point for an upcoming fall project of theirs.</p>
<p>Following is some detail from the <a href="https://github.com/matthewcornell/paper-header-annotator-2/blob/master/README.md">README</a> on the GitHub fork I created. You can play with it at <a href="https://annotator-demo.herokuapp.com/">annotator-demo.herokuapp.com</a>.</p>
<div id="readme" class="instapaper_body blob">
<h2>Description</h2>
<p>This is a simple <a href="http://flask.pocoo.org/">Flask</a>-based web app that demonstrates using <a href="http://fabricjs.com/">Fabric.js</a> to implement a direct-manipulation SVG annotation tool. It has a bare-bones UI and layout (i.e., no Bootstrap or equivalent) and no production-level features such as authentication, authorization, document owners and assignments, error handling, concurrent access, deployment, etc. However, it does support most of the front end features desired, plus the connection to the back end. The original prototype supported adding new svg files by dropping them into the repository directory, which a DAO would pick up, creating corresponding json files as needed. This demo uses a fixed repository of a few files to keep it simple. The point is mainly to show the front end proof-of-concept using Fabric.js.</p>
<h2>Motivation</h2>
<p><span class="full-image-float-right ssNonEditable"><span><img style="width: 200px;" src="http://www.matthewcornell.org/storage/post-images/annotator-making-mix-nets.png?__SQUARESPACE_CACHEVERSION=1441385179777" alt="" /></span></span>The fuller version of this tool was used to help a group of people annotate thousands of PDF files in SVG format as generated by Mozilla Labs' <a href="https://github.com/mozilla/pdf.js">pdf.js</a>. The SVG retains formatting and so is browser-renderable, but the markup is more useful for <a href="https://en.wikipedia.org/wiki/Information_extraction">information extraction</a> than that of PDF. The ultimate goal of the project was to create a gold standard for comparing the IE output to. Along with saving a document's annotations, users could ask the server for the text bounded by a particular annotation rectangle - a non-trivial problem with these complex SVG files (the 's generated are quite chopped up).</p>
<p>The decision to to write a tool was made after I researched existing annotation tools including some amazing ones:</p>
<ul>
<li><a href="http://annotatorjs.org/">Annotator</a></li>
<li><a href="http://zurb.com/playground/javascript-annotation-plugin">ZURB JavaScript Annotation Plugin</a></li>
<li><a href="http://annotorious.github.io/">Annotorious</a></li>
<li><a href="http://www.annotationstudio.org/">Annotation Studio</a></li>
</ul>
<p>Unfortunately they lacked the customization and power user features we needed for the high-throughput workflow necessary to annotate 1000s of SVG files. I then surveyed JavaScript graphics libraries (both Canvas- and SVG-based) to find one to support the rectangle-based UI we wanted. I settled on the excellent Fabric.js (the <a href="http://fabricjs.com/demos/">demos</a> are awesome) after having looked at general purpose ones (of which there are many) including:</p>
<ul>
<li><a href="http://raphaeljs.com/">Rapha&euml;l</a></li>
<li><a href="http://createjs.com/easeljs">EaselJS</a></li>
<li><a href="http://d3js.org/">D3.js</a></li>
<li><a href="http://jqueryui.com/">jQuery UI</a></li>
<li><a href="http://processingjs.org/">Processing.js</a></li>
<li><a href="http://paperjs.org/">Paper.js</a></li>
<li><a href="http://snapsvg.io/">Snap.svg</a></li>
</ul>
<p>I also looked at diagram-oriented libraries, but they felt like too much work compared to a straightforward <a href="https://en.wikipedia.org/wiki/Canvas_element">canvas</a> wrapper.</p>
<ul>
<li><a href="https://jsplumbtoolkit.com/">jsPlumb Toolkit</a></li>
<li><a href="http://neyric.github.io/wireit/docs/">WireIt</a></li>
<li><a href="http://www.jointjs.com/">JointJS - the HTML 5 JavaScript diagramming library.</a></li>
<li><a href="http://www.draw2d.org/draw2d/">Draw2D touch: Blog</a></li>
<li><a href="http://adrai.github.io/flowchart.js/">flowchart.js</a></li>
<li><a href="http://js-graph-it.sourceforge.net/">js-graph.it homepage</a></li>
<li><a href="http://js.cytoscape.org/">Cytoscape.js</a></li>
</ul>
<p>In the end I liked Fabric best for its inbuilt support of handles and grouping, and its solid level of activity. And it worked pretty well, modulo a number of gotchas. It's a really nice piece of work, as are many of these. The JavaScript graphics library scene is definitely alive and well.</p>
<h2>Code tour</h2>
<p><span class="full-image-float-left ssNonEditable"><span><img style="width: 200px;" src="http://www.matthewcornell.org/storage/post-images/annotator-pressure-effect.png?__SQUARESPACE_CACHEVERSION=1441385491483" alt="" /></span></span>While I don't have my client's permission to share the code, I'll sketch out the implementation for the curious.</p>
<h2>Routes</h2>
<p>At the top level of this standard Flask MVC app is app/routes.py, which has four URIs/controllers:</p>
<ul>
<li><code>/</code>: app/templates/index.html template. Lists the SVG files in the fake <a href="https://en.wikipedia.org/wiki/Data_access_object">DAO</a>.</li>
<li><code>/docs/&lt;fileName&gt;</code>: app/templates/edit.html template. Described below.</li>
<li><code>/docs/&lt;fileName&gt;/annotations</code>: REST API endpoint for getting a document's annotations as a JSON array. Finds and reads the JSON file corresponding to the SVG file and returns it as 'application/json'.</li>
<li><code>/docs/&lt;fileName&gt;/text</code>: placeholder endpoint for calculating the text bounded by a rectangle. Pulls the rectangle bounds from the query parameters and returns a fake string.</li>
</ul>
<h2>edit.html</h2>
<p>This view has a simple vanilla HTML editing section at the top with three hard-coded annotation types, and the editing area underneath. The latter layers the <a href="http://fabricjs.com/docs/fabric.Canvas.html">Fabric canvas</a> on top of the SVG file, whose element is inserted dynamically via <code>loadSvg()</code> in app/static/edit-document.js (see below). Finally, edit.html sets the variables needed by edit-document.js, based on the Flask-injected <code>fileName</code> variable, and loads the Fabric and edit-document.js.</p>
<h2>edit-document.js</h2>
<p>This file does the heavy lifting. On load it links up the button and shortcut actions to their methods and then calls <code>loadSvg()</code> to load the SVG document and its annotations. It does so using an AJAX call because of a browser SVG loading bug where the SVG size is reported incorrectly until it's completely loaded. Once it's loaded, <code>loadSvg()</code> inserts the SVG element behind the Fabric canvas, sets its size, and then initializes the Fabric canvas, saving it as an application property on the canvas element for easy access. Once that's all done, the function calls <code>loadAnnotations()</code>, which performs the AJAX call to <code>/docs/&lt;fileName&gt;/annotations</code> and then creates <a href="http://fabricjs.com/docs/fabric.Rect.html">Fabric Rect</a> objects for each, using the correct color for the annotation type.</p>
<p>One tricky bit was handling the line connecting linked Rects. Lines are managed explicitly (there is no built in 'connector' feature in Fabric), so their endpoints must be dynamically adjust during rectangle moves and resizes, and pointers to/from them must be saved as properties on the Fabric objects (i.e., a Rect needs to know all of its Lines, and a Line needs to know its two endpoints' Rects).</p>
<p>The rest of the code manages all the fiddly aspects of the app - resizing, duplicating, linking, etc. And of course button state must be updated based on selection changes. The only other mildly interesting thing is the AJAX call to get an annotation rect's text - a straightforward call to the <code>/docs/&lt;fileName&gt;/text</code> endpoint.</p>
<h2>User Documentation</h2>
<p>The  UI is a straightforward direct manipulation one where users work with  rectangle objects. Click to select, drag to move, drag resize handles to  resize, click the delete rectangle button to remove, etc. The only  feature that's non- obvious is how to add and remove links between  rectangles. To add a link, select exactly two rectangles with the same  label and no existing link and then click the add link button. To remove  a link, select two rectangles with an existing link and the click the  remove link button. Keystroke shortcuts are supported for power users:</p>
<h2>Keystroke shortcuts</h2>
<h3>Types</h3>
<pre>          1: set current type to Title
 shift  + 1: filter all but Title
          2: set current type to Abstract
 shift  + 2: filter all but Abstract
          3: set current type to Author
 shift  + 3: filter all but Author
     Escape: reset filter
</pre>
<h3>Create</h3>
<pre>+: create new annotation using current type
</pre>
<h3>Move &amp; Resize</h3>
<pre>                     arrow key: move selection 1px
            shift  + arrow key: move selection 10px
           option  + arrow key: resize selection 1px
 shift  +  option  + arrow key: resize selection 10px
</pre>
<h3>Delete/Duplicate</h3>
<pre>backspace|delete: delete selection
    control  + d: duplicate selection
</pre>
<h3>Select</h3>
<pre>          tab: select next
 shift  + tab: select previous
</pre>
<h3>Text Feedback</h3>
<pre>x: display text for selection
</pre>
</div>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35458857.xml</wfw:commentRss></item><item><title>Proposal: An online skeptical toolbox</title><category>critical_thinking</category><category>projects</category><category>skepticism</category><category>technology</category><dc:creator>Matthew Cornell</dc:creator><pubDate>Sat, 29 Aug 2015 14:36:30 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/8/29/proposal-an-online-skeptical-toolbox.html</link><guid isPermaLink="false">434480:8122567:35457355</guid><description><![CDATA[<p><span class="full-image-float-right ssNonEditable"><span><img style="width: 250px;" src="http://www.matthewcornell.org/storage/post-images/4667896076_a3ae9e43fa_b.jpg?__SQUARESPACE_CACHEVERSION=1440859118708" alt="" /></span></span>For some time I've been collecting project ideas to help my overall goal of "computing in the <span class="il">service</span> of <span class="il">humanity</span>" (a phrase I recently picked up). While I definitely want to find consulting work in this area, starting up a personal project in the meantime is important to me. Out of a bunch of ideas I've decided to start with an online skeptical toolbox.</p>
<p>What finally kicked me in the butt were two things: 1) The publishing of Google's paper <a href="http://arxiv.org/abs/1502.03519v1">Knowledge-Based Trust: Estimating the Trustworthiness of Web Sources</a> (which got a <a href="https://www.google.com/search?hl=en&amp;gl=us&amp;tbm=nws&amp;authuser=0&amp;q=Knowledge-Based+Trust%3A+Estimating+the+Trustworthiness+of+Web+Sources&amp;oq=Knowledge-Based+Trust%3A+Estimating+the+Trustworthiness+of+Web+Sources">good bit of coverage</a>), and 2) the recent <a href="http://www.csicop.org/">Committee for Skeptical Inquiry</a> article <a href="http://www.csicop.org/specialarticles/show/online_tools_for_skeptical_fact_checking">Online Tools for Skeptical Fact Checking</a> by <a href="http://www.csicop.org/author/tamar_wilner">Tamar Wilner</a>. To that end I've created a <a href="https://github.com/matthewcornell/skeptical-toolbox">GitHub project</a> that at the moment has two files: the <a href="https://github.com/matthewcornell/skeptical-toolbox/blob/master/README.md">proposal itself</a> and a somewhat-organized <a href="http://www.xmind.net/">XMind</a> <a href="https://en.wikipedia.org/wiki/Mind_map">mind map</a> file <span class="css-truncate-target css-truncate"><a id="c23eced4389f7c4a3ec6435cae975a25-a639d3b09c3313857176117f4ebc511a2b0b0d81" class="js-directory-link" title="skeptical-toolbox.xmind" href="https://github.com/matthewcornell/skeptical-toolbox/blob/master/skeptical-toolbox.xmind">skeptical-toolbox.xmind</a></span> that lists some detail.</p>
<p>As I said in the <a href="https://github.com/matthewcornell/skeptical-toolbox#implementation">Implementation</a> section, the tools cover a range of complexity, which affords our   quickly rolling out something useful using the simpler ones, and   progressively introduce more sophisticated tools as the toolbox   develops. I plan on using <a href="https://www.python.org/">Python</a> + <a href="http://flask.pocoo.org/">Flask</a> to write it in (same as I used for <a href="http://www.peepweather.com/">PeepWeather</a>), but I'm quite open to other languages and frameworks, such as <a href="https://www.ruby-lang.org/en/">Ruby</a> on <a href="http://rubyonrails.org/">Rails</a> if someone steps up and convinces me. One thing in Python's favor is the popular <a href="http://www.nltk.org/">Natural Language Toolkit</a> with its <a href="https://en.wikipedia.org/wiki/Information_extraction">information extraction</a> tools, especially around <a href="https://en.wikipedia.org/wiki/Named-entity_recognition">entity recogition</a>.<br /><br /></p>
<p>What do you think? It is very early days (day zero, I suppose) but I hope that sharing this will generate some thoughts. If you're interested in helping build this - awesome! Just comment here or <a href="http://www.matthewcornell.org/contact">send me a line</a>.</p>
<p>(Image: <a href="https://www.flickr.com/photos/see-through-the-eye-of-g/4667896076/">Memory Belt</a>)</p>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35457355.xml</wfw:commentRss></item><item><title>Available! Versatile Software Engineer, 20+ Years Experience, Object-Oriented &amp; Extreme Programming</title><dc:creator>Matthew Cornell</dc:creator><pubDate>Tue, 11 Aug 2015 17:37:28 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/8/11/available-versatile-software-engineer-20-years-experience-ob.html</link><guid isPermaLink="false">434480:8122567:35440719</guid><description><![CDATA[<p><span class="full-image-float-left ssNonEditable"><span><img style="width: 250px;" src="http://www.matthewcornell.org/storage/post-images/8050383428_c8fd9bf019_o.jpg?__SQUARESPACE_CACHEVERSION=1439319756231" alt="" /></span></span>After taking a break from computing in 2009 and creating a <a href="http://www.matthewcornell.org/display/Search?searchQuery=edison">new kind of social platform</a> - one based on treating life as an experiment - I happily returned to <a href="https://en.wikipedia.org/wiki/Computer_science">CS</a> research in 2011. I was asked to re-join the <a href="https://en.wikipedia.org/wiki/Artificial_intelligence">AI</a> group at UMass where I previously helped build <a href="https://kdl.cs.umass.edu/display/public/Proximity">Proximity</a>, a platform for <a href="https://en.wikipedia.org/wiki/Machine_learning">machine learning</a> research. However, that position's funding has ended (as is the way of research's ebb and flow) and I'm excited to move on to the next adventure.</p>
<p>In those four years I got a lot done, including exploring ways to scale up the Ph.D. students' algorithms [1], coaching students in the <a href="https://en.wikipedia.org/wiki/Extreme_programming">Extreme Programming</a> <a href="https://en.wikipedia.org/wiki/Agile_software_development">agile software development</a> methodology that I love so much, implementing lab infrastructure improvements (I consolidated multiple aging servers into a single modern cluster, and got the lab using the commercial <a href="https://en.wikipedia.org/wiki/Wiki">wiki</a>, <a href="https://www.atlassian.com/software/confluence/">Confluence</a>), and of course collaboratively designing and coding the lab's new causal learning in <a href="https://www.python.org/">Python</a> (a language I love, BTW, though I still enjoy <a href=" https://www.java.com/">Java</a>).</p>
<p>As a pleasant surprise, before the funding ended in May, at the last minute another CS professor asked me to help his <a href="http://www.iesl.cs.umass.edu/">lab</a> get some urgent work done during the summer, which I was happy to do. While there I benchmarked and started an optimization effort for their new <a href="https://en.wikipedia.org/wiki/Information_extraction">information extraction</a> pipeline, led the team in meeting a crucial grant deadline, and prototyped a web-based PDF annotation tool [2] for creating a <a href="http://stats.stackexchange.com/a/19051">gold standard</a> to evaluate their algorithms' performance against. I was introduced to <a href="https://nodejs.org/ ">Node.js</a>, continued learning <a href="http://www.scala-lang.org/">Scala</a> (I wrote some for my GraphX work), and JavaScript for the web app.</p>
<p>Looking forward, I've started my job search for my next project, with the goal of finding work that's challenging, engaging, and meaningful. I've not had to do one in a while, so the process itself is a kind of experiment. What approaches will work? What is out there? Who is doing cool work? Exciting, and a little scary.</p>
<p>So: if you know someone looking for a versatile software engineer with lots of experience who writes excellent code, then please - <a href="http://www.matthewcornell.org/contact">drop me a line</a>. My LinkedIn profile is <a href="https://www.linkedin.com/in/matthewcornell ">linkedin.com/in/matthewcornell</a>.</p>
<ul>
<li>[1] I explored multiple <a href="https://en.wikipedia.org/wiki/Massively_parallel_%28computing%29">MPP</a> approaches including single-node SQL using <a href="http://www.postgresql.org/">PostgreSQL</a>, distributed SQL using <a href="http://www.cloudera.com/content/cloudera/en/products-and-services/cdh/impala.html">Impala</a> on the <a href="https://hadoop.apache.org/">Hadoop</a> ecosystem and <a href="http://www.vertica.com/">Vertica</a>, and graph databases such as <a href="https://spark.apache.org/graphx/">GraphX</a>, <a href="http://neo4j.com/">Neo4J</a>, and <a href="https://giraph.apache.org/ ">Giraph</a>.</li>
<li>[2] The prototype allows users to load a PDF file as a browser-renderable SVG, draw and edit rectangular regions overlaid on the text, and save and load them from a server. I used the standard web technologies of JavaScript and JSON, all running on <a href="https://www.playframework.com/">Play Framework</a> and its Java API. I used <a href="http://fabricjs.com/">Fabric.js</a> for the direct manipulation UI, which I really enjoyed working with (check out the awesome<a href="http://fabricjs.com/demos/"> demos</a>).</li>
</ul>
<p>(Image from <a href="https://www.flickr.com/photos/boston_public_library/8050383428/">Fountain on Boston Common</a>.)</p>
<ul>
</ul>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35440719.xml</wfw:commentRss></item><item><title>Smith College Guest Lecture, "How to build a simple production web site in Python"</title><category>peepweather</category><category>presentation</category><dc:creator>Matthew Cornell</dc:creator><pubDate>Wed, 22 Apr 2015 14:21:13 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/4/22/smith-college-guest-lecture-how-to-build-a-simple-production.html</link><guid isPermaLink="false">434480:8122567:35315564</guid><description><![CDATA[<p><span class="full-image-float-left ssNonEditable"><span><img style="width: 250px;" src="http://www.matthewcornell.org/storage/lego-computer.jpg?__SQUARESPACE_CACHEVERSION=1429714902050" alt="" /></span></span>Last week I had a fun gig giving a one-hour guest lecture for <a href="http://cs.smith.edu/index.php">Smith College</a>'s freshman <em>Intro Computer Science through Programming</em> class. The topic was "How to build a simple production web site in Python" in which I was invited to share the process and technology that went into building <a href="http://www.peepweather.com/">PeepWeather.com</a> . The rationale was that it would give the students a taste of what is possible to create using some relatively straightforward Python frameworks and libraries, hopefully providing some motivation for those wondering what good their current and (necessarily) bounded assignments are.</p>
<p>I had a great time meeting the challenge of putting together a presentation that had breadth and was motivating, but at the same time understandable to the audience (with a the right amount of stretching to keep them engaged). It went well, with some excellent questions during and after the talk. Here I want to share a little about the concepts and technology I covered in case you're considering giving a similar presentation of your own site. For the curious, <a href="https://drive.google.com/file/d/0B7Ywd0BmSUF7N015aHFmZHhXdVU/view?usp=sharing">here's the PowerPoint as a PDF</a> (unfortunately the video did not work out).</p>
<h2>Technologies</h2>
<p>I named these technologies, describing each pretty briefly, except for diving deeper into Flask:</p>
<ul>
<li><strong>Python web apps</strong>: <a href="http://flask.pocoo.org/">Flask</a>, <a href="https://en.wikipedia.org/wiki/HTML">standard web tech</a> (HTML, CSS, cookies, JavaScript), and the common JavaScript libraries: <a href="http://jquery.com/">jQuery</a> for sanity, <a href="http://getbootstrap.com/">Bootstrap</a> for layout, and <a href="https://twitter.github.io/typeahead.js/">typeahead.js</a> for autocompletion.</li>
<li><strong>Application-specific</strong>: XML and <a href="https://en.wikipedia.org/wiki/Representational_state_transfer">REST</a> (for the <a href="http://graphical.weather.gov/xml/">weather.gov API</a>), <a href="http://pillow.readthedocs.org/">Pillow</a> for the widget image, <a href="http://rhodesmill.org/pyephem/">PyEphem</a> for the daylight calculation, and <a href="https://erikflowers.github.io/weather-icons/">Weather Icons</a>.</li>
<li><strong>Deployment</strong>: <a href="https://www.heroku.com/">Heroku</a> for hosting, <a href="http://www.git-scm.com/">Git</a> for pushing, <a href="http://gunicorn.org/">gunicorn</a> for Flask on Heroku, and <a href="https://www.namecheap.com/">Namecheap.com</a> for domain registration.</li>
</ul>
<h2>Concepts</h2>
<p>I covered the following at various levels of detail:</p>
<ul>
<li>Web frameworks (routing, templates)</li>
<li>The <a href="https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller">Model-View-Controller</a> design pattern (and <a href="https://en.wikipedia.org/wiki/Software_design_pattern">design patterns</a> in general)</li>
<li><a href="https://en.wikipedia.org/wiki/Object-oriented_design">Object-oriented design</a> using <a href="http://www.agilemodeling.com/artifacts/crcModel.htm">Class-Responsibility-Collaborator</a> cards</li>
<li><a href="https://en.wikipedia.org/wiki/Test-driven_development">Test-driven development</a></li>
<li><a href="https://en.wikipedia.org/wiki/User_experience">User interface design</a></li>
</ul>
<h2>The Code</h2>
<p>I jumped into the code at the appropriate points, <a href="https://www.jetbrains.com/idea/help/code-folding.html">folding</a> as needed so I didn't overwhelm. I stuck to high level basics, highlighting the concepts they currently know including classes and methods. Mainly the focus was on how the concepts got implemented in Python, but no significant code detail; there simply wasn't time. (Note that IntelliJ IDEA's <a href="https://www.jetbrains.com/idea/help/intellij-idea-viewing-modes.html#d673772e18478">presentation mode</a> worked great for the projector.)</p>
<h2>Motivation</h2>
<p>I tried to make two points to get the students excited. First, they are programmers and so they have a powerful skill to <strong>create web sites</strong>! I suggested they pay attention to any thoughts that come up like "Wouldn't it be great if you could __?" or "Why isn't there a site to __?" Second, I pointed out that putting together even a small site is a fantastic portfolio piece that impresses prospective employers. It shows that they can have ideas and, better yet, bring them into reality on their own. I wrapped up by saying that I had a ton of fun writing PeepWeather and learning the various technologies. It's extremely gratifying to bring something into reality (if that's the right word for something as virtual as a web site :-)</p>
<p>Overall I enjoyed both the talk's preparation and presentation, and I hope to give it at the other <a href="https://en.wikipedia.org/wiki/Five_College_Consortium">Five Colleges</a>.</p>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35315564.xml</wfw:commentRss></item><item><title>Loving the wonderful book Code Complete 2</title><dc:creator>Matthew Cornell</dc:creator><pubDate>Fri, 27 Mar 2015 13:08:13 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/3/27/loving-the-wonderful-book-code-complete-2.html</link><guid isPermaLink="false">434480:8122567:35285846</guid><description><![CDATA[<p><span class="full-image-float-left ssNonEditable"><span><img src="http://www.matthewcornell.org/storage/post-images/cc2e-cover-small.jpg?__SQUARESPACE_CACHEVERSION=1427462848154" alt="" /></span></span>After two false starts [1], I am working my way through Steve McConnell's book Code Complete: A Practical Handbook of Software Construction, Second Edition (links: <a href="http://www.stevemcconnell.com/">author</a>, <a href="http://cc2e.com/">book</a> , <a href="http://www.amazon.com/gp/product/0735619670/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=0735619670&amp;linkCode=as2&amp;tag=masidbl-20&amp;linkId=A25BEWE53F3RWZHS">amazon</a>). He describes the topic as "an extended description of the programming process for creating classes and routines." Extended is right!</p>
<p>There are very good reasons why it is in the top 10 in software design at Amazon: After 1/2 way through, I've found the book is unique and very well-written. (I am not alone in this assessment [2].)</p>
<p>It is unique because it goes into a level of detail about coding that is specific and <strong>really</strong> deep. McConnell has given incredibly clear thought into the minutae of programming, and then shared it with fine writing. My boss chucked when I showed him that there are four <em>chapters</em> (table of contents <a href="http://www.stevemcconnell.com/cc.htm">here</a>) on variables, one dedicated solely to naming them. Awesome! I've been writing software for a long time, and it was gratifying to find that McConnell has put into words things learned intuitively, such as <em>Think about efficiency</em>:</p>
<blockquote>
<p>... it's usually a waste of effort to work on efficiency at the level of individual routines. The big optimizations come from refining the high-level design, not the individual routines. You generally use micro-optimizations only when the high-level design turns out not to support the system's performance goals, and you won't know that until the whole program is done. Don't waste time scraping for incremental improvements until you know they're needed.</p>
</blockquote>
<p>But beyond being reminded of these (and there are many, many of them, for example: "Initialize each variable close to where it's first used"), a big payoff is the new-to-me ones, of which there are many. The book is simply too detailed to summarize the main points, so I'll just share some tidbits that jumped out at me:</p>
<ul>
<li>"Upstream Prerequisites" ("programmers are at the end of the food chain. The architect consumes the requirements, the designer consumes the architecture, and the coder consumes the design.") His point: "the overarching goal of preparation is risk reduction. By far the most common project risks in software development are <em>poor requirements</em> and <em>poor project planning</em>." (Note: I found the StackOverflow question <a href="http://stackoverflow.com/questions/704855/software-design-vs-software-architecture">Software design vs. software architecture</a> answered terminology confusion I had about those terms: "I think we should use the following rule to determine when we talk about Design vs. Architecture: If the elements of a software picture you created can be mapped one to one to a programming language syntactical construction, then is Design, if not is Architecture.")</li>
<li>Programming IN a language (limit thoughts to constructs the language directly supports) vs. INTO a language (decide what thoughts you want to express then determine how to do so using the language's tools).</li>
<li>"Wicked problems" (like software design): can be clearly defined only by solving it or solving part of it.</li>
<li>"Design is a sloppy process": is about tradeoffs and priorities; involves restrictions; is nondeterministic; is a heuristic process; is emergent.</li>
<li>Software's Primary Technical Imperative: the importance of managing complexity by dividing a system into subsystems. "The goal of all software-design techniques is to break a complicated problem into simple pieces. The more independent they are the better."</li>
<li>For the sake of controlling complexity, you should maintain a heavy bias against inheritance (and prefer containment): Inherit when you want the base class to control your interface. Contain when you want to control your interface.</li>
<li>Aim for loose <a href="http://en.wikipedia.org/wiki/Coupling_%28computer_programming%29">coupling</a> (the manner and degree of interdependence between software modules) and strong <a href="http://en.wikipedia.org/wiki/Cohesion_%28computer_science%29">cohesion</a> (the degree to which the elements of a module belong together).</li>
<li>Routine length (he generalizes procedures, functions, and methods as "routines"): Allow them to grow organically up to 100-200 lines (excluding comments &amp; blank lines). (My routines tend to be smaller - I'll have to give this more thought.)</li>
<li>Routine parameters: pass &lt;= ~7 of them. More implies too much coupling.</li>
<li>Design Practices - Capturing Your Design Work: code documentation, wiki, email summaries, camera (whiteboard) vs. a drawing tool, flip charts, <a href="http://www.c2.com/cgi/wiki?CrcCard">CRC cards</a> (I love 'em), UML diagrams.</li>
<li>The Pseudocode Programming Process: It was helpful to see this named and formalized. "Once the pseudocode is written, you build the code around it and the pseudocode turns into programming-language comments. This eliminates most commenting effort. If the pseudocode follows the guidelines, the comments will be complete and meaningful." Cool!</li>
</ul>
<p>And many more. Bottom line: If you are serious and committed about improving your craft, read the book! How about you? What coding books have helped you be a better programmer?</p>
<p>[1] There were two reasons my first two attempts to read this hefty 900-pager failed. 1) I wasn't fully committed, and 2) I didn't have a concrete plan to read it. I resolved the former when I realized I'd been focusing on my work projects to the exclusion of professional development [3] (other than reading a handful of blog posts a week) and needed to shake things up. The second block was simpler to address - break the book up into manageable chunks and commit to reading one every day. I summarized this generally in <a href="http://www.matthewcornell.org/blog/2005/9/20/reading-books-the-gtd-way.html">Reading Books The GTD Way</a>, but in this case I did a <a href="http://en.wikipedia.org/wiki/Back-of-the-envelope_calculation">back-of-the-napkin analysis</a>, working backward from an estimated chapter velocity:</p>
<p>900 pages, 35 chapters</p>
<ul>
<li>1 chapter/hr -&gt; 35 hours</li>
<li>1 hour/workday, 3 workdays/week -&gt; 3 hours/week</li>
<li>-&gt; ~12 weeks (~2.8 months)</li>
</ul>
<p>I've been recording my minutes/page progress, which ranges between 1 and 2 1/2. With an average chapter being about 30 pages, this works out to be about 60 minutes/page. Good estimate! (Actually, I'm readming more chapters than three per week, and at my current rate I will be done at the end of April.)</p>
<p>[2] A helpful review was <a href="http://grovr.blogspot.com/2011/10/code-complete-review.html">Matt Grover's blog: Code Complete Review</a>, which pointed me to Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin (Amazon link <a href="http://www.amazon.com/gp/product/0132350882/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=0132350882&amp;linkCode=as2&amp;tag=masidbl-20&amp;linkId=WISSST5GMIHIHFIW">here</a>). That book looks excellent too, though it apparently has less detail, weighing in at a svelte 500 pages.</p>
<p>[3] I was surprised to find few articles on the Web on the importance of professional development for software engineers, at least via a quick Google search. The main one was <a href="http://www.cio.com/article/2838445/careers-staffing/10-professional-development-tips-for-programmers.html">10 Professional-Development Tips for Programmers</a>. It is structured annoyingly as a slide show, meaning you have to click through to see the content (an SEO and advertising gambit?), so I've collected the titles for you:</p>
<ul>
<li>Staying Current Requires Continuous Learning </li>
<li>Problem-Solving Skills</li>
<li>Communication and People Skills</li>
<li>Networking and Personal Branding</li>
<li>Code Documentation and Neatness</li>
<li>Master Naming Functions</li>
<li>Get Familiar With Agile</li>
<li>Get Familiar With a Native Mobile Platform</li>
<li>Project Management Skills</li>
<li>JavaScript, CSS and HTML5 Skills</li>
</ul>
<p>Searching the two programming-specific sites I like (<a href="https://news.ycombinator.com/">Hacker News</a> and <a href="http://www.reddit.com/r/programming">reddit.com/r/programming</a> - what are your favorites, BTW?) was more fruitful. Here are a few:</p>
<ul>
<li><a href="http://workplace.stackexchange.com/questions/3506/professional-development-is-wide-or-deep-knowledge-better?rq=1">work experience - Professional development, is wide or deep knowledge better?</a></li>
<li><a href="https://www.linkedin.com/pulse/20140828225722-19987-9-levers-of-success-in-a-turbulent-economy">A Simple Tool to Find Your Professional Development Sweet Spot</a></li>
<li><a href="http://blog.smartbear.com/code-review/using-peer-code-reviews-as-a-professional-development-tool/">Using Peer Code Reviews as a Professional Development Tool</a></li>
<li><a href="http://wayback.archive.org/web/20090630033454/http://blog.briandicroce.com/2007/12/29/ten-simple-tips-to-become-a-valuable-software-professional/">Ten Simple Tips To Become A Valuable Software Professional</a></li>
<li><a href="http://www.globalnerdy.com/2013/10/11/staying-sharp-maintaining-your-development-skills/">Staying Sharp: Maintaining Your Development Skills </a></li>
</ul>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35285846.xml</wfw:commentRss></item><item><title>A few ideas around citizen involvement and government/media oversight</title><dc:creator>Matthew Cornell</dc:creator><pubDate>Sun, 22 Mar 2015 20:03:10 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/3/22/a-few-ideas-around-citizen-involvement-and-governmentmedia-o.html</link><guid isPermaLink="false">434480:8122567:35279244</guid><description><![CDATA[<p><a href="https://www.flickr.com/photos/propagandatimes/5389779608"><span class="full-image-float-right ssNonEditable"><span><img style="width: 150px;" src="http://www.matthewcornell.org/storage/post-images/5389779608_b9f74e55de_o.jpg?__SQUARESPACE_CACHEVERSION=1427054935209" alt="" /></span></span></a>While going through old files, I dug up a few single-paragraph proposals from 2004 I wrote for a research lab I worked for at the time. They did not go forward, but I thought I'd share them with you. When I discovered them, I was surprised because I'd forgotten my interest in these areas has been around since then. Another surprise is that the algorithms and technical approaches to possibly implement some of these ideas are just now emerging from computer science research labs - [1] and [2], for example.</p>
<p>Right now I'm exploring ways to restart this interest, and I hope that posting these archived ideas here might jiggle something loose. Cheers!</p>
<p>References:</p>
<ul>
<li>[1] <a href="http://www.newscientist.com/article/mg22530102.600-google-wants-to-rank-websites-based-on-facts-not-links.html#.VQ8mB2Y3SXE">Google wants to rank websites based on facts not links</a></li>
<li>[2] <a href="http://maroo.cs.umass.edu/pub/web/getpdf.php?id=1194">Navigating Controversy as a Complex Search Task</a></li>
</ul>
<h2>1. Increased Corporate Influence of the Federal Government</h2>
<p>Specifically, using technology to enable citizens to monitor for abuses and actions that limit its effectiveness. The first example that comes to mind is the commercialization of legislation: Wealthy corporations and individuals who influence laws to their benefit [1]. The result is laws that ignore the greater good (which is usually in the longer term) in favor of shorter term creation of profits. What can our group do?&nbsp; Let's apply our ideas to the job of examining this cycle of influence. It seems that the data are available and amenable to relational analysis. Is the problem challenging? I'm not sure; it depends on how clear the patterns are.<br /><br />References:</p>
<ul>
<li>[1] <a href="http://www.motherjones.com/politics/2004/05/bundles-influence">Bundles of Influence</a></li>
</ul>
<h2>2. Increased Bias in Consolidated Media Companies</h2>
<p>As citizens I believe we are facing challenges to fundamental aspects of our society as a result the the recent creation of large media conglomerates that control a high percentage of mainstream media [1]. The problem here is that corporations are biasing the news in directions that favor profits over journalism. What can we do? One idea is to create a system that can indicate the bias of news stories. I imagine something like <a href="https://news.google.com/">Google News</a> that lists major news stories along with a bias indicator. The hope is that people reading their favorite writer on a topic will be able to get a summary of the story's bias. A final idea would be to look for stories that are *not* covered by mainstream media, i.e., an automated version of [2].<br /><br />References:</p>
<ul>
<li>[1] <a href="http://www.mindspace.org/liberation-news-service/archives/000284.html">Bill Moyers' Buzzflash interview on Media reform</a></li>
<li>[2] <a href="http://www.projectcensored.org/">Project Censored - The News that Didn't Make the News and Why</a></li>
</ul>
<h2>3. Reduced Rigor in Major Media News Reports</h2>
<p>Another problem related to 2) above is to address the current propensity of some writers to report facts as taken from second sources without investigating the original sources [1]. To solve this we might analyze stories written about a single news event in order to determine what sources they derived their information from. For example, do they all link to a single source, or are sources more diversified? Also consider the recent avalanche of stories (many misinformed) about DARPA's Total (now 'Terrorism') Information Awareness program. Could a software system using our research have been able to determine that a small number of influential stories was propagating around the net with little additional analysis performed by those referring to them? Finally, another idea is to create automated versions of some of the 'fact check' web sites currently available ([2], [3]).<br /><br />References:</p>
<ul>
<li>[1] <a href="http://wayback.archive.org/web/20041204134540/http://www.projectcensored.org/newsflash/mainmedia_failsitself.html">Mainstream Media Fails Itself</a></li>
<li>[2] <a href="http://www.salon.com/2004/04/28/claimvsfact/">Center for American Progress: Claims vs. Facts Database</a></li>
<li>[3] <a href="http://www.factcheck.org/">Annenberg Political Fact Check</a></li>
</ul>
<p>&nbsp;</p>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35279244.xml</wfw:commentRss></item><item><title>The Intrinsic Poverty of Thumbs Up/Down Popularity Voting on the Web</title><dc:creator>Matthew Cornell</dc:creator><pubDate>Sun, 08 Mar 2015 16:30:18 +0000</pubDate><link>http://www.matthewcornell.org/blog/2015/3/8/the-intrinsic-poverty-of-thumbs-updown-popularity-voting-on.html</link><guid isPermaLink="false">434480:8122567:35261340</guid><description><![CDATA[<p><span class="thumbnail-image-float-left ssNonEditable"><span><a href="javascript:showFullImage('/display/ShowImage?imageUrl=%2Fstorage%2Fpost-images%2FPeople_at_work_in_Wartime-_Everyday_Life_in_Wartime_Britain_1940_D1039.jpg%3F__SQUARESPACE_CACHEVERSION%3D1425832345938',800,528);"><img src="http://www.matthewcornell.org/storage/thumbnails/4823311-26022909-thumbnail.jpg?__SQUARESPACE_CACHEVERSION=1425832345939" alt="" /></a></span></span>I'm tired of the malnutrition of discourse in web sites' comments sections. Pick a controversial topic [1] on a popular site, go to the comments section, and if you're like me, you'll start feeling some emotions. Disbelief, disgust, and maybe a lack of faith in how we've (dis)organized our collective thinking ability. Comments by readers on climate change stories are perhaps the worst of the lot - name calling, <a href="http://changingminds.org/techniques/propaganda/propaganda.htm">propaganda</a> , but perhaps most fundamentally, a dearth of <a href="http://forbesindia.com/article/rotman/the-need-for-a-critical-thinking-revolution/23642/1">critical thinking</a> skills. You simply see very, very little <a href="http://changingminds.org/disciplines/argument/argument.htm">solid argumentation</a> taking place.</p>
<p>Some examples are <a href="http://www.washingtonpost.com/news/energy-environment/wp/2015/03/05/the-gops-climate-change-skepticism-in-one-groan-worthy-video/">The GOP&rsquo;s climate change skepticism, in one groan-worthy video</a>, <a href="http://www.nytimes.com/2015/02/19/us/politics/faulted-for-avoiding-islamic-labels-white-house-cites-a-strategic-logic.html">Faulted for Avoiding &lsquo;Islamic&rsquo; Labels to Describe Terrorism, White House Cites a Strategic Logic</a>, and even my local paper has some doozies, e.g., <a href="http://www.masslive.com/news/index.ssf/2015/03/tony_robinson_shooting_protest.html">Tony Robinson shooting: Protestors hit Madison, Wisconsin streets (video)</a> (with gun control at least being a bona fide controversy).<br /><br />I am just starting to look into the research around "hindrances due to basic human limitations" (check out Table 1 [2] from <a href="http://www.skepdic.com/essays/Haskins.html">A Practical Guide To Critical Thinking</a>) and I'm undecided whether it's possible to change minds through technical means (e.g., requiring comment authors to express some kind of well-structured, if simple, argument along with each claim), but it struck me that the <strong>thumbs up/down</strong> voting feature that's common to most of these sites exacerbates the problem. Let me take a naive look at this, the most meager form of interaction (clicking is literally the least you can do to interact) and maybe re-evaluate the value of popularity's limited usefulness on the web.<br /><br />I think at its most basic, the <a href="https://en.wikipedia.org/wiki/Thumbs_signal">thumbs signal</a> expresses one person's opinion of a human artifact (person, place, thing, or utterance, say) where likability, approval, and desire are common interpretations. As a web tool, <a href="http://www.designingsocialinterfaces.com/patterns.wiki/index.php?title=Thumbs_Up/Down_Style_Ratings">Thumbs Up/Down Style Ratings</a> says to use it when "A user wants to express a like/dislike (love/hate) type opinion about an object (person, place or thing) they are consuming / reading / experiencing," with the value being:</p>
<blockquote>
<p>these ratings, when assessed in aggregate, can quickly give a sense for the community's opinion of a rated object. They may also be helpful for drawing quick qualitative comparisons between like items (this is better than that) but this is of secondary importance with this ratings-type.</p>
</blockquote>
<p>From the individual's perspective, I can see a few uses in online discussions:</p>
<ul>
<li>the satisfaction of expressing one's opinion,</li>
<li>the ability to get attention from others,</li>
<li>being able to see how others voted, which might lead to</li>
<li>challenging, or more likely reinforcing, ones opinions and beliefs, and</li>
<li>finding others possibly in your <a href="http://www2.sunysuffolk.edu/westn/humangroups.html">tribe</a> (and perhaps more importantly, those <em>not</em> in it)</li>
</ul>
<p>I suppose that from the site's perspective, the feature's value is engagement; people are excited by the above uses.<br /><br />Regarding quality of discourse, what is the value of this feature? And how does it enhance or hinder critical thinking? To my mind, its contribution is all negative because opinion does not equate to fact. (However, I think I'm out of sync on this, given the current cultural and political climate where opinion is perhaps valued higher than fact.) Let's simplify the question and phrase it in terms of popularity. What is the value of popularity in online discourse? My thinking is, none. A solid argument is not based on personal feeling, it is based on evidence, support, quality of sources, etc. This is literally (well, should be) <a href="http://www.education.com/reference/article/critical-thinking/">grade school material</a>.<br /><br />It's a problem, and my question is, can we replace the thumbs signal interaction feature with a better one, something that ideally counteracts the <a href="https://en.wikipedia.org/wiki/Echo_chamber_%28media%29">echo chamber</a> effect? A nascent thought I had was to keep the voting feature but apply it to portions of explicitly-presented logical arguments instead of the individual unstructured utterances that currently make up comments. Briefly, this would require that comments link to an unbiased and well-structured argument (I list some sites below [3]), and it is there, rather than the comments themselves, that people would vote. However, the difference is that they'd be voting on specific elements of the argument (such as a source's trustworthiness) which we might be able to interpret as a self-disclosed bias, perhaps proudly proclaimed.</p>
<p>Without getting into more detail here (I just wanted to get the thumbs up/down limitations written up here), I wonder if sharing and comparing biases instead of opinions might lead to tools to help bring a tiny fraction of people a tiny bit closer to understanding each other. Instead of "Hey John, check out the asshole liberal comments on this article," would it be useful to hear "Hey John, here's my opinion on this article's argument"?<br /><br />Again, I realize I'm fighting human nature [2], but what if a computer program could analyze two opposing parties' beliefs and, for example, find an area of the argument they agree on? Or find a slightly diverse group of people whose beliefs are different but close (think of a belief search space) and somehow bring them together.<br /><br />Naive? Probably, and this post is rough, but I'd love to hear your thoughts on any of it.<br /><br /><br />[1] I should put "controversial" in quotes for those topics like climate change that are no longer controversial from the scientific - i.e., reality-based - perspective. And I am quite looking forward to watching the <a href="http://merchantsofdoubt.org/">Merchants of Doubt</a> documentary (<a href="http://www.rottentomatoes.com/m/merchants_of_doubt/">Rotten Tomatoes review</a> and <a href="http://www.amazon.com/gp/product/1608193942/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=1608193942&amp;linkCode=as2&amp;tag=masidbl-20&amp;linkId=6HTIWOWXJVB6HYIN">Amazon book</a> links), by the way. Seen it yet?<br /><br /><br />[2] From <a href="http://www.skepdic.com/essays/Haskins.html">A Practical Guide To Critical Thinking</a>: <strong>Hindrances Due To Basic Human Limitations</strong>:</p>
<ul>
<li>Confirmation Bias &amp; Selective Thinking</li>
<li>False Memories &amp; Confabulation</li>
<li>Ignorance</li>
<li>Perception Limitations</li>
<li>Personal Biases &amp; Prejudices</li>
<li>Physical &amp; Emotional Hindrances</li>
<li>Testimonial Evidence</li>
</ul>
<p>[3] Argument tools:</p>
<p><a href="http://debatewise.org/">Debatewise - where great minds differ</a></p>
<ul>
<li><a href="http://justify-app.appspot.com/">Justify</a></li>
<li><a href="http://www.argunet.org/editor/">Argunet Editor - Argunet</a></li>
<li><a href="http://www.debatepedia.org/en/index.php/Welcome_to_Debatepedia%21">Debatepedia, the Wikipedia of Debates</a></li>
<li><a href="http://debategraph.org/Stream.aspx?nid=61932&amp;vt=bubble&amp;dc=focus">DebateGraph</a></li>
<li><a href="http://www.procon.org/">ProCon.org - Pros and Cons of Controversial Issues</a></li>
<li><a href="https://www.truthmapping.com/">TruthMapping</a></li>
</ul>
<p>&nbsp;</p>]]></description><wfw:commentRss>http://www.matthewcornell.org/blog/rss-comments-entry-35261340.xml</wfw:commentRss></item></channel></rss>