http://greaterdebater.com/blog/gabe/<p>This is where I talk about GreaterDebater development, general programming or anything else that comes to mind. </p> <p>You can email me at gabe at this domain dot com. </p>en-usFri, 24 May 2013 04:37:13 -0000http://feedproxy.google.com/~r/ThatsDebatable/~3/1d_FChI7ZfA/by gabe at Mar 26, 2013 <br /> <div> <p>Today I want to talk about two pillars of the American cultural experience, cars and guns. If you think about it, they're really quite similar. Both can be very useful and fun if used properly. Both are very dangerous when used by people who are inexperienced, drunk, or mentally ill. Both are responsible for thousands of deaths in the United States every year. (30,470 firearm related deaths in 2010 <a href="http://en.wikipedia.org/wiki/Gun_violence_in_the_United_States">source</a>, 32,885 automobile deaths in 2010 <a href="http://en.wikipedia.org/wiki/List_of_motor_vehicle_deaths_in_U.S._by_year">source</a>) </p> <p>Yet, while cars are heavily regulated, any attempt to impose restrictions on who can buy guns, when and where they can buy them, and what sorts of guns they can buy are met with howls of outrage. Every state maintains a registry of what cars are on the road and who owns them. Calls for a similar registry of guns have been met with panicked cries that such a measure would be merely a precursor to wholesale confiscation of guns. If that were true, shouldn't all the cars have been confiscated by now?</p> <p>Of course the real issue at the center of the debate is that the right to bear arms is enshrined in the Bill of Rights and the right to drive cars is not. But that doesn't mean we should blindly accept what was written in a document hundreds of years ago. At the time the Bill of Rights was written, a gun was a much more important and necessary part of daily life than it is today. You needed a gun to hunt and provide food for your family. Many people lived in remote areas where they had to protect themselves against criminals or hostile natives. The country did not have the large, organized standing military that we have today, so being able to enlist ordinary citizens quickly to fight was important. Not one of these issues is still pertinent today.</p> <p>Today people are easily as dependent on cars as people in colonial times were on guns. They are necessary for much of the population to find and maintain employment so they can support themselves and their families. Yet, we are able to manage to procure and maintain this necessary item despite its government regulation. Opponents of gun control complain that a registry would place an undue burden on private gun sales. Meanwhile, thousands of cars are sold by private sellers every year despite having a similar burden.</p> <p>So, if you're against gun control, but you don't have any similar objections to the way cars are regulated, why not? Check out <a href="http://www.greaterdebater.com/80355/">the thread</a> over on GreaterDebater and leave your thoughts there. I think the two issues have a lot of similarities and the only thing holding us back is an emotional attachment to a centuries-old ideal that isn't nearly as relevant in the modern world.</p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/41/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/41/http://greaterdebater.com/blog/gabe/post/41/http://feedproxy.google.com/~r/ThatsDebatable/~3/ltAySruLDLs/by gabe at Sep 22, 2012 <br /> <div> <p>Just a short note to document some missing dependencies for anyone else trying to google the same problem I just had.</p> <p>This first time it failed it couldn't find a file called ft2build.h. This was fixed by installing libfreetype6 and libfreetype6-dev. </p> <p>After that it failed when it couldn't find png.h. This was fixed by installing libpng-dev</p> <p>In summary, before running pip install run:</p> <pre><code>sudo apt-get install libfreetype6 libfreetype6-dev libpng-dev </code></pre> <p>and it should go off without a hitch. I assume these would be needed installing outside a virtualenv as well.</p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/38/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/38/http://greaterdebater.com/blog/gabe/post/38/http://feedproxy.google.com/~r/ThatsDebatable/~3/tjN7cL_tuyc/by gabe at Jul 02, 2012 <br /> <div> <p>Every day, on Twitter and Facebook and comment threads all across the Internet, thousands, if not millions of statistics are being cruelly mistreated. These helpless statistics are ripped away from their proper context and used to support or attack viewpoints they may have nothing to do with. Others are dropped into the middle of a hostile argument and then heartlessly neglected by their caretakers.</p> <p>"What can I do? I am just an extremely limited sample size of one man or woman." you may be saying. Well, have no fear. By working together we can have a significant (p &lt; 0.01) impact on the plight of even the most abused statistics.</p> <p>Many cases of abuse are caused by simple ignorance. Someone will see an exotic new statistic on TV or in the newspaper and he will bring it home and start putting it to work in his own arguments without knowing how to correctly interpret it. A misinterpreted statistic can then be made to perform all manner of vile deeds, even turning against its own underlying fact. These abusers can often be stopped by calmly offering to educate the offender about proper statistical care. (NOTE: If you encounter a professional statistics abuser, such as a politician or journalist, do not attempt to intercede on your own. Contact an SPCS professional.)</p> <p>Remember: these statistics cannot speak for themselves. It is up to us to give them a voice. Don't let the next abused statistic you see become just another statistic.</p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/35/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/35/http://greaterdebater.com/blog/gabe/post/35/http://feedproxy.google.com/~r/ThatsDebatable/~3/tm_Y_onuGJg/by gabe at Dec 16, 2011 <br /> <div> <p>I've been following the news about SOPA, the Stop Online Piracy Act, for the past couple of weeks. Yesterday there was an interesting development when 83 of the most prominent engineers responsible for creating the Internet signed an <a href="https://www.eff.org/deeplinks/2011/12/internet-inventors-warn-against-sopa-and-pipa">open letter</a> voicing their opposition to SOPA. This in and of itself is hardly surprising. Since the law was introduced anyone with a shred of technical acumen has stated that a.) It will not work. The law will not hinder piracy. b.) It <em>will</em> be hugely detrimental to the normal operation of the Internet.</p> <p>But, this made me think. If a law like SOPA were to be passed, how would it be implemented? How would it be enforced? I think it's safe to say that at some point somebody is going to have to write some code or possibly build some hardware. Maybe some of these technologies can be bought off the shelf from, say, China or North Korea, but at the very least someone is going to have to administer the servers that make this all work. Who's going to do that? The politicians? The lawyers? Entertainment industry executives? No. The task is going to fall to the very people who have been the most vocal opponents of the law from the start. What if they refuse?</p> <p>If SOPA were to pass and your job would require you to enforce its provisions, you should quit. If you currently work in IT or software development for a company advocating for the law, for a lobbying firm that is promoting the law, for the campaign of one of the representatives sponsoring or supporting it, you should quit. The organization paying you is actively trying to use your skills make people less free. There's a perpetual shortage of talent in the industry, right? Surely, you can find another job that does not require you to be an instrument of government oppression, that does not ask you to dismantle the infrastructure you've spent your career building and maintaining. I know it may seem like a drastic measure, but freedom, as we are so often reminded, is not free. If a free Internet is important to you, you have to be willing to make sacrifices to defend it, or it will cease to exist. Be happy that you can fight for freedom on economic terms instead of having to put your life on the line.</p> <p>If your current position would not be involved in complying with SOPA, but you're in charge of hiring people, you could let it be known that any experience that included building technology for the enforcement of SOPA would immediately disqualify an applicant from getting a job at your company. (Assuming, of course that they participated willingly, not the folks I just told to quit their jobs in the previous paragraph.) I don't think this would be unreasonable or unfair. Deliberately building something that nearly every expert in the field has condemned as a detriment to the Internet represents such a staggering lack of professional judgement that it <em>should</em> disqualify you from ever working again in this profession. As engineers we spend most of our education and careers focusing on what we <em>can</em> build, and very little time thinking about what we <em>should</em> build. Unlike doctors or lawyers we (mostly*) do not have professional licenses or ethics boards to report to. This does not mean we cannot act unethically, or that we should not consider the social ramifications of the things we make. An engineer who would build the infrastructure to make SOPA a reality should be treated exactly like a doctor who would willingly commit malpractice. He should be blacklisted from the profession.</p> <p>I know this isn't a foolproof plan. If there's enough money on the table, someone will come out of the woodwork to take the job. If the task receives enough scorn from the rest of the industry, though, you can be sure that it won't be the best and the brightest working on this. Anything that results will be that much less effective for it. Remember politicians and lawyers can bloviate and scheme all they like, but ultimately it is engineers who have to bring their plans into existence. We are the gatekeepers between dreams and reality, and when it comes to the politicians and executives, they need us far more than we need them.</p> <p>* Of course, there are Professional Engineer (P.E.) licenses. But for the majority of Internet related work I believe they are not required. For what it's worth, I do happen to have a PE and work in a field where it's required. It is expected and understood that you would refuse to design something for a client that would be harmful or unsafe for the people using it. Indeed, you would lose your license (and thus your livelihood) if you did so.</p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/27/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/27/http://greaterdebater.com/blog/gabe/post/27/http://feedproxy.google.com/~r/ThatsDebatable/~3/ZcXUgJcCE6k/by gabe at Jul 01, 2011 <br /> <div> <p>By hook and/or crook I managed to wangle an invitation to Google+ yesterday. After messing around with it a little, I really like what they've done and I think it has great potential. </p> <p>The circles are clearly the defining feature. The ability to compartmentalize friends into different groups and share things with only certain groups (or combinations thereof) is what everyone has been wanting from social networking sites for a while now. I think they might have solved the dilemma everyone has faced when their boss or parents want to friend them. To be fair, I don't think they're accomplishing anything you couldn't already do with lists and privacy settings on Facebook, but they've made it so easy and intuitive and central to the product that it stands a much greater chance of actually getting used. It's a lot like the difference in the security models between *nix systems and Windows that I mentioned in <a href="http://greaterdebater.com/blog/gabe/post/14">Facebook is the New Microsoft</a>. Facebook started with the assumption that you would want everything you shared to be visible to all of your friends, all the time. This made sense when the only people on it were college students. Now, privacy settings on Facebook are a confusing mess that was bolted on after the fact. Google+ has taken the opportunity to start fresh and build from the ground up with privacy in mind and the result is great. </p> <p>Circles are such a new concept though, that I think it could use a little more explanation in some parts. It's not always entirely clear how things are going to behave. For instance, when I share a post with a given circle, it sends an email to people in that circle who aren't on Google+ yet. Now, if I later go back and add more people to the circle, are they going to get an email about the thing I shared with that circle? Do I have to re-share to them in order for them to see it? Are people getting emailed when I add them to my circle if they're not on Google+? As you might be able to tell, I'm a little concerned about spamming my friends with Google+ nonsense. I don't <em>think</em> I am, but it would be nice if it were a little more clear on this point. </p> <p>Google+ grabs all of your contacts from gmail, giving you a pool from which to start populating your circles. This is helpful, but it's a little sloppy at the moment. Gmail has the annoying habit of saving practically everyone who's ever emailed you as a "contact" so there can be a lot of chaff to filter through here to find the people you want. It also only shows you their display name by default and there doesn't seem to be an easy way to see the email address underneath it. It would make sense to be able to manage these contacts inside Google+. It seems like that would be the sort of thing it should do, but there's no option yet to remove people or merge redundant entries together. I guess as they join Google+ their profile will override the information from your contact list. It would be nice if the people in your circles were mirrored into your contacts, essentially replacing your address book entries with their Google profiles wherever possible. Maybe that's on the horizon. </p> <p>I've only used the mobile app a little bit, but it's looking very well put together too. I'm looking forward to the feature that automatically uploads any photos you take (private by default!). Getting information off of a phone is always an annoyance and it will be nice to have that taken care of without even having to think about it. I have a feeling this is going to be murder on my phone's battery in areas without much signal, but I guess that's just the price you pay. It's easy enough to turn off if it becomes a problem. </p> <p>The mobile app introduces "Huddles" which seem to be a sort of ad hoc chat room you can set up with your friends. Curiously I haven't been able to find this feature anywhere on the website. I'm not sure if it's intended to be mobile-only or if they just haven't integrated it into the web yet. If enough people get on Google+ I think Huddles could be a viable replacement for text messaging without the character limits and outrageous rate plans. With an Android Huddle widget it would be just as easy to use and have the advantage of being able to seamlessly converse with any combination of people. I guess that remains to be seen. </p> <p>Overall, I'm optimistic about Google+. They've nailed down a great feature set. It's just a matter of gathering enough steam so they don't become one of those great products that fail because nobody uses them. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/25/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/25/http://greaterdebater.com/blog/gabe/post/25/http://feedproxy.google.com/~r/ThatsDebatable/~3/89i3_th6kW0/by gabe at Jun 30, 2011 <br /> <div> <p>I heard on the radio the other day that Top 40 songs have gotten more self-centered recently. Supposedly, songs in the past used to be about ideas and now they're just all about the person singing the song. The DJ picked a few examples, found that it was the case, and declared this to be true. </p> <p>Is it though? It seems like it wouldn't be that hard to test. I plan to do just that in the following manner. <p><br /> </p> <ol> <li> Get a list of every Billboard #1 single going back as far as possible </li> <li> Get lyrics for each song </li> <li> Count the occurrences of first person personal pronouns (I/me/my/myself/mine) in each song as a measure of self-centeredness. </li> <li> Compare the me-ratio(as I'm calling it) across the decades. </li> <li> ????? </li> <li> Profit?<br /> </li> </ol> </p> <p>That doesn't seem too hard. What I'm calling the me-ratio here is the number of occurrences of 1st person personal pronouns divided by the total number of words in the lyrics. We could question whether the me-ratio is the best metric, but it's at least easily measurable, so I figure it's as good a place as any to start. I don't see how you could sing primarily about yourself without saying words like "I" more than usual. I suppose you could refer to yourself in the third person, but we'll assume this is rare enough to not be much of a factor. I would also argue that any song that includes "I" a lot more than others is inherently pretty self centered. This premise leads to some interesting conclusions, as we'll see. </p> <p>I only looked at #1 songs to try to take popularity into account. If popularity were ignored, I might find that one era produced a whole lot more selfish songs than another, but if it turns out that nobody liked those songs, I couldn't really claim that pop music as a whole was really more selfish then than at another time. </p> <h2>Top Ten Most Self-Centered Songs</h2> <ol> <li><p><em>I Knew I Loved You</em> by Savage Garden<br /> Me-ratio: <strong>22.8%</strong><br /> Date: February 26, 2000 </p> </li> <li><p><em>Help Me, Rhonda</em> by The Beach Boys<br /> Me-ratio: <strong>21.5%</strong><br /> Date: May 29, 1965 </p> </li> <li><p><em>I Think I Love You</em> by The Partridge Family<br /> Me-ratio: <strong>21.1%</strong><br /> Date: November 21, 1970 </p> </li> <li><p><em>Always on Time</em> by Ja Rule featuring Ashanti<br /> Me-ratio: <strong>20.8%</strong><br /> Date: February 23, 2002 </p> </li> <li><p><em>Love Me Do</em> by The Beatles Me-ratio: <strong>20.6%</strong><br /> Date: May 30, 1964<br /> </p> </li> <li><p><em>Because You Loved Me</em> by Céline Dion<br /> Me-ratio: <strong>20.0%</strong><br /> Date: March 23, 1996 </p> </li> <li><p><em>You're Sixteen</em> by Ringo Starr<br /> Me-ratio: <strong>19.8%</strong><br /> Date: January 26, 1974<br /> </p> </li> <li><p><em>...Baby One More Time</em> by Britney Spears<br /> Me-ratio: <strong>19.6%</strong><br /> Date: January 30, 1999 </p> </li> <li><p><em>Take on Me</em> by a-ha<br /> Me-ratio: <strong>19.0%</strong><br /> Date: October 19, 1985<br /> </p> </li> <li><p><em>All My Life</em> by K-Ci and JoJo<br /> Me-ratio: <strong>19.0%</strong><br /> Date: April 04, 1998 </p> </li> </ol> <p>The top ten at least, don't show any clear prevalence of newer songs than older. Every decade is represented at least once. Indeed, if we look at the data for all of the number one songs since 1958 there doesn't seem to be any evidence of the me-ratio increasing over time. </p> <p><a href='/static/img/out.svg'><img src='/static/img/out.svg' alt="Me-ratio over time" width=700 height=370 title="click to embiggen"></a> </p> <p>What does seem to have happened over the past 10 years or so is that there aren't many songs that have a me-ratio of zero, meaning they don't mention the singer at all, but the peaks have also dropped off some. The lyrics refer to the singer more regularly, but the frequency does not peak as highly as it has in the past. </p> <p>One of the more interesting things I noticed from the top ten is that they mostly appear to be love songs. At first this would appear to be contradictory as a song about how the singer loves someone else shouldn't be considered self centered. But on closer examination I think this makes sense. For example, look at the number one song, <em>I knew I loved you</em> by Savage Garden. When you look at the <a href="http://lyricsfly.com/search/view.php?f9a319f8ed&amp;view=7160&amp;artist=Savage%20Garden&amp;title=I%20Knew%20I%20Loved%20You&amp;hl=%20&amp;opt=title">actual lyrics</a>, there's almost no information about the object of the singer's affection. The whole song is about <em>him</em> and how he feels about this person he loves, whom he apparently didn't even need to meet before falling completely in love with. This is actually pretty clever because it means the listener can more easily imagine the song is about the object of his or her own affection, but that's somewhat beside the point. </p> <p>In contrast, one song that I was sure would be at the top was Mims' 2007 hit "This is Why I'm Hot". I mean, what song could be more self centered than that, right? But the me-ratio came in at a relatively tame 13.2%. When you look at <a href="http://lyricsfly.com/search/view.php?0ee40be9e0&amp;view=602237&amp;artist=MIMS&amp;title=This%20Is%20Why%20I'm%20Hot&amp;hl=%20&amp;opt=main">the lyrics</a> Mims covers a wide range of topics, from cars, to cities to women. Granted, they all relate back to being reasons for why he's hot, but they aren't actually about himself, per se. He also spends part of every chorus explaining not only that he's hot, but that <em>you</em>, in fact, are not. So, the song is about the listener, almost as much as the singer, in some respect. </p> <p>In short, is there a trend in music becoming more self-centered? No, I don't think so. Additionally, the songs that <em>are</em> self-centered might not be the ones you think. </p> <p>Code used in gathering this data and performing the analysis can be found in <a href="https://bitbucket.org/ggetzie/selfish-music/overview">my bitbucket account</a> </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/23/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/23/http://greaterdebater.com/blog/gabe/post/23/http://feedproxy.google.com/~r/ThatsDebatable/~3/IIp7SaBUOSQ/by gabe at May 06, 2011 <br /> <div> <p>I never thought this day would come. For years now, I've chuckled at the naysayers who lamented that "Reddit has gone downhill". People have been saying that for about as long as I can remember, and I've been there for over four years. Sooner or later, though, I guess we all become what we hate, because here I am.</p> <p>Reddit has gone downhill. </p> <p>Maybe that's not even quite the right way to put it. It's more like Reddit has hit a plateau. From my perspective, it's tough to tell if it's actually changed or if it's always been like this and I'm tired of it now. Maybe it's because I've been there so long, but there's just nothing new anymore. There's only a few basic kinds of posts. I'm talking about /r/AskReddit specifically, which has been my favorite subreddit by far since its inception. I've long since unsubscribed from /r/reddit.com and most of the larger subreddits. With a few exceptions, most of the other subreddits are only occasionally interesting. I've always been a proponent of the "If you don't like it, downvote and/or hide it" philosophy, but with 90% of the posts falling into one of a few categories that create the same predictable responses every time, I'm left with no option but to remove myself entirely. One man downvoting can only do so much, and it's become clear that my own tastes are no longer in sync with the AskReddit community. To save you some time, here's a summary of what you'll find on AskReddit, every day, for the foreseeable future.</p> <ol> <li> <p>The question that's not really a question. This really belongs in the rants and raves section of Craigslist, or maybe your livejournal. As an example <a href="http://www.reddit.com/r/AskReddit/comments/gu1nu/why_does_my_cat_insist_on_throwing_up_on_the/">Why does my cat insist on throwing up on the carpet with a perfectly good wooden floor available?</a>. I don't mean to pick on you, guy who submitted this, but you know perfectly well why your cat throws up on the carpet. It's because it's a cat and it throws up wherever it happens to be when it feels like throwing up. This is not a thought-provoking or inspired question, it's some dude bitching about his pet. Again, I don't mean to single this post out. If you look at the front page, about 25% of the submissions are like this. I include in this all variants of "Does Anyone Else". These are not questions. They are not discussions. They are pleas for affirmation and attention and they are tedious. I don't want to ramble on about this too much, as it could easily be a post unto itself.</p> </li> <li> <p>The "What's your favorite X" question and its partner "What's your least favorite X", one of these usually following the other. Here X is most often some form of media such as movies, TV shows, books, video games, music, or occasionally quotes. Sometimes they are phrased creatively, like "Which 3 books would you take to a desert island?" or "If you could only pick one song to represent music, what would it be?". It doesn't matter. It's the same list every time, practically one of them every day. It's this more than anything that leads me to suspect that Reddit's voting system is fundamentally broken. Maybe it's just that there's so many people now and new ones are arriving at such a rate, that every time there's a new incarnation of one of these there's a whole new voting block that hasn't seen all the previous versions ready to vote it to the front page. Then again, maybe it's the same block of people who want to make really, really, really sure that we all know we should watch Firefly and Arrested Development and read Catch-22.</p> </li> <li> <p>The advice seeker. There's a few flavors of this post and few, if any, of them are worth reading. A short summary includes:<br /> a. The person who clearly needs professional medical/legal/financial advice, but has decided to ask Reddit instead.<br /> b. The person who has already decided what to do, but wants several hundred people to parrot the answer back to him/her.<br /> c. The person who wants someone to type out the first result in Google for them.<br /> d. The person who has decided to entrust a major life decision to a bunch of anonymous strangers. (often b in disguise)<br /> </p> <p>This is a pretty broad category, so the questions vary widely in the amount interesting discussion the generate. As with any discussion, even a relatively boring question sometimes derails into an interesting thread. As a general rule, the more unusual the situation, the more interesting the discussion tends to be. More often than not, however, these tend to have one obvious answer that quickly gets voted to the stop, and several hundred other comments that rephrase it.</p> </li> <li> <p>The hypothetical. This is another broad category. In fact, with the infinite possibilities hypothetical questions offer, it <em>should</em> be the most interesting. But usually the same few questions are endlessly repeated like "If you had three wishes what would they be?", "What would you do with (some large amount of money)?", "What super-power would you want?". Other times these can be just thinly veiled attempts to get the hivemind to reiterate its political opinions: "If you could tell every person in the world one thing, what would it be?", "Conservatives: Why are you like that???".</p> <p>With a little imagination, these can sometimes be great, such as <a href="http://www.reddit.com/r/AskReddit/comments/ajoal/would_it_be_ok_for_a_centaur_to_poop_anywhere/">my favorite AskReddit question of all time</a>, but the common ones tend to have predictable answers, and the political ones quickly devolve into the usual echo chamber.</p> </li> <li> <p>Story time. These at least generate new content each time, even if they are the same topics like "What are your worst/best work stories?" (sometimes one profession specifically); worst/best relationship stories; grossest, awesomest thing that's ever happened to you; biggest coincidence you've ever experienced, and so on. This is one place where the largeness of the community works in its favor. It's pretty unlikely you'll see the same stories retold when the threads resurface. Even though these aren't very thought provoking, or inspired, this is where AskReddit provides most of its entertainment (for me, anyway).</p> </li> </ol> <p>Obviously, this isn't a comprehensive list. It isn't meant to be. Feel free to enumerate other types you've observed in the comments below. There are always outliers that don't fall into any category and these are often great, fitting with the spirit of "thought-provoking, interesting questions" that AskReddit is supposed to be all about. It just seems that lately these have been fewer and farther between, and the Internet is a poorer place for it.</p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/24/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/24/http://greaterdebater.com/blog/gabe/post/24/http://feedproxy.google.com/~r/ThatsDebatable/~3/oEieCZkT5uo/by gabe at Mar 29, 2011 <br /> <div> <p><a href="http://crooksandliars.com/karoli/donald-trump-lets-tax-rich-pay-national-deb">Crooks and Liars</a> is parading around a <a href="http://articles.cnn.com/1999-11-09/politics/trump.rich_1_donald-trump-trump-said-trump-trump-said-the?_s=PM:ALLPOLITICS">twelve year old article</a> wherein Trump proposed a one time 14.25% tax on the top 1% of Americans (then defined as households with a net worth of greater than $10 million) to pay off the national debt then $5.7 trillion.</p> <p>The rest of Trump's politics aside, this doesn't sound like such a horrible idea, if the math works out. In 1999, if a 14.25% tax on the net worth of the top one percent would have paid off the $5.7 trillion national debt at the time, that assumes that the top one percent had a combined net worth of 5.7 trillion / 0.1425 or $40 trillion. For the same 14.25% tax to pay off the national debt today, the top one percent would have to have a combined net worth of 14.2 trillion / 0.1425 or almost $99.7 trillion.</p> <p>According to the <a href="http://www.federalreserve.gov/releases/z1/current/z1r-5.pdf">Federal Reserve</a> the total net worth of all households and non-profit organizations in the US is $56.8 trillion as of Q4 in 2010. Right away, there's a problem. Clearly a 14.25% tax on the net worth <strong>everyone</strong> would not be enough to pay off the national debt.</p> <p>Now the paper that has caused all the latest hand-wringing about wealth inequality in America is <a href="http://www.levyinstitute.org/pubs/wp_589.pdf">this one</a> by economist Edward Wolff. According to this paper, the top 1% control 34.6% of the net worth (including home values, which the Federal Reserve seems to include). So, that puts them with about $19.7 trillion. A tax on the top 1% that paid off the entire national debt would have to be a 72 percent tax of their net worth. That seems a little steep, even for a one time payment. Would the benefit they get from having the country out of debt be worth it? Probably not. If we all chip in, a one time tax of 25% of everyone's net worth would be enough to pay off the debt. I guess if you have a negative net worth, you wouldn't have to pay though. Imagine the rush to take out loans if <em>that</em> were announced.</p> <p>There's also practical problems that would arise from a tax on net-worth. As mentioned, this includes home values, so a big chunk of most people's net worth is not very liquid. Getting the money to actually pay the tax would likely be a big problem.</p> <p>Crooks and Liars is trying to paint this as an issue that Trump has flip-flopped on. Looking at the numbers, he'd have to be an idiot <em>not</em> to have changed his mind about it. Revising one's stance in the face of new evidence is a quality we should encourage in our leaders. I don't really have an opinion about Trump as a presidential candidate either way, but it would be nice if the pundits providing political commentary could spend some time doing a little fact checking and presenting a rational argument, rather than vapid "he said that, but now he says this!" commentary that is ultimately of no value.</p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/22/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/22/http://greaterdebater.com/blog/gabe/post/22/http://feedproxy.google.com/~r/ThatsDebatable/~3/sU8Qd0060DE/by gabe at Jan 24, 2011 <br /> <div> <p>For a while now I've been interested in data mining and visualization. When someone posted a link to a <a href="http://www.reddit.com/r/MachineLearning/comments/e81go/ufo_dataset/">dataset of UFO sighting information</a> on Reddit, it seemed as good an opportunity as any to jump in and get my feet wet. There's a lot that could be done with this data, but to start I decided to make a heat map indicating the number of sightings by county, throughout the US. </p> <p>With the help of <a href="http://flowingdata.com/2009/11/12/how-to-make-a-us-county-thematic-map-using-free-tools/">this excellent guide</a> over at Flowing Data, this didn't seem too hard at all. The only hiccup was that I needed to determine the county and state from the location field in the data. For the most part, I was able to stitch together data from several other datasets to resolve the city and state format of the location field into the county and state information needed to locate the sighting on the map. Unfortunately, this narrowed down the data from around 63000 sightings to around 43000. For the records where no county and state could be identified, most were either outside the US, or the city name was not connected to a county name in any of the datasets I dug up. Obviously, having a more comprehensive listing of all city names in the US along with the corresponding county in which they reside would be helpful, but working with free data, you take what you can get. </p> <p>Another confounding factor was that the location field was somewhat noisy and non-uniform. The location might be something like "New York City, NY" (which would be ideal as this could be split on the comma into city and state) or it might say something like "Las Vegas (side of the highway north of town), NV" which fails when simply split on the comma, so even though that particular city and state would be in my data, it wouldn't be identified. I was able to improve the rate at which counties were correctly identified a little by removing all parenthetical remarks, and special-casing any location that already had the word "county" in it. </p> <p>But anyway, without any further ado. Let's have a look at the map. Click on it for a larger version. </p> <p><a href='/static/img/ufomap.png'><img src='/static/img/ufomap.png' alt="UFO Heat map" width=555 height=370 title="click to embiggen"></a> </p> <p>I think one of the more interesting things to note is that the sighting density seems to roughly correlate with population. The stereotypical idea of a UFO sighting, at least in my mind, is of someone all alone on a dark road out in the middle of nowhere, and is also associated with more rural populations. However, it seems like the more people there are are in a given area, the more likely it is, rougly, that some of them have seen, or thought they saw, a UFO. </p> <p>I think there's a good deal of information to be mined. I'd also like to get around to seeing if there's any particular day of the week or time of year when sightings are more prevalent. Maybe Friday and Saturday nights when people are more likely to have been drinking? It would also be interesting to not if there's any similarities in description based on sightings that take place around the same time, or in the same place. </p> <p>If you're interested, you can find all the datasets I used to patch together the data for this analysis at the following locations: </p> <p><a href="http://infochimps.com/datasets/d60000-documented-ufo-sightings-with-text-descriptions-and-metad">UFO Dataset</a><br /> <a href="http://infochimps.com/datasets/united-states-zip-code-database-postal-code-latitudelongitude-ci">US towns and zip codes</a><br /> <a href="http://www.census.gov/geo/www/gazetteer/places2k.html">US county and FIPS code data</a><br /> <a href="http://commons.wikimedia.org/wiki/File:USA_Counties_with_FIPS_and_names.svg">Blank USA County map</a><br /> <a href="http://www.itl.nist.gov/fipspubs/co-codes/states.txt">All county FIPs codes</a><br /> </p> <p>If you'd like to see the code I used to create the map, it's available on my <a href="https://bitbucket.org/ggetzie/ufo-data-code">bitbucket account</a>. The numerical FIPs codes ended up being unnecessary, so it's a little more convoluted than I might like, but it works well enough I suppose. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/19/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/19/http://greaterdebater.com/blog/gabe/post/19/http://feedproxy.google.com/~r/ThatsDebatable/~3/sOh8rb5svjI/by gabe at May 20, 2010 <br /> <div> <p>Is it me, or are the similarities between Facebook and Microsoft becoming downright eerie? I know Microsoft is a major investor in Facebook, so it's probably not all coincidental. But sometimes it seems like Facebook is running the MS playbook step by step, including the mistakes.</p> <h3>Origins</h3> <p>Gates and Zuckerberg both dropped out of Harvard to start their companies. Superficial and coincidental, sure, but a little weird in the Kennedy-had-a-secretary-named-Lincoln-and-vice-versa way. Additionally, both founders have been depicted in the press as being cold and calculating, with concern for little else besides the success of their company. Gates' image has softened quite a bit since he's retired to pursue philanthropy, but Mark Zuckerberg has a lot in common with the Bill Gates of the 80s and 90s, for better or worse.</p> <p>Aside from the founders, the companies started off in similar ways. Microsoft's breakthrough product, DOS, was not developed in-house but purchased from another software firm. Zuckerberg is rumored to have "stolen" the Facebook idea from a previous startup he worked with. Whether there's any substance to such claims, it's safe to say the the origins of both companies include some element of controversy. There's a deviation from the archetypal Silicon Valley success story where a group of genius programmers work maniacally to build a better mousetrap. I don't mean to imply in any way that the founding teams of either company aren't very talented or that they don't work very hard, but both have been haunted with a historical footnote that there was an extra circumstance that helped catapult them into greatness.</p> <h3>Facebook and Microsoft have both been highly dependent on network and lock-in effects</h3> <p>Microsoft and Facebook are hardly the only two businesses to exploit these effects, but they are probably the two who met with the most resounding success in doing so. In the early days of Microsoft, being "IBM-compatible" was a major selling point for computers. Part of what this meant was that the computer ran DOS, the operating system IBM licensed from Microsoft. As the number of people with IBM-compatible computers grew, consumers and businesses became increasingly dependent on being able to share data among a common set of programs. The effect continued as Microsoft Office became popular. It was difficult, if not impossible, to send someone a file created in a different office suite and expect them to be able to open it. Eventually, anyone who wanted to have any hope of communicating had to start using Office. Facebook is in a similar position as even the long-time holdouts are starting to cave simply because they <em>need</em> Facebook to get in touch with their friends. Facebook had also until recently taken a walled-garden approach to curating its community. The information on the site was largely closed to non-members. It's illuminating to consider that Microsoft Office has also moved to a more open file format now that the competition is safely crushed. Is Facebook's recent move to grow over its walls and extend its tendrils out onto the open web an assertion of their dominance in the social networking space? On the other hand, many of the walls remain intact. You can't send a message to a friend on Facebook unless you also have a Facebook account. You also can't follow your friends' status updates or receive invitations to events planned on Facebook without an account of your own. <a href="http://radar.oreilly.com/2009/03/facebook-in-2010-no-longer-a-walled-garden.html">Predictions</a> of Facebook's imminent openness may be premature.</p> <h3>Facebook and Microsoft both push their business models to the limits of legality</h3> <p>Microsoft famously leveraged its success in the operating system market in an attempt to seize control of the booming new market for web browsers thus controlling how people interacted with the Internet in the same way they controlled how people interacted with the computer. The result was a <a href="http://en.wikipedia.org/wiki/United_States_v._Microsoft">landmark antitrust case</a> that cost Microsoft dearly in terms of consumer goodwill. Facebook now finds itself in a similarly powerful position. They are the default repository for personal information. They are also becoming the default way for people to interact with each other via the Internet, much like Windows became the default way the people interacted with a computer. The question on everyone's mind is: What is Facebook going to do with all this power? They have a truly staggering amount of data about consumer preferences all willingly submitted by the consumers themselves. Facebook clearly wants to capitalize on this data-mining bonanza in the most profitable way possible, but they are already drawing the ire of lawmakers <a href="http://bits.blogs.nytimes.com/2010/04/27/senators-ask-facebook-for-privacy-fixes/">in the US</a> and <a href="http://www.theregister.co.uk/2010/05/14/facebook_privacy_rebuke/">in Europe</a>. Unlike Microsoft, who already had a wildly successful business selling operating systems and office suites before they overplayed their hand with Internet Explorer, Facebook <em>needs</em> to capitalize on every advantage at its disposal in order to satisfy the outsized expectations (and venture capital funding) that have been placed on them.</p> <h3>Privacy : Facebook :: Security : Microsoft</h3> <p>Microsoft's dominance of the software market made them a high profile target for writers of viruses and other malware. The sheer size and homogeneity of the Microsoft ecosystem created the perfect environment for viruses to flourish. Unfortunately, the core of the Windows operation system was not designed to operate securely in the highly networked world that it eventually inhabited. Windows always worked best under the assumption that the user was the sole user of the computer and should have maximum privileges to make any changes. This made it easy for malicious software to install and replicate itself. The neverending torrent of spam sent out by botnets of infected Windows computers is the legacy of this broken security model. Only recently, with Windows Vista and 7 has Microsoft started to make some headway in producing a more secure operating system.</p> <p>In much the same way, the original Facebook was not built with privacy in mind. Originally, users could only join if they could provide a college email address. With private networks there was not as much of a need to control what information was displayed. As the site has grown, the developers tacked on a bewildering array of privacy controls. The complexity and constant revisions of the privacy policy have left many users unsure of what information they're sharing and with whom. As Facebook approaches 500 million users it presents a target similarly large and uniform as the Windows install base. Aside from the exact same problem of being an <a href="http://blogs.msdn.com/tzink/archive/2010/01/29/new-facebook-worm.aspx">attack vector</a> for the spread of malicious software, the Facebook network provides a new set of problems with the spread of more human-readable information. <a href="http://valleywag.gawker.com/tech/your-privacy-is-an-illusion/bank-intern-busted-by-facebook-321802.php">Some users</a> are learning the hard way that you might not want <em>all</em> of your Facebook friends to see everything you post on Facebook. The nature of the site -- you mostly interact with people you know (or used to know) in real life -- causes users to let down their guard more than they might on the open Internet. As a result cons like the <a href="http://www.allfacebook.com/2010/01/facebook-scams/">Western Union scam</a> make hijacked Facebook account a particularly hot commodity for thieves. In time, users will come to realize that they can't trust Facebook any more than any other communication medium, just like we've all had to train ourselves not to open email attachments from untrusted sources.</p> <h3>What's next?</h3> <p>I don't claim to have a particularly accurate crystal ball, and in truth, I've probably already pushed this analogy as far as it can go. If it were possible to directly compare the trajectories of these two companies, I'd say Facebook today is probably equivalent to early 90s Microsoft. They're just coming into their own as the market leader and they're looking for the best ways to throw their new weight around. Unfortunately, Facebook has a much more fickle and mobile user base than Microsoft ever did. Microsoft's customers avoided competitors due to the need for compatibility. The high cost of a second computer to try out a new operating system, or the technical expertise required to run two operating systems on the same computer, made it impractical for most to experiment with alternatives. Such is not the case with social networks. It's trivial for anybody to maintain two or three different accounts. Users migrate away gradually by starting accounts on new services and gradually spending more time there as more of their friends join. Eventually, they close the old account or just abandon it. (I don't think I ever officially closed my Friendster account, did you?) So far, user outrage has managed to stymie most of Facebook's attempts to profit off of their vast store of data. This hasn't seemed to slow their growth any, though. The majority of the users don't seem to care about the privacy issues, and if they do experience some negative consequences it's likely they'll adjust their use of Facebook rather than abandoning it altogether. In the meantime, Facebook seems to be homing in on an unobtrusive way to realize the immense profits their investors are expecting. Facebook has already passed the point where they can spectacularly flame out. They've got enough momentum that failure is more likely to be a painful, drawn out process. With a vocal contingent of savvy users, Facebook can't afford to ignore the privacy issues like Microsoft ignore security for so long. Facebook will find success by giving users the most control over their own data and demonstrating that the advertising based on that data adds value for the users.</p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/14/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/14/http://greaterdebater.com/blog/gabe/post/14/http://feedproxy.google.com/~r/ThatsDebatable/~3/zW4aaapNrbE/by gabe at Apr 05, 2010 <br /> <div> <p>One thing I really slacked on while developing GreaterDebater was testing. Maybe that's not the right way to put it. It's not like I <em>didn't</em> test. In fact, I did a lot of testing. A lot of painstaking manual testing. For every change. So, in a way it was really the opposite of slacking. It was working a whole lot harder than I really needed to. It was also dumb. What I did slack on, was figuring out how Django's unit testing framework worked. Well, no more! In fact, it's really easy to set up. Writing tests can also be somewhat painstaking when it comes to ensuring you've covered every possible input, but at least that pain is an investment that repays itself every time you make a change to the program and you can know right away if you've broken something that used to work.<br /> </p> <p>As usual the <a href="http://docs.djangoproject.com/en/dev/topics/testing/#topics-testing">Django documentation</a> is excellent. I highly recommend it for learning everything there is to know about testing with Django. What I hope to do here is give a brief overview of what you need to do some basic testing. Partly as a reminder for myself on future projects, and hopefully to help out anyone in the same position I was before I started. </p> <h2>Getting Started</h2> <p>If your Django app is relatively new, the startapp command already created a tests.py file when you ran it to set up your app. This file includes some helpful examples, like so: </p> <pre><code>""" This file demonstrates two different styles of tests (one doctest and one unittest). These will both pass when you run "manage.py test". Replace these with more appropriate tests for your application. """ from django.test import TestCase class SimpleTest(TestCase): def test_basic_addition(self): """ Tests that 1 + 1 always equals 2. """ self.failUnlessEqual(1 + 1, 2) __test__ = {"doctest": """ Another way to test that 1 + 1 is equal to 2. &gt;&gt;&gt; 1 + 1 == 2 True """} </code></pre><p>If you don't already have a tests.py file, the important things to note are that you have to import the <strong>TestCase</strong> class from django.test and create a new class which subclasses it. Each method defined in this class will correspond to one test, usually the test of a corresponding function in your <strong>views.py</strong> file. To test a particular function from <strong>views.py</strong>, define a method with the name <strong>test_functionname</strong>. </p> <h2>Test data</h2> <p>Now, I'm guessing your application probably manipulates some sort of data. The Django test runner is kind enough to setup and destroy a complete test database for the purposes of running your tests. There's two important consequences to this. First, the database user specified in your settings.py file has to have the necessary permissions to create and destroy databases. So, make sure that's the case. Second, if you already have a test database set up on your development machine (or wherever), with data you've inserted during your manual tests, the test runner won't have access to any of it. </p> <p>To get some data for your tests, you can try exporting the data already in your test database with Django's dumpdata command. The test runner can then load it as a <a href="http://docs.djangoproject.com/en/dev/topics/testing/#fixture-loading">fixture</a>. Honestly, I just couldn't get this to work. Some googling about the errors I was getting seemed to indicate that I wasn't the only one having problems with it. The update to <a href="http://djangoadvent.com/1.2/natural-keys/">natural keys</a> announced for Django 1.2 seemed like it would help, but I've still been unsuccessful, so far. In any event, I plan on keeping an eye on future developments to see if there's any improvements in this area. </p> <p>In the meantime, I went ahead and did it the hard way. I created a separate file called <strong>testsetup.py</strong>. I imported all the models for my project and defined a function also called <strong>testsetup</strong>. All this function does is create a few instances for each model with all their various possible configurations. By defining a method called <strong>setUp</strong> in the new class I created in <strong>tests.py</strong> and calling this function there, all of the necessary data will be added to the test database before the tests are run. </p> <pre><code>class ViewTest(TestCase): def setUp(self): testsetup() </code></pre> <h2>GET Requests</h2> <p>The easiest of your views to test are the ones that only handle GET requests. All you have to do is retrieve the correct item from the database and display the information. Let's use the canonical book/author/publisher example. Suppose we're testing a view that displays the details for a particular book. </p> <pre><code>def test_book_detail(self): testbook = Book.objects.all()[0] url = '/books/id/' + str(testbook.id) response = self.client.get(url) self.assertContains(response, testbook.title) </code></pre><p>The first thing I want to stress, is that I think it's important that no specific test data is used in the tests. Always try to select objects you're going to use for your tests in a generic way. This way it won't matter how your test data gets moved around or changed over time. In this example I just picked the first book returned in the queryset for all books. What you <em>don't</em> want to do is something like this: </p> <pre><code>testbook = Book.objects.get(title="The Test Book") </code></pre><p>That assumes your test database has a book with that title in it, which it may or may not. Over time it's bound to get mighty confusing trying to keep the data in your test database synchronized with the data in your tests, so it's best to save yourself the headache. </p> <p>The Django <strong>TestCase</strong> class comes with a built in client to make all your requests. For a GET request all you have to do is pass it a URL and it will return the response generated by that request. The <strong>TestCase</strong> class also defines some new <a href="http://docs.djangoproject.com/en/1.1/topics/testing/#assertions">assertions</a> in addition to the standard ones Python provides. <strong>assertContains</strong>, used in this example, takes a response object and verifies that it has a particular piece of text in it; in this case, the book's title. Optionally, you can specify a particular response code, and number of repititions for the piece of text. If you're expecting a particular url to not be found you might test: </p> <pre><code>badurl = '/books/id/notabookid' response = self.client.get(badurl) self.assertContains(response, "Book not found", status_code=404) </code></pre> <h2>POST requests</h2> <p>Of course all of this data has to get into your database somehow. You know you don't want to put it all there yourself, so you're probably going to have your users do it. This likely means you'll have some forms on your site that you want them to submit. For our example, let's pretend authors are submitting their own books to the site. Let's assume a few things about how this function works, so we know what we're testing. </p> <ul> <li> The user must be logged in to submit a book </li> <li> A book requires a title, an author, and an ISBN field </li> <li> Upon successfully submitting a book, we want to redirect the user to the detail page for that book. </li> <li> The ID field for the <strong>Book</strong> model is auto-incremented (Django's default) </li> </ul> <p>Simple enough, right? Django provides all the tools you need to test every aspect of this function. </p> <pre><code>def test_addbook(self): url = '/books/add/' author = Author.objects.all()[0] user = author.user # User not logged in response = self.client.get(url) self.assertEqual(response.status_code, 403) self.client.login(username=user.username, password='password') # Valid user response = self.client.get(url) self.assertEqual(response.status_code, 200) # Invalid Form response = self.client.post(url, {'title': "Book Title", 'author_id': author.id, 'ISBN': "invalid ISBN"}) self.assertFormError(response, "BookSubmitForm", 'ISBN', "ISBN field must contain a number") # Valid submission newid = Book.objects.aggregate(Max('id'))['id__max'] + 1 redirect = '/books/id/%d' % newid response = self.client.post(url, {'title': "Test book", 'author_id': author.id, 'ISBN': 123456}, follow=True) self.assertRedirects(response, redirect) self.assertContains(response, "Test book") </code></pre><p>First you'll want to test a GET request for the url, to make sure the user can see the form properly to fill it in. Also check that the form is not visible to users who aren't logged in. In this case, that means simply making sure the response returns a 403 Forbidden status code. To log in as a user, simply use the <strong>self.client.login</strong> function with the appropriate name and password. It's easy enough to pull a random user from the ones you set up in the <strong>testsetup</strong> function, but I've violated my own rule a little by specifying a value for the user's password. In fact, I'm not sure there's any other way to do this. For the sake of simplicity, I'm just giving all of the users created in <strong>testsetup</strong> the same password. For POST requests, simply add a dictionary with all the required form fields and values. You can make sure the form validation is catching errors properly by submitting an invalid form and using the <strong>assertFormError</strong> assertion to verify that the appropriate form validation error occurs. This assertion takes 4 arguments: the response, the name the template uses for the form, the form field in question, and the error text (or a list of strings for multiple errors) that is expected. Finally, since the id for new books is just autoincremented, we can assume the id for the book we create will be one more than highest one in the database yet. By plugging this into the book detail url, we have the address of the page we'll be redirected to when the book is successfully submitted. If you want to use <strong>assertContains</strong> or anything else to check the contents of the final page you get redirected to, be sure to add <strong>follow=True</strong> in the post function. This way it will return the final response, not the intermediate redirect response. </p> <p>I suppose now is a good time to mention that the Django test runner will return the database to its original state (after populated by the <strong>setUp</strong> method) after each test. That means that any books you add during the <strong>test__addbook</strong> method will be deleted when that test is over. You won't have access to them in, for example, <strong>test_deletebook</strong> that you might write afterwards, so keep that in mind. </p> <p>I hope this has been a helpful introduction to Django's testing facilities. I've only scratched the surface of what it can do, but I think it's a good place to start. Like I said, the <a href="http://docs.djangoproject.com/en/dev/topics/testing/#topics-testing">documentation</a> is up to Django's usual high standards, so definitely make the most of it. </p> <p>Happy Testing! </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/12/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/12/http://greaterdebater.com/blog/gabe/post/12/http://feedproxy.google.com/~r/ThatsDebatable/~3/Moyw8AJU3b4/by gabe at Mar 23, 2010 <br /> <div> <p>I like them quite a lot, thank you. In fact, they're pretty great! Ever since my esteemed colleague over at <a href="http://kil22.blogspot.com/2009/04/and-all-that-jazz-apples.html">Killa Blog</a> introduced me to Jazz apples almost a year ago, I've kept a close eye on the produce section at my local grocer. This past weekend a new challenger appeared. </p> <p><img src="/static/img/pinklady.jpg" alt="Pink Lady apple"/> </p> <p>It seems Australia, ever the bitter rival of New Zealand, is not content to let those lousy Kiwis dominate the world of boutique apple varietals. The <a href="http://en.wikipedia.org/wiki/Pink_lady_apple">Pink Lady</a> is a bit more tart than the Jazz apple, but very very tasty. She'll be appearing in my own lunch bag at least until plums and nectarines are in season. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/11/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/11/http://greaterdebater.com/blog/gabe/post/11/http://feedproxy.google.com/~r/ThatsDebatable/~3/6I_91p7unGU/by gabe at Mar 22, 2010 <br /> <div> <p>I knew this day would come. GreaterDebater experienced it's first wave of spam this past week. In a way, it's almost flattering that the site is considered worth spamming now. In another much more prevalent way it's incredibly annoying. I know I'm only about the billionth or so person to complain about spam, but I have to wonder if the fact that spam is a worthwhile activity for some people is not an indication that there's something seriously broken about the way the Internet works. </p> <p>The blessing and the curse of the Internet has been the ability to widely distribute information with very low overhead. This has simultaneously made more knowledge available to more people than at any point in history and also presented serious challenges for anyone who was making a living copying and distributing information that can now be delivered digitally. Spam and piracy exist at opposite ends of the digital distribution spectrum. In the case of piracy, you have information that many people want, but the producers of this information want to carefully control the supply to maximize the value. With spam, you have information that <em>nobody</em> wants, yet the producers want to disseminate it as widely as possible. I'm not sure that this points to a common solution, but I think it helps to know the two problems spring from similar sources. If there were a communication paradigm that balanced the needs of information consumers and producers this might help, but I don't think it's the only solution. </p> <p>The thing about the spam I've received so far is that it does not appear to be selling anything. For one thing, most of it hasn't been in English. Don't get me wrong, I would love for GreaterDebater to have a fully international audience, but that just isn't the case yet. Attempting to advertise your foreign language product via a site which is pretty obviously only English speaking doesn't make a lot of sense. Since this particular brand of spammers isn't trying to get people to part with their money directly, I can only assume they're trying to promote sites which will attempt to install malware. This is really only a superficial difference, though. In all probability the malware will be used to assimilate the victim's PC into a botnet and send out ... more spam. </p> <p>It's hard to imagine that spam filters could get much more accurate, or that the response rate for spam could get much lower. Increasing the cost of sending email would undoubtedly produce more harm than good. Few, if any, of the products advertised via spam are legitimate and there's no shortage of outright scams. If the only cost of doing business is sending the spam, it doesn't take much response to make it profitable. </p> <p>I don't know what the solution is. I hope one day we'll move past spam. Right now an enormous amount of bandwidth is wasted delivering messages that are unwanted or outright harmful. Not to mention that the spammers' lives are being wasted producing nothing of value. If we could harness these resources towards more productive ends, who knows what we might be able to achieve? </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/10/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/10/http://greaterdebater.com/blog/gabe/post/10/http://feedproxy.google.com/~r/ThatsDebatable/~3/CWW27jKOz48/by gabe at Mar 15, 2010 <br /> <div> <p>I think every programmer must have some bit of code they wrote that they absolutely hate. Maybe it's something that's brute force where it doesn't need to be. Maybe it's a block of code that's been copied and pasted a few more times than is advisable. Sure, it works. It's perfectly <em>functional</em>. But you know it's in your codebase, lurking. It's waiting for the perfect time to strike, when it can rear up and bite you in the ass at the worst possible moment. Of course, you didn't <em>want</em> to write it that way. You'd rather have spent a few days pondering over the most elegant way to express yourself. There wasn't time, though. Anyway, this is good enough. For now. You'll come back to it later. </p> <p>I think this would make a great interview question (though I've never heard of anyone asking it): "What's the worst piece of code you ever wrote?" First, you'd get to see their ability to spot shortcomings in their own work, which is always a valuable skill. Second, it will give you some insight into what aspects of good code they value most. Is the source of their discomfort something like poor modularization? Maybe they can't abide code that isn't properly indented. If something trivial is their biggest grief, perhaps they're not worth hiring. If they can't think of <em>any</em> of their own code that they hate, they're either a phenomenal programmer, or they just haven't given it much thought, probably the latter. </p> <p>My own current source of shame is a particularly atrocious rats' nest of conditionals that determine what gets displayed when someone views an debate on GreaterDebater. The debate itself doesn't change of course, but the options available to act on it depend highly on who's looking at it. The rules go like this: </p> <ol> <li> If the user isn't logged in, and the debate is still ongoing tell them they should log in if they want to vote. </li> <li> If the user is logged in, and isn't one of the people in the debate, and the debate hasn't ended, and this user hasn't voted yet, give them the option to vote. </li> <li> If all of the above is true except the user has already voted, showthem the current tally of votes with the option to change their vote. </li> <li> If the debate is over, always show the outcome and the vote tally. </li> <li> If the debate is ongoing and the user viewing it is the user whose turn it is, show them their options for replying. </li> <li> If 5 is true but they haven't yet accepted the challenge to the debate, then show them the options to accept or decline. </li> <li> If 5 is true but their opponent has offered a draw, then show the options to accept or decline the draw </li> <li> If the user is a participant in the debate and it's <strong>not</strong> their turn, just show the debate so far without any actions. </li> </ol> <p>Phew. All that is realized as the following code. </p> <pre><code>if arg.status in range(0,3): # The argument hasn't ended current = True if current and (request.user.is_authenticated() == False or request.user == arg.whos_up(invert=1)): # The viewer is either not logged in or a participant and it's not his turn # Don't show any controls show_arg_actions = False else: show_arg_actions = True if request.user == arg.defendant and arg.status == 0: # defendant hasn't accepted or declined the challenge yet # and is viewing the argument, show the options # to accept or decline the argument new_arg = True if current and request.user == arg.whos_up() and arg.draw_set.all(): # A draw has been proposed and the recipient is viewing the argument # show the option to decline or accept the draw show_draw = True if current and new_arg == False and not arg.draw_set.all() and request.user == arg.whos_up(): # No draw is pending, the person viewing the argument # is a participant and it's his turn show the options # to respond show_actions = True if current and request.user.is_authenticated() and not request.user in [arg.plaintiff, arg.defendant]: # The person viewing the argument is a registered user # and not a participant, show the voting box show_votes = True return render_to_response("items/arg_detail.html", {'object': arg, 'incite': arg.incite, 'comments': arg.argcomment_set.order_by('pub_date'), 'new_arg': new_arg, 'voted_for': voted_for, 'last_c': last_c, 'current': current, 'pvotes': votes.filter(voted_for="P").count(), 'dvotes': votes.filter(voted_for="D").count(), 'show_actions': show_actions, 'show_votes': show_votes, 'show_arg_actions': show_arg_actions, 'show_draw': show_draw }, context_instance=RequestContext(request)) </code></pre><p>This is actually an improvement from the first attempt. The show_* variables are in turn evaluted in <strong>another</strong> conglomeration of <em>IF</em> tags in the template. Suffice to say, the template is not very designer-friendly. </p> <pre><code>{% if show_arg_actions %} &lt;div id="arg_actions"&gt; {% if current %} {% if new_arg %} &lt;!-- The argument has been proposed but not accepted or declined, the defendant is viewing the page --&gt; &lt;div class="arg_query arg_responses" id="arg_responses"&gt; Begin this debate?&lt;br /&gt; &lt;ul class="flat-list"&gt; &lt;li&gt;&lt;a href="#" onclick="respond_challenge({{ object.id}}, {{ request.user.id }}, 0); return false;"&gt;Accept&lt;/a&gt;&lt;/li&gt; &lt;li&gt;&lt;a href="#" onclick="respond_challenge({{ object.id}}, {{ request.user.id }}, 1); return false;"&gt;Decline&lt;/a&gt;&lt;/li&gt; &lt;/ul&gt; &lt;/div&gt; {% endif %} {% if show_draw %} &lt;!-- A draw has been proposed, the user viewing the page is currently up in the argument --&gt; &lt;div class="arg_responses arg_query" id="draw_query"&gt; &lt;p&gt; A draw has been offered &lt;/p&gt; &lt;a class="menu" href="#" onclick="respond_draw({{ object.id }}, {{ request.user.id }}, 0); return false;"&gt;Accept&lt;/a&gt; | &lt;a class="menu" href="#" onclick="respond_draw({{ object.id }}, {{ request.user.id }}, 1); return false;"&gt;Decline&lt;/a&gt; &lt;/div&gt; {% endif %} {% if show_actions %} {% include "items/turn_actions.html" %} {% endif %} {% if show_votes %} &lt;div id="vote" class="arg_responses"&gt; {% if voted_for %} {% include "items/vote_div.html" %} {% else %} &lt;p&gt; Who's winning? Cast your vote below. &lt;/p&gt; &lt;a class="menu" href='#' onclick="vote({{ object.id }}, {{ request.user.id }}, 'P'); return false;"&gt;Vote for {{ object.plaintiff.username}}&lt;/a&gt; | &lt;a class="menu" href='#' onclick="vote({{ object.id }}, {{ request.user.id }}, 'D'); return false;"&gt;Vote for {{ object.defendant.username}}&lt;/a&gt; {% endif %} &lt;/div&gt; {% endif %} {% else %} &lt;div id="vote" class="arg_responses"&gt; {% include "items/vote_div.html" %} &lt;/div&gt; {% endif %} &lt;/div&gt; {% else %} {% if not request.user.is_authenticated %} &lt;div id="arg_actions"&gt; &lt;div class="arg_responses"&gt; &lt;a href="/users/login?next={{ request.path }}"&gt;Log in or register&lt;/a&gt; to cast your vote for the greater debater! &lt;/div&gt; &lt;/div&gt; {% endif %} {% endif %} </code></pre><p>What bothers me the most about this strategy is how the logic is split up between the view and the template. It should really be in one or the other, ideally it will be in the view in keeping with the Django practice of having as little logic in the template as possible. I can remove all of the conditionals from the template by breaking out each endpoint into a small template of its own. By choosing the appropriate template in the view, we can pass the rendered HTML to the main debate template directly, thus removing all of the conditionals from the template. This greatly simplifies the template. In fact, the entire mess of template conditionals above can be replaced with a single line: </p> <pre><code>{{ arg_actions }} </code></pre><p>The view will pass on the rendered HTML as the arg_actions variable. </p> <p>Of course, all of this complexity has to go somewhere. In this case it's being shoved into the view. At least Python is better equipped to handle it than the Django templating language. I'm still not entirely sure this is the best way to handle this situation, but I am glad the template is cleaned up. The code in the view now looks like this: </p> <pre><code>if arg.status in range(0,3): if request.user.is_authenticated(): if request.user == arg.whos_up(): if arg.status == 0: # The challenge has been proposed but not accepted argt=loader.get_template("items/arg_new_respond.html") argc = Context({'object': arg, 'request': request}) arg_actions = argt.render(argc) elif arg.draw_set.all(): # A draw has been offered argt=loader.get_template("items/draw_actions.html") argc = Context({'object': arg, 'request': request}) arg_actions = argt.render(argc) else: # normal turn argt=loader.get_template("items/turn_actions.html") argc = Context({'object': arg, 'last_c': last_c, 'user': request.user}) arg_actions = argt.render(argc) elif request.user == arg.whos_up(invert=1): # The user is a participant, but it's not his turn arg_actions = '' else: # The user is not a participant in this debate if voted_for: # The user has already cast a vote argt = loader.get_template("items/vote_tally.html") argc = Context({'pvotes': votes.filter(voted_for='P').count(), 'dvotes': votes.filter(voted_for='D').count(), 'object': arg, 'current': True, 'voted_for': voted_for }) arg_actions = argt.render(argc) else: # The user hasn't voted yet argt = loader.get_template("items/vote_div.html") argc = Context({'object': arg, 'request': request}) arg_actions = argt.render(argc) else: # debate is in progress, tell user to log in to vote argt = loader.get_template("items/arg_login.html") argc = Context({'request': request}) arg_actions = argt.render(argc) else: # debate has ended, show the final vote tally argt = loader.get_template("items/vote_tally.html") argc = Context({'pvotes': votes.filter(voted_for='P').count(), 'dvotes': votes.filter(voted_for='D').count(), 'object': arg, 'current': False, 'voted_for': voted_for }) arg_actions = argt.render(argc) return render_to_response("items/arg_detail.html", {'object': arg, 'incite': arg.incite, 'comments': arg.argcomment_set.order_by('pub_date'), 'last_c': last_c, 'arg_actions': arg_actions }, context_instance=RequestContext(request)) </code></pre><p>So, that's how my most hated code got transformed into perhaps my second-most hated code. I still feel like the deeply nested if statements here are a <a href="http://www.amazon.com/gp/product/0201485672?ie=UTF8&tag=greater0c-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0201485672">code smell</a><img src="http://www.assoc-amazon.com/e/ir?t=greater0c-20&l=as2&o=1&a=0201485672" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> that indicate there's a better way to handle this. I'm just not quite sure what it is yet. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/9/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/9/http://greaterdebater.com/blog/gabe/post/9/http://feedproxy.google.com/~r/ThatsDebatable/~3/WAcerpZDtIs/by gabe at Mar 08, 2010 <br /> <div> <p>Google is the undisputed champ when it comes to internet advertising. There's no shortage of companies on the web pursuing the same model. Obviously, not every product can be offered for free and funded with advertising, at some point somebody has to pay for the products being advertised in order for there to be any money to buy advertising with. Google's great insight has been the ability to specifically target advertising towards what people are looking for at the time. This advertising is much more valuable to both consumers and advertisers and it has enabled Google to make a plethora of excellent services available for free. How far could this model be taken though? </p> <p>Could it be extended to housing? Affordable housing is difficult to find in many major metropolitan areas. Suppose apartment buildings started adding screens inside the apartments that display advertisements in exchange for a lower rent. I know what you're thinking: it's outrageous and nobody would willingly put a <a href="http://www.amazon.com/gp/redirect.html?ie=UTF8&location=http%3A%2F%2Fwww.amazon.com%2Fs%3Fie%3DUTF8%26x%3D0%26ref_%3Dnb%5Fsb%5Fss%5Fi%5F0%5F4%26y%3D0%26field-keywords%3Dtelevision%26url%3Dsearch-alias%253Delectronics%26sprefix%3Dtele&tag=greater0c-20&linkCode=ur2&camp=1789&creative=390957">screen that displays advertisements</a><img src="https://www.assoc-amazon.com/e/ir?t=greater0c-20&l=ur2&o=1" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> in their home. Of course the television provides entertainment in exchange for viewing ads. Clearly people have shown they're willing to trade their attention in small slices when they get something in return. I think the only question is how much you would have to give before an in-home billboard becomes an attractive proposition. </p> <p>With a little extra technology this system could provide additional value for the consumer as well as the advertisers. Let's say you belong to discount club at your local supermarket. They're probably already tracking what you purchase and it would be trivial to link up your data at the store with your home billboard system. In my experience most purchases fall into some sort of a pattern, there are items you buy every week, some every other week or on some other frequency as they run out. Over time the store could easily pick up on the patterns in your purchases. </p> <p>Surely, this information is already shared with advertisers in one way or another. However, the advertisers' ability to act on this information is limited by current media. How much would it be worth to them to get an ad for, say, ketchup into just the homes of <em>people who are about to buy ketchup</em>? </p> <p>With all this information floating around between your home and the grocery store, it would be exceptionally easy to make up your shopping list. If the store knows what you're running out of they could add it to the list. In this case it would be a snap to run down the list of things you usually buy, evaluate what you might be running out of, and add anything new you might want to get. No doubt, someone's already working on some clever machine learning algorithms to analyze your purchases and suggest related products you might want to buy. </p> <p>As more stores start to incorporate <a href="http://freepizza.cc/2008/11/25/handheld-grocery-checkout-scanners/">handheld scanners</a> there's no reason you couldn't use one to scan your discount card and have it download your grocery list. It could check items off your list and even alert you if you get to the register without picking up something you meant to buy. (Can you tell this happens to me all the time?) The database the scanner is accessing knows what items the store has on sale. It could alert you if there's a coupon for something on your list or even if there's a similar item on sale at a lower price. </p> <p>At this point it's starting to sound like something people might actually <em>pay</em> for, rather than something that should be given away for free in exchange for some advertising, and this is just an example from one industry. </p> <p>People seem to hate advertising most when it's irrelevant. There doesn't seem to be quite so much resistance to advertising that is a.) unobtrusive and b.) related to something the viewer wants to buy. The in-home billboard could have a strong advantage in the relevance category with the right source of information. Making the sign sufficiently unobtrusive would mean striking a careful balance between not annoying the person who has to live with it and making it worthwhile for advertisers to spend money on it. I'd imagine that something like a poster-sized e-ink screen (or anything not back-lit) could fit the bill. </p> <p>Between the conveniences it could create and maybe a couple hundred dollars off of the monthly rent, I think targeted in-home advertising could be an attractive proposition for some. What would it take for you to live in a house with a personalized billboard? </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/8/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/8/http://greaterdebater.com/blog/gabe/post/8/http://feedproxy.google.com/~r/ThatsDebatable/~3/m1uQrLlYqmc/by gabe at Mar 01, 2010 <br /> <div> <p>Python is the language of one, and only one, obvious way to do things. This isn't quite the case when it comes to concatenating string though. There's three obvious choices. First, you can use the + operator. Since strings are considered a sequence type, along with lists and tuples, this works just like you would expect: </p> <pre><code>&gt;&gt;&gt; str1, str2, str3 = 'abc', 'def', 'ghi' &gt;&gt;&gt; str1 + str2 + str3 'abcdefghi' </code></pre><p>String formatting is also an option. This comes with a host of advantages, including type coercion when necessary. In this simple case it looks like this: </p> <pre><code>&gt;&gt;&gt; "%s%s%s" % (str1, str2, str3) 'abcdefghi' </code></pre><p>Finally, there's the join method available on every string. This method takes a list of strings and joins them together with the calling string in between each element. If this method is called from the empty string, the elements of the list are joined with nothing between them. For example: </p> <pre><code>&gt;&gt;&gt; ''.join([str1, str2, str3]) 'abcdefghi' &gt;&gt;&gt; ', '.join([str1, str2, str3]) 'abc, def, ghi' </code></pre><p>If we want to write idiomatic python, we're <a href="http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#building-strings-from-substrings">advised against</a> using the addition method in favor of using join. String formatting is not mentioned. Presumably, the flexibility of the formatting system is overkill for such a simple application. This is easy enough to test though, so let's see how the performance compares on a common application, say building a url with a username and some kind of id number. </p> <pre><code>def join_test(s, i): return ''.join(['/user/', s, '/', str(i), '/']) def format_test(s, i): return "/user/%s/%i/" % (s, i) def plus_test(s, i): return '/user/' + s + '/' + str(i) + '/' </code></pre><p>Now we just need to use Python's timeit module to build some timers. When the timer's timeit method is called, it will run its function 1,000,000 times and report how long it took. </p> <pre><code>import timeit jointimer = timeit.Timer("string_test.join_test('test', 5)", "import string_test") print "Join method took %f seconds" % jointimer.timeit() formattimer = timeit.Timer("string_test.format_test('test', 5)", "import string_test") print "Format method took %f seconds" % formattimer.timeit() plustimer = timeit.Timer("string_test.plus_test('test', 5)", "import string_test") print "Plus method took %f seconds" % plustimer.timeit() </code></pre><p>Simple enough, right? Let's look at the results. </p> <pre><code>Join method took 1.115780 seconds Format method took 1.275534 seconds Plus method took 0.756690 seconds </code></pre><p>Whoa! Now here's where I got confused. It turns out the much-maligned "plus" method of string concatenation is actually the fastest by a pretty wide margin. Has it been improved considerably since the Python idioms were determined or am I just doing something wrong? Occam's razor would suggest that I'm doing something wrong. Let's look more closely at the idiom and figure out what's different. </p> <blockquote><p>Start with a list of strings: colors = ['red', 'blue', 'green', 'yellow'] We want to join all the strings together into one large string. Especially when the number of substrings is large... Don't do this: </p> </blockquote><pre><code>result = '' for s in colors: result += s </code></pre><blockquote><p>This is very inefficient. It has terrible memory usage and performance patterns. The "summation" will compute, store, and then throw away each intermediate step. Instead, do this: </p> </blockquote><pre><code>result = ''.join(colors) </code></pre><blockquote><p>The join() string method does all the copying in one pass. </p> </blockquote><p>Well, for one thing, the usage we're being warned about here is a list of strings being summed by a loop. Let's add another test that matches that scenario exactly and see how it performs. </p> <pre><code>def loop_test(s, i): l = ['/user/', s, '/', str(i), '/'] result = '' for s in l: result += s return result looptimer = timeit.Timer("string_test.loop_test('test', 5)", "import string_test") print "Loop method took %f seconds" % looptimer.timeit() </code></pre><p>And the result: </p> <pre><code>Loop method took 1.427076 seconds </code></pre><p>Now, that makes more sense. The overhead of the loop adds to the time considerably. In fact, if we modify the plus_test function to add the overhead of making a list, this alone accounts for a large portion of the difference in performance. </p> <pre><code>def plus_test(s, i): l = ['/user/', s, '/', str(i), '/'] return '/user/' + s + '/' + str(i) + '/' Plus method took 1.259691 seconds </code></pre><p>So, if you can avoid building a list and just sum the strings together directly, this is the fastest way to concatenate them. However, if they're already in a list, using join is your best bet. The format method is pretty slow even without the overhead of making a list, but it's probably going to be a bit more readable when there's multiple substitutions with different types involved. </p> <p>Of course, you should always profile your code to see where the bottlenecks are before making any changes. If you're starting from scratch though, it doesn't hurt to use the fastest method available. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/7/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/7/http://greaterdebater.com/blog/gabe/post/7/http://feedproxy.google.com/~r/ThatsDebatable/~3/-LT10HfXxkQ/by gabe at Feb 22, 2010 <br /> <div> <p>Other social news sites(<a href="http://reddit.com">Reddit</a>, <a href="http://news.ycombinator.com">Hacker News</a>, <a href="http://digg.com">Digg</a> to name a few) use a voting mechanism to rank stories and comments. On the whole, these work pretty well. The top rated stories and comments are often the most interesting, or at least the most amusing. It seems that there is always a certain level of disagreement about the proper voting etiquette though. Inevitably something gets voted to the top that some people feel should not have been. Said people make their feelings known, trying to educate everyone else on why this story shouldn't be at the top. Better stories belong there. Presumably, the ones that they themselves like. </p> <p>Sure, these sites have guidelines to explain how and when you should vote. In a perfect world, everyone would read and abide by them. Unfortunately people have their own conceptions of when they should vote things up or down. Here's a partial list of my own reasons: </p> <p><strong>Up</strong><br /> </p> <ul> <li> It made me laugh </li> <li> It answered the question the post was asking </li> <li> It already said what I opened the comments to say </li> <li> It makes a good point and I agree with it </li> <li> I don't agree with it, but it makes a good point </li> <li> It sounds like it will generate some interesting discussion, so I want more people to see it. </li> <li> I enjoyed it and I wish its makers success, so I want to contribute to giving it publicity. </li> <li> Mostly, it made me laugh </li> </ul> <p><strong>Down</strong><br /> </p> <ul> <li> It's incorrect </li> <li> It's abusive </li> <li> It's harmful <em>e.g. telling a new Linux user to run rm -rf /</em> </li> <li> I don't agree with it, and its point is made poorly </li> <li> I'm bored of seeing this sort of thing </li> <li> It failed to make me laugh </li> </ul> <p>I imagine most people have similar lists of reasons for voting the way they do. Some of the reasons are good according to the site standards, some are not. (Of course there's also always the people who decide "I just submitted a story so I'll vote everyone else's story down, so mine is relatively more popular) </p> <p>I don't think any of these reasons are bad. No doubt everybody else feels the same way about their own list of reasons for voting the way they do. However, this leaves some confusion for what it means to be voted to the top on one of these sites. Was your link or comment the funniest? The most insightful? The one everyone agrees with? It could be any of these, or all, or none. </p> <p>On GreaterDebater topics are sorted by the rate at which they are accruing new comments. Writing a comment is a little more involved than simply voting, so the topics at the top are those that most inspire conversation. This way the main metric used for sorting topics is their level of engagement. </p> <p>Does this contradict the decision to use voting to determine the outcomes of debates? I don't think so. The decision to cast a vote for one or the other participant in a debate is a lot less ambiguous than the one to vote a topic up or down. The voting system for debates may yet prove to have it's own set of perils but they likely won't involve the promotion of uninteresting debates. </p> <p>Longer comments have a greater impact on the ranking of a topic than shorter ones. Comment length is not necessarily an indicator of quality, but it is often correlated. For instance, it does tend to indicate the amount of thought put into a comment. Topics that inspire fewer thoughtful comments will rise faster than those that only accrue short "me too" style comments. </p> <p>Of course, not all comments will be in support of a topic. Sometimes people will leave a comment to express their dissatisifaction or disagreement with a particular issue. I don't think this is a problem though when it comes to the possibility of having a topic with which many people disagree at the top of the list. As long as it inspires disagreement that creates interesting discussion, there should be no complaints about a topic's position. It's quite often that I end up learning more from the comments on social news sites than I do from the articles themselves. I hope to encourage that same atmosphere here at GreaterDebater. </p> <p>In general, I think the best way to find out how someone feels about something is to look at what they do, whenever possible, rather than ask them directly about it. In retrospect it's pretty simple. Rather than ask people which topics they find most engaging, simply look at the ones that they're most engaged with. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/6/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/6/http://greaterdebater.com/blog/gabe/post/6/http://feedproxy.google.com/~r/ThatsDebatable/~3/QI1yW11g_b8/by gabe at Feb 15, 2010 <br /> <div> <p>For now, let's disregard the cynical response: "If you're arguing online, you've already lost". Sure, there's no shortage of unproductive flaming happening on the internet, but let's assume for the moment that it's possible for people to disagree constructively. </p> <p>The question then is that in the fortuitous event of such a disagreement, how should it be resolved? In a perfect world, sooner or later one party would be convinced and accept... </p> <p>Ok, I'll wait until you've stopped laughing. </p> <p>... </p> <p>So, it's exceedingly rare for a participant in a debate to admit error and concede the point to his opponent. Nevertheless it was my first instinct for resolving debates at GreaterDebater. For in what way can you say you've really succeeded in a debate if your opponent remains unconvinced? I figured debates would go on until either one party conceded or a certain period of inactivity had passed. I soon realized that it would be very easy for a debate with this scheme to degenerate into a war of attrition. If the only requirement is to keep making replies until your opponent has stopped, this favors persistence over a valid argument. Idealism is all well and good, but we can't always expect the rest of the world to live up to our ideals. It's the nature of a debate that there will always be people who merely want to win, rather than to seek the truth. </p> <p>In other formats, debates are often moderated by a panel of judges. Ideally, these judges are experts in the subject matter being debated and/or the art of debating. In theory, this sounds like the best solution, but it's difficult to implement. GreaterDebater isn't limited to any one subject, so a wide variety of experts would be needed. There will (hopefully) one day be a large volume of debates. Given the unpredictability of the subject matter and the scale at which I hope to operate, arranging expert judges for every debate would impose a crushing amount of overhead on the process. Maybe this is something that could be arranged under special circumstances, but the level of coordination required would greatly hamper the spontaneous nature of posting comments on the internet. </p> <p>Finally, I came up with the approach of general voting. With a diverse enough user base any biases the individual voters might have should cancel each other out. What's left should be a reliable indicator of the how convincingly each side argued his or her case. Voting has proven to be effective at promoting insightful comments at sites like <a href="http://reddit.com">Reddit</a> and <a href="http://news.ycombinator.com">Hacker News</a>, so it should be too much of a stretch to suppose it could also pick a quality argument. With a time limit to ensure debates don't go on forever, a debater with a solid case should be able to win over a simple majority of disinterested third parties, even if he can't convince his opponent. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/5/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/5/http://greaterdebater.com/blog/gabe/post/5/http://feedproxy.google.com/~r/ThatsDebatable/~3/yRXDrtGJTvA/by gabe at Feb 08, 2010 <br /> <div> <p>Finding a url inside a chunk of text is no easy task. <a href="http://www.codinghorror.com/blog/archives/001181.html">Coding Horror</a> and <a href="http://daringfireball.net/2009/11/liberal_regex_for_matching_urls">Daring Fireball</a> have covered regular expressions for matching urls in a variety of circumstances. I went with the Coding Horror regular expression, mainly because the Daring Fireball version wasn't around when I was working on this. Using Python's regular expression library, the code to get all the urls in a block of text looks like this: </p> <pre><code>urlre = re.compile("(\(?https?://[-A-Za-z0-9+&amp;@#/%?=~_()|!:,.;]*[-A-Za-z0-9+&amp;@#/%=~_()|])(\"&gt;|&lt;/a&gt;)?") urls = urlre.findall(html) </code></pre><p>That pulls out all of the urls in a text and puts them in a list. From there you can iterate through the list and replace the url in the source text with the appropriate html. </p> <p>But that only works if you're only allowing plain text as input. What if you're using markdown for formatting? In this case there might be urls in the text that are already a part of a link. You don't want to add additional html around these or you'll break the user's formatting. There's two ways you could go, I suppose. You could either process the text before you send it to the markdown processor and add markdown formatting around the urls, or you could look at the html output you get and find the bare urls in that. I went with the latter option. This way the function isn't tied to markdown specifically, and it's useful in more applications. </p> <p>The obvious case you want to avoid is where the user has entered a normal link as markdown. In that case, the markdown formatting will take care of adding the appropriate html. </p> <pre><code>&lt;a href="http://example.com"&gt;link&lt;/a&gt; </code></pre><p>But maybe some helpful user has used markdown to link a url for us. That will output some html like this: </p> <pre><code>&lt;a href="http://example.com"&gt;<a rel="nofollow" href="http://example.com">http://example.com</a>&lt;/a&gt; </code></pre><p>In this case we want to ignore both instances of the url. <br /> </p> <p>Finally, you have to account for the sadistic case. This is the possibility that someone will make a regular markdown link with a given url, then paste in that <em>same</em> url elsewhere in the text. You have to ignore the instance where it's linked legitimately, but catch the instance where it's a bare url and turn it into a link. The resulting html will look like this: </p> <pre><code>&lt;a href="http://example.com"&gt;<a rel="nofollow" href="http://example.com">http://example.com</a>&lt;/a&gt; ... ... <a rel="nofollow" href="http://example.com">http://example.com</a> </code></pre><p>There are two steps to solve this. The first part is the regular expression shown above. This actually has two groups. The first group </p> <pre><code>(\(?https?://[-A-Za-z0-9+&amp;@#/%?=~_()|!:,.;]*[-A-Za-z0-9+&amp;@#/%=~_()|]) </code></pre><p>matches the url while the second group </p> <pre><code>(\"&gt;|&lt;/a&gt;)? </code></pre><p>matches either "&gt; or &lt;/a&gt; after the url, if they're present. When your regular expression has multiple groups urlre.findall returns a list of tuples. The first element of the tuple contains the matched url. There will be an entry in the list for every occurrence of a url in the text, with duplicates included. The second element in the tuple will be either "&gt; or &lt;/a&gt; if they are present immediately after the url. If the second element is not an empty string, we know the url is already part of a link, so we can ignore it. </p> <p>Then there's the sadistic case to contend with. You want to replace a url with a link <em>only</em> when it's not preceded by =" or "&gt;. Instead of using the simple string replace function, the substitute function from the regular expressions library will allow us to only replace certain instances of the url. A negative lookbehind will ensure that the url is replaced only when it's not already part of a link. </p> <p>The complete autolink function is shown below. </p> <pre><code>def autolink(html): # match all the urls # this returns a tuple with two groups # if the url is part of an existing link, the second element # in the tuple will be "&gt; or &lt;/a&gt; # if not, the second element will be an empty string urlre = re.compile("(\(?https?://[-A-Za-z0-9+&amp;@#/%?=~_()|!:,.;]*[-A-Za-z0-9+&amp;@#/%=~_()|])(\"&gt;|&lt;/a&gt;)?") urls = urlre.findall(html) clean_urls = [] # remove the duplicate matches # and replace urls with a link for url in urls: # ignore urls that are part of a link already if url[1]: continue c_url = url[0] # ignore parens if they enclose the entire url if c_url[0] == '(' and c_url[-1] == ')': c_url = c_url[1:-1] if c_url in clean_urls: continue # We've already linked this url clean_urls.append(c_url) # substitute only where the url is not already part of a # link element. html = re.sub("(?&lt;!(=\"|\"&gt;))" + re.escape(c_url), "&lt;a rel=\"nofollow\" href=\"" + c_url + "\"&gt;" + c_url + "&lt;/a&gt;", html) return html </code></pre><p>Hopefully, a later version of markdown will have autolinking baked right in. If you're using <a href="http://github.github.com/github-flavored-markdown/">GitHub Flavored Markdown</a>, you've already got it. I haven't seen a Python implementation for that yet, though. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/4/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/4/http://greaterdebater.com/blog/gabe/post/4/http://feedproxy.google.com/~r/ThatsDebatable/~3/roqVn1M7-34/by gabe at Feb 01, 2010 <br /> <div> <p>Welcome to the first installment of <strong>That's Debatable</strong>, the official GreaterDebater blog. I guess this is the part where I talk about the sorts of things I intend to be posting here. For the time being, I'm planning on using this as a catchall blog for everything I want to write about. I suppose time will tell if this is the best idea or not. On the GreaterDebater front, I'd like to blog about: </p> <ul> <li> New and in-development features for GreaterDebater. Feedback is always appreciated. </li> <li> Some of how GreaterDebater works and things I've learned along the way. Hopefully someone can learn from my mistakes and blind alleys. </li> <li> Any other GreaterDebater news and announcements. </li> </ul> <p>On a more general note, I'm going to blog about: </p> <ul> <li> Other projects, unrelated to GreaterDebater. I have some ideas I hope to start working on soon. </li> <li> Reviews of books, movies, video games and other media. </li> <li> Anything else that comes to mind that I think might start some interesting discussion (either in the comments here or at GreaterDebater). </li> </ul> <p>So there it is! Now to get started on a non-metablogging blog post. </p> </div> <br /> <div> <a href="http://greaterdebater.com/blog/gabe/post/1/">view comments</a> </div> http://greaterdebater.com/blog/gabe/post/1/http://greaterdebater.com/blog/gabe/post/1/