<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0"
					xmlns:content="http://purl.org/rss/1.0/modules/content/"
					xmlns:wfw="http://wellformedweb.org/CommentAPI/"
				  >
<channel>
<title>the_codist()</title>
<link>http://thecodist.com/</link>
<description><![CDATA[Thinking About Programming]]></description>
<language>en-us</language>
<pubDate>Sat, 25 Jun 2011 15:46:14 -0700</pubDate>
<item>
<title>Go Ye Out and Build a New Facebook </title>
<link>http://thecodist.com/article/go_ye_out_and_build_a_new_facebook</link>
<description><![CDATA[<p>Everyone, or at least 1 out of every 10 people in the world, has a Facebook account.</p>

<p>People seem to like connecting with friends and relatives, keeping up with what&#39;s new in their lives, and sharing what&#39;s going on.</p>

<p>But do those two things have to intersect? I don&#39;t think so, despite the enormity of Facebook&#39;s size and ubiquitousness. Most people I know on Facebook don&#39;t really like it much, but stick around because it&#39;s the only game in town. Generally everyone is aware of how Facebook is dying to make money off of them, the complexity of the settings, the constant changes, the fear that the next little game or app that you give rights to will steal everything you own. It feels like a benevolent dictator that you are never quite sure might not suddenly become a monster.</p>

<p>The history of business has shown that it is quite rare for someone to dominate indefinitely no matter how large the market share might be. The history of internet related businesses is full of people who ran the table only to be run out of town by the next big thing. Facebook is huge yet as vulnerable as anyone.</p>

<p>For one thing, Facebook doesn&#39;t really make money, in 2010 something like $1 per account, which is chump change. So if they have an IPO they need to generate a lot more income to justify the business, which means making it off the users. No on yet has figured out how to make billions and billions of dollars off of people who have no interest in paying anything to talk with their friends.</p>

<p>So why try to compete with a giant? Why approach venture capitalists (or even your own pockets) with ideas of competing with the largest non-search, non store company?</p>

<p>Because it&#39;s fun to do what other people think is impossible. Taking on giants is what the internet is best at, witness the succession of huge companies that didn&#39;t exist 15 years ago, 10 years ago or even 5 years ago. Creating new websites and web systems is so easy today any competent bunch of people can develop at idea into an internet destination in a very short time. With enough smart people trying enough ideas all at the same time something is bound to be successful. </p>

<p>I doubt anyone will ever come up with a pure internet company that makes Apple kind of income and spread of products; when I read about how much money people invest in stuff these days I wonder how the companies that result will ever justify the investment of anyone other than the early investors. But it doesn&#39;t stop them from pumping money into everything. So why not another Facebook? It&#39;s just as likely something will come along and replace it as it did MySpace or the death of AOL. It might take a while and lots of tries or happen overnight, you never know.</p>

<p>After all what is Facebook offering besides all the connections you have with friends? Games, most of which you can get elsewhere. Funny little apps? Chat which you can get elsewhere. An API for applications to let you connect with your friends; sure it&#39;s convenient but nothing something else could not provide. Basically Facebook wins because everyone is on it. It&#39;s like saying horses are the transportation of the future back in 1890 because everyone is riding them. But today you don&#39;t have to invent the automobile or even have a factory.</p>

<p>Some people are trying to replace it, but the numbers are way to small in my opinion. The folks at <a href="Diaspora">https://joindiaspora.com/</a> are certainly trying the hardest but I wonder if they have investment at all.</p>

<p>Personally I would rather have something that was simply (1) private first (2) allowed me to share text and media with my friends (3) had a nice sharing API. After all, what matters to almost all of my friends is the network of friends, not where that network is hosted. Ideally what I want (and probably most people) is ability to get these things but not be locked into one site or system or company. After all I can call my friends on the phone regardless of the carrier (though sadly there are fewer and fewer options there); I have a phone number and list of my friend&#39;s phone numbers and that is sufficient to make a network. I just want something like that but on the internet.</p>

<p>There is absolutely nothing in Facebook that makes it the only option for social connecting. Even Twitter, the other darling of the social set, has nothing special other than volume. Ultimately both of them are simply two ways of connecting people together but hardly the only possibilities.</p>

<p>So never give up on an idea simply because the competition has 700 million subscribers. Pick up a stone, stick it in your slingshot and fire away.</p>
]]></description>
</item>
<item>
<title>What&amp;#39;s My Motivation? </title>
<link>http://thecodist.com/article/what_39_s_my_motivation</link>
<description><![CDATA[<p>The stereotypical actor always wants to know this before performing a scene. Why am I doing this? Why is the character saying that? How did we get into this scene? The idea being that understanding what the character would be doing or saying in the real situation is helpful to making the acting seem real.</p>

<p>Over the years I&#39;ve been a programmer (almost 30) I&#39;ve generally been aware of what motivates me in projects, jobs and coding in general. Sometimes I know too well that my motivation gets me into trouble sometimes!</p>

<p>I have this desire to fix broken things, solve problems others abandon as too hard, and look for solutions no one else sees. I can look back at why I took certain jobs or contracts, why I started little companies, why I injected myself into situations where failure was almost assured, and even what technologies I became interested in.</p>

<p>What gets me into trouble is fixing broken things irritates the people who broke them; solving problems others abandon as impossible makes them look stupid, and seeing the solution that I know will work often is at odds with the status quo. Even though I generally get along with most people, there are some kinds of people who simply will hate you for doing better than they do, or even doing something different than they do.</p>

<p>Being motivated by these potential failures means you have to be ready and willing to suffer the failure if you can&#39;t succeed. Lots of people hate to fail and do anything to avoid it; often it&#39;s hard for me to convince them that I like taking chances and even arguing in favor of them despite the odds.</p>

<p>What doesn&#39;t motivate me is a daily grind, just filling out a box in an org chart, picking up my check, or working on a doomed project where there is no hope and I have no ability to change anything. I&#39;d rather quit or work for cheap on something where I can be motivated and make a real difference. Sadly this motivation has messed me up financially more times than I can count!</p>

<p>I have way too many examples of how this strange desire to risk failure has affected me.</p>

<p>The best code I ever wrote came out of just such a situation. I worked for a consulting firm around 1999. We got a contract to build a web based product search site for Consumer&#39;s Digest (don&#39;t look for it, they never paid for it and it got pulled after 18 months). I wasn&#39;t involved in the project. Due to the ridiculously complex Automobile feature/package rules the previous firm had given up after 6 months saying it was impossible. A month before CD had promised the site would be up, our team was in panic mode as the average search time was 70 seconds on production hardware, and nothing they did made it go down. So I volunteered to come up with a solution. However I could not change the database, change any code other than the search feature itself, and the hardware was already purchased. Despite that I was able to figure out a solution, test it, write the code and insert it into the rest of the site with no database changes, and the average search time was well below a second. The site went live on time and was popular until we killed it since CD never paid up.</p>

<p>What&#39;s more there was never a single bug reported in the search code over the entire 18 months it was up. Under ridiculous pressure and impossible odds I made it work. It&#39;s not like that happens every time of course but for whatever reason this kind of problem kicks my motivation into high gear.</p>

<p>Later on I worked for a Financial services company (as a Software Architect) which had Java web applications to serve the customers as well as the field agents. For the prior three or so years before I started the Java programmers had been blamed for the terrible performance of the applications. The database backing these was the company AS/400 which that group said was really fast and self-optimizing so the issue had to be Java. Nothing the Java team said made any difference.</p>

<p>This bothered me for while before I came up with a way to prove what the issue was (which was clearly the database to the Java team). The AS/400 ran a version of DB2 which the Java applications used SQL to talk to, but the underlying data was managed by a non-SQL 4Gen tool. The Oracle DBA helped me out by cloning the production data into a compatible schema on an old PC running Oracle. I then wrote a benchmark Java application that communicated either with the Production DB2 or the wimpy Oracle PC and simply obtained a single row from each table in the schema while measuring the time. Low and behold the little Oracle PC ran rings around the production DB2 in some cases by an order of magnitude. I packaged up the code and the charts and sent them around to everyone that mattered.</p>

<p>The AS/400 DBA that had insisted there was nothing wrong with his hardware had no way to explain how an obsolete PC could beat his 6 million dollar baby, so he ran the benchmark Java application while we looked at the AS400 console. It turned out the DB2 was running in a 90MB partition on the AS/400 and every time it was accessed swapped memory like crazy. All of the companies 150,000 customers and 800 agents were sharing 90MB of RAM! So they admitted it was too small, upped the partition to 1GB and suddenly the unchanged Java applications ran like the wind. Problem solved!</p>

<p>Then the company went into politics as usual mode, made stupid decisions, created uselessly complex software methodologies and I couldn&#39;t stand it anymore and I found another job.</p>

<p>At my last employee type job at a healthcare claims processing company I was also an architect but when it came to deciding on how to replace the company&#39;s main software system I sided with the CTO who thought we should have a backup plan and not depend on a third party middleware system that was pretty skanky. I pushed this to the architecture team and they got so pissed at me I had to leave. They went ahead and worked on the new system using their plan for more than a year and it never worked. Eventually a new CTO came along and canned the effort, replacing it with a new system not too different from my backup plan (though I wanted to build it and this didn&#39;t exist at the time). So some vindication but I sometimes miss the great salary and 5 weeks vacation the company offered. There is always a downside to my motivation!</p>

<p>So just yesterday I turned down a promising job with a Silicon Valley startup basically since I don&#39;t believe in the product, the development is very rigid, and the potential for making a real difference seems negligible. Probably 3x the money I am making now, but at the game company I am at now I can and have made a difference, the challenges are (too) many, and I love the product.</p>

<p>I guess I am basically an idiot but too old to give up on what motivates me as a programmer. If I ever lose it and just cash a paycheck and stop caring I will quit programming.</p>

<p>Though it would be nice to make 3X the money again!</p>
]]></description>
</item>
<item>
<title>The Application Updatability Curve</title>
<link>http://thecodist.com/article/the_application_updatability_curve</link>
<description><![CDATA[<p><img src="http://thecodist.com/assets/curve.png" alt="Application Updatability Curve" /></p>

<p>Like people, software has a lifespan -- eventually everything reaches a point where nothing works anymore and the end comes. Unlike people however companies often push software far beyond its natural life and keep trying to update it despite the ever increasing cost to do so.</p>

<p>This curve represents the lifespan of most applications. Early on there is not much code written and thus every feature is fairly expensive since it requires you to code everything new (though it can be much cheaper if you start with other people&#39;s code). As the application is shipped or reaches maturity, adding new functionality is fairly easy; the original programmers might still be around, the code is well understood and hopefully cleanly written and documented. As time goes by and people leave, documentation and code knowledge goes stale, and the code begins to get brittle the ease of creating new code becomes much more difficult. Eventually the code is so crappy every change is hard to test, every bug fix or new feature adds more instability and new bugs appear every day. At that point development slows to a crawl or features are cancelled by the bushel. Cost per feature becomes very high.</p>

<p>Now cost in this picture may not simply be the cost of development, it can just as easily be programmer burnout and turnover, customer frustration at poor quality and lack of progress, and even reduced income.</p>

<p>Note that I am talking about essentially adding features not simply bug fixes or basic maintenance. Some software rarely needs new features but in most cases due to competition, market pressure or simply the need to create new income you have to add to the application&#39;s feature set. The problem comes when the cost of adding (or the inability to deliver) new functionality reaches a point where it really isn&#39;t feasible despite the demands of management or the market.</p>

<p>Me and my team worked on Deltapoint&#39;s Deltagraph for 5 years from 89-94 during which we deliver 5 major releases for this market leading product. After all the changes including a port to Windows by the publisher, the C code was no longer very good and we wanted to redo the application in C++. At the time Deltapoint was making something like 5 million per year in sales so we thought it should have been affordable. The decided it wasn&#39;t worth the investment and we went our separate ways. Eventually they sold off the product and went into a different direction.</p>

<p>Now 22 years after the initial development the product is owned by <a href="http://redrocksw.com">Red Rock Sofware</a>. During that time no rewrite was ever done, and I am sure the code is beyond ugly at that point. In fact the UI still has the same crappy icons I stuck in temporarily back in 1989! I can imagine the curses of all the engineers forced to work on this today. I don&#39;t blame them if they have a dartboard with my name on it. The number of new features added to the codebase since 1994 is very small compared to those we added in those 5 years; much of the application seems identical to my last checkin back then.</p>

<p>The SABRE reservation system was a marvel back in the 1960&#39;s. By 2000 the code was so decrepit SABRE had to hire people from Russia, as it was hard to find anyone in the US who still wrote in the IBM macro assembler it was written in. I heard stories where people would collect unused bits from various data structures to create new data items since they couldn&#39;t easily expand the records (might be just a story but it seemed possible). It took them something like 12 years to finally replace it but today the system is a modern SQL database based system.</p>

<p>It is very difficult to convince management that their software is becoming impossible to update. People like to think that all it takes is more or better or smarter programmers or something to change an old codebase. I liken it to trying to run a dead horse in the Kentucky Derby -- but no matter how much you want it the thing won&#39;t run.</p>

<p>People often fail to understand that rewriting an application may allow newer/better/faster technology; use open source or even purchased frameworks and libraries; and maybe simply have the opportunity to do things write the second time. Yes, I heard the saying "it&#39;s better to keep the old bugs you know rather than create new ones you don&#39;t" but that supposes your current team is more incompetent that the original. If that is the case then what are the odds they can successfully update the old brittle application?</p>

<p>Software can be successfully reincarnated as something new; it just takes an investment and desire to move your software back to the original start of the curve and then capture the easy middle. If you refuse you may have to pay the price and live with the consequences of being unable to be keep up with the market, piss off your customers and deal with the turnover of frustrated programmers.</p>

<p>Of course in a perfect world every application is written so well that it lives successfully forever. Sadly most of the software in the world is neither perfect nor will it live happily ever after. More commonly I would expect there are a lot of dead horses out there being kicked until the have some semblance of life.</p>
]]></description>
</item>
<item>
<title>The Stone Age to Wells Fargo Online Banking, We Want Our Technology Back </title>
<link>http://thecodist.com/article/the_stone_age_to_wells_fargo_online_banking_we_want_our_technology_back</link>
<description><![CDATA[<p>Every month I pay my mortgage to CitiMortgage via Wells Fargo&#39;s online banking. Last month it was lost and now I am in customer "service" hell.</p>

<p>Apparently WF sends mortgage checks by mail to CitiMortgage despite both being among the world&#39;s largest banks, both of whom have been doing electronic transmission of money for decades. Usually it works. But what happens when the payment is lost, or sent to the wrong place, or some other problem?</p>

<p>Once Citi started complaining about the missing payment I check online at Well&#39;s bill pay application and it indicated the payment had been sent. So I filled out the inquiry form. After a few days someone tried to call my cell phone (while the battery was down) and left a message saying they had Citi on a conference call and to call them back, along with a reference to the inquiry (a number). He also stated that Citi would not tell WF anything since they weren&#39;t the mortgage holder unless I got on the line.</p>

<p>Once I saw the message I called the phone number which was just the main number at WF, requiring a long trip down the menu system to get the right place.</p>

<p>Now the fun started. I gave the service person the reference number, and after several minutes of silence he finally admitted he couldn&#39;t find it despite running two queries. It took several minutes as the place he worked had hundreds of people sharing a single network connection and generally everything took minutes to execute. Apparently the reference number was no longer active, as normally if they don&#39;t get you on the phone after one try they kill the inquiry. So he had to search some archive system which took another 10-15 minutes before he came back and found the inquiry, but none of the details were attached (apparently they aren&#39;t archived).</p>

<p>He made a new inquiry and put a note in it saying "please try to call more than once" and told me "that&#39;s how we usually write it up". Maybe they will call on Monday or sometime this week.</p>

<p>So other than a good way to piss me off, what can we learn from this?</p>

<p>Unlike most customers I have worked for financial service companies (with a bank), analyzed service organizations and developed and architected applications for these industry areas. I understand how the sausage is made.</p>

<p>Customer service and similar organizations in big companies are always considered an expense, and developing productive technology for their employees not directly earning revenue is considered a waste of time and money. Often the leadership of the company is completely disconnected from the actual employees stuck in these organizations and are unaware (or simply don&#39;t care) how poor their service appears from the real customers.</p>

<p>When I went to fix the refund process at cheaptickets.com (long ago bought by someone since) around 2000 the process of getting a refund took an average of six months during which no one had any idea of the status should a customer call in. The company did not care about the refund department whatsoever, assuming the pissed customers could be easily replaced by new ones. When this became apparent it was costing real money, they wanted something new.</p>

<p>What I found was 4 ladies who worked long hours on a Rube Goldberg type system they had put together from whatever they could find in which not only did it take months to grant or deny a refund, but in such a way that people either got money they didn&#39;t deserve or people got nothing when they should have been refunded. Plus the system was so slow that the credit card companies would often support a claim because they never heard back from the overworked quartet.</p>

<p>When I left with a mostly complete project the average time took a week and status was available via a web page to anyone in the company. The 4 ladies couldn&#39;t believe how easy their job had become.</p>

<p>While doing work for SABRE (the reservations company) around the same time I was asked to profile the Air Pricing group, about 250 people who processed the ticket prices and rules from the many airlines that used SABRE. My analysis should each worked needed to understand 17 different systems in order to do their job, which generally took 6 months to become productive at. The company provided no actual tools to help in the work, relying on outside systems, several manually built access databases, an Excel spreadsheet (one copy used for syncing the work!) and a lot of manual text editing of raw data. It was horrible. It also followed an internal profile which came to the same conclusion, that the company needed to invest in providing a streamlined data system collect all the data (most of which was available via files) and provide a single system for everyone to use.</p>

<p>The management didn&#39;t like the first profile or mine and afterwords requested another internal one; I guess they were waiting on someone to say everything was fine.</p>

<p>Well Fargo, like most banks, has mainframes with software likely written in the 70&#39;s and 80&#39;s at their core. Providing a web interface is very difficult to do on systems not intended to be web interactive. From looking at their online banking system you can see how absolutely crappy it is on many levels. It looks like something from the mid-90s. Each section of the online banking appears to be a separate application (witness the single signon mess when you switch to Bill Pay for example). User interface design seems to be different in different places, as if there are multiple groups responsible, and who probably refuse to work together. The inquiries, for example, vanish once you have created one; there is no way to track them yourself. Of course I found out even the CS people can&#39;t track them either. Likely there is no simple way to exact the information from wherever they are stored so there can be no way to provide them to a customer.</p>

<p>If you read the VP/CTO&#39;s resume they trumpet the awards the online banking has won. Like paying for awards is difficult (there is prominent best buy award from a magazine that works this way, which we found out about when we did their website in the early 2000&#39;s).</p>

<p>Providing poor tools to customer service people along with insufficient bandwidth is a clear sign customer service is not important. Building a website that is ugly, inconsistent, and slow is par for the course for most large financial institutions. Having to send paper checks between banks is another sign of a lack of interest in the customer (why is a utility payment might be sent electronically right away but my mortgage is not). But then again I&#39;ve been on the inside and it&#39;s not surprising at all.</p>

<p>Sure I would like to get away from Wells Fargo, but at the moment I am somewhat stuck there.</p>

<p>It&#39;s odd that customer service winds up becoming customer disservice. I do wish all companies with customer service departments required their executives spend a week a year working in one.</p>

<p>It&#39;s not hard to make happy customers, even when things go wrong. It&#39;s not hard to make happy customer service employees either. When you think pissed off customers don&#39;t matter, that there is always another sucker coming along, you run the risk of eventually running out of them.</p>

<p>Then it&#39;s too late.</p>
]]></description>
</item>
<item>
<title>I See the Future of the Games Industry and It&amp;#39;s Procedurally Generated </title>
<link>http://thecodist.com/article/i_see_the_future_of_the_games_industry_and_it_39_s_procedurally_generated</link>
<description><![CDATA[<p>Being in the game industry I see how much effort goes into content generation. You could say most games are content glued together with a thin layer of programming. Yet creating content is a labor intensive and thus expensive process that only grows as technology allows (and customers demand) more complex and richer games.</p>

<p>I am amazed at how rich the environment appears in iD&#39;s new game, <a href="http://idsoftware.com/">Rage</a> and wonder at the supposed 20GB of data backing it up. The amount of effort required to generate such a AAA title must be staggering.</p>

<p><a href="http://www.swtor.com/">Star Wars The Old Republic</a> is rumored to be approaching $100 Million in cost. Unbelievable that games now rival Hollywood in how much it takes to build a major title.</p>

<p>There are also millions of games developed by small teams or individuals, most of which have at least some art and other content. For many it&#39;s fairly minimal, such as a word game for the iPhone like the one I&#39;ve written. For an MMO like the one I work on the content can be pretty large (ours is a small team but the game has been live for 10 years) and for something like Rage enormous.</p>

<p>Of course content is a broad category: it begins with art, from 2d painting to 3d object creation, to animation, story, level design, ai and even web and marketing design. It&#39;s the stuff the makes the game what it is. Programming (my part of this) can be anything from simple scripting in an existing engine such as Unity Pro or Unreal to building your own engine and technology like iD does with Rage. Without content though the programming is an empty shell.</p>

<p>If a top level AAA title involves tens of millions of dollars to build and generates gigabytes of content how does the developer/marketer/publisher make enough money to justify the cost? Most AAA titles generate the bulk of their revenues in the first few weeks after release. Games which feature solo play generally play for less than 10 hours to complete them. Once the game is finished, a customer might dabble in multiplayer play for a bit before diving into the next title. In such a game incredibly detailed and rich areas might takes weeks to develop and are seen for all of a few seconds.</p>

<p>In traditional MMO games of course the catch is that you pay as you go, either per month or as a free to start but pay to have fun design. Unless the gameplay itself is rich and non-repeating the game company must continually develop and ship new play areas or risk losing customers to the next big thing. Our gameplay is fairly non-repeating and addictive but with our small staff it&#39;s impossible to produce enough new content to grow the playerbase. As time passes the content becomes stale looking and no game can attract new players no matter the gameplay.</p>

<p>So content costs a fortune and you can&#39;t build enough of it, you can&#39;t ship it easily because it is too big, and customers always want shiny new things as if they were human Magpies. What to do?</p>

<p>One choice is to focus on games which require little content, especially art, such as the little games I am working on for the Apple App Store. That isn&#39;t much of a solution for most companies though. </p>

<p>Another choice is to build a game where the customers do most of the content creation, like <a href="http://www.minecraft.net/">Minecraft</a>. It isn&#39;t easy to design such a game but if you do it well the game can grow for a long time. Even then you need to continually improve the game itself and what it allows the customers to create but that is much easier from a cost standpoint than creating the actual content itself. Games which allow customer modding and include level design editors also extend the life of the product, though rarely does that result in additional income for the original developer unless it spurs more sales.</p>

<p>A final and far more complex choice is to write code that generates content either upfront or during gameplay. This is an area still far from mature which many people fear will result in either fewer content creator jobs or bland content. Given the insatiable demand for game content and the inability of the industry to support ever increasing budgets for games automation and generation is going to happen eventually. As in the early industrial revolution hand crafting of everything has to give way to using more machines and leaving the rich details to the content designers.</p>

<p>That paragraph scares the pants off of lots of people in the industry.</p>

<p>In the next article I will generate more content on the details (but I swear it will be hand crafted) and what I think can be done.</p>
]]></description>
</item>
</channel>
</rss>