wtamatap

Quantity vs. Quality of Requirements

noreply@blogger.com (Mick) — Sat, 17 Oct 2009 11:53:00 +0000

I have had to get better lately at writing requirements. In a recent project, there were two mistakes that came about because the requirements were not robust enough. The mistakes were caught late in development, and the problem was that the developer could not complete the requirement as stated. Ironically, it was the developer that had not thoroughly considered the requirement when agreeing to the task, and postponed all thought about it until it was too late. We had to notify the customer that we could not deliver what was agreed to.

It is well known that the requirements must be reviewed carefully with the customer to ensure their desires are well represented. The requirements must be reviewed with equal care with the entire development team, including third party contractors, to ensure that the requirements are technically feasible. Testers can bring clarity early, and developers often connect gaps in the requirements from their own familiarity with the systems.

It had not been my intention to produce requirements in a vacuum, but it worked out that way on a particular project. The third-party contractor reviewed the requirements, gave an estimate for the work and then, five months later when we finally moved to develop the project, the contractor could not deliver. It was like they threw a number out at us without considering how they might solve the problem.

So I am instituting explicit reviews to give people an opportunity to challenge the requirements. It had been my habit to distribute them and solicit comments, but that's too passive. I have to challenge people to challenge the requirements.

At the same time, I've needed to produce them faster, and so I have found myself cranking lots of words in short, focused times. This is usually at the end of the day when my goals for the day are at risk, and I need to produce. I am simply surprised how much I get done in the last forty-five minutes before I leave, but other people are winding down, and the interruptions stop, and I can sit and think.

I wish I had an office with a door, but at least I don't sit outside the men's room, where it's busy, noisy, and smelly.

Ogres Have Layers, Like Onions

noreply@blogger.com (Mick) — Sat, 10 Oct 2009 01:03:00 +0000

I use Ubuntu on my laptop. Specifically, I use Xubuntu. I really like, but on rare occasions, I need to use Windows to get to Internet Explorer so that I can make a remote desktop connection; specifically, it's making a Terminal Services connection, and that just won't run under Linux.

In the past, I have used rdesktop on Xubuntu to connect to a Windows box on my network, but I can't use this here to cross the firewall involved. But I could then leap from my Windows box across the firewall to the desktop I desired.

I decided to cut out the middle man, sort of, and installed Windows under Sun VirtualBox on my Xubuntu laptop, and it worked great. It took the better part of a day to get all the service packs installed, but, now that it works, it's a very cool bit of technology to fire up the Windows virtual machine from this Linux laptop, and put it into full screen mode, and then use Remote Desktop from there to cross that firewall using Terminal Services.

It's a little slice of cool to put all of that together. It's one fairly amazing technology piled on top of the other. Like the layers of onions.

A Bigot's Approach to Web Development

noreply@blogger.com (Mick) — Sun, 04 Oct 2009 13:56:00 +0000

I have mentioned (at length) that I am working on an application for Facebook, and using Dojo Toolkit for the browser interface. It works great in Firefox. I did all my development and unit testing in Firefox. Dojo Toolkit is supposed to handle the compatibilities between browsers. That's really great when it works.

I have also mentioned that the Dojo Grid does not render correctly in IE 6 or 7. Like an idiot, I never tried it until one of my Facebook friends mentioned that it doesn't work. So I have finally checked it out with IE8 and discovered, much to my horror, that the grid doesn't render at all.

I scoured the Dojo Toolkit website for clues, and I did find a couple, but nothing that applied to my situation. I'm hoping they magically release a new version but, in the meantime, I'm going to release a very different version with simpler controls. It breaks my heart a little bit, but I need to get something released.

What I've learned is that I would have been much better off paying attention to the browser of my user base (IE -- still 70%) and not the one I like (Firefox). I was bigoted; but I've got that out of my system.

The Ways of Analysis

noreply@blogger.com (Mick) — Wed, 23 Sep 2009 10:53:00 +0000

I have a variety of project at work, and so I must use a variety of techniques in conducting my analysis. What pangs me is that not all techniques are as fun as others. Here is my rough draft list:

New Solution
Modify Existing Solution
Implement Existing Vendor Solution
Implement Off-The-Shelf Solution
Implement Hosted Solution
Modify Hosted Solution

That's fairly reasonable order in terms of fun. The most fun is to solve a problem with a brand new solution, the least fun is to request a fix or change to something completely out of your control. It's hard to make your needs known and to schedule the solution. That's just an all around pain.

When you're solving a problem that, at least where you work, has not yet been solved, you have the thrill of invention combined with the discovery of a business domain. You are Edison and Magellan, rolled into one. If you're very lucky, you get to be Patton, as well, and lead the troops to victory.

Of course, the "solution" itself has a range of fun-ness. Some tiny, pain-in-the-butt utilities are no fun when only two people use it, and they don't even like it; it's difficult to get those modified when everyone would rather ignore it, and use something else.

I happen to be working in the realm of content management at the moment, so that's a reasonably fun set of solutions. I ain't complaining. If I were to complain, it'd be because I'm not able to use my Cooper training at the moment, or to program in Python.

Command Line Blogging

noreply@blogger.com (Mick) — Wed, 02 Sep 2009 10:32:00 +0000

Occasionally, my creative side and my technical side complement each other. I am preparing to write a long, long story—a work of fiction. I intend on serializing the story on a blog, but I don't want to sign into Blogger.com and leave all the original work there, in its system. It also doesn't feel right to just cut and paste (although, really, it would be simple enough). I want to have the text on my Ubuntu laptop where I can be responsible for it.

My first thought was to post the blog entries directly from VIM, but the nice person who figured out how to do it did not post his work anywhere that I can find it. It really used Python to talk to google and blogger, which was fine. There is also a version that uses Ruby in a similar fashion, but I don't want to get started with Ruby at this moment.

So I found out that there is a Python recipe for talking to blogger, and I may need to wrap that in some command line options, but I'll use that to post entries to Blogger.com. I can even do it from vim with the good old bang (!).

Blog Post From the Desktop

noreply@blogger.com (Mick) — Tue, 01 Sep 2009 02:12:00 +0000

For a variety of reasons, I am interested in posting to this blog from the desktop. Ideally, I'd like to create text files that I store on my hard drive, then post them to Blogger.

I looked at vim-blogger, but it requires VIM with Python compiled into it (I suppose I should just do that and quit goofing around).

I was very hopeful for TomBoy with its Blogpost plugin, but it has not been able to connect to Blogger.

Any ideas appreciated.

Increasing Timeouts

noreply@blogger.com (Mick) — Fri, 28 Aug 2009 10:32:00 +0000

I made a silly mistake the other day with a database query. On my test system, it took a rather long time. Three Minutes. Three minutes are an eternity if you're waiting for a database query. Minutes give you time to think that your entire career has been a mistake, and you should have entered the priesthood. Three minutes gives you enough time to mentally undress everyone in your cubicle isle, and then you wish you'd stayed in the priesthood fantasy.

In spite of the long run time for this query, it actually seemed reasonable to me, so I moved it into my production system. I thought that, because the production hardware is more robust than the test system, it would run faster. I was wrong. In fact, it took longer. In fact, it took ten minutes and only stopped because of the timeout setting on the web server running database queries. In ten minutes, if you're waiting for a database query, you can question the validity of the U.S. Constitution, and wonder if the Founding Fathers, some of them slave owners, drunkards, or adulterers, were reasonable human beings with a legitimate right to create a new form of government. But ten minutes is also long enough to realize that, given similar circumstances, you'd probably own slaves, drink too much if only to avoid the diseased and putrid water, and sleep with anything that wanted to sleep with you, so maybe the separation of powers in the federal government in fine just the way it is, especially due process and double jeopardy. (You never know when you might incriminate yourself after an arrest, and risk standing trial twice for murder.)

I decided to consider the query once more. It was, in fact, three queries in parallel, so I ran each one separately and found the offending code. I compared it to the others that ran so much faster, and realized I was missing one of the where clauses to limit the amount of data. This query runs against a single table with sixty million records in it. I added the clause and, lo and behold, the query ran in a single second.

I am somewhat embarrassed at the speed improvement. This is a hey-your-fly-is-open kind of moment, and oh by the way, I'm not wearing underwear. I'll get past it.

I went to make coffee after that, and I had to brew a fresh pot. It takes four minutes to brew fresh coffee. Four minutes! You know what I can consider in four minutes? I questioned the Emancipation Proclamation, the Declaration of Independence, and the Magna Carta. But my reasoning will be revealed in another post.

W8 Monitr is dead -- Long Live W8 Loss

noreply@blogger.com (Mick) — Thu, 27 Aug 2009 10:53:00 +0000

This is a notice that my interactive web site, w8monitr.com, will no longer be available at that address. I'm pulling it down for re-tooling.

However, I have also had a series of articles there that discuss my weight loss, nutrition, and healthy lifestyle. Those I want to persist, and they'll be available over at MickeyHadick.com in the articles section.

Dojo Toolkit Grid Failure in Internet Explorer

noreply@blogger.com (Mick) — Sat, 15 Aug 2009 11:27:00 +0000

For my facebook application, I use a grid embedded within a border container. I use several grids, in fact, each on its own tab control. I have been developing and testing with Firefox exclusively. I invited a friend to try the app, and the crazy knucklehead discovered that it fails upon opening. He can't touch the thing, and it turns out he uses IE.

If I refresh the web page, it actually works fine. However, each grid fails consistently the first time. It's not like we can ask people to immediately refresh the page the first time they visit.

I checked the dojo forums, and there are others with the same problem. They have suggested various solutions that worked in their case, but none of those have worked for me. Anyone have any experience with the dojo toolkit grid failing with IE?

Please let me know.

Classic Debugging Situation

noreply@blogger.com (Mick) — Sat, 25 Jul 2009 16:30:00 +0000

I released a new version of List Lust on Facebook. One of the new features is a ready-for-print layout of the data, and it fails in one, isolated set of circumstances. The real trick is that it's difficult to get the log running on my webserver, so I don't see the error message directly; just the 500 error displayed in the browser.

But because the error scenario is isolated, and can walk through the code with what I know about the data in mind. Wish me luck.

VIM and Tabs

noreply@blogger.com (Mick) — Mon, 20 Jul 2009 16:06:00 +0000

This is just a short homage to a great page that reminded me of my ignorance: Vim Tips. I use gvim, and I use tabs a lot, but I hadn't gone beyond any defaults to make it better. This web page has it all.

Python and CSV Processing of Unicode Text

noreply@blogger.com (Mick) — Sat, 18 Jul 2009 12:13:00 +0000

I had a cool moment this week in that we had a challenge at work to process a file of comma-separated-values with one million records in it. Each record had twelve values, and we only needed the eighth one. The fact that it had over a million records made Excel quit when we tried to use that. I volunteered to extract what we needed using Python.

I happened to have Python 2.5.2 installed. I knew there was CSV module available, but had only used it once, and that was to create a CSV. A quick search revealed this web page. I also perused EffBot.org, which was even better. But I immediately hit a snag because the source file turned out to be Unicode.

The Python docs have an example on how to deal with Unicode, but I a) didn't know the encoding, and, b) preferred if it wasn't in Unicode at all.

To figure out the encoding, I used this web page, which was nothing short of brilliant. I was able to list out the BOM using a simple read of the first four bytes, and deduced it was UTF-16.

I then used the other examples to encode the text as simple ASCII while reading the CSV, and I was oh-so-happy.

Because this was bottom-up software development, I had to wrap the small chunk of code in something manageable, so I used this as an example, although there are plenty of others. This was a nice skeleton example with command line arguments. We expect more of these files, so I have to manage this on a regular basis.

I finished it off by having the program create new files of no more than 100,000 records, and just included the one value we want, and included a heartbeat so that I knew it was working.

VIM and Me

noreply@blogger.com (Mick) — Sun, 28 Jun 2009 16:20:00 +0000

Have I mentioned that I returned to my roots by forcing myself to use VIM. It's really not a burden, but did challenge me to recall long forgotten thought patterns. Your brain works differently when you use VIM, and that's what I wanted to experience again.

One of the tools that kept me away from VIM was the Programmer's File Editor (PFE). It had a number of cool macro features that are straight forward and highly effective. However, VIM has a macro feature too, and I'm enjoying it.

There are better tools out there, and maybe UltraEdit is better overall, but I kind of like being a Unix Zealot again, and using VIM as much as possible is a big part of that.

The real point of the post is that, among many great tutorials on VIM, here is a nice, comprehensive one: Using VIM Like a Pro.

Making my Facebook Application More Viral

noreply@blogger.com (Mick) — Fri, 26 Jun 2009 12:17:00 +0000

I have basic functionality in my facebook app, but the friends to whom I've suggested it don't use it. It seems that it flat out is not infectious. I had harbored dreams that it would be like those "Living Social" applications. Just before I started inviting my friends, the "Top Five" application was released, and I definitely had that sick feeling in my stomach about because it worked well, was cute and clever with the pictures, and moved from friend to friend like a cold through a children's daycare center.

What I'm working on now is the newsfeed features so that, if any of my friends ever do use it, it will be streamed on their wall. I'm hoping that may encourage other users.

Nested facebook screens

noreply@blogger.com (Mick) — Fri, 12 Jun 2009 11:11:00 +0000

My iFrame Facebook application suffers from nested screens on a few occasions, and I haven't been able to fix that yet. I don't understand enough of the subtleties of HTML, iFrames, and anchor to know what is going on. And the way Facebook embeds your application deep within their DOM structure, I'm at a loss.

So a call to the lazy web: anyone have any luck with this. If you want to see what I'm talking about, add my application at:

http://apps.facebook.com/toptenbeta/

and when you are through, my app will be nested in a facebook double-image. Refreshing the browser makes it all good.

The way I invite friends does the same thing.

I need help.

Multi-Friend Selector

noreply@blogger.com (Mick) — Wed, 03 Jun 2009 10:50:00 +0000

I had said I'd post my multi-friend selector code, and someone finally asked, so here it is:

http://wiki.developers.facebook.com/index.php/Multi_friend_selector

I'm delivering it directly from my web server, rather than messing with KID.

I'm going to figure out how to do it entirely in javascript, next. That should be better.

Also note that the xd_receiver is being served from a sub-directory of the main web server.

Rendering a Multi-Friend Selector in a Facebook iFrame Application using XFBML

noreply@blogger.com (Mick) — Thu, 02 Apr 2009 22:59:00 +0000

I struggled with the problem of rendering a multi-friend-selector in a Facebook iFrame application. In fact, it was stopping me ice cold in my project, because I couldn't translate the PHP examples into TurboGears, Python, and DojoToolkit. Well now I've done it.

The most embarrassing part is that I ultimately stepped through the XFBML introduction on Facebook to make it work. It was really all there, I just had to remove my fear of the unknown and persist. I made it a bit easier for myself by serving up the HTML directly from my TurboGears controller using Python, and that helped me work through it.

It also depended on the Django example of Facebook integration to encode the embedded XML within the text.

If interested, drop me a line. I'm not publishing it here only because it's a lot of code, and will not look good. I'd rather put it on the Facebook developers wiki.

The Big Celtx Workaround -- WebDAV on Xubuntu

noreply@blogger.com (Mick) — Mon, 23 Mar 2009 12:05:00 +0000

My next big problem is using Celtx 2.0 on Xubuntu. It works pretty well, but the feature needed to typeset the script into industry standard format needs to be online, and that is not working on my Xubuntu system (a problem shared with other Linux and Windows users). It is vexing and stubborn.

So I'm attempting to install WebDAV to create my own online repository. I'll use my laptop to edit, and when I need to print, I'm going to copy the script to a web site with WebDAV and use a different computer to access it and print.

Silly, but it'll get me past this problem. I've lost about four days spinning my wheels on this.

I'm posting the instructions I'm using for posterity sake (to help me remember in the future). This is what I used....

http://www.linuxquestions.org/linux/answers/Applications_GUI_Multimedia/Setting_up_davfs2_with_the_Ubuntu_package

and

http://www.howtoforge.com/davfs_ubuntu

The First Big Workaround

noreply@blogger.com (Mick) — Sun, 22 Mar 2009 11:56:00 +0000

I'm struggling with my two main interests right now: the web project for Facebook, and using Celtx as scriptwriting software for Linux (Xubuntu).

The first one is my problem in figuring out how to invite friends to use my (wonderful) application on Facebook. There are suggested ways, but I need a solid example on how to do this for an iFrame application. Nothing is complete, and I haven't the time to figure out the missing pieces. I'm really quite desperate on this one.

How desperate? I am asking for help here on the internet. That's how bad. I'm sure that if I had a job I didn't really care about, I'd take a few days off and work it out.

If you have an example on how to render a multi-friend selector with an invitation to use a Facebook application in an iFrame, do let me know. Thanks in advance.

When Persistence Becomes Silly Obsession

noreply@blogger.com (Mick) — Sun, 15 Mar 2009 11:47:00 +0000

I noticed that it has been exactly six months since I declared in a posting that my web project was beginning to take shape. It really is, now, starting to take shape, but I made a huge detour in making it Facebook compatible.

From a strictly programming-technology point of view, the project uses Dojo Toolkit to present the interaction framework, which includes tabs, grids, accordion things, drop-down tooltip dialogs, and modal dialogs. Those elements use AJAX to talk to the Python and TurboGears server with a PostgreSQL database, which is now hosted on WebFaction. It does all of this in a Facebook iFrame. Oddly enough, I now feel like I'm more of a JavaScript programmer than I am a Python programmer, and I really miss Python.

My final mystery is figuring out the XFBML necessary to create an "Invite Friends" dialog. I really think I want to do it with Facebook's AJAX object, but I haven't found great documentation on it. There is some documentation, here, though. So I'm going to give that a try right now.

Persistence of the Programmer

noreply@blogger.com (Mick) — Fri, 27 Feb 2009 01:55:00 +0000

I haven't blogged in a month, but that's because the few scant moments I can spare for this hobby I have been struggling with several Dojo features. The most troubling has been drag and drop (DnD).

As is often the case, the simple examples work great, especially those using declared, rather than programmatic, HTML. They drag, they drop, and everything is great. But when I tried to make things happen they way I wanted them to, to implement the functionality I wanted, using javascript and the Dojo libraries, and the fancy-schmancy creator functions, errors are getting thrown left and right.

So here is what I learned about Dojo DnD: you have to destroy any existing sources before creating a new one to attach to the same DOM node. In some program architectures it wouldn't matter, say, if the web page was reloaded all the time, but my approach is a single page and javascript does all the interaction. That one source was hanging around, and trying to reuse the DOM node failed.

It took me four weeks to figure that out as I experimented with different techniques, and eliminated possible causes. I'm not really proud of my accomplishment, but I am surprised at my own persistence.

Dojo is really a fun approach to web programming. I'm using the grid, tabs, border container, tool tips, dialog boxes, and DnD, all jammed onto a single HTML page. It's probably a mess, but because I wrote it, I can still find things.

Dojo Toolkit: ItemFileReadStore vs. QueryReadStore in a Grid

noreply@blogger.com (Mick) — Sun, 25 Jan 2009 18:02:00 +0000

My web project uses a Dojo grid backed by a data store. The data stores calls a web service (a URL) to retrieve data. The data store passes it to a model (dojox.data.grid.DojoData) which then lays out the data for the grid. I had a heck of a time getting the grid to refresh its data when the server data was updated.

Given a grid with a JavaScript variable name of "grid", I tried grid.refresh(), but to no avail. I then tried grid.model.refresh(); again, nothing. I also tried grid.model.store.fetch(). Still no update to the grid on the screen.

The books and articles make it sound like all the plumbing takes care of everything, and it will notice these things. In fact, the grid examples often use the ItemFileReadStore to fetch the data. However, as this article explains, ItemFileReadStore will only fetch once. It's nearly impossible to force it to go back to the server.

I pursued the suggestion of using QueryReadStore instead. By watching the server traffic, I confirmed that grid.model.store.fetch() did retrieve the data, but again the grid view was unchanged. Finally, I added grid.model.refresh()and the grid view was correct.

It took me about a day of exploring to get it right. I hope I never, ever forget.

By the way, I'm reading "Mastering Dojo" by Gill, Riecke, and Russell. It's quite good, and if I had more time to concentrate an what they wrote, would answer all my questions.

Dojo Toolkit Examples on DojoCampus

noreply@blogger.com (Mick) — Fri, 23 Jan 2009 11:55:00 +0000

I was struggling mightily with Dojo, specifically to get a TabContainer to work inside a BorderContainer, and even before that, to get just a BorderContainer to display correctly. It was a mess, and entering the example provided in the book of Dojo didn't do much. That was very annoying, and made me wonder if I had gone down the wrong road.

Enter google. Persistent searches for examples brought me to the DojoCampus.org, specifically the Explorer, where I found some examples that worked great.

And just like that, I'm happy and productive again, troubled only by own lack talent.

Goal-Directed Design: Analyzing the Interviews

noreply@blogger.com (Mick) — Thu, 22 Jan 2009 00:32:00 +0000

After the interviews, the next step in goal-directed design is to analyze them for the underlying behaviors and characteristics. Based on the interview strategy, and what was discussed in the interviews (especially unanticipated characteristics), determine up to 25 characteristics common to your interviewees, but which differ in degree.

For instance, all your users have an age, so that is the characteristic, and you would plot their different ages on a horizontal scale.

AGE:
|--------------------------------------------|
Young......................................Old

Your users have a certain amount of experience using computers:

EXPERIENCE:
|--------------------------------------------|
Little...................................A Lot

If the educational backgrounds of the users is pertinent to the solution, plot that as well:

EDUCATION:
|--------------------------------------------|
High School...............................Ph.D

And so on. It is the particulars of the interview pool and the questions asked that determine the specific data items.

Once all the interviewees have been plotted on these scales, you then look for patterns. As clusters emerge, the characteristics of the archetypal user is defined.

The next step is to define the primary and secondary personas, basing those personality traits on the behaviors and characteristics determined by the patterns discovered in the interviews.

It is the strength of these personas that give greater strength to the design.

TurboGears, Visits in the Identity Module, and Facebook

noreply@blogger.com (Mick) — Mon, 19 Jan 2009 12:29:00 +0000

For the record, one of the problems I was experiencing with creating an iFrame application with TurboGears and Dojo Toolkit for Facebook was that the Visit cookie was not being sent. It took me weeks to realize this, and I had simply ignored it for a while to work on other things, but finally had enough functionality to really need to solve that problem.

A little background for the search engine crawlers: the Identity module for TurboGears generates a unique Visit identifier, and that must be sent as a cookie from the browser to associate the user with the login. It preserves the state of the login, essentially, with that cookie. I am trying to run an application in a Facebook iFrame, and could not preserve the state of the login. It's also very, very difficult to debug this, even with FireBug.

I suspected that the cookie with the Visit key was not being sent, and this led me to the APP.CFG settings, where I realized the settings were wrong--but fixing the settings did not fix the problem. I did more research, and found the answer in docs.turbogears.com. The answer, as it often is, was there the whole time.

I could manually send the Visit key in each GET or POST request as a parameter. I needed two settings:

visit.source - set it to "form" (or "cookie, form")
visit.form.name - set it to the name of the parameter to be sent ("tg_visit")

I now include that parameter on every request, and get the key in the KID template by referencing:

${tg.identity.visit_key}
${tg.config('visit.form.name','tg_visit')}

I set those to JavaScript variables, and use them with everything sent back to the server.