Liquid Maze

Simplest way to logging

2008-07-28T18:43:00+00:00

I use NLog for logging. I used to keep Logger property in every class I needed to log from and then have Winsdor container inject it with appropriate logger.

All of that changed when I switched to AutoFac and I wanted to simplify working with logger, especially since not all classes I want to log from are managed by the DI container.

The best solution I came up with is to use extension methods in .NET 3.5. Here's the code:

C#-Code: Logging extension method

using System; 
using NLog; 
namespace Logging 
{ 
    public interface INeedToLog 
    { 
    } 


    public static class LogExtensions 
    { 
        public static Logger Log( this INeedToLog needToLogObj ) 
        { 
            var type = needToLogObj.GetType(); 
            return Logging.GetLogger( type ); 
        } 
    } 


    public static class Logging 
    { 
        public static Logger GetLogger( Type typeToLogFrom ) 
        { 
            return LogManager.GetLogger( typeToLogFrom.Name ); 
        } 


        public static Logger GetLogger( object objectToLogFrom ) 
        { 
            var typeToLogFrom = objectToLogFrom.GetType(); 
            return LogManager.GetLogger( typeToLogFrom.Name ); 
        } 
    } 
}

I simply inherit from INeedToLog from any class in which I need to log. Then extension method Log becomes available and I can do something like this.

C#-Code: Log usage

this.Log().Info( "Info text" );

This method for logging might be intrusive into interface hierarchy, but it's very simple to set up, even if you don't use DI. I like that!

Re-blogging

2008-06-29T13:06:50+00:00

Wow, people are still tuned to my blog, even though more than a year has passed since last time I blogged. Well, I'm back, but I don't know how often I'll write new posts.

I moved to a new blogging software (BlogEngine.net) and renamed blog just to Liquid Maze (to keep it simple). The old feed will continue to function, but a new feed is available now.

I'll probably write when I feel like it instead of trying to adhere to some kind of schedule. That didn't work out too well for me. Nonetheless I'll try to keep the quality of content high.

So, stay tuned...

Do less

2007-05-16T21:20:00+00:00

I just read Getting Real book and I really liked it. It was written by guys at 37 Signals, creators of Ruby on Rails and a series of successful online services. The book is about how to be more practical by staying simple and focusing only on currently problems. Among other things it shows that by doing less you can be more productive.

In other words most of the crap we are doing, or I'm doing, doesn't really matter and has no real effect on my life. I agree with that. I knew about the concept for a long time, but was never able to do much about it. After reading the book and thinking about it I'm starting to get it (maybe).

Doing less is doing just enough, not everything. There's no perfect or complete state.

People often talk about Pareto principle. In the context of this post it means 20% of all actions produce 80% of the value. According to it I just need to focus on that valuable 20% of the actions. I always understood this principle, but could never apply it practically. It's an interesting concept, but I must be missing something.

Solving only problems at hand seems more practical and easier to do. When I first saw the idea I thought, "Don't I always try to solve problems at hand?" As I read and learned about it, I realized that, no, not really. I make several mistakes.

Try to optimize my time by solving problems early. Cost of most problems grows over time, but most of the problems I think of never happen or are completely different in reality than I thought they are.
I act on what's urgent. In many cases urgent isn't important, but I'm afraid to lose an opportunity. What I came to realize is that many things that appear urgent, are not urgent at all and can be done at any time.
Do something without understanding of what I'm trying to do. I try not to do that anymore :)

I don't fully agree with the book on solving problems as they appear. We can anticipate for them to happen and do something early to make it easier to solve them in the future. In a way it's prevention instead of solving something that isn't real yet.

With software development I try to follow good design practices to make my code easier to understand and maintain. When I don't, I usually need to go back and change it later on, thus increasing the total cost. If I don't separate concepts and make sure my classes are independent it's hard to replace an algorithm later when I run into performance problems or switch to database from using files. Speaking about performance, premature optimization can be bad, but if I don't think about it at all until I'm having problems I can push myself into a dead-end and have to rewrite large portions of the program.

Normal life (like I have any) is the same way actually. I can't solve all my health problems, but I can help prevent them and make it easier to handle them if they appear by eating healthy and exercising.

Don't solve problems you don't have, let that which doesn't matter slide, but don't completely close your eyes. Problems will appear. Be flexible, expect the problems to happen, expect that you will have to change your code. Most importantly, be smart about it. It's possible to go too far with anything, prevention including. Be practical and stay real, and do less, just what's really important.

Estimation, Part II

2007-05-08T17:41:00+00:00

In the last post I talked about estimation points and why they can result in more accurate estimations. In short summary, relative estimation is simpler and more accurate. That's what estimation points are, relative size of one task compared to the other. How long it takes to complete each point needs to be approximated from several initial iterations. Today I want to introduce additional techniques to make estimation easier.

It's usually easy to see if one task is twice as big as the other. If another task is 1 point, the twice-as-big current task is 2 points. What if the current task is about 10, 20, 100 times larger? It's impossible to say if it's 100 or 101 points. It's hard to distinguish between 20 and 21 and even between 10 or 11. It's also not important because estimations are inaccurate. 10 or 11 are close enough for errors in estimation to make it possible for 11 points task to be smaller in reality than 10. By making a task 10 or 11 we are just guessing that it's somewhere in 8 to 15 points range.

To make things easier we can group different ranges to hold all items close in size. For example all tasks in 8 to 15 range will go into 10 group, 15-25 to 20 group and so on. Two series of groups are popular - based on Fibonacci sequence (0, 0.5, 1, 2, 3, 5, 8, 13, 20, 40, 100) and on 2^n sequence (0, 0.5, 1, 2, 4, 8, 16, 20, 40, 100) or something along those lines. Around 20, groups become 20, 40, etc. in size. It doesn't really matter if it's 20 or 21, but it's easier for people to think in rounded numbers.

I tried to use this system with Sider. It was easy and fun. I didn't feel anxiety to say if something is 8 hours or 9 hours. Instead I just chose appropriate group. I also didn't have to guess if my hour is really an hour. I knew I can figure it out later.

The problem for me was that I couldn't really switch to thinking in points instead of hours and days. I almost automatically decided that 8 points is 8 hours - 1 work day. I ended up comparing each task to something that I thought would take a day or an hour.

At this stage I've no idea if my estimations are accurate and when I will complete the next Sider release. In several iterations I hope to have a better picture and a better idea when the release will happen.

I also want to mention teams. Most of the time people work in teams and they need to give estimations as a team. I usually work alone so I can't really share any experience or talk about related problems. I suggest you check out Planning Poker, a procedure and a game created to help people overcome frequent problems and give accurate estimation together with their team.

To learn more about estimation I highly suggest listening to Agile Estimation podcast.

Estimation

2007-05-02T15:33:00+00:00

People need to estimate as part of planning. It can be time (tasks), money (budgeting) and so on. The point is to understand the value of something. We can then decide if it's worth the time or money or in what order should we do/get things. When working with tasks we focus on more valuable actions first and if there's time we complete the rest later. We also want to know when a project will be done.

So, software developers are often asked to estimate. We often miss and underestimate. Why? Here's my list of the possible reasons. There are many books written on the subject. I have hardly read any of them so I probably miss something.

We've never done it before, so we have no idea how long it will take. We just guess.
We lie, because we wish we would have been better developers and could develop it quicker, or because we want to tell somebody what they want to hear, not our opinion.
We fail to fully grasp the scope. We estimate what we see, but there are a lot of hidden additional work.

In the first case it's just a matter of experience, but I don't think this is a big factor. In either case there's not much we can do about it. I'm stuck with whatever experience I have.

In the second case, we just need to learn to be honest and face the truth, even if it's not pleasant. It might be hard to acknowledge the problem and do something about it now, but later on it will be more expensive and more unpleasant. This is applicable to all aspects of life. People hide from reality, hoping that the problem will go away on its own. It might, but if it doesn't, it grows like a weed.

We can actually do something about the last case. When we make mistakes in estimation we are consistent about it and we can compensate for it. That's exactly what an estimation point system and velocity are designed to do in Agile Development.

While listening to Agile Estimation podcast I finally understood how point estimation works. Each point is just an abstract unit used to measure relative size of the task. It assumes that it's easier for people to estimate relative size of two objects or tasks when they compare them side-by-side than to estimate an object on its own using standard units (time, length, etc.) The size of 1 point is unique for each project and each team.

When estimating we don't know yet how long it will take us to complete each point, but we can say that 4 points task is twice as large and will probably take twice as long as 2 points task to complete.

Velocity is how many points are completed during each iteration (a week, 2 weeks or however long an iteration is). At first we don't know how big it is, but after a few iterations we can look back and guess how many points we will be able to complete in the future. This makes it possible to project when the project will be done, while compensating for estimation mistakes.

There's more I want to say about estimation and I will continue writing about it in my next post.

Understanding Sider

2007-04-20T19:49:00+00:00

I've always wondered how to present and market Sider, because I sometimes have a problem defining it to myself. Is it a tool to build tools, or a platform to work with information, or both, or something else? I'm having this problem because in a way it's idea is new and unique, but I also don't have a lot of experience in software development and design. My partner, Igor, who came up with the idea, understands the project much better, but I'm the one who's doing most of the work and marketing.

Taking a fresh look at the design I see it as an engine, similar to a game engine. Instead of dealing with graphics and animation, it deals with information, logic to process it and views to show it. Through modules and packages it can support new types of information and new ways to deal with it. For example a module can allow users to manage a software project using Agile practices, or store and organize notes, and so on.

What's the difference between a platform and an engine anyway? It's the scope they deal with. A software platform is a set of technologies used for building a large variety of applications. An engine focuses on a specific type of problems and many common activities associated with it. In both cases software runs on top of the platform or an engine. A platform is just a broader and more abstract engine.

What's the best way to present this idea to the user? Without extensions Sider is useless to most people, just like game engine is useless to players without a game on top of it. Game developers usually market games to regular people and engines to other developers. Many players are not even aware of the engine and that it can run other games.

This creates a problem with Sider, especially since it's fresh and hasn't been proved yet. Many users won't care that it's a flexible engine. They're interested in concrete solutions, not in potential. At the same time I always wanted to charge for the use of the engine, not extensions (at least initially). I hope this will help Sider to stand out as new extensions will become available for free. I will need to make people aware that it is an engine and that's what they are paying for.

In addition anybody can extend it. History shows that some users (power users I call them) will be interested in customizing it and adding new features. Most people will just want to use them, but extensions created by power users are important to Sider success.

I think initially, it's best to focus on promoting extensions and scenarios they support, just mentioning that more will be available. When they are available, expand marketing to include them. In time, after several iterations and adding new extensions I can try to expand and capitalize on the fact that there will be support for a large number of things, because in the past we have added many.

Back to blogging and Sider

2007-04-16T16:58:00+00:00

I'm back to blogging. I'm gonna start by changing how I write posts, making them more un-official, like a journal. Not sure yet how that will work out.

I'm also back to working on Sider. I decided to make its development process more transparent. I want to be open and honest about what I'm working on, what mistakes I make and how I fix them.

I'm in the process of putting all project documentation in one place on the wiki where it's available to anybody. I started to review and publish Sider design documents. That shouldn't take too long, at least for the initial phase where I create an outline of all features. I'm not sure yet if I will publish everything, but I'm sure that most of it will be online. The wiki is opened for registration, so anybody can contribute. I have also opened read-only access to Sider issues.

In addition I recently learned about Scrum and want to try it as a project management methodology. I'll have more user stories (which I seem to like, unlike the use cases that are too detailed for my taste) and I'm going to estimate all remaining work using abstract points, then use velocity to approximate when the next release will be available. I will probably write here about my experience and thoughts.

Taking a break

2007-02-19T21:39:35+00:00

I haven't written at the blog in a while. I realized that my interest has faded. I've been trying to push myself to create new posts, and that's just no fun.

I'm going to take a break from it. I don't feel that I'm done with the blog, so I'm expecting to get excited about it again. I hope it won't be longer than three or four weeks. Maybe more, maybe less, but I don't really know.

Feel free to stick around if you're OK with waiting for a little bit for new posts.

Thanks to all people who have being reading my posts for their time and support.

EH added to Mindjet Research Accelerator

2007-01-18T20:57:00+00:00

You can now use Mindjet Research Accelerator to search through EH forums and blogs. Research Accelerator is a plug-in to allow people search for information using different web services from inside of MindManager, MS Office applications and IE. You will need to download and install plug-in, and then add Community Server Research Service.

I'd like to thank Michael Scherotter from Mindjet for inviting and adding us to the list of supported websites.

It's nice that Mindjet is working on providing wider support for finding and importing data. It can make you more effective at finding and integrating additional information with your documents.

It would be really cool if imported information could be updated dynamically. However, integrating updates and doing version control in an application where you can edit data is hard. The problem is similar to what source control programs (SourceSafe, CVS, SubVersion, etc.) try to solve. Information in them can be changed in several places and it then needs to be merged in one place.

There are many applications specializing at gathering information from many sources (news aggregators, for example). Most of them are web-based and can gather and show news, blogs, etc. on a single page. Yahoo is probably the most popular service. The problem is usage. These applications can gather information, but they don't allow user to edit it, customize it (add custom notes) or connect several related articles. MindManager is now one step closer to supporting such scenarios.

Blog changes

2007-01-17T02:51:00+00:00

I decided to change my blog format a little bit. I don't post as often as I need for the blog to be interesting, engaging and actually successful. Too often several weeks pass before I get to write a new post and I think by the time that happens, you lose interest in what I have to say.

My goal is to do 3-4 posts a week. Up to this point it was much less frequent because I turned blogging into a long and annoying experience.

It took me about 3-4 hours to complete one post. I wanted to completely cover the subject I wrote about, but in many cases I started out with an undeveloped idea. Thinking everything through and then writing a large post took a while and I made a lot of mistakes, which had to be corrected later. I guess, by making my blog topics large I lost focus, and doing anything without one is a hard, prolonged, and frustrating process.

I'm going to try and learn from my mistakes to post more often. I plan to limit time I dedicate to each post to about an hour. Idea is to make sure I don't feel like I'm spending too much time on it, and force me to write smaller, but more focused posts. This means I will often have to split larger topics into smaller parts, sometimes taking several posts to cover a single topic.

Existing tools that work with information

2007-01-03T18:45:00+00:00

In the last post I divided existing information management tools (mentioning some in examples) into either open or strict-type categories. Today I continue by describing what I think is done right and what I would want to see improved.

Done right

Open tools

I'm happy with open tools. Most of them have been in development for over a decade and are helpful with general problems. Microsoft has done a good job with its Office and there are several alternatives, such as OpenOffice. You can use Microsoft Word (or something similar) if you need to write anything (a report, an article, or a note). Other examples are Adobe, Macromedia (which is part of Adobe now) and other companies, all providing good tools to work with pictures and graphics.

These tools are useful in many information management scenarios, but in most instances, they can handle only cases with simple data structure. When amount or complexity of information goes up open tools become harder and harder to use and strict-type tool designed for specific problem becomes more and more appropriate.

Simpler design

Another thing I like is a general shift towards less complex design. Applications like this with less features are becoming more popular, especially web applications. This is a good change from old feature racing (where every program would try to have more features than competition), although I don't agree that cutting features out is always appropriate. I think Einstein's, "As simple as possible, but no simpler" is a good guideline in design. Eventually we might end up with applications that are too simple to be practical.

What can be improved

Strict-type tools

There exist a lot of strict-types tools that try to address specific problems. Many of them are good, but I think most need to be better. When the tool fits the problem, great, however it often doesn't. Designing and developing a good tool is difficult. It's not easy to understand the problem (because usually there's a set of problems to address) and to come up with the best solution (there can be many solutions, each with its plusses and minuses, and on top of that different people might have different preferences on how to do something). Finally it's hard to balance resources (both development and of the user PC). As a result, a lot of strict-type tools have good potential, but can see a lot of improvements.

All tools have common problems

Almost all tools have very similar problems. Few tool supports substantial source control (saving history of changes to see and compare document modifications through its lifetime). Usually there's limited support for collaboration and security. The list goes on, but the whole situation is improving.

Strict-type vs. open

Strict-type tools are often not strict enough on one hand, but on the other are too strict, making them harder to use. For example, wiki supports flexible structure to store notes in a graph, with one note being able to reference any other including its parent. They can be strict and often don't allow notes with the same names, their wiki-text is often too simple (creating tables is too hard), etc. At the same time, note (page) doesn't have any text structure, so it's impossible to enforce certain structure, where user has to fill out some fields for a new note.

What can be done

I don't really have a good answer. Designers need to always keep in mind user needs and make interface simple and intuitive. Things are moving forward. Every new generation of tools gets better. We are starting to see more flexible, context-based interface and support for larger, more complex problems. With Internet emergence demand for online tools is growing, and developers have to address that as well.

Our solution, so to speak, is Sider. We're trying a different approach, creating an environment to help with applications development. We believe in strong-type tools are the best solution for many information management problems, but at the same time developing specific tool from scratch requires a lot of effort. Sider addresses this by supporting higher-level languages to define logic, structure and interface, and to bind them together (through XML and browser technologies). It also promotes reuse of parts of existing tools through inheritance, composition, referencing and other mechanisms.

We also want to address a set of common problems I have mentioned previously. Since all tools will run on a single platform, it's possible to share horizontal features (like security or capturing document history) with all Sider-based tools.

Sider 0.5 Technology Preview release

2006-12-15T21:18:00+00:00

Whew, finally this version is finished and available for download. It took a lot of effort and much longer than I ever imagined. I've made many mistakes, but also learned a lot.

This is a technology preview release. It demonstrates the platform with most important ideas implemented. It's possible to define different types and views, but there are only few extensions available. Read release notes for details.

A little bit about us

Sider is the work of two people. My name is Slava Ivanyuk and I do most of the development. I write all code, documentation, organize design documents, etc. My partner, Igor, mostly helps by giving ideas and testing Sider. It was his idea to develop a flexible platform with potential to support any type of information.

What's next?

My next steps are to take a short break, do a post-mortem (maybe I'll blog about it) and start moving toward next release. We plan to improve Sider to the point where it has wider practical use. That means adding some features, but mostly focusing on creating extensions. Here's a list of few major things we will be working on. We'll finish adding more details to wiki roadmap sometime soon.

Move to .NET 2 - At least evaluate the move, but most likely we will do it. It offers many improvements over .NET 1.1, not only in general but also to work with XML. Eventually Sider will move to it and the sooner the better. My only concern is possible delay it might introduce. Most likely, we will move to it in several stages.
Persistent user options - Right now, most customization is done through extensions. We want to add application-wide settings.
Support packages - A package is a collection of different extensions. Each package would focus on providing support for specific scenarios, like note management, project management, collections, etc.

Upcoming packages

These are just a few ideas of popular uses we want to support with Sider.

Note management - There're many programs to work with notes, but none of them get it quite right. We want to have a flexible system that would support graphs (like wikis) of notes, tagging, filtering, etc.
Journal - People often use notes to track progress, to capture thoughts on the topic over time, instead of creating a system for referencing information. Journal will feature special notes, with views heavily oriented on dates and calendar.
Project management - Extensions to allow you put project plans, notes, documents in a single place. There will be support for several methodoliges, including agile (XP, Scrum, combinations), waterfall, etc. But main focus will be on making it possible to mix notes and other documents with tasks, issues, stories. There are often many ways to view project documentation: tasks and related notes, list of features and related tasks, etc. I believe some software (such as Trac) represent moves in the right direction, but they aren't quite getting everything right. Of course, they offer many options that won't be possible with Sider in the near future (online collaboration, source management integration, etc).

I also want to go back to Xelog. I like the idea behind it, but I don't like how heavyweight it ended up. I will need to fix that before moving on.

With the next post, I'll go back to discussing information management tools. I had some ideas and suggestions on how to improve my classification for them.

Sider roadmap

2006-12-13T21:28:00+00:00

I started working on Sider roadmap. It's helpful to have a good roadmap to maintain focus and work on projects. It's something I should have done it much earlier.

I have ideas about Sider 4 versions ahead. At some point, I want to put them all in writing for everyone to see, but with upcoming release I'm short on time and might not be able to finish it. If not I will complete it shortly after release.

My goal is to have full design doc online (preferably in wiki). I want for people interested in Sider to have an understanding of the platform and maybe share ideas on what can be improved.

I have only a portion of it done and it's not in the form I would to put it online yet. I'm working on improving it and will start moving it to wiki, but there's so much to do, that I'm not sure when I will put it online yet.

I came to realize importance of design documents through my recent experience. I used to think I could get away without one. No, not really. Writing design document forced me to think about different aspects of Sider and understand the whole picture much better. I had hard time focusing on my work (jumping between different things) without it and my overall design is not nearly as good as I want it to be. After I created a large portion of the document, development became much easier and more enjoyable. I started to get things done quicker and understand the whole project scope better.

My main argument against writing document was bureaucracy. I was afraid I would end up with a large, hard to read or navigate document. I didn't want to spend all this time creating something useless. Maybe I just wasn't ready for it. Anyway, when writing it I used [wikipedia:MindManager] instead of word processor. [wikipedia:Mindmap] helped to put all my ideas down in writing and quickly organize them, but I think wiki is actually more suited for reading the document with its support for free linking between different pages.

Information management tools

2006-12-08T20:06:00+00:00

In our lives we constantly deal with information. I've talked about what people try to do with it, but there are limits to both our physical and mental activities. To make everything easier we invent tools. There are many different kinds of tools to work with information, yet there're a lot of common aspects to their usage and design.

All tools help us reach our goals, help us with our pain, etc. Many people have different definition of what tools (programs) should do, but in the end it's all the same. A tool must solve more problems than it creates or there's no point in using it (although, sometimes it's hard to see all problems that are going to be created, before some of the existing ones are solved). As you can see from my previous post I assume people have two major goals when managing information, to understand it and to share it.

Every information management tool does only one thing. It stores information and shows it back to us. There can perform a lot of different operations, but they simply extend our abilities, allowing us to work with more information simultaneously, more accurately and to visualize it differently.

I'm mostly focusing on computer-based tools, but there are other kinds, such as writing/drawing on a physical medium (i.e. paper and pen). Paper is very flexible, but not as scalable as computers and doesn't support data manipulation well. It's easy to put just about any type of information on paper: text, pictures, diagrams, numbers, graphs, etc. and more importantly it's easy to combine different information. However, it's hard to change it once it's written down. In addition capturing, organizing (combining sheets of paper) or finding what you need becomes harder and harder as the ammount of information increases.

Before we can use any tool, we need to transfer information into it. We often understand it better in the process because in many cases we organize information as we input it. For example, when we talk to somebody about a problem we are trying to solve, we actively analyze it. As a result we can find solution without other person saying anything because we finally organized information in a pattern we came to recognize.

I want to classify information management tools by how general or strong-typed they are. On one end of the spectrum there's pen and paper (allowing us to enter information in a free form without limitations). Less extreme examples of open tools are Word and Excel. They work with specific structure of the information (linear text or tables), but the actual structure is quite open. It's possible to store a lot of different information in Word, ranging from books and articles to notes and to-do list. Excel support for tables is suited for a lot of uses.

On the opposite end of the spectrum there are tools work strictly with specific information (strong-typed tools). For example Outlook can be use to manage tasks. Each task has a strict structure and tasks are organized in a list. Most of the tools fall somewhere in-between, and there are good and bad aspects to being more open or more strong-typed.

Open tools are easier to adopt for different information scenarios. It's possible to analyze mathematical data, to work with finances and to manage a small project with Excel. Even though the tool is open, it's usually easier to work with information organized in a strict structure. The beauty of open tools is that they allow the user to invent a simple system to simulate strict-type tool (often without realizing they are doing it) and so well-designed open tools are very popular. Good tool will actually provide features to help simulate such strict-typeness (through templates, custom attributes, and so on).

There are two problems however: scalability and potential lack of features. It becomes harder to implement features for tools that support larger variety of information. For example operations to work with plain text will be simpler than operations to work with rich formated text.

Scalability problems appear because users try to use open tools not designed to handle such scenarios. Excel might be suitable when managing small project, but it becomes harder to use when the scope of the project grows. Big chunks of information need to be organized and grouped for the human to be able to comprehend it. We can work with 10-20 items in a list, but working with 200, or a 1000 items in a list would be very hard. Designing and implementing support to organize information of arbitrary type and structure is hard. If it's not done right (assuming it's even possible) user will be forced to adjust to how the program works, and it loses its openness.

Strict-type tools work much better than open tools with information that matches their design. For example, a project management application would have more potential than Excel or Word when used to manage projects, from capturing information (tasks) to analyzing and tracking progress. It could also support different information structures (such as arranging tasks in a tree), users, and so on, and provide more operations, such as grouping progress by iterations, tracking time, filtering. When information fits well with the tool it can be very easy to use, but even a small deviation makes it harder. The tool imposes on the user what information it can work with and if its design doesn't exactly fit the need there's a problem. We try to adjust it for what we need, but it's most likely not as flexible as an open tool (and it wasn't designed to be as flexible). For example if project management tool doesn't support time estimation, and we need it, we can add our estimation in parenthesis after the task title.

Strong type tool is easier to use and provides more options, when information fits it and when the tool is designed for the tasks you want to perform. Open tools are more flexible. They allow you to invent your own way to simulate strict types and structure information. When available I prefer strong-type tools, they are easier to use and don't force me to invent something. In the best cases it fits me, my information and my problems like a glove. If it doesn't, I fall back to open tools (like Word and Excel).

I was going to talk about design, what is done right and what can be improved in the existing tools, but I felt it was important to think about what kind of tools are out there first. Wow, this post is probably one of the longest I've ever written and it took me many hours to think about all this and write it out. I really hope some good will come out of this series on information management. Anyway, I'm planning to address tools design in the next post.

1st Sider release is coming

2006-12-02T17:52:00+00:00

I'm going to release Sider on 12/15/2006. This is going to be a demo of the platform ideas. I want to show that it's not only possible, but also easy to develop new tools to work with information using Sider. It won't be very useful from practical point of view and will feature a single package to store notes in a tree-like structure. There're some things I want to fix before release, as well as restructure wiki documentation a little and create website for Sider.

Information usage

2006-11-18T15:00:00+00:00

In my last post I talked about information and what it is. Today's post is about how people use it and work with it.

We use information to decide what to do and to communicate. Let's try to see what we can learn from this description, even though it's pretty simple and abstract.

All our actions are based on our understanding of the world around us. Most of the time we process information to improve that understanding to make better decisions based on it. We work with information to understand it better, to find meaning in it. We simplify it, arrange it in familiar patterns and relate to existing information (effectively incorporating what we learn with what we already know).

Communicating information is sharing it with other people (or ourselves when we forget something). Effective communication can be hard, even if you are just leaving notes for yourself. We need to understand information involved (like we do when deciding what to do) or remember and reproduce exact data (but people are usually pretty bad at that). In addition we need some way to store and present information. Quite often we present it in the same form as it is stored. When we write in the notebook we see everything in the form it is written (stored).

It seems that how we work with information is similar whether we want to communicate it or do something. In either case we need to understand it well. Having a good method to store and present it is also very helpful. This is a list of goals we usually have when working with information:

Simplify - Like computers, people have limited memory and can process only so much information effectively. We want to reduce amount of data we work with to make it easier for us. To do that we can remove any duplication and irrelevant data, and enumerate similar data by assuming it is the same. The downside is that we can get less accurate picture as we eliminate data, so balance of how much information is simplified is important
Capture information - Store it on some media. This is important both to share it and to analyze it. It's important for analysis because we, again, have a limited memory problem and can often process only a portion of data. We need to store the rest of the data while we are working on a part of it to be able to come back to it later.
Capture relationships - Describe how smaller pieces of data relate to each other and how new information is related to existing information (where it fits in the larger picture).
Display - How we see the information determines what it means to us and how we work with it. We often choose the most suitable way to show it depending on what we want to do. For example people are much better at understanding pictures then numbers. Yet numbers are often much more accurate then pictures.

Correct tools can greatly help us with any of the above operations. We have been inventing them since the beginning of mankind. Actually they are so common now that we use a lot of them without much thought, writing in a notebook for example. Yet I think tools and how we use them can be greatly improved to make our lives easier. In the next post I want to explore that to see what's done right, what's done wrong and can be improved.

Taking a closer look at information

2006-10-31T19:45:00+00:00

Today I want to start a series of articles on information management. I want to look at what information is, how it relates to our lives, why it's important to think about it, how people work with it, etc. My goal with this and the following posts is to better understand the nature of the information to design better information management tools.

To understand what it is we need to take a look at data. Data is a description of an object, relationship between objects or a process. I couldn't come up with a better definition. Numbers, letters, pictures, sounds, smells are all data and they represent something to us.

Most of what we perceive in the world is gradual. There are no distinct borders between colors. On the spectrum one color changes into the other. Time is continuous as well. We created minutes and hours to measure it. We enumerate and simplify data naturally (often without thinking about it) to make it easier for us to work with it. Different pieces of data describe a small characteristic of an object, relationship, process. Looking at a brick we can figure out its size, material it is made from, weight, etc. All of these describe different aspects of the brick. We enumerated information on the brick. Without doing it it would be hard to calculate how many bricks we will need to build a house or how tall it can be before it will collapse. Use of correct tools can help us capture this information and use it more effectively.

Information is a collection of data organized into a pattern. Humans are very good at recognizing patterns and finding associated meaning. Keep in mind that data and information are objective, but their meaning is very subjective. A brick is a brick no matter who looks at it. Associated meaning (how we see it) can differ. One person might decide to use it for construction, another to use it as a weapon, while third uses it as a paperweight.

When looking at the information we might recognize it. In that case we automatically recall associated meaning. When we look at a red, rectangular stone we recognize it as a brick. Actually we like to recognize patterns and understand the information we are presented with.

Patterns we fail to recognize are noise and are very, very boring because we think they are useless. TV snow is a random pattern of black and white dots. We might recognize that something is wrong with TV, but we probably will not be able to understand all signals that force dots to be arranged in one pattern and not the other.

In the next post I want to talk more about how people work with information. This is directly related what kind of tools can help them do their job better.

Problems with native XMLHTTP in IE7

2006-10-22T00:44:00+00:00

Microsoft has finally released IE7. It offers a lot of improvements over IE6 for both users and developers. A lot of bugs related to layout, CSS and memory leaks were fixed. I got really excited when I installed it thinking that most of my problems are over, but life is never perfect. I spent almost a whole day trying to figure out how to work around a new problem that appeared with IE7.

New Internet Explorer introduces native XMLHTTP (used to communicate with server behind the scene without page refresh) support. Up to and including IE6 developers had to use ActiveX object that came with MSXML. Native support makes IE more compatible with other browsers (script is easier to reuse between browsers), but more importantly it is faster and allows XMLHTTP to work even when ActiveX is disabled.

This new feature became a source of frustration for me. Microsoft is tightening security screws and new XMLHTTP denies local files access. In general I think this is wise, but there are exceptions where access to the local files is needed.

When I'm developing JavaScript web controls I often test them by simply opening a file from hard drive. With the enhanced security DojoToolkit doesn't work locally since it uses XMLHTTP to load packages.

I know only one way around this - disable native XMLHTTP support in IE options. This forces DojoToolkit to use old ActiveX MSXML object.

Another problem is with Sider. It hosts WebBrowser control to show document with HTML views. All files are on the hard drive and native XMLHTTP refuses to load them. This made the application pretty much unusable.

Again, the only solution I found was to disable native XMLHTTP and force script to use old ActiveX object. However, I can't force users to change IE options just for my application. Microsoft, fortunately, provided a way to customize them for a specific application and I can disable native XMLHTTP only for Sider.

To disable it for a specific program use this registry key:

[HKEY_LOCAL_MACHINE\SOFTWARE\
Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_XMLHTTP]
"ProcessName.exe"=dword:00000000

I'm not too happy about having to disable new features only because there's no way to configure them. Native XMLHTTP is faster and I would like to have an option of testing it. It's not essential for me now, but at some point it might be more important. For now I can look for an alternative on the Internet from time to time, hoping that there will be a better solution to loading local files through XMLHTTP than simply disabling native XMLHTTP feature.

Xelog 0.1 released

2006-10-15T14:37:00+00:00

I have released the first Xelog version. You can look at Xelog change log to see how it looks or download it. In this version I focused on implementing the most important things: schema for the change log XML, allow auto-start by opening Xelog XML file in the browser, and support for different views with ability to change them on the fly.

Current views are simple and static. I considered dynamic views to support on the fly sorting and such, but decided against it. I wanted to release as soon as I could and to have a working proof of concept, but I also wanted Xelog to be useful. I think it is and I'll use it with all my projects.

My goals for Xelog are to help users read and understand project changes and help developer to record those changes. Having different views and ability to switch them on the fly is a user-oriented feature and I will continue to add more of them in the next releases.

For the developers I want to provide views to allow change log editing on the fly. I will probably add it at least partially in the next release. For now you would need to edit XML file manually, but there's a Xelog schema to help you. If you are using a sophisticated XML editor it might be able to use it to supply auto-completion.

With the next release I also want to add support for different formats, such as text, PDF, RSS. I'm not sure yet how that will work and I have a feeling that RSS support might require a server-side solution.

Last, but not least, I want to allow developers to customize how to show the change log by generating different links to the XML file (or maybe HTML file that will launch Xelog). I'm thinking of using URL arguments to configure Xelog behavior.

So, check out Xelog, play a little with it and I'd love to hear about your thoughts. Would you consider using it? Do you think there's something missing that would make it more useful for you? I'm open to suggestions and ideas.

I want to focus on Sider and release a technology preview version. I'm not really sure when I'll release next Xelog version, but I don't want to delay it for too long.

Impact of updates on the user

2006-10-12T13:08:00+00:00

I was working on Xelog and had an idea. A change log describes what has changed in anapplication. It lists new features and bug fixes. But I don't think it's really about logging project changes. Its main audience is users and they want to know how new updates affect their experience with the program. Most of them don't really care about changes deep inside the application, only about things that are visible on the outside. A user doesn't really care about things like code refactoring (and he shouldn't), unless he's using this code directly. He only cares the benefits it provides, for example refactored code can lead to improved performance (and this can be important if performance hindered program's use).

Many, many projects only talk about changes to the project, but don't really talk about what it means to the user. Often it's possible figure that out, but what user thinks in many cases is quite different from what developer thinks. This can lead to unintended confusion.

I added support to Xelog to document this aspect of the changes. Each entry in the change log needs to have an impact flag to indicate how much a user is affected by the change. It can be something minor, such as a spelling fix, or something that can break things, such as public interface change, document format change and so on.

In addition to the notes describing the change and impact flag, there's a new text field to allow developer describe how the new update affects the user. Its purpose is to allow developer explain what effect he thinks the change will have. It's a good place to describe how to adapt to the new data format or how to use new feature to its fullest.

I've updated old views to display impact information for each change. In addition, I'm going to add views focused at showing change log with emphasis on this user impact. I want to help anybody reading the change log to understand which changes are important and provide new, cool functionality, and which are minor and thus can be ignored.