This relation is about to end by WolfS♡ul

Many web frameworks are structured around the MVC pattern for server web development. The relationship between the use of MVC and server web development is fatally doomed. Here is why.

MVC in itself is a wonderful pattern. In case you do not know about it, here is a quick snippet from your preferred source.

Successful use of the pattern isolates business logic from user interface considerations, resulting in an application where it is easier to modify either the visual appearance of the application or the underlying business rules without affecting the other. In MVC, the model represents the information (the data) of the application; the view corresponds to elements of the user interface such as text, checkbox items, and so forth; and the controller manages the communication of data and the business rules used to manipulate the data to and from the model.

It is a pattern that has brought success to many software projects. It is popular and used in many places.

In the traditional sense of web development, it would look close to this:

In this case, the browser is treated as a view, the application server contains the controllers and the database or any other storage mean is treated as the model. The problem with this pattern is that it is getting further and further way from the real usage. What has changed is the browser. It is less of a simple viewer today than it used to be.

In case you have not noticed, the browser is the new operating system. It can store data, manipulate and process data in multiple ways and it can shows more rich data than before. The browser is not exclusively used to show simple linked and static pages anymore. It is running full blown applications. With the explosion of Javascript, you can expect this trend to keep growing.

This is what we are going towards for web development:

This does not eliminate the MVC pattern. It could still be used for the application running in the browser, but I think it will kill the most of the MVC server web development as we know it today.

In order to develop the application generator or provider, we will need better tools than what we have now. Maintaining a bunch of CSS, HTML, Javascript and server side code files is not realistic anymore if you want to scale, develop rapidly and have a pleasant developing experience.

This is where tools like Google Web Toolkits and Pyjamas. We need more of those tools.

Not everyone likes what IBM is doing by Looking Glass

I have started using Rational Team Concert (RTC), version 1 and 2, at work for my latest project. RTC is an IBM product based on the Jazz Foundation platform which integrates work item tracking, builds, source control, and agile planning. It tries to manage most of what a software development team will need to do with the same tool.

In that sense, it is similar to Team Server Foundation by Microsoft. You might remember my rant about it.

RTC is geared towards big enterprises. The initial cost for the standard edition is probably enough to justify not to use it for most small and medium teams or companies.

Like many IBM products, RTC tries to do everything from defect management, automated builds, planning to source code management. I must admit that overall, it does manage this feat to a good level.

Without any other delays, here is what I dislike about RTC.

It sends whole files on the wire instead of deltas

Imagine you have a 100 Megs file that needs to be modified and sent back in your central repository multiple times in a day. Let’s also pretend that you are simply changing 10 characters in that file each time. With RTC, whenever you send those changes are sent back in your central repository, the whole file will need to be transmitted on the network instead of the 10 characters.

This is simply unacceptable. It is a waste of network resources and a waste of my time because I need to wait for this transfer to complete. It seems like some operation depends on other ones and you cannot perform them in parallel. Whenever I tried to do too many long operations at the same time with RTC, it shows a model progress windows that completely disabled my ability to use my IDE.

A simple diff algorithm on the client side or on both side would use a few of my heavily underused CPU cycles and it would be way more efficient. This is one of the base concept of most version control system out there. Subversion has been doing this for years now.

It is centrally based

It seems like most people have not discovered the joys of distributed version control yet. During the course of my normal usage of RTC, I have had network failures and disconnects with the RTC server. As soon as I was disconnected, I was left in the cold dark state of anger because I could not do much about managing my source code. For that reason alone, it should be distributed.

It is treating me as an idiot who do not know what he is doing

Whenever I try to do something that might have some repercussions, it shows me a popup asking me if I am sure I want to do that. Here is a typical scenario:

1. I check-in some files while keeping some files unresolved (not checked in).
2. I associate a work item with the new changeset because it is a precondition to delivering a changeset on the project.
3. I try to deliver the changeset.
4. I get a popup asking me if I am sure I want to deliver those files without including the other files that are unresolved.
5. I confirm this action by clicking on yes.
6. There are errors in my project that triggers a precondition failure that makes me unable to complete the delivery.
7. I override that precondition so I can complete my delivery.
8. I click on the “Retry” button so my delivery can be retried with the precondition overridden.
9. I get another popup asking me if I am sure I want to deliver those files without including the other files that are unresolved.
10. I confirm this action by clicking on yes.

Those popups are everywhere within RTC. Many of them are modal ones too. I am not an idiot. I know what I am doing. Stop bugging me and provide facilities to revert my mistakes.

You cannot easily name a revision in the past

There are two named revision concepts in RTC: snapshot and baseline. In short, a baseline is for a component (a kind of project division) and a snapshot is a group of baselines. The problem is that you cannot create them for revision in the past. You can only create them for your current workspace. If you forgot to create a snapshot for your latest deployment, it is just too bad for you. If you really want to create it, you will need to update each file to the state where it was when you did your deployment and create a snapshot with the files in your current workspace. The ugly side of this is that you cannot update your workspace to look like a revision in the past easily either.

Read-only file updates make your project out of sync

For our project, we had some files that were read-only but needed to be updated once in a while. Every time we had to update those files and they were accepted on the developer’s PC, it would put the projects in the related baseline out of sync. It is a state where you cannot do any operations with RTC without having multiple popups telling you that you need to resync your projects. The read-only file updates is only one way to get your projects out of sync. We manage to reach that state in different scenarios.

In other to correct that problem, we had make the files writable and perform the resync operation. This was a real pain.

Considering the whole thing, RTC is not such a bad product. It just has many quicks that are unpleasant and difficult to bear with. If you have the money and want an enterprise product, go for it. If you do not, there are multiple free solutions out there that can do most of what RTC is doing and that can do it better.

My main problem is that I want to do everything. I want to live it and enjoy it. Here is a short list of thing I would like to do:

• Invest more time in training for programming and development skills.
• Invest more time to make my work job enjoyable for everyone by planning some more activities with my colleagues.
• Invest some time having fun.
• Invest some time taking care of my general health, doing sports and some body building.
• Invest more time helping out open source projects. So far, I am still evaluating which ones.
• Invest more time in my personal software projects.
• Invest more time in my family members’ web projects.
• Invest more time blogging.
• Invest more time keeping up with the news about software development.
• Invest more time hanging out with my friends.
• Invest more time taking care of my girlfriend.
• Invest some more time with work related out of work projects.
• Invest some time influencing politician about privacy, copyright and new digital realities.
• Keep up with the daily stuff.

Right now, I know I am probably spending too much time on gaming. On average, I might spend about 14 hours a week on it. That item should be scrached from the list, but one of my philosophy is that I should have fun everyday. I will not have time to have fun once I am dead.

As for the rest, I just seem not to get enough of each. Am I trying to do to many things at once ? It seems like everyone is doing it. I should be able too right ?

Do you have any tips or cure for me ? I am open to everything.

Me and Guido van Rossum at PyCon 2009 in Chicago

I have been in Chicago for PyCon 2009 this weekend and I must say that it was a tremendous conference. These guys know what they are doing and they are doing it well.

First, the hotel was alright. There is not much to say about it except for the fact that I had to walk 10 minutes to get to the main hotel where the conference was. From what I got, they had to reserve two hotels because they wanted something bigger and they had a contract with a previous hotel.

The food was pretty good. We had a continental breakfast every morning. We also had a great lunch every day that was served fast and professionally.

Most of the talks were interesting. Some of them had either a monotonous speaker or were less captivating than I though they would be. Since all speakers were giving their own free time for this, the quality was high. We were asked to vote after each presentation by dropping a bingo chip in either a green bucker for good, a yellow bucker for so-so and a red bucker for bad. It was an original way of measuring appreciation but my feeling is that way too many people abused it. The logistic of organizing all this was mostly perfect. There was nearly no downtime for people to setup and start their presentation at the right time.

Everyone was given the opportunity to speak during the lightning talks. They were scheduled 12 hours before talking place and all that was required to speak was a laptop to show something and write your name on a sheet. Given my future implication in some python projects, I might do one next time I go.

Another nice thing about PyCon was the Open Spaces. It is a nice concept where people write some topic on small paper cards and place them on a board to setup the time and the place of the meeting. People than go have a look at the boards and plan to attend whatever they like. They had something like 12 smaller rooms that could be used for this. All this was going all around the clock. On Friday night, I played board games with other Python users up to 2 in the morning. It was all fun.

I have had the chance to meet with so many inspiring people. I met with Guido van Rossum for a quick picture. I was so impressed how friendly, open and easy going he is. He attended the conference just like the rest of us without special treatment (from my point of view). Everyone was down to earth and simple, including him. I also had the chance to meet with Python users from Québec.

The community around Python is fantastic. They are a bunch of friendly, intelligent and open minded guys (the PyGirls were drowning in the sea of males) sitting on a diamond mine. There is much to discover by digging below the surface on a social and technological level.

When I left, they were starting their sprints, a developing session, to advance various open source projects while being face to face with people that are often sitting in front of their computer around the globe. Have I had more money and time, I would have stayed to help some of these projects.

I am now back to reality. Time to get those Python skills ready for the next PyCon. See you there.

I am really excited about it. It will be my first real development conference and an international one.

I will have the chance to meet or at least see Guido van Rossum, leaders and contributers to many open source projects I use and I admire: SQL Alchemy, Django, TurboGears and others. I will also have the chance to watch conferences by the developers of EVE Online, a MMORPG game developed with Python.

It is going to be a wonderfull 3 days.

No matter where it is going, it has to run fast by Hamed Saber

Most developers I know are worried about their software performance. They are worried to the point of nonsense. With every decision they will think about, they will immediately balance out the arguments regarding performance whether it is an architecture decision, a technological choice or a nested loop.

This does not make sense on many levels. First of all, performance is more often overrated. A fully functional and correct slow software is always better than a fast buggy or incomplete software. Clients are often asking for highly performing software but in most cases, they are satisfied with a fully working version even if it does not run as fast as they first though.

Developing with a performance mindset often create solutions which contains more bugs, are harder to maintain and are harder to understand and read. Here is a example of this. Compare these two C code snippets:

int i, sum = 0;
for (i = 1; i <= N; i++)
    sum += i;
printf ("sum: %d\n", sum);

int sum = (N * (N+1)) >> 1;
printf ("sum: %d\n", sum);

They both execute the same task. The second one is optimized for performance. Which one would you prefer for ease of maintenance, ease of understandability and ease of reading ? The first one of course.

Another reason why you should not think about optimization too much is because you cannot know which part of your application is or will be the performance bottleneck. You might put a lot of efforts with a specific section of code that is only called once in a while and fast enough.

I am not saying you should not do performance optimization, but that should be the last thing you should thing of. In my mind, I only see two reasons for doing it:

1. Customers are complaining about speed
2. You want to gain a competitive advantage (just like the current crop of upcoming browsers)

If you know any other good reason for doing it, let me know.

When you have to optimize your code, the first thing you have to do is use a code profiler to find where is the problem. There are many tools available for this and I am pretty sure there is one for your language. Here is a simple profile output I ran a few days ago for Mercurial.

         1239498 function calls (1237075 primitive calls) in 18.604 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   18.604   18.604 :1()
        1    0.000    0.000    0.000    0.000 ConfigParser.py:106(Error)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:118(NoSectionError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:125(DuplicateSectionError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:132(NoOptionError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:141(InterpolationError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:149(InterpolationMissingOptionError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:162(InterpolationSyntaxError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:166(InterpolationDepthError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:177(ParsingError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:189(MissingSectionHeaderError)
        1    0.000    0.000    0.001    0.001 ConfigParser.py:203(RawConfigParser)
        5    0.000    0.000    0.000    0.000 ConfigParser.py:204(__init__)
        3    0.000    0.000    0.000    0.000 ConfigParser.py:211(defaults)
        3    0.000    0.000    0.000    0.000 ConfigParser.py:214(sections)
       11    0.000    0.000    0.000    0.000 ConfigParser.py:219(add_section)
       20    0.000    0.000    0.000    0.000 ConfigParser.py:229(has_section)
[...]
       10    0.000    0.000    0.000    0.000 {setattr}
        2    0.000    0.000    0.000    0.000 {signal.signal}
        1    0.000    0.000    0.000    0.000 {sys._getframe}
        1    0.000    0.000    0.000    0.000 {sys.exit}
        2    0.000    0.000    0.000    0.000 {sys.getwindowsversion}
        2    0.000    0.000    0.000    0.000 {thread.allocate_lock}
        1    0.000    0.000    0.000    0.000 {win32api.GetCurrentProcess}
        1    0.000    0.000    0.000    0.000 {win32api.GetFullPathName}
        1    0.000    0.000    0.000    0.000 {win32api.RegOpenKey}
        1    0.000    0.000    0.000    0.000 {win32api.RegQueryValue}
        5    0.020    0.004    0.020    0.004 {win32file.CreateFile}
       25    0.474    0.019    0.474    0.019 {win32file.ReadFile}
        1    0.000    0.000    0.000    0.000 {win32file.SetFilePointer}
        1    0.000    0.000    0.000    0.000 {win32process.GetModuleFileNameEx}
        2    0.000    0.000    0.014    0.007 {zip}

Using different runs, you can find out which method is called the most, which one it using the most CPU time and which one is the most expensive with profilers. Once you know that, you can start optimizing the sections which are the most problematic.

Once you are done with your optimizations, you have to run your profiler again with the new code and compare the results with the unoptimized version. That way, you can see if you are making progress or not.

I have come to think that while managing your code, you should use a separate branch for the optimized version and the unoptimized one. Since the unoptimized one is probably going to be easier to read and understand, it will also be easier to maintain that is a benefit you should keep.

Stop worrying about performance and enjoy clean code.

In one of my previous post, I wrote about the distributed way of managing source code and what differences it make compared with the common solutions that are used today. Before going any further, here are the benefits I see when using a distributed revision control mechanism:

• Offline support (commits, branches, merges)
• You are your own working copy and you can share changes with others in the way you want
• You are your own working copy and you can break it apart, test it, try experimental things. You are free to play with it.
• Flexible structure for changes flow (someone has to pull changes from you - aka the Linux model, everyone push/pull to a central repository - aka the central model, someone has to pull changes from you to check that they meet a certain quality and push them somewhere else, etc)

This is great if you are only managing source code, but there are many different things you have to manage for a software project to be successfull. How does it translate in the distributed model ? Some projects like the ones under the Mozilla umbrella are using a distributed revision control software for their code and some central models for other part of the their projects. This is fine, but there are some alternatives.

I discovered some interesting distributed aware tools to complement your distributed revision control software for documentation and bug, issue or task tracking.

• Distributed wiki: ikiwiki, MoinMoin MercurialBackend branch
• Distributed bug, issue or task tracking: Bugs Everywhere

It is still early to tell if we are going to see a push towards even more distributed tools like these. We can surely benefit from using these tools in a proper way. I will keep an eye onto it.

Holding tight by jbelluch

How hard should you hold onto your ideas? Do you really need full control of everything happening around you? What about trusting people and finding the answers with them?

It is time to let go. Tomorrow is too late. Now is the time.

Back when I started building my first program, I never thought I would enter a religion war zone by making my initial steps in the software development world. There should be a warning sign for this on every tutorials and introduction textbooks about programming.

Warning ! You are making your first steps in dangerous zone. Prepare yourself or you will get sucked in as well.

Everyone seems to know what is best for everyone else. Client will tell you how you should use the session to get a better performing application. Your colleague will tell you how you should you the database to get the best performance possible.  Your boss will tell you which colour you should use to make your application usable.

Developing software the hard way by Cliff Hanger

Let’s start with clients. Most clients I worked with wanted total control over the scope, the schedule and the cost of the project. They know how to do software development. One of the problems, is that they were given that control. More precisely, they were given the illusion of it. Giving that much control to your client is like developing upside down while being tight up. It is not going to work.

I think the first step toward redemption is to educate and explain to your clients what it is to develop software. It is surely not like manufacturing hammers.

One point I would like to emphasis is the requirements. I have seen way too often requirements made of all must-haves. I think it should have a good proportions of must-haves and nice to haves so that the scope become variable and can be adjusted to meet more important schedules or costs.

People in the software industry are known to have strong opinions. They will argue for hours over whether that technology is superior or better than another. They often suffer from the silver bullet syndrome, but the usual answer is it depends. Every solution has a cost, some advantages and some weaknesses. We should learn them and give educated answers about those technologies.

Ancient philosophers by Lawrence OP

Being a software developper gives you a whole lot of possibilities. You are often given confidential documents. You have access to confidential data that most people would not have access too.

I was once maintaining an online shop where every credit card was stored with a simple symmetric cipher with the key in plain text in the code. I could have use all those credit card records if I was unethical but I did not. You should probably not trust at least 50% of every online shops with your private data these days.

I recently created a personal training plan to get better at software development. I searched for some references on the web and I found this document. It is called the Software Engineering Code of Ethics and Professional Practice. It is a highly interesting text which I endorse. I suggest you read it too and endorse it in your own way.

Since I am not part of the official software engineering community, I never had to follow any code of conduct. This code is well written. It express so many good ideas that I can only make it my own by approving it. If you are in the software development field and you do not have any ethical guidelines, have a look and pass it down to your colleagues.

Communication difficulties by Tania Paz

Some weeks ago, I realized something important. Software development is first and foremost a task about communication. It is not the kind of communication happening on network links. It is about human communication.

A programmer will typically have to speak or communicate with his fellow programmers, the analyst, the architect, the projet manager, the customer, the tester, the documentation writer, the graphist, his boss, his coworkers’ bosses, the customer’s boss, the project owner and many others. There are many potential pitfalls.

Analyzing it from my point of view, I think many programmers have social skill problems.  When I say “many”, I do include myself in that group. I think certain kind of personalities makes you like computer science or at least, it makes it more enjoyable. The environment in which you grow up is also a factor which may drive you to this discipline.

On the Myers-Briggs personality scale, I am an INTP. Which can be resume with the following:

• I for introversion which oppose extroversion
• N for intuition which oppose sensing
• T for thinking which oppose feeling
• P for perception which oppose judgement

In short, I was not easily interacting with people. It has lead me to a slow progression toward social skills. Those skills which make you good at interacting with people and communicating with them. Those same skills which make you understand how people are feeling, what they are going through and adjust yourself to this reality. Those same skills which are essential in software development.

There is only one way out: trying to get better.

This is what I have been trying to do since than. I think so far it is working. Even my girlfriend has noticed.

My first goal was simple. “I will try to engage a discussion in a kindly manner with whoever I meet.” It seems simple, but for someone who has tried to stay in his bubble for as long as possible, it can be hard. So far, I think I have met that goal even though I still have some defiling moments when I just do not want to speak with anyone.

Recently, I bought How to Win Friends & Influence People. This book was recommended in a software developement book. I thought it would make sense to buy it and have a look. I did not buy it to make more friends or manipulate people. I bought it for the social lessons it contains. I have read a few pages so far and I will probably post an update when I finish it.

Another possibility I am currently thinking about is taking some communication classes. Those kind of open courses where you can go in whenever you want.

Social and communication skills are often disregarded by software developpers. I think it is time we change our view and see just how usefull they are.