Francisco Garcia Rodriguez

You Are The Evil One

2012-02-01T00:00:00+01:00

Last weekend shopping with a friend in the supermarket I had a revelation right in front of the meat section about how hard is being a fair person. The following dinner was such a bloody discussion about the stupid belief of people in fairytales, unicorns or that politicians and banks are the most evil forces of this world.

Our fight just started because of some stupid chicken breast. While I was looking at the prices around the butcher's section, my friend came proudly with a frozen bag and slapped my face with the brilliant deduction of the day: This costs 1.60€ and what you are looking at costs 2.40€

Although I do not make a fixed amount of money every month, I would say my friend makes more than I do. We both do some regular donations but do not agree on our goals. My friend's small Jiminy Cricket is at ease donating money for nature and wild animal life, something which I find rather stupid considering that chicken choice.

Before converting into a flexitarian I was a vegan concerned about animal rights because of an overdose of PETA's videos. Yeah, I know, it is a real shame but meat is not easy to give up and it is not always like that. Oh really? You might well believe also that your chicken gets a massage in the belly every morning and some classical music during the evening. Now get serious and make your numbers. Most chickens that people buy comes from a place where 10 to 20 chickens are manufactured per square meter. If you have 1,500 chickens in 100 squared meters would you notice if one get sick? or when half of them are dead? You just load all of them on antibiotics everyday just in case those bastards die on you and ruin your production quota.

How much more do you think it actually costs to raise one chicken outdoors? think about fresh air, some space to run, maybe green grass, sunlight and real food instead of compound feed. Say maybe, that you need 1 squared meter for each one. Also the cost of living of the local farmer. How is he supposed to sell that? 10 times more? No matter how you look at it, raising an animal with some decent life costs more. Do you really think it is the difference between 1.60€ and 2.40€? Well, that day I was looking at the real local, natural farming edition, and that my friend, was 10.20€ bucks, same quantity. That is the real cost of supporting a local farmer, demanding some dignity for the life of that chicken, avoiding the environment and human health impact of a meat factory and dreaming for future generations with effective antibiotics. Hardly any of my donations goes for animal life hence I am very evil, some vegan friend hate me for being so cruel eating animals and many others shame on me because I am the dumbass who throws away money after being brainwashed from yellow journalism.

So what is the lesson from that little chicken? That nobody cares, because all that matters is getting the best price. A bio egg costs 40 cents more, and that is also the price margin according which many people decide if you are an idiot.

Leaving poor chickens aside, the fashion industry is another great example. Some months ago another friend of mine was so proud because she bought some jeans for 5€. Yes, I know, so sad there is so much injustice in the world, and I am sure she slept just fine that night without thinking about a 6 years old child sewing the button with bleeding fingers. Who can believe that is the real cost of a pair of jeans? This time we need a factory, material, machines... and the minimum wage for the worker to carry a decent life. In a wonderful world the worker could get 30% of the sale, being realistic 10%, but in the real one, I do not believe it is more than 1%.

Is there any BIO or FairTrade equivalent for the fashion industry? Clearly those jeans should cost more than 5€. What is the real value of clothing?. Maybe you should buy from a trustworthy brand claiming that they produce locally, not in exploited countries, but there are a lot of ways to cheat on that. You do everything abroad with slave labor force, sew one button locally and put Made At Home Sweet Home because Made in China is not cool. In the fashion industry it is more fashion to write Made in Italy, that is exactly the reason for Italy having the largest Chinese concentration in Europe 1

It seems that injustice in the world is because of greedy people. You know, those politicians and those bankers. Oh yeah, that last group is the worst of all evil. You can see that everywhere, on TV, on Internet... because the world financial crisis is a direct result of their evilness. Most people give freely their money to those bankers. As far as the politics goes, most people promote them. I say promote and no vote because either they vote the big parties financed by the banks they hate, or just do not vote, which is exactly the same thing, because the smaller parties, the ones less manipulated and with people who actually want to do something, have a hell of life to get votes enough.

Personally I am not that much scared of the greedy banks. I am more concerned about normal people. Sometimes I believe that people who feel smart getting the lowest price without thinking on the meaning of that price might not be that much better. Would they do anything different having the power? Many people still believe that a fair job is a stable 8 hour life-long one, not very stressing and a style of life based on products produced by slave labor force.

Are banks and corporations really evil? How many people financing a house or a car actually pays a higher interest to an ethical bank instead to one giving lower prices because of their higher profits financing weapons and slave force/polluting companies? Since when in this world is giving people exactly what they want a crime?

Most people price demands do not allow local production by people with a similar quality of life. They also demand better work conditions and more environmental protection. How are they surprised when labor is outsourced to a place you can enslave and pollute? Even most amazing is when they hate foreign policies when the most crucial thing we must do right now is exporting our social and environmental values to them so everybody can compete under a more fair game.

And the most intriguing questions of all: Why on Earth did I place a picture of Gandhi on this post if I wrote most of the time about chickens? Indeed, this post originated from a dead chicken! Well, the reason is because for Gandhi the final responsibility of the solution went something like this:

Be yourself the change you want to see in the world

Please, before complaining again about how unfair life is, make of yourself an example. This is not all or nothing. You can be ethical one week a month, or you can buy the happy chicken for your kids while you eat the factory version. Think twice about what you buy. Do your research and an effort to become a more responsible consumer. Deciding the right thing or choosing fair products is never easy. Support local artisans, farmers and people trying to make a decent life by producing something on their own. Be yourself that change.

Your Credit or Your Life

2011-11-22T00:00:00+01:00

During the last months I've been exposing myself more to the entrepreneur world. I am not sure if this is one reason for which more people is asking me about becoming a freelancer or building a new startup. Todays post is not only for all of you wannabe independent out there, it also applies to anyone concerned about how money affect life.

If you are planning to take the lead of your career you will find plenty of literature out there and blog posts. Let me summarize most of what I learned about life and money so far.

The most expensive things in life are:

Laziness

Ignorance

Rush decisions

You alone are responsible for covering those points because no one will solve them for you. The bad news are that it might take your whole life getting it right.

I have noticed one difference between Americans and Europeans, it might be one of those things that the American school system implanted into my brain, which could explain why I am so concerned about it: Most Europeans completely underestimate the importance of good credit.

People building their own company are all the time leveraging credit on their side and use it as one of their most valuable tools. Great ideas cannot wait to be implemented until you grow the money, neither good opportunities are open for too long.

The other people who do not have to deal with banks so often, ignore such nuisance convinced that solid private finances are only about being solvent: saving some money in a bank account and never spending what they do not have.

Anyone wanting to fly solo, must learn more about private finances. Forget about a rent or public health care, you are completely on your own and better be ready to know what to do with that extra money. You must learn something about savings, taxes, investment and credit. The later is the one people usually neglect, so that is the only thing I want to write about.

Nobody like the idea of the extra money gone with a loan. They rather pay in advance everything and assert the idea of how evil and stupid borrowing money is. Yeah, to the hell with credit... this is what most freelancers never think/care about before jumping on the entrepreneur bandwagon. They do not notice how their credit score sinks deep into hell until it is too late.

What are the implications of that stupid thing about credit? Well, if you try to upgrade your phone contract you might be surprised that you cannot get it. Your phone company will check your credit, which reflects your non permanent job status and decide not to risk themselves giving you a phone that you might not pay, even if it is free with a 10€/month contract.

Since you are new into freelancing, you might decide to spend a lot on new gadgets, office stuff... maybe go wild and decide to pay 600€ in 12 smaller payments. Guess what, you will not get it. Even if you had 100 times more money ready in your your bank than the money you ask for, nobody will trust you, not even your very own bank. For them you are Mr Nobody and Mr No Job

Companies do not care at all how much money you have saved, they have no idea of how free you are to use it. So stop dreaming about solving your problems sending them your last balance, it will just not work. What they care about is your financial experience and commitment paying debts, how often and how much other institutions trust in you: your credit history

If you ever plan to buy a house, buy car or face a medical situation where the best solution is not paid, you will need to ask for a loan. Having bad credit does not mean that no one will give you money, only that you will have to pay much more. Loan interest rates can move between 6% to 30% depending on your credit score. If you are not good at financial math, you should know that a 10 year loan at 6% with monthly payments mean that at the end you pay back to your bank the money plus 33% in interests. With a 30% loan you pay 216%. If you are surprised with these numbers or do not understand why 5 times more of interest rate cost you 7.2 times more wait until you need a loan.

Being independent means that most likely your whole life your credit will be worst than a normal person with a permanent job contract, paid insurance and paid retirement. Even if you had 1,000 times more money in your bank account than them. In the very moment you become a freelancer, you will stink to every bank on Earth.

Giving advice about improving your credit is difficult because it depends on the country and so far it is not something traveling with you to a new country. These are some things for improving your score that usually work everywhere and you should do before quiting your job.

Ask your bank for an increase in your account overdraft margins. Anybody should maximize this once a year. With time and a good credit you should get them between 6,000 and 10,000 margin, but normal employed people who never cared about credit will likely get 500 worth trust their first year. Offer them to accept higher overdraft fees in exchange of a wider margin, but never ever use it.
Get yourself a credit card or two. Prepaid ones do not count, only real ones, however can use them without the credit part.
Make a phone contract where you must pay a minimum every month. Mostly anything with your signature and commitment to pay will work, even your electricity bills.
Next time you are offered buying something with monthly payments, say yes.
Never close an account or contract where you were trusted money or your commitment to pay without thinking twice about it. Once you close it, your whole credit history related with that contract will vanish.

The above points are things that any financial institution can research about you. Ask around for your local credit agencies and ask them for your credit history and learn what they are tracking and what you can improve.

Financially most people strive only for economic independence, the bye mom and dad level. Next level is when you are able to afford a family. There is one more personal finance level which you should never lose the sight of: the economic freedom level. That is when you reach a point where you neither suffer nor endure nuisance bringing home a paycheck. You still work for one, but you do not suffer for it and do not care if your lifestyle is the same as now for the rest of your life.

You can either be born with money, inherit it, marry it, steal it, work for it or a combination of that. Most people spend half of their life destroying their health for money and the other half destroy their savings with their health costs. However some decide to live working on something they love without dreaming for retirement. It is not only having luck neither being independent, although that helps a lot. It takes some self reflection too on what you really want and some effort on learning how to deal better with the money you make. At the end, the economic freedom does not only take money, but also experience and maturity. Usually that also takes longer than you might expect.

Misunderstanding Software Quality

2011-11-02T00:00:00+01:00

The word quality applied to software is full of false stereotypes and misconceptions. You should blame the materialistic view of other engineering disciplines have forced unto us. People hear quality and instantly they make a connection with high quality materials. All high quality building, car, machines, etc. have high quality steel. The main expectation from a high quality restaurant is a tasty dish with fresh ingredients and from a high quality hotel a big room furnished with luxus furniture.

So what do most people expect from high quality software? Programmers themselves and even IT managers have problems defining that word. In our field, even facing the doubt of what it is exactly, we all are 100% sure of at least one thing: it must be awfully expensive.

Take this further and you will immediately find a division within most IT companies. On one side you have the humble programmers facing all that mumbo jumbo from the QA department made of business graduates, social workers and God knows what other clans who insist in the importance of having the right page margins for their printed test reports. I have fond memories of a QA department that every time, after writing down three remarks forced us to make a new release and rewrite all manual test reports again before continuing with something with so many errors. For these programmers quality is all that nonsense that prevents real work getting done.

On the other side you have the managers, for whom the QA department is the least of their problems. It is usually all that whining and crying about how the code is such rotten ball of mud that cannot be fixed without being rewritten from scratch, or at least some serious refactoring. If the programmer is young then the solution is as easy as dumping evil Windows and alienating with the heavenly light force of Linux, open source software and super powerful language X.

Going along the lines of physical products and classical marketing mindset there is also a disturbing term but very lucrative: software tester. You know, those guys who receive the final product, or something that resembles it, and make sure that the quality is good enough for the users.

All I have been telling you so far is how most people experience and/or approach software quality. In my humble opinion the root of all this evil misconception when adding the word quality to software is thinking in a final product. Software engineering unlike many other engineering disciplines has hardly anything to do with a final product but with a process. The marketing department is the one that give it a face and make it a real product, software engineers only know about a process.

Think for a moment about the term high quality doctor. It sounds stupid and for sure it is a term you would never use. You might prefer a term such as good doctor, but I want you to even think about the difference between a good doctor and a quality doctor.

Quality for a doctor is usually a must and not an option. A quality doctor must find the best diagnostic and avoid spending money on expensive tests unless necessary (pressure of their hospitals and insurance companies). In a battle field they must endure stress, perform with scare resources and still be creative. Outside the industrial world their status and income depends on how long their patients stay healthy. Mistakes can be expensive and when they occur, they come back to hunt them down until the patient dies or sues.

Think about that last sentence. Here there is also a false association with physical products. Everybody knows that a low quality thing breaks sooner than a high quality one. This is also true with software, but there is one big difference. If a physical product deteriorates the consumer must face the cost of replacing it or fixing it. With software it is usually the other way around. Even if the company is not liable for solving the deterioration, its reputation and future sales are at stake. Software erosion cost money to the manufacturer, and this is solved with quality. This is becoming even more critical when most of our industry is switching from the make money selling new versions style to the make money providing a service.

The fact that Software Engineering is such a young discipline make it also more vulnerable to propaganda. I remember as a kid advising my parents which car to buy because the one I saw on TV could go under a dune and fly over a river (even though we did not have any of those around). Since the beginning big SW companies have been bombarding us with their advertisement and how all our problems would go away with their tools, languages and methods.

These days I am relieved that Unix came into existence and brought some freedom and common sense into our world. I am not only thankful because of the open tools but for the freedom to combine them, scripting, functional programming and above all a mindset looking for an alternative way of doing software beyond pure OOP. If you are old enough, you might remember how much money and obsession was placed into awfully expensive tools molding you into UML, one single language and one single OO programming style. If so, you will have noticed that today we face far less advertisement for software building tools. Our profession is evolving and we no longer have to pay for expensive tools (although there are amazing and unique paid ones).

One thing were I do not agree with most people is that high quality software must always be well tested and well document. Personally I believe that real quality lies in the balance between minimum cost and maximum profit from the economical and user experience point of view.

Most software projects are just trials to see if they work and can be sold. Here you can loose your hair and go wild for a while to see if it is really viable. As soon as you see a future, start taking more serious your process. So no, no that lovely documentation, automation, lovely UML diagrams and testing from day 0. Mmmm, maybe the UML diagrams could be ignored even when so many people insist they are a trait of high quality.

Although being a TDD believer, I am OK with non tested prototypes. What I am not so OK with is that awful false delusion of working during months or years without tests or documented code because it is just a prototype. The story goes like this:

No, we do not care about X because we just have a protype
(any pointy-haired person at this very moment)

Suddenly one day some manager from high above commands a day of work for the removal of the word prototype in all the documentation. This achieves in less than 24 hours the final product ready for testing and finishes the excuse for breaking all the broken software engineering rules and the crimes against science, software and above all, that group of life forms called programmers. So you see, at the end they are not liable to be sent to hell, sorry.

A prototype is code you throw away and only reuse what you learnt from the problem domain, that is, requirements and design. Even if you write it in something as low level as C++ or Java it is probably not a prototype.

High quality software uses continuous refactoring because of software erosion. Sadly when many people hear that word, the first thing that comes to their mind is a whole new fork that paralyzes the project and a bunch of programmers adding spaces to perfectly align their comments, painting nice diagrams fulfilling their UML porno with object-oriented-pattern fantasies.

There are far more refactoring patterns than the famous 23 object patterns. Halting a software project for more than a couple of days to improve something is not refactoring, it is a crime and the symptom that something really bad is happening.

Software quality is not something that must be sold or produced at a high cost. If you spend more time automating, writing tests or thinking about the problem than it is really necessary you got it wrong too, that is not quality either.

Software engineering is all about the process of building software. Since the user do not get the process but the result, software quality is something that mostly affects the manufacturer.

High quality software can be sold cheaper and adapted faster than the one of your competitors. The choice of technologies is realistic with the market and available pool of engineers.

Managers must find out which complains of their programmers are really necessary, but never ignore them completely because past the big ball of mud and garbage the next stage is the one of the big nightmare: Everything is so messed up that only the veteran programmers can understand the project.

If a programmer cannot be replaced or rotated at a reasonable cost, it is not quality software. Are you telling me that I need more than one month before I can do something useful for the project? Then the software quality really smells bad.

Projects in the nightmare level stage are burn out projects where programmers are constantly complaining about their life at work and specially at home. Managers should work hard to avoid this, because reached this point it is hard to find people who want to stay. The best guys fly away, and you are left mostly with the ones who do not care if the software breaks every day and new bugs cannot be avoided. The whole story about a minimum design, structure and quality is also to be able to hire and retain good workers.

Real quality is a combination of intellectual and creative work balanced at the lowest price possible to attain something reasonable, so please, starting today stop thinking that high quality software is something necessary expensive, it is just something awfully hard to achieve. Or do you want me to explain to you now what is a high quality painter?

Apple's Dirty Secret

2011-09-17T00:00:00+02:00

No matter how much you hate it or love it (or both), Apple is one of the most amazing companies out there. Even most amazing still is how its greatest achievements are completely ignored. Of course they are the ones selling iPhones, iPads, MacBooks and making megabucks out of them, but it is not the most amazing thing they have made.

The first and foremost complaint about its products is how overpriced they are. Many people keep away from them as a strategy to preserve their sanity and reproach Apple fanboys of being victims of some form of Stockholm Syndrome. Actually they are absolutely right, for that price Apple hardware is awfully overpriced, but why is everybody so obsessed with the belief that they are paying for hardware?

Most people have doubts about this world sanity whenever they look at the price of an iPhone and grasp a minimum understanding of its current market share. Most of my engineer pals are complaining all the time that a camera, a chip, a gps and a touchscreen packed together is not that much money at all. They are also completely right.

Out there many companies such as Nokia mocked Apple. "Are they crazy? Do they believe they can make phones now?". That was their reaction for a long time. Most likely they also thought something like: "I am going to pack this super camera, this super processor, this supper screen... and beat the hell out of Apple" and that is exactly what Nokia and others did. What could possible have gone wrong? Are the consumers becoming idiots? What is wrong with them?

There are many reasons to explain what happened. From a sociological point of view, one of our greatest human needs is the feeling of identity. For many centuries that was the feeling of patriotism and religion. Now people find that boring, they might not even be religious because being atheist is more fashion. Today brand loyalty, fashion and sport teams cover that deep desire of belonging to something. They have proven also to be much more profitable than religion for a wider margin of individuals. Except for satanical sects, there is already a long established monopoly of religions.

There is however one recurring mistake from an engineering point of view that many big companies still make. It is something that was defined almost one century ago. It is only happening now, but is so recent and new that there is no concious awareness of it. I am talking about ignoring the effects of ephemeralization.

In 1938 Buckminster Fuller coined the term ephemeralization to describe the increasing tendency of physical machinery to be replaced by what we would now call software (...) The iPhone and the iPad have effectively drilled a hole that will allow ephemeralization to flow into a lot of new areas. No one who has studied the history of technology would want to underestimate the power of that force.

Paul Graham in Tablets

Most people have an old conception of engineering products. They expect products to evolve in a physical way. Something you can touch, feel and measure. A car for example, is expected to evolve with a more efficient engine, having lighter but more resistant materials, being faster... but just like many other engineering products, there is a limit after which physical evolution does not affect usability that much.

I personally do not own a car, maybe some people smarter than I am might favor some brands, but the most important thing I would value from a car is having a GPS navigator. That is exactly a functional feature. For me and probably most people, any car cover the minimum material and physical qualities I demand.

Now imagine that all GPS devices can only work when placed inside one specific car brand. How do you think the streets of your city would look like? It sounds like an stupid idea, but something similar is happening now with that software patent mess going around, specially the tablets industry.

Still today the reaction of most Apple competitors is trying to release cheaper and more powerful products. Indeed Nokia and Samsung do have far more capable hardware. Their struggle for surviving proves the ephemeralization of cell phones. People already have the minimum hardware they need from any company. Their choice is made now on what they can do with it, and that is mostly provided by software.

So far most people using computers cared most for raw power. They wanted a solution that could open their word processor or web browser and not waiting for the cursor to move. Hardware during this time was quite expensive and what many people wanted could cost very well more than two months of salary. That was not a time when you cared that much for software. Hence everybody have this strong hardware-price connection in their minds.

All this time Apple has been more of a luxury product. There is something called software that they do really well, and doing it well is not only expensive, it also decides the overall user experience. It is hard to convince people of this when you have a market hungry for RAM, CPU speed, graphics card and all that hardware... until now.

People find awfully difficult the idea of paying big money for something you cannot touch. Apple marketing strategy was so far building amazing software and packing it into an expensive brand and overpriced hardware. Did you ever wonder why Apple tried so hard to bind their software to their hardware? That way the could transfer huge amounts of money to their software department and deceive the world as a hardware selling company.

During most of its life, Apple was more successful with professionals who benefited the most with the productivity increase, such as musicians and graphics designers. It was easier to lure this people with a lovely design and a standardised set of tools for their activities. Now that hardware is so cheap, Apple is testing their biggest deception strategy with a broader consumer market, and indeed, many high-end users are feeling that their Macs do not show that huge gap they had before. Obviously, it is far more profitable selling to the masses than to a reduced set of people.

No matter how much I admire Jonathan Ive for his designs, I do believe that the software will be the main engine of our products during many years. Maybe this was already obvious, but with the arrival of tablets the ephemeralization of many things will speed up. Your car might need already more than 100 million lines of code, that navigator is going to be integrated into your phone, and so many other devices. I as a software engineer, for sure will enjoy the ride.

Posting with Jekyll

2011-09-04T00:00:00+02:00

Transform your text into a monster (Jekyll website)

I always knew I should be posting more on my blog, but it was always so alien to me. I managed to set up first a WordPress blog, shortly after I switched to Drupal and that was it, almost no more dedication.

The thing is that it was really painful to write posts. All these click here and click there, with all the content spread across a database. For some text oriented guy who love to keep his work clear and under control, that was unthinkable. I knew everything I created was going to be inside that database, but just thinking about extracting that info scared me off.

Finally I decided to give Jekyll a try. I could have also gone for Hyde, the Python remake of Jekyll, but I wanted to have some contact with Ruby again. One more reason for Jekyll is that it is very well supported by my favorite coding comunity: GitHub. Hence anything said by Tom Preston catches my attention.

In one day I've been able to switch from Drupal to Jekyll. There are still some rough edgeds to solve, but I managed to build an initial new theme, migrate my Disquss and Google Analytics account along several minor details. Right now I am very interested in playing with HAML and SASS just to find out what this Ruby people are up to.

All my posts will be now in a simple text file formated with Markdown and a YAML header for the meta-info. So far it is amazing having all my posts with a such clean separation from the content manager. Not only I am convinced that now my work is less dependent on one CMS, but I can use the tools I am more comfortable with.

Since a picture is worth a thousand words, just see yourself how one of my posts looks like (with lists, code blocks, quotes, links...) and think all that was rendered from such a lovely text file like this one. All my blog structure can also browsed.

All this time I have been killing flies with a cannon. Any static site is much faster than anything dynamically generated. At the time I chose Drupal over WordPress was because I wanted to learn a more powerful framework, but these are programmed in PHP and now I realize how much better any solution can be developed with Ruby or Python. A static site also means that I can go for cheaper hosting and I will have to play much more with Javascript. That is a good thing because my Javascript-fu is rusty. Every month that goes by I am more convinced that it is becoming a very valuable language that complements any web project.

Most of the time and for most of people a dynamic CMS will be the best solution. Once I had client for whom I migrated their site to a CMS so their own employees could edit their web page contents with an easy WYSIWYG solution. Until that day they needed to call a webdesigner for every change.

However, if you are an Unix programmer, you must try some static content manager. It is far more natural for people like us to edit our posts with vi/emacs/textmate or whatever without even an internet connection. Add your favorite version control system and branch your drafts or other people contributions. Have a copy of your blog replicated in your computers. Filter your posts with grep, spellcheck with aspell, or script massive changes with regular expressions, that is how blogging should be for any programmer. Because no matter how good any of the other dynamic solutions get, your posts will never get the same freedom and workflow quality that you have with your daily tools.

What is the most compatible way to install python modules on a Mac?

2011-08-28T00:00:00+02:00

The following post is an extract from a Stack Overflow question and was automatically generated with the web scraping scripts at scraps.

Question

Question Source

I'm starting to learn python and loving it. I work on a Mac mainly as well as Linux. I'm finding that on Linux (Ubuntu 9.04 mostly) when I install a python module using apt-get it works fine. I can import it with no trouble.

On the Mac, I'm used to using Macports to install all the Unixy stuff. However, I'm finding that most of the python modules I install with it are not being seen by python. I've spent some time playing around with PATH settings and using python_select . Nothing has really worked and at this point I'm not really understanding, instead I'm just poking around.

I get the impression that Macports isn't universally loved for managing python modules. I'd like to start fresh using a more "accepted" (if that's the right word) approach.

So, I was wondering, what is the method that Mac python developers use to manage their modules?

Bonus questions:

Do you use Apple's python, or some other version? Do you compile everything from source or is there a package manger that works well (Fink?).

Any tips or suggestions here are greatly appreciated. Thanks for your time. :)

Answer

Answer Source

Your question is already three years old and there are some details not covered in other answers:

Most people I know use HomeBrew or MacPorts, I prefer MacPorts because of its clean cut of what is a default Mac OS X environment and my development setup. Just move out your /opt folder and test your packages with a normal user Python environment

MacPorts is only portable within Mac, but with easy_install or pip you will learn how to setup your environment in any platform (Win/Mac/Linux/Bsd...). Furthermore it will always be more up to date and with more packages

I personally let MacPorts handle my Python modules to keep everything updated. Like any other high level package manager (ie: apt-get) it is much better for the heavy lifting of modules with lots of binary dependencies. There is no way I would build my Qt bindings (PySide) with easy_install or pip. Qt is huge and takes a lot to compile. As soon as you want a Python package that needs a library used by non Python programs, try to avoid easy_install or pip

At some point you will find that there are some packages missing within MacPorts. I do not believe that MacPorts will ever give you the whole CheeseShop. For example, recently I needed the Elixir module, but MacPorts only offers py25-elixir and py26-elixir, no py27 version. In cases like these you have:

pip-2.7 install --user elixir

( make sure you always type pip-(version) )

That will build an extra Python library in your home dir. Yes, Python will work with more than one library location: one controlled by MacPorts and a user local one for everything missing within MacPorts.

Now notice that I favor pip over easy_install. There is a good reason you should avoid setuptools and easy_install. Here is a good explanation and I try to keep away from them. One very useful feature of pip is giving you a list of all the modules (along their versions) that you installed with MacPorts, easy_install and pip itself:

pip-2.7 freeze

If you already started using easy_install, don't worry, pip can recognize everything done already by easy_install and even upgrade the packages installed with it.

If you are a developer keep an eye on virtualenv for controlling different setups and combinations of module versions. Other answers mention it already, what is not mentioned so far is the Tox module, a tool for testing that your package installs correctly with different Python versions.

Although I usually do not have version conflicts, I like to have virtualenv to set up a clean environment and get a clear view of my packages dependencies. That way I never forget any dependencies in my setup.py

If you go for MacPorts be aware that multiple versions of the same package are not selected anymore like the old Debian style with an extra python_select package (it is still there for compatibility). Now you have the select command to choose which Python version will be used (you can even select the Apple installed ones):

$  port select python
Available versions for python:
    none
    python25-apple
    python26-apple
    python27 (active)
    python27-apple
    python32

$ port select python python32

Add tox on top of it and your programs should be really portable

A modern backup idea for your Unix system

2011-08-27T00:00:00+02:00

Backups are just like flossing your teeth: you should do it often, but at the end, you only do it during some weeks after you are into deep trouble.

What is the best solution for things you never like to do? The one that is as fast and effortless as possible, ideally fully automated. Because of that I usually suggest people to go for a USB powered disk. Don't get one with a power supply, here the comfort factor is essential. Any minor nuisance can trigger that famous "I will do it later...", so make yourself a favor: make your task as dumb as possible.

Now it is time to consider how to get your data there. Forget about tar, zip and all those compression tools. Compression takes time and these days you get plenty of space with any external disk. Go for a file mirror based tool such as rsync. Only if you go later for a two-way based synchronization go for unison).

Most backup posts I have seen are really classic, with the incremental, rotation and compress fluff, even the ones that mention rsync. What many do not mention is the magic "--link-dest" option. What is so magic about it? Well, it is better seen than shown.

I want to backup my Documents folder (the final "/" matters):

$ rsync -av --link-dest=/full/path/Documents/ Documents/ Documents.1

This will copy the full folder into Documents.1, but if you try to get their sizes you will see something strange:

$ du -h --depth=1  (Linux)
$ du -hd1          (Mac OS X)
110M    ./Documents
  0B    ./Documents.1
110M    .

Did anything went wrong? Well, go and see inside that empty folder

$ du -hd0 Documents.1
    110M    .

Now it is full? What happened? Try something else:

$ dd if=/dev/zero of=Documents/file_5MB bs=1024 count=5120
$ rsync -av --link-dest=/full/path/Documents/ Documents/ Documents.2
$ rm Documents/file_5MB

Here you created a new file with 5MB, synced and deleted it from your Documents. How does it look like now?

$  du -hd1
     110M    ./Documents
       0B    ./Documents.1
       5M    ./Documents.2
     115M    .

OK, now I owe you an explanation. The rsync command with the --link-dest parameter is making a duplicate of the Documents folder with hard links. Every file inside of Documents.1 is just a reference, so if you list the size of every folder, the command du will not count the size of the same file in different locations.

Inside every folder it will count the real size, but outside it will just give you the size of the first folder.

Later we added a 5MB file, synced and deleted it. So when listing everything only this extra difference was counted within Documents.2. Unix file system allocation is based on a reference counting algorithm. Hard links increase the reference count and only when you delete all the references the file is really deleted.

The real magic here is that with every sync you get a full backup at the cost of an incremental one! Every folder looks like a full copy, but only the real differences are what take space.

Now I leave up to you the exercise of finding the right command to save your backups!

Is it reasonable for modern applications to consume large amounts of memory?

2011-07-24T00:00:00+02:00

The following post is an extract from a Stack Overflow question and was automatically generated with the web scraping scripts at scraps.

Question

Question Source

Applications like Microsoft Outlook and the Eclipse IDE consume RAM, as much as 200MB. Is it OK for a modern application to consume that much memory, given that few years back we had only 256MB of RAM? Also, why this is happening? Are we taking the resources for granted?

Answer

Answer Source

Yes, it is perfectly normal. Also something big was changed since 256MB were normal... and do not forget that before that 640Kb were supposed to be enough for everybody!

Now most software solutions are build with a garbage collector: C#, Java, Ruby, Python... everybody love them because certainly development can be faster, however there is one glitch.

The same program can be memory leak free with either manual or automatic memory deallocation. However in the second case it is likely for the memory consumption to grow. Why? In the first case memory is deallocated and kept clean immediately after something becomes useless (garbage). However it takes time and computing power to detect that automatically, hence most collectors (except for reference counting) wait for garbage to accumulate in order to make worth the cost of the exploration. The more you wait the more garbage you can sweep with the cost of one blow, but more memory is needed to accumulate that garbage. If you try to force the collector constantly, your program would spend more time exploring memory than working on your problems.

You can be completely sure than as long as programmers get more resources, they will sacrifice them using heavier tools in exchange for more freedom, abstraction and faster development.

The Monk and The Philosopher

2010-06-11T00:00:00+02:00

My scientific career was the result of a passion for discovery ... In short, science, however interesting, wasn't enough to give meaning to my life. I came to see research, as I experienced it myself, as an endless dispersion into detail, and dedicating my whole life to it was something I could no longer envisage

Those were the first words from Matthieu Ricard that catch my attention. Finally I found an outstanding book from a scientist explaining Buddhism. This man, for all of you who doesn't know him, was a great biologist who suddenly decided to quit his scientific career to become a Buddhist monk.

The book is a continous speach betweek Matthieu and his father, Jean-François Revel a philosopher from whom I recomend his book "Neither Marx nor Jesus" about why two ideal-utopical societies have failed throughout history. Some parts may be some heaving reading because of the philosophical terms and some chapters target the Buddhist metaphysics concepts. However the whole book is a great insight into Buddishm and helps you understand some differences between the Western and Eastern societies... or maybe between Christian and Buddhist societies? At the begging I thought I knew the basis of Buddhism well enough, but the book seemed to discuss one topic I am interested.

Why Buddhism is becomming popular in the Western world? In a scientific world where religion seems to lose ground, a secular spirituality fills the human need of spiritual meaning without the religious framework which seems unpleasant for a growing number of people.

It has a wonderful discussion about how Western world has evolved with a science driven society while Eastern has been focused in the mind and personal growth as a way to achieve the enlightenment. It also clears several misconceptions about Buddhism and how it can enrich a cold science world. Philosophy is sadly loosing ground in Europe, maybe because many science fields have become independent and don't need it. It was fun to realize that at the beginning, philosophers were like Buddhist in the Western history. They were knowledgeable people seeking personal growth, people who were valuable advisors and affected whole societies. However with time they lost influence and focused on studying knowledge and society.

This book was everything I wanted. It was a great chance to discover the work of many philosophers I didn't know and others I just knew their name. A wonderful discussion of father and son, Philosopher and Buddhist monk about religion, science, Western society evolution and its growing lack of spirituality and humanity.

Fast comparison of char arrays?

2010-05-28T00:00:00+02:00

The following post is an extract from a Stack Overflow question and was automatically generated with the web scraping scripts at scraps.

Question

Question Source

I'm currently working in a codebase where IPv4 addresses are represented as pointers to u_int8. The equality operator is implemented like this:

bool Ipv4Address::operator==(const u_int8 * inAddress) const
{
    return (*(u_int32*) this->myBytes == *(u_int32*) inAddress);
}

This is probably the fasted solution, but it causes the GCC compiler warning:

ipv4address.cpp:65: warning: dereferencing type-punned pointer will break strict-aliasing rules

How can I rewrite the comparison correctly without breaking strict-aliasing rules and without losing performance points?

I have considered using either memcmp or this macro:

#define IS_EQUAL(a, b) \
    (a[0] == b[0] && a[1] == b[1] && a[2] == b[2] && a[3] == b[3])

I'm thinking that the macro is the fastest solution.

What do you recommend?

Update
I just read the article Squeezing performance out of memcmp usage which explains how the compiler (Visual Studio, but perhaps also GCC) can optimize !memcmp(..) calls.

Answer

Answer Source

I would go for memcmp()

It is more portable
I usually try not to be smarter than my compiler/language. You are trying to compare memory contents and (depending on compiler options too) the implementation of memcmp() should be the most efficient way to do that.

Also think that if your compiler does not inline memcmp() you will suffer the function context switch

Are you sure you need to optimize that hard? Have you already checked that your program spend most of its time doing that type of operations?

The Most Important Language for a Programmer

2010-04-11T00:00:00+02:00

Software engineers with low coding karma choose their favorite language based on money and job stability expectations. As a result you will often hear C++ or Java.

Fortunatelly there is a big group of people out there who take programming back to their homes as a hobby who will wisely reply back something like Python, Ruby or Scheme. I love this type of people and it is great working with them, but there is one single language that is essential for any programmer: English!

Joel Spolsky made this quite clear:

... The difference between a tolerable programmer and a great programmer is not how many programming languages they know, and it's not whether they prefer Python or Java. It's whether they can communicate their ideas. By persuading other people, they get leverage. By writing clear comments and technical specs, they let other programmers understand their code, which means other programmers can use and work with their code instead of rewriting it. Absent this, their code is worthless. By writing clear technical documentation for end users, they allow people to figure out what their code is supposed to do, which is the only way those users can see the value in their code. There's a lot of wonderful, useful code buried on sourceforge somewhere that nobody uses because it was created by programmers who don't write very well ...

Although Joel Spolsky's comment applies also to great coding style (naming, asthetics, abstraction...) it uncovers the importance of writting code for other people. During the first years most programmers (me included) try to squeeze that tiny extra code branch for the sake of speed optimization. Sooner or later we get frustated when we must work on existing code because the truh is that writing code that a human being can understand is awfully more dificult than doing it for a machine. Even more scary is that code must be read countless times more than it is written. If you ever must certify or test other people code... let's just say life goes on after work.

I am a native Spanish speaker, document my work in English and live in a German speaking environment. I know that many of my clients would like me to produce German documents, which I always gently decline. It will be a long (very long!) time until I can produce German documentation with the quality of a native speaker. In any case, I encourage my clients not to write their documentation in anything else than English. The official lingua franca for software projects has always been English and anyone ignoring this fact will severly limit the pool of resources and talents that can be invested.

Nevertheless, the difficulty is not the language itself, but being able to communicate ideas in a clear, concise and direct way. For a software engineer I would also demand writing in an pleasant and telegraphic way. Why? We must digest thousand of manuals that usually are verbose and poorly written. Everybody hates reading manuals, but reply any question with RTFM

This was one of the reasons I started this blog. Not only it is a great excercise for my English (which still must be improved a lot), it helps me to practice my writing skills. Before entering college I took two years of literary expresion (as a hobby) and so far most people told me that looks like a waste of time.

A proud programmer should care about the quality of his written communication skills. Discover that most code editors do not spell check the comments and that poor human-friendly-code will be the blood and tears of many colleages. In any case, just to get you started, always think about this quote before writting any single line:

Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live

XML for Compilers

2010-03-27T00:00:00+01:00

Thinking about how to implement a tool for measuring source code smelliness, my main concern was extracting the syntactic elements I wanted from a programming language.

Obviously plain regular expression matching are not the way to go. I decided to opt for a reduced AST representing the common elements of most imperative languages: functions, variables and some type of packaging (name spaces, files and/or classes).

Even when I could easily build an AST on my own with Flex and Bison, I started looking for projects that had already taken care of that. I discovered many tools with no apparent code reuse, they all implemented their own parser for the same language. It did not take too long to realize how useful it could be using XML, a context free grammar that could represent every detail I wanted.

Like most of the Aha! momments of my life, just a short glimpse on the internet made me realize that not only people thought about that before I did, they did long time ago YZKK01TCDJR04. So I will just sumarize here (and force myself to think) about what I want from XML.

Portable representation

XML is the bridge that eases the flow of data structures among different systems because binary representation leads to several communication problems.

A compiler is the king of the tools dealing with our code but there is a whole array of development related applications that also demands (or benefits of) an understanding of our code: debuggers, static analysers, IDE environments, editors, code browsers, bug trackers, documentation systems, requirement management, version control...

Just for the first type of applications we came up with debugging data formats such as DWARF and similar variants. What about the rest? Well, every single of them seem to parse code on their own. When they just don't care, they lack functionality which could help programmers.

Even if the LLVM library model makes code reuse easier, a common way to represent the basic elements of a language could reduce the development effort of those tools.

There are also plenty of commercial tools which will not disclose their data structures. These tools could work with the XML representation of their targeted language and the integration with any other tool would be immediate.

Defined conversions

Now you might be thinking that the differences among languages, tools and lack of a common standard are hard stuff to chew. Well, that is where XLST can help. Just look at this small piece of XML The example was taken from the Wikipedia article about XLST:

010    <?xml version="1.0" ?>
020    <persons>
030      <person username="JS1">
040        <name>John</name>
050        <family-name>Smith</family-name>
060      </person>
070      <person username="MI1">
080        <name>Morka</name>
090        <family-name>Ismincius</family-name>
100      </person>
110    </persons>

With XLST you could define a set of rules for translating that into something like this:

010    <?xml version="1.0" encoding="UTF-8"?>
020    <root>
030      <name username="JS1">John</name>
040      <name username="MI1">Morka</name>
050    </root>

The application to developer tools would be great because the XML output of a compiler could be reshaped and filtered providing just the basic elements that other tools need. Even it would be possible to have be different transformations depending on the targeted tool.

Binary mapping

There will be a concern about the efficiency of these methods. People with a compiler background, used to highly optimized solutions, might be skeptical thinking about the costs of using XML.

Even if we prevent several other tools from parsing over and over again the code, it just doesn't look OK to translate the internal binary AST into the text of an XML file, transform and back again into binary.

Well, that is what XML static binding solves. It defines the binary representation of the XML nodes. A compiler can offer the XML generation for the sake of portability. However the binary representation of a full or reduced AST could be produced and other tools will have a common medium to understand the structure of such dump.

Today compilers are envisioned as a single tool in every programmer's computer. There is a possibility for the current distributed compilation systems becoming more common than we think. Just the same, other developer tools dealing with documentation and requirements might also evolve into distributed environments. The XML representation will be a great asset.

Conclusions

Most programming languages have common elements such as functions and variables. With a unified representation, some tools could be expanded and provide us with more information, no matter the underlaying language.

Imagine a bug tracker that automatically list the functions modified when solving a bug. A VCS listing the history of functions not just the history of files, also it could define modification rights classes. A documentation and/or requirement tool adapting to a user defined notation. Is there any easier way to encourage the development of such tools?

The Smell of Your Code

2010-03-27T00:00:00+01:00

You should not be surprised that a guy who deals with garbage also feels concerned about the smell of your code. If you want to save the click to the Wikipedia article, I can summarize the concept rather quick: Code smell is all those symptons or things you see in code that raises a warning flag in your mind.

If you ever used a static analysis tool to verify your code, you already have covered some of the basis. Those tools will prevent you from shooting yourself with a completely valid syntax (think C):

010   if (entry = valid)
020      result += entry;
030      result = 0;

Once I believed that static analysis was a patch to the ugliness of poorly typed languages whereas properly designed ones never needed to cover for this type of defects. Considering all the factors of smelly code I was wrong. There are plenty of issues in any programming language that can be improved with the right analysis.

The Freudian Approach

Programs must be written for people to read, and only incidentally for machines to execute Abelson & Sussman, SICP

How would you feel if you encounter these variables in someone else code:

SYSTSTS and SYSSTSTS
open and Open
obdoK and obdcK
noEntries and numEntries

Now you might understand the difference between a normal static analysis and code smell. Languages like Ada are not case sensitive because of the second example, but clearly misses the rest. I find the psychological concept quite exiting. We come up with Fortran and C as our first intermediators between the human mind and the machine mind. Other languages followed (functional, logic, object, aspect...) trying to offer a better adaptation of the way we think.

A simple tool with a word proximity algorithm could warn you about those cases. By having a strict naming convention extra checks could be carried. Last line could be further improved enforcing a dictionary of abbreviations.

From a Freudian point of view. This is just the tip of the iceberg. More complex metrics can analyse code complexity, design cohesion, object coupling. All are elements affecting our ability to work with code.

Style matters but no one cares

Most programmers don't use code beautifiers as often as they should, maybe just the good veterans. However simple beautification is only one step towards a unified looking code.

Add naming conventions and syntactic restrictions to the mix, make it part of the compilation process so any fault stops the building process: You will have a tool teaching you the right style. No more discussions about the name of that variable, the position of that bracket or where is that required in the standard. Programmers have too much information to read, a tool enforcing a common style can save a project from overwhelmed programmers who cannot afford learning that long list of apparently silly things.

A strict coding style is greatly underrated. Without a good tool it is easy to ignore some rules, but the larger the project the more important this is. Several huge projects are implemented in a language with a poorly designed packaging system: C. A solid naming convention to emulate a name space is far more important in that case.

However, overall, no matter how good your language is, code is just more maintainable if it looks uniform. At a very minimum, comments should be written in the same human language.

Code metrics on steroids

I believe that the combination of several tools, maybe with the help of XML, can yield far useful metrics for any programmer. Consider this scenarios:

Your bug tracking system counts how many times the class you are debugging had issues in the past. Software projects have a tendency to have trouble maker classes and it is easier to track them if your system knows about your code structure.
Recently some of the automated tests have failed. A VCS can spot recently modified functions related with that test (linking requirements and tests; also the functions used by the test).
Most VCS tools can identify the authors of every line. You could add to your classes a list of experienced programmers, most active ones and modification history. Sometimes it is not that easy to know whom you may ask.
The reliability of one class can be evaluated measuring how often it is used, when was the last modification time, the length of the applied fixes and how old it is. If many people have been used one class without any problem, it should be less likely to some will raise.
When many requirements are implemented within the same class, you start getting the idea that something is failing in your design. Maybe bad coupling. The class tries to achieve too much and its maintenance is clearly troublesome.

Now combine all the above with some classic metrics: your smelliness index!. If just your debugger/IDE could display that information in some smart and attractive format... you should become a very happy programmer!

Evil Side

In some dark corner of company, some manager might have the evil idea of using metrics for the competence appraisal of his/her programmers. That would be like killing the goose of the golden eggs! Software projects don't fail because of the talent of its programmers, they fail because of the competence of their managers. The work quality of a programmer is likely to be affected by the tools, the time and decisions from other people.

Most programmers will love to improve their work and get the best metrics if these prove to be useful. However within a company it will be quite likely that they are told to focus on something else like new features and never in quality refactoring.

Conclusion

Although I started commenting just on code smell, my objective is to raise your awareness on the importance of good metrics. Another of my concerns is the poor connection between different development tools. I hope this inspires you to support some free software project. If you are interested in these missing connections you should read my post regarding XML for compilers.

If you want to read more...

Weinberg98

McConnell04

AHDT99

Inforich or Infobloated?

2010-03-21T00:00:00+01:00

In an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it
Herbert Simon (1916 – 2001)

Around 1996 I had my first experience with the Internet. Back then you didn’t have as many net-terms as we do today, but one which immediately catch my attention was “Info-rich”.

The word describes the access to knowledge within the information society and the internet literacy of people. As counterpart, little or no access at all is known as Info-poor. Later known as the digital divide, it is not only a new distinction in today society classes but also a way to describe the gap between developed and developing countries.

Lately I’ve been concern about a little ability of mine which I share with many people. Somehow I manage to capture a huge amount of articles and information I want to digest, not only from the Internet but also TV programs, newspapers, books, courses… and my inbox just keeps growing. My efforts to satisfy my curiosity about several subjects has only lead me to a poor time management. That’s why I know I don’t fulfill my efforts on my projects, not even my free time.

I am starting to develop my skills on attention allocation. Otherwise I will no longer consider me as info-rich but as info-bloated. Some state where the information becomes useless because the quantity which keeps coming into my inbox only makes me information bloated. As time goes by, this started to affect my own personality. Just like information, I try to grasp many personal projects to the point I stop being efficient. Or was it the other way around?

Now I process information just like I take money decisions. Before putting something in my inbox I formulate the same question: Is my attention worth the cost? An inoffensive article might seem inocuous. However they build up to a great attention comsuption with huge but unseen costs.

Kent Thune post on attention economy and finance, from whom I discovered the Herbert Simon’s quote, has been writing about this issue which I want to focus more on personal time management.

The short message here is to encourage you to GTD yourself. If David Allen’s methology doesn’t apply to you find something which does. I keep a constant effort to apply it on my life and I do believe his advice and tips are priceless either in my professional as my personal life.

In order to become a happy organization-freak you must learn to process all information attacking you. The key point is to fine-tune your skills to reject information and digest what you decided to let through. Then some extra organization tips will help but two are a must.

The Flight from Truth

2009-11-06T00:00:00+01:00

The foremost of all the forces that drive the world is falsehood. More than any before it, twentieth-century civilization has depended on information, teaching, science, culture - in short, on knowledge as well as on a system of government which, by its very definition, seeks to make knowledge availabe to all: democracy. [snip]Those who act have better data on which to base their actions, and those on the receiving end are much better informed about what those who act are doing.

It is therefore interesting to inquire whether this preponderance of available knowledge - with its detail, its abundance, its ever broader and swifter disseminiation - has enabled humanity to guide itself more judiciously than in the past. The question is all the more inportant since the perfecting and accelerating of the techniques of transmission and the steady increase in the number of individuals who benefit from them will make the twenty-first century an age in which, even more than in the twentieth, information will be a central element of civilization.

Jean Francois Revel

Ada Love Story

2009-10-27T00:00:00+01:00

Programs must be written for people to read, and only incidentally for machines to execute Abelson & Sussman, SICP

Right before my first programming assignment at my University, we all had big expectations regarding the language we had to work with. Back then Visual Basic, C++ and Delphi were the big guys, but I as much as my colleagues agreed, maybe that was a too much for starting. C seemed the most promising option, it was after all, the language of the hackers. All the real macho programmers could code any thing they dreamed in it.

You cannot imagine our surprise when we found out that our assignments were to be made in something called Ada. Greatly concerned about the quality of our education, we started our own research.

A language from 1983? Come on! It is older than C! It looked like some evil board of professors wanted to punish us with a Cobol or Fortran look alike. Sadly, even those languages were heard of, maybe as dinosaurs, but even so, much more than it. What was it good for anyway? That thing was born in the hands of the USA army, but they seemed to ditch its development. At first sight, not a single company cared about it.

Ever since all went just bad. Suddenly programming became harder. That stupid compiler could not stop complaining at every single line of code. The semester went through with a great deal of pain. After such experience, the question about which programming language was used in every other course had quite a deep meaning along it.

However I've always had this nasty habit of calling for trouble with the most challenging solutions. So I started picking up language and compiler oriented courses. I've been always convinced that understanding what is going on between the language and the machine can help me to become a far better programmer. That way you can quickly find the most efficient way to execute your code, the safest coding style, all your options... and quite specially, to understand those moments when you have a hard time trying to understand why your code is not doing what is supposed to do. You might very well call it: the Coding-X-Vision Superpower.

On the other hand, compilers are the most exciting tools to work on because you can put to work everything you know about computer science. You must have highly complex data handling, concurrent/distributed programming, come up with all kind of optimizations. The design and robustness must be perfect because compilers are supposed to live an eternity while having always room for improvement. No other thing will demand a better understanding of the operative system and the processor. If on top of that you love the job well enough to be turned on with some assembly lines of code. What else might you dream of?

Much to my surprise I noticed that some of the courses used Ada in their lab assignments. It caught my attention that some things could be only done in it, and even if something was done in another language, it had its origin in Ada.

Before Ada 2005 object oriented programming in Ada didn't look as elegant as what you can do with Java, but even so, Ada was the first standardized language to support it. If you use exceptions, you should say thanks to Ada. What if you want to use threads or develop a distributed application? Hats down for Ada again. Not that the beloved C/C++ cannot do it, they have libraries for that, it is just that Ada defines that in its own standard, plus, they had it even before C. Standards translate into extreme portability which software that behaves exactly the same everywhere. One of Ada big points is that all its features are exquisitely defined and well documented. No place for surprises! No Sir!.

The story went on from distributed computing to language and compiler design. I started to learn about those ambiguous legal constructions in the main languages and the gaps in their definitions. Something compiled with no problems but then something failed. If by then you tried looking in the standard you might find the expression behaviour undefined. Well, that is a very bad word you will not find in Ada!

When I had to choose a project to finish my career, I went for Ada, and although that made my life more difficult than necessary, I definitely developed a great admiration for the language. Maybe it took me way longer than any other project, but I had the chance to work with two great engineers: José Fortes, highly skilled in compilers and Javier Miranda, a tireless GNAT hacker.

My experience with my professors and Ada has changed the way I think about programming, refined my tastes and influenced my career life before it really started. Although I hope this article raises your interest for the language, I will not try to convince you to use it. Sadly it is a greatly misunderstood language and you will not find many opportunities to work with it (neither do I). I just wish that more people could value Ada for what it is, and how it has affected so many other languages that despite being more famous, still cannot beat it.

Why Garbage Collection?

2009-09-28T00:00:00+02:00

Lisp has jokingly been called "the most intelligent way to misuse a computer". I think that description is a great compliment because it transmits the full flavor of liberation: it has assisted a number of our most gifted fellow humans in thinking previously impossible thoughts
Edsger Dijkstra, CACM, 15:10

Just as filthy as garbage collection sounds, it is the annoying feature many computer scientists disregard. The most common opinions are that it encourages bad programming practices, it will prevent you from learning important mechanics about the memory consumption of a program and finally, even the most complex problem can be solved with explicit memory deallocation.

Well, all of that is absolutely right. So why care about it? Why on Earth would I try to ease its adaption in such an excellent language as Ada GarciaEtAl07? In my opinion the following assumptions might explain much of its unpopularity:

Companies care to do software right
Companies only hire good people
There is always time and money for the best solution Some would argue that the best solution is the one that matches best the allocated time and money
Garbage collection encourages lazy programming

Software Engineering is all about dealing with complexity through several layers of abstraction. Why not making memory handling part of that? Software always evolves into bigger and more complex creatures. Since memory handling is a recurring error, spending time fixing such type of errors or taking care of a proper memory deallocation is a quite time consuming task. As many Lean methodology adventists would argue, it is not something that really adds value to the final product

The need to understand how a program uses memory is clear, but the driving economics factor behind any company makes even more clear that garbage collection is important. Releases can be done faster with less software bugs. Arguably companies care more about releasing something than how it has been achieved. Hence I do believe that if Ada had garbage collection, it would not be better, but it would raise more interest.

C and C++ are facing this dilemma. Many programmers want garbage collection supported into their favorite language, but there is no common agreement about how this should be done. Ellis and Detlefs have long argued about it EllisD94, and yet, the Cxx0 standard has postponed the issue for a future revision.

What is happening? I think the current solution is partly to blame. A conservative garbage collector boehmWebBoehm88 can be easily deployed with non cooperative languages such as as C++. It doesn't need changes in the compiler and mostly, only demands replacing the malloc() and new() calls with an alternative function. This have covered the needs of many garbage collection advocates, but has not convinced the mainstream.

Some software products like Firefox have successfully used this alternative, and Zorn has proven that this type of collectors really work well berger01composing Zorn93. However in my humble opinion, I believe it is only a problem of faith. Conservative collectors make many heuristic assumptions. They explore the whole memory trying to guess what looks like a pointer. Obviously, computer scientists feel nervous loosing control of their memory to a mechanism that looks more like black magic. Touching here and deallocating there without delimited control. Furthermore they are not very portable because of their tendency to rely on special mechanisms of the underlaying operative system and/or microprocessor (paging, protected memory regions, etc)

Zorn also made some promising discoveries comparing implicit and explicit memory handling, but I miss one important point. There is a huge code legacy in Ada/C/C++ doing explicit memory handling. It is quite likely that when using a conservative collector, it will expand without control to the memory managed by such code adding the cost of implicit memory deallocation to the current implicit one.

Hence I have my doubts that conservative garbage collection is the answer that we all will see in future language standards. I believe that the secret lays within implicitly deallocated containers provided by the languages themselves. By providing garbage collection to a delimited subset of a program, one will have a better control of the processes, along an estimation of the increased memory and CPU usage. Such solution should ease the mind of its detractors.

However it will not be an easy task. The road is full of conflicts and semantic restrictions. Such containers will demand more information about the memory than old languages currently provide. The essential need of a garbage collector is its ability to extract the pointers to collectable objects within a specific type layout and the program stack. If the languages don't ease the implementation of custom collectors, there will be little hope to any improvement in the current situation.

Probably we will need two standard revisions of our languages to see garbage collection made common. The first one should provide the means to build precise collectors and the second define the semantic restrictions to use a garbage collected container.

Addressing this issue is a good way to extend the life of Ada/C/C++. Since the complexity of software will evolve, I can think of two historic events:

Assembly has mostly been replaced by higher level languages such as C. Despite many people defending real machine code programming, C became the standard. Now even object oriented languages such as C++ and Java are making their way into the embedded market pushing away C.
When the gap between CPU and hard-drives grew too large, complex file systems emerged with more efficient storage-retrieval algorithms and new features never thought of.

In the same way, the gap between CPU and memory speed can grow so large that advanced memory algorithms will emerge. New advantages not considered so far might surprise us. For example, with an increasing demand in portable computing, it will be interesting if memory structures could re-arrange themselves. An optimized object layout will improve cache hits and reduce the memory regions that need to be powered, saving on energy.

In one academic study the choice of a memory management strategy made a program consume 40 times more energy Diwan02energyconsumption. Garbage collectors proved to be quite energy inefficient by then. It is curious that Java is the first choice for many portable devices, but more research is trying to close the gap CSKVIW02.

Obviously such achievements will take some time. However our future researchers might lack a more fine-grained memory control from our languages, like the ability to pin-point pointers within memory. Improving languages and compilers to provide the needed infrastructure for garbage collection seems a good excuse to start. Future advanced memory algorithms will benefit of this one way or the other and maybe, in that future it will look unconceivable that programmers take care of their own memory.

If you want to read more...

07whatevery

AltedFr10

The Memory Management Reference
http://www.memorymanagement.org/

Those Nutheads Writting Blogs... Oh, My!

2009-07-05T00:00:00+02:00

The greatest hope of internet generation is that you can share your thoughts with everybody in the world. The greatest letdown of the same generation is that nobody cares. Still, that doesn't keep us from trying.
Ted Dziuba

I don't have any doubt that the first thought of many of my friends reaching this blog will be something like "I knew this guy was nuts!"

Yeah right. Why would any normal person keep a blog? Don't only nerds and such sort of socially troubled people do it? Come on! It is a lot of work for just exposing one's private life and simple minded thoughts. Isn't it?

Well, keep thinking that way and you will be missing a great tool for improving your career, personal growth, social connections and much more. It is the single most important thing you can do in social media if you take that seriously.

Unless you understand the reasons for reading a blog, I have my doubts that you will understand my reasons for blogging. However for explaining this, I can give you a short but detailed list of my reasons.

It is an extension of my mind

The life of a computer scientist, like those of any other living entities, is full of new discoveries. In my case I have a problem to have a grasp of all of them. Whenever I learn something new I try to write some notes about it and put them together in this blog. Whenever I am I can read them, and if any friend of mine is battling with the same thing I can point them here.

You will never get a better review

If four eyes sees better than two (supposing that none of the subjects is one-eyed), how better can see thousand of eyes? Any of my personal values or opinions can be discussed by anyone. I would be excited even if someone makes a bad review. As far as the guides I wrote about before, if I am missed something, hopefully someone will tell me.

Your best loved ideas might expand

When I wrote my master thesis I was really excited, so much that I decided to make it available on the internet. I would feel really proud of it if someone tries to continue that work. Meanwhile I push some things on my free time and anyone can criticise it and is free to come with ideas that can make it better.

It makes me think

When I write I force myself to put straight my thoughts and make some logic out of it. Sometimes it becomes an objectives declaration and I realize not only the power of putting down with words my plans, but showing the whole world about it. Overall and for the same reasons, sometimes it becomes quite a psychological test.

It is a great connection

I expect people with similar thoughts and interests to keep reading this post. I have no doubts that with them I will make more complete my knowledge about the things I write. Specially, when they start responding with smarter comments than my posts.

Still Alive Proof

I am a lowsy penpal, most of my distant friends hardly ever get news about me, except for the important dates. Blogging is a way to show them I am alive and with a personal section they can read about my life.

I don't speak English

At least not natively. I have already been accused of high treason among some peers, but come on, I already know how to write in Spanish. A good command of the English language is always a nice asset to have. Being an Engineer myself I am forced to write technical documents in English. Although anyone can understand what I write about, I would rest better thinking that my documents are something decent and error free. This blog is my biggest tool for learning properly the language, but sometimes feels like overly bold to write so much in it.

Need to practice my writing

I spent two years in a literature course and some years later I had to write my master thesis. My best experience out of it was learning how important it is to craft effective sentences and choose the right words. Between the course and my thesis I had not written too much and I realized that one not only needs a lot of practice to write properly, but needs to keep doing it. My skills are not as good as back then, but I have no doubt that within some years I will be able to look back my first posts and cringe when comparing the level I will achieve.

The Magic of the Blogosphere

2009-06-06T00:00:00+02:00

When the Internet was born many people understood how important it was the ability to publish any content and reach the whole world at almost no cost. The freedom and the resources have ever since been there, but maybe this was never really exploited until the blogs appeared.

A blog by definition, is just a personal journal published on the internet. Usually they are focused in one specific subject. Think one word about any hobby, political trend, professional activity... anything! and you will find someone blogging about it.

The big difference from any other of your reading sources (and the reason because people don't usually take it seriously) is that most of them are written by normal people. Anyone can have a blog for free in a few minutes. This usually comes along low quality, non reliable, non professional and many other non desirable things. However such freedom gives a voice to interesting people who would never get noticed.

On the other hand books, articles, magazines or any other communication source have an industry behind them. You get contents usually with higher quality and made by professionals, but not without the control of some kind of institution. Today corporate behemoths, usually with some political hand behind, own the major outlets of our news, opinions and entertainment. What you read has been previously classified as acceptable and profitable. Only in some cases this is really an advantage, like when an academic institution ensures that its articles meets a minimum of quality, veracity and grounded research.

Blogs when you need information

Once you add some blog reading to your internet routine, it will quickly become your best source of information. Look at it as a specialized newsletter that filters everything you can find in the internet. Some might become your entry point to the social circle you needed to find someone to help you, work with you or learn with. Good blogs will point you to places and people with a common interest. Many people find the right partners for their projects this way, even scientists are finding out that they can share, develop and review their work through blogging.

Blogs for entertainment

I bet this must be the best reason for many people to surf the internet, right after sex. Latest news about their favourite topics, jokes and funny videos.

If what you are looking for are good stories I must tell you that they are hard to find but I have come across some really talented bloggers out there which I truly enjoy reading every so often. However, this is an area that I have not fully explored. Blogs are good for learning new things because they are full of fresh ideas, independent and with many points of view, even it their contents are of just cheap quality. However when it comes to reading for leisure I care far more about the literary quality. I might be somehow demanding because I spent two years of my life learning about literature, but a good novel can be a piece of art and I still have a long list of classics to read yet.

Blogs for bibliographies

Most people think of bibliographies as something boring, and sometimes I think they are right. If you walk into a book store you will mostly find two types: The ones about famous some-times-interesting people who died a long time ago and the ones about boring living people who have money enough to pay someone to write about them.

However blogs, and specially those written with more personal entries, have something special. They are from normal people which which would never publish their thoughts unless being so plain easy. This way you might find out about the life of a person that fits your interests. Let me explain this with an example.

Imagine you are a young entrepreneur in front of a small company and willing to digest any material which can help you lead better your company. Sure you can find books about management, leadership and even about the specific market of your company. However, something really valuable that is hard to find out of the blogosphere would be the personal diary of people who started like you, who battled like you, suceeded and is "currently" writing frequently about it.

It can be people who succeeded in something you are trying to achieve, a mother telling the world her challenges raising her kids in this world, the man who decided to quit smoking and run a marathon. Sure that you will be exited to read the experiences of someone whose objectives resembles yours. Although there are books about anything, I believe that in this case there are many small voices worth listening to.

Start reading!

Choice is wide and variety great. You will find such a quantity of contents that you will have to develop new skills to digest such outrageous amount of information. Many times you will discover that there are many people out there with too much free time. You might get that feeling when you find something like a 12 hours detailed log about the activities of a fly in someone's garage, with pictures, videos, animations and detailed diagrams. Except those cases, I assure you that there is such a thing as serious, high quality, professional blogging. But once again, the value of blogs is not in their quality, but in their independent opinions, fresh ideas and unique perspectives. Some of their contents are made with various voices from different parts of the world coming together. A blend of the thoughts of people with different backroungs, experiences, stories, religions...

But be aware that freedom always comes not exempt of surprises. Blogs can be hoaxes like the story of Kaycee Nicole or part of some marketing campaign. Professional blogs are also cash machines and the bigger they are, the more chances their contents might be moved because of money.

Dear John, the machines are already taking control

2009-02-22T00:00:00+01:00

Year 2029, the world as we know today no longer exists. Intelligent machines with human form have taken control and only a small human resistance group struggles to survive.

Are we there yet? Well, not quite, but a similar fight is already going on.

After reading Is Google making us stupid? by Nicholas Carr I realized about our current struggle in the Great Invisible War Against Technology. It was the triggering point that gave me the idea of this blog when I was still under the influence of The Monk and the Philosopher . That was the moment when my passion for Psychology and Philosophy applied to Computer Science was awoken.

Carr made an amazing point in his article. Internet is changing the way we read and many of us are having a hard time to focus in long texts. However with examples about how Nietzsche work changed when he got his first typewriter, one could expand the concept to a bigger idea: Technology is changing our values and the way our brain works.

We are not only what we read ... we are how we read ... the style of reading promoted by the Net, a style that puts efficiency and immediacy above all else, may be weakening ... our ability to interpret text, to make the rich mental connections that form when we read deeply and without distraction
Maryanne Wolf and Nicholas Carr

Our writing tools are also working on our thoughts
Nitzsche

Socrates was alarmed about books, he was concerned about people basing their education on them because they are only a medium to convey information. It is only through dialog one could gain wisdom and a better insight on knowledge. Therefore he never wrote a single one. The only thing we know about him is because Plato, his student, decided to write his conversations.

Today most of our education is based largely on printed material and I will not argue how a valuable tool it has become. However and just like Carr pointed out, now we struggle to grasp some insight among so much information (Information Oberbloat), our capability to concentrate in long materials has been diminished.

Living in an age where most of our knowledge comes from movies and internet, I would like to mention Star Trek. I was quite grown-up when I became a fan of the series. Today it might be shocking to endure such awful special effects, but once you contemplate the show as a historical workpiece, it is possible to take out many technology-philosophy concepts such as soul existence, abortion, religion or human genetic engineering .

The whole show was an attempt to say that humanity will reach maturity and wisdom on the day that it begins not just to tolerate, but to take a special delight in differences in ideas and differences in life forms. We tried to say that the worst possible thing that can happen to all of us is for the future to somehow press us into a common mould, where we begin to act and talk and look and think alike. If we cannot learn to actually enjoy those small differences, take a positive delight in those small differences between our own kind, here on this planet, then we do not deserve to go out into space and meet the diversity that is almost certainly out there.
Gene Roddenberry

Technology can solve all of the problems of mankind, including the problems that stem from human nature such as war, hatred, greed, etc
Scott Whitmore

The three main characters Kirk, McKoy and Spock were the portraits of different types of people. One of the most recurrent events were the fights between Dr. McKoy and Spock about human nature and logic. Most of their discussions represented the importance of human concepts to discern between the best actions versus the right actions.

Later the series evolved with complete different players. Still though, the show was like classic Greek mythology, every actor represented different human virtues and defects, most of them being quite immature. The role of Spock was replaced by an android called Data which still represented after all, the conflict between logic and humanity.

However one great addition was a new enemy, the Borg race, a machine civilization which slowly assimilated every living creature in the universe and whose moto was "Resistance is futile".

We don't need to wait so many years for such futuristic fight. Sadly, machines do not need to be that advanced to affect humanity. Many humans do not remember important dates if it is not in their electronic agenda, their friendship circle is extended by hundreds of Facebook contacts and sometimes it is their only social circle. Our ability to concentrate deeper in content, art and relationships is getting harder and harder. The only difference of this fight is that it is not led by violence, but by seduction.

When I think how future might be alike, I take for granted that society will be technologically more advanced. However when I try to imagine how will be the human being of the future I am completely lost. Still looking back in our history I recognize similarities with our current world. Ancient Rome and Egyptian societies were led by commerce, law and labor. There were social layers, justice and injustice. Faith fights. Buddies met after work for some beers. Parents tried to teach their children about life while children taught their parents what life is about. Happiness was based on security, money, love and health... Will we always be like this type of human beings? Or will the rich and poor worlds drift apart as far as Diaspar and Lys did?

He found part of his answer among the children, those little creatures who were as strange to him as any of the animals of Lys ... Sometimes it seemed to him that they were not human at all, their motives, their logic, and even their language were so alien. He would look unbelievingly at the adults and ask himself how it was possible that they could have evolved from these extraordinary creatures who seemed to spend most of their lives in a private world of their own.

And yet, even while they baffled him, they aroused within his heart a feeling he had never known before. When-which was not often, but sometimes happened-they burst into tears of utter frustration or despair, their tiny disappointments seemed to him more tragic than Man's long retreat after the loss of his Galactic Empire. That was something too huge and remote for comprehension, but the weeping of a child could pierce one to the heart.

Alvin had met love in Diaspar, but now he was learning something equally precious, and without which love itself could never reach its highest fulfilment but must remain forever incomplete. He was learning tenderness.

Several science fiction works have based their plot on the use of genetic engineering to enable parents selecting special traits in their offspring. Society becomes divided between genetically favored humans and those freely bred. It is easy to imagine that as technology evolves, certain achievements of Diaspar such as advanced liberal eugenics could be emulated producing new human beings in their adult stage. Could you imagine a world without children? With people who never met one? It might be feasible but... Do you have hopes for a future in a world with such type of human beings?

This was the way that life had once begun; these noisy, fascinating creatures were human children. Alvin watched them with wondering disbelief-and with another sensation which tugged at his heart but which he could not yet identify. No other sight could have brought home to him so vividly his remoteness from the world he knew. Diaspar had paid, and paid in full, the price of immortality.

Our society has still many fights ahead, not only technological. Daniel Goleman showed us the importance of emotional intelligence. Dr. Carol Craig has criticised how modern school systems are producing a wave of narcissists children. Is there any connection on how technology affect us? On its unifying effect? Behind the scene, there is a small resistance group made of psychologists and philosophers fighting those fronts.

In this Internet Era, every corner of the world is interconnected. Our economy, culture, virtues, defects and sense of humanity are merging across the globe. Our victories and defeats against the machines are having a worldwide effect. I do not expect people considering an amish lifestyle. Technology will advance, shape us along our values, redefine our concept of humanity, and any resistance will be futile.

If you want to read more...

The Most Disturbing Presentation Of The Year
If Shakespeare thought life is a theater, this guy will convince you that life is just a game

Ph.D Kelley L. Ross - There Is No Spoon: The Matrix
A professional philosopher comments on the philosophical aspects of The Matrix

Welcome to the Golden Age of Narcissism
http://drsanity.blogspot.com/

Why the drive for teachers to boost children's self-esteem is killing our Blitz spirit
Dr Carol Craig - The Daily Mail