Engineering Thoughts

Time Travel and free will

2011-12-08T13:53:00.002-05:00

http://www.hawking.org.uk/index.php/lectures/63

This is an interesting article on Quantum Mechanics. It's worth a read on it's own. However, while reading it and it started to deal with Time Travel, this really got my attention.

One is what I shall call, the consistent histories approach. It says that one has to find a consistent solution of the equations of physics, even if space-time is so warped, that it is possible to travel into the past. On this view, you couldn't set out on the rocket ship to travel into the past, unless you had already come back, and failed to blow up the launch pad. It is a consistent picture, but it would imply that we were completely determined: we couldn't change our minds. So much for free will.

This basically says we can't really change the past via time travel. What happened has already happened even if you came back in time. Probably my favorite time traveling movie: 12 Monkeys takes this approach to Time Travel.

The movie stars Burce Willis and he is an interesting character. During his childhood, he vividly remembers being at the airport and seeing a man being shot while his girl friend cries.

Anyways, a deadly virus is released on Earth wiping out most of humanity. Bruce Willis is sent back in time to try an figure out what happened and and hopefully stop it. During the wild adventure, he ends up at the airport with his female assistant. A man is trying to smuggle the deadly virus on a plane. He tries to stop the man, but airport security ends up shooting him and he dies in the arms of the woman. A small child looks on and sees it happen. No surprise... this was him as a child. He saw the time-travelling version of himself beng shot.

He could not change history... everything happened exactly as it did. The virus was released. The man and woman were shot at the airport.

Which Hawkins described as per-determination... which in his words is a conflict with free will.

Now, I've just never seen this great conflict with free-will and time.

For example, in the movie, Bruce Willis appears to be making his choices at each stage. It just so happens that those choices lead him to the outcome.

What's interesting about this whole discussion is its impact on religion. In Islam there is the notion that God knows everything that will happen. This can lead to an acceptance of per-determination and fatalism. Some Muslims have fallen into this trap. I can't do anything about my future as it is already determined.

For another movie reference, watch the movie Kingdom of Heaven. In this movie, the Christians are the people who believe in Fatalism. They just charge into battle and think God will determine the outcome. It is Saladin of the Muslims who believes in preparation and strategy for battle.

In any case, just because God knows the outcome doesn't mean you have no choice. We are seeing more and more how space and time are intertwined. Time is just another dimension. Imagine God existing outside of time so God can see the entirety of time. Does God being able to see the outcome change your choices in the present? Absolutely not. You have the entirety of choice. No action you can take can be against God's will.

Similarly, the issue exists with respect to time travel. At each point, the individual has complete choice as to what to do. The fact that these choices have already been accounted for in the time doesn't change the choices that were made. Now admittedly, there is still hard to accept. This is especially true of hypothetical situations. For example, if myself from the future comes back in time and kills John. I see this happen. Now as live my life and times passes. Do I face the choice to go back in time and Kill John. What would i do it?

Hmmm, I think I have to clear this up a bit :P

The Euro Failure - Progressivism is Enlightened Ignorance

2011-11-03T11:12:00.007-04:00

http://www.theglobeandmail.com/news/opinions/margaret-wente/the-euro-a-nice-idea-until-the-people-got-in-the-way/article2223194/

Doesn't it seem odd now. Every pundit, analyst, commenter, and Joe SixPack seems to agree that the Euro was destined to fail. You can't run a common monetary policy without a common fiscal/political policy. How obvious it is. Which of course begs the question:

How did all these brilliant Europeans come up with such a ridiculous idea?

It's a question that aches at the heart of the modern society. The answer is pretty systemic if you ask me. It is at the heart of progressivism. Note by progressivism, I don't just mean 'leftest'. Progressivism stretches far into the right as well.

Essentially, the core of progressivm bases its roots in the idea that 'might has always made right'. There are no 'natural rights.' Everything is just a man made system we impose on ourselves. So we should not tie ourselves to any political ideology (like communism or libertarianism or capitalism...). We should simply do whatever it takes to advance society. Progressive thought dictates that since 'might always makes right anyways' we should just give up, and simply put the best people in charge. The smartest and brightest and most intelligent of society should of course then be placed in positions of power. I mean, if society is destined to be ruled by the rich or the warlords... we might as well preempt all that and have society be led by those whose mission is progress (academics, scientists, bureaucrats, teachers, economists, bankers...)

That is essentially the core of progressive thought. Behind the rather appealing idea, is actually a crucial flaw. They reject any notion of rights, human nature, history... It decides to be ignorant of everything that is human. Preferring humanity to be thought of as perfectly malleable. The same way that we can mold steel into any shape we want. So too, can we mold society into what the progressives want.

I call it Enlightened Ignorance.

What do I mean. Let's start with something seemingly benign. Progressives believe in the idea of rational administration. That is to say the smartest and brightest should lead society. So we apply it to the field of say transit. The expert in transit, looks at a city and decides we need to do the following for optimal transit usage.

Build high density residential
Build high speed rail connecting cities
Build subway and light rail to connect people

This is all accompanied by thousands of pages on the benefits, cost saved, efficiency... I mean only an idiot would reject such an enlightened plan. Yet missing from the entire thought process of a progressive is anything that has to do with humanity. Questions like the following are totally absent from their mind. They don't want to be bothered by them.

Will people be willing to pay the higher taxes to support the infrastructure
What will you do about zoning where people don't want to live in high density
How do you plan to curtail the power of public sector unions that run this massive transportation system. You create systemic risk of your society being completely shutdown on the whim of a union demand.

And yet, I would argue this are the question far more pertinent to society. Society has collapsed due to human factors and revolutions. I haven't heard of a society that has collapsed because it doesn't have optimal transit usage. You see by being willfully ignorant of all human factors, progressives tend to build up a lot of risk as they pursue their grand plans. Eventually, these risks come to fruition and what you get is collapse. A far worse scenario than a more pragmatic political system that takes human factors into account.

Such was the case with the Euro. Some academics and elites drew up a plan of how the new Europe would be. They designed rules and systems and institutions all carefully worked out. Except of course... they completely ignored the human factor. What happens if a government, elected by the people doesn't want to obey the rules of the Eurozone? For example, the 3% deficit rule was ignored. What happens if they choose not to be subject to the dictates of the Eurozone, as we see in Greece now? Can you really have all Greeks and Italians behave like the Germans? Will the Germans be willing to give of their wealth to the Greeks? Will the Greeks be willing to take the austerity measures?...

The result is obvious. Their best laid plans are destined to collapse.
Again, progressives view society to be as malleable as everything else in nature. The problem is humans are not so easily malleable. What happens when a human doesn't agree with you. Can you force them? Protests, social unrest, rebellion, have been pretty common in human history. Indeed, such was the reason for the mass atrocities of Communism. People didn't agree... so you might have to slaughter tens of millions of people. I see no appetite for such slaughter among progressives... so they simply have these grande plans that cannot be implemented in the greater whole of society.

The worst part of progressivism is their tendency to design grande schemes, which as I say cannot often be implemented over the long term, but to accomplish them, they have to destroy the current system. The end result is actually a system that is worse off.
I often liken this to some of the failed colonial projects. Suppose you have a primitive society that has its backwards ways, but they're living as a society. Some colonial power comes along... decides that this is too backwards and we need to 'fix this society'. To do so, they first destroy the local structure... and then attempt to impose a new system wholesale upon it. When that new system fails, the people are left in a worse off position. The new system is broken and causes strife... and they don't have the old system which at least kept society moving.

Such is one of the biggest problem with progressives is their inability to move slowly. To introduce new ideas into society slowly, see how they play out, and adjust accordingly taking into account the human factors.

Going back to the transit example above. A solution that takes into account human factors would be far more resilient. You would for example acknowledge the automobile as a resilient aspect that let's your society function without being behold to a transit union. Instead of solely focusing on transit efficiency, you also take into account resiliency.

Progressivism is enlightened ignorance. Nothing is easier than thinking if you can just implement your vision of society, that everything would be great. Everyone you meet seems to think so. Every king, dictator, warlord, theocracy, supreme council, first year university student... Everyone can think up the perfect transit, healthcare, education, space research... plan. The hard part is and has always been dealing with the human factors. Who gets to rule? Who gets this job? Who gets how much?...

That simple question 'Who gets to rule' is probably the most important question. Again, it is simply denied by progressives. They prefer the administrative state where agencies rule apart from society. Democracy is a mere inconvenience for progressives. Who better to know about your healthcare than experts on healthcare? They think in terms of institutions and systems... not in terms of people. Yet society is composed of the people. How do control these agencies, or who ends up in them is not something progressives bother themselves with.

Progressivism is the very denial of the most important questions that have existed for thousands of years of civilization. However they frame this behind a veneer of intellectual pragmatism. They claim to be pragmatic and not beholden to any political ideology. Yet, they do not do this by being pragmatic. They do this by denial of all human factors. If they took human factors into account, they'd be then forced to address the eternal questions of politics and rule, and would then cease to be progressive.

The point I'll make here is that whatever efficiencies/improvement you think are gained by progressivism are miniscule compared to the reality that they choose to ignore all human factors and thus have built up a huge set of even greater problems for society.

Again this is not just a problem on the 'left'. The whole economic collapse was due to progressivism on the 'right'. We all recall Alan Greenspans famous words regretting his belief that financial institutions beholden to shareholder value would be in the best position to regulate themselves. What a complete abject failure. The reason... as a progressive, Greenspan rejected any notion of humanity.

The mortgage broker as a person doesn't give a rats behind about long term stability of the financial institution, they care about making their money. So even if the goal of the 'institution is to enhance share holder value', that has nothing to do when on the ground, the mortgage broker doesn't care.
They preferred to trust in models of the financial academics and completely ignore the human factors like greed, desire, risk aversion, fraud...

The same can be said for progressive support of free trade. They didn't bother to take into account the human factors of it. How would society deal with the real or perceived job losses or declining standard of living for certain groups? They simple had a grand vision of efficiency and trade.

You see again, the progressive mind doesn't want to deal with anything human. As an ideology progressivism is enlightened ignorance. Even ideologies like communism attempted to deal with human factors. They often prescribed violence or force, but at least they had an idea of what they were willing to do to acheive their goals.

More libertarian arguments place a heavy value on human factors. They probably talk too much about the nature of humanity, while acknowledging their desire to leave outcomes to what each person, group, society, makes of it. I came to side with liberty not out of a 'natural' right regime, but out of a pragmatic approach that looks like overall outcomes and includes human factor. Rights and blind justice and rules are the best way society has to organize itself taking into account the humanity of individuals.

Progressives by in large live with tunnel vision within a specific field, unable to gain a holistic view of society. It is such irony that an ideology premised on such intellectualism can be so ignorant... hence.

Progressivism is Enlightened Ignorance

The Dangers of the Insite Ruling: Judges as experts

2011-10-13T16:32:00.000-04:00

http://www.theglobeandmail.com/news/national/british-columbia/bc-politics/landmark-insite-decision-threatens-peace-between-judges-and-legislators/article2196941/

Legal battles are always interesting. People typically cheer the legal system when they agree with the outcome, and they disagree with it or cry judicial activism when they disagree with the outcome.

However, the judiciary is supposed to about the rule of law. There is a reason why Justice is blind. It is not about whether you like the outcome or not, but about creating a society based on the rule of law so people understand how it operates and can go about their lives according to its rules.

And so we have the Insite ruling in Canada. Essentially, Vancouver had opened a safe-injection facility allowing drug users to safely inject themselves with drugs. The federal government opposed it as these drugs are illegal substances. The court ruled in favor of the Insite facility.

Now we need to look at their reasoning. Their reasoning is in reality very dangerous to our democracy.
I fully support Insite and I support the outcome of the decision.

I'm not a lawyer, but a decision based on the law would have been acceptable. Something along these lines:

Healthcare in Canada is provincial issue. The provinces are in charge of healthcare. Drug addiction is a healthcare issue and so a safe-injection facility can be considered a medical facility falling under provincial jurisdiction which can override federal illicit substances regulations.

That is a legal ruling. The judges are sticking to their field. Reading and interpreting legal texts. This is what judges and lawyers are trained to do.

Yet, this is not how the judges ruled. Here's a key aspect of their ruling.

It is arbitrary, undermining the very purposes of the CDSA, which include public health and safety. It is also grossly disproportionate: the potential denial of health services and the correlative increase in the risk of death and disease to injection drug users outweigh any benefit that might be derived from maintaining an absolute prohibition on possession of illegal drugs on Insite’s premises.

Notice here, we have judges weighing benefits and harms of medical issues and greater societal problems. These are legal judges untrained in these areas. They are not medical doctors or scientists; nor can they predict the future. Many would argue that in this case it was 'obvious' that the harm caused by the prohibition outweighed the desire for prohibition. I'm not arguing that point. But understand that the repercussions of this law. You will now have people making cost/benefit arguments in front of judges who have no training in such fields. In many cases the scientific evidence will be vague or not there. These judges will be left to arbitrarily predict and weigh costs and benefits based on their own personal biases.

A classic example of this refer to the United States supreme court where Justice Breyer dissents against school choice.
http://www.youtube.com/watch?v=qjAYuMLDGyI

Essentially his 'logic' goes like this.
The US constitution has freedom of religion.
The purpose of this is to prevent religious strife and violence.
School choice will segregate people by religion
This will result in religious strife.
Therefore we can ban school choice...

To some this seems straightforward. But it all rests on two gigantic assumptions.
Namely:

1. School choice will segregate people by religion
2. This will result in religious strife.

Notice the danger here. Much like the Insite ruling, he is making decision outside his legal areas of expertise.
He just assumes that school choice will result in segregation of people by religion.
He just assumes this will result in religious strife.

Now in the insite case, the cost/benefit analysis might have been backed by 'more science', but the legal rational remains the same. We have judges trying to predict outcomes in the future. We have judges injecting their own personal belief systems. It completely destroys the rule of law.

There is little conclusive evidence that school choice causes religious violence. Bryer doesn't need to look that far in life. Just look North to Canada. Both British Columbia and Alberta have school choice programs. Are these two provinces experiencing any more religious violence than Ontario or provinces without school choice? One could make an equally compelling rational argument that school choice actually decreases strife. It is known that people will literally leave their community to be in a 'good school'. In the US, this is often know as 'white flight'. Parents will do a lot for their kids. It can be argued that school choice would allow families to stay in the community and simply send their kids to a different school. Denying school choice forces families to actually segregate themselves more by moving out of the community entirely just so their kids can be in a 'better school'.

The whole point is that there is no conclusive scientific proof or conclusive rationality either way. Yet, here we have Justice Bryer thinking he can predict the future and is an expert in religion, sociology, class, education... and to base his ruling on this assumption.

To call this the rule of law is absurd and the insite ruling is the downfall of the rule of law in Canada.
From now on, you will have people bringing cost/benefit cases to the court in areas. Rarely will there be absolute scientific proof either way, but judges will be free to assume whatever outcomes they want and rule not according to the rule of law, but rule according to the cost/benefit to society as they see fit. Many will say, it's a slippery slope argument... well it is... and as we see from Justice Bryer... one that the legal profession will overstep. Similarly, we have the cases of the Canadian Human Rights Tribunals which have become their own monster trampling over free speech... once again... removing the rule of law, in favor of judges trying to predict outcomes and harms and benefits.

I'm all for the Insite facility as the provinces right to run Healthcare in the manner it wishes. But this ruling opens Pandora's box and removes the rule of law from society in favor of rule by man (arbitrary judgments). These are issues that must be handled by the democratic and legislative process.
I don't agree with Harper's tough on crime and drugs approach. But the way to change that is to change governments, lobby, protest, have the provinces fight back for provincial rights...

We'll regret this ruling as the downfall of the rule of law in Canada.

What does it mean to live in a free society?

2011-08-02T10:20:00.000-04:00

The word freedom is thrown around a fair bit these days. So I decided to ask myself, can we come up with a workable definition of a free society? Let's not go to extremes like anarchy. Let us acknowledge that there will be a government and it will infringe on people's freedoms. Let's also not go the extremes like communism where just because we infringe on some rights doesn't mean we should infringe on all rights.

So here's what I came with as a reasonable state of a free society.

A free society is one in which the default position is freedom. It is up to the government to prove the harm caused by individual freedom in any area is so great that the government must intervene.

Now naturally what is 'harm' and how do you define 'great'? I'm not going to attempt to give any kind of quantifiable definition to those questions. However just rephrase your thought process around this statement. It is up to the government to prove the harm is so great. It is not up to you to prove you should have the right.

So let's see how it applies to common questions in life. Should we have school choice? The position taken today is often counter to the statement. People ask charter and other independent schools to prove they are better than public schools. Rather the question should be are public school so significantly better and independent schools so harmful, that the government must be in charge and monopolize this entire part of life? Then you look at the data and independent schools are on par if not better than public schools. You look at issues like social stratification and what not and you find that societies with school choice are not anymore divided. Ontario is no less divided than British Columbia or Alberta or Sweden...

Or you look at the issue of drugs. Is the harm so great that the government must ban the substance?

In a free society one should be able to go to court with the only argument being 'I want to do this'. Let the government prove what I want to do is so harmful that they must deny me that right.

That would be a good definition of a free society.

The case for a guaranteed income

2011-05-06T04:34:00.002-04:00

http://www.theglobeandmail.com/news/national/how-paying-peoples-way-out-of-poverty-can-help-us-all/article2011940/

It's always easy to be suspicious of a Guaranteed Income. Paying money to people without them working? What would happen to work ethic? What would happen to our economy?

Well, let me make a case for it and then I'll talk about some of the downsides.

If there's one thing we know, it is that we don't live in a free market society. The government spends money all the time. It goes into debt. It prints money. It spends money on unproductive uses all the time.

So if the government can spend billions on a war on drugs just to employ 100k lawyers, police officers, prison guards... and that this war on drugs produces nothing of value to society, why not spend 1/10th of the money on a guaranteed income. For the price of 1 police officer involved in the drug war, we could support 10 people on a guaranteed income scheme (assuming they each took 10 000 per year).

Indeed, the idea of a guaranteed income is probably among the least harmful redistribution schemes. It is probably why many great economics like Milton Friedman and Frederik Hayek support it. The cost are direct; easy to calculate; easy to administer; treat people fairly...

Consider the alternative where the state hire very well paid people to 'help' the poor. These people employed by the state then have a very large self-interest in maintaining their position of privilege in government work.

Indeed as I've said before with that increased use of automation and computing in our society, real productive jobs are going to be harder and harder to come by. It seems to me far more dangerous to create a divided society with those extorting government privilege to create work that is often completely unproductive than to simply give people a guaranteed income.

Now the danger is quite simple. We do not live in utopia yet. There is still lots of productive work that needs to be done. Farmers are needed to harvest food. Industrial workers are needed to provide our energy. Factory workers are needed to make our widgets. Indeed, that is why we need money. So we can purchase these goods.

So who is going to do these productive jobs when they can sit at home at collect the guaranteed income from the government?

Note the jobs I chose above. Those are all jobs people would rather not do. It is easy for an academic to support the guaranteed income without reservation as they would likely still be doing their job regardless. It's interesting fulfilling work. As would many engineers, teachers, doctors...

Yet, find me the person who would rather go work on oil rig in the middle of the Atlantic?

I don't know... but we as a society clearly don't care about money and equality in law. It's completely immoral that the government gives money to certain groups of privilege in the public, legal, or corporate sector when 80% of what they do is non-productive and then can't spare a relatively small amount as a minimum income.

Do you see Western Society adopting free market and independent living anytime soon? Nope. So we might as well have a guaranteed income.

Of course I'm a much bigger fan of work sharing and deflation.
But hey, a guaranteed income and inflation is okay too.

2011 Canadian Federal Election - The fall of the Liberals

2011-05-04T11:44:00.002-04:00

The 2011 Canadian federal election surprised many, including myself.
The conservatives gained a majority government after years of minority rule.
The NDP gained major seats to become the official opposition.
The Liberals were decimated.

I was truly suprised the Conservatives managed a majority.
It came as a surprise to many that the Liberals have fallen in this election. On that point, I was not surprised.
Each political party has an underlying vision of how a country should operate. What's happened is the vision of the Liberal party is no longer believed by the vast majority of Canadians. It's backers mainly include special interest groups and some academics.

So what is the intellectual vision of the Liberal party. Essentially, they believe in the globalization with the idea that education is the key to economic prosperity. The idea that if we focus on education that Canada can compete globally and rise... all the while expanding free trade. This idea sounded attractive and plausible many years back. It gave rise to the Chretien and Martin years. It gave rise to the Dalton McGuinty's. Yet most Canadians would say it has been a failed vision.

All this focus on 'education' and Nortel was all but obliterated while the Liberals in power in Ottawa. All this focus on education and Ontario hasn't really thrived under McGuinty. The intellectual wheels being the education=prosperity movement have come off. I would argue for good reason. I don't know how you can claim to be an intellectual and actually claim that education=prosperity in a globalized world.

There are a few things that stop this.
1. Innovative jobs requiring education are there. Yet, they need few very skilled people. They are not mass employers. For example, Google does everything it need with roughly 25000 people. It has search abilities, online video, world processing, mobile applications... They key to understanding is that a relatively small number of skilled people can serve billions. This is what innovation and computer get you. Think back to another time. The days of local video stores. Each community needed their own video store. This is amplified due to globalization.

2. The whole world is educated. Education provides an advantage when the other people are not educated. If you're the only one who can read in your community, you will do well. But in a world where so many countries have educated people capable of creating the products and services that an innovative economy depends on, education doesn't carry the same advantage. You can take this to the extreme as it was during the colonial period. Educated 'Western' people did all the high end work. They relied on the largely uneducated colonies to do the low end resource collection work.

The reality is that education with an innovation economy is really only beneficial to 'small export oriented states'. Places like Singapore, Finland, Sweden... They have very small populations who can specialize heavily and export to the world. It really can't be a model for a large country. Imagine Silicon Valley as its own country. That small area would be very wealthy with all the export and innovation. Yet spread over the wealth of 300 million Americans, it is nothing.

The Liberal policy of continuing immigration with this broken idea of the education innovation economy has collapsed. Industry collapsed and all the Liberals say is 'More Education!'. As far as social welfare goes, the Liberals really aren't generous enough to make a difference in people's lives.

This vision has proven false, and so we see groups diverging into two groups.

1. Conservatives - have a vision which means making Canada more competitive, lowering taxes, lowering costs, lowering regulations, decentralization, lowering the cost of living, maintain trade

2. NDP - Big government, protection of industry...

At least that's the vision. I myself might have voted NDP had they actually produced a platform to rebuild Canadian industry. However, I think the NDP is going to suffer the same problems as the Liberals. They're going to move too far to this broken platform of Canada can still keep free trade and be competitive. I think the NDP is scared of making any big changes to Bay Street and so while they talk the talk, I don't think they'd make any big policy changes that would actually rebuild Canadian industry. But a lot of people voted for them on their vision. As I said, I don't think they will do anything to back up that vision. They're going up just like the Liberal party, trying to beef up the public sector and social spending, but collapsing private industry.

I have my doubts of the conservatives as well. Can they really make Canada competitive again? I don't know. I really have my doubts that they'll truly make the reforms needed. But at least they're not promising to drive Canada into more long term entitlement debt.

Now some might say the Liberals are the center and Canadians are now more polarized. I don't see it that way. The Liberal vision has collapsed. They are not the center or middle way. They are the bureaucratic way. A party only of benefit for those connected to the bureaucracy. Poor private sector workers have seen no benefit from the liberal party. A lot of them turned to the NDP and I suspect will continue to turn to the NDP.

The same divide is viewed on immigration. The NDP focussed much more on family reunification as their form of immigration... at least from a message level. This played well with immigrants and we can all understand families trying to stay together. The Liberals maintained their old philosophy of immigration is good just for the sake of immigration. They lost the immigrant vote. I wonder if the Liberals forgot that immigrants who can vote and immigrants who are already here. Ask many immigrants who are already here and capable of voting and ask them if they feel they need more competition for jobs, more people in already congested areas like Toronto... You'll start to see serious reservations about the immigration policy amount immigrants already here. You don't need to tell someone from India about the problems of too many people and too few jobs :P They left India for that reason. Why would they vote to recreate their problems in Canada?

At best, I view the Conservative government as holding the line on economic collapse.... before we'll have to make some serious reforms.

Just turn a blind eye

2010-10-04T22:07:00.002-04:00

So I was reading The Globe and Mail and came across this article.
margaret-wente - forget-legalization-of-prostitution-just-turn-a-blind-eye/article1738924/

In context, it concerns the prostitution. It's not really illegal per se in Canada, but all acts associated with it are. Making it hard to run an open prostitution business. Wente suggests it might be good to just leave the situation as is... a vague legal concept. Keep prostitution taboo and even in legal limbo. It might be better than trying full legalization and all the unknowns that might open up.

Surely not something legal purists would want. People all want the law to be clear and cut.

Yet, the more I thought about it, the more I realized there is value in turning a blind eye... in keeping the law vague. I'm not suggesting I support turning a blind eye... just that it is a policy worth contemplating.

Don't Ask Don't Tell... a Good Policy
Don't ask Don't Tell was US government policy for several years. It basically stated that homosexuals could be in the US military as long as they kept their sexuality private. Of course it was challenged in court and people felt it discriminated against homosexuals.

It was in a sense a legal version of 'turning a blind eye'

The interesting thing about getting rid of don't ask, don't tell is that is actually brings about another kind of discrimination. It creates discrimination against heterosexuals.

How... well follow the logic here.
Why do we separate Men and Women? We do so as we desire privacy. There is a certain reality of sexual tension. Now maybe you wish this wasn't the case, but it is the reality of the world we live in. We have men/woman washrooms, separate sleeping areas...

Now then, how does this reality blend with homosexuality. If we would think it 'wrong' to force men and women to shower together due to the sexual tension... how can we then force women to shower with a lesbian... or how can we force men to shower with people who are gay?

It is a logical conundrum. Well there are ways to resolve this in typical liberal accommodation.
Well we can have separate washrooms, sleeping areas for men/women/homosexuals... or maybe even men/women/gays/lesbians.

We could also resolve it by getting rid of men/women separation and telling people to get over their sexual tension.

Great... now imagine being the US Military and have to design these separate areas into your military bases, equipment... It's a logistics nightmare.

Look at the can of worms just opened by getting rid of don't ask don't tell. Maybe turning a blind eye was a good idea. It's not perfect. Just don't publicize the fact that you're homosexual and people won't know to feel sexual tension. Sure, someone could be homosexual... but you would not be confident and thus would not feel as much sexual tension as knowing someone is for sure.

Now of course, all this is a non issue if we get rid of the entire separation of the sexes :P Yet are we prepared to do that as a society? Don't ask don't tell, makes a lot of sense when viewed from this perspective.

Indeed, maybe we should have left don't ask don't tell in there for a while...a couple of generations. Maybe we'd get over this whole sexual tension issue. We should also remember that the military remains a voluntary force, so if you really have a problem keeping your sexuality private, you don't have to join.

Other Areas
Indeed, while I am a libertarian, the reality is we sometimes don't know what a world of legal prostitution, drugs, no speed limits... would bring. Such a world has not existed. Maybe we should move slowly on such issues and part of that process is the old turning a blind eye. Sure the official speed limit is 100 Km/h on the highway, but 'we all know' you can drive 120 km/h. The world isn't ending because of this legal vagueness. It leaves you with a bit of fear to watch your speed. You never know when a speed trap might be there. You certainly wouldn't tempt a police officer by blatantly driving past them at 120 while they're driving 100.

Conclusion
Indeed, we might not want to be so quick to seek legal purity in these areas. We might wish to trust our democratic and legislative institutions to wobble our way through. We can and should experiment and see how things turn out... but lets not seek legal purity in all things. Maybe let a city/state/province try out legalization... see how it works. lets other try it and different kinds of controls.

Yet, let's not take everything through legal purity and impose policies that we have no idea what the consequences will be.

Why the 'Experts' should never rule

2010-09-07T05:22:00.009-04:00

Maybe it's just me, but I'm finding an overwhelming number of regular people who believe in 'trusting the experts' when it comes to policy. It's very strange actually. It is almost taken as a given that the experts should be in charge.

Here, I'll present a few arguments why the experts SHOULD never be in charge.

Experts really don't know everything
Well this is any easy one, Experts really aren't that good. We saw it with the recent financial crises. Here we had some of our brightest minds in the financial world; countless models, countless authorities monitoring the economy... and yet... the experts didn't predict it correctly. Whereas I met countless regular people who simply said... this housing market is crazy... it has to end. Countless regular people who said, we're making too much money just on playing with money. This is going to end. In this regard, the opinion of experts and regular people were pretty much equally relevant when it actually came to results.

I believe this problem stems from treating everything as science. Being an engineer, I trust science and the scientific method every day. If I want to make a chemical compound, build a airplane, I trust science. I trust the numbers. The laws are fixed and well understood. The same cannot be said for economic and social issues. These are complex systems and involve vague concepts like human behavior. People have tried to turn these into sciences. Hence terms like social sciences and financial engineering.

Yet, let me say unequivocally, any attempt to turn economics or social science into a science will be met with abysmal failure. Both these areas cannot submit themselves on a broad scale to the scientific method... and thus can hardly be called science. The key element missing is REPEATABILITY. Anyone who took a science class in elementary school should be familiar with repeating an experiment multiple time. In real sciences, you can repeat an experiment. This let's one see what effects variables have.

If I was trying to figure out gravity, I would run lots of experiments timing how quickly an object fell to the ground. I would try different sized objects, different mediums, different weights, different parts of the world...only then would I have a reasonable concept of what gravity is. Otherwise, I might just drop a feather once and it falls slowly due to air resistance, but I don't take that into account and so I miscalculate the force of gravity.

The problem with the social and economic issues is that they generally cannot be repeated under controlled conditions.
For example, PHDs have tried to study the GREAT DEPRESSION. People have literally devoted their lives to it. Yet, what can they actually say with any certainty after all these years? Not much. You can find supporting facts about any position you take in terms of government spending, monetary policy... Even if you think you know the cause of past events, its impossible to try different actions and see what effect that would have. And why is this? Simply because the event happened once and cannot be repeated. So we cannot isolate the variables and rerun the experiment.

Can we learn anything from the great depression about our current economic crises? Not much. Equally important as variable isolation is the reality that conditions changed. Wages, family structure, technology, work force, free trade... have all changed so much that whatever you think you can learn from the great depression is all but worthless.

Indeed, Nicholas Taleb's book 'The Black Swan' discusses this as it relates to economic policy and our poor ability to model economic systems. It is very difficult to predict future events from past experience in these kinds of cases.

Ditto for most social issues. The conditions change on the ground so often, that we cannot rerun experiments. If we were completely unethical, we could discover some real science in the social sciences. Imagine cloning two people as identical twins, isolating them in control environments, and then raising them in different ways... and repeating this over and over. We might be able to learn something factual. Of course we would have to do this on mass to see what genes have an impact... But it would be so unethical. Indeed, evil regimes like the Nazis were able to perform such experiments and learn real facts about the human condition that we could have never learned while behaving ethically. So most social science is rather flimsy relying on modeling or surveys... Hardly anything concrete. And the impacts of bad social science are horrendous. Never is this more evident than in education.

Just look at the residential school issues faced by the Native Americans. Colonial Europeans thought Native parents were incapable of providing proper education to their kids, and the solution was for the kids to be removed from the care of their parents and the kids raised in government schools. It turned out to be an absolute disaster and the children collapsed socially being raised without their parents.

So again... due to the non-repeatability of events, social and economic policy can never be run by experts as they really have few facts on their side. The same cannot be said for real hard sciences where a person who has studied aerospace engineering knows factually a lot more about airplanes and I would trust them to make proper choices.

This is not to say social issues and economics are not worth studying. The certainly are. They just do not produce reliable results of scientific quality and thus should not be given the merits associated with real science. They should be considered just another input into policy making. Indeed financial modeling and social studies are excellent tools. They often provide very good results within a limited spectrum. Yet should they be used to set policy? Absolutely not.

For example, I would hope financial firms try to model things. If they get it right, they might make more money. The firms that model it incorrectly lose out. Ditto for the social sciences. Companies that adopt good mamagement techniques thrives. Others will fail. Again, these are very useful tools... but they should not be government policy.

Do I think social and economic policies can ever become sciences? Well no I do not due to life constantly changing and their non-repeatability. That all said, it is science fiction. I recall reading the Foundation trilogy by Isaac Asimov as a young child. In it, a brilliant scientist was able to 'predict' the future by massive modeling of large societal patterns. But that's all science fiction :P

Experts are always politically controlled
Experts do not magically hold power. They are granted power by political bodies. As a result, the experts will never be free to actually have pure policy. They will always be subjected to political pressure from unions, businesses... other special interests... and yes... the voting public.

Just to take a very simple case. The experts come up with a grand transportation policy... and they want to fund it via higher taxes. The voting population rejects it as they don't want the tax.

The experts will learn what they have to say to get into positions of power. The experts really are not free to be experts. They will always have to tow the political line. Indeed, experts often have differing opinions on everything. All the government does is pick the experts that agree with their position and say... this is expert advice.

The only way around this is to have a scientific dictatorship without democratic oversight. Such as Plato's Philosopher Kings dream :) Unless you're willing to have a dictatorship, you really cannot have a society run by experts. Indeed, many western countries are trying to setup mini dictatorships within the public sector with unelected bureaucrats and committees writing policy.

Experts are not pure at heart
Well what is wrong with an expert dictatorship? Won't they make all the best decisions? Ah, but the experts are not pure of heart. They are not saints. Experts have as much greed, lust for power... as anyone else. How can you 'trust' that an expert is truly giving their non-biased honest opinion and not just writing policy to benefit themselves?

Indeed, we have inherited this idea that the experts are objective from the history that experts held no power in the past. Science was allowed to prosper freely predominantly because it held no power. There is little reason to lie about the force of gravity. There is every reason to not be truthful if it affects your money and power. Would a scientist be alarmist about a natural disaster like asteroids hitting or global warming if their funding is tied to it... or their prestige?

This basically comes back to the great liberal Joha Action who said:

"Power tends to corrupt, and absolute power corrupts absolutely. Great men are almost always bad men."

Indeed this was a key element of the enlightenment. The lesson was not that religion should not be in charge. The lesson was that nobody should be in charge of society. Whoever is in charge of society will be corrupt. That is the true lesson of the enlightenment. Those who wish to have an expert run society are regressive; wanting to return to the dark ages.

In the end of the day, an expert run society is no different than a theocracy. They both submit the powers of society to a group of people who dictate what must be done. Oh, but experts have peer review you say! And priests have an ethical code that prevents them from molesting children... yet tell that to thousands of children sexually assaulted by the Catholic Church... while even the good priests stood by silently letting it happen. Here we go again... experts are just people. Having people follow the purest and goodest form of their ideology is a dream. Power will corrupt them and then you have a horrible situation of corrupt people with a lot of power.

Indeed, I fear the corruption of science itself the more it is granted power. Science will not purify politics. Politics will corrupt science to the point where we can no longer trust science. And I do not wish to go there. The only reason science has gained the trust it has is because it has been deprived of power.

Empowering Experts Deprives People of solving their own problems
Sadly, empowering experts means regular people are disenfranchised towards solving their own problems. Just to take a simple case. Most policies require money. So an expert run society will take money from the regular people and give it to the experts to spend. This deprives the regular person of money to use to solve their own problems.

If you are a regular person in a western country and you do not think the public school system is appropriate for you, you do not have enough money to send your child to a private school as you have already paid so much money into the public school system via taxes. You are deprived of the ability to solve your own problem.

An Expert Run Society is not Resilient
It's odd how little this word is actually used. We always hear about efficiency as if we're all little machines optimizing efficiency. Yet far more important than efficiency is resiliency. How well does a system stand up to 'bad events'. Let's take a computer example here. If I ask a regular person about computers they would think that Google has amazing high tech, high end computers. Yet Google is actually the opposite. They have lots of little computers that are designed and expected to fail. In this way, it is very resilient as a system. At any given time one computer may go down... but as a whole the system thrives.

An expert run society might get a policy right and it might be the most efficient policy. Yet this has to be weighed with the question... what if they're wrong? What is the consequence? What is the consequence of setting bad education policy at the national level? You ruin the entire education for an entire generation of children? See the Native Indian Residential school issue. Whereas if you leave things at a local level, you might end up with small failures here and there... yet the overall system remains strong as different schools will adopt different policies... so nothing can bring down the entire system.

Indeed, some might argue this was the very cause of the economic crises. The experts ran the banks and built economic models and ratings. It was most likely highly optimized. Yet did they ever ask... what happens if they're wrong? Well you know the rest... the entire economic system collapsed.

Expert run systems rarely take into account resiliency as that often means having lots of different policies and being humble enough to say 'I don't know'.

Science cannot give you values
This last point is extremely important. Even if experts were saints and even if they somehow got everything right, you could still never have a society run by experts because life is full of different domains. Weighing the 'value' of these different domains is purely a task in morality. There is nothing objective about it.

What percentage of your budget do you spend on healthcare, education, transit, technology, culture, arts, regulations. An expert on any individual field no doubt wants to solve every problem in their field and wants all the resources of a society to do it. Yet balancing how much resources go to each problem is pretty much an arbitrary moral decision where once again Joe the Plumber's opinion is just as valid as a PHDs.

Let's take something like Global Warming. How many resources should we devote to solving it? For poor countries like China, how does the value of bringing a billion people out of poverty compare with solving global warming? Should we even just let global warming occur and just deal with the consequences by moving people away from the coats...

Some might suggest you try to maximize the value to all people. This is like philosophy 101. It is called utilitarianism, but now once again, we're into philosophy which tends towards morality. You might be able to make a theoretical case of maximizing benefit in cases like global warming by estimating the number of lives saved. Yet when it comes to abstract values, you cannot really assign a number to it.

For example, how do value 'fun' or 'freedom'. Everyone is going to value it differently. To me life is an end unto itself so I value freedom significantly more than say, healthcare. If an expert came to me and said if we impose these restrictions and ban these kinds of pleasure foods, we could extend life by 10 years. I'd just say, I value freedom and living life more. Others will have other opinions... all completely arbitrary.

In trying to value different domains, the experts are once again no better than regular people.

Conclusion
Indeed, as Buckley said, he would rather be ruled by the first 2,000 names in the Boston phone book than by the Harvard faculty. A wiser saying we should all pay attention to.

The problems with growth dependency

2010-08-28T19:30:00.002-04:00

I've been trying to put my thoughts to words on this for a while as it relates to the current economic crisis. Yet, before I do, I'll make two definitions and distinctions that I think are very important.

Free Market: A system where individual make choices as to what products/services they wish to use and how much to pay and charge for products and services they make.
Capitalism: A system whereby economic growth is the target of society composed of fractional reserve banking and a dependency on moderate inflation.

I'm not saying what I describe above are the 'true' definitions of such terms if they even exist. But for the purposes of this blog, that is what I mean when I talk about the free-market versus capitalism.

The problem we are facing now is a problem with capitalism. Our entire society is dependent on growth. Our pension system requires constant growth. Our banks are dependent on ever growing debt and profit. Our society expects people to constantly get raises. Our society expects home prices to go up...

Yet, growth is not something guaranteed. It is merely something that happens once in a while when it needs to. Like a Christmas Bonus, it is nice to have, but it is not something you would want to be dependent on. Yet, we have all but institutionalized growth dependency into our societies and we seem incapable of having a discussion on anything that tries to remove this dependency.

Our corporations are legally obligated to try and keep growing and growing and growing... which is fundamentally impossible. Once you have a market satisfied... what happens? You either enter new markets or you live with a good normal profit. This is why I don't foresee much real growth in the overall market. You will probably see various companies taking over, some rising, some falling, but overall, the markets are satisfied.

Now you will get new industries and new innovations, but overall these are not going to be grand markets and they will simply steal market share from other markets. So the overall shape of the market itself is stable.

Let's ask a simple question. Why save the housing market?
If home prices are too high that people are foreclosing... maybe home prices should drop.

I would dare say that within the next 100 years, the fundamentals are such that entire housing market will collapse. This is because housing is a simple function of supply and demand.
Supply being the number of homes/condos...
Demand of course being the number of people.

The population is stabilizing throughout the world. And this is very good. Everyone recognizes the benefits of lower/stable populations. Too many people results in too much poverty like in India/China. Most of the developed world would be in population decline were it not for immigration. So a collapsed housing market is not a bad thing.

Imagine everyone affording a home on the cheap. Wonderful no? Of course what would happen to the banking, mortgage, and housing industries?

Technology itself is in a constant state of collapsing prices. I work in the industry... it is pretty much everything we do is to make things more efficient and cheaper. And again, this is also very good. Cheap TVs, cheap communication like cell phones... It is great no?

In economic terms, both are forms of deflation. Deflation sounds good no? Lower prices are great. So why do economists and pretty much every government say it is bad? Well again... we are dependent on debt as per capitalism. Debt is very bad once deflation kicks in.

Suppose I take out a loan for $300 000 for a house and I'm earning $100 000 per year.
But deflation kicks and things get cheaper... but if things get cheaper... that also means my salary gets lower as everyone is being paid less.

So in 10 years.

I have this huge mortgage with say $220 000 left on it, and now I'm earning 50 K/year and the house in only worth $200 000.

That's not lasting long. Someone is going to have to eat this loss. Either I keep paying on this loss, or I foreclose and the bank eats the loss.

This affects anyone who is in debt... and who are the biggest debtors but government.
Let's say a government has racked up a debt of 1 trillion dollars with a GDP of 2 trillion dollars.
Meaning it's DEBT/GDP ratio is 50%. Not bad.

Now suppose deflation kicks in and GDP drops to 1 trillion dollars.
Suddenly DEBT/GDP ratio is 100%!

Keep it going and it will be impossible to pay off that debt.
Such is the problem economists and governments and others see with deflation. They are damn right it is bad.

So what do government want to do. They want to inflate... by pumping money into the economy to make old debts worth less. So is inflation bad? Well it raises prices. But monetary inflation does not need to do this IF YOU CAN KEEP COSTS GOING DOWN while inflating the monetary base.

For example, let us say an apple cost $1.
The government uses monetary inflation which devalues your money so now it costs $2.00. Well people would see that and freak out and you will end up with Zimbabwe or Argentina or WW2 Germany.

Yet, what if the government uses monetary inflation... but then via technology progress or simply cheaper labor, the cost of the apple goes down. So it still costs $1.00.

That is the idea.. and to an extent it can work as long as the rate of price decrease matches the rate of monetary inflation. Yet once again, there is a limit. Technology only moves so fast :)

But remember, deflation is not bad by default.

DEFLATION IS ONLY BAD ON DEBT DEPENDENT (CAPITALIST) ECONOMIES.

The kind of deflation we face is a deflation of reality. People are reaching their needs. They have the housing they need. They have the entertainment... that they need. The markets aren't growing.

Once your population reaches a certain standard of living, your citizens aren't going to spend infinitely on things they don't need. I might buy 3 new jeans each year. That's enough for me. I don't need any more.

Now a free market adjust to all these things quite easily. Demand goes down, wages go down, life goes on. Deflation is not a problem for a free market society. Those who are in debt, just have to eat their losses.

But once again, the problem we face is we have institutionalized debt dependency in our capitalist society in the western world.... so we are incapable of transitioning to the new reality of deflation.
In this sense, modern capitalism faces the same problem of any centrally planned economy...

What? Am I comparing modern capitalism to Soviet style central planning. You betcha!

The principle problems with any centrally planned system are two fold.

The inability of the planners to predict the future and account for it accurately.
The inability of the planners to take on special interests dependent on the present

Both of these are problem with modern capitalism. They did not see the big recession. Our governments certainly don't plan for it... just look at our pensions, deficits... that are all dependent on growth and thus don't take into account any possibility of deflation.

Government, public sector unions, and big banks are the big players in most western countries and they are all completely dependent on the current debt based system. They cannot change without suffering and they prevent any change that is needed for society to move forward.

In a way this explains why it is we haven't achieved the more 'relaxed' society that robotics and automation promised.

For example, as we automate more and become more efficient, we should theoretically, be able to allocate more people to be nurses, doctors, farmers and we would all be richer as a society. We would all 'get more stuff' and 'have better services'.

The problem is that we do not let the free market push labor in such a direction. The public sector expects to earn a premium over other people. This was fine when the autoworker made 80K then they can be taxed and the teacher can make 80k, and they can both feel well off buying the labor of the restaurant worker making 25K.

When the autoworker is automated, the gap between the teacher and the restaurant worker must drop.
This is not allowed to happen.
Ideally, as we automate more... the 'public sector' jobs should begin job sharing as that is where the 'need' is... but it won't happen is public sector unions always expect pay increases and a premium position in society. They will never accept the deflationary aspect that technological progress and efficiency guarantees.

I should emphasize again that this is not just a problem with the public sector. The banking sector is equally trapped in this.

A recent trend for example has been for government to try and simulate their economy via the banks. They try and ensure banks are lending to companies to try and force growth. However, as I said, with so many markets satisfied, it is not really doing anything productive.

So what is the way forward? I don't know. Our society is so dependent debt and inflation, that I think we'll just keep fighting reality to force it to meet these dependencies. We'll inflate our debt away, we'll prop up the housing market, we'll force immigration just to drive the housing market.... knowing full well we'll have to face the stabilizing of population eventually... just not now :P

It's like a giant ponzi scheme, and every country has invested too much to cut its losses. They will just keep trying to prop it up until it explodes in a giant catastrophe.

Sub-urban transit

2010-07-24T20:59:00.001-04:00

http://spacingtoronto.ca/2010/01/08/fixing-the-suburbs-go-transit-hubs-and-oakville-midtown/

The reason we have day to day traffic... getting people to and from work. It is the inter-regional traffic that creates the most jams.

I'll say this much.
While politicians talk of residential intesification, what we really need is commercial intensification.

It's fairly easy to ship mass numbers of people from a suburb to a core. That is not really the issue with traffic. It's easy to get everyone to Union station (main part of Toronto). That's a trivial problem to solve. Everyone just drives, walks, bikes, busses to their local Go Station (inter-regional rail), and its easy and fairly cost effective.

The problem is that businesses are scattered... making it hard to get from the GO Station to where you work.

So building up around GO-Stations is a great idea. The problem/question is whether or not they will tackle the reason businesses choose to go further and further away... that is property and land costs. Both of these are issues controlled by government planning.

Whatever your moralizing, companies are forced to compete and price is a factor. They don't have the luxury of infinite tax payer funds. This is especially true of back-office, manufacturing... jobs.

It is one of the reasons Toronto lost many businesses to Mississauga / Markham / Vaughan... and unfortunately none of those places showed any sense in concentrating their businesses to make it easy to connect to inter-regional transit.

As I look at many sub-urban areas... it really wouldn't take much to get companies to move closer to transit hubs. Proper zoning, parking garages... would make a world of difference. I think Markham has done some great work with some of the large employers to get them to move to more appropriate locations. Heck, give some tax incentives if need be.

Much more work needs to be done to
a) attract companies (especially the large ones) to locate next to transit hubs. Allocate more commercial land near such hubs lowering the costs
b) gain the trust of business so they know once they move, they won't be hit with heavy taxes...
c) Focus heavily on inter-regional transit. If you live in a suburb, your best choice is probably to drive around locally. Don't waste so much money on local transit in a suburb like Oakville. But focus heavily on inter-regional transit and hubs. Let the local transit develop naturally as things scale.
d) Focus on getting businesses to the GO-stations. Funding for new civic buildings, parks... is going to be tough in this economy.

Epigenetics

2010-06-06T19:32:00.000-04:00

I found out about epigentics while browsing this video.

http://fora.tv/2010/03/18/The_Genius_in_All_of_Us_David_Shenk#chapter_13

I've read up more on it, and it's quite interesting. I'm not going to go into the details here. There's a pretty good article on wikipedia http://en.wikipedia.org/wiki/Epigenetics .

When I took high school biology, we covered genes and reproduction. Basically we are all carry a set of genes. When we mate, our offspring get the genes of the mother and the father. And sometimes there is also a random mutation; a mistake in copying the genes. Through random mutations, over billions of years, you get evolution as the random mutations give rise to 'better' versions of the species who in turn get to reproduce more. So good genes get passed on. Bad genes die over time.

I was never really comfortable with that being the only mechanism at play here. The easy one is skin color. Supposedly the more your 'race' was exposed to sun, the darker your skin as ur prevents sunburns. But in areas without enough sun, you need lighter skin to absorb enough vitamin D. I'm a 'brown' person. If I were to go to Sweden. And I stayed there for generation upon generation. Only mating with other 'brown' people. In a billion years, would we still be as 'brown'? Would there be no mechanism for the human genome to adapt to this situation?

So epigenetics brings an interesting feedback mechanism here. Basically you have to separate the gene from the gene expression. Imagine each gene has an on/off switch. Epigenetics is the ability to turn on/off the various genes. The actions/enviroment can alter then gene expression of your offspring. So if I moved to Sweden, I would be exposed to less vitamin D. My body would detect this. This would be passed on via epigenetics. So the expression of my off spring's gene would take this lack of vitamin D into account and try to lessen the skin tone.

Now it's not as simple as this. But you get the idea. Epigenetics is limited as it can only impact the genes you currently have.

What does this mean for individuals. Well that you can actually impact the gene expression of your kids.
Let's imagine you have history of being fat. Your parents were fat. You're fat.
Does that doom your child to being fat?

Not necessarily. Suppose you work out hard and try and stay in good shape. Then the epigenetics will take that into account and try and prime your child to handle that workload and perhaps increase their metabolisms... and they might end up not being so predispoed to being fat.

Ditto for intelligence, allergies, manual ability...

How Did Apple Get Developers

2010-06-02T21:52:00.001-04:00

Recently Apple took over Microsoft in terms of marketshare. It got me thinking a bit about Apple.
The one question that stood out in my mind is how did they get developers to write brand new applications for their phone in a language most wouldn't have encountered before.

Microsoft for example has always tried to get things compatible. So of course, what is their development language for Windows Mobile? Well it's the same ones you use to program for Windows (visual c++, c#...).

Many other phones hooked up with Java, another well known language. Symbian developers use C++.

So of course the question is how did Apple manage to get huge numbers of people to use XCode? Better still who was the executive at Apple this made this choice. No doubt in the meetings, they pondered using C++ or other languages.

Recently as well, Apple is trying to outlaw developing for the iphone using multi-platform compilers. Basically 3rd party program where you program in one language, and it does all the work and spits out versions in xcode for the iphone, another for Java, Symbian...I'm sure there's some less pure motives at work at Apple. Perhaps platform lockin and the like.

However, it's hard to argue with success. Perhaps there is something to Apple forcing developers to write applications specifically for their devices. It forces a certain level of quality.

If I were to pick the one failing for windows mobile, it would be they chose the compete opposite of this. They tried to make it an easy transition from desktop to mobile. Windows CE devices which ran the early versions of PDAs copied the desktop windows 95 style interface... complete with a start button. They made no attempt target development to the devices.

The IPhone for example uses an exclusive touch interface. Now let us imagine I had application in Java and used keyboard input. Let's imagine a world where Apple support Java applications. I'd just redeploy and users of my software would just use the IPhone's virtual keyboard. It will get the job done...Yet, by forcing me to develop in a new language, I'd have to see how to work with my application in touch. It could completely change the UI for the application.

The same can be said about other choices for the IPhone platform, such as 'lack of multi-tasking' It changes the way people design their software at a fundamental level.

All that said, what is amazing is that Apple pulled it off. They somehow managed to get people to write for their platform and throw out everything else. That is simply amazing. Far too often developers look for universal solutions to minimize work. What Apple has shown is that people are actually willing to invest in rewriting when new paradigms emerge. Can everyone do it? I doubt it... but Apple did it.

Movies that work to not make sense

2010-05-29T00:35:00.000-04:00

I'm what people would call a very forgiving movie and show watcher. I let things slide.
If there's a plot hole that's not essential, I overlook it.
If they forget to explain something... no problem.

One of my favorite tv series' is Babylon 5. It just has just an amazing story arch. Yet I know I have trouble recommending it to people. It has bad acting and even poorer visual effects.

More recently there was Lost. I'm sure the writers really had no idea how to resolve everything yet they reolved the overall character issues.

But sometimes it seems a show goes out of its way to annoy people. Someone actually put work into it that it leaves me puzzled.

Let's take this past year's hit; Avatar.
It is one of the those movies that I found great. It had a simple plot that worked and amazing visuals... and few tear jerker moments. Then came the final fight scene. Bare in mind this is movie based on a futurastic human race with huge robots and big guns invading a planet to get its resources from some 'native indian' like people. So we have the captain in a huge mech-warrior style suit. Guns armed and ready... and he is ready for the final fight with a primitive armed with a knife.

So the captain... in his mech-warrior suit... pulls out a giant knife! That means some writer actually thought this up. They mad some animator draw it and animate it. No one bothered saying... this is retarded. Why would a futuristic mech-warrior carry a giant knife.

Another Example: The new Star Trek Movie.

This was a great movie. The plot is basically this. A Romulan space ship from the future travel back in time hunting Spock and Kirk and them fight him off. Great simple plot. Yet, why is he after Spock? Heck, being my forgiving itself, I'd be willing to accept if they even just left it as a mystery. But they don't. Some writer actually took the time to think this up...

He is hunting spock because in the distant future, the Romulan homeworld is about to be destroyed by natural events. Spock being the genius scientist he is, is trying to figure out a way to save the planet. He can't do it. Romulus explodes. So this Romulan nut in the future, goes on a vengeful quest to kill Spock because he didn't save their planet? Really?

It wasn't even that big a deal in the movie. they maybe had a few lines on this.
Here writers... here's some more reasonable explanations that acheive the same effect of getting a futuristic romulan ship on a mission to kill spock:

1. Spock is actually responsible for the destruction of Romulus... maybe he is doing an experiment that goes wrong.
2. Maybe there were two theories on how to save Romulus, but they could only try one. Spock's way is chosen and he could not do it. More reasonable as it keeps Spock the good guy.

None of these ruin a movie for me... but it just puzzles me how these slip through. They're not people forgetting to explain thing or minor plot holes in a complex plot. These are little things poeple actually had to think up and do and it makes no sense. Great movies though :P

Professions and High Caliber Individuals

2010-05-28T23:33:00.001-04:00

I always like to try and find arguments against myself. So I thought I'd ponder the issue of professions. If you're a Libertarian minded person, you are probably against professions. A pharmacist, no matter how valuable their knowledge... really seems more like a glorified cashier. All their conflicting medication knowledge could pretty much be boiled down to a simple computer database.

Most functions of a doctor could probably be done by lesser health professionals quite easily. If they can't handle it, they can pass it to a doctor. Do I really need to book a visit with my doctor so he can prescribe me the same medication he has for the past 10 years?

But then I ask, would we have the high-caliber individuals in these professions were it not for the job protection they receive from state regulation. Would we get the great brain surgeons or great chemists? Without a path of excellent set out, how would you get a great brain surgeon?
Would a talented person even choose the medical field if there were no job guarantees? Would they invest years upon years of education were they not all but guaranteed a decent living?

We can kind of see this issue developing in engineering and software. As the entry requirements have dropped and there is ever less professionalism. Talented people shun the field. The low-level jobs still get done. But everyone company is desperate for the top engineer and developer and architect. Where do we get such people? They don't appear out of nowhere.

So if you hire someone with just enough know how to change some SQL for a database. They will be cheap for sure. They will get the job done 95% of the time. Yet, can you build such a person into a high-caliber individual capable of architecting a distributed database? Nope. As this spreads throughout industry... there are fewer and fewer capable of being high caliber individuals. Those who might be capable are not built to their full potential as it becomes hard to isolate them.

The point is some jobs need high calibre individuals and they cannot be broken down.
You cannot replace a lawyer with a requirements analyst and a team of secretaries.
You cannot replace a good developer/engineer with a product manager, project manager, requirements analyst, a team of mediocre programmers and a million testers.

Just think back to high school. It's not like you could take 10 C students and if you put them in a group they could somehow solve a complex calculus problem. Whereas some other A students just 'got it'.
I know my limitation for example. I'm 'good' but not great. I'm always amazed at people who understood probability courses. I never 'got it'. I got it enough to get through university, but I never felt like I truly got it the way I understand programming or Algebra...

I actually taught computer science to high school. I was always fascinated by how people learn. Some 'got it' and some didn't. Some people can think sequentially and algorithmically. Some don't understand variables. I see now people with 10 years experience who still seem to suffer these problems. I'd love to know how people's brains are shaped to think in such a way. I don't know. I can't explain how some people understand calculus and derivatives... while others can go through all of high school, attend university, and still can't do fractions.
In any case, it is all besides the point.

Now a very legitimate counter-argument to this is that this is only a problem because high caliber individuals have easy alternatives. They can shun engineering and software because there is still a guaranteed path for them in the medical profession, legal profession... If we lived in a libertarian world, they would not have this easy path and they would be in the field and take charge. I don't know if this is valid or not... but it is something to keep in mind.

Now the Western in general has decided the way to create these high caliber individuals is to not build them in industry, but to build them in the university system. They do this primarily because it is the only control mechanism they have. Free trade has all but gutted many industries. Barriers to entry are seen as preventing business. So the only avenue left for government support is the university system.

Once again, we have a problem with this... and we can see it play out. Why are so many grad student in Western universities from Asia? It's quite simple, most Western people shun the advanced academic streams because it's a pretty ridiculous career path. Spend the next 10 years in some of the hardest math and science, only to have no guarantee of a job... and unless you're a super high-caliber individual capable of founding your own company, you really don't have an advantage over anyone else. So they don't make the investment.

It is much easier to make a commitment to a field and grow in it if you have a good chance of a secure long term job,
A young student thinking of medicine might not know if they're capable of being a brain surgeon. You will never know unless you're pushed and try it out. But they will enter the medical field knowing they can at least be a decent family doctor.

How would this look in Engineering and software. Well first of all, all the 'admin' job would have their requirements raised. So operating a router would require more credentials and training.... not just a CCNA. This is like your doctor job protection... where only doctors can prescribe and diagnose... even though a nurse could probably do the same job 80% of the time. From there, you move up to more specialties.

Just pondering.

My Lost Theory

2010-05-24T00:38:00.002-04:00

Well today was the series finally of Lost. I never really expected them to 'answer' things, so here's my theory.

I'm probably in the minority here. But I always thought the smoke monster (aka smoke) was not a bad guy. Let's look at the back drop here.

A young pregnant woman crashes on the island.
She gives birth to Jacob and Smoke.
She is then killed by another woman (aka fake mother) in cold blood.
This woman then raises Jacob and Smoke.

She teaches Jacob to protect the light, not to talk to other people, and not to leave the Island.
This is turn causes Jacob to conflict with Smoke as Smoke just wants to leave the Island. Smoke in his quest for freedom kills the fake mother. Jacob then pushes Smoke into the light turning him into the Smoke Monster.
Jacob now cannot let Smoke leave the Island as the fake mother told him that the light cannot leave the Island and Smoke touched the light.

So what do we have here. Everything Jacob believes is taught to him by the fake mother who cannot be 'holy' as she is a murderer in cold blood... perhaps just crazy. Jacob is essentially a brainwashed child who creates a religion out of this. Smoke just wants to leave the damn place.

I wish they had explored this more. Everyone just seemed to take it for granted that Smoke was evil... I'm surprised the show didn't even explore this or hint at it.

In any case, now to theorize I guess. I think the Island is just a place you go before you pass to the next stage of life. The plane crashed. Everyone 'died'. But they end up in this middle state between life and what lies beyond... that is the Island. You stay there until you reconcile your life and can move on.

Good people pass on. 'Evil' people are stuck there for eternity. Like Michael is condemned to the Island as a whisper.

Following this through would imply that Smoke couldn't accept this death. His real mother died while pregnant with him and Jacob. He wanted to return to a living life.

I'm not sure what that means in terms of destroying the Island. If you destroy the Island, you destroy the middle ground and then all we have left is 'regular life' and no one has a chance to reconcile their life and move on. Jack and the rest reconcile, find love, then enter heaven.

Now how does this all work with the alternate life? I don't know. Before the ending, I always the island existed outside of time as evidence by its time traveling ability. o when the bomb went off, it created 2 time lines. The 1st one is their normal time line. The 2nd one is a time line in which the bomb sinks the island in the 1970s... so that is the reality where the island does not impact their lives.

Interviews... Resumes are useless

2010-05-22T00:08:00.001-04:00

Well we've been interviewing for a new position at work. It's been an interesting experience to say the least. If you haven't tried hiring anyone, you probably don't have an appreciation of how hard a recruiters job probably is. I almost have sympathy... almost :P We scan resumes and people know how to write them. They come in with all the abbreviations, Masters degrees, years of experience at major companies... yet some how don't know about variable scoping? I compared some of these resumes to my own... and I think... I couldn't tell us apart were I an outsider. It's odd though; I find myself doing this weird classifications. I'm almost to the point where if I had someone with just a Bachelors Degree and someone with a Master's degree, I'd put the person with the Bachelor's degree ahead. It shouldn't be that way, but that's how these people are interviewing. I feel there must be some great people applying for jobs, but we can't filter them from the thousands upon thousands we get. Makes you appreciate the word of mouth applications :). At times I feel like saying

Why are you applying for a software development job, when you can't program?

I mean, I tend to self-screen my jobs. I certainly would not apply to a job I'm not qualified for. If I saw 'Google seeking senior developer with specialty in search algorithms.' I wouldn't apply. I know I'm not that smart. On the other hand, maybe it's developers like myself who do not know how to promote ourselves. But then I remind myself not to begrudge people for applying. Everyone needs a job and they have families to take care of. I suppose were I in a bind, I'd apply for everything as well.

To study Computer Science or not?

2010-03-21T23:18:00.001-04:00

Recently, a younger relative asked me if they should go into Computer Science. I thought about it for a while. I tried to shrug off some of my bad experiences in the field. Eventually, I settled on this answer: Don't take computer science, but do take it as a minor. I dare say, I'd almost recommend getting rid of computer science as a major discipline at most universities. Here's why. 'Programming' is not the most important. What you are programming is the key. You have to know what you are programming. Hence, why programmers end up having to know everything about your business :P Get a business or finance degree with a minor in computer science if you plan to work in that field. Get an computer engineering degree (as I did) if you plan to work in networking or server side processing. Get a health sciences degree with a minor in computer science if you plan to go into that field. The reality is that the core of programming is problem solving. You'll figure out if you're good at it or not once you take a few courses in school. If you find yourself copying all the assignments, for the love of god, don't go into the field :P This is not a field where you do the same thing every day. This is a field where everyday is a new assignment. And this is not a field where you can just get by and it doesn't affect other people. You will be part of a team. In any case, their are very good programming specific issues, but you should be able to cover those in a few algorithms and data structure courses. There is only a small subset of work that is worthy of a computer science degree on its own. If you plan to work in search or graph theory or any of those more mathematically intensive fields, then you should enroll in computer science. Yet the jobs for those are rare and few and far between. Perhaps only at Google :P What other tidbits of advice would give. Well I am still a young kid in the field, but I've learned a whole lot, from my experiences and from my mistakes. 1. Get involved a lot in university. I made this mistake by not being nearly involved as I would have liked. University is where you will meet your first circle of career friends. Part of this was the reality that my university took on twice its normal load of students... and it just seemed like we were being pushed through like cattle. The other part was my fault of not making as great an effort. 2. Find out what makes you uniquely good. I'm a great programmer. Unfortunately, only other programmers care about code. However, it is strange that few programmers have the communication skills I do. Strange of course... because I had a stutter and still struggle with it on occasion. Nonetheless, that is my selling point and it's taken me a few years to realize it. My future career moves will definitely be customer facing. 3. Try different career roles during your coop terms. I made this mistake of taking on mainly programming roles during my coops. It would have been interesting to try out other roles, like program management... ... So that is my little advice for what it is worth.

Why Microsoft was successful... the last 5%

2010-02-08T22:12:00.000-05:00

It's easy to find people who criticize Microsoft. But it helps to step back and ask... what about the competition? Shouldn't there be better products out there? There have always been big players in the game. What about HP, IBM, SUN... Is Microsoft just that good at marketing?

My answer is simply that Microsoft gave the users what they wanted. By users, I mean bother consumers as well as developers. They did it better than all the other companies. To an extent, they keep doing it. I sit around wondering why other companies don't get it.

Let's go through a few examples here:

Sun created Java well before Microsoft came to market with .NET. I'm not going to dwell on technical and performance issues. I've worked with them both and they both work *well* enough. Yet we look at deployment. .NET application appear as regular executables and dlls. Java... well you need to have your classpath set, worry about including jar files separately... You can use manifest files and web-start. I've ran into too many situations where Java's xmx/xms memory setting just too often for my comfort. Java's support for programmers was always weak. On the desktop,l it took them years upon years to even support system tray icons. AWT/Swing were average at best for GUI development. Integration with legacy code is weak as well. Contrast that with .NET that can load dll's by default.

What would it have taken SUN to resolve these minor issues? Almost no effort. Yet I suspect what was at play here was some engineers notion of purity. Let's not infect our Java with integration with the OS. Let's not infect our Java with legacy. Let's not infect our Java with fancy GUI or easy to use IDEs.

Or there's Exchange versus Lotus Notes. I've worked with Lotus and you can definitely see the history and the vision. I like to say Notes was probably envisioned by brilliant computer scientists. Everything is a field in a database. You can customize it to do anything. Supposedly great security and replication. A great collaboration platform. Yet, what does it actually do?

It doesn't do very much... and it doesn't to it very well.

I care about email, calendaring, document storage... In all the actual uses of notes, there are much better products. I open Microsoft Outlook and its core of Mail and Calendaring works well. Heck, it's clear that is what Outlook is supposed to do. There are parts of Microsoft Outlook that suffer from the same problems as Lotus. For example, Exchange can also store arbitrary documents in public folders. Lord help any company that bases their document storage on that though.

This is another classic problem. You let the big picture be designed by computer scientists, but forget to assign good people and resources to the final product. IBM spends a lot of effort on customers. I don't think it's their lack of focus or caring about the customers. I simply think its IBM not understanding that all software is design. It's not enough to 'design' the big picture... and then the implementation is 'trivial'. All of software is design.
http://yaminb.blogspot.com/2009/09/problem-with-design-and-implementation.html

Microsoft is far from perfect and has strayed many times. Yet it has often found its way back. Post Windows XP, Microsoft engaged in a little of what plagues Lotus. There were promises of Win FS... the new database file system. I'd put the Windows Registry in this area as well. A universal settings storage system with permissions and everything! Sounds brilliant... now try working with it and you'll come praying for old fashioned ini files. Microsoft has found its way back I think by popularizing the APP_Data folders and the like.

Microsoft just knows how to make life easier for developers as well. I've been working interop with Exchange. Everyone else is going web services. Which is great. Microsoft goes one better. They do web services... and they also provide a DLL to mask/cache the webservices.

Unit Testing's False Security

2010-01-19T13:32:00.004-05:00

Unit Testing is software development where developers write code to test other code. They key point here is the word unit in unit test case! Let us take a very idealized case. Suppose I have a calculator class (unit).
class Calculator    int add( int a, int b)    int divide (int a, int b);
   int sqroot( int a);
To unit test the 'calculator' unit, You can write some unit test cases for it.

Calculator_test {     bool test_add()
    {
         assert( add(5,3) == 8 );
         assert( add(-1, 6) == 5);
    }

    bool test_divide()...
    bool test_sqroot()...
}

It is very nice and clean. I start out with this clean example as this how people end up on the wrong path. This is so clean and testable... let's just apply it to everything. It's a symptom of the plague of computer science which has spent that past decades obsessing on formally proving a program 'correct'. Unit testing works very well when the code under test is very organized and INDEPENDENT.
Unit testing poorly organized code generally doesn't improve its quality. Make sure the code is organized properly into self contained units that can be tested independently.

If you're working on a poor code base that is not nicely organized into units, I'd forgo the unit testing on it. I would suggest simply make sure new code added is created in units and has appropriate unit testing. Focus on cleaning up the poor code itself.

Independence is another big point. Let me write that sentence again. INDEPENDENCE IS ANOTHER BIG POINT. Unit testing is most effective when you can test each unit independently. Notice in the calculator class above, I am not calling anything outside the calculator class.  I can test the calculator without thinking of anything else. There are no external dependencies.

If you are developing an API, this tends to come naturally and you unit testing is more useful.

If you're dealing with an application, finding code that is so well organized into units is... hmmm.. challenging... Finding code that is independent is even rarer. Suppose we are dealing with a server. It gets commands from a client. To actually test the server, you need to get input from the client. This is normally done via a simulator of some kind... things get even messier if the server or client need to maintain state information. To create an accurate simulator, you would actually need to build a fully functional client. You're now venturing well passed the ideal unit case above... and well into functional testing. Now, you can use mock objects. Mock objects simulate what the a real object would. For example, suppose you are communicating with a database. Instead of actually talking to a database, you simply talk to your mock object which would return some data that you create. Writing your code to support 'mock' objects is good practice and generally makes for better object oriented design. However, mock objects do impose significant overhead as you must now write them. You will probably spend more time coding the simulators/mock objects of some kind than actually fixing problems in the code.

Which brings me to my next point. Unit Testing is code. Why are we unit testing? Because developers have bugs in their code.... Is there a chance the unit test code will be buggy? Absolutely!  I get nervous when my unit test starts to look complicated or it is getting too big.  You don't want be be spending your time fixing unit test bugs instead of fixing actual bugs in the application.

On a side note... these cases tend to be example of venturing into functional testing and just trying to hack it onto a unit testing framework. There is nothing wrong with this. I write functional tests all the time and just attach them onto a unit testing framework as it is very convenient to run and get reports.

The final point is very important. Unit testing only checks conditions you code it to test for.
So, if you are not doing good error checking in your regular code, chances are the same checks will be missing in the unit test code.

A quick example of this. In the unit test for the add function in the Calculator above, I do not test the border conditions.
If a do add( 2147483647, 1), the result will not be 2147483648. It will actually be -2147483648 due to the overflow.
If I didn't think up this scenario the coding the actual add() function, it is unlikely I would test for it in the unit test for it.

Unit testing should not be an afterthought. It should be done as an aid to the developer as they are writing software.
Many developers do unit testing informally typically coding a function and then running the debugger and inspecting variables or logs to make sure things are correct. Unit testing is just a formalization of this process.

In general, I like unit testing, but it should not be confused with functional or regression testing. I personally find it very useful when building APIs or libraries, but less usefull on the application end. Generally speaking, on the application side, you tend to need simulators and there are various dependencies that make unit testing less desirable. It is much more in the realm of general function testing and automation.

I always find it useful to have a unit-testing framework (nunit, junit..) from the start of building an application. Many times, you can ever hook onto this framework to run functional tests or various automated tests. However, let's not lose sight of the reality that they are not unit tests in the idealized sense. At the end of the day, you must handle the reality that you have limited developers, limited time, limited testers... You have to choose how much effort you put in all areas of software development and testing. Focus on what unit-testing is good for. If you are writing an API or purely computational math, then unit-testing will work wonders. If you're writing applications or interacting with a lot of 3rd party components, you probably need to focus much more on functional testing.

A little research always worth it...

2009-11-24T22:17:00.001-05:00

So often, a little research in software is all it takes to find that nice premade functiont that solves much of what you want to do. I would like to say lazy programming is good programming. But it's not laziness. It's knowing that code that is written and tested is better than new code you will write.

Sometimes when you find that rare function, it just makes you day. My first Eureka moment came early on in my career during one of my coop terms. I was working on some old code that had a lot of sprintf. I had to do some wrapping around it, and I couldn't figure out how to wrap the sprintf function.

sprintf(buffer "X has a value of %d and Y has a value of %X\n, x,y);
The declaration of sprintf is:

int sprintf ( char * str, const char * format, ... );

Notice the ..., as a variable parameter list.  So I made my wrapper function:

MySprintf(char * buffer, const char * format, ...)

But how do I call sprintf given the variable parameter list.

The answer:  a function called vsprintf that takes in a variable argument list.

Problem solved.

More recently I was supporting some old ASP web code and the original programmers were manually encoding/decoding HTML. There was a bug in that, I tracked it down... the readily available function Server.HTMLEncode would have been better to use...

Or more recently I was working some new library and I had written some code that was *logically* correct. Yet it would occasionally fail. One of the people on my team took a second look and found a more appropriate API and it worked all the time.

I know it gets darn near impossible to know every single library and API we are sometimes subjected to. Yet, as Judge Judy says... "if it doesn't make sense, it's probably not true." I try and give the original authors of APIs and libraries the benefit of the doubt. There must be a better way via the library to get things done. Sometimes you find out there is no other way and the library authors just didn't expect your situation. But in general, it is worth the search.

Who is the victim?

2009-10-13T18:46:00.002-04:00

The meaning of freedom is interesting. Everyone wants freedom and everyone has a different meaning of what freedom is.

The enlightenment definition of freedom is what most libertarians would subscribe to. That is freedom from the coercive force of government. You can't have the state enforcing a religion because that would violate my personal freedoms. You can think of this form of freedom as the 'leave me alone freedom'.

The other kind of 'freedom' involves positive rights. You require something of someone else. For example, universal health care is a kind of freedom knowing that you don't need to worry about paying for health care. Yet, the actual work requires doctors, nurses, medical research and engineers to provide the work for you. Or social security requires other workers to give up their money so you can retire nicely.

White Freedom, Black Slavery
One the most extreme examples of positive rights involves the freedom gained by white people during slavery. It liberated white people from tedious manual work and allowed them to pursue a higher standard of living; free from chores and enough free time to pursue art.

Indeed, it wasn't that long ago that South Africa had apartheid. This too was a freedom for white people. The restrictions on colored people allowed white people to hold high positions and have access to cheaper labor. This privilege was not exclusive to white people. Asian people were considered 2nd tier and they too enjoyed the privilege of a black underclass.

You have to be very careful about positive rights. You have to look at who is your slave?

The Modern Western World... Finding The Victim
The Western world, coming from a tradition of slavery and colonialism is used to having an advantageous position. Unfortunately, the modern world has broken down many of the institutions that used to keep people as subordinates. Yet, the tradition has not died. It just exists in a very temporary bubble, slowly collapsing in on itself.

Consider welfare. Most western countries have a generous welfare system that allows most of their citizenry to not worry about starving. Yet who works on the farms in the western world? It is not Western people. They generally have to import people from the 3rd world as temporary laborers to work on the farms. This allows the cost of food to remain low. The 3rd world is the unseen victim.

Though as mentioned before, this is not coercive. Nothing holds the 3rd world down. There isn't a colonial government forcing Mexicans to work as cheap labor in the United States. This is a temporary measure as the 3rd world raises, the western world will have to reform its welfare system as it no longer has access to 3rd world cheap labor.

The Rich as Victims
The easiest way out for defenders of the Western Welfare state is to suggest we can make the rich victims. That is we place a high tax on the rich and that should cover the welfare of the poor. Problem solved. Unfortunately, no one enjoys being a victim and so as these policies have been put in place, the results have been quite predictable. Over the years, the average person in the western world have realized they can either be givers into the system (engineers, private sector worker... ) or takers (public sector workers, receivers of welfare...). The result is a natural tendency to favor being a taker. The long term result is very simple; the 'rich' in the western world will not be able to produce enough wealth to pay for all the welfare. It is no surprise that virtually no Western Nation has been without large deficits since the introduction of these welfare program. Even during the era of high taxation on the rich and a young workforce, we still ran deficits. Today, with less industry, fewer risk takers, and an aging population, the result is even less wealth being generated.

My fear is the new victims will be the underclass service worker. Made victims through regressive taxation, such as property and sales taxes to pay for the central government spending on the public sector and corporate connections. All in the name of helping the poor of course.

Indeed, for a while, this seemed like it would almost work. Again, the West used the 3rd world as the victims. It allowed immigration that brought in low wage workers to do the work no Western person would do. These immigrants were not brainwashed into relying welfare, so they made good workers. Less recognized, though equally as valid was the reliance on the 3rd world for high-value jobs (engineers, doctors, accountants...) . If you doubt this, just look at the number of immigrants in Silicon Valley. These are you high-value victims. Yet, this could not last forever. Why be a high-end victim in the West, when you could be in India and live a much higher standard of living.

This is not even getting into making victims of their own children, using debt to offload their standard of living.

The West Without Victims
It will be interesting to see the West without victims. What will happen to all those positive rights when the best of Asia isn't coming to the West to build its industry? Will young Western folks go into industry and be the victims for the welfare state? Will Western people be willing to work on their own farms?

Indeed, behind the veil of your average Western socialist is really a colonialist demanding some lesser being (Asian, Latin, African) provide them with their cheap labor so they can have an easy life. They demand cheap food, but are unable to work cheaply enough on the farms to produce that cheap food. They want top notch medical care, but don't know how a person making 20K/year can afford doctors making 250K.year. They just assume the high-end Asian labor can work hard enough producing enough wealth, so they can be taxed to offset the doctor's salary.

I don't know. I do know that things are changing very fast. India and China are both starting to stand on their own two feet. China being much more independent than India.

What I can say is that Western people are going to have to realign their mentality to fully comprehend positive rights. They will no longer be able to count on the 3rd world as victims. They will have to fully cost out the positive freedoms to their own people.

The 3 Little Pigs and the Financial Collapse

2009-10-11T00:52:00.001-04:00

"“Those of us who have looked to the self-interest of lending institutions to protect shareholders’ equity, myself included, are in a state of shocked disbelief,” Alan Greespan (2008).

This little admission came during the after shocks of the great financial crises. Greenspan who had pushed for deregulation for financial institutions appeared stunned. Socialists and everyone on the left appeared vindicated by this statement. Capitalism has failed! We need more government was word of the day.
Libertarians and fiscal conservatives ranting about how what we have is not real capitalism, so don't blame capitalism when government involvement in the first place is the problem!

The strange thing is that with respect to this financial crises, both of them (the libertarians and the big government folks) are right. Not on the same side of course, but both are right. The only wrong are the people defending the status quo. Strange as it may seem, I think we'll end up with the status quo even after all the new rules post financial crises are put in place.

Alright, what am I smoking today? Let's look at the big picture for a second.

The Car Analogy

We all love car analogies even as they are never perfect. It really does model our current problem with the financial industry today.

I love driving, but where I live, and in many parts of the world, if you drive, you have to have car insurance. Is it really fair to mandate insurance. I've paid into it year after year, and never made a claim. Nonetheless, as a society, it is fairly easy to see why insurance is often mandated.

Imagine if we didn't mandate car insurance. A regular person like myself gets into a car accident (we know it happens often enough). As a result of the crash, the person I hit ends up paralyzed. He sues me for a million dollars that I don't have. As a result, my financial life is ruined and I go bankrupt trying to pay off this debt. Maybe we don't live in a country with universal health care, and the paralyzed person can't even get the proper health care he needs because I can't pay. We're both ruined. So much better to just mandate insurance.

Alright, so we have these insurance companies who insure us against these bad situation that are likely to happen to one of us, eventually. In this private insurance market, insurers have to assess risk... to determine premiums. This is common sense and any private insurer will do this.
Been caught drinking and driving 10 times, speeding 50 times, and you drive a Subaru WRX STI? I think your rates will be higher than a person with clean driving record. Heck, you might not even be able to find an insurer willing to cover you with that risky behavior.
Do you know what the magic word is here. This private insurer is REGULATING you. They have to have regulations to make sure you act right, because if you don't, they have to pay for your accidents.

So far, pretty reasonable right?
So this is the system we had for a long time. Now suppose we live in a country where the government ran car insurance. Alan Greenspan takes over and has new ideas. This here is his 'logic'.

I don't think we need to regulate drivers.
I mean, drivers don't want to get into an accident. It costs them money and maybe even their life.
So, it is in their best interest to regulate their own driving. I say we start removing the regulations on drivers

I think we can all agree, this wouldn't last very long before it broke down. Alan Greenspan suddenly seems lacking if the elusive common sense.

Relating to the Financial industry
Alright, let me just tie in the car insurance example above to the collapse of the financial industry.

Like car accidents that we know are going to happen and can be devastating, societies recognize that bank failures are going to happen. Bank failures are not a desirable outcome, so governments around the world introduced something called bank insurance. It lives by different names in different countries. In the USA, it is called FDIC insurance. There is also implicit insurance of companies that are too big to fail. Everyone 'knows' the government will bail out such institutions. This includes institutions like AIG that form the backbone of the financial services industry.

Yes, also consider the Federal Reserve a part of the government for this article. It acts as an insurance as well preventing runs on banks by being a lender of last resort. I just had to throw that in there, as it is all the rage these days.

So you see, the government is acting like the car insurance company, and the financial institutions are acting like drivers. The government insures the financial institutions against collapse.

What do car insurance companies do? Yes, they impose restrictions and REGULATIONS on you to make sure you act right, to decrease the chance of you screwing up. So it is perfectly free market capitalism for an insuring entity to regulate the behavior of the insured. So government, being the insuring entity of financial institutions... has... guess... what... A FREE MARKET, CAPITALISTIC DUTY, to REGULATE financial institutions.

Yes, for you libertarians out there, in a fantasy world, the government would not be insuring banks... but they are... and if they are... they had better do it responsibly. This means regulating them. If the financial institutions are engaging in too much risky behavior (sub prime loans, mortgage backed securities...), it is the free market duty of the government to step in and regulate them as it is insuring them.

Libertarians and Socialists are both right

Let me just make sure of one big thing here. Libertarians are absolutely right that government caused the problem. You see, government introduced what is called a Moral Hazard. They decide to insure financial institutions. This causes people and other businesses to 'trust' these financial institutions with their money. When the government removes regulations, the financial institutions can take on more and more risk. People don't remove their money as they don't care about this extra risk, as it is guaranteed by the government. In a very strange twist of fate, the government was actually encouraging the financial institutions to take risky behavior. For example, the government was trying to 'help' low-income folks by encouraging the banks to give poor people loans. This was risky behavior. Let's not even get into artificially low interest rates.

This is like your car insurance company encouraging your to drink and drive :P

I won't go into how a libertarian banking system would work. Suffice to say, if we lived in that world without a government guarantee of banks, I certainly would not trust the banks with all my money. Maybe this is why libertarians are fond of some kind of a gold standard as they could keep some of their wealth in gold in their possession.

To see this plain and simple. Suppose I start up a bank. It has no government guarantee of anything. Would you be willing to trust me with your money? Trust me :) I won't steal your money or rip you off :) I most certainly won't use your deposits to make high risk bets in the hopes of a high return, that I will take a nice cut of :) You certainly don't need to worry about my high risk bet failing and I go bankrupt and you can't get your money :) The correct answer is you don't trust me.

Now suppose I have the FDIC insurance sign and I am backed by the government. Do you trust me a bit more? You might be tempted to put your money with me now.

So as we can see both the libertarians and the socialists are both right.
Socialists are right to suggest that financial institutions need more regulation.
Libertarians are also right that government was the problem.
Sounds pretty reasonable eh?

Working with What you Have
This represents a classic case of you have to work with what you have to develop a working system.
As an engineer, my mind is hardwired to think this way. I don't have the luxury to live in an ideal world. I have to make things work with what I have now.

I might not like some software architecture that was chosen 10 years ago, but while I work at this job, I will always try and make the best of it. Making it as solid as possible. I might hate perl, but while I write perl, I will write the best perl I possibly can. It's the only way to be productive.

To the libertarians I say, fight for your ideal, but let's deal with the present. We have a banking system insured by the government. It is correct to fight for more regulations on the financial industry.
This is not hypocrisy; one of the most overused words.

Let me use a little example here. Suppose we are stranded on an island. On the far part of the island are some heavy bricks. Closer to us, are some trees. I tell you that we should build a house of bricks. You disagree with me and I am outvoted and we decide to build our shelter of sticks. Alright, the decision is made, even though I think a house of sticks is a bad choice, I am still going to try and build the best house of sticks I can. I will always remind you that we should have built the house of bricks, but as long as we have the house of sticks, I am going to help build the best house of stick possible... because well... I have to live in the house of sticks.

So libertarians, you might have wished society built the house of bricks, but they chose the house of sticks.
As long as you live in the house of sticks, you better help make it as solid as possible. That means pushing for more financial regulations. Never let up trying to convince people to build the brick house, and maybe you can ever get enough people together to build a separate brick house. But all that is not relevant while we are living in the house of sticks.

To the socialists... all I can say is listen a bit to the libertarians and fiscal conservatives when they point out how government was the cause. Listen to them when they point out how flawed the straw house is. It might help you when the big bad wolf comes.

Capitalism did not failed. Freedom did not fail. We didn't have any of those in the financial system to begin with. The very fact that government is insuring these entities makes it unfree and non-capitalistic. All that happened was a bunch of naive people (Greenspan...) forgot this little fact and decided to apply the rules of the free market to a system that was not a free market. Greenspan is like the little piggy who bought plans for a brick house, but built it out of sticks..

There is a right way to build a brick house. (free market banking)
There is a right way to build a house of sticks (regulated banking system).
Building a house of sticks as if it were a house of bricks is just asking for it to collapse. (deregulated government backed banking system)

I wonder if this 3 little pigs analogy will make sense in the morning... or to anyone else :P

The Engineers and the Market

2009-09-22T00:24:00.003-04:00

I suppose articles like this are bound to appear in tough economic times. So it was that I spoke to a few my friends going through layoffs in the high-tech industry in Ottawa; Silicon Valley North, or so they'd like to refer to themselves. What always amazes me is the sheer adherence to the 'Free Market' that so many Engineers and business people in high-tech take. There is this acceptance that jobs can be done cheaper elsewhere, industries move on, the company needs to be profitable, employees are only useful if they're productive... that you just don't find in many fields. It is admirable to an extent. I suppose a left over from the meritocracy that Engineers have always dealt with. A certain pride in being able to get things done. If you aren't performing then you're useless and don't deserve a salary.

I myself lean libertarian. If I had my way, freedom would be maximized, government wouldn't be allowed to go into debt, school and health vouchers would replace government run systems... I really do think it is the best way to run a country. The fairest of all. Every other system puts a certain group of people in a position of power over the rest and they typically exploit it for their own gain. Yet, we must recognize one thing many colleagues refuse to come to terms with. We do not live in a libertarian or even a free-market state. So why should we follow such rules that the rest of society does not follow? Is this a religion worth losing your livelihood over?

The Not-So Free State

I suppose I must first justify that the rest of society does not operate under these rules. Let me first state, I'm not here saying it is wrong or right. There might be very good reasons to have government or monopolies heavily involved in fields like education or healthcare or transit. I leave that to political discussions as it is beside the point. This is the world we live in. However, we must simply acknowledge that these fields operate under a vastly different set of rules than the rest of society.

First we have the public sector, which is increasingly a large part of the GDP of many industrialized nations. These jobs are typically unionized with a pension backed by the government and wages not based on a 'market' rate, but a wage based on some abstract negotiation with the government. In the private sector, people generally look for the best value they can. By value I of course mean some compromise between price and quality. If there are two restaurants and both are of the same quality, but one is cheaper... then the cheaper one wins out and the more expensive one goes under. Not only that the choice is an individual one. Maybe you really value quality and are willing to pay for it. Maybe you don't. It is your choice. This notion of choice simply does not exist in the public sector. Let us take say transit workers. How much more quality do you get if the transit worker is being paid 50K versus 80K per year? Ditto for education and everything else provided by the public sector.

You also have a state monopoly on these which means you cannot go anywhere else for competition. So you have a class of society with no competition getting paid from the public purse and negotiating with the government for their wages. This might even be fine except virtually all Western government are rolling in debt. This means they are spending more than they can afford. This is like a company paying more for its employees than it is taking in. In the private sector, this means layoffs, wage cuts, and cost reductions. In the public sector, this means going into debt and burdening the rest of society. In a well functioning society, as private sector wages fell, so to would public sector wages and benefits.

In short:
The public sector pays based on what people think they deserve.
The private sector pays based on what people are willing to pay and afford.

Then we have what I call the regulated professions. These are you doctors and lawyers. They maintain their status largely by restricting their trade. Once again, I'm not here to argue the merits of it. Simply to point out the disparity. If I want to take some drug, I have to go to a doctor to get a prescription for such a drug. This kind of regulation does not exist in the private sector. If I can fix my computer, I do it on my own. I certainly don't need to take it to an MIT Computer Engineer with 8 years of advanced schooling. The same thing goes for car repair. Could you imagine the cost of an oil change if it had to be done by a government certified professional auto mechanic? One could certainly make the case that the professionals have more quality. Yet, once again. The market is dictated by value. If you could get your oil changed by some self-taught person for $30 or have it done by a certified professional for $200, you would probably choose the local shop. The 'added value' of such a professional is not worth the extra $170.00 for most people. That keeps professional salaries in the true private sector affordable. In regulated professions, you simply cannot practice without having the license. It stifles competition. The professions limit their numbers and to a large extent control societies abilities to function without them. Lawyers are heavily involved in making the law. It is certainly not in their best interest to make the law simple so a laymen could navigate the system. Indeed, it is in their best interest to make the legal system so complex and convoluted that you need a lawyer for every legal purpose. Ditto for taxation and accountants.

Public vs Private Sector... The False Compromise

I suppose many out there are just saying "so what." Of course there's a public sector and a private sector. Isn't that the compromise we reach? The so called mixed-market economy. The great balance between communism and the free market.

Indeed, if the mixed market were managed properly, this wouldn't be a problem and the compromise would work. This is of course one of the reasons why I am a Libertarian. I do not believe humans are capable of managing the economy. We have far too much self-interest. At the end of the day, any managed economy has some group of people in charge of handing out the money. These people are not saints, and as such they hand out money to their causes and their people. If you are not one of their allies, you are a loser in such a society. Even the most well-intentioned democratically elected leader cannot manage the economy. There are a million and one bureaucrats in every decision making process. It is really these bureaucrats who make the decisions on a day to day basis.

So what most of the western world has created is a two-tier economy. This is a failed compromise as you essentially have the state treating its citizens vastly differently. Work for the public sector or a well connected bank... you get access to government money. If you don't... who are you again? In such an economy its perfectly acceptable to say things like manufacturing should compete based on cost. You will often hear statements like the auto worker doesn't deserve 60-70K per year. It's 'unskilled labor'. Whereas bureaucratic jobs which require an education need to pay a lot more? Who makes this judgment? Why should a more educated person get more money? What use is a bureaucrat to me? I'd rather money go to people in productive jobs like sweeping the streets than another paper pusher. It's not the market. It's based on what people perceive.

I knew some people in the now declining auto sector. One moved to become a teacher. In his view, the sheer boredom of the repetitive assembly line caused him to move into teaching. In his view, being an assembly line worker was harder than his job now as a teacher. Yet, to some in society being a teacher deserves a good wage because they are 'educated'. In many parts of the country as wages have fallen in manufacturing and other areas, has this been mapped into corresponding drops in the public sector wages? The answer of course is no, as they are the ones holding access to government money. A totally free society manages the issue of who gets paid how much very easily. If a job has a lot of capable candidates, its wages are lowered. A shortage in a job, and wages go higher.

The Pitfalls

I have a little faith that in the long term, these things will even out. Yet, transitions are not always smooth. Nor do they always leave society intact. I will now examine some of the pitfalls of the status quo. Most of them stem from the 'Big Thinkers' in Western society who care little for the details of life. There are those in Western capitals who simply view the high dollar of the western world as 'economic room.' At any point, we can print money, devalue our currency, and voila... we're back in competition. Apart from the transition in the near term of loss of jobs comes several details many of the 'big thinkers' forget. These details might end up preventing the nice recovery they think will happen once they restore wage competitiveness.

Cost of Movement

First and foremost, there is a significant cost in moving operations. There are capital costs such as building and equipment. There are costs such training and recruiting people. As such even if the Western World could become competitive on wages, it would have to not just match the cost of labor in Asia and elsewhere, but actually undercut it by a significant margin to cause companies to risk the costs associated with moving an operation back to the Western World.

Choice of Careers

The choice of career is very important. In a vicious way, it's a feedback system that is only going to make things worse for the Western World; especially in the high-tech area. Think back to your high school days; probably the last time you spent significant time with all kinds of people. What percentage of your high school class do you think could make a significant contribution to an engineering or high-tech firm? Let's be optimistic and say 15%. If Engineering as a career is not competitive, this top 15% is not going to choose it. They're going to go into medicine, law, teaching, nursing, the government... any field that is protected or gets government money. So this leaves Western employers with a shortage of TOP talent. The talent is there. It is just not choosing to go into the field.

The opposite is true in places like India of course. There, an engineer is a great job that offers you more money than being a doctor if you get to work for a US firm. So you can imagine the best and brightest in India going into high-technology. Then this only furthers the drive to go to India/China to get the work done.

The government could drop the value of our currency right now making cost not an issue, and most companies would still do business in Asia for this reason. Engineering, relative to other professions a skilled person could achieve is just not attractive. I don't blame the companies for doing it either. They have to compete with the world. How long will this taint the profession of Engineering in the Western World?

Now many say that Engineering in the Western World still has lots of positions for really talented folks. This is quite true. If you are a really talented engineer, the jobs await you. Yet, let's look at it from the point of view as a career. If your only hope of making a decent run at an engineering career is to be the *best*; you've basically turned the field to resemble the entertainment or sports industry. It's only really a career if you make the big time. That is why few parents encourage their kids to think of being an actor or a baseball player as a career. If you are the best, then you'll love it. If you're just good at what you do, your career will amount to nothing. This is one of the biggest dangers of thinking that the Western World can get rid of the lower-talent jobs. Few people are going to choose a career where they face the choice between being the best or no job at all. If budding engineers could at least be comforted that if they didn't make it trying to build a company to take on Microsoft, they could at least get a normal 9 to 5 job doing network support or something mundane, more people would take the risk.

You Need Big Companies and Industry Specific Knowledge

This point is similar to the one above. Some Western countries have this idea that entrepreneurship is the wave of the future. You can ship the mass jobs over to Asia and we can be the innovative ones. However, as above by limiting the low-skilled jobs, you not only drive top talent away from the field, but you also lessen hiring opportunities of big companies. There is so much knowledge in your big companies and they act as a fertile training ground for your employees. Everything from project management to dealing with customers and coordinating deliverables and dealing with legacy components and dealing with imperfection are hardly things you can learn in the classroom.

There is also a lot of industry specific knowledge and relationships. You don't just come out of school a seasoned sales professional or a seasoned engineer capable of architecting the new processor for Intel. These are all skills that take time in a specific industry to learn. Most often, you start off as a junior and learn along the way. With fewer low-end jobs available, the Western world is not creating these seasoned professionals. Trying to start a new company without seasoned professionals is hard. You might even be able to have a great idea, but to turn that into a profitable company requires so much more. You also tend to lock yourself out of niche fields. For example, it is fairly easy for anyone to start the next web application. The industry specific knowledge is quite minimal. Facebook is a great web application, but it's not really advanced engineering. let's take say graphic cards (Nvidia/ATI). Do you really have a need to go learn the details of that industry without working in it? I doubt it. Without being in the field, you probably can't come up with too many ideas on how to advance it. Thus your entrepreneurship becomes extremely shallow and suffers.

On a side note, this is also why it is hard to recreate silicon valley as many bureaucratic cities have found out. Without having a strong base of tech companies and seasoned professionals, you can have all the office space, tax incentives, cool committees, and other bureaucratic gizmos that you wish, you're still missing the key ingredient (a concentration of great people with industry knowledge). Note, I'm not just talking about engineers here, but also sales, marketing, venture capitalists...

I'd be more than willing to wager that if nothing changes, in the next 50 years, Asia will be the hotbed of innovation as few in the Western World would have received the industry training to make the innovative contributions.

What Can We Do About It

I'm sure I've just listed pitfalls that anyone in the industry thinks about. Yet what can we do about it? I'd say we must first realize we do not live in a free market. We should not constrain ourselves to it; especially as the rest of society does not operate in that manner. I reiterate once again.... if by chance we manage to elect Ron Pauls in Western government, then retract everything I say as we would be in a free society and the imbalances would not exist. By all means continue to fight the good fight in the name of efficiency and freedom. Yet, it has to be comprehensive to all (or at least most) of society. Seeing as to how I don't see us moving towards more freedom, I suggest those of us in high-technology give up our freedom loving high-horses and join the game. Doctors aren't giving up their monopoly yet. School choice is still a dream. Transit is still heavily subsidized... The law and tax codes keep getting more and more needlessly complex.

I recognize many of these solutions are not all serving to the gods of technological progress or efficiency. Heck, many of them might even be downright protectionist or harmful to the economy in other ways.
If we were having a theoretical discussion on some idealized society, I'd blast cannon balls through these proposals. I'm sure anyone could.

I'm sure someone will point out the 'broken window fallacy'.
http://en.wikipedia.org/wiki/Parable_of_the_broken_window

They're absolutely right. All I can say is that in a world where everyone else is going around breaking windows to 'drive' their business, I want to break some windows too to bring my industry some business.

I'm not here to argue on the merits of protectionism or government spending. It is bad. Plain and simple. The merging of corporate and public interests is also bad and rife with corruption.
I'm just saying having a two-tier economy is worse for us.

I'm talking about people making a living at the end of the day in an industry. If you think you can convince the rest of society to embrace liberty, I'll gladly join that fight instead.

So onto the solutions. They are listed in order in increasing government intervention (decreasing freedom).

Let's Not be Afraid of Lock-in or Monopolies

Many tech people love to bash Microsoft. Yet, they have remained a great player in the game and continue to be just about the best company to work for in terms of quality of life and how they treat employees (I do not work for them. Just from what I have heard). By last count, they employ almost 100 000 people. They do this because they have lots of money and a constant cash flow. You can argue that they did it by monopolistic practices or vendor locking via proprietary methods... and I will say... yes... and? We've got to be the only players in the game of life that don't want to give ourselves an advantage. Of all the methods I will discuss, this one is the least harmful as far as I am concerned. We're not stealing (taxing) anyone's money or preventing people from doing what they wish.

Openness is wonderful. Yet, this is not new. Much of the older R&D in the field was funded precisely because of monopolies. The old ATT/Bell saga was a telecom monopoly. That funded a lot of R&D; C and C++ were both built from that legacy. That constant reliable cash flow means a lot. It is from this legacy that we tend to have the attitude towards open source. When you have a monopoly or you're selling hardware, it is easy to hand the source over for free. Today, ATT like most service providers does not doo much in terms of R&D. All that has shifted to equipment providers who are never quite comfortable with their cash flow. Thus the main successful players have to force their way into constant cash flow either by strong arm sales tactics or vendor locking. I simply support these moves as a means of providing sufficient cash flow to the industry. I'd certainly rather see these more tolerated than some of the more government interventionist measures.

Strong arm sales tactics and vendor lockin are not guaranteed either. Google defeated Microsoft with the web. So I say, let them have their way. Driving costs in our industry down to 0 might be great for consumers, but does little for us workers.

Make Engineering a True Profession in the High Tech Field

I can't get a prescription without seeing a doctor. Maybe you shouldn't be able to operate that Cisco router without a professional engineering degree and an appropriate accreditation. The internet itself as at stake! Who knows what bits you might send down the tubes. Don't even think of setting up that home router without it being certified by professional engineers. You might not enable encryption, thus leaving you exposed. It is a matter of security that I must certify your home router setup. Don't you even think of touching software. That requires the brilliant services of professional engineers with residency training! Yes, this means restricting our labor. We see wages dropping too much, we 'adjust' the numbers of people entering the program. Yes, this means bothering people with needless regulations. This would at least ensure people who invested in their engineer career can get the basic jobs even if they are not able to get their dream R&D job. It would help draw the best and brightest into the field once more as it offers some guarantee of a job.

Embrace Managed Trade

There is no such thing as free-trade in the modern world. Typically it only applies to certain industries. Sure, as a Canadian we have free trade in goods. Yet the entire medical industry is protected by government monopoly. What if I could get a surgery done cheaper in India? I don't suppose the Canadian government would think of reducing costs this way. Not to mention the myriad of provisions that countries have with each other. China has all kinds of provisions that make technology transfer easier, such as mandating a percentage of R&D be done in the country if you want to do business there. No one in the world is playing by these imaginary free-market rules.

So why not borrow a few of these ideas. Why not mandate a percentage of R&D has to be done in your country for a corporation to sell their products in your country? Perhaps tie this percentage to how much sales the company does in your country. At least the knowledge and industry links will be spread out among countries.

Embrace Government Aid

First, there is indirect government aid. Cash for clunkers? Why only for cars? If we see demand for CPUs going down, offer cash for Pentiums :) Many countries use such tactics to drive other industries. Japan for example has consistently fed its own industry by banning resale of 2nd hand products, increasing efficiency requirements... All of these drive innovation and keep the industry moving forward even if people themselves don't feel the need to buy newer technology. We should embrace laws such as those out-dating older models forcing people to upgrade.

There is also more direct government aid. These help keep cash flow going to industries. When it comes to national security you can mandate all kinds of provisions that allow you to guarantee business contracts to your local economy. Not to mention more direct aid such as that given to the auto sector or Wall-Street recently. Yet I'd rather not have it come to bail outs. It's far better to have constant cash flow. We could have government mandates to constantly upgrade the network infrastructure. Governments could buy local. Maybe government contracts could have saved Nortel and kept it in business. Japan has done this a fair bit as well with many so called bridges to no-where projects. Sadly even in Japan, this might be changing with the election of a more leftist government, which promises an end to such projects, but offering free money to people. Pardon me if I think the society should at least get something instead of just handing out cash to families. You can employ people and get that levy built or upgrade your wireless network...

There is also something to be said of industries needed constant cash flow to keep the talent in the company for when it is needed. This is one of the excuses the military often uses. You can't just stop buying nuclear subs for example. If you stop buying them and the industry goes bust, it will cost orders of magnitude more to restart construction and you probably won't have the industry specific talent ready to immediately start building them. So to be ready, you need your military suppliers to stay 'healthy' in case war breaks out and you need to up production. So it is with other parts of the high-tech industry. They need their constant cash flow to stay afloat, so you might as well place lots of order to make sure they stay healthy.

The Problem With Design and Implementation

2009-09-22T00:13:00.016-04:00

I originally posted this story on osnews.com

I'm editing it a bit to clear up some of the points.
As I've done more reading and reflection on this topic, there's a better read from someone more authoritative than my simple self.

Martin Fowler
http://www.martinfowler.com/articles/newMethodology.html

It's a pretty good site in general for anything related to the software process...

The Problem with Design and Implementation
I've been developing software for quite a few years. One of the issues that seems to come up again and again in my work is this concept of design and implementation. I recall it being a significant part of my education at the University of Waterloo's Computer Engineering program as well. The message was always the same. Never write code first. First you must design software by writing a design document, flow charts, pseudo-code, timing charts... then it's merely a trivial matter of implementing it. Make note of the attitude here given towards implementing. The real work is in the design, and it's just a trivial matter of implementing it. It sounds so simple doesn't it? Now, how often does this work out in real life?

Before I attempt to counter this attitude on this attitude, let me first say that I am probably the last person that just starts hacking away at code. I really do believe in design and properly thinking about a problem first. So keep this in mind and for the rest of this article; understand that I am in no way advocating hacking. Requirement gathering, functional specification, high level design documents, specific algorithm usage and all the other wonderful process oriented documents are absolutely essential.

However, if I were to summarize the article in one phrase it would be this: The final DESIGN is code. That is ultimately what matters.

To people in software, most of what I say here is going to seem obvious. Why am I spending so much space writing about something so obvious? Well because many people, even many in software, have missed this point. Let me also suggest that you read this article focusing on the mentality of the people in the business and how it works out in practice. Many of the terms I take issue with may seem reasonable from an abstract academic point of view, but this article in written from a practical perspective.
The Origin
I believe the origin of this attitude stems from older fields in engineering. The civil engineer designs a bridge and the construction workers build (implement) the bridge. The mechanical engineers design a car and the autoworkers assemble (implement) the car. So it was natural to try and overlay this well known idea onto the field of software. One designs software and then another implements. You can see the constant theme here. The implementers are thought to be like robots. All they need to do is follow the instructions in the design and you will end up with a good product.
The Philosophical Problem
The major problem with this is that ALL of software is design. 100% of software is design from the high level architect-like design to the low-level design of a for-loop. The implementers of software are not human! I knew you suspected as much given how odd many programmers are. No, the implementers of software are actually 'perfect' machines. They are the compilers (interpreters, preprocessors... are all included in the generic use of the this word). For almost all purposes, the compiler is perfect.

It is rather strange actually. It is as if people do not recognize the very thing the computer brings to the table. It gets rid of human implementers. It makes them obsolete. Thus, it can do the same task perfectly over and over again without error. Software is like a civil engineer having an army of robots capable of reading his design and perfectly building the bridge every time. What an amazing world that would be. Every screw is tightened to the exact specification. Every weld done perfectly. Every piece of steel cut to the exact precision. That is what we have in the world of software. The perfection of implementation. Yet, we do not recognize this. Rather, we have decided we cannot live in a world without human implementers. So we erroneously transferred this concept over to the field of software and created the notion of implementation where none is needed.
What is Design?

Yes, all of software is design.  There is no implementation.  Pardon me as I stress this over and over.
There is only high level and low level design.
To mirror other fields of engineering:
A civil engineer also has high level design, such as choosing the type and shape of bridge.
A civil engineer also has lots of low level design, such as choosing the kind of screws, where they go, where to weld...

All parts of the design are essential and are 100% design. So it is in software. The high-level architecture (choosing components, designing interfaces...) are all essential. So is the low-level design of individual for loops, error checking... I dare suggest most of the problems I deal with on a day to day basis are in problems in the low-level. Low-level software should not be dismissed as dummy work. This is the guts of a program.
The Real World Problem
Every line of source code is design. Software is the equivalent of the blue prints to a bridge. The only complete design is actual source code. Once you realize this, you will begin to understand why so many software projects go wrong. It is not enough to hand over a design document or specification to a code-monkey and expect everything to come out okay. The key issue here is to understand that no traditional design document or specification is complete. If it were complete, you would have been better off just writing the source code yourself.
After all, what is source code, but the specification of what the program should do? Most modern programming languages resemble English and written language enough that a well-written program reads as well as a specification. Modern programming languages are not cryptic or needlessly verbose like assembler. Programming languages have gotten so good and have gotten rid of so much of the fluff that they essentially have become a very good way to represent algorithms. As I look through my source code today, about the only fluff are the import or include statements at the top of the file. Well written libraries and proper design abstract the rest of the fluff.
Suppose I were to write a function that divides two numbers. I could either write a specification as follows:

Program Inputs:  32 bit signed integer A, 32 bit signed integer B
Program Outputs: A/B as an integer ignoring any fractional component
Error checking:  If B is 0 then the program will throw an exception.

That is what a properly written specification would be. I could of course have just written the source code myself as follows.

int DivideNumbers( int a, int b)
{
if( 0 == b) throw new Exception ("Illegal divide by zero");
return a/b;
}

Is the English specification any clearer than the actual code? I highly doubt it. You might as well just look at the source. Does the English specification provide anything of value that the source code does not? Combine this with the fact that if you need to make changes, you are relying on the specification and now you run the risk of having the specification be out of date with the source. Now imagine a program with thousands upon thousands of functions. Have you ever worked at a place where every function is defined in a specification to the detail above BEFORE any code is written? Of course not! Everyone recognizes the insanity that would be. It would be mindlessly repetitive to fully specify something in English and than translate that in source code.
Of course this even assumes it is reasonable to think a design can be perfected when written once. Indeed, software is an iterative process as is most designs. You design something, test it out, makes changes, rinse and repeat. Software makes this process amazingly simple given our debuggers which act is simulators. Certain fields in engineering have been done so much that standard designs make it seem as if they somehow possess more quality. We certainly aren't building many new styles of bridges for example. Yet given any new problem, all fields in engineering face an iterative process filled with bugs. We've been doing civil engineering for centuries. Somehow things like the Big Dig in Boston still cause lots of problems.
Source code is a valid specification on its own.
Yet, we seem to cling to this notion of the specification that 'just needs to be implemented'. Let us expand this little example a bit. Pardon the extreme simplification here. Much more likely than the full specification you see above, is something like the following or even no specification at all:

Program Inputs:   A,  B
Program Outputs: A/B

This leave the 'code-monkey' needing to fill in the blanks as the specification is not complete. The programmer might choose to return -1 instead of throwing an exception if B is 0. He might not even bother to do error cheeking and just rely on the language itself to handle the issue. He might choose to use long, double, float data types instead of integer. No one knows. This is for a very simple function. Imagine the blanks the programmer has to fill in for anything more complex. Never is this more obvious than in any application that needs a graphical user interface (GUI). The impact of this is immediately seen by users. Don't blame the programmer here. I highly doubt the GUI was specified completely before they started coding. Then when things go wrong you have people wondering what went wrong. Why couldn't that programmer just implement what we wrote in the spec? The programmers say the spec was incomplete. Again, the only complete specification is really the source code itself. Everything else is really just an incomplete specification. No one is really to blame except the broken process itself.
The Academic Problem
This is not just a problem in industry with the stereotyped MBAs not understanding software. Often, you hear from engineers that universities should not be teaching software in a specific language, but they should be teaching abstract notions of algorithms and data structures. I agree, but how do you propose students express algorithms or data structures? Yes, this is why programming languages were invented. To allow us to express algorithms and data structures in a human readable format! You have to learn the language to express your ideas. How do you test your algorithms and data structures? By writing it in a specific language and running it. The power of a good programming language is essential to your learning. You can set break points, inspect variables, make changes and see the results immediately.
Indeed, by insisting that specific programming languages are not 'valid' ways to specify something, academia only reinforces this notion of design and then implementing it. While this might be true in some abstract notion within academia it has disastrous effects when carried over into the rest of the world.
Let's have a look at a simple algorithm one might encounter in academia. Here is Wikipedia's description of the Euclidean Algorithm to find the Greatest Common Denominator.
Euclid's GCD Algorithm

The Euclidean algorithm is iterative, meaning that the answer is found in a series of steps; the output of each step is used as an input for the next step.[21] Let k be an integer that counts the steps of the algorithm, starting with zero. Thus, the initial step corresponds to k = 0, the next step corresponds to k = 1, and so on.
Each step begins with two nonnegative remainders rk−1 and rk−2. Since the algorithm ensures that the remainders decrease steadily with every step, rk−1 is less than its predecessor rk−2. The goal of the kth step is to find a quotient qk and remainder rk such that the equation is satisfied
rk−2 = qk rk−1 + rk
where rk « rk−1. In other words, multiples of the smaller number rk−1 are subtracted from the larger number rk−2 until the remainder is smaller than the rk−1.
In the initial step (k = 0), the remainders r−2 and r−1 equal a and b, the numbers for which the GCD is sought. In the next step (k = 1), the remainders equal b and the remainder r0 of the initial step, and so on. Thus, the algorithm can be written as a sequence of equations
a = q0 b + r0b = q1 r0 + r1r0 = q2 r1 + r2r1 = q3 r2 + r3…
If a is smaller than b, the first step of the algorithm swaps the numbers. For example, if a « b, the initial quotient q0 equals zero, and the remainder r0 is a. Thus, rk is smaller than its predecessor rk−1 for all k ≥ 0.
Since the remainders decrease with every step but can never be negative, a remainder rN must eventually equal zero, at which point the algorithm stops.[20] The final nonzero remainder rN−1 is the greatest common divisor of a and b. The number N cannot be infinite because there are only a finite number of nonnegative integers between the initial remainder r0 and zero.

Brings back memories of university doesn't it? Now you read that and yes, that can be implemented in any programming language. Here is one of the implementations done from the same Wikipedia article. It's a simplified version and might not match up exactly to the English/Mathematical language above. Go with the flow here.

function gcd(a, b)
if a = 0
return b
while b ≠ 0
if a » b
a := a − b
else
b := b − a
return a

Granted it is in pseudo-code, but it wouldn't take much effort to convert it to C# or any other language. If I were writing a program what would be the point of me spending several paragraphs describing the procedure for the algorithm in some mathematical and English language, when I could just express it directly as source code? Can you imagine handing off that written mathematical specification to a 'code monkey' to just implement? He'd have more trouble understanding what you wrote than if you had just written the code yourself.
And this is the same problem in every other realm from network protocol specifications to HTML standards. Every specification that is complete is better off just written as source code directly. I can almost guarantee you that it will be more understandable as well.
Small Tangent
Let's go off on a tangent here as this is one of the reasons people find it so hard to implement specs and standards. What is the correct HTML standard? The answer really is whatever reference renderer you use. Remember again that to fully specify the HTML standard, it would have to be long and detailed enough to essentially be source code. A small example I just recently ran into is Read-only text boxes. Firefox renders it the way I like. A read only text-box is 'greyed out'. In IE, it looks the same as a writable text box. Not being a web-programmer, this was news to me. I suddenly feel old and outdated as I am sure many of you are thinking that is so 1999. The HTML specification of course says nothing of the 'right' way to do this. This is natural of course as to get the HTML to render the exact same the specification would be so long detailing colors, border widths, bevels, how round corners should be, gradients... I'd hate to be in that committee.
These days, it seems WebKit is becoming one of the 'standard' renderers. You must behave as WebKit behaves. So why not use Webkit directly? Hence, we see many more browsers moving towards it rather than playing a game of constant catchup. Have a look at the alternative if you wish. Microsoft has released many specifications of its formats. They are insanely long and I doubt they are fully specified. I'd be more than willing to wager that to actually implement any of their standards, you would have to launch their renderer (MS Word), see how it renders something, and then copy it. I do not fault Microsoft here. It's just the reality of writing specifications. Having worked in the networking field, we did this all the time. Some part of the specification is vague or strangely worded or missing. What do we do? We hook it up to the 'standard' Cisco box and see how it behaves. Then we make sure we behave the same way. Sometimes you are lucky in that as more and more people do this, the 'specification' is updated to include all the little vagaries people ran into along the way. We're actually quite fortunate in networking as the protocols themselves are fairly straight forward and they rarely change. Case in point is IPV4. Despite all the problems with it, it is still the dominant protocol. Contrast that in other fields where change flows much quicker.
Granted, often times companies don't want to just give you the source code as they spent time and money developing it. They might want your software to be incompatible, so that they can claim your software is broken and less reliable than theirs. One of the beauties of open source is that you don't need to read the specification. You can just link the source directly.
My only point here is to suggest that to FULLY specify something, the actual source code is often the most concise and best method of doing it.

Only Implementation Activities
Now, are there any implementation activities in the software realm? I suppose porting an application between similar languages using similar libraries would be implementation only. Some aspects of GUI design are also implementation only. However, most of these are disappearing as they are simply areas that compilers are lacking and boy do we like writing compilers to get rid of this tedious tasks. A lot of GUI design is moving towards some form of specification (XAML...) that can then be read or compiled into a program itself without you needing to touch 'code'.

This is the progression that has been constant. We keep moving up the chain if you will, removing all the redundant aspects of software and leaving only 'pure' design. Decades ago if you programmed in assembler, there might have been an 'implementation' stage where you were simply doing a repetitive task transferring the design of a for loop into assembler commands. However, it is the compiler that has relieved you of this tedious, repetitive tasks. If you are coding in assembler today it is only because you lack a compiler for a higher level language or you are doing optimizing design.

Solutions
I believe the solution to this problem lies in changing attitudes. First and foremost we need to change the language used by us and the academic world. I never want to see the phrase 'design and implementation' ever mentioned again. Even if 'we' understand the academic difference that the design can be conceived independent of programming language, when this reaches the wider management and business community, it is reduced to the very problem discussed here. We should only speak of high level and low-level design.

Next, we need to recognize the validity of source code itself as a valid specification. That is exactly what it is. You don't see civil engineers trying to write an essay to describe a bridge when an auto-cad blueprint is the real design. We need to treat our source code as the specification that it is. That means writing it neatly and cleanly. Putting separate objects in separate files. Just as the civil engineer doesn't scribble over his designs, neither should we leave our source code mangled with undecipherable variable names. You should be able to read it with ease. I suggest we as software engineers bare a great deal of responsibility here. Often times our source code resembles a blue print with eraser marks, numbers scratched out, pages duct tapes together, big arrows leading to other documents and a big TODO that leaves out critical functionality... We should also try and use programming languages that make being a valid specification easy. C# for example has the ability to have a lot of meta information and attributes within the source code itself. I hope this process continues and is expanded in more languages. The more expressive the language, the more we can offload onto the compiler.

We should use valid design documents were possible. It should aid in the source code specification, but should not be thought of as the specification itself as that is not what the implementers (compilers) use to produce the final result. Putting emphasis on useful design documents will allow for more man hours to be spent here instead of on useless items. I tire of seeing someone copying the header file of a c++ class, pasting it into a word document and claiming that is valid documentation. Designing GUI prototypes, timing diagrams for network protocols, class and interface maps, overall design choices... are all essential in guiding the source code.

Last, but not least is something I haven't touched on too much, but it is implied. Software is not trivial work. As such getting good people and training them is essential. You cannot separate the specification or design or knowledge from the code. The latest fad is something called a 'subject matter expert (SME)'. I say this is a fad as it assumes that you can separate the person doing the coding from the person that 'knows stuff.' In the networking world, this is the person that knows the protocols and specifications down to a tee. No worries though. They don't program. That is just the trivial matter left to the mundane implementers. I had such an experience recently at a company I worked for. They had a PHD who was supposed to be a subject matter expert. Of course there were the programmers actually writing the code, debugging issues, working on interoperability, coding which bits need to be set, what values go in what fields, and so on. The programmers ended up having to know more about the specification than the SME himself. Do I really need to rewrite the article on the futility of separating the SME from the programmer? In the end, it is your programmer who will debug issues, solve problems... They need to be experts in the subject matter. I suppose you could have an SME who does not code. It's just that the programmer would have to bother them for every line of code. In the end, the SME would probably be better off just writing the code themselves.

If you are hesitant to make the leap of realizing that to fully specify something, you essentially need to code it or that a programmer needs to be a subject matter expert... let me summarize this article with an old philosophical question.

Can you have thought without language?
Some of best languages we have found to precisely specify many routines are the programming languages. No other written language offers as much clarity or conciseness as a modern programming language.

About the Author
Yamin Bismilla is a graduate of the University of Waterloo's Computer Engineering program and is currently a software developer in the Toronto, Canada Area.