Ken Egozi's blog

Determining SQL Server edition

2011-11-01T14:21:18Z

Thanks to http://support.microsoft.com/kb/321185 and to Ariel (@Q) who have read it more carefully than I did, I learnt that there is a SERVERPROPERTY that you can query:

SELECT SERVERPROPERTY ('edition')

I expected to find Developer, but found Express instead.

MySQL error 105 - Phantom Table Menace

2011-10-10T16:30:46Z

MySQL is weird

The weirdest problem happened to a college today.

When creating the database schema during integration tests run, he got “Cannot create table FOO error 105” from MySQL.

There *used* to be a table named FOO with a VARCHAR primary key. The schema then changed so that the primary key of FOO became BIGINT. There is also a second table in the system (call it BAR) which has a foreign-key into FOO’s primary key. A classic master/details scenario.

However, the table BAR was obsoleted from the schema.

The integration tests runner is dropping all tables and recreating them before running the test suite. It is inferring the schema from the persisted classes using NHibernate’s mapper and the Schema creation feature of NHibernate.

Sleeves up

We cranked open the mysql console and started to look around:

When doing “SHOW TABLES”, the FOO table was not listed.
CREATE TABLE FOO (`Id` BIGINT) - fail with error 105.
CREATE TABLE FOO (`Id` VARCHAR) – success !!
huh?
DROP TABLE FOO – success
encouraging !
CREATE TABLE FOO (`Id` BIGINT) - fail with error 105 – again
huh ???
DROP TABLE FOO – fail with “cannot delete … foreign key …”
but SHOW TABLES still does not list FOO
huh ?????
DROP DATABASE dev; CREATE DATABASE dev;
now everything works.

Back to work

Luckily this was not a production database, and even more lucky – the said DB change (change that PK from VARVHAR to BIGINT) would need to run on production within a separate DB instance that can be recreated on deploy.

And while we’re at it

Way can’t MySQL store non-indexed columns in an index?

How do you pass values from Controller to View with MVC3

2011-10-08T13:28:22Z

The scenario:

Given a blog application, with the following layout

with two possible usages – a post page:

and a homepage:

Let’s define the view model:

PostData:
  string Title
  string Body

PostView
  PostData Post; 

HomepageView
  PostData[] Posts

LayoutView
  Tuple<string, int>[] Archive
  Tuple<string, int>[] TagCloud
  string[] Similar

The views:

_Layout.cshtml – obvious
Post.cshtml – given a PostData instance will render Title and Body
PostPage.cshtml – given a PostData, will call Post.cshtml and then render “add comment” form
Homepage.cshtml – given PostData array, will iterate and call Post.cshtml for each post

How data moves around:

Controller is passing PostView (or HomepageView) *along with* LayoutView to the views
Post.cshtml should only see its parameters, not the layout’s (which are passed but are not interesting within the post template).
same goes for the other views
All views should be able to “see” a shared parameter named “IsCurrentUserAdmin”

Given that I want typed access to the view parameters in the view (for the sake of intellisense and refactorings), how would I model and pass the data around?

I’ve pseudo-code-grade written two options: the first is to use inheritence in the view model to achieve type-ness, on the expense of flexibility (composition is difficult with class hierarchy, and you need to be aware of and grab the viewModel instance in various places). The second is flexible (use the ViewData dictionary) but getting type-ness is cumbersome and partial (strings scattered around, casting needed etc.)

see https://gist.github.com/1272269 if the gist widget does not load in-place

I do have a solution that works for me

With the many years that I’ve been writing complex web apps using various ASP.NET frameworks and almost always with c# based, static-typed view engines, I have a solution that works very nicely for me.

But I want to be aware of the MVC3 canonical / textbox way

So for all you MVC3 ninja’s out there – please describe your way of doing it.

I will describe my approach in an upcoming post and I’d appreciate any input on it

Re-blog

2011-10-07T22:06:00Z

My blog has moved to AppHarbor, and while doing that I also changed the engine for a completely custom thing (running on custom-y stuff like WebOnDiet and NTemplate), to a wee bit more conventional codebase based on MVC3 and Razor, with lots of nuget packages replacing custom code that I wrote myself.

The packages file now contains AntiXSS, AttributeRouting, Castle.Core (for my good pal DictionaryAdapterFactory), elmah, MarkdownSharp, mongocsharpdriver, XmlRpcMvc and XmlRpcMvc.MetaWeblog (awesome!)

BTW, expect a post on using the DictionaryAdapterFactory to make handling Controller=>View data transport truly awesome.

What’s missing here? IoC !

yeah I did not bother with that now. I have my tiny 15LOC thing and this blog does not need anything of this sort.

Some things might still break. Files I used to host for downloading would probably won’t work now. I will fix that soon I hope, time permitting.

note to self – reshuffle the tags here on the blog. I need to re-tag may entries. Maybe I’ll let site visitors suggest tags?

ASP.NET MVC3 Model Validation using Interface Attributes

2011-08-22T21:33:10Z

After reading Brad Wilson’s post on that, I thought to myself:

Brad is 100% correct regarding the way the CLR treat interface attributes, but this does not mean the users should not be able to use validation attributes on model interfaces

So I sat down to extend the model validation to do just that: (see https://gist.github.com/1163635 if it is broken here)

Now I know it is hacky – it should not go on a FilterAttributes() method. If I had access to the sources I’d have added a virtual “GetValidationAttribute” method on DataAnnotationsModelMetadataProvider… (hint hint hint)

Count number of times a file has been download via HTTP logs - DOS edition

2011-08-19T20:07:11Z

Apparently you can also do this with DOS, but it is not nearly as easy as it is with Unix shells or Poweshell.

DOS corner:

Good old DOS have FINDSTR which is quite similar to ‘grep’ but does not have anything to resemble ‘wc –l’.
However, the FIND command can count occurrences so

type log_file | find /c url

would work – for a single file.

in order to do that for all files, an on-disk concatenated version would need to be created and removed. Here’s a batch file to accomplish this:

@Echo off
pushd C:\inetpub\logs\LogFiles\W3SVC1
if exist all.log del all.log
copy /a *.log all.log > nul
type all.log | find /c %1
del all.log > nul
popd

yuck? indeed

Count number of times a file has been download via HTTP logs - IIS and powershell edition

2011-08-18T09:43:56Z

Following ben hall’s post (great blog – subscribe!) on counting file access data from Apache logs using *nix commanline’s grep, I thought to see how easy would it be to get the same with Powershell (I’ve been wanting to learn PS for ages – I have a few Windows servers I manage, and I love *nix commandline and miss it on these machines).

Smoking pipes

I also wanted to be more idiomatic – looks like shell scripting (both with bash and friends, as well as with Powershell) favor piping outputs to inputs.

for e.g. – the command that Ben is using – reading the file using grep’s switch,

grep –o url log_file | wc -l

would be rewritten as

cat log_file | grep url | wc -l

Not being a unix master myself, I cannot tell which is better, but personally I prefer the latter; instead of needing to remember grep’s switch, I pipe cat into grep.

It becomes even better once going through multiple files – I have no idea what grep’s syntax for that is, but I can pipe ‘ls’ into ‘cat’

WRONG
cat does not except input as a pipe. You need to resort to other syntaxes (see http://stackoverflow.com/questions/864316/how-to-pipe-list-of-files-returned-by-find-command-to-cat-to-view-all-the-files)

anyway,

Powershell-ing

First – ‘count strings’ is done by Select-String (this is the one of two parts I needed googling for)

the equivalent for ‘cat’ is … (wait for it) … ‘cat’

so,

cat log_file | select-string url

will return all relevant lines

So how do you count lines (equivalent to ‘wc –l’) ? - there’s a command called ‘measure’

Now we are at

cat log_file | select-string url | measure

IIS logs vs apache’s

Not sure about how Apache does things, but with IIS, the access logs are split by default on a daily basis. All logs per a website are within a designated folder (, so one needs to iterate over all of them to count access to a given url.

That’s where cat’s piping impediment could have been bothering, however with powershell – things are a bit easier – you can actually pipe ‘ls’ into ‘cat’ !

show me the codez already

ls C:\inetpub\logs\LogFiles\W3SVC1 | cat | select-string "e.js" | measure

does the trick !

Actually – there’s a little trick here to take this a step forward

As opposed to unix shells, powershell is object oriented, and has richer models over simple text lines.

meaning – the output of ‘ls’ in powershell is not simply the file-names, but rather FileInfo objects

So, what’s so good about it you ask?

select-string, when fed with FileInfo instances, knows to look for the string within the file’s content instead of its name!

so we can drop ‘cat’ for that purpose and the final version is:

ls C:\inetpub\logs\LogFiles\W3SVC1 | select-string "e.js" | measure

Windsor 3 is here

2011-08-15T12:02:19Z

See the announcement and grab the latest beta bits.

http://groups.google.com/group/castle-project-users/browse_thread/thread/632ef8b47395736b

What’s new?

http://docs.castleproject.org/Default.aspx?Page=Whats-New-In-Windsor-3&NS=Windsor

Major kudos to Krzysztof Koźmic for running the Windsor show so smoothly.

btw – a new design to the castle project website is about to be launched. It looks awesome! I’ll update once its out

Read recommendation–perspective on skills series

2011-08-03T08:07:35Z

Josh Coffman over at Computerist Solutions is running an interesting series of interviews, asking people a few short and concise questions about their coding habits, preferences etc. Quite insightful, and he’s bringing in interesting people who are extremely bright, very experienced, or both. And (at least until now) they are as polyglot as it gets.

Do you know a web architect

2011-07-12T10:45:38Z

Following the worst job description ever, I’ve received an email I got today from a recruitment company, titles the same as this post.

Here is a (translated from Hebrew, structural and grammar weirdness transferred along) copy of the content:

We have an opening that might interest you or one of your friends.

Requirements:

- At least ten years developing/architecting WEB

- Experience with HTTP, JAVA SCRIPT, AJAX, XHTML, SOAP, DOM and more

- Startup/Multimedia/Video experience would make a substantial edge

And this is how I responded to that email:

Dear XYZ

If you’d be able to supply answers to these following questions:

- Who is the hiring company, what are they like?

- What does this company do? what is its primary product/market/services?

- What would be the hired person’s responsibilities?

then I’d be able to say if this would interest me or my friends. Without that, there’s just not enough data to know

Blah blah clarification Blah blah

2011-07-10T21:43:52Z

You’re missing the point

After writing my last the post, I kind-of learned that not everyone got my point correctly. For e.g., someone who did not get my point right, took it personally perhaps, and started bashing me and the .NET crowd (as if I was a formal representative or something) over at Google plus.

Anyway

My points:

I don’t like people from one discipline bash other people and disciplines
Especially when it is acted loud an impolitely
I happens to be that *some minor comrades* in the rails community is more loud in doing that
And I* find it offensive

**according to the comments on my previous post, and after talking to a few people that whished to remain anonymous, I am not the only one.

Not my points:

Technology X is better than Y. It is about flavors and personal preferences. Productivity and quality has lots more to do with the actual team than the technology choices.
Attacking anyone in specific**.

**I’ve been asked to name the persons and/or groups where I was getting the offensive vibe, but I am not in the business of name calling and finger pointing. The people who are offensive already know that, and they are also very unlikely to change anyway so it won’t do any good.

My personal take

Maybe I am too offensive myself. Perhaps I should have titled that last post something like “The Ruby train is sometimes a bit too loud”. Changing it now will hurt the context though, but I will try to be more aware.

Back to being friends everyone?

The Ruby train goes blah blah blah

2011-07-08T05:20:49Z

In reply to the latest and greatest article about how the ruby train is all choo choo. This started as a comment on the said post, but grew too large into justifying a post here.

A couple of pre-words on me in case you got here from that article and you’re not familiar with me.

I've been doing web related and (other kinds of) software development for quite a few years. Just in the last couple of years I did so with Java, .NET, Ruby, Javascript (and to a lesser extent also with python and php, even perl). I actually moved from “fat-client” to web with .NET (circa 2003) and discovered rails later (but quite early – @mid 2006) so I fall into the “.NET devs discovering rails” group. My machine of choice (actually bought it myself) is Macbook Pro. You cannot put any fanboy hat on my head.

And I think that Rails is a cool platform, I really do.

And I like the ruby language. A lot.

BUT

here is my take on the takeaways from that article:

1. RoR is mainstream in Silicon Valley startup scene. Even non devs (business guys and VC types) that I talked with were all chanting about "how ruby is cool and all startups must use rails". It is far from mainstream on any corporate environment, at least the ones I've been in contact with. Why? I have a idea but there isn't enough room here for that. But it is getting a huge traction outside of early adopters circles (along with other disruptive stuff like node.js, tornado, lift etc

2. Rails *is* more enjoyable from *crappy* vanilla .NET environments. But vanilla rails (with no gems) is much worse. Now if you use modern tools and libraries on .net (monorail/fubo/nancy/windsor/nhibernate/etc.) then .NET is at least as "enjoyable" if not more, since you do not have to deal with crap-load of conflicting, questionable-quality gems. As for productivity - this is a whole different story - it is more about the developer's abilities and mindset than about the tools he's using. I know COBOL devs that are super productive, and Rails guys that are destructive at best. Perhaps others think differently than I do, but you should scrap the "every, single, one" bit (or at least the undo the colouring). I might just be that one, and I know more than a few more.

3. Far from being a fanboy MS (with regards to server dev tools) has loads to offer. C# is wildly more advanced and feature rich than any other mainstream language, and the upcoming things in c#5 making it even better. And the BCL of .NET is better and richer than perhaps any other built-in class library today in orders of magnitude. The VM , typesystem and core runtime are very robust and filled with features. Real features, like the advanced concurrency controls etc. So, true that Rails comes with nice sugars that appeal to some people, but once you need to do more than "read from DB render to html" you start hitting walls. While on the other end, if you start with a solid baseline (like .NET's or Java's) you are far less likely to get stuck, and it is easy to find the right sugar to sparkle on-top (there are countless snappy MVC frameworks for Java and .NET, and language choices - even compatible rubies, but far more options). Java 7 got dynamic dispatch, and if Java 8 will at last get closures (and if it will be supported in the runtime and not in the compiler) then we are looking also at good news from Oracle-land.
Btw - if stagnation is the issue - when would the rubyists finally give up on 1.8, and on the hybrid 1.9, and get a proper 2.0?

4. Almost every professional I know that is involved in .NET, is *not* identifying himself as a .NET developer, but as a Software Developer, and they are open for ideas, and they are not religious about their tools. On the other hand, apart from one, all of the Rails related professionals I know of are behaving as if they are the warrior-priests of DHH, spreading arrogance (we saw the light - you will too!, my life were so bad, but now it's all flowers and barbecue, blah blah blah) FUD and loudness everywhere (whitespace is crap! static typing is crap! tabs instead of spaces? crap!). And they ALL identify themselves as "Rails developers".

It is much worse than what the Java community used to be before they started opening up for stuff like groovy, scala and clojure.

My takeaways:

1. As a developer – always strive to be better. Know the tools of the trade. Know the tradeoffs of the tools.

2. As a person – always strive to be better. Know that the are (usually) no black-and-white decisions. know the tradeoffs.

Installing older node.js with latest npm on osx

2011-06-28T19:05:40Z

At the time I write this, Heroku support Node 0.4.7, however the latest release is 0.4.8

Installing in the usual way (brew install node) will bring 0.4.8 in, so instead I opted to download the .pkg for 0.4.7

Node.js was installed correctly, however, it came with an old version on npm (the package manager for node.js).

Searching around for npm installation brought up downloading the sh script and executing it. The snippet is curl http://npmjs.org/install.sh | sh

For *nix noobs – this mean that the content of the file at http://npmjs.org/install.sh will be pushed as input for sh which is the runner for .sh scripts.

The problem is that the script tries to delete old npm versions, but it fails to do so for lack of permissions. Usually one prefixes with `sudo` to make this work, however it still did not work for me. What I ended up doing is breaking the process in two – first download the script into a local file (curl http://npmjs.org/install.sh > install.sh) and then `sudo sh install.sh`

I should be starting a “tips” section on the blog.

Agile Architecture talk by Johanna Rothman

2011-06-28T19:03:19Z

Last week I went to IASA Israel meeting, where we got to listen to Johanna Rothman of the Pragmatic Programmer fame, talking about the role of architects in the Agile projects world.

I’ve been taking notes during. Consider them a transcript + stream-of-thoughts.

here we begin:

so, is an architect role on an agile team should be considered as an

Oxymoron?

A common mistake is to think that architecture is about doing frameworks and designs up-front !

Some people think that the fact that in agile-style managed projects the architecture evolves with time, means that there is no place for an architect on such projects.

This will cause the evolvement to be scattered, and a guiding hand is needed to make all of the small changes drive the overall design towards a meaningful direction which will support the growth of business value.

Feature-itis

Pushing more and more features in expense of sparing time for determining long-term architecture goals will lead to a hole filled with technical and architectural debt, causing future features and adjustments be more complicated than needed, and hurdling the project in the long term.

My observation – feature-itis is even worse because of it usually comes with the lack of proper user-testing. A feature imo is a hypothesis, that has to be proved, so a proper ground (a/b testing, explicit and auditable metrics etc.) has to be in place, and developed alongside the features being cranked out .

Concurrent projects

A product (and at a higher level – a program) is often made out of a few projects, each lead by a different person/division/etc. A project is made of many features and feature-sets. Just like in software, you need to lower coupling between projects, and make feature-sets cohesive. This needs to happen both on the product level (the Principal PM’s responsibility), but also on technology level, which is the architect’s responsibility. Developers usually concentrate on a feature, sometimes on a feature-set. Testers have broader sight, but they would usually be working too details, thus not seeing the overall picture also. It is the responsibility of the architect(s) to see that the project is making sense technically across the whole product .

Some Scrum Master observations:

Scrum is not the only agile FW
A Scrum Master is not a project manager
He does not manage technical risks
He is not the architect

Communication Paths

equals (N*N-N)/2

hence with 10 people there are 45 communication paths!

Architect on an agile team?

So what does the architect has to do, which no one else on a Scrum team usually do?

Wayfinding (what is waiting ahead), roadmap exploration
Scouting (potential risks and paths)
Architectual spike / vertical prototype – with bigger scopes than a ‘traditional’ XP design spike, and would not fit in the regular sprint/release cycles

Timing

Often managers ask: “what is the latest responsible moment for making architectural decisions”?

Johanna’s view is that the correct question is : “what is the *most* responsible moment for making architectural decisions”?

You should not try to postpone important decisions, but instead make them at the most critical times.

My observation – for any project of non-trivial scope, the proper time for architectural decisions is always. That is, there should always be at least one person who stops and think about larger picture, about future paths, etc. (the things listed a few paragraphs back) and it has to happen all the time as the project evolves, not at some predefined spots.

The worst job description ever

2011-06-20T20:39:54Z

The following screenshot is taken from what is probably the worst job description I have ever seen. And be reminded that I used to work for the HQ of a large recruitment firm, been a recruiting manager and technical screener for candidates, and that my wife has years of experience in HR. I have seen my fair share of crap, but this is the worst of it

My eyes are still bleeding, be warned before you click to the fullsize image.

the only missing acronym is WTF

The hungry fox jumped over the lazy mac

2011-06-19T18:08:37Z

And I use mainly chrome for day-to-day. the FF window is only open for a secondary gmail account, and the occasional Firebug session

Object Oriented Mess

2011-06-09T14:25:52Z

Following the “why I do not like (certain widely used paradigms in the ) Ruby (community), I would like to bring yet another specimen representing the same problem.

DISCLAIMER:

I’m writing this on a 10hour flight, that got extended by 3 more hours of waiting at the gate because apparently it take 3 hours to fix “a software problem with the passengers cabin lighting system”. I’m not joking you.
I’m tired, pissed off, and terribly missing my family whom I did not see for almost three weeks. Hence – I might get distracted, digress here and there, and sound a bit impatient at times. Leave now if you possess a soft soul (but do click on the flashy ads on the sidebar before you do that).
This is also going to be a much longer post than my usual ones I’ve had for the last two years +. I hope to make that happen more often, as I have some more free time now that I left Delver. More on this on a (near) future post.

It’s not you, it’s all of you

Since I’ve been doing mostly rails and ruby lately, my examples would probably have a ruby color spread all over then. Don’t get me wrong though, this is not going to be an anti-Ruby rant. People certainly are doing the exact same thing in the Java and .NET worlds, especially people who have “read” Evans book but never really groked it. They are 100% certain that OOP is about “making your objects smart” and forgetting what an object in that sense should actually represent (hint – encapsulate business goals and processes in well defined boundaries). Perhaps the way OOP is taught universities is to blame (Animal Kingdom? really?) as they concentrate on the technicalities of Is-A vs Has-A and not on how to approach modeling in the first place. In order to make things worse, I will use the terms ‘class’, ‘object’, ‘instance’ and ‘whuuzaa’ interchangeably.

So, where was I? ah – another concrete example for bad design caused by oversimplified OOP implementations

.url

(or .family_name, whateva)

OOM – Object Oriented Mess

So you have your domain objects (or Models if you’ve never heard about other frameworks but Rails) and since you did not read Evans book you think that you should “encapsulate their logic” and by that you refer to the .url property. Cuz the User class (/object/model/entity/whuuzaa) *represents a real user* and *that is the model* for the user. So in your application (say a website) you’d want to include urls to the user’s profile page, so you’d be giving your view engine (template, jsp, view script, xib, you-get-te-point) a bunch of Users, and then you’d call .url on each of them – and voila – the “generate user profile url” problem got completely abstracted !. You’d even have a IHaveUrl interface (or UrlContainer if you’re from java, or a creepy act_as_urlifiable Module if you’re from … enough with that) and then you can enjoy the fun of calling .url on *anything*.IT WOULD JUST WORK.

Until it doesn’t.

One model to rule them all

Is there really a single way to represent a User in your system? I bet that a User instance used to populate the User Profile page is quite different and a lot richer than a User instance that would be used in a recurring list of the User Profile’s friends list, which would require perhaps only name, image, and guess what – url !

What’s that thing doing here?

Within a recommendation-engine service, that runs complex calculations on the users and their related activities on the site to get relevance scores, the url does not mean anything. If the User Entity is used there then the .url method is just hanging there meaning nothing, contributing nothing while perhaps adding complexity and confusion

Oops, sorry, I didn’t mean to say that

And to make it worse – on other parts of the site (think an admin back-end) a user’s url can mean a completely different thing – like a url to the whereabout of the user’s administration page, or whatnot.

Hey there are other people here

Put the url example aside, if you do a feature, which is relevant for a certain part of the application, yet you choose to expose your code to central part of the system (such as a central User class), then you end up stepping on other people’s toes. And they most certainly will step on yours. At some point in the future some bugs in your feature will creep in, and you’d have hard time locking the problem down, before noticing that someone else used the code you exposed on that entity for the sole purpose of your feature, for a whole different thing, giving that code a different meaning, which inevitably clashed with yours.

So what would YOU do Ken?

Encapsulating features in their own set of classes would be the way to go. The surface of the system that need to be aware of a feature should be kept to minimum.

I think that it would be a safe bet to say that the process of generating a profile page url for a given user (or any other ‘thing’ in the system) should live within the boundary of the presentation code, probably as a helper or a static method (don’t get me started on the never-ending testability argument – that’s a whole different post. Just make your code work already and stop arguing about testability. already !) You might want to add some syntactic sugar there to get the .url thing working (like a c# extension method or a Ruby module) but I’d avoid even that. It is good avoid a code that might have dual meanings. I’d take <%= Urls.ProfilePage(name, id)%> (maybe with a couple of overloads) over <%= user.url %> any day of the week. Especially on Fridays when I don’t have time for surprise-bugs-oh-crap-my-weekend-is-going-to-be-short-again.

You’re the best reader in the world

Btw, if you got to this point without either getting bored or pissed at me, then hurray to you. You are either a good friend, totally drunk, or both.

jQuery pwned by Microsoft

2011-05-11T12:22:43Z

Just read it on the InfoQ weekly e-mail:

It does not mean much, but I did find it amusing.

Technology stack choice for startups

2011-03-28T13:42:25Z

There has been some rants lately going on about “you shouldn’t use .NET (and J2EE)”, and “shouldn’t hire .NET people” (see here, here and here for e.g.)

So Ron went ahead to ask on the alt.net mailing list “is specializing in .NET risky”?

My short answer is: specializing in a single technology only is somewhat risky, as the world keeps changing.

I would never hire a “.NET developer” or a “J2EE developer”, or a “Ruby developer”. I will hire a great developer, and I do not really care which language he used the most lately, as long as he is great in it. Here in delver we had many success stories with great people coming in with having almost no experience in .NET or Java (which are the major technologies we use here), but php/python/c++/whatever instead. Because they are great people, they quickly covered the gap, and became productive in no-time.

Because, at the end of the day, even if you use technology A, and you hire someone who is an expert in technology A, then that one would still need time to adjust to the specific paradigms, techniques, and spirit of the code used in your company. So why limit yourself (and to limited people - ones who care religiously about language instead of caring about being great)?

Anyway, this is my take on the specific MySpace story:

I've seen the Scoble article on Myspace being killed by MS technologies, and I call bullshit.

It is classic "Not My Fault Lets Blame Someone Else" syndrome.

Specifically, MySpace were driven by bad business decisions (like using a 3rd party package for its core functionality - the 'social engine'), and bad management and leadership (it is owned by an old-media corporation, which is a firm believer in pre-web2.0 concepts - re the new iPad newspaper that does not allow talkbacks or any other feedback mechnism !?!?!?? )

It has *nothing* to do with specific technology. Not even with bad system architectures.

As for the "cannot find good XYZ people" - this is again a fallacy. Good (and especially superb) people, which is what you want in a startup (or anywhere for that matter) would always be polyglot. They would adapt to any technology. Skills has nothing to do with a specific syntax, or familiarity with certain libraries. Cuz these change rapidly anyway.

As for the "Hollywood is not a place for startups" - well probably yCombinator is currently only suited for the valley, however a startup that already has a seed investment, and has a reasonably crafted business plan, does not *have* to be in the "meet all the dudes in the coffee shops" scene. Plus LA is about a couple of hours flight away from the valley, or 6 hours drive.

And people *will* come work for you if your product is right, your company is great, compensation is good, location is appealing - more or less in that order.

As for "it does not scale" - it is never about the technology stack. It is architectures and processes that do not scale.

My bottom line - choosing the stack has very low impact on the overall success probability of a company, because it has no impact on business decision. What is important is the possibility for a company to quickly adapt to changes, and be able to deliver features and changes as fast as possible so that it could get feedback from users and grow accordingly.
So what does matter is the skill set of the first technical person(s) in the company, whichever stack they are most rapid with. Later additions to the team will be made of excellent people, which will adapt to the stack if the product and the company is compelling enough.

so worry about making your product fricking compelling, and your company a fricking great place to work at. Leave religious debates to silly youtube clips.

Fetching from many urls concurrently - the right way

2011-03-15T22:50:30Z

I’ve recently bumped into a post describing how to access a bunch of urls. The post was describing usage of Parallel.ForEach as a mean to parallelize such code.

It goes down to describe the the improvement over the iterative, single threaded version, was bounded to the number of cores.

Multithreading is definitely a way to increase performance of long running, independent jobs. However, there is a major difference between jobs that are CPU bound (such a complex computations) and jobs that are IO bounds (file-system, web services, DB calls, whatnot). First, there is no reason to keep all those threads waiting for the IO call to complete, and second - the number of available threads is limited, so the improvement is bound to the number of cores.

The way to achieve a better improvement for the said scenario and similar others, is to use non-blocking calls that are using IO-completion ports instead of threads.

I’ve ran three versions of the code - serial, parallel, and Async IO based. The results are stunning. For 300 files, the serial one took ~3.5 minutes, the parallel to ~30 seconds, and the Async IO base took just under a second!

Notice the ServicePointManager.DefaultConnectionLimit = 1000; part, which tells the application how many concurrent outgoing connections are allowed. A modern windows host allow up to ~64k file descriptors (the stuff that amongst other things allow this behaviour) so a 1000 is not a problem.

I also like CountdownEvent very much. Easier than ManualResetEvent with a counter and Interlocked.Decrement calls.