Bitter Olives

Collaborative coding

2009-04-25T14:57:00.002+02:00

I would promote this idea to a higher level than just 'snippet sharing'. I can envision something called collaborative coding.

Collaborative coding embraces the uniquely identified code snippets but enhances it with a timestamp of modification and user comments.

Say,

/* Calculates the fibonacci number for the parameter
guid: {1c125546-b87c-49ff-8130-a24a3deda659}
date: 2009-04-25 19:00
comm: First version */

int fibonacci(int n) {
if (n<=2) return 1;
else return fibonacci(n-1)+fibonacci(n-2);
}

/* Calculates the fibonacci number for the parameter
guid: {1c125546-b87c-49ff-8130-a24a3deda659}
date: 2009-04-25 19:25
comm: Can't use recursion as my stack is too small, changed to an iterative version */

int fibonacci(int n) {
if (n<=2) return 1;
int j = 1;
int k = 1;
int ans = 0;
for (int i = 3; i <= n; i++) {
ans = j + k;
j = k;
k = ans;
}
return ans;
}

Then tools can show diffs over time of the different versions of snippets, with comments (if available) and be made to push back the snippets to available repositories.

You could actually end up having conversations in code. The identifier should identify function, and diff algorithms can show what has been changed and hopefully why.

WWW2009

2009-04-24T20:41:00.006+02:00

I was lucky enough to get funding to go (without having sent a paper nor being an active participant in the research community) to the last day of WWW2009 Madrid!

Lucky indeed as I hadn't been before in such a global and well organized conference. I'm sure there were some problems (the people in charge apologized in the closing ceremony) but I personally didn't see any.

I went to the following talks:

- Invited talk about web infrastructure by Pablo Rodríguez the Internet Scientific Director at Telefonica. He explained the problems current internet architecture which mainly consist of obsolete, generic protocols, where distribution is hard. He suggested to specialize protocols (where you route content, not hosts) and push things down a layer or two (where every router has a terabyte storage to do an akamai-for-the-people).

- Invited talk about web search in web2.0 by Ricardo Baeza-Yates, VP of Research for Europe and Latin America at Yahoo!. He showed a pretty interesting summary of the recent research done by him and his team on searching text/images using mainly user generated content (tags, tags of objects in pictures). Summing up, using user generated content can enhance search results significantly.

And some highlights from the papers presentations:

- Social searching: Porqpine

- Synonym extraction: Ways to detect that queries like "vaio 720 laptop" and "vaio 720" are synonyms in an efficient way

- A framework to manage digital rights: Interesting idea, a good model to license, but sadly it requires everyone to use a centralized website and to download an extra application to access the actual content. It just doesn't pass the grandma test.

- A reference implementation of the web coverage processing service standard, Rasdaman (plus some web tools which use it to justify its presence on WWW)

- Leveraging web search engines to query databases: A clever approach to enhance search results with structured database results, typically useful when searching for entities which are usually in structured DBs (movies, electronic devices and so on)

- YUI 3. I wasn't present at the talk, but looks nice

All in all, it was a very good experience and I now have a CD with 215 pdf files to review (between papers and posters)... that's some hard work! If I manage to do any of it and see something worth, will comment on it. Pics here.

Bach's passion

2009-04-18T11:38:00.007+02:00

It's amazing how a good communicator can inspire. Especially a passionate communicator who shows how much he cares about the subject at hand. If you haven't seen this, you should: James Bach's talk @ Google.

What I found really interesting is the combination of subjects he manages to make relevant for his field. We all should follow his lead and find inspiration and guidance from more than a single discipline.

Learn, not just about technology, but about social sciences which make the technology relevant and also make the development of the technology easier. For example, if you could communicate better and understand people's limitations on what they can perceive, you would be able to gather better requirements, or capture bug reports more efficiently, or just develop humane software (*). Not to mention the applications outside the world of software (love life, friends, networking your career).

In fact, this all points to aspiring to becoming a generalist, the more you can aprehend about different subjects the more you can then combine in different ways under different contexts. Current schooling does not aim to this, but you can always take care of your education (in addition to or instead of the regular, your call). In the words of the great Paul Lutus "do not let your schooling interfere with your education".

(*): More on humane software some day.

The case against the case against everything buckets

2009-03-30T21:55:00.003+02:00

Alex Payne writes a good piece about how everything buckets are awful.

He defines everything buckets as applications where you can store every imaginable piece of data and then search over the RTF or PDF documents it generates. Or, in his own words:

These applications claim to be “your outboard brain” or “your digital filing cabinet” or similar. They go by many names: Yojimbo, Together, ShoveBox, Evernote, DEVONthink. There may be differences in their implementation and appearance, but these applications are all of the same sinister ilk. They are Everything Buckets.

He then continues on how these kind of applications suck because they try to do many things at once and fail miserably at most if not all of them. Additionally, he says filesystems already have all the features that allow you to organize all the information you want (you can even have tags via symlinks, he stresses). There's no need to pay for these lousy implemented, do-many-things-wrongly apps.

Computers like structured data and so filesystems are a good way to store hierarchical information without the need for expensive indexing tasks current operating systems usually perform (Spotlight, Beagle, and so on).

This is all good and well, though there are people who think hierarchical filesystems suck.

Anyhow, even if everybody agreed that current filesystems are Sliced Bread 2.0, this well construed argument lacks a pretty important consideration, which is people want the computer to organize their information for them, even if current applications blow. Alex is too quick in dismissing the concept, only because there is no good implementation of it.

All these arguments could have well been used against web search 12 years ago, and I need not mention what happened to it. A good everything bucket would not prevent you to do work with your computer when you want to, nor would it corrupt its data, additionally it would cross reference the data and automatically insert what you throw at it into relevant buckets with little user input.

Alex's advice is right in that, as of now, you are better trying to organize your information manually via the filesystem (YMMV), but he's wrong in that you will never be able to have a good everything bucket application, and even more wrong when he says everything buckets shouldn't exist.

The need certainly exists. The fact that current individual personal computers suck at managing unstructured information and take too much time at creating structure from the chaos is merely accidental.

Let us create a good Everything Bucket soon enough.

UPDATE: As always, I arrive late to the party... and, of course, some people like current everything buckets. I still think they can and should do a lot better.

Excel Interop COMException HRESULT 0x800A03EC via C# (.NET 2.0)

2009-03-09T14:42:00.003+01:00

Today I'm writing some code to read Excel files via the interop assemblies. I was reusing working code and it suddenly started failing with a COMException which said the error code was HRESULT 0x800A03EC.

I'm using the following code:

string excelLocation = @"C:\test.xls";
Microsoft.Office.Interop.Excel.Application _app = new Microsoft.Office.Interop.Excel.Application();

Workbook wbook = _app.Workbooks.Open(excelLocation,
Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing,
Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing,
Type.Missing, Type.Missing, Type.Missing, Type.Missing);

Worksheet sheet = (Worksheet)wbook.Sheets[1];

/** rest of the code **/

and I get the COMException in the line where I open the workbook.

So, I open the file via regular Excel and I find out it finds the file somewhat corrupt, Excel tells me it had fixed some font and sheet name problems with the file and that if I want to make that fix permanent to save the file. I save the file and then the code works!

But wait, this is not the proper solution if you want a truly automated processing of the files. Given that I don't control the Excel generation process I cannot fix the corruption on the source side of things, so I'll have to fix it on my side.

I started to dig into the Open method parameters and found that the last parameter is suggestively called 'CorruptLoad' so, the solution:

string excelLocation = @"C:\test.xls";
Microsoft.Office.Interop.Excel.Application _app = new Microsoft.Office.Interop.Excel.Application();

Workbook wbook = _app.Workbooks.Open(excelLocation,
Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing,
Type.Missing, Type.Missing, Type.Missing, Type.Missing, Type.Missing,
Type.Missing, Type.Missing, Type.Missing, true);

Worksheet sheet = (Worksheet)wbook.Sheets[1];

/** rest of the code **/

And that opens it without throwing an exception and gets all the data in the sheets, at least for this particular type of file corruption.

Wondered this too?

2009-01-13T17:53:00.002+01:00

I had always wondered why you only see 3 GB only. Usually simplifying explanations to you need 64 bits OS so see 4GB.

Real explanation is here

There is no hope

2008-12-20T16:51:00.003+01:00

If one's to give any credibility to this study, then campaigns like this one stand no chance at all to light up a fruitful dialogue.

Nonetheless I'm glad things like this are happening and the voice of the non believers is making itself heard next to the believers' voice, who usually tend to shout way too loud for anything else to be heard.

Visual Programming Revisited

2008-12-20T00:01:00.005+01:00

I find Subtext (soon to be called Juncture) highly interesting, the basic premise is true (we are restrained by text) and the solution looks like going in the right direction, as less work and easier reasoning are always welcome. I still wonder how will that extend to huge systems, but I'm willing to let it pass for now.

Check the video on it and you'll be surprised. You might love it or hate it, but it will surprise you. That's just how it is with great ideas.

You should also check Jonathan Edwards' (the author of this experiment) blog, Alarming Development.

Flames

2008-09-04T08:15:00.010+02:00

As much as I hate the pussyfooting, heartwarmingly tender posture of the peace loving, criticism avoiding please don't shout and be quiet crowd:

When we do make mistakes, just point them out politely, there is no need to shout.

and regardless of how much I love the tough, impolite, harsh and rough posture of the flaming crowd:

The common refrain (enumerated in "Why Shout?") is "Can't we all just get along?" The answer is no. No, we cannot. Indeed, every community I've seen in which people "just get along" (usually through strict moderation) degenerates into a feel-good circlejerk devoid of any useful critique. You need only look at a place like DeviantArt to see this in action.

The alternative to this is a community in which criticism is largely unmoderated, inevitably leading to scathing critiques and outright flames.

I think there is a third way.

Disregarding the classical flame concept, which by definition contains personal insults, I propose a new classification:

a) technical flames, where mistakes are pointed out in a rough or hostile manner with no time wasting for politeness (note that in the example the insults are given to the technical contents, not to the person arguing for them) and

b) personal flames, where no technical content is given.

Couldn't a moderation system take care of the latter while allowing the former?

I believe it could. It just needs the proper moderators. Of course there will be controversial content and you have to draw the line somewhere, but just by doing nothing when unsure, that is, by allowing the content stepping on the line, everything should be okay.

I also believe it would be desirable. I'd rephrase the "there's no need to shout" for "there's no need to call somebody an idiot just because he is one". Just let the facts show him as one if you feel that kind of urge. It's harder, but a lot classier.

So true

2008-09-01T16:11:00.002+02:00

This is so true. The original source is less interesting, and in my opinion it reads better without the original context.

What are you doing to be above it?

Naming inconsistencies

2008-09-01T12:11:00.005+02:00

Look at this, forget everything else you know about these commands or the products behind them and think for a while:

root@parrot:~# my
myisamchk mysqlaccess mysql_client_test_embedded mysqldumpslow mysqlimport mysqlshow mysqltestmanager-pwgen myisam_ftdump mysqladmin mysql_convert_table_format mysql_explain_log mysql_install_db mysql_tableinfo mysql_tzinfo_to_sql myisamlog mysqlbinlog mysqld mysql_find_rows mysqlmanager mysqltest mysql_upgrade myisampack mysqlbug mysqld_multi mysql_fix_extensions mysqlreport mysqltest_embedded mysql_upgrade_shell mysql_print_defaults mysqlcheck mysqld_safe mysql_fix_privilege_tables mysql_secure_installation mysqltestmanager mysql_waitpid mysql mysql_client_test mysqldump mysqlhotcopy mysql_setpermission mysqltestmanagerc mysql_zap

root@parrot:~# pg
pg pg_ctlcluster pg_dump pg_lsclusters pgrep pg_upgradecluster pg_createcluster pg_dropcluster pg_dumpall pg_maintenance pg_restore
(pg and pgrep are not postgresql related)

Same things happen if you start looking at the command line options (try mysqldump --help and pg_dump --help).

So, which one is easier to use? Which one makes more sense? What does this tell you about the two projects? ($)

To me, it says one has organized, well thought, planned development, which transpires the whole product, even to the user interfaces. The other one has haphazard, randomly planned development, which probably percolates from the user interfaces to the code (*), or viceversa.

I'm not saying this demonstrates MySQL as an inferior product. I'm saying this demonstrates MySQL is less organized, cares less about user interfaces and probably is less organized in a lot of other areas as well. I cannot be sure about that, but it sure is the impression it gives to me.

Of course this is not privative to the MySQL project, same naming inconsistencies plague PHP, for instance.

I guess what I'm really trying to say here is that you must take care about all pieces of a project, the more care you take, the better it will look and the easier will be to make it as good as it looks.

What impression is your project giving?

($): Of course one single fact cannot tell the whole story, so please apply standard disclaimers.
(*): I must confess I haven't looked at mysql's code so I cannot know.

Win32Exception error code 267 "The directory name is invalid"

2008-08-31T11:15:00.003+02:00

I'm currently working on an "Automated Run As" tool. Its goal is helping admins which, like me, have to give users a means to execute one or two programs as Administrator and would like to do so without having to surrender an admin's password.

So, I'm developing on Vista and I just whipped up a small proof of concept prototype, that'd run calc.exe as a different user, using ProcessStartInfo and Process. This worked fine when I executed it as myself (a rather pointless exercise, I must admit), but when I created a new user and tried to run it as him, I stumbled upon a Win32Exception complaining that the directory name is invalid, native error code 267. I was instsantly baffled, as I knew of no supplied directory name that could be invalid. I then tested the code on an XP machine and it worked!

I started googling on it to no avail, many reports of that error but no conclusive solution, or on different contexts. Finally, after a while it dawned on me, I wasn't specifying the WorkingDirectory property of the ProcessStartInfo class, as soon as I added the lines

FileInfo fileInfo = new FileInfo(path);
startInfo.WorkingDirectory = fileInfo.DirectoryName;

to my code, it was allowed to run code as different than logged in user.

As to why, I haven't looked deep enough yet, I think that it may have something to do with default values of the property or with default behavior when it is not specified, plus default permissions (which were tightened on Vista, if I'm not mistaken).

Do *you*, my non existent reader, know why?

Moving from a Workgroup based network to an Active Directory one

2008-08-22T20:01:00.007+02:00

Back from vacation! I spent this week (yes, you read that correctly) migrating our 10 machine workgroup based network to an Active Directory setup. Despite the long time taken I must confess it was not too painful, as most of that time was spent first designing the user/group/permission setup I wanted to enforce in our network (previously everyone was an admin and we had a mess on the network) then reconfiguring the damn application we have to run on all the machines and waiting for data to move, machines to reboot and so on. I learned more about Windows this week than in my previous 15 years working with computers (mainly on Linux :-)), I honestly didn't want to know that much.

First the good news, I used a Windows Server 2003 (x64) as a server and all our machines are XP Pro so there was much pain avoided. I've read that in an heterogeneous or bigger setup lots of issues arise.

So, the steps I followed and stones I had to kick were, roughly:

1) Configure Windows Server 2003 DNS, making it serve a fake domain just for AD, thus avoiding to configure zone transfers between our ISP's DNS and this server. Also I make this DNS to cache our ISP's one for the clients to be able to navigate normally.

2) Configure Active Directory, define shares and some other stuff, the wizard is fine for that. Afterwards define the groups and users you'll use.

3) Configure each machine network connection to use the Windows Server's DNS.

4) Go on each machine to System, Computer Name, Change, pick Domain and put the name of the domain. It'll ask you the password of a Domain Admin so the machine can be added. Reboot and the machine's now a member of the domain!

5) I wanted for each user to be able to log on on any machine and carry his profile with him, so I learned about and configured Roaming Profiles. This is done by first defining a share that'll store profiles and having write permissions for the users' group, let's say \\server\profiles. Then you must log on to the machine as an administrator (and also a DIFFERENT account than the one you want to copy) to copy the profile of the user of the machine by going to System, Advanced, User Profiles and click the button and then "Copy To" for the profile you're interested in. On the popup we select the share we created plus the username, for instance, \\server\profiles\jdoe and in the lower part of the popup we can tell which domain user can use this profile, in this case jdoe@domain.local. Then wait for the data to move to the share. Afterwards configure the account on the domain controller to use the profile located on \\server\profiles\jdoe (This is under the Profile tab, if I remember correctly)

Caveat: When the Copy To button is grayed out it means that the account the profile belongs to is either logged on or a process is running with that account's credentials (this last one took me a while to figure out).

6) Now we can test the profile works by logging out of the admin account and logging in as the domain user in the machine.

Okay, so, what's next? We have all the machines in the domain and the roaming profiles set up. Why have them in the domain if we won't use it, no? So

7) Define and configure the Global Policy. This is too huge to get into details but basically you can configure here everything from registry access to installed software to firewall setup to password policies for all the machines in the domain AND for individual users (which would override the machine's one if they are both set up for the same parameters). To do this it is recommended to download the Global Policy Management Console (GPMC) which can be downloaded here, and a relatively good reference is here.

Caveats: It needs .NET Framework 1.1 and it only works on 32 bit machines. Good news is it can be used from any machine in the domain logged in as administrator. Also you must remember to activate the policy with the command gpupdate /force on each machine you want the policy to get activated right now as opposed to when it reboots or somebody logs off and on

8) Test your applications, see if you need extra rules on the firewall or permissions on the folders, etc.

Pending issues:
a) Remote software installation (ie, not needing the msi supported by the Global Policy) or else how to convert any app to a GPO usable msi.

b) Write a tool to run programs as different accounts without sharing the administrator password.

c) Find out about controlling a user's session without having him log off from the machine, all I've managed so far is to use remote desktop to log off the user, which is not nice, sometimes I'd like to enter her own session.

Phew, that was quite a bit of information :-)

Hope this helps somebody.

ConBLUcurrenEScy

2008-07-30T13:26:00.005+02:00

If you can read the title then you know what I'm talking about.

We use an application which allows many machines to log in with the same user. This application automatically saves changes on a database table and "locks" stuff in a boolean column called locked from the same table (ie, no native locks are actually used). I bet you can imagine the potential for disaster this combination allows.

So, I've been pretty much the last hour very busy trying to manually restore the state of a corrupted database. I see you, my non existent reader, immediately asking things like:

a) Why didn't you restore a backup, moron?
b) Do you know squat about ACID?
c) Do you, in general, have a clue at all, asshole?

a.- Because backups are done nightly and the lost work was from this morning. (This reminds me
I should set up some transaction log backups so as to lose at most 15 or 30 minutes, have to buy more hard drives!)
b.- I have no control over the application, sadly. I must report these bugs and hope they fix them. Proprietary world hell...
c.- I have to!

Well, current disaster is: some values disappear as the application loads values from the DB at startup, does changes locally in memory, then saves upon exit or at regular intervals. Two users sharing credentials doing this and only the last to exit will get its values saved.

Of course I'm telling people not to use the same credentials, but we all know how we are so that's not a safe solution.

I really hope there is some reason besides 'users should not share credentials' for all this nonsense.

For my future reference, I must use:

"SELECT * FROM scInfoTbl WHERE father NOT IN (SELECT cId FROM scInfoTbl)"

to find which cs are orphans. Orphans kill the application, as it is not ready to handle
an orphan c (crashes with a NPE).

As you can deduce, scInfoTbl defines a tree of c, with the father column pointing to the parent leaf (or root) of the tree. Of course a leaf cannot be an orphan, so why check it? Grr...

Saving a swift

2008-07-29T20:41:00.006+02:00

A friend found a baby common swift on the ground, he let us have it and now we are on a quest to try and save it. These beasts are quite wonderful, they fly for months in a row, only touching the ground to breed like once a year. Saving this chick will probably be very hard as they are quite sensitive to human fungi and bacteria and they can't damage their wings or feathers in the least.

Anyhow first step for tomorrow morning: see if we can find the proper food (crickets, wax moth larvaes, bee drones) here in Madrid.

I'm going to document the process and put some pictures. Hopefully all will go well and we'll see how he flies away in 4 or 5 days or so. If not, we at least informed ourselves and did what we could.

PS: This site seems to be a wonderful place to learn about this amazing bird.

Hey!

2008-07-29T18:21:00.002+02:00

Hello folks.

It's known that the only way to get better at something is by doing it. So, this is my first (in truth, like my fourth, but let's pretend for literary sake) attempt at getting better at expressing written thoughts.

Sadly, another requirement to express better thoughts in writing is to have thoughts in the first place and in that department I'm kind of lacking. My mind is so quiet I'm the envy of zen practitioners. But, going back to the first phrase, this will serve me to practice thinking as well.

Speaking of thoughts, I wonder, will any people get here without any publicity at all, if so, how? Why are you reading this? Would you like to comment? Need I publicize this to get an audience? Please, leave me a note saying how did you end up here.

See you, probably after holidays... or before, who knows.

PS: Thanks niq for the blog's name :-)