Machines Plus Minds

On big technical meetings, or why the end of the UDS is a bad idea

2013-02-26T15:15:00.000-08:00

Canonical has just announced that the Ubuntu Developer Summit will no longer be face to face and every six months. Instead it will be entirely online and virtual, using Google Hangouts. (Here is the announcement.)

On the surface, this seems like a good idea: It's cheaper monetarily, it appears to open things up to people who are unable to travel, and it makes it easier to make complete records.

However, I think it's a bad idea, for several interrelated reasons.

Some decision making needs face-time to happen. For whatever reasons, internet-only communication is not enough for a good enough "meeting of the minds" for sticky or subtle engineering and design decisionmaking.

The IETF, who probably have the longest history of any organization ever of online internet-enabled collaboration, worked out long ago that while day to day collaboration can be done over email and text chat, some technical decison-making HAS to be done face to face. Thus, the IETF meets every 6 months.

Likewise, at the old MySQL AB, even though the entire company was famously completely distributed, we also figured out that despite being on email and IRC with each other every day, we had to meet ever 6 months, for face-time decision making. Thus, the whole company met every year, and then each team or group met together at least one other time over the year.

And then, most anyone who actually does a working attendance (as opposed to just helicoptering in to give an executive keynote, or being whisked off to a secured conference room to have a private upper executive meeting) at a technical conference knows, most of the ACTUAL work at a conference or at a technical design summit happens in the hallways, over dinner, in serendipitious meetings, in people introducing people to each other, and in impromptu engineering meetings.

These are the reasons that the OpenStack community meets together every 6 months, for our own design summit. The keynotes, the vendor booths with their signboards and handouts, the standard podium-and-rows-of-seats are, at best, a sideshow, from where the real work is getting done, the reason for the summit: the circles (not rows) of seats for the design summit meetings, and for the hallways, informal dinners, and social mixers, where all the individual meetings and necessary social processing happens.

I started to make a list of all the times I personally was part of such unstructured un-"planned" events at conferences that had significant impact, and the list grew too long, so I cut it from this post.

Email and IRC and etherpad are awesome tools, and I commend Ubuntu, as well as most other large collaborative open source projects, such as OpenStack, for using them. Likewise, Google Hangouts seem to be pretty awesome, and I'm glad that Canonical is trying them.

However, they do not replace face to face large group meetings, and cannot solve the problems that such gatherings can.

I wish Canonical and Ubuntu well, but this is a mistake that I hope does not damage them too much. /p

Maven's role...

2013-02-13T02:13:00.004-08:00

"Maven is a great tool to make sure that your expensive and critical application servers are running an independent copy of every single version of every single 2nd and 3rd party Java library ever written."

Why I love the Hallway Track, or instigating a junk OpenStack cloud

2013-01-31T17:15:00.000-08:00

I just had an experience that reminds me why I find physically going to open source conferences valuable and rewarding.

I am here at the last day of Linux.conf.au 2013 in Canberra. Earlier today, Tim Berners-Lee delivered his keynote. Afterwards, we all moved over to the main public hall for afternoon tea.

I happened to overhear a trio of young university students talking about the huge presence of the OpenStack project at this LCA, and expressing some misconceptions about the project. Two of them had never even heard of OpenStack before seeing it presented here at the conference.

As one may do in the "hallway track" of conferences like this, I jumped in, and introduced myself, and gave them a better overview of what OpenStack is and what it tries to do, while handing out business cards

"You mean with this OpenStack, I can run my own cloud?"

"Yes. You do have to supply the hardware."

"Well, our department is throwing out heaps of old PCs. We could gather them up, haul them down to our student computer club, and install it on them..."

I encouraged this line of thought, and pointed out that having ops experience and dev experience with OpenStack is right now really good for getting a job.

THAT got their attention.

"I could get a job with HP if I do this?"

"You could get a job at lots of places. Lots of companies are getting into OpenStack, and they are hiring."

When I left them for the next talk, they were talking about getting in touch with all the other Australian university computer student clubs, each club installing OpenStack on recovered junked PCs, and joining them all together as availability zones.

I like to hope I've instigated something fun here. Or at least made some people's lives more interesting.

Thoughts on Google, YubiCo, and "The War on Passwords"

2013-01-18T15:40:00.001-08:00

There are a lot of articles going around the blogosphere today about Google "Declaring War on the Password", and showing picturers of a YubiKey.

While I am a fan and proponent of improved trustworthyness of authentication, especially with using 2 factor protocols like HOTP and TOTP and devices like the YubiKey, I am curious as to what all the hubub is about today.

What keeps Google Authenticator and YubiKey from easily working together right now is the fact that Google uses TOTP and YubiKey implements HOTP. They are almost the same protocol, with one important difference. TOTP is time based. That's what the T stands for. Every fixed internal (usually 30 seconds) a TOTP token generates a new password, which means that token needs to know what time it is, which means it needs a clock. While a HOTP device like a YubiKey just needs to keep a counter, and generates a new password every time it's button is pressed.

So the Google and Yubi partnership means one of three things.

Google is going to support HOTP on the Google Two Factor Login service, or
Google and YubiCo have figured how to to put an extremely low power clock and extremely small battery into a new version of the YubiKey, or
Google and YubiCo have written a USB device driver that speaks to the YubiKey when it's plugged in and tells it what time it is to generate the correct password (which means that driver needs to be installed on every Windows/Linux/MacOS/ChromeOS device you want to use the token on)

I look forward to seeing which one it is. My money is on option #3, with the added guess that it will probably only be supported, at least initially, only on machines running Chrome or ChromeOS.

Mark's Stories: "The carpets are so clean, we don't need janitors!"

2012-08-04T07:25:00.001-07:00

(edit: Hello to the folks who are coming here from YCombinator Hacker News. Feel free to comment here as well as on YCHN. I've posted a few more details about this story at YCHN.)

At one company I worked at, one of the problems it didn't have was IT.

When someone was hired, by the time they got to their new desk, there was a computer on it with the correct image on it, their desk phone worked, their email worked, the calendaring and scheduling worked, and all necessary passwords and ACLs were configured. The internal ethernet networks all worked, were fast, and were properly isolated from each other. The wall ports were all correctly labeled, and there where the right kinds of wall ports in each cubical and conference room. The presentation projectors and conference room speaker phones all worked. The printers all worked, printed cleanly, were kept stocked, and were consistently named. The internet connections were fast and well managed. Internal and external security incidents were quickly recognized and dealt with. Broken machines were immediately replaced with working and newly imaged replacements. If someone accidentally deleted a file, getting it back from backup typically took less than an hour. Software updates were announced ahead of time, and usually happened without issue.

The IT staff did not seem noticeably bitter, angry, harried, or otherwise suffering from the emotional costs traditionally endemic to that job role. In fact, they were almost invisible in their skill and competence.

So, of course, came the day when the senior executives said "the carpets are just naturally clean all the time, we don't need all these janitors!". IT was "reorganized" into a smaller staff of younger and much less experienced (and probably cheaper) people.

Of course, it all went to shit. New employees would go a week before they had machines, phones, passwords, and ACLs. Printers ran out of paper, projectors ran out of lightbulbs, servers ran out of storage, networks got misconfigured, and so forth. The total time lost and wasted across the whole company was most certainly greater than the savings of laying off the expensive and skilled IT staff.

This is not to say that the reorganized IT staff were stupid or lazy. They worked very hard and ran themselves ragged trying to keep up with the cycle of operations, while trying to skill themselves up in their "spare time" and with a slashed training budget.

The lessons I learned from this experience speak for themselves.

What lessons that may have been learned by any of the other people involved, especially the executives who made these decisions, I cannot say.

Mark's Stories: The Closed Source Drivers

2012-08-03T13:38:00.002-07:00

I was working on a highly constrained consumer electronics device, a little "satellite device" that spoke to the main device over a CATV RF coax cable and also received commands from an IR remote control. My code was failing in bizarre ways. I adopted an extremely paranoid defensive programming stance, filling my code with asserts and doing paranoid cross checking of all inputs. This didn't make the device work. Instead it now consistently did not work, instead of inconsistently, because the cross checks and asserts would usually (but not always) trip before it would crash. It also started to run out of memory because of the all the paranoia code I had added.

I asked for the source code for the driver for the IR receiver, and for the driver for the CATV RF digital transceiver, and for the peer code that was driving the cable digital that ran on the main device.

The driver for the CATF RF digital transceiver was handed to me the first time I asked. And by "handed to me" I mean that I was pointed to where it was sitting in the source repo.

The business partner / hardware supplier who was supplying the IR glue and drivers, after giving me a runaround, finally just flat out refused, citing trade secrets, confidentiality, secret sauce, and similar bullshit.

So, I finally "stole" the source code with a disassembler. And found the sources of many of my problems. It was complete shit. "Unexpected" input from the silicon would cause wild random pointer writes. And random sunlight on the receiver optics would cause it. "Expected" input of undefined remote commands wasn't much better, generating and handing back blocks of garbage with incorrect block length headers.

I ended up writing, nearly from scratch, a replacement IR receiver driver.

The peer device driver code was written by a developer in a different group in my same company. I finally got the P4 ACLs to read it after loudly escalating, over the objections of it's developer and his group manager. It was also complete shit. I cannot even begin to remember everything that was wrong with it, but I not only figured out may of the sources of my own pain, I also found a significant source of crash and lockup bugs that afflicted the main device.

I was not allowed to rewrite the peer code, as it was not in my remit. However, I was able to sneak in and check in a large number of asserts using the excuse that they were "inline documentation".

On, and the device driver for the CATF RF digital transceiver? The source code I got for the asking, without a fight? When I reviewed it, it was easy to understand, efficient, elegant, and as far as I could tell, bug free.

In the end, I made my part work. It just took over two months instead of the original guesstimate of less than two weeks. This caused a schedule slip in the release of the satellite box. Which would have been a more serious problem, except…

Except there was also major schedule slip for the main box. A significant reason for that slip was because the peer code that I had filled with asserts was now itself crashing with assertion failures instead of emitting garbage to crash my code.) I was lucky that I was not more officially "blamed" for that. The reason why I wasn't, was mainly because the people who understood what I did understood the problem, and the executives who didn't understand what the problem was were also too clueless to blame anyone, let alone me.

My lesson learned from this experience is: if someone is refusing to show the source to suspect driver code, citing trade secrets, confidentiality, secret sauce, partnership agreements, or similar excuses, it's not because they are protecting their magic. It's because they have screwed up, and they are trying to hide it.

A second rule of thumb I have is: source control systems that have complex read ACLs, e.g. don't allow any arbitrary developer to check out and review any arbitrary source code file, are expressions of moral failure.

On LinkedIn's claim of salting passwords

2012-06-13T22:49:00.001-07:00

In the wake of the leak of their password database, LinkedIn issued a blog post: An Update On Taking Steps To Protect Our Members, wherein they claim they had an existing project to salt the passwords, and that that "transition was completed prior to news of the password theft".

That cannot be true. The transition could not have been "completed". One cannot "transition" a bashed password to a salted properly hashed password. The original plaintext password is required to generate the salted and properly hashed data.

Best case, LinkedIn was in the process of slowly migrating accounts over, like so: When a user logs in, look in the new salted hash database. If there is no entry, then use the old unsalted hash to verify the user, then compute the new salted hash, store that, then delete the old unsalted one.

Doing it this way has the disadvantage that it doesn't protect the people who haven't yet logged in, and there are a LOT of LinkedIn users who log in to update their profile only occasionally, such as when they change jobs.

It does have the "advantage" that it's invisible, allowing a slow migration "on the down low".

However, I'm not hugely impressed.

Go get your whooping cough shot

2012-05-16T16:37:00.003-07:00

I just went on a half-hour binge reading about pertussis, as a result of reading that WA state has an epidemic outbreak that the public health agencies are struggling to stop.
Reading people's first hand accounts of having it will cross your eyes in empathized pain. One person described that she has given birth, has suffered a compound fracture of a leg, has passed a kidney stone, and has had whooping cough. Only one of them made her wish for death. It's been described as feeling like an asthma attack while someone is punching you in the ribs. Oh, and most all cough medicines do pretty much nothing for it.
You, reading this. You. Right now, pick up your phone, and call your doctor's clinic. Ask them if you're up to date, and if you're not, go get your freaking DTaP shot. You can get one at Walgreen's for less than the cost of a high-end Starbucks drink. You owe it to yourself, you owe it to public health, you owe it to your friends and coworkers, and you owe it to every pregnant woman, every newborn, and every immunocompromised person you share this biosphere with.

Is there a "C for JVM"?

2012-03-07T16:21:00.001-08:00

The accusation that C is "structured assembly language" has merit. It is not hard for a competent developer to map C to CISC assembly, running the compiler in his head. This is not a weakness, it is a strength, and is the reason why it remains a tool of choice when performance and size count.

As near as I can easily see, nobody has designed an equivalent language for the JVM. Something that maps the language constructs closely and clearly to the underlying semantics of the actual procedural execution model. With no "object code blowups", where doing some small change to the source causes a massive inflation of object code side.

Does such a language exist?

(BTW, the Java language itself is not it.)

Any random CS masters student can design a language that abstracts away an underlying reality. A few rare and gifted language designers can do it well. But it takes a rare genius to embrace and clarify an underlying reality, and make it approachable and useful.

On my way to LCA2012

2012-01-12T16:54:00.000-08:00

I'm starting the first leg of a literal `round-the-world business trip.

I'm flying from my home in Seattle, via LAX, to Melbourne Australia. There I will meet up with my friend Stewart Smith, the Director of Engineering at Percona. He is fellow survivor of MySQL & Sun, and a fellow contributor to Drizzle.

Other good friends of mine who are converging on Australia for LCA 2012 are Sarah Novotny, Monty Tayler, and Jacob Applebaum.

Why am I going to Australia? Geeks into open source who are "in the know" know the Linux.conf.au conference to be one of the best open source conferences in the world. This year it will be held in Ballarat, which is not far from Melbourne. This will be my 4th LCA, having attended past ones in Brisbane, Wellington, and Tazmania

At that conference, I will be speaking in the SysAdmin Miniconf, to demo OpenShift, Red Hat's cloud PaaS. (Sign up with the promo code LCA2012.)

This is only the first leg of this trip. After LCA, I will be heading to Bangalore India to present at JUDCon:India.

And after that, I will keep heading west to Europe, to present at FOSDEM in Brussels.

And then who knows where I will go next?

Thoughts on travelling and OkCupid matching

2012-01-03T19:38:00.000-08:00

I've had an OkCupid profile about 10 years, and have answered over 1200 questions in their database.

When I travel, I sometimes reset my location in OkC to be the city I'm in. It makes for an interesting view into the place that I'm at.

When I'm in Seattle, there are surprisingly large number of people who match me at 95% or above. When I am traveling, there are relatively few people who match that closely. Often, there are none. Even in very densely populated areas with an order of magnitude more people to match against.

I find that very interesting.

Did I end up living in Seattle because it's friendly to "people like me"? Or is it the other way around, and being around Seattle has made me more like the people here I match so closely with?

And then there are even more interesting questions one could ask and answer with the OkC database. Does this kind of "lots of close matches" effect happen in every city? How much dispersion is there in a given city? After all, even if there are many people I match closely with, there are a LOT more in Seattle that I don't match at all. How do other cities compare that way?

Hopefully the data geeks inside OkC will someday answer these questions.

Ideas for Foursquare

2011-12-22T10:52:00.000-08:00

I have a list of ideas for Foursquare:

One, they should open source their Android app, so people can improve it.

Two, when a mobile app checks in, it should also send the mac of the wifi it's connected to, the mac & signal strengths of all the other wifi's it sees, the cellphone tower id the phone is connected to, the signal strengths of all the other towers it sees.

Three, the app should be able to search for locations based on that data. Why wait for a GPS lock and/or a couple of round trips trying to turn that information into a latlon and THEN search for locations near that latlon. Cut out the middleman data.

Four, the app should keep a local cache associating wifis & tower ids to checkin locations. When my phone can see the local cafe's wifi access point blazing away, it should be able to instantly display the location I checked into the last time it saw that AP, without having to wait for a couple of network round trips and searches from the 4sq servers.

Five, the Android app should integrate with the Map app just like Google Latitude does. When I bring up Maps, there should be an overlay named 4sq that has pinpoints for all my friend's checkins in the last couple of hours.

Six, they should start contributing their mapping information to OSM. Or at least give the user the option to click a couple of checkboxes "Do you want to contribute your information to OSM?"

Seven, they need to work with organizations and with serious crowdsource volunteers better. For example, make it easy for an airport authority, a university, a mall, a public transit org, etc to create the canonical entries for airport gates and terminals and shops, for buildings and classrooms, for mall locations and shops, for bus and transit stops.

2011-12-12T18:01:00.001-08:00

I hate it when example or teaching code does something like:


import foo.bar.*

import foo.baz.*

And then starts just using the symbols defined in foo.bar and foo.baz willynilly. This is *stupid*. The point of example code is to teach me. Doing this in example code does not teach me which symbols are in which package, nor does it teach good form or proper idiom. The only thing it teaches is rage against who ever wrote the examples. Most all of the teaching docs and example code for Java are worst chronic offenders.

Best practice for "releasing" Open Source Software.

2011-11-14T17:36:00.001-08:00

Many projects practice "open source" via the "release the source" technique. Sometimes, in the process of being underwhelming, what is "released" is a bare tarball that lacks build instructions, and metadata such as change history, internal documentation, and bug and feature commentary.

This is not a "best practice".

A slightly nicer way to do it is to maintain a read-only repository, such as a public SVN or GIT server, and on occasion write out an approved and sanitized version of the software for the great unwashed to pull down.

This looks nicer than just bare tarball drops, but actually isn't any better.

The next step is "public development". This turns software development into a public performance. It turns out that the proverb "Sunlight makes the best disinfectant" is true for software quality as well as for politics. Such projects keep their real operational version control system world readable, and keep their bug tracker and development mailing lists and documentation fully public. Sites like GitHub and Launchpad make it trivial to do this.

There are some costs a project has to pay to make this work. They have to make sure that "tip always passes", e.g. that they have a good enough test suite, continuous integration system, and merge processes. But, consider: any project, open source or closed, that doesn't have these things is unlikely to be generating high quality work at all.

The next step after "public development" is open development. Such projects accept participation and contribution from "outsiders". When fully expressed, there is no such thing as an "outsider", everyone is a contributor.

Drizzle: Seeking tiny contributions, leading to big things

2011-11-07T17:24:00.000-08:00

The Drizzle project regularly gets people asking what they can do to get involved in the project.

One very easy way to brush up on your C++ skills and dip your toe into our open development process is to fix minor warnings.

We are very proud that Drizzle builds with zero warnings for with "gcc -Wall -Wextra".

But we can be even better! Our JenkinsCI system has a target that is even more picky, and also a target that runs cppcheck.

Go to one of those pages, pick a build log off the build history, find a warning that you think you can fix, and then ask us in the #drizzle channel on Freenode how to send your fix to us.

After you've done that a few times, you'll be ready to fix some low hanging fruit.

We've had people graduate from this process into becoming a Google Summer of Code student, and eventually having a full time paying job hacking on Drizzle and other open source software.

And it all starts with writing a simple warning fix.

Repost: Cloud computing is to computing, what the Internet is to telecommunications.

2011-10-31T12:48:00.000-07:00

(originally posted 2011-03-03)

Twenty years ago, if I wanted a reasonably fast data connection between a computer in Seattle and one in San Francisco, I had to call The Phone Company. Contracts would be negotiated and signed, Purchase Orders would be sent, Expensive Machines would be shipped, Work Orders would be generated and executed on, and well-trained well-paid Union Men would provision and test the link, which would be from a specific geographic point, to another specific geographic point. And I would be presented with a monthly Expensive Bill.

Ten years ago, that started to change, dramatically. All that complex hardware, cabling, installation, and cost-recovery got abstracted away by TCP/IP. Today, to get a much faster and much more flexible connection, I just click on a hyperlink, and or start a VPN, and I have a connection that lats a few seconds to a few hours, for just as long as I need it, and then the underlying real hardware forgets completely about me and my data, and gives some other random person the link they need.

Ten years ago, if I need to run "back office" software for an company, or if I wanted to run a web site, I would again have to do the whole Contracts, Purchase Order, Expensive Machine, Work Order and so forth. And again, there would be a big monthly bill, plus a big capex spend too.

About 5 years ago, that started to change, dramatically. All that "stuff" got abstracted away. With the type of a command, or the click of a UI button, machine instances spin up to do my work, and when I am done with them, the underlying real hardware forgets completely about me and my workload, and gives some other random person the work they need.

Cloud computing is to computing, what the Internet is to telecommunications.

Repost: I forget not everyone understands Open Source

2011-10-31T12:47:00.000-07:00

(originally posted 2011-03-13)

When you surrounded by something, that something eventually becomes obvious, then it becomes assumed knowledge, and then finally it becomes invisible.

I have been involved in Open Source since the late 1980s, and it is sometimes hard for me to remember that is what is obvious and assumed by someone in my position, is not obvious to everyone.

Early this week, I received an email from a group who are literally half a world away from me, physically and culturally. They use Eucalyptus, and had a question about it. Specifically, they wanted to know if they could modify it, if they could add modules and features that they needed.

Like I said, what seems obvious to me, is not obvious to everyone.

I wrote back to them, and explained that Eucalyptus is "Free Libre Open-Source". Most of it is licensed under the GPL Gnu Public License. That means that they can download it for free, can try it for free, run it on as many machines, with as many users, for as long as they want, for free. And yes, they can also look at the source code, modify and patch it, add more code, and write new modules for it, without having to get permission from anyone.

Closing The Eucalyptoid, and saving some posts

2011-10-31T12:45:00.000-07:00

For a while, I was writing blog posts for Eucalyptus Systems. Since I have moved on from them (and I wish them well, they are a great company with a great product), I've closed the personal/role blog I used for those posts. But two of those posts, I want to keep around, and so will repost them here.

2011-10-28T17:43:00.001-07:00

I think that this is the last straw, and I'm now really completely done with Perl as anymore more than a write-once short-script language.

CPAN and /usr/bin/cpan as installed by native package management apparently do not work out of the box on stock MacOS, on stock Solaris, on stock Illumos, on stock Ubuntu, on stock Debian, on stock Fedora, on stock RHEL, or on stock CentOS.

Perl is now worse than the various implementations of the JVM at assuming that an entire box is going to be turned into a "Perl machine" and that the admin of that machine has any interest in keeping track of all the ways that Perl is special, and in all the ways that Perl wants to make that machine "special".

I mostly gave up on Perl about 3 years ago, and about once a year I go back and give it other try, and each time the experience is worse.

Some good advice

2011-10-27T10:01:00.001-07:00

A few days ago I was interviewed by a Georgia Tech student about my job and career, and what advice I may have for students of computer science and programming.

Take on as little student loan debt as possible. And if someone will not pay you to get a post-grad degree, don't waste the debt and the time. Keep out of debt. You never want to feel stuck somewhere so to make rent and pay bills.
Learn to write. You learn to write by writing. Take writing classes, read about good writing, and practice writing. It doesn't matter what kind of job you get or life path you take, you need to know how to write.
Get involved in some open source projects, and make real contributions to them. The Google Summer of Code is a good thing to get involved in. A portfolio of demonstrated contributions to open source projects is more impressive than a GPA on a new resume.
Get involved. Find your local makerspaces, hackerspaces, and barcamps. Volunteer and participate. Go to Ignite. Speak at Ignite.
Always be fluent in at least two programming languages, and practice learning new ones. Languages and frameworks come and go, learning new ones is forever.
When getting a job, beware of the non-compete and copyright assignment clauses in the employment contract. Push back on them. If they are non-negotiable, too onerous, are enforceable, beware and be careful of taking that job. Keep your list of "personal and outside projects" ready to attach as an appendix.

Wanted: A Netflix for physical books

2011-09-12T11:13:00.000-07:00

The tech rumor mill has been churning for a few days about "Amazon launching a Netflix for books". While a neat concept, it is for Kindle books only. This plays to Amazon's strengths, but is not what I have been wanting for years.

For years I have been wishing for a "Netflix for Books", for physical books.

Here is how I envision it working:

A large municipal library, or a consortium of them working together, set up a site and paid service very similar to Netflix, only for books.

I, as a user, select how many books at a time I want to rent. There would be different monthly payment levels, just like Netflix.

Books in my queue get checked out from the library or via inter library loan. They get mailed to me, along with a return mailer. Postage would be USPS Book Rate, of course. Unless I am willing to pay extra for Priority or Express mail.

I read the book, keep it for as long as I want (maybe with a one year maximum), and then either return it with the return mailer or by dropping it off at the library like a regular book.

I could keep using my muni library "for free", or use this service for the convenience factor. It could even be a source of much needed funding for the amazing public library system that we all too often take for granted, and do not use enough.

I want this.

Google Plus, minus minus

2011-08-11T21:51:00.000-07:00

I've started telling friends and family who have been asking me about Google Plus to not bother, until G fixes their broken "real name" policy. Plus is turning into a place where it's mostly white techy males talking about white techy male topics. And speaking as a white techy male, that's pretty boring.

Regarding Autotools, and attempts to replace it

2011-07-21T12:32:00.000-07:00

In the world of software development on Linux, especially in the open source world, there is a set of packages called "autotools", which abstracts and manages portability between compilers and other aspects of the software build environment. Nobody likes it. In fact, most people who use it hate it. There are many attempts to replace it.

And the problems with autotools are getting worse, because it itself was never designed to have cleanly portable control files between versions. Back when most everyone just FTPed down a tarball, and then ran the prebuilt ./configure, it worked pretty well. Now that people pull a projects raw repo over SVN, BZR, or GIT, and then have to run liptoolize, aclocal, automake, autoconf, etc themselves, and who knows what version of autotools is locally installed, all hell breaks loose.

With respect to all the autotools replacements, such as cmake, Ant, etc, and all the other ones mentioned in Eric Raymond's recent blog post: they ALL are some multiple combination of horribly slow, enforce their own special "one true way" of laying out source trees, are specific to what languages they will deign to handle, have abysmally bad error messages when there is a problem (and being worse than autotools in this respect is an amazing achievement), require the installation of a JVM and a huge pile of buggy poorly documented class files, require the installation of a huge pile of buggy poorly documented Python modules, require the installation of a huge pile of buggy poorly documented Perl modules, cannot intelligently detect and handle optional build dependencies, cannot cross compile, cannot build out of a read-only source tree, cannot build out of tree, and/or cannot build shared object files.

I wish to gods and monsters that this was not true, but it is. And until the writers of the competing build chain systems understand why all this stuff is important, and are willing to support it, autotools will stick around, and people will continue to use it.

This is not to say that it cannot be used better. One of Monty Taylor's herculian tasks the Drizzle project has been pandora build, which is a refactoring and rewrite to the years of cargo-cult accumulated cruft that has infested most autotools based open source projects.

SQL identifier case, Unicode case, and boost::to_upper, all in a fight that nobody will win

2011-07-12T14:14:00.000-07:00

The SQL standard says that table and row names are case insensitive. Drizzle table names are in Unicode with UTF8 encoding. boost::to_upper et al mangles UTF8 case. Down this path there is going to be a lot of pain. My own inclination is to tell the SQL Standard to realize that its 2011, no longer 1961, and break all the apps that are lazy about identifier case, but other people will probably disagree.

Multi Master Replication is back in Drizzle

2011-07-06T19:51:00.000-07:00

This last March, David Shrewsbury put together a basic implementation of multi-master replication into DrizzleDB link. We were able to actually merge it with the mainline trunk, but then had to remove it until we had refactored and fixed more things. Today, we were able to put it back in.

It's worth reading Shrew's original blog posts, and then trying it out.