Platte daddy

Removing hard-coded cookies from ExpressionEngine

2011-12-07T11:15:00.000-05:00

A client is using ExpressionEngine 2 as just a CMS and not using its membership features, and we're doing everything we can to improve performance, both back-end and front-end.

On the front-end side, the cookie setting takes up some of the precious bytes that get transferred before the browser can start downloading assets.

I have turned off everything I can turn off that adds cookies to ExpressionEngine's output, but there were still two cookies hard-coded into the output even for logged-off users: exp_tracker and exp_last_activity.

The code that does this is in expressionengine/libraries/Session.php:

$this->EE->functions->set_cookie('tracker', serialize($tracker), '0');
$this->EE->functions->set_cookie('last_activity', $this->EE->localize->now, $expire);

This was a bit tricky to find, and I didn't see answers on Google, so there it is for posterity.

Note that EllisLab staff says making this change "can affect your ability to upgrade, and your ability to receive support." They also point out that you would need to re-apply such changes after an upgrade.

Web performance

2011-10-05T09:51:00.001-04:00

Several of my clients have asked me to do performance tuning for their websites. I find it fascinating work, and rewarding to deliver improvements that help machines serve humans rather than the other way around.

I'll try to post nuggets of value as I come across them, regardless of whether they seem basic or advanced to me.

A great one to start with this first day is WebPagetest. It loads a page with an empty cache, then a filled cache in one of many browsers and produces amazingly detailed reports.

I have been doing a lot of tuning on one of my clients' sites, but mostly through my own Mac Chrome on a pretty fast connection. I had WebPagetest run the site on IE8 on a slower connection and got quite an education.

OO programming in JavaScript, neatly summarized

2010-04-12T08:56:00.002-04:00

This article from Douglas Crockford outlines the basics and feel of programming with objects in JavaScript very succinctly.

On Documents vs. Streams, by Bryce Harrington

2010-04-07T10:13:00.002-04:00

This article was written by Bryce Harrington, and gave me a fresh perspective on interfaces when it was published and discussed on InfoQ. It's currently available only through Google's cache, so I'm republishing it here to preserve it.

The other day I was trying to figure out what was making my desktop computer run so slow. This was pretty unusual on Linux. Way back when I used to use Windows, I'd encounter such things due to running too many programs, so of course I looked at what was running.

But these days I don't really run that many intensive applications. I hardly ever use spreadsheets, presentation tools, word processors, or so on like I used to. Really the only things visible in my task bar were Firefox and Terminal. Killing off Firefox made no difference. And even more oddly, rebooting had no effect - within minutes my load average was up in the 20-30's.

Of course, the culprit was that I had a bunch of automatic cron jobs scheduled to occur at the same time. Reordering the timings got the system back on track.

But this led to an epiphany moment.

This was my *desktop* machine. The last time I'd seen this cron job I/O blocking madness was on servers. What was I doing with so many cron jobs on a *desktop*??

I looked at the individual cron jobs, and found that each and every one of them had something to do with my job. Disabling any one of them would reduce my work productivity. I could no longer do without one of these silly cron jobs than I could have done without Microsoft Excel a decade ago.

And then, as I looked around my desktop at the other things running on it - internet radio, IRC chat, mutt - that I noticed that I hardly ever use traditional "document-centric" applications. Everything running had something to do with dealing with _streams_ of information.

The realization sank in at that point, that our computer UI paradigm really stinks for what we actually use our computers for.

The prevailing UI paradigm today is built around the notion of document authoring. It expects that the main thing you do is create spreadsheets, word documents, presentations, and so on. There is a task bar to remind you of what documents you're editing, there is cross-application cut and paste so you can put pieces of one document into another. You can place documents on your desktop surface itself, so you can organize your work. You can define which applications to use for which types of docs. You can set up a default printer to put your documents to hard copy. You can set up system-wide fonts to use in documents. You can put icons to apps and even documents onto your panel. And on and on.

Occasionally, some of that is actually useful to me. But most of the time, for most of the work I do, it's all irrelevant. To pick one example, I used absolutely none of that in order to write this blog post.

Really, what I mostly do today is stream management. And I suspect this is true for the vast majority of people. I don't deal with writing documents, but with changes to documents. I put comments onto things. I slap patches onto things. I tweak the states of things. Once in a rare while I may author a completely new thingee, but even there I usually end up working with it as a stream of changes that I build up over time (and usually in collaboration with a few other people who stream changes to me).

Thinking about user interfaces this way, it occurs to me that there are a LOT of different kinds of streams. Streams of emails. Streams of spam to take *out* of my email stream. Streaming radio broadcasts. Streams of bug reports. Streams of software updates to apply. Streams of chatter on IRC. Streams of youtube video URL's from friends and family to check out. Streams of updates to my weather applet. Even my todo list is more like a stream of things coming in and going out, than like a static document I can print and save and be done with.

Each one of those silly cron jobs bogging down my computer were critically important for me, because THEY were the tools I used for helping me stay atop all these streams. They summarized my bug streams in various ways. They filtered my email streams into more organized sub-streams. They flagged tasks (and took care of certain tasks for me). They took care of updating the tools I used day to day.

It's weird that for all the document tools at my fingertips, it's a stodgy old *server* tool that is helps my productivity the most. I wonder how non-technical people not knowledgeable about crontab and scripting deal with such things?

This all got me to a key question... Since the purpose of our desktop UI is to make our work easier and more efficient, then if today's knowledge workers are, like me, more stream-oriented than document-oriented, then doesn't it stand to reason that we ought to re-think our UI design to optimize it for making stream management easier and more efficient? How would such optimization be done? How would such a UI look and feel? What kinds of toolkits would be needed?

Merlin Mann's How to Succeed in Business

2009-03-31T13:56:00.002-04:00

From John Gruber and Merlin Mann’s talk on making money by being authentically enthusiastic:

Give away more stuff than you think you should, and make it easy for people to get.
Focus on diverse secondary revenue streams, and always have your eyes open for new and replacement ones.
Don’t do stuff that seems profitable, but potentially messes up the reason people like you.

should be || should_not be

2009-03-20T12:15:00.002-04:00

Stephen Eley has done something very clever on the rSpec list, it begins thus:

should be || should_not be: that is the expectation:
Whether 'tis nobler in the parser to interpret
The outputs and side effects of outrageous duck typing,
Or to inherit against a sea of matchers
And by declaration extend them?

And on from there, to a satisfying conclusion.

javascript_named_routes

2009-03-20T10:18:00.002-04:00

Fred Polgardy has just released a tasty bit of code that will allow JavaScript to access Rails named routes in a very straightforward way.

Check out his javascript_named_routes Rails plugin.

How much process, and why?

2008-12-10T14:05:00.002-05:00

The best write-up in my memory of how agile processes fit into projects hackers build.

Don't steal from me, Senator Obama

2008-09-25T11:27:00.002-04:00

I signed a petition asking my legislators not to approve a bailout. I got a response from Senator Obama similar to the one quoted elsewhere.

Dear Senator Obama:

I contacted you earlier writing against the bailout, and received your reply.

Given your commitment to a bailout, I'm glad you're looking for ways to improve matters. But you are planning to remove thousands of dollars from my pocket, either as taxes or as devaluation of the dollar, to pay for mistakes made by private citizens.

You do not have my consent to do this. It is stealing from me.

Merely waving your hands and saying "liquidity" and "worsening downturn" is not enough explanation for why you're planning to take money from me. You owe us better than that.

Furthermore, you point to a need to rescue the economy. Have you considered the effects of inflation on the economy? Talking about bailing out mortgages without any discussion of the inflation this will create makes it look like you are only interested in protecting the rich. I'm sure you don't want that.

Another topic that isn't being discussed: is it the right answer to go on creating a socialist regime in the US? Buying up companies and personal loans? This isn't the America I grew up in. It sounds more like China. Are you for or against this approach?

The closest you came to answering my call to vote against a bailout plan was "I agree that urgent and decisive action is necessary during this time of economic turmoil". Please come back with a better answer for your actions than that.

Eating well, or, please hold the sugar

2008-09-16T11:40:00.003-04:00

Two members of our family were suffering new tooth-decay problems, and my wife Nicole had been reading about a dentist who researched the profound difference in health between people following traditional diets versus Western diets.

Nicole decided to take the plunge and learn how to feed us a traditional diet. We have removed refined flour and sugar from our diet, and added lots of whole, organic, delicious vegetables, milk products (unpasteurized, like God—and Louis Pasteur!—intended), honey, and meat. There are lots of changes she's making as she learns, which I'll also share over time, but she's posting interesting articles on her traditional-food blog.

This is incredible food, and our health is improving dramatically as we restore the levels of nutrition available to our bodies from the red zone to which modern Americans are accustomed, into normal ranges so our bodies have something to work with. Dental health, skin health, energy, mental function, sleep patterns, emotional health, you name it...every one of us is doing well and still improving rapidly.

I was generously offered some chai at work this morning. I took a few ounces. It was delicious, but I noticed an immediate and all-too-familiar feeling of weakening. I used to regard this as normal, but now I realize that I would feel this whenever I ate or drank anything with refined flour or sugar. As long as I stay clear of sugar and flour, I don't crave it, and this kind of experience is quite the reinforcement of my desire to stay away.

So my diet is very different now. This may sound inconvenient, and sometimes it is. But don't cry for me—I'm eating delicious homemade butter, homemade ice cream (sweetened with dehydrated cane juice and/or honey), rich Indian dishes...

Pasteurization was developed for wine. On hearing that pasteurization was to be used on milk, Pasteur is reported to have said, "What are they doing to my wonderful food?"

Back to article

Images: thrashing vs. process

2008-09-15T19:22:00.005-04:00

Steve McConnell illustrated an article about software process with these images. I'm interested in this effort—what is a success, and what could be improved?

How much to spend on security

2008-09-15T09:14:00.002-04:00

I love it when someone can give me a sound way to quantify wiggly and abstract things. Bruce Schneier wrote this about Security ROI:

…a company should implement only security countermeasures that affect its bottom line positively. It shouldn't spend more on a security problem than the problem is worth. Conversely, it shouldn't ignore problems that are costing it money when there are cheaper mitigation alternatives. A smart company needs to approach security as it would any other business decision: costs versus benefits.
The classic methodology is called annualized loss expectancy (ALE), and it's straightforward. Calculate the cost of a security incident in both tangibles like time and money, and intangibles like reputation and competitive advantage. Multiply that by the chance the incident will occur in a year. That tells you how much you should spend to mitigate the risk. So, for example, if your store has a 10 percent chance of getting robbed and the cost of being robbed is $10,000, then you should spend $1,000 a year on security. Spend more than that, and you're wasting money. Spend less than that, and you're also wasting money.
Of course, that $1,000 has to reduce the chance of being robbed to zero in order to be cost-effective. If a security measure cuts the chance of robbery by 40 percent -- to 6 percent a year -- then you should spend no more than $400 on it. If another security measure reduces it by 80 percent, it's worth $800. And if two security measures both reduce the chance of being robbed by 50 percent and one costs $300 and the other $700, the first one is worth it and the second isn't.

Taking back my corner of the web

2008-09-04T20:12:00.002-04:00

I've decided to abandon the idea of focusing this blog on a narrow set of topics. I have very wide-ranging interests and strengths, and I don't think anybody's benefiting from the slow trickle of weak ideas that's left when I restrict this blog to agile + Ruby. I may even wind up posting more frequently on those topics.

You may like this, you might not, but I wanted to give any subscribers fair warning.

Quick letter about electronic voting

2008-01-20T22:42:00.000-05:00

Electronic voting has been thoroughly shown to be unreliable and manipulatable. As a professional software developer, I can attest to the difficulty of creating systems that cannot be tampered with. I can also add that many reports I have read of defects in voting machines have suggested a lack of competence in a task that would be difficult for the very best developers.

On the other hand, voting techniques that leave a paper trail give us a backup plan when machines fail us. It is long past time to add laws that encourage and aid officials in implementing paper trails for our elections. Inaction makes our elections look illegitimate, even if there is indeed no manipulation at all.

Managing to capacity

2008-01-17T15:19:00.000-05:00

Jason Yip says managing to capacity is a prerequisite to other work on process:

When going down the path of Agile/Lean, there are two main topics:
Learning to manage to capacity
Learning how to improve process capability
If you can't accept the first point, the second point won't make any difference.... in fact, because of overburdening, you'll make the second point harder.

assert_efficient_sql

2008-01-16T16:05:00.000-05:00

The ever-brilliant Phlip Plumlee with a way to lint SQL "pessimizations":

assert_efficient_sql - O'Reilly Ruby

To ensure your test cases call efficient MySQL…
def test_my_case
 assert_efficient_sql do

   # just wrap them in this block!

 end
end
The assertion intercepts and copies out your MySQL SELECT statements, then calls EXPLAIN on each one, and inspects the results for common problems.
The goal is test cases that resist database pessimization, even as you change your data relations, to add new features. If you run your tests after every few changes, you can easily detect which change broke your database’s indices and relations.
This article is a reference for this assertion’s options.

Phlip is soliciting help with implementing this feature across more database engines.

Four busted laptops, one measly task

2008-01-10T01:26:00.000-05:00

Good friends of ours from out of state are staying with us tonight. They're fellow geeks, and he's a fellow developer.

Our task: copy 1.5 MB from a CD to his wife's MacBook.

Our challenge: his wife's MacBook does not currently read optical disks.

Tools available to solve the problem: a PC laptop with a failing hard drive, my work MacBook Pro that also has issues with optical disks, our family iBook that works but has no screen backlighting at present, and the destination MacBook, which, again, isn't reading optical disks.

Solution: I fortunately had SSH enabled on the iBook before its backlight went out. I logged into that, and inserted the CD into the iBook. Checking in /Volumes, I found that the CD wouldn't mount until I logged in to the GUI. I'm sure there are dozens of ways I could have done that from the command line, but it went along with the silly mood of the whole thing, so I went ahead and logged in blind.

Sure enough, during the GUI login process the CD appears in /Volumes. We make a tar file and my friend copies it down.

Easy peasy.

PS: Oh, and the eject key on the iBook didn't cause ejection. Turns out a quick drutil eject does fine.

Track down warnings in Ruby test runs

2008-01-04T11:20:00.000-05:00

Got warning or other spew in your Ruby test runs? To track down the source you can have a setup method output the test name:

p name

This makes it very easy to track down the source of the unwanted output.

This works because each test run is its own instance of your TestCase subclass.

Daniel J. Bernstein advocates for Ruby?

2007-12-03T10:11:00.000-05:00

It's not your typical advocacy piece for Ruby (in fact it's not about Ruby at all), but Daniel J. Bernstein has written a paper reflecting on the success of his qmail mail server, which has a security guarantee (in the form of a $500 bounty) that has gone unclaimed since March 1997, and which runs several prominent mail houses.

He notes several aspects of his development process and architectural decisions that contributed to qmail's success. Interestingly, many of his comments relate to his use of concepts in his C code that are usually associated with higher-level languages. Here he complains about C's dangerous handling of integers:

Another surprise for the programmer is that y can be much smaller than x after y = x + 1. This happens if x is the largest representable integer, typically 2^31 − 1; y will then be the smallest representable integer, typically −2^31.
...
Most programming environments are meta-engineered to make typical software easier to write. They should instead be meta-engineered to make incorrect software harder to write. An operation that is not exactly what I normally want should take more work to express than an operation that is exactly what I normally want.

Ruby does get many details like this right, and good object-oriented practices can help too. For example, I kept thinking of the Tiny Types pattern while reading this.

I'm reading this a bit at a time and still parsing what Bernstein's ideas mean to me, but the segments on spending some performance dollars to buy quality abstractions were refreshing to see from a developer who's met with measurable success.

Spying on assert_select

2007-10-25T15:35:00.000-04:00

When testing your views with assert_select, it's nice to get information on what response text failed to match, but it's too much to just throw @response.body into the test failure message:

assert_select('div#whatever-id p.classname', 'small stuff', @response.body)

Instead you can use nested assert_selects and the elements they yield:

assert_select('div#whatever-id') do |whateverId|
 assert_select('p.classname', 'small stuff', whateverId)
end

That will dump out just div#whatever-id and its contents into your test failure message.

A solution for developing remotely: Vim's netrw

2007-07-12T16:01:00.000-04:00

I'm doing some work that has to be developed remotely, so I wanted a way to edit locally instead of doing everything in a terminal window. I tried using MacFUSE and sshfs to make the remote directory appear on my own desktop, and it worked after I figured out the options I needed, but opening files in gvim was very slow.

So I took my friend Ed Marshall's advice and tried Vim 7's netrw support. Bingo! It works via scp (which is (almost?) always available when there's ssh access) and is dreamy.

To edit a file /path/to/file.rb owned by a user joe on a machine example.com:

vim scp://joe@example.com/path/to/file.rb

Brief pause while the file gets retrieved, but then it's copied to your /tmp directory and everything's full speed. The only downside I ran into was that inter-file navigation wasn't as good. To get a file browser for the files in ~/path/to, I could then do:

:sp %:h

or to edit ~/path/other_file.rb (again from a window opened on ~/path/to/file.rb):

:sp %:h:h/other_file.rb

:sp means "split window". % means "the current file". :h means "the parent directory of the preceding file path", and can be repeated.

I'd love to hear more about navigating in Vim + netrw—I haven't even gotten my feet wet yet. I just wanted to share the nice experience I had right out of the box.

Hidden in plain sight: script/runner

2007-07-11T17:55:00.000-04:00

I am loving the opportunity to work with my co-workers in the home office. We have "geek-out" lunches in the office once a week or so, and invariably all learn something.

So today I learned about script/runner, a file that's in every Rails app that I never looked into for some reason. Turns out I wish I had. It's a replacement for calling the Ruby interpreter that gives your script your app's Rails environment preloaded, so you can just write Ruby code without having to monkey around with loading Rails yourself. Just call something like

script/runner my_file.rb

script/runner "puts 'My Ruby code here'; puts MyModel.count".

What are your favorite "hidden-before-your-eyes" features of Rails?

Welcome Dave Astels!

2007-07-10T14:08:00.000-04:00

I'm excited to welcome Dave Astels to the Obtiva team. Dave is a very active member of the Agile community and has shared some ideas over the years that I have found very valuable for better understanding TDD. It's an honor to have him working with us.

Obtiva is shaping up to be quite a strong software house with a promising future.

Land rush

2007-06-25T07:38:00.000-04:00

I have a workplace running gag with a friend at a client wherein when we encounter a Rails misfeature (especially the ones that have been solved carefully and elegantly in Java or elsewhere), we rip on Rails mercilessly to make ourselves feel better. My friend came up with this one almost unprovoked, I thought it eminently bloggable:

"It's like the land rush. We are all so excited about the free land that we forgot about all the bears and Indians waiting to kill us."

How to stub out Rails helper methods with Mocha?

2007-06-18T17:31:00.000-04:00

UPDATE The code given doesn't actually work. I'd love to hear if anyone knows of code that will do this. I hacked around the problem a very ugly way.

Rails helper methods are defined in modules and mixed into objects...of what class? Turns out it's anonymous copies of ActionView::Base. How does one mock or stub methods on such a beast? Here's a solution I found:


class MyController
  def rescue_action(e) raise e end
  attr_reader :template
end

class MyControllerTest < Test::Unit::TestCase
  def setup
    @controller = ...
  end

  def test_should_stub_helper_method
    @controller.template.expects(:random_helper_method).returns('')
    ...
  end
end

Thanks to Matthew Dieter for posting about this. I had to add the attr_reader line to get this to work.