After switching to Apache we needed a way (previously accomplished with PerlEx from ActiveState) to run certain scripts persistently. I did some research and discovered that using mod_perl in win32 was feasible and you can indeed turn it on for parts of your site. Yet again, all was well with the world.

Unfortunately as time passed and we started using a deeper stack of Perl (we originally were just using DBI, the Perl database layer, and sometimes using Template::Toolkit, one of the most major Perl templating systems,) we started seeing Apache crashing or leaking memory. Unlike IIS crashes the cause and time till crash was unpredictable, but after some work the issue was found and fixed.

During this time I and all but one of my coworkers switched to the most excellent Strawberry Perl, the Windows Perl one might say. Logging in to the servers to install packages with PPM quickly soured for me and I spent probably 3 days worth of my time (half of which was unpaid!) trying to find a way to use Strawberry persistently. If we could use Strawberry on the servers installing dependencies would boil down to

But I couldn’t seem to get mod_perl to build, mod_fcgid and mod_fastcgi were both unworkable (for me anyway,) and I couldn’t get lighttpd + FastCGI to work. So I gave up on that endeavor.

Nearly four months have passed since the crashing Apache issue was originally solved and just days ago we deployed our first Catalyst project (we have two more in the pipeline now!) We deployed onto mod_perl and Apache on Windows. I would never recommend deploying onto Windows, but I also realize that there are business reasons to do so and sometimes it’s just what you have to do.

And all was well with the world…for two page requests. It turns out that somewhere in our stack of Perl, Apache, mod_perl, and Windows there was an issue that made the server consistently crash after nearly every other request. I did some research, and even built up a replica of our deploy on my machine (linux) to see if the issue was generic mod_perl. If it were a problem in Linux it would be much easier to get free help from the community, but alas, it ran perfectly on my machine.

While I was driving to a friend’s recently I had a thought; why not just use the Catalyst development server or some other Perl based server and just proxy to it with Apache? Heck, it’s actually very similar to one of the recommended ways to deploy Catalyst in the Catalyst book.

Before I get into the details of this I need to point out that we are not using HTTP::Prefork, thanks to Windows. If you have a large site you really should not use Windows, for numerous reasons. That was the conclusion that my boss and I came to anyway.

First off, here is the Apache configuration we ended up with:

So basically all we do in this configuration is have Apache serve the static files and then proxy the requests to a couple of catalyst dev servers. I used Srvany.exe and a couple of .bat files to start the catalyst dev servers. It works much better than using mod_perl, and each server sits at about 90M a piece. If we ended up getting a huge site and for some strange reason needed to keep our outfacing server windows, we could actually serve the catalyst parts on a linux server and have apache proxy to those, so it scales very nicely.

Anyway, here’s to the next 6 months of serving!

I am proud to announce the next bit of mstware! Log::Contextual is a small module for making your life easier when it comes to logging. Instead of bringing yet another logging infrastructure into the mix (see Log::Log4perl and Log::Dispatch), this module is a thin wrapper around any logging system you choose to use. (Note: we are working with authors of major logging packages to work seamlessly with L::C, but at the time of writing most need some form of adapter.)

There are a few major features worth noting. First off, ridiculously convenient interface. Once you’ve set up your logger (presumably in the startup of your app or whatever) all your logging code will look like the following:

Another great thing is that, like Devel::Dwarn, all of the logging functions are identity functions; that is, they return their arguments. That means you can do cool things like the following:

Of course, in Perl you may be passing around complex references and the above will get cumbersome fast, so we added shortcuts specifically for logging out data structures:

The automatic stringification is done with Data::Dumper::Concise, so you will get reasonably indented output for free.

In a separate package we are going to provide a module to basically turn the logging functions into no-ops at compile time, thus giving you the ability to have your code run just as fast if it never had the logging functions in in the first place. I’ll post more on that once it’s released.

So what are you waiting for? Go log stuff!

The ResultSet::Random helper had the wrong function used for MySQL. That was fixed thanks to an RT from pldoh.

get_namespace_parts from the util package was unnecessarily strict. Thanks to melo for the prodding to do that.

I refactored some of the code in core DBIx::Class so that I can more easily detect is_numeric with Row::NumifyGet, instead of requiring the user to specify it. Normally DBIx::Class autodetects it based on column type, but that code wasn’t quite generic enough until now. Nice!

And then the most exciting bit is a new helper entirely for the suite: Row::ToJSON. Basically I was sick of doing this:

Of course that can be shortened to:

But I still have to make that stupid columns list! This shiny new helper makes a TO_JSON method that will simply include all of your columns except for the “heavy” ones like TEXT, NTEXT, or BLOB. Of course you can have finer-grained control than that by explicitly saying to include (or not) a column in it’s configuration. See the docs for all the nitty gritty details.

First off, you might want to check out the wikipedia article about cryptographic hash functions. The gist of it though is this: a password stored in plain text is obviously compromised if the passwords file gets into the hands of evildoers. You can “hash” the passwords and they are now harder for the attackers to transform into plain-text. If your password is good it is nearly impossible, but basically what can happen is that the attacker uses the algorithm to generate hashes for every word in the dictionary or whatever and now they basically can crack all the basic passwords.

You can take it a step further and “salt” your passwords (wikipedia salt article.) A simple way of doing that is just to concatenate some string onto the end of all of your passwords. This will make dictionary attempts useless unless they know your salt. Typically when using a salt the salt is kept secret.

And then you can have a unique salt per password. Imagine a scheme where the salt is $username$id. It would require the attackers to basically generate a dictionary per user!

The scheme we’ve settled on uses Eksblowfish. The DBIC Component for it actually uses a 16 character randomly generated salt for every password. Nice!

Ok, so how does one apply such sweet, sweet code? First, (always) set up your model. This is a slightly trimmed version of ours:

Easy peasy! The encode_check_method option for password basically puts a method in your result class that you can call with a plaintext password and it returns true or false if the password is legitimate. The nice thing about that is that if you decide to switch to some other kind of hashing, your controller stays the same. Model code for the win!

Next up, the Catalyst configuration. This was what took me a while to find, but thanks to mst I finally found it yesterday. The package we use for auth is the same one everyone uses in Cat: Catalyst::Plugin::Authentication. The docs that I was looking for specifically were the ones for Catalyst::Authentication::Credential::Password. So after reading those docs, the following is the catalyst config snippet one would use for these nice passwords:

Note: the password_type of self_check is what tells the controller to just call $result->check_password($plaintext).

So there you have it. That’s all the code you need for secure passwords with Catalyst. If you make a new project and your users passwords get compromised it is your fault.

Have a nice day

It wasn’t until recently that I realized just how much time I spent working on grids and their related records (representation of the rows of a grid.) Although even if I’d known just how much I do this at the beginning of the project, I certainly didn’t know DBIx::Class as much as I do now, in addition to the other 4 non-core modules that I’ll mention.

Because I’ve been working all these super long hours (12-14 a day), often after a frustrating day I’ll try to code something more relaxing and rewarding at home or at work but not during work. So today I decided to finally take the plunge and do what I’ve been pondering for a few months and write some code to generate Ext scaffolding for me. So I’m going to walk you through the script that I wrote (and will probably work on more as time goes by.)

This isn’t a full module, not even in my repo yet, so it’s all just in one file. Here’s the boilerplate intro:

That’s pretty basic. We use a bunch of modules that I’ve used in at least 1 other project before and was happy with.

That’s the code to parse any kind of Catalyst config file and the grab a new schema based on it. It’s a ton of biolerplate but I live with it.

So next up is a basic inflection function that gives us all the different forms of a word we might need. It starts with either “single_foo” or “SingleBar” and gives us six variations based on that:

So that uses String::CamelCase and Lingua::EN::Inflect combined with join and split mostly.

We’re *almost* ready to generate a record. But first we need to define a mapping from the data type in the database to the data type that Ext uses:

Ok, now let’s look at the code to generate an Ext.record:

So what’s going on here is that we get the source from the schema. The source could be considered something like a table definition, although it can also point at a view or whatever too. Then we start generating the string representing our record, and then we use a join/gather combo to get the column data the way we want it.

We could certainly just use a more complex map instead of the for+gather that we have, but I personally feel than any map where you *must* use the block form is cumbersome. So we join together all of the strings that gather took, and then append the end of the definition, and voilà, we have a record!

Here’s example output on our Customer source:

If you’re still following along you should be fine with the next, much more impressive functionality.

I’d like to have the script do as much as possible to speed up my work, so let’s see how far we can take this. First, I made a mapping from type to renderer, so that things that are datetimes use our custom datetime renderer, same with booleans:

Next, I wrote a function that would try it’s best to guess how wide a column should be based on it’s header and the average width of the strings inside of it:

Now finally we have the grid function:

We see nearly the same structure here that we did with the record, except much more intricate. Notice the code to generate the width uses the magic constant 7. As I played with this I found that 7 seemed to work for the font that Ext uses by default (Arial?) Optimally I would actually use some kind of metrics package to ask it for the width of all of the strings that I generated in the function above and average that, instead of averaging character lengths. But this seems to work really, really well, so the ROI is pretty good.

Also note the code to pick and insert the renderer. It’s not complex or anything, but it yields very convenient results. Here’s the output of running this one on the same source we used for the record:

Note that there are probably some things that we could do a little better. For instance, we should probably check for text fields and not include them in the grid. We could also be clever and look for anything ending with _id and have it hidden by default. But in general, the above is very nice results. At the very least you are probably going to need to reorder the columns (although you could give some kind of grid_order_id to the column definition in the Schema::Result code, but I don’t think that’s worth it.)

So I hope someone else enjoyed reading this as much as I did writing it. After I’ve exhausted all of the little Ext things I can use this for (form, etc) I will probably look into messing with DBICx::AutoDoc to add in Dia outputs.

At $work we do as much “view” type code as we can in JavaScript with the ExtJS framework. I have personally found it to be a great framework to work with, although often it is lacking in the non-UI department. One thing that at first I really liked about Ext was their Template and XTemplate classes. But as time went on I got more and more annoyed with those modules.

I’ve always thought that Template-Toolkit was a really nice templating library to work with. I hate templating html because of all the weird little gotchas having to do with CSS and whatnot, but doing almost all that kind of work in JavaScript I have started missing Template-Toolkit. At some point I heard about Template::Tiny and this past Saturday I thought, “that’s like, 160 LOC…I could probably port that to javascript!”

So I did! It’s not really done yet, but the current code is at github. I need to finish porting the test suite from Perl 5 to JavaScript so that I can ensure correctness (I am certain I screwed up some stuff.)

Alias mentioned in his post that I use XRegExp (600 LOC) to help out with the Regular Expression support in JavaScript for this module. I actually wasn’t going to, since the only thing I needed it for was the /x flag, or to be more clear telling the parser to ignore whitespace, but I want to keep Template.Tiny in sync with the Perl version, and I really don’t want to strip out all the whitespace by hand. If someone takes issue with the dep they can fork away

So once it is fully ported I fully intend to use it entirely from now on instead of the Ext templating. But take note, Ext templates and Template.Tinies (what?) really solve different problems and have different goals. The following is a list of Pros and Cons of each:

Ext.(X)Template

Pros:

• Can be compiled to JavaScript, for speed
• Allows complex expressions in “if” blocks (see Example 1)
• Interesting “topicalizing” feature (Example 2)
• Crashes when you leave out a variable (no mystery as to why a field is blank) (Example 3)
• Can execute arbitrary JavaScript code in a template for complex stuff (Example 4)
• Neat automatic “current item” style variables when you are iterating over an array (Example 5)
• Basic Math Support
• Ability to call functions associated with Template object
• Very cool builtin Renderer support (Example 6)

Cons:

• No else if. If you want something like that you must do if !expr. Lame.
• Crashes when you leave out a variable (mostly that’s just annoying)
• This is almost entirely subjective, but having xml as a templating thing is kinda gross.
• Not really open source, so I can’t use it in personal projects and have people use my code in a corporate setting

Example 1:

Example 2:

Example 3:

Example 4:

Example 5:

Example 6:

Template.Tiny

Pros:

• Has IF/ELSE (Example 7)
• Doesn’t crash on undefined fields
• Nicely Licenced (Perl License)

Cons:

• No complex expressions, math, or external javascript support
• No topicalizing
• Doesn’t crash on undefined fields (could be nice for debugging)
• Probably slower (haven’t checked that yet)

Example 7:

I am planning on making a wrapper for TT for our stuff that will allow an anonymous function that will do data transformation like above. But as you can see above XTemplate really has more to offer, it just annoys the heck out of me on a regular basis

Next, in the base class we use for all of our Result classes in our Schema, we override _inflate_to_datetime to rebless the returned value into our subclass:

And lastly, in our JSON view, we ensure that convert_blessed is on so that json will automatically call our TO_JSON method:

And that’s all there is to it! Thanks Perl for allowing me to rebless my objects

which expands to:

Problematically, DateTime has no TO_JSON method. I see two solutions to this, both of which kinda suck.

Monkey Patch

I could do something like:

but we all know that Monkey Patching is Sketch Towne City. Since perl is a prototypical language there is certainly a way to do something like:

but I couldn’t really figure out how to do it. And even if I could that (probably easy with Class::MOP) I’d still have to make hooks to do that to all to all of our DateTime objects (which still sucks.) And then I could effectively do the same thing…

Subclass DateTime

And then do some kind of trickery in DBIC-land to make the DateTime instantiation able to use other classes. But that’s not as simple as it might sound (due to design issues in DBIC that are not easy to solve as far as I can see.)

I’m pretty sure that there is a good solution that I’m missing. What is it? Can anyone tell me?

1. unsubscribe from from the vanilla feed.
2. Add the OPML to my feed reader and then unsubscribe from the non-english and spam feeds
3. work on the IronMan code base to add another feed of when people add their feed, and then subscribe to that so I know when people join
4. profit!

Anyway, if anyone wants to help with that second to last one I’d appreciate it since plagger is CRAZY.