Planet Perl

Release Announcements.

2010-02-10T01:31:56+00:00

Turning and turning in the widening gyre
The falcon cannot hear the falconer;
Things fall apart; the centre cannot hold;
Mere anarchy is loosed upon the world...
Surely some revelation is at hand;
Surely the Second Coming is at hand.
—William Yates, The Second Coming

I've got 3 release announcements to do:

Strawberry Perl January 2010.

It's been out for over a week - I really should start writing these release announcements beforehand. This is an "upgrade version" - The only new module is GDBM_File. There are a number of non-user-facing internals changes, however. Things just got too busy over the holidays to do the user-facing changes that were planned.

5.10.1.1 is at: http://strawberry-perl.googlecode.com/files/strawberry-perl-5.10.1.1.msi
Other releases are mentioned at: http://strawberryperl.com/releases.html

"Strawberry Perl plus Padre" 0.56.

This is the new version of what has been previously called "Padre Standalone". In this case, it's been updated to the latest versions of both Strawberry Perl and Padre.

http://strawberryperl.com/download/padre/strawberry-plus-padre-0.56.msi http://strawberryperl.com/download/padre/strawberry-plus-padre-0.56.zip

"Merge Module" for Strawberry Perl.

We're starting to build a merge module that can be used in other Windows Installer packages. The infrastructure to do this was one of the things that got added to Strawberry Perl in this version.

If you're wondering "What is a merge module?" - a merge module is a Windows Installer "sub-package" that can be included in other packages.

More documentation about how to include Strawberry Perl in their own software will be posted within the next few days.

Plans for April:

Relocatability on install

"Strawberry Professional"

64-bit versions

perl 5.12.x (assuming it gets released soon enough)

Non-admin installation and support

Problem with Strawberry + Padre 0.56

2010-02-10T01:29:57+00:00

I included a development version of the wxWidgets libraries in the Strawberry+Padre builds, and it turns out that the development version causes Padre to crash upon exit.

Here's what to do about it, until Padre 0.57 is released and a new build is created:


rem These 2 lines delete the old version, just to be safe.
del /F c:\strawberry\perl\vendor\lib\Alien\wxWidgets.pm
del /F /S c:\strawberry\perl\vendor\lib\Alien\wxWidgets\*.*

rem The next command should all be on one line, 
rem and will install the fixed libraries.
perl -MPAR::Dist -e"install_par q(http://www.strawberryperl.com/download/padre/Alien-wxWidgets-0.50-MSWin32-x86-multi-thread-5.10.1.par)"

rem This rebuilds the Wx perl connector. 
cpan -f Wx

(The .par file has now been updated to use wxWidgets 2.8.10 instead of the 2.9.0 that was used.)

Benchmarking MooseX::Method::Signatures

2010-02-09T18:07:33+00:00

I've been seeing some talk about MooseX::Method::Signatures and its speed. Specifically, Ævar Arnfjörð Bjarmason said says that MXMS is about 4 times slower than a regular method call. He determined this by comparing two different versions of a large program, Hailo. This is interesting, but I think a more focused benchmark might be useful.

Specifically, I'm interested in comparing MXMS to something else that does similar validation. One of the main selling points of MXMS is its excellent integration of argument type checking, so it makes no sense to compare MXMS to plain old unchecked method calls. Therefore, I made a benchmark that compares MXMS to MooseX::Params::Validate. Both MXMS and MXPV provide argument type checking use Moose types. That should eliminate the cost of doing type checking as a variable. If you don't care about type checking, you really don't need MXMS (or MXPV).

The benchmark has two classes with semantically identical methods doing argument validation. One uses MXMS and the other MXPV. All method calls are wrapped in eval since a validation failure causes an exception. I also tested both success and failure cases. My experience with Params::Validate tells me that there's a big difference in speed between success and failure, and the results bear that out.

Here's what the benchmark came up with:

               Rate   MXMS failure   MXPV failure   MXMS success   MXPV success
MXMS failure  262/s             --           -41%           -81%           -94%
MXPV failure  448/s            71%             --           -68%           -90%
MXMS success 1393/s           431%           211%             --           -69%
MXPV success 4545/s          1634%           915%           226%             --

First, as I pointed out, there's a big difference between success and failure. I can only assume that throwing an exception is expensive in Perl. Second, the difference between MXMS and MXPV is much greater in the success case. This makes sense if simply throwing an exception is costly.

It seems that in the success case, MXPV is about 3 times faster than MXMS in the success case. I think the success case is most important, since we probably don't expect a lot of validation failures in our production code.

Benchmark code

Devel::NYTProf - Perl profiling links needed

2010-02-09T08:29:17+00:00

A conversation on IRC this morning...

07:46 XXXX damn... is it normal for DProf/Profiler to not work correctly with moose stuff?
07:46 aaaa people still use dprof?
07:47 XXXX people who might not be aware of alternatives, sure
07:48 XXXX what would you suggest instead then?
07:49 aaaa Devel::NTYProf is THE profiller these days :)
07:52 XXXX unfortunately googling perl profiling doesn't take you anywhere near it :(

So, I'd like to talk about Perl profiling, or even profiling Perl. Devel::NYTProf doesn't make the claim that it is THE Perl profiler, but it really does seem to be the best profiler for Perl that is currently available.

If you've not come across Devel::NYTProf the Perl profiler please go and give it at try.

If like me you think it's fantastic, please link to http://search.cpan.org/dist/Devel-NYTProf/ in a blog post or from your website with some appropriate link words, so we can let Google know about it, and therefor the rest of the world.

Can you help identify ambiguous CPAN distributions?

2010-02-09T02:45:06+00:00

Hello, Perl community. As I work on converting legacy CPAN Testers (CT1.0) reports to the new CPAN Testers 2.0 (CT2.0) format, I’ve encountered a curious conundrum and could use some volunteer help.

CT1.0 indexes reports based on the distribution name and version, e.g. “Foo-Bar-1.23″. This is an unfortunate historical accident, since PAUSE does not prevent uploads with the same file name to different author directories:

JDOE/Foo-Bar-1.23.tar.gz
JQPUBLIC/Foo-Bar-1.23.tar.gz

CT2.0 will index reports based on the full unique distribution file path. I’m currently working on a heuristic to link any given legacy test report (on “Foo-Bar-1.23″) with the correct distribution file path for that distribution name and version for the conversion to CT2.0.

For the most part, it works. Usually, there is only one distribution file path on BackPAN that matches. Sometimes there is more than one possibility, but I’ve worked out ways to resolve the ambiguity by comparing the possibilities to information in the 01mailrc files or the 02packages.details file.

But there are about 50 distribution name-version pairs on BackPAN that my heuristic fails to resolve. Since this is a one-time conversion from CT1.0 to CT2.0, all I need is a mapping file with entries like this for these ambiguous cases:

    YAML-0.39    INGY/YAML-0.39.tar.gz

If you think you can help — either through some automated approach or just by volunteering your human brain to do some basic research to identify the “authoritative” path (e.g. historical author list in the distribution documentation files), that would be a great help for me so I can keep plugging away on the conversion code and other todos.

Even confirming that the candidates on BackPAN have the same md5 sum would be helpful since then even if we guess the wrong author, the test results are still “good” for the mistaken distribution file.

Here is the list. The name-version pair is followed by an indented list of possible paths for that pair.

Attribute-Memoize-0.01
  DANKOGAI/Attribute-Memoize-0.01.tar.gz
  MARCEL/Attribute-Memoize-0.01.tar.gz
B-Generate-1.12_03
  JCROMIE/B-Generate-1.12_03.tar.gz
  JJORE/B-Generate-1.12_03.tar.gz
Bundle-Cobalt-0.01
  HARASTY/Bundle-Cobalt-0.01.tar.gz
  JPEACOCK/Bundle-Cobalt-0.01.tar.gz
CDDB-0.9
  FONKIE/CDDB-0.9.tar.gz
  KRAEHE/CDDB-0.9.tar.gz
Catalyst-Plugin-Session-Store-File-0.07
  ESSKAR/Catalyst-Plugin-Session-Store-File-0.07.tar.gz
  KARMAN/Catalyst-Plugin-Session-Store-File-0.07.tar.gz
Catalyst-Plugin-Static-0.05
  MRAMBERG/Catalyst-Plugin-Static-0.05.tar.gz
  SRI/Catalyst-Plugin-Static-0.05.tar.gz
Catalyst-Plugin-Static-Simple-0.14
  AGRUNDMA/Catalyst-Plugin-Static-Simple-0.14.tar.gz
  MRAMBERG/Catalyst-Plugin-Static-Simple-0.14.tar.gz
Crypt-SSLeay-0.51
  CHAMAS/Crypt-SSLeay-0.51.tar.gz
  TAKESAKO/Crypt-SSLeay-0.51.tar.gz
Curses-UI-0.72
  MARCUS/Curses-UI-0.72.tar.gz
  MMAKAAY/Curses-UI-0.72.tar.gz
Curses-UI-0.73
  MARCUS/Curses-UI-0.73.tar.gz
  MMAKAAY/Curses-UI-0.73.tar.gz
DateManip-5.20
  PHOENIX/DateManip-5.20.tar.gz
  SBECK/DateManip-5.20.tar.gz
Finance-Bank-HSBC-1.04
  BISSCUITT/Finance-Bank-HSBC-1.04.tar.gz
  MWILSON/Finance-Bank-HSBC-1.04.tar.gz
Finance-Bank-HSBC-1.05
  BISSCUITT/Finance-Bank-HSBC-1.05.tar.gz
  MWILSON/Finance-Bank-HSBC-1.05.tar.gz
Locale-Object-0.73
  EMARTIN/Locale-Object-0.73.tar.gz
  FOTANGO/Locale-Object-0.73.tar.gz
MARC-0.81
  BBIRTH/MARC-0.81.tar.gz
  ESUMMERS/MARC-0.81.tar.gz
MARC-1.13
  ESUMMERS/MARC-1.13.tar.gz
  PETDANCE/MARC-1.13.tar.gz
Mail-Thread-2.41
  RCLAMP/Mail-Thread-2.41.tar.gz
  SIMON/Mail-Thread-2.41.tar.gz
Math-MatrixReal-1.1
  ANDK/Math-MatrixReal-1.1.tar.gz
  STBEY/Math-MatrixReal-1.1.tar.gz
Maypole-Authentication-Abstract-0.6
  BOBTFISH/Maypole-Authentication-Abstract-0.6.tar.gz
  SRI/Maypole-Authentication-Abstract-0.6.tar.gz
Maypole-Config-YAML-0.1
  BOBTFISH/Maypole-Config-YAML-0.1.tar.gz
  SRI/Maypole-Config-YAML-0.1.tar.gz
Maypole-Loader-0.1
  BOBTFISH/Maypole-Loader-0.1.tar.gz
  SRI/Maypole-Loader-0.1.tar.gz
Maypole-Plugin-Authentication-Abstract-0.10
  BOBTFISH/Maypole-Plugin-Authentication-Abstract-0.10.tar.gz
  SRI/Maypole-Plugin-Authentication-Abstract-0.10.tar.gz
Maypole-Plugin-Component-0.05
  BOBTFISH/Maypole-Plugin-Component-0.05.tar.gz
  SRI/Maypole-Plugin-Component-0.05.tar.gz
Maypole-Plugin-Config-YAML-0.04
  BOBTFISH/Maypole-Plugin-Config-YAML-0.04.tar.gz
  SRI/Maypole-Plugin-Config-YAML-0.04.tar.gz
Maypole-Plugin-Exception-0.03
  BOBTFISH/Maypole-Plugin-Exception-0.03.tar.gz
  SRI/Maypole-Plugin-Exception-0.03.tar.gz
Maypole-Plugin-I18N-0.02
  BOBTFISH/Maypole-Plugin-I18N-0.02.tar.gz
  SRI/Maypole-Plugin-I18N-0.02.tar.gz
Maypole-Plugin-Loader-0.03
  BOBTFISH/Maypole-Plugin-Loader-0.03.tar.gz
  SRI/Maypole-Plugin-Loader-0.03.tar.gz
Maypole-Plugin-Relationship-0.03
  BOBTFISH/Maypole-Plugin-Relationship-0.03.tar.gz
  SRI/Maypole-Plugin-Relationship-0.03.tar.gz
Maypole-Plugin-Transaction-0.02
  BOBTFISH/Maypole-Plugin-Transaction-0.02.tar.gz
  SRI/Maypole-Plugin-Transaction-0.02.tar.gz
Maypole-Plugin-Untaint-0.04
  BOBTFISH/Maypole-Plugin-Untaint-0.04.tar.gz
  SRI/Maypole-Plugin-Untaint-0.04.tar.gz
Net-DNS-0.02
  ANDK/Net-DNS-0.02.tar.gz
  MFUHR/Net-DNS-0.02.tar.gz
Net-SSH2-0.07
  AWA/AWA/Net-SSH2-0.07.tar.gz
  DBROBINS/Net-SSH2-0.07.tar.gz
NetPacket-0.04
  ATRAK/NetPacket-0.04.tar.gz
  CGANESAN/NetPacket-0.04.tar.gz
PDL-2.3.2
  CSOE/PDL-2.3.2.tar.gz
  KGB/PDL-2.3.2.tar.gz
PNGgraph-1.11
  DMOW/PNGgraph-1.11.tar.gz
  SBONDS/PNGgraph-1.11.tar.gz
POE-Session-Attributes-0.01
  CFEDDE/POE-Session-Attributes-0.01.tar.gz
  JSN/POE-Session-Attributes-0.01.tar.gz
Plucene-1.19
  SIMON/Plucene-1.19.tar.gz
  STRYTOAST/Plucene-1.19.tar.gz
RT-Extension-MergeUsers-0.02
  JESSE/RT-Extension-MergeUsers-0.02.tar.gz
  KEVINR/RT-Extension-MergeUsers-0.02.tar.gz
SNMP-1.6
  GSM/SNMP-1.6.tar.gz
  WMARQ/SNMP-1.6.tar.gz
SXIP-Membersite-1.0.0
  KGRENNAN/SXIP-Membersite-1.0.0.tar.gz
  TOKUHIROM/SXIP-Membersite-1.0.0.tar.gz
Scalar-Defer-0.13
  AUDREYT/Scalar-Defer-0.13.tar.gz
  NUFFIN/Scalar-Defer-0.13.tar.gz
Term-Prompt-0.02
  ALLENS/Term-Prompt-0.02.tar.gz
  DAZJORZ/Term-Prompt-0.02.tar.gz
Term-Prompt-0.05
  ALLENS/Term-Prompt-0.05.tar.gz
  DAZJORZ/Term-Prompt-0.05.tar.gz
Test-Warn-0.07
  BIGJ/Test-Warn-0.07.tar.gz
  MPRESSLY/Test-Warn-0.07.tar.gz
Time-0.01
  JPRIT/Time-0.01.tar.gz
  PGOLLUCCI/Time-0.01.tar.gz
Tk-Wizard-Bases-1.07
  LGODDARD/Tk-Wizard-Bases-1.07.tar.gz
  MTHURN/Tk-Wizard-Bases-1.07.tar.gz
UUID-0.03
  CFABER/UUID-0.03.tar.gz
  LZAP/UUID-0.03.tar.gz
Win32-EventLog-Carp-1.21
  IKEBE/Win32-EventLog-Carp-1.21.tar.gz
  RRWO/Win32-EventLog-Carp-1.21.tar.gz
YAML-0.39
  INGY/YAML-0.39.tar.gz
  KING/YAML-0.39.tar.gz
finance-yahooquote_0.19
  DJPADZ/finance-yahooquote_0.19.tar.gz
  EDD/finance-yahooquote_0.19.tar.gz
libapreq-1.33
  GEOFF/libapreq-1.33.tar.gz
  STAS/libapreq-1.33.tar.gz
pg95perl5-1.2.0
  MERGL/pg95perl5-1.2.0.tar.gz
  YVESP/pg95perl5-1.2.0.tar.gz

Why Ruby is prettier and Padre changes the Perl community

2010-02-09T02:13:57+00:00

Why is PHP so much easier for newbies?

Why does Java have the best IDE tools?

Why is Ruby prettier than Perl?

Why does Perl have the best package repository?

As I've played through Mass Effect 2 over the last few weeks, I see some interesting parallels.

In the Mass Effect universe, human technology is bootstrapped by the discovery of an ancient abandoned alien observation outpost on Mars, and the further discovery that the dwarf planet Charon is really an abandoned but active interstellar jump gate covered in ice.

Other similar species have done the same, resulting in a galactic community of around a dozen civilisations all based around the same basic technological underpinnings.

Despite these civilisations believing a recently (50,000 years) extinct civilisation built the gates, it turns out the technology is perhaps millions of years old.

Every 50,000 years, the synthetic AI race that built them returns from hiding in intergalactic space to wipe out all of the existing advanced species based on "their" technology, and reset the galaxy for the next set of civilisations to rise.

In a conversation between the game's protagonist and one of these old AIs, we are lambasted by the AI for taking the shortcut on their technology. The jump gates and other technology is left in place intentionally, so that each new generation of civilisations take a controlled and predictable development path, making it easier to destroy them.

The AI posits that it is the overcoming of adversity on your own that drives true technological advancement, and that easy routes make you (technologically) weak.

I think you can see something similar in the development of the different programming languages.

Java is long and wordy, taking a long time to type. The need to work around this limitation resulted in the proliferation of powerful IDEs, resulting in the annual 20 million line of code Eclipse release train.

PHP as a web language would have been stillborn if it didn't deal competently and quickly with the need to easily deploy code, the result of which is that you can effortlessly just change .html to .php, add a hello world tag, and upload via FTP as normal (something Perl still can't do well).

Python's need to gain mindshare against an entrenched Perl led to a huge focus on being easy to learn, to a simplification of the language, and to hugely popular things such as the PyGame library and game competitions.

Faced with the lack of truly great package repository, and with a web-heavy community, Ruby became the "prettiest" language. Creating an elegant website is both expected and required if you are going to gain mindshare for an idea.

And Perl's messy syntax and difficulties in the area of maintaining large codebases, combined with a pragmatic sysadmin-heavy community, resulted in an unmatched packaging system that allowed code to be maintained in small pieces, with enormous volumes of support infrastructure around it.

The ease of publishing and trend to smaller package that the CPAN allowed conversely means that the Perl community has never really had the need for pretty and elaborate websites, and the smaller package size means that we lack the giant headline libraries that make the payoff from website work better.

Our bias towards a pragmatic tech-savvy sysadmin userbase means we haven't really provided anything like the focus on learnability that has driven Python's gradual dominance in the mindshare of the young. It takes a certain rigour in your prioritisation to intentionally remove and dumb down existing powerful features so that the language is easier to learn.

Even for Strawberry, which focuses on the userbase with the lowest traditional knowledge, we intentionally have the smallest and most maintainable website possible and we don't even have the kind of introductory screencasts that we really really need (which should be easy but which I never seem to find the time to do).

If you throw a bunch of Perl coders against some PHP coders in a website competition, it is not unexpected that when both sides play to their strengths you will see something like http://geo2gov.com.au/html?location=e.g.+1+Oxford+Street from the Perl coders and something like http://www.hackdays.com/knowwhereyoulive/postcodes/view/2000 from the PHP coders.

The former required a massive amount of data extraction, transformation, aggregation, a gigabyte-sized PostGIS database, and deployment via a Linux virtual appliance to Amazon EC2 to allow for strategic load-shedding.

The latter required the ability to turn data into presentable and understandable information for real humans, and to make it pretty enough that they WANT to look at it.

Driving true technological progress, then, may often be about identifying weaknesses that are hard to solve but aren't completely impossible (and don't have any crippling long-term conceptual flaws at an economic or project-management level).

The three best projects I have driven - PPI, Strawberry, and (in part) Padre - all share this property. All three of these represent hard but not impossible problems, and require an awareness about which issues are intractable and which issues merely exist because there's been no need to solve them any better.

Padre in particular has suffered greatly from issues with Wx quality and threading. But given the low takeup of both threading and Wx it was reasonable to move forward on the basis that these would be fixed once there was something depending on them, and driving a need to fix them.

All of our early problems are gone now, and there is continued pressure to find ways to improve our use of (and the efficiency of) Perl's native ithreads.

Similarly, the creation of Strawberry required a lengthy year-long process of fixing Win32 bugs in all kinds of toolchain and low level modules, because we'd never had a proper working developer feedback loop before.

Similarly, Perl's current push for marketing and blogging and websites is directly resulting from Python's success in mindshare capture.

So my question for you to ponder this week is the following:

What can you see that Perl as a whole struggles to do well, for which a good solution is not impossible, and is only being held back by smaller problems which would go away if there was a working candidate solution put in place that needed those small problem solved.

Building RPMs from CPAN Distributions

2010-02-08T15:16:54+00:00

Regular readers will know that in the past I've shown some interest in building RPMs from CPAN distributions. It's been a while since I did much work in this area (although I do still release the occasional module to my RPM repository.

Over the weekend I was at FOSDEM and I attended Gabor's talk on packaging CPAN modules for Linux distributions. This has rekindled my interest in this area and I spent most of the train journey back from Brussels hacking around the area.

There's one thing that has been bothering me in particular recently. The standard RPM building mechanism (or, at least, the way it's configured in Fedora and Centos) does something incredible brain dead when trying to work out what other modules the current module depends on. It does it by parsing the source code and looking for "use" statements. This means that a module that might only be used in really obscure cases is going to be listed as a mandatory requirement for your module.

Gabor and I actually saw an example of this over the weekend when the Fedora packaging team raised a bug against Padre because it requires Win32::API. Padre, of course, only uses Win32::API when being used on Windows. And for that reason Win32::API is not listed as a dependency in its META.yml.

And that's, of course, where the RPM builders should be going to get a list of dependencies. META.yml contains the list of other modules that the author wants the module to depend on. This should be seen as the definitive list. Of course, there might be errors in that list - but that should be addressed by raising a bug against the module.

I've poked at this problem a few times, trying to work out how the RPM system parses the code and trying to replace that with code that looks at META.yml instead. But the RPM system uses a baroque system of interdependent macros and eventually they all lead to a piece of rather clunky Perl code. So each time I've approached this problem, I've backed off again.

The problem became more urgent when I wanted to package Plack for Fedora. Plack supports all sorts of hosting environments and therefore includes "use" statements loading a number of modules that most people will never use. Fedora includes Apache2, so Apache::Request (which is for Apache1) will never be available. It's not listed in META.YML, but it is used by one of the modules. The RPM build system was therefore insisting that it should be present. An impasse was reached.

Then I decided to turn the problem on its head. RPM building has two steps. You create a spec file for the RPM and then you build the RPM using the spec file and your original tarball. I started wondering if I could ensure that the spec had all of the requirements (from the META.yml). Once I'd done that I would only need to find some way to turn off the RPM build system's default behaviour.

People packaging CPAN modules for Fedora (and Centos) use a program called 'cpanspec' to generate spec files. I started digging into the code there in order to find out how to insert the list of correct dependencies.

Only to find that it has already been done. cpanspec is already doing the right thing and generating a list of 'Requires' statements from the data in META.yml.

Then all I needed to do was to see if I could turn off the (broken) default RPM build behaviour which was adding spurious extra dependencies. That proved to be easy too. It's just a case of adding %__perl_requires %{nil} to your .rpmmacros files.

So now all of my RPMs will have only the correct dependencies listed. This makes me very happy.

I suppose I should go back and rebuild all of the older ones too.

Oh, and because I've worked out a really easy way to generate this - here's a spreadsheet listing which CPAN modules are available as RPMs for Fedora. I plan to keep this list up to date (and make it much longer). [Link now fixed]

p.s. More about my trip to FOSDEM and the Perl marketing push there over the next couple of days.

Spock and Data look at CPAN

2010-02-08T02:03:32+00:00

In the two Star Trek series, the characters of Data and Spock pose a question: What would it be like to be able to make decisions on a largely, or even purely, logical basis? Not widely known is that this is a question with an answer based on evidence.

Because of tumors, surgery, and accidents, some unfortunate individuals lose all or most of their ability to make decisions based on emotion. Particularly striking are patients with ventromedial (VM) damage. Patients after VM damage seem to function normally. Their IQ is not impaired, and is sometimes above average. VM patients also do well on multiple-choice tests aimed at measuring their ability to make the kind of decisions needed to cope with everyday life. But VM patients do not make real life decisions well. They cannot hold jobs. They cannot manage their own financial affairs. Everyday decisions, even ones they make successfully on multiple-choice tests, they are unable to make in real life. When family is not available to care for them, people with VM damage live out their lives in institutions.

Data and Spock could not function as Starfleet officers. Why? Antonio Damasio studied VM patients for many years, and he gives his answer in the book Descartes' Error. Written for the non-specialist, the book reveals how we actually make complex decisions, including complex technical decisions. It is well worth studying.

Imagine looking for a module on CPAN. You look at what the modules claim to do. You note the author's name. Do you know him from elsewhere? What did you think of his other work? You look at the documentation. Well written? Does it give you the feeling that the writer knows what he is talking about? You read the reviews. You assess not just what is said, but how the review says it. Is the reviewer thoughtful or reckless? Well-informed or a newbie? Bigoted or open-minded?

These decision variables, according to Damasio, are matched to "somatic states". Somatic states can be thought of as snapshots of the body state, internal and external. Body chemistry is a major component in a somatic state. Somatic states embody (pun intended) emotional value judgements. Somatic states are the variables in the forebrain's decision-making calculator.

Our somatic state calculator is able to make complex decisions in situations where the data is not just uncertain, but deeply uncertain. Real-life uncertainty cannot be characterized as a Bell curve distribution of well-defined variables. Real-life uncertainty goes deeper than that. Deep uncertainty cannot be plotted as a distribution. In a deeply uncertain situation, we are uncertain, not only about the shape of a probability distribution, but also about the meaning and significance of the distribution itself.

Even a relatively structured real-life decision, like selecting a CPAN module, is deeply uncertain. The human brain can make quick and relatively accurate decisions in the face of deep uncertainty. An unemotional thinker could not effectively choose among CPAN modules, and could not make accurate decisions about which programming language to use. Data, Spock or a computer might be able to deal with technical subproblems if the subproblems were structured to eliminate deep uncertainty, but their logic would get lost in the technical decisions that actually occur in real-life.

Damasio theorizes that the basic "variables" of our decision making are body-images. He uses this hypothesis to outline a mechanism to explain how human decision-making came to be through evolution. Earlier brains (perhaps pre-human) would have started by associating somatic states with actions and/or things in their environment. Perhaps their memory would have contained action-to-somatic-state pairs like these:

    "Tried to eat porcupine" => "Still hungry and boy do I hurt"

While informative, this by itself does not tell an organism whether it should avoid all porcupines, or whether it should merely be more cautious when they are on the menu. It would be an evolutionary advantage to be able to analyze fact-to-somatic-state data entries like the above and calculate with them in complex ways. Logic would be one tool for doing that. A creature with some degree of ability to reason logically would have a survival advantage.

Logical decision-making seems to have evolved from emotion. It remains rooted in it. The role of emotion in decision-making is not a useless vestige like the appendix. We can't cut it out without destroying our decision-making mechanism.

Closing apologies: I hope Star Trek fans will forgive my simplified picture of Spock and Data. While Spock and Data do not have the usual humanoid emotional lives, they do have emotional lives, emotional lives which play major roles in some episodes. In fact, the two series seem to spend more time on Spock's and Data's emotional issues than on those of anyone else on the Enterprise. For Spock and Data to function as well as they do, they would need to be emotional. It might be that, even in fiction, there is no way of getting around this.

Damasio's theory is complex, and like Spock and Data, it resists simplification. I hope my summary will encourage others to read Damasio directly. Descartes' Error is written for a general audience, and most of it is about his somatic marker hypothesis.

Help keep the world safe from SQL injection

2010-02-06T20:55:45+00:00

A while back, I put up bobby-tables.com as a repository for showing people the right way to handle external data in their SQL calls. Whenever someone pops up on a mailing list or IRC and they're building SQL statements using external tainted data, you can just refer them to the site.

In the past few days, I've spiffed up the site (with design help from Jeana Clark) and added pages on Perl and PHP. I need more examples, though. It's 2010, and there's no reason anyone shouldn't know about parameterized SQL calls.

The site source is hosted on github, so if you have any contributions, please fork it and let me know about your applied changes, or you can email me directly.

Thanks!

P.S. In the next few days, I hope to fire up some redesign on perl101.org, too.

An English-only Planet Iron Man

2010-02-06T12:55:04+00:00

I’m very happy to know that Perl has global appeal from seeing all the non-English Perl blogs aggregated on Planet Iron Man, but since I’m a (typical American) monoglot, I’d prefer an Iron Man feed with only English articles. So I made one.

It’s available at http://feeds.dagolden.com/ironman-english.xml. It updates hourly from the master feed.

And for the curious, or for anyone who wants to adapt this for other languages, here’s the Perl program that I whipped-up to create the feed:

# feedfilter.pl - downloads and filters the Perl Ironman feed for English
# entries. Results sent to STDOUT.
#
# The heuristic filters out entries unless the content is mostly latin
# characters and English is close to the best guess of a language.  Short
# entries with code seem to confuse Lingua::Identify, so we take entries that
# seem "close-enough".  Tuned via trial-and-error.
#
# Copyright (c) 2010 by David Golden - This may be used or copied under the
# same terms as Perl itself.

use 5.008001;
use strict;
use warnings;
use utf8;
use autodie;

use IO::File;
use Lingua::Identify qw(:language_identification);
use Time::Piece;
use URI;

use XML::Atom::Feed;
$XML::Atom::ForceUnicode = 1;
$XML::Atom::DefaultVersion = "1.0";

# Global heuristic tuning
my $latin_target = 0.95;  # 95% latin chars
my $lang_fuzz = 0.02;     # English within 2% probability of best language

run();

#--------------------------------------------------------------------------#

sub latin_ratio {
  my $string = shift;
  my $alpha =()= $string =~ /(\p{Alphabetic})/g;
  my $latin =()= $string =~ /(\p{Latin})/g;

  return 0 if ! $latin || !$alpha; # !$alpha probably redundant
  return $latin / $alpha;
}

sub run {
  my $in_feed = XML::Atom::Feed->new(URI->new("http://ironman.enlightenedperl.org"));

  my $out_feed = XML::Atom::Feed->new;
  $out_feed->title("Planet Iron Man: English Edition");
  $out_feed->subtitle( $in_feed->subtitle );
  $out_feed->id("tag:feeds.dagolden.com,".gmtime->year().":ironman:english");
  $out_feed->generator("XML::Atom/" . XML::Atom->VERSION);
  $out_feed->updated( gmtime->datetime . "Z" );
  for my $l ( $in_feed->link ) {
    $out_feed->link($l);
  }

  for my $e ( $in_feed->entries ) {
    my $content = $e->content->body;
    my $latin = latin_ratio($content);
    my %lang = langof($content);
    my $best = [sort { $lang{$b} <=> $lang{$a} } keys %lang]->[0];
    $lang{en} ||= 0;
    $out_feed->add_entry($e)
      if $latin > $latin_target && ($lang{$best} - $lang{en} < $lang_fuzz);
  }

  binmode(STDOUT, ":utf8");
  print $out_feed->as_xml;
}

CPAN Testers 2.0 end-January update

2010-02-04T23:45:11+00:00

The bad news is that we’re still about two weeks behind schedule. The good news is that we’re not falling further behind and in some areas, we’re already ahead.

As I wrote in the last update, a number of my early-January tasks for revising the Metabase libraries didn’t get done and were blocking progress on other fronts. That work is now pretty much complete and I’m ready to turn my attention to the actual migration of legacy reports into a Metabase repository. Once that’s done, we’ll be able to test using it to feed the cpantesters.org databases that Barbie has been preparing for the conversion.

CPAN Testers 2.0 activity in the last couple weeks:

I revised Metabase framework libraries to separate user profile information and user authentication into separate facts. This also meant revising the user-profile generation program to match.
I implemented new Metabase::Resource classes to standardize extraction indexing data from Metabase::Fact resource strings. (E.g. cpan://distfile/DAGOLDEN/Capture-Tiny-0.07.tar.gz can be indexed under author “DAGOLDEN”, distribution name “Capture-Tiny”, and so on). This wasn’t on the plan, but I discovered that Ricardo and I had never actually gotten around to implementing, so it had to go from stubs to working code.
I confirmed that despite all the Metabase framework changes, I could still launch a local Metabase server and send CPAN::Reporter test reports via Test::Reporter::Transport::Metabase. (Thanks to Florian Ragwitz and Matt Trout for patches and guidance respectively on updating the Metabase web server for a more modern Catalyst runtime).
Barbie converted cpantesters.org backend databases to index on GUIDs instead of NNTP IDs. Existing legacy report had their NNTP IDs mapped to GUIDs.
Barbie and I agreed to have cpantesters.org get updates directly from the Amazon back-end rather than going through a web server. This postpones the need to implement search capability through the web until after launch.

The exact semantics of direct search against the back-end have yet to be worked out, but I’ve decided to hold off on that until we have a Metabase of historical records to experiment against.

At this point, the critical path is the conversion of old articles to the Metabase and the deployment of a web server to inject new reports into it. I’m working on the first, and hope to have both of those done by mid Feb. That leaves us a tight two week period for testing, so stay tuned for the next update.

Perlbuzz news roundup for 2010-02-01

2010-02-04T16:00:00+00:00

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

Famous Perl One-Liners Explained, Part IV: String and Array Creation (catonmat.net)
Run invdividual Test::Class methods in vim (blogs.perl.org)
A regular expression tip every day from @regextip (and not just Perl regexes)
A few resources for women in open source (itworld.com)
The next QA hackathon: What do you need? (blogs.perl.org)
YAPC::NA 2010 dates announced: June 21-23, 2010 (news.perlfoundation.org)
Pod: Now with sane web links (justatheory.com)
Creating mailing labels with PostScript::MailLabels (blog.worldlabel.com)
Ten ways to destroy your community (coffee.geek.nz)
Conferences on the cheap (blogs.stpcollaborative.com)
Perl 5.11.4 available (xray.mpe.mpg.de)
Using Perl to find best Wii bundle pricing (blogs.perl.org)
Now's the chance for North Americans to easily get to a Nordic Perl Workshop (blogs.perl.org)
What is Moose? (perltraining.com.au)
Pretty Perl (blogs.perl.org)

Using Devel::REPL to develop a mail cleanout script

2010-02-04T11:17:06+00:00

So, weeks and weeks on from when I planned, I'm finally getting down to sorting my e-mail out. First thing is to pick an IMAP client library. A quick poke for IMAP client libraries on CPAN reveals a few, notably IMAP::Client, Net::IMAP and Net::IM...

Read and post comments | Send to a friend

CPAN Testers Summary - January 2010 - The Wedge

2010-02-04T10:49:36+00:00

One month to go and work is progressing well on the transformation to CPAN Testers 2.0. Over the last month many changes to the websites have been visible, but just as many changes have been happening behind the scenes. The Metabase is a key part of the transformation, and although work has been going well, it is reaching the point where it'll need some serious testing prior to switch over on 1st march 2010. If you have the time, please join the cpan-testers-discuss mailing list or contact David Golden to let him know what you can help with. See David's CPAN Testers 2.0 mid-January update blog post for a more detailed status update.

In order to reduce the load on the perl.org servers, after the announcement of the switch over, the CPAN Testers agreed to back off their smokers to ease the pressure on the perl.org mail servers. The cpan-testers mailing list is a very high volume list, and takes up a lot of resources to manage it. Many of the testers throttled back their smoke bots and we did see a dramatic reduction in test reports being submitted. We were aiming for around 5,000 a day maximum. Within a day or two we were successfully below the target.

However, not all went well. One smoker bot suddenly appeared to go AWOL, and the tester didn't seem to be responding to direct requests to throttle the smoker. Worse still the bulk of reports being produced were bogus. While some PASS reports got through, most were failing due to what appears to be a bad combination of environment and old toolchain software. As this was now polluting the pool of reports at a considerable rate (for every good report submitted, 1 or more was submitted by the bad smoker), something needed to be done to reduce or halt the effects. Several authors were rightly concerned that this would make their distributions look bad on the CPAN Testers Reports site. Thankfully, a new site (more on the later) is in the works which will make this easier to manage, but in the interim a further measure was put in place. I now have the ability to blacklist runaway smokers, by invalidating their reports as the come in. This then means the reports are ignored by the Reports site, the Statistics site and the rest of the eco-system. I also manually marked all the smoker's reports during January as invalid.

It transpires that the tester was on holiday and had started off his smokers before he went, without checking to ensure the reports they were sending were valid. Once this tester has upgraded to use the right tools, I'll remove him from the blacklist. However, it is good to know that we can now quickly stop any future runaway smokers before they can do much damage to the reports and statistics.

Normally one story would be the only excitement of a month, but there was more to come. On 17th January, the CPAN Testers server started show effects of being under attack. In the early hours of 18th January, the server locked up, and required manual intervention to reboot it. Once back online, an investigation through the logs revealled that the MSNBot, as used by Microsoft, had been hitting the server at a rate of knots. In fact, so much so, that the logs began rapidly filling up again after the reboot. After initially blocking the range of IPs, which grew as the day went, I wrote an article and posted to the CPAN Testers blogs to warn anyone who might be using the CPAN Testers server. Little did I realise that the story would spread like wildfire around the world on numerous IT related news networks and blogs! I did get an apology from someone representing the Bing team, but it should never have got to that. Reading many of the comments on various blogs, although a small minority took delight in having a kick at Perl, the majority of posts were in support of the ban, and many even had their own experiences. While I may have been the first to shout loudly, CPAN Testers definitely weren't the first to be knocked out by Microsoft.

Over a week later, with the ban still in place, and the robots.txt changed to ban all access to msnbot, every hour now the msnbot blasts the server for about 5-10 minutes at the rate of between 4-8 requests a second, mostly from the same 2 IP addresses. So even after banning the bot (it gets a 403) and having an apology from Microsoft, the bot still hasn't learnt to get itself under control. If Microsoft ever want people to take Bing seriously as a search engine, then they need to start acting responsibly, otherwise they are likely to find themselves banned from a good portion of the internet.

One thing I would like to make clear about the incident, is that all the monitoring of the server is done completely voluntarily. Over the last month this has taken up a lot of spare time, which often wasn't there to begin with. However, the server itself is *NOT* a High-Availability setup, and is *ONE* server on its own. No redundancy (apart from the RAIDed disks) with the web server, database and processing tools all sharing the same physical hardware. If it takes 2 seconds to return a web page, its likely that the server is under considerable load to process incoming reports, running backups or generating web pages, RSS feeds, JSON/YAML files to keep the rest of the eco-system (including CPAN/CPANPLUS, search.cpan.org, etc) able to keep up to date. Taking it out of action is not something that is taken lightly.

The original post was perhaps rather emotionally put together, and I apologise to anyone who may have got caught in any flak for that. However, I had just woken up and spent much of the morning trying to get the server back online while getting the kids ready for school and heading out to work! With it being a Monday morning too, hopefully it was understandable that a rant ensued. I'll be taking several deep breaths, if (though hopefully not) it happens again!

As mentioned earlier in the post, and in the previous summary, I did plan to release a new site during January. The CPAN Testers Administration site is still planned to go live, just not yet. With all the changes to the underlying software for CT2.0, there are some changes required for the Administration site that also need to be done. As this isn't live yet, I now consider it a low priority to getting CT2.0 completed, and will now wait until after CT2.0 has gone live, before finishing off the release.

In the last weekend of January, the biggest changes to the current databases went into effect, with the Metabase GUID now being used. Although the full extent of the change won't be seen until we're using the Metabase for submitting reports, this first shift is an important one. There were a few glitches as I brought the processing tools back online, as I soon discovered little parts that were affected by the change that I hadn't anticipated. Thankfully the errors were minor and all were quickly fixed. The server is now catching up on processing from the weekend, and I anticipate all will be back to normal service within the next day or two.

To reduce the processing load, as mentioned in a previous post, the database backups are now happening a little less frequently. The CVS backups have now been disabled, with the uploads and release databases both backed up once a day (usually between 00:30 and 02:30 Central European Time). The cpanstats database is currently backed up once an hour, but seeing as the bzip version seems to be only popular with a few people (one being Yahoo! Slurp :)) and only downloaded at most once a day by any single IP (including Yahoo! Slurp ... see Microsoft, some search engines can get it right), I'm considering only generating the bzip version only once a day. I'll watch the logs and see if there are any changes, but if aren't I will likely adjust the backups inline with current requests for the files.

Along with the backup changes, various other daily server processes have been reviewed and many have been rescheduled to reduce server load. The end result has been to reduce the nightly overheads and hopefully the server will be in a better position to process reports once the CPAN Testers switch to the Metabase and unleash their smokers from the current limiters.

Last month we had a total of 162 tester addresses submitting reports. The mappings this month included 21 total addresses mapped, of which 7 were for newly identified testers. Another low mapping month, due to work being done on CPAN Testers as a whole.

A long summary this month, but then a lot has been happening. Expect updates throughout the month as various parts of CT2.0 undergo testing, and we start to see the results of all the hard work of the past couple of years. The future is nigh.

Cross-posted from the CPAN Testers Blog.

microsoft customer service madness

2010-02-04T00:00:10+00:00

In January, I ordered an Xbox 360 Play & Charge Kit from Amazon. It's a NiMH battery pack with a USB cable to connect the controller (also a charger) to the Xbox. It arrived a few days later. While the cable would power the controller, its "charging" light wouldn't come on and it didn't seem to be charging the controller. The kit came with a piece of paper that said, "In case of problems, do not return! Contact Microsoft!"

It didn't say how to do that, but I found a way to do it online (it wasn't very easy, either) and after a few emails, they told me to phone them. On the phone, they gave me a mailing address. "Send the kit here and we will send you a replacement." That sounded fair, so I did so.

Today, I got a package in the mail from Microsoft. I opened it, and it contained an Xbox 360 Quick Charge Kit. This is a slightly different product. It's more expensive, but does not allow you to play while you charge. Without two battery packs, it's not very useful, and it only comes with one.

I called Microsoft to explain their error, and after a lot of explaining, the representative asked me for my tracking number for the item I sent to their returns facility. I said, "I never got one." He hissed and said he'd try to help, but that without a tracking number, they couldn't prove that the facility had ever received my original item.

"Well," I said, "you sent me a replacement, so surely you did it after receiving something. Also, the replacement is in a box labeled with my RMA number."

This didn't mean much to him. He said I should go to the local USPS office and ask for my tracking number. I assured him that this was not possible. USPS does not have a tracking number for every parcel, unless one is requested. He confirmed this with his supervisor. Instead, he said, I could just give them my "drop off confirmation number," which would be on my receipt.

"I sent this over a week ago. I do not have the receipt anymore." He said I should go to the USPS and ask them for my receipt. I tried to explain that the post office would have no means to find a record of my business with them. The guy kept repeating that I could just go ask USPS for my receipt and they'd have it... somehow. I'm pretty sure he thought I was insane for suggesting otherwise.

Eventually I said, "You know, I have the tracking number of the package in which my replacement came." This excited him, and he took the number and put me on hold. When he came back he said, "Great! Now we know you received a package from us! All I need now is the tracking number for the package you sent us." He seemed to think that I had the number, but was reluctant to give it.

"Look," I said, "I have a box, from you, with a tracking number showing you sent it and I received it. It has my RMA number on it, which refers to what I sent in. The box's shipping label has the name of the incorrect part. How on Earth is this not enough information?"

He said, "I'm going to go the extra mile here and really be on your side. I'm going to bypass our procedures and file this request anyway. It will probably be rejected, but I promise you I'll call you back. When I promise I'll do something, I'll do it!" I'm pretty sure I'm never gonna hear from this guy again. I'll call them on Tuesday and they'll deny having any knowledge of my case or possibly of what an Xbox is.

On the bright side, at least I ended up with something more expensive than what I sent them. If they had sent me a small plastic widget-cover, I'd be pretty angry. As it is, I just think they're hopelessly disorganized.

Perl for Windows statistics

2010-02-03T17:34:20+00:00

Adam Kennedy pointed me to the download count of Strawberry Perl for Windows. this shows that in the last 3 months there were approximately 54K downloads of 5.10.x and 4.5K downloads of 5.8.x That would be about 650 downloads a day.

Back on 7/7/2007 Jan Dubois mentioned that ActiveState has more than 4,500 downloads of ActivePerl 5.8.x per day for Windows and more than 1,000 downloads of ActivePerl 5.6.1 per day.

Unfortunatelly I don't have newer numbers from ActiveState but it would be interesting to see how have the numbers changed? Also it would be nice if we had numbers from the other Perl distributions for Windows.

On Nagios, Thunk, Shinken and wrapper included marketing

2010-02-03T11:35:39+00:00

Nagios is probably the most famous and used monitoring program on the market. It's free, GPL and has nice features such as object representation of data, inheritance, plugin systems, passive testing, built-in Perl interpreter, result caching, pipe interface, alert delegations and so on and so on.

The web interface of Nagios is, however, incredible ugly. It's written in CGI the way the early CGI scripts were written. When you make a change to a server via the web interface, you get a few screens (avoiding Javascript is a benefit for some cell phones, I guess) and the old and quickly-annoying "You have done the action you wanted, please click this back link we created to go backwards" screen. You won't simply get automatically directed back to where you were before with a new message at the top in bright green saying "Action X done" or something like that. That would be too easy and Web 2.0. It uses frames (yuck!) to show the sidebar, you don't see the content of comments in hover, only when you click on the comment to get to the comment screen to view the comment. It's literally a pitfall and at least one company where I worked at rewrote the entire interface in ASP and .NET (I know, I know...) by parsing the Nagios log.

For that reason, it has always been a bit difficult (though possible) selling Nagios to the enterprise when your boss isn't tech savvy, and other programs, not much better or worse (OpenNMS for instance) find their way since they have a much better user interface.

A new fork of Nagios has begun with a PHP interface, calling Icinga. The point is to accept a lot of patches that were difficult to get into Nagios (that has only one actual developer - Ethan Galstad), and provide a beautiful web interface with Javascript. At least one Nagios community members sees the entire fork's point is the web interface, and assumes there's a good chance it will be merged back into Nagios, keeping the core as it is.

Apparently, there is a Perl Catalyst-based project to revamp the Nagios web interface, called Thunk. It is available on Github and also has a demo page. It's incredibly fast and seems promising. However, there is no website for it. Currently the only face it has is the Github page which seems generic and perhaps non-welcoming for some people who consider using it. You can also view the demo, but it still has the basic dull look of Nagios' oldschool interface.

Another new project relating to Nagios is Shinken, which is a Python rewrite of Nagios. Apparently someone thought that Nagios is great, except it's written in C, which makes it a barrier to include other peoples' work. I personally disagree for a variety of reasons which I won't go into. What does seem interesting is that Shinken is very welcoming, even though (at least to me) it seems like an exercise in futility. I'm assuming it will rapidly develop a stable core userbase, and the website is to thank, IMHO.

One more issue to note: Ethan Galstad is now developing Nagios XI, an enterprise solution. The "solution" boasts a new web interface, which I suspect will be the leading selling point of it.

So:

Nagios has a terrible interface.

Nagios XI has a pretty interface (and a free iPod Touch with every purchase, according to the site).

Icinga say they will put more patches in that were rejected or ignored for Nagios (but will keep core compatibility). However the strong point of Icinga (and where most efforts are now going to) is the web interface.

Thunk attacks the interface problem directly, but doesn't seem it will be adopted much because it doesn't even have a face.

Shinken tries to replace Nagios by rewriting the core to be in Python, but the best reason to adopt it (as with OpenNMS) is the web interface.

Now, of course there's a difference in Python, Perl, C and all that. However, in marketing, the selling value goes to the more presentable. In a System Monitoring conference I would have a pretty difficult time selling anything with just a Github page. It all boils down (in marketing terms) to the interface.

More design love please...

2010-02-03T09:08:52+00:00

This is a follow up, or rather extension to xSawyerxs Marketing the Entire Box (including the wrapper)

I completely agree, design is the first thing people notice about a project. If you do not put effort into the look of your projects web page then whether you like it or not it will have an affect on the perception of your project (and indirectly on Perl).

If someone (my list of projects is way too large at the moment unfortunately) has time I think there are two projects which would make a massive difference.

Provide a Creative Commons licensed design(s) that anyone can use for a Perl project. Lots of us are not designers, so even if we'd like to make our projects look better we don't know how!
See if it is possible to get CPAN redesigned, this is already the home to thousands of Perl projects, so improving this will have a massive impact.

Many of the Apple applications websites have a similar feeling (simple and clean), this is because they all use iWeb. Good design does not mean lots of design. It just means a clean page, and separating content under headings (rather than having a single page with everything just listed).

Marketing the Entire Box (including the wrapper)

2010-02-02T13:35:49+00:00

For a long while now I've been wondering about some observations I've made of Perl, Ruby, Python and PHP in marketing terms. I'm going to discuss them here in detail, and I hope it will gain us some insight into better marketing understanding or at least not bore anyone.

I've understood through the years that projects with beautiful websites have a better chance of getting picked up by users (even when the project itself is purely command-line) and definitely gives much more credit to the encompassing layer (like the programming language itself).

I've also noticed that a rather large portion of Ruby-related projects have very beautiful websites. You might have noticed this yourself as well. I tried to understand how this situation occurred and had a few discussions at $work with people whose opinion I appreciate, one being a Python and PHP programmer (or, as I like to see him, a programmer who knows PHP) and the other being a Ruby, RoR and overall Javascript whiz.

At first, my thoughts were that most Ruby programmers major in Rails, so they're into web, while the major web frameworks in Perl (Jifty, Catalyst, Mojo, Mojolicious, Mojolicious::Lite, Dancer, CGI::Application, etc.) only flowered in recent years, which could lend the thought that most Perl programmers aren't web-oriented or web-aware. I mean this obviously as "aware" in higher extent than most users. As mst eloquently expressed, we're still into IRC (me included, of course).

Quickly that theory was squashed when I entered the Django website. For a web framework, it is horrible. Especially for a web framework that speaks of cleanliness. The website design is horrendous. Same goes for the HAML website. PHP as a very successful web-oriented language shows interesting results. Most PHP programs/scripts come in various index sites (much like oldschool PERL scripts), few PHP frameworks have websites and fewer have beautiful websites. Most of them are awful. Frontpage awful.

So apparently, dealing with web doesn't mean you create beautiful websites. Yeah, I guess anyone could say that, but to put it in better words: knowing web doesn't mean you do web. So what is the reason it is almost given that a Ruby project (as small as it might be) will have a website, and it will be beautiful?

A thought that formed rapidly through the conversations was that Ruby programmers see the marketing as relating to not just the product, but its wrapper. That is, that many Ruby programmers understand at a very core level (more than most programmers - at least me) that the website which shows the project is the actual wrapper of the project and is just as important, if not more so. When I thought of a "nice wrapper", I just thought of a detailed POD and --help output. Obviously, this isn't the case for many Ruby programmers. Those Ruby programmers think "I cannot ship this application without wrapping it nicely, it is just incomplete."

I began to see the major difference between many Perl programmers and many Ruby programmers. I haven't ever written a website for a project I worked on. Evidently, perhaps many of them don't merit a website of their own, but perhaps some of them do! One of the prime examples for this, IMHO, is LessCSS which has a beautiful website for a rather simple filter, which could definitely be accomplished just as well in other languages such as Perl. I thought how simple it would be to write a version in Perl, stepped up to CPAN and here is CSS::LESSp and Apache2::Filter::LESS::CSS. I have to say that CPAN is indeed comfortable for me, but as a design, it isn't very attractive or compelling. CPAN is an index site, not a front page wrapper for your project.

When I see the LessCSS website, I'm seeing beauty which attracts me not only to LessCSS (which I don't really have any use for right now), but also (and this is key) to Ruby itself.

Returning to my quest, I had already sat down with our resident Ruby/Rails expert and a cup of tea and flux-seeds crackers. What I still wondered about was how can a Rails programmer know Ruby, Rails, Javascript (usually) and web design all at once and on such a high level for each to produce such beautiful websites. It's definitely not common, or is it?

We've laid the foundation that many programmers use prepared (sorry, there is no "pre-prepared" in English) templates and layouts, or hire an actual professional web designer. Hiring a web designer is not a cheap cost. Why are these people spending good sums of money on professional web designers to create (usually static) websites for such small programs? I couldn't understand it.

Then he mentioned some conventions he went to. When someone shows her peers and future employers projects she has done, the first impression is on the looks of things, not the sound or ability of them. If it's beautiful, she got to first base. If it works much better, but it doesn't even have a face, it stands much less chance of even getting a shot. This is what I was missing.

Marketing my project is not just marketing a code that does a task, it's marketing me!

Dancer doesn't just market a small web framework, it markets Perl and Plack (the encompassing technologies), but also the developer(s) of Dancer. Same goes for Catalyst, same goes for Moose and KiokuDB and Module::Starter and Test::More and on and on and on.

A response I'm expecting is "well, obviously!" and my answer would be the same, I do know it, however Understanding it is much more important than knowing it. To understand it means to write a website for whatever project I have done which I think can elevate me, and which I think can elevate Perl, or some other encompassing technology (like Moose or Dancer).

This morning I imagined having a space for project websites. Actual beautiful static website for each Perl project. I don't have the time to set up an infrastructure for hosting and all that. However, I did purchase PerlProjects.net and I will be giving NS hosting for free to anyone who wants to set up a website at <YourProject>.PerlProjects.net, no matter how small. Also, I want to set up a main site at PerlProjects.net which will be an index of... Perl Projects.

I do believe that having individual beautiful websites for projects (even the small or relatively exclusive ones) will help market Perl better, and any technology (of Perl or not) you're using in your project. This will help boost interest and consequently the number of programmers who know Perl will go up and of course job offerings will follow. If we can present an image of "Perl projects come in beautiful wrappers", we can present an image of "Perl is beautiful", and that is what we should strive for - at least in marketing terms.

If you feel like helping with PerlProjects.net (design, design, design) or want your own subdomain and free NS hosting, let me know!

Cultured Perl Blog

2010-02-02T13:31:26+00:00

A couple of years ago I thought that one thing the Perl community was missing was a network of blog sites about Perl. I'm not talking about the individual blogs that are being shown off to such good effect by the Iron Man project, I'm talking about a set of multi-author blogs that covered particular facets of the Perl world. Something like a Perl-specific version of LifeHacker or BoingBoing. To that end, I registered a number of domains and set about installing Movable Type.

That bit was easy. That bit I can do. The next bit is harder.

The next bit involves getting authors interested in writing for the blogs on a regular basis. That bit I didn't do so well at and none of the blogs florished.

One of them didn't even get going. That was Cultured Perl. The idea behind Cultured Perl was that it would discuss Perl culture. That's all the non-technical bits of the Perl world. Perl Mongers, Perl conferences, things like that. I had a few authors signed up, but nothing ever really happened.

So why am I telling you this? Well, the Cultured Perl domains are up for renewal. And I'm trying to work out whether it's worth keeping them.

Would you be interested in reading a Cultured Perl blog? And would you be interested in writing for it?

Showing Perl on non-Perl conferences, getting money from TPF for swag

2010-02-02T10:57:12+00:00

On this weekend I'll be on FOSDEM along with several other Perl Mongers. In addition to just trying to enjoy the talks we'll be promoting Perl. I am really happy that The Perl Foundation was ready to provide us with $500 to buy conference swag we can give away.

Similarry, a month from now we will be present at CeBIT in Hannover, Germany where we will have a booth to promote Perl and Perl based projects. I've just heard from Karen Pauley that TPF agreed to provide us another $500 for further materials to give away on CeBIT.

This is awesome and this probably means that if you are ready to talk to people on non-perl related events telling them about Perl you could also get some money from TPF in order to have some hard material to give away.

For FOSDEM we are preparing some round tuits and a postcard listing the Perl events in Europe that will take place in the coming months.

For CeBIT we are also preparing some further marketing materials - more business oriented - that we will be able to hand out.

There are further ideas to prepare various fun promotional materials one can give away during a chat with a fellow developer and we have also discussed that we should have a Linux liveCD full of Perl applications ready to be used by anyone. (Also include Windows binaries in the form of Strawberry Perl for those using Windows)

Of course almost none of the the above is my work. There are several people involved in the preparation: Rich Sands organzied the tuits and the postcards for FOSDEM, Salve J. Nilsen is working and on community intro cards, Renee Backer is working hard on the materials for CeBIT just to thank a few of them.

We are also expecting some help from the Enlightend Perl Organization (EPO) as they are preparing beer-mats to give away.

Getting involved

Most of the conversation about this is done on the events mailing list and some data is collected on the TPF wiki. If you would like to help in preparing the materials or if you'd are planning to attend a non-perl event and would like to have some fun or business oriented material to give away, please join us on the events mailing list to discuss the details. You can find information on the TPF wiki.

DBIx::Class 0.08001 released (and a podcast)

2010-02-01T18:15:28+00:00

For those of you who're interested in me demonstrating just how badly I interview, you can hear a perlcast podcast interview with me about DBIx::Class which was recorded last month. Better still, after 7 dev releases, four of them RC grade, a lot...

Read and post comments | Send to a friend

$obj->blessed

2010-02-01T18:10:48+00:00

I've been meaning to write about this gotcha for a long time, but somehow forgot. This was actually an undiscovered bug in Moose for several years:

use strict;
use warnings;

use Test::More;

use Try::Tiny qw(try);

{
    package Foo;

    use Scalar::Util qw(blessed);

    sub new { bless {}, $_[0] }
}

my $foo = Foo->new;

is( try { blessed($foo) }, undef );

is( try { blessed $foo }, undef );

done_testing;

The first test passes. blessed has't been imported into main, so the code results in the error Undefined subroutine &main::blessed.

The second test, on the other hand, fails. This is because blessed has been invoked as a method on $foo.

The Moose codebase had several instances of if ( blessed $object ), in packages that did not import blessed at all. This worked for ages, because Moose::Object, the base class for most objects in the Moose ecosystem, didn't clean up that export, and therefore provided an inherited blessed method for pretty much any class written in Moose.

I think this example provides a very strong case for using namespace::clean or namespace::autoclean routinely in your classes.

To cover the other half of the problem, the no indirect pragma allows the removal of this unfortunate feature from specific lexical scopes.

FOSDEM

2010-02-01T13:12:23+00:00

This weekend is the annual FOSDEM conference in Brussels. I really enjoy FOSDEM but, for reasons I don't really understand, this will be the first time I've been since 2005. It will also be one of the rare occasions where I attend a conference without giving a talk - the organisers turned down my proposed talk on Modern Perl.

I like FOSDEM because it's not just a Perl conference. It's about the wider open source movement. In fact Perl is a really small part of of the conference. In many years it has been completely unrepresented. One of the things I mentioned in my "M Word" talk at the London Perl Workshop was that Perl needed to be better represented at non-Perl conferences. With that in mind, the Perl Foundation has booked a stand at the conference and various volunteers (including me) will be there telling people about how wonderful Perl is.

The main driver behind this push to get Perl represented at other conferences has been Gabor Szabo and he'll also be at FOSDEM giving a couple of talks. One is a lightning talk introducing people to Padre. The other is about packaging CPAN modules for Linux distributions. Those of you with long memories might remember me talking about this at YAPC in Copenhagen. I'm hoping that attending Gabor's talk will galvanise me into having another go at my project to automatically build RPMs of many more CPAN modules than are currently available.

So, as you can see, there are plenty of good reasons to be at FOSDEM this weekend. And that's even before considering that it takes place in one of my favourite European cities. I might even treat myself to a Kwak in one of the bars on the Grand Place.

If you're at FOSDEM next weekend, please stop by the Perl stand and say hello.

Rainy Day Outage (update four)

2010-02-01T07:08:53+00:00

Good news everyone!

The few remaining services that were out should be back shortly.

We moved the failed server from Robert's house to Ask's office today and finally got enough parts replaced that the server is running again. As we hoped all data is intact. It's currently copying all the data off to a couple of 2TB disks. Actually, I just checked and the only missing services right now are some pm.org sites and some of the historical mirrors /archives (very old versions of perl, some cpan-testers mails).

The pm.org sites should be back by the morning; the rest might need a few days for us to do the sneakernet thing to get the data to a fast enough network connection that copying hundreds of GB isn't too slow.

Test Automation using Perl classes

2010-02-01T04:28:38+00:00

The conference season is warming up and so I'll start offering my Test Automation using Perl training class so I can have an excuse to go to the workshops and conferences. Here is the schedule:

March 8-11, Berlin, Germany, after CeBIT where we have a Perl booth
March 15-18, Tel Aviv, Israel
Apr 13-16 Vienna, Austria, right after the QA Hackathon
May 11-14, Stockholm, Sweden, (no workshop here but I cannot make it to NPW in Rekyavik on 1-2/May)
June 1-4, Stuttgart, Germany, just before the German Perl Workshop
June 15-18, Columbus, Ohio, USA, the week before YAPC::NA

The first one in Berlin is a bit tight in the schedule now but I hope there will be enough people interested to warrant opening the class.

The class is 4 days long. Details can be found here: Test Automation using Perl. For pricing please contact me directly via e-mail.

Related announcement in German

Test::Class::Most

2010-01-31T19:25:10+00:00

Test::Class::Most.

hmm, Nordic Perl Workshop...

2010-01-30T21:20:43+00:00

It came to my attention via brian d foy's blog post that the next Nordic Perl Workshop will take place in Iceland - which sounds very interesting. At the same time I decided that I need to take a longish break after working very hard for the last two years. So I will combine my holiday with NPW!

My current plan is bike from Vienna via Prague to Berlin, spend a few days there to visit some friends, take a plane to Rekjavik, attend NPW, and then either cycle a bit more through Iceland or treck a bit (probably combined with public transport). Then take the plane back to Berlin and a train back to Vienna.

The plan still has some issues:

Is it totally insane to cycle through Iceland in May (if any Icelandic person could comment on this, I'd appreciate that..).
Maybe cycling from Vienna to Berlin takes to long or is boring. Then I could take the train to Berlin, and cycle from there to Copenhagen, fly to Iceland, and go back to Berlin.

Anyway, I'm really looking forward to this!

Testing with PostgreSQL

2010-01-30T16:22:57+00:00

My new personal project has a PostgreSQL database. Here's how I'm handling testing.

Rainy Day Outage (Update Three)

2010-01-30T08:26:20+00:00

Not a lot of new news.

We got a new motherboard for the system, and have installed it.... but it's not working. Which leads us to believe that its actually the powersupply thats shot. We're on the hunt for a compatible powersupply locally, but its probably not worth pouring more money into this box, so we're working on Plan B, which is to get the 3ware RAID controller installed in another system and get the data off the drives. This is complicated by the fact that we need to connect eight drives at a time and that the controller is PCI-X, a no-longer popular form factor.

If anyone in the Los Angeles area happens to have an idle server we could borrow for a few days with a PCI-X motherboard and at least 8 PATA bays, we'd love to hear from you. Drop us a line to webmaster at perl.org.

(We're trying to avoid Plan C, which involves taking images of all the drives one at a time and then gluing them together.)

We've initiated a plan to get rt.perl.org up and running again ASAP, and are dedicated to getting the pm.org (and other) data off these drives.

Writing a perl REPL part 3 - lexical environments

2010-01-29T22:14:55+00:00

(this is a continuation of the series started in this post, so you may want to start there) New tool for today: Rocco Caputo's amazingly handy Lexical::Persistence module. We're going to use this to persist lexical variables - i.e. those declared...

Read and post comments | Send to a friend

Devel::REPL part 4 - script options, rc files and profiles

2010-01-29T22:14:52+00:00

Ok, so this post is spectacularly late. Work's been busy, my business partner got married and I got to annoy all sorts of brilliant people at OSCON (my slides are up if you care). And I -did- actually write the code that this post depends on a whi...

Read and post comments | Send to a friend

FreeBSD hates Capture::Tiny

2010-01-29T21:59:15+00:00

At least, FreeBSD 7.1 does.

Could anyone with access to perl on FreeBSD 7.1 please test Capture::Tiny? It now passes test on most platforms, but FreeBSD 7.1 still sometimes fails.

I suspect the FAIL reports I’m seeing are something specific to either CPANPLUS or BinGOs’ smokers, but I need more data points (PASS or FAIL) on that OS to get a better idea of what the issue is.

Thanks!

Oh, and if you haven’t seen the alpha CPAN Testers Analysis site, check this out: Capture-Tiny-0.07 failure analysis. Cool stuff!

Perl 6 Design Minutes for 27 January 2010

2010-01-29T21:00:38+00:00

The Perl 6 design team met by phone on 27 January 2010. Larry, Allison, Patrick, and chromatic attended.

Larry:

tweaked definition of when a series operator is considered infinite
nailed down more list assignment semantics with respect to interators
clarified how ($a, $b, @a) = 1..* works
KeyWeight deletion criterion kept consistent with other KeyHash types
negative keyweights are allowed to fail at pick time
"mostly eager" now assumes unknown closure generators are probably infinite
random whackage on List, Seq, Parcel, Capture, Iterator, Nil etc.
List is now simply the iterator role, and doesn't do Positional
Seq takes over Positional duties for reified (or reifiable) value lists
think of Seq now as a constant Array (but also lazy like Array)
Iterable now means you can ask for an iterator, but doesn't do List
Array, Seq, etc do Iterable, but not List
only actual iterators do List
Nil is defined as a suitable sentinel for both list and slice iterators
continued to rethink that with pmichaud++ et al
we'll probably end up with an EMPTY special exception object to be the iterator sentinal
proposed an E operator to go with it to make testing for EMPTY across multiple iterators very fast
other than that, mostly just bug whacking, no major refactors
still thinking about doing real LTM for STD
did lazify Cursor's fnum->fate translations for shorter LTM candidates in preparation for smarter LTM
we don't need special objects for the items that get matches
we do need to think more about the hyper cases
how to do list processing using balanced trees of delegated sub refs
don't want to build in serial assumptions where we don't need them

Patrick:

made the Rakudo #25 release last week
it was much easier to make the release than explain what we were planning to do instead
also working on iterators and lists
NG branch is blocking on that
worked on the design in my head for three weeks
realized that we were doing iterators completely wrong the other night
Larry's making some useful changes to the spec in response
there are still some unclear spots in the spec
we need an implementation to figure those out
my biggest question is the relationship between List, Parcel, Itertor, and array
as of this morning, I think I have it
that code seems to be working and efficient
so far it's working well
continuing with that
wrote a very short range iterator prototype that colomon has used
also write a map iterator that works
coming up with examples for the zip operator was nice
good ideas for what we need to be able to do
objects that can iterate have a .iterator() method
to interpolate that into a list, .list() returns a flat Parcel for that iterator
Parcels know how to generate Iterators
those know how to handle Iterators of Iterators
I suspect that's how we do hyper iteration
change Parcels to understand that
adding pieces back into the ng branch
next I have to fix slurpy parameters
many of our builtins need that
need to figure out Jonathan's code to do that
after that, I'll do arrays
that should remove the blockers on the ng branch

Allison:

working on Pynie
Francois has helped greatly to update it for Plumage

still working on the TT #389 fix
think I have the right design, just need time to implement it
working on a potential new time for #parrotsketch

Allison:

thinking about hackathons
would be nice to have a Rakudo hackathon at YAPC::NA

Parrot will come up; didn't it come up about half the time last year?

Patrick:

it was all Parrot

Allison:

you'll have an influx of Rakudo interest two months after Rakudo Star

Patrick:

probably will have one before then
but can tell people "Go to YAPC; we'll show you how to help in person there"

Perl QA Hackathon 2010 Venue & Vienna.pm Sponsorship

2010-01-29T16:49:24+00:00

The Perl QA Hackathon 2010 will take place in the lovely MetaLab , a grass-root non-profit hack-space in the middle of Vienna (more on Wikipedia). There will be lots of space, workplaces, sofas, wireless and wired network and a big fridge full of drinks.

Vienna.pm has also proud to announce that we will sponsor the hackathon with 10.500 Euro. We will reserve a small part for catering during the event, but most of the money will go into paying for transport and hotel of invited guests.

If you want to hack on a QA / Toolchain project, please add yourself to the Attendees page of the wiki.

Thomas Klausner,
on behalf of Vienna.pm and the Perl QA Hackathon 2010 team

Planet Perl

Release Announcements.

Problem with Strawberry + Padre 0.56

Benchmarking MooseX::Method::Signatures

Devel::NYTProf - Perl profiling links needed

Can you help identify ambiguous CPAN distributions?

Why Ruby is prettier and Padre changes the Perl community

Building RPMs from CPAN Distributions

Spock and Data look at CPAN

Help keep the world safe from SQL injection

An English-only Planet Iron Man

CPAN Testers 2.0 end-January update

Perlbuzz news roundup for 2010-02-01

Using Devel::REPL to develop a mail cleanout script

CPAN Testers Summary - January 2010 - The Wedge

microsoft customer service madness

Perl for Windows statistics

On Nagios, Thunk, Shinken and wrapper included marketing

More design love please...

Marketing the Entire Box (including the wrapper)

Cultured Perl Blog

Showing Perl on non-Perl conferences, getting money from TPF for swag

Getting involved

DBIx::Class 0.08001 released (and a podcast)

$obj->blessed

FOSDEM

Rainy Day Outage (update four)

Test Automation using Perl classes

Test::Class::Most

hmm, Nordic Perl Workshop...

Testing with PostgreSQL

Rainy Day Outage (Update Three)

Writing a perl REPL part 3 - lexical environments

Devel::REPL part 4 - script options, rc files and profiles

FreeBSD hates Capture::Tiny

Perl 6 Design Minutes for 27 January 2010

Perl QA Hackathon 2010 Venue &amp; Vienna.pm Sponsorship

Perl QA Hackathon 2010 Venue & Vienna.pm Sponsorship