walkah

Using Drush to Mass Delete Comments

2012-04-13T00:00:00-04:00

Like many kind, good-natured geeks, I (yes, I was talking about me) host a number of small sites for friends. Many of those sites run Drupal and several don't get a lot of attention. That is, until spammers find their way around Mollom. I've seen this happen a few times, but most recently on a Drupal 7 site. In this particular case, the spam comments had been trickling in over a few weeks undetected. Following a burst of recent activity (enough to notice via monitoring), I checked in to find > 60,000 spam comments.

If this has ever happened to you: you're not alone. At this number of comments, using Drupal's interface (50 comments at a time) isn't really usable. Also, these things tend to happen in bursts - so chances are good there's a block of comments that are all spam (i.e. there haven't been any legitimate comments that you want to save since it started). So, I whipped up a small script here:

<?php $first_comment = 38898; $num_delete = 50; do { $cids = db_select('comment', 'c') ->fields('c', array('cid')) ->condition('cid', $first_comment, '>=') ->range(0, 50) ->execute() ->fetchCol(); print "deleting ". count($cids). " comments...\n"; comment_delete_multiple($cids); } while (count($cids) == $num_delete);

Here's how to use it:

Place the code in a file (say, comment-rm.php) in your Drupal directory.
Find the cid of the first spam comment (exercise left to the reader) and set $first_comment to that value.
Run drush scr comment-rm.php and go grab a coffee.

Hope that helps someone, but at least now I can find it again next time.

Beware: This deletes comments forever, be careful.

Why Vagrant

2011-07-05T00:00:00-04:00

Few things are as frustrating in the life of a (web) developer as those unfortunate times when, despite passing QA and testing, your latest feature, bugfix or completed task stops working when it gets to production. Nobody likes to have the sweet feeling of "go live" victory wasted. While any number of things can cause the embarrassment and wasted time, one way to significantly reduce the number of potential errors is to keep your development, staging and production environments as close to each other as possible.

Lately I've been dealing with my own special, self-inflicted challenge: I have two machines (a thinkpad x201 and an older, pre-unibody macbook pro) that run two different operating systems (Ubuntu 11.04 and Mac OS 10.6, respectively). Now, why I do this is probably a longer discussion, but I do - and I like it. Just to add a little more (minor) variation, my personal projects, experiments (and in fact this blog) all run on linode instances (running Ubuntu 10.04). The end result is, my development and "production" happens across 3 different OS versions. Standard package installations of development tools (regardless of which technology I'm exploring at the moment) are rarely ever the same release version. Throw in some subtle and potentially maddening differences between linux and os x (case insensitive filesystem, what?) and I've lost too much time debugging my own fixes.

As it turns out, I do an okay job of emulating my own little development team (except in productivity, of course).

In a more real world scenario: I have been doing some work lately with Myplanet Digital (a fun team located in Toronto - and they're hiring). One of their portfolio client projects is a fairly large, complex Drupal implementation. The production version is hosted on Acquia cloud hosting, they host their own QA / CI / testing infrastructure, and their (ever-growing) development team uses a mess of mac os x and windows versions. They are continually looking for ways to streamline their development process - and avoid any needless debugging time.

Vagrant Chefs to the rescue!

I've developed a growing interest in two tools to help solve this dilemma: Chef (for environment management and provisioning) and Vagrant (for local, VirtualBox based virtual machine management). Vagrant is super easy to get running, like it says on the home page:

$ gem install vagrant
$ vagrant box add base http://files.vagrantup.com/lucid32.box
$ vagrant init
$ vagrant up

Assuming you have a reasonably functional ruby environment and have a recent version of VirutalBox installed, that's all there is to it! Chef (for provisioning), however, has taken me a bit more time to get my head fully around. You can see my experiments happening live(ish) on my github account.

But I don't use lucid32!

One interesting thing about the Acquia managed hosting platform, is it's all built using Ubuntu Hardy (8.04). In Internet years, it's rather old but has some significant differences from Lucid (and most current packaged versions of the LAMP stack) - PHP 5.2 (vs. 5.3) and MySQL 5.0.x (vs. 5.1.x). To do this right - to actually replicate the production environment for development - it's important to have these versions in sync. There are subtle differences between these versions that can trip you up. Enter veewee. A few weeks ago, I provided a pull request for veewee that added a "hardy32" template - for building a vagrant box with the same version of Ubuntu found on Acquia hosting. For the lazy, I've posted the base box to my dropbox account.

Overall, I've found it requires a bit of tinkering (I'll try to share more as I go), but the result is that I can deploy code with a bit more certainty and that is worth it. I'm here tinkering, so you don't have to.

I should also mention, that all Drupal developers should checkout the vagrant project on drupal.org for a nice general solution for Drupal apps.

Dev Ops!

Every Drupal Site is an Install Profile

2011-05-20T00:00:00-04:00

I've been working with this hypothesis while working on Drupal 7 sites. I don't think it's necessarily revolutionary, but I would like some feedback. Currently, I'm approaching every Drupal site build as its own install profile.

Back when we were running Bryght, we recognized the need for "Installation Profiles" as a way to focus the highly configurable, but largely baffling initial experience with Drupal into making sense. We had lofty dreams of catering to various verticals with streamlined, elegant experiences that took the immense power of Drupal and made it make sense for new users. That was Drupal 4.5... we were maybe ahead of our time. Needless to say, a lot has happened with Drupal since then.

With the release of Drupal 7, one of the things people aren't talking about as much are the vast improvements to install profiles. For the first time ever, Drupal core ships with more than one install profile, which has also meant that for the first time people are consciously aware that such things actually exist. In D7, install profiles behave much more like other Drupal packages (i.e. modules and themes). They have .info file and .install files. They are far easier to create than ever before.

Clearly, I'm not the only one who sees the importance. Development Seed and now Phase2 Technology have invested a lot in install profiles such as OpenAtrium. For creating Drupal products such as Atrium, install profiles are important and central. What I am talking about, however, is to create an install profile for every Drupal site you build. In updating this very blog to D7, I created an install profile for my site to test (and tinker with) my theory.

The theory

What is a Drupal Site? Well, once you've determined a version, core remains unchanged across most sites (you know, don't hack core and all). So what makes a Drupal site your Drupal site is: 1) a theme 2) your selection of contrib modules and 3) any custom code / modules you may have written. If we dig a bit deeper, the things unique about a Drupal site are typically: 1) a theme 2) any custom code and 3) the list of contrib modules / libraries in use. It just so happens that an install profile can nicely encapsulate this information for us. So why bother?

A Drupal site rarely only gets installed in one place. Best practice suggests that we use development, staging and production for our websites. If we work on a team, "development" will actually be a separate install for each developer on the team. Being able to reliably install and replicate your site will actually make things much easier - whether it's just you or your whole team.
Drush Make works well like this. Your profile can have its own makefile (as openatrium does). Drush make will recursively make install profiles, and will add all contrib modules and libraries into your profile's directory. Maintaining nice separation.
Make the most of multi-site. Since your core version remains constant (at least for major versions) across all the sites you work on, why do you keep installing core? In my development environment (which I will write more about soon), I have 2 virtual hosts d6.dev and d7.dev. All of the sites I'm working on fall under one of the two. Thanks to drush make, I can have a single makefile for each major version that recursively grabs each profile (i.e. site or group of sites) that I'm working on for that version. The same can be used in production or not - the install profiles can move independently to production.

The end result is, I have very small custom repositories - containing usually a drush make file, a custom module or two and a custom theme. Checkouts are a breeze and I have a clear manifest of any other other code I'm using.

Objections

Now, historically this isn't how people have approached Drupal sites. Historically it's not even how I've approached it. Generally, we have single repositories with all of core, our modules etc. Generally, a single repository containing all of the core, contrib and custom code - for years in subversion and now folks are transitioning to git. An argument for continuing this way (I received from webchick herself), is that a single repository makes it easy to see things like if someone has made changes directly on production (?!?!) or that just by having a single checkout all developers / installations have the exact same code.

While I think "hot fixes" in production are a bad idea (no matter how small the site), the latter objection (exact same code) is worthy of a little more discussion. Keeping things in sync with Drush Make is a bit more work (always pin your versions!), I think it's a worthwhile habit to establish. Yes, git is fast enough that those long, painful svn checkouts are largely a thing of the past so having all of your code in a repository isn't as punishing. For me, the hosting considerations above are significant, but another thing came up for me recently:

If you build a lot of Drupal sites, chances are good that there is some overlap. Say there is a patch that you need for a favourite contrib module, drush make means that you can explicitly apply the patch (and maintain it outside of your repository - or pull directly from the issue queue). Similarly, using independent feature module features allows for similar mixing and matching. Each module (or theme) should have a single, canonical source and a Drupal site is simply the combination or collection of them.

It seems to me there are real gains in being explicit with makefiles, being smart about hosting and re-using core, and being modular in our repositories (using drush make to pull it all together).

Am I crazy?

Ch-ch-ch-changes

2010-01-06T00:00:00-05:00

This year promises to be an exciting one - and one that proves to be full of changes galore. As I mentioned in my last post, much of my 2010 change began late in 2009.

Cue the Bowie...

Leaving Lullabot

Although it's been largely quiet (and really not a huge deal), before the rumours spread too far: I've left my position as the Director of Education for Lullabot. I'm leaving behind a totally awesome team and a wonderful job (in the midst of a recession). Why on earth?!

Let me be very clear: Lullabot isn't in danger, stopping Drupal training, nor is there any backroom drama. The 'bots are wonderful people and chances are very good that we'll continue to collaborate in the future (at the very least, there's still hugs).

This was a very personal decision - and one that was a long time coming. For the morbidly curious, it boils down to three things (and those of you who know me well, know it always comes down to three things):

Travel: Anyone who is friends with a 'bot on Dopplr or Tripit knows that the job entails a lot of time on the road. With over 230 days on the road in the past 2 years, I needed and my kids deserved a break. While we (Lullabot and I) largely worked around this - it's still just part of the gig.

Drupal: I stood up in front of a rather large group of Drupal folks almost a year ago and explained why I hate Drupal - so it's obvious, right? While I feel the points I tried to make still face the community at large (such as smallcore/drupal is not a product, or even rethinking the maintainer structure), I don't actually hate Drupal (as those of you who grok sarcasm might have noted).

However, Drupal has been my full-time job for 6 years. In that time, the community (and the software) has grown and changed considerably. It has been an amazing ride. As Dries mentioned to me on the phone a few weeks ago, "once a Drupal guy, always a Drupal guy". This is probably true - I have no intention of leaving the community, but I am ready for some new challenges.

Open Web: One thing people may have noticed is that when I have had the chance to hack on Drupal lately - it tends to involve "open web" or "open standards" implementations (notably, OpenID etc). Many folks have also noticed that my contributions have trailed off lately. When your "after hours" time starts including more things like "sleep" - your after hours projects take a hit.

I'd like to get back to building cool, new stuff. While I certainly get a lot out of teaching people how to make the most of the tools available, I'm passionate about building the next tools (which doesn't exclude Drupal). These are interesting times on the internets, I wanna have my nose in the middle of it.

Now what?

Officially, I will be freelancing (technically have been for a few weeks). I've already got some interesting things lined up that I'm excited to start talking about soon.

Open, Social for the rest of the web

2009-04-22T00:00:00-04:00

This past weekend, I had the privilege of being one of the chosen attendees for Social Web FooCamp. Needless to say, I was flattered and had an amazing time (thanks again, @daveman692 and @davemorin ) . One thing, however, became very apparent: the conversation, currently, is being dominated by the 'big players' (Google, Yahoo, Facebook and Myspace predominantly). In several discussions I found myself increasingly dropping the phrase:

... on the rest of the web

the big guys

First off, this is not a critique of the Google's and Facebook's of the internet. They are incredibly valuable to the growth of the openweb. The fact that Google, Yahoo and Myspace all three have various OpenID and OAuth initiatives in the wild and are actively pursuing additional ways to open their data is awesome (and Facebook wants to get there). It helps raise awareness and bring (slash confirm) "legitimacy".

The big guys also have resources. They can attend the conferences (and camps!) and have dedicated resources to write the standards, participate in the discussions and help shape the future.

However, they are only part of the discussion.

perspective

The issues the major providers face are different from the rest. They have a few sites with large numbers of users (hundreds of millions). Out here on the rest of the web, we have millions of websites, each with a "small" number of users (hundreds or thousands). We all understand the necessity for open data, identity, standards and protocols, but our reasoning tends to be slightly different.

The big guys recognize the benefit of exposing their data and most are providing OpenID and various levels of OAuth. How many are consuming it?

Sure, the big players want to be the primary authority for your identity and your information. In some cases, it is their business. But, rather than ranting against 'the man', I ask: have we - the rest of the web - given them a compelling reason to yet?

open source platforms for the open web ###

It's one thing for a major site (with hundreds of millions of users) to act like a silo, but on the rest of the web it amounts to isolation.

Those of us working on open source web platforms have an enormous potential for influence here. Implementing the various open standards "from scratch", while possible, is not realistic or even necessary. Increasingly, individuals have Wordpress blogs or perhaps their company, organization or club has a Drupal site. Web developers are increasingly turning to these platforms, or development frameworks such as Rails and Django. These platforms all have a real opportunity to bake in implementations of these open standards. The DiSo project offers a central place for co-ordination around these efforts.

We have data - gobs of it. We also, collectively, have the users and, in most cases, have more authoritative information about them (we know ourselves, our employees and our members).

We - the rest of the web - need to join the conversation: attend the events, participate in the mailing lists, and build the code to power the open, social web.

Why I Hate Drupal

2009-03-09T00:00:00-04:00

I'm back home from a truly spectacular DrupalCon DC and have been reflecting this morning on some of the feedback from my talk: Why I Hate Drupal.

See the slides and watch the video

I first got the idea for this talk several months ago watching the DjangoCon 2008 keynote Why I Hate Django by Cal Henderson. I had several ideas for things to address, but aside from the session description I intentionally said very little about my talk publicly. This, of course, lead to some interesting speculation and negative feedback. All part of the plan.

As it turned out, I was not lynched and nothing rotten was thrown.

What I was not expecting (and what the video doesn't capture), though, was all of the interesting discussion that followed. I was overwhelmed by the positive response and the number of people who agreed with several of the points I tried to make:

Drupal is not a product. To grow into a "movement", we should focus on becoming a better platform, adopt some better practices around development, be a better framework, and create more space for the creation of "products" (install profiles, etc) on top.

What do you think? How to we "fix" this project?

DrupalCon is Coming with lots of OpenID

2009-02-27T00:00:00-05:00

In just a few days, most of the drupal community will be headed to Washington, DC for DrupalCon. As the conference draws closer, I always get excited to see friends I don't get to see and share exciting ideas, but this time there is a lot of growing interest and activity around OpenID.

As has become a bit of a tradition, I'll be giving my 4th OpenID talk. This year, I'm hoping to focus a bit on the exciting new developments from the OpenID community and looking at some of the things being built on top of OpenID (like the OpenID/OAuth hybrid model and the DiSo project).

Also, Chris Messina will be one of the keynote presenters - also talking about online identity. We had Chris on the lullabot podcast this week - be sure to check it out!

Finally, for those of you coming to DC - I'm going to round up interested parties on Saturday for an OpenID code sprint. Hope to see you there!

image module: end of an era

2009-01-30T00:00:00-05:00

A few weeks ago, I very quietly made a (personally) significant move in my Drupal life. I "officially" transferred the project ownership of the drupal image module to Daniel "sun" Kudwien. Truth be told, I haven't been the primary maintainer of the module for over a year. That job has been admirably handled by drewish. However, both drewish and I have a lot else on our plates and one of Drupal's most used modules has not gotten the focused attention it deserves.

Now, as Angie will tell you, Daniel is a fantastic Drupal contributor - worthy of the praise he receives. But, I'd like to give a personal shout out: he has helped to take the image issue queue from over 12 pages long down to 3.

Nice work, sun. The community thanks you :-)

happy 8th birthday, drupal!

2009-01-15T00:00:00-05:00

Man, we're getting old!

Today (January 15th) is the 8th anniversary of the day Drupal 1.0 was released. Although Dries had no idea at the time - it was a move that would not only change his life, but mine too...

January 2009 also marks the 5th anniversary of my starting to work on Drupal full time (after a few years of "hobby" involvement). My first project (at the time, actually a re-launch) still stands as one of my favourites: http://www.terminus1525.ca/ . Since then, Drupal has defined my career: from co-founding Bryght to my current life as a Lullabot. The community is home to some of my best friends and people I love.

Five years - full-time. No wonder I feel old.

Using Drupal or Look, Ma, I'm on Amazon!

2009-01-09T00:00:00-05:00

In my usual heard-it-hear-last style... I was out to dinner this week with my friend, Phillip, who was apparently unaware that Lullabot had written a book(?!?) and I was one of the authors.

Well, for most of last year (plus) most of us over at lullabot spent sleepless nights putting together Using Drupal. It went to press in early December, and is indeed on amazon and even on shelves.

I'm personally pretty proud of the book. It's the first Drupal book by O'Reilly and the first to take a comprehensive look at building a "real" Drupal site with heavy emphasis on CCK, Views and the rest of drupal contrib.

I have to say, too, that O'Reilly was a lot of fun to work with. There's a reason they have a reputation for having the top tech books. If you missed it, @eaton and I did a live webcast with O'Reilly which was their biggest ever. Kool-aid for everyone!

If you don't have a copy yet, what's wrong with you? ;-)

Done it With Drupal - Back to Life

2008-12-15T00:00:00-05:00

My last several weeks (months?) have been pretty darn hectic preparing for Do It With Drupal. Well, we've done it with Drupal - and it was awesome! It was really fun bringing the whole Lullabot team to make it happen. The speakers were awesome, the venue was great and the people rocked.

For those of you who missed it, the event was pretty well documented via the twitter backchannel and flickr photos. Thanks everyone!

I'm back home now... oh - and I moved! I'm now living in a beautiful house at St. Clair and Dufferin.

So it's time to settle in for the holiday season, finish getting unpacked and settled and enjoy a little breathing room. To kick it off, I'm heading to #hohoto tonight. Looking forward to partying with my local Toronto people!

Drupal Theme Workshop in Toronto

2008-11-11T00:00:00-05:00

I'm excited to announce that members of the Toronto Drupal User Group have put together an exciting one day, free workshop on Drupal theming.

I will be there to present in the afternoon. Please come out and say "hi". Also, Lullabot will be sponsoring lunch (thanks guys!).

It seems like Toronto has gone Drupal crazy lately. I am loving all the local events!

The event takes place starting at 10am on November 22nd, 2008 at the Centre for Social Innovation. Be sure to check the website for full details. See you there!

VectorLover Drupal theme

2008-10-31T00:00:00-04:00

Earlier this week I gave a free workshop for PHUG here in Toronto. We had a lot of fun and our scheduled one hour turned into 3 or 4 (oops!). I had a lot of fun with the group and the night received good reviews.

During the evening, we took a pure HTML and CSS design and converted it into a Drupal theme. The design is called VectorLover - freely available from styleshout.com.

I took some time this week to clean up our work, and am making "VectorLover" available for download here. Please contact me if you have any questions or comments. Enjoy!

(Note: Due to the license, this theme will not appear in the Drupal repository. Sorry!)

PHUG Workshop: Drupal Theming in an Hour

2008-10-22T00:00:00-04:00

October is a busy speaking month - but I'm excited to say I've been asked back to present another (free) PHUG Workshop on Drupal.

Back in April, I did an Intro Workshop. This time, I'll be showing off theming in Drupal 6:

In this workshop, we will show the process of taking an HTML & CSS design and converting it into a fully working Drupal theme. Along the way, we’ll look at the 3 main aspects of Drupal theming, some best practices and a few tricks. Drupal 6 makes the whole process easier than ever, so get started making your Drupal site look not like a Drupal site!

The workshop is Tuesday Oct 28 2008 @ 630pm at Seneca @ York Campus in room 2112. See the workshop announcement for full details.

FSOSS is back for 2008!

2008-10-17T00:00:00-04:00

One of my favorite annual free software conferences is back again! The 7th annual Free Software and Open Source Symposium is happening next week, October 23rd & 24th. I've spoken at this event 3 years in a row - it's a great event and local in Toronto!

I'll be running 2 sessions again this year:

Drupal and Views 2: Powerful Websites in Zero Lines of Code: This workshop will cover some of the power available to Drupal site builders thanks to the awesome new Views 2 module. 2 hours, 1 site, 0 lines of code. Rock!
Using Drupal: Community Powered Code to Run Your Site: This year's presentation will focus on the secret weapon of Drupal: the community. Drupal's success comes down to the power of the people behind it!

Check out the full schedule and register now!. Hope to see you there!

Drupal Meetup in Waterloo

2008-10-02T00:00:00-04:00

As Khalid announced, I'll be heading out to Waterloo, ON tomorrow evening to give an introduction to module development with Drupal. Khalid has asked me to brush one of my old favourites: Pants module. It's a nice, fun way to get your feet wet with module development.

It will be my first visit out to the (relatively) new Waterloo Drupal Group. I'm looking forward to seeing everyone and hopefully meeting some new faces!

For full details on the event, see this post and sign up!

For those of you here in Toronto who can't make the trip, I hope to see you at next week's Toronto meeting.

drupal at sxsw

2008-08-15T00:00:00-04:00

I'm certainly not the first to post about this, but it looks like Drupal will have a strong showing at SXSW Interactive 2009. There's even a front page post on drupal.org to bring awareness. A group of us from Lullabot went last year and we had an absolute blast. If you haven't been, sign up now!

The panel picker is live and I suggest you *all* vote for Drupal with Its Pants Off (you know you wanna). What's really exciting, though, is the long list of other Drupal panels on the list.

Looks like there'll be lots of Drupal in Austin next March... even http://sxsw.com/ is Drupal powered!

My hotel is booked. See you there!

DrupalCon Szeged is coming

2008-08-14T00:00:00-04:00

Sadly, I won't be there... but you should be!

I have to sit this one out due to scheduling conflicts, but the posted schedule looks really good. Of course, the Lullabot team members going are all over the schedule. Everyone should attend webchick's intro to testing and be sure to attend the testing party - she has promised chocolate!

Scanning through the schedule, it's nice to see that fellow Toronto Drupal Users Group member, Emma Jane, will be giving a presentation on Drupal for small business networks.

So, while my socks and I will only be there in spirit, you should be there in person. Register now!

Now running Drupal 6 and Views 2

2008-08-11T00:00:00-04:00

I finally "re-launched" my blog last night, after tinkering for a few months. I'm now running (at the time of writing) Drupal 6.3. More interesting, however, is that my site is almost entirely powered now by Views 2.

The biggest visible change is the home page. Inspired in part by Dave Shea's lovely blog, I wanted to make my front page shorter. So now, I'm displaying the latest full post, with 9 previous titles only. This is all done with views (using the awesome new "attachment" display type).

The other interesting bit is that I'm using the latest version of twitter module so that the "twitter" block on the right is actually views2 powered as well (and gets cached).

I'm sure I'll keep tweaking, but I dig it. How about you?

XRDS-Simple for Drupal

2008-06-04T00:00:00-04:00

Last night I put together a simple module for Drupal to implement the XRDS-Simple spec. It is based loosely on the xrds-simple wordpress plugin.

XRDS-Simple is an important piece of the DiSo project. From the XRDS-Simple spec::

XRDS-Simple provides a format and workflow for the discovery of resources metadata, and other linked resources. As web services continue to grow, applications utilize a wider range of web services and resources across multiple providers. XRDS-Simple allows providers to document their resources in a machine-readable way, which can be automatically discovered by consumer applications.

So, check it out: http://drupal.org/project/xrds_simple .