A threat analysis for computing systems is...

Systematic detection, identification, and evaluation of areas or spots of vulnerability of a facility, operation, or system.

Let's look at some ways we can dip out toes into a threat analysis. This is not all inclusive and you'll need to go well beyond these ideas but they are a place to get started.

You'll notice I suggest documenting many of the aspects discussed. Documenting them helps to communicate the system and details to others who can offer insight and it helps to visualize what's happening.

Diagram Your System Architecture

Above is a simple example of a CMS based website, such as a common Drupal site. In the diagram diagram document all the components, even elements that browsers download from 3rd parties or components that aren't user facing.

Once you have the diagram look at all the interconnects between the different parts. For example, you may serve your pages over https but the connection between the web server and the MySQL isn't over an encrypted connection. That could offer a route to peek in on data.

Also, look at who can access what ports on what servers. If Memcached is accessible to anyone who knows the IP of the server the data in it can be retrieved by anyone. In this picture private networks, cloud security groups, or some other protection should be in place to protect anyone from ever accessing Memcached or MySQL.

Looking at the system can help you identify places to secure communications. It's an easy place to start identifying tasks.

Data Storage

Many sites store information about customers. This ranges from mundane settings through personally identifiable information (PII). Imagine an e-commerce site where someone gets into the database or can even just monitor the traffic between the web server and the database. They'll know customer names, email address, home address, and more.

An easy way to get a view of the data you're storing is with an old fashioned table. List out everything from the content that's displayed to the private details (even those sent to web services like a credit card processor).

Once you have this information you can combine it with the accessibility of the system and start to get an idea how open data is for hackers. It's also an easy place to start finding tasks to make the data more secure. For example, can information like addresses be encrypted?

Security Update Plan

Software is insecure. The more complicated the software the more likely there are holes in it. The software powering the Internet has regular security updates to fix the problems as they are found. By regular I don't expect a week to go by without needing to update one thing or another.

How do you update software? Is it automated? Is it often? How often? For example, it's great to install updates to a CMS but what about the web server, the database, the operating system they are on, and everything else in the system?

Document how you handle updates and then look for ways to improve on and automate the updates.

Reviewing Logs

Hacking attempts happen. Some people will even scan the entire Internet to see what's open. It's not all the hard to scan the entire Internet and you can do it at a slow pace in under a day.

Part of handling threats isn't just handling them but identifying bad situations quickly and reacting. That's where good logging practices can come in. Look at the log review practices and automate as much as possible.

Make sure to log everything (minus information such as passwords) and review those logs. If an IP address keeps trying to access your systems but fails to authenticate you should know about it. If systems are accessed when you don't expect them to be or from locations you don't expect, you should know about it.

Just the beginning...

These few things are just the beginning but a good place to start. Initially, this can raise a number of places to improve the system while getting some of the security thinking more in place.

Continue Reading »

These elements have had quite a history.

Bold and Italics Before HTML5

Bold was a visual styling and that should be in CSS rather than tags, right? So, we have the strong tag. The same type of thing happened for the italics tag. In the HTML4.01 spec and XHTML specs there were no longer bold or italics tags.

But, they didn't go away. Not in use.

Strong and Bold

In HTML5 <strong> and <b> both exist and have a different meaning.

The <b> tag is for "offset text conventionally styled in bold". If you read deeper into the details you'll see it adds, "without conveying any extra emphasis or importance".

<strong> is different. It "represents a span of text with strong importance." There is semantic meaning of importance here. In fact, a <strong> tag within another <strong> tag has even more importance. There is nested importance. The html5 spec is enlightening on this.

Emphasized and Italics

Just like <strong> and <b> we have changes to <em> and <i>. The <i> tag is for "text conventionally styled in italic". There is no semantic meaning. <em> "represents a span of text with emphatic stress". Like <strong>, <em> is a tag with semantic meaning and that matters when nested.

Does The Spec Represent The Practice?

I'm not really sure. We'd need to go to those who read the markup while caring about semantic meaning to find out. I know that many tools don't take this into account. For example, CK Editor has buttons for italics and bold but uses the <em> and <strong> tags. This may be a part of HTML5 that just isn't widely understood or used.

Continue Reading »

The 2012 Data Breach Investigations Study by Verizon shows that in 855 data breaches they examined, 71 percent occurred in businesses with fewer than 100 employees.

When I read this finding I was reminded of all the sites I'd seen successfully breached over the past several years. The sites of friends. The sites of non-profits. The sites I was asked to take a look at.

This inspired me to present at DrupalCamp MI on the topic of security. In the 45 minute time slot I could only cover a fraction of the material I would like to have. For example, I didn't talk about social engineering types of issues. I hope someone presents on that soon.

The slides for this presentation are available on Speaker Deck and Slideshare.

One thing I came to realize is that I'd like to see more sessions and trainings at future DrupalCons and DrupalCamps. It's a good time to raise awareness and teach people practical methods and tools to use in this space.

Continue Reading »

Know Yourself

Knowing the details about your personality, strengths, and weaknesses can really help you navigate your career. Consider using toole like Strengths Finder and learning your Myers-Briggs type.

None of these is perfect or going to tell you everything. But, they can provide you useful insights into yourself.

Focus On Your Strengths

When I was growing up a common piece of advice was work on your weaknesses to become more well rounded. This was bad advice.

Focus on your strengths. Avoid the things you are weak at. Good ways to avoid them are to outsource that work or partner with others who are strong where you are weak.

You can't remove doing all work you are weak at. If you can't avoid it spend as little time doing it as possible.

Don't Hitch Your Horse To Just One Wagon

Focusing solely on Drupal development is a bad idea. Before you jump to the comments to argue with me hear me out.

Few technologies stand the test of time. If you go back 5, 10, or 15 years on the web you can see how much things have changed. Good projects from the past have come and gone. Careers tend to last longer than any piece of technology.

Doing just Drupal work can pigeon hole you into just Drupal work. You might find yourself looking for work and people aren't interested. Or, you want to work on a project and they want someone more well rounded.

What does this mean practically? If you are a Drupal back-end developer write something that's not in Drupal. Maybe a PHP CLI application using Symfony. This is a good place to start and you'll learn a world of other technologies without ever leaving PHP.

If you focus on site building put in some time to using Wordpress, learn about information architecture, spend time learning customer experience, and so forth.

If you are a front-end developer learn other templating systems. PHPTemplate, the system in Drupal, is a one off solution. Learn other setups like Twig, haml, or one of the other popular options. Use SASS or Less.

If you're feeling really daring, pick up another programming language or learn DevOps.

Learn, Learn, Learn

Technology is always changing. What's important in technology is always changing. Go back 5 years. There was no focus on front-end performance. Now it a sub-industry of development.

Always keep learning. Put aside some time each weak for continuous learning.

A 40 Hour Work Week

Don't work too much. 40 hours average per week is enough. For some of us it might go up to 50 hours per week. Don't go higher than that if it's within your power. I include side development and free time contributing in this category.

When not working on code do other things. Spend time with family. Have hobbies. Relax. The life we live has so much more to it. People to love and be loved by. Beauty to take your breath away. Friends to have a beer with. Enjoy these things.

By doing this it's easier to focus when at work. Creativity is more likely to flow. There is a desire to get it done rather than dillydally. The focus, creativity, perspective, and added joy you have will show up in reviews, interactions with others, and your career path.

Take Part In Mentoring

Have a mentor. Mentors have experience they can share. The kinds of things you can't learn from Google. Well, not yet anyway.

They can talk to you about your career. They can help you see things in ways you don't know about. They can point you to technologies you should look at. They can introduce you to others. When jobs come up they can be references. When they know of good jobs they can point you to them.

Have a mentor. Pick one carefully.

Stay Focused

There is so much technology. So many shinny things to distract us. Don't let the distractions take over. Not every problem is yours to solve. Don't go down all the rabbit holes. You might end up in wonderland and have a great time. But, if you can't get your work done it's a problem.

I allow myself some distractions. They are fun and some turned out to be useful. But, the time is limited and the types of things are limited. Just like my backlog, I try to keep a prioritized list of the ones I'll let myself do and a time limit I can spend on them.

Keep the main things the main things. Don't let distractions stop that.

This list isn't trying to tell you where to go. We all have different paths. The idea is to go down your own path without getting stuck and have a good time doing it.

Continue Reading »

While The security conscious players like Google, Microsoft, Dropbox, Stackoverflow, Apple, and many others have all implemented this in appropriate places, many of the sites we regularly visit have not yet done so. Some sites not implementing frame protections include drupal.org, wordpress.com (though Wordpress itself has it for login and admin pages), reddit, Evernote, and many others. Since Drupal doesn't have support by default for X-Frame-Options I assume most Drupal sites have not yet implemented this.

If you are interested in learning how to implement X-Frame-Options keep reading as we'll dive into a few ways to do this.

The Basics

A great place to learn the basics is on the Mozilla Developer page about X-Frame-Options. For the purposes here we'll only look into the options that have wide adoption. These are:

• DENY: When the X-Frame-Options http header is set to this value a page can never be embedded in a frame/iframe.
• SAMEORIGIN: In this case only the originating domain can embed pages in a frame/iframe. This is specific to a domain including the subdomain. Pages on foo.example.com cannot embed pages from bar.example.com if this value is used.

Apache Setup

Apache can be setup to use one of these values by default. If mod_headers is installed than this setting can be used:

Header always append X-Frame-Options SAMEORIGIN

Nginx Setup

For nginx in the server or location configuration you can use something like:

add_header X-Frame-Options SAMEORIGIN;

In PHP

In any PHP application the header can be set before page content is sent. This is done using the header function.

header('X-Frame-Options: SAMEORIGIN');

Drupal

I want to touch on Drupal for a moment because it's the second most widely used CMS. I would talk about Wordpress, but they already covered their basis on this. Drupal has a custom way to deal with headers through the use of drupal_add_http_header. Using this function you could do:

drupal_add_http_header('X-Frame-Options', 'SAMEORIGIN');

This could be used for specific paths or site wide. If you are using the Overlay module you'll want to use SAMEORIGIN rather than DENY or the overlay will not work.

Alternately, the module seckit provides some security options including this one. I have not used this module and make sure you thoroughly check it out before using it.

Continue Reading »

Architecture

One of the surprising things about enterprise applications is that I've found one architect in the application often doesn't know how the whole thing works. The software is complicated and applications can end up with numerous people who have architect in a title. Drupal has a similar experience. With all of the things going on, even the wisest of contributors and initiative leads don't know how the whole thing works.

Drupal is complicated like enterprise software.

Monolithic

It recently struck me that Drupal is a fairly monolithic application. A trend in applications is to have small parts that operate together to create a whole. For example, one application provides the user interface. That application talks to an API server. The API server schedules things to happen via a queue. Workers grab items from the queue and act on them. In this picture each of the different types of things is a different application.

Or, you could have a small application. I think of Wordpress as a fairly small application and it comes in around 200k lines of code. For a small application it does a lot.

Drupal is nearing 950k lines of code. I won't be surprised when it passes 1 million lines of code. This is just core and no one just builds a site with core anymore.

Drupal also does a lot. Sites can have a message queue (using Drupal), caching (using database caching through Drupal), and so much more. It's big.

Drupal Specific Terms and UI

I've had the opportunity to teach people about Drupal who have had years using web applications, have worked with content and other CMS, and some who write applications. I've learned there are a lot of Drupalisms. A whole vocabulary has grown around Drupal and you have to have domain knowledge to use Drupal, build with it, or just understand conversations around it. In addition the UI isn't intuitive to outsiders. Someone needs to understand the product down to the major version to perform everyday tasks.

These are both elements I've found in enterprise software. In many ways these custom aspects lock users into the product because it raises the barrier to go to other products due to change. If successful it can help build an ecosystem around the product (more on that later).

Major Version Update… What?!?!

Have you tried to update major versions of a Drupal site? How about updating when you jump a major version? Going from Drupal 7 to 8, much less Drupal 6 to 8, is going to be a huge effort for anyone who tries to take it one. The architecture changes that happen between major versions means custom code needs to be rewritten, contributed modules need to work and have an update path, and the data needs to be migrated because it often can't just be updated.

This can be costly. It can end up in the hands of consultants to handle the upgrade just to keep the same base functionality. That doesn't even mean adding support more responsive design.

Ecosystem - Consultants and Partners

Many enterprise software applications will have an ecosystem around them. There will be partners that sell the software. There will be partners that support the software. There will be consultants who help those who implement the software do so. When you have a complicated piece of software someone new to implementing it needs the ecosystem.

Does this sound at all like Drupal (with the exception of selling Drupal)? Drupal has an amazing ecosystem around it with many fantastic people. And, many companies could not implement Drupal as a solution without them.

I'm Not Complaining

I could continue on this thread but many people won't continue to read (short attention spans). I'm not complaining but merely observing what Drupal has become.

Continue Reading »

At HP Cloud we just launched a new homepage design (pictured above). The process to craft this landing page is one I think is worth exploring because it can help anyone with their landing pages or even broader products.

The Process and People

I recently wrote about a simple product development process. This is the process I applied to the creation of this new homepage. Following this process had some major benefits that really appeared late in the process. While following a process to design a landing page where design is late in the process may seem odd it actually makes the design work easy while crafting something that works.

Being that HP Cloud, like many organizations, has a number of stakeholders it was important to include the right people at each step while not bringing in everyone for all the steps. When you need to get work done only the people who will contribute to that need to be involved in the development.

Figuring Out The Target Audiences

HP Cloud is a web thing so aren't all web users the target audience? The answer is obviously no. The hard question is identifying who the target audience is going beyond base demographic information.

For example, if you are targeting a product at Drupal developers you might not be targeting content managers. Or, vice versa. Knowing who the target audience is up front is really important. At HP Cloud we aren't targeted at Drupal developers but we do know who our audience is.

Before moving on we had a clear and limited list of target audiences for this landing page. This was mapped to the list of personas I maintain because they are really helpful to agree on the details of these audiences.

Goals Goals Goals

When crafting a landing page the goal isn't to talk about yourself. Goals allow you to aim for a target, measure success, and compare as you tweak things. Once we had target audiences we gathered up the goals for the homepage. Having target audiences before goals I find to be very important. Goals aren't just goals. They are goals for a target audience.

The Content

Once we had gathered the audiences and goals it was time to craft the content. The fabulous folks in marketing wrote some new content that was:

1. Written towards a target audience.
2. In prioritized order.
3. Written to meet the goals.

When we shared the content, before there was ever a design, there was push back. People would ask, where is my thing on the homepage? Having the target audiences, goals, and prioritized content made it easy to talk through this. When something wasn't on the homepage (or any landing page) it's easy to ask how it helps meet a goal for a particular type of audience. If there is no easy justification it's easy to justify it not being on the page.

The Design

Once all of this is done the design work began. I intentionally held off design to this point so everything would be in place to make it easy. Instead of wondering, going back with lots of questions, or having to be creative on content I was able to just design. Having content in prioritized order was really nice because what you see on the page is the prioritized order. If/when we go to a responsive or mobile design we can flow things based on the priority.

Building It

Once the design was complete and signed off it was time to build it. This is a Drupal site. Ryan Peters took the design and turned it into something editable by our content managers using the base platform we already have in place.

Continue Reading »

This process is meant to be a simplification bordering on over simplification. Let me break down each of these parts.

Persona / User

Before you can build anything you need to know who you are building it for. Creating something for an open source developer is radically different than creating something for someone in the office of the CIO at an enterprise. They talk differently, have different needs, approach a situation with a different context, and so forth.

This all needs to begin with an understanding of who a user type is at a non-trivial and non-demographic level. The best way I've seen to do this in a manner that can be used by a UX developer, a marketer, someone in sales, and so forth is to use Personas.

Goals

Once you know who your users are you need goals. And, "tell customers all about our stuff" is a terrible goal. The goal should be for a persona not you.

Some good goals could look like:

• A product manager signs up for our email list where they can learn about product updates.
• A potential developer customer completes the online registration form.
• A developer can easily navigate to service documentation.

These should be specific to the personas and what they are willing to do. You don't want a goal that would be out of the nature for a Persona.

Content / Experience

Once you have goals and know who your users are content and an experience can be crafted. It's important to wait until this point to craft content and an experience so as to do it well. For example, content about a product written to a developer or someone in the office of the CIO at an enterprise will be different. Or, a goal of getting an individual developer to sign up for an account will be very different from that of someone in enterprise management.

Design / Veneer

Once you have content and an experience you can design it. You can create the pretty icons, figure out page layout, pick your fonts, and so forth. And I mean this in all the ways that go beyond veneer.

Continue Reading »

On the ofter side of the argument is that all code needs to be commented. In many discussions, including many I've contributed to, what and why code needs to be commented isn't clear. And, the truth is the lines of code themselves don't usually need to be commented. But, there is a lot that does need to be commented. Here I'll touch on 3 types of code comments that have little to do with the lines of code themselves.

Why Does The Code Do What It Does?

Have you ever started to work on a project that was already well developed but had few or no comments? I worked on a codebase like that in the past year. I kept asking the question, why did they do it that way? In order to figure that out I had to bug (and in some cases annoy) people to figure it out. I didn't want to change something important because of my newness and ignorance of the codebase I was working on.

Code comments that explain why something was done in a certain way are really helpful. That context you have in your head while writing code isn't something the next person will have who touches the code. And, you might not remember it when you work on the code a year from now. Adding that context as code comments can be really helpful.

For example, take a case where code works against an API and while writing it you learn a nuance about the API. Will you always remember that nuance? Will the next person know it who hasn't done the work you did? Adding a code comment explaining what's happening can be very helpful in the future.

Another example is to document your assumptions and business rules. Sure, there might be a ticket number in the commit message where the code was added. But, how often do people look at code that way? If details about some assumption or reason are in a code comment it makes it much more clear.

References

Code comments should reference relevant documents, other code, specs, etc. If someone comes along later to work on the code it would be great if they could see the reference used and go read it themselves. It's much easier and faster than trying to search the web for the right version of the right one. Or, if you go backand what it returns plus any nuance to using it. While the name of a method may tell me what it does that doesn't mean I know what the inputs are and all possible returns.

If I think of code as a black box, which I often do, I want to know about the inputs and outputs. This is where header comments on functions and methods come in. They explain the inputs and outputs for those who don't know or remember the internals and don't need to read the code to use it.

This is popular enough that IDEs and some text editors are able to use these comments to help as we code on the fly.

In Practice

I've worked on a lot of code in the past couple years including both overly commented and code without comments. In practice having code comments explaining these three things has made my life a lot easier. And, the absence of them has made it made more difficult to work on a codebase meaning it's taken more time as well.

Well commented code is just easier to work in.

Continue Reading »

On the flip side, when developing a product it is helpful to understand target audiences. You can look to see what features a target audience needs and evaluate how well multiple options meet that need for that audience.

I've recently started to ask these questions about Drupal. As a popular CMS that is used to power a significant number of sites this is a topic I expected someone would have dove into already. And, while I've found a number of opinions on the topic I did not find much of an overview. So, after a little digging here is what I found.

Before we dive into the target audiences I would like to note that this is an attempt to document where Drupal is at today and moving towards. This is not an attempt to cover the past and I'm trying not to give my opinion but instead document what's happening.

If you're looking for opinions on the topic of who Drupal should target you can read what Leisa Reichelt and Larry Garfield have written.

Content Readers

This is the largest target for Drupal and the one I think I've read the least about being targeted. When we're creating Drupal and using Drupal to build sites we have to keep this group in mind. There's no reason to publish content if you aren't trying to get people to read it, right?

There an easy place to see this in Drupal as a target. Look at how there is a Drupal 8 initiative for mobile. This recognizes the migration of content consumers to different devices. While there are definitely parts of this that play well for other audiences it very nicely targets this audience.

Content Authors and Editors

Drupal is a CMS and someone has to write and manage content for readers. I think this audience has been highlighted through efforts like WYSIWYG in core among numerous other things. Note, if you want to help with WYSIWYG in core there is an issue for it.

Admins

I'm calling Admins the people administer the site. They handle module and core updates. They are the ones who may create the user accounts, manage mail settings, etc.

Drupal has long targeted this group. In Drupal 7 the built in update manager is one example. How Drupal has a pattern of settings being stored and changed through the UI making it easy for admins is another.

Site Builders

When I started using Drupal this target didn't appear on the radar. This target has now become one of the most significant and power that Drupal targets. Through the development of modules like CCK (now fields in core), Views, and several others Drupal has put the power to build features, data models, content displays, and more in the hands of people with ideas without requiring them to learn to program. This is a very powerful idea and one Dries has said openly Drupal is targeting.

Developers

Developers is an easy group to describe, right? In all seriousness developers is the most controversial group where the Drupal community has the strongest opinions in my experience. Since developers is really a mix of sub-groups I wanted to try and document those.

Front End Developers and Themers

Drupal is targeting front end developers in an interesting way. There are really two groups of these being targeted right now. The more well known of these is typically called themers. Themers are a group of people who can map a design to the Drupal theme (templating) system. They know CSS, how to create templates in the Drupal way, how to override default Drupal markup, etc.

Then there are front end developers from the web at large. In my past with Drupal I've not really noticed these front end developers being targeted. With some of the contrib work in Drupal 7 and some of the efforts going into Drupal 8 I see this group starting to rise up. Some of the decisions being made in the development of Drupal 8 is with this group in mind.

I will personally be interested to see how these slightly different groups who do the same types of things work out as target audiences through some of the changes Drupal is making right now.

Professional Programmers

Drupal 8 has definitely moved into the realm of professional programmers. It can be seen in how Drupal has added more complex programming concepts. This should not be a surprise since lots of core work comes from consulting companies, companies that have services built on Drupal, and so on. Lots of professional programmers are working on Drupal. Because of this the target audience of programmers is moving to people who can work with these more complex concepts.

Now, I'm not going to go into what a professional programmer has in their expectations other than to note Drupal is being driven by programmers who do it professionally.

Also note, I do consider there to be a difference between software engineering and programming which is not lost here. But, the difference between software engineering and programming is best left for another post.

DevOps

The popularity of Drupal has lead it to being used on more complex deployments and the rise of DevOps has put it on a lot of radars. Drupal has started to show it is targeting this group through things like the new core release schedule and conference tracks for DevOps.

A Group Drupal Is No Longer Targeting

When I first started using Drupal it was something a weekend warrior programmer could jump in and work with. I know a number of core developers who jumped in this way and eventually became professional programmers because of this. But, over time the ways Drupal is being used have changed, the people contributing to core have changed, the use cases Drupal is being applied to solve have changed, and the people who have been around driving things have themselves changed.

Drupal no longer targets the weekend warrior programmer. In a subtle nuance, much of what those programmers used to do (and what I did when I started using Drupal) can be accomplished through site building.

This doesn't mean weekend warriors are no longer a target. It doesn't mean that weekend warrior programmers are still not around. It just appears that they are no longer a target audience in practice.

Continue Reading »

QueryPath, put simply, is jQuery written in PHP for server side manipulations. Now that I can easily use anonymous functions and closures I can write and read functionality that looks more like the jQuery I've been writing for years.

Anonymous Functions and QueryPath

Before PHP 5.3 was around a form of anonymous (a.k.a. lambda) functions were around through the use of create_function(). Unfortunately, this method had a number of drawbacks making it something I avoided using.

In PHP 5.3 anonymous functions were introduced in a familiar manner. I find the easiest way to look at these is by example.

$foo = function($bar) {
  print $bar . "\n";
}

$foo('Hello World');

Now, let's apply this style function to QueryPath.

$doc = qp($file);

$doc->find('.foo')
  ->each(function($index, $item) {

    // $item is a DOMElement
    $bar = qp($item);

    // Do something to each item in here.
  });

The each method can accept a number of things. This can include a callback function, an anonymous function, or anything that's callable.

In this case using an anonymous function turns out to be fast. In my case and tests it ran faster than a normal callback function.

Closures and QueryPath

According to Wikipedia:

A closure is a function or reference to a function together with a referencing environment

The existing environment is a really important part here. Let's look at an example.

function get_iterator($foo) {

  $bar = $foo * 2;

  return function ($index, $item) use ($bar) {
    // Do something here.

    // $bar is available from the parent environment.
  };
}

$a = get_iterator(2); // $bar is available when $a() is called as 4
$b = get_iterator(4); // $bar is available when $b() is called as 8

$doc = qp($file);
$doc->find('.foo')
  ->each($a);  // $bar inside $a() is available as 4.

The environment from get_iterator() is available to the function it returns even outside the original context. It's important to note that this is the environment at the time get_iterator() was called to create the function.

There are many ways closures can be used. For example, they can help setup an environment for anonymous functions to be used and passed around.

QueryPath and Drupal

The context for my QueryPath code has been within Drupal and there is a module for that.

Continue Reading »

Looking at Drupal 7 and 8

Thinking of modern Drupal development versions 7 and 8 came to mind. For Drupal 7 Dries had a list of priorities that read:

• Better media handling
• Custom content types in core
• WYSIWYG editor
• Better performance
• Better tools to structure/organize content
• Basic Views like module
• Automatic upgrade functionality
• Improved node access system
• Better internal APIs
• Better external APIs (import/export, web services)
• Usability

While these were his priorities you may notice several of these did not happen.

And for Drupal 8 he has setup a number initiatives that include or have included:

• Configuration Management
• Web Services
• Design
• Multilingual
• HTML 5
• Mobile
• Views-in-core
• Layouts

When I look through these two lists I see a lot that is developer focused. Better internal and external APIs are definitely targeted at developers and those are 2 of the 11 goals for Drupal 7. When it comes to Drupal 8 configuration management and web services are both things that have a definite impact on coders. The html 5 initiative was for front end developers. Without even digging down into these initiatives I can see a top down approach of developers as a target audience.

Where Is It Going Wrong?

If the top down picture shows a focus on developers than why are people complaining? I can't point a finger at Dries for not setting some form of direction around these. It's not a matter of formality because there has been formal direction in some form.

So, where is it going wrong? Why are developer complaining (and sometimes rightfully so)?

Maybe a good place to start is to ask what developers Drupal is and should target. There is quite a gap between a weekend warrior, someone who does that by themselves for small organizations, and someone who is building a service on it or doing large scale applications. That is both in their needs and their background as a developer.

Then we can ask questions like, is Drupal targeted at me? What can Drupal do to have better internal APIs for this or that type of developer?

Two Warnings

In this whole thought exercise there are two warnings.

First, we all need to be careful not to blame. In my time in the Drupal community I've learned there are a lot of people who care, are well meaning, and are trying. No one does the best or right thing all the time (or can even agree what that is).

Instead I like to think of this as an exercise in incremental improvement and looking at it from a place of observing the system to spot those opportunities. This is also a way to get a view of what's happening without coming up with an opinion based on frustration with a particular problem.

Second, I hear people say Drupal is for anyone. Or, at least for all developers. This is obviously not the case. For example, a Java developer is not expected to be excited about Drupal. I'm reminded that you can't please all the people all the time but you can displease all the people some of the time.

Coming to a picture of who Drupal is for can help to meet the needs of those people even better while diverting people trying to solve a problem that's not appropriate for Drupal to other places that meet their needs. It can even be a way to help identify others who compliment Drupal and have a better exchange with those projects.

Continue Reading »

This all starts with a new function to Drupal 7 called field_get_items(). This function is designed to get the items while handling the language for you. It works for entities beyond just nodes as well. An example use would be:

$items = field_get_items('node', $node, 'field_foo');

As an optional 4th argument you can pass in the language code if you are looking for a specific language.

If you look inside field_get_items() you'll see that it handles the language for you. This function can also be used on any field of any fieldable entity.

Though, if what you're trying to do is style the output of the field you might be better off writing a field formatter.

Special thanks to Dave Reid who was the original person to teach me about this function a lot time ago.

Continue Reading »

Backup and Migrate + HP Cloud

An easy way to regularly backup sites in Drupal is to use the Backup and Migrate module. Being an HP Cloud user I use the HP Cloud module enabling Backup and Migrate to put my backups in object storage. Using Backup and Migrate I don't just want to backup my database but my public and private files as well. To do that there is the Backup and Migrate Files module.

Note: I'm a fan of having backups in multiple locations. Don't just store your backups with one provider but maybe also have them in a second provider or copy them locally. I've seen it where a backup goes bad and you need it.

Store It In Git + Bitbucket

Part of backups is the codebase to our sites. If there are custom themes, custom code, or something else with the site that isn't just a straight project (like Wordpress or Drupal) you can have a problem if you think you can reconstruct it.

My solution is to store all my sites in git repositories. I'd first thought of using Github for all of these but the basic (free) accounts don't allow you to have private repos. While I like sharing as much code as makes sense there are things I want to keep in private git repos. For this there is Bitbucket. You can have a lot of private repos.

For anyone familiar with the past for Bitbucket you might not know that it is now owned by Atlassian who is quite successful with their products. They appear to be a strong parent company who has grown to be strong without shady sales people (or even good ones).

Note: there are lots of other good git repo management solutions. For enterprise there is Github Enterprise, there are open source solutions you can install on your own server, and other competitors to Bitbucket and Github. I've chosen Bitbucket for personal things because it was the most cost effective solution for my case that I found.

Backing Up Personal Computer Files

I'm a Mac and Linux user which means I can't just use time machine on the Mac to do backups. But, that is where I start. I do use time machine to create backups for some of the Mac computers in my home. In addition one some computers I use Carbon Copy Cloner to make duplicate images of my hard drives. This is great for the case that a hard drive goes bad and I can literally remove the bad one and swap in the cloned drive to be back up and running.

For personal files suck as markdown documents (I only use word documents when I have to) I keep them backed up and shared between computers with Dropbox. This comes in handy beyond backups.

The scripter in me has come up with several scripts that use rsync to keep backups of files in other locations as backups.

There are also a number of services that do backups. Of them the most highly recommended one shared with me is Crash Plan. It does cross system as a reasonable price.

What Do You Use?

These are the solutions in my arsenal. What do you use and how well does it work?

Disclaimer: I work for HP on HP Cloud and work on the Drupal module providing the integration.

Continue Reading »

For the record I'm going to keep these stories anonymous. While I think these stories are useful to help us see why we would want backups I'm not going to name my friends and cohorts for the bad or painful things that have happened to them.

Years of Blog Posts Lost

A friend of mine used to host his blog with a shared hosting provider. The hosting provider listed on their site that they did backups of the content and it was included in a package. After being with the hosting provider for a couple years they had a catastrophic outage. The codebase and the database were lost. You'd think the codebase (an open source project) would be simple to reconstruct and the host had backups of the database - which wasn't just marketing BS. But, this outage also corrupted the backups. It was all lost.

This was a lesson for me in having backups separate from your service provider.

A Config Glitch Corrupts A Database

Imagine a site that uses multiple servers and is hosted in a cluster. Deployment management is all nicely scripted. One day a server config update is deployed and it all goes wrong. A regular running script does something that corrupts a database. It was a race condition that would only show up in an outage. Now the site is down. To make matters worse a recent update to the backup scripts caused the compression of the backups to generate corrupt files and no one was the wiser. The content is corrupt.

This case didn't turn out entirely bad. There was a second set of backups that fired just at deployments and updates in an automated manner. These were used to bring the site back up and visitors were basically unaffected.

The lessons I took away from this story are to test your backups semi-regularly and to create backups right before any kind of update that you test before doing the update.

Bye bye Family Photos

Backing up web stuff isn't the only smart thing to do. For example, I know someone who only had their family photos on their laptop. It was a computer used everyday and it didn't seem to have any problems. In fact, no computer this person had owned for the past couple decades had ever had a problem. Then one day part of the hard drive started to go corrupt. It went bad before they realized what happened. Loosing your music and documents is annoying. Loosing photos of your kids is something different all together. I know several people something like this has happened to.

What I took away from this was to make sure my personal stuff was backed up.

Next Steps

After writing this I realized it might be useful to share some of the backup strategies I've used or am aware of. Look for a follow-up post with some simple actions people can take to start backing up their stuff.

Update: The post with backup solutions is up.

Continue Reading »

Dieter Rams, for those of you who don't know, is an award winning designer from decades ago. Even though the advice is decades old it still applies to web and physical products today. The design of Dieter Rams is still influencing designs today. What he did is long lasting.

The 10 Principles of Good Design

Without further adieu…

• Good design is innovative.
• Good design makes a product useful.
• Good design is aesthetic.
• Good design makes a product understandable.
• Good design is unobtrusive.
• Good design is honest.
• Good design is long-lasting.
• Good design is thorough down to the last detail.
• Good design is environmentally friendly.
• Good design is as little design as possible.

Continue Reading »

$('#foo', document);

If you iterate over a set of 100 items and create a new jQuery object from each of these you end up creating 300 jQuery objects with this method. Let's take a look at what's happening.

Note: If your jQuery is in noConflict mode you will need to use jQuery instead of the alias $.

The Constructor: jQuery and jQuery.fn.init

The source code to look at is for the call jQuery which creates a new object using jQuery.fn.init. In 1.8.1 it looks like (the difference to 1.4.4 is rootjQuery):

jQuery = function( selector, context ) {
  // The jQuery object is actually just the init constructor 'enhanced'
  return new jQuery.fn.init( selector, context, rootjQuery );
}

jQuery.fn.init is a function that deals with all kinds of special cases that might get passed into the jQuery constructor. Knowing how these special cases work can help us create fewer jQuery objects making our JavaScript perform faster.

The Standard Case

Most of the time if you use jQuery the right way you'll end up creating two objects every time you construct something new with jQuery. One of them is thrown away and eventually caught by garbage collection. But, it's good to know about these throw away objects because they do take time and resources for a JavaScript engine to handle.

jQuery('#foo')

Creating a new jQuery object only using a single id creates just one jQuery object. This is a special case jQuery has been optimized for. This is different from the case jQuery('#foo', document) which creates 2 jQuery objects. The case where there is only the id assumes document is the context (for in browser uses).

jQuery(context).find('.bar')

If you have a case where you looking for a selector on something other than document than you can create one less object by using jQuery(context).find('.bar') than jQuery('.bar', context). The reason for this is quite simple. When you call jQuery('.bar', context) the special case is to do jQuery(context).find('.bar'). In addition to one fewer object you also avoid some internal jQuery logic.

Even if you are searching on document and you pass that in as the second parameter you should use the optimization here. $('.bar', document) creates 3 jQuery objects and this optimization works to speed it up and create one less object.

Element names and 1.4.4

In jQuery 1.4.4 there is an optimization no longer in the latest jQuery release. If you make a call like $('div'), where you are searching by the name of an element type, it creates just one jQuery object. This is a special case. In jQuery 1.8.1 two objects are created like almost all cases in jQuery.

Understanding vs. Writing Good Code

Understanding the jQuery constructor and writing elegant high performance code are two different things. If you want to write high performance jQuery it's useful to know much more of the jQuery source than this, how to take advantage of it, and to know plain JavaScript which, being native code, is faster than user space code.

For anyone who wants to really understand jQuery I recommend reading the source.

Continue Reading »

This idea of a development mode vs a production mode can be seen all over. If you visit the jQuery website you'll see the download has a toggle for the development or the production version. In Symfony, the frameworks whose components are making their way into Drupal 8, it has the difference between app.php (the front controller) and app_dev.php in symfony standard. A production and development version of the app. These are just two examples.

Lets take a look at how we can use the Speedy module to bring some of this concept to Drupal 7.

What Is Speedy?

Speedy is a module that supplies minified JavaScript for Drupal core. Minified JavaScript for Drupal core files is not something that works out of the box. To turn minified JavaScript on and off Speedy adds a toggle to the performance page inside Drupal. This is the feature we are looking to take advantage of.

Not sure why why minified JavaScript helps with performance? You can read more at "Why Minify JavaScript?"

Using Speedy for a Dev/Prod Toggle

Speedy stores the value of the dev/prod toggle in the Drupal variable speedy_js_production with the default value of TRUE. By default we assume the site is in production mode. To take advantage of it you just need to use variable_get('speedy_js_production', TRUE).

Let's look at an example for inserting a minified or original JavaScript file.

if (variable_get('speedy_js_production', TRUE)) {
  drupal_add_js('path/to/foo.min.js'); // The minified file
}
else {
  drupal_add_js('path/to/foo.js'); // The larger non-minified file
}

This will work even if the Speedy module is not installed because of the default value of TRUE. If the Speedy module is installed someone will be able to toggle to the non-minified script for testing and development.

Note: I realize there are more optimal ways to write this than to use an if/else setup. It's used here to make the example clear.

Drupal 8

There is an issue to "Add a Production/Development Toggle To Core" for Drupal 8. Whether this will go into Drupal 8 or not is still undecided. It will take some work. If you'd like to help out that issue is where to go.

Continue Reading »

GiveCamp is a series of camps where technical people connect with a non-profit for a focused sprint and hand off a product at the end. For example, a team of 6 might spend a weekend building a website for a non-profit and hand it off to them at the end of the weekend.

Are you a developer, Drupal site builder, designer, DevOps, organizer, sponsor type, or someone else in this arena who wants to help? If so consider volunteering at a GiveCamp or helping to organize one.

I for one will be volunteering at the Ann Arbor GiveCamp coming up in September.

Continue Reading »

HP Cloud PHP Library

The foundation for all of this is a PHP library that currently supports identity services, object storage, CDN, and DBaaS. Our versioning philosophy is that generally available services, like object storage, are semantically versioned. Service still in beta, like DBaaS, may have change but only if absolutely necessairy.

This library is being developed as we need it for our services. So, not every service is covered in the 1.0.0 release. But, we are using our own cloud so expect more services added to it. If you want to contribute please feel free. This project is on github and we look forward to pull requests and issues.

If you want are interested in an introduction to these services I wrote a 5 part series for the HP Cloud blog.

The Drupal Module

To make Drupal play nicely with HP Cloud we released a Drupal module that ties the PHP library into Drupal. For example, you can set public or private files to be served out of object storage rather than the file system. This works for image styles, automatically generated js and css files, and normal uploads. If a container is CDN enabled all these files will be served over CDN as well.

Feel free to dive in and take a look.

Moving Forward

Our goal is to share what we can and grow these codebases over time. As more people pick them up and use them we look forward to incorperating even more improvements.

If you have any questions about these please feel free to ask.

Continue Reading »