While a number of our users have requested this feature from us for awhile, we’ve been hesitant because a number of studies have shown that when a captcha is used on a form, it severely impacts the conversion rate. This is because it also slows down and turns away legitimate users, who often get frustrated by the extra test.

In an effort to balance the needs of our users who want to insure the integrity of their data with our desire to promote best practices for high converting forms, we implemented a smart captcha system in Wufoo that tries to automatically detect abusive behavior and only show a captcha test in those situations. Here’s a video screencast we made to show off the feature in action.

We’re also very proud to be using reCaptcha to help power our new anti-bot system. They’re used by web sites all over the world and we think it’s great that we’ll also be able to help digitize texts that are being scanned and stored in digital format for preservation and accessibility.

By default, all forms in Wufoo are set to automatic protection, but you can also set the captcha system to always show or never show the captcha on your form as well. You can find this feature’s options under the form settings tab when you edit your form.

If you’re up or interested, you can follow our progress during the downtime on our Tumblr Status Blog and/or Twitter feed.

Why are we moving?

Over the past two months, we’ve unfortunately had a number of incidents that can be sourced to our current data center. As some of you are aware, we had three separate instances of power outages that resulted in the site coming offline. Normally, these outages would be minimized by the use of a generator used onsite at the data center once our UPS runs out, but something apparently happened on their end that we’ve never had satisfactorily explained.

Our trust in this data center was lost for two reasons: First, the incident happened multiple times and the time frame between each incident was enough that we believed the problems should have been diagnosed and addressed, but weren’t. Second, and more importantly, there was little communication explaining what happened and why it happened. And as you know, that’s just not how we run our business.

Ultimately, it is not fair for all Wufoo users to trust their business and the reliability of their forms to a data center we do not trust. It’s unfortunate because the data center has provided rock solid reliability up until these incidents, so the communication issues were a surprise to us. Rather than avoiding the problem and hoping it doesn’t happen again, we’re moving centers in hopes that our new relationship will instill new confidence.

Thanks for all of your support,

— The Wufoo Team

Update

And we’re all done. Everything went over as planned and we actually finished a half hour ahead of schedule. Many thanks to Bitpusher for helping us make this an extremely smooth transition!

Here’s a breakdown of what it was before to what it is now:

• AD Hoc : 200MB -> 250MB
• Bona Fide : 500MB -> 1GB
• Carpe Diem : 1GB -> 3GB
• Ad Infinitum : 3GB -> 10GB

There you go. Enjoy!

It’s a really great tutorial and the best part is that he included a nice screencast of the method to show everything step by step. If you’re wondering about the reason for the code acrobatics, it’s because we currently don’t allow JavaScript snippets on public forms for security reasons. However, we’re glad to see Boris show a safe and fairly painless work around to get the advanced tracking Google provides. Thanks Boris!

We hate to admit it, but if you ever used line graphs on the old system, you probably know that it was basically just shy of terrible. The biggest problem was the lack of flexibility in regards to the X-axis. We didn’t let you specify how to handle that data over time and this made the graphs not very good. And so one thing you’ll notice when you create a line graph in the new Report Builder is that we’ve added an additional option called Date Range to help you choose the time scale you want your data based upon.

By default, this option is set to “All”, which means that it’ll look through the dataset (based on your Data Settings in Step 2) and graph all information that’s ever been collected for that field’s choice. We’ve also included a lot of other premade Date Range settings (ie. Today, This Week, Last 30 Days, etc.) that we thought would be useful for users wanting a relativistic look at their data from the present moment. And for power users that need absolute control, we also provided a way to graph a custom date range based on your own start and end dates.

A Smarter Axis

One of things you’ll notice about our new graphs is that we automatically calculate and use the proper time scale for the X-axis of the graph based on the date range specified. So if you’re looking at data for a particular day, we’ll show you the data in regards to entries per hour or if you’re looking at data over several weeks, we’ll show you that data in regards to days.

We’ve spent a lot of time fine tuning the system for better line graphs and we think you’ll like not having to worry about all those little details.

In the new Report Builder, there will be 5 types of widgets (Graphs, Charts, Numbers, Text and Datagrid) that you can add, mix and match in your reports. Because each widget type is uniquely different in their presentation and function, they have their own unique set of features and settings. The following is a look at the redesigned interface settings we’ve created for each report widget in the new Report Builder.

Graph Settings

These graph settings will be very familiar to those familiar with the old Report Builder. We’ve added a twist to one of the graph types, but we’ll talk about that in another post.

Chart Settings

One feature we’ve added to our Field Charts is the ability to selectively show certain columns if it’s using a Field Chart type that’s dedicated to showing choices. The column options will allow you to choose between All Columns, Percentage Only and Count Only.

Number Settings

The Number Settings work and feel just like our old Big Number settings, but with a different arrangement to enhance clarity and take better use of the space in the controls.

Datagrid Settings

One of things we’ve done with the new datagrid settings is do a much better job of separating the fields from the entry specific information (ie. Date Created, IP Address, etc.) and payment information (ie. total, status, merchant, etc.). This will make choosing the appropriate data to highlight a lot easier. We’ve also added a new feature that allows you to specify ahead of time which column to sort your datagrid ahead of time and whether you want it sorted in ascending or descending order. Because the datagrid widget is a resource intensive and complicated data browsing tool, you will only be able to have one datagrid on a report at a time.

Text Settings

This is a new widget type that we’ve added to the new Report Builder to add flexibility to the system. You can now easily add static text to your reports to help explain your data to your users. As you can see here, you can supplement the formating of this text with standard HTML.

Our vision for this new system was ambitious and so we’ve had to work on this project in stages. First, we started by rewriting the backend of the reporting engine so that it can take advantage of our new API system. This created a solid foundation for all the new stuff we wanted to incorporate and also had the nice side effect of making our export engine a lot more reliable. Earlier this year, we finished phase two, which launched a whole new set of widgets (field charts and big numbers) so you can show off your data in lots of new and exciting ways. The last step on this project, which we’re coming very close to releasing here, was to do a complete code and interface overhaul of Wufoo’s Report Builder.

While we’re still testing and putting the finishing touches on the new system, we thought it would be nice to take some time over the next few days to give a little preview here on the blog of the new interface and features. One of the biggest changes that you’ll see in the new interface is that it’s a lot more visual. Like Wufoo’s Form Builder and Theme Designer, the new Report Builder will be divided into two parts: a control interface that contains all the settings that you can manipulate in your report and a dynamic preview that’ll allow you to see what you’re designing. Today, we’re going to focus on the new control interface.

As you can see, we’ve decided to divide the report creation process in Wufoo into four steps with each step having it’s own specific settings. Navigating through the steps are easy. As you finish each step, just click on the next step’s title and it’ll slide out to show you it’s settings. We’ve worked really hard to make the feel of this interface very fast and very natural.

Report Settings

This is a fairly straightforward first step that’ll be familiar to everyone on the old system. This is where you’ll give your report a name and description. One new feature we’ve added here is the ability to allow or not allow your users the ability to export the data from the public report. This is good for allowing reports to show off your stuff without giving full access to your users.

Data Settings

Just like in the old Report Builder, the Data Settings are where you specify what form you want the report to base all its data processing from. There’s been a lot of goodness added to this step. One feature we’ve added from the Entry Manager is to make filters context sensitive. For example, if you’re filtering by date, different conditions will show appropriate to that property (is before, is on, is after) in addition to a date picker. This smart processing makes filtering your data a good deal more intuitive and the options available to you a lot more accessible.

Layout Settings

Again, very exciting stuff here. As you can see, reports in Wufoo will be able to take advantage of multiple premade layouts. This means you won’t have to stack all your widgets one right after each other. Instead you can use different zones to structure your widgets where you’d like.

Additionally, like in the Form Builder, we’ve got a dedicated section that’ll allow you to add your widgets where you want them by clicking or dragging them to the new Report Builder’s live preview. We’ve added all the same polish that you’ve come to expect from Form Building experience and we can’t wait for you to try it out.

That’s it for now. We hope you like what you see and we’ll be back with more soon.

This means that if you’re collecting images or documents through your form, you no longer have to log into Wufoo to access your stored files each time you need them. Now you can export your data into Excel and follow clickable links to access your files whenever you want them.

And Excel is not the only export format receiving this upgrade. CSV and Tab-Delimited exports will also include the links as well, except they’ll just be the url to the file on our servers. While not clickable like their Excel brothers, these links are still helpful for machine-parsing your File Upload info data.

So, what happened exactly?

During both outages, the problem was the same — all power went out at the data center and this resulted in all sites and services hosted there to go down in addition to ours. The downside to a power loss on this scale is that all core level services were affected. This therefore significantly increased the time to get all servers online. Networking on critical level servers had to be brought up first, and then all application level servers had to go through a crash recovery process.

We’re still working on getting a better understanding from our data center as to how and why this happened, and how they’re going to make sure this isn’t going to happen again. When we picked this data center, their power system was one of the key criteria. The main elements of their power equipment (UPS, generator and power control) are all good systems with ample capacity. We’re still trying to understand what it was about how these are put together, or how they’ve been managed, that led to the failure that we had.

What triggered the power outage?

In both cases, there was a significant reduction in the voltage to the building from the local electric utility.

Doesn’t Wufoo’s servers have a backup power supply?

We have redundant circuits on two separate power systems. Additionally, our UPS systems turned on as expected, and provided power for another hour after the outage occured. The third, and final, resort is to use a generator when a failure like this happens. Once an outage gets to the level that we had earlier this week, we must rely on the the company that runs the data center to get emergency systems like a generator up and operational. We don’t have the details yet, but there was apparently some difficulty in switching over to generator power, so the UPS reserves on our systems therefore ran out. Again, we’re seeking more details about this.

Once power came back, why was Wufoo down for so long?

It is tough to prepare for a worst case scenario, and we can assure you everyone at BitPusher (our server management team) was moving as fast as they could. Realistically, recovery from a situation like this will always take at least an hour since all core servers need to be restored and all application servers have individual crash recovery processes that have to complete. On Tuesday, the process went smoother and it took approximately an hour from when power was restored.

What improvements are you going to make?

Continuing in the current environment without significant changes that ensure this won’t happen again is unacceptable. We are examining three different directions: better power in the existing facility, moving our infrastructure to a different facility, and working with third-party hosting partners.

One thing, we’re also working on is creating better communication strategies and methods for communicating with our users during such incidents. While we responded (as always) to every single email sent in during the outage with updates as they came in, it’s good for you to know how to passively observe our actions behind the scenes.

Many of you followed our updates on on our Twitter page and our Wufoo Status blog and we think everyone liked how that worked out. Currently, we’re working on ways to enhance the Wufoo Status blog so that it provides daily updates and a feed on our uptime status and additional information. More on that to come as we enhance those processes.

Why was there no downtime page?

Normally, Wufoo has a styled downtime page that appears when a form is unavailable. These pages make the embedded forms look more professional. During these outages, there was no downtime page and every request to our servers timed out. The reason for this is because our current downtime system relies on the load balancer. Since the load balancer lost power also, we had no downtime page to serve. We’re currently looking at our options for restoring downtime functionality from another facility.

Is my data safe?

Yes. There was no data loss during the power outage and all servers successfully completed crash recovery without issue.

This experience hurt our company. We would like a refund.

We completely understand and again apologize for such inconveniences. If you feel that these outages directly interfered with your primary purpose for using Wufoo, please contact us via support and we would be happy to give you a refund for this month’s services.

End Note

During the outage, many of you were actually very supportive and we’re extremely grateful for such understanding and support. We’ve always believed that we have the best kinds of people using Wufoo and many of your actions this week served as a testament that this really is true. Our entire team sincerely thanks all of you for such patience and understanding and hope we can live up to such good treatment by minimizing these incidents to few and far between.

If you’re curious as to why we didn’t jump on the train sooner, it’s because some dude in Spain was squatting our name (we know, very weird). Many thanks to our friends at Twitter for helping us sort that out.