AppHarbor Blog

Announcing SPDY support

Today we're announcing SPDY protocol support for your AppHarbor applications. SPDY is an experimental protocol pioneered by Google which is designed to minimize latency of web pages. It is available immediately on all shared and dedicated load balancers in the US and EU regions. Among other things it allows for efficient use of available bandwidth by multiplexing concurrent streams over a single TCP connection and compressing HTTP headers. Here's an example of SPDY in action from last year's Google I/O:

The performance benefits that can be realized with SPDY of course depends on your application and user demographic. If your web pages include many resources (javascript, css, images etc) served directly from your domain you'll likely see some pretty significant performance improvements in supported browsers. For instance, SPDY can eliminate the need for CSS sprites as all images can be downloaded in parallel. If you use a CDN the performance benefits will be limited as the resources fetched externally won't be served through SPDY (unless the CDN supports it). You'll also need to use HTTPS since SPDY relies on SSL. AppHarbor offer piggyback, SNI and IP-based SSL to help you with that.

Chrome and newer versions of Firefox and Opera supports SPDY and it seems like Internet Explorer 11 will support it too. AppHarbor supports draft 2 of the SPDY protocol. We also recommend reading the SPDY white paper and this CloudFlare blog post to learn more about SPDY.

To enable SPDY on an existing application simply click the button on your application's settings page. SPDY is enabled by default on all new applications. Since the protocol and the nginx implementation we use are still considered experimental it'll continue to be configurable for a while.

You can test your application on SPDYCheck.org or you can install the SPDY indicator for Chrome or Firefox to confirm that SPDY is enabled.

Cloudinary add-on now available

We've just added Cloudinary to the add-on catalog. Cloudinary makes working with images in the cloud a breeze and offers a wide array of image manipulation services. They also have a .NET SDK available making it easy for AppHarbor customers to take advantage of their services.

Working with images can be a hassle and we know some of you already use Cloudinary to take the pain out of transforming images. We're excited to offer easy integration with this simple and flexible service and hope you'll find it useful for your applications. Cloudinary has published thorough documentation and samples showing how to use the add-on with AppHarbor - make sure to take a look when you try it out.

Introducing Vertical Scaling - Make Your Workers Better, Faster and Stronger

Today we're introducing vertical scaling of workers, so you can get web and background workers with more resources. You can now get workers with two, three and four times the limits of a normal worker unit. We’ve tested this feature for the last couple of months with some large customers and the results have been great.

An application is scaled on the app’s subscription page. When you scale your workers vertically you’ll get better response times, request throughput, application stability and ultimately an improved user experience. Scaling vertically also allows you to deploy larger and more complex applications on AppHarbor.

Vertical scaling can do wonders for your apps, but we still want to encourage developers to design apps that are ready for horizontal scaling. The principles outlined in “The 12 Factor App” by Adam Wiggins are excellent guidelines to archive that (among other things), and if you haven’t read it already we strongly recommend it.

The impact of vertical scaling depends on how resource intensive the app is. Here is a screenshot from New Relic showing how appharbor.com performed after tripling each worker's resource allocation:

About 80-100ms (~40%) was cut of the average response time, and notice that the New Relic apdex score is greatly improved. In the meanwhile the request throughput was doubled. You can try for yourself, and see how vertical scaling makes your apps perform better.

Some scenarios where this feature is particularly useful:

You've exhausted the resources of a single worker unit but can’t scale horizontally.
You have already scaled out to multiple workers, and need to handle even more requests.
You want to improve page load times.
You have CPU/RAM intensive web or background workers.
You need to scale more to more than 20 regular web or background workers.
You application requires many concurrent connections and a higher request queue limit - for example for apps that extensively uses WebSocket and long-polling.
Background workers - redundancy might not be as critical here and a single, large worker will often perform significantly better.
Each of your background workers runs a large number of processes. With vertical scaling you get fewer, larger workers that can better run many processes.

Remember that all prices on AppHarbor are prorated and you only pay for the time you have a resource provisioned, not for the entire month - so you can scale your app vertically as well as horizontally cheaply and when you need it.

With today’s release we’re also changing the plans a little - the plans now include “worker units” rather than “workers”. This means that pricing remains the same, but you can use the worker units to scale both up and out. For instance if you’re on the Yacht plan, which includes 4 worker units, you can now scale to two workers with two worker units allocated for each - or 1 large worker with 4 worker units.

We’re very excited to make this feature available immediately. It’s a powerful and flexible addition to AppHarbor’s “scaling toolbox”. We welcome any and all feedback so we can continue to it improve it, and if you need help or advice on how to scale your app you’re more than welcome to reach out.

Now go scale those apps.

Hosted Graphite now in add-on catalog

Today we're welcoming the Hosted Graphite add-on. Hosted Graphite offers simple, cloud-based Graphite hosting to support real-time metrics from your applications.

A Graphite.NET client library is available here and is can be installed using the NuGet package too. Provision the add-on and configure the client to start sending metrics:

var apiKey = ConfigurationManager.AppSettings.Get("HOSTEDGRAPHITE_APIKEY");
using (var client = new GraphiteTcpClient("carbon.hostedgraphite.com",
    port: 2003, keyPrefix: string.Format("{0}", apiKey)))
{
    client.Send("users", users.Count());
}

Metrics can also be sent via UDP and you can read more about it in the documentation.

Have fun!

WebSocket support for .NET and node.js apps

We're happy to finally introduce (experimental) WebSocket support today. You can now select the "Beta" region when creating an application to try our latest environment, and we hope you'll help us test it with your own .NET and node.js apps.

The "Beta" region runs Windows Server 2012 application instances in the US (AWS US-EAST) with IIS8, node.js (0.8.20) and iisnode (0.2.3) and is otherwise identical to the current environment. Along with nginx this allows you to leverage WebSockets for your .NET and node.js apps.

We're still ironing out some kinks before we're ready to fully migrate to WS2012 and IIS8 and the WebSocket-enabled version of nginx was just released - so it will be a little while before we feel confident about migrating all apps. With the new region you can try how your applications actually work before we make the move to the updated environment. It will be updated frequently and is generally less available than our regular US and EU regions, so make sure not to use it for your production apps.

Here are a couple of sample apps using WebSockets on AppHarbor:

JabbR chat app clone based on SignalR - just create a pair of test credentials when trying it: You can use the JabbR repository as-is, just provision a SQL Server database, set the connection string "Alias" to "Jabbr" and make sure to enable file system write access on the application. Source code is available here.
Node.js app using Socket.IO: Very simply websocket sample using iisnode. Source code available here. Note that you currently need to configure the app with a web.config file and may have to disable view precompilation under "Settings". More about node.js on AppHarbor here.

That's it! Take it for a spin and we look forward to your feedback.

Loader.IO add-on now available

Today we've added the loader.io add-on to the add-on catalog. Loader.io allows you to test and measure web application performance under load. With loader.io you can among other things run tests with up to 10,000 concurrent connections with SSL support. And it's free!

Get started by installing the add-on from the catalog and go to the loader.io admin page. Before you can use it you need to configure either your application or DNS, so your ownership can be established. You can read more details on Loader.io when you create tests.

Loader.io is a new product from SendGrid labs, a group under SendGrid, the e-mail platform, which is also available in the add-on catalog. SendGrid Labs aims to "help developers be more successful by providing technologies and services we've identified as pain points while focusing on building and scaling our engineering organization as we've grown as a company" says Mike Rowan, VP R&D for SendGrid.

We're excited to have the loader.io add-on available and are looking forward to see what comes out of SendGrid Labs in the future.

CloudBird RTM

CloudBird, the cloud RavenDB service recently "peeled off the beta label" and now welcome production applications to use their service.

Andrew Smith from CloudBird commented on the launch:

We're delighted to launch our CloudBird AddOn in the EU, offering production-ready RavenDB hosting in the cloud. We're launching with full support for RavenDB 2.0 and the ability to easily upgrade existing 1.0 databases.

We have a variety of plans to suit your storage, performance and replication requirements starting at just $9.99 a month, as well as a free Starter plan to get your project off the ground.

We perform twice-daily internal backups for all databases and most plans also offer the ability to do hourly incremental exports to your own Amazon S3 bucket.

It's great to see more providers for RavenDB and we're happy to have the CloudBird service available in our add-on catalog. We wish them all the best with the launch!

Visual Studio Git Integration

Microsoft today announced that they'll be supporting Git in Visual Studio and Team Foundation Server. We care a lot about Git and providing a great user experience to developers, so we think it's great that Microsoft is adding first class support for Git in Visual Studio. Many developers have asked us for a way to deploy directly from Visual Studio, and this is one way to do it.

Luckily Microsoft has provided a CTP for VS 2012 so you can try it right away. A couple of steps are necessary to get started, but it's all covered in the beginning of this tutorial. Also make sure to read Scott Hanselman's blog post about the integration. In this post we'll just cover a couple of steps necessary to set the integration up with AppHarbor and share a few initial impressions.

Adding your AppHarbor repository is straight forward: If you have already set up a a remote AppHarbor/GitHub/Bitbucket repository in your local Git repository it should work without any further configuration. If not you simply need to copy the "Repository URL" from your application on AppHarbor. Then go to the "Connect" page in the VS Git tool, select "Clone" and paste the URL underneath:

You're now set and can start using Visual Studio to deploy to AppHarbor.

Our first experience with the integration is that it seems to work pretty well and is stable. It doesn't allow developers to do too much Git magic/harm and generally applies some sensible defaults (auto-crlf on by default, .gitignore file). You can store your credentials the first time you connect to your AppHarbor repository, so you'll only have to write your password once which is very convenient. The default Visual Studio merge tool is well-integrated (and arguably better than the default "Git Gui") and will likely make more Windows developers feel comfortable about resolving Git merge conflicts.

Git integration for Visual Studio is a big step in the right direction and a very welcome addition to our favorite IDE. We're excited that git support on Windows is evolving at a rapid clip, and if you haven't already we recommend that you also check out GitHub for Windows.

Browser uploads to AWS S3 and Google Cloud Storage using CORS and ASP.NET MVC

Ever-increasing requirements to application scalability and availability often requires applications to be deployed to multiple machines. Many cloud hosting providers, such as AppHarbor, even encourage developers to write apps that are mostly stateless and designed to scale horizontally.

This poses challenges when building applications that interacts with files: Files that are uploaded to a web service needs to be available to all instances that needs to serve them. This is difficult to archive if the files are written to the local file system of the instance that the file is uploaded to, which is also why we and other hosts restrict file system write access by default.

Amazon S3 and Google Cloud Storage supports CORS (Cross-origin resource sharing), which allows us to work with files in a flexible, asynchronous manner and provide our users with a better experience. In this post we'll show how to leverage CORS and the direct upload options available, and introduce a small new .NET library, Nupload, that makes working with GCS, S3, CORS and ASP.NET MVC even easier.

We'll walk through 5 steps to upload and serve files:

Configuring CORS on S3 and GCS
Server side form and signature generation.
Asynchronous, CORS-enabled upload of files directly to S3 or GCS using Javascript.
Notifying our application about successful uploads.
Server side signing of URLs to serve private content to a client.

The source code is available on GitHub and includes a sample application - the sample application is also deployed to AppHarbor [so you can go and play with it right away. In this blog post we'll use the code in the sample application, but you can also just download the sample project and try it out for yourself. You'll need to set your own AWS or GCS credentials and configure the CORS policy of your bucket.

Before diving into the code lets walk through the configuration of CORS policies on AWS and GCS respectively.

Step 1: Configuring CORS policy

Below are a couple of samples for CORS configuration. Please note that in both cases you'd likely want to set the allowed origin parameters to something more restrictive than "*" - like the domain name of your application.

AWS configuration:

Go to the AWS S3 management console, create a bucket and click the "Add CORS configuration" button. The CORS configuration can look similar to this:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <AllowedMethod>POST</AllowedMethod>
        <AllowedMethod>PUT</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

For security reasons we also recommend that you use AWS IAM to generate credentials that are only allowed to perform the bucket operations you need them for.

GCS configuration:

Use the gsutil tool to configure your bucket with a CORS configuration like so:

gsutil setcors corsconfig.xml gs://corssample

Where "corssample" is the name of your GCS bucket and corsconfig.xml contains something like:

<?xml version="1.0" encoding="UTF-8"?>
<CorsConfig>
    <Cors>
        <Origins>
            <Origin>*</Origin>
        </Origins>
        <Methods>
            <Method>GET</Method>
            <Method>HEAD</Method>
            <Method>POST</Method>
            <Method>DELETE</Method>
            <Method>PUT</Method>
        </Methods>
        <ResponseHeaders>
            <ResponseHeader>*</ResponseHeader>
        </ResponseHeaders>
        <MaxAgeSec>10</MaxAgeSec>
    </Cors>
</CorsConfig>

With this in place we're ready to upload files to our bucket using Nupload, ASP.NET MVC (2 or later supported out of the box) and CORS.

Step 2: Server side form and signature generation

S3 and GCS both work by POSTing files using a signature for authentication. The signature ensures that even though the clients have access to certain configuration fields, they won't be able to modify them. We need to do sign our signatures with private keys, so for security reasons this is best handled on the server side.

First we can install the Nupload library in our MVC web project. MVC2 and forward is supported out of the box, but it can easily be extended to other web application types. Nupload is available as a NuGet package which can easily be installed:

 Install-Package Nupload

Using Nupload we can now configure the client uploads to our cloud storage provider like so:

public ActionResult Create()
{
    var randomStringGenerator = new RandomStringGenerator();
    var objectKey = string.Format("uploads/{0}/${{filename}}", randomStringGenerator.GenerateString(16));
    var bucket = "corssample";

    var maxFileSize = 512 * 1024 * 1024;
    var objectConfiguration = new AmazonS3ObjectConfiguration(objectKey, AmazonS3CannedAcl.Private, maxFileSize);

    var credentials = new AmazonCredentials(
                    ConfigurationManager.AppSettings.Get("amazon.access_key_id"),
                    ConfigurationManager.AppSettings.Get("amazon.secret_access_key"));

    var uploadConfiguration = new AmazonS3UploadConfiguration(credentials, bucket, objectConfiguration);

    return View(uploadConfiguration);
}

Here an AmazonS3UploadConfiguration object, which implement the IObjectConfiguration interface) is passed to the view. Object access settings are configured and can be set to any of the S3 Canned ACL grants.

If you prefer using GCS we've also included a GoogleCloudStorageUploadConfiguration implementation in Nupload, but you're free to add your own implementation to suit your requirements.

By using the upload configuration that is passed from the controller to our view, we can now implement the upload form in the "New" painting view:

@using Dupload

@using (Html.BeginAsyncUploadForm(new Uri(Url.Action("create"), UriKind.Relative), Model,
    new { id = "fileupload", data_as = "imageUrl" })) {

    <input id="file" multiple="multiple" name="file" type="file" />
}

The BeginAsyncUploadForm HtmlHelper extension method is included as part of the Nupload package. It takes care of extracting and writing all the required form attributes and fields based on our upload configuration. The first argument is the application's "callback URL", which will be set to the data-post attribute. The data_as attribute is also set for use later on.

This will render HTML similar to this:

<form accept-charset="utf-8" action="https://corssample.s3.amazonaws.com/" data-as="imageUrl" data-post="/Paintings/create" enctype="multipart/form-data" id="fileupload" method="post">
    <input id="key" name="key" type="hidden" value="uploads/osqSbYGNNWJXH4Go/${filename}">
    <input id="acl" name="acl" type="hidden" value="private">
    <input id="policy" name="policy" type="hidden" value="eyJleHBpcmF0aW9uIjoiMjAxabY0xMi0xMVQwMjo0MDo0NVoq$LCJjb25kaXRpb25zIjpbWyJzdGFydHMtd2l0aCIsIiRrZXkiLCIiXSxbInN0YXJ0cy121XRoIiwiJGNvbnRlbnQtdHlwZSIsIiJdLFsic3RhcnRzLXdpdGgiLCIkYWNsIiwicHJpdmF0ZSJdLFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLDAsNTM2ODtwOTEyXSx7InN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyI6IjIwMSa9LssiYnVja2V0IjoiY29yc3NhbXBsZSJ9LHsiYWNsIjoicHJpdmF0ZSJ9XX0=">
    <input id="signature" name="signature" type="hidden" value="P/cuc28kR4gc/sLeK1U7XS3gmGd4=">
    <input id="content-type" name="content-type" type="hidden" value="application/octet-stream">
    <input id="AWSAccessKeyId" name="AWSAccessKeyId" type="hidden" value="AKIAJMTMMPVNWB73LE4A">
    <input id="success_action_status" name="success_action_status" type="hidden" value="201">
    <input id="file" multiple="multiple" name="file" type="file">
</form>

We're now ready to implement the client side logic that'll upload files to our storage provider.

Step 3: Uploading files from the client

Files can be uploaded in a number of ways and in this example we'll use the jQuery FileUpload plugin. Here is some of the javascript included in the sample project:

$('#fileupload').fileupload({
    dataType: "xml",
    add: function (e, data) {
        var file, types;
        types = /(\.|\/)(gif|jpe?g|png)$/i;
        file = data.files[0];
        if (types.test(file.type) || types.test(file.name)) {
            data.context = $(tmpl("template-upload", file));
            $('#fileupload').append(data.context);
            data.form.find('#content-type').val(file.type);

            return data.submit();
        } else {
            return alert("" + file.name + " is not a gif, jpeg, or png image file");
        }
    },
    done: function (e, data) {
        var content, domain, file, path, to;
        file = data.files[0];
        to = $('#fileupload').data('post');
        content = {};

        var location = $(data.result).find("Location")[0].textContent;
        content[$('#fileupload').data('as')] = decodeURIComponent(location);
        $.post(to, content, function (data, statusText, xhr) {
            $.get(xhr.getResponseHeader("location"), function (painting) {
                $("#uploadedPaintings").append($(tmpl("template-painting", painting)));
            });
        });
        if (data.context) {
            return data.context.remove();
        }
    },
});

Let's go through some of the steps:

When files are added the "add" callback is invoked. In this case we're testing the file ending, but you can of course put any validation logic you like here.
If the file type is valid we set the "content-type" form field to the actual mime type of the file. By default the content-type field is set to application/octet-stream, but we want to set the specific content type so the cloud storage provider can use it for serving content.
Finally the form is submitted and the upload is initiated.

Step 4: Notifying our application about the uploaded files

When the files have been uploaded we want to notify our application about it. That is also handled by the javascript above:

The done callback is invoked when the upload finishes. At this point we use the response from our storage provider to get the exact file URL. We then POST a request to the data-post (callback) URL. The parameter name for the object URL is set to the value of the data-as attribute, in this case "imageUrl".
Finally a javascript template is rendered to show the user the new file.

The controller action that takes this request could look similar to this (assuming the Painting model has an ImageUrl parameter):

[HttpPost]
public ActionResult Create(Painting painting)
{
    if (ModelState.IsValid)
    {
        var imageUri = new Uri(painting.ImageUrl);
        painting.Name = Server.UrlDecode(imageUri.Segments.Last());

        _databaseContext.Paintings.Add(painting);
        _databaseContext.SaveChanges();

        Response.RedirectLocation = Url.Action("details", new { id = painting.Id });
        return new HttpStatusCodeResult((int)HttpStatusCode.Created);
    }

    return View(painting);
}

We simply set the URL decoded name of the Painting object, save it to the database and redirect the client to the "Details" action.

Our application now knows about the new Painting that was uploaded, and it can be served directly from the cloud storage provider by specifying the image url. In case you want to modify files (like creating thumbnails) we recommend that you do this asynchronously, for instance by using AppHarbor's background workers.

Step 5: Authorizing file downloads

In a lot of cases we don't want files to be publicly accessible. We want to restrict access only to authorized users. As you may have noticed we actually set the access grant to "Private" in step 2, which means that only the bucket owner has access to the file. This also means that although a client can upload files like we saw in step 3, he cannot retrieve it afterwards even with the URL. This gives us much of the same access control that we'd have if we stored the files locally.

We can authorize access to private files in our bucket by using signed URLs. Signed URLs allows us to give our clients a URL with a signature parameter, which authorizes access to the resource given a signature expiration and a method. We've included "URL signers" for AWS and GCS in Dupload, which can be used like so:

public ActionResult Details(int id)
{
    var painting = _databaseContext.Paintings.Find(id);

    var credentials = new AmazonCredentials(
                    ConfigurationManager.AppSettings.Get("amazon.access_key_id"),
                    ConfigurationManager.AppSettings.Get("amazon.secret_access_key"));
    var urlSigner = new AmazonS3UrlSigner(credentials);
    var signedUrl = urlSigner.GetSignedUrl(new Uri(painting.ImageUrl), TimeSpan.FromMinutes(20));

    return Json(new
    {
        id = painting.Id,
        signedUrl = signedUrl,
        name = painting.Name
    },
    JsonRequestBehavior.AllowGet);
}

This will give us a signed URL, which will be valid for 20 minutes which should be plenty of time to retrieve an image.

In the example application we use the returned JSON object from the Details action to render our template. The template can look like so:

<script id="template-painting" type="text/x-tmpl">
    <div class="painting">
        <img src="{%=o.signedUrl %}" alt="{%=o.name %}">
        <span>{%= o.name %}</span>
    </div>
</script>

That's it. The application can now handle uploads to Amazon S3 using CORS, notify our application about the new file upload and afterwards serve it using a signed URL.

We hope you'll find good use of the Nupload library and please feel free to contribute with anything you think is missing. We can also recommend watching this railscasts episode if you'd like to learn more about uploading to Amazon S3.

ElephantSQL now in add-on catalog

We have a new add-on ready for you: ElephantSQL – cloud-hosted PostgreSQL as-a-service.

PostgreSQL is a stable, mature and high-performance open source RDMS and the guys at ElephantSQL have done a great job of packaging that into an awesome service.

We have updated our PostgreSQL-NHibernate example MVC app to also work with ElephantSQL and there's additional .NET sample code and documentation on the ElephantSQL website.

ElephantSQL has a free plan to get you started, try it out right now!