Working with cutting edge technology at GoldFire Studios has afforded me the opportunity to share what I've learned with the community by speaking at development conferences all over the country. I was thrilled to have the opportunity to take those lesson overseas when I spoke at the incredible Nordic.js conference in Stockholm, Sweden in October.

While I've spoken at events like SxSW, TEDx, ThunderPlains, LibertyJS and more, none could quite match the unique flair that was Nordic.js. Whether it was the saunas, giant ball pit, barber shop or tattoo parlor, the quirkiness of this conference made it truly one to remember.

But, it wouldn't be a great conference if it didn't have great speakers, and it absolutely delivered on that front as well. I was honored to be included in such a lineup where I, along with the 1200 in attendance, learned from a diverse set of speakers from all over the world.

My talk was actually inspired by our work on CasinoRPG. We ran into growing pains as the user-base expanded as well as server failures and other issues that would cause frequent outages. We needed to build a fully distributed system to achieve a more reliable and stable experience for our players.

In my talk, I demonstrate the basics of the approach we took and how it can be applied to a wide variety of applications. This begins at a high level and then deives into a practical live coding demo that puts the concepts to work. Besides the above video, you can also find the slides and code here.

Exocraft launched on the web just over a year ago, but this was only the beginning of its journey. We've been hard at work making new features, overhauling the gameplay and fixing countless bugs as we approached the full rollout of the game across all platforms.

Today is the next step of that journey as we launched on Steam, the world's largest PC/Mac games store! You get the same great experience with full cross-play support, meaning you can login to your account and play from Steam or the web with everything staying in sync. This also means that everyone playing on PC, Mac and web can play with each other. This will extend to mobile when that experience launches later this spring.

Download Exocraft on Steam

As always, we give high praise to all of the open source maintainers out there that have made it possible for us to have a Steam launch like this. As was the case with CasinoRPG, we again employed Electron as the rendering engine for the desktop versions of the game. Greenworks was another vital project that allows the Steam overlays to work with Electron. We of course continue to do our part with extensive open source onctributions back to the community, including howler.js and democracy.js.

The launch of Exocraft.io and the subsequent flood of traffic revealed the flaw of our reliance on Amazon Web Services -- bandwidth cost. Within a month of launching, our costs surged nearly 10x entirely due to bandwidth costs on our EC2 servers (it would have been worse if we weren't using CloudFlare for our static assets). So, we set out to find an alternate provider that could offer similar performance at a more predictable cost.

Methodology

No benchmark can be a perfect representation of real-world performance, but tracking multiple metrics over time gets us close enough to make some reasonable decisions. We were specifically interested in network speeds, CPU performance and disk I/O.

The virtualized nature of cloud hosting makes benchmarking over a period of time vital to getting the full picture. In order to consistently capture these values across various providers, we wrote and open-sourced a Node.js CLI tool called cloud-bench and made it available through npm. With this tool, you can easily specify a time-frame to benchmark and return a CSV with a variety of metrics:

• Network CDN Download (Mbps)
• Network Ping (seconds)
• Network Download (Mbps)
• Network Upload (Mbps)
• CPU Time (seconds)
• Read IOPS
• Write IOPS
• I/O Ping

We made the tool general-purpose not only so that it would work on a variety of cloud solutions, but also so that others could use it to test for their own purposes. Simple usage examples are included in the repo's README, making it trivial to run extended benchmarks of any cloud platforms you are considering using.

Clouds Tested

With our goal being to achieve lower and more predictable costs, we had to eliminate major players like Google and Azure, as they suffer from the same high bandwidth pricing as AWS. So, our focus turned to lower-cost providers that offer bundled bandwidth with low overage costs. The prices were all fairly similar, so all we needed to narrow down was which offered the best mix of performance across our three main metrics.

• Amazon EC2 (C5 - 2 vCPU & 4GB RAM - $62/mo)
• DigitalOcean (CPU Optimized - 2 vCPU & 4GB RAM - $40/mo)
• Linode (Linode 4GB - 2 vCPU & 4GB RAM - $20/mo)
• Packet (Tiny - 4 CPU & 8GB RAM - $50/mo)
• Vultr (60GB - 2 vCPU & 4GB RAM - $20/mo)

This isn't a perfect comparison of servers for a few reasons. First, Packet offers dedicated hardware, but the included Intel Atom C2550 doesn't stand a chance against the Xeon processors the others offer. We also tested CPU optimized servers on Amazon EC2 and DigitalOcean simply because we are looking for the best CPU performance. Even though neither Linode nor Vultr offer a CPU optimized tier, we wanted to test the options we would actually be using if we went with each provider.

Results

We ran benchmarks for 24 hours in parallel on all providers to get a sense of average performance and consistency over a full day. The following charts display the results for the main metrics collected.

The CDN download test used 5 different CDNs to benchmark download speeds from various locations around the world. DigitalOcean was the clear winner with an average speed of 47 Mbps, though all but Vultr were in a reasonable range.

Ping was calculated using the speed-test module, which requests the nearest edge node (meaning it could have been in the same datacenter). Lower is better, and all hosts were reasonably consistent.

Download speed also used the speed-test module, but in this case you can see that DigitalOcean and Amazon EC2 doubled the speeds of the other hosts. However, we must keep in mind that it is possible speedtest nodes were hosted in these datacenters.

Upload speeds paint a similar picture as downloads, though all of the hosts were even closer in range.

This test performed 2.5M random hashing and array operations and calculated the time it took to complete. Amazon EC2 slightly edged out the rest, though DigitalOcean was a close second. Linode and Vultr were nearly twice as slow, with Packet coming in nearly 5x slower (again, expected with Packet's dedicated, but slower CPU). This test really shows that the CPU optimized tiers in EC2 and DigitalOcean aren't just marketing.

All of the disk speed tests used fio to calculate IOPS. You want a higher number here, and all of the hosts outside of DigitalOcean seemed to have a cap. The Amazon EC2 cap was so low that it actually was not able to complete the test and returned no results.

Write speed had a much smaller range than read and actually favored Packet and Vultr, though by a smaller margin than DigitalOcean had on read. It is important to note that DigitalOcean's read and write IOPS were much less stable, likely due to the lack of limits like the other providers.

This was calculated using the ioping tool and is similar to network ping, but measures disk latency instead. Again, Amazon EC2 failed this test, with DigitalOcean showing by far the best consistency and speeds across the 24 hours.

The averages for each metric and each provider are outlined in the following table.

Conclusion

The goal of these tests was to find a way to lower our costs without sacrificing too much in performance. DigitalOcean accomplished both of those and more by besting EC2 in every category but CPU (which was quite close as well). We've since moved Exocraft.io to DigitalOcean and have realized an over 80% reduction in cost, which will only grow as our bandwidth usage increases. In addition to these performance/cost gains, we found DigitalOcean's interface and support to be much more approachable than the alternatives.

Some will argue that the best price-to-performance possible is with dedicated servers. This certainly can't be denied, but that ignores the many benefits hosting in the cloud offers. We've written our backend and structured our infrastructure to take full advantage of the ease at which cloud servers can be scaled up and down (both vertically and horizontally). In fact, we wrote and open-sourced another library called healthcare.js that allows you to specify your DigitalOcean infrastructure as configuration and automatically grow/shrink/heal your servers with no single points of failure.

So, while cloud hosting might not be a perfect fit for everyone, if your use-case calls for it, know that you have more viable options than just the "big three." I'd encourage you to use cloud-bench to do your own testing and find the solution that best suits your project's needs.

As we discussed in our previous post about CasinoRPG launching on mobile, our goal has always been to make CasinoRPG truly cross-platform. The idea from day one was to allow our players to access the full CasinoRPG experience no matter their device or platform of choice.

The mobile launch on iOS and Android was a big milestone, and we are excited to announce that just over a month later we've reached another big milestone. Steam is the world's largest PC/Mac game store, and CasinoRPG is now available on Steam as a free download!

Download CasinoRPG on Steam

This is the same great desktop experience that you are used to from the browser or Windows Store, but now available as a desktop application on both PC and Mac. Being the largest games store for desktop, Steam opens us up to an entirely new community that have yet to discover CasinoRPG, and we hope to bring them into our own growing community.

From a technical perspective, we made use of several open-source projects to bring CasinoRPG to Steam. First of which is Electron, which uses the rendering core of Chrome to allow desktop applications to be built with web technologies. Greenworks is another project we relied on, which allows an Electron games to connect to all of the Steam services and features. A Steam launch simply would not have been possible without the open source community, which is why we put a lot of effort into releasing our own open source projects such as howler.js, pokersolver and democracy.js.

Development began on CasinoRPG way back in the fall of 2012 with the goal of creating a truly cross-platform gaming experience. We focused on browsers because we were (and still are) believers in the power of the open web. Our Kickstarter re-emphasized this cross-platform future with stated goals of releases on both iOS and Android.

We are nearing the end of 2017, and CasinoRPG is still a largely desktop-centric experience. However, that is all about to change in a big way! We are excited to announce the release of CasinoRPG on both the iOS App Store and Android Google Play.

It hasn't been easy taking a large screen experience and making every aspect of it work on phones/tablets. We've been slowly updating every bit of the game over the last year-plus to seamlessly scale to all screen sizes. This isn't just a companion app, it is the full CasinoRPG experience.

You can play all of your favorite casino games, manage and design your casinos, access global/private chat and so much more! The most exciting part is that you don't need to start fresh on any platform. Simply login on any device with your same user account and pick up right where you left off. Unlike with many games, there is no segregation between platforms. Whether you are on your phone or computer, you are able to play with everyone in real-time.

As for how we pulled this all together, the magic lies with the open-source Cordova project. The vast majority of the work in making this possible was within the CasinoRPG code-base itself. This means that the mobile enhancements translate to the browser (as we announced earlier this year). However, we've found that mobile browser limitations have led to substandard gameplay experiences.

Moving to the app stores was the logical next step to provide the best experience for our players, and Cordova has made that possible without rewriting the entire game from scratch. We are still able to provide the same updates at the same time to all players since the vast majority of updates go directly to the web view. However, we now get access to the reach and simplicity of the App Store and Google Play.

This is a big step towards our original goal of allowing our players to play anywhere they want. But, we aren't done yet! We have a Steam version of the game in the works and we are also looking into bringing CasinoRPG to consoles in 2018. Best of all, we are able to continue using web technologies to enable these truly cross-platform gaming experiences.

When I started GoldFire Studios nearly a decade ago, the goal was always to build a community of games and the diverse people that play them. It is this goal that led us to build the GoldFire platform that we relaunched earlier this year. It is also this goal that drives us to continue creating unique social gaming experiences like the one we are announcing today.

Exocraft

With each new game, we like to explore concepts that push creative and technical boundaries. Exocraft is no exception and adds a genre to our portfolio that we love, but have yet to explore. That being said, Exocraft is still browser-first and heavily community-driven, just like everything else we've done. It is a massively multiplayer game where everyone shares the same world, albeit wrapped up in simpler mechanics than the sprawling city and casinos of CasinoRPG.

Exocraft is a sci-fi game set in a distant desert world where you can team up with friends and explore the entire planet in search of the valuable crystals it holds. Through strategy and a bit of luck, you'll gather, battle and mine with your fleet of drones in an exciting action strategy setting. Your loot from these activities can then be used to design and customize your unique spacecraft to work your way to the top.

A closed beta period is starting today, exclusively for our CasinoRPG community. Watch for open beta in the coming weeks and a full launch in early 2018.

CasinoRPG

We've been methodically growing our team and infrastructure over the last few years before even thinking about a new game. CasinoRPG and the incredible community it has spawned remains our top focus, and we've worked hard to allow ourselves to continue exploring creative expression while keeping the pedal firmly down on CasinoRPG's continued development.

When work started on CasinoRPG, GoldFire Studios was just two people. We have since tripled in size and have many incredible volunteers that make it possible for us to support independent game development. We've also worked hard to make GoldFire 2.0 a great home for our games so that minimal work is required to release and support new ones, such as Exocraft.

By growing our team and streamlining our processes and infrastructure, we are able to better serve a growing community and continue to release great new features, updates and fixes in a timely manner. There is still so much more to come for CasinoRPG, and we are excited for all of you to experience it!

Today's CasinoRPG update includes several new Arcade Machine items, which give you exclusive access to the Exocraft beta. By reaching level 10 in Exocraft, you'll also receive a limited edition Exocraft Arcade Machine item! All you have to do to enter the beta is find an Arcade Machine (or place one in your casino) and walk up to it like you would a slot or video poker machine.

One of our goals at GoldFire Studios has always been to make our games available everywhere that our players might want them to be, whether on our primary platform of the web or one of the many app stores. We’ve been making a conscious effort to ramp up those efforts in 2017, starting with the launch of GoldFire 2.0 in February, which includes full mobile and cross-platform support.

Today, we are taking the next step and launching CasinoRPG on Steam Greenlight. If you aren’t familiar with Steam, it is the world’s largest game distribution platform for PC, Mac and Linux. Getting onto Steam will help us to reach a much wider audience and make the game more accessible. Many players want all of their games to be in their Steam library, so we want to be there too.

However, launching on Greenlight doesn’t yet get us onto Steam. This is where we need your help. Greenlight is an opportunity for the community to vote on games they want to see included in the Steam Store. The more “yes” votes we get, the better our odds are of being accepted onto the platform. So, any help we can get from all of you in voting and sharing this news will go a long way towards those goals being met.

Vote on Steam Greenlight

Share on Twitter

Share on Facebook

In early 2013, I set out on a journey to simplify browser audio for our then in-development HTML5 multiplayer game, CasinoRPG. It quickly became clear that this was a problem many in the development community faced, and howler.js evolved into much more than the simple helper it set out to be. Today it is used by the likes of Google, Disney, LEGO, NASA, DC, EA, Ubisoft, Paramount, Warner Bros, Universal and thousands of other developers all over the world.

Browser audio has come a long way since that weekend project got started, and so has our understanding of how to best utilize it. Using all of the knowledge gained by the incredible community howler.js has spawned and through our own development experiences, howler.js v2.0 has been completely rewritten from the ground up to be a truly all-in-one solution for browser audio (the API changes were kept as minimal as possible, and you can find the migration guide and changelog here.

New Structure

The same goals remain: audio in the browser should be simple and effective by defaulting to Web Audio API and falling back to HTML5 Audio (now only necessary in Internet Explorer for the most part). In addition, howler.js is now a modular library, allowing for a lean core that can be extended to add more advanced functionality such as the wealth of tools available as part of the broader Web Audio API (spatial audio, filters, etc). We are launching with a spatial audio plugin, but hope to add more with the help of the community in the future.

One of the pain points of 1.x was the confusion surrounding playback of multiple sounds, sprites, etc. This mainly came about because the original release was much simpler than what things evolved into. So, with all of this in mind, a 3-layer structure became an obvious solution.

You still have the global context with Howler, but now there's the idea of groups and single instances of audio. Each time you play a sound, it keeps track of its own state, inheriting from the Howl group. This allows you to more explicitly control both the group of sounds and the individual sounds, providing more intuitive and predictable results. This also allowed for the introduction of a whole host of new helper methods that can control one or many sounds, such as once, playing, duration, state, rate and more.

VIEW FULL CHANGELOG

New Website

In addition to re-writing the library, it became apparent that howler.js deserved to have a home of its own rather than living on this blog. The idea was to showcase the library and its uses in an easily digestible format on any device/platform (just like the library itself). Plus, it was a fantastic excuse to break out my design chops, which had been growing dusty by the day!

It isn't just a new website, but also a new logo as well. Lane Brown, our fantastic art director here at GoldFire Studios, put my 1.x attempt at a logo to shame with this new mark, bringing the howler monkey and his thumping headphones to life.

The site is very minimal: highlights of major features, various live demos and a showcase of sites/apps using howler.js. The website is also hosted open-source on GitHub, so I encourage contributions, especially if you've got a cool app that I missed!

Thank you to the dozens of contributors who helped make this release a reality! The open-source community is something quite remarkable, and we continue to be excited to be a small piece of it.

UPDATE: After speaking with CloudFlare's CTO, I've been informed that they don't cache audio files (even static ones) by default. I'm in the process of re-doing the large file test and will update the results accordingly.

UPDATE 2: The charts for the large file tests have been updated and CloudFlare has gone from dead last to fairly even with MaxCDN. The lesson is that if you are going to use CloudFlare, be aware that you need to enable caching of audio files using the Page Rules feature.

Nearly two years ago we ran a 24-hour benchmark comparing CloudFlare vs CloudFront when improving load times for our HTML5 MMORPG. Since then we've seen the emergence of HTTP/2 and a bevy of CDN's stepping up their games, prompting a new set of benchmarks on a broader scale.

We are again using the Node.js CDN benchmark CLI that we open-sourced previously, but this time we are benchmarking: CloudFlare, CloudFront, MaxCDN, CDN77, KeyCDN and CDNsun. These are some of the most well known and widely used providers that serve more than just the "enterprise."

They also offer HTTP/2 support now or in the near future and free SNI SSL support, which is a must-have for our use-case. Some even have baked-in Let's Encrypt support, which isn't vital but is a nifty nice-to-have.

The Test

In order to benchmark a CDN provider, you must record performance over time and not just after a handful of downloads. The main purpose of a CDN is global speed, so it is also important to put multiple POPs to the test when gathering your data.

cdn-bench combined with a few Digital Ocean droplets in different zones makes this trivial. Using the below command, we ran a 24-hour benchmark of all 6 CDNs from 3 locations around the world: Oklahoma City (in our office), London and Frankfurt.

node cdn-bench
    --files https://cloudflare.example.com/large,https://cloudfront.example.com/large,...
    --interval 600
    --limit 144
    --out results.csv

This downloads the listed files, one after the other, every 10 minutes over a 24 hour period. It then appends the download times to a CSV file for analysis. We loaded the same 3 files into each CDN: a small image (12 KB), a medium image (199 KB) and a large audio file (4.1 MB).

The Results

After the tests completed, we combined the CSV output from each server and averaged them into the below charts (click the legends to enable/disable each CDN). These charts help to visualize the relative performance of each CDN over a 24-hour period, which we found particularly useful when comparing the consistency and speed throughout the day.

We also calculated average download times and standard deviations for each. We found the standard deviations especially important, as it is indicative of how consistent our customer experience might be.

Conclusions

The results aren't quite as clear-cut as they were on our first test. CloudFlare seems to have fixed their consistency issues, especially on smaller files. However, they were consistently awful when it came to downloading the large file. CloudFlare appears to be the winner since last time, fixing their consistency issues and improving overall speed and standard deviation across the board. CloudFront (our previous winner) didn't show much difference, if only a little less consistent than before.

The overall winner appears to be one of the standout names in the industry: MaxCDN. They seem to be a solid choice for any type of workload, though we found their pricing to be somewhat deceptive and not nearly as competitive as some of the other providers (unless you have "enterprise" scale). They also don't yet support HTTP/2, though promise it'll be coming later in 2016.

In the end, we decided to move some of our small files back to CloudFlare (where bandwidth costs nothing) and everything else to KeyCDN. Besides free, KeyCDN's pricing was as good as it comes. They were also quite respectable on speed and consistency, include HTTP/2 support and include the aforementioned Let's Encrypt integration.

In the end, we decided to move everything back to CloudFlare from CloudFront. You can't beat free bandwidth, they already have full SSL and HTTP/2 support and the speeds across the board are on-par with the best.

Ultimately, you couldn't go wrong with most of these options depending on your workload and budget needs. This wasn't an all-encompassing test, so if there are other CDN's you think should be tested, we'd love to see your results in the comments!

I've always had a fascination with startup offices. Creating the right environment that props up your unique culture is vital to long-term success. Not only is it important for the creative process, but also for the competitive market of tech hiring.

Being a bootstrapped startup, our first two offices were pretty limited. We are still mostly bootstrapped today, but we've found success over the last year that has allowed us a little more freedom--yet still tightly within a startup budget of around $2,000. This is how we went about assembling a $12,000 office for under $2,000.

Finding the Right Space

The design costs for a new office may seem daunting, but they pale in comparison to the long-term cost of leasing a space. It is vital that you not only find a space to grow into, but also one that doesn't lock you in past your reasonably foreseeable runway.

View from the new GoldFire HQ over downtown OKC.

We knew we wanted to be in the downtown area where there's a palpable energy present. However, the market in downtown OKC has been on a rocket-ship of late. Luckily, with a little patience and 3 months searching, the perfect opportunity revealed itself.

Blank Canvas

We ended up with a beautiful space (or rather, a space that we believed could become beautiful) with an incredible view. We also managed to get favorable terms on a short-term lease, the negotiation of which is outside the scope of this post and will vary by market.

Gotta love that pink carpet!

Original work space.

The office may not have looked like much when we landed it, but we negotiated all-new carpet and paint throughout. This is where we started our design process with Interior Design for iPad.

Basic design mockup for the new GoldFire offices.

Desks/Chairs

There are areas of an office where you can get away with cheaper items, but desks and chairs (which have a direct impact on productivity) aren't those items. Good, sturdy desks with plenty of surface area are a must, as are comfortable chairs rated for long hours.

However, pre-built desks from quality sources can easily cost $1,000 or more. So, I put on my DIY-er hat and started looking through the IKEA catalog. Sure, they have relatively cheap desks available, but they are cheap for a reason.

The great thing about IKEA is that they also sell their furniture in pieces. You can get six-foot, solid wood butcher-blocks for around $90. Combine that with stainless steel hairpin legs from Etsy and you've got yourself some functional and beautiful desks for only $190/ea.

The end result of our IKEA/Etsy mix-and-match desks.

Unfortunately, there's not much mixing-and-matching to be done with desk chairs, so a good chunk of the budget will go into these. There's some great chairs available in the $100-range, and there are some immaculate chairs available in the $1,000-range.

Obviously, the high-end was out of the question, but we wanted to find that sweet spot in the middle. With a little patience (key word in this whole process), we found some generous coupons at Staples and picked up some $300 chairs for $200/ea (TempurPedic TP9000).

Plenty of space to grow.

Accent walls in bright GoldFire red.

Before/after of donated filing cabinet now in GoldFire colors after a few coats of spray paint.

Other Rooms

With our work area filled out and still $1,220 left in the budget, we turned to the more exciting areas. To fill out the break room, kitchen and conference area, we turned to Craigslist, friends and family.

We were able to score a Killer Instinct arcade cabinet ($1,000 value), modern TV stand ($1,500 value), 42" TV ($1,000 value), refrigerator ($300 value) and two sturdy tables ($1,000 value each) for just $300! We then filled in the gaps from sites like Overstock and Amazon for another $500 for 6 chairs and a side table.

New entrance view.

Arcade cabinet from Craigslist.

"Hall of Games" - what will we add next?

Break room with comfy chairs, Xbox and N64.

Decorations

Furniture is of course important, but a startup office is largely about the culture. This means the decorations are just as important and are also an area where you can really stretch the dollar.

We still had a budget of $420 left, so we got to work scouring Craigslist, eBay and local vintage video game stores. We were able to find 16 frames on eBay for around $150 and a pack of vintage posters for practically nothing. We also picked up 5 authentic arcade marquees for $50 to really set the kitchen apart.

Kitchen and conference room in one.

Plenty of snacks, stickers and marquees.

The centerpiece is our logo on the front wall. Laser-cut acrylic or metal would have cost anywhere from $1,000-$3,000. Instead, we found an online wood-cutter that cut it from MDF-board for just $70. We then bought 2 cans of glossy white spray paint for another $6 and had 3D logo letters on our wall for a fraction of the cost.

3D logo letters for $76 instead of thousands.

Overview

Setting up a new office space that both feels at home and is a productive space to work can be a daunting task--especially if your budget is severely limited. However, with a little bit of creativity and even more patience, you can create a space to be inspired in, and one that can make all the difference as you progress through your startup journey.

If you noticed the big empty space in our work room and feel like you could be just the thing to fill that space, we are hiring! Check out our jobs page for details.