First things first - If you haven’t registered for the Polyglot Conference yet, get registered ASAP as it could sell out!

I have a workshop coming up on Friday of this week at the Vancouver BC Polyglot Conf. In addition I’ll be attending the conference itself and looking to have a lot of conversations and open spaces sessions around distributed systems, technologies, platform as a service, other *aaS and generally anything for modern, distributed, scalable architecture and systems. Hopefully I’ll get to converse with all of you about as many topics as possible  - it’s always a great time to dig into these topics at conferences like these. It’s where the real next steps in technology come from!

In the coming days before and after the workshop I’ll be blogging material I have put together for the workshop. Previously I’ve put together one blog entry on backing up distributed systems ala the Riak variety. Check it out “Backup Riak – Learning About Distributed Databases :: Issue 001“.

The Workshop, for those of you interested in surviving a Zombie Apocalypse (or just learning about Riak) can sign up still, there’s a few spaces left! Below is a the basic outline of what we’ll be touching upon. There’s a LOT more topics beyond this, but we’ll save those for open space sessions. If there’s anything specific you’d like me to cover, feel free to throw a comment on the blog and I can easily add 1-2 topics of interest if attendees would like!

• Introducing Riak, a database designed to survive the Zombie Plague. Riak Architecture & 5 Minute History of Riak & Zombies.

Architecture deep dive:

• Consistent Hashing, managing to track changes when your kill zone is littered with Zombies.
• Intelligent Replication, managing your data against each of your bunkers.
• Data Re-distribution, sometimes they overtake a bunker, how your data is re-distributed.
• Short Erlang Introduction, a language fit for managing post-civil society.
• Getting Erlang

Installing Riak on…

• Ubuntu, RHEL & the Linux Variety.
• OS-X, the only user centered computers to survive the apocolypse.
• From source, maintained and modernized for humanities survival.
• Upgrading Riak, when a bunker is retaken from the zomibes, it’s time to update your Riak.
• Setting up

Devrel – A developer’s machine w/ Riak – how to manage without zombie bunkers.

• 5 nodes, a basic cluster
• Operating Riak
• Starting, stopping, and restarting
• Scaling up & out
• Managing uptime & data integrity
• Accessing & writing data

Polyglot client libraries

• JavaScript/Node.js & Erlang for the zombie curing mad scientists.
• C#/.NET & Java for the zombie creating corporations.
• Others, for those trying to just survive the zombie apocolypse.

Cheers!

Adam Baldwin is presenting…

Introducing NodeSecurity.io

Adam Baldwin

Adam Baldwin is a web app hacker, team lead at ^Lift Security and the CSO for &yet. Adam has presented at various security & dev conferences in the past including, DEFCON, Djangocon, Toorcamp and RealtimeConf.

The node.js community is growing at an amazing rate. At the time of writing there was 27,757 modules publised on npm. Have you ever stopped to think just what you are putting into your project when you npm install somebody else’s module? Do you trust that code? This is an insane project to find out the answer to that question.

This talk will introduce the nodesecurity.io project, it’s goals, current results in hopes of inspiring involvement and receiving feedback directly from the node community!

James Halliday is presenting…

beep boop

James

Oh hello. I write too much code. I co-founded browserling. Here are some pretty pictures.

Unix philosopher and methodological reductionist etc.

Learn how to make computer sounds in node and the browser with the same api.

Using just a single function that takes a parameter t, time in seconds, and returns an amplitude between -1 and 1, inclusive, you can create music!

You can use this basic approach to write songs and synthesizers. In javascript. Yay!

Ryan Jarvinen is presenting…

Clustering Node.js on OpenShift

Ryan

Ryan Jarvinen is an Open Platform Advocate working with RedHat’s OpenShift team. He lives in Oakland, California and is passionate about open source, open standards, open government, and digital rights. You can reach him as ‘ryanj’ on twitter, github, and IRC.

Learn how to automate builds, deployment tasks, and application scaling as we use OpenShift’s platform architecture on-demand to build your own git-based release pipeline, including: development, testing, staging, and cloud-scaling production environments for node.js.

Slides:

• http://darkslides-rjdemo.rhcloud.com
• https://github.com/ryanj/darkslides/

Posts:

• https://www.openshift.com/blogs/secret-free-source-on-paas
• https://www.openshift.com/blogs/release-management-in-the-cloud

An adaptation of this talk was presented recently at HTML5DevConf in SF - http://html5devconf.com/sessions.html#r_jarvinen

Intro to Sails.js

Mike McNeil is presenting…

Mike

Mike autobiogrophies himself as, “I’m Mike, a developreneur based out of Austin, Texas and connoisseur of fine code. I’m also the creator of Sails.js, the open-source BaaS framework which allows front-end developers to build robust, scalable APIs using only JavaScript.

My first startup was in social television, where I saw the need for more efficient, easy-to-use solutions for realtime social features. Because of that, I got involved in Node.js early on, and after building a few early apps, recognized the need for an MVC solution to normalize patterns. Early last year, I founded Balderdash, a UX-focused mobile and web studio, which has given me an excellent opportunity to build out and utilize Sails.js in production.“

Sails.js makes it easy to build custom, enterprise-grade Node.js apps. It is designed to resemble the MVC architecture from frameworks like Ruby on Rails, but with support for the more modern, data-oriented style of web app development. It’s especially good for building realtime features like chat.

Sails empowers UX and design teams to build hi-fi prototypes in no time without waiting for the back-end to be finished. This means focusing more resources on the user experience, which means better products. One Sails.js project at a time, companies move their legacy architecture over to a simpler, more efficient Node.js cloud. Each new client-side code base is more maintainable, since it’s built using the universal language of the internet: a RESTful JSON API.

Chris Dickinson is presenting…

Implementing Git in JavaScript & the Browser: A Case Study

Chris

Chris describes himself as “I make silly things with JavaScript: I particularly love bit-twiddling and WebGL-based projects. I live in Portland OR and work at Urban Airship as a JavaScript engineer.”

Git is one of my favorite things to hack on. It’s long been my goal to get a working (workable?) implementation of git running in pure JS, in the browser. My first attempt two years ago failed; and for a long time I’ve let the thought bounce around in the back of my head.

Spurred on by the recent interest in js-git, I recently restarted the journey towards an in-browser git, in order to help creationix deliver the best possible js-git. Newly armed with browserify and the small-module ethos, I’ve come much closer to a working git in browser and Node, and in the process have really put browserify and its shims through their paces.”

This talk will be comprised of:

A quick intro to the git object model and transport protocol
How browserify and the small module ethos have enabled great successes in the project.
Difficulties encountered in the process, both with Node.JS itself and with browserify, and how I’ve worked through them.
How I’ve diagnosed and worked through various performance issues.
Where is this project going?

Horse JS is presenting…

JavaScript, This is Confusing

….Horse JS Tweets, nuff’ said.  Horsing around…

Are you signed up?  BUY YOUR TICKET FOR NODE PDX HERE

Want to learn more? http://nodepdx.org/

Want to know the dates? http://nodepdx.org/

Want to know who else is speaking? Stay tuned here or go check out http://nodepdx.org/!

Zach Bobb is presenting…

Building a Computer In Your Browser

Zach the TriMet Ticket Man!

Zach is a mobile engineer with GlobeSherpa working hard to bring you the app that will let you buy TriMet tickets on your phone.

Want to learn how computers work under the hood while learning Web technologies? Come learn about the Von Neumann 51, a series of tutorials that will take you through building a computer from the ground up in JavaScript, HTML5, and CSS3.

The talk will start by laying out some of the motivation behind the project and then demonstrate some of the components people will build through the tutorials: logic gates, adder circuits, displays, etc. The end of the talk will demo a prototype of the vN51.

Some of the technologies in this talk are:  D3 | A JavaScript library for dynamic data-driven SVG graphics HTML5 Canvas | Pixel manipulation for display output

Paul Jungwirth is presenting…

Handling Errors with Cluster and Domains

Paul

Paul will jump into,

“Node’s callback pattern makes error handling difficult: throwing an exception kills the entire node process, terminating all current requests, and every callback initiates a new stack, so stacktraces are terse and don’t indicate how you got where you died. You can solve these problems using some newer features of Node called clusters and domains. This talk with explore using these tools for better error handling.“

Forrest Norvell is presenting…

Do as I say, not as I do: Node in the real world

Forrest

Forrest is a software engineer who has spent the better part of the last two years working with Node full-time, first working on bringing Singly’s Locker Project up to web scale, and then exploring the wilds of monitoring the performance of Node applications for New Relic. He has learned an unreasonable amount about JavaScript, V8, libuv, and making it all work well in the real world.

Forrest has spent the last year plumbing the depths of Node for New Relic, as he brings support for Node to New Relic. It’s a process that has required ingenuity, guile, and the willingness to break a lot of otherwise very sensible rules. He will take you on a tour of some of the more powerful but dangerous tools in the JavaScript toolbox: monkey patching, working with Node’s internals, and mixing synchronous and asynchronous code without setting your hair on fire.
Charlie Key is presenting…

Building a Multiplayer World for Pillow Pets

Charlie

Charlie is Co-founder of Modulus, a premier Node.js hosting solution. He has spent the last six-years working in the software, where he has created over a dozen production websites and applications for many global brands.

He is also a core contributor on Pulse game engine and one of members of the team who developed Pillow Pets World.

The talk will dive into the design goals, architecture and end result of creating this massive virtual world. Come see how Pulse (a HTML5 game engine) and Node.js were combined to create a fast, expandable, mobile ready world. Targeting a game at 10,000,000 people? Then come and find out how Pillow Pets World was built. Pillow Pets World is a virtual world built for millions of kids. Scalability and performance were aspects that were included from the start.

The technologies included in this talk are:

Pulse | HTML5 Game Engine Node.js | Scalable small servers Socket.io | Real-time communication using Web sockets Redis | Small in memory storage used for pub/sub communication between servers

Are you signed up?  BUY YOUR TICKET FOR NODE PDX HERE

Want to learn more? http://nodepdx.org/

Want to know the dates? http://nodepdx.org/

Want to know who else is speaking? Stay tuned here or go check out http://nodepdx.org/!

Adam Ulvi is presenting…

Put a Sensor On It!

Adam Ulvi

Adam is a Portland native (straight out of the Simpsons) with a penchant for software and systems integration. 8-bit gamer, seasoned professional, perpertual noob. Specialization is for insects.

Learn how easy it is to create your own monitoring system! Hobbyist components and a rich ‘maker’ community puts advanced system designs well within the reach of your average software wonk. Stop planning and start building!

Our case study is ‘GroMon’, a solution for monitoring a tiny indoor lettuce garden. Our wireless sensor keeps track of temperature and humidity, if the plants get too hot or too cold then we are notified via text message.

We will discuss the design goals and architecture, as well as component selection, prototyping and debugging steps. With a little bit of programming skill and patience, anyone can build this network. Learn how to easily extend this solution for your own use.

Our stack is Node.js running on a Raspberry Pi. We connect over Bluetooth to an Arduino hosting a single sensor. All components can be purchased off-the-shelf, no soldering is required and the total cost is around $80. Code and bill of materials is available on GitHub, let’s hack!

Aron Racho is presenting…

Jive Purposeful Places SDK – A NodeJS Bromance

Aron

Aron Racho is a Senior Software Developer at Jive Software. Aron’s background has primarily been in Java, about 10 years in. He is a relatively recent convert to Javascript, and server-side Javascript in particular. Though relatively new to NodeJS, he has been smitten by its fluency, flexibility, and superb design. One might even say a Bromance has been started.

Ok, get ready, this description for this sessions is HUGE!

Jive Software’s latest cloud release enables 3rd party developers to easily push data marshalled from external systems of record such as Salesforce into Jive. Our aim is to publish a developer framework and API which makes it drop-dead simple — and fun — to get up-and-going from scratch, or easily integrate into an existing framework. We chose NodeJS precisely for those reasons:

• High developer adoption
• High velocity development — javascript! no compilation required
• Best-in-class IDE support (IntelliJ for example)
• Native support for REST and HTTP
• Excellent package management system (NPM == maven the Good Parts)
• Tons of great libraries
• Amazing documentation, well organized, very easy to get started instantly

Our framework is built on Express, and is designed to be programmed by “convention: fill in the blanks with logic specific to your application, and as long as you’ve satisfied the contract, the framework automatically:

• Wires up routes required for configuring your integration
• Notifies your listeners for integration life cycle events (integration created; destroyed; updated, etc.)
• Executes recurrent tasks you’ve scheduled
• Handles persistence of required objects. We have support for 3 types of persistence out of the box — in-memory, file, and MondoDB.

The framework is designed for developers who want to as quickly as possible start integrating a 3rd party service with Jive, with minimum setup.

For those interested in integrating Jive into an existing NodeJS Express app, we are going to make the underlying API available to developers, allowing them finer grained control over the setup of their integration. This API will be the same one underlying the mechanics of the aforementioned framework.

For my presentation, I will be describe how we used NodeJS as the basis for this framework and API. Please note at the time of this proposal, we’ve created the framework, and are now in the process of refining the API so that it can be used independently of the framework.

Christopher Meiklejohn is presenting…

An Introduction to Functional Reactive Programming

Chris Meiklejohn

Christopher Meiklejohn is a Software Engineer with Basho Technologies, Inc. where he focuses on building rich web applications for Riak using Erlang and JavaScript. Before joining Basho, he worked at Swipely, a loyalty program startup based in Providence where he maintained critical infrastructure components written in Ruby. Christopher currently serves as one of the maintainers of Rubygems.org.

Chris also knows a thing or three about this show… called The Wire… he could probably speak entirely in quotes solely from The Wire and still make complete sense. So throw a quote out, I bet he’ll catch it.

There is no doubt that todays web applications continue to grow in adoption, replacing their desktop counterparts in all areas of computation. Essential to their growth is their ability to provide near-native performance and rich user experiences. As these applications grow in essential complexity, they also grow in accidental complexity due to the imperative callback processing style found in most web applications. The asynchronous nature of most of these applications also further compounds the issues due to guarantees around message ordering, and a level of indirection required in callbacks handling events.

Functional reactive programming is one approach for mitigating accidental complexity, using a declarative and composable data-flow model. During this talk, we’ll look at the history of functional reactive programming, some JavaScript implementations of FRP, and finally some ClojureScript implementations of FRP and examples on how to get started with using functional reactive programming.

Max Ogden is presenting…

Minecraft.js

Max Ogden – Penciled and colored!

Max used to live here in Portland, and at some point defected to work on noble causes with Code for America. Since then he’s been an Oaktown Coder (Oakland, the other city near San Francisco).

In January of 2013 he started the Voxel.js Project. Since he and contributors have generatored nearly 100 node modules related to 3D game development and distribution. Examples including voxel rendering and first person controls and physics. On the main voxel-engine the project has received over 50 pull requests from 20 contributors. All of this since January. Max is going to dive into this effort and what inspiration people have taken to dive into game development with JavaScript.

Brock Whitten is presenting…

Mighty Messaging Patterns

Brock Whitten

Best know for having co-created PhoneGap, Brock went on to work at Joyent where he created the Public API for the beloved (and now sunset) No.de Platform. He’s now working on the Harp Platform where he’s worked with a team to have created a dead simple publishing platform that uses Dropbox as its deployment mechanism. Much has been learned, he’s read to share.

He describes his course as, “Messaging is the lifeblood of distributed systems yet it is often treated as an afterthought when applications are architected. Few get passed the point of tacking on a message queue to fire and forget tasks with no visibility into what is happening on the system. I’m here to tell you we can do better. That Messaging can do more for us than just put tasks in the background. By combining basic messaging paradigms we can build powerful distributed systems with full awareness of what is happening around the network. And we can do it all in pure JavaScript.

In this talk, I will start with a crash course on the basic messaging patterns push/pull, pub/sub, and request/reply and then show a real example of how we have combined these patterns to build a custom message broker that we have used to build a fully distributed and modular architecture for the Harp Platform. I will share details about what we have learned and common pitfalls to avoid when building a messaging system for your needs.

Basic outline for the talk:

• how messaging can be useful
• crash course on the basic message patterns
• how to get started with zeromq/axon
• common pitfalls when in production
• proven trade secrets we have learned

By the end of the talk, my hope is that everyone will have a new appreciation for what can be achieved with massaging and will know where to begin when attempting to integrate messaging into their next project. I feel this aspect of building modern web applications is often overlooked and viable techniques need to be shared and discussed.“

Are you signed up?  BUY YOUR TICKET FOR NODE PDX HERE

Want to learn more? http://nodepdx.org/

Want to know the dates? http://nodepdx.org/

Want to know who else is speaking? Stay tuned here or go check out http://nodepdx.org/!

Have a last minute request, idea, comment or a speaking proposal? http://nodepdx.org/

Scott Hanselman is presenting…

Azure for the non-Microsoft Person – How and Why?

Scott Hanselman

Scott is a web developer who has been blogging at http://hanselman.com for over a decade. He works on Azure and ASP.NET for Microsoft out of his home office in Portland. Scott has three podcasts, http://hanselminutes.com for tech talk, http://thisdeveloperslife.com on developers’ lives and loves, and http://ratchetandthegeek.com for pop culture and tech media. He’s written a number of books and spoken in person to almost a half million developers worldwide.

For a dose of Scott, check out how to scale to infinity by doing NOTHING!

Join Hanselman as he digs into the open source SDKs of Windows Azure. Let’s access Azure from the command line and deploy and redeploy with Git. We’ll fire up Linux VMs, setup Mongo and run node.js apps in the cloud. We’ll look at things like SendGrid and New Relic. The future of the cloud is open and it’s a hybrid. This very technical session will cover Windows and Mac, .NET as well as pinching pennies in the cloud.

Tracy Abrahms is presenting…

Punching Rocks: my intimate adventure with rock climbing and becoming a programmer

Tracy Abrahms

Tracy tells a story, “A funny thing happened at the rock gym… I kept running into programmers. Rock climbing is a constant challenge. Physical? Hardly. Tired muscles is a concern once you’re two pitches up and can’t figure out the next move. I’m afraid of heights! The psychological and mental tenacity required to complete a wall feels eerily similar to the daily challenges of the Programmer. You will commit yourself to situations that you pretty much HAVE to find a way out of. Sound familiar?

Malcom Gladwell’s 10,000 hours–I’m not that far in. How are people learning to program nowadays? MOOCs, tutorials, workshops, communities(meetup.com), books, standard CS degrees(what of non-tech degrees?), internships, apprenticeships, code schools. What am I doing? What have I done? How many callouses have I build in the process? Experiences shared from my own perspective and others I have met on my journey have shown me a number of great ways to help move forward those willing to take up the challenge.

Finally, what can I do, along with knowledgeable and helpful Node.js programmers, to build the knowledge base and accessibility into the community? How do I get programmers hooked? And how can I convince all of these brainiacs to get out and punch a few rocks?“

Matthew Lyon is presenting…

Let’s Make Music

Matthew Lyon

I got to work with Matthew @mattly for a short period while I was helping out at AppFog a while ago. I also have had the great fortune of catching Matthew talk about JavaScript magic and more!

Also a homebrewer, hacker, artist, photographer Mattew Lyon is coming to give us a lesson on making music. He describes his course as thus, “Music happens over time and so does asynchronous code. Therefore, let’s write music in Javascript! We’ll build a network sound sequence server that will talk to freely-available softsynths and possibly an HTML5 Audio API, covering how to model things such as drum patterns, melodies, oscillators and pattern banks.“

J Chris Anderson presenting…

Hands On Realtime Text Analytics

J Chris Anderson

J Chris just recently returned to Portland because of many different reasons after hacking in the San Francisco area. If you know what livable streets are (re: Bike Portland and if you intend to stay and live in Portland, you probably should get in on this, livable streets are too good to merely pass up), you’ll want to keep tabs on J Chris’ efforts ongoing, I definitely will be! J Chris also refers to himself as a hacker dad, couchbase cofounder & mobile architect.

With “Hands on Realtime Text Analytics” J Chris will present to us  as described:

“A common problem in large scale computing, is coordinating workers when they can be scattered across compute nodes. For workloads like this, atomic operators like increment and decrement reduce contention between distributed processes.

In this talk I’ll show a full text analysis tool which ranks words in the Twitter firehose. By storing each token in a key based on its characteristics, we can provide word rankings both globally, as well as over time and space.

We’ll show the running application, and take a tour through the code, as well as discuss cluster sizing and how it is impacted by variations in the input data stream.

For instance a tweet in English from San Francisco might say “Go Giants” so counters for 2012:go and usa-sf:2012-07:giants (among a few dozen others) are incremented. Even using memory like this, the counts from a full corpus of English text would only take a few gigabytes to hold, making this architecture more efficient than a traditional index-and-rollup approach.“

Are you signed up?  BUY YOUR TICKET FOR NODE PDX HERE

Want to learn more? http://nodepdx.org/

Want to know the dates? http://nodepdx.org/

Want to know who else is speaking? Stay tuned here or go check out http://nodepdx.org/!

Have a last minute request, idea, comment or a speaking proposal? http://nodepdx.org/

Backing Up and Recovery of Riak (Nodes)

I’ve been asked approximately 423,983,321.7 zillion times how this is done. So here’s a quick summary and respective links to the best ways to backup Riak, how to recover nodes.

When backing up Riak there are two key things that need copied to the backup storage; the ring and data directories. Each of these things are specific based on the backend used with Riak. In addition to the core backup containing the ring and data, another good thing to backup is the configuration directory. When recovering this comes in useful.

For the locations of data, it depends slightly based on the operating system being used. The two big variances are OS-X and Linux Distros. On OS-X the data path, ring data and configuration are located at the locations listed below:

• Bitcask data: ./data/bitcask
• LevelDB data: ./data/leveldb
• Ring data: ./data/riak/ring
• Configuration: ./etc

For each specific distro, there are slight variations on where the locations are, for a full list check out the Basho Riak docs on backups. But on Linux distros the paths are as follows:

Debian and Ubuntu

• Bitcask data: /var/lib/riak/bitcask
• LevelDB data: /var/lib/riak/leveldb
• Ring data: /var/lib/riak/ring
• Configuration: /etc/riak

Fedora and RHEL

• Bitcask data: /var/lib/riak/bitcask
• LevelDB data: /var/lib/riak/leveldb
• Ring data: /var/lib/riak/ring
• Configuration: /etc/riak

Other Operating System Paths

Freebsd

• Bitcask data: /var/db/riak/bitcask
• LevelDB data: /var/db/riak/leveldb
• Ring data: /var/db/riak/ring
• Configuration: /usr/local/etc/riak

SmartOS

• Bitcask data: /var/db/riak/bitcask
• LevelDB data: /var/db/riak/leveldb
• Ring data: /var/db/riak/ring
• Configuration: /opt/local/etc/riak

Solaris

• Bitcask data: /opt/riak/data/bitcask
• LevelDB data: /opt/riak/data/leveldb
• Ring data: /opt/riak/ring
• Configuration: /opt/riak/etc

When backing things up, it’s important to note that each node could have slightly inconsistent data. The data however is rebuilt by the Riak read-repair system once it is recovered and brought into use.

Backup Jobs

One of the easiest ways to backup Riak is to setup a cron job with your choice of cp, rsync or tar. Then just get those files onto whatever your choice of backup medium. An example tar cron job to backup a Bitcask backend is shown below (snagged from the documentation) just to give you an idea of where to start.

tar -czf /mnt/riak_backups/riak_data_`date +%Y%m%d_%H%M`.tar.gz /var/lib/riak/bitcask /var/lib/riak/ring /etc/riak

For a leveldb back end the most important thing to note is that the node must be stopped. The basic workflow of backing up a node in this manner is to stop the node, backup the data, ring and configuration and then start the node back up.

Backup Recovery / Restoring

When recovering data on a node that is replacing an existing node that has the same name (fully qualified or IP) then follow the steps below:

1. Install Riak
2. Restore the old node’s configuration, data & ring.
3. Start the node

Once you’ve got the node started back up it’s a good idea to do a ping or status against the node to verify it is in a good state.

If node names have been changed there are additional steps.

1. Mark the original instance down

   riak-admin down 

2. Join the restored cluster  

   riak-admin join 

3. Replace the original with 

   riak-admin cluster force-replcae  

4. Get the cluster plan built 

   riak-admin cluster plan

5. Commit the changes 

   riak-admin cluster commit

6. Change the -name setting in the vm.args configuration file to match the new name.
7. Change & verify that the IP reflects the instances IP in the app.config for http and protocol buffer interfaces.

Cluster Backups via Riak Enterprise Multi-Data Center (MDC)

In the above sections I wrote about the traditional backup approaches. This is very similar to the way RDBMS are backed up. However, with a distributed system like Riak there is another great alternative if you’re utilizing multiple datacenters and Enterprise Riak. In this version of Riak, which is basically Riak with additional features and capabilities, one of the possible backup scenarios is to use the Multi-Data Center, or MDC, to replicate a duplicate cluster and use it as an active, real-time and always ready backup.

One workflow that is an exceptionally effective way to provide backups is to setup the “backup” cluster beside the current operative cluster. As an example, if your cluster is operational in AWS and it is running in X region and Y zone then you’d want to put the backup cluster in that same region and zone. Once you’ve setup Riak Enterprise and MDC, then just setup a full sync. Once the full sync is done you can then remove the backup cluster and it provides a point in time backup of the data.

riak-repl start-fullsync

It’s easy to schedule full sync operations to low usage periods and it is also possible to pause and resume full sync operations.

riak-repl resume-fullsync<br />riak-repl pause-fullsync

The variations on backing up data with Riak Enterprise and MDC are pretty expansive. Doing a point in time, maintaining a secondary live copy of the data, using the replication as a data dump to another cluster or even just using the MDC replication to dump all of the data to a single instance.

File System Snapshots

One other technique that is extremely efficient, fast and thorough is snapshotting the file system. The backup workflow for snapshots is extremely easy. First stop Riak, then snapshot, then start Riak again. Of all the methods, snapshotting is one of the easiest of the options. Just like setting up a cron job, automating snapshots based on some pre-defined schedule and meshing that with automated start and stop of Riak provides a very thorough backup.

With these options, have fun strategizing your stratagems into strategies for backups.

Diskettes

One of the oldest, tried and true backups is the old diskette. The bestest way to backup with diskettes is to backup each node on three diskettes each. The send one of each diskettes to a geographically dispersed to a bank lock box or other secure facility. Do this for each node, and if need be use as many diskettes for each node as needed. A particularly useful method is to use the sharded zip strategy to stripe a backup across many diskettes. Once each lock box has a copy of the node for each node in the cluster, you’ll have one of the most secure backups in existence. Nothing compares to the diskette backup!

References:

1. Basho Docs – Backups
2. Basho Docs – MDC Full Sync

Jason Denizac presenting…

Programming With a Purpose

Jason Denizac

I met Jason over a year ago while he was in Portland at a coding conference. Since then we’ve kept in touch, and everytime I’ve enjoyed seeing him speak. This guy is from a place far over the hills and valleys somewhere in that central part of the United States, but I’m stoked he’s out here on the west coast now bringing more brainpower to the region.

Jason described his talk, “As programmers we are wizards. Our job is to manufacture super powers. Like with the Manhattan Project, wielding such great technological power entails moral implications which we ignore at our peril. But we can recognize this power, embrace it, and use it for great good. Node gives us a tool kit to confront great problems and share in solving them. Specifically: radical decomposition, horizontal reuse, and positive community norms around testing and documentation.

Audience participation requested: be prepared to share your expertise in a real world problem space like education, civics, social equity, environmental conservation, healthcare, or your own favorite “intractable” problem. If time and scheduling permits, I’d love to have a series of lightning talks in which people could introduce other developers to the problem domain.“

This should be a good one.

Thorsten Lorenz presenting…

Module Driven Programming

Thorsten Lorenz

Thorsten Lorenz has been creating nodejs projects for over two years mostly in his spare time. The main reason why he can’t (and won’t) stop is, that ideas can be transformed into something that works in a very short amount of time when implemented in JavaScript. Aside from these projects, he also contributes to projects important for the community, one of the recent additions is source map support to browserify. These source maps even allow debugging transpiled languages (i.e. CoffeeScript) right in the browser.

Thorsten threw a bullet point list of the key points for his presentation, which follows:

• what constitutes a module
• why it is desirable to build smaller modules
• challenges and patterns for separating the application into independent modules
• process of pulling out a module from an application
• replpad case study
• how to become module driven phase1 and phase2
• browserify and how it enables to even manage your client side modules with npm
• quick primer on tools like npm init, pkginit, travisify and npm link that help with module driven development

Peter Braden presenting…

Teaching Robots to See With Javascript

Peter Braden

Peter Braden is the author of the node-opencv bindings that allow node scripts to interface with this powerful computer vision library. Peter related to us that he’s both excited and terrified for a future of seeing drones and robots. Currently he works as a web consultant-of-all-trades at frozenridge.co.

In his talk he’ll be diving into, “As we start to use javascript to control quadrocopters and robots, it becomes important that they can see with images from cameras onboard. In this talk we’ll take a step back and look at the field of computer vision; look at some of the exciting practical problems that can be solved with CV techniques, and look at how we can do this with javascript. Whether it’s detecting and recognising faces, building a picture of surroundings from a video stream, or tracking shapes, we’ll look at how we can make our javascript programs see. And all going well, we’ll have a live demonstration of a drone navigating based on onboard camera data.”

…

…

…ready to get your tickets?

Are you signed up?  BUY YOUR TICKET FOR NODE PDX HERE

Want to learn more? http://nodepdx.org/

Want to know the dates? http://nodepdx.org/

Want to know who else is speaking? Stay tuned here or go check out http://nodepdx.org/!

Have a last minute request, idea, comment or a speaking proposal? http://nodepdx.org/

…and no, that isn’t sarcasm, it’s the summer time now so we’re allowed to have sun and warm weather! With that, the speakers for Node PDX!

Ward Cunningham presenting…

My Sensors Love Node.js

Ward Cunningham

Who’s Ward? In his own findings, “The Oregonian describes me as the Old Growth of the Silicon Forest. I appreciate the thought but move way too fast to be compared to trees. Think of me as your intellectual immune system separating good technology infections from bad. Ditch that fever. Go with node.“

As Ward describes this presentation, ”I’ve tested a half-dozen home sensor integration technologies over as many years and learned something important about architecture with each generation. I’ve replaced Arduino hardware with Teensy which offers much better USB support. I’ve replace C++ with Perl then with Ruby/Sinatra and now Node/Wiki each time feeling the fresh air of a more friendly and dynamic environment. I’ve plotted results with ascii-art, java-2d, flot and now d3.js which can be a career in itself. I’ll share the good parts of each of these and suggest how you will know when it is time for you to move on.“

Nexxy presenting…

Realtime Hardware with Node.js

Nexxy

Key facts about Nexxy:

• Nexxy has been hacking on things in one way or another SINCE THE BEGINNING OF TIME.
• Nexxy is also known for her occasional use of hyperbole as a literary device.
• Nexxy first began hacking arduinos with node while she was working with a vegan strip club named Casa Diablo.
• Nexxy is now enjoying her work with Ninja Blocks as they take over the world with the internet of things!
• Nexxy inexplicably decided at the last second to write this section of the proposal in 3rd person, bullet form.

…and I have recreated this proposal section here for your information!

Join Nexxy & all for another installment of “Realtime Hardware with Node.js” as we take a look at just how exactly one should go about making a fool of themselves on stage with a bunch of electronics. We will cover the basics of getting started with hardware, demonstrate some cool tech, and conclude with a super rad (slightly hazy) and interactive dance party of epic (modest) proportions — if everyone cooperates. Questions are welcome throughout the presentation and audience members are encouraged* to participate.

* bribed with stickers

Jerry Seivert presenting…

Know Your Environment

Jerry Sievert

Jerry is a conousier of fine beer, drinks and other assorted things. The relevant list of coarse includes Lego, trains and JavaScript. His own words of these things, being a drink or beautifully coded up application, “I love to build things!” summarizes up Jerry’s love of building perfectly!

Jerry will pose the question, “What if your software knew about its environment and could react?” With very basic electronics skills, and the ability to read a datasheet, you can be well on your way to a smarter and more responsive application. We will discuss some common hardware protocols and how to interface your code with them to build something all “teh aWeSOME“!

Hannah Fousanon presenting…

Optimizing Single-Page Javascript Websites for SEO

Hannah

Hannah Fouasnon is a cofounder and lead engineer at DJZ based in San Francisco. While a full time hacker, she’s currently focusing on creating DJZ’s next single-page javascript website optimized around playing media and games. Former projects include Luckysort, a big-data, text-analytics company based in Portland, OR, where she implemented a large portion of the node.js architecture.

Hannah will cover how to create single-page javascript websites and enable developers to more easily support advanced web client features. For example, the persistent playing experience on soundcloud is supported by a single-page backbone.js architecture. One of the downsides when deciding on this architecture is that web crawlers have trouble indexing content, which can hurt seo and facebook sharing.

This presentation covers how we solve this problem at DJZ. The example will be a simplified angular.js application optimized for seo using primarily phantom.js. Viewers can follow along with code on github (link to come).

Are you signed up?  BUY YOUR TICKET FOR NODE PDX HERE

Want to learn more? http://nodepdx.org/

Want to know the dates? http://nodepdx.org/

Want to know who else is speaking? Stay tuned here or go check out http://nodepdx.org/!

Have a last minute request, idea, comment or a speaking proposal? http://nodepdx.org/

@adron Delete the “.vagrant” directory.

— Mitchell Hashimoto (@mitchellh) April 26, 2013

@adron @mitchellh @jeffsussna @dberkholz @monkchips @sogrady did you delete your .vagrant subfolder and try again with –provider flag?

— Troy Howard (@thoward37) April 26, 2013

With that quick delete of the hidden vagrant directory I gave it a shot again with the provider flagh.

rm -rf .vagrant/
$ vagrant up --provider=vmware_fusionBringing machine 'default' up with 'vmware_fusion' provider...
[default] VMware requires root privileges to make many of the changes
necessary for Vagrant to control it. In a moment, Vagrant will ask for
your administrator password in order to install a helper that will have
permissions to make these changes. Note that Vagrant itself continues
to run without administrative privileges.
Password:
[default] Box 'bosh-solo-0.6.4' was not found. Fetching box from specified URL for
the provider 'vmware_fusion'. Note that if the URL does not have
a box for this provider, you should interrupt Vagrant now and add
the box yourself. Otherwise Vagrant will attempt to download the
full box prior to discovering this error.
Downloading or copying the box...

Which this seemed to work. Downloading the helpers and such started and I waited patiently.

A Few Thoughts…

Needing to delete a hidden file struck me as one of those completely arbitrary and random solutions. It worked, which is awesome, but it working is a completely counter intuitive solution. I did a ‘destroy’ previously along with a number of things that were somewhat not intuitive. At this point the steps were fine, I had to ask for help, and I got help really fast. That’s awesome, but needing to go through those steps was unfortunate and ties back around to @jeffsussna‘s tweet earlier.

Hey #developersaskingmakers dudes: how come UX for Dev/Ops tools still sucks? @dberkholz @monkchips @sogrady

— Jeff Sussna (@jeffsussna) April 26, 2013

Anyway, as soon as I did this I decided Virtual Box it is. As it went through a 40 minute download of an image (??) it finished and displayed…

$ vagrant up --provider=vmware_fusionBringing machine 'default' up with 'vmware_fusion' provider...
[default] VMware requires root privileges to make many of the changes
necessary for Vagrant to control it. In a moment, Vagrant will ask for
your administrator password in order to install a helper that will have
permissions to make these changes. Note that Vagrant itself continues
to run without administrative privileges.
Password:
[default] Box 'bosh-solo-0.6.4' was not found. Fetching box from specified URL for
the provider 'vmware_fusion'. Note that if the URL does not have
a box for this provider, you should interrupt Vagrant now and add
the box yourself. Otherwise Vagrant will attempt to download the
full box prior to discovering this error.
Downloading or copying the box...
Extracting box...te: 119k/s, Estimated time remaining: --:--:--)
The box you attempted to add doesn't match the provider you specified.

Provider expected: vmware_fusion
Provider of box: virtualbox

… because the image isn’t available for vmware according to Vagrant, so for now, with some solutions and more questions I’m just going to go with the Virtual Box Solution and get back on track with the larger picture blog entry I’m writing. Thanks to @brianmmclain, @mitchelh, @jeffsussna and @thoward37.

$ vagrant plugin install vagrant-vmware-fusion
Installing the 'vagrant-vmware-fusion' plugin. This can take a few minutes...
Installed the plugin 'vagrant-vmware-fusion (0.6.1)'!
$ vagrant plugin license vagrant-vmware-fusion license.lic
Installing license for 'vagrant-vmware-fusion'...
The license for 'vagrant-vmware-fusion' was successfully installed!
$

Now mind you, you’ll need to go out and buy the VMware Fusion Plugin from Hashi Corp. From my perspective I was happy with this purchase just to get the stability improvements of VMware Fusion.

Vagrant & Riak

For my first example of the new plugin I forked and then cloned the Bosh Riak Repository. Once that was cloned I simply opened a terminal and navigated to the path of the repository and tried…

vagrant up

…and immediately got the message, “Failed to load the “vagrant-vmware-fusion” plugin. View logs for more details.” Noooooooooooooooez! I was sad. But dove into the logs by executing vagrant up with…

VAGRANT_LOG=INFO vagrant up

…and there was the error amid the log was something about the VMware Plugin requiring Vagrant the latest version.

Doh! I’d forgotten to upgrade first. I installed the upgrade via the downloads for 1.2.2. Once I upgrade I ran into the same error. A quick ‘vagrant -v’ to check the version showed 1.2.2 was installed. Not knowing the special secret sauce at this point, I figured I’d just reboot. It had been about 20-30 days since I had, so who knew what weird service or something needed to be restarted. I guessed correctly and after a restart vagrant kicked off the download of the vagrant image for the Bosh Riak deploy. It went by fast, and since it didn’t spit out an error about not loading the plugin I thought everything had worked…

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.

...more stuff here...

…or so I had thought. Scrolling back up through the log I realized Vagrant had NOT used the plugin for VMware Fusion. I was still stuck with VirtualBox. I went through the plugin install again to see if it just needed re-applied. At this point, since I’d already installed Virtual Box previously I figured maybe I’d just keep plunging forward and mess with the vmware plugin later, however I’d REALLY like to have all my virtualized images running via Fusion. Not sure what I missed I decided to give it one more try…

$ vagrant plugin install vagrant-vmware-fusion
Installing the 'vagrant-vmware-fusion' plugin. This can take a few minutes...
Installed the plugin 'vagrant-vmware-fusion (0.6.1)'!
Adrons-MacBook-Air-3:Downloads adronhall$ vagrant plugin license vagrant-vmware-fusion license.lic
Installing license for 'vagrant-vmware-fusion'...
The license for 'vagrant-vmware-fusion' was successfully installed!
$ cd ~/Codez/riak-release/
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...

Ugh… so I almost give up at this point. I read this, it seems ironic.

@jeffsussna @dberkholz @monkchips @sogrady I feel as though Vagrant’s UX is quite good. No? — Mitchell Hashimoto (@mitchellh) April 26, 2013

Being this isn’t the best user experience I stumble forward trying something else. I get a suggestion from a fellow coder (thx Brian) to try this.

$ vagrant up --provider=vmware_fusionAn active machine was found with a different provider. Vagrant
currently allows each machine to be brought up with only a single
provider at a time. A future version will remove this limitation.
Until then, please destroy the existing machine to up with a new
provider.

Machine name: default
Active provider: virtualbox
Requested provider: vmware_fusion
$

Well, that didn’t work either. It looks like maybe if I just blew away my VirtualBox images it would work? Well, funny, being I don’t have any VirtualBox images running or installed. Well, just to make sure I checked the VirtualBox directory. Nothing. I went ahead and deleted the entire VirtualBox Application. Tried again.

$ vagrant up --provider=vmware_fusionAn active machine was found with a different provider. Vagrant
currently allows each machine to be brought up with only a single
provider at a time. A future version will remove this limitation.
Until then, please destroy the existing machine to up with a new
provider.

Machine name: default
Active provider: virtualbox
Requested provider: vmware_fusion
$

Ok, I’ve no idea now. Any help would be greatly appreciated! This same thing appears to work fine under VirtualBox, but with the plugin added and VirtualBox removed I’m still not able to get this to work. Help!