Building — and adopting — a component library in the context of a vibrant business is no easy feat. Here are a few things we’ve learned.

At a small startup, the timing is never quite right to start a component library. Extremely quick iteration is not usually associated with putting together a standardized set of components and conventions to use across your app, as your components and conventions change all the time in the frenzy of finding product-market fit!

Nevertheless, component libraries can be tremendously valuable in separating design concerns from feature development and business logic, eventually leading to faster-than-ever iteration. In this article we’ll explore what went into our component library — we called it Ryu — and what we left out, the tradeoffs we made along the way, and the vision for its future. Let’s go!

What does a component library consist of?

In essence, our component library consists of colors, sizes, fonts, line heights, and even media breakpoints and z-indices. When building a component library you don’t generally come up with these values, but rather you take them from what Design has already fleshed out — probably in Figma. The role of our component library is to mirror Figma as closely as possible, so that developers need not worry about getting the design aspect of their feature work, and can instead focus on getting business logic and its edge cases right. In this sense, the component library can and should be the bridge that melds together Design, Product, and Engineering, effectively acting as the source of truth for our design system is.

To that end, we need an understanding across the organization that, while Design has the last word on how the experience we offer looks and feels, Engineering requires it be standardized into a core set of functionality like inputs, buttons, tooltips, menus, tables, drawers, and anything else the designs require. This distinction essentially buckets design work into two categories: new core components or modification to existing ones, and feature work that relies exclusively on core components and their many variations. Product’s role in this collaboration is keeping both sides accountable and ensuring new designs are smoothly implemented so that feature work can leverage the latest. Aligning on this across the organization is key in ensuring we offer a consistent experience, where tweaks to components are leveraged across the product instead of making a change that only applies to one particular project or feature.

This strict separation between larger feature work and the components that make up those features affords your team the opportunity to stop and think about how changing the design system fits with the rest of the design and the app. It also forces engineering to be more involved in the design process, offering its perspectives on UX and feasibility.

So we have variables, tighter collaboration and feedback loops, and a strict separation of design concerns and feature work. An important point to make next is building a hard boundary between the main codebase and your component library.

Isolation

You might be tempted to argue in favor of keeping the component library in the main codebase — just put the components in their own directory but keep them in the main repo. This is a good first step, but it’s not good enough. Having a separate repository is beneficial in many ways:

• Feature work doesn’t get to make drive-by “small tweaks” to the component library as a side effect
• Designers can interact with the component library in isolation without having to worry about business logic, in a codebase that’s typically a lot easier to digest
• Satellite efforts like internal tools, an engineering blog, or documentation sites can easily leverage the component library if it’s isolated
• The component library can’t be tainted by the main application, and isn’t constrained to it, meaning there’s no intermingling of business logic and design concerns
• It’s a forcing function to properly document your component interfaces, which self-reinforces as better interfaces for its consumers
• In a similar vein, you’re also forced to think about the design system in the larger scheme of things. It’s not just meant for your core application anymore, so this forces us to think about the essence of what a component is trying to achieve, and to create simpler, more general purpose interfaces

Once you decide to isolate the component library and can leverage it across the company, it’s important to draw lines. The component library interface is to be fiercely protected, lest it degrade as a result of trying to appease every potential use case. When designing its interfaces, always take a step back and think about the use cases. What purpose does this new setting serve? Is there a more general way of achieving the same functionality that doesn’t back us into a corner? Do we even need to make a change here, or is there a solution that doesn’t involve any changes to the component library? And of course, the cardinal sin of component design — would we end up with multiple ways of achieving the same outcome?

Good component libraries are like a good wine, they need time to breathe. It’s important to recognize that you don’t need to support every use case. Think of the context your component library is born in. It is unlikely the component library will be the first project the company embarks on. By the time you start thinking about developing a component library, a vibrant product development team is churning away features and likely already has an existing set of components they’ve outgrown. These components were iterated on heavily, the experience they provide is likely really good, and if you’re thinking about starting a component library afresh, chances are their interfaces have become less than ideal due to a combination of small tweaks done as part of feature work and a proclivity to favor speed of iteration above nearly all else. This is not necessarily a bad thing, priorities change, and you now have a deep well of insight to draw from.

So when bootstrapping a component library, we can start with the obvious. Tackle the aspects of existing components that have a clearly superior solution, and ignore others for a while. As the library builds out and time lapses, patterns emerge, become obvious, and then you can incorporate them. Punt on problems you don’t have a good solution for. Rather than implementing a bad interface, do nothing. It’s not like the component library will suffer from it: you’re preserving the status quo, consumers can always go use the old components that support every use case under the sun. Your goal isn’t to support everything yet, but to end up with a package that eventually can support almost every use case, and in order to do that you need a solid foundation that doesn’t overextend itself.

Consumption

Let’s say you expose your component library as an npm package (private or otherwise). What else should you have? Even when the library is internal to your company, it’s always a good idea to treat the project as if it were open-source.

Yes, this means maintaining a hand-crafted changelog, so that developers can keep appraised about what’s happening in the UI library and how it evolved over time, and proper versioning, cutting releases the team can adopt at their own pace.

As with any open-source project, you should take great care of building an approachable interface. Extensively using TypeScript could also be a great way of nudging consumers towards addressing breaking changes.

One of the ultimate goals of a component library is lowering the bar for UI contributions, so that — for example — folks working on the backend can contribute bits and pieces of UI work without having to know much about design or the patterns we use, since all of that is encapsulated in the library.

It also enables for faster iteration by virtue of being unable to make tweaks to the components as you go, and instead forcing the organization to make changes to the component library separately (which lets us weigh those changes on their own merit). This helps streamline processes and funnels all design changes through a dedicated channel.

Having clear documentation that covers all major use cases (and then some more) is crucial in getting adoption. Documentation is the opportunity to abstract developers away from Figma and focus them towards feature work. A lack of proper documentation can be catastrophic for adoption, given folks would not know how to use the library or what the preferred style or patterns are, resulting in them rightfully defaulting to whatever alternative existed before the component library came to life.

Documentation is also an excellent way of talking with Design. Designers who can code make non-trivial contributions to the component library and tighten the link between Figma and code. Without a component library, designers will rarely feel adequate contributing style changes, fearing they might cause some undesirable side effect. The component library, being more isolated, makes contribution feel safer. The more examples you have for consumers, the better the experience a designer has contributing can be.

There are well known tools for writing documentation sites, but when it comes to a component library, we argue that nothing beats using the component library itself as the building blocks for its own documentation site. Right off the bat this gives you a second consumer, with its own set of needs. It also forces you to write an application from scratch using the component library. At first it might look underwhelming, but you can use those rough edges to your advantage, sharpening the component library itself.

Architecture

To reiterate, you probably aren’t going to build a component library in a vacuum. Ramp was 799 days old when we started building ours. We are building it for the long term, and it’s equally important to build on the context we already have. To that end, we picked technologies we are already using in the main application, and which we’re already comfortable with, such as React, TypeScript, Babel, styled-components, and downshift. By leaning on these technologies, we were able to iterate quickly.

We already had a color palette, and standardized sizing units, breakpoints, and so on. When it comes to the library, we made the conscious choice to add a layer of indirection here. Instead of naming colors, for instance, we assigned semantic meaning to them. We assigned a color to signify destructive outcomes, a color to signify constructive outcomes, and so on. Having this layer of indirection decouples design from implementation, accomplishing two things. First, it lets us change implementation details — such as the exact hue we use for a “destructive” button — without changing the interface consumers interact with. Second, it means consumers don’t have to worry about it. Third, semantic meaning and indirection are crucial to make theming implementations feel effortless. (Fine, three things).

We can take it a bit further and have a “destructive” background which translates as a light red tint, or a “destructive” button that’s bright red. Consumers don’t care about any of this. All they care about is having the component library render the correct interface for whatever Design decreed to be the correct “destructive” aesthetic for any given component. We can apply this concept across the board when it comes to magic numbers. Instead of knowing magic numbers, consumers talk in terms of whether something is large or small, so it makes sense for that to be the interface.

At this point you might argue: “but what if consumers want to display something slightly differently”, and that hits a big aspect of having a well-crafted component library: it constrains consumers’ choices, in such a way that there’s very few ways of accomplishing exactly what they want — so that everyone’s code looks the same — and so that there’s as little customization going on as possible — so that everyone’s feature looks and feels the same.

That is to say: “they can’t, and that’s a feature not a bug”.

We also already had many components that had been heavily iterated on, and this is a trickier but crucial decision point: do you port these over or take the opportunity — and spend the time — to rework them? Both options have some merit, let’s explore.

Porting over existing components has the advantage of being quick. Once migrated to the new repo, you can easily port over the usage of existing components to the new ones. That said, existing components may be intertwined with business logic or deeply ingrained in the core codebase, complicating the extraction. At the same time, at the end of this exercise, you might not have a lot to show for. Sure, components are now isolated, but they still carry the technical debt that accumulated over years of quick iteration, and it’ll be just as hard to clean up their interface now as it was before the migration. At least you’ve stopped the bleeding: contributors are no longer able to make changes to core components as part of their normal workflow, they still should be able to change the core components, but they’re forced to go through a dedicated process: contribute to a different repository, triggering a discussion with the maintainers of core components in Engineering, Design, and so on.

Reworking components as you port them into the component library has its own set of considerations. For one, it’s a markedly slower process. You’re taking something that was iterated on for years and condensing its interface down to the essence while being careful not to permanently close the door on any legitimate use cases. It’s also an opportunity and a license to rewrite the codebase over time and in a way that makes business sense.

(Will it even be the same codebase once you’ve fulfilled your vision, though!?)

Rewiring the interfaces can give you an edge: with the beenfit of hindsight, its a lot easier to craft a consistent set of interfaces. You might make it so that — for instance — all components used to render lists of actions a user can perform take them in a standardized way, such as an array of button props. Such unified vision makes the library a breeze to use: consumers know to expect that virtually every component taking a list of actions expects button props. Consumers can thus write functions wiring business logic into the format your component library expects, and reuse them across different components and use cases.

Synergy isn’t something we could obtain by copying existing components and refactoring a bit. It’s going to be made a whole lot better by starting again from scratch. Adoption of these new components gets trickier, but we’ll get to that.

Synergy

One concrete example of synergy within the confines of the library we’re building, can be told starting with our file upload button RyuButtonFiles. This is a button users can click to upload files. It takes all the same props as a regular RyuButton, plus a few more props that are specific to file upload buttons.

Ryu also has a menu component. It renders a button and, when clicked, some menu items in a dropdown list. The button has a certain default look and feel, but you can customize the button props for it. Menu items also take an object that’s quite similar to the props buttons take. Consuming RyuMenu might look something like this:

Now, suppose we want to mix the two. We want to render an upload button as a menu item. This is not a very common use case, and most component libraries I can think of don’t have that use case in mind. Generally menu items can handle a click event and little else. Some are lucky to be able to handle links, but most try and get away with history.pushState instead.

Taking advantage of the as prop — which we heavily constrain usage of, through TypeScript — from styled-components, we can render a file upload button among our menu items. This prop essentially lets us render a component as something else, and even though Ryu doesn’t expect you to use it often, we recognize it as a good way of supporting use cases that might otherwise go ignored.

This renders a button that, when clicked shows a couple menu items, one of which is a label styled as a button styled as a menu item, that when clicked shows a system dialog for file uploads.

In a similar vein, buttons as well as many other components in Ryu take an iconType. Typically, libraries might take a string and render an icon represented by a certain identifier. Ryu takes the iconType a little bit further, by allowing certain icon types to be animated. One such icon is a loading indicator. Instead of taking another prop for loading indicators, you’d set iconType='loading'.

For example, imagine how consumers could potentially come up with some sort of function to convert a RyuMenu item to an upload button that behaves in the precise way they want, with loading and error state handling, and without compromising the core Ryu interface.

It’s hard to imagine how we’d build a synergistic interface like this had we not decided to build the component library from scratch. Further, we think having a deep well of existing and maturely iterated components to draw inspiration from actually led to a far stronger interface than what we could’ve built if we had the component library from day one. As I write these words, it is day 973 and we’re just getting started!

Adoption

Getting a core product to start using a component library built from scratch is no easy feat. Folks have an existing workflow, they probably know the components they usually work with intimately, and change is hard! When we add that the new library is a hard fork of existing components, that means every instance where we consume the old components needs to be ported over, and that may take a long time. You risk a split where neither approach fully wins, and you end up in a worse situation than what your starting point was. To avoid this scenario you need to constantly reinforce the advantages, and the urgency, of adopting the new solution.

The advantages should be manifold. This is why we’ve decided to do a component library in the first place. Maintainability improvements, separation of concerns, more Pull Requests from infrequent contributors who aren’t steeped in day-to-day front-end work, quicker iteration, more consistency, streamlined process, easier theming and accessibility, less code duplication, tighter coordination between Product, Design, and Engineering, and any other lofty goals we may have.

There’s many strategies we can pursue to drive adoption of a component library. We can pick a component and replace all instances with the newer one, eliminating the need for the older version altogether. We can choose a part of the application and refactor it to use the new stuff, and then it can serve as a point of reference. The most important aspect of adoption is that we don’t keep on adding new debt, that is, new code shouldn’t be using the old components unless there’s a good reason to do so. The default mode should be using components in the new library, and over time we can remove references to old components.

At the initial stages, adoption is far more important than completeness. That is to say: it is better at this stage to only have 5 components that can only accomplish a subset of the use cases the original components supported, but have fully adopted them; than to have 25 components that aren’t in use at all. Aligning the team behind adoption goals, rather than any one person taking it on on their own, is the best way to set the library up for success. To this end, complete and precise documentation is a must.

Closing

We started this article talking about the timing to get started with a component library, talked about the trade-offs in creating one from scratch, and discussed the challenges along the way. As we continue to grow at an accelerating speed, we’ll continue improving productivity by driving adoption of the component library across our applications (core app, our blogs, internal tools, etc.), and incorporating new components and designs along the way.

Our small but burgeoning engineering team is always on the lookout for passionate folks. If you made it this far and are considering new roles, you can reach out to me on Twitter (feel free to DM me @nzgb) or apply to our Frontend Software Engineer position directly. We have lots of interesting challenges and great chances for having a significant impact in the future of Ramp!

Originally published on the Ramp Engineering blog.

Handling data in React can be treacherous if you don’t know your way around. Learn some common patterns for fetching, storing, and retrieving data in this guide to help you avoid messy code traps.

React’s flexibility means you can handle data in a lot of different ways. This guide will teach you patterns for fetching, storing, and retrieving data in React without the stress of maintaining a complex system.

As part of my mentorship business, I have the unique opportunity to work with React daily across a wide range of projects. Teaching others how to build their own software products, I’ve come to find that most problems in React can be solved with some very simple techniques.

While some cases may call for a full-blown architecture using something like Redux (or other fanciness), a lot of the time React’s built-in lifecycle methods and local state do the trick. In my opinion, the mark of a really great developer is one who can solve a problem with as few moving parts as possible. While it can feel great to build a monument to engineering in the form of a complex system, often it’s just overkill.

In this tutorial, I’m going to share what I call my “data survival kit:” the most common patterns and hacks I use in my day-to-day work for managing data. By the time you finish, you’ll have everything you need to build data-driven UIs that offer a polished user experience.

Loading Data via Hooks

As most folks are building “database apps,” loading data is arguably one of the most common tasks to perform. Ultimately, where your data is coming from dictates your need for this. This pattern is best utilized in apps using a REST API or an RPC (remote procedure call) to fetch data.

For example, if you’re loading data via GraphQL, it’s likely that you’ll use this technique sparingly (e.g., when you need to fetch data programmatically as opposed to on page load), relying on tools like Apollo to load your data for you.

import React, { useEffect, useState } from 'react';

function Posts() {
  const [posts, setPosts] = useState([]);

  function getData() {
    fetch('https://jsonplaceholder.typicode.com/posts').then(async (fetchedPosts) => {
      const postsAsJson = await fetchedPosts.json();
      setPosts(postsAsJson);
    });
  }

  useEffect(() => {
    getData();
    const pollForData = setInterval(() => getData(), 5000);
    return () => {
      clearTimeout(pollForData);
    };
  }, []);

  return (
    <div>
      <h4>Posts</h4>
      {posts.map(({ id, title, body }) => (
        <div key={id}>
          <h3>{title}</h3>
          <p>{body}</p>
        </div>
      ))}
    </div>
  );
}

export default Posts;

This example is relatively new but is ultimately just a refactor of a previous approach where you’d use componentWillMount() or componentDidMount() to use React’s new hooks feature. The idea is simple here: when our component loads into memory, go and fetch the data it will need and put it onto state.

To make this work, we leverage the useEffect() hook to say “when our component loads, get our data and then set up a poll interval to refetch every five seconds.” useEffect() is essential, here, because side effects (e.g., fetching data) are not allowed in the body of a functional component due to their ability to produce “confusing bugs and inconsistencies in the UI.”

The idea at play here is that useEffect() will allow us to update the state of our functional component via the updatePosts() method returned from our call to the useState() hook. To prevent useEffect() from running on every render of our component, we pass an empty array [] as its second argument (this can contain values to conditionally fire useEffect() when they change—learn more here).

In order to clear out the interval when our component unmounts, useEffect() accepts a return value of a “clean up” function (this function behaves similar to componentWillUnmount() in a class). The end result is that our data will be fetched and put onto state when our component initially loads and again every five seconds until our component unmounts.

The reason I like this pattern is that it makes data fetching easy to understand for beginners and veterans alike. It also makes tasks like polling/refetching to keep data up to date easy—no need to call to an out-of-scope refetch function or Redux action. Technically speaking, too, this is handy because it helps us to avoid getting tangled up in global state—something that should only be used if it’s absolutely necessary.

Autosaving Using State

As applications have evolved, features like “autosave” have become common enough for users to expect them. Fortunately, React—and by proxy, it’s state—makes this sort of feature painless for us to implement.

The way I like to accomplish this is by write input changes to state directly (a controlled component) and then after a delay, make a call to write to the database. Depending on the UI and the amount of data involved, I’ll either send up a PATCH for an individual field, or, a PUT for the entire object it’s part of.

For the delay, I like to use this delay function that I picked up a few years ago—it’s simple and has served its purpose well:

const delay = (() => {
  let timer = 0;
  return (callback, ms) => {
    clearTimeout(timer);
    timer = setTimeout(callback, ms);
  };
})();

The basic premise of this function is that it’s a self-clearing setTimeout(). So, when it’s called, if it’s called again before it’s timeout ms, it clears itself to avoid overflowing JavaScript’s call stack. After the specified delay in ms has passed, the function performs like a regular setTimeout, executing the code it contains (i.e., once a user stops typing for 3000ms, then make the call).

class UserProfile extends React.Component {
  state = {};

  handleLiveUpdate = (event) => {
    const { name, value } = event;
    const { updateProfile } = this.props;

    this.setState({ [name]: value }, () => {
      // After handleLiveUpdate has stopped being called for 3 seconds, call to update database.
      delay(() => {
        // Example #1: Updating the database via GraphQL mutation.
        updateProfile({
          variables: {
            [name]: value,
          },
        });

        // Example #2: Updating the database via Meteor Method.
        Meteor.call('users.updateProfile', { [name]: value }, (error) => {
          if (error) {
            alert(error.reason);
          }
        });

        // Example #3: Updating the database via HTTP PATCH.
        fetch('https://app.com/api/users', {
          method: 'PATCH',
          body: JSON.stringify({ [name]: value }),
        });
      }, 3000);
    });
  };

  render() {
    return (
      <form>
        <h4>User Profile</h4>
        <div>
          <label htmlFor="firstName">First Name</label>
          <input type="text" name="firstName" value={this.state.firstName} onChange={this.handleLiveUpdate} />
        </div>
        <div>
          <label htmlFor="lastName">Last Name</label>
          <input type="text" name="lastName" value={this.state.lastName} onChange={this.handleLiveUpdate} />
        </div>
        <div>
          <label htmlFor="emailAddress">Email Address</label>
          <input type="email" name="emailAddress" value={this.state.emailAddress} onChange={this.handleLiveUpdate} />
        </div>
      </form>
    );
  }
}

The idea here is that as the user makes changes to the form we want to autosave, we immediately set state. Then, in the “background,” we utilize the delay() function to say “after they’ve stopped typing, persist the current state in the database.” The database part depends on how you handle data in your application.

Because I spend a lot of my time working in Meteor and GraphQL with the boilerplate I maintain, Pup, I’ll either use Meteor’s Meteor.call() convention to send my data to the server for storage, or, call to a mutation if I’m relying on GraphQL. If I’m building a mobile app, I’ll rely on fetch() or a library like axios() to talk to my REST API.

Using Local Storage to Persist Unsaved Data

One of the worst bits of UX is having a large form without a backup. As a user, there’s nothing quite as disheartening as filling out a large form and accidentally hitting refresh to find all of your work is gone.

Fortunately, the trick to getting around this is pretty simple (and utilizes a similar approach to the autosave example above). This pattern necessitates that our data live on our component’s state, allowing us to maintain the illusion of a user’s data being persisted without hitting a database.

import React from 'react';
import store from 'store';

class UserProfile extends React.Component {
  constructor(props) {
    super(props);
    this.state = store.get('myApp.userProfile') || {};
  }

  handleUpdate = (event) => {
    const { name, value } = event;

    this.setState({ [name]: value }, () => {
      delay(() => {
        store.set('myApp.userProfile', this.state);
      }, 500);
    });
  };

  render() {
    return (
      <form>
        <h4>User Profile</h4>
        <div>
          <label htmlFor="firstName">First Name</label>
          <input type="text" name="firstName" value={this.state.firstName} onChange={this.handleUpdate} />
        </div>
        <div>
          <label htmlFor="lastName">Last Name</label>
          <input type="text" name="lastName" value={this.state.lastName} onChange={this.handleUpdate} />
        </div>
        <div>
          <label htmlFor="emailAddress">Email Address</label>
          <input type="email" name="emailAddress" value={this.state.emailAddress} onChange={this.handleUpdate} />
        </div>
      </form>
    );
  }
}

This should look pretty familiar. Everything here is identical to the autosave approach save for a few small details.

First, we’ve introduced a library store which will help us to get cross-browser access to local storage (or a comparable browser cache that’s available to our user). Our usage of the library is limited to two calls: one when we load our component/page up for the first time and whenever a user changes their data.

The first takes place in our component’s constructor() function. Here, we’re setting our default state value relative to the current value of our local storage key (here, myApp.userProfile is the name of the key we’ve chosen to store our data in local storage). What we expect is that store.get() will return an object containing properties that reflect the state values our UI requires.

  handleUpdate = (event) => {
    const { name, value } = event;

    this.setState({ [name]: value }, () => {
      delay(() => {
        store.set('myApp.userProfile', this.state);
      }, 500);
    });
  };

In order to get those values into local storage, we use a handleUpdate method on our component that uses the same approach to our autosave pattern. Here, we immediately set the user’s input on state and then after a short delay, we update the local storage value via store.set('myApp.userProfile', this.state).

Two details here: notice that we’ve reduced our delay significantly to 500ms. This is because it’s far less expensive to hit our local storage than it is to make a trip to the server. Also, notice that we’re setting the entirety of the current this.state value onto local storage when we change any input. This, too, is inexpensive for performance and also saves us a messy constructor() loaded with calls to store.get() for each individual field.

Reaching Into a Child Component via Refs

One of the bittersweet features of React is the ability to nest child components. It’s bittersweet because while it makes composition easy—bringing together multiple components in a cohesive UI—it can make tasks like getting data out of child components a chore.

A simple hack that I like to leverage for this is accessing child components via refs. While it’s more common to pass a function via props to a child component that handles tracking the child’s state on the parent (or at least, notifying the parent when its internal state changes), this can get messy and cumbersome.

Instead, if all we care to know is the current internal state of a child component, refs make our lives a hell of a lot easier. Let’s consider the example of a job application. We have some basic form fields we’d like to grab along with two lists: the candidates strengths and weaknesses.

import React from 'react';

class List extends React.Component {
  state = {
    items: [],
  };

  handleRemoveItem = (id) => {
    this.setState(({ items }) => ({
      items: items.filter((item) => item.id !== id),
    }));
  };

  handleAddItem = (event) => {
    event.persist(); // Use event.persist() so we don't lose React synthetic event in nested function below.
    this.setState(({ items }) => ({
      // randomIdGenerator() is used for example here and doesn't exist.
      items: [...items, { id: randomIdGenerator(), item: event.item.value }],
    }));
  };

  render() {
    return (
      <div>
        {this.state.items.map(({ id, item }) => (
          <li key={id}>
            {item}
            <button onClick={() => this.handleRemoveItem(id)}>
              <i className="fas fa-remove" />
            </button>
          </li>
        ))}
        <form onSubmit={this.handleAddItem}>
          <input type="text" name="item" />
          <button type="submit">Add Item</button>
        </form>
      </div>
    );
  }
}

To manage those two lists, we use a nested component with its own state called <List />. Internally, the component gives the applicant an input to add as many list items as they choose. As a standalone component, this doesn’t present us with any issues.

import React from 'react';
import List from './path/to/List';

class JobApplication extends React.Component {
  state = {
    firstName: '',
    lastName: '',
    emailAddress: '',
  };

  handleSubmitApplication = (event) => {
    const { submitApplication } = this.props;

    // Example call to a GraphQL mutation here to perform the submission.
    submitApplication({
      variables: {
        ...this.state,
        strengths: this.strengths.state.items,
        weaknesses: this.weaknesses.state.items,
      },
    });
  };

  render() {
    return (
      <React.Fragment>
        <h1>Job Application</h1>
        <form onSubmit={this.handleSubmitApplication}>
          <div>
            <label htmlFor="firstName">First Name</label>
            <input type="text" name="firstName" value={this.state.firstName} onChange={this.handleUpdate} />
          </div>
          <div>
            <label htmlFor="lastName">Last Name</label>
            <input type="text" name="lastName" value={this.state.lastName} onChange={this.handleUpdate} />
          </div>
          <div>
            <label htmlFor="emailAddress">Email Address</label>
            <input type="email" name="emailAddress" value={this.state.emailAddress} onChange={this.handleUpdate} />
          </div>
          <label>What are some of your strengths?</label>
          <List ref={strengths => (this.strengths = strengths)} />
          <label>What are some of your weaknesses?</label>
          <List ref={weaknesses => (this.weaknesses = weaknesses)} />
          <button type="submit">Submit Job Application</button>
        </form>
      </React.Fragment>
    );
  }
}

Where things can potentially get messy is when we render the <List /> component inside of a parent. Here, <JobApplication /> represents that parent. In the render() we can see two instances of <List /> being generated.

Traditionally, we could add a prop to <List /> called onUpdate() that was called internally by <List /> passing up the current items. Where this becomes problematic is in having to track the list’s state both internally as well as on the parent (or having the parent feed the child its data—no bueno unless we need to load data from the database).

To simplify this, we add a ref to each instance of the <List />, assigning it back to our <JobApplication /> component as either this.strengths or this.weaknesses. What’s great about this is that now, we have direct access to these components from within <JobApplication />.

When our applicant submits the form, all we need to do get its items is call to either this.strengths.state.items or this.weaknesses.state.items.

Bonus: Controlling a Child via the Parent

You may be wondering, does this mean I can control the child component via refs as well? Yes. You want to be careful with this, however, as it can produce unexpected side effects. For example, when it comes to updating a component’s internal state relative to a parent’s data, it’s best to pass changes via props.

Sometimes, though, this isn’t always feasible or wanted. For example, consider our example above. After the application is submitted, let’s assume we want to “reset” the form. Because our <List /> components maintain their state internally, this means we need to manipulate their state from <JobApplication />.

import React from 'react';
import List from './path/to/List';

class JobApplication extends React.Component {
  state = {
    firstName: '',
    lastName: '',
    emailAddress: '',
  };

  handleSubmitApplication = (event) => {
    const { submitApplication } = this.props;

    // Example call to a GraphQL mutation here to perform the submission.
    submitApplication({
      variables: {
        ...this.state,
        strengths: this.strengths.state.items,
        weaknesses: this.weaknesses.state.items,
      },
    }).then(() => {
      this.strengths.setState({ items: [] });
      this.weaknesses.setState({ items: [] });
    });
  };

  render() {
    return (
      <React.Fragment>
        <h1>Job Application</h1>
        <form onSubmit={this.handleSubmitApplication}>
          <div>
            <label htmlFor="firstName">First Name</label>
            <input type="text" name="firstName" value={this.state.firstName} onChange={this.handleUpdate} />
          </div>
          <div>
            <label htmlFor="lastName">Last Name</label>
            <input type="text" name="lastName" value={this.state.lastName} onChange={this.handleUpdate} />
          </div>
          <div>
            <label htmlFor="emailAddress">Email Address</label>
            <input type="email" name="emailAddress" value={this.state.emailAddress} onChange={this.handleUpdate} />
          </div>
          <label>What are some of your strengths?</label>
          <List ref={strengths => (this.strengths = strengths)} />
          <label>What are some of your weaknesses?</label>
          <List ref={weaknesses => (this.weaknesses = weaknesses)} />
          <button type="submit">Submit Job Application</button>
        </form>
      </React.Fragment>
    );
  }
}

Here, when our applicant submits the form, as soon as we get an “all good” back from the database, we call to this.strengths.setState() and this.weaknesess.setState() to reset their internal state.

Again, use this wisely—I consider this a hack which means it should be applied with caution and ample experimentation/testing.

Conclusion

Using data in React doesn’t need to be complicated. In fact, one of the joys of using React is that it can help you produce interactive UIs with very little effort. If you’re already committed to React, it’s worth considering how you might simplify your own use of it. More often than not, the problems I see in React applications involve unnecessarily complex data patterns.

One of the things I enjoy the most about coding is discovering patterns. Being aware and mindful of the patterns emerging in your codebase can make it easier to keep that codebase consistent, readable and easy to navigate. Most patterns will remain unspoken, and some, that prove notably elegant, are named and promoted as best practices. Others might even be candidates for a basis of a new abstraction, although I recommend restraint as it’s worth remembering that abstractions are expensive.

If you’re not familiar with the topic or React hooks, there is plenty of introductory as well as in-depth materials out there. I can also shamelessly recommend my talk on hooks from a recent React Copenhagen meetup. What follows assume at least some understanding of what hooks are and how to use them.

Pattern extinction level event

Discovering new patterns is relatively rare when working with established technologies, but every now and then a new idea comes up that stirs things up within some ecosystem. Enter React hooks. Hooks enable a drastically different way to build stateful components in React, which means that they also invalidate ¹ a lot of the established patterns. However, being an exceptionally well thought-through and composition-friendly abstraction, hooks enabled new, elegant patterns to emerge.

Over the last few months, I’ve been building apps with hooks and I’ve been having a lot of fun discovering these patterns. I’d like to share one, that I’ve grown especially fond of. For the lack of a better name, and for sole the purpose of this blog post, let’s call it the Choco pattern (from Context, HOok, COpomonent). The pattern comes useful when you need a UI widget which is top-level, centralised and unique across the application and needs to expose some functionality to the rest of the components in the app. It might sound vague at the moment, so let’s try to go through solving a concrete problem where I noticed the Choco pattern manifest itself.

Exhibit A: Feature toggles

Let’s try to build a simple feature toggle functionality for a React app. There should be UI where the user can toggle features on and off and we want to expose these settings to any component interested. We can identify parts we will need:

1. a context provider exposing the feature toggles anywhere in the component tree,
2. a component rendering the UI for toggling features,
3. a place to store the feature toggle state and the associated logic.

Our first attempt could look like this:

This code is fairly simple and works so we could stop here. However, there is a big opportunity for refactoring, by moving all the code related to the “feature toggle concern” to a separate module, instead of mixing it with the rest of our app. Let’s do just that.

The biggest change here is extracting the state and its related logic into a custom hook. We can now clearly see the Choco pattern here, with the Context, HOok, and COpomonent exported from the new module.

I made a choice here to keep the initialFeatures dictionary with the application code instead of moving it to the newly created module. One could argue either way, but by keeping that dictionary out of the feature toggle module and dependency-injecting it, we make that module completely generic and a candidate for possible extraction into a library.

Again, we could easily end the refactoring here. In fact, that’s the level of decoupling I’d recommend for in this scenario (more on that later). We can, however, take it one step further and introduce an abstraction to hide some of the implementation details and reduce the API surface of the module. Here is one way to do this:

The biggest difference to notice is that the fact we’re using the native React context is now an implementation detail, not exposed to the rest of the app. By adding our own abstraction on top of it we also gained a more fine-grain control over what properties of the context are exposed. In this case, we could expose only the feature toggle dictionary, while keeping the function to toggle them private.

Another thing to notice is that by wrapping the context provider in a component, we had an opportunity to render the FeatureToggler in that component and take that responsibility away from the module’s consumer. This might be a viable option if we decide to e.g. render the toggler in a portal to make it overlay the page after the user presses some specific key combination. However, in this example I chose not to - this way is much more flexible as the consumer can choose where and how to render the toggler.

What’s interesting is that we kept the same type of exports from the feature toggle module (Context, HOok, and COpomonent), but they are consumed in a different way. The hook is now taking care of exposing the features to the components. The context (or more precisely the wrapped context provider) now also encapsulates the state and related logic.

Did we improve this code in comparison to the previous iteration? I think the answer depends on how we plan to use it. If we decided to extract the feature toggle module into a small library, then that last version of the code is most likely a better choice. It would be easier to document and use the API with a smaller exposed surface. Hiding the implementation details might also enable adding more functionality later on, without introducing breaking changes.

However, if we keep it as a module within the codebase the abstraction creates an unnecessary indirection that the next developer would need to unpack to understand. We created a black box that would need to be explored as opposed to the previous version, which while verbose, used the core React APIs most likely already familiar. Since we don’t extract and version the module, breaking changes are not a big deal as we could quickly make necessary adjustments in the places where the module is consumed.

Exhibit B: Toast notifications

It wouldn’t be much of a pattern if it didn’t repeat. To see how it manifests again and again in this kind of problems, let’s look at one more example. We are going to build a simplified toast notification system. We want to be able to ask for a notification anywhere in our component tree and then show it in a global bar. Again, let’s identify what pieces we are going to need:

1. a context provider exposing a function that can be used to ask for a toast,
2. a component rendering the toast bar,
3. a place to store the notifications state and related logic

We already know what to expect so let’s skip the first draft and go directly to the version with a separate module:

It’s our Choco pattern on full display! And as in the previous example, we can refactor this to make the API cleaner and more controlled. It could look like this:

As in the feature toggle example, I chose to only expose some part of the context value to the component tree through the custom hook. In this case, the array of toasts remains private, while the functions to add and remove toasts are exposed.

Exhibit C: User guide

To cement the pattern further, let’s look at one last example. We will build user guide functionality. We want to add a help button in some places in our app, which when clicked, opens a user guide widget that explains the part of the interface in question. You know the drill by now, let’s see what we’re going to need:

1. a context provider exposing the help icon button component
2. a component rendering the user guide UI
3. a place to store state (i.e. which piece of the interface the user asked for) and related logic

Our first iteration could look like this:

Hopefully, the structure is intuitive and familiar by now. The only significant difference is the type of functionality exposed to the other components. In the case of the guide, we choose to expose a component, while in the previous examples it was a method or state data. Anything goes!

Following the previously established procedure, the refactored version of the same code could look like this:

Spot the pattern

Of course, I slightly cheated with the way I presented the refactoring evolution here. I already knew the pattern and made the examples follow it from the very beginning. In fact, when I first got to implement these features using hooks, I didn’t immediately see the Choco pattern and my initial implementation attempts were chaotic and inconsistent. But with time, I started to see the same code paths and elements showing up repeatedly and I was able to streamline and clean up my implementations.

Discovering and embracing patterns can be a powerful technique that can make penetrating and understanding a codebase easier. I’m always on a lookout for concerns and features that share similar requirements and characteristics and try to implement them in a consistent way, without introducing abstractions prematurely.

I’m really eager to find out what other concerns are candidates for using this pattern and what other patterns using hooks will surface. Let me know what you think!

Mocks are a great way of preventing AJAX calls in tests, but they can also help you isolate side effects and impurities that can create complicated tests.

As you learned in Part 1, mocks are a great way to handle external data or any data that is likely to change. Mocking external data will likely be your most common use case and for a good reason. You want your tests to stick as closely to your code as possible this includes all dependencies. Still, there are times when a dependency creates a testing specific problem. In other words, the code works as you would want it to in production, but in a testing environment can make consistent, predictable tests hard or nearly impossible.

Some of your third party code may have side effects or some form of impurity that will complicate your testing. Maybe the code has certain expectations of the DOM. Maybe it will change slightly depending on the order of operations. In all cases, the code is out of your control, but you need it to be predictable.

Creating Predictable Outcomes with Mocks

Suppose you have an application that manages employees or users. You need to make a simple function that will create a new employee, but you want to make sure that the employee always contains certain fields and will have a unique id.

Your code will look something like this:

import uniqueId from 'lodash/uniqueId'

export function createEmployee(details) {
  return {
    name: '',
    position: '',
    ...details,
    key: uniqueId()
  }
}

In this case, you are using the Lodash method uniqueId to, well, create a unique ID. Specifically you are creating a key that you can use if you do any manipulations on an array of employees. The key, though, is a minor thing.

For the most part, your tests should focus on other aspects of your code such as ensuring default fields while allowing those fields to override. Here are two simple tests.

import { createEmployee, } from './utils'

describe('createEmployee', () => {
  it('should create a blank employee', () => {
    const blank = createEmployee()
    const expected = {
      name: '',
      position: '',
      key: '1'
    }

    expect(blank).toEqual(expected)
  })

  it('should create use existing details when creating employee', () => {
    const blank = createEmployee({ name: 'Bill' })
    const expected = {
      name: 'Bill',
      position: '',
      key: '2'
    }

    expect(blank).toEqual(expected)
  })
})

Notice anything strange in the expectations? In the first case, the key is '1' in the next test it’s '2'. Here’s where the problems start to sneak in. Every time you add a test, you need to iterate the key. The uniqueId function is doing exactly what you would want in production, but in the testing case, it’s creating a situation where your tests always have to run in the same order.

Suppose you needed to change the function slightly to handle cases where you get an object with a field of firstName and you want to use as the name field:

import uniqueId from 'lodash/uniqueId'

export function createEmployee(details = {}) {
  const { firstName: name, ...rest } = details

  return {
    name: name || '',
    position: '',
    ...rest,
    key: uniqueId()
  }
}

Your tests now look like this:

import { createEmployee, } from './utils.next'

describe('createEmployee', () => {
  it('should create a blank employee', () => {
    const blank = createEmployee()
    const expected = {
      name: '',
      position: '',
      key: '1'
    }

    expect(blank).toEqual(expected)
  })

  it('should create use existing details when creating employee', () => {
    const blank = createEmployee({ name: 'Bill' })
    const expected = {
      name: 'Bill',
      position: '',
      key: '2'
    }

    expect(blank).toEqual(expected)
  })

  it('should convert firstName to name', () => {
    const blank = createEmployee({ firstName: 'Bill' })
    const expected = {
      name: 'Bill',
      position: '',
      key: '3'
    }

    expect(blank).toEqual(expected)
  })
})

What happens if you change the order? Suppose you wanted to only run the third test:

import { createEmployee, } from './utils.next'

describe('createEmployee', () => {
  // … other tests …

  it.only('should convert firstName to name', () => {
    const blank = createEmployee({ firstName: 'Bill' })
    const expected = {
      name: 'Bill',
      position: '',
      key: '3'
    }

    expect(blank).toEqual(expected)
  })
})

You’d get an error. But it’s not because your code is wrong. The order has just changed.

  expect(received).toEqual(expected)

  Expected value to equal:
-   {"key": "3", "name": "Bill", "position": ""}
  Received:
+   {"key": "1", "name": "Bill", "position": ""}

  Difference:

-   Expected:
+   Received:

    Object {
-     "key": "3",
+     "key": "1",
      "name": "Bill",
      "position": "",
    }

  …

So how can you fix it? In this case, you have a couple of options.

The easiest option is to override it inside the test itself. By using mockImplementation.

import { createEmployee, } from './utils.next'
import uniqueId from 'lodash/uniqueId'

uniqueId.mockImplementation(() => '1')

describe('createEmployee', () => {
  it('should create a blank employee', () => {
    const blank = createEmployee()
    const expected = {
      name: '',
      position: '',
      key: '1'
    }

    expect(blank).toEqual(expected)
  })

  it('should create use existing details when creating employee', () => {
    const blank = createEmployee({ name: 'Bill' })
    const expected = {
      name: 'Bill',
      position: '',
      key: '1'
    }

    expect(blank).toEqual(expected)
  })

  it('should convert firstName to name', () => {
    const blank = createEmployee({ firstName: 'Bill' })
    const expected = {
      name: 'Bill',
      position: '',
      key: '1'
    }

    expect(blank).toEqual(expected)
  })
})

Notice what’s happening. First, you import the code with import uniqueId from 'lodash/uniqueId', then you add a mockImplementation to the function you want to control. Jest will now use the mockImplementation rather than the actual code anytime you call the function.

In other words, you now have a function that returns the result you want everytime. After that, you can write each test expect the exact same key. After all, you don’t really care what the key is, you just want it to be there.

Another option is to just define the return value using mockReturnValue. The steps are the same, but instead of returning a function, you just declare the desired return value.

import { createEmployee, } from './utils.next'
import uniqueId from 'lodash/uniqueId'

uniqueId.mockReturnValue('2')

// Tests are the same

Those are both perfectly good solutions. You may reach for mockImplementation in situations where you need to check parameters before you decide what to return.

The problem with both solutions is that you have a little extra boilerplate for every single test suite that uses the uniqueId. To solve that, you can use a similar technique to mocking out the API.

Create a new directory called __mocks__ at the same level as your node_modules directory. This will almost certainly be the root of your project. Inside __mocks__ create a file structure that emulates your import. In this case, you need a Lodash directory that contains a uniqueId.js file.

| __mocks__
| + lodash
|   + uniqueId.js
| node_modules

Inside, uniqueId.js export either your mockImplementation or your mockReturnValue. The only difference is that you have to create a jest.fn() first. It’s not strictly required that you return jest.fn() you can just return any function. However, by returning a mocked jest.fn() you have more options to change the return value (more on that in a bit).

export default jest.fn().mockImplementation(() => '1')

Now that you have mocked the function at the import level, you don’t need to do anything in your actual test. The mock is detected for you.

import { createEmployee, } from './utils.next'

describe('createEmployee', () => {
  it('should create a blank employee', () => {
    const blank = createEmployee()
    const expected = {
      name: '',
      position: '',
      key: '1'
    }

    expect(blank).toEqual(expected)
  })

  // Other tests

})

Changing on the fly

Of course, if you want to change if for an individual test you can still do so. Suppose you had a helper function that sorts the employees by key. This would, essentially, reset the order.

The function is pretty straightforward:

export function sortEmployees(employees) {
  return [...employees].sort((a,b) => Number(a.key) - Number(b.key))
}

This makes testing a little more tricky. Remember, you just mocked the function so it always returns the same value. And that is good for all the previous tests, but now you need the few different unique ids.

Fortunately, once you have it mocked, it’s easy to make changes at runtime… As a reminder here’s the mock in __mocks__/lodash/uniqueId.js:

export default jest.fn().mockImplementation(() => '1')

The important thing is that you are exporting a jest function. That means you can import that mocked function and change it on the fly.

First, import uniqueId into your test:

import uniqueId from 'lodash/uniqueId'

Now, in your test anytime you want to change the value, you declare what you want it to return. As a safety, use the method mockImplementationOnce this will ensure that the method will only return the alternate value, well, once. Then it will return to the original mock implementation. This will prevent some confusing bugs from sneaking into your code.

import uniqueId from 'lodash/uniqueId'

uniqueId.mockImplementationOnce(() => '2')

uniqueId()
// 2

uniqueId()
// 1

With this power you can create a few employees with different unique ids. After that the test is simple:

import { createEmployee, sortEmployees } from './utils.next'
import uniqueId from 'lodash/uniqueId'

describe('createEmployee', () => {
  // … tests

  it('should sort employees by id', () => {
    const blank1 = createEmployee({ name: 'Olivia'})

    uniqueId.mockImplementationOnce(() => '2')
    const blank2 = createEmployee({ firstName: 'Xander'})

    uniqueId.mockImplementationOnce(() => '3')
    const blank3 = createEmployee({ name: 'Bill' })

    const expected = ['Olivia', 'Xander', 'Bill']
    const employees = [blank2, blank3, blank1]

    const mapped = sortEmployees(employees).map(employee => employee.name)

   expect(mapped).toEqual(expected)
  })

  // … more tests
})

In this situation, you are creating different unique ids for only this one specific set of assertions. Everything else falls back to the predictable unique id.

Ignoring Black Boxes

There’s more ways you can use mockImplementation to change or modify individual library methods. But occasionally, you want to completely remove a piece of external code. You may have a heavy piece of code that itself has a large number of external dependencies. Rather than import the large code, it may be easier to bypass it altogether. This is a good technique when you have a piece of code that’s indepedent from the rest of the code, but still is included.

In other words, whenever a sales or marketing person forwards you an email with code you “just have to drop in” — a tracking pixel, a buy now widget or so on — you may have something you need to ignore.

Suppose you have a piece of code that adds a map of some sort. Assume the stripped down version of the code you are importing looks something like this:

const tiles = [...Array(1000)].map(() => '<div></div>')
const container = `
  <div class="container">${tiles.join('')}</div>
`

export default function render() {
  const mapContainer = document.querySelector('#map')
  mapContainer.innerHTML = container
}

Nothing too difficult here, but it does make a few assumptions about the DOM. It runs a querySelector and expect to see the innerHTML.

Suppose that you have a small React component that list some ride options and also renders the map. The map is effectively an image as far as you are concerned. You just need to make sure that a div exists so you can add the map to it at runtime.

import React, { useEffect } from 'react'
import renderMap from 'streetMap'

export default function RideFinder({ rides }) {
  useEffect(() => {
    renderMap()
  }, [])
  return (
    <div>
      <h3>Find a Ride</h3>
      <ul>
        {rides.map(ride => <li key={ride.id}>{ride.name}</li>)}
      </ul>
      <div id="map"></div>
    </div>
  )
}

In your test, you don’t care about the map. It’s a blackbox, but you do want to make sure you are creating the correct number of list items.

import React from 'react'
import { mount } from 'enzyme'
import RideFinder from './RideFinder'

function flushPromises() {
  return new Promise(resolve => setTimeout(resolve, 10))
}

describe('RideFinder', () => {
  const rides = [
    {
      name: 'River Trail',
      id: 1,
    },
    {
      name: 'Downtown Bikepath',
      id: 2,
    }
  ]

  it('should render ride finder', async () => {
    const wrapper = mount(<RideFinder rides={rides}/>)
    const renderedRides = wrapper.find('li')
    await flushPromises()
    expect(renderedRides.length).toBe(2)
  })
})

Note there are a few complications. To make sure useEffect runs in enzyme, you need a small timeout to ensure everything finishes rendering. There are other testing options than enzyme, but enzyme remains popular.

After running this code, you get an error:

Uncaught TypeError: Cannot set property 'innerHTML' of null

The DOM manipulations are causing problems. In this case, you may want to mock out the map code.

As with lodash above, make a file of the same name as the module in the __mocks__ directory. In this case, you are using the main default, to you don’t need an additional directory. Together, it would look like this:

| __mocks__
| + lodash
|   + uniqueId.js
| + streetMap.js
| node_modules

Inside of streetMap.js simply return a mock function. Remember, in this case, you don’t really care what it does. You just want to make sure it runs.

export default jest.fn()

At this point, your test will run with no problems. However, if you want to ensure that the your code will run the blackbox, even though you don’t care about the output, you can still assert that it works. As with Lodash, import the code you want to check, then add an expect function to see if it is called:

import React from 'react'
import { mount } from 'enzyme'
import RideFinder from './RideFinder'
import renderMap from 'streetMap'

function flushPromises() {
  return new Promise(resolve => setTimeout(resolve, 10))
}

describe('RideFinder', () => {
  const rides = [
    {
      name: 'River Trail',
      id: 1,
    },
    {
      name: 'Downtown Bikepath',
      id: 2,
    }
  ]

  it('should render ride finder', async () => {
    const wrapper = mount(<RideFinder rides={rides}/>)
    await flushPromises()
    expect(renderMap).toBeCalled()
  })
})

You now have a test that ensures that blackbox code is called while preventing the code from running itself. Since you have no control over the code, and it is fairly independent, this is a good strategy to ensure coverage without overcomplicating your test suite.

Using mocks responsibly

Mocks give you a lot of power to bypass code. However, that power can also be dangerous. As soon as you bypass code, you are creating tests that do not fully execute your code. If you make false assumptions in your mocks, errors will begin to sneak into your code. And I can tell you from experience, this can happen quickly and tracking it down can be frustrating.

As a rule, only use mocks in situations where the side effect is very isolated. As you can see with the Lodash functions, you don’t want to mock everything, just the piece that causes problems. If you use other Lodash methods, you want to ensure they run properly.

Same with the blackbox code. Nothing in the rest of your code is dependent on the results. So you can safely mock it out without creating too many additional assumptions in your tests.

Mocks can be a powerful tool that can get you more code coverage with a simple interface. Use them with caution, but use them confidently when you do.

Have you ever tried to do a code review on a PR that merges a large release branch or feature branch back into mainline, fixing merge conflicts? It’s not pretty. The diffs are often and easily very big, — 50k+ LOC big — have hundreds of commits, and the actual changes made by the engineer resolving the merge conflicts are virtually impossible to spot.

This article covers one nifty trick I’ve been using lately in these cases that lets you review exactly the conflict resolution, while ignoring the mountains of changes that don’t represent any conflicts (and can thus be safely ignored).

Check out the PR locally

If you haven’t done this before, you can just add the author’s remote (provided they’re using a fork, otherwise you can skip this step) like so:

$ git remote add nico https://github.com/nico/repo.git

Then we can fetch the PR branch, let’s assume it’s called fix-merge-conflicts and it was merging branch-with-conflicts into master, fixing the conflicts along the way:

$ git fetch nico fix-merge-conflicts
$ git checkout fix-merge-conflicts

Assess the situation

The git log should indicate they’ve merged a couple branches. The commit hashes are the key bits here.

$ git log -1 HEAD
commit a9c9ed7gcc0a76670d86df4b732f1f219ccb48de (HEAD -> fix-merge-conflicts)
Merge: aecf730802 ba9918dad5
Author: Bloodninja <bloodninja@usenet.org>
Date:   Mon Apr 8 12:32:38 2019 +0000

    Merge remote-tracking branch 'branch-with-conflicts'

And now we can verify that these indeed belong to each of the branches we care about:

$ git branch --contains=aecf730802
  master
* fix-merge-conflicts
$ git branch --contains=ba9918dad5
  branch-with-conflicts
* fix-merge-conflicts

Now for the magic

git checkout -b what-were-they-thinking
git reset --hard aecf730802
git merge --no-ff ba9918dad5 # this will conflict
git add .
git commit -m WIP

At this point, what-were-they-thinking is the version of the conflict resolution where you just committed things exactly as git merge left them, conflict markers and all.

And, presto, the following command lets you see exactly what they did.

git diff what-were-they-thinking fix-merge-conflicts

Obviously all the commit hashes, branch names, and the author are made up here, and you’ll have to use the actual output you get in each command to feed that into the following command, but you already knew that so I’ll just stop mansplaining and typing these words now!

Do you have any other quick and dirty tricks like this that would blow most people’s minds? This one certainly blew my mind, although I can’t claim credit for it. All credit goes to my brilliant co-worker Joe Gallo, who taught me this nifty trick and is a git mastermind!

Testing can be simple. In fact, it is simple. Well, it is simple until impurities slip in.

Code that would be easy to test becomes a nightmare as soon as you get impure data (like date checks) or complex external dependencies (such as DOM manipulations or large 3rd party libraries). The part that tends to frustrate developers most is when they have code that’s easy to read, easy to write, but difficult to test. No one wants to spend more time on their tests than they do on their code.

Consider an example. Let’s say you needed to write some code that checks to see if a given date is two weeks from today. If they date is far enough away, you get an empty string. If the date is less than two weeks away, you get an error message:

import moment from 'moment';

export const message = 'Date must be two weeks from now'

export function isAtLeastTwoWeeks(date) {
  const isLater = moment(date).isAfter(moment().add(2, 'weeks'));
  return isLater ? '' : message;
}

That’s pretty simple code. But how do you write a test for it? Things can get tricky fast.

Here’s an even more basic example. You want to fetch some data from an API and display the first couple items:

export function getRecent() {
  return fetch('/albums')
    .then(response => response.json())
    .then(albums => {
      return albums
        .reverse()
        .slice(0, 2)
        .map(({title}) => title);
    })
}

Simple to read. Simple to understand. But how would you test it without hitting the live API?

There’s an unfortunate belief that testing is hard. Early in my career, I heard one developer estimate that it doubled development time. And he was arguing for testing, that it was worth the extra time. The problem is that most testing is easy. But testing impure functions can be very, very difficult.

Fortunately, the Jest testing framework has simplified a lot of the complexities behind testing. The secret is a tool called a mock. A mock is effectively a shortened version of a function. Instead of running the function you stub out the result you want. Your code will return what you want instead of running the live code.

In other words, you say what an API should return. The mock will bypass the live API and return the data directly. Instead of relying on a date function that changes every 24 hours, you effectively freeze it in place by returning the same date every time.

Warning

Mocks are a great tool, but they should be a last resort. If your tests use a significant amount of mocks (or spies or stubs or other testing trick), consider refactoring first. After you have isolated the complexities as much as possible, then it’s time to use mocks.

Jest: A Brief Intro

Jest is a testing framework developed at facebook, it’s used most often with React, but it does work independently.

Jest offers runtime testing tools. an expect library and system for mocks and spies right out of the box.

At it’s most basic, a Jest test will look nearly identical to mocha or jasmine or any other framework. In fact, Jest started as a fork of Jasmine.

import { double } from './double';

describe('Some Test', () => {
  it('should add two number', () => {
    expect(double(1)).toEqual(2);
  });
});

In this case, toEqual is the assertion.

That’s pretty much all you need for the purposes of this article. There are plenty of extra features that you can read about in the documentation. You can also look there for installation and configuration. If you want to follow along, check out the repo for this article.

Creating Testable API Functions

Return to your API code. Now that you have your code and you have your test suite all set up, it’s time to write tests. As a reminder, you have a piece of code that fetches API data and parses the results.

export function getRecent() {
  return fetch('/albums')
    .then(response => response.json())
    .then(albums => {
      return albums
        .reverse()
        .slice(0, 2)
        .map(({title}) => title);
    })
}

The first step in creating a mock is isolating the pain point. In this case, the problem is the fetch block. Specifically this part:

return fetch('/albums')
    .then(response => response.json())

The rest of the code isn’t so bad. It’s just a series of array manipulations.

The next step if figuring out what kind of data from the hard part you expect to pass down to the easy part. Again, all you need to do is look at what the live API will return by either hitting it directly or by getting the contract from some other team. In this case, you plan to get the albums of the great doom metal band Sleep. The data you expect to get will look something like this:

[
  {
    "year": 1992,
    "title": "Holy Mountain"
  },
  {
    "year": 1999,
    "title": "Jerusalem"
  },
  {
    "year": 2014,
    "title": "The Clarity"
  },
  {
    "year": 2018,
    "title": "The Sciences"
  }
]

Now, you should split the function. Pull the hard part away from the easy part. That will make your tests cleaner and easier to mock.

As a quick side note, some consider it bad practice to break up your code to make it more testable. I would disagree. Testability tells you something about the complexity of your code. The harder it is to test, the more complex it tends to be. When you are refactoring your code to make it more testable, you just changing the complex to the simple.

Since you’ve isolated the complex part, the next the step is pulling the fetch call out into a separate function. Call the new functionservice and since it now has a separate responsibility go ahead and move it into a new file called service.js. To keep things clean, move it to a separate directory called api this will be important later when you set up the mock.

export default function service() {
  return fetch('/albums')
    .then(response => response.json())
}

Now you need to import the service and replace the original fetch method. Your original function will be a little smaller, but you still are able to perform the exact same actions as your did before:

import service from '../api/service';

export function getRecent() {
  return service()
    .then(albums => {
      return albums
        .reverse()
        .slice(0, 2)
        .map(({ title }) => title);
  })
}

At this point, you should start to see an opportunity for a mock. Remember, a mock is just replacing a live piece of code with a simple function that returns the data you expect.

In this case, you have a function service that returns a Promise that returns some sort of array. You happen to know this service hits an API, but that doesn’t really matter. It can be loading data from localstorage as far as you’re concerned.

Time to jump into the test. Here’s what it’ll look like:

import { getRecent } from './recent';

describe('getRecent', () => {
  it('should get albums', async () => {
    const albums = await getRecent();
    const expected = [
      'The Sciences',
      'The Clarity'
    ]
    expect(albums).toEqual(expected);
  })
})

Since getRecent returns a promise, you can use async/await syntax. This code says, run the getRecent function and once it resolves, make sure the albums are as expected.

Jest will wait for a promise to resolve if you return. That means you can also write the test like this:

import { getRecent } from './recent';

describe('getRecent', () => {
  it('should get albums', () => {
    return getRecent()
      .then(albums => {
        const expected = [
          'The Sciences',
          'The Clarity'
        ]
        expect(albums).toEqual(expected);
      })
  })
})

Either way, you still need to account for the service. Currently, it will still try (and probably fail) to hit a live endpoint.

Now, it’s time to start mocking the results.

Mocking an API Result

There are multiple ways to mock data, but the easiest is by creating a manual mock at the same level as the code you need to bypass.

In practical terms, you are going to make a file of the same name that returns data that you want instead of executing a function to get that data.

Start by making a directory called __mocks__ in the same directory as service.js Suppose you had a directory called api that contained services.js, you’d add your __mocks__ in the same api directory like this:

| api
  | __mocks__
    | service.js
  | service.js

Inside __mocks__/service.js create a function with the same name, but that returns the data you want.

const albums = [
  {
    year: 1992,
    title: 'Holy Mountain'
  },
  {
    year: 1999,
    title: 'Jerusalem'
  },
  {
    year: 2014,
    title: 'The Clarity'
  },
  {
    year: 2018,
    title: 'The Sciences'
  },
];

export default function service() {
  return Promise.resolve(albums)
}

Just as with the actual service function. This one returns a Promise but unlike the original function, this one has the data hard coded. It will never change.

Now that you’ve set up your mock, you need to alert your test that you want to use the manual mock instead of the real function. To do that, you need to use jest.mock(path) where the path is the location of the file you want to mock.

Suppose you had a file structure like this:

| albums
  | recent.js
  | recent.spec.js
| api
  | __mocks__
    | service.js
  | service.js

In your test, add the line jest.mock('../api/service'). Notice, you are not adding the path to the __mocks__ directory, you are using the same path as if you were importing the function. Jest will find the mock for you. You are using the same path as you would in recent.js.

import { getRecent } from './recent';

jest.mock('../api/service');

describe('recent', () => {
  it('should get albums', async () => {
    const albums = await getRecent();
    const expected = [
      'The Sciences',
      'The Clarity'
    ]
    expect(albums).toEqual(expected);
  })
})

Now, you are not hitting a live API. You are hitting an internal simulation. And it only takes one line of code. You don’t need network connectivity. You don’t need to worry about results changing. You don’t need to spin up a separate service. Everything is happening internally.

What’s next?

Mocking API data is one of the best uses of mocks. Nearly all JavaScript projects need to be able to access remote data. However, mocks can do a lot more. They can also deal with complicated side effects such as date time or DOM manipulations. In my next article, I’ll show you how to mock out moment and other libraries that return data that changes depending on the day or even the location of the test.

Let’s convert a mock API endpoint for signing up new users in a mobile app into using the action pattern.

When I first started writing software on the web, my code was a mishmash. Every project was loaded with unnecessarily long files and code left commented, thrown to the side of the road like an abandoned vehicle. The theme of the day was: unpredictability.

Under ideal conditions—the happy path—I was able to make my code work. But what I couldn’t do was make my code work consistently. One time through my code would work, then the next an anonymous 500 Internal Server Error would send me spiraling for days.

I was able to squeak by, but the thought of having to continue answering emails from clients that read “this isn’t working…” was a life I didn’t want to lead.

Taking off my beginner’s hat, I started to see what other, more experienced programmers were up to. I’d heard about Bob “Uncle Bob” Martin in passing, eventually discovering his Clean Coders series. I was hooked. For the first time, he was answering questions others on my path hadn’t.

My primary question? “How do I organize complex code?” As far as questions go that was a ball of yarn, but over several videos he explained the parts I was missing:

• Using explicit names that can’t be mistaken.
• Breaking your code into functions that do one thing.
• Using TDD (test-driven development) to guide yout work.

Still green, some of this made sense and some of it didn’t. The other problem was that Bob’s language of choice was Java, not JavaScript. This meant that I was able to grasp what he was saying at a high level, but practically I was still stumped.

Several iterations later…

Eventually, what Bob taught started to sink in. As I gained experience, I slowly started to organize my code into a pattern (supported by a short list of rules):

1. Any code that involves multiple steps should be moved into its own file/module.
2. That file/module should be given a name that describes what those steps lead up to.
3. Each step in that code should be a single function with a name that describes exactly what it does (even if it’s longer than we prefer).
4. If the code fails, it should be easy to see exactly where it failed without a lot of backstepping.

What started out as an informal set of rules for myself eventually evolved into a concrete pattern. After years of iteration and putting it through the paces on client and personal projects, in 2017 the action pattern was christened.

How Actions work

For the remainder of this tutorial, we’re going to convert a mock API endpoint for signing up new users in a mobile app into an action. Our goals:

1. Understand the structure of an action.
2. Learn how to use JavaScript Promises with actions.
3. Find a greater “why” for using actions.
4. Understand how writing tests is simplified by using actions.

Converting Our Existing API Endpoint

Our app, Doodler (a paid social network for artists), handles its signups via an existing Express-based API. When a new user signs up in the app, a request is made to their API at https://doodler.fake/api/v1/users/signup.

At that endpoint, the following steps take place:

• A new user is created in the users collection.
• A new customer is created on Stripe.
• A customer is created in the customers collection.
• A welcome email is generated.
• A “new user” message is sent to the company’s Slack.

Together, these five steps represent the action of signing up a new user. Because some of the steps are dependent on prior steps, we want to have some way to “stop” our code if earlier steps fail. Before we get into the weeds, let’s take a look at the code we have now:

/* eslint-disable */

import mongodb from '/path/to/mongodb';
import settings from '/path/to/settings';
import stripe from '/path/to/stripe/api';
import imaginaryEmailService from '/path/to/imaginaryEmailService';
import slackLog from '/path/to/slackLog';

export default {
  v1: {
    '/users/signup': (request, response) => {
      mongodb.connect(settings.mongodb.url, function (error, client) {
        const db = client.db('production');
        const users = db.collection('users');
        const customers = db.collection('customers');

        users.insert({ email: request.body.email, password: request.body.password, profile: request.body.profile }, async function (error, insertedUser) {
          if (error) {
            throw new Error(error);
          } else {
            const [user] = insertedUser;
            const userId = user._id;
  
            const customerOnStripe = await stripe.customers.create({
              email: request.body.email,
            });

            customers.insert({ userId, stripeCustomerId: customerOnStripe.id }, async function (error, insertedCustomer) {
              if (error) {
                throw new Error(error);
              } else {
                imaginaryEmailService.send({ to: request.body.email, template: 'welcome' });
                slackLog.success({
                  message: 'New Customer',
                  metadata: {
                    emailAddress: request.body.email,
                  },
                });

                response.end();
              }
            });
          }
        });
      });
    },  
  },
};

Looking at this code, assuming that all of the parts in use work on their own, it’s plausible that this code will work. What’s distinct about this code, however, is that it’s not terribly organized. It contains a lot of nested calls and not much flow control (i.e., if something fails, the whole house of cards falls).

This is where we start to tiptoe up to the “works” vs. “works well” chasm. Unfortunately, it’s code like this that leads to a lot of wasted time chasing down and fixing bugs. It’s not that the code doesn’t work, it’s that it works unpredictably.

You’re probably saying “well yeah, all code is unpredictable.” You’re not wrong. But, if we’re smart we can significantly reduce the amount of unpredictability, giving us more time to focus on fun stuff—not fixing mistakes of the past (either of our own making or someone on our team).

Introducing the Action Pattern

First and foremost, it’s important to understand that the action pattern is vanilla JavaScript. It’s a pattern to follow, not a library or framework to implement. This means that using actions requires a certain level of discipline (the majority of which can be automated via snippets in your IDE).

To get started with our conversion, let’s look at a skeleton version of an action and then build it up to handle our new user signup.

/* eslint-disable consistent-return */

const actionMethod = (someOption) => {
  try {
    console.log('Do something with someOption', someOption);
    // Perform a single step in your action here.
  } catch (exception) {
    throw new Error(`[actionName.actionMethod] ${exception.message}`);
  }
};

const validateOptions = (options) => {
  try {
    if (!options) throw new Error('options object is required.');
    if (!options.someOption) throw new Error('options.someOption is required.');
  } catch (exception) {
    throw new Error(`[actionName.validateOptions] ${exception.message}`);
  }
};

export default (options) => {
  try {
    validateOptions(options);
    actionMethod(options.someOption);
    // Call action methods in sequence here.
  } catch (exception) {
    throw new Error(`[actionName] ${exception.message}`);
  }
};

Actions are designed to be read from the bottom up. At the bottom of our file, we export a function known as our handler. This function is responsible for calling to all of the other steps in our action. This helps us to accomplish a few things:

1. Centralize all of our calls to other code in one place.
2. Share response values from each step with other steps.
3. Clearly delineate the order of steps in our code.
4. Make our code more maintainable and extensible by avoiding nested spaghetti code.

Inside of this function, the very first thing we do is call to validateOptions passing in the assumed options argument passed to the handler function (or, what we export from our file as our action).

With validateOptions we start to see a few other sub-patterns of actions appear. Specifically, the name of the validateOptions function is called exactly what it does. It’s not vldOpts or validateOps or anything that leaves room for confusion. If I were to drop another developer into this code, and ask them “what does that function do?” they’ll most likely respond sarcastically with “uhh, validates the options?”

The next thing you’ll notice is the structure of validateOptions. Immediately inside of the function body, a try/catch statement is added, with the catch portion taking the exception and throwing it using the JavaScript Error constructor. Notice, too, that when this error is thrown, we tell ourselves exactly where the error is happening with [actionName.validateOptions] followed by the specific error message.

In the try block, we do what our code says: validate our options! The logic here is kept simple on purpose. If our action requires that options be passed and requires specific properties to be defined in those options, we throw an error if they don’t exist. To make sure this clear, if we were to call this action now like this:

actionName();

We’d get the following error in response:

[actionName.validateOptions] options object is required.

This is a serious boon on development. We’re telling ourselves exactly what we need up front so we can skip the “what did I forget to pass now?” roulette.

If we move back down to our handler function, we’ll see that after our options have been validated with validateOptions, our next step is to call actionMethod, passing options.someOptions.

This is where we get into the actual steps or functionality of our action. Here, actionMethod takes in options.someOption. Notice that because it’s the second step called in our handler, it’s defined above validateOptions (our first step).

If we look at actionMethod things should—purposefully—look pretty familiar. Here, we repeat the same pattern: give a clear name for our function, run our code in a try/catch block, and if our code fails, throw an error telling ourselves that it came from [actionName.actionMethod].

Refactoring our signup

Feeling undewherlmed? Great! That’s what we’re after. Writing clean code shouldn’t be difficult or excessively esoteric. Now, let’s start to refactor our signup endpoint into an action. Let’s clean up our skeleton, adding some legitimate checks to validateOptions:

const actionMethod = (someOption) => {
  try {
    console.log('Do something with someOption', someOption);
    // Perform a single step in your action here.
  } catch (exception) {
    throw new Error(`[signup.actionMethod] ${exception.message}`);
  }
};

const validateOptions = (options) => {
  try {
    if (!options) throw new Error('options object is required.');
    if (!options.body) throw new Error('options.body is required.');
    if (!options.body.email) throw new Error('options.body.email is required.');
    if (!options.body.password) throw new Error('options.body.password is required.');
    if (!options.body.profile) throw new Error('options.body.profile is required.');
    if (!options.response) throw new Error('options.response is required.');
  } catch (exception) {
    throw new Error(`[signup.validateOptions] ${exception.message}`);
  }
};

export default (options) => {
  try {
    validateOptions(options);
    // Call action methods in sequence here.
    options.response.end();
  } catch (exception) {
    throw new Error(`[signup] ${exception.message}`);
  }
};

A few things have changed. Notice that instead of actionName, we’ve now given our action a name: signup.

Inside of validateOptions, we’ve set some real expectations, too. Remember that in our original code, we reuse the request.body object several times. Here, we think ahead and make the assumption that we’ll just pass the body part of the request (the only part we utilize). We also make sure to validate that each of the properties of the body are present.

Finally, we also want to validate that the response object from our endpoint is passed so we can respond to the request within our action.

The details of this are mostly arbitrary; the point here is that we’re ensuring we have what we need before we put it to use. This helps to eliminate the inevitable “did I pass that yet?” question as well as subsequent time wasted debugging to figure it out.

Adding additional steps as functions

Now that we have our handler function set up as well as our validateOptions, we can start to port over the core functionality for our action.

/* eslint-disable consistent-return */

import mongodb from '/path/to/mongodb';
import settings from '/path/to/settings';

const connectToMongoDB = () => {
  try {
    return new Promise((resolve, reject) => {
      mongodb.connect(
        settings.mongodb.url,
        (error, client) => {
          if (error) {
            reject(error);
          } else {
            const db = client.db('production');
            resolve({
              db,
              users: db.collection('users'),
              customers: db.collection('customers'),
            });
          }
        },
      );
    });
  } catch (exception) {
    throw new Error(`[signup.connectToMongoDB] ${exception.message}`);
  }
};

const validateOptions = (options) => [...];

export default async (options) => {
  try {
    validateOptions(options);
    const db = await connectToMongoDB();
  } catch (exception) {
    throw new Error(`[signup] ${exception.message}`);
  }
};

First, we need to establish a connection to our database. Recall, we need access to the users and customers collection from MongoDB. Knowing this, we can streamline our code by creating an action method connectToMongoDB whose sole job is connecting us to MongoDB, giving us access to the databases we’ll need to do our work.

To do it, we wrap our call to mongodb.connect using the action method pattern. By wrapping this code with a JavaScript Promise, we can ensure our connection is complete before we try to use it. This is necessary because we’re no longer running our subsequent code accessing the database inside of mongodb.connect's callback. Instead, we resolve our Promise passing the db connection along with the two databases that we’ll need: users and customers.

Why is this important? Consider this: our connection to MongoDB could fail. If it does, we not only want to know why, but we want our code to be easily debugged. With nested spaghetti code, this is possible, but adds mental weight.

By encapsulating our call—and any failures—inside of a single function, we eliminate the need to track down errors. This is especially helpful when the errors themselves are unhelpful or ambiguous (R.I.P to souls who get an ECONNRESET). The difference between ERR ECONNRESET and [signup.connectToMongoDB] ERR ECONNRESET is night and day. The error may not be clear, but we’ve told ourselves exactly who’s responsible.

Back in our handler function, we utilize the async/await syntax to ensure that we’ve received a response from MongoDB before we continue with the rest of our action (i.e., we’ve achieved what our callback gave us without opening an Italian restaurant).

/* eslint-disable consistent-return */

import mongodb from '/path/to/mongodb';
import settings from '/path/to/settings';

const createUser = (users, userToCreate) => {
  try {
    return new Promise((resolve, reject) => {
      users.insert(userToCreate, (error, insertedUser) => {
        if (error) {
          reject(error);
        } else {
          const [user] = insertedUser;
          resolve(user._id);
        }
      });
    });
  } catch (exception) {
    throw new Error(`[signup.createUser] ${exception.message}`);
  }
};

const connectToMongoDB = () => [...];

const validateOptions = (options) => [...];

export default async (options) => {
  try {
    validateOptions(options);

    const db = await connectToMongoDB();
    const userId = await createUser(db.users, options.body);
  } catch (exception) {
    throw new Error(`[signup] ${exception.message}`);
  }
};

Next up is creating our user. This is where the magic of actions start to show. Down in our handler function, we add our next step createUser beneath our first step connectToMongoDB. Notice that when we need to reference the value returned by a previous step in future steps, we give it a variable name that represents exactly what’s being returned.

Here, const db suggests we get access to our database in that variable and const userId suggests we expect a user’s _id back from createUser. In order to get there, we know that we need to connect to the users collection in MongoDB and we need the user information passed in the request.body to create that user. To do it, we just pass those values as arguments to createUser. Clean and tidy.

const createUser = (users, userToCreate) => {
  try {
    return new Promise((resolve, reject) => {
      users.insert(userToCreate, (error, insertedUser) => {
        if (error) {
          reject(error);
        } else {
          const [user] = insertedUser;
          resolve(user._id);
        }
      });
    });
  } catch (exception) {
    throw new Error(`[signup.createUser] ${exception.message}`);
  }
};

Focusing just on our createUser definition, we can see that we take in that db.users argument as users and options.body as userToCreate (remember, this should be an Object with email, password, and profile as properties).

Using the same Promise approach, we call to users.insert and rely on our resolve and reject to handle the respective error and success states of our call to users.insert. If our insert is successful, we get the _id of the insertedUser and resolve() our Promise with it.

Pay close attention. Because we’re calling resolve(user._id), this means that back in our handler function, our const userId = createUser() is now “truthful” because once that Promise resolves, we’ll get the userId in return, assigned to that variable. Sweet!

Completing our action

At this point, we’re familiar with the core concepts of an action. Once the full conversion is complete, here’s what we get:

import mongodb from '/path/to/mongodb';
import settings from '/path/to/settings';
import stripe from '/path/to/stripe/api';
import imaginaryEmailService from '/path/to/imaginaryEmailService';
import slackLog from '/path/to/slackLog';

const logCustomerOnSlack = (emailAddress) => {
  try {
    slackLog.success({
      message: 'New Customer',
      metadata: {
        emailAddress,
      },
    });
  } catch (exception) {
    throw new Error(`[signup.logCustomerOnSlack] ${exception.message}`);
  }
};

const sendWelcomeEmail = (to) => {
  try {
    return imaginaryEmailService.send({ to, template: 'welcome' });
  } catch (exception) {
    throw new Error(`[signup.sendWelcomeEmail] ${exception.message}`);
  }
};

const createCustomer = (customers, userId, stripeCustomerId) => {
  try {
    return new Promise((resolve, reject) => {
      customers.insert({ userId, stripeCustomerId }, (error, insertedCustomer) => {
        if (error) {
          reject(error);
        } else {
          const [customer] = insertedCustomer;
          resolve(customer._id);
        }
      });
    });
  } catch (exception) {
    throw new Error(`[signup.createCustomer] ${exception.message}`);
  }
};

const createCustomerOnStripe = (email) => {
  try {
    return stripe.customer.create({ email });
  } catch (exception) {
    throw new Error(`[signup.createCustomerOnStripe] ${exception.message}`);
  }
};

const createUser = (users, userToCreate) => {
  try {
    return new Promise((resolve, reject) => {
      users.insert(userToCreate, (error, insertedUser) => {
        if (error) {
          reject(error);
        } else {
          const [user] = insertedUser;
          resolve(user._id);
        }
      });
    });
  } catch (exception) {
    throw new Error(`[signup.createUser] ${exception.message}`);
  }
};

const connectToMongoDB = () => {
  try {
    return new Promise((resolve, reject) => {
      mongodb.connect(
        settings.mongodb.url,
        (error, client) => {
          if (error) {
            reject(error);
          } else {
            const db = client.db('production');
            resolve({
              db,
              users: db.collection('users'),
              customers: db.collection('customers'),
            });
          }
        },
      );
    });
  } catch (exception) {
    throw new Error(`[signup.connectToMongoDB] ${exception.message}`);
  }
};

const validateOptions = (options) => {
  try {
    if (!options) throw new Error('options object is required.');
    if (!options.body) throw new Error('options.body is required.');
    if (!options.body.email) throw new Error('options.body.email is required.');
    if (!options.body.password) throw new Error('options.body.password is required.');
    if (!options.body.profile) throw new Error('options.body.profile is required.');
    if (!options.response) throw new Error('options.response is required.');
  } catch (exception) {
    throw new Error(`[signup.validateOptions] ${exception.message}`);
  }
};

export default async (options) => {
  try {
    validateOptions(options);

    const db = await connectToMongoDB();
    const userId = await createUser(db.users, options.body);
    const customerOnStripe = await createCustomerOnStripe(options.body.email);

    await createCustomer(db.customers, userId, customerOnStripe.id);
    sendWelcomeEmail(options.body.email);
    logCustomerOnSlack(options.body.email);
  } catch (exception) {
    throw new Error(`[signup] ${exception.message}`);
  }
};

A few things to point out. First, all of our additional action methods have been added to our handler, called in sequence.

Notice that after we’ve created a customer on Stripe (and returned that customer as const customerOnStripe), none of the steps after this need a value from the previous steps. In turn, we just call these steps independently without storing their return value in a variable.

Notice, too, that our sendWelcomeEmail and logCustomerOnSlack steps remove the usage of an await because there’s nothing for us to wait on.

That’s it! At this point, we have a complete action.

Wait, but why?

You’re probably wondering “didn’t we just add a ton of extra code to do the same thing?” We did. But something important to consider is how much context and clarity adding that (negligible amount of) extra code gave us.

This is the point of actions: giving us a consistent, predictable pattern for organizing complex processes. That’s a mouthful, so another way to think about this is reducing maintenance cost. Nobody likes to maintain code. Often, too, when we’re tasked with maintaining a “legacy” codebase, it tends to look more like the code we started with.

What this translates to is cost. Cost in time, money, and for the people doing the work: peace of mind. When code is a tangle of pasta, there’s a cost to understanding that code. The less structure and consistency, the higher that cost.

With actions, we can significantly reduce the amount of thinking that goes into maintaining our code. Not only that, but we also make it incredibly easy to extend our code. For example, if we’re asked to add the ability to log the new user in our analytics system, there’s little to no thought involved.

[...]
import analytics from '/path/to/analytics';

const trackEventInAnalytics = (userId) => {
  try {
    return analytics.send(userId);
  } catch (exception) {
    throw new Error(`[signup.trackEventInAnalytics] ${exception.message}`);
  }
};

const logCustomerOnSlack = (emailAddress) => [...];

const sendWelcomeEmail = (to) => [...];

const createCustomer = (customers, userId, stripeCustomerId) => [...];

const createCustomerOnStripe = (email) => [...];

const createUser = (users, userToCreate) => [...];

const connectToMongoDB = () => [...];

const validateOptions = (options) => [...];

export default async (options) => {
  try {
    validateOptions(options);

    const db = await connectToMongoDB();
    const userId = await createUser(db.users, options.body);
    const customerOnStripe = await createCustomerOnStripe(options.body.email);

    await createCustomer(db.customers, userId, customerOnStripe.id);
    sendWelcomeEmail(options.body.email);
    logCustomerOnSlack(options.body.email);
    trackEventInAnalytics(userId);
  } catch (exception) {
    throw new Error(`[signup] ${exception.message}`);
  }
};

This means that instead of wasting your own time and energy, you can implement features and fix bugs with very little stress. The end result is a happier you and happier stakeholders. Good deal, right?

While it’s a minor detail, just so it’s clear, let’s look at how we actually use our action back in our API:

import signup from '/path/to/signup/action';

export default {
  v1: {
    '/users/signup': (request, response) => {
      return signup({ body: request.body, response });
    },  
  },
};

This would be an appropriate time for a Bill Cosby “puddin’ face” GIF, but, well…you know.

Testing our action

The final “wow” of actions is how easy they are to test. Because the code is already in steps, an action tells us what we need to test. Assuming we’ve mocked the functions in use inside of our action (e.g., stripe.customers.create) an integration test for our action might look like this:

import signup from '/path/to/signup/action';
import stripe from '/path/to/stripe';
import slackLog from '/path/to/slackLog';

const testUser = {
  email: 'test@test.com',
  password: 'password',
  profile: { name: 'Test User' },
};

describe('signup.js', () => {
  beforeEach(() => {
    stripe.customers.create.mockReset();
    stripe.customers.create.mockImplementation(() => 'user123');

    slackLog.success.mockReset();
    slackLog.success.mockImplementation();
  });

  test('creates a customer on stripe', () => {
    signup({ body: testUser });
    expect(stripe.customers.create).toHaveBeenCalledTimes(1);
    expect(stripe.customers.create).toHaveBeenCalledWith({ email: testUser.email });
  });

  test('logs the new customer on slack', () => {
    signup({ body: testUser });
    expect(slackLog.success).toHaveBeenCalledTimes(1);
    expect(slackLog.success).toHaveBeenCalledWith({
      message: 'New Customer',
      metadata: {
        emailAddress: testUser.email,
      },
    });
  });
});

Here, each test represents a verification that the step in our action completed as expected. Because we only care that our action performed the steps, our test suite is dirt simple. All we need to do is make a call to our action with some input (in this case, we pass a testUser object as the options.body value in our action).

Next, we verify that our steps complete. Here, we verify that given a user with an email test@test.com, our action calls to stripe.customers.create passing that same email. Similarly, we test to see of our slackLog.success method was called, passing the message we’d like to see in our logs.

There’s ample nuance with testing, of course, but hopefully the point here is clear: we have a very tidy chunk of code that’s remarkably easy to test. No confusion. No time wasted “figuring it out.” The only true cost would be the time mocking out the code called by our action if we hadn’t done that already.

Wrapping Up

So there you have it! Actions are a wonderful way to clean up your codebase, make things more predictable, and save yourself a ton of time in the process.

Because actions are just a JavaScript pattern, the cost to test them out in your own app is zero. Try it, see if you like it. Most importantly: see if they improve the quality of your code. If you’re struggling to write code that performs predictably, give this pattern a try. You won’t regret it.

Proper state handling in React will make your components simple and maintainable. Poor choices will give you lots of headaches in the long-term.

There are plenty of options for managing state in a React app. But there’s very little guidance about which one you should use in any situation.

Let’s fix that.

The solution you pick to manage state should fit they way you want to use the items you hold in state. Your choice should also make it easy for you to update, reuse, and refactor your code. In other words, your choice of state management can make your life easy or hard.

Here’s a non-exhaustive list of things you should consider:

1. How quickly can you add new state code?
2. How well encapsulated is your stateful data?
3. How easy is it to pull the data into a new component?

Let’s use this list as a basis for comparing different ways of handling state. To do this, we’ll make the same simple component using three different forms of state handling. By making the same thing three ways, you’ll get a few nice perspectives.

First, you’ll see that any approach can solve any problem. This is important because if you know that you can solve nearly any problem with any form of state management, you can turn your attention from the technical problem of getting it working, to the more abstract problem of making it work well in your over all application.

Second, by making the same thing three ways, you’ll start to see the subtle differences which can become big headaches as an applications grows.

What are the three ways to manage state?

1. Local State: State stored directly on the component.
2. Global State: A global store. In this example, you’ll use redux.
3. Context: A newer API that lets you access state higher up the tree.

Using the three points of comparison and the three ways of managing state, you’ll be able to build a nice table that will let you compare the different options.

Alright. Let’s get started.

You’ll be making a very simple counter. It’s just two things: the current count and a button to add to the current count. As mentioned you’ll have one for each type of state, but they all act exactly the same way.

So far, nothing too exciting. The first thing you are going to do is make the presentation layer. This will be a simple reusable component that displays the count and the button. Making this independent is not only a good practice, it will also a show you how easy it is to swap different forms of state handling.

import React from 'react';

export default function Display({ addOne, count, title}) {
  return(
    <div className="counter">
      <h3>{title}</h3>
      Current Count: {count}
      <button onClick={addOne}> Add One! </button>
    </div>
  )
}

Now that you have your basic display component, you can start to build the stateful components.

Local State

The first component you’ll build will store state locally, directly on the component.

All you need is a basic class with state as a property and a method to add to state. Pass both the count from the state object and the addOne() method into your presentation component and you’re all done.

import React, { Component } from 'react';
import Display from '../Display/Display';

export default class StateCounter extends Component {
  state = {
    count: 0
  }

  addOne = () => {
    this.setState(state => ({count: state.count + 1}))
  }

  render() {
    return (
      <Display
        addOne={this.addOne}
        count={this.state.count}
        title="Local Counter"
      />
    )
  }
}

At this point, one thing should jump out: it’s easy. Even without knowing much about other forms of state handling, you can see this one is very simple to set up. But it gets even better.

With the introduction of React Hooks, you’ll remove nearly all the boilerplate code.

Here’s an updated version using React hooks that’s nearly identical to the official docs:

import React, { useState } from 'react';
import Display from '../Display/Display';

export default function StateCounterHooks() {
  const [count, add] = useState(0);
  const addOne = () => add(count + 1);
  return (
    <Display
      addOne={addOne}
      count={count}
      title="Local Counter"
    />
  )
}

As above, count is the stateful data, and add() is a method for updating state. It would really be difficult to make this easier.

Global State

The next component you’ll build will use a global store. In this example, you’ll use redux since it’s by far the most popular, but many of the issues would be the same if you were using other options like MobX or even Apollo and GraphQL.

We’ll move quickly past some of the complexities of implementing redux. If you need a refresher, check out the official docs.

To start make an action to add an item to the global store.

export function add() {
  return ({
    type: 'ADD_ONE',
  })
}

Next, make a reducer to actually store the count.

function count(state = 0, action) {
  switch (action.type) {
    case 'ADD_ONE':
      return state + 1;
    default:
      return state;
  }
}

export default combineReducers({
  count,
})

If you’ve worked with redux before you may have ignored this part, but it’s important to take a moment to think about what you just did here.

You just created a global state object with the a property of count . That property name is now unique for this particular action. If you wanted to create another property with a similar name you’d need to either give it a unique name such as clickCount or encapsulate one, the other, or both inside a nested object.

You’ll return to this idea in a bit. For now, use connect to pull the state into your display component.

import React from 'react';
import { connect } from 'react-redux'

import { add } from '../../store/actions';
import Display from '../Display/Display';

export function ReduxCounter({ addOne, count }) {
  return(
    <Display
      addOne={addOne}
      count={count}
      title="Redux Counter"
    />
  )
}

const mapStateToProps = ({ count }) => ({ count });

const mapDispatchToProps = { addOne: add }

export default connect(mapStateToProps, mapDispatchToProps)(ReduxCounter);

Again, we won’t explore the details, but as a summary, the component is tapping into the global store so that you can both read from it, using mapStateToProps, and update it using mapDispatchToProps. Other than that, your display component doesn’t care where the data comes from.

So far, the thing that should jump out immediately is how much work is involved. There are more files, more imports, more complexity. That’s not a problem if the advantages outweigh the extra work — and they often will — but it is still something to consider.

Context

The final component will use context. This is a fairly new form of state handling that was introduced in React 16.3. Context is similar to local state in many ways, but it has a slightly different implementation.

To start off, you need to create a context. This will set up a couple defaults that you will use when you create an implementation.

import React from 'react';

const CounterContext = React.createContext({
  addOne(){},
  counter: 0,
});

export default CounterContext;

This sets the initial state of 0 and adds a noop function that you will override later.

To actually use context, you will first need to make a Provider. A provider is the base component that will hold the state object and the methods for updating the state.

import React, { Component } from 'react';
import CounterContext from './CounterContext';

export default class ContextProvider extends Component {
  state = {
    count: 0,
  }

  addOne = () => {
    this.setState(state => ({count: state.count + 1}))
  }
  
  render() {
    const { count } = this.state;

    const value = {
      addOne: this.addOne,
      count,
    };

    return (
      <CounterContext.Provider value={value}>
        {this.props.children}
      </CounterContext.Provider>
    )
  }
}

If you scroll up, you’ll notice this looks very similar to the local state component. The big difference is that you are combing the state and the update function in single object which you then pass down into a Provider component that wraps everything else. In this case, you are wrapping this.props.children but you can also wrap a component directly.

To use this component, you’d wrap some other components. In this case, you’ll wrap a single component.

import React from 'react';
import CounterProvider from '../ContextCounter/CounterProvider';
import OtherComponent from '../ContextCounter/OtherComponent';

export default function Counter() {
  return(
    <div>
      <CounterProvider>
          <OtherComponent />
      </CounterProvider>
    </div>
  )
}

Notice, you are not passing any props to OtherComponent . More clearly, you are not explicitly passing anything to OtherComponent. The provider is holding the state information so that you can tap into it later.

Even if you go several components deep, you will still be able to access the provider’s state. So if OtherComponent returns another component:

import React from 'react';
import AnotherComponent from './AnotherComponent';

export default function OtherComponent() {
  return(
    <AnotherComponent />
  )
}

And AnotherComponent returns yet another component. That information is still hanging in the background.

import React from 'react';
import ContextDisplay from './ContextDisplay';

export default function AnotherComponent() {
  return(
    <ContextDisplay />
  )
}

Ok, that’s far enough. It’s time to pull out the state and do something with it.

To get the information you need to create a Consumer. The consumer uses render props to pull out that single value object. Remember, value contains count and addOne . You can use destructuring as a short hand.

Now that you have the state and the function to update state, you can pass it into your display component:

import React from 'react';
import CounterContext from './CounterContext';
import Display from '../Display/Display'

export default function ContextDisplay() {
  return(
    <CounterContext.Consumer>
      {({ addOne, count}) => {
        return (
          <Display
            addOne={addOne}
            count={count}
            title="Context Counter"
          />
        )
      }}
    </CounterContext.Consumer>
  )
}

Comparisons

As you saw, any form of state handling can do the same thing. In reality, if you put effort into it, you can make any state handling system work for any piece of data in your application.

Of course, who wants to put effort into it? The goal is always less effort, better results. With that in mind, it’s time to compare the different approaches.

Simplicity

This is the easiest comparison. Nothing beats local state particularly when you use React hooks. It’s quick, it does that job and it’s easy to refactor into a more complicated system if necessary. As a rule, you should always start with local state. If you need something more complicated, you can refactor as you go.

For this reason, Local State wins for simplicity.

Encapsulation

Next, let’s explore how well the data is encapsulated. If data is well encapsulated, you can reuse a component multiple times without any concern that another component may accidentally change your state. Encapsulation isn’t necessarily a goal in itself. There are times where you absolutely want different components to access and modify shared data (more on that in the next section).

As with most things, it’s about predictability. If you expect a component to be independent, the data shouldn’t be open to modifications by other components.

To explore encapsulation, you’ll need to add multiple versions of a component to see how they interact.

To start off return to the original page that has all three components.

The code looks something like this:

import React from 'react';
import StateCounter from '../StateCounter/StateCounterHooks';
import ReduxCounter from '../ReduxCounter/ReduxCounter';
import CounterProvider from '../ContextCounter/CounterProvider';
import OtherComponent from '../ContextCounter/OtherComponent';

export default function Counter() {
  return(
    <div>
      <StateCounter />
      <ReduxCounter />
      <CounterProvider>
          <OtherComponent />
      </CounterProvider>
    </div>
  )
}

You have a simple parent component that wraps one instance of each component. To test encapsulation, all you need to do is add a second instance.

Start off with the StateCounter component. The updated code looks like this:

import React from 'react';
import StateCounter from '../StateCounter/StateCounterHooks';
import ReduxCounter from '../ReduxCounter/ReduxCounter';
import CounterProvider from '../ContextCounter/CounterProvider';
import OtherComponent from '../ContextCounter/OtherComponent';

export default function Counter() {
  return(
    <div>
      <StateCounter />
      <StateCounter />
      <ReduxCounter />
      <CounterProvider>
          <OtherComponent />
      </CounterProvider>
    </div>
  )
}

When you open that in the browser, you can easily see that the data is well encapsulated. Every time you click the add button, it only updates that count on that particular component.

Now try adding a second redux component.

import React from 'react';
import StateCounter from '../StateCounter/StateCounterHooks';
import ReduxCounter from '../ReduxCounter/ReduxCounter';
import CounterProvider from '../ContextCounter/CounterProvider';
import OtherComponent from '../ContextCounter/OtherComponent';

export default function Counter() {
  return(
    <div>
      <StateCounter />
      <ReduxCounter />
      <ReduxCounter />
      <CounterProvider>
          <OtherComponent />
      </CounterProvider>
    </div>
  )
}

Open this in the browser and notice how different it is. Every time you click on **Add **, you update the data on both components.

In this case, these components are not independent. You can alter the data in any component that use the same action.

Now, there are certainly ways around this. You can have an id on the global store for each component. You can use an array. You can use a different namespace. But the fact is that global stores are best for global data. Encapsulation is not a primary goal.

How about context? Well, this is were things get a little more tricky. What does it mean to add a second component? Does that mean you add a second consumer? In this case, that would mean adding a second OtherComponent under the same provider. Or do you add a second CounterProvider?

Why not both? First, add another provider. Then in the existing provider, add a second instance of OtherComponent.

import React from 'react';
import StateCounter from '../StateCounter/StateCounter';
import ReduxCounter from '../ReduxCounter/ReduxCounter';
import CounterProvider from '../ContextCounter/CounterProvider';
import OtherComponent from '../ContextCounter/OtherComponent';

export default function Counter() {
  return(
    <div>
      <StateCounter />
      <ReduxCounter />
      <CounterProvider>
        <OtherComponent />
      </CounterProvider>
      <CounterProvider>
        <OtherComponent />
        <OtherComponent />
      </CounterProvider>
    </div>
  )
}

Let’s see what happens.

When you click on the first component, the counter iterates, but the other counters do not. But when you click on the second context counter, it will change the third counter. Similarily, if you click on the third counter, it will change the second.

This means that context is somewhere in the middle. It can be encapsulated when you want it to be, and it can be global when you want it to be.

Now, this claim comes with a big caveat. The ability to share date or keep it private all depends on how you order the components. Anything in the same hierarchy as the provider will share data, anything outside will not. This means that you may have to move providers up or down depending on how you want to use the consumers.

Still, at this point, you can finish filling out the chart. Local state is well encapsulated and context can be. They each get a check. Global state is not well encapsulated to it does not get credit.

Availability

That just leaves one more area of concern: availability. The beauty part is that you don’t need to write any more code. Sometimes you want your data to be easy to access.

Consider a shopping cart. Pushing users to a sale is the most important thing your code can do. Consequently, nearly every part of your app will potentially need to know about the cart (how much is in it, the total, etc). This is data you want to make easily accessible.

In this case, you can use what you learned above. Local state, for example, could work, but you would have to store it so far up the component hierarchy that you’d be passing it down a lot of props.

Global state, on the other hand, was designed specifically for this. It’s an easy thumbs up .

Context as you saw above can either be global (at least depending on where you put it in the hierarchy) or encapsulated. Go ahead and give it credit.

The final comparison looks like this:

One thing that’s not reflected in this chart is the large collection of third-party code that ties in with a global store. If you want to use observables with your store, then you’re better off using redux-observables than trying to roll your own solution.

Building and Refactoring

You may look at this table and think that local state or context are somehow better. That’s not true. The best solution is the one that works best for your data and your app.

Still, there are a few common patterns that may help the decision making process.

First, always start with local state. As you saw above, local state is the easiest to build and prototype. When you start with local state you don’t have to deal with all the complexities of setting up a Provider or writing actions and reducers. Even if you plan on moving to a more widely available state management, local state is a great way prove your concept.

Next, use context if you want to maximize reusability and local state is not sufficient. After you go through the hard work of building a component, you’ll want to reuse it as much as possible. That may even mean reusing it across different project and teams.

By choosing context, you reduce the number of additional dependencies your independent component will need. In other words, it will be much easier to drop the new component into an existing project. The component will be internally independent. You gain all the advantages of widely available components while avoiding extra code.

Finally, don’t worry about the rules. If your team uses global state and has a well defined structure in place, you’ll do more harm by using a different solution. Internal consistency and clear communication should always be the first priority. If your new component breaks the pattern, you may save yourself a few minutes of coding, but your teammates will lose their ability to quickly skim and understand a component. New patterns aren’t bad, but they should add value.

Each of these solutions exist for a reason. Your job is to find the fit that makes your code easiest to build and maintain.

GraphQL is all the rage, but what exactly is it? In this in-depth walkthrough, we take a look at what GraphQL is, how to use it, and why you should use it.

Learn why GraphQL is all the rage! We’ll walk through the implementation of a schema for a popcorn company’s API, learning about types, queries, and mutations as we go.

I love popcorn.

As I’m writing this, I’m snacking on a bowl of multi-colored kernels, lightly seasoned with Flavacol (hint: this is the movie theater’s secret) and I’m feeling a twinge of hubris in my eye…

“I should start a popcorn company…”

Well, that’s a terrible idea, Ryan! But, for the sake of this tutorial, it’s the best idea you’ve had all week. In this tutorial, we’re going to learn how to create a simple, GraphQL-based API for a popcorn company.

All we need is a name. Seeing as how I’m quite fond of sarcasm, how about…

Sarcastic Kernels: The Popcorn You Love to Hate to Eat™

Perfect! And like any self-respecting nerd, it only seems right to start writing some code before we come up with a sound business plan. To get started, let’s wrap our heads around what GraphQL is and how it compares to REST.

What is GraphQL?

GraphQL is a client-side query language coupled with a pattern — formally known as a “schema” — for organizing the creation, reading, updating, and deleting of data in your application (yeah, that CRUD).

We say “application”, here, and not database because GraphQL is data-source-agnostic, meaning it doesn’t care where your data lives.

From the outside looking in, GraphQL can seem quite scary. Does the “Graph” part mean I have to learn about graph databases? Does the QL (query language) mean I have to learn I have to learn a brand new programming language?!

Not quite. To calm your nerves, the harsh truth is: GraphQL is just a dressed up GET or POST request.

Wait, what?!

Yep. While GraphQL as a whole does introduce some new concepts for organizing and interacting with your data, behind the curtain, GraphQL still relies on a good ol’ fashioned HTTP request to do its magic.

Rethinking REST

Where GraphQL separates from a more familiar REST API is due to its flexibility. With REST, done properly, endpoints are typically designed from the perspective of a resource, or, a type of data in our application.

For example, a GET request to /api/v1/flavors would be expected to send us back a response that looks something like this:

[
  {
   "id": 1,
    "name": "The Lazy Person's Movie Theater",
    "description": "That elusive flavor that you begrudgingly carted yourself to the theater for, now in the comfort of your own home, you slob!"
  }, {
    "id": 2,
    "name": "What's Wrong With You Caramel",
    "description": "You're a crazy person that likes sweet popcorn. Congratulations."
  }, {
    "id": 3,
    "name": "Gnarly Chili Lime",
    "description": "The kind of popcorn you make when you need a good smack in the face."}
]

Now, there’s nothing terribly wrong with this, but let’s consider our UI, or rather, how we intend to consume this data.

If we wanted to display a simple list UI where all we had were the kinds of popcorn that are available (and nothing else), we might end up with a design like the one below.

And… we’d be in a bit of a jam. We can choose not to use the description field, sure, but are we just going to sit around and act like we didn’t send that to the client as well?!

Haha! What do you think this is? Fantasy land? Of course we are! And when someone asks “why is the app so slow for users?” in a few months, we’ll just say “be right back, going to grab a coffee!” and then split town, never to be seen again.

It’s not entirely our fault, though. To be fair, REST is the data-fetching equivalent of going to a restaurant and being asked “What do you want? We’ll give you what we have.”

Jokes aside, in a real application this can be problematic. For example, we may display different traits for each kind of popcorn like pricing information, brand details, or dietary restrictions (“vegan popcorn!”). Rigid REST endpoints make delivering specific traits based on context a headache, leading to unnecessary performance overhead and frustration.

How GraphQL improves REST

On the surface, this may seem like a small problem. “Who cares if we’re sending unnecessary data to the client?” Well, let’s add some context. GraphQL was invented at Facebook. Facebook serves millions of requests per second.

Translation? Every optimization counts.

Instead of saying “here’s what’s available,” GraphQL inverts the problem and asks “what do you need?”

We can get back a response from GraphQL specific to the context where we’re consuming data without having to add a one-off endpoint, perform multiple requests, or write convoluted conditional code.

How does GraphQL work?

Like we hinted at above, at it’s core GraphQL relies on a simple GET or POST request for moving data to and from the client. Unpacking that, GraphQL has two types of requests when it comes to reading (the R in CRUD), creating, updating, and deleting (The CUD in CRUD): queries and mutations.

All of those queries and mutations are sent as either a GET or POST request to a GraphQL server at a URL like https://myapp.com/graphql. More on that below.

Understanding Queries

Queries are what you’d expect: a request for some data. We have the UI, and we need to fill it with data, so we make a query to the server. With a traditional REST API, our query would come in the form of a GET request. With GraphQL, we introduce a new syntax for requesting data:

{
  flavors {
    name
  }
}

Wait, you mean JSON? Or is that a JavaScript object?

Neither.

The “QL” part of GraphQL stands for query language. Quite literally, this is a brand new language for writing data queries. That sounds more complicated than it is. Let’s break down the above query.

{
  // The fields we want to query are written here.
}

All queries start from the “root query” and are known as fields. To save yourself a headache: it’s best to refer to this as “query fields in my schema.” Don’t fret, we’ll learn more about how those are defined in a bit. Here, we ask to query the flavors field on the root query.

{
  flavors {
    // The sub-fields we want for each flavor are written here.
  }
}

When querying a field, we also need to specify the sub-fields we want for each object in the response (even if we expect just a single object to be returned).

{
  flavors {
    name
  }
}

The end result? When we send this query to a GraphQL server, we get back a neat, tidy response like this:

{
  "data": {
    "flavors": [
      { "name": "The Lazy Person's Movie Theater" },
      { "name": "What's Wrong With You Caramel" },
      { "name": "Gnarly Chili Lime" }
    ]
  }
}

Neat, right? To make this clear, if we ran the following query on another page:

{
  flavors {
    id
    name
    description
  }
}

we’d get back a response like this:

{
  "data": {
    "flavors": [
      { "id": 1, "name": "The Lazy Person's Movie Theater", description: "That elusive flavor that you begrudgingly carted yourself to the theater for, now in the comfort of your own home, you slob!" },
      { "id": 2, "name": "What's Wrong With You Caramel", description: "You're a crazy person that likes sweet popcorn. Congratulations." },
      { "id": 3, "name": "Gnarly Chili Lime", description: "A friend told me this would taste good. It didn't. It burned my kernels. I haven't had the heart to tell him." }
    ]
  }
}

Super powerful! Same endpoint, with a different response tailored to the context.

If we wanted to get a single flavor, GraphQL queries accept arguments, too:

{
  flavors(id: "1") {
    id
    name
    description
  }
}

Here, we’ve hardcoded the specific id of a flavor we want to query, but we can also have a dynamic id:

query getFlavor($id: ID) {
  flavors(id: $id) {
    id
    name
    description
  }
}

On the first line, we give our query a name (this is arbitrary; we could replace getFlavor with pizza and this would still work) and define the variables that query expects. Here, we expect a possible variable id to be passed as an ID scalar type (more on these below).

Regardless of using a static or dynamic id to make our request, here’s the response we can expect:

{
  "data": {
    "flavors": [
      { "id": 1, "name": "The Lazy Person's Movie Theater", description: "That elusive flavor that you begrudgingly carted yourself to the theater for, now in the comfort of your own home, you slob!" }
    ]
  }
}

Nice! Hopefully your hamster is starting to spin. This is already great, but where GraphQL really starts to shine is with nested fields. Let’s assume we had another field in our schema called nutrition that told us just how unhealthy our sarcastic kernels are:

{
  flavors {
    id
    name
    nutrition {
      calories
      fat
      sodium
    }
  }
}

What this may look like is us having a nested nutrition object on each of our flavors. Not quite! With GraphQL, we can combine separate but related data sources into a single query, getting a response that gives us the benefit of nested data without having to denormalize everything in the database:

{
  "data": {
    "flavors": [
      {
        "id": 1,
        "name": "The Lazy Person's Movie Theater",
        "nutrition": {
          "calories": 500,
          "fat": 12,
          "sodium": 1000
        }
      },
      ...
    ]
  }
}

That’s a serious boon on productivity. But what about updating data, does GraphQL give the same advantages?

Understanding Mutations

Where queries fetch data, mutations are responsible for making changes to data. Alternatively, too, mutations can be used for a generic RPC (remote procedure call) for miscellaneous tasks like sending a user’s data to a third-party API.

mutation updateFlavor($id: ID!, $name: String, $description: String) {
  updateFlavor(id: $id, name: $name, description: $description) {
    id
    name
    description
  }
}

Mutations rely on a similar syntax to queries. Here, we define a mutation updateFlavor with some variables: id, name, and description. Just like with our queries, we “wrap” a mutation field (defined on a similar root mutation) using the keyword mutation, followed by a name describing the mutation and a set of variables to pass along.

Those variables include what we’re trying to change or mutate. Notice, too, that after executing a mutation, we can ask for some fields back.

In this case, we want to get back the id, name, and description after they’ve been mutated. This helps with things like optimistic UI, negating the need for a request following the update.

Writing a schema and attaching it to a GraphQL server

So far, what we’ve been looking at is how GraphQL functions on the client. This is how we make requests, but how do we respond to them?

The GraphQL server

In order to make a GraphQL request, we need to have a GraphQL server to send it to. A GraphQL server is a regular ol’ HTTP server (if you’re a JavaScripter, think Express or Hapi) with a GraphQL schema attached to it.

import express from 'express'
import graphqlHTTP from 'express-graphql'
import schema from './schema'

const app = express()

app.use('/graphql', graphqlHTTP({
  schema: schema,
  graphiql: true
}))

app.listen(4000)

By “attached,” we mean that requests received by that server are passed through the schema and then back to the client, kind of like an air filter in your house.

The “filtering” process that takes place is relative to the query or mutation you send from the client. Both queries and mutations are resolved using functions associated with the fields defined on our root query or root mutation in our schema.

Above, we can see a mock HTTP server being created with the JavaScript library Express. Utilizing the graphqlHTTP function from the express-graphql package by Facebook, we “attach” our schema (here, assumed to be defined in another file) and start our server on port 4000 (i.e., http://localhost:4000/graphql is where we’ll send requests from the client).

Types and resolvers

With a running server, we need to define the schema that we attach to it.

Recall that earlier, we talked about defining fields on either a root query or root mutation.

import gql from 'graphql-tag'
import mongodb from '/path/to/mongodb’ // For example. Assuming `mongodb` gives us a MongoDB connection.

const schema = {
  typeDefs: gql`
    type Nutrition {
      flavorId: ID
      calories: Int
      fat: Int
      sodium: Int
    }

    type Flavor {
      id: ID
      name: String
      description: String
      nutrition: Nutrition
    }

    type Query {
      flavors(id: ID): [Flavor]
    }

    type Mutation {
      updateFlavor(id: ID!, name: String, description: String): Flavor
    }
  `,
  resolvers: {
    Query: {
      flavors: (parent, args) => {
        // Assuming args equals an object like { id: '1' }
        return mongodb.collection('flavors').find(args).toArray()
      },
    },
    Mutation: {
      updateFlavor: (parent, args) => {
        // Assuming args equals an object like { id: '1', name: 'Movie Theater Clone', description: 'Bring the movie theater taste home!' }

        // Perform the update.
        mongodb.collection('flavors').update(args)

        // Return the flavor after the update.
        return mongodb.collection('flavors').findOne(args.id)
      },
    },
    Flavor: {
      nutrition: (parent) => {
        return mongodb.collection('nutrition').findOne({
          flavorId: parent.id,
        })
      }
    },
  },
}

export default schema

When it comes to defining fields in a GraphQL schema, there are two parts: typeDefs and resolvers.

typeDefs contain the type definitions for the data in our application. For example, earlier we talked about retrieving a list of flavors. In order to do that, we need to do three things:

1. Tell our schema what a flavor’s data looks like (above, by defining the type Flavor type).
2. Define a field on the root type Query field (above, the flavors property on the type Query value).
3. Define a resolver function on the resolvers.Query object corresponding to the field we defined on the root type Query field.

Focusing on the typeDefs, this is where we tell our schema about the shape of our data. In other words, we tell GraphQL about the different properties a piece of data might contain.

type Flavor {
  id: ID
  name: String
  description: String
  nutrition: Nutrition
}

The type Flavor definition says that “a flavor can contain an id as an ID, a name as a String, a description as a String, and nutrition as Nutrition.”

For that last one, nutrition, we pass the name of another type defined in our typeDefs. Here, type Nutrition describes how nutrition data is shaped in our application.

Notice that we’re not saying in our database. A database is assumed in the example above, but your data can come from any data source. Even a third-party API or a static file!

Just like we did for type Flavor we specify the names of the field a piece of nutrition data will have, assigning what GraphQL refers to as scalar types to each property. As of writing, GraphQL recognizes five built-in scalar types:

• Int: A signed 32‐bit integer.
• Float: A signed double-precision floating-point value.
• String: A UTF‐8 character sequence.
• Boolean: true or false.
• ID: A unique identifier, often used to refetch an object or as the key for a cache. The ID type is serialized in the same way as a String; however, defining it as an ID signifies that it is not intended to be human‐readable.

In addition to these scalar types, we can also assign custom types to a property like we did with the Nutrition type on the nutrition property of the type Flavor above.

type Query {
  flavors(id: ID): [Flavor]
}

On our root type Query (the “root query” we talked about earlier), we define the name of a field that can be queried. When defining that field, too, we specify any arguments we might expect along with the type of data we expect to be returned.

In this example, we expect a possible id argument to be passed as an ID scalar type and expect an array of objects resembling the Flavor type in response.

Wiring up a query resolver

With our flavors field defined on our root type Query, next, we define what’s known as a resolver function.

This is where GraphQL more or less “stops.” If we look in the resolvers object of our schema file and then in the Query object nested under that, we can see a property flavors assigned to a function. This flavors here is the resolver function for the flavors field defined on our root type Query.

typeDefs: gql`…`,
resolvers: {
  Query: {
    flavors: (parent, args) => {
      // Assuming args equals an object like { id: '1' }
      return mongodb.collection('flavors').find(args).toArray()
    },
  },
  …
},

This resolver function takes a few different arguments: the parent query if one exists, the args passed to the query if any exist, and a missing context argument which gives us miscellaneous “context” data (e.g., the current user if we provide them when our server starts).

Inside our resolver, we do whatever we need to do to resolve the query. This is where GraphQL “quits caring” and leaves retrieving and returning data up to us. Again, this could be a call to a database, an API, a static file…anything.

While GraphQL doesn’t care where our data comes from, it _does _ care about what we return. We can return a JSON object, an array of JSON objects, or a Promise (which GraphQL will resolve for us).

Here, we use a mock call to a MongoDB database collection called flavors, passing in our args (if any exist) to a .find() call and returning what it finds as an array.

Resolving nested fields

What may not be clear above is how our nested nutrition data is resolved. Remember: we’re not actually storing the nutrition data on each flavor but assume it lives in another database collection/table.

While we did tell GraphQL that our type Flavor might include some nutrition data in the shape of the type Nutrition, we didn’t explain how to actually resolve that data. Again: the nutrition data for a flavor is assumed to be in a different collection than the flavor data.

  typeDefs: gql`
    type Nutrition {
      flavorId: ID
      calories: Int
      fat: Int
      sodium: Int
    }

    type Flavor {
      […]
      nutrition: Nutrition
    }

    type Query {…}

    type Mutation {…}
  `,
  resolvers: {
    Query: {
      flavors: (parent, args) => {…},
    },
    Mutation: {…},
    Flavor: {
      nutrition: (parent) => {
        return mongodb.collection('nutrition').findOne({
          flavorId: parent.id,
        })
      }
    },
  },

If we look close at the resolvers object on our schema, notice that we have Query, Mutation, and Flavor. These correspond to the types we defined in the typeDefs above.

Looking at the Flavor object, we can see the field nutrition being defined as a resolver function. What’s unique about this is that we’re defining this _on the Flavor type directly. In other words, we’re saying “this is how we want you to resolve the nutrition field for any queries utilizing the type Flavor.”

Inside, we do a familiar MongoDB query, but notice that we utilize the parent argument passed to the resolver function. The parent here is each iteration of the flavors field. For example, if we ask for all flavors at once like this:

{
  flavors {
    id
    name
    nutrition {
      calories
    }
  }
}

For each flavor returned by flavors, we’d pass it through the nutrition resolver defined on resolvers.Flavor as parent. If we look close, we can see that we utilize the parent.id field, referring to the id of the flavor we’re currently iterating (looping) over.

We pass that parent.id to our database query, matching it to a (presumed) flavorId property on each nutrition item.

Wiring up mutations

Conveniently, our knowledge of wiring up queries maps over perfectly to mutations. In fact, the process is nearly identical. If we take a look at our root type Mutation, we can see that we define a field updateFlavor accepting the arguments we specified on the client:

type Mutation {
  updateFlavor(id: ID!, name: String, description: String): Flavor
}

Here, we’re saying “we expect the updateFlavor mutation to accept a possible id as an ID (the ! tells GraphQL that this is required), name as a String, and description as a String.” Additionally, once our mutation completes, we expect some data in return resembling the Flavor type (i.e., containing an id, name, description, and/or nutrition).

{
  typeDefs: gql`…`,
  resolvers: {
    Mutation: {
      updateFlavor: (parent, args) => {
        // Assuming args equals an object like { id: '1', name: 'Movie Theater Clone', description: 'Bring the movie theater taste home!' }

        // Perform the update.
        mongodb.collection('flavors').update(
          { id: args.id },
          {
            $set: {
              ...args,
            },
          },
        )

        // Return the flavor after the update.
        return mongodb.collection('flavors').findOne(args.id)
      },
    },
  },
}

Inside our resolver function for the updateFlavor mutation, we do what you might expect: interact with our database to change or update the flavor.

Notice that immediately after we perform the update, we make a call back to our database to find the same flavor again and return it from our resolver. Why?

Remember: on the client, we expect a return value after our mutation completes. In this example, we expect the flavor we just updated to be returned.

Couldn’t we just return the args object? Yep! We could. The reason we choose not to in this case is that we want to be 100% certain our database update succeeded. If we go fetch the data again and see that it’s changed: all is well!

Why would I want to use GraphQL?

Though it may not look like much, at this point we have a functioning—albeit simple—GraphQL API up and running!

As with any new tech, though, what may not be abundantly clear is why you’d even want to use this. To be fair, this is a lot of moving parts. Why shouldn’t we just stick to REST or talking to the database directly?

You want to reduce the number of requests made from the client

Where a lot of apps get bogged down is in the number, frequency, and complexity of HTTP requests. While GraphQL doesn’t completely eliminate requests, utilized properly, it can significantly reduce the number of requests you make from the client (in many cases, down to just one).

Whether you’re running an app with tons of users or an app with lots of data (e.g., an app for handling medical records), using GraphQL will definitely speed up client performance.

You want to avoid denormalizing data just to compensate the UI

In applications with a lot of relational data, the “denormalization trap” can be quite common. While this works, it’s by no means ideal and can slow things down unnecessarily. With GraphQL and nested queries, the need to denormalize your data is significantly reduced.

You have multiple data sources to talk to from different apps

This problem can be solved in part with a traditional REST API, but it still leaves a problem: consistent querying from the client. Assuming you have a product with a web app, iOS app, Android app, and developer API, it’s likely that you’ll have to rig up querying for each of those platforms.

This translates to developing knowledge of multiple client implementations for HTTP requests, an inconsistent means for performing queries, and messy, platform-specific endpoints in your API (don’t you “holier than thou” me, friend, you know you’ve done this before!).

Is GraphQL perfect? Should I ditch my REST API today and switch to it?

Of course not! Nothing is perfect.

Where GraphQL most obviously falls short is its complexity. Writing a GraphQL schema introduces a lot of mandatory steps in order to wire up your data. As you’re learning, this can be frustrating because what is missing from your schema may not always be obvious and errors on the client and server may be unhelpful.

Further, consuming GraphQL on the client is not standardized beyond the GraphQL query language. Different libraries exists for this, the most popular being Apollo and Relay, each with their own idiosyncrasies.

GraphQL is also just a specification. Packages like graphql (used internally by the express-graphql package from our examples) are just an implementation of that specification. In other words, different implementations for different programming languages may interpret the specification differently. This can lead to problems for you and your team if you use multiple languages across projects.

GraphQL is an impressive step forward in handling data, though. It’s by no means a silver bullet, but it’s certainly worth experimenting with. A great way to get started is to think about a particularly messy part of your data process and try to implement it using GraphQL.

This is the great news: GraphQL can be implemented incrementally. You don’t have to go all in to start taking advantage of it. This is a great way to get buy in from your team and stakeholders and create an opportunity to get your hands dirty.

Keep in mind: GraphQL is ultimately just a tool for doing a job. It’s not “killing” anything. That said, it’s worth familiarizing yourself with it and starting to apply in your apps. Anywhere you struggle with performance overhead or complex UIs like data dashboards, news feeds, and user profiles is a great place to get started.

Happy coding!

In recent years, JavaScript engines have improved on all fronts. The performance of JavaScript has reached a level where it can easily compete with programming languages that have traditionally been considered more appropriate for high-performance computing. This is not only true for V8, the JavaScript engine inside of Chrome and Node.js, but for all major JavaScript engines, including ChakraCore, the engine inside of Edge, JavaScriptCore, the engine inside of Safari, and SpiderMonkey, the engine inside of Firefox.

Not only did the peak performance improve, but engines also managed to deliver more consistent and predictable levels of performance. Given that JavaScript is a highly dynamic language, all of this performance is based on choosing the right heuristics in the engine. JavaScript engines use techniques like speculative optimization and inline caching to speed up execution of the likely path.

But heuristics can also easily work against you, and it helps to be aware of them. So today I’m gonna share some background on two subtle performance pitfalls in the V8 JavaScript engine. Being aware of these potential pitfalls might help you to resolve issues if you get bitten by these more easily.

Optimization limit

The compilers built into V8 - both the TurboFan optimizing compiler and the Ignition bytecode generator - are so-called method JITs, meaning the unit of compilation is always a method, aka a function in JavaScript speak. The optimizing compiler is able to include the bodies of other methods when it finds hot call sites and sees potential for further optimizations via doing this, which is commonly referred to as inlining. Contrast this with other runtimes that use so-called tracing JITs - LuaJIT and PyPy are popular examples here - where the unit of optimization is an arbitrary sequence of consecutive bytecodes that have been executed repeatedly previously.

In both method JITs as well as tracing JITs, there’s always an upper limit on the size of the input that the optimizer can deal with. For method JITs this limit is naturally defined by the size of the function itself. TurboFan also has such a limit, which is currently 60 KiB of bytecodes (the kMaxBytecodeSizeForOpt constant in runtime-profiler.cc). That means if you have functions whose generated bytecode is bigger than 60 KiB, they will not ever be optimized by TurboFan, even if they are considered hot (i.e. invoked very often).

Let’s consider an example, using eval to dynamically generate functions. (We intentionally don’t use the Function constructor here since that doesn’t allow us to put a name onto the function that is syntactically recognizable by the JavaScript engine, and that would make the investigations really challenging).

function generate(n) {
  let s = "(function add" + n + "(x) { return 0";
  for (let i = 0; i < n; ++i) {
    s += "+x";
  }
  s += "; })";
  return eval(s);
}

This generate function constructs a new JavaScript function object which adds up the parameter passed to it n times. So when you call generate(10) you’ll get a function object that looks like this:

function add10(x) {
  return 0+x+x+x+x+x+x+x+x+x+x;
}

Now let’s take a concrete example of the above and run it in Node.js:

// add10.js

function generate(n) {
  let s = "(function add" + n + "(x) { return 0";
  for (let i = 0; i < n; ++i) {
    s += "+x";
  }
  s += "; })";
  return eval(s);
}

const add10 = generate(10);
add10();

Looking at the generated bytecode for the add10 function, using the --print-bytecode flag with the node shell, we see something like this (output is from Node v10.15):

$ node --print-bytecode add10.js
…
[generated bytecode for function: add10]
Parameter count 2
Frame size 8
   19 E> 0x279c523204b2 @    0 : a5                StackCheck
   26 S> 0x279c523204b3 @    1 : 0b                LdaZero
         0x279c523204b4 @    2 : 26 fb             Star r0
         0x279c523204b6 @    4 : 25 02             Ldar a0
   34 E> 0x279c523204b8 @    6 : 34 fb 00          Add r0, [0]
         0x279c523204bb @    9 : 26 fb             Star r0
         0x279c523204bd @   11 : 25 02             Ldar a0
   36 E> 0x279c523204bf @   13 : 34 fb 01          Add r0, [1]
         0x279c523204c2 @   16 : 26 fb             Star r0
         0x279c523204c4 @   18 : 25 02             Ldar a0
   38 E> 0x279c523204c6 @   20 : 34 fb 02          Add r0, [2]
         0x279c523204c9 @   23 : 26 fb             Star r0
         0x279c523204cb @   25 : 25 02             Ldar a0
   40 E> 0x279c523204cd @   27 : 34 fb 03          Add r0, [3]
         0x279c523204d0 @   30 : 26 fb             Star r0
         0x279c523204d2 @   32 : 25 02             Ldar a0
   42 E> 0x279c523204d4 @   34 : 34 fb 04          Add r0, [4]
         0x279c523204d7 @   37 : 26 fb             Star r0
         0x279c523204d9 @   39 : 25 02             Ldar a0
   44 E> 0x279c523204db @   41 : 34 fb 05          Add r0, [5]
         0x279c523204de @   44 : 26 fb             Star r0
         0x279c523204e0 @   46 : 25 02             Ldar a0
   46 E> 0x279c523204e2 @   48 : 34 fb 06          Add r0, [6]
         0x279c523204e5 @   51 : 26 fb             Star r0
         0x279c523204e7 @   53 : 25 02             Ldar a0
   48 E> 0x279c523204e9 @   55 : 34 fb 07          Add r0, [7]
         0x279c523204ec @   58 : 26 fb             Star r0
         0x279c523204ee @   60 : 25 02             Ldar a0
   50 E> 0x279c523204f0 @   62 : 34 fb 08          Add r0, [8]
         0x279c523204f3 @   65 : 26 fb             Star r0
         0x279c523204f5 @   67 : 25 02             Ldar a0
   52 E> 0x279c523204f7 @   69 : 34 fb 09          Add r0, [9]
   55 S> 0x279c523204fa @   72 : a9                Return
Constant pool (size = 0)
Handler Table (size = 0)
…

Since Node.js runs a lot of its own JavaScript, you might get pages of output here. Search for the phrase [generated bytecode for function: add10] where the following bytecode dump contains a sequence of 10 Adds with Star and Ldar bytecodes in between. There’s a blog post on Understanding V8’s Bytecode from my former colleague Franziska Hinkelmann if you’re curious to learn more about this topic.

Removing the irrelevant parts we see this concrete bytecode output:

  0 : a5                StackCheck
  1 : 0b                LdaZero
  2 : 26 fb             Star r0
  4 : 25 02             Ldar a0
  6 : 34 fb 00          Add r0, [0]
  9 : 26 fb             Star r0
 11 : 25 02             Ldar a0
 13 : 34 fb 01          Add r0, [1]
 16 : 26 fb             Star r0
 18 : 25 02             Ldar a0
 20 : 34 fb 02          Add r0, [2]
 23 : 26 fb             Star r0
 25 : 25 02             Ldar a0
 27 : 34 fb 03          Add r0, [3]
 30 : 26 fb             Star r0
 32 : 25 02             Ldar a0
 34 : 34 fb 04          Add r0, [4]
 37 : 26 fb             Star r0
 39 : 25 02             Ldar a0
 41 : 34 fb 05          Add r0, [5]
 44 : 26 fb             Star r0
 46 : 25 02             Ldar a0
 48 : 34 fb 06          Add r0, [6]
 51 : 26 fb             Star r0
 53 : 25 02             Ldar a0
 55 : 34 fb 07          Add r0, [7]
 58 : 26 fb             Star r0
 60 : 25 02             Ldar a0
 62 : 34 fb 08          Add r0, [8]
 65 : 26 fb             Star r0
 67 : 25 02             Ldar a0
 69 : 34 fb 09          Add r0, [9]
 72 : a9                Return

Looking at the output above, we see that the generated bytecode for add10 is 73 bytes in size (72 is the offset of the Return bytecode and that instruction is 1 byte in size). Given what we learned before about the 60 KiB limit in TurboFan, this function should be easily optimizable by V8. Let’s try and see what happens if we put that function in a hot loop.

// add10-optimized.js

function generate(n) {
  let s = "(function add" + n + "(x) { return 0";
  for (let i = 0; i < n; ++i) {
    s += "+x";
  }
  s += "; })";
  return eval(s);
}

const add10 = generate(10);

let result = 0;
for (let i = 0; i < 5 * 1000; ++i) {
  result += add10(i);
}

Running this snippet inside of Node, passing the --trace-opt command line flag, we see something similar to the following output:

$ node --trace-opt add10-optimized.js
[marking 0x32fc83347751 <JSFunction add10 (sfi = 0x32fcde157b91)> for optimized recompilation, reason: small function, ICs with typeinfo: 10/10 (100%), generic ICs: 0/10 (0%)]
[compiling method 0x32fc83347751 <JSFunction add10 (sfi = 0x32fcde157b91)> using TurboFan]
[optimizing 0x32fc83347751 <JSFunction add10 (sfi = 0x32fcde157b91)> - took 0.823, 0.514, 0.016 ms]
[completed optimizing 0x32fc83347751 <JSFunction add10 (sfi = 0x32fcde157b91)>]

You may see messages about other functions being optimized or marked for optimized recompilation as well, just ignore those. The interesting line is the last line, which says that add10 was successfully optimized by TurboFan. This matches our expectations. Note that every modern JavaScript engine gives you this behavior.

// add10000.js

function generate(n) {
  let s = "(function add" + n + "(x) { return 0";
  for (let i = 0; i < n; ++i) {
    s += "+x";
  }
  s += "; })";
  return eval(s);
}

const add10000 = generate(10 * 1000);

let result = 0;
for (let i = 0; i < 5 * 1000; ++i) {
  result += add10000(i);
}

Let’s see what happens if we increase the size of the generated function such that the bytecode for the function exceeds the limit in TurboFan. We choose a rather arbitrary number of 10,000 additions for this example. Running…

node --trace-opt add10000.js

Doesn’t print anything to the console, meaning that add10000 is not even considered for optimization by TurboFan (it’s a bit unfortunate that V8 doesn’t say anything in this case, but just silently continues). In fact TurboFan is not even involved here! Instead, the so-called RuntimeProfiler that profiles bytecode execution immediately decides that the function is too big to be considered for optimization and disables optimization of the function (in the RuntimeProfiler::ShouldOptimize method).

Running node again with --print-bytecode we see the culprit:

$ node --print-bytecode add10000.js
…
[generated bytecode for function: add10000]
Parameter count 2
Frame size 8
   18 E> 0x2e785b7dcf22 @    0 : a0                StackCheck
   24 S> 0x2e785b7dcf23 @    1 : 0b                LdaZero
         0x2e785b7dcf24 @    2 : 26 fb             Star r0
         0x2e785b7dcf26 @    4 : 25 02             Ldar a0
   32 E> 0x2e785b7dcf28 @    6 : 32 fb 00          Add r0, [0]
         0x2e785b7dcf2b @    9 : 26 fb             Star r0
         0x2e785b7dcf2d @   11 : 25 02             Ldar a0
   34 E> 0x2e785b7dcf2f @   13 : 32 fb 01          Add r0, [1]
…
20026 E> 0x2e785b7f52aa @ 99208 : 00 32 fb ff 0d 27 Add.Wide r0, [9997]
         0x2e785b7f52b0 @ 99214 : 26 fb             Star r0
         0x2e785b7f52b2 @ 99216 : 25 02             Ldar a0
20028 E> 0x2e785b7f52b4 @ 99218 : 00 32 fb ff 0e 27 Add.Wide r0, [9998]
         0x2e785b7f52ba @ 99224 : 26 fb             Star r0
         0x2e785b7f52bc @ 99226 : 25 02             Ldar a0
20030 E> 0x2e785b7f52be @ 99228 : 00 32 fb ff 0f 27 Add.Wide r0, [9999]
20033 S> 0x2e785b7f52c4 @ 99234 : a4                Return
Constant pool (size = 0)
Handler Table (size = 0)

The function add10000 generated 99,235 bytes of bytecode instructions. That certainly exceeds the 60 KiB limit imposed by the RuntimeProfiler.

Background

There’s an ongoing discussion in v8:8598 regarding the issue of this optimization limit. In general, there’ll always be a limit of some form, since it’s all about trade-offs in the engine. In V8’s case, the limit allows TurboFan to store various counts (i.e. the number of inputs and outputs of entities in the intermediate representation) as 16-bit integers vs. 32-bit integers. Other engines and languages have similar limits. For example, in Java the 64 KiB method limit is even part of the JVM bytecode specification.

Take-away from this section

Make sure to split large functions into smaller building blocks. This is generally good advice for maintainability, but also helps the JIT to properly optimize everything that’s relevant to your application. Smaller functions also generally play well together with the inlining machinery inside the JIT, and often reduce the cost of compilation and optimization. Imagine you have a bigger function that does 10 different tasks;ven if you only need 1 or 2 of those tasks, you still pay the cost of compiling everything that’s in the function (at least compiling to bytecode), plus since inlining heuristics also take into account the size of the inlinee, it’s a lot less likely that such a function is inlined at a hot call site.

It’s worth pointing out that normally you don’t run into the 60KiB limit when writing JavaScript, except in some really extreme cases. But tools generating JavaScript programmatically (i.e. parser generators) can hit this limit easily.

Double fields

The V8 engine uses a technique called pointer tagging to encode arbitrary JavaScript values. The trick here to realize that while pointers can be used to address any single byte in memory, this is not needed for JavaScript objects in memory, which usually are aligned to word boundaries (i.e. 4-byte aligned on 32-bit architectures and 8-byte aligned on 64-bit architectures). So the least significant bits in any valid object pointer are zero. V8 uses these bits to encode additional information. In particular it uses the two least significant bits to distinguish between three different kinds of values:

A Smi is a small integer in the 31-bit range, i.e. a value between -1,073,741,824 and 1,073,741,823¹, shifted up by one bit and padded with a 0 in the least significant bit. A HeapObject pointer is the address of an object in (managed) memory, with the two least significant bits set to 01. That means when V8 wants to get to the real address of the object it has to subtract one from the value. There’s also the WeakHeapObject, which has the least significant bits set to 11, and is essentially like HeapObject, except that the reference is treated weakly by the garbage collector.

For the purpose of this article we only care about Smi and HeapObject. Given that small integers are the most common number values in JavaScript programs, it was considered important to have an efficient value encoding for those. That’s why there’s the special Smi encoding: so that small integers can be stored efficiently, for example inside of objects.

const a = {
  x: 42,
  y: "Hello"
};

Just consider the object a above. It has two properties x and y, and x has a small integer value, whereas y has a String value. V8 is going to represent that object in memory like this:

Here a is represented as a JSObject with a Shape that holds the information about the properties of a, and the actual values of the properties are in the instance a (you can read more about shapes in our previous articles JavaScript engine fundamentals: Shapes and Inline Caches and JavaScript engine fundamentals: optimizing prototypes). Due to the value encoding mentioned above, the small integer 42 can be stored efficiently inside of the JSObject whereas the String value has to be represented as a separate entity in memory and pointed to by a HeapObject pointer.

Now for Strings that seems kind of obvious and makes sense to have them allocated as separate entities², but what about other number values for example? In JavaScript numbers are 64-bit double precision floating point values so they can encode a lot of values outside the 31-bit integer range. In the basic pointer tagging scheme used in V8, all numbers outside this range have to represented as separately allocated HeapObject entities.

const b = {
  x: 2 ** 32,
  y: 1.5
};

In the example of b above, we assign x an integer value 4,294,967,296, which is outside the 31-bit integer range, and y is assigned a number that is not even an integer at all. Neither of these values cannot use the efficient Smi encoding, and thus need separate heap-allocated entities, so-called HeapNumbers in V8 speak:

As you can imagine this would be quite costly when you think of applications that operate on data structures that mostly consist of numeric values outside the 31-bit signed integer range, i.e. points, vectors, etc, and often update the values of these properties. In such cases every update to these properties would have to allocate a new HeapNumber entity, since these entities are by their nature immutable (since they are passed around and thus can be referenced from many different places).

const c = {x: 1.1};
c.x = 1.2;
c.x = 1.3;

Running this simple snippet results in the creation of three HeapNumber objects, one for each of the different numbers 1.1, 1.2 and 1.3.

Always having to allocate new HeapNumbers can be quite costly specifically for numeric applications, as it causes a lot of traffic on the garbage collector and may lead to bad cache locality.

To mitigate this problem for applications that operate on lots of objects with numeric properties, V8 has a concept called field representation tracking, which tracks for each field in an object what the representation of the values were so far. Specifically V8 tracks whether a field (as described by a concrete shape) has always stored numbers so far, where we distinguish between states Smi and Double (the former meaning that only values in the small integer range were seen, the latter means that also other number values outside that range occurred).

Fields with Smi representation are already stored efficiently as described above, but the optimizing compiler TurboFan can use the field representation information to generate more efficient code (i.e. it doesn’t need to check whether the field contains an integer since that’s known from the representation on the shape). For Double fields a new MutableHeapNumber entity was introduced that looks like a HeapNumber, but is tied to the instance and can be updated in-place.

const c = {x: 1.1};
c.x = 1.2;
c.x = 1.3;

Looking at this example with the update again, we see that only a single storage for the double values need to be allocated and it’s just updated to the most recent value in-place:

Notice how the Property information now tags the property 'x' as a Double field.

This optimization is done by V8 under the hood, as a developer you don’t even notice that it’s going on. And even better, on 64-bit architectures V8 can store the double values inline into the instance itself, using a technique called double field unboxing. I’m not going to describe that here, since the details of that are quite complex. Take-away so far should be that this goes on automatically and usually does the right thing for you.

But as with any heuristic, there are trade-offs. You might already ask yourself how that work with the dynamic nature of JavaScript? Specifically since a field can hold any arbitrary value at any given point in time. Like what if you decide to store a string into c.x at some point?

c.x = "Hello";

This is perfectly valid in JavaScript, and V8 handles this by having a lattice of field representations like this:

For each field V8 will choose the best possible representation given the value that’s being stored, and it will progress through the lattice as necessary. Like in the case of storing a string into a field that currently has a Double representation tag, it’ll progress the field to Tagged representation, which essentially says anything. Ideally fields should have one of Smi, Double or HeapObject representation tags (highlighted in orange above), i.e. you should not mix numbers and other values, for best performance in most cases. You may already know about the concept of elements kinds in V8, which is very similar to the field representation mechanism. Elements kinds are concerned with array-indexed properties, whereas field representations are used for non-array indexed properties.

Mixed number and non-number fields

When you store both number and non-number values into a field, its field representation is going to be Tagged for sure, since that’s the only valid representation for fields that hold both number and non-number values. If this is really what you intended to do, good. But if you intended to operate on number values really, then mixing in non-number values, including undefined or null primitives is a bad idea. Performance-wise the JavaScript engine can skip a bunch of checks if it knows that a field has Smi or Double representation. Consider this simple example

function distance(p1, p2) {
  const dx = p1.x - p2.x;
  const dy = p1.y - p2.y;
  return Math.sqrt(dx * dx + dy * dy);
}

which computes the distance between two points p1 and p2. Without any additional information about the representation of the fields x and y, the engine would need to check that the results of the four property accesses p1.x, p2.x, p1.y, and p2.y all yields numbers (and handle the case where they are not).

Specifically don’t ever pre-initialize number fields with undefined or null, unless there’s a good reason why this would be necessary for your application. That being said, don’t do something like this

class Point {
  constructor() {
    this.x = null;
    this.y = null;
  }

  setX(x) { this.x = x; }
  setY(y) { this.y = y; }
}

i.e. use null as a sentinel value for number fields. Prefer actual number values, i.e. maybe just 0 (if you plan on using small integers) or 0.1 or even NaN (if you plan on using floats).

class Point {
  constructor() {
    this.x = 0.1;
    this.y = 0.1;
  }

  setX(x) { this.x = x; }
  setY(y) { this.y = y; }
}

Smi to Double migrations

Even if you pay attention to what was said above, there’s still another subtle problem that you might run into with number fields. For example the React team got bitten by this recently. The subtle issue here, without going into too much detail, is that Smi and Double field representation aren’t compatible: The former requires the number to be encoded as part of the value in the field, the latter requires the number to be stored in a dedicated MutableHeapNumber entity. So going from Smi to Double involves a process called instance migration, where instances that used to have the value stored as a Smi need to be migrated to have the field value stored in a MutableHeapNumber instead.

This process involves creating entirely new shapes for the instances, and lazily migrating instances from the old shape to the new shape as the engine hits the instances. (It would be too expensive to stop the world and find all the instances in managed memory at the time when the migration is kicked off). Since shapes are organized in trees (as explained in JavaScript engine fundamentals: Shapes and Inline Caches), this might involve the creation of new shape subtrees. Consider this example of two objects o1 and o2 with Smi fields x and y:

const o1 = {};
o1.x = 3;
o1.y = 4;

const o2 = {};
o2.x = 5;
o2.y = 6;

This creates appropriate shapes and transitions between these shapes as outlined below:

Now imagine we change the value of o2.x to value 5.5, which cannot be represented as a Smi like this:

o2.x = 5.5;

Now V8 has to create a whole new subtree starting from the shape that introduced the property x, and have o2 use the new shape. The instance o1 still points to the previous shape, which is now marked as deprecated.

As the diagram shows, the old shape subtree is essentially disconnected from the root shape, and instead a new shape subtree is put in place. All the shapes in the old subtree are marked as deprecated, and will eventually be garbage-collected once no more instances use them. In the meantime however there’s additional overhead essentially keeping two separate trees of metadata alive.

The next time o1 is seen by any IC (inline cache), it automatically migrates the instance away from the deprecated shape to its so-called migration target, which is the corresponding shape in the new subtree. For example, if you now perform a property access like o1.y it’ll automatically do this self-healing. You don’t even need to store to it; loading any property from it is sufficient. And this also forces o1.x into a Double field representation, so that afterwards o1 and o2 use the same shape again:

Now imagine what happens when this has to be done for thousands of objects with plenty of properties on them, which is kinda what happened in facebook/react#14365, except that in their case, it was some bad interaction with Object.preventExtensions() on top, where a bug in V8 causes it to miss the proper migration target somehow, and so V8 ended up allocating a new unique shape for each and every FiberNode instance in this application. The relevant issue v8:8538 is still under investigation.

The actionable advice for you here is to make sure that whenever you’re going to use a field to hold arbitrary number values (including values outside the small integer range), initialize this field with a non-small integer number. That way, you get the correct shapes from the beginning, and you avoid the migration cost (and troubles). One easy way to accomplish this is to store NaN initially³, which was also the fix for the React issue:

class Point {
  constructor(x, y) {
    this.x = NaN; this.x = x;
    this.y = NaN; this.y = y;
  }
}

But also like with all performance advice: Don’t do this blindly everywhere. A lot of your code is probably fine and unaffected performance-wise, even if objects undergo migrations sometimes.

Take-away from this section

Try to avoid mixing field values of different types, i.e. don’t mix numbers, strings, objects, other primitives, unless that’s what you intended to do. Specifically don’t pre-initialize number fields to null or undefined, but choose sensible default numbers (use NaN if in doubt). And try to initialize double fields (aka fields that are supposed to hold number values outside the small integer range) with double values outside the small integer range up-front, i.e. if in doubt put a NaN there first, and then store the actual initial value.

Conclusion

You should trust that the JavaScript engines usually do the right thing for you automatically. But as with everything, it’s good to be aware of some of the details, so in case you bump into problems, you’ll find your way around it, and sometimes giving the JavaScript engine an additional hint can be beneficial if the heuristics would otherwise fail badly.