Jack Hsu

The Reader monad and read-only context

Wed, 10 May 2017 00:00:00 -0400

In this two-part series, I want to take a functional approach to building React applications.

There will be mathematical theory sprinkled throughout the series, and hopefully by the end of it you will pick up some useful techniques!

Series contents:

Part 1 - Deconstructing the React Component
Part 2 - The Reader monad and read-only context

In the previous post, we looked at React through a more functional lens and rebuilt parts of the React component by using a View box that contains the computation Props -> Element.

There are two remaining React component API that I want to replicate: context and state. And in this post we will learn how to add a read-only context for our application using yet another box object.

So onwards and upwards!

Motivation behind context

Say that we have a copyright notice View as follows.

const copyrightNotice = View(({ author, year }) => (
  <p>© {author} {year}</p>
))

copyrightNotice.fold({ author: 'Bob McBob', year: 2017 }) // <p>© Bob McBob 2017</p>

In order for this View to be used in our application, we will need to pass the props author and year down from the root View.

As the number of props grow in the application, the more data we will have to manually pass down. Not only is this a tedious process, but it also means that the root View will need to take in every single prop as its own input just so they can be passed down to descendents.

But wait, there is a better way!

Meet the Reader

Let’s start by defining a new Reader box that contains a computation of the type Context -> a. What is a here? It really could be anything, but for now we can focus on computations that return a View.

The Reader defines one function runReader that takes in the value of the context, and returns the result of the computation.

const Reader = computation => ({
  runReader: ctx => computation(ctx)
})

Now, we can use the Reader like this.

const footer = Reader(({ author, year }) => 
  copyrightNotice.contramap(() => ({ author, year }))
)

footer
  .runReader({ year: 2017, author: 'Bob McBob' })
  .fold() // <p>© Bob McBob 2017</p>

Here, the Reader’s computation returns a View, which is our previously defined copyrightNotice, that has its input props contramapped from the Reader context. So, when we run the reader with a context value, we get the value (View) out of the box, which we can fold down to get the final element.

Take a moment to let that sink in.

Now, just like we did with View previously, we can make the Reader a pointed functor.

const Reader = computation => ({
  runReader: ctx => computation(ctx),
  map: f => Reader(ctx => f(computation(ctx)))
})

// Running this reader will just return `x`.
Reader.of = x => Reader(() => x)

Then, we can use map to wrap the notice with a <footer> tag.

import { map } from 'ramda'

// We have to map twice since the first map is over the Reader,
// and the second map is over the View.
const footer2 = footer.map(map(y => <footer>{y}</footer>))

footer2
 .runReader({ year: 2017, author: 'Bob McBob' })
 .fold() // <footer><p>© Bob McBob 2017</p></footer>

But what else can Reader hold?

Remember that the Reader contains the computation Context -> a, where a can be anything?

So far we’ve returned a View from the computation, but it can really be any value, even a function!

Let’s look at a function that changes the color of an element within a View.

const makeColor = ({ color }) => view => view.map(element => 
  <div style={{ color }}>{element}</div>
)

We can wrap this up in a Reader.

const withColor = Reader(makeColor)

To use this Reader, we need to run it with a context, then apply it to an element in the View.

// Get the color function inside withColor
const colorize = withColor.runReader({ color: 'red' })

// Get the view inside footer2
const footerView = footer2.runReader({ year: 2017, author: 'Bob McBob' })

// Run the view through the function, which returns another view.
const result = colorize(footerView)

// Fold to get the element out
result.fold() // <div style={{ color: 'red' }}}><footer><p>© Bob McBob 2017</p></footer></div>

Rather tediously, we have to run both readers before can apply the colorize function to the footer view.

Of course, there is a better way to do this. And that is with the ap function.

const Reader = computation => ({
  runReader: ctx => computation(ctx),

  map: f => Reader(ctx => f(computation(ctx))),

  ap: other => Reader(ctx => {
    const fn = computation(ctx)
    return fn(other.runReader(ctx))
  })
})

The ap of a Reader will call its resulting function with the resulting value of another Reader. We can visualize this like so.

This means that we can simplify our usage of withColor to this.

// Apply the result from footer2 to the result of withColor.
const x = withColor.ap(footer2)

// We only need to run the Reader once with a shared context.
x.runReader({
  color: 'red',
  year: 2017,
  author: 'Bob McBob'
}).fold() // <div style={{ color: 'red' }}}><footer><p>© Bob McBob 2017</p></footer></div>

A bit of theory: The Reader is now an Applicative in addition to being a functor.

An Applicative A must obey the following laws.

Identity: A.of(x => x).ap(v) === v
Homomorphism: A.of(f).ap(A.of(x)) === A.of(f(x))
Interchange: u.ap(A.of(y)) === A.of(f => f(y)).ap(u)

For the Reader just substitute A with Reader in the above laws.

We can verify that these laws do indeed hold in this test.

Chaining the Reader context

Let’s consider another view for displaying the page title.

const pageTitle = View(({ title }) => <h1>{title}</h1>)

Now, say we want to read the title from the context, just like we did for the copyright notice.

const header = Reader(
  ({ title }) => pageTitle.contramap(() => ({ title }))
).map(map(x => <header>{x}</header> )) // Wrapping title with <header>
                                      // Remember that we have to map twice,
                                      // first over the Reader, then over the View.

But, how can we use both the header and footer2 together? Maybe we map over the View inside the header?

const combined =
  // Map over the view inside header
  header.map(headerView => (
    // Then map over the view inside footer
    footer2.map(footerView => (
      // Concat both views together
      headerView.concat(footerView)
    ))
  ))

This solution works, but isn’t ideal since running combined.runReader returns yet another Reader. Therefore, in order to get the resulting element out we will have to run the Reader twice!

const ctx = {
  year: 2017,
  author: 'Bob McBob',
  title: 'Awesome App'
}
combined.runReader(ctx).runReader(ctx).fold()

Imagine having to use invoke runReader once for each usage of the context. Such drudgery! Luckily for us, there is a much better way to do this. We need to add the chain function.

const Reader = computation => ({
  runReader: ctx => computation(ctx),

  map: f => Reader(ctx => f(computation(ctx))),

  chain: f => {
    return Reader(ctx => {
      // Get the result from original computation
      const a = computation(ctx)
      
      // Now get the result from the computation
      // inside the Reader `f(a)`.
      return f(a).runReader(ctx)
    })
  }
})

The chain function takes in a function f, similar to map. However, f for chain is a function that takes the result from the previous computation as the input, then returns another Reader from it. And when the resulting Reader is run, the context will be passed to all of the chained computations.

This allows us to easily combine the header and footer boxes.

// Takes a view, then returns combined view in a Reader.
const addFooter = other => (
  // Map over the view inside Reader
  footer2.map(footerView => (
    other.concat(footerView)
  ))
)

// We need to use chain here since the result of the function is another Reader.
const combined2 = header.chain(addFooter)

Indeed, we now only need to run the Reader once.

combined2.runReader({
  year: 2017,
  author: 'Bob McBob',
  title: 'Awesome App'
})

A bit of theory: The Reader is now a Monad.

A Monad M must obey the following laws.

Left identity: M.of(a).chain(f) === f(a)
Right identity: m.chain(M.of) === m
Associativity: m.chain(f).chain(g) === m.chain(x => f(x).chain(g))

The M in our case is the Reader.

We can verify that these laws do indeed hold in this test.

But what is a monad exactly?

You can think of a monad as any object that implements the chain function, and obeys the monad laws.

Note that you may see bind or flatMap being used instead of chain in other libraries or articles. They are usually the same as chain, and can be treated as aliases. You might also note a similarity between chain and the Promise then method. They are indeed almost the same. However, in the case of Promises, the computation is not lazy, and has an immediate effect when constructed, whereas the Reader does nothing until runReader is called.

Why didn’t you just ask?

Let’s define a helper function for our Reader monad.

// When asked, just return the context as the result.
Reader.ask = () => Reader(x => x)

Now, we can start the chain by asking for the context.

const a = Reader
  .ask()
  .chain(x => Reader.of(x + 1))
  .chain(y => Reader.of(y * 2))
  .chain(z => Reader.of(`Got ${z}!`))

a.runReader(1) // 'Got 4!'
a.runReader(0) // 'Got 2!'
a.runReader(-1) // 'Got 0!'

We will also add a helper for this dot-chain sytle of code using generator functions.

// Adapted from https://github.com/DrBoolean/freeky/blob/master/monad.js
const Monad = {
  do: gen => {
    let g = gen() // Will need to re-bind generator when done.
    const step = value => {
      const result = g.next(value)
      if (result.done) {
        g = gen()
        return result.value
      } else {
        return result.value.chain(step)
      }
    }
    return step()
  }
}

The Monad.do function allows us to write “normal-looking” JavaScript code.

const b = Monad.do(function*() {
  const ctx = yield Reader.ask() // This will chain!

  // Now this is just "normal" JS
  const x = ctx + 1
  const y = x * 2
  const z = `Got ${y}!`

  // Return the final value in the chain
  return Reader.of(z)
})

b.runReader(1) // 'Got 4!'
b.runReader(0) // 'Got 2!'
b.runReader(-1) // 'Got 0!'

Assembling all the pieces

Recall that the app from the previous post rendered a message along with some header and footer content.

Let’s use our new friend, the Reader monad, and provide some internationalization (i18n) support for that “Hello ___!” message.

We’ll start by defining a message bundle and a formatMessage function.

import IntlMessageFormat from 'intl-messageformat'

const messages = {
  'en': { hello: 'Hello {name}!' },
  'es': { hello: 'Hola {name}!' },
  'fr': { hello: 'Bonjour {name}!' },
  'zh': { hello: '你好{name}!' }
}

const formatMessage = locale => id => values => {
  return new IntlMessageFormat(messages[locale][id], locale).format(values)
}

formatMessage('fr')('hello')({ name: 'Alice' }) // 'Bonjour Alice!'

Now, we can create context-aware functions using formatMessage.

const formatWithLocale = Monad.do(function*() {
  const { locale } = yield Reader.ask() // Read the locale from context
  return Reader.of(formatMessage(locale))
})

// Formatting "hello" is just applying the `id` 'hello' to `formatWithLocale`.
// Of course, we'll need to wrap 'hello' in a Reader as well.
const formatHello = formatWithLocale.ap(Reader.of('hello')) // <-- Remember this!

// Usage
const f = formatHello.runReader({ locale: 'fr' })
const g = formatHello.runReader({ locale: 'zh' })
const h = formatHello.runReader({ locale: 'es' })

f({ name: 'Alice' }) // 'Bonjour Alice!'
g({ name: 'Alice' }) // '你好Alice!'
h({ name: 'Alice' }) // 'Hola Alice!'

Also, recall our three morpshisms on elements from the last post. We’ll use them in the app as well!

const clap = x => <span>👏 {x} 👏</span>
const uppercase = x => <span style={{ textTransform: 'uppercase' }}>{x}</span>
const emphasize = x => <span style={{ fontStyle: 'italic' }}>{x}</span>

Finally, we can build our entire app as follows.

// New context-aware view for displaying a message.
const message = Reader.of(View(({ message }) => <p>{message}</p>))

const app = Monad.do(function*() {
  // Let's colorize our header shall we? 
  // Note: header and withColor were previously defined
  const headerView = yield withColor.ap(header)

  // This will translate our message!
  const sayHello = yield formatHello

  // The message needs to be mapped using the context-aware `sayHello` function.
  const messageView = yield message.map(x =>
    // The `name` prop will be provided when we `fold` the view.
    x.contramap(({ name }) => ({
      message: compose(
        clap,
        emphasize,
        uppercase,
        sayHello // <-- run the function and get a translated message back
      )({ name })
    })
  ))

  const footerView = yield footer2 // Note: footer2 was previously defined)

  // Combine everything together, and we have our app!
  return Reader.of(headerView.concat(messageView).concat(footerView))
})

While we’re at it, let’s crank it up to eleven by styling the app.

// Again, we're mapping over the Reader, then mapping over the View.
const styledApp = app.map(
  map(x => (
    <div style={{ fontFamily: 'Helvetica', textAlign: 'center' }}>{x}</div>
  ))
)

// Now we can render it by providing the context and props.
ReactDOM.render(
  styledApp
    // Execute reader computation with context.
    .runReader({
      color: 'green',
      locale: 'zh',
      title: 'Awesome App',
      author: 'Bob McBob',
      year: 2017
    })
    // Pass props to resulting view.
    .fold({ name: 'Alice' }),
  document.querySelector('#root)
)

When we render the app, this will appear on the screen.

The runnable source code for the full example can be found in this repository.

Summary

In this post, we learned about Applicatives and Monads and created the Reader to provide read-only context for our application.

In the next post we will wrap up this series by adding support for stateful computations. So stay tuned!

If you want additional resources on Monads, I recommend these.

Deconstructing the React Component

Sun, 30 Apr 2017 00:00:00 -0400

In this two-part series, I want to take a functional approach to building React applications.

There will be mathematical theory sprinkled throughout the series, and hopefully by the end of it you will pick up some useful techniques!

Series contents:

Part 1 - Deconstructing the React Component
Part 2 - The Reader monad and read-only context

Functional programming – category theory in particular – teaches us a lot about program construction. Much of the tools that functional programmers reach for are very generalized, and widely applicable to various domains. It is this learn once, appy everywhere idea that I think would appeal to many folks in the business of making software. (Not to mention that functional programming is fun!)

React has many patterns that grew out of the community. I want to take a step back and view these patterns from a new angle, and see if we can formalize them with mathematical theory.

You will need basic React knowledge in order to understand this series. I will discuss concepts such as components, elements, and higher-order components (HOC).

If you want to jump ahead a bit, the repository with the final, full examples are on GitHub. Word of warning though, there isn’t much documentation, so it’s probably easier to read the posts. ;)

So let’s get started!

React from first principles

An element in React describes what should be rendered in the UI.

const x = <span>Hello World!</span>

These elements are immutable since you cannot modify them after they are created. So what can you do with immutable elements? Pass them to functions of course!

Let’s look at three pure functions that take in an element and outputs another element. We say that these functions are pure because they do not have any observable side-effects – just data in, and data out.

const uppercase = element => (
  <span style={{ textTransform: 'uppercase' }}>{element}</span>
)

const clap = element => (
  <span>👏 {element} 👏</span>
)

const emphasize = element => (
  <span style={{ fontStyle: 'italic' }}>{element}</span>
)

A bit of theory: The type for an element is React.Element, which are objects in category theory. Other examples of objects are String, Boolean, etc.

The pure functions uppercase, clap, and emphasize are morphisms in category theory. A morphism is a mapping between two objects -- in this case React.Element -> React.Element.

We can also create computations that allows us to capture data in our elements.

const Greeting = ({ name }) => <span>Hello {name}!</span>

Nothing fancy here. Just a plain old functional component in React. You could render it via JSX <Greeting name="Alice"/>, but I prefer to treat it as a normal function for now – Greet({ name: 'Alice' }).

Now we’re ready for our first contrived example!

import ReactDOM from 'react-dom'

function contrivedEx(name) {
  const a = Greeting({ name })
  const b = uppercase(a)
  const c = emphasize(b)
  const d = clap(c)
  return d
}

ReactDOM.render(contrivedEx('alice'), document.querySelector('#root'))

This will render “👏 HELLO ALICE! 👏” on the screen.

The function contrivedEx has two responsibilities:

Run the computation Greeting to get an element out.
Map the resulting element through multiple functions, then returning the final result.

Let’s formalize these two concepts.

Look at my shiny new box!

We can wrap the original Greeting computation in a box, which I will call View.

const View = computation => ({
  fold: props => computation(props)
})

// Throw in some computation, and now we have a boxed value`.
const greeting = View(({ name }) => <span>Hello {name}!</span>)

To get the value out of the box, we just need to fold it.

greeting.fold({ name: 'Alice' }) // <span>Hello Alice!</span>

As you can see, the fold function let’s us extract the value out of the box.

Hmm, but now when we want to map over our elements, we have to keep folding it down.

This is very tedious, so let’s define a mapping function that can operate on the values within our boxes.

const View = computation => ({
  // I've shortened this since `props => computation(props)` is the same as `computation`
  fold: computation,

  map: f => View(props => f(computation(props)))
})

The map function will return a brand-new box that runs the original computation through the provided function f. Note that map is pure a function, and we can still maintain a reference to the original box.

const superGreeting = greeting
  .map(uppercase)
  .map(emphasize)
  .map(clap)

superGreeting.fold({ name: 'Alice' }) //  <span>👏 <span style={{ fontStyle: 'italic' }}><span style={{ textTransform: 'uppercase' }}><span>Hello Alice!</span></span></span> 👏</span>

// Our original box is unmodified.
greeting.fold({ name: 'Alice' }) // <span>Hello Alice!</span>

All computations are delayed until we call the fold function. This is a really useful property since we can perform multiple transformations without needing to work with concrete values.

We can visualize map as follows.

The morphisms can operate on values (elements), or we can bring them into the world of boxes (Views) and map from box to box. You can extract an element at any point by folding it down, but now we don’t have to until we really need it!

A bit of theory: The View box that we just create is called a Functor in category theory. You can think of functors as any object that provides the map function.

There are two mathematical laws that functors must obey:

Identity: a.map(x => x) === a
Composition: a.map(x => f(g(x)) === a.map(g).map(f)

We can use these laws to our advantage when composing applications. For example, our previous superGreeting view can be optimized by calling map only once, using the composition compose(clap, emphasize, uppercase).

Because of the composition law, we know that Greeting.map(uppercase).map(emphasize).map(clap) is identical to Greeting.map(compose(clap, emphasize, uppercase)).

While we’re at it, let’s also make the View a Pointed Functor by allowing us to make a single value into a View using the View.of function.

View.of = x => View(() => x)

// Folding the view back down from a `View.of` just returns the original element.
View.of(<span>Hello</span>).fold() // <span>Hello</span>

A quick aside on compose. I’m using a library called Ramda that provides a lot of utilities for functional programming, include compose. The composition is applied from right-to-left, so f(g(x)) is the same as compose(f, g)(x). The mathematical symbol “∘” also denotes a composition – .e.g. f ∘ g === compose(f, g).

Okay, moving on!

Formalizing higher-order component concepts

So, isn’t the View box kind of like higher-order components (HOC)? Which we already have in “normal” React.

// Normal React HOC
const red = Component => () => <span style={{ color: 'red' }}><Component/></span>

// A map in our view
const red2 = view => view.map(x => <span style={{ color: 'red' }}>{x}</span>)

Here, we have two functions that when given a component or view, returns something that is like the original input, but in red.

The difference between the HOC and the View is that the latter formalizes the concept of mapping its value through the map function.

But wait, there is something that HOCs can do that Views cannot (yet).

For example,

const withColor = color => Component => props => (
  <Component {...props} color={color}/>
)

The withColor HOC takes a color (e.g. red, #fff, rgb(0,0,0), etc.), then a component that uses the color prop, and then returns a component that is the original, but with the color prop already provided.

const Color = ({ color }) => (
  <span style={{ color }}>{color}</span>
)
const Red = withColor('red')(Color)

<Red/> // <span style={{ color: 'red' }}>red</span>

Whereas the red HOC is mapping over the resulting value, the withColor HOC is mapping over the input props. And since view.map maps over the value (element), there is no chance for us to map over the original input to a view’s computation.

So is there anything we can do?

Of course! We just need to formalize this new concept.

Mapping over inputs

Remember that the View wraps a computation defined as Props -> Element. Let’s take a look at a concrete example with our previous color computation.

const color = View(({ color }) => (
  <span style={{ color }}>{color}</span>
))

color.fold({ color: 'blue' }) // <span style={{ color: 'blue' }}>blue</span>

Here, we have Props that is some data that contains color, and Element is the resulting <span> based on the color passed in.

What we want to do now, is to map over the input props before passing them into the computation. This can be done by defining a nifty new function called contramap.

const View = computation => ({
  fold: computation,
  
  map: f =>
    View(props => f(computation(props)))

  contramap: g =>
    View(props => computation(g(props)))
})

Note that with map we call f with the resulting value, but with contramap we call g with the input props, and then that result is passed to the computation as input.

So, with our new function, we can do the following.

const blue = color.contramap((props) => ({ ...props, color: 'blue' }))

blue.fold() // <span style={{ color: 'blue' }}>blue</span>

Awesome! We can now map over results and contramap over inputs!

A bit of theory: The View is now also a Contravariant, in addition to being a functor. You can think of contravariants as any object that provides the contramap function.

And just like functors, contravariants also have similar mathematical laws:

Identity: a.contramap(x => x) === a
Composition: a.contramap(x => f(g(x)) === a.contramap(f).contramap(g)

The main difference between map and contramap is that composition with the latter is reversed. We can visualize the difference with these two pictures

Functor composition

Contravariant composition

No need for HOCs

Now that we’ve captured both map and contramap use cases of HOCs, we no longer need them! We only need to consider whether we are mapping over result or input to decide whether we want map or contramap.

By formalizing these concepts we can be more explicit and precise about what we are mapping over, and how we are mapping. Additionally, we have mathematical laws to help us compose our program, which is awesome!

Quick Recap

So far we’ve seen how we can:

Put a computation of the form Props -> Element into a box (View).
Take the value out via fold.
Map over the resulting element via map.
Map over the input props via contramap.

But what if we want to combine multiple Views together?

Combining multiple boxes

Let’s step back from the world of React for a bit and consider an array.

const a = [1]
const b = [2]
const c = [3]

We can see that arrays are functors since they implement a map function that obeys the functor laws – you can verify this for yourself!

But arrays have something our View does not: a way to combine two arrays into a new array.

a.concat(b).concat(c) // [1, 2, 3]

So let’s add this concat function to our View.

const View = computation => ({
  fold: computation,
  
  map: f =>
   View(props => f(computation(props)))

  contramap: g =>
   View(props => computation(g(props))),

  // Borrowed from Brian Lonsdord's talk Oh Composable World
  // See: https://youtu.be/SfWR3dKnFIo?t=21m51s
  concat: other =>
    View(props => (
      <React.Fragment>
        {computation(props)}
        {other.fold(props)}
      </React.Fragment>
    ))
})

A bit of theory: The View is now also a Semigroup, which are objects that provide the concat function.

Semigroups have one law:

Associativity: (a.concat(b)).concat(c) === a.concat(b.concat(c))

Note, that View doesn't technically obey the associativity law since the ordering of the nested divs are different. We will correct this later on in this post, but for now let's say that concat ordering is not affecting how the combined View appears on the screen.

And now we can concat like a boss!

const header = View.of(<h1>Awesome App</h1>)
const greeting = View(({ name }) => <p>Hello {name}!</p>)
const footer = View.of(<p>© Bob McBob 2017</p>)

const main = header.map(x => <header style={{ color: 'red' }}>{x}</header>)
  .concat(greeting.contramap(() => ({ name: 'Alice' })))
  .concat(footer.map(x => <footer style={{ color: 'blue' }}>{x}</footer>))

const centered = main.map(x => <div style={{ textAlign: 'center' }}>{x}</div>)

ReactDOM.render(
  centered.fold(),
  document.querySelector('#root')
)

Yes, the map and contramap are a bit gratuitous… But, if you run the program, you should see this.

Just for fun, we can also make View a Monoid by providing an empty function.

View.empty = View.of(null)

A bit of theory: The View is now a Monoid, and it provides the View.empty function that has the following laws.

Right identity: a.concat(A.empty()) === a
Left identity: A.empty().concat(a) === a

Remember how we filtered out null values that result from the computation? This is why the right and left identity laws hold. Otherwise, React will generate HTML comments for null children, thus resulting in different markup.

Summary

Phew! I think this is a good stopping point for this post. Let’s see what we’ve done so far.

We added a new “box” called View that holds a computation to generate a React element.
We added a fold function that extracts a value out of the box.
We added a map function that maps over the value in the box.
We added a contramap function that maps over the input in the box.
We added a concat function that combines two boxes together.

And along the way we learned about Functors, Contravariants, Semigroups, and Monoids.

But we are not done yet since our View still hasn’t recreated two key features of React components: context and state.

In the next post, we will replicate the React context using the Reader monad.

For more resources on related functional programming topics, I recommend the following:

Functional Programming Jargon
Functors, Applicatives, And Monads In Pictures by Aditya Bhargava
Oh Composable World! (video) by Brian Lonsdorf
Professor Frisby Introduces Composable Functional JavaScript (course) by Brian Lonsdorf
Brian Lonsdorf on Medium and Twitter

Don't Fear the Type System

Tue, 24 Jan 2017 00:00:00 -0500

There’s been a lot of talk about static types recently. I’ve especially noticed that folks who usually work with dynamically typed languages have become big proponents of static typing – and I count myself amongst them.

Being a fan of Robert C. Martin (aka Uncle Bob) for a long time, it was a surprise for me to see that he has written not one, but two blog posts on the subject of The Dark Path. He refers to the trend of languages becoming more “strongly” typed as this dark path. Something I disagree with.

Now, I’m not against people holding differing opinions, and it is important that we remain open to opposing views. What I disagree with is the spread of fear through misinformation.

Fear is the path to the dark side.

Fear leads to anger.

Anger leads to hate.

Hate leads to suffering.

– Yoda

In this post, I want to address some of the concerns made against “strong” static types. And I want to present a few reasons for why you may want to consider languages with more powerful type systems, such as Kotlin, for your own projects.

The power of a type system

Firstly, I don’t really like using the term strong typing. It’s ambiguous, and not that useful when discussing type systems. For the purpose of discussion, I describe the power of a type system as the level of expressivity it affords the programmer.

In a dynamic language like Ruby, you cannot encode any type information in code, thus it is not very powerful. Take this simple example of an add function in Ruby.

def add(a, b)
  a + b
end

The only guarantee that we can get with this add function is that it will run. The only static check that can be performed is that the program has valid syntax.

This means that add(1, '2') is a valid program, but if you run it then TypeError will be thrown.

Now, if we look at the same function in Java, we end up with something like this.

BiFunction<Integer,Integer,Integer> add = (a, b) -> a + b;

If we invoke the function with add.apply(1, "2"), then the program is invalid and will not compile because "2" is not an Integer – even though it is syntactically valid!

The difference between the Ruby and Java functions is that in the latter function, we were able to encode constraints about the valid types that it accepts. This is what we refer to as a static type system.

The main advantage of having a static type system is that we get more guarantees about our program without having to run it. By constraining add to only integer parameters, we rule out a whole class of programs that erroneously call it with unsupported types.

There is, however, still a way to compile a valid program that will result in a runtime exception.

// Passing null reference is still a valid Integer!
add.apply(1, null);

Running this program will result in a null pointer exception (NPE).

To deal with nulls, most programmers employ a number of patterns and tests, such as guard-clauses and using preconditions to reject invalid parameters early. You can see examples of these patterns in the book Confident Ruby – FYI, it is a great book if you work with Ruby.

But, can we do better?

Null References: The Billion Dollar Mistake

The inventor of null reference, Tony Hoare, has famously referred to it as the “Billion Dollar Mistake.”

The reason that null is so dangerous is that it subverts all static type checking in many programming languages. The onus is on the programmer to be very careful at all times, because there is no guarantee that they will not receive a null reference at any point in the program.

For this reason, many newer programming languages include the concept of nullable types.

Let’s see how the add function can be implemented in Kotlin (another language that runs on the JVM).

var add = { a: Int, b: Int -> a + b }

If we try to call the function with add(1, null) then we will get a static type error, since we did not declare b as a nullable type Int?. In case we want to allow nullable types, we need to explicitly declare them, which will force us to handle null references whenever we use those values.

// This is not valid because we did not handle the null case of b.
var add = { a: Int, b: Int? -> a + b }

// This is valid since we handled nullable b.
var add = { a: Int, b: Int? -> if (b != null) a + b else null }

And since Kotlin infers the return type correctly as Int?, you cannot use it without checking for null again.

var add = { a: Int, b: Int? -> if (b != null) a + b else null }

// Won't compile because return type from add(1, null) is nullable.
add(1, null) + 2

And just as going from Ruby to Java provides more guarantees about the correctness of our programs, so too does going from Java to Kotlin provide even more guarantees of correctness. By introducing nullable types, Kotlin eliminates a whole class of runtime errors.

Note: There is an issue of potential null errors coming from the Java side, even though they are all checked on the Kotlin side. This is because Kotlin does not assume all values coming from the Java side to be nullable, so you will still need to guard against this yourself.

Whose job is it to prevent defects?

The main thesis in the Dark Path blog post is that the responsibility is on the programmer to prevent defects, and that we shouldn’t keep adding language features just to prevent defects.

Defects are the fault of programmers. It is programmers who create defects - not languages.

He goes on to claim that languages are adding features such as nullable types, because programmers are not testing their code.

Why are these languages adopting all these features? Because programmers are not testing their code.

I completely agree that programmers are responsible for preventing defects, and that testing is important. I, however, also think that the static type checking is a great tool for eliminating whole classes of errors that we don’t have to write tests for.

If we can specify function parameters are integers, then the static type checker can let us know when we mistakenly call it with strings. And in the same vein, if we can specify function parameters as integers, then the type checker can let us know when we mistakenly call it with nulls.

In fact, with more powerful type systems, there are a whole class of tests that you don’t have to write!. Example-based testing only guarantees that the code is correct for the given examples, but types make guarantees about all programs in the language. (There are also other types of testing, such as propery-based testing, but that is out of scope for this post.)

Of course, you can override the null safeties in Kotlin using the !! operator (e.g. add(1, null)!! + 1), and then you’re back to runtime NPEs. But so too can you simply skip or delete failing tests because you are too lazy to fix the program properly. The discussion should not be types versus tests, but types and tests.

The main benefit of types and tests are that they both constrain what a valid program can be. Tests can provide guarantees about runtime constraints and behaviour, and types provide guarantees about compile-time constraints.

Note: I left out behaviour guarantees in a type system since in most languages, this is not possible. However, there are languages with dependent types, such as Idris, that can achieve this with theorem proving.

Constraints liberate. Liberties constrain.

Often times we as programmers want the freedom to do whatever we want. Constraints simply get in our way when we are trying to do our job!

I think this mentality is misguided. If we program without constraints, then our brain is forced keep track of all possibilities in our program. Can I trust an add(a, b) function in Ruby to do the correct thing? Will it blow up if I call it with nil or str? What am I getting in return? An int? Can it be nil?

What if I pass the result of add into another function? Now I have just compounded the problem even further!

Without contraints, our programs are impossible to reason about and the task of understanding it can be a huge mental drain. But with constraints, we can liberate our mind to focus on higher level abstractions.

Contraints a la carte

When we want to add constraints in the system, we can either do it via static types or by adding tests.

In dynamic languages such as Ruby, we only have tests to work with, so we typically turn to TDD. This helps us immensely because we are forced to think about valid types and behaviour before writing code.

In Java, we can have some language-level constraints via static types, but we can also add more constraints via tests. In our tests, we no longer need to consider testing add with non-integers since the program will not compile anyway. Furthermore, we also do not need to test that the return type must be integer, since that is guaranteed statically as well – except for the null case. This removes some burden on the programmer to write certain classes of type tests.

Finally, in Kotlin, we can express that the add function cannot receive null references, thus there is little value in testing for those cases. And of course, we should still be testing for behaviour that cannot be encoded in the type system.

When it comes to preventing defects, both tests and static types can help greatly, so we should be taking advantage of both!

Expressivity of a type system

So far, we’ve seen how constraints are helpful when writing software. The more powerful the type system is, the more constraints we can encode into our programs.

We can also say that a powerful type system, such as the one found in Kotlin, provides an increase in expressivity. For example, if I designed my function to handle null inputs, then I can express that information statically using nullable types.

The expressivity of Kotlin goes beyond nullable types though. There are many other constraints that you can encode into your program.

We will examine one more featured of Kotlin, the sealed class, and how it can help write better software.

Sealed classes

Sealed classes can be used to represent restricted class hierarchies. When a class is marked as sealed, it can only be extended by its nested classes.

Here’s an example of a Either<A,B> type, which can have either a left value of type A and a right value of type B. It is biased towards the right, meaning that mapping over its value will only map the right side. The left side can be considered as an error or exceptional value.

sealed class Either<A,B>() {
    class Left<A,B>(val value: A): Either<A,B>()
    class Right<A,B>(val value: B) : Either<A,B>()

    override fun toString(): String = when(this) {
        is Left -> "Left: ${this.value}"
        is Right -> "Right: ${this.value}"
    }
}

The sealed class hierarchy allows us to express a tagged union. That is, we can write a type, which can take on any value within the sealed set of subclasses. In this case, we can use do the following.

var x: Either<Int,String> = Either.Left(404)
println(x) // Prints: "Left: 404"

x = Either.Right("Success!")
println(x) // prints "Right: Success!"

The advantage of using a sealed class is that the when expressions can be statically checked to be exhaustive. If we miss a case in a when expression, the compiler will inform us that it is not total. To gain this exhaustive guarantee, we need the constraint that the base class will not be extended further by addtional subclasses.

Again, the constraint of disallowing extension gives us more expressiveness. In this case, we can now define a tagged union using sealed class hierarchies.

Exhaustive pattern matching

Let’s look at how we can take advantage of the exhaustive pattern matching by implementing two functions:

The of function which will return a boxed right value.
The flatMap function which will apply a transform function to the boxed right value – ignoring left value.

Here is the of function, which returns a boxed right value.

sealed class Either<A,B>() {
    // ...

    companion object {
        fun <A,B>of(b: B): Either<A,B> {
            return Either.Right<A,B>(b)
        }
    }
}

The companion object can be thought of as an object containing static methods on the base class. We can now use the new function as follows.

// Left Int value could be an error code
val x: Either<Int,String> = Either.of("Hello")

And, here is the flatMap function to transform the right boxed value.

sealed class Either<A,B>() {
    // ...

    // There is actually an error in this implemention!
    fun <C>flatMap(transform: (x: B) -> C): Either<A,C> = when(this) {
        is Right -> Right(transform(this.value))
    }
}

We are pattern matching against the type of Either, and when we see Right, we return its value. Pretty straight-forward right?

But not so fast! There is an error here. The compiler will complain that the when expression is missing the Left branch. To fix this, we need to add a check for Left as well, to make the pattern match exhaustive.

sealed class Either<A,B>() {
    // ...

    fun <C>flatMap(transform: (x: B) -> C): Either<A,C> = when(this) {
        is Left -> Left(this.value)
        is Right -> Right(transform(this.value))
    }
}

And we can now use the flatMap function as follows.

val exclaim = { s: String -> "$s!" }
val upper = { s: String -> s.toUpperCase() }

val x: Either<Int,String> = Either.of("Hello")

println(x
        .flatMap(exclaim)
        .flatMap(upper)) // Prints "Right: HELLO!"

x = Either.Left(404)

println(x
        .flatMap(exclaim)
        .flatMap(upper)) // Prints "Left: 404" because flatMap ignores left value

(As you may have noticed, the Either type is a monad.)

Now, we could have achieved similar results in Java using a final class, but it would be impossible to guarantee the exhaustiveness of pattern matching, which is achieved using when expressions in Kotlin. This is where tests comes in of course, but tests can only provide guarantees for the given set of examples, whereas types can provide language-level guarantees.

If we specify another subclass in a sealed class, then there is no way we would have tested for that in our test suite. This is where the type system can help us out tremendously.

Moreover, without nullable types, even a simple expression such as Either.Right(...).flatMap(upper) can explode on us, because we forgot to check for null reference on the input to upper (since null is a valid String).

Closing

There has been some discussion on the value of more and more powerful type systems – such as in Kotlin. The concerns regarding additional language-level constraints are valid, but I truly believe that adding more language-level constraints will provide us with more freedom in the long run.

Just as testing allows us to constrain the types and behaviour of our programs, so too can static type checking provide even stronger guarantees. Testing shows that the program behaves correctly given a set of examples, whereas static types provide guarantees for all programs.

If programmers are given the ability to encode their design into their code, it will create less ambiguous programs. The decrease in ambiguity helps us reason about our code much better, therefore making less mistakes.

I hope you enjoyed this post. And if you haven’t already, I hope you will consider coming down The Dark Path with me and try out one of these languages!

Suggested Resources

Kotlin Language Reference
Gary Bernhardt’s write-up of types provide good insights into static vs dynamic, and strong vs weak.
Null References: The Billion Dollar Mistake (video)
Constraints Liberate, Liberties Constrain (video)

Additional Guidelines For (Redux) Project Structure

Mon, 12 Dec 2016 00:00:00 -0500

In this series we are looking at code organization in the context of a React and Redux application. The takeaways for the “Three Rules” presented here should be applicable to any application, not just React/Redux.

Series contents

Part 1 - Three Rules for Structuring (Redux) Applications
Part 2 - The Anatomy Of A React & Redux Module (Applying The Three Rules)
Part 3 - Additional Guidelines For (Redux) Project Structure

The Three Rules I presented in the first post are purposely minimal. I don’t believe in being too prescriptive on project structure, since a lot of it will depend on the type of project or personal preferences.

What I want to do in this post is to present other (hopefully helpful!) guidelines for those looking for help with code organization.

What to do with common components?

In a lot of applications, there will be common components that are used everywhere. Some examples could be <Image> or <Title> components that are not related to any feature per se.

My recommendation for these components is to do one of two things.

Keep the common components outside of the modules/ directory.
Make a core module that includes all of these components.

Introducing a separate components folder

In this case, your application structure may be as follows.

src/
  components/       <-- These are common components
    Image.js
    Title.js
    index.js
  modules/          <-- These are the feature modules
    todos/
      components/
      actions.js
      reducer.js
      ...

By moving common components outside of the modules, it may send a stronger signal that these components are not coupled to any part of the application state.

Using a core module for common concerns

src/
  modules/          <-- These modules make up the entire application
    core/
      components/
        Image.js
        Title.js
        index.js
    todos/
      components/
      actions.js
      reducer.js
      ...

Using a core module to group common components reduces the number of “things” in system (everything is a module!), but perhaps at the cost of having a wider definition of what a “module” is – e.g. it may or may not manage a slice of the application state.

Regardless of whether you use a core module, or separate out common components, just make sure you don’t end up throwing too many components in there. If a component is part of a new feature then make sure a new feature module is created!

Exporting and testing connected components

A connected component in a Redux application is one that is wired up to query from the state (using selectors), and can dispatch actions.

I tend to expose connected components from my modules. The reasoning behind this is that the consumers of those components don’t necessarily want to wire up an unconnected component each time that they are used.

For example, the public interface of a <Todo> component should be its ID, and perhaps any additional options or callbacks.

<Todo
  id={123}
  theme="dark"
  size="large"
  onUpdate={...}
/>

This hides all of the internal details of data flow within the todos module.

Contrast the above to an unconnected component.

<Todo
  todo={...}
  actions={...}

  theme="dark"
  size="large"
  onUpdate={...}
/>

In the unconnected case, the consumer of the component has the additional responsibility of providing the queried todo object, as well as any required actions functions (e.g. action creators). This means that you would have to keep repeating the same code of importing action creators and selectors, and connecting them each time a <Todo> is used!

Now, there is one downside of exporting connected components. It makes unit testing more complicated, because each time you test the component, you have to instantiate the Redux store with the correct reducers, etc.

One way to deal with this is to always test these components with stores. This has the benefit of being closer to integration tests, however, it makes testing pure renders a lot more cumbersome. That is, if you just want to test that the component renders correctly, given a set of inputs, then having to deal with stores is not ideal.

A simple solution is to export both the connected and unconnected components. The unconnected component can be exported only within the module for testing purposes.

For example, a test for <Todo> may be as follows.

// In todos/components/__tests__/Todo-test.js

import { shallow } from 'enzyme'
import test from 'tape'
import { Todo } from '../' // This named export is unconnected.

test('it renders correctly', function (t) {
  const todo = { id: 1, text: 'Finish writing post' }
  const wrapper = shallow(
    <Todo todo={todo}/>
  )

  t.ok(/Finish writing post/.test(wrapper.html()), 'renders text')

  t.end()
})

Given that the component is as follows.

// In todos/components/Todo.js

// Exporting named, unconnected component for testing purposes.
export const Todo = ({ todo, actions }) => (
  <div>
    <p>{ todo.text }</p>
    ...
  </div>
)

// The default export is the connected component.
export default connect(...)(Todo)

And in the todos/index.js we would only export the connected component.

// In todos/index.js

export Todo from './components/Todo'

Normalizing application state

The last guideline is to always normalize your application state. This makes it much more natural to work with connected components and selectors from other modules.

For example, the todos module can hold its records in an object/map. A map is a good structure to use here since it allows easier lookup and deletion.

{
  "todos": {
    "byId": {
      "1": { "id": 1, "text": "Item One", "done": false },
      "2": { "id": 2, "text": "Item Two", "done": true },
      "3": { "id": 3, "text": "Item Three", "done": false },
      ...
    }
  }
}

Then, in other modules (such as projects), we can reference Todos by their IDs.

{
  "projects": {
    "byId": {
      "100": {
        "id": 100,
        "name": "Some Project",
        "todos": [1, 3] // These IDs should exist in the `todos` state.
      },
      ...
    }
  }
}

By normalizing data, we can avoid stale data issues, since we never duplicate the same object twice in the system. Any references to an entity has to use its ID, not an object reference or a clone.

And as noted earlier, this normalization of state plays very nicely with connected components.

const Project = ({ project }) => (
  <div>
    <h2>{ project.name }</h2>
    <p>
      Todos:
      {
        project.todos.map(todoId => (
          <Todo id={todoId}/>
        )
      }
    </p>
  </div>
)

Reducing to normalized states

This also means that we need to handle loading normalized data. You can either do this using a saga – where you use put to dispatch the appropriate loading actions – or you can use thunks.

// Using saga
function* requestProject(id) {
  const response = yield call(fetch, `/projects/${id}`)
  yield put({ type: 'todos/LOAD', payload: response.todos })
  yield put({ type: 'projects/LOAD', payload: [response.project] })
}

// Using thunks
const requestProject = (id) => {
  return async (dispatch) => {
    const response = await fetch(`/projects/${id}`)

    // Dispatch events to both todos and projects module so we can store the normalized data.
    dispatch({ type: 'todos/LOAD', payload: response.todos })
    dispatch({ type: 'projects/LOAD', payload: [response.project] })
  }
}

If you are interested in learning more about sagas, I have a blog post about them here.

Closing

In this series, we began by looking at the Three Rules for structuring applications.

Organize by features
Create strict module boundaries
Avoid circular dependencies

Then, in the second post, we went into in-depth examples of how to structure each module with our application.

And lastly, in this post we explored additional guidelines to help with code organization.

Where do common/unconnected/dumb components fit in?
Exporting and testing connected components
Normalizing application state

I hope this series has proven useful to you. Feel free to leave a comment, or reach me on Twitter!

Why You Need Types

Tue, 23 Aug 2016 00:00:00 -0400

Dynamically typed languages like JavaScript provide a lot of expressiveness and power to the programmer. By not having to think about strict types, a program is more maleable since it will run no matter what, allowing the programmer to write code very quickly.

The problem with dynamic types is that it slows down development of an application over time. This decrease in velocity can be attributed to a couple of factors.

Bugs introduced by type mismatches (such as null types) take time away from feature development.
Each time existing code is examined, the programmer needs to figure out what the intended types are again.

But Why Worry About Types At All?

Programs should be written for people to read, and only incidentally for machines to execute.

- Abelson and Sussman in “Structure and Interpretation of Computer Programs”

The first question you may ask is why we should bother with types at all? The answer to this is simple: Types are like the weather. There is nothing you can do about it. Weather happens.

Whenever you write down JavaScript code, you are already thinking about types, just unable to write them down.

For example, say you implemented an add function as follows.

const add = (a, b) => a + b

The types for a and b are implicitly number, but there is no way to write this down in code.

Or how about a greet function as follows.

const greet = (name = 'World') => console.log(`Hello ${name}!`)

Can you guess the type for the parameter name? If you guessed string then you are close but not quite correct. The correct type is an optional string – meaning it can be undefined.

Without explicit types, we will inevitably make a mistake and use the wrong ones. Is add(null, 2) a valid invocation? Obviously not! But what about add('1', 2). Maybe? It is hard to know the original intention of the function’s author.

In short, it is very difficult for another programmer (or the future you) to understand what the originally intended types are for every function in an application.

Types To The Rescue!

Let’s look at both functions again with type declarations (using Flow).

const add = (a: number, b: number): number => a + b

// `name?` means it is optional
const greet = (name?: string = 'World'): void => console.log(`Hello ${name}!`)

It is now clear that add('1', 2) is not a valid invocation of the function. Furthermore, the Flow type checker will actually throw a useful error.

(We also know that add(add(1, 2), 3) is valid, because the return type of add is number!)

As for the greet function, it should be clear which of the following statements are valid or not.

greet() // Valid with since name is optional
greet('Alice') // Valid since 'Alice' is a string
greet(1) // Invalid since 1 is not a string
greet(null) // Invalid since null is not a string

add(greet(), 2) // Invalid since greet returns void, which is not a number

Note: You do not need to know the implementation of each function to know whether the types are valid or not. Knowing the input and output types, that is the interface, are enough.

Getting Rid Of That Dreaded Null error

The single most common error in a JavaScript application is likely the null is not a function error, or some variation of it. This class of error is completely due to type mismatches: Not dealing with nullable types properly.

For example, given a function such as the following.

const updateEmail = (person, email) => {
 person.contact.email = email
}

It isn’t clear whether there is a bug in that code or not.

However, if I were to add some type annotations to it, then it is very clear that there is a bug!

type IPerson = {
  id: number,
  contact?: {
    email: string,
    address: string
  }
}

const updateEmail = (person: IPerson, email: string): void => {
  // `contact.email` may be undefined, thus is unsafe!
  person.contact.email = email
}

In fact, you don’t even need to manually check the code since Flow will check for you.

Awesome!

Aren’t Types Too Constraining and Troublesome?

There are two common concerns of using a type system:

They are too troublesome to set up.
They constrain the programmer too much.

While both are valid concerns, Flow type – and TypeScript to some extent – provide an almost seemless experience.

For starters, if we take the original add function, and simply add the // @flow comment to the module, then we can already run it through the type checker!

// @flow
const add = (a, b) => a + b

Of course, without type annotations, the type checker is limited in usefulness. It can catch errors such as add([], 2) but not add(null, 2), since the latter null + 2 operation is (unfortunately) valid JavaScript.

Note: Flow can also catch unary or nullary calls of the add function: e.g. add(1) and add().

As for contraints caused by static types, it simply isn’t true. If you want dynamism, you can use the any type to forgo type checking. In fact, the any type is implicitly the only type available in pure JavaScript!

// @flow
const add = (a: any, b: any): any => a + b

add(null, null) // Valid
add() // Valid
add([], {}) // Valid
add(add, add) // Valid

Adding Types To Your Project Today!

Now that I’ve convinced you that types are really great, then the next step is setting it up in your own project. I recommend Flow if want to perform a quick experiment, since the set up is faster than the alternatives. TypeScript is also great, but it will require a bit more work (e.g. configuring tsconfig.json and renaming .js files to .ts). In a real project you will need to weigh the pros and cons of each technology choice.

Setting Up Flow

You only need to do two things to get Flow running in your project.

Firstly, install the flow binary globally.

npm install -g flow-bin@0.30

Secondly, go to your project root and run:

flow init # This creates the .flowconfig file

That’s it! You can now verify that Flow is working in your project.

flow check

This should produce no errors since you have not added the // @flow comment to any modules yet.

Note: To run Flow code you will need to strip out type annotations using Babel. Follow this guide for more information. Alernatively, you can use the comment syntax if don't want to use a transpiler -- thanks to Gleb for this tip.

Gradually Adding Types

The next step is to start adding types to your modules. I recommend starting with modules that don’t have any dependencies (i.e. no imports), or as little imports as possible. If a module has dependencies, then you will need to type those first before typing the dependent module.

If you are working with third-party modules, you will need to install those type definitions before you can import them. The best way to get third-party type definitions is using flow-typed.

npm install -g flow-typed

You can search and install type definitions using this binary.

flow-typed search redux
flow-typed install -f 0.30 redux

The -f option specifies the Flow version you want to install the definition for.

To automatically include the installed definitions, add an entry to the [libs] section in the .flowconfig file.

[libs]
./flow-typed/npm/

If you cannot find type definitions using flow-typed , you can simply provide them yourself.

Start by adding a new [libs] entry to the .flowconfig file.

[libs]
./flow-typed/npm/
./manual-typings/

Then define the third-party modules in the ./manual-typings/ folder. For example, the lodash.throttle module can be defined in the ./manual-typings/lodash.throttle.js file as follows.

declare module 'lodash.throttle' {
  declare var exports: any
}

Here, I’m being lazy with the types and simply declared the default export as any, which means it will not be type checked. Using the any type for module declaration is a good starting point for your project, and you can always go back to define the correct types when you have the time to.

// Better definition
declare module 'lodash.throttle' {
  declare type ThrottleOptions = {
    leading?: boolean,
    trailing?: boolean
  }
  declare var exports: (fn: Function, wait?: number, options?: ThrottleOptions) => void
}

Wrapping Up

So should you start using types in your projects? It is really up to you! If you do choose to start using types though, you will find that your application becomes easier to maintain because:

The type checker will catch type mismatches for you, resulting in less bugs in production.
It will be easier to read existing code, because it will be immediately obvious what the valid inputs and outputs are.

It is also seemless to integrate types into your application through Flow (or TypeScript). For even more powerful type systems in the front end, you can also check out Elm and PureScript – both of which are in the ML family of languages.

Three Rules For Structuring (Redux) Applications

Sun, 28 Feb 2016 00:00:00 -0500

Series contents

Part 1 - Three Rules for Structuring (Redux) Applications
Part 2 - The Anatomy Of A React & Redux Module (Applying The Three Rules)
Part 3 - Additional Guidelines For (Redux) Project Structure

As our applications grow, we often find that file structure and organization to be crucial for the maintainability of application code.

What I want to do in this post is to present three organizational rules that I personally follow on my own projects. By following the rules, your application code should be easier to reason about, and you will find yourself wasting less time on file navigation, tedious refactoring, and bug fixes.

I hope that these tips will prove useful for developers who want to improve their application structure, but don’t know where to start.

Three rules for project structure

The following are some basic rules for structuring a project. It should be noted that the rules themselves are framework and language agnostic, so you should should be able to follow them in all situations. However, the examples are in React and Redux. Familiarity with these frameworks is useful.

Rule #1: Organize by feature

Let’s first start by going over what not to do. A common way that projects can be organized is by object roles.

Redux + React:

actions/
  todos.js
components/
  todos/
    TodoItem.js
    ...
constants/
  actionTypes.js
reducers/
  todos.js
index.js
rootReducer.js

AngularJS:

controllers/
directives/
services/
templates/
index.js

Ruby on Rails:

app/
  controllers/
  models/
  views/

It may seem reasonable to group similar objects together like this (controllers with controllers, components with components), however as the application grows this structure does not scale.

When you add and change features, you’ll start to notice that some groups of objects tend to change together. These objects group together to form a feature module. For example, in a todo app, when you change the reducers/todos.js file, it is likely that you will also change actions/todos.js and components/todos/*.js.

Instead of wasting time scrolling through your directories looking for todos related files, it is much better to have them sitting in the same location.

A better way to structure Redux + React project:

todos/
  components/
  actions.js
  actionTypes.js
  constants.js
  index.js
  reducer.js
index.js
rootReducer.js

Note: I will go into the details of what's inside the files in the next post.

In a large project, organizing by feature affords you the ability to focus on the feature at hand, instead of having to worry about navigating the entire project. This means that if I need to change something related to todos, I can work soley within that module and not even think about the rest of the application. In a sense, it creates an application within the main application.

On the surface, organizing by feature may seem like an aesthetics concern, but as we will see in the next two rules, this way of structuring projects will help simplify your application code.

Rule #2: Create strict module boundaries

In his Ruby Conf 2012 talk Simplicity Matters, Rich Hickey defines complexity as the complecting (or interleaving) of things. When you couple modules together, you can picture an actual knot or braid forming in your code.

The relevence of complexity to project structure is that when you place objects in close proximity to one another, the barrier to couple them lowers dramatically.

As an example, let’s say that we want to add a new feature to our TODO app: We want the ability to manage TODO lists by project. That means we will create a new module called projects.

projects/
  components/
  actions.js
  actionTypes.js
  reducers.js
  index.js
todos/
index.js

Now, it is obvious that the projects module will have a dependency on todos. In this situation, it is important that we exercise discipline and only couple to the “public” API exposed in todos/index.js.

BAD

import actions from '../todos/actions';
import TodoItem from '../todos/components/TodoItem';

GOOD

import todos from '../todos';
const { actions, TodoItem } = todos;

Another thing to avoid is coupling to the state of another module. For example, say that within the projects module, we need to grab information out of todos state in order to render a component. It is better that the todos module exposes an interface for projects to query this information, rather than complecting the component with todos state.

BAD

const ProjectTodos = ({ todos }) => (
  <div>
    {todos.map(t => <TodoItem todo={t}/>)}
  </div>
);

// Connect to todos state
const ProjectTodosContainer = connect(
  // state is Redux state, props is React component props.
  (state, props) => {
    const project = state.projects[props.projectID];

    // This couples to the todos state. BAD!
    const todos = state.todos.filter(
      t => project.todoIDs.includes(t.id)
    );

    return { todos };
  }
)(ProjectTodos);

GOOD

import { createSelector } from 'reselect';
import todos from '../todos';

// Same as before
const ProjectTodos = ({ todos }) => (
  <div>
    {todos.map(t => <TodoItem todo={t}/>)}
  </div>
);

const ProjectTodosContainer = connect(
  createSelector(
    (state, props) => state.projects[props.projectID],

    // Let the todos module provide the implementation of the selector.
    // GOOD!
    todos.selectors.getAll,

    // Combine previous selectors, and provides final props.
    (project, todos) => {
      return {
        todos: todos.filter(t => project.todoIDs.includes(t.id))
      };
    }
  )
)(ProjectTodos);

In the “GOOD” example, the projects module is not concerned with the internal state of todos module. This is powerful because we can freely change the structure of the todos state, without worrying about breaking other dependent modules. Of course we still need to maintain our selector contracts, but the alternative is having to search through a whole bunch of disparate components and refactor them one by one.

By artificially creating strict module boundaries, we can simplify our application code, and in turn increase the maintainability of our application. Instead of haphazardly reaching inside other modules, we should think about forming and maintaining contracts between them.

Now that the projects are organized by features, and we have explicit boundaries between each feature, there is one last thing I want to cover: circular dependencies.

Rule #3: Avoid circular dependencies

It shouldn’t take too much convincing for you to believe me when I say that circular dependencies are bad. Yet, without proper project structure, it is all too easy to fall into this trap.

Most of the time, dependencies start out innoculously. We may think that the projects module need to reduce some state based on todos actions. If we are not grouping by features, and we see a large manifest of all action types within a global actionTypes.js file, it is all too easy for us to just reach in and grab what we need (at the time) without a second thought.

Say, that within todos we want to reduce state based on an action type of projects. Easy enough if we have a global actionTypes.js file. However, we will soon learn that this is no easy feat if we have explicit module boundaries. To illustrate why, consider the following example.

Circular dependency example

Given:

a.js

import b from './b';

export const name = 'Alice';

export default () => console.log(b);

b.js

import { name } from './a';

export default `Hello ${name}!`;

What happens with the following code?

import a from './a';

a(); // ???

We might expect “Hello Alice!” to be printed, but in actuality, a() would print “Hello undefined!”. This is because the name export of a is not available when a is imported by b (due to circular dependencies).

The implication here is that we cannot both have projects depend on action types within todos and todos depend on action types within projects. You can get around this restriction in clever ways, but if you go down this road I can guarantee you that it will come to bite you later on!

Don’t make hairballs!

Put another way, by creating circular dependencies, you are complecting in the worst possible way. Imagine a module to be a strand of hair, then modules that are inter-dependent on each other form a big, messy hairball.

Whenever you want to use a small module within the hairball, you will have no choice but to pull in the giant mess. And even worse, when you change something inside the hairball, it would be hard not to break something else.

By following Rule #2, it should make it hard for you to create these circular dependencies. Don’t fight against it. Instead, use that energy to properly factor your modules.

Now that we have our three rules, there is one last topic I want to discuss: How to detect project smells.

Litmus test for project structure

It is important for us to have the tools to tell us when something smells in our code. From experience, just because a project starts out clean doesn’t mean it’ll stay that way. Thus, I want to present an easy method to detect project structure smells.

Every once in a while, pick a module in your application and try to extract it as an external module (e.g. a NodeJS module, Ruby gem, etc). You don’t have to actually do it, but at least think it through. If you can perform this extraction without much effort then you know it is well factored. The term “effort” here remains undefined, so you need to come up with your own measure (whether subjective or objective).

Run this experiment with other modules in your application. Jot down any problems you find in your experiments: circular dependencies, modules breaching boundaries, etc.

Whether you choose to take action based on your findings is up to you. Afterall, the software industry is all about tradeoffs. But at the very least it should give you a much better insight into your project structure.

Summary

Project structure isn’t a particularly exciting topic to discuss. It is, however, an important one.

The three rules presented in this post are:

Organize by features
Create strict module boundaries
Avoid circular dependencies

In the next post, we will dive deeper into code examples and learn how these rules can be applied to a React and Redux project.

The Anatomy Of A React & Redux Module (Applying The Three Rules)

Sun, 28 Feb 2016 00:00:00 -0500

Series contents

Part 1 - Three Rules for Structuring (Redux) Applications
Part 2 - The Anatomy of a React & Redux Module (Applying The Three Rules)
Part 3 - Additional Guidelines For (Redux) Project Structure

In this post, we are expanding on the three rules of structuring applications and diving deeper into the contents of the different files with a Redux and React application.

This post will be specifically about React and Redux, so feel free to skip to the next post if you are not interested in these libraries.

In-depth example and recommendations

Recall the three rules presented in the previous post:

Organize by features
Create strict module boundaries
Avoid circular dependencies

Now, let’s take a look at our TODO app again. (I added constants, model, and selectors into this example)

todos/
  components/
  actions.js
  actionTypes.js
  constants.js
  index.js
  model.js
  reducer.js
  selectors.js
index.js
rootReducer.js

We can break these modules down by their responsibilities.

Module index and constants

The module index is responsible for maintaining its public API. This is the exposed surface where modules can interface with each other.

A minimum Redux + React application should be something like this.

// todos/constants.js

// This will be used later in our root reducer and selectors
export const NAME = 'todos';

// todos/index.js
import * as actions from './actions';
import * as components from './components';
import * as constants from './constants';
import reducer from './reducer';
import * as selectors from './selectors';

export default { actions, components, constants, reducer, selectors };

Note: This is similar to the Ducks structure.

Actions & Action creators

Action types are just string constants within Redux. The only thing I’ve changed here is that I prefixed each type with “todos/” in order to create a namespace for the module. This helps to avoid name collisions with other modules in the application.

// todos/actionTypes.js
export const ADD = 'todos/ADD';
export const DELETE = 'todos/DELETE';
export const EDIT = 'todos/EDIT';
export const COMPLETE = 'todos/COMPLETE';
export const COMPLETE_ALL = 'todos/COMPLETE_ALL';;
export const CLEAR_COMPLETED = 'todos/CLEAR_COMPLETED';

As for action creators, not much changes from the usual Redux application.

// todos/actions.js
import * as t from './actionTypes';

export const add = (text) => ({
  type: t.ADD,
  payload: { text }
});

// ...

Note that I don’t necessarily need to use addTodo since I’m already in the todos module. In other modules I may use an action creator as follows.

import todos from 'todos';

// ...

todos.actions.add('Do that thing');

Model

The model.js file is where I like to keep things that are related to the module’s state.

This is especially useful if you are using TypeScript or Flow.

// todos/model.js
export type Todo = {
  id?: number;
  text: string;
  completed: boolean;
};

// This is the model of our module state (e.g. return type of the reducer)
export type State = Todo[];

// Some utility functions that operates on our model
export const filterCompleted = todos => todos.filter(t => t.completed);

export const filterActive = todos => todos.filter(t => !t.completed);

Reducers

For the reducers, each module should maintain their own state as before. However, there is one particular coupling that should be solved. That is, a module’s reducer does not usually get to choose where it is mounted in the overall application state atom.

This is problematic, because it means our module selectors (which we will cover next) will be indirectly coupled to the root reducer. In turn, the module components will also be coupled to the root reducer.

We can solve this issue by giving control to the todos module on where it should be mounted in the state atom.

// rootReducer.js
import { combineReducers } from 'redux';
import todos from './todos';

export default combineReducers({
  [todos.constants.NAME]: todos.reducer
});

This removes the coupling between our todos module and root reducer. Of course, you don’t have to do it this way. Other options include relying on naming conventions (e.g. todos module state is mounted under “todos” key in the state atom), or you can use module factory functions instead of relying on a static key.

And the reducer would look as follows.

// todos/reducer.js
import t from './actionTypes';
import type { State } from './model';

const initialState: State = [{
  text: 'Use Redux',
  completed: false,
  id: 0
}];

export (state = initialState, action: any): State => {
  switch (action.type) {
    case t.ADD:
      return [
        // ...
      ];
    // ...
  }
};

Selectors

Selectors provide a way to query data from the module state. While they are not normally named as such in a Redux project, they are always present.

The first argument of connect is a selector in that it selects values out of the state atom, and returns an object representing a component’s props.

I would urge that common selectors by placed in the selectors.js file so they can not only be reused within the module, but potentially be used by other modules in the application.

I highly recommend that you check out reselect as it provides a way to build composable selectors that are automatically memoized.

// todos/selectors.js
import { createSelector } from 'reselect';
import _ from 'lodash';
import { NAME } from './constants';
import { filterActive, filterCompleted } from './model';

export const getAll = state => state[NAME];

export const getCompleted = _.compose(filterCompleted, getAll);

export const getActive = _.compose(filterActive, getAll);

export const getCounts = createSelector(
  getAll,
  getCompleted,
  getActive,
  (allTodos, completedTodos, activeTodos) => ({
    all: allTodos.length,
    completed: completedTodos.length,
    active: activeTodos.length
  })
);

Components

And lastly, we have our React components. I encourage you to use shared selectors here as much as possible. It gives you the advantage of having an easy way to unit test the mapping of state to props without relying on component tests.

Here’s an example of a TODO list component.

import { createStructuredSelector } from 'reselect';
import { getAll } from '../selectors';
import TodoItem from './TodoItem';

const TodoList = ({ todos }) => (
  <div>
    todos.map(t => <TodoItem todo={t}/>)
  </div>
);

export default connect(
  createStructuredSelector({
    todos: getAll
  })
)(TodoList);

Other responsibilities

There are many other things that could be included in a module. For example, you may be using redux-saga, which should definitely be included in your module structure – where each module can optionally expose a root saga.

Other things that you may include are route handlers (or containers) for use with react-router – or a similar routing library.

Summary

We’ve seen in this post how to break down a React/Redux module into individual core responsibilities.

Index and constants - The public API and constants.
Actions and action creators - Information that flow through the application.
Model - Types and utilities for the model.
Reducers - Updates module state.
Selectors - Queries module state.

In the last post of this series on code organization, we will discuss additional guidelines that fall outside of the “Three Rules.”

The Little Idea of Functional Programming

Wed, 13 Jan 2016 00:00:00 -0500

A little known fact about me is that I took Wing Chun lessons for four months back in 2006. A student starting in Wing Chun will begin by learning the first form, called Siu Lim Tao (小練頭). This form’s name translates to “little idea,” and it provides the foundation in which succeeding forms and techniques depend upon.

I’ve spent a lot of time thinking about the little ideas behind the art of programming. In particular, the topic of functional programming keeps cropping up every once in a while. Some people love it, some people hate it, while others just have no idea what functional programming is all about.

When you ask people about functional programming, you might hear jargons like monads, morphisms, or semigroups. Others may say functional programming is all about map-filter-reduce. And you may also hear talks about purity, and side effects.

What I want to do in this post is to take a look at a couple of simple concepts that make up the little idea behind functional programming. I will tie the concepts back to code examples in JavaScript. There will be some ReactJS code, so some familiarity with it helps, although not necessary.

Also, a big shoutout to Brian Lonsdorf for his Class Coding with Prof. Frisby video series, which inspired the code examples in this post.

Implementing YouTube instant search

Before we begin exploring functional programming concepts, let’s take a look at our sample problem - a YouTube search.

The YouTube search app will allow the user to type in search terms into an input box. As the user types, video results will appear below the input box.

OK, now we are ready to begin!

The Little Idea

There are three concepts that capture the essence of functional programming.

Data in, data out
Code as data
(Function) Composition all the way down

Other functional programming concepts and techniques can be built upon this little idea.

Concept #1 - Data in, data out

The first concept is the separation of data and behaviour. Data by themselves cannot do anything, but functions can be used to take them in as input and then output some new data. You can picture a function as a box that take in some shape, and spits out another.

In the illustration above, function f takes a triangle as input, and outputs a square. Function g takes in a square and outputs a circle.

When working with data in, data out, it is important to enforce function purity. A function is pure if its output is solely determined by its input. That is, given the same input, it must always return the same output. Furthermore, a pure function must not have any side effects. An effect is something like writing to database, changing a global variable, or throwing exceptions.

With this idea in mind, we can implement a couple of functions to be used in our app.

Code - Mapping search term to URL

As the user is typing into the input box, we need a way to use that value and transform it to an URL to fetch the results.

Here, we take in a String as input, and ouput an Url. Note that Url is simply an alias for String but has better semantics.

// this type alias will only work if you use TypeScript or FlowType
type Url = String;

// makeUrl :: String -> Url
const makeUrl = (term) =>
  `https://www.googleapis.com/youtube/v3/search?part=snippet&maxResults=24&type=video&q=${term}&key=xxxx`; // xxxx is the Google API key

Now that we can map the user’s input value to an URL, we can fetch an API call to fetch the response JSON. So let’s look at the VideoJSON object, and think about how we can build Video objects from them.

Code - Hash to Video

The YouTube API returns a VideoJSON as follows.

{
  "id": {
    "kind": "youtube#video",
    "videoId": "9Q7Vr3yQYWQ"
  },
  "snippet": {
    "title": "Led Zeppelin - Stairway to Heaven Live (HD)",
    ...
    "thumbnails": {
      "high": {
      "url": "https://i.ytimg.com/vi/9Q7Vr3yQYWQ/hqdefault.jpg"
      },
    },
    ...
  },
  ...
}

Therefore, we can map it to a Video type as follows.

// This is the tagged constructor.
// e.g.
// const v = Video('1', 'http://...', 'http://...', 'Some Video');
// v.id; //=> '1'
// v.title; //=> 'Some Video'
// v instanceof Video; //=> true
const Video = daggy.tagged('id', 'thumbnail', 'url', 'title');

// toVideo :: VideoJSON -> Video
const toVideo = json => {
  const {
    id: { videoId },
    snippet: { thumbnails: { high: { url: thumbnail } }, title }
  } = json;
  return Video(videoId, thumbnail, `https://www.youtube.com/watch?v=${videoId}`, title);
};

Note: I am using Haskell's type declaration notation in the comments above the functions (e.g. toVideo :: JSON -> Video). It describes the input and output of the declared function.

Concept #2 - Code as data

The second concept is more commonly referred to as higher-order functions. It simply means that functions can be used as input to other functions; and that functions can output new functions.

For example, given the toVideo :: JSON -> Video function above, we can use it to map over an array of JSON objects.

// Array of JSON objects
const jsonObjects = [{ "id": "1", ... }, { "id": "2", ... }, ...];

// Map over them to create array of Video.
const videos = jsonObjects.map(toVideo);

videos[0].id; //=> "1"
videos[1].id; //=> "2"
videos[2].id; //=> "3"

Here, the Array.prototype.map method takes in a transformation function as input, and returns a new array with the original elements transformed by the input function.

The downside of the Array.prototype.map method is that you can only call it in the context of the input data. As an alternative, we can use Ramda’s map function, which will take in a transformation function as input (same as before), and then output a new function that can be called with mappable objects (such as Array).

const mapVideos = map(toVideo);

mapVideos(jsonObjects); //=> [Video("1"), Video("2"), Video("3")]
mapVideos([]); //=> []
mapVideos([{ "id": "4" }]); //=> [Video("4")]

This new mapVideos function is now completely modular and can be used with any array containing VideoJSON objects to get an array of Video objects.

Note: The output function from map is called a transducer. Transducers are composable transformations that are decoupled from input data. You can watch Rich Hickey's talk on transducers to learn more about them.

Concept #3 - Composition all the way down

The third concept is that we can compose two functions (f and g) together (g ∘ f) as long as f’s output matches g’s input.

In the illustration above, we created a new function h that composes f and g. This new function takes a triangle as input (input of f) and outputs a circle (output of g). We can describe this in JavaScript using the compose function.

const h = compose(g, f); // h(x) === g(f(x))

Taking this idea further, if we had another red function as follows – taking in a nested data that includes triangle, then outputs triangle.

Then, we can compose red and h together to take in the nested data, and output circle.

This allows us to start with simple functions and use composition to build up more powerful functions. The advantage of this approach is that the functions are completely modular. All you have to do is make sure that the inputs and outputs match while composing.

So, let’s compose a few functions for our app!

Code - Mapping JSON response to videos

The mapVideos function that we previously implemented takes an array of VideoJSON objects as input, and ouputs an array of Video objects (mapVideos :: [VideoJSON] -> [Video]). However, the YouTube API actually nests the array of video JSON objects inside a structure like this:

{
  ...
    "items": [
      { "id": ... },
      ...
    ]
}

We can retrieve the VideoJSON objects with the prop function, which returns a property (by name) when called with an object.

prop('items')({ items: [{ id: 1 }] }); //=> [{ id: 1}]
prop('items')({ items: [] }); //=> []

Now, if we pass the response JSON to prop('items'), we will get an array of VideoJSON objects, which is exactly what mapVideos takes as input. This is all we need to define the toVideos function.

// toVideos :: VideoJSON -> [Video]
const toVideos = compose(mapVideos, prop('items'));

toVideos({ items: [{ id: 1 }, { id: 2 }]}); //=> [{ id: 1 }, { id: 2 }]
toVideos({ items: []}); //=> []

Building a function to search videos

Let’s recap what we have so far.

A function makeUrl that takes user input String and outputs an Url:

// makeUrl :: String -> Url
const makeUrl = (term) =>
  `https://www.googleapis.com/youtube/v3/search?part=snippet&maxResults=24&type=video&q=${term}&key='xxxx'`;

A function toVideos that takes ResponseJSON and outputs [Video].

// toVideos :: ResponseJSON -> [Video]
const toVideos = compose(mapVideos, prop('items'));

So we just need a function that can take in Url and output ResponseJSON. Well, it’s not as simple as that since we are dealing with an API call that goes over the network, thus we cannot return a simple value. In this case, I am opting to use the Task type, which represents a delayed computation. Think of it as a Promise, but does not have an effect right away.

Usage example:

// Create new task that will resolve with 'hello' after 100 ms
const t = new Task((reject, resolve) => {
  setTimeout(() => resolve('hello!'), 100)
});

// The effect only happens when `fork` is called.
t.fork(
  // Rejected handler
  (x) => console.error(x),

  // Resolved handler
  (s) => console.log(s)
);

Now the function httpGet is defined as follows.

// httpGet :: String -> Task Error ResponseJSON
const httpGet = (url) =>
  new Task((rej, res) =>
    fetch(url)
      // Since json() returns another promise,
      // we need to chain it down to the resolve function.
      .then((resp) =>
        resp.json().then(res)
      )
      .catch(rej)
  );

And then we can define the final searchVideos function as follows.

// searchVideos :: String -> Task Error [Video]
export const searchVideos = compose(lift(toVideos), httpGet, makeUrl);

Note: The lift function is used inside the composition. This is needed since the return type of httpGet is Task Error ResponseJSON, we need to lift the ResponseJSON -> [Video] transformation function into a function that can operate on the future resolved value of the Task. The type for lift(toVideos) is M ResponseJSON -> M [Video], where M is a mappable object -- in this case we are using a Task.

Wiring up the UI

Now comes the easy part of putting the UI togther to use the searchVideos function we just composed.

Here is the main App component:

class App extends Component {
  constructor(props) {
    super(props);
    // Keep the results in app state
    this.state = { results: [] };
  }
  render() {
    return (
      <VideoSearch
        onSearch={(term) => {
          // Returns type `Task Error [Videos]`
          searchVideos(term).fork(
            // If there is an error, just log it.
            (e) => console.error(e),

            // Otherwise, update app state.
            (results) => this.setState({ results })
          )
        }}
        results={this.state.results}/>
    );
  }
}

And here is the less interesting VideoSearch component.

const VideoSearch = ({ onSearch, results }) => (
  <div>
    <input placeholder="Search for videos"
           onChange={ ({ target: t }) => onSearch(t.value) }/>
    { videos.length === 0
        ? <div>No videos found</div>;
        : <div>
            { results.map(v => (
                <div key={v.id}>
                  <Embed video={v}/>
                </div>
            )) }
          </div>
    }
  </div>
);

The main thing to note is that the both the App and VideoSearch components are very simple. All of the complexity is delegated to the searchVideos function, which as we’ve seen, is made up of a bunch of smaller functions.

Dealing with changing requirements

Let’s consider a new requirement: We want to display a message to the user if no search term is entered.

To implement this, we will need a way to determine if the search has not been performed. We can use Either to achieve this, and change the searchVideos function to return Task Error (Either Empty [Video]) instead of Task Error [Video].

Either a b means the structure may contain a value of type a (Left) or type b (Right). The Left value represents an exceptional value, which in our case we will use to represent an empty search state.

// Left value means the search has not been performed, and the default message is contained in the object.
const Empty = Either.Left('Search for YouTube videos');

/* Usage example */

// This is a function that maps Videos inside an Either object to their titles.
// mapTitle :: Either l [Video] -> Either l [String]
const mapTitle = lift(map(prop('title')));

// Right value means we have performed the search and the results are available.
const s1 = Either.Right([
  Video('1', '...', '...', 'Led Zeppelin - Stairway to Heaven Live (HD)')
]);

mapTitle(s1); //=> Either.Right(Led Zeppelin - Stairway to Heaven Live (HD))
mapTitle(Empty); //=> Either.Left('Search for YouTube videos')

Handling the case of not having an URL

Now, to determine whether a search can be performed in the first place, we can create a new function maybeMakeUrl :: String -> Maybe Url. Note that we are returning a Url in the context of a Maybe.

Maybe can be of type Maybe.Just or Maybe.Nothing, with the latter representing the case of no value. (Note how we avoided using null or undefined)

// maybeMakeUrl :: String -> Maybe Url
const maybeMakeUrl = (term) =>  (
  term && term.length > 0
    ? Maybe.Just(makeUrl(term))
    : Maybe.Nothing()
);

/* Usage example */
maybeMakeUrl(null); //=> Maybe.Nothing
maybeMakeUrl(''); //=> Maybe.Nothing
maybeMakeUrl('foo'); //=> Maybe.Just(https://www.googleapis.com/youtube/v3/search/?q=foo&...)

HTTP GET using a Maybe URL

So now that we have a function that returns Maybe Url, we also need a new version of httpGet that handles Maybe Url instead of just Url.

The Maybe type provides a cata method that will allow us to pattern match whether we have Just Url or Nothing (think of it as using x instanceof Maybe.Just and x instanceof Maybe.Nothing).

// maybeHttpGet :: e -> Maybe String -> Task Error (Either e JSON)
const maybeHttpGet = (e) => (url) =>
  url.cata({
    Just: compose(lift(Either.Right), httpGet),
    Nothing: () => Task.of(e)
  });

In the Just case, we are using the lift function to transform the ResponseJSON object inside the Task to an Either.Right value.

In the Nothing case, we are returning a Task that contains the empty type e (Empty in our case). This is similar to using Promise.resolve(...), but for Tasks.

Note: The cata method uses a concept called Catamorphism (from Greek components "down" + "shape"). It provides a generalized way of collapsing a type based on its structure.

If you want to learn more about catamorphism, I found the Introduction to recursive types post of F# for fun and profit to be a good read.

Refactoring the search videos function

Now that we have the two new functions maybeMakeUrl and maybeHttpGet, we can change searchVideos to the following.

// searchVideos :: String -> Task Error (Either String [Video])
const searchVideos = compose(
  lift(lift(toVideos)),
  maybeHttpGet(Empty),
  maybeMakeUrl
);

Notice that all other functions remain unchanged (makeUrl, httpGet, toVideo, toVideos). We simply introduced two new functions, and refactored the final search function to handle Maybe and Either.

The lift(lift(...)) call is made because we need to lift the toVideos transform function twice:

Once for lifting the transform to operate with Either Empty ResponseJSON.
Second time for lifting the lifted transform to operate with the future resolved value of type ResponseJSON.

Refactoring the component to work with Either

And finally, we refactor the SearchVideos component to reduce the rendered element based on whether we have a right value (search performed), or a left value (search not performed).

results.cata({
  // This is the same as before, simply render the videos.
  Right: (videos) => {
    if (videos.length === 0) {
      return <div style={styles.fill}>No videos found</div>;
    } else {
      return (
        <div style={styles.results}>
          {
            videos.map(v => (
              <div key={v.id} style={styles.video}>
                <Embed video={v}/>
              </div>
            ))
          }
        </div>
      );
    }
  },
  // This is the new empty case, where we display the default message to the user.
  Left: (message) => <div>{ message }</div>
})

You can view the full source code by visiting the repository on GitHub.

The power of the functional approach is that what works in the small also works in the large. We started with a simple function makeUrl, and eventually ended up with the more powerful function searchVideos. Along the entire way we were using the same approach of thinking about input/output types, and function composition.

A nice benefit of having these pure functions is that we can easily use and test each of them separately. For example, the lift(lift(toVideos)) function can be called with input of type Task Error (Either e ResponseJSON).

const json = { items: [ {
               id: { videoId: '9Q7Vr3yQYWQ'},
               snippet: {
                 title: 'Led Zeppelin - Stairway to Heaven Live (HD)',
                 thumbnails: {
                    high: { url: 'https://i.ytimg.com/vi/9Q7Vr3yQYWQ/hqdefault.jpg' }
                 }
               } } ] };
const t1 = Task.of(Either.Right(json)); // input
const t2 = lift(lift(toVideos))(t1); // output

// Getting Video array out of the output task.
const either = t2.fork(identity, identity);
const videos = either.cata({ Right: identity });

videos[0].title; //=> 'Led Zeppelin - Stairway to Heaven Live (HD)'

Summary

In this post, I presented the little idea behind functional programming in terms of three concepts.

Data in, data out - separating data and behaviour
Code as data - using functions as data
Composition all the way down - composing simple functions to form more powerful functions that are composable themselves

These three basic concepts will help you keep your application more modular by allowing you to mix and match functions together (like a Lego!) as long as their types match up. This functional approach is something that works both in the small and in the large.

If you are interested in learning more functional programming, I’ve included some videos below that were influential to me.

Further Resources

View the final source code for the example on GitHub.
Ramda - Functional utility library for JavaScript
[Video] Check out Classroom Coding with Prof. Frisby for more functional programming with ReactJS.
[Video] Functional Programming for OO Development - Jessica Kerr
[Video] Destructuring Functional Programming - Gilad Bracha
[Video] Functional Programming from First Principles - Erik Meijer
[Video] Functional Programming Design Patterns - Scott Wlaschin
[Video] Railway oriented programming: Error handling in functional languages - Scott Wlaschin

Managing Side Effects In React + Redux Using Sagas

Sun, 03 Jan 2016 00:00:00 -0500

Redux is a Flux-like framework that has exploded in popularity within the React community. It reduces complexity by enforcing a unidirectional data flow, the use of single state atom, and pure reduce functions for state updates.

For me, there has always been one thorn in the React+Flux setup, which is that more complicated processes involving coordination of action creators and side-effects are hard to handle. Solutions using component lifecycle methods (componentDidUpdate, componentWillUpdate, etc.), and action creators returning thunks do work, but they seem out of place in certain situations.

To illustrate what I mean, let’s take a look at a simple Timer app. Note, the source code of the full solution can be found here.

The Timer App

This app will allow users to start and stop a timer, as well as provide the ability to reset it.

We can think of the app as a finite state machine that transitions between two states: Stopped and Running (as shown in the simplified diagram below). While the timer is in Running state, it will update the timer every 1 second.

So let’s go over the basic app setup first, then I’ll show how sagas can help manage side-effects using processes outside of action creators and components.

Actions

There are four actions in our module.

START - Transitions the timer to Running state.
TICK - Increments the timer on each tick.
STOP - Transitions the timer to Stopped state.
RESET - Resets the timer back to zero.

// actions.js

export default { start: () => ({ type: 'START' })
               , tick: () => ({ type: 'TICK' })
               , stop: () => ({ type: 'STOP' })
               , reset: () => ({ type: 'RESET' })
               };

State Model and Reducer

The timer state consists of two properties: status and seconds.

type Model = {
  status: string;
  seconds: number;
}

Where status is one of Running or Stopped, and seconds is the accumulated seconds count since the timer has started.

Then the reducer is implemented as follows.

// reducer.js

const INITIAL_STATE = { status: 'Stopped'
                      , seconds: 0
                      };

export default (state = INITIAL_STATE, action = null) => {
  switch (action.type) {
    case 'START':
      return { ...state, status: 'Running' };
    case 'STOP':
      return { ...state, status: 'Stopped' };
    case 'TICK':
      return { ...state, seconds: state.seconds + 1 };
    case 'RESET':
      return { ...state, seconds: 0 };
    default:
      return state;
  }
};

Timer View

The view is straight-forward if we keep it side-effect free. It renders the current time and status, and also invokes corresponding callback functions when the user clicks on Reset, Start, or Stop buttons.

export const Timer = ({ start, stop, reset, state }) => (
  <div>
    <p>
      { getFormattedTime(state) } ({ state.status })
    </p>
    <button
      disabled={state.status === 'Running'}
      onClick={() => reset()}>
      Reset
    </button>
    <button
      disabled={state.status === 'Running'}
      onClick={() => start()}>
      Start
    </button>
    <button
      disabled={state.status === 'Stopped'}
      onClick={stop}>
      Stop
    </button>
  </div>
);

Problem: How to Handle Periodic Updates?

The app can now transition between Running and Stopped states, but there isn’t a mechanism for scheduling periodic updates to the timer.

In a typical Redux+React app, there are two ways we can handle the updates.

The view can call the action creator periodically.
The action creator can return a thunk that will dispatch TICK actions periodically.

Solution 1: Letting View Dispatch Updates

For #1, the view will have to wait for state to transition from Stopped to Started, and then start the periodic dispatches. This means we have to use a stateful component.

class Timer extends Component {
  componentWillReceiveProps(nextProps) {
    const { state: { status: currStatus } } = this.props;
    const { state: { status: nextStatus } } = nextProps;

    if (currState === 'Stopped' && nextState === 'Running') {
      this._startTimer();
    } else if (currState === 'Running' && nextState === 'Stopped') {
      this._stopTimer();
    }
  }

  _startTimer() {
    this._intervalId = setInterval(() => {
        this.props.tick();
    }, 1000);
  }

  _stopTimer() {
    clearInterval(this._intervalId);
  }

  // ...
}

This does work, but it makes our view stateful and impure. Another problem is that our component is now responsible for more than just rendering HTML and capturing user interaction. It now causes side-effects, which makes the view and application as a whole harder to reason about. This may not be a huge deal in a small app such as this one, but in a larger application you want to keep side-effects at the boundaries of the system.

So what about using thunks?

Solution 2: Using Thunks in Action Creator

An alternative to the view approach is to use thunks in our action creator. We can change the start action creator to the following.

export default {
  start: () => (
    (dispatch, getState) => {
      // This transitions state to Running
      dispatch({ type: 'START' });

      // Check every 1 second if we are still Running.
      // If so, then dispatch a `TICK`, otherwise stop
      // the timer.
      const intervalId = setInterval(() => {
        const { status } = getState();

        if (status === 'Running') {
          dispatch({ type: 'TICK' });
        } else {
          clearInterval(intervalId);
        }
      }, 1000);
    }
  )
  // ...
};

The start action creator will now dispatch a START action as soon as it is invoked. Then, a TICK action is dispatched every 1 second, as long as the timer is still Running.

The issue I have with this approach is that the action creator is doing too much. It is also harder to test this action creator because it is no longer just returning data.

Better Solution: Using Sagas to Manage the Timer

The project redux-saga reifies side effects into artifacts called Effects. The Effects can be generated by another artifact called Sagas. The concept of sagas, as far as I know, comes from the world of CQRS and Event Sourcing. There are some debates on what sagas are, but you can think of them as long-live processes that interacts with the system by:

Reacting to actions dispatched in the system.
Dispatches new actions into the system.
Can “wake itself” using internal mechanisms without actions being dispatched. e.g. waking up on interval

In redux-saga, a saga is a generator function that can run indefinitely inside the system. It can be woken up when a specific action is dispatched. It can dispatch additional actions, and has access to the application state atom.

For example, if we want to dispatch periodic TICKs whenever the timer is Running, we can do the following.

function* runTimer(getState) {
  // The sagasMiddleware will start running this generator.

  // Wake up when user starts timer.
  while(yield take('START')) {
    while(true) {
      // This side effect is not run yet, so it can be treated
      // as data, making it easier to test if needed.
      yield call(wait, ONE_SECOND);

      // Check if the timer is still running.
      // If so, then dispatch a TICK.
      if (getState().status === 'Running') {
        yield put(actions.tick());
      // Otherwise, go idle until user starts the timer again.
      } else {
        break;
      }
    }
  }
}

As you can see, a saga uses normal JavaScript control flow constructs to coordinate side-effects and action creators. The take function wakes the saga up when the START action is dispatched. The put function causes the new TICK actions to be dispatched. And the call function allows us model the wait effect in a structure that does not cause it to run – similar to a Task.

By using a saga, we are able to keep our view and action creators as pure functions. It also allows us to model state transitions using familiar JavaScript constructs.

Wrap Up

Sagas are a way to manage side-effects within the system. They are a good fit when you need a long-running process that coordinates multiple action creators and side-effects.

Sagas react to actions, as well as internal mechanisms (e.g time-based effects). They are especially useful when you need to manage side-effects outside of the normal Flux workflow. For example, a user interaction could lead to further actions that do not require further interaction from the user.

Lastly, whenever you can model your solution as a finite state machine, sagas are worth a try.

If you want to see the full source code of the Timer app, see the repository here.

Have you tried using sagas yet? What are your thoughts?

Avoid Unnecessary Indirection

Sat, 21 Nov 2015 00:00:00 -0500

In programming, indirection is the ability to hold references to something, as opposed to the value itself. In object-oriented programming, indirection is used for dynamic dispatch and delegation.

For example, we can use delegation as follows.

class Delegator {
  constructor(delegatee) { this.delegatee = delegatee; }

  doSomething() { return this.delegatee.doSomething(); }
}

const a = {
  doSomething() { console.log('Doing something...'); }
};

const b = new Delegator(a);
b.doSomething(); // Delegates to `a`

This allows us to change the behaviour of a.doSomething depending on the late-bound object b.

While indirection is indeed a powerful tool, we should be careful not to overuse it. I will show what I mean through an example that I will refactor in steps. Each refactor will strife to make the code cleaner and more maintainable. And finally, I will do a comparison of the before and after, and see why the “after” may not be the best.

Example: Traffic Lights

Consider a class that controls traffic lights and their various states.

class TrafficLight {
  constructor(initialState = 'stop') {
    this.state = initialState;
  }

  changeTo(state) {
    this.state = state;
  }

  signal() {
    switch (this.state) {
      case 'stop':
        this.turnOnLamp('red');
        break;
      case 'caution':
        this.turnOnLamp('yellow');
        this.ringWarningBell();
        break;
      case 'proceed':
        this.turnOnLamp('green');
    }
  }

  nextState() {
    switch (this.state) {
      case 'stop':
        return 'caution';
      case 'caution':
        return 'proceed';
      case 'proceed':
        return 'stop';
    }
  }

  turnOnLamp(color) {
    console.log(`Turning on ${color} lamp`);
  }

  ringWarningBell() {
    console.log('Ring ring ring!');
  }
}

(Example adapted from one in the Confident Ruby book.)

The TrafficLight class has four main responsiblities.

Keeps track of the current state.
Can switch to another state using changeTo(state).
Can signal() to turn on different lamps, and in some cases may ring a warning bell.
Knows what the nextState() is given the current state.

Here is an example of usage.

const light = new TrafficLight();

light.state; // 'stop'

light.signal();
// Log: Turning on red lamp

light.nextState(); // 'caution'

light.changeTo('caution');
light.signal();
// Log: Turning on yellow lamp
// Log: Ring ring ring!

The first code smell is that there are switch statements inside signal and nextState. Why is switch a smell?

Firstly, it makes the method behaviours static, therefore inflexible. This means that if we ever needed to change their behaviours then we will have to add more case blocks, which also means that we cannot dynamically add or change cases during runtime.

Secondly, the structure of both switch statements look awfully similar. My gut tells me there is a new abstraction that is screaming to come out.

Let’s refactor!

Refactor #1: Eliminate Switch Statements

The insight here is that we can group the color and next state data into state objects. Therefore, the responsiblity of which colored lamp to turn on and the next state is removed from TrafficLight (through delegation).

const States = {
  STOP: {
    color: 'red',
    get next() {
      return States.CAUTION;
    }
  },
  CAUTION: {
    color: 'yellow',
    get next() {
      return States.PROCEED;
    }
  },
  PROCEED: {
    color: 'green',
    get next() {
      return States.STOP;
    }
  }
};

class TrafficLight {
  constructor(initialState = States.STOP) {
    this.state = initialState;
  }

  signal() {
    this.turnOnLamp(this.state.color);

    // Still need a special case for ringing warning bell on `caution` state.
    if (this.state === States.CAUTION) {
      this.ringWarningBell();
    }
  }

  nextState() {
    return this.state.next;
  }

  // ...
}

Looking good so far.

Now, the second code smell is the special case of calling ringWarningBell() when we are signaling States.CAUTION. Let’s create a State base class that will handle the special signal cases through polymorphism.

Refactor #2: More Delegation!

The idea here is that we can move all of the signalling logic to the State class. This way we can take further advantage of delegation and keep our TrafficLight class nice and clean.

// Base class that provides a default `signal` method.
class State {
  signal(trafficLight) {
    trafficLight.turnOnLamp(this.color);
  }
}

class Stop extends State {
  get color() { return 'red'; }
  get next() { return States.PROCEED; }
}

class Caution extends State {
  get color() { return 'yellow'; }
  get next() { return States.STOP; }

  // Special case is now fully encapsulated in the Caution class.
  signal(trafficLight) {
    super.signal(trafficLight);
    trafficLight.ringWarningBell();
  }
}

class Proceed extends State {
  get color() { return 'green'; }
  get next() { return States.CAUTION; }
}

const States = {
  STOP: new Stop(),
  CAUTION: new Caution(),
  PROCEED: new Proceed()
};

class TrafficLight {
  // ...

  signal() {
    this.state.signal(this);
  }

  // ...
}

The refactoring so far has reduced signal from 11 LOC to 1, and nextState from 8 down to 1 LOC. Another cool part is that a new domain concept has been capture via a new class State. Looks good to me!

Or is it?

Too Much Indirection

If you compare before and after of the TrafficLight class, I would argue that the former is much easier to reason about.

Reasonability of code is an often discussed topic in programming. But what does it actually mean?

Code is easy to reason about if its output can be easily predicted given some input. That is, I can predict what happens without having to expend a lot of effort tracing through it.

For example, referentially transparent functions with immutable objects are easier to reason about because I don’t need to run through all possible mutations that can occur in the system. I know that given the same input, I will always get the same output. I also know that the objects I pass into the function cannot be mutated, leading to behaviour changes.

Looking at the signal() method from the initial example, I can tell you without much effort what it will do given the different states. Can I say the same with the final example?

Scenario: What Happens When I Signal?

Let’s step through the same scenario with the initial code, and the final, refactored code.

Initial state is caution (or States.CAUTION).
I call light.signal().
What happens?

Let’s take a look at the initial method in question. Look for the numbers attached to each step below.

class TrafficLight {
  signal() {
    // 1. I know `this.state` is 'caution'...

    switch (this.state) {
      case 'stop':
        this.turnOnLamp('red');
        break;

      // 2. Ah! Here it is.
      case 'caution':

        // 3. Okay, it turns the yellow lamp.
        this.turnOnLamp('yellow');

        // 4. Oh, it rings the warning bell.
        this.ringWarningBell();

        // 5. Case is done.
        break;

      case 'proceed':
        this.turnOnLamp('green');
    }

    // 6. Done!
  }
}

Constrast this with the final refactored code. The step numbers are out of order and might be hard to follow, but this is what happens with indirection in real code. ;)

class TrafficLight {
  // ...

  signal() {
    // 1. I know `this.state` is `States.CAUTION`...

    // 2. I need to look up what `States.CAUTION.signal` does
    this.state.signal(this);

    // 10. Returned from `signal` and I'm done!
  }

  // ...
}

const States = {
  // ...

  // 3. Now I need to look up what Caution class does.
  CAUTION: new Caution()

  // ...
};

class Caution extends State {
  // 7. Oh, the yellow lamp will turn on!
  get color() { return 'yellow'; }

  // ...

  // 4. Ah, here is the signal method!
  signal(trafficLight) {
    //  5. Okay, what does this do? Let me look at State parent class.
    super.signal(trafficLight);

    // ...

    // 8. Now it will ring the warning bell.
    trafficLight.ringWarningBell();

    // 9. Method is done! Let me jump back to `TrafficLight`.
  }
}

class State {
  signal(trafficLight) {
    // 6. Ok, it turns the lamp on... but what color?
    //    I need to go back up the Caution class...
    trafficLight.turnOnLamp(this.color);
  }
}

The difference in scanning the initial example versus the final one is that the the former is completely contained in one method, and can be scanned from top to bottom. In the refactored code, we had to jump all over the place, and keep track of where we have jumped from.

Another thing to note is the the signal method of State is tightly coupled to the TrafficLight implementation, so we did not create any meaningful abstraction layers either.

Does this mean the refactored code is necessarily worse? No, I don’t think so. And it’s not necessarily better either. It really depends on the requirements and personal preference. The refactored code might capture the state transitions and colour requirements in a more cohesive manner. It might even increase the ease of extension.

One thing is for certain, the refactored code is more complex than the original. What might look like clean code initially (small functions/methods, no switch statements, etc.) might not be desirable.

Identifying the Necessity of Indirection

Here are a couple of ways to help identify unnecessary or bad indirection.

Very Tightly Coupled Objects

If you notice that some objects always change together, then it might be a sign that you got the abstractions wrong the first time around. This can be done by analyzing git log for example.

Sometimes for the sake of keeping code clean and DRY, we introduce the wrong abstractions early on. These bad abstractions can lead to bugs, and make our code very hard to reason with. If it often better to leave code duplication alone until you arrive at the insight to refactor properly.

Too Many Object Interactions

A good abstraction allows you to focus in on the task at hand, without complecting it with all the other things going on in the system. So if you find yourself constantly having to load a bunch of objects in your head in order to reason about the system, then it might be another sign that something is amiss.

Acting On Bad Indirection

Once you’ve identified bad indirection, the best course of action is usually to undo the damage and inline the code. Some editors will help you inline your functions, variables, etc., with a quick command. Once inlined, it is much easier to see the bigger picture.

You can then decide whether you want to refactor further by creating better abstractions.

Conclusion

Indirection is a valuable tool. When used correctly, it allows us to create powerful abstractions that allow us to extend and maintain our system more easily. However, like most things of value, indirection comes at a cost. It increases complexity.

Does this mean we shouldn’t use indirection? Of course not! You will not get very far without it. You should think a bit before refactoring, and make sure you are actually making your code better.

If you end up with unnecessary indirection, don’t be afraid to inline your code again. This way you can more easily reason about your system as a whole. You may then get the insight needed for your next refactor.

Happy coding!

Edit (2015-12-13): Fixed typos in code example.

Jack Hsu

The Reader monad and read-only context

Motivation behind context

Meet the Reader

But what else can Reader hold?

Chaining the Reader context

But what is a monad exactly?

Why didn’t you just ask?

Assembling all the pieces

Summary

Deconstructing the React Component

React from first principles

Look at my shiny new box!

Formalizing higher-order component concepts

Mapping over inputs

No need for HOCs

Quick Recap

Combining multiple boxes

Summary

Don't Fear the Type System

The power of a type system

Null References: The Billion Dollar Mistake

Whose job is it to prevent defects?

Constraints liberate. Liberties constrain.

Contraints a la carte

Expressivity of a type system

Sealed classes

Exhaustive pattern matching

Closing

Suggested Resources

Additional Guidelines For (Redux) Project Structure

What to do with common components?

Introducing a separate components folder

Using a core module for common concerns

Exporting and testing connected components

Normalizing application state

Reducing to normalized states

Closing

Why You Need Types

But Why Worry About Types At All?

Types To The Rescue!

Getting Rid Of That Dreaded Null error

Aren’t Types Too Constraining and Troublesome?

Adding Types To Your Project Today!

Setting Up Flow

Gradually Adding Types

Wrapping Up

Further Reading

Three Rules For Structuring (Redux) Applications

Three rules for project structure

Rule #1: Organize by feature

Rule #2: Create strict module boundaries

Rule #3: Avoid circular dependencies

Circular dependency example

Don’t make hairballs!

Litmus test for project structure

Summary

The Anatomy Of A React & Redux Module (Applying The Three Rules)

In-depth example and recommendations

Module index and constants

Actions & Action creators

Model

Reducers

Selectors

Components

Other responsibilities

Summary

The Little Idea of Functional Programming

Implementing YouTube instant search

The Little Idea

Concept #1 - Data in, data out

Code - Mapping search term to URL

Code - Hash to Video

Concept #2 - Code as data

Concept #3 - Composition all the way down

Code - Mapping JSON response to videos

Building a function to search videos

Wiring up the UI

Dealing with changing requirements

Handling the case of not having an URL