It’s very typical when a start of the journey is frustrating, and there is nothing better than the advice from the people who already been there.

I never thought about that question carefully, but I formulated something quick for him. But later I decided to elaborate on it and share it with a bigger audience.

There Is No Recipe

The thing is – there is no just simple recipe. If there is, I guess it would sound like that – “practice, practice, practice”.

It’s not a secret that you have to dedicate much time to your craft. And I really mean a lot.

If you are starting your career and be lucky enough to have a job, you have to come back home and work on side projects. If you are a student, you come back after the school and work on side projects, watch courses and read good books.

Through my career, I realized, it were the side projects which contributed to my experience the most.

Here is the thing, at the workplace you have to perform. At the work, you have a lot of constraints – existing codebase, architecture, set of libraries and frameworks, the team rules, etc.

It’s not always an educational environment at the workplace. You have to solve business goals, not primarily to educate yourself. Rarely, both things are coming in parallel, but it’s rather an exceptional case.

So, prepare yourself mentally to work a lot and spend evenings with your IDE opened, instead of watching a favorite series on Netflix or going out (even thought, it’s also very important to do from time to time).

JavaScript Foundation

JavaScript is the assembly language of the web. Without a proper understanding of the language, you have nothing to do in the front-end world.

The good news is, you don’t have to be Douglas Crockford to be a successful developer. I’ll tell you what – I’ve spent quite sometime in the profession, but details of language opened up to me way later. It’s totally normal if you don’t understand the depths of language. It will come.

However, there a fundamental parts that you have to be very comfortable with:

• Data types
• Arithmetic operations
• Boolean and conditional operators
• Understanding of OOP paradigm (prototypical inheritance)
• Understanding of functional paradigm (functors, applicators, immutability)

I also recommend, to start learning Node.js directly.

All modern tools for front-end require Node.js. Developing simple applications and running than from command line is way more convenient comparing of dealing with the browser.

Browsers and DOM

Besides that, you should be able to understand how browsers work and in particular understand the concept of the DOM.

• How DOM functions and it’s basic components
• Events, events propagation, and handlers
• DOM API, window object
• DOM selectors

Here you should be able to write simple, pure JavaScript applications by listens to controls and perform simple DOM manipulations, as setting values, classes, properties.

UI Development

In my career, I came from backend and desktop applications UI and the first sight HTML, and CSS seems to be easy. It’s not even a real programming language, just a set of declarations, right? Not really.

Even if your goal is not to be a designer, you should develop useful skills and a taste for good design. So the best thing to do is to steal cool ideas.

No kidding, just pick up the site you like the most and try to copy it. Remember, the famous guy said – «Un artista copia, un gran artista roba» (Good artists copy, great artists steal).

Make your hands dirty with HTML & CSS.

Be fluent in creating, simple and good looking interfaces. Find a freelance job of turning PSD or Sketch into HTML / CSS. You will get exceptionally important skills.

Client / Server Communication

Browser applications don’t exist in isolation. They do talk to servers to get the data and push data back.

Without going too crazy, you will hugely benefit from understanding the following things:

• Basics of the HTTP protocol
• The concept of REST (HTTP verbs and resources)
• fetch API
• Optionally: the concept of GraphQL and how it differs from REST

There are tons of open API’s you can do cool stuff with. Pick one and build the simple application against it.

I remember one of the projects I did alexbeletsky/github-commits-widget. A simple app that connects to Github API and makes a widget of contributions to a project. I had a lot of fun with it.

Development Frameworks

And only after the first three parts, which on its own might take a year, or even more I would suggest you take a look at some UI development framework.

The JS ecosystem is famous to create “fatigue”. Tools are complex, configurations are confusing, don’t start with React or Angular, just because you heard that they are cool.

Maybe till the time you are comfortable with JS/HTML/CSS those things are dead already (front-end development framework has a tendency to survive for 3-5 years and then entirely overtaken by competitors.. don’t bother with that race).

However, if you are reading the article 2018, I would recommend you to focus on React.js. The ecosystem is stable, million of tutorials, open source projects, meetups. You won’t feel the lack of information, that’s for sure.

Having all three parts behind your shoulders will make you confident with any front-end frameworks because all of them are designed to simplify complicated things or making things more performant. The foundation of development, as DOM manipulations, HTML/CSS would remain the same.

Even if the framework could encapsulate any particular detail, you will be confident enough to understand, what’s actually going on there.

At the end I would just share the list of resources that helped or influenced me during my own journey:

• JavaScript Jabber Podcast
• Eloquent JavaScript Book
• JavaScript: The Good Parts
• React Podcast
• EggHead Video Courses

Hope you find that helpful and I wish you the best in this interesting and exciting journey!

Enjoy the ride.

Originally published on Medium in Product & Engineering blogfoster blog

For web-based products, measurements typically mean collecting of different user behavior metrics. Based on these metrics we can see, what users are doing in the application, their behavior has a direct influence on a product.

Many projects I participated, had “Add Analytics” as a bottom-most task in development backlog. Ignoring the task, made it hard to implement later stages. You need something like analytics.trackEvent('something happened', { data: 123 }); in many places, making code looks ugly and quite easy to miss some important details.

Redux

In my previous article, I already mentioned the Redux similarities to Event Sourcing architecture paradigm. One of the most interesting side-effects of Event Sourcing is Audit Log.

Audit Log gives you an ability to look back and see what happened. As the original idea of Redux, it allows you travel back in time.

If you follow Redux rules, everything that happens in your application is represented as an action. In terms of analytics, it means you already have all events defined, you don’t need to find the place where to inject analytics.trackEvent() code.

Take a look into this code,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import { createConstants } from '../utils';

const constants = createConstants(
  // account actions
  'ACCOUNT_FORM_RESET',
  'ACCOUNT_FORM_CHANGE',
  'ACCOUNT_FORM_ERRORS',
  'ACCOUNT_FORM_ERRORS_CLEAN',
  'ACCOUNT_ACCESS_TOKEN_FETCHING',
  'ACCOUNT_ACCESS_TOKEN_FETCHED',
  'ACCOUNT_ACCESS_TOKEN_FETCH_FAILED',
  'ACCOUNT_EMAIL_CONFIRMED',
  'ACCOUNT_EMAIL_CONFIRMATION_FAILED',

  // create website actions
  'CREATE_WEBSITE_BLOG_URL_CHANGED',
  'CREATE_WEBSITE_BLOG_TITLE_CHANGED',
  'CREATE_WEBSITE_BLOG_TITLE_RESOLVE_FAILED',
  'CREATE_WEBSITE_STATE_LOADING',
  'CREATE_WEBSITE_STATE_LOADED',
  'CREATE_WEBSITE_WIZARD_STEP_NEXT',
  'CREATE_WEBSITE_WIZARD_STEP_PREV',

  // ...

Those are constants for action types. Our goal is to turn those action types, into meaningful analytics events.

Analytics Server

At blogfoster, we are using custom analytics solution. Essentially it’s an HTTP API, which receives an event and stores and distributes it for further processing. It makes it easy from integration perspective both for the web and mobile clients.

Even the back-end is quite complex there, the interface is rather simple,

1
2
3
4
5
6

HTTP POST /v1/event

{
  eventType,
  payload
}

Middleware

If you have worked with Express.js or Koa.js frameworks, then you should already be familiar with a middleware concept. Middlewares are functions that could be injected into response-request pipeline and designed to produce different side-effects (e.g. request modification, logging, redirections, performance measurements, etc.)

Redux introduces the concept of middleware as well, which makes library not only flexible but applicable for real-world applications. We know that Redux is essentially all about synchronous workflow, but the nature of web applications is asynchronous (events, client/server communications) and this is where middleware shines.

The API of Redux middleware is pretty elegant; it’s not hard to implement own functions. If we inject the function, that can analyze the action and send data to analytics server, we will reach our goal, without touching any application logic.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

import { client } from '../analytics';

const handleAction = (store, next, action, options) => {
  if (!action.meta || !action.meta.analytics) {
    return next(action);
  }

  const { eventType, eventPayload } = action.meta.analytics;

  client(options).track(eventType, eventPayload);

  return next(action);
};

export function createAnalytics(options = {}) {
  return store => next => action => handleAction(store, next, action, options);
}

As you can see, the handleAction() function checks, if there is meta data related to analytics exists in action, it will pass it to client that would when to make HTTP request to analytics server.

Configuring store

We need to apply middleware function now, it’s done via applyMiddleware() Redux helper,

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import { createStore, applyMiddleware } from 'redux';
import { createAnalytics } from '../middleware';

const middlewares = [
  thunk,
  createLogger(),
  createAnalytics({ host: 'ANALYTICS_HOST', token: 'ANALYTICS_TOKEN' })
];

const enchancer = applyMiddleware(...middlewares);

export default function configureStore(rootReducer, initialState) {
  return createStore(rootReducer, initialState, enchancer)
}

Augmenting Actions

I came up with a small utility function, to minimize modification of existing action creators as much as possible,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

import constants from '../../constants';

export function loadedDashboardState(website, status, news, revenues) {
  return trackable({
      type: constants.DASHBOARD_STATE_LOADED,
      payload: { website, status, news, revenues }
    },
      'Dashboard opened',      // <- event name
      { name: website.name }   // <- additional event data
    );
}

export function loadingDashboardStateFailed(err, resp) {
  return trackable({
      type: constants.DASHBOARD_STATE_LOADING_FAILED,
      payload: { err, resp }
    },
      'Dashboard loading failed',
      { error: resp.text }
    );
}

Where trackable(),

1
2
3
4
5
6
7
8

export function trackable(action, event, properties = {}) {
  const analytics = {
    eventType: event,
    eventPayload: properties
  };

  return { ...action, meta: { analytics } };
}

That’s it. Now go through all action creators and wrap actions you need to log into trackable() function, so they became trackable, and middleware would send them to the analitycs server.

Recap

Even if you don’t think about analytics from the very beginning, Redux architecture allows you to plug it in whenever you want (or need it). Based on Audit Log you can watch and turn actions into corresponding analytics events. Middleware functions would allow to hook all actions and turn them into analytical events.

PS. During my writing time, developers from Rangle.io participated React-Europe conference and gave a nice talk about the topic above. So, I would recommend you to check it out for further details.

Originally published at blogfoster Engineering.

Higher-order components concept goes back to higher-order functions, functional programming concept, describing the function that takes other function(s) and returns a function. In exactly the same way, higher-order component takes another component(s) and return a component.

Where might it be useful? Let’s consider a few practical cases.

Authentication and Authorization

Web applications have different areas, which are accessible depending on user’s authentication and access control.

There are guest routes, like /signup, /signin, /forgot-password etc, as well as private routes, /dashboard, /user-profile and so on.

For example,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

const routes = (
  <Route path="">
    <Route path="/signup" component={SignUp} />
    <Route path="/signin" component={SignIn} />
    <Route path="/welcome-on-board" component={Welcome} />
    <Route path="/forgot-password" component={ForgotPassword} />
    <Route path="/signout" component={SignOut} />

    <Route path="/" component={App}>
      <Route name="dashboard" path="dashboard" component={Dashboard} />
      <Route name="profile" path="user-profile" component={UserProfile} />
    </Route>

    <Route path="*" component={FourOFour} status={404} />
  </Route>
);

What we want to do: Everything under / URL is protected from unauthorized access.

One solution could be to extend App itself, with componentDidMount and componentDidUpdate and check if the user is authenticated. If not, redirect the user to /signin. It sounds right from the very beginning, but this solution has two drawbacks.

Firstly, it violates the single responsibility principle, what now mixes up functionality to App component, that originally doesn’t belong to it. And secondly, in case we want to introduce another route with similar behavior, that logic has to be present there as well, that would lead to some code duplications.

Let’s introduce a higher-order component for that,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

import React, { PropTypes } from 'react';
import { connect } from 'react-redux';
import { push } from 'react-router-redux';

export default function requiresAuth(Component) {
  class AuthenticatedComponent extends React.Component {
    static propTypes = {
      user: PropTypes.object,
      dispatch: PropTypes.func.isRequired
    };

    componentDidMount() {
      this._checkAndRedirect();
    }

    componentDidUpdate() {
      this._checkAndRedirect();
    }

    _checkAndRedirect() {
      const { dispatch } = this.props;

      if (!this.props.user) {
        dispatch(push('/signin'));
      }
    }

    render() {
      return (
        <div className="authenticated">
          { this.props.user ? <Component {...this.props} /> : null }
        </div>
      );
    }
  }

  const mapStateToProps = (state) => {
    return {
      user: state.account.user
    };
  };

  return connect(mapStateToProps)(AuthenticatedComponent);
}

We are using Redux here, but in general, the concept doesn’t depend on Redux at all.

So, requiresAuth is a function that takes Component and returns AuthenticatedComponent. AuthenticatedComponent wraps the original component, plus it checks if the user already authenticated, in case he is not, it will redirect to /signup.

Updated routing,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

const routes = (
  <Route path="">
    <Route path="/signup" component={SignUp} />
    <Route path="/signin" component={SignIn} />
    <Route path="/welcome-on-board" component={Welcome} />
    <Route path="/forgot-password" component={ForgotPassword} />
    <Route path="/signout" component={SignOut} />

    /* App component is wrapped with requiresAuth()  */
    <Route path="/app" component={requiresAuth(App)}>
      <Route name="dashboard" path="dashboard" component={Dashboard} />
      <Route name="profile" path="user-profile" component={UserProfile} />
    </Route>

    <Route path="/config" component={requiresAuth(Config)}>
      <Route name="page" path="page" component={PageConfig} />
    </Route>

    <Route path="*" component={FourOFour} status={404} />
  </Route>
);

By extending requiresAuth with additional parameters, it’s very easy to add some ACL functionality,

1
2
3
4
5
6
7
8

<Route path="/app" component={requiresAuth(App, { role: 'user', redirectTo: '/signin' })}>
  <Route name="dashboard" path="dashboard" component={Dashboard} />
  <Route name="invoices" path="user-profile" component={UserProfile} />
</Route>

<Route path="/config" component={requiresAuth(Config, { role: 'admin', redirectTo: '/app' })}>
    <Route name="page" path="page" component={PageConfig} />
</Route>

Now, _checkAndRedirect() function is modified to take a role into account,

1
2
3
4
5
6
7
8

_checkAndRedirect() {
  const { dispatch, user } = this.props;
  const { role, redirectTo }  = options;

  if (!user || !user.role === role) {
    dispatch(push(redirectTo || '/signin'));
  }
}

Redirections

Another case is a redirection of a user. A redirect is forced for some users in specific conditions. For instance, we might want a user to complete some actions during his onboarding, and prevent access to a particular route, if these actions are not completed.

Otherwise, if onboarding is completed, we want to redirect a user to the dashboard.

To make it possible, we introduce checkBoarding() function and a corresponding CompletedComponent.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

export default function checkBoarded(Component, { withValue, redirectTo }) {
  class CompletedComponent extends React.Component {
    static propTypes = {
      state: PropTypes.object.isRequired,
      dispatch: PropTypes.func.isRequired
    };

    componentDidMount() {
      this._checkAndRedirect();
    }

    componentDidUpdate() {
      this._checkAndRedirect();
    }

    _checkAndRedirect() {
      const { dispatch } = this.props;

      if (this._shouldRedirect()) {
        dispatch(push(redirectTo));
      }
    }

    _shouldRedirect() {
      return this.props.state.user.boarded === withValue;
    }

    render() {
      return (
        <div className="boarded">
          { !this._shouldRedirect() ? <Component {...this.props} /> : null }
        </div>
      );
    }
  }

  const mapStateToProps = (state) => {
    return {
      state: state.account
    };
  };

  return connect(mapStateToProps)(CompletedComponent);
}

Routing,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

const routes = (
  <Route path="">
    <Route path="/signup" component={SignUp} />
    <Route path="/signin" component={SignIn} />

    // applied here..
    <Route path="/welcome-on-board" component={checkBoarded(Welcome, { withValue: true, redirectTo: '/app/dashboard'}) } />

    <Route path="/forgot-password" component={ForgotPassword} />
    <Route path="/signout" component={SignOut} />

    // .. and here
    <Route path="/app" component={checkBoarded(requiresAuth(App), { withValue: false: redirectTo: '/welcome-on-board' })}>
      <Route name="dashboard" path="dashboard" component={Dashboard} />
      <Route name="profile" path="user-profile" component={UserProfile} />
    </Route>

    <Route path="/config" component={requiresAuth(Config)}>
      <Route name="page" path="page" component={PageConfig} />
    </Route>

    <Route path="*" component={FourOFour} status={404} />
  </Route>
);

Setting up context

Some routes (or more typically, group of routes), require some data context. Say, /website/1 and /website/1/banners, /websites/1/revenues all requires that website with id:1 is loaded and available as part of the state.

Instead of loading the same data in each component, we might use higher-order component.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

export default function websiteContext(Component) {
  class WebsiteContextComponent extends React.Component {
    static propTypes = {
      dispatch: PropTypes.func.isRequired,
      state: PropTypes.object.isRequire,
      params: PropTypes.shape({
        websiteId: PropTypes.string.isRequired
      })
    };

    componentDidMount() {
      this._setCurrentWebsite();
    }

    componentDidUpdate() {
      this._setCurrentWebsite();
    }

    _setCurrentWebsite() {
      const {
        dispatch,
        params: { websiteId },
        state: { websites }
      } = this.props;

      // prepare context data here..
      const website = _.find(websites, { id: websiteId });

      dispatch(loadContextWebsite(website));
    }

    render() {
      const { state: { website } } = this.props;

      return (
        <div className="website-context">
          { website ? <Component {...this.props} /> : null }
        </div>
      );
    }
  }

  const mapStateToProps = (state) => {
    return {
      state: state.application
    };
  };

  return connect(mapStateToProps)(WebsiteContextComponent);
}

Routing,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

const routes = (
  <Route path="">
    <Route path="/signup" component={SignUp} />
    <Route path="/signin" component={SignIn} />

    // applied here..
    <Route path="/welcome-on-board" component={checkBoarded(Welcome, { withValue: true, redirectTo: '/app/dashboard'}) } />

    <Route path="/forgot-password" component={ForgotPassword} />
    <Route path="/signout" component={SignOut} />

    // .. and here
    <Route path="/app" component={checkBoarded(requiresAuth(App), { withValue: false: redirectTo: '/welcome-on-board' })}>
      <Route name="dashboard" path="dashboard" component={Dashboard} />
      <Route name="profile" path="user-profile" component={UserProfile} />

      <Route name="websites" path="websites/:id" component={websiteContext(Websites)}>
        <Route name="banners" path="banners" component={Dashboard} />
        <Route name="revenues" path="revenues" component={Revenues} />
      </Route>

      </Route>
    </Route>

    <Route path="/config" component={requiresAuth(Config)}>
      <Route name="page" path="page" component={PageConfig} />
    </Route>

    <Route path="*" component={FourOFour} status={404} />
  </Route>
);

Compose all the things

We sometimes need to apply a few higher-order components on the same route. As from routing above, we can see that /app route both requiresAuth() and checkBoarded(). Sooner or later, you might have 3-4 function calls there; that makes Routes a bit bulky.

Fortunately, higher-order components are easily composable,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

const CheckedWelcome = composeComponents(
  Welcome,
  [
    (c) => checkBoarded(c, { withValue: true, redirectTo: '/app/dashboard' }),
    (c) => requiresAuth(c)
  ]
);

const CheckApp = composeComponents(
  App,
  [
    (c) => checkBoarded(c, { withValue: false, redirectTo: '/welcome-on-board'}),
    (c) => requiresAuth(c),
    (c) => requiresSmth(c, { options: {} })
  ]
);

const routes = (
  <Route path="">
    // much more clean route definition
    <Route path="/welcome-on-board" component={CheckedWelcome} />

    <Route path="/app" component={CheckedApp}>
      // ...
    </Route>
  </Route>
);

composeComponents() is a simple reducer utility function,

1
2
3

export function composeComponents(component, wrappers = []) {
  return wrappers.reduce((c, wrapper) => wrapper(c), component);
}

So, higher-order components is a great concept that aims to replace Mixins as composition mechanism. My experience showed they are great as containers wrappers, but in general, they have many practical applications.

Originally published at blogfoster Engineering.

Each update destroys previous value, there is no history what value had been stored a minute ago, an hour ago or 5 years ago.

Events and Immutability

Are there any alternatives, that allow mitigating that drawback?

One of the oldest professions in the world (not what you think) already have an answer for that. In accounting the only way to modify the balance is to introduce a credit or debit record.

If an accountant makes a mistake of placing $100 debit, she simply can’t erase an entry. Instead, she introduces a contra-operation, new entry that says $100 credit, that should compensate balance and return it to the previous state.

That principle makes a foundation for Event Sourcing architectural style. In Event Sourcing, there is an initial state and series of events, each event represents a change it introduces to the state. Every event is stored, and can not be deleted or modified, state is derived from events. Both are immutable.

The state of a whole system can be described with such formula:

1

Snext = F(S, event);

To get the state of the system for given moment of time, we need to take an initial state and replay all events that happened before.

UI as Function of State

An imperative approach is dominant now, and front-end applications are not the exception. We are using MVC / MVVM on a client, fetching data from a server, imperatively modifying it and storing back.

The state of UI is distributed all around, there is no central place, where it’s stored. It’s rather a mix of states in models, states in DOM with a bunch of side-effects.

What if we imagine such central place and the whole UI of our application, as a function of it?

1

UI = F(S);

Means, UI is a function of a state.

React is a library that gives the ability to treat UI like that. React represents the UI as a hierarchy of components, each of the components is a function, that takes some properties as input and returns a markup. The whole UI is a composition of such functions.

But how to deal with the state to avoid the problems we already had with an imperative approach to UI?

Predictable State

Dan Abramov, in his famous talk of Time Travel Debugging came to the idea, of bringing event sourcing principles of state handling to the client, solving one of the most painful issues.

Redux represents the state as,

1

Snext = R(S, action);

Combining it with UI part,

1
2

UI = F(S);
UI = F(R(state, action));

F – React component function, R – Reducer, a function that transforms the current state, based on action that happened in an application.

In Redux architecture, we have a store that holds the application state and provides an interface for changing that state, by dispatching actions. An Action holds it’s type as well as payload, which reducer uses to produce next state.

Like in Event Souring, actions are just plain objects, they can be logged, serialized, stored, and later replayed for debugging or testing purposes.

React with Redux

Even if Redux stresses it’s not designed to be used especially with React, React and Redux make a perfect couple.

React allows you to treat the UI as a function of the state, Redux allows you to manage that state in a predictable way. React-Redux binding connects React component to Redux state, and every time it changes, component gets new state and UI re-rendering triggers.

In other words, with each change, the whole UI is re-rendered. Because of React re-rendering is based on virtual DOM diff algorithms, complete re-render doesn’t bring huge performance penalties, comparing to other frameworks.

Unidirectional data flow makes it easy to understand what’s going on in complex interfaces. As I jumped in with Redux with-out considerable experience with React itself, I would say once state part as a problem is solved, React part becomes a simple one.

Originally published at blogfoster Engineering.

Very quickly we grew up from 0 to 1.000 users that gave us huge motivation to go on. Both of us had good-paying jobs at that time but we decided to follow our dream. It was a dream to build a company that makes our world a little bit better place. A company where you feel the meaning and purpose of life by solving complex problems and working on something you truly love.

Mark Zuckerberg once said: “Move fast and break things”. At Likestore we moved very fast, breaking nothing (well, almost nothing). It could be silly to talk about culture in the team of two but we had a great culture there, especially technical one. Without a doubt, Likeastore was the best project we worked so far.

Today we are shutting down.

Many of you have noticed the lack of progress in the latest months. Silently project started to die and we feel very sorry for that. Likeastore worked without any attention from engineering. We wanted to make it run as long as possible using its stability.

But let’s be honest to ourselves, without marketing and growth, without incredible user support and passion, every product is doomed. We were naive and energized but sooner or later you meet reality. Reality punched us in a face and taught some valuable lessons.

We want to say huge thanks for all our users and supporters. A lot of people helped this project, a lot of users shared their feedback, we really, really appreciate it!

PS. We decided to open-source everything we worked on for last 2 years, including application, synchronization engine, and many useful libraries, to say thanks to the OSS community and hopefully help other developers.

There are many things you need to consider, but primarily – business and technological risks. For companies, with an established customers base, value proposition and product, taking a risk of a massive rewrite is always very high. For startups, speed to market and responsiveness to change is a priority, so they are pivoting and rewriting products from scratch.

Our approach

At blogfoster, we are kind of in the middle. The young company, hitting product-market fit, having customers and making revenues. We thought about a combined approach. Enhance our existing front-end application, by substituting some pieces of it, using a technological stack for further growth and development speed.

We had a common opinion about the technology that we would like to pick up. Having a good awareness of which technologies are taking off now, we had the confidence to bet on them.

We agreed on the following: React for handling UI, Redux for managing state, Webpack as bundler, and ES2015 with Babel, as programming language. On the server side, Hapi.js for the REST API, MySQL and Sequilize as a data storage and access.

Since almost all of libraries and tools, mainly a front-end ones, mentioned above were new to us, our CTO Dominic, proposed an approach of starting up with developing of boilerplates both for new server and client, so we build our applications based on them.

The goal was to build some simple application, which would involve quite typical web applications operations, e.g. routing, rendering, fetching and sending data to a REST API. Since signin/signup is the entrance of all applications, it makes sense to implement it as a first prototype.

I was pretty sure, I need about two days on that.

It took me a week to get it in shape so that we’re confident to continue with it. React, Redux, Webpack, Babel are quite hard to combine, if you are doing it at the first time.

Honestly, I had a lot of frustration with the boilerplate implementation. On the way, I seriously thought a few times if it is the right approach we are going to take.

Webpack

Webpack appeared to be a huge piece of software, with very unintuitive API. A configuration of Webpack is more difficult, comparing to grunt/gulp + browserify + tools setup. I started to see Webpack as huge kitchen sink. Transpiring, bundling, optimizing, hot-reloading, server deployment – all is in the box.

But once it’s configured, you realize it is worth.

First of all, it works quite fast. Our application is still small, but it takes only 1.6s to fully rebuild it. Secondly, Webpack is much more than simple a build tool. It introduces you to another way of working with web resources such as sources, styles and images.

Finally, Hot Reload is something that makes development with Webpack really fun. Having the experience of updating a portion of the applications without refreshing the page is totally awesome.

Babel and ES2015

Babel 6 has been released at the end of October. I started to use it from the very beginning, but it was quite a mistake. A lot of libraries still had dependencies on Babel 5 and refused to work properly. I had a bunch of cryptic error messages in a console, read a bunch of GitHub issues before I realized that. That forced me to step back for react-hot-reloader, instead of react-transform since it wasn’t yet ported to Babel 6 and we still have some small issues after Babel updates.

But now, if you ask me – would you like to switch back to ES5, I would likely say no. I wasn’t happy when I heard the first time that ES6 will have classes. But, implementing React components as classes, makes a lot of sense.

Moreover, spread operator, fat arrow functions, and new scoping rules make programming with ES2015 as a much more pleasant process.

Redux

Redux is one of the most popular JavaScript libraries ever. Taking the functional programming approach, combined with a nice API and a great timing of the release, Redux is de-facto default Flux implementation for React-based applications.

It requires a lot of paradigm shifts, cause it’s very different from MVC/MVVM libraries I used to work with.

Redux philosophy is based on the fact, that the whole application state is represented as one big object. The application’s UI, becomes a function of this state object. The only way to change the state, is to dispatch an action. An action contains it’s type and some payload, which is passed to a reducer function. The reducer function produces a new instance of the state. Based on the updated state, the application re-renders their UI.

As you see, all that approach is pretty much synchronous. It was quite hard for me to understand how to introduce asynchronous behavior there. I was very happy once my redux-thunk implementation started to work.

Besides, hooking up Redux to React the application was not super straightforward. I spent some time with an understanding of the configuration of the storages, connected components, etc.

I’m still having a lot of questions. I not really happy with all the code you need to write if you want to introduce new actions. And I don’t know yet how to test properly all that stuff.

But Redux gives a nice tooling around, plus better confidence of what’s going on in the application. Since all actions could be logged, and state never overwrote it’s much more easy to debug and introduce new functionality.

React

0.14 is the latest React release, released in October.

First of all, it just works. I feel so great dealing with React, since it brings so less surprises. The concept of a component is simple and very native.

I’m not sure yet if I use everything right. Designing the component hierarchy is still a question, that requires a bit more practice.

I’m happy with the error messages that React provides. The messages pointing you directly to the exact problem. Compared to other libraries, which produce nothing more as encrypted error messages (yes, angular). Making UI’s with React feels great.

Currently, we started to involve our designers to work directly with JSX instead of HTML and we are looking forward to see the impact there.

Second, React is likely the biggest change to front-end development in the last 5 years. Virtual DOM, diff tree and partial re-rendering – quite a masterpiece library. I can’t say it feels totally perfect for me, but it feels really good.

ESLint

I have to mention ESLint. Previously, I used JSHint, but unfortunately, they decided not to support latest language features until official support.

But ESLint actually becomes so much nicer. It so simple to configure and it’s so helpful, that I can not imagine to develop without it anymore.

Both Atom and Sublime have ESLint support, and it will be the only choice if you want to go the ES6/React/Webpack way with linting.

Conclusions

After the boilerplate has been concluded, we started with the implementation of our new application based on that. It works fine so far, not ideally, but good enough. During the flight, we skipped our idea of changing pieces of existing app and started new project from scratch.

Greenfield development + latest libraries and tools make whole development process nice and fun.

Originally published at blogfoster Engineering.

For a quite long period, my default setup was like – NPM for all server-side components and Bower.js for client-side stuff. It worked quite good together, especially in conjunction with Require.js.

But recently, I switched to NPM only setup. That allowed me to use CommonJS style both front and back ends and got rid of task runners as Grunt and Gulp.

CommonJS

Doing a lot of back-end coding I really get used to CommonJS style, which is simple, practical and powerful. CommonJS modularization is plain as,

1
2
3
4
5

var module = {
};

// export the module
module.exports = module;

and

1
2

// use the module
var module = require('./module');

Node.js highly popularized Common.js approach. And NPM appeared to be one of the best package managers ever developed.

But for front-end, NPM was not that suitable. First of all, because of browsers do not understand Common.js style of modules. Second, npm modules are being placed to ./node_modules folder, with is typically the root of project, that made things complicated to refer those scripts from index.html and configure server to server static resources from ./node_modules folder.

Browserify came to change this.

Browserify

Browserify is JavaScript transpiling tool, with allows to:

1. Use Common.js style for front-end applications.
2. Allowed to use various npm packages directly in a browser.

A lot of pure front-end frameworks and libraries were published to npm. In fact, it’s more and more used for front-end libraries as according to NPM developers.

Of cause, nothing comes for free. The cost of Browserify is a build step. You have to introduce to transform Common.js application into a bundle which is loaded and executed by a browser.

Moreover, some packages are still being hosted on Bower. Even, it’s possible to use Bower packages together with NPM packages (by means of debowerify transformation) I quickly realized, that this approach is very unpractical.

So, during the last few months my main contributions on GitHub was adopting of some front-end packages, to be conformant to Common.js and publishing them on NPM after.

The setup

The NPM now is the only one package manager to use, both for front-end and backend.

All dependencies are mentioned on one file package.json instead of several files.. and you will never forget to run bower install, since npm install will do install everything.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

  "dependencies": {
    "async": "^0.9.0",
    "babelify": "^5.0.4",
    "body-parser": "^1.9.0",
    "colors": "^1.0.2",
    "cors": "^2.4.2",
    "envify": "^3.4.0",
    "express": "^4.9.5",
    "method-override": "^2.2.0",
    "moment": "^2.8.3",
    "morgan": "^1.3.2",
    "node-logentries": "^0.1.4",
    "react": "^0.13.1",
    "respawn": "^1.0.1"
  }

The server side code,

1
2
3
4
5
6
7

var express = require('express');

// configurations...

app.listen(port, function () {
    logger.info('editor-app listening on port ' + port + ' ' + env);
});

The client side code,

1
2
3
4
5
6
7

var React = require('react');

var Component = React.create({

});

module.exports  = Component;

The same style, feels nice and clean.

No task runners

NPM is not only package manager. It’s also very simple task runner (or scripts runner). There is a special section, called scripts responsible for that. Originally it’s been used to run some package related tasks (run tests, perform pre/post install configurations).

Traditionally Grunt.js and Gulp.js are ones that used dealing with compiling static assests. But CLI interface of many tools and stream piping could replace them.

As example, I’ll show pretty typical setup:

• build JavaScript with browserify and apply minification
• build Sass with node-sass and apply minification
• run watcher and rebuild changed assets

Building JavaScript

The browserify need to be installed globally,

1

$ npm install -g browserify

In package.json, in scripts section, will add such script,

1
2

  "scripts": {
    "build-js": "browserify public/js/app.js -o public/build/app.js",

Since I also use several transformations in my applications, the Browserify have to properly configured for that. It check’s if package.json contains browserify section, and if it does – will use it contents as own options.

1
2
3
4
5
6

  "browserify": {
    "transform": [
      "babelify",
      "envify"
    ]
  },

Building Sass

As with Browserify, node-sass need to be installed globally,

1

$ npm install -g node-sass

Then package.json changes,

1
2
3

  "scripts": {
    "build-js": "browserify public/js/app.js -o public/build/app.js",
    "build-sass": "node-sass public/sass/main.scss public/build/main.css",

Minification

Now, let’s minify both assets using one of the best concepts ever – Unix pipes.

We need two packages additionally, uglifyjs and cleancss.

1

$ npm install -g uglifyjs clean-css

And corresponding scripts,

1
2
3
4
5
6

  "scripts": {
    "build-js": "browserify public/js/app.js -o public/build/app.js",
    "build-sass": "node-sass public/sass/main.scss public/build/main.css",
    // minifying
    "build-min-js": "browserify public/js/app.js | uglifyjs -o public/build/app.min.js",
    "build-min-sass": "node-sass public/sass/main.scss | cleancss -o public/build/main.min.css",

Watching

Now, let’s apply watching. One of the watchers I use for several years already, that works best for me is nodemon.

nodemon is not only able to watch file changes and restart node processes if necessary, but also running custom commands.

1
2
3
4
5
6
7
8

  "scripts": {
    "build-js": "browserify public/js/app.js -o public/build/app.js",
    "build-sass": "node-sass public/sass/main.scss public/build/main.css",
    "build-min-js": "browserify public/js/app.js | uglifyjs -o public/build/app.min.js",
    "build-min-sass": "node-sass public/sass/main.scss | cleancss -o public/build/main.min.css",
    // watching
    "watch-js": "nodemon -e js -w public/js -x 'npm run build-js'",
    "watch-sass": "nodemon -e scss -w public/sass -x 'npm run build-sass'",

Complex tasks

Typically, you need to run few tasks simultaneously, e.g. build all scripts or watch all application. We can combine commands, by running background tasks,

1
2

"build": "npm run build-js & npm run build-sass",
"watch": "npm run watch-js & npm run watch-sass",

To build full app,

1

$ npm run build

or to watch it,

1

$ npm run watch

Using dev-dependencies

Global packages are not cool, cause they might be missing on developers box there application is cloned. Moreover, dependency management becomes a bit more complex, if you need to work on different projects depending on different tools versions. As @bcomnes very correctly noticed, all tools as browserify, uglify etc. can be installed as local dev-dependency, which are fetched during npm install and will be available to call inside project folder as global commands,

1

$ npm install --save-dev browserify uglifyjs node-sass clean-css

So, the package.json got such update,

1
2
3
4
5
6

"devDependencies": {
  "browserify": "^9.0.7",
  "clean-css": "^3.1.9",
  "node-sass": "^2.1.1",
  "uglifyjs": "^2.4.10"
},

Takeaways

• Once you implement front end library, consider UMD so your modules are reusable in any environment.
• Embrace CommonJS, use Browserify/Webpack/Whatever to use CommonJS (or ES6) on client.
• For many cases, you really don’t need Grunt or Gulp, all is easily doable with NPM scripts.

As example, please refer to brick, the project I currently use as boilerplate for new applications.

Spending much time on backend and working with Node.js/CommonJS I liked simplicity of config pattern and wanted to reuse that pattern on frontend. It’s really straightforward and framework-agnostic approach.

Config pattern

Config pattern is something I frequently use for Node.js apps. It simply the folder with index.js file, containing such code:

1
2
3
4
5
6
7
8
9

var env = process.env.APP_ENV || 'development';

var config = {
  development: require('./development.config'),
  production: require('./production.config'),
  staging: require('./staging.config')
};

module.exports = config[env];

The folder contains such files, development.config.js, staging.config.js, production.config.js etc.

The config files, simply export the object with configuration,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

var config = {
  connection: 'mongodb://localhost:27017/notifierdb',
  accessToken: '1234',

  hook: {
      url: 'http://localhost:5000/api/notify/',
      token: 'fake-hook-token'
  },

  logentries: {
      token: null
  },

  transport: {
      // ...
  }

  // ...
};

module.exports = config;

The module actually needed in configuration would require the config folder, for instance:

1
2
3
4
5
6
7
8
9
10
11

var request = require('request');
var config = require('../config');

function fetchActions(user, callback) {
  var apiUrl = config.api.url;
  var token = config.api.token;

  request({url: apiUrl + '/actions/' + user, headers: {token: token}, callback);
}

module.exports = fetchActions;

Browserify

Browserify is a pre-processor (or call it transpiler) for javascript. It embraces CommonJS style of modularity and brings popular npm modules on client site. It’s simply awesome project (many thx to substack for it) that makes a lot of sense to me.

With Browserify you can write browser javascript in the same way you do on backend with Node.js. It implements require() call and bundles the application consists on different modules into one, executable in browser.

Usually, browserify is not run manually, but rather by means of JS build tools as grunt or gulp.

I’m going to adopt Node.js config pattern for frontend with browserify.

Prepare the configuration

By analogy of node applications, let’s create a config folder inside our js folder.

index.js file is exactly the same as mentioned above, config files contains application dependent settings.

With example of Angular.js application, here is the service that used as data provider,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

var config = require('../config');

function drivers ($http) {
  var apiUrl = config.api.url;

  return {
      fetch: function () {
          var url = apiUrl + '/api/drivers';

          return $http.get(url);
      }
  };
}

module.exports = drivers;

If you just try that, it would work as charm.. but only for development environment.

Setting up NODE_ENV

The problem though, browserify have no idea what to put into process.env.NODE_ENV variable, since it is undefined – development configuration is always selected.

Fortunately, browserify architecture supports, so called transforms, middleware components capable to customize browserify behaviour. One of handly transform function is envify by Hugh Kennedy.

What envify allows to do is basically replacement of process.env.NODE_ENV to particular value, eg.

1
2
3

if (process.env.NODE_ENV === "development") {
  console.log('development only')
}

with NODE_ENV set to “production”, will appear as

1
2
3

if ("production" === "development") {
  console.log('development only')
}

This is exactly what we need, to make config/index.js work properly.

Since I use grunt, I’ll show how to integrate envify in the workflow (approach for gulp would be similar),

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

var envify = require('envify/custom');

module.exports = function (grunt) {
  grunt.initConfig({
      browserify: {
          dev: {
              files: {
                  'source/build/app.js': ['source/js/app.js']
              },
              options: {
                  browserifyOptions: {
                      debug: true
                  },
                  transform: [envify({
                      NODE_ENV: 'development'
                  })]
              },
          },
          stage: {
              files: {
                  'source/build/app.js': ['source/js/app.js']
              },
              options: {
                  transform: [envify({
                      NODE_ENV: 'staging'
                  })]
              }
          },
          prod: {
              files: {
                  'source/build/app.js': ['source/js/app.js']
              },
              options: {
                  transform: [envify({
                      NODE_ENV: 'production'
                  })]
              }
          }
      },
  // ...
}

Now, once the grunt build:dev or grunt build:prod is run, it will populate correct NODE_ENV value and the rest of transpiled application would work as expected.

There few products on market that can do that, Pingdom and New Relic probably the most know. But they are paid and not that flexible as I want. So, I’ve created likeastore/heartbeat that for a month or so helps me to provide quality service.

It’s really easy to setup and deploy on any server or Dokku/Heroku environment.

Overview

The core of heartbeat is about ~300 lines of code, so it’s not a big deal to look inside to understand it principles of work. But in general, it’s continuously running loop, that executes the small jobs who runs particular check strategy and report results if anything is wrong.

To make use of it you just need to clone the repo and update config/index.js with your configuration.

Notifications

Heartbeat can use 2 types of notifications at the moment. Mandril based emails and Twilio based SMSs.

Both requires you to create accounts there and provide API access keys.

That’s what’s configured on transport section of configuration:

1
2
3
4
5
6
7
8
9
10

transport: {
  mandrill: {
      token: 'fake-token'
  },

  twilio: {
      sid: 'fake-sid',
      token: 'fake-token'
  }
}

instead of fake-token and fake-sid you provide your real values there.

You have to specify the recipents for notifications, that happens in notify section of configuration:

1
2
3
4
5
6
7
8
9
10

notify: {
  email: {
      from: 'heartbeat@likeastore.com',
      to: ['devs@likeastore.com']
  },

  sms: {
      to: ['+3805551211', '+3805551212']
  }
}

As you can see, you specifying from and to emails, as well as phone numbers to send SMS.

Monitoring

Now the main part, configure monitoring options. The monitor section of configuration is responsible for that. At the moment, it provides provides 5 different monitoring strategies.

• HTTP
• JSON
• MongoDB
• Ping
• Resolve

HTTP

The HTTP sends GET request to specified URL and checks the response code. If response code !==200 the error notification is reported.

1
2
3
4
5
6
7
8

http: [
    {
        url: 'https://likeastore.com'
    },
    {
        url: 'https://stage.likeastore.com'
    }
]

http is an array of urls to monitor.

JSON

The JSON strategy suits the best for HTTP API’s. It could request particular URL receive JSON response and compare it to expected result.

1
2
3
4
5
6
7
8
9
10
11

json: [
    {
        url: 'https://app.likeastore.com/api/monitor',
        response: {
            "app":"app.likeastore.com",
            "env":"production",
            "version":"0.0.52",
            "apiUrl":"/api"
        }
    }
]

MongoDB

MongoDB is a commonly part of web applications infrastructure. You need to have a connection string and then you specify the checking query. It could be find query or insert query.

1
2
3
4
5
6
7
8
9

mongo: [
    {
        connection: 'mongodb://localhost:27017/likeastoredb',
        collections: ['users'],
        query: function (db, callback) {
            db.users.findOne({email: 'alexander.beletsky@gmail.com'}, callback);
        }
    }
]

Resolve

Resolve is something that makes heartbeat a little different. It usually the thing that one domain names is configured for different ip addresses. As it name stands, resolve would resolve all ip’s associated with domain name and ping them separately. If even one of them fails, the notification is created.

1
2
3
4
5

resolve: [
  {
      name: 'google.com'
  }
]

nslookup for google.com returns,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

› nslookup google.com
Server:       192.168.1.1
Address:  192.168.1.1#53

Non-authoritative answer:
Name: google.com
Address: 173.194.113.198
Name: google.com
Address: 173.194.113.199
Name: google.com
Address: 173.194.113.200
Name: google.com
Address: 173.194.113.201
Name: google.com
Address: 173.194.113.206
Name: google.com
Address: 173.194.113.192
Name: google.com
Address: 173.194.113.193
Name: google.com
Address: 173.194.113.194
Name: google.com
Address: 173.194.113.195
Name: google.com
Address: 173.194.113.196
Name: google.com
Address: 173.194.113.197

All that addresses will be checked separately.

Ping

Finally it’s ping, the simples operation. It just pings given ip address.

1
2
3
4
5

ping: [
  {
      ip: '37.139.9.95'
  }
]

Setting up the interval

You can specify the interval of checking. By default it’s 10 seconds, but you can do more frequent checks if you want.

1

interval: 5000

Running and deploying

If the configuration is set, you can run heartbeat by,

1

$ node app.js

The log optionally could be forwared to Logentries, by setting up config/index.js#logentries section.

Once you need to setup infrastructure for notifications in your application, it will be really easy to do. It provides transport of Mandrill, Twilio, Android and iOS push notifications.

Starting up

The project already has comprehensive README that explains initial steps. At the moment, the server is available as github repository that you need to clone and prepare entry point for it. I’m thinking to pack the things into one npm module. If you see it makes sense, please let me know.

To start, you need to clone the repo,

1

$ git clone https://github.com/likeastore/notifier.git

And then create app.js file, there all setup of notifer taking place.

1
2
3
4
5
6

var notifier = require('./source/notifier');

// TODO: configuration

// start the server
notifier.start(process.env.PORT || 7000);

Let’s go straight to configuration part.

Events, Actions and Executors

The notifier is HTTP server, that receives event and turns that event into corresponding action. One event could create as many actions as needed. Actions is something that later is going to be executed.

The basic setup consits of 3 parts: receive event, resolve event and execute it.

1
2
3
4

notifier
    .receive('incoming-event', function (event, actions, callback) { /* ... */ })
    .resolve('created-action', function (action, actions, callback) { /* ... */ })
    .execute('created-action', function (action, transport, callback) { /* ... */ });

Say, we have user-registered event which we need to send a welcome email,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

notifier
  .receive('user-registered', function (e, actions, callback) {
      actions.create('send-welcome', {user: e.user}, callback);
  })
  .resolve('send-welcome', function (action, actions, callback) {
      db.users.findOne({email: action.user}, function (err, user) {
          if (err) {
              return callback(err);
          }

          if (!user) {
              return callback({message: 'user not found', email: action.email});
          }

          var data = {
              email: action.user,
              user: _.pick(user, userPick)
          };

          actions.resolved(action, data, callback);
      });
  })
  .execute('send-welcome', function (action, transport, callback) {
      var vars = [
          {name: 'USERID', content: action.data.user._id},
          {name: 'USER_NAME', content: action.data.user.displayName || action.data.user.name}
      ];

      transport.mandrill('/messages/send-template', {
          template_name: 'welcome-email',
          template_content: [],
          message: {
              auto_html: null,
              to: [{email: action.data.email}],
              global_merge_vars: vars,
              preserve_recipients: false
          }
      }, callback);
  });

• recieve – receives event from outside and turns the event into corresponding action.
• resolve – step to extend action with additional data, email, name of user etc.
• execute – uses the transport to transmit the event

Transports

Mandrill provides support from emails notifications,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

.execute('send-welcome', function (action, transport, callback) {
  var vars = [
      {name: 'USERID', content: action.data.user._id},
      {name: 'USER_NAME', content: action.data.user.displayName || action.data.user.name}
  ];

  transport.mandrill('/messages/send-template', {
      template_name: 'welcome-email',
      template_content: [],
      message: {
          auto_html: null,
          to: [{email: action.data.email}],
          global_merge_vars: vars,
          preserve_recipients: false
      }
  }, callback);
});

Twilio is used for sms,

1
2
3
4
5
6
7

.execute('send-verify-sms', function (a, transport, callback) {
      transport.twilio.messages.create({
          to: a.data.phone,
          from: '+12282201270',
          body: 'Verification code: 1111',
      }, callback);
  });

For Android push notifications,

1
2
3
4
5
6
7
8
9
10

.execute('send-android-push-notification', function (a, transport, callback) {
  var tokens = [];
  tokens.push(a.data.token);

  transport.android.push({
      message: {key1: 'value1', key2: 'value2'},
      tokens: tokens,
      retries: 3
  }, callback);
});

For Apple push notifications,

1
2
3
4
5
6
7
8
9
10
11
12

.execute('send-ios-push-notification', function (a, transport, callback) {
  var tokens = [];
  tokens.push(a.data.token);

  transport.ios.push({
      production: false, // use specific gateway based on 'production' property.
      passphrase: 'secretPhrase',
      alert: { "body" : "Your turn!", "action-loc-key" : "Play" , "launch-image" : "mysplash.png"},
      badge: 1,
      tokens: tokens
  }, callback);
});

Configuration

Before the run, you need to make sure the notifier is configured properly. Security tokens and connections strings to MongoDB. If you use Logentries, you can provide token and all logs will be submitted there.

The accessToken is a shared secret between server and client, to prevent unauthorized access.

The config is using ENV variables on production and hardcoded values for development and staging.

Clients

notifier can be used by any HTTP client, in most simple case curl,

1

$ echo '{"event": "incoming-event"}' | curl -H "Content-Type:application/json" -d @- http://notifier.likeastore.com/api/events?access_token=ACCESS_TOKEN

We already have Node.js client DemocracyOS/notifier-client and plan to have browser client as well. But in essence it’s just HTTP post with event payload and access_token in query string, so basically it can be used from any language and platform.

Once I logged on to DigitalOcean account, I’ve seen message from administrators that server is sending a lot of traffic (UDP Flood), very likely compromised and therefore stopped. The link they give as instructions to fix the problem was quite comprehensive, but the most important info was in comments. A lot of people who were hit by problem had ElasticSearch deployed on their machines.

It appeared, if ES is left on default configuration with port opened outside, it will be easy target for bad guys, both of Java and ES vulnerability. Basically, I had to restore server from scratch, but this time I’m not goint to be naive, server have to be properly secured.

I’ll describe my setup that involves: Node.js, Dokku / Docker, SSL.

Clean DigitalOcean image

I had to reinstall my machine from scratch, so instead of creating plain Ubuntu 14 server, but I decided to go dokku/docker path. Something that I tried before and very happy with the results. DigitalOcean offers pre-packed image with dokku/docker already on board. As usually it takes just a few seconds to spin up machine on DO.

The plan was the following: deploy ElasticSearch instance inside Docker container, disable dynamic script features of Elastic, deploy Node.js based proxy server with custom authentication by Dokku and link those containers, so only Node.js proxy will have access to Elastic. Finally, all traffic between will by crypted by SSL.

The benefits are obvious, no more :9200 is outside the machine, one potential vulnerability with dynamic script feature is disabled, only authenticated client could access Elastic server.

Deployment of ElasticSearch in Docker

First we need to have Docker image of ElasticSearch. There few ES plugging for Dokku, I decided to install one by myself, since I thought it is easier to configure. There is a great instruction from Docker here.

1

$ docker pull docker pull dockerfile/elasticsearch

Once the image is there, we need to prepare volume there Elastic stores data and configuration.

1
2

$ cd /
$ mkdir elastic

Elastic folder should contain elasticsearch.yml file, with all required configurations. My case is very simple, I have a cluster of one machine, so default configuration applies to me. One thing, that I mentioned above – I need to disable dynamic scripting features.

1

$ nano elasticsearch.yml

The content of the file is just one string,

1

script.disable_dynamic: true

Once it’s done, we are ready to launch the server inside Docker container. Since, it could be done few times (during configuration and debugging), I’ve created a small shell script,

1

docker run --name elastic -d -p 127.0.0.1:9200:9200 -p 127.0.0.1:9300:9300 -v /elastic:/data dockerfile/elasticsearch /elasticsearch/bin/elasticsearch -Des.config=/data/elasticsearch.yml

Please note the important thing, -p 127.0.0.1:9200:9200 – here we are binding :9200 to be only accessible on localhost. I’ve spend some hours to close :9200 by iptables without any success, but that thing works as expected. Thanks a lot to @darkproger and @kkdoo for great help.

-v /elastic:/data will map the containers volume /data to local one /elastic.

Front-end Node.js server

Now, we need to deploy front-end proxy server. It would proxy all traffic from http://localhost:9200 into outside world, securely. I’ve created small project, based on http-proxy called elastic-proxy. It’s very simple one and can be easily re-used.

1
2

$ git clone https://github.com/likeastore/elastic-proxy
$ cd elastic-proxy

The server itself,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

var http = require('http');
var httpProxy = require('http-proxy');
var url = require('url');

var config = require('./config');
var logger = require('./source/utils/logger');

var port = process.env.PORT || 3010;
var proxy = httpProxy.createProxyServer();

var parseAccessToken = function (req) {
  var request = url.parse(req.url, true).query;
  var referer = url.parse(req.headers.referer || '', true).query;

  return request.access_token || referer.access_token;
};

var server = http.createServer(function (req, res) {
  var accessToken = parseAccessToken(req);

  logger.info('request: ' + req.url + ' accessToken: ' + accessToken + ' referer: ' + req.headers.referer);

  if (!accessToken || accessToken !== config.accessToken) {
      res.statusCode = 401;
      return res.end('Missing access_token query parameter');
  }

  proxy.web(req, res, {target: config.target, rejectUnauthorized: false});
});

server.listen(port, function () {
  logger.info('Likeastore Elastic-Proxy started at: ' + port);
});

It proxies all the requests and only by-pass ones, who provide access_token as query parameter. The access_token value is configured on server, by applications environment variable PROXY_ACCESS_TOKEN.

As you already prepared Dokku all you need to do is to push application to your server.

1

$ git push master production

After the deployment, you should go to server to configure applications environment,

1

$ dokku config proxy set PROXY_ACCESS_TOKEN="your_secret_value"

I wanted to have SSL and it’s very easy to configure with Dokku. Just place your server.crt and server.key to /home/dokku/proxy/tls folder.

Proxy have to be redeployed afterward, to apply configuration changes. Try proxy by hitting it’s URL, in my case https://search.likeastore.com. If everything is good, it will reply,

1

Missing access_token query parameter

Link containers

We need to link both containers, so Node.js application is able to access ElasticSeach container. I really liked dokku-link plugin, that does exactly that’s required in very easy way. So, will install it

1
2

$ cd /var/lib/dokku/plugins
$ git clone https://github.com/rlaneve/dokku-link

Now, we need to link containers,

1

$ dokku link proxy elastic

And application have to be redeployed again. If everything is good, you will be able to access the server by https://proxy.yourserver.com?access_token=your_secret_value and see the response from ElasticSearch,

1
2
3
4
5
6
7
8
9
10
11
12

{
  status: 200,
  name: "Tundra",
  version: {
      number: "1.2.1",
      build_hash: "6c95b759f9e7ef0f8e17f77d850da43ce8a4b364",
      build_timestamp: "2014-06-03T15:02:52Z",
      build_snapshot: false,
      lucene_version: "4.8"
  },
  tagline: "You Know, for Search"
}

Reconfigure the client

Depending on a client you use, you need to apply a little tweak to configuration. Basically, we need to use access_token for all our request. For Node.js applications,

1
2
3
4
5
6
7
8
9
10
11

var client = elasticsearch.Client({
  host: {
      protocol: 'https',
      host: 'search.likeastore.com',
      port: 443,
      query: {
          access_token: process.env.ELASTIC_ACCESS_TOKEN
      }
  },
  requestTimeout: 5000
});

Again, ELASTIC_ACCESS_TOKEN is env variable to hold your_secret_value token.

Now, restart the application, make sure everything is running and exhale.

PS. I would like to say thank to DigitalOcean for support and a little credit I received as downtime compensation. Thats awesome.

Today, I want to share some info on where are we now and lessons I learned so far.

Focus on search

One of the most significant books on startups, defines startup as temporary organization aims to find sustainable business idea. It’s not about coding, testing or anything related to development. It’s all about the search.

The biggest mistake of Likeastore, we never did any kind of search before the project started. It was a cool idea that made sense to me and I wanted to make it happen. Development is very expensive process. It’s not possible to do good looking and well functioning product without long hours of work. And we worked hard.

Many times I heard opinion “work smarter, not harder”. That principle transformed to something we know now as Lean Startup, that basically says “do as less as possible before your idea is validated”. The only way to validate it is to show it to people and talk to that people after. There is no truth inside the organization, founders might have vision but that vision could be wrong.

Engaging with users, processing the feedback and correction of idea is something that called is Customer Development. Customer Development is vital part of search. Unfortunately, we never accomplish any kind of customer development and currently we suffer that. “Likeastore is so cool idea” – we still receiving that kind of emails from our users, that stays for 3 weeks and then slipping away.

Working smarter is hard. For me, that’s completely different set of skills and habits. The founder is the one who is able to tweak himself for new conditions and focus on search.

Coding is easy part

Being the programmer and spent over 10 years in software development I was sure that I’m doing the most complex part of work. All those business guys, product owners and so on, seemed to be auxiliary people, sometimes ones who are just interfere the work.

Working in the company, together with business people, your area of responsibility is primarily the code. But at the end of the day, you don’t know what’s happening after deployment. How customers work with your code, are they happy.. are they ready to pay the money? Business people care about it and plan on what exactly need to be done further. You turning their ideas into code and get salary not because your code is good, but because business works.

Working in startup is different. No more product owner who says what to do next. You are the product owner. And generating ideas is hard. Talking to customers is hard. Search of product/market fit is hard.

Being able to sit and spend 10 hours for programming is fun for me, but it turned out it’s the most easiest part of journey.

Measure everything

The code is deployed, some traffic lead to site – now what? In Likeastore we were blind for quite long time without good understanding of what actually happening inside the application.

The truth is, without data it’s not possible to make decisions. We are now utilize Mixpanel, Google Analytics, Seismo custom dashboards etc. in order to understand what’s works or not.

Build, Measure, Learn – the mantra of Lean Startup. What we focused on previously is Build part only and that’s not right. Startup is ability to make educated guesses, guesses should be based on learnings, learnings are based on measurements.

We are at very stage, but now working hard to create good framework for analytics and measurements.

Build your own culture

Even if your are very small, culture plays major role. At the early stage you affect the culture, later stages culture affects you. At Likeastore, we care about marketing and development culture much.

Being based on open source we try to contribute back to open source. I’m really happy with our github profile and only predict it’s going to grow. We try to open source as much as possible, sometimes even solutions that quite specific to Likeastore.

Our later obsession with metrics and numbers will be part of culture as well. As founder you should see where you want to be in 2-3 years, not only in terms of product, but the way other people know about the company. For me, saying “I work on Likeastore” means a lot and I want it mean a lot for everyone works here.

Listen what they say

It’s a common practice for business accelerators accepts teams with at least 2-3 founders. I didn’t really understand why.

There are few things here. First of all, if you are not able to inspire at least one person with your idea it’s likely going to fail. Second, founders should complement each others skills. There are rule of 3 that ideal startup team should consists of hacker, hustler, designer.

But there is more important aspect. Founders have to talk all the time, brainstorm and share ideas, discuss competitors etc. You need at least 2 people for discussion. More importantly, you should be able to share your idea with other people. Once we’ve joined business accelerator, first advice that I received – “talk to other teams, they will help you”. I didn’t take that advice seriously and I was really wrong. Startups are huge community and you either part of that community or out.

You have to both talk a lot and listen what they say. Co-founders, users, colleagues – they know something you don’t. It doesn’t mean you have to do everything they say, but definitely listen and analyze.

We are now team of 3 and it feels really good. Each of us have own experience, so we need to combine and make something real out of it. Don’t keep secrets, share the stuff. Doesn’t matter what exactly your role is, talk to the team, talk to other guys, feel the influence.

Don’t close the door of your garage.

The most important thing for every startup are metrics. In web application we use Mixpanel together with Google Analytics and Seismo. For browser extension I also needed some simple metrics, like “usage”, “clicks”, “shares” etc. Unfortunately, if you just try to reference mixpanel-2.2.min.js it won’t work.

Instead, you will see error message in console,

1

Mixpanel error: 'mixpanel' object not initialized. Ensure you are using the latest version of the Mixpanel JS Library along with the snippet we provide.

There are few reasons for that. One is described in SO question. But after I did same manipulations as answer mentioned it still didn’t work. For extension I only have content script, there is no place there I can put <script></script> tag with markup. It seems like mixpanel-2.2.min.js have some dependencies that just prevents it to start normally inside extension, not usual web application.

I had no time to investigate the problem deeply, but instead found workaround that works.

Mixpanel HTTP API

Besides platform-dependent libraries, Mixpanel provides access to HTTP API. This is the most “pure” way of communication between services, so I decided to try it.

Since, I already used jQuery in my application, I ended-up with such module,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

;(function (window) {
  var api = 'https://api.mixpanel.com';
  var app = window.app = window.app || {};

  var mixpanel = {};

  function init(token, user) {
      mixpanel.token = token;
      mixpanel.distinct_id = user.email;
  }

  function track(event) {
      var payload = {
          event: event,
          properties: {
              distinct_id: mixpanel.distinct_id,
              token: mixpanel.token,
              browser: app.browser.name
          }
      };

      var data = window.base64(JSON.stringify(payload));
      var url = api + '/track?data=' + data;

      $.get(url);
  }

  app.analytics = {
      init: init,
      track: track
  };

})(window);

Here, I’m using this implementation of base64 (copied from mixpanel-2.2.js). So, to track the event you just HTTP GET to Mixpanel API, with a payload of base64’ed JSON string of event and properties.

Initialization

To init mixpanel you need both token and distinct_id. The distinct_id is unique value, that Mixpanel generates itself and stores the value to cookie, to identify the user. If you send events from different applications, say web app, mobile web extension etc. – I would recommend to user mixpanel.alias() call, so you “link” unknown distinct_id with some user’s attribute (id from database or email would work).

On our web application, once user logged on we call mixpanel.alias() with email of user, so all other applications could use that as distinct_id of user.

The entry point of content script is run() method, where user is fetched from server and initialize mixpanel module,

1
2
3
4
5
6
7
8
9
10
11
12

var ready = function (user) {
  app.analytics.init('[MIXPANEL_TOKEN_HERE]', user);
};

return {
  run: function () {
      $.get(api + '/users/me')
          .done(ready)
          .done(search)
          .fail(login);
  }
};

Tracking

Once content script injects it’s HTML into target page DOM, it’s possible to listen to DOM events and call .track() function.

1
2
3
4
5
6
7
8
9
10
11

self.bindEvents = function () {
  self.$.find('.item a').click(function () {
      app.analytics.track('search extension results clicked');
  });

  self.$.find('.ls-more a').click(function () {
      app.analytics.track('search extension more clicked');
  });

  app.analytics.track('search extension');
};

Conclusions

I consider Mixpanel HTTP API as very nice substitution of library, in those rare cases when library is not applicable. This would work perfectly if you need simple events tracking. If you interested, feel free to check sources.

I knew about search oriented storages as ElasticSearch, Sphinx, Solr, etc. but the migration process always seemed a bit scary to me. After a bit of elaboration I decided to go with Elastic and the migration was quite fun and not so difficult as I initially expected.

What’s wrong with MongoDB?

We are using MongoDB 2.4 and full text search is experimental feature there. By default, it’s turned off on production instances of MongoHQ. Anyway, it’s very easy to turn it on and run text commands against it. But we met it’s bottleneck when indexed collection start to grow (>3M documents).

We are doing a lot of inserts and if you have full text index, each insert triggers index re-calculation. Few weeks ago I’ve noticed huge performance issues of our DB. After first glance of monitoring information it’s was pretty clear that Locked index is >110% and a lot requests got timed-out. Application became low responsive.

It could be potentially solved by moving on more powerful server (more CPU and RAM) or by sharding database into few instances. Both costs money. Costs are something that I would like to avoid, if possible. It was time to move to specialized solution.

This is the reason to deploy ElasticSearch – put less strain on MongoDB and to use as an asynchronous index.

Migration plan

Before starting out, I’ve consulted with my guru Vinay Sahni and his experience of ElasticSeach in SupportFu. The general feedback on ElasticSeach was really nice. They don’t use any rivers instead use after_save hook of their ORM, so each time new document is created it’s pushed to ElasticSearch index.

I decided to go similar way, so my migration plan was:

1. Export all existing documents to ElasticSearch.
2. Correct the code and after each new insert send update to ElasticSearch.

In my situation it’s a bit simpler, by the fact, I have only one collection to index and major part of DB operations are inserts not updates, otherwise I believe using of river has more sense (and a more effort to setup).

Deploy instance on Digital Ocean

I’ve spin-up Digital Ocean droplet and used that great instructions for installation (btw, I recommend to install from Debian package, since it’s easier to run elastic as init.d service).

Just in 10 minutes you have working ElasticSeach cluster, ready to rock.

Migrate MongoDB data to ElasticSearch index

Then I needed to move about 12GB of data from my production MongoDB instance to Elastic. To accomplish that, I’ve created really simple tool – elaster. Elaster streams MongoDB collection into ElasticSearch index, based on configuration you provide.

I’ve installed nodejs on ElasticSeach server, cloned elaster there and run,

Both elaster and application code use elasticsearch npm package.

The whole process took ~2 hours.

Update the application code

Right after documents are inserted to MongoDB and initialized with _id, they are ready to be saved to index. I use bulk operation for that,

1
2
3
4
5
6
7
8
9
10
11
12
13

index: function (items, state, callback) {
  if (!items || items.length === 0) {
      return callback(null, items);
  }

  var commands = [];
  items.forEach(function (item) {
      commands.push({'index': {'_index': 'items', '_type': 'item', '_id': item._id.toString()}});
      commands.push(item);
  });

  elastic.bulk({body: commands}, callback);
}

Simple search query

ElasticSearch comes with very nice Query DSL. I’m very new to the technology, so I’ve managed to create really simple query, which already gives quite nice results.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

function fullTextItemSearch (user, query, paging, callback) {
  if (!query) {
      return callback(null, { data: [], nextPage: false });
  }

  var page = paging.page || 1;

  elastic.search({
      index: 'items',
      from: (page - 1) * paging.pageSize,
      size: paging.pageSize,
      body: {
          query: {
              filtered: {
                  query: {
                      'query_string': {
                          query: query
                      },
                  },
                  filter: {
                      term: {
                          user: user.email
                      }
                  }
              }
          }
      }
  }, function (err, resp) {
      if (err) {
          return callback(err);
      }

      var items = resp.hits.hits.map(function (hit) {
          return hit._source;
      });

      callback(null, {data: items, nextPage: items.length === paging.pageSize});
  });
}

It just uses query_string and filter term, to filter out results for given user. The interface of Search API remained the same, just instead of querying MongoDB, now we query ElasticSearch.

Summary

The main conclusion is – ElasticSearch is kind of technology that works out of the box. It’s very complex inside, but have great interface that hides all the complexity. First and good enough results can be archived very quickly.

In the same time, the technology requires time and effort to understand it and to use it right. I have absolutely no experience with full text search, so it’s really journey for me.

The maintenance package is now available on npm and you welcome to use it. But I would like to share the way I developed and test it.

So, maintenance is a function that takes app instance and options as a parameters. Then it augments app with additional endpoint (if there is such preference) and injects middleware function that would render maintenance page in case of mode is set to true.

It’s really small piece functionality, but still I wanted to make sure, it’s gonna work in my application without annoying restarts of servers and debugging the stuff. And TDD is right approach to solve the pain.

TDD is quite commonly trade-off of unit tests and integration tests. And it’s always up to developer which direction go in particular case. Let’s take a look on my case:

If you go unit test way, it would be something like that:

• call maintenance function, pass mock of express instance
  • verify that new route is added after function completes
  • verify that all route.callbacks now have callback to check the mode
  • verify that route.callbacks[0] contains mode check function
• call route.callbacks[0], mock req and res object
  • verify that res.render called with maintanance.html argument
  • etc..

Is all of that kind of suck? It is, since it would take too much effort for mocking the stuff.. But more important, all of that is nothing more as just testing of implementation details of Express.js, but not behavior details of application.

Now, let’s consider integration tests way for same stuff:

• run application in normal mode
  • send HTTP GET and make sure that response is fine
• run application is maintenance mode
  • send HTTP GET and make sure that response contains maintenance page
• run application in normal mode
  • send HTTP GET and make sure that response is fine
  • sent HTTP POST to maintenance endpoint
  • send HTTP GET and make sure that response contains maintenance page

Does it look like real behavior testing? Indeed, test acts as user and checks that application actually behaves right, doesn’t matter of implementation details.

With spending more time with Express.js and Node.js I see much more value in integration way of testing. It’s very easy to spin-up a server and send HTTP requests and check responses.

If you are interested, check out specification and test application.

The year, seems to be a huge period of time. It is, but I have to say it’s only few last months (from the middle of December till today), we are fully dedicated to project. It’s been great ride so far, so I wish to share some interesting moments.

What we started with

Let me remind you, Likeastore is a web application that allow users to connect their social accounts and collect all their favorite items from them. We are 2 founders, me and Dima who committed themselves to build product that matters.

As founders we felt that “likes” are kind of new way of bookmarking. Then you favorite something on social network you do 2 things – you send appreciate to author content, as well as marking the content as interesting to you.

The problem that we try to solve. First, there are too many sources of information (social networks) where we spend time and receive content from, so tracking all social likes is extremely difficult. And second, it’s very hard to find exact favorite item since social networks provide poor search functionality.

So, our initial hypothesis for application was – people want to collect their likes in one place and be able to search them.

Where are we now

After successful demonstration on hackathon, we’ve reimplemented application from scratch and did first private beta in May 2013. It was pretty painful, since after 10 minutes of fly my inbox was full of error messages. But, general impression of people we spoken was positive. We’ve crossed the line from hack to side project.

Our public beta released at July 2013. All major issues were solved, the infrastructure established and UI/UX of product were done. We were lucky enough to be featured by Codrops and after by SmashingMagazine twitter, which gave us first 500 beta users.

At the end of 2013 we’ve dropped our full time jobs, joined business accelerator and totally focused on a product. I would say, that was the point then we crossed the next line, from side-project to startup.

We are at the full speed, full time, 2 people dedicated to Likeastore. And since then, we did major step forward. Let me share some of current statistics:

• 6.370 signups (+ ~4.000 from January till today)
• 2.700.000 likes collected (+ ~1.500.00 from January till today)
• ~200 active users (+185 from January till today)
• ~600 feedback emails received
• 452 collections created

How does it feel like

First of all, time flows 2x faster compare to anything I was involved before. You put yourself in a very constraint environment: lack of money, lack of time, not clear whats happens tomorrow. Multitasking is second name of startup, you don’t have time to be focused on one task during the day.

Every day you feel ups and downs. The mood is changing constantly. You have growth and receive users feedback – it feels great. You talk to investors or mentors that find weaks and leaks of idea – it feels bad. And it’s all the time like that.

Founders job here is to supporters to each other, motivate for further wins.

What we learned so far

Product / Market Fit – basically all you do in startup is finding a balance between value you can propose and market that meets that value. It doesn’t matter what the initial idea is, more important is to be able to change direction and put maximum efforts to find the fit. The search could be blind or educated, educated search is probably more successful.

You will never know about that fit, before you try. Almost no idea will survive after it hits the customers. Steve Blanks calls that as “getting out of the building”, talk to people and show your product before you even built it and spend a lot of money for it.

The product have to solve real customer pain or need to survive.

Metrics are important – all decisions have to be taken on metrics. Retention, Cohort analysis, Viral coefficients – that was all new words for me and only now I realize how metrics are important.

Startup dashboard that aggregates major metrics and KPI’s is something that founders spend a lot of time with, finding new strategies and forecasting startup future. Measure everything.

Marketing and Growth – current market is already very tight. People are constantly releasing new web and mobile applications, solving different problems, targeting different customer groups. You are a drop of water in the ocean and you have to be noticed to have growth.

I see marketing as car’s engine starter – you built the car, put powerful engine, comfortable chairs and high quality audio system. But without starter, engine won’t start and car won’t move.

At least basic marketing skills are vital for startup. If founders are more technically oriented (as we are) someone either have to learn it, or you have to hire specialist to help.

Smooth pivot

We are growing, average 10% per week. In the same time, we have quite low retention coefficient. People are quite excited about idea of application, but very fast they simply forget about and don’t come back.

To be successful on B2C segment, you have to have hundred thousands of users to be sustainable. Likeastore is very far from sustain state as for now.

In the other hand, startup is great place to change your mind. I change my mind all the time. The change of mind in startup have a fancy word for it – pivot. Pivot is the change of direction that startup does, to find it’s product / market fit.

We are going to pivot Likeastore in a very smooth way. As we previously was “Dropbox of Likes”, just allowing to collect and search.. we now want to try something like “Pinterest of Likes”, allowing people to collect best content into sharable collections, following that collections and collaborating on them.

Ultimate goal

We are putting our bar much higher. Our ultimate goal now is to gather 20.000 users at the end of the May 2014. If it happens, that would mean the idea of Likeastore is alive, otherwise the chances it survives are very low.

Both me and Dima really believe in what we do. We have the vision, will try to turn it to reality and either pivot or try something completely new.

I’ve changed my mind on testing with Node.js/Express.js development. For API’s I prefer “end-to-end” testing: setting up user account, authentication, HTTP calls to server, real calls to DB and serving JSON payload back. API’s have to be tested from consumer point of few to be able to give some meaningful results.

Tools and Frameworks

Pretty standard setup: mocha, chai, request.

Mocha is time proven tool for testing Node.js applications, Chai is good enough expectation framework and Request as one best HTTP clients I even worked with.

Prepare application for testing

Dependencies above is installed via npm install and should be saved to package.json file (with --save option). Your project structure should have test folder inside, which mocha is using as default to look tests inside. Few files should be added there, current structure I have that works.

1
2
3
4
5
6
7
8

/test
  /specs
      auth.spec.js
      ...
  common.js
  mocha.opts
  utils.js
  runMocha.js

/api folder is the one that would contain specifications for you API, mocha.opts – contains global mocha configuration, common.js is common require file, that all tests are using, utils.js – test helper that would contain everything you need during testing, runMocha.js utility that would be tests entry point.

mocha.opts

1
2
3
4
5
6
7

--require ./test/common.js
--reporter spec
--ui bdd
--recursive
--colors
--timeout 60000
--slow 300

Mocha options allow to require some additional javascript file, as well as setting up global Mocha settings as what reporter to use, timeouts etc.

common.js

1
2

global.chai = require('chai');
global.expect = global.chai.expect;

To do not require chai in each spec files, it’s possible to require it once and place to global scope.

runMocha.js

1
2
3
4
5
6
7
8
9
10
11
12

process.env.NODE_ENV = process.env.NODE_ENV || 'test';
process.env.TEST_ENV = process.env.TEST_ENV || 'test';

var exit = process.exit;
process.exit = function (code) {
  setTimeout(function () {
      exit();
  }, 200);
};

require('../source/server');
require('../node_modules/mocha/bin/_mocha');

The secret sauce is last 2 lines. Since require is synchronous we first “call” API server to get up and after that “call” mocha engine to start testing. So, inside each tests we can do real HTTP calls to real HTTP servers. No mocks.

package.json

1
2
3
4
5

// ...
"scripts": {
    "test": "node test/runMocha",
  },
// ...

Package file should contain script to call API tests with simple npm test command.

Test driven API

Mocha is using BDD (behaviour driven development) approach to testing. Comparing to classical TDD, BDD encourage to write specifications in plain English, which works good especially when you just starting with particular feature.

What I typically do is just writing down specification of API, without any thinking of how to implement it. Give you example of something that I worked with recently.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42