Astrails - We create awesome web applications.

Why printing business cards when you can use an app

2019-10-09T09:53:44+00:00

When it time came to print a new batch of business cards. I tried to find the latest design we’ve used a few years back and update them.

Than, suddenly, I though: what if I had an app that allows sharing a business card. I searched App Store and found a few not trivial apps that probably only engineers can use.

I had strong vision of how the app should look like and what should it do.

It should allow having as many cards as I need. Well, I hold a few positions in a few companies Astrails work with.
It should not keep my personal information somewhere on remote servers, who knows how well the data is protected there.
And most important: it should be super easy to the receiving party to save the card, no extra apps should be required for that.

Imagine that you’re starting a meeting with 3 participants.

Asking their phone numbers/emails and sharing your contact from a phone is not an option. It will take a lot of time and introduce an unnecessary hustle. The sharing should be as simple as passing a printed card.

So, the answer for the last question is QR code with embedded vCard, default camera app since iOS 12 and Android 9 recognizes QR codes. And you should only show your phone to participants.

That’s how Virtual Business Card was built. There is a video there, so feel free to check it out.

Right now it only works on iOS, Android version is coming soon.

The Engineering Perspective: A Brief History of Web Development. Part 4.

2018-04-18T15:04:16+00:00

This is the 4rd and a last post of the series, Part 1, Part 2, Part 3.

The initial release of AngularJS happened just one week after the release of Backbone.JS, but was pretty much ignored for a while. It took some time to understand that in addition to Javascript, HTML and CSS, frontend developers that wanted to succeed with Angular had to learn ng-extensions too. Within a few years, Angular became very popular among developers. The only problem was that it was very easy to get started with it, but pretty complicated for more advanced things.

Exactly one year later, ex-Rails core engineer Yehuda Katz released Ember. The first version was super opinionated and had a high entry barrier. Instead of trying to solve some small tactical issues, Ember directly targeted full-featured client-side web applications.

Two years after that, Facebook released the first version of React - a breath of fresh air for frontend development. No monolithic applications, no misplaced MV (whatever) patterns, no extensions. Yes to Daft Punk’s Random Access Memories, album released that same year. The concept of React was very simple: representation is a function of an application state. Just implement the render function as a function of a state, include optional lifecycle callbacks, and React will take care of the rest.

Later, Facebook introduced Flux, frontend application architecture with unidirectional data flow. Dan Abramov’s react-redux based on Redux is probably the best implementation of the same concept of unidirectional data flow. It took about a year and a half for the community to catch up and by the end of 2014, React became extremely popular.

And at the beginning of 2015, Facebook released React Native, a framework for building native apps with React that introduced the concept of “learn once write many”. iOS and Android applications will not share 100% of the code, but it is possible to reach 60-90% depending on the requirements. Compared with 0% code sharing when developing ObjectiveC/Swift for iOS and Java for Android, this is quite an achievement. Especially when taking into account that development is mostly done using React and Javascript.

Angular 2 entered the beta stage in late 2015 and was released in September 2016. Its breaking changes and a few concept shifts deprecated most of the knowledge and experience that developers had accumulated while working with Angular. Some developers were angry about these changes because they had to re-invest time to basically writing everything again from scratch using Angular 2. But some were happy because the new version was better engineered and more fun to work with.

Initially released in 2014, Vue gained popularity around the same time that Angular 2 was released. It was what Angular JS should have been from the beginning.

At the end of 2015 and most of 2016 there was a popular joke: two new Javascript frameworks were probably released while you were reading this sentence. And another three have just died.

Q&A

So, what should I use now?

React. Unless you have someone on board who is familiar with Angular, then use Angular or Vue and consider React anyway.

What should I learn if I am just entering this field?

React. It will allow you to develop desktop/mobile web apps, native mobile apps, and even Windows applications using react-native-windows.

What’s next?

Progressive Web Apps are the next stage in web frontends. It deserves its own series of posts.

The Engineering Perspective: A Brief History of Web Development. Part 3.

2018-04-18T15:04:16+00:00

This is the 3rd post of the series, Part 1, Part 2, Part 4.

Ok, so now initially loaded web pages have almost no HTML code, but have tons of Javascript and CSS instead. This is good, because these assets could be served from CDN which in theory makes the whole page load much faster.

But the page load time is not the optimization objective. What really matters is the time that it takes for the page to become usable, in other words - when users can start consuming the content on the page.

So besides the time that it takes to load all of the assets, we also have to consider the time spent inside the Javascript to fetch relevant data using APIs and to render it on the client side by updating the DOM. It’s user CPU time that renders the content in this case. User CPU seems free from an application vendor perspective since the user pays for it and for the electricity to power it. But that’s not 100% true.

The first iPhone was launched about 2 years after the original AJAX piece was published. And that triggered a new epoch of mobile browsers. But mobile browser CPU time is not free. It doesn’t cost money directly. The price is the user’s disappointment with battery life. Now users don’t blame lazy developers who wrote suboptimal code, they blame mobile device vendors for using bad batteries.

Anyway, Prototype JS was released in the same month as the AJAX article and attracted a lot of attention. It allowed direct DOM manipulation, contained cross-browser abstractions, and was bundled with its own Javascript effects library. Prototype JS was probably the most important advancement in frontend development since the invention of CSS in 1996. In any case, the first version of jQuery was released 18 months later. It becomes a de-facto standard for the next 9 years because it was direct DOM manipulation done right.

A crazy mix of server-rendered HTML pages and lots of jQuery based mostly on spaghetti code to update these pages, plus a few JSON/XML APIs here and there, was characteristic of Web 2.0 development for many years. By that time, the prestige of frontend development had significantly declined. Backend engineers looked down on HTML/CSS/JS folks that spent most of their time showing and hiding divs and changing the colors of different elements.

Barack Obama inaugurated as the 44th President of the United States of America just 4 months before the first release of Node.js. Node.js was accepted quite sceptically as a nice try to bring Javascript to server side, and there was no significant adoption of Node.js in web development. Probably a release of Express a year later became a trigger of a wide adoption of Node.js. Today Node.js is a quite popular choice for backend development, especially taking into account that modern frontend frameworks allow the same Javascript code to perform rendering on both sides: server and client.

The state of frontend development was now where backend development had been in the late 1990s - a zoo of approaches and technologies. Different groups tried to come up with a frontend framework that would straighten out the mess. Jeremy Ashkenas came up with the most successful one at the time, Backbone.JS. Released in 2010, this client-side MVC framework restored some order to jQuery-based frontend apps. A few years later, Marionette came along to attempt to enhance the functionality of BackboneJS and to become a Rails-style opinionated frontend framework.

Then more and more engineers began to develop frontend-based apps. And the fun was about to begin.

The Engineering Perspective: A Brief History of Web Development. Part 2.

2018-04-18T15:04:16+00:00

This is the 2nd post of the series, Part 1, Part 3, Part 4.

So the main question is - what does the backend code do in order to render its output? Static assets aside, the response will most probably include data that was fetched from some sort of database. If the output is a full HTML page, it should also include proper HTML markup around the data. This is called server-side rendering because the content of the whole HTML page is rendered by the server.

The bigger the HTML page the more expensive rendering will be.

In earlier web applications, each request type was handled by a separate script file. The script was interpreted line by line to render a web page. Several approaches were introduced to make the code more maintainable and MVC was one of the most successful ones. It’s worth mentioning that Rails, a Ruby-based MVC implementation that was initially released the same year that Zuckerberg launched Facebook, contributed significantly to the wide adoption of the MVC design pattern in web development. Model-View-Controller frameworks for many languages were released in the late 2000s.

MVC separates the concerns of the backend code. Requests are handled by a controller, it uses models to fetch data. The data is passed to views which are responsible for rendering pages. It’s a much better solution than a single script file that does all of the above. But tons of CPU and memory are still used to prepare HTML pages that became bigger and bigger.

And then along came Asynchronous JavaScript + XML, aka AJAX, almost nine years after the FastCGI specification was published and in the same year that the 3rd episode of Star Wars was released. The AJAX approach suggests fetching extra data from the server to update the current screen without reloading the entire page.

Web servers can render many types of content in response to an XMLHttpRequest - including HTML segments that can replace an existing part of the web page, Javascript code that can do anything (including UI effects and replacing parts of the HTML page) once it is evaluated by the browser, and pure data that is serialized in XML/JSON format.

Javascript code running in a browser uses XML/JSON data to update the content of a page. Beforehand, Javascript was used mostly for UI effects. This marked the beginning of a significant shift in the concept of web development. It was the beginning of client-side rendering. This shift actually affects backends as well, that no longer have to render expensive HTML pages. Instead, their main function is serving APIs. It almost makes the V-portion of the MVC framework redundant, because JSON format doesn’t include anything but the data itself. Now server CPUs and memory can be used for better purposes.

The Engineering Perspective: A Brief History of Web Development. Part 1.

2018-04-18T15:04:16+00:00

Two months after the end of WWII in Europe, Vannevar Bush published an essay called As We May Think. He introduced a concept called memex, which became the forerunner and inspiration for hypertext.

Forty-six years later, Tim Berners-Lee released the very first web server, called CERN httpd, and a browser that worked on NeXT. Later that same year, Nicola Pellow developed a text mode browser for other platforms. At the same time, on the other side of the planet, Pablo Escobar surrendered himself to the police. CERN httpd’s latest version 3.0 is available on GitHub.

Two years later, NCSA published a specification for calling command line executables. The output of these executables could be rendered back via the web server over an HTTP protocol as a response. This was basically the birth of the dynamic web and ultimately resulted in a formal CGI standard.

Three years of extensive use of this solution showed that launching a separate process for each request and tearing it down at the end of the request was a waste of resources. Let’s ignore the forgotten Netscape server API for a minute and note that Fast CGI was the greatest achievement since CERN httpd. Requests from the web server could be routed to a FastCGI server (via a socket, TCP or named pipe). The FastCGI server keeps data-rendering processes running for a series of requests, so that responses are returned to the web server and then to the client. This approach significantly reduced server load and response time.

Another popular approach that was actively developed during that time is called embedded interpreters. Some well-known examples are Apache’s mod_perl (released in 1996) and mod_php. They basically keep the interpreter running constantly in order to save launch/tear down overhead and to transfer the script code to the interpreter process, which in return renders back the response body. Interpreters usually have access to the web server’s internal APIs and act as extensions of the web server.

CGI, FastCGI and embedded interpreters are still widely used today (Apr 2018).

But all this has nothing to do with frontends. I just mentioned it to give you an overview of how backends render their outputs to a browser. Of course I skipped some basic things like sessions, caching, reverse proxies, database connections because they are less relevant to the big picture.

The big picture is quite simple. A request is received by a web server, which somehow gets the output of the backend application code and renders it back to a client application. In most cases that client application is a browser. As for static HTML / CSS / Javascript assets, the backend application code just reads content from a file (database, Redis, Memcache, tape, punch card or whatever storage is used) and sends the content back.

This is the 1st post of the series, Part 2, Part 3, Part 4.

Astrails noname - AI-based names generator

2018-03-21T12:12:12+00:00

AI-based names generator.

Earlier this months we released noname, an AI-based project names generator.

It uses a neural language model trained on CrunchBase data.

There will be a separate post(s) covering the whole flow of preparing a data set, training a model, serving a model, etc.

The backend is a CherryPy-based server serving JSON API to make predictions based on models implemented in Keras with TensorFlow backend.

The webapp is implemented in React obviously ;-)

Building and deploying React applications

2017-06-14T12:12:12+00:00

Building and deploying React applications.

Another presentation I was giving at Applied Materials office about React applications.

It’s trivial today to start writing and debugging some React code, but it’s not 100% clear how to properly deploy the application, manage versions and what implications that has on the build configurations. Especially if you want to allow different versions for different users in order to perform some A/B testing, testing new features in production environment, come up with some UI experiments, or gradually roll out new features for a subset of users.

In this presentation I hopefully covered all that.

Building and deploying React applications from Astrails

Code is here. It shows the Rails example but it’s super easy to do the same in Node or Python, slides cover the differences.

Machine Learning: Make Your Ruby Code Smarter

2016-12-08T12:12:12+00:00

Machine Learning: Make Your Ruby Code Smarter.

I was giving this presentation at RailsIsrael 2016 conference. I covered the basics of all major algorithms for supervised and unsupervised learning without a lot of math just to give the idea of what’s possible to do with them.

There is also a demo and ruby code of Waze/Uber like suggested destinations prediction with fast neural networks on Ruby.

Slides are here.

Code is here.

Video and screen cast should be published at here shortly.

Migrating from Flux to Redux

2016-06-17T12:12:12+00:00

Migrating from Flux to Redux.

I was talking about migrating from Flux to Redux last Wednesday at Reacts Israel meetup.

Video and screen cast should be published at ReactJS-IL shortly.

TL;DR: When I started to work with React back in Apr-2015 there were many libraries to manage the application flow. I decided to start with classical FB’s Flux implementation to understand what’s missed there. Eventually react-redux and redux solved most of issues I had with Flux. This talk is about practical aspects of migration from Flux to Redux.

Slides are here.

Code is here.

Start with tag #flux, then #redux, then switch to master branch.

One Huge Timesaver

2015-12-07T20:30:32+00:00

Content Creation Flow (5 mins reading time).

Today we’re going to talk about an effective way of defining new product features.

A product feature can be defined in two ways – from either a marketing or engineering perspective. The marketing approach means explaining how the feature benefits the customer, and the engineering perspective means explaining how that feature works.

Some product managers may miss this distinction and explain a new feature to their engineering team from a marketing perspective. As a result, engineering may work really hard and possibly proceed in the wrong direction, losing time and money for the company. READMORE

For example, let’s take the well-known concept of an online marketplace that sells several products and describe an exciting new feature from a marketing perspective. Our marketplace will now display a list of 10 featured products on its home page.

Well, engineering will have lots of questions. Who will feature the products? How will they be featured? How long will they be featured? Who will un-feature the products? All of these issues have to be clarified.

At Astrails, we’ve come up with the concept of Content Creation Flow. It’s a way of first explaining how data is created in the system and then understanding how it’s consumed, instead of vice-versa. Also, at any given time we can only use terms that were defined beforehand (i.e. objects that have already been created). It makes the concept a little harder to define, but so much easier to understand.

For example:

Facts - application admin users moderate the products; a products back office UI is used to edit/update the products.

Flow - application admins should be able to mark a product as “featured” through the back office UI. The ten most recently featured products should appear on the home page.

In this example, the creation process is described before the consumption process.

This makes it easier for engineering to understand and implement the feature. In our example, each product will have a featured timestamp. In order to feature a product, application admins will trigger the timestamp to be set to the current time. The 10 most recent products should appear on the home page in descending order of those timestamps. We’ll cache these 10 products and invalidate the cache when any product is changed or when a different product is featured by the application administrator. Crystal clear. For engineering.

This concept can and should be applied to wireframing and designing user interfaces as well.

Let’s assume again that we’re building a new online marketplace. The designer starts by designing a home page instead of starting with the inner pages, and presents a first draft showing 10 featured product boxes. Each box has a product image, product name and a short description.

Engineering then receives the homepage design and discovers that there is no short product description field at all. So, what now? Engineering can add this field, but they don’t have a UI for the seller to provide a short description. The web designer can remove the field and use the seller’s name instead. In either case, precious time has been wasted.

A better way to develop the product would be to design the seller’s product-editing page first. Once that is ready, designers will know that the only fields supported are product name and long product description. Long descriptions can obviously not be used in a small product display box, so the designer will use the seller’s name in the homepage design in the first place.

Everything is clear from the engineering perspective - the data to be consumed has already been created and the engineers know how to query it.

So, we discovered that feature specs are described more clearly using Content Creation Flow. Designs and UI can also be produced more efficiently according to this flow.

It turns out that engineering can use the same flow as well and save lots of time.

How? Let’s imagine the following situation. Engineering got specs for some new exciting feature. The specs are 90% complete. The engineers are ready to start coding but there’s an editing screen that hasn’t been designed yet. They decide not to wait. They’ll just seed some data, render other pages that use seed data and get back to that editing screen later. After a while, it turns out that the editing screen has more fields then they expected, with complex relations between them and seeds are not good enough. Now engineering needs to refactor the code and throw away lots of their work before they even start adding the new fields. Result: time wasted.

If they had started with the editing screens, all of the fields and relations that will be used later would have already been defined.

There’s nothing wrong with seeding some data for development, but it’s probably not a good idea to do so before it becomes clear what the data will look like. So, creation before consumption. Always.

If you have any questions about content creation flow, or any questions at all, feel free to email me at boris@astrails.com.

Homebrew Yosemite Issues

2014-12-04T17:09:17+00:00

There seems to be quite a few issues with Homebrew compilation after upgrade to Yosemite.

While some of them are easily fixed by upgrading relevant GCC packages, one problem kept me from upgrading my macvim for a while.

Short version: if after upgrading all the gcc related packages you still get errors on Yosemite, try upgrading python as well.

Dragonfly, ImageMagick and memory bloat

2014-07-28T06:16:23+00:00

We usually use dragonfly to handle user generated assets in almost all the projects. But sometimes dragonfly with ImageMagick doesn’t play nicely in a limited environments like heroku.

We were getting tons of R14 - Memory quota exceeded errors after analyzing even small images using ImageMagick’s identify command.

Here is how we solved it. READMORE

First of all the context.

We use direct S3 upload on the client side in order to reduce the heroku servers load. Client goes to the Rails server with a sign request and gets back a policy, a signature and a key (aka path) of the resource to be uploaded to S3. There are more details about jQuery-File-Upload flow here.

Once the file is uploaded client goes to ImagesController and creates a record for the image.

class My::ImagesController < InheritedResources::Base
  before_filter :authenticate_user!
  actions :create, #...
  respond_to :json

  # ...
end

“create” action receives the only parameter original_image_uid which is passed to the model.

class Image < ActiveRecord::Base
  belongs_to :user
  dragonfly_accessor :original_image

  def original_uid=(value)
    self.original_image_uid = value
    self.original_image_width = original_image.analyse(:width)
    self.original_image_height = original_image.analyse(:height)
    self.original_image_size = original_image.file.size
  end
end

This is where all the (image-) magic happens. Before the model is saved we analyze the image width, height and the file size in order to use later according to the application business needs.

First call to the original_image will download the file from S3, files can be upto a few megabytes, so it takes about a second in the production environment. Than original_image.analyse calls the ImageMagick’s identify command and cache its results.

So everything is quite straightforward. But, we started to get R14 errors on heroku after the images#create requests. We were under impression that some huge memory leak eats up all the memory, but it turned out that it was not garbage collected memory bloat that happens right after the identify command returns.

It looks like ImageMagick’s identify tries to get as much memory as possible with no particular reason from my perspective. So we had to fight these bloats in a few different ways.

First is to run garbage collection. Check out gctools, the only thing we had to do is to add these lines to config.ru

require 'gctools/oobgc'
if defined?(Unicorn::HttpRequest)
  use GC::OOB::UnicornMiddleware
end

It works with unicorn running on ruby 2.1. Learn more about it in the Aman Gupta’s blog.

And everything got back to normal, no R14 any more because the memory was cleaned up properly after each request.

But, why should we allow identify to take so much memory at the first place? And here comes the solution: passing limits to the identify command.

Another line of code added to initializers/dragonfly.rb

Dragonfly.app.configure do
  plugin :imagemagick, identify_command: "identify -limit memory 0 -limit map 0"
  #...
end

So, the ImageMagick doesn’t eat so much memory any more, and even if it does the bloat will be garbage collected after the request.

A few things worth mentioning

2014-07-20T21:53:50+00:00

I’m going to start a series of short digest blog posts that will cover a few things worth mentioning. I sumble upon a lot of things reading different sources, here I will share the most interesting ones. Well, at least most intersting for me. READMORE

Here we go.

Tools.

http://breach.cc/

A browser for the HTML5 era Entirely written in Javascript. Free. Modular. Hackable.

The browser is really nice especially its minimalistic design. Chrome Development Console works as usual, so it can really be an alternative for Incognito Tabs/Windows when debugging web applications that require to work on flows involving different logged in users.

https://github.com/google/ios-webkit-debug-proxy/

The ios_webkit_debug_proxy allows developers to inspect MobileSafari and UIWebViews on real and simulated iOS devices via the DevTools UI and WebKit Remote Debugging Protocol.

I’m really an Apple fun, but Google’s DevTools rock.

[http://macdown.uranusjr.com/] (http://macdown.uranusjr.com/)

MacDown. The open source Markdown editor for OS X.

I used MarkdownPro for a while, but this one is really cool and free.

http://iterm2.com/

iTerm2 is a replacement for Terminal and the successor to iTerm.

2.0 is released.

Security.

http://words.zemn.me/csp

Detecting login state for almost any website on the internet

I’m not sure if it worths it to re-implement Location-based redirects in all the projects I participated but big guys probably have to consider this.

http://minilock.io/

File encryption software that does more with less.

In short, users have only remember the passphrase, keys pair will be generated automatically based on the passphrase.

Development.

Where were a lot of buz about 2 charting libraries recently: http://fastly.github.io/epoch/ and http://www.chartjs.org/. Both are cool, I’m looking forward to testing both of them in a next project that will require charting.

http://component.kitchen/

Component Kitchen - Great ingredients for your web apps

Tons of useful code, all searchable.

http://customelements.io/

Custom Elements - a web components gallery for modern web apps

More or less the same.

http://www.polymer-project.org/

Welcome to the future - Web Components usher in a new era of web development based on encapsulated and interoperable custom elements that extend HTML itself.

Most exciting thing for front end development ever happened since Backbone.js. I’m trying to use it in the mobile verion of http://isratracker.com which will comes out shortly.

Design.

http://www.google.com/get/noto/

Google Noto Fonts - Beautiful and free fonts for all languages

Beautiful indeed, looks even better then Roboto.

http://designairs.com/inside-brand-evolution/

Inside our Brand Evolution

An interesting reading about airbnb rebranding.

Reading

http://venturebeat.com/2014/07/18/how-yo-became-one-of-the-most-viral-apps-of-all-time-step-by-step/

How Yo became one of the most viral apps of all time — step by step

http://www.logolounge.com/article/2013logotrends/

2013 Logo Trends

Writing Markdown With Style in Vim

2013-08-12T08:03:25+00:00

Boris recently purchased iA Writer to edit Markdown, which is our primary format used internally for readmes, proposal documents, contracts etc.

iA Writer looks very nice indeed. The typography is beautiful, and I especially like how it outdents headings and lists:

Still, I can’t help but find it impossible to do any serious text editing without the full Vim power by my side. READMORE

So I decided to try to at least approximate the experience of using iA Writer with Vim.

It turned out you can make the text look pretty decent by using a good font and bigger linespace. The “outdent” is out of the question of course (if anyone knows how to accomplish it I’d be very glad to hear that), but the rest is actually pretty good, with some extra formatting for lists, links etc.

I don’t like the ‘focus mode’ in iA Writer, so I didn’t try to replicate any of this, and not sure if its possible. Instead I use a different background color for the current line. Here is the result:

As you can see its not bad ;), and its Vim!

I’ve put the config into ~/.vim/writer.vim file that I can ‘:source ~/.vim/writer.vim’ when I’m working with Markdown:

set background=light
set nonu
set laststatus=0
colorscheme default
hi FoldColumn guibg=white
set foldcolumn=12
set linespace=8
set guifont=Source\ Code\ Pro\ Light:h20
set tw=80
hi Normal guibg=gray95
hi NonText guifg=gray95
hi FoldColumn guibg=gray95
hi CursorLine guibg=gray90
hi Title gui=bold guifg=gray25
hi MarkdownHeadingDelimiter gui=bold guifg=gray25
hi htmlSpecialChar guifg=black
hi markdownBold gui=bold guifg=gray25
hi markdownItalic guifg=gray25 gui=underline
hi markdownUrl guifg=#2fb3a6
hi markdownAutomaticLink guifg=#2fb3a6
hi markdownLinkText guifg=#317849
hi markdownUrlTitle guifg=#317849
hi markdownBlockquote guifg=#317849 gui=bold
hi markdownId guifg=#2fb3a6
hi markdownIdDeclaration guifg=#317849 gui=bold
hi markdownListMarker guifg=#317849
highlight Cursor guibg=#15abdd