The diary of a software developer :: Jakub Chodounský

Suggesting email address using string similarity algorithm

2020-01-14T02:56:07Z

Recently, we noticed a large number of unconfirmed accounts. Those were not spam accounts they looked like legit users. Are we losing customers that early in the funnel? When we looked closely we noticed a similar pattern:

john.doe@gnail.com
jane.doe@yahoo.co.nz
tim@outlook.con
susan@gmail.co.nz

All of those email addresses have a mistyped domain. That’s not great. I could imagine a customer refreshing their inbox, waiting for a confirmation email, and cursing the internet.

What could we do to improve the onboarding experience? What if we could detect typos and suggest a correct email for them?

Did you mean john.doe@gmail.com?

That would be really cool. And luckily for us there is a few neat algorithms that can help us with that.

In computer science this is an exercise of approximate string matching. And we will use a string distance (or sometimes called a string metric) to quantify how close two words – email domains in our case – are to each other.

Levensthein Distance

The most common string metric is Levenshtein distance (also called edit distance). It tells you the minimum number of edits – insertion, deletion, substitution – that need to take place to get to the same word.

For example gnail.com and gmail.com would have Levensthein distance of 1. You need to substitute one character to get to the same word. Those two strings are very similar.

We can normalise the distance between 0.0 and 1.0 where 0.0 is not similar at all and 1.0 is an exact match. That’s a good idea to do for different string lengths. For that, we divide the edit distance by the length of the longest string and subtract that from 1. For our example it would end up to be 0.88.

Levensthein distance is usually implemented with dynamic programming using a variation of Wagner-Fischer algorithm. Below is a Javascript version from Andrei Mackenzie.

var getEditDistance = function(a, b){
  if(a.length == 0) return b.length;
  if(b.length == 0) return a.length;

  var matrix = [];
  var i;

  for(i = 0; i <= b.length; i++){
    matrix[i] = [i];
  }

  var j;
  for(j = 0; j <= a.length; j++){
    matrix[0][j] = j;
  }

  for(i = 1; i <= b.length; i++){
    for(j = 1; j <= a.length; j++){
      if(b.charAt(i-1) == a.charAt(j-1)){
        matrix[i][j] = matrix[i-1][j-1];
      } else {
        matrix[i][j] = Math.min(matrix[i-1][j-1] + 1,
                                Math.min(matrix[i][j-1] + 1,
                                         matrix[i-1][j] + 1));
      }
    }
  }

  return matrix[b.length][a.length];
};

Trigram

Levensthein distance is a nifty algorithm but we found it didn’t work particularly well for our dataset. Also, it not easy to scale but for this use case it doesn’t matter.

There is a few other alternatives. We decided to go for Jaccard distance over Trigram vectors. It sounds scary but don’t be afraid.

Trigrams are three consecutive letter groups from a string. They are often used in natural language processing as they are relatively cheap to make and provide you with additional context of neighbouring characters.

Firstly, we decompose a string into trigrams.

trigram('hello');
['  h', ' he', 'hel', 'ell', 'llo', 'lo ']

And the code for the above.

var trigram = function(string) {
  const padded = `  ${string} `.toLowerCase();

  let gramStart = 0;
  let gramEnd = 3;
  const grams = [];

  while (gramEnd <= padded.length) {
    grams.push(padded.substring(gramStart, gramEnd));
    gramStart += 1;
    gramEnd += 1;
  }

  return grams;
}

After that, we can calculate the distance. It is a ratio of matching trigrams to their union.

var distance = function(aGrams, bGrams) {
  const matches = aGrams.filter(value => bGrams.includes(value));
  const uniqueGrams = [...new Set(aGrams.concat(bGrams))];
  return Number((matches.length / uniqueGrams.length).toFixed(2));
}

var compare = function(a, b) {
  const aGrams = trigram(a);
  const bGrams = trigram(b);
  return distance(aGrams, bGrams);
}

With the final number we need to decide what threshold is a possible match. I’d like to say that we did an extensive testing on our dataset to get the perfect distance but we haven’t.

Instead, we looked at Postgres pg_trgm extension and used their default of 0.3. And it worked out great.

Conclusion

Last step is hooking these functions up to our registration form. We extract the domain from the input, lowercase it, and remove any white space.

Then, we run it against a dictionary of our confirmed email domains that have at least a few users registered. That covers most of our basis and runs pretty quickly on a client.

And with that, we save a customer or two from abandoning the registration.

Resources

The Great Computer Experiment: Ruby Edition

2019-10-21T08:53:19Z

Can a second-hand desktop computer from 2013 run specs? How much does a touch bar speed up an integration suite? Do I need 6 cores to run rails new blog?

In this article I’ll put 4 machines and 5 different operating systems through a true test. What am I trying to achieve? I don’t know. But I intend to share my experience with you.

Computers

Recently, I’ve been playing with different computers and operating systems. I used to be a C# developer on Windows for quite a few years and after switching to Ruby I started using Macs. The direction Apple took with its latest laptops made me explore other options and I tried Linux on a desktop PC. And I was surprised how powerful and cheap desktops are these days.

Here are the machines that feature in the showdown. Prices are for New Zealand.

Custom-built PC (2019)

i7-8700 (3.2-4.6Ghz, 6 cores with 12 threads)
32GB RAM
Samsung 970 EVO Plus NVMe SSD
Ubuntu 18.04

Price: $1 500 ($950 USD) new

This is my main office workstation. It’s custom-built with decent components but nothing lavishly outrageous. It’s a perfect compromise of snappy performance and cost. It runs basic Ubuntu 18.04 installation with no customisations. Good reliable work horse.

Macbook Pro 13” (2013)

i7-4558U (2.8-3.2GHz, 2 cores with 4 threads)
8GB RAM
macOS 10.14 Mojave

Price: ~$700 ($450 USD) second-hand

The price is for a similar machine on the second-hand market in New Zealand. This has been my day-to-day machine for the last 6 years. I still use it when working from home or on the go.

Double amount of RAM would be better and maybe more cores but I’m not planning replacing it any time soon. It is a solid laptop with a great keyboard and beautiful screen. Recently, I was tempted by some of the new ThinkPads but I’m too scared of Linux on a laptop. If there is something I don’t want to do it’s tinkering with touchpad drivers, keyboard backlighting, and wonky wi-fi.

Budget Dell (2013)

i5-4570 (3.2-3.6GHz, 4 cores with 4 threads)
Crucial MX500 SATA 3 SSD
16GB RAM
Windows 10 WSL 2 Ubuntu 18.04
Ubuntu 18.04

Price: $250 + $70 RAM + $95 SSD ($260 USD) second-hand with new SSD and RAM

This PC is Dell OptiPlex 9020 that was released mid 2013 – close to my Macbook Pro. Machines like this are great if you are short on budget. They are build to last and cheap. I was lucky and got mine for free from an office giveaway and immediately replaced HDD for SSD. It came with Windows and I started using it for photo editing with Adobe Lightroom and Photoshop at home. Since then I added extra RAM and GPU to make Lightroom snappier (GPU is not included in the price though).

I will use it as a guinea pig for WSL 2 and bare metal Linux to compare the two. Is it worthy alternative for my trustworthy Macbook? Is WSL ready for serious work? Can we finally develop Rails on Windows?

Macbook Pro 13” (2018)

i7-8559U (2.7-4.5Ghz, 4 cores with 4 threads)
16GB RAM
macOS 10.14 Mojave

Price: ~$4 800 ($3 000 USD) new

Last year’s top of the line Macbook comes with a hefty price tag. I’m not a big fan of the keyboard and touch bar and this nugget belongs to my friend who helped me out with running some of the benchmarks.

It will be interesting to compare it to the custom-built PC.

Benchmarks

For each test I ran it 4 times, then discarded the worst value, and averaged the rest. While running the tests I was doing some light web browsing (if there is such a thing) and ran Slack to simulate happy developers’ day.

These are not your typical synthetic benchmarks but a real world performance. Including hackernews and epilepsy-inducing giphies.

Internal project

This is the most important test. The codebase that I work on day to day is the benchmark. It is a standard Rails monolith with React front-end and the suite is a combination of

rubocop,
webpacker,
rspec (single-threaded),
feature specs (headless chrome) and
jest.

Linux is the clear winner here. The 2018 Macbook Pro is ~10% slower than the PC. And WSL 2 is 1.4x slower than running native Linux. What a bummer.

What surprised me is how good the old Dell with native Linux is. It ran the suite in almost the same time as the custom-built PC from 2019. I don’t have a good explanation for it – I was expecting it to be worse. The CPU is slower (3.6GHz vs 4.6GHz) and the disk is slower (budget SATA 3 SSD vs NVMe).

Ruby

Next task is CPU intensive Ruby 2.6.5 compilation.

The 2013 Macbook Pro is showing its age here. The combination of 3.2Ghz with macOS are not cutting it. Again the bare metal Linux dominates and surprisingly the new 2018 Macbook Pro is lagging behind.

Devise

The popular gem for rails authentication has a small set of unit and integration tests. No parallelisation here either.

Linux PC is up by a mile but the WSL 2 on a budget hardware is beating 2018 Macbook Pro.

ActiveAdmin

This suite is a good mix of unit, integration, and UI specs. Also, it makes use of some parallelisation so the results might be interesting.

First win for macOS on the 2018 Macbook Pro – just by a second ahead of the Linux PC. Solid performance with budget Dell and WSL 2 is acceptable.

Mastodon

Last one is running the social network test suite with a mix of unit, integration, and UI tests. It is a good representation of standard Rails project.

Unfortunately, it’s missing numbers from the 2018 Macbook Pro but let’s compare the remaining.

The Linux PC is claiming the title back here and budget PC is lagging behind by more than 40s. There is a performance hit on the WSL 2. With 2013 Macbook Pro you are better off running the suite in your CI pipeline.

Conclusion

Firstly, all these machines are perfectly capable of developing Ruby and Rails applications. While coding you might be running a spec or two and leave the whole suite to your CI. That makes even a 6 years old laptop acceptable while on the road.

What surprised me is how good the old Dell with bare metal Linux performed. You can get this machine for so little money and have an excellent PC for work.

Linux in general is not such a hassle as it used to be. As long as you keep things simple and stick to the vanilla settings. I’m not too demanding and I avoid common pitfalls (high DPI displays, power management, customising UI, etc.) so it works for me. Not having an official OneDrive client and Office sucks though.

WSL 2, while still in preview, is a decent option and I didn’t run into any issues with it. In combination with the new Windows Terminal and Visual Studio Code it is a joy to work with. It might be the end of Macbook domination in the laptop space for web developers. Still you are getting a performance hit but Windows ecosystem might be worth the trade (or not … depending on your personal preferences).

Progressive web application as a share option in Android

2019-03-23T21:28:57Z

Progressive Web Applications are becoming more and more powerful and now can be registered as a share intent on Android devices. This means that any Android application can share directly to your web app through the standard native sharing dialog.

It also works the other way around. Your PWA can trigger the native share dialog and send data to other applications. This brings them one step closer to native apps.

In this article we will focus on becoming the share target.

Can a progressive web app be registered as a share option in Android?

For a long time this wasn’t an option. But with the release of Chrome 71 on Android there is an experimental support of the Web Share Target API. Your PWA can appear in the sharing menu. How cool is that?

Let’s have a look at what you need to get your website into the Android sharing dialog:

Manifest
Service Worker
Share target

Becoming progressive

The first steps are really about becoming a progressive web application. The share intent will be registered when the user “installs” the app by saving it to the home screen – either from a popup or manually.

Important note: Your website needs to be served over HTTPS.

For that you will need a website manifest and a registered service worker. Firstly, let’s have a look at a simple manifest.json.

{
  "short_name": "Sharing Demo",
  "name": "Web Target Sharing Demo",
  "icons": [
    {
      "src": "/android-chrome-192x192.png",
      "type": "image/png",
      "sizes": "192x192"
    },
    {
      "src": "/android-chrome-512x512.png",
      "type": "image/png",
      "sizes": "512x512"
    }
  ],
  "start_url": "/",
  "background_color": "#ffffff",
  "display": "standalone",
  "theme_color": "#ffffff"
}

You can read about all the different values and options in more detail here. After that, we will need to link to our manifest in our HTML files. So add the following into head section of your page.

<link rel="manifest" href="/manifest.json">

The Second thing for an app to become a PWA is a registered service worker with a fetch event. It doesn’t need to do much, it just has to be there. The simplest way to do that is to create service-worker.js with an empty fetch listener.

self.addEventListener('fetch', function(event) {});

It won’t do anything but it is everything we need for now. Next, we will register the service worker in our page.

if ('serviceWorker' in navigator) {
    navigator.serviceWorker.register('/service-worker.js')
      .then(function(reg){
        console.log("Service worker registered.");
     }).catch(function(err) {
        console.log("Service worker not registered. This happened:", err)
    });
 }

Nice work! It wasn’t too hard and we changed a simple website into a progressive web app. If you are using HTTPS you should be able to see the Add to home screen popup when you visit your website.

You can use Chrome Dev Tools to check on the manifest and see if the app can be installed. Go to Application -> Manifest or use Lighthouse and see the Installable section of the report.

Converting your website to a progressive web application was actually the hard part. Registering the share intent so your app will appear in the native dialog is super easy.

You need to add the following lines into your manifest.json that we created previously.

  "share_target":
  {
    "action": "/share",
    "params":
    {
      "title": "title",
      "text": "text",
      "url": "url"
    }
  },

Every time a user will share something through the dialog the request will go to /share endpoint of your website with query parameters of title, text, and url.

Note: If you are sharing something from mobile Chrome the URL actually comes through as a text.

Now we put everything together and this is the final version of the manifest.json file with registered share_target section.

{
  "short_name": "Sharing Demo",
  "name": "Web Target Sharing Demo",
  "icons": [
    {
      "src": "/android-chrome-192x192.png",
      "type": "image/png",
      "sizes": "192x192"
    },
    {
      "src": "/android-chrome-512x512.png",
      "type": "image/png",
      "sizes": "512x512"
    }
  ],
  "share_target":
  {
    "action": "/share",
    "params":
    {
      "title": "title",
      "text": "text",
      "url": "url"
    }
  },
  "start_url": "/",
  "background_color": "#ffffff",
  "display": "standalone",
  "theme_color": "#ffffff"
}

After that, you can use any method of reading the query parameters that suits your needs. Either server-side like this with Ruby on Rails:

def share
  @url = params[:url]
  @title = params[:title]
  @text = params[:text]
end

or client-side with a bit of good old javascript:

var parsedUrl = new URL(window.location.toString());
console.log('Title shared: ' + parsedUrl.searchParams.get('name'));
console.log('Text shared: ' + parsedUrl.searchParams.get('description'));
console.log('URL shared: ' + parsedUrl.searchParams.get('link'));

I’m a big fan of progressive web apps and the web share target makes the integration with your mobile workflow much smoother. It comes with a few caveats of being experimental so the API will most likely change or it could totally disappear. Also, keep in mind that (at the moment) it’s supported only on Android devices and your Apple users won’t benefit from it . The proposal takes iOS into account though so maybe some time in the future. I’ve been waiting for this feature for a while and I’m super-excited that it made it to the Android community.

Reading list 2017

2018-03-24T20:26:40Z

As I shared my reading list last year I’d like to do the same for 2017. Hopefully, you’ll find an interesting book or two to add to your list.

Technology

Coders at Work

This book is full of interviews with people that were influential for computer science, algorithms, and operating systems. I was expecting more people closer to application programmer’s reality. Don’t get me wrong, I have a huge respect for people like Donald Knuth or Guy Steele, but it’s not what the majority of us do every day. I was seeking some inspiration and found none. Well, at least I got some insight into a few groundbreaking projects like Netscape, Javascript, Erlang, or Unix.

The Complete Redux Book

This book is, more or less, a written version of Dan Abramov’s Redux course where he builds the library from scratch. It’s a quick read that helps you understand the core principles of Redux architecture.

Business and self-development

I’ve read too many junk business and self-help books over the last few years. The ones that give you an insight into an unknown field are worth reading. But the ones that are like “I have it all figured out”, “What worked for me will work for you”, and “(Don’t or Do) Follow your passion” seems to be a waste of time. Read one and skip the rest.

Ten-Day MBA

The book looks at various aspects of the business—from marketing, ethics, accounting to operations, finance, strategy, and others. It’s a great read for a programmer as you are usually working with those teams and the book will give you a common language and a better understanding of each other. And also, MBAs really like to use acronyms and things like the plan of five P or a five S strategy—mention it (or a quote The Art of War) in a meeting and you’ll be taken seriously.

So Good They Can’t Ignore You

A book that tells you that you won’t be happy by following your passion. That is a good news for people who are not sure what their passion is. You pick something, get better in it, build a career capital that allows you to get more freedom, and find a mission that you can fulfill with the capital you built. That’s how you gonna love what you do. It makes sense, more sense than blindly following your passion and dreaming of becoming a lumberjack or a farmer in your cubicle.

The Subtle Art of Not Giving a F*ck

An antidote to all of those self-help be happy books—you are not special, you need to lose, you need negative experiences. You won’t achieve much by only being positive. Care about things that are dear to you and ignore what TV and social media tell you, etc, etc.

Buddhism dressed up a little for millennials

And it’s orange and has a swear word in the title so it’s gonna be a best-seller.

Traction: A Startup Guide to Getting Customers

A book with concrete advice on how to grow your audience through different channels. Each channel has a small case study and tells you at what stage and type of startup it worked. Not all the channels will be applicable to your business but this gives you an idea which ones to try out.

Traction

I’m gonna admit it. I wanted to read the startup Traction but I picked up this one instead by mistake. And I realized that there is something fishy going on when I was already in a third of the book. Sunk cost fallacy kicked in and I finished the book.

The result of that is gaining better vocabulary when talking with C-level execs. Apart from that, it gives you a framework to apply in your company to improve the company. If you are a CEO. Oh well, I’m not a CEO. So I can’t really tell you if it’s working or not. Sorry.

The 4-Hour Week

Firstly, the tone of this book makes you cringe. And apart from that, there is not really much that would stick with me. It tells you to define your goals, take mini-retirements/sabbaticals, be effective, cut out unimportant, automate, don’t multitask, and many other things that I can’t argue with.

Also, it tells you to learn to leave things unfinished—I should’ve read that before accidentally going through the CEO Traction, hey?

Corporate Confidential

An eye-opening introspection into a corporate culture and HR practices that is both chilling and depressing to read. But, it’s still better to know the other side than to ignore it. You can decide whether you want to play along or face the consequences. It’s coming from an American background but it applies to Europe and New Zealand to a certain extent as well.

The Barefoot Investor

With a cheesy title comes a book with cheesy text and big letters. I’ve read this one because of work. The style might suit some and the general advice is not too bad. The benefit of the book is targeting Australians and capturing the specifics of their retirement plans, taxes, and others. Oh boy, and those testimonials at the end of each chapter—does anyone apart from the author likes to read them?

Non-fiction

Vietnam: A History

A long and very thorough description of events that lead to the US involvement in Vietnam and, eventually, to the war. Stanley Karnow lived there through those years and worked there as a journalist. The book is filled with personal experience with key actors and gives a lot of detailed background but doesn’t really go into many details of why.

Innovators

A decent audiobook to listen to—well narrated. Isaacson describes stories around the major breakthroughs that led to the current state of computer technology. From Ada Lovelace to the internet. He focuses on the people and teams and gives lots of details about the personality of each inventor.

The Short History of Nearly Everything

An interesting book covering the history of various fields of science from physics, cosmology, to chemistry, and paleontology. It tells you how we’ve came to know as much as we do now. The style is accessible and, despite the range of the topics, you are not getting lost. It is fascinating and informative read.

Liar’s Poker

It is an account of greed in the investment banking industry in the 80s at Wall Street that the author lived through. The story is told by a graduate, the author, joining Salomon Brothers and becoming a bond salesman. He hated investment bankers but he still kept screwing the clients and getting the “fat paychecks with bonuses” coming each year to his bank account.

Never before have so many unskilled twenty-four-year-olds made so much money in so little time as we did …

There is a slight hint of what led to the financial crisis in 2008 which is described in another of the author’s books—The Big Short. Lastly, Lewis has a real talent for explaining complex financial instruments with simple and easy to understand analogies (e.g. mortgage-backed securities).

Salt: A World History

I didn’t have any idea how much a common thing like salt influenced the world history. Back in the day, having access to enough salt allowed you to preserve more food. More food led to feeding more people. Feeding more people get you bigger armies. Armies won wars. Winning wars gave you more power.

Another interesting part of the book is how different cultures used different means of salting and preserving the food. From fish sauce in the Roman empire and still in use in Asian cuisines with soy sauce to a plain use of crystals.

Salt was worth more than gold.

Masters of Doom

A book that goes to the roots of id Software and takes you to the journey of the most influential games of the 90s—Wolfenstein 3D, Doom, and Quake. It draws the relationship between John Carmack, John Romero, and Tom Hall. It’s a great story and if you played those games at that time you won’t be able to put the book down. I listened to an audiobook and it was perfectly narrated.

Eating on the Wild Side

This is probably the geekiest book that I read last year. It tells you where our modern vegetable came from, what we lost and gained through the selection and breeding, and how to choose the most nutritional varieties. Also, it shares many other tips like how to store it (don’t put tomatoes in the fridge), or what is the best way to prepare so you get the most nutrients (cook your tomatoes). I love learning about things that you can use (and eat) every day.

Everybody Lies

It shows you what random facts you can learn by studying Google searches dataset. Those would be presented with some funny and some quirky moments. At the end, there is a discussion about the limitation of the data analysis.

Touching the Void

A chilling story about two climbers facing a tough decision. It goes to a description of what goes through one’s head in an almost impossible situation. It’s a raw encounter with powerful unforgiving mountains.

Fiction

I, Robot

A few short stories from a near future about robots—it is an entertaining book filled with logic around three laws of robotics that Asimov defined. The book was written in 1950 but somehow feels up to date.

Fear and Loathing in Las Vegas

A surreal story of seeking American Dream by going to Las Vegas with a pack full of drugs. Two guys on drugs doing crazy things described in a great style and language. I loved the book but I hope it is not based on a true story. I’m slightly suspicious that it is (at least partially). There is a movie version with Benicio Del Toro and Johny Depp—a great adaptation of the book.

Recommendation

Those two are not very entertaining but they will help your career. The Ten-Day MBA gives you a solid understanding of different parts of a company. It will help you make better decisions as a programmer and have more empathy for others.

The latter was depressing but if you aim to climb a corporate ladder it’s an eye-opening experience into a big companys’ culture and politics.

Masters of Doom

I just loved that book — totally recommended if you haven’t read or listened to it yet. Growing up with these games and following the authors in the game magazines throughout the 90s makes it even more fun.

I love learning about ordinary things and I would have never thought that salt had so much impact on the history of mankind. And Bryson’s book gives you an entertaining peek into the history of science with lots of details presented in a cohesive way.

One is a surreal road trip to a pop culture of the 70s filled with substance abuse, and the other is a cruel raw encounter with a mountain. Both books are like a bizarre car wreck — you are chilled to the bone but unable to turn away.

And what have you read last year? What book would you recommend to put on the reading list for the next one?

Tips for getting a programming job abroad

2018-03-13T06:42:38Z

A few years ago I moved from the Czech Republic to New Zealand. I didn’t have a network, a job offer, or a scheduled interview.

There are lots of benefits of moving to a different country. You are opening yourself to different ideas, mindset, and experience. You can learn the language, better understand a foreign culture, and adopt new ways of doing things. It is a scary step but one worth doing.

I had never quit a job without another offer. And now I was moving across the globe with anything in my hands. In the end, I managed to find a great job. Sorted out a residency. And have an awesome career. I’d like to share a few tips that will make it easier if you decide to do the same.

Research

It is important to do your own research. Firstly, you need to figure out how much the cost of living in your destination is and how much money you might earn.

I found Numbeo to be a great source of information. You can compare the numbers in your current location and adjust them if they are not that accurate. Remember that you will learn by trial and error so the first few months might be more costly than you planned.

Also, searching for “Cost of living in [destination]” might lead you to other useful resources from expats, travel bloggers, and even governments.

Now it is time to figure out how much you might earn. There are heaps of global sources on this topic like PayScale or Glassdoor but the best place to start is a local job board. Some posts will have a salary range and you will get the feel on how many jobs are out there. For New Zealand there are two major boards: Seek and TradeMe.

Another great place to get information about wages is a report from local recruiting agencies. A few of them publish salary reports for different IT positions and technologies.

Money is important but don’t forget to take into account other factors that are dear to you. It could be pollution, commute time, work-life balance, ocean, mountains, sun, health care, active community, and many others. For that, I also recommend Numbeo’s indexes: quality of life, pollution, and a bit more time spent on Google. Seriously, I didn’t know that Wellington is one of the windiest cities in the world.

While you are scanning the job boards you will get an idea about the popularity of your tech stack. More opportunities equal to less stressful transition. And remember, your first position doesn’t have to be a dream job. A stable job that helps you with your visa situation and a good work-life balance can be a great choice for the first few years while you are settling in the new country. On the other hand, if you are highly specialized (e.g. research in computer graphics algorithms) and there would be a company that could use your skill (Wellington is a home of movie effects) it could be easier.

When you want to work in another country you will need a work visa. If you are moving from one EU country to another it is very simple thanks to the Schengen area. Similarly, New Zealand has a pact with Australia and I bet other countries do as well.

A few countries (Australia, New Zealand, Canada, and others) have a working holiday program which might be a good starting option if you match the entering criteria.

Getting the visa can be the biggest hurdle but there is a shortage of good programmers all around the world so companies might go above and beyond to get your skills. Keep in mind that migrants are still a big risk for the company and getting them onboard is time-consuming and not that cheap.

Building a network

To find a great job it helps to have a network. The best positions could come from your friends and colleagues you worked and built great relationships with. But how can you get around that when you have no connections and starting from the ground?

There are a few things that you can do before you arrive at the new country. It won’t be as effective but it will do the trick.

Start making connections by cold emailing a few people from local communities. Find out if there is a meetup group for your stack (an example could be the WelliDotNet group in Wellington). Another place where programmers hang out “locally” could be Slack channels, IRC, or Facebook groups. Do a bit of a legwork and find those places. If all fails there is always LinkedIn. People are generally pretty helpful even to a complete stranger. A polite and informed stranger though.

Choose people that seem to be active in the community—explain your current situation and show them you did some research and put an effort into the message. Ask a few and easy to answer questions and start building that relationship. Best people to contact could be past presenters, organizers, administrators, or just the ones who seems to be answering others’ questions. Get in touch with at least a few different people to “diversify” so you limit a chance of reaching out people inside a bubble.

Hi Jane,

my name is John and I’m a .NET programmer from Prague. I’ve been programming for over 5 years and mostly in bigger companies in healthcare. I’m looking to move to New Zealand and was wondering if I could ask you a few questions.

I love working with diverse people that care, on a product that helps. Bigger companies might be a better choice for me as I’d like to have some stability to get my family settled first.

It seems that there are a few big .NET companies in Wellington. Xero, TradeMe, and Datacom. Are those nice places to work? Would you recommend any others? Should I stay away from some of the local companies?

There is also lots of opportunities in Auckland, but people seem to prefer Wellington. Would you recommend Wellington over Auckland as well?

Thank you for your time,
John

Once you moved it’s nice to get in touch with those that helped and buy them a coffee or a beer. A simple “thank you” will also do if you run into them at one of the meetups.

Applying

From my experience companies and recruiters won’t take you seriously until you move (unless your destination is just a short flight away). You can start sending CVs and cover letters but don’t feel bad if you are not getting any response. It will change once you arrive.

Always mention your visa situation in your CV and note it in your cover letter as well. If you have a work visa it will be to your advantage and if not at least you are not putting the company on the spot with an unpleasant surprise (some of them can’t afford to sort the visa out for you).

When you are new to the country you might fall into a certain bias and getting your first job will be harder than the next one. Employers might worry that you don’t share the same work culture and ethics, and will have a language barrier. Because of that, you might not get jobs that you are a perfect fit. If that happens to you don’t worry too much about it and don’t get discouraged.

Another point to consider is mentioning your family (or partner) situation. That’s a fair point because a common thing I’ve seen is that programmers (who get the job) arrive with their partners that can’t find any job. The partner becomes unhappy in a few months, and want to leave back to their country. The programmer will usually follow them and the employer loses a member of the team they invested in and need to start hiring again. To mitigate the risk show commitment to staying in the country and if you are not alone mention what’s the plan for your significant other. You don’t have to be too specific—just the fact that you thought about it shows a lot.

Lastly, it is easier to follow up with the leads you gathered through remote networking than applying to recruiters and job ads. But don’t frown upon any of those and use all the channels that are available.

It won’t be easy but if you persist you will find a good place to work for.

Tips

I can’t stress enough the importance of having enough money in your bank account. At least a few months worth of rent and food (and possibly a return ticket back home) will help your confidence and greatly reduce the already stressful experience. It will allow you to take your time to interview for different jobs, find a place to live that you will enjoy, and helps to settle in. You want to take enough time and don’t rush anything.

Make sure you have a plan for your family and your significant other. If they are following you and they won’t have a job it will be very hard for them to be happy. Remember that they won’t have a network of longtime friends and family to lean on as they would in your original country.

If you want this experience to be a success and not a stressful adventure do your research upfront and be prepared. And even if you do your homework there will be lots of surprises.

Lastly, when you have multiple offers to choose from you should think about a few additional things then you’d normally do. If you are on a temporary work visa is the employer willing to help you with getting the permanent one? You might be better off joining a bigger team to expand your professional network than a small startup with just a handful of people. Also, don’t underestimate the social aspect of the job—it might be the place where some of your work relationships transform into your first local friendships. Maybe in a diverse environment that supports immigration you will find people in similar situation and that will bring you together.

Working in a different country is hard, stressful, but in the end, it is worth it. Good luck!

Garbage collection in C#

2017-05-01T21:06:59Z

When you are programming, no matter the task on hand, you are manipulating some data. These are stored in basic types and objects and they live inside computer memory. Eventually, the memory fills up and you need to make more room for new data and discard the old one.

You can do it either by hand, like C and C++ programmers (used to) do, or use a languages with a mechanism that does it for you. In C# we are fortunate enough to have a garbage collector that takes care of our memory.

Basic concepts

In short, garbage collector tries to find objects that are no longer in use by the program and delete them from memory.

The first garbage collected language was LISP written by John McCarthy – one of the founders of artificial intelligence – around 1959. So the concept has been around for a while.

When you have a garbage collector in place you are making a trade off of safe memory allocations and programmer’s productivity for performance overhead. Garbage collected language will never be suitable for real-time critical applications like air traffic control, but you can get solid performance out of it – I’ve seen successful high frequency trading platforms written in C#.

Garbage collection is not only responsible for cleaning the memory and compacting it, but also allocating new objects.

.NET and CLR are making use of tracing garbage collector. That means that on every collection the collector figures out whether the object is used by tracing every object from stack roots, GC handles and static data.

Every object that could be traced is marked as live and at the end of tracing the ones without the mark are removed (swept) from the memory and it gets compacted. This is called a simple mark and sweep algorithm.

It gets a bit more complex than that but this is enough for having a simple mental model of what’s happening under the hood.

If the garbage collector halts the program when running we call this event stop the world. That guarantees that no new objects are allocated during collection, but it also means that there will be a delay in our program. To minimise that disruption incremental and concurrent garbage collectors has been designed.

Incremental collection makes the pauses shorter and works in small increments and concurrent doesn’t stop the program at all. For that you are paying with the overall longer collection time and the use of more CPU and memory resources than traditional stop the world.

Generational collection

In 1984 David Ungar came up with a generational hypothesis that layed the foundations of GC that are in use these days.

Young objects die young. Therefore reclamation algorithm should not waste time on old objects.
Copying survivors is cheaper than scanning corpses.

This hypothesis gave birth of the generational garbage collectors – first used in Smalltalk and used in all modern platforms today – C# is not an exception.

The memory is divided into several generations based on the age of objects. The collection happens in the youngest generation and the surviving objects are promoted to the generation above.

In C# we have 3 generations:

Generation 0 is the youngest generation with short-lived objects in which the collection happens the most often. New objects are placed here with the exception of large objects that go straight to the Generation 2.
Generation 1 serves as a buffer between the short-lived and long-lived objects.
Generation 2 is the place for the long-lived objects and it gets collected less often.

When there is a collection over Generation 2 we call it full garbage collection as it goes through all objects in managed memory.

To keep the balance between generations most modern platforms are using a hybrid approach of generational cycles (minor cycle) and some variation of full mark and sweep (major cycle).

The CLR automatically adjusts itself between not letting the used memory get too big and not letting the garbage collection take too much time by setting thresholds for new object allocation.

The collection gets triggered when

memory is low,
threshold is exceeded or
GC.Collect is called manually.

When the collection is triggered every thread is suspended, so it can’t allocate new memory, and the GC thread goes into action.

As you already know the collector uses static data to know whether the object is alive or not. As the static fields are kept in memory forever (until the AppDomain gets unloaded) this might lead to a common memory leak when it refers to a collection and you keep adding objects to the collection. The static field keeps the collection alive and all the objects inside alive as well. Be careful about that and possibly avoid the usage of static fields if you can.

Characteristics

Modern garbage collectors have different characteristics and as with many things in programming – different trade offs. Based on the application requirements they could be tuned towards less interruptions and better user experience or maybe you want to run a fast server-side application that scales.

What characterizes a Garbage Collector?

Program throughput: CPU time spent collecting vs CPU time doing useful work.
GC throughput: amount of data the collector can clear per CPU time.
Heap overhead: additional memory the collector produced during collection.
Pause times: time needed for collection.
Pause frequency: how often does garbage collection happen?
Pause distribution: short pauses vs long pauses, and how consistent the pauses are.
Allocation performance: how much time does the new object allocation take? Is it predictable?
Concurrency: does the collector make use of multicore CPUs?
Scaling: can your GC handle larger heaps?
Warmup time: can a collector adjust to the characteristics of your application? How long does it take to adjust?

Configuration

You can influence the type and behavior of your application by setting garbage collection to a different type.

There are two major types – workstation and server. They could be configured by setting <gcServer enabled="true|false" /> element in your application configuration.

<configuration>
   <runtime>
       <gcServer enabled="true"/>
   </runtime>
</configuration>

The workstation setting is tuned for client-side applications. Low latency is preferred as you don’t want your forms in WPF application become unresponsive during a long pause caused by garbage collection. In this mode it favours user experience and less CPU usage.

On the other hand, the server configuration is designed for high throughput and scalability of server-side applications and it uses multiple dedicated threads for garbage collection with the highest priority. It is faster than workstation but that also comes with more CPU and memory usage. This configuration assumes that no other applications are running on the server and prioritise the GC threads accordingly. Also, it splits the managed heap into sections based on the number of CPUs. There is one special GC thread per CPU that takes care of the collection of its section. You can run this mode only on a computer with multiple CPUs.

By default, the workstation GC mode is active.

Another option you can choose is to use concurrent collection by using the <gcConcurrent enabled="true|false" />. It will give you more responsive application with less pause frequency and time.

<configuration>
   <runtime>
       <gcConcurrent enabled="false"/>
   </runtime>
</configuration>

In concurrent mode, there will be an extra background thread that marks the objects alive when your application is executing its code. When it comes to collection, it will be faster as the set of dead objects has already been built.

The concurrency only affects the generation 2. Generation 0 and 1 will always be non-concurrent as they both finish fast and are not worth the extra overhead. When concurrent option is set, a dedicated thread will be used for collection.

To improve performance of several processes running side by side, you can try using workstation garbage collection with the concurrent collection disabled. This will lead to less context switching and therefore better performance.

Performance

The memory is divided into small object heap and large object heap. Large objects are the ones above 85 000 bytes and they go straight into the Generation 2.

In general, fewer objects on the managed heap is less work for garbage collector. That is applied especially to the large object heap that gets collected less often.

If you feel like GC causes you troubles there are various tools that you can use to debug it – memory performance counter, WinDbg or tracing ETW events.

Triggering collection manually with GC.Collect is usually more counterproductive than beneficial unless you really know what you are doing. It might help in certain situations when you want to release a lot of memory at once (e.g. a large dialog is closed and won’t be used again) – but unless you have diagnosed a memory problem, don’t go there.

If you are calling GC.Collect manually too frequently you will certainly see a decrease of performance in your application.

When you get into situation where the latency of your application is critical – like in the middle of the high frequency trading decision loop – you can set a low latency mode. This will put the GC into a low intrusive mode when it will become very conservative.

GCLatencyMode oldMode = GCSettings.LatencyMode;
System.Runtime.CompilerServices.RuntimeHelpers.PrepareConstrainedRegions();
try
{
    GCSettings.LatencyMode = GCLatencyMode.LowLatency;
    ...
}
finally
{
    GCSettings.LatencyMode = oldMode;
}

The collection of Generation 2 will be paused for the time in low latency mode. That leads to shorter and less frequent pause times and therefore decreased program latency. But that comes with the price that you might run out of memory if you are not careful. The best guideline is to keep those low latency sections as short as possible and don’t allocate too many new objects, especially on the large heap. In this case you might want to call GC.Collect where appropriate to force the Generation 2 collection.

Don’t forget that the latency mode is process wide, so if you are running multiple threads it will be applied to all of them.

Managed and unmanaged resources

In CLR all the code that is written is managed and it will be garbage collected. C# gets compiled into CIL and runs on top of CLR so all your C# code will be garbage collected.

That sounds easy enough until you need to use unmanaged resources.

And you will use them on almost daily basis as these are database connections, COM interops, network and file streams and many others. Unmanaged resources are not garbage collected and you need to free them from your memory by hand.

If you write a type that uses an unamanged resource you should

implement the dispose pattern and
provide a mechanism to free the resource in case consumer forgets to call Dispose.

The basic implementation of a dispose pattern with the use of SafeHandle could look like

public class DisposableResourceHolder : IDisposable
{
    private SafeHandle resource;

    public DisposableResourceHolder()
    {
        this.resource = ...
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    protected virtual void Dispose(bool disposing)
    {
        if (disposing)
        {
            if (resource != null) resource.Dispose();
        }
    }
}

To free the resource you can use Object.Finalize or SafeHandles. For more details check out Cleaning up Unmanaged Resources.

As a consumer of a type with unmanaged resource always use the using pattern.

using (var s = new StreamReader("file.txt"))
{
  ...
  s.Close();
}

How can you tell the object needs to be disposed? It implements the IDisposable interface. Some static analysis tools can help you notice the cases where you might have forgotten to dispose your objects.

What about other languages?

The first versions of Ruby used a simple mark and sweep technique which isn’t the best performer but it was simple for the authors of C-extensions to write native extensions. That simplicity was the key to Ruby’s growth in the beginning and took it where it is these days. Generational collection was introduced in Ruby 2.1 (current version is 2.4) to improve the throughput of programs – quite late in terms of language maturity. After that Ruby 2.2 introduced incremental marking which addresses long pause times by running the GC in short increments.

Programmers in C++ don’t have any garbage collection and they need to do memory management by hand. There are techniques to make that easier – smart pointer are now part of the C++ 11 standard and are a tool that automatically deletes memory from the heap. Smart pointer can be implemented with reference counting that ensures that the object is deleted as soon as it is no longer needed – as opposed to tracing garbage collection when the unused object waits for the next cycle of collection.

Java is very similar to C# and uses tracing generational garbage collection – the heap is structured to a young generation (eden, S0 and S1), old generation and permanent generation. GC uses minor and major cycles to clean the generations and promote objects from one to the other. Both the collections of young and old generations are the “stop the world” event so all the threads are suspended until they finish. There are different types of collectors on JVM – serial, parallel, concurrent mark and sweep and its replacement G1 in Java 7.

Python doesn’t use tracing garbage collection but a reference counting with periodical cycle detection to solve the case of two dead objects pointing to each other. But as opposed to classic reference counting it combines it with the generational approach and uses the reference count instead of the mark and sweep algorithm. It doesn’t deal with memory fragmentation but tries to avoid it with allocating objects on different pools of memory.

In Javascript there isn’t a unified approach to garbage collection. It is in hands of browser vendors and they do it differently – Internet 6 and 7 used reference counting garbage collectors for DOM objects. From 2012 almost all modern browsers are shipped with tracing mark and sweep algorithms with some extra improvements – generations, incremental collection, concurrency and parallelism.

Conclusion

C# and languages on top of CLR are garbage collected. They use generational garbage collection with 3 generations and an advanced tracing mark and sweep algorithm to figure out which objects are still alive.

You can change the characteristics of the collector by changing to a server or workstation mode and setting up a concurrency mode.

Most of the times triggering garbage collection manually is not a good idea unless you really know what you are doing. And in cases where latency is important you can use latency mode to increase your program throughput and eliminate pauses.

Lastly, there are unmanaged resources like file streams, network, database connections that you need to take a special care of. Most of the times you’ll be implementing and using the IDisposable pattern so you don’t cause any memory leaks in your application.

If you liked this article you might enjoy C# Digest newsletter with 5 links from the .NET community that I put together every week.

Resources

Comments

Justin Self: Nice Post. Minor note: Large objects (over 85,000 bytes) get collected during a Gen 2 collection but are not really part of the Gen 2. They are part of the Large Object Heap.

Automate your macOS development machine setup

2017-01-19T20:30:21Z

My laptop was running a bit slow and there was an OS upgrade that I’ve been postponing for quite some time. So, on one rainy Friday afternoon, I decided to get my hands dirty and do a clean install of macOS Sierra. And why not automate it so I don’t have to think about it next time?

One of the benefits is that my team mates can use chunks of the configuration that are relevant to our work and use it for themselves.

Let’s have a look at the tools that we will use.

Dotfiles

It’s a bunch of configuration files and directories that are in your home directory. For example vim uses .vimrc or a git uses .gitconfig. You want to bring them with you from your old machine.

Some people store them as a git repository but I just chuck them on Dropbox. Make sure that you are not revealing any sensitive files like your private SSH keys or infrastructure setup.

When you setup your new machine you create symlinks to the ones that you like to use. It can get pretty fancy or you can keep it simple with ln -s.

Brew

Homebrew is a package manager for macOS. And you can create your own bundles to set up whole environment.

Apart from that there is an extension for installing desktop applications — cask. You can search for the recipes with the following commands.

brew search vim
brew cask search google-chrome

When you find the mix of applications you bundle them together into a Brewfile.

cask_args appdir: '/Applications'

tap 'homebrew/bundle'
tap 'caskroom/cask'
tap 'homebrew/services'

brew 'vim'
brew 'tmux'
brew 'git'
brew 'ruby-build'
brew 'rbenv'

cask 'google-chrome'
cask 'vlc'
cask 'dropbox'

I like to keep my Brewfile in my Dropbox folder with the rest of the dotfiles. Now with the file you can run brew bundle and it installs everything for you. Easy.

Everything together

We have our configuration with dotfiles and we can install all the necessary applications with brew and cask. How about putting everything into one script that we can run and leave the computer do its thing?

I created an install.sh file that I keep with the rest of the files.

#!/bin/bash

xcode-select --install

# dotfiles
ln -s ~/Dropbox/dotfiles/.bash_profile ~/.bash_profile
ln -s ~/Dropbox/dotfiles/.gitconfig ~/.gitconfig
ln -s ~/Dropbox/dotfiles/.tmux.conf ~/.tmux.conf
ln -s ~/Dropbox/dotfiles/.vimrc ~/.vimrc

# brew stuff
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
cd ~/Dropbox/dotfiles
brew bundle
cd ~

git clone https://github.com/VundleVim/Vundle.vim.git ~/.vim/bundle/Vundle.vim
git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm

# ruby
rbenv install 2.2.5
rbenv global 2.2.5

gem install bundler

echo "******************** Done ********************"
echo "Don't forget to configure SSH properly with key and config"

Firstly, it installs XCode command line tools — if you need to install the full version you might need to do that through AppStore. I haven’t found any way around that yet.

After installing XCode the script takes care of the dotfiles and links them into your home directory. And when those are sorted it starts the brew magic.

There is a few custom commands afterward — installing plugin managers for tmux and vim and installing ruby with rbenv. You can also use terminal to customize macOS behaviour or do other things that you would normally do by hand.

Conclusion

And with that a fresh OS install is not a pain anymore. After finishing the installation and encrypting your disk you copy your backed up Dropbox folder into your home directory, run

. ~/Dropbox/dotfiles/install.sh

and you can go for a surf, come back and have your computer ready. Unless you need to compile Qt with Webkit. Then you might go for two surf sessions at least.

Comments

Victor Maslov aka Nakilon: My relevant gist

Gabriel Filipe Gizotti: fresh helped me a lot

ege: Brewfile also let you add apps from Mac App Store thanks to github.com/mas-cli/mas. But doesn’t work with 2FA-enabled account yet. I hope devs find a solution.

brew 'mas' / brew install mas
mas '1Password'

Phil Pirozhkov: I find it really hard maintaining the up to date list of formulae by hand and came up with the following:

Show installed formulae that are not dependencies of another installed formula:

brew leaves >! .formulae
brew cask list >! .casks

Install:

brew install $(< .formulae )
brew tap caskroom/cask
brew cask
brew cask install $(< .casks)

Reading Diary of 2016

2017-01-12T06:18:39Z

I enjoy going through other’s reading lists and find inspiration what to read next. Here is mine from 2016.

Team Geek

A book about collaborating in a team by two former Googlers is a great read for everyone — not just team leaders. It sums up basic principles of being a valuable team member and gives you some guidance in challenging social situation. In the end it’s all about communication.

It is not a revolutionary book and you might say it’s a common sense, but it’s a quick read and a well-written one.

Peopleware

On the similar notion as Team Geek this book is about software development teams and projects. It discusses working environment (strongly discourages open offices), distractions (do you have your slack notifications on?), hiring and turnover (turnover is expensive and always let the team have say in your hiring) and dives deeply into creating jelled teams.

The Passionate Programmer

A book about programmer’s career — with lot’s of different aspects discussed. How to be marketable. You should treat your career like a product. Learn the domain and understand the business and always know how you are making money for the employer.

Growing Object Oriented Software Guided by Tests

Steve Freeman and Nat Pryce take a small trading platform and goes through a journey of adding requirements in a TDD manner. One takeaway I liked from the book is to setup a fully functioning skeleton before diving too deep into functionality. Otherwise the style was not for me and it took me ages to finish it.

Chaos Monkey

A true story from Silicon Valley that gives you some insight into VC funding and all those accelerators. A startup company goes through Y Combinator and the team gets acquired by partially by Twitter and Facebook. The author stays in Facebook during it’s IPO phase and among other things shares what he earned. It’s a good view behind the curtains of startup industry and you can decide if it’s for you or not.

Kind of educational

Rework

Jason Fried and David Heinemeier Hansson don’t waste words and in this short book provide a guide on how to run a business. As a few other books it seems mostly as a common sense. It advices against rigid MBA practices and suggests to always have your customer at the first place.

Economic Facts and Fallacies

A second look at facts that seems to be obvious and well-accepted such as the rate of crimes in poor neighbourhoods, income discrimination of women, academia or third-world countries. The views are backed by an extensive research.

Cockpit Confidential

A brief look under the hood of airplanes, airlines and everything related to flying. Did you know that a rough landing is not a sign of a bad skill? Or what is the difference between captain and first officer? What are the career advances for pilots?

The Art of Happiness

A dialogue between Howard C. Cutler and Dalai Lama about western unhappiness. It is kind of meh. There is another volume called The Art of Happiness at work — I wonder what his holiness who never set a foot into the corporate environment has to advice.

How to Make People Like You in 90 Seconds or Less

One of those books that focuses on a few tricks to improve your perception by others to set up a good rapport. It is an ok read (and a short one) with a few things that is nice to keep in mind when meeting people.

Confession of a Wall Street Analyst

A personal journey of Daniel Reingold — one of the top telecommunication Wall Street analysts — through the worst crysis in the industry. He openly discusses insider trading, CEOs fudging their numbers and other illegal activities on Wall Street.

Guns, Germs and Steel

This is one of the best books I read last year. It answers the question why Euroasian civilisation dominated the world. It starts from its location and climate, continues through crops and animals that are available for domestication and compares it to other continents. From that it moves to the development of germs in more dense population and how it played its role in wiping out a few entire civilisations.

Barbarian Days

William Finnegan talks about seeking perfect waves from 1960s when surfing wasn’t such mainstream sport as it is now. He starts in Hawaii, goes through Indonesia, Tonga, Samoa and Australia. For a surfer it is a beautiful book.

The Ultimate Hiker’s Gear Guide

Andrew Skurka — a long-distance hiker — talks about hiking gear. A few useful advices that might save you some wasted money.

Escape from Camp 14

A chilling story of North Korean concentration camps that is told by a guy who was born there and escaped. In the end it turns out that there might be some troubles with his credibility but still — it is a reminder of dangers (or the fortunate lack of them) in different societies.

Drive

A book on motivation and its three stages: first — food, shelter, reproduction, second — rewards and punishments and third — autonomy, mastery and purpose. Therefore from a certain point money is not a good motivator anymore. Also, coined the terms extrinsic (you get rewarded for doing something) and intrinsic (you want to do something) motivation.

How to Win Friends and Influence People

Apparently this is a classic, but it falls down into the bucket of common sense advices for me.

The Story of Tea

This book has everything that you need to know about tea. It starts with the history of tea and goes nicely into the relationship of Western civilisation and tea culture. It also gives you the answer why English drink tea with milk.

It continues with the different teas description and goes into deep details of their manufacturing. After that it goes into the tea drinking culture and health benefits.

Even if you are not a tea enthusiast you will find a interesting pieces of knowledge in the cultural, geographical and historical chapters. And skip the tea specific ones.

The Tea Enthusiast’s Handbook

As opposed to the Story of Tea, this one is a bit more practical and is focused on buying, drinking and steeping the different types of teas.

The Art of Empathy

Actually I haven’t finished this book. I couldn’t get over the author’s style and the message from first 50 pages was not worth it to push through it.

Fiction

And I read some fiction too.

I would recommend Dune, American Gods and Lords of Flies — I loved these books. Fight Club seems slightly overrated and Chrysalids and Neuromancer are not my cup of tea.

Progressive Web Applications

2016-12-22T19:52:09Z

Every time there is a chance to build a mobile application with javascript, web developers are like “yeah, this is the way; the future of building mobile apps” and jump on the bandwagon of the saucy java intent avoidance.

[WHATEVER JS FRAMEWORK] will change the way we build mobile apps Every web developer ever

We’ve seen it with jQuery Mobile, Apache Cordova (Phonegap), Ionic, NativeScript and of course React Native.

How do progressive web apps fit into the picture?

What is it?

The fancy term was coined by Google where this technology originated. It sounds complex but progressive web apps are just web pages that use modern browser features – Web App Manifest, Service Workers and HTTPS. They are usually complemented with Cache Storage and IndexedDB.

They are linked with the browser and those technologies are currently experimental. At the moment you are mostly limited on Android platform as Safari doesn’t have a full support of service workers. You can check out the readiness of it at Is ServiceWorker Ready? site.

The web manifest is a simple file with a few information about your app – icon, name, description and others. When you add the site to your phone home screen this is the file where the icon comes from.

Service worker sits in between the browser and the server and is able to manage the network communication, work with the cache API and manage request when offline. It only works over HTTPS protocol.

Progressive web applications can mimic the material design of Android applications, but in the end you are in charge of your HTML and CSS. For that you could use Polymer or one of the native look emulating javascript libraries like Ionic.

Why would you want to use it?

Firstly, they won’t replace a mobile app. Or a website. They are a complement to them and are not faster to build or easier to maintain. They solve a different problem.

They help to overcome the barrier to install and launch apps. No need to go to the shop, search for the app, click install, agree on permissions and launch. You have them immediately in the browser and user can bookmark them to the home screen.

According to one of the Google presentations each step involved in installing an app has at least 20% drop out rate – meaning that with each click you lose 20% users. This is a huge! And for certain types of business like big e-shops, airlines or news it should be the reason to explore this area.

When using your cellphone your connection will be flaky and slow even if you don’t live in New Zealand. Removing that from the equation leads to better conversions and more sales on flaky networks. Users won’t leave your website because they’ve been staring at a blank screen for the past 3 minutes.

Apart from improving the speed of the app you also have the ability to work purely offline. That might be convenient for example for an airline app to show checked tickets.

Prosper Otemuyiwa did a nice write up of existing use cases and how they improved the business.

Where to learn more?

Offline Web Applications on Udacity by Google – a short but useful introduction into Service Workers, Cache Storage and IndexedDB by Michael Wales and Jake Archibald.
Rossta’s Progressive Web Application on Rails guide with a few additional resources to follow.
serviceworker-rails is a gem from Rossta that provides the basic pieces for your offline app and integrates it into Rails asset pipeline.
The Offline Cookbook by Jake Archibald shows different architecture patterns and approaches with their translation to Javascript.
Mozilla’s Service Worker Cookbook is a great collection of patterns and examples on how to solve common problems like deferring POST requests while offline.
IndexedDB Promised wraps IndexedDB interface into a promise based architecture – nice to have if you want to interact with it. It keeps the interfaces same with the original so you can still use documentation for original IndexedDB API.
Instant-loading Offline-first from Progressive Web App Summit 2016 where Jake Archibald’s demonstrates the capabilities of Progressive Web Apps, converts an existing online application into a fully functional offline web app and compares the results.
Building Progressive Web Apps Today from Chrome Dev Summit 2016 where Thao Tran shares some success stories from customers like Flipkart and Alibaba and gives some guidance on how to make the money rain.

Conclusion

Building a progressive web application requires the same amount of effort as creating a mobile app. You can’t easily convert your website into an app without major changes to the architecture and most of the early adopters solves it by creating and serving a mobile version different to the desktop one.

You shouldn’t think of it as a replacement for a mobile app, but as a complement to your website that will help with customer engagement, conversions on flaky networks and remove the barrier to install on the phone.

It is not a faster and cheaper way to develop mobile apps but it provides another way to support your business and generate more money.

Comments

Sintaxi: As one of the inventors of Cordova/PhoneGap allow me to set the record straight. We never intended for anyone to think PG was anything more than a temporary measure. Hence the mantra: “the ultimate purpose of PhoneGap is to cease to exist” –Brian LeRoux.

If anything this article should be acknowledging the role Cordova/PG served as a conceptual if not functional proof of concept for progressive web apps.

Jakub: You guys did a hell of a good job with Phonegap though. I think it is a wonderful piece of technology and it still has a great number of use cases.

People like to think about these as silver bullets to mobile development which I dont think exists.

What articles C# developers liked in 2016

2016-12-18T21:20:34Z

One of the benefits of running C# Digest – a weekly newsletter from .NET community – is to see which articles were the most popular in the C# space and what the programmers liked to read in the last year.

This year I shared over 486 articles through the weekly emails and I bet you are wondering which ones were the top 5.

Common Multithreading Mistakes in C#

Multithreading, concurrency and parallel programming are big topics in the .NET community and sometimes even seasoned developers don’t get them right. In this series of posts Ben Bowen explains common mistakes and how to avoid them. We are eager to see part 3 about unsafe assumptions.

.NET on Linux – bye, Windows 10

One of the big things of 2016 was .NET Core and Microsoft officially supporting OS X and Linux platforms. Piotr Gankiewicz shared his experience of setting up and running a competitive development environment on Linux Mint and lots of people liked it. I’m glad to see .NET on other platforms as the power of some UNIX tools is enormous and programmers can only benefit from this mutual symbiosis.

12 Visual Studio 2015 extensions you might find useful

Productivity tips and improving your development experience in Visual Studio is a popular topic. It seems that every .NET developer loves to be more efficient and deliver better quality code and use the best tools possible. In this article Hamid Mosalla shared his 12 favorite extensions for Visual Studio. Check them out!

What’s some C# bad practices that you know and what’s the workaround?

Number two is actually not an article but a reddit discussion about C# bad practices. Some might argue that it doesn’t belong to a newsletter, but reading these battlefield proven stories might actually improve the way how you think about your design as opposed to read another article about Javascript fatigue.

And it only proves that C# developers care about their code and want to avoid common pitfalls. Good on us!

Performance Considerations for EF 4, 5, and 6

Performance is a hot topic in .NET world. You can write very efficient and fast applications in .NET with the speed of development a C++ developers could only dream of.

The most popular article of this year is this deep dive into Entity Framework performance and database techniques by David Obando and Eric Dettinger. Great read guys, thank you — I bet you saved a few CPU cycles and disk spins out of our database and application servers.

Conclusion

These were the most read articles that I shared with you over 2016. I recommend reading through them (even twice if you already did) as you can learn a heaps.

Make sure that you don’t miss the golden ones in 2017 and subscribe to the weekly newsletter to keep your knowledge up to date.

The diary of a software developer :: Jakub Chodounský

Suggesting email address using string similarity algorithm

Levensthein Distance

Trigram

Conclusion

Resources

The Great Computer Experiment: Ruby Edition

Computers

Custom-built PC (2019)

Macbook Pro 13” (2013)

Budget Dell (2013)

Macbook Pro 13” (2018)

Benchmarks

Internal project

Ruby

Devise

ActiveAdmin

Mastodon

Conclusion

Progressive web application as a share option in Android

Becoming progressive

Adding web share target

Reading list 2017

Technology

Business and self-development

Non-fiction

Fiction

Recommendation

Tips for getting a programming job abroad

Research

Building a network

Applying

Tips

Garbage collection in C#

Basic concepts

Generational collection

Characteristics

Configuration

Performance

Managed and unmanaged resources

What about other languages?

Conclusion

Resources

Comments

Automate your macOS development machine setup

Dotfiles

Brew

Everything together

Conclusion

Comments

Reading Diary of 2016

Somehow computer related

Kind of educational

Fiction

Progressive Web Applications

What is it?

Why would you want to use it?

Where to learn more?

Conclusion

Comments

What articles C# developers liked in 2016

Common Multithreading Mistakes in C#

.NET on Linux – bye, Windows 10

12 Visual Studio 2015 extensions you might find useful

What’s some C# bad practices that you know and what’s the workaround?

Performance Considerations for EF 4, 5, and 6

Conclusion