Appearance

Terminal & Editor font: Hack

Here is a test pattern I am using to evaluate fonts:

o0O s5S z2Z !|l1Iij {([|})] .,;: ``''"" 
a@#* vVuUwW <>;^°=-~ öÖüÜäÄßµ \/\/ 
the quick brown fox jumps over the lazy dog
THE QUICK BROWN FOX JUMPS OVER THE LAZY DOG
0123456789 &-+@ for (int i=0; i<j; ++i) { }

This is how the pattern looks like with Hack, my favourite programming font:

All characters are very distinguishable, e.g. 0 and O, 1 and l. Subpixel hinting is also excellent.

colormake

Simple wrapper around make to colorize its output.

sudo apt install colormake

redshift

Install redshift-gtk, which is similar to f.lux but is more robust on my graphics card. I use this config for redshift (geoloc provider has problems).

In file ~/.config/redshift.conf:

; Global settings
[redshift]
location-provider=manual
 
; Linz, Austria
[manual]
lat=48.306940
lon=14.285830

Tools

Terminator

Multi-window terminal with lots of features.

sudo apt install terminator

ripgrep

Extremely fast grep tool, similar to ack but much faster, even faster than silver searcher. It searches through my 48GB subversion folder in 1 second. Also available for Windows! See here for installation instructions.

I’ve also added some aliases in ~/.bash_aliases:

# only sarch for cpp files, smart case handling.
alias rgc='rg -tcpp -S'

# only cpp, and only in test subfolders
alias rgct='rg -tcpp -g "*/test/cpp/**/*" -S'

# only cpp, everything but test subfolders
alias rgcnt='rg -tcpp -g \!"*/test/cpp/**/*" -S'

geany

Replacement for gedit, for basic editing tasks. It’s not excellent but usually does the job. I’m not a vim/emacs guy.

Visual Studio Code

Nice and highly configurable editor, also great in Linux. Just open a directory and you are ready to go. Get it here. Essential extensions are:

• Guides – An extension for more guide lines
• vscode-icons – Icons for Visual Studio Code
• Dracula Syntax Theme – A dark theme for many editors, shells, and more.

There are many others, but these are the bare minimum I use.

glances

Glances gives a fantastic overview of what’s going on in the system. Using you can see at a glance if e.g. your memory, disk space, swap is running out. The version you’ll get with apt is unfortunately quite outdate, so I am installing it with pip (see the homepage).

Useful Time Savers

Clear Terminal buffer

In ~/.bash_aliases, put this:

alias cls='printf "\033c"'

While Ctrl+L just clears the screen, cls will now clear the whole buffer. Convenient for a large build job and you want to scroll up to find something without scrolling too far.

Sleep Until

I use this ruby script to sleep until a given time. It’s in ruby so I can also easily use it in Windows:

#!/usr/bin/ruby

# sudo gem install chronic
require 'chronic'

exit(0) if ARGV.empty?


t = Chronic.parse(ARGV.join(" "))
now = Time.now
t += 60*60*24 if t < now
delaySeconds = t - now
puts "Sleeping #{delaySeconds.to_i} seconds until #{t.rfc2822}"
sleep(delaySeconds)

I use this to schedule builds, so that when I arrive at the office everything is already built with the latest version so I can start working right away.

$ sleepuntil monday && svn up && make -j4 build
Sleeping 71605 seconds until Tue, 6 Dec 2016 03:00:00 +0100

bash prompt with runtime and errorcode

See my Linux Bash Prompt post. It’s awesome.

Optimizations

ccache

Use ccache, and use it on an SSD disk. Here are my settings from ~/.ccache/ccache.conf:

max_size = 16G
compression = true
compression_level = 1

I’m using this for a huge C++ project where I build debug, release both in 32bit and 64bit, so a large ccache helps. Using compression_level is only a very minor slowdown, but practically increases cache size by a factor of 3-5 or so.

Use tmpfs for /tmp

Much faster compilation. Add this to /etc/fstab:

tmpfs	/tmp	tmpfs	mode=1777,nosuid,nodev	0	0

Compressed RAM

Especially in virtual machines or low-end machines where memory is constrained, zram provides memory compression.

sudo apt install zram-config

Reboot your machine, and everything is well configured. For more information, see

• zram: Compressed RAM based block devices
• Transparent Memory Compression in Linux (2013)

Disk Cleanup

Remove libreoffice

sudo apt remove --purge libreoffice*
sudo apt clean
sudo apt autoremove

Remove old Kernels

sudo apt install byobu
sudo purge-old-kernels

Source: How to Easily Remove Old Kernels in Ubuntu 16.04

Virtualbox

Dynamic Disk Size

Don’t do it. It’s much slower. If you have to, be aware of this:

I’m using Linux in a Windows Host, my .vdi image is on an SSD. When using dynamic disk size, this unfortunately grows the space continuously when compiling a lot, because deleted files won’t shrink the .vdi image. But it is possible to do this with e.g. zerofree (a hazzle), or better with TRIM:

• In the Windows host, enable SSD and TRIM support for the image (see here):
  

  VBoxManage storageattach ubuntu --storagectl "SATA" --port 0 --discard on --nonrotational on

• In Linux guest, perform TRIM, you’ll see the .vdi disk shrinking while this runs:
  

  sudo fstrim -va

• Add a nightly task to crontab with sudo crontab -e, there add this line:
  

  @midnight fstrim -a

I also tried to mount the image with the discard option (see here), but it caused my Linux to hang so I disabled it again

Mount shared folder

Add the user to the vboxsf group:

sudo usermod -G vboxsf -a martinus

Edit /etc/fstab e.g. like this:

Share_VB        /media/sf_Share_VB      vboxsf  defaults,noauto 0       0

GNOME keyring Without Password Prompt

It’s unfortunate when a scheduled svn up asks for password because it has not been entered for a while. To disable the password question, do this:

1. sudo apt install seahorse
2. Follow this guide.

Optimize Disk Scheduler: use noop

When using virtualbox, it’s best to use the noop scheduler. See this. From here:

1. cat /sys/block/sda/queue/scheduler
2. sudo gedit /etc/default/grub
3. Change the line

   GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

   into

   GRUB_CMDLINE_LINUX_DEFAULT="elevator=noop quiet splash"

4. sudo update-grub

#include <iostream>
#include <stdint.h>

int main(int, char**) {
	for (uint32_t i = 1; i < 3; ++i) {
		uint32_t a = i * 0xfbd1e995;
		uint64_t b = a;

		std::cout << a << " 32bit" << std::endl;
		std::cout << b << " 64bit" << std::endl;
	}
}

Compile and run in 64bit, Release. This is the expected output (as with g++, clang++, or in debug):

4224838037 32bit
4224838037 64bit
4154708778 32bit
4154708778 64bit

This is what I get with 64bit, Release:

4224838037 32bit
4224838037 64bit
4154708778 32bit
8449676074 64bit

Also see my stackoverflow post. The compiler seems to optimize too much, and directly uses a 64bit number instead of making sure the 32bit value is casted to 64bit. I have already reported the bug to Microsoft, and wait for feedback.

• Red ✘ if the previous command has failed, otherwise a green ✔.
• Shows running time of the previous command
• Separate line for path and command

Example

Installation

Add this to your .bashrc:

function timer_start {
  timer=${timer:-`date +%s.%3N`}
}
 
function timer_stop {
  local ELAPSED=$(bc <<< "`date +%s.%3N` - $timer")

  local T=${ELAPSED%.*} 
  local AFTER_COMMA=${ELAPSED##*.}
  local D=$((T/60/60/24))
  local H=$((T/60/60%24))
  local M=$((T/60%60))
  local S=$((T%60))

  timer_show=
  [[ $D > 0 ]] && timer_show=${timer_show}$(printf '%dd ' $D)
  [[ $H > 0 ]] && timer_show=${timer_show}$(printf '%dh ' $H)
  [[ $M > 0 ]] && timer_show=${timer_show}$(printf '%dm ' $M)
  timer_show=${timer_show}$(printf "%d.${AFTER_COMMA}s" $S)
  
  unset timer
}
 
trap 'timer_start' DEBUG
PROMPT_COMMAND=timer_stop
 
PS1="\e[0m\n\[\`if [[ \$? = "0" ]]; then echo '\e[1;32m✔'; else echo '\e[1;31m✘' ; fi\` \e[1;32m\h\e[0m \e[1;94m\w\e[0m \e[93m\${timer_show}\e[0m\n\$ "

Updates

• 2017-04-28: Now prints days, hours, minutes, seconds. Much better readable for long running tasks.
• 2016-11-04: Initial version

Sources

• How can the last command’s wall time be put in the Bash prompt?
• How to pass results of bc to a variable
• What is your favorite Bash prompt? [closed]
• Customize your Bash prompt
• Convert seconds to hours, minutes, seconds

As promised in part 2 of this series, I have some interesting benchmarks of the different hash map variants. In this part I will discuss the results, and present my conclusions. Click any benchmark to get a much larger graph.

How I Benchmark

I’m inserting 100 million int -> int pairs into different hashmap variants, and every 100000 inserts I am taking measurements (insertion time, memory usage, time to lookup 1 million elements, where half of the lookups fail.). As a hash function I am using std::hash. My computer is an Intel i5-4670 @ 3.4 GHz, 16GB RAM, Visual Studio 2015 Update 3, 64bit.

Insertion Time Benchmark

In this graph the reallocations are very clearly visible. All variants are doubling the size of the internally used array when it gets full. As expected in part 2, it can be seen that insertion time slows down when “Robin Hood with Infobyte” gets full, and also when “Robin Hood with Infobyte & Fastforward” gets full. This is due to the fact that more and more data needs to be moved around so the hashmap can stay sorted.

Dealing with the fastforward field has a clear overhead associated with it, insertion is quite a bit slower.

For “Robin Hood with Infobits and Hashbit”, insertion time is very fast: mostly because the offset is limited to 16, if it would get larger the hashmap’s size is doubled. Hopscotch insertion is almost as fast, just a tad slower.

All variants are much faster than std::unordered_map. The comparison is a bit unfair, because in my implementations I do not have to support the same interface as std::unordered_map does.

Memory Usage

Robin Hood with Infobyte is the most compact representation. I am resizing the map when it gets 95% full, or when an overflow occurs which is highly unlikely when using an appropriate hash function. There is just 1 byte overhead for each entry.

The second best representation is “Robin Hood with Infobyte & Fastforward”. This has exactly the same reallocation properties as when just using the infobyte, but it uses one additional byte per bucket.

When limiting the maximum offset as in “Robin Hood with Infobits & Hashbits”, resizes are occuring much quicker (maximum offset is 15 instead of 127), so the memory usage jumps up earlier.

For Hopscotch this reallocation seems to happen still a bit earlier. The overhead in my implementation is 16 bit, so the line overlaps a lot with the Infobyte & Fastforward variant.

All my implemented variants need much less memory than std::unordered_map. While std’s implementation allocates new memory for each insert, in all my variants the memory stays constant until reallocation occurs.

Lookup Time

This is probably the most interesting benchmark for most use cases, and I find this graph extremely educational and interesting.

First, the Robin Hood with Infobyte variant: Lookup time starts pretty fast, but it gets slower and slower as the hashmap gets full. This is due to the longer and longer chains that needs to be traversed to find the correct element. So the highly compact format is bought with ever increasing lookup (and insertion) times. Still, lookup is almost always faster than lookup for std::unordered_map.

The Robin Hood with Infobits & Hashbits variant shows the same behaviour, lookup time goes up when the hashmap gets full. This increase is capped early though due to the earlier reallocations. So faster lookup times can be bought with earlier reallocations and more memory usage.

To improve lookup times compared to the infobyte variant, I have introduced the fastforward byte to quickly skip directly to the elements that belong to a bucket. That’s the gray line. When the hashmap is quite empty it seems that extracting this fastforward information has quite a bit of an overhead. When implementing this variant I did not expect such a large overhead. It seems to work very well though in improving lookup time when the map gets very full, as can be seen around the 60 million elements line. Here, lookup for the variant with just the infobyte takes 0.0676 seconds, and when using fastforward byte it takes 0.0487 seconds, 38% faster. Also the lookup time is quite linear.

Last but not least, the Hopscotch variant. After spending quite some time tuning, this variant turned out as the fastest of all. Lookup time does not have the spiky behaviour that robin hood has, and it is extremly fast. One trick I am using is that I using that I didn’t see anywhere else mentioned, is that I shift the whole hop field and can stop the while loop as soon as the hop field is zero. So when only one element is there and it is in the first bucket, I need to check just one bit of the hopfield. When the element is not there, hopfield is zero so no bits have to be checked at all.

Summary

I’ve summed up the average insertion time, average lookup time, and average memory usage over all the different samples. std::unordered_map is the reference with 100%. Here are my conclusions:

• The HopScotch implementation is the best overall for most use cases, if you have a reasonable hash function. Insertion time is just a tad slower than the fastest variant, memory usage is a bit higher than the other variants but still almost 3 times better than std::unordered_map, and lookup is blazingly fast with almost twice the speed as unordered_map. 0.0404 seconds to look up 1 million elements, or just 40.4 nanoseconds per lookup on average is not too shabby.
• If your hash function is bad or compactness is of utmost importance, use the Robin Hood with Infobyte variant.
• If the std interface is important, stick to std::unordered_map. While it uses lots of memory, lookup times seem to be ok and fairly consistent.

That’s my tour of hashtable implementations. You can find my code on Github at martinus/robin-hood-hashing. It still needs lots of cleanup, and most importantly, the ability to remove elements…

In part 1 I have discussed Hopscotch and Robin Hood Hashing table. Since then I have implemented several hashmap variants that combine some tricks from these two variants, and have some interesting results to share. Here are the variations that I have implemented:

Robin Hood with Infobyte

Robin Hood Hashing uses the hash value to calculate the position to place it, than does linear probing until it finds an empty spot to place it. While doing so it swaps out entries that have a lesser distance to its original bucket. This minimizes the maximum time a lookup takes. For a lookup, it is only necessary to lineary probe until the distance to the original bucket is larger than the current element’s distance.

In this “Infobyte” implementation variant, instead of storing the full 64 bit hash value I directly store the distance to the original bucket in a byte. One bit of this byte is used to mark the bucket as taken or empty. The bit layout is defined like this:

When storing four elements a, b, c, d, where a, b map to hash position 1 and c, d to hash position 2, this is how they would be stored:

The advantage is that I have just 1 byte instead of 8 overhead, and linear probing is faster because I need only comparison operation on a single byte instead of calculating offsets with the hash. A byte is more than enough for the offset, since on average the distance to the the original bucket is very small. Here is the full code for lookup:

const Val* find(const Key& key) const {
    size_t idx = _hash(key) & _mask;
 
    // IS_BUCKET_TAKEN_MASK is 128 (1000 0000 in binary)
    std::uint8_t info = Traits::IS_BUCKET_TAKEN_MASK;
    while (info < _info[idx]) {
        ++idx;
        ++info;
    }

    // check while info matches with the source idx
    while (info == _info[idx]) {
        if (key == _keys[idx]) {
            return _vals + idx;
        }
        ++idx;
        ++info;
    }

    // nothing found!
    return nullptr;
}

This search is very simple and straightforward, and fast.

Robin Hood with Infobits & Hashbits

Using a full byte for the offset (actually 7 bit) is fairly large. This would allow an offset of 127, which would lead to a very long and slow linear probing. My idea then is to limit the offset to a much lower maximum number. The remaining bits of the byte can be used to store a few bits from the hash, so that we can use these few bits to reduce the number of key comparisons. Also these hashbits could be used when resizing the hashmap so we would not have to rehash all elements each time we resize. After some tuning, this is the bit layout I am using for this variant:

The much smaller offset means that when inserting an element and the maximum offset is reached, I have to resize the hashmap. This is intentional, since very large offset would lead to slow searches. This way this hashing variant is more similar to Hopscotch. Here is the full code to lookup an element with this byte:

const Val* find(const Key& key) const {
    auto h = _hash(key);

    // create info field: offset is bucket is taken, offset is 0, with hash info.
    std::uint8_t info = Traits::IS_BUCKET_TAKEN_MASK | ((h >> _level_shift) & Traits::HASH_MASK);

    // calculate array position
    size_t idx = h & _mask;

    // find info field
    while (info < _info[idx]) {
        ++idx;
        info += Traits::OFFSET_INC;
    }

    // check while it seems we have the correct element
    while (info == _info[idx]) {
        if (key == _keys[idx]) {
            return _vals + idx;
        }
        ++idx;
        info += Traits::OFFSET_INC;
    }

    return nullptr;
}

The find code is almost exactly like in the pure Infobyte variant, except that I am or’ing 3 hash bits into the info byte, and instead of

++info

info += Traits::OFFSET_INC

Robin Hood with Infobyte & Fastforward

In the first variant of Robin Hood Hashing finding an element will be slow as the hashmap gets full, because a lot of elements need to be skipped before finding the full one. Robin hood hashing has the property that it always keeps elements that belong to the same bucket together. The infobyte introduced in the first variant basically acts as a backpointer to the bucket that element originally belongs to, and it is possible to introduce another byte that is stored at the original bucket, that acts as a forward pointer to where the first element is stored for this original bucket.

Using the above example where a,b maps to 1 and c,d to 2, fastforward byte looks like this:

When looking up element c, the fastforward byte can be used to skip one element, so the

while (info < _info[idx]) { ... }

const Val* find(const Key& key) const {
    size_t idx = _hash(key) & _mask;

    const auto ff = _info[idx].fastforward;
    std::uint8_t info = Traits::IS_BUCKET_TAKEN_MASK | ff;
    idx += ff;

    // check while info matches with the source idx
    while (info == _info[idx].info) {
        if (key == _keys[idx]) {
            return _vals + idx;
        }
        ++idx;
        ++info;
    }

    // nothing found!
    return nullptr;
}

I am using a struct that contains two bytes: info field, and fastforward. Now I have two bytes overhead for each entry, but can quickly skip lots of buckets which should be especially useful when the hashmap is very full.

Hopscotch

Hopscotch can be seen as quite similar to the robin hood hashing with fastforward, except that it replaces the info and fastforward byte with a bitmask. With some tuning I am now using a 16 bit field, where 1 bit is used to define if the current bucket is full, and the other bits define which offsets to the current bucket are taken.

If the first bit is set, the current bucket is taken; but not nessarily by the bucket from this hop bitfield. This bit is not strictly necessary because that information is also encoded in the different hop tables, but it speeds up insertion code. With 16 bits available, this limits the hop bits to 15 though. As with “Robin Hood Hashing with Infobits & Hashbits”, when the hop bits are full, the hashmap has to be resized.

Benchmark Results

I’m leaving the most important information for part 3: how do all these variant perform? What’s best?

A while ago I’ve spent significant time researching and implementing a fast Hopscotch hash table for C++. My current source code can be found in my github repository at martinus/robin-hood-hashing. After spending some time optimizing, I am mostly happy with the results. Insertion time is much faster than std::unordered_map and it uses far less memory.

Benchmarks

In this simple benchmark I have measured time while sequentially adding an entry (int -> int) to the hashmap, similar to incise.org’s benchmark. As a hash function, I am using std::hash:

Whenever a jump occurs, the hashmap got too full and it is reallocating. Interestingly, std::unordered_map has to allocate memory whenever it inserts an element (since it’s a linked list), while the Hopscotch hash table only allocates once and uses that memory.

I am using Visual Studio 2015 Update 3, 64 bit, Intel i5-4670 @ 3.4GHZ. Here is a comparison table (best values bold)

Insertion is really fast and much more efficient, query time is also a bit faster than std::unordered_map, even though we need to check the hop bitmap of 32 elements.

Robin Hood Hashing vs. Hopscotch

After contemplating a while, I have come to the conclusion that Hopscotch is just a bad version of Robin Hood Hashing. Here is my reasoning:

1. Hopscotch’s bitmap naturally has to be quite sparse. For a perfectly full hashmap, where each bucket contains a corresponding entry, of the 32 hop bits there will be just a single bit that is set to 1. Wikipedia has a nice representation:
2. Is there a better way to represent the hop bitmap? On way would be a linked list of offsets. Unfortunately, linked lists are not very cache friendly.
3. To store the linked list more efficiently, we can put them into an array. How about just storing the offsets next to the buckets? So the above example from a) could be stored like this:
   Will this work? Surprisingly, yes! When querying for an element, we just need to sequentially check the offsets. Say we want to check if ‘b’ is in the map:
   1. We hash ‘b’ and get index 6.
   2. The offset at index 6 is 0: That means at index 6 is an element that actually belongs there: It’s not b, so we have to continue checking.
   3. At index 7 we would expect an offset 1 if it contains an element that was indexed to 6. Since it is taken by something else, it will have a higher value, so we are certain this is not the bucket we are looking for.
   4. At index 8 we expect an offset 2 if it contains an element that was indexed to 6. It is to, so check the value, and bam! We found the element.

   If we query an element that is not in the map, we just need to check hop size offsets.

4. All this sounds increasingly similar to Robin Hood Hashing. Once we have the offset, we can use it as clever as robin hood hashing does: It enables us to use the cool “take from the rich” algorithm. So now we can ditch the hop size, and just keep swapping elements exactly like robin hood hashing does.
   

   What now?

   With these insights, I believe I have a great idea to implement a highly efficient variant of the robin hood hash table, that takes some ideas from the hopscotch implementation. In my next post, part 2, I will explain these ideas and (hopefully) have a fantastically fast and memory efficient hash table in my repository.

   ]]> http://martin.ankerl.com/2016/09/15/very-fast-hashmap-in-c-part-1/feed/ 0 http://martin.ankerl.com/2016/06/06/analyzing-keto-calculator-demographics/ http://martin.ankerl.com/2016/06/06/analyzing-keto-calculator-demographics/#respond Mon, 06 Jun 2016 14:36:48 +0000 http://martin.ankerl.com/?p=1533 Continue reading "Analyzing Keto Calculator Demographics"
   ]]> Recently my Keto Calculator has grown huge. So big, that in January 2016 alone I’ve had 222,781 page views. That is a record for my site, and I have no reason to believe that it will slow down anytime soon. In my quest to make all the visitors as happy as possible, I’ve done quite a bit of analysis of my visitors to find out what I need to do with my homepage. Here is some of the data that I have gathered using Google Analytics.

   Each graph is color coded: blue stands for mobile devices, orange for tablets, and green for desktops.

   Where Do the Users Come From?

   I have created a bar chart showing the total number of sessions for January 2016, sorted by top countries.

   

   By far the most visitors come from the US, followed by Canada, UK, and Australia – all english speaking countries. The first non-english speaking country is Germany, and only with 0.69% of all sessions.

   For fast access for the majority of my users, my hosting server should sit in the US. I have recently switched to T1Hosting which is both cheaper and faster than my previous webhoster. Also, they were really fast to respond to my questions. So far I am pretty happy with them!

Translation the calculator is currently not really useful. I am sure that translating it into German would increase the number of users from Germany, but it does not look like the effort would really be worth it. What I have done is add an automatic translation feature to the top of the calculator to provide some usability for non-English speakers. Unfortunately the quality is quite bad.

Almost half of my users come from mobile devices! To be precies, 48.8% mobile, 41.7% desktop, 9.5% tablets. That means it is absolutely essential that the keto calculator works well on mobile and on the desktop. It took me quite a while to make it look good on mobile, and now I’m happy with the result. I’ve done most of the testing with Chrome DevTools, which are absolutely fantastic!

How Old Are My Visitors?



Most visitors are 25-34 years old. I guess that’s a typical result for a website with nutrition as a topic. Interestingly, the graph does not show any data for younger than 18 year olds. I have updated the calculator with some safety checks so that even when somebody very young tries to use the calculator it will give reasonable advice. If you are under 18, it will show a warning. Under 15 the calculator even refuses to give you any results.

What Gender Are My Visitors?



There are a lot more female visitors! It seems that females are much more sensitive about nutrition topics. Also, female tend to use more mobile devices.

It would probably help to have some gender specific advise on the keto calculator, especially for women. E.g. breastfeeding comes up from time to time. Unfortunately I know nothing about these topics, so it’s best to go to the dedicated subreddit /r/xxketo with these kind of questions.

How Big Are the Screens?



Lots of visitors use smartphones, and the screen resolutions reflect this. The sizes range from quite narrow with just 360 pixels to quite large with 1440 pixels. I’ve tuned the layout to be nicely visible on all these ranges. Thanks to @media it gracefully switches between these layouts and looks quite readable on very wide screens and very small screens.

]]> http://martin.ankerl.com/2016/06/06/analyzing-keto-calculator-demographics/feed/ 0 http://martin.ankerl.com/2016/01/22/google-analytics-experiments-is-buggy/ http://martin.ankerl.com/2016/01/22/google-analytics-experiments-is-buggy/#respond Fri, 22 Jan 2016 13:34:41 +0000 http://martin.ankerl.com/?p=1516 Continue reading "Google Analytics Experiments is Buggy"
]]> Google Analytics has a very interesting nice feature: it is possible to use it for A/B testing, and in a nice client-side way that just uses javascript. I am using this to improve my keto calculator website. An interesting feature is that google uses a multi-armed bandit implementation, which optimizes the A/B split ratio while optimizing.

Unfortunately, Google’s implementation is quite buggy. Here are a few problems that I ran into while testing:

Inaccurate Number

While testing, Google Analytics shows a “Probability of Outperforming Original”. This number can is often incorrect, and does not match with what is actually used for the homepage A/B split ratio. It seems that the numbers in the Analytics UI is calculated based on data that is only partly up-to date. In reality the probability should is at least 99%, but the UI shows 84.1%. The number is updated about twice a day, and it jumps around wildly even though there is not much change in the relative ratio between A and B test.


Incorrect Graphs

The above graph is sometimes not up to date. It seems that the graph data can be delayed for 1-2 days. Sometimes it shows the current days as no data, even though it’s already recorded in the “Adsense” view.

By the way, when optimizing for Adsense Revenue, it seems that the “AdSense” view is much more accurate than the “Conversions” view. What really counts is the “AdSense eCPM” value (which by the way should be renamed to RPM).

Incorrect/Inaccurate Results with Multiple Variants

I’ve tried to use the multi-armed bandit to find the optimum variant out of 20 different ones. It seems that this has failed completely. While having a look at the experiments.js file that google integrates, it can be seen that some variants had a probability to be chosen of 0, so they were not at all considered any more. With multi-armed bandit Google usually has a selection probability for each variant that is exactly the probability of it outperforming the original. The javascript file for the above screenshot contains this data:

"api_version": "V1",
"method": "experiments.get",
"data": {
	"kind": "cx/api#experiment",
	"id": "xxxxxxxxxxxxxxxxxxxxxx",
	"self_link": "/cx/api/V1/experiments/xxxxxxxxxxxxxxxxxxxxxx",
	"updated": "2016-01-22T00:36:33.752Z",
	"participation": 1,
	"items": [{
		"kind": "cx/api#combination",
		"id": "0",
		"weight": 0.019,
		"disabled": false
	}, {
		"kind": "cx/api#combination",
		"id": "1",
		"weight": 0.98099999999999998,
		"disabled": false
	}]
}

So even though the screenshot says a 84.1% probability of outperforming, the file has a selection probability of the variant of 98.1% which should be more correct. In my test with 20 variants, about 10 variants had a weight of 0, which means that they were never shown to the user. When I noticed this, I added my own selection code (in 50% of the cases, just randomly present a variation to the user) so that the 0 probability variants have a chance to be actually visible to some users, and while this had some effect on the results, it still seemed that the multi-armed bandit just can not deal with so many variants at the same time. Some variants which are actually very similar got a completely different probability of outperforming the original.

Verdict

All in all Google Analytics is really excellent for A/B testing, but has a few quirks that one should be aware of. When A/B testing, a Chi-Square test for comparison is always helpful. E.g. a Chi-Squared test of click through ratio of the above example (ok, it’s not the same as testing AdSense Revenue like google does, but the difference should be marginal in my case) shows that with a confidence level of 99% we can say that the variant is more successful.

]]> http://martin.ankerl.com/2016/01/22/google-analytics-experiments-is-buggy/feed/ 0 http://martin.ankerl.com/2016/01/04/body-fat-comparison-pictures/ http://martin.ankerl.com/2016/01/04/body-fat-comparison-pictures/#respond Mon, 04 Jan 2016 19:22:52 +0000 http://martin.ankerl.com/?p=1406 Continue reading "Body Fat Comparison Pictures"
]]> Here is a collection of visual body fat charts. Use these to estimate your personal body fat for the keto calculator. See below for men.

Women







As you can see, even the same body fat percentag (here twice 15%) can look quite different, depending on your muscles and where the body fat is positioned. The minimum possible body fat percentage is 8-9%. This low is not sustainable, and very unhealthy! If you are female, don’t go that low.

Men









As above with the female pictures, here are also two images with the same body fat percentage: 10% can look quite different, depending on how muscular you are and where your body fat is positioned. The lowest male can go os 3-4%, but this is not at all healthy at all. It is only possible to reach such low body fat for a very short period of time. I’d say the lowest you should go to still be healthy is 10%.

Back to the Keto Calculator

Now when you have made up your mind where about your body fat percentage might be, head back to the keto calculator and enter your number.

Sources

• Leigh Peele – Body Fat Pictures and Percentages
• ai beauty
]]> http://martin.ankerl.com/2016/01/04/body-fat-comparison-pictures/feed/ 0 http://martin.ankerl.com/2015/12/22/beautiful-git-logs/ http://martin.ankerl.com/2015/12/22/beautiful-git-logs/#comments Tue, 22 Dec 2015 07:35:55 +0000 http://martin.ankerl.com/?p=1346 git has a very configurable logging options. I’ve played a while with the configuration, and found an awesome alias that looks just beautiful. It only works as of git 1.8.3 (March 24, 2013) because it uses auto coloring.

git config --global alias.l 'log --graph --pretty=format:"%C(auto)%h%<(3)%d %s %C(bold blue)(%cr, %an)%Creset" --abbrev-commit --all'
git config --global alias.ls 'log --graph --pretty=format:"%C(auto)%h%<(3)%d %s %C(bold blue)(%cr, %an)%Creset%n" --abbrev-commit --all --find-copies -M --stat'

git l



git ls



]]> http://martin.ankerl.com/2015/12/22/beautiful-git-logs/feed/ 2