One of the most memorable part for me was the sound track, so I really wanted to have that in the game.

Graphics

This year I decided to go with vector graphics. I had a simple path-to-custom-array converter tool I made for my rather failed Trado Polo Explores game for js13k Games 2023, so I built upon it.

The current tool is taking a bunch of SVG files, extracts the path elements from it and processes their d (the path definition) and style (fill color, fill style, stroke color, etc. parameters) attributes, then it scales them to a 100×100 grid, rounds the coordinates and then save all the points in an array, along with the . It creates an array from the styles (keeping only the unique ones) and assigns these styles to the path.

The 100×100 grid and coordinate rounding is introduced to use less data, it generally leads to no visible artifacts, except for large shapes.

Let’s say we have this triangle on a 300×300 canvas (the arrows and coordinates are there for the explanation only).

This results in the following SVG file:

The converter will pick this up and will give us a result like this:

This definition is then later used to reconstruct an SVG that is very close to the original one (shown using dotted line and gray coordinates), like this:

The difference between the SVG and the converted image definition might not look impressive at first, but when having a real-world file with multiple shapes and the coordinates with fractional numbers, also when combining multiple files, it can add up quickly. Check this example of four files:

When compressed using ZIP and then advzip the four SVG files result in a 4,031 bytes ZIP, the JS results in a 587 bytes ZIP.

When compressing all the SVGs used in the game it results in a 81,712 bytes ZIP, in JS it is 4,459 bytes.

An interesting side note: ZIP needs to store metadata of every file and that takes up some space too. Also, ZIP is compressing every file separately. So – although the result is nonsense – if I just copied all SVG files into one file, the file size was the same but ZIP compressed it to 18,248 bytes instead – that is less than 1/4 of the previous 81,712 bytes!

Music

The music starts as a MIDI file, but that would have been too much for this project – both in features and in size. So I went with converting it to the bare minimum – a custom format denoting only the tempo, timestamps of notes, and the pitch of the notes.

To play it back I am generating sound samples with ZzFX to use them as instruments, and a simple loop to play them at the right time.

This way the original MIDI starting at 3,410 bytes results in music definition of 2,492 bytes that can be compressed down to 594 bytes. (The MIDI could be compressed to around 1,100 bytes.)

Compression

I write the code in TypeScript to have the advantage of strong(er) typing and get rid of many of JavaScript’s annoyances unique features, and the help of my IDE (VSCode).

For the build I just merge all TS source files into one file and use TSCC to compile the TypeScript to JavaScript, inserting JSDoc annotations and other useful things for further processing, then use Google Closure Compiler to compress it.

The HTML is usually really simple, so I just create a cleaned up version manually.

For CSS I have a simple oneliner to get rid of the white spaces and the unnecessary newlines.

After this I put all the files into one single index.html, because it results in smaller ZIP, as each additional file needs additional metadata.

For the final compression I use AdvanceCOMP‘s advzip, which is a program that re-compresses ZIP files with different parameters, trying to squeeze the ZIP file even smaller.

With fairly conservative settings (3000 iterations) it can save 5-8% compared to the “high” compression setting of standard ZIP. This can be a huge deal with size limited projects, with the js13k limit of 13,312 bytes it can squeeze an additional 1,150 bytes in. Higher settings can lead to even more savings, although typically only a few more bytes – but there were years when I needed those extra 1-2 bytes to fit into the limit.

This year my standard ZIP file was 13,940 bytes (628 bytes over the limit), advzip re-compressed it to 12,919 bytes (394 bytes under the limit).

This size consisted of the following parts: JavaScript (7,204 bytes), graphics data (4,459 bytes), music data (594 bytes), HTML (346 bytes), CSS (315 bytes).

Note: these sizes are approximations as there is no way to really determine the exact size of a part of the file taking up, so I compressed these parts separately and then scaled their sizes to the final ZIP size. (They were compressed to a bigger total separately. Also, when I tried by replacing/removing parts, that lead to smaller html/css size approximations.)

I have compiled a chart that shows all steps of the build process, starting from source files, then assembling, compiling, minimizing the parts and finally compressing them.

After the competition ended I started to experiment with Roadroller (see the dotted line) and I might go with it next year.

Results

This post turned out to be rather technical and compression focused – but I won’t lie, I wanted to do such a post for some time now.

I had a few features and details that either made it into the game but is not shown (eg. more color palettes ), or I wasn’t able to finish (eg. some extra elements on the opening screen, more levels, final level), but I think for a jam game it turned out to be fun, and for me that’s really important and I am happy with how it turned out.

Alley Kitty landed on the 20th place, out of 176 games – with some pretty strong entries this year too!

See you again in 2026! Maybe that time I’ll have a simple idea, for a change… :)

LUTs are Look Up Tables for colors, used mainly for color grading, they basically map colors to different colors and can be represented as a regular PNG. Ffmpeg can apply LUTs to videos, and it can also generate you one to edit to your liking and then apply it.

You might have a clip that you shot but for some reason the colors were a bit off, or you just want to do some fine-tuned effects. With the following command you can generate a snapshot of the video at a specific time (0:00:04 in this case) along with a LUT.

ffmpeg -f lavfi -i haldclutsrc=8 \
  -i input.mp4 -ss 0:00:04 -frames:v 1 \
  -filter_complex "[1]scale=-1:512[b];[0][b]hstack" \
  haldclut_correction.png

Then you edit it in your photo editor, but make sure to only apply effects that are affecting the pixel color values (color temperature, curves, contrast, sepia, etc.) but does not alter the image (i.e. blurring).

Then after saving the image, with the following command you can apply it to the whole video:

ffmpeg -i input.mp4 \
  -i haldclut_correction.png \
  -filter_complex haldclut \
  -pix_fmt yuv420p \
  -c:v libx264 -preset slow -crf 18 \
  -c:a copy \
  output.mp4

And that’s it, no guesswork included. And as this is still ffmpeg you can automate it, chain more edits, etc. and have full control over the quality of the output. (This example results in a fairly high quality MP4, the videos in this post are more compressed (have worse quality) to conserve bandwidth.)

If by any chance you come across a video that looks underexposed to you, you can also fix that with a simple adjustment.

And of course, you can go further if you want.

You can really apply anything you’d like, and I just found out that you can also have a video as input for the LUT – I won’t start playing with that for now. But please share it with me if you do :)

Notes

Of course you can also apply any effects to the PNG that “corrupts” the LUT – it might result in a glitchy, fuzzy, artistic video – and that might even be what you’re after! You really can’t do any harm by trying, ffmpeg will happily render you something.

The post is based on this answer for the question How to apply a LUT filter with FFmpeg on StackOverflow.

The -pix_fmt yuv420p might not be necessary when encoding the output but ffmpeg with libx264 might default to yuv444p which might cause problems when playing the video in browsers and older media players. Also, when exporting to web with libx264 encoding, adding -movflags faststart should reduce the time needed to start the playback (more here), but this is unrelated to the post.

The screenshot on the right of the generated image is simply ignored by ffmpeg (“Padding on the right (910px) of the Hald CLUT will be ignored”), it is just there so you can see the results of your edits.

While ffmpeg can work with several types of LUTs, the Hald CLUT that is used in this post is probably the most widely used one. There are tools like cluttool to convert from Hald CLUT to 3D LUT (3DL) and LUT-Convert to convert from Hald CLUT to CUBE. Ffmpeg also supports 3D LUT and probably CUBE, too.

The second example is a few seconds of House Of The Dragon (season 1, episode 7) that was criticized for being too dark, I used it only for illustration and educational purposes, please don’t sue me, HBO. I’ll remove it if you ask.

When I first was on the U3 subway in Vienna last year I noticed how many segments the characters have on these displays. It looked absurdly many, but I really liked the shape of the characters, so the next time when I was around I took a few pictures and started to draw it in Inkscape. I counted 84 segments in the end.

Needless to say, at this point I was already in love with these shapes. I was also joking about making a font out of it, but… well…

Creating the font

After looking around for a bit I found Birdfont, a free open source font editor. Checking out the Intro to Birdfont tutorial by Spencer Saunders was enough to get me started.

Birdfont has a few sample fonts but you can really open any TTF file with it. As I had no idea what would be the correct shape and dimensions in the SVGs, I have opened a TTF font and exported a wide and full height letter with accents (“Ë” in my case) and based my characters on its shape (although it turned out to be a bit too low in the previews).

Birdfont has a really great approach when importing the characters from SVGs. Basically you can import a directory containing one SVG per character with an easy to follow naming scheme:

• the base of the file name is the character to import (C.svg will imported as “C”)
• everything after # is treated as comment (C#v0.2.1.svg will be “C”)
• special characters (or anything, really) can be imported based on its Unicode codepoint (U+<codepoint>.svg, so U+2e.svg will be “.”, U+5b.svg will be “[“)

It can also set the character width based on the SVG document size. These were my SVG Folder Import settings:

Although I had to manually unselect a few files, but that was just a few clicks.

It can also export EOT and SVG formats, the OTF is unfortunately a “plus version” feature. It can also tune spacing, kerning, ligatures, but I didn’t need these.

After making a few characters I exported a TTF to play around with. It looked great! So I added the rest of the characters I had a photo of and also found some photos online (see the Sources in the readme), and then guessed the rest of the characters based on these. The full character set as of v0.12 looks like this:

I did a side-by-side comparison with a photo and also a bit of a test run.

And well, I could not resist to try it in a terminal…

Downloading the font

I put everything in the wu3-segments GitHub repo, you can download the font files under the releases tab:
https://github.com/gheja/wu3-segments/releases/

Feel free to use it and I’d appreciate any feedback you might have!

Further?

At this point I thought the font was quite complete and I didn’t expect any major changes – except… derloris on Mastodon showed me two photos they took with lowercase letters displayed!

I did not expect this. Although they look a bit janky (the spacing is all over the place) now I feel the font is not quite as complete as I thought. And maybe now I should add these characters, too…

Some notes

Collecting all the characters needed for a font seems to be an impossible task, as only a few station names and the numbers are displayed, apart from occasional messages like “Use public transport, save the climate!” (“Öffis nützen, Klima schützen!”, which actually led to misbehave/crash the mobile apps and other connected services). So long story short, I had to guess several characters (surprisingly Q, X, Y are amongst those).

If you have any pictures of any letters that are missing or marked as guesses, I’d happy to update the font, see the About section of the readme for details.

Posting about it on Mastodon (okay, tooting is the correct verb) was a big part of kicking this project forward. Each of my posts about it got more and more positive reactions and encouragement: my first impressions, kind of joking about making a font, releasing the preview TTF font file, they all got great reactions.

Marcin Wichary made a really nice Segmented type appreciation corner webpage to play around with segmented fonts, I think you might like it.

One strict requirement was to have it running locally on my machine, offline, without contacting any external services.

While searching I have found quite a few outdated libraries, examples and outdated or unnecessarily complex tutorials.

In this post I am using an up-to-date Ubuntu, up-to-date Python and the latest face_recognition library (that uses the dlib machine learning library).

I also try to focus on getting it working as simple as possible and then add the extras and optimizations.

All versions are available in the GitHub repository: https://github.com/gheja/face-recognition-test/

The goal

My goal is to have known people recognized on photos, given a (smaller) set of photos of them.

The second goal is to find unrecognized faces and group them by person. Then we can mark them as known people and find them in other photos.

Terminology used

Face detection – finding one or more faces in an image.

Face recognition – finding the person for a detected face from a known set of faces.

Clustering – arranging detected faces into groups based on facial features (without knowledge of the person).

Facial encoding – an array holding numeric data based on facial features extracted from a face.

Setting up the environment

My test environment is a live Ubuntu 20.04 LTS system.

First we need to enable the universe apt repository that contains the community-maintained free and open-source software to be able to install pip (the package installer for Python). This will also update the apt cache so no need for apt update.

Then we will install python3-pip for managing the Python packages and cmake to be able to compile some modules.

After this we can install the face_recognition and the scikit-learn packages and we can start working. The latter is needed only for clustering.

sudo add-apt-repository universe
sudo apt install python3-pip cmake

pip3 install face_recognition scikit-learn

Finding and recognizing known faces

To recognize faces first we need to have a list of known faces. The script searches the known faces in the known_faces directory, grouped by person in a sub-directory, holding the files. For example:

known_faces
  Alice Test
    alice_1.jpg
    ms_test.jpg
  Bob Test
    bobby.jpg

First we import the required modules, save the current path to a variable (the script will use this as the base directory when searching for files), then prepare the global variables to hold the known faces and names:

import os
import numpy as np
import face_recognition

base_dir = os.getcwd()
known_face_encodings = []
known_face_names = []

Let’s create a load_known_faces() function that gets a list of all directories and files and opens the files one by one to find the faces in each, then picks the first and stores its encoding in known_face_encodings and their corresponding names in known_face_names.

def load_known_faces():
	print("Loading known faces...")
	
	os.chdir(os.path.join(base_dir, "known_faces"))
	
	for root, dirs, files in os.walk('.'):
		for f in files:
			filename = os.path.join(root, f)
			
			print("  {} > {}".format(root, f))
			
			image = face_recognition.load_image_file(filename)
			
			# Search for all the faces
			face_locations = face_recognition.face_locations(image)
			
			# Pick the first one and get its encoding
			face_encodings = face_recognition.face_encodings(image, [ face_locations[0] ])
			
			# Store the processed face
			known_face_encodings.append(face_encodings[0])
			known_face_names.append(os.path.basename(root))

With this we have all the known faces loaded, next task: recognizing them on unknown pictures.

Let’s create a recognize_faces() function that searches for the images to be processed in the test_images directory, then loads them one by one, searches all the faces in each, extracts the facial features and then calculates the likeliness to every known face. It then picks the nearest match (if good enough) and displays the corresponding name and confidence value.

def recognize_faces():
	print("Detecting and recognizing faces...")
	
	os.chdir(os.path.join(base_dir, "test_images"))
	
	for root, dirs, files in os.walk('.'):
		for f in files:
			filename = os.path.join(root, f)
			
			image = face_recognition.load_image_file(filename)
			
			print("  {}: processing...".format(filename))
			
			# Search for all the faces
			face_locations = face_recognition.face_locations(image)
			
			# Then calculate their encodings
			face_encodings = face_recognition.face_encodings(image, face_locations)
			
			# Loop through each face found in the unknown image
			for (top, right, bottom, left), face_encoding in zip(face_locations, face_encodings):
				name = "Unrecognized"
				confidence = 0
				
				# Find the distance of the face to every known faces. Distance is 0.0 at an exact match
				# and gets higher with the distance, can be more than 1.0
				face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
				
				# Use the known face with the smallest distance to the new face
				best_match_index = np.argmin(face_distances)
				
				# If there is a possible match
				if face_distances[best_match_index] < 0.6:
					name = known_face_names[best_match_index]
					
					# Confidence is the opposite of distance
					confidence = 1 - face_distances[best_match_index]
				
				print("    found {} with confidence {:.2f}".format(name, confidence))

Then just call these functions.

load_known_faces()
recognize_faces()

And that’s it for the basics and if all goes well. I have uploaded the script as recognize_v1.py.

Now it’s a good time to add some checks if images could be loaded correctly, faces could be detected, etc., see recognize_v2.py.

Highlighting recognized faces on pictures

I think the next logical step is to not just print the detected faces but show them on the picture.

Fortunately the face_locations array holds the top-left and bottom-right coordinates of the faces, we also got the best matches for them and the Pillow library (python module named PIL) can easily draw on images.

First we need to import the PIL and PIL.ImageDraw modules:

import PIL
import PIL.ImageDraw

Then after loading the image in recognize_faces() we create a PIL.ImageDraw.Draw to work with:

# Convert the image to a PIL-format image so that we can draw on top of it with the Pillow library
# See http://pillow.readthedocs.io/ for more about PIL/Pillow
image2 = PIL.Image.fromarray(image)

# Create a Draw instance to draw to
draw = PIL.ImageDraw.Draw(image2)

Then in the loop over the faces we draw a rectangle around the current face and write the confidence value and the name of the best candidate, optionally with colors depending on the quality of the match:

if confidence >= 0.5:
	color = (0, 190, 0)
	color2 = (255, 255, 255)
elif confidence >= 0.35:
	color = (0, 120, 190)
	color2 = (255, 255, 255)
elif confidence >= 0.25:
	color = (0, 0, 120)
	color2 = (255, 255, 255)
else:
	color = (30, 30, 30)
	color2 = (90, 90, 90)

# Draw a box around the face
draw.rectangle(((left, top), (right, bottom)), outline = color)

# Draw a label with the name below the face
text_width, text_height = draw.textsize(name)
draw.rectangle(((left, bottom), (right, bottom + text_height)), fill = color, outline = color)
draw.text((left + 6, bottom), "{:.2f} {}".format(confidence, name), fill = color2)

And right after finishing the processing of the image we save the copy with the highlights in the annotated_images directory:

# Remove the drawing library from memory as per the Pillow docs
del draw

# Save the annotated image
outputName = os.path.join(base_dir, "annotated_images", f)
image2.save(outputName)

Note: the annotated_images directory is not emptied before run, the files there will be retained or overwritten.

The full source code is available as recognize_v3.py.

See an test run of this version in the last section of this post below.

Clustering faces automatically

It would be nice to collect all the unrecognized faces and group them by person, wouldn’t it? This is what clustering is for.

Sidenote: So far we searched for people we “labeled” manually earlier for the script, this is supervised learning. If we want to group similar faces, we either need to supply how many groups (people) we want or use unsupervised learning to do it automatically. In this post we want the latter by using DBSCAN (density-based spatial clustering of applications with noise).

First we need to import the DBSCAN clustering from scikit-learn library (python module named sklearn):

from sklearn.cluster import DBSCAN

Then add an array to hold the unrecognized faces:

unrecognized_faces = []

Then in recognize_faces() add the faces to the array:

if name == "Unrecognized":
	unrecognized_faces.append({
		"face_location": (top, right, bottom, left),
		"face_encoding": face_encoding,
		"filename": filename
	})

And now everything is ready for clustering.

Clustering is really simple thanks to the scikit library, it just needs the faces in an array and two parameters: min_samples – telling how many faces needed at least to form a group, and eps – telling how far the faces needed to be at most to be considered in that group.

After this we copy the faces into separate PNG files (to retain as much detail as possible), grouped by person in directories. The directories will be named person_0, person_1, etc. with an exception for the rest: unclustered.

For example, after processing three test pictures with 6 faces in total we should get similar results:

test_images
  group_photo_1.jpg
  group_photo_2.jpg
  group_photo_3.jpg

clustered_faces
  person_0
    face_1.png
    face_2.png
    face_5.png
  person_1
    face_3.png
    face_6.png
    face_7.png
  unclustered
    face_0.png
    face_4.png

def cluster_unrecognized_faces():
	print("Clustering unrecognized faces...")
	
	if len(unrecognized_faces) == 0:
		return
	
	encodings = []
	
	# Create an array suitable for DBSCAN conaining only the encodings
	for face in unrecognized_faces:
		encodings.append(face["face_encoding"])
	
	# Group at least 3 faces ("min_samples") with a maximal distance
	# of 0.5 ("eps"). From a few runs these numbers seems to be working
	# well for me. Your case may vary so try to tune these if you don't
	# like your results.
	#
	# For more info see:
	#   https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html
	clt = DBSCAN(metric="euclidean", n_jobs=-1, min_samples=3, eps=0.5)
	clt.fit(encodings)
	
	print("Saving clustered faces...")
	
	face_index = 0
	for face in unrecognized_faces:
		person_index = clt.labels_[face_index]
		
		if person_index == -1:
			name = "unclustered"
		else:
			name = "person_%d" % (person_index)
		
		filename = os.path.join(base_dir, "test_images", face["filename"])
		clustered_filename = os.path.join(base_dir, "clustered_faces", name, "face_%d.png" % (face_index))
		
		# Create the directory for this person if needed
		if not os.path.exists(os.path.dirname(clustered_filename)):
			os.makedirs(os.path.dirname(clustered_filename))
		
		try:
			image = face_recognition.load_image_file(filename)
		except:
			print("{}: failed to load, skipping.".format(filename))
			continue
		
		(top, right, bottom, left) = face["face_location"]
		
		# Convert the image to a PIL-format image so that we can draw on top of it with the Pillow library
		# See http://pillow.readthedocs.io/ for more about PIL/Pillow
		image2 = PIL.Image.fromarray(image)
		
		# Crop the image to the face
		image2 = image2.crop((left, top, right, bottom))
		
		image2.save(clustered_filename)
		
		face_index += 1

As a last step we need to call this function after recognize_faces() collected the unrecognized faces:

cluster_unrecognized_faces()

And we’re good to go.

After running the script we can examine the grouped faces, correct them manually if needed and move to a sub-directory of known_faces to be able to recognize these people next time.

Note: the clustered_faces directory is not emptied before run, the files there will be retained or overwritten, the new and old faces might mix for subsequent runs. It is a good idea to clear it by hand before running the script.

The full source code is available as recognize_v4.py.

Optimizations

As the locating and encoding of faces take the most significant time it is a good idea to store their results instead of running them on each images every time – for this I added pickle and store the data in known_faces_cache, if there is a stored version available the program uses it, otherwise it processes the image and stores the result for next use. See recognize_v5.py.

Processing a huge image can take considerably longer time so adding a resize that limits to 1200×1200 pixels might be a good idea, this is what load_image() does. See recognize_v6.py.

Also, dlib (the library that face_recognition uses) supports two methods for face detection: HOG+SVN (histogram of oriented gradients and linear support vector classification) and CNN (convolutional neural network). The first one is called “hog”, it’s reasonably fast on CPU and is the default; the second one is “cnn”, it is more accurate for faces in non-ideal angles, uses more memory and it is strongly recommended to have a GPU with CUDA support. To use “cnn” modify the face_recognition.face_locations(image) to face_recognition.face_locations(image, model="cnn"). Also, “cnn” supports batch processing, too. Read more at CNN based face detector from dlib and the docs.

My test runs and conclusion

I started my tests by adding several photos (5 to 8) of a few actors and actresses, all out of character and from the last few years (2018-2020). Sources were like Wikipedia, Oscar galas, etc.

After running the script I moved the clustered images from clustered_faces to known_faces with the proper names – without manually correcting anything else.

Then I collected a few photos, mostly from posters, stills from movies and episodes, etc. and ran the script.

It surprised me how accurately it recognized the faces. So I added a few more pictures, with non-ideal angles, partially covered faces, ones with sunglasses, older ones (shot 10 or even 29 years earlier) and… see for yourself below.

(Click on a photo to open gallery.)

The success of recognition with sunglasses and on 11 years old photos surely surprised me, especially with the high confidence rate.

But what happens if we have have incomplete data? For example if we only know a subset of the people we try to recognize on photos? Well, I have removed some of the actors from the known_faces directory and ran the script again.

(Click on a photo to open the gallery.)

This looks way worse. But of course, whether this is a spectacular failure or a hiccup depends on the parameters and the intended use.

All the incorrectly recognized faces are with less than 0.5 confidence (distance higher than or equal to 0.5), tuning this parameter can get rid of them but then some of the correct recognitions will be cut to.

And also, intended use: for a just for fun project you’d just need to tune them and then handle the clusters of unrecognized faces. For something more serious, don’t. Just don’t use facial recognition.

Recognizing a face based on a 2D image (from a video or photo) is way too easy to trick and also unreliable, unsuitable for any serious use. A few articles from 2020 alone:

• Wrongfully Accused by an Algorithm – New York Times
• Facial recognition fails on race, government study says – BBC
• Facial-recognition software fails to correctly identify people ‘96% of the time,’ Detroit police chief says – Business Insider
• Outrage over police brutality has finally convinced Amazon, Microsoft, and IBM to rule out selling facial recognition tech to law enforcement. […] – Business Insider

So while it is fun and can yield surprising results, it heavily depends on the quality of the input, fine-tuning and the algorithms used – even for the big guys like Amazon, Microsoft and IBM.

But by all means, try it, play with it, create your next pet project – but be aware of the possible downsides as well.

Copyright info: Each photo was used for educational purposes only and I tried my best to have the correct copyright info displayed, please contact me and I will correct them or remove the photo if needed.

The data packet – although it is only 59 bits – holds a lot: the current date and time, day of week, current time zone (CET or CEST), upcoming daylight saving time change and leap seconds, civil danger/emergency warnings, weather forecast and parity bits. (The weather forecast is in a proprietary format, though, more on that later.)

It is transmitted on 77.5 kHz at a whopping 1 bit per second rate using amplitude modulation, the 59 data bits are followed by a sync period as well. The frequency is too low for my SDR so I could not catpure directly but here is a capture made using WebSDR (opened in Audacity, commented by me):

It looks clearly here there are short (100 ms) and long periods (200 ms) of low amplitude and a significantly larger period (~1800 ms) of high amplitude – according to the the specifications – they are 0 and 1 bits, and the sync signal, respectively.

The bits always start on the beginning of a second, so the data for the 1st second is always starting right at the beginning of the minute.

The format of a DCF77 packet

0-00010100101001-00010-1-1100100-1-100000-1-010010-001-10001-00000100-0-0
| |------------/ |:|:| | |-----/ | |----/ | |----/ |-/ |---/ |------/ | |
| |              |:|:| | |       | |      | |      |   |     |        | minute mark (always 0)
| |              |:|:| | |       | |      | |      |   |     |        parity for prev. section
| |              |:|:| | |       | |      | |      |   |     years
| |              |:|:| | |       | |      | |      |   months
| |              |:|:| | |       | |      | |      day of week
| |              |:|:| | |       | |      | days
| |              |:|:| | |       | |      parity for hours
| |              |:|:| | |       | hours
| |              |:|:| | |       parity for minutes
| |              |:|:| | minutes
| |              |:|:| minute start bit (always 1)
| |              |:|:leap second announcement
| |              |:|CET
| |              |:CEST
| |              |summer time announcement
| |              call bit
| civil warning bits and weather information
start of minute (always 0)

The values are encoded digit by digit in BCD:
  a b c d e f g h  =>  a*1 + b*2 + c*4 + d*8 + e*10 + f*20 + g*40 + h*80
  0 1 0 0 0 0 1 0  =>  42
  1 1 1 0 1 0 0 1  =>  97

Hours: in 24 hours format

Day of week: 1 is Monday ... 7 is Sunday

Years: within century (two digits)

More info in the GitHub repository: https://github.com/gheja/dcf77-decoder

I created a JavaScript decoder as well: https://gheja.github.io/dcf77-decoder/tools/decode_js/decode.html

Jumping on it

I ordered a receiver module from eBay that only converts the signal to digital levels but does not decode them – after all, that’s the fun part and I want to do that. It turned out that the low amplitude corresponds to high level, high amplitude to low level for this specific module.

In my limited experience the signal is really sensitive to noise and shielding, so it needs some work to be reliable in non-ideal conditions. For more info see Noise section below.

The documentation of the DCF77 receiver module has suggestions on antenna orientation as well:

Hardware

I tried to go with a simple solution but… I ended up using an ATMega328P for the microcontroller with an USBasp programmer, USB-TTL converter for communication, and the DCF77 receiver module mentioned earlier.

I read conflicting specifications for the DCF77 receiver (one stated 3.3 volts is fine, the other stated 3.0 volts as the maximum) so I added an LM317T voltage regulator to supply the 3.0 volts to it to be on the safe side (also I don’t have a spare one).

And just to be sure I don’t fry anything I always connected only one device to my computer.

Software

On the software end I decided to go with a simple state machine with the following states: waiting for signal, receiving signal. I said it’s simple (it actually had more states but they were unnecessary).

So the program reads the logic level on one pin with some basic noise reduction – read it, wait a little, repeat 200 times, assume it’s “1” if 75% of reads were “high” (confusingly meaning the received radio signal was “low”).

The program measures the length of the “high” signal so I fine-tuned (guessed) the delay above to repeat the reading and have enough precision to distinguish between the meanings:

• 0-69: invalid
• 70-84: logic 1
• 85-94: logic 0
• 95-169: invalid
• 170-189: sync
• 190+: invalid

Then based on the current state it processes the received data: if it waits for the sync and got the sync then it resets everything and switches to “receiving signal” state, if it is in receiving state it stores the bit just received.

If an error occurs anywhere (received an invalid signal) or the 59th bit is received it goes back to “waiting for sync” state. In “waiting for sync” it also checks if the data received in the last minute was valid and if it is then decodes and sends it over serial.

There is a debug option to send more detailed info over serial, see the readme for this.

I have also captured a few samples, both correct and invalid ones, check the samples.txt.

Accuracy

For the test I removed most of the messages and added a line with “!” only to be sent out as soon as sync is detected. Then checked the the accuracy of my laptop’s clock using ntpdate and on time.is, created a quick CodePen demo showing the current time and started receiving:

It turns out the time is off by around 120 milliseconds. I believe it is mostly due to my noise filtering (see Noise below) and the time it takes to transmit the data over the 9600 baud serial port. It might worth a try to tune the noise filtering but it would be much easier to just acknowledge the delay and sync the (hypothetical) other device to the received time +0.12 seconds.

Noise

We have radio controlled clocks at work and they don’t really work. They need to be placed in the windows of a specific side of the building (in the direction of the transmitter) to sync the time so they regularly take a sun bath at weekends of DST changes. And then only 90% of them syncs, the rest need manual reset and will sync overnight. Their manual recommends placing them 2 meters away from electric wiring and steel – good luck in an office, really.

I started experimenting with it in the middle of the living room, it seems to be quite noisy, but instead of moving to a more ideal location I decided to add some noise filtering in my code. It results some delay but makes the reading way more reliable.

Also, my laptop charger (while charging) adds a lot of noise, it renders about around 90% of readings unusable.

In the current version the serial port takes time to transmit the data at 9600 bits/second, in some cases longer than the pulse length of “zero” (100 ms) which prevents the correct detection of the edge.

Furhter?

That’s all for now. There are places for further optimization but I think this is enough for a proof of concept implementation.

Some interesting stuffs

Erik de Ruiter created a DCF77 Analyzer/clock that decodes and displays the data, as well as the raw bits and some more.

There are some GPS to DCF77 modules, too that create DCF77 signal based on date and time received from GPS satellites. For places with poor coverage or just fun.

The weather data is transmitted in a proprietary format, supplied by Meteotime. To decode it you need a HKW581 Meteotime decoder chip, the price includes the lifetime license.

As an interesting fact, people managed to reverse engineer the Meteotime format, an example is MeteoDecode but the keys were removed on request, also Arduino Projects4u has a description and example, too.

More

https://www.ptb.de/cms/en/ptb/fachabteilungen/abt4/fb-44/ag-442/dissemination-of-legal-time/dcf77/dcf77-amplitude-modulation.html

http://arduino-projects4u.com/home-weather-station/

https://github.com/ottojo/MeteoDecode

Decoding these messages by eye might seem tricky at first but it can be done quickly once you know the how. Although there are softwares to do this automatically, it is a fun excercise and gives a better understanding of the messages in the air.

Intro

Softwares I use in this post and find useful:

• Gqrx SDR – to inspect the radio spectrum and capture the messages
• Audacity – to visually inspect the captured data and decode by eye
• RTL 433 – receiving and decoding messages on the command line
• Domoticz – to (automatically) register devices and decode their messages, draw graphs and even control some of them

I have a Nooelec NESDR Mini that I used in the screenshots below but it should work with any Software Defined Radio (SDR).

Although I am talking about 433 MHz, the message format is independent from the frequency – however if you plan to buy any device see Note 1 on the bottom of the page.

The easy way

Just start a decoder, wait for your message to arrive and you’re done.

Hey, that’s cheating!

The line format

There are two formats commonly used by these communications: Pulse Width Modulation (PWM) and Pulse Position Modulation (PPM). They both encode binary values (zeros and ones) but differently. I have added vertical lines to highlight the beginning of the values.

PWM encodes the data by sending a short pulse then a long pause or a long pulse then a short pause. Depending on the protocol used one means one the other zero. Note that the transmissions have a fixed length.

PPM encodes the data by sending a fixed length pulse, then a short or long pause before sending the next – the short period encodes a zero, the long encodes a one – or reversed, depending on the protocol used. Note that the two values have different lengths.

Capturing the message

I am using Gqrx as it is a fairly standard SDR on Linux and works just fine.

First you need to start the processing (1), then find the frequency you’re intereted in using the tuner (2). If you find the frequency you’re looking for, set the receiver to AM and filter to Normal (3) then you can start recording using (4). The message is marked with (5) in this screenshot.

The output will be a WAVE audio file (.wav) that can be opened using any sound editor software.

Note: you might need to adjust gains to have a clear signal, try to fiddle with LNA gain on Input controls tab and Gain on Audio tab.

Checking the captured message

I used Audacity in this post but any sound editor will do the job, you only need to zoom in on parts of a WAVE file.

After opening the recording you should see something like this.

You should see some noise and some structure (1), the latter is the message you’ve captured. Select and zoom in to it using (2).

Notice that the message seems to be repeated – there are larger gaps and then the packet starts again (gap, 1 peak, 3 peaks, …).

It is very common for a message to contain the same packet multiple times, as the trasmitters are not synchronised, all of them send as they like, collisions are common. Repeating the data increases the chance of a clear reception.

In this case the wireless thermometer sends the package 7 times, leaving a long gap between each time, according to its protocol. Zoom in on the first one.

Decoding by eye

After we successfully found the useful part of the message all we need to do is decoding it. Easy, right?

If you don’t know the packet format but know the measurements transmitted (as this is a temperature and humidity meter) you can guess the format. Most of the sensors send the data in a very basic format, usually the temperature in 0.1 °C and humidity in percent.

By fiddling these values, one at a time, capturing and decoding the packets we can find the bits that change and find out their meaning.

There are a few things to consider when guessing the format:

• modulation: PPM or PWM
• gap length – bit value pairing: is the short zero and long one or the other way around
• bit-endiannes: the order of bits in bytes. 131 (128+2+1) can be encoded two ways 10000011 (big endian) or 11000001 (little endian)
• checksum: if there is a part of the message that changes when any of the other part changes, that might be a checksum and can be ignored when eye-decoding (not common)
• rolling code: if there is a part that changes with all transmissions it might be rolling code, can be ignored as well (not common)
• sequence number: if a part increases with every transmission it might be a sequence, can be ignored as well (not common)

First of all, the length of the pulses seems to be uniform but the gaps change. It looks like Pulse Position Modulation (PPM), let’s assign a value for the gaps – zero for the short ones, one for the long ones.

So we know the following parameters (either because it is a switch or because it is displayed on the device):

• the sensor was set to channel #2,
• temperature was 12.3 °C,
• humidity was 45%.

Let’s capture and decode more packets, while keeping an eye on the parameters:

-------------------------------- data --- chan  temp  humi
10010000 00011001 00000111 10110010 11010    2  12.3    45
10010000 00011001 00000111 11000010 11010    2  12.4    45
10010000 00011001 00000111 11010010 11010    2  12.5    45
10010000 00011001 00000111 11010011 00100    2  12.5    50
10010000 00011001 00000111 11010011 11000    2  12.5    60
10010000 00011001 00000111 11010011 11000    2  12.5    60
10010000 00011010 00000111 11010011 11000    3  12.5    60
10010000 00011000 00000111 11010011 11000    1  12.5    60
10010000 00011000 00000111 11100011 11000    1  12.6    60
10010000 00011000 00000111 11100100 00010    1  12.6    65
10010000 00011000 00010000 00110100 00010    1  25.9    65
xxxxxxxx xxxxxxcc xxxttttt ttttthhh hhhhx

By noting which bits change when a parameter changes we can find and mark the correlations – see the last line, I marked “c” for channel change, “t” for temperature, “h” for humidity.

In the case of this device we can conclude the following with great certainty:

• channel is sent on bits 15-16, value is “switch position – 1”, big endian,
• temperature is sent on bits 20-29, value is in 0.1 °C, big endian,
• humidity is sent on bits 30-36, value is in percent, big endian,
• the rest of the bits are still unknown to us.

To increase certanity we should test more, but I am happy with these results for now, feel free to continue.

Decoding another device

I also have an RF remote with several buttons. I started a capture the same way as described before and pressed three buttons:

… opened it in Audacity:

… as I held the buttons for a short amount of time, the remote sent the packet 3-4 times. Cutting those and the noise here are the three buttons:

These packets look different than before, the pulses are different in length but with the gaps the bits seem to be consisten in length – so this looks like PWM.

Note: as this is binary data the difference in amplitudes (height of the signal) does not matter, it is an artifact of the sender/receiver.

Let’s assign one and zero values for the short/long pulses and start decoding by eye:

So it seems that significant part of the packet is constant, only four bits (marked in teal) varies, these must be the values for the buttons.

We will validate these findings in an upcoming blog post.

Using software to receive and decode the packets

So after decoding by hand let’s see a software doing this for us. I use RTL 433 here, it grabs the first SDR it finds and tunes automatically to 433.92 MHz – handy.

Currently it knows around 145 devices (or packet formats), adding -A switch shows a dump of the received package and some analytics it uses to parse it.

First of all, the colored part (that I showed earlier) shows the parsed data, (1) indicates it is an on-off keying (OOK) encoded packet, (2) shows it is indeed PPM and (3) is the actual decoded data.

Another great advantage is that RTL 433 being an open source software, you can check out the decoder and the documentation for the protocols. For the Prologue sensor looks like this:

More info here: https://github.com/merbanan/rtl_433/blob/master/src/devices/

More

Dave posted in detail about setting up a remote controlled Balsam Hill Christmas tree from scratch in his OpenHAB home automation system, with Alexa integration. It is a good read, if you enjoyed this one I highly recommend checking his post out, too: Automating a Balsam Hill Artificial Tree

Coming up next

In a future post I will share how to send packets by copying the ones we captured and also creating our own.

Notes

Note 1: In Europe the most common frequency these devices use is 433.92 MHz, which is usually referred as “433 MHz” and sometimes “434 MHz”. There are two other frequencies typically used, 433.42 MHz and 868.3 MHz. In the US 915 MHz is typical (for more precise definitions see ITU and search for your country). It is important to note that not all receivers support multiple frequencies, you might need separate receivers for each.

I have a DS1820 one-wire sensor plugged on a Linux machine that polls the temperature every minute then stores it in a CSV. Since I recently installed Domoticz on a Raspberry PI, I wanted to show the values there as well. I’ll show what you need to do this.

Open the Setup menu and choose the Hardware option. Enter a name you like and select the HTTP/HTTPS poller type, you should see the following:

Data Timeout: leave it Disabled

ContentType: the type of received data, application/json

URL: the full URL including scheme, host name and path where you will have the JSON file with the latest data.

Command: this is a filename (with extension) for a script that will handle the downloaded JSON, which Domoticz will search at …/domoticz/scripts/lua_parsers/<command>
Tehre’s a basic example called example_json.lua, for a this setup it will be fine.

Refresh: refresh interval in seconds.

After adding you should see it immediately in the list:

Click on Create Virtual Sensors button to add a sensor, give it a custom name, select the type of data you want to log and select OK. It should appear in your devices list:

Note the Idx of the sensor (not the ID nor Unit, the first number) which is 15 in my case. You will need to put this in the JSON file on remoteserver in a field named id – yes, this is inconsistent.

The JSON file you need to provide on remoteserver can be as simple as this:

{ "id": 15, "temperature": 12.3 }

Tip: you can output this with any script, you can consider it a string, just replace the value with the latest measurement. You also need to update it periodically.

As soon as you added the poller, Domoticz starts to poll the URL you supplied and passes the result to the script you supplied.

If in the next few minutes you start to see the data in Domoticz, congratulations, you have it working. If not, check the Troubleshooting section below.

Note 1: Domoticz executes the script one time for every HTTP/HTTPS poller “hardware” and you can have multiple sensors configured for one poller. In this case you need to supply multiple id/idx values and multiple values for measurements and need a custom script updating each sensor.

Note 2: In the example script the id is read from the JSON and blindly supplied to Domoticz which will update the sensor having that as idx. This may overwrite the data of any other sensor if configured improperly.

Unfortunately Domoticz is only supplying the hardware idx as hwdId variable but not the idx of the sensor(s) on that poller, so if you want to prevent it you need to hardcode the idx of the sensor(s) in your script, e.g. you can create remote_1_json.lua with the content:

local idx = 15
local temperature = domoticz_applyJsonPath(request['content'], '.temperature')
domoticz_updateDevice(idx, '' , temperature)

Then you can entirely skip the id field in JSON:

{ "temperature": 12.3 }

Just make sure you update the script’s name on the web interface.

Note 3: you might need to add the following header on the remote host to your response:

Content-Type: application/json

Troubleshooting

Error: CLuaHandler (updateDevice from LUA) : Incorrect parameters type 

One of the field received in JSON is invalid. The id should be an integer, temperature should be a float.

Error: CLuaHandler: cannot open /home/.../domoticz/scripts/lua_parsers/example_json: No such file or directory 

You need to provide a full file name with extension in the Command field of the Setup > Hardware page. Also, make sure the script exists.

CLuaHandler (updateDevice from LUA) : idx=15 nvalue= svalue=12.3 invalue=0 signallevel=12 batterylevel=255 

This is normal, the script parsed the JSON and issued an update for the sensor. If you can’t see the update on the interface, double check the idx or id parameter you have in the JSON file.

Bokosan is a reverse Sokoban where boxes are on a goal position and the player needs to pull them away while avoiding deadly spikes and falling into pits.

The game has 20 levels, single player mode, time challenge mode with online leaderboard, customizable player appearance, sound, 4 music tracks (6 in full version) and a few more features.

It is an entry for the js13kgames gamedev competition, where the goal is to create a web browser game using only JavaScript/CSS/HTML, with final size (after ZIP compression) below 13 kB, in a month. It was created for the 2015 jam for the theme “reverse”.

This year I worked together with my brother, Zsolt, he composed the music and created most of the levels.

Play it here.

Graphics

I’ve decided to go with raster graphics, stored in PNG with low color count and replacing colors in code if needed. I’ve tried several other methods to store graphics (putting each sprite in a string, etc.) but the overhead for decoding was just too big.

(The image is scaled down, click here for full version.)

The final PNG contains the Jostix font by Ray Larabie (only uppercase letters, numbers, punctuation marks), 13 tiles, 9 character animation frames and 3 other graphics; it is 700×48 pixels, 20 colors and the final size is 1600 bytes after TinyPNG compression.

It might worth to note that for character animation I used oldschool tricks of mirroring and rotating to reduce the frames. I have only drawn the standing and step with right foot frames and then “stand, step, stand, mirrored step” sequence. Also, I just rotated the “walking up” frames for the “walking right”, as well as the “walking down” for “walking left”. For pulling I did the same but used one frame only and played the sounds of steps.

The colors of customizable parts of the character (hard hat, shirt, pants) are replaced during render (and not on the sprite sheet) as each entry in the challenge leaderboard can be different.

Quick note: when opening an HTML from local disk and accessing its contents in JavaScript, like reading pixels of an image (rather than putting in DOM and showing to user, or using drawImage()) the Cross Origin protection kicks in and prevents the operation. Although when opening from a web server there is no such problem, this feature can be disabled but it’s strongly not recommended. Worth to note that JS engines track the data they handle so reading a canvas where a drawImage() copied a portion of a protected image will result in an error as well.

Sound effects

For the sound effect we used jsfxr to generate them.

I have added a few minor optimizations for jsfxr before compressing the source via Closure Compiler, and after that manually edited the compressed code to be more suitable for ZIP.

Music

The music was created entirely by my brother, Zsolt.

I have created a small tracker that implements the basic features of an XM, handles only one pattern and loops it, but also have a map that describes which channels should be played on which loop (and also an option for intro that will be only played once at the beginning) – this way a song can be made up of a short period of music but can be dynamic and changing on each loop. Also created a converter that reads an XM file and creates the JavaScript arrays for the player, so Zsolt could use MilkyTracker to compose.

The samples were produced using jsfxr just like the sound effects (they are actually in the same array) and are altered on the fly for player needs (changing playback speeds to create the notes).

I have also created a frontend for playback later, check it out.

Level design

The levels were by designed mostly by Zsolt, with a few exceptions.

They are stored as simple strings – ZIP is really good at compressing text.

The levels are locked by default. Only the already solved and the next two unsolved are unlocked, as an attempt to prevent the player from getting stuck on one level.

And as the steps are counted for each level it is just natural that achievement hunters appear, so we added an “A+” badge for them for completing the level with at most the minimal steps we completed them during development, displayed in the level selector.

Challenge mode

There is a challenge mode with online leaderboard showing the name, time and character profile picture for each record.

There are 6 challenges, each consist of 3-4 already completed levels, the player plays them one after the other and the time is recorded and shown on the leaderboard.

Statistics

The game collects statistics about each play: play time, moves made, pulls done, levels started, levels finished.

The statistics is stored on the server side, the games recevies the current snapshot of the statistics and the one from a minute before, and then interpolates them during the next minute – this makes the display much more active.

I loved to see the numbers rolling during the vote period, unfortunately the statistics are no longer available on the jam website. [Although I made a few screenshots, but I could not find them yet.]

Update: since then I abandoned the domain, the server and uploaded the game to itch.io so there is no server to handle the statistics.

Server side

The server keeps track of challenge leaderboards and statistics. It does not do anything more, really.

Update: I am no longer running the server, check last section.

Compression

I basically go with the idea of writing readable code and leaving the low level optimization and compression to the tools.

Most of the time this works just fine as for a really simplified example…

if (x == 3)
{
    return true;
}

return false;

… is exactly…

return x == 3;

… but the former can be read more easily and the tool will rewrite it to the latter anyways.

For code compression I used Google Closure Compiler in two phases: first is the advanced optimization without whitespace removal, then the whitespace removal. This leaves a chance for (easier) manual/scripted optimization to be more suitable for the next tool (ZIP in this case).

The spritesheet is compressed using TinyPNG.

I used as few files in the compiled version as possible, as each file adds at least 130 bytes to the final ZIP file – this can be a lot and it increases with file name length.

For the final ZIP file the rules dictate to be a standard ZIP file, so no 7zip or other advanced formats are permitted.

To optimize the final ZIP I used AdvanceCOMP’s AdvZIP which basically recompresses an existing ZIP file but keeps compatibility.

About 90% of the build process – starting from readable code to entry ZIP – was automated in a build script. This way I could track the effect of each changes on the final size. The rest was manual editing of code just before Closure Compiler whitespace removal.

Fun fact: the last ZIP before the submitted one was 13313 bytes, 1 byte over the limit. I saved that byte by changing the player name input from “Player name:” to “New name:”.

Final thoughts

I think Bokosan is the most polished game I’ve created until today (alone or in a team), it has the most content and feels the most complete. It also got a lot of positive feedbacks and it was really fun to catch my collagues playing it, trying to get the “A+” badge on all levels.

However, the game has not ranked in the Top 10 in Desktop nor Mobile category, in Server category it finished at 6th place and in Facebook special on 3rd place.

A huge thank you to Andrzej Mazur for organizing js13kgames competition once again, Ray Larabie for the Jostix font, and Markus Neubrand for Jsfxr!

This last part was what got me, I was really curious how others fit games in such a small space – what later turned out to be much more than it looked on the first sight.

As the competition was already halfway or was over when I found it (I can’t really recall) I could only examine others’ entries and decide I will participate next year, in 2013. I also tried to gather some ideas and set up a concept before the competition start date – strictly not a line of code was written.

There is a theme each year (which is optional but worth bonus points if honored), for 2013 it was bad luck – and I skipped it entirerily…

Okay and now about my entry.

My concept was to create a 2 player networked tower defense/tower attack game.

I must admit I am really bad at naming things, I just called the main (and only) client side file 13312.js and not long before the deadline I came up with the exceptionally uncreative name untitled13.

Of course to fit in 13 kB entries usually do not use existing engines, they are written from scratch – a thing I don’t mind at all, I’ve been always curious about the details and liked the control one has when starts from the ground up.

For graphics and animations I decided to go with a fixed color palette and dynamically generated content. All the objects consist of one or more layers rendered by a function that takes a string that describes a series of commands and their parameters. It turned out only two commands were necessary, a “draw a closed polygon filled with a color” and a “draw an alpha overlay” for shadows. The parameters were simply X and Y coordinates of the points, represented as characters in the base64 character set (64 distinct values), multiplied by the size of sprite. (For example: sprite size is 64×32, string is “AAAzjj”, therefore the coordinates are 0;0, 0;25, 35;17).

The map consists of starting points for the attacker, paths for the units to walk on, path intersections with switches, a few objective tiles for crystals the defender has to defend and concrete blocks to build the towers upon. There are three types of monsters with different parameters (speed, health) and three towers with different parameters (attack type, damage).

The networking approach is pretty simple but the most interesting part: the initial state is known to both players (the map is fixed in the entry version, in later versions it is randomized but the seed is shared), the mechanics are known as well and the actions can be described easily. So I came up with the idea to run a whole game instance for both players, create delayed action queues for them and synchronize them. When one player clicks a button the action is added to their queue with the configured delay, sent to the other player’s game, and a bit later executed on both instances at the same moment.

To get this working I needed reliable advancement of simulation steps, one that is not affected by skipped frames. I created a tick() method that steps the simulation by exactly one time slice (one simulation step), so when a frame is requested by the browser I calculate how many tick should have happened up until then since the last frame and call tick() that many times. (I usually go with this method to keep games frame rate independent – skipped frames are less worse than game slowing down.)

As both players are running the same game (only playing different sides), are aware of every details (only hidden from them), all events are happening at the exact same time, the game should play out identically for both of them. Later I learned this is called simultaneous simulation and it is the core of great games like Age of Empires, Starcraft and a lot more.

In this form however there is a serious flaw. There is no guarantee that the actions will reach the other player before they are due, it is blindly trusted that it will. For example: Let the action queue delay be 10 frames. The defender clicks to build a tower at frame #100, this will be executed at frame #110. If the network lag is say 7 frames then the other player gets the notification at frame #107, still 3 frames before execution, the tower gets built at frame #110 in both instances and shoots the attacker just before it reaches the crystal. But if the network lag is more than the queue delay, say 13 frames then at frame #110 the tower gets built in one instance, then the notification arrives at frame #113 for the other player, tower gets built at frame #113 when it’s already too late, the crystal was reached and the game is over. In the defender’s instance the crystal is saved, in the attacker’s the crystal is destroyed – the game is out of sync.

To address this the other instance must be notified about no action as well periodically to make sure nothing is missed. And when there is no info, the game should pause and wait until it arrives. This is a feature I had no time to think through and implement before submitting my entry.

The server part is running in node.js (my very first application) and it is very basic, it just matches the players by their IDs and passes the messages between them.

Random minor thoughts: The map is a hexagonal grid, which made finding neighbour tiles for concrete block and fog calculation ugly, but it looks so much better than a square grid – although it is handled like that. The map has fog to hide tiles that are too far away from units/towers. The UI is usable but confusing and unhelpful at first.

The project is on GitHub, the version I submitted is in the js13kgames-2013-entry branch. I like to commit often (and to have descriptive commit messages) so the submitted version had 381 commits, but for some reason I have not uploaded the final ZIP file, it was around 11255 bytes of 13312, so I used up 84.5% of the limit.

The game was better received than I expected, it ranked 1st place in Server category and 9th place in Desktop category.

The competition had awesome prizes including various licenses for game engines and gamedev related services, ebooks, physical books and vouchers. It was organized by Andrzej Mazur who tirelessly did an epic job, I’d like to thank him again for organizing and hosting this event, finding sponsors, arranging prizes and of course organizing the event annually since then, so thank you Andrzej!

Check out js13kgames.com for previous and upcoming competitions.

One of the most common is the skewed print, like in the following images.

This is most likely because of the stepper motors skipping steps. The problem can be traced back to one of two reasons: a belt touching the pylon or too high printing speed. Fortunately both problems can be fixed easily.

First check all your belts during a print. See if any of them is touching the plastic of the pylon, rubbing against it. If yes then check the pulley/carriage attached to the belt, find where the belt is fixed to it. Try to push the belt a little to the front or the back. If this does not help you can also try to loosen the screw holding the bearing in place on the top of the tower a bit.

If you see no problem with the belts it can be a problem with the stepper motors moving the belts. If the printing speed is too high then the motors tend to skip steps. The 101hero is not a fast printer, the stock motors support up to 14 mm/s printing and travelling. Unfortunately in some cases this speed is still too high for the motors even with stock motors and stock GCode files (from 101land.com).

If you have the model you should try to reduce the speed of the printing by configuring the printing as well as travelling speeds to no more than 14 mm/s, maybe down to 10 mm/s.

If you do not have the model but only the GCode file (the ones that can be downloaded from 101land.com) you need to add a command to reduce the printing speed to a certain percent. Open the “101hero” file on your SD card with a text editor, find the line starting with G28 (“Move to origin/home”), and add a line right after that stating “M220 S75”. This command is “Set speed factor override percentage” and the parameter is the percentage (14 mm/s * 75% = 10.5 mm/s).

If printing with these settings fixes the problem then you should try to use higher printing speeds (or higher percentage) to see what is the speed your printer is still reliable at.

If you still have the problem leave a comment here or post in the 101 Hero (Unofficial) Facebook group.

This part is about assembling the printer step by step.

First connect only the PSU to the controller and switch it ON and OFF, the red LED should light up when it is ON.

Now connect the motors and end-stops to the controller (do not assemble anything yet) and attach the SD card slot. The intended order of the pylons as well as the motor-end-stop pairings are below:

After connecting these put a simple “go home” file on your SD card and then power it up. The motors should start moving the carriages (all three at the same time) until one hits its end-stop. Then the other two should stop and only that should move back and forth triggering the end-stop a few times. After this the remaining two should do the same one by one.

If all three carriages hit the end-stops then congrats, you have working motors and end-stops, this is a good start! If not… well then you are in the unlucky group of quite a few people, including me. In this case check out the 101 Hero (Unofficial) Facebook group.

You can restart the procedure by pushing the reset button or power cycling the controller board.

Now on to assembling the printer!

At the top of the pylon arrange the red-black cable of the end-stop so it goes through the shaft:

At the bottom of the pylon the red-black cable should go through the shaft just like above and arrange the cables of the motor like on the picture:

Make sure for each pylons that the spring on the belt is tight to provide tension…

… the belts are properly on the wheel at the top…

… and the bottom:

If the belts are off the wheels try to put them back carefully. If that is not enough then you can loosen the motor a bit by loosening the two screws holding it in place (3mm Hex Socket aka Allan heads). If you have to remove the belt tensioner spring here’s a video how to put it back.

Next, assemble the bottom plate and the three pylons (4 pcs of M4x12 self-tapping screws (PH1 head) for each pylon):

And then add the top plate (2 pcs of M4x12 self-tapping screws (PH1 head) for each pylon, NOTE: the two inner holes remain empty on each end on the top):

And now fix the effector to the rods. The facing of the head does not matter but it can be useful to see the fan (to see if it is working) and the door (for change filament) from the front. The first screw…

… the second screw (you can safely leave it hanging)…

… and the rest:

Now stick some painter’s tape to the glass plate (you don’t need to cover the whole glass just make sure you have no overlaps of the tape) put it on the bottom plate and fix it in place using the paper clips like this:

Now connect the rest of the cables to the controller box:

Next step is to load the filament. The extruder assembly has a door you can open, this door pushes the filament to the extruder motor which is supplying the filament to the heated nozzle. Open up the door and put the filament in at the top of the extruder assembly, guide it into the tube after the motor, like this:

When completed close the door and tighten the screws (2 pcs M3x12 self-tapping screws (PH1 head)).

After completing this step the printer should look like this:

Which is a mess. The cable management is pretty much non-existent for the 101hero by default. People use different cable organizing methods, the most popular is the spiral wrap but personally I like the zip tie method which is nothing special just zip tying the cables at a few centimeters interval, like this:

After completing this step you should cut the ties short, of course. One more thing I did was fixing the bunch of extruder cable to the top plate to an unused hole like this:

You need to leave about 25 cm (10 inches) of cable between the fixing point and the extruder assembly so the hot end can reach the farthest point safely.

And that’s all about assembling the printer. The next post will be about how to calibrate the 101hero.

First of all, the 101hero team’s goal was to create “The World’s Most Affordable 3D Printer” as they said and not the the printer you should start your 3D printing adventures with and this can be seen in a few places. Nevertheless, the printer is functional and the prints can turn out surprisingly nice if one is willing to spend enough time and don’t mind the hassle but if you’re new to 3D printing and you want a printer that works out-of-the-box, this one is not the best choice.

I backed the IndieGoGo run and have received the printer in January, 2017. I am new to 3D printing this is the first one I bought and the first one I ever printed with, In these posts I will share my experiences, tips I find useful and caveats.
The printer comes in two versions: the CV (Consumer Version) and DV (Developer Version). The only difference is that the DV has a USB port and can be connected to a computer while the CV can only print from SD card (which is not part of the package).

Assembling the 101hero is pretty straightforward but before getting started check all the components that should be in the box:

• 3 pylon assembly – they are identical except for the length of the wires and labels, there should be an end-stop microswitch (to detect that the carriage reached the top of the pylon), two slide rods, a belt, a carriage (fixed to the belt, sliding on the two slide rods, fixed to effector rods) and a motor
• 1 controller box
• 1 SD card slot
• 2 plates – one for the base and one for the top
• 1 extruder assembly with effector – the “head”
• 1 bunch of filament – the printer comes with black or white, I bought the green separately
• 1 roll of painter’s tape – to help adhesion of the first layer printed
• 1 glass plate
• 3 paper clips – to fix the glass plate to the base
• 6 pcs M3x10 machine screws (PH1 head) – for the carriage-straw/strand and straw/strand-extruder assembly
• 18 pcs M4x12 self-tapping screws (PH1 head) – for pylon-base and pylon-top
• 2 pcs M3x12 self-tapping screws (PH1 head) – for door fixing on extruder assembly (identical to the ones used for end-stops in early editions)

In early editions the end-stops are hitting against M3x12 self-tapping screws with PH1 heads, I have read that later they switched to Hex Socket (aka Allen) types.

Before assembling the printer I would like to tell you some basics and recommend to do further checks.

When the printer is powered it looks for a file called “101hero” on the SD card and starts to execute it (i.e. printing, moving, etc.). The file is a G-code file and has no extension, it is just “101hero” (when saving the file using Notepad on Windows to prevent it appending “.txt” change the “File type” drop-down to “All files (*.*)” or put the file name between quotes).

The button on the controller box simply resets the printer that causes it to re-initialize and looking for the file and execute it.

The pylons are identical except for the cable lengths and the labels that are on them, that is you can have them swapped and even replaced from a different package, they will work. One important note however – the end-stops are used to detect when the carriages reach the top of the pylon so the motor-end-stop pairs need to be kept consistent, if they are not then the controller will keep moving the carriage until you turn the power off.

A big warning: the power is not only sourced from the power connector but from USB as well – that is as long as your printer is connected over USB it will power the board – regardless of the power switch! Yeah, if you slide the switch to OFF the board will still be powered, although the printer will not operate properly due to the low current but you can damage the printer if you short something.

One more big warning: the controller has no flyback diodes to protect itself from inductive spikes. Simply put: only move the carriage slowly by hand because the motor generates current that will be fed back to the controller. If the current is too high (due to moving fast) it might damage your controller. Although I have not tested it yet, it might also be fed back over USB damaging your computer as well.

The printer has basically no customer support. As of March, 2017 the 101hero team is still busy shipping packages and really unresponsive to support questions, however there is a really active group on Facebook, the 101 Hero (Unofficial) and a forum at 101user.com.

Now it is time to start connecting things, meet me in part two.

In June I was out for shooting again. I mounted the lens, found some bright spot but despite all attempts I could not focus it into one point. I was wondering about what could I possibly do wrong until… no, that couldn’t be, could it? Could that spot not be a star at all?

A few pictures and corrections later I was astonished by the fact that I could capture a planet (other than our own, of course) with my DSLR and a cheap lens. Of course, the image quality was bad but, that spot was Saturn.

These pictures were taken directly from the camera, they are only cropped and stitched into one file. A few days later I spotted Saturn again:

And then I checked out Venus…

… and then Jupiter.

And I will not tell you what this is in the last picture:

Actually it is not just the Sun but Venus as well from the Venus transit on 6th June 2012.

All images above were just cropped but not scaled, therefore the size of the bodies can be compared to each other.

And since then I have a new hobby… :)

I have visited earlier Pillow Fight Days and have shot a few photos there, but now I thought creating a video would be more fun.

When I am making a photo or video I like to in the middle of the crowd. Given this, my 8 mm fish-eye lens and a try at recording the videos at 50fps, this is the result:

When watching it in full-screen with the default HTML5 player the “720p50” option should come up. If that’s not the case try this link or try to download the original video and watch it with your dekstop/mobile player.

This was the first time I used Kdenlive for video editing and I immediately loved its straightforward interface and workflow, also its ease at rendering to file – it is a really powerful yet simple editor, I can only recommend it.

Personally I like the outcome and I think I will experiment further.

As I already have a PowerBank (basically a rechargeable battery with USB ports to charge a phone on the go) it looked like the ideal solution to use this as a power source. But then after a few searches of USB powered or at least 5 volts powered Wi-Fi access-points that meet my requirements to create a secured WDS (more on this in a later post) seemed to be too high. I found a few of them but they either were ridiculously priced or were given bad reviews.

At this time I realized that I had a good old Linksys WRT54GL router laying around. As this is a fairly common model with a quite favorable hackability factor, I did a quick research and found that this might be the best device for me.

The OpenWRT can be installed on this router and from software point of view it has all the features I want: WPA2 support, WDS (Wireless Distribution System – mesh, roughly), admin interface over HTTPS and is running Linux. Well, this last part was not strictly on the requirement list but hey, a device running Linux is always good to see (and opens up a lot of new opportunities).

On the hardware part I found out that the power adapter says it is supplying 12 volts and 0.5 amperes to the router. I found a few places where they told that just after the power jack there is a power regulator (a switching mode buck converter) that converts the voltage down to 3.3 volts. After taking apart I saw that this is indeed in place. Good thing about these buck converters that the input voltage range is usually wide, I found that it is about 3.7-16 volts, so it is more than suitable for my battery powered operation where the voltage drops over time when connected directly to the battery.

After finding out this wide range of input voltage I reconsidered my battery selection. Instead of using the PowerBank I decided to use a more universal solution – AA batteries. These can be found virtually anywhere and can be used for a lot more purposes when they are not used in this project. Also, hooking up 4 of them gives me 6 volts, but if I need more power then 8 of them is fine as well (giving 12 volts).

Here I used 2700 mAh batteries, 4 of them gives about 13 Wh (watt-hour) power (1.2 volts x 2.700 ampere-hour x 4), 8 of them gives about 26 Wh. If the router is consuming the maximum the original power adapter can provide (6 watts) then I can use the router for 2 hours straight from 4 batteries. But I really doubt that even a peak consumption ever reaches 6 watts.

Unfortunately it seems that the barrel plug power connector is not too stable (or maybe just the one I bought was of poor quality), when moving the router around a little a momentary loss of power (and therefore a reboot) was fairly common. So… as the router was already taken apart, I have soldered a pair of wires to the main board for the power and found a hole on the bottom of the case where I could bring the cable out without having to drill it.

It turned out to be working perfectly in the end.

In my next post I will write about making another WRT54GL mobile and setting up a WDS (Wireless Distribution System) to extend the range of the mobile Wi-Fi network.

It is a RaLink RT5350 (360 MHz MIPS24KEc) based board with 8 MB SPI Flash (although Vonger, the creator of the project upgraded to a 16 MB Flash for the Indiegogo batch just because he’s a nice guy) with two 10/100 Mbps Ethernet interfaces, one USB 2.0 interface, a bunch of serial interfaces (UART, I2C, I2S, PCM, JTAG) and over 20 GPIOs. All these broken out to standard 1.27 mm connectors while keeping the size at 25×25 mm!

There is a dock to provide some connectors: an Ethernet, a USB, a micro USB (for the power) and a micro SD card slot – this measues 25×25 mm as well, of course.

VoCore is running the OpenWRT firmware. It acts as an access point by default, running a network called “VoCore” (this is unfortunately an open network (i.e. no encryption set up) so changing security settings as soon as possible is recommended).

Vonger, the creator of the project is running a blog at vonger.cn, he was constantly updating it with the latest status of the VoCore development and production, both bad and good news. Thanks to this it seemed that we were witnessing all the little details, it felt like we (the backers) were parts of the whole process :)

The VoCore can be ordered from vocore.io/store. If you would like to have the VoCore main board only (with all the I/O breakouts but no connectors at all) it is USD 20, but if you’d like to have the Dock as well, plus a USB to TTL converter (note: this is missing from my pictures below!) to re-flash the firmware it is just USD 45.

I believe that the VoCore has a lot of potential thanks to its small size (25×25 mm) its low power consumption (0.6-1.2 Watts) its flexibility (20+ GPIOs and the whole bunch of connector break outs) and low cost.

I think this is a must have board for any geeks out there who have thought about hacking some hardware once in a while.

Oh, and one more thing: VoCore is open source, both software and hardware. All the schematics, circuits, firmware, 3D model for the shell/case, everything is available to download from the VoCore’s site.

All the other program code (i.e. the game itself and all the instructions to make it work) is located on the GameBoy Cartridge alongside with the textures, sounds and all the rest.

So the cartridge is basically the part of the system, the CPU reads it directly when it needs the next instruction and executes the code from there without copying it to its own memory.

The cartridges usually consist of the following components:

• ROM: Read Only Memory
  Stores the program code and all the resources needed for the game. It is programmed once and then never written again.
• MBC: Memory Bank Controller (optional)
  The Memory Bank Controller switches banks of the addressable memory space (32 kilobyte in total). The lower 16 kilobytes are always the first 16 kilobytes of the ROM while the upper 16 kilobytes are switchable between several blocks (i.e. part of the ROM or the RAM). There are several MBCs out there with their own purposes. This is a really long story, I’ll write more about this later.
• RAM: Random Access Memory (optional)
  This is a readable and writable storage, when a game has high-score list, saved game state, etc. it holds them information in the RAM. This is a standard SRAM so this is volatile – it needs constant power to keep the data alive this is why cartridges with RAM need batteries as well.
• Battery: (optional)
  The battery provides power for the RAM to keep the stored data alive.
• Connector:
  Last but not least, all the cartridges have a 32 pin connector that provides a connection between the console and the components of the cartridge.

The interface between the console and the cartridge consists of 32 pins (direction is written from console perspective, out: from console to cartridge, in: from cartridge to console):

• pin 1 [out]: VCC (+5 V)
  This is the supply pin for the cartridge and its components (i.e. the ROM, RAM, MBC).
• pin 2 [n/c]: not connected or CPU clock
  I have not seen this pin connected so far but some documents claims this is a direct output of the CPU clock. I ran a logic analyzer on this pin, it showed a high-low pattern alternating at 1.04 MHz (0.46 µs high followed by 0.50 µs low).
• pin 3 [out]: !write
  This is a signal line that is low when a write request is made to RAM. Otherwise this is high.
• pin 4 [out]: !read
  This is a signal line that is low when a read reqeust is made to ROM, MBC or RAM. Otherwise this is high.
• pin 5 [out]: !RAM select
  This line is low when the RAM or the MBC is being accessed. Both of them supports reads and writes. Otherwise this is high.
• pin 6..21 [out]: address selection bit 0..15
  These bits sets the address that is being written or read in the RAM, ROM, or in the MBC registers.
• pin 22-29 [out/in]: data in/out bit 0..7
  When the console wants to read from the ROM, RAM or MBC these pins are inputs, their values being set by the remote component. When the console wants to write data these are outputs, set to the data by the console.
• pin 30 [out]: !reset
  When the console reboots it sets this pin high after about 4-5 ms. This delay is probably introduced to give the clock a few cycles to stabilize.
• pin 31 [n/c]: not connected or analog audio input
  I have not seen this pin connected so far but some documents claims this is an analog audio input. (I think I’ve read something about a direct sound channel on the GameBoy CPU, but that might be a different story.)
• pin 32 [out]: GND
  The ground connection for the cartridge and all the components.

In the following posts I will write about the different cartridge and MBC types, and also will post schematics and circuits to build a few different homebrew GameBoy cartridges. Stay tuned.

One of the base concepts was to build a multiplayer game. It was challenging despite I’ve written some basic games, also mulitiplayer ones, I’ve never tried to write a real-time strategy game yet (not counting a quick POC). The synchronization of the game between players was the most interesting part of it, and therefore my favorite one. I was not intended to sync the whole game state (map, objects, internal states, etc.) continuously as this can easily become a huge set of data even for tiny games at high refresh rates so I decided I will come up with a better, more resource-friendly solution.

I’ve been thinking about the game: it has some mechanics, player events events and also their effects on the gameplay and concluded one could be separated from the other – of course it turned out that I was not the first who came up with this approach. So instead of syncing everything, I was thinking about synchronizing just the map in its starting state and a list of events.

All these events were always originated from the players (i.e. buy an item, toggle a switch) or from the game mechanics (i.e. an enemy is close enough to be fired on, an object loses health, objects collide). As the game on both ends are always identical the result of mechanics should be always identical too, therefore all I left to deal with were the events from the players.

So basically all I was doing is this: my game had so-called “ticks” that were the updates for the game internal states (this event was happening with a guaranteed interval, i.e. 15 times per seconds, or if late (due to lack of CPU resources or bad timing) all the affected ticks ran anyways to catch up), and when a player did an action (i.e. clicked a switch) I stored a record in a queue about it and a “process this on the nth tick” property. I’ve also sent this record over the network to the other player. This way I introduced a slight delay (this nth tick was always about 300 milliseconds in the future) but during this delay I sent the event to the other player and when the nth tick came both players made the change on their isolated environment in the exact same moment (speaking in game time) and therefore both ends have seen the same new state, the game is deterministic.

Later I’ve learned – in a Gaffer on Games post and in another one about network programming in Age of Empires (both are must reads!) – this is a so called simultaneous simulation approach with one big difference. The handling of the lag on the network. This was the worst thing during the development and tests of the game. As I’ve been syncing future events if the notification got stuck somewhere between the players it could easily come later than it must have been executed. As the tick it was intended to be run at was passed (and the other player have executed at that time) the game became out of sync. And this is bad, really bad. (I’ve made some efforts to handle this, but these turned the game code much more complicated, i.e. to be sure that important events happen on both sides I added them as an event to the queue (and sent that over the network), that’s why the little creatures pause for a while before exploding at the target markers. But this is bad, don’t do this. Keep reading for a simpler solution instead.)

In a real simultaneous simulation the players should be aware of the lag in the propagation of events so instead of just saying “here, run this on the nth tick” there needs to be a “I have nothing for you to run on the nth tick, carry on”. And if the other player does not specify what to do (or not to do) on the nth tick when it comes the simulation should pause while it is unsure about what to do. Which is not nice but far much better than becoming out of sync. Obviously working with 15 ticks per seconds and expecting the network to transfer the data for the next tick is nearly impossible (talking about transmitting over the internet as this would permit about a 66 milliseconds deadline on a client1-server-client2 route, also sending at least 15 events every second) and would result frequent pauses, and that would be really annoying. So instead of scheduling events for any tick of the game a rule should be applied to limit the ticks that can accept events. Saying events can be scheduled only for every 5th ticks reduces the count of transmitted packets dramatically, also extends the deadline to a more sane value.

Also, when a pause occurs due to network lag the rule of scheduling could change i.e. not every 5th but every 10th tick should accept events. And likewise when the network seems reliable for a long time this number could decrease.

But this topic will be covered in another post, first I’ve to try this approach. Although I think this should work nicely as games like Age of Empires and Warcraft used this method pretty reliably.

Recommended reading:

• Mark Terrano: 1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond
• Glenn Fiedler: What every programmer needs to know about game networking
• Glenn Fiedler’s Game Development Articles and Tutorials

So I had to come up with a short domain name – I already have one but that is under the .hu TLD – that gives away something about the purpose of itself… It was ridiculously challenging to find one that is not already taken in the .com/.org/.cc world. In the end I’ve chosen the r42.cc. The first part stands for revision42, something that is probably reached by a successful project (and of course, 42 is the answer), and the cc stands for creative community.

So if I put here some photos and designs in the future you should check it for this URL and open it to see some related content.

Lately I’ve started to play Munzee, a game that is the little brother of Geocaching in my point of view. It is working by the same idea, you’ve given the coordinates and you have to find the hidden treasure. Although here is no treasure at all, just a QR code stuck to a surface – usually at the back of (traffic) signs and bottom of traffic lamps. So finding it is not that great challenge after a while…

The idea is the following: what if we stuck the QR codes on non fixed objects, but moving ones? I.e. you could stick it on a train (somewhere hidden, not on a window or seat, so where you could only find it if you’re looking for it). It would be more challenging, and also you could track where the object travels (could be illustrated on a “spotted here” map) also you could meet something you met earlier at a different place.

I think this could add another layer of fun to the game.