Silviu Tantos (razius)

Auto-refreshing Google Chrome on file changes.

2014-01-19T10:58:00+01:00

I am known for being an lazy ass, one of the ways of coping with this horrible disease is automating any repetitive task that takes more that 5 minutes of work. So, in my free time, when I’m not dreaming of eating cake on a magical dessert island, I have the habit of bashing away at my keyboard to write shell scripts that ease my life.

During the process of writing a HTML presentation for my school and while trying to find ways to procrastinate and avoid it, I head this little voice in my head: “Hey, psstt, you, wouldn’t it be neat if you didn’t have to spend those 5 seconds refreshing the browser window each time you save that file?”.

After consulting my trusty “Is it work the time chart?” and seeing that I have lots of free time, with the weekend ahead of me, I’ve decided to listen to my little procrastination demon and dive in to see how I can shave those 5 seconds off.

It turns out it’s pretty easy to do, there are two nice system utilities on Linux that you can easily glue together by writing a bash script.

inotifywait

inotifywait is a command-line utility that uses the inotify Linux kernel API which provides a mechanism for monitoring filesystem events. The API can be used to monitor individual files, or to monitor directories.

When a directory is monitored, inotify will return events for the directory itself, and for files inside the directory. Events are fired, for example, for file and directory creation, deletion, access, modification, etc (for a full list, read the manual).

inotifywait, as the name suggests, will listen and wait for an event returning a code if the events for which we are listening to are triggered.

xdotool

xdotool is a command-line utility that lets you automate keyboard input and mouse activity, move and resize windows, etc. It basically lets you control your GUI application.

TuxRadar has a really nice tutorial about it if you want to find out more.

Tying it all together

Ubuntu doesn’t come with inotifywait and xdotool installed by default, but you can install them using apt:

1	apt-get install -y inotify-tools xdotool

Now for the bash script.

#!/bin/bash
#
# Watches the folder or files passed as arguments to the script and when it
# detects a change it automatically refreshes the current selected Chrome tab or
# window.
#
# Usage:
# ./chrome-refresh.sh /folder/to/watch /some/folder/file_to_watch.html

TIME_FORMAT='%F %H:%M'
OUTPUT_FORMAT='%T Event(s): %e fired for file: %w. Refreshing.'

while inotifywait -q -r --timefmt "${TIME_FORMAT}" --format "${OUTPUT_FORMAT}" "$@"; do
    CHROME_WINDOW_ID=$(xdotool search --onlyvisible --class google-chrome | head -1)
    xdotool key --window $CHROME_WINDOW_ID 'CTRL+r'
done

It waits for an event, when detected it will search for the current Google Chrome window or tab and send the refresh key shortcut (CTRL+r) to it then loop again in the listening state.

I’ve also made a gist for easy reference. I’ll probably update that more when I’ll want to add new features.

Note: Yes, I know about LiveReload but I find it way to complicated for my needs and it’s also not available on Linux (my OS of choice). Plus, in the process I learned something new and I will definitely be using that knowledge in the future.

Ditching Wordpress and becoming one of the cool kids

2013-12-21T10:58:00+01:00

I’ve been a Wordpress user and developer for a long long time but lately this nagging idea kept crawling in the back of my head, why not switch my website to a statically generated one? Well, a 3 hour train journey and a lot of boredom finally pushed me to just make the switch and to write a short post about it.

What is a statically generated website?

Compared to dynamic web pages, where the web page is rendered by server-side logic and usually require a database, with statically generated websites you feed files written in a markup language like Markdown, reStructuredText, Textile, etc. to a static site generator which spits them out as a static website that is ready for deployment.

They are useful when the content doesn’t vary based on parameters provided by a user, like in the case of a personal web page where the content is delivered to the user exactly as it was stored.

Why bother switching?

reStructuredText

I could write my articles as simple flat files using the reStructuredText format, goodbye storing of articles in a database. Having the articles as simple text files would allow me to easily edit them using my editor of choice, sublime text, so no more editing of text in a form on a web page, copy/pasting code snippets in it and painfully adjusting the indentation. Just take a look at the source of this article and see how easy it is to embed different code snippets, it feels just like editing a source file. I would also have syntax highlighting powered by the excellent pygments syntax highlighter.

print("Hello, World!")

#include <stdio.h>

int main(void) {
    printf("Hello, world!\n");
    return 0;
}

public class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello, world!");
    }
}

package main

import "fmt"

func main() {
    fmt.Println("Hello, world!")
}

Plus, having the articles as flat files would allow me to keep the entire website in a git repository really easy.

Faster page loads

Because there is no need to build the web page each time upon serving it, page load would be faster. I made a test of a page load before and after the switch.

Page load speed for Wordpress

Page load speed for static website

Not bad, a whole second. Yes, it’s not really a fair comparison, I also changed the theme to a much simpler one which is a big boost too, but still.

Github pages

I could host my page for free using Github pages, which is a free hosting for static pages offered by Github, using git repositories for file storing. No, I’m not killing the little curious sysadmin in me, it’s just that for my humble page administration is starting to feel like a hassle.

The geek factor

In the end, why not? I keep this website to toy around with it and this would allow me to play around more with Python, Jinja2 and reStructuredText plus I could test a new flow for writing articles.

Making the switch

I picked Pelican over the two most popular static web site generators, Jekyll and Octopress because being a python developer, and Pelican being python based, it would feel a bit more natural than Jekyll or Octopress which are Ruby based.

If you think something else would suit you more, you can take a look at a complete list of static website generators at staticsitegenerators.net

Installing Pelican, it’s as simple as running:

1	$ pip install pelican

Next, kickstart a new project. Do do this run pelican-quickstart, it will ask us a few questions about you site to generate a config file named pelicanconf.py and create a project skeleton with some helper scripts inside.

1	$ pelican-quickstart

You can edit you pelicanconf.py to further tune your configuration, take a look at the manual for the available options or you can check out my pelicanconf.py for some inspiration.

All the content goes into the content folder and I like to keep all my articles and posts in separate folders by setting the following in pelicanconf.py:

PATH = 'content'
PAGE_DIR = 'pages'
ARTICLE_DIR = 'articles'

You can export your articles from Wordpress into an XML file by going to Tool -> Export and then run pelican-import to generate the appropriate text files into the content folder.

1	$ pelican-import --wpfile -o content/ wordpress-export.xml

You can preview your files by running make devserver, this will start a webserver that serves your build html files under http://localhost:8000/ and a process that watches the content folder for file changes and rebuilds the served html files.

1	make devserver

Hosting on Github

Instead of using Github pages, where you are required to have your html pages under the project root and because pelican puts them under the output folder, I chose to use project pages. With project pages you keep the html files in a separate git branch called gh-pages and Github will publish those pages for you under GITHUB_USERNAME.github.com/GITHUB_PROJECTNAME. from that branch. Ex. my web address would be http://razius.github.com/razius.com, don’t worry, you can use your custom domain too.

Notice the difference between the master branch and the gh-pages branch.

Luckily, you don’t have to maintain this branch manually, there’s a script called ghp-import which will manage it for you. It copies a directory to the gh-pages branch, that is the site’s document root.

1	pip install ghp-import

Now with ghp-import installed you can publish your project pages by simply running make github command which will build the html pages for publishing under the output/ folder, call ghp-import to copy the files under that folder to the gh-pages branch and do a git push to push the new changes to Github.

1	make github

You can also use your own custom domain, I keep my CNAME file in a static folder called files and I’ve added the file’s path to EXTRA_PATH_METADATA in pelicanconf.py so that pelican copies it each time on build.

STATIC_PATHS = ['files']
EXTRA_PATH_METADATA = {
    'files/CNAME': {'path': 'CNAME'},
}

PS: Yes, the title is a bit ironic.

Building a messaging framework for Django

2013-11-17T20:58:00+01:00

As you might have noticed, one of the key usability features that we were missing on Iconfinder was the ability to display so called “flash messages.” Flash messages are one-time notification messages shown to the user after processing a form or handling other types of user input, so the user knows if things went as expected. Changing your password without being told Your password was updated. can be a little confusing, so it was long overdue that we did something about something that obvious.

Example of flash messages.

Note: This is a cross-post of an article that I originally wrote on my company’s blog a while back.

Context is king

Most popular web frameworks already feature an easy drop-in way of showing these flash messages. Iconfinder is a heavy Django shop, and of course Django comes with its own messaging framework built in for just this purpose. But, like most other drop-in solutions, it rarely fits the bill of a larger site.

The built in messaging framework has a problem in that it’s not contextually bound - in other words, it’s not associated with any one form or page. This means, that messages from a specific form could under odd circumstances be shown in the completely wrong place, which is arguably more confusing than not being shown any message at all.

Furthermore, this also means that we can’t create “out of band” messages to be shown from asynchronous jobs, such as when we’re processing icon data, as that would just show up somewhere where the user might not expect it. What was left for us to do then, was to roll our own solution that would make it possible for us to do all the things we wanted.

Now, contextual is a pretty ambiguous term (which is why academics love using it) so we of course had to pick something more “tangible.” Django sites basically contain a bunch of views, which have the responsibility of displaying a specific kind of page, so as a starting place, we decided to bind messages to a specific view. As each user is also identified by a session, it made sense to start out with these two things defining the context.

Transient messages in Redis

Given the contextual nature of the messages, it is very likely that a user might have more than just one message waiting for him - especially if he or she is doing a lot of stuff at once. Therefore, the standard solution of using cookies to store the messages made no sense, as it would mean that a lot of pointless data was sent over the wire all the time. So, we have to store the data somewhere. Our primary PostgreSQL database is the first option, that comes to mind, but given that messages are shown once, that would mean that we’d be inserting and deleting a lot of data - something a relational database is usually not made for. Luckily, there are options out there that are basically made for this kind of workload, like for example our favorite key-value store, Redis.

Redis is essentially in-memory, but we do snapshots every minute and run a master/slave pair to make sure we don’t loose too much data in case of failure, but the loss of a couple of messages won’t be the end of the world, which makes it the perfect tool for the job.

The beauty of Redis is, that we can get away with a simple key structure: the base key is the application name, followed by the session key and finally by a view identifier. This makes insertion and retrieval of values very easy. The keys are also set to expire after a default period of inactivity, just to make sure we don’t have any orphaned messages lying around a week after you’ve changed your password.

messages:user-session-key:view-identifier

When a new message needs to be shown, we simply serialize it along with a bit of meta data and store it in a sorted set with the message’s timestamp as a score. This way the messages can be easily sorted by time or be purged using a garbage collection scheme if we run out of space.

The exposed API

One of the keys to keeping a large code base maintainable is to expose functionality through a stable, consistent API. The contextual messaging system adheres to this principle by exposing to simle methods: one to add a message and the other one to retrieve all the messages for a session in a view. If you’re a Pythonista, the following should look familiar to you:

from messaging.models import Message

Message.add(session_key = 'current-users-session-key',
            text = 'This *is* the message you are looking for.',
            title = 'Hello',
            status = Message.INFO,
            autoclose = True,
            view = my_view)

messages = Message.get(session_key = 'current-users-session-key',
                       view = my_view)

A perfect case for a mixin

Django provides class based views which makes it easy to structure views and expose common functionality by harnessing inheritance and mixins. Because in most of the cases we would need to add and retrieve messages for the current view, to make it even easier we created a mixin that would provide some abstracted functionality to our view classes. The retrieval of messages for the current view is handled by the mixin so messages are ready to be consumed in the templates without having to call Message.get() in each view.

Adding a message is also a lot easier as we can figure out the view name and session from the view:

class DashboardView(MessageViewMixin, TemplateView):
    def form_valid(self, form):
        self.add_message(text = 'Your settings have been changed successfully.',
                         status = Message.SUCCESS)

Doesn’t get much easier than that, huh?

Don’t be afraid to write code

If there is a functionality built into the framework that you need but it doesn’t fit you use case, don’t be afraid to write your own solution. In most cases you only need a minimum viable product that fits your needs really well rather than a full fledged library that can send a rocket to the moon.

Launching services after Vagrant mount

2013-11-10T14:40:00+01:00

I recently had to set up a Vagrant project where all the configuration files for services like nginx, supervisor, mysql where symlinks pointing to files residing on the Vagrant synced folder. This turned out to be a bit of a problem, when the Vagrant box boots, one of the last things it does is mount the synced folder, so those services would fail to start because of missing configuration files.

To fix the problem you need to ensure that all services are started after Vagrant had the chance to mount the synced /vagrant/ folder and to do this there are a couple of ways depending on the type of init script that the service has.

Upstart

There is an upstart event that Vagrant emits each time it mounts a synced folder that is called vagrant-mounted so we can modify the upstart configuration file for services that depend on the Vagrant synced folder to listen and start after the vagrant-mounted event is emitted.

# nginx
description "nginx http daemon"
author "Silviu Tantos "

# Listen and start after the vagrant-mounted event
start on vagrant-mounted
stop on runlevel [!2345]

env DAEMON=/usr/sbin/nginx
env PID=/var/run/nginx.pid

expect fork
respawn
respawn limit 10 5

pre-start script
        $DAEMON -t
        if [ $? -ne 0 ]
                then exit $?
        fi
end script

exec $DAEMON

System-V

If you are not running Ubuntu as the guest OS or you don’t want to convert your init script to upstart, there’s still hope, albeit a bit more complicated. Underneath Vagrant uses VirtualBox’s shared folder for it’s synced folder so we can hook-up to udev events and trigger a script execution after the VirtualBox mount event has fired, this of course will also works for VirtualBox’s shared folder not just for Vagrant’s synced folders.

Finding out the udev event

This step is not needed as I already wrote the udev rule, it’s just to show how I came up with it. If you’re not interested in it just skip to writing the udev rule.

Fire up a terminal and listen to udev events:

root@vagrant:~# udevadm monitor --property --kernel
monitor will print the received events for:
KERNEL - the kernel uevent

Fire up another terminal and trigger a mount:

root@vagrant:~# mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant` /vagrant /vagrant

Now we should see the event in the first terminal window:

root@vagrant:~# udevadm monitor --property --kernel
monitor will print the received events for:
KERNEL - the kernel uevent

KERNEL[4234.545610] add      /devices/virtual/bdi/vboxsf-13 (bdi)
ACTION=add
DEVPATH=/devices/virtual/bdi/vboxsf-13
SEQNUM=1407
SUBSYSTEM=bdi
UDEV_LOG=3

Writing the udev rule

We can see that the subsystem is bdi and the action is add so we can write a udev rule for the event. I use /etc/udev/rules.d/50-vagrant-mount.rules as the file name and path.

# Start on mount
SUBSYSTEM=="bdi",ACTION=="add",RUN+="/usr/bin/screen -m -d bash -c 'sleep 5; /etc/init.d/nginx start'"
# Stop on unmount
SUBSYSTEM=="bdi",ACTION=="remove",RUN+="/usr/bin/screen -m -d bash -c 'sleep 5; /etc/init.d/nginx stop'"

This will run each time a VirtualBox shared folder is mounted. It spawns a screen session (of course you need to have the screen package installed) to prevent the command from being killed by systemd when parent udev exits, sleeps for 5 seconds to make sure Vagrant had the chance to mount the synced folder and exits successfully and finally start the service.

Don’t forget to type in the full paths otherwise udev won’t be able to find the command. For more information consult the udev manual.

Move fast and break things

2013-10-14T19:14:00+02:00

Move fast and break things is one of the founding mantras of Facebook and I think it defines a company culture that can apply to almost any technology startup. It is a culture who’s principles embody, promote and encourage communication, constant and fast iterations, tight feedback loop between the implementation and the design, short code-to-deployment cycles, experimentation and efficiency.

These are also just a few of the reasons why I chose to be a part of the Iconfinder team and to do my internship at a startup company instead of a traditional one. I have the perfect environment for rapid professional growth, I can acquire new skills and sharpen my old ones at a much faster pace, I feel challenged everyday and I have the pleasure of working with the newest technologies.

In this move fast and break things culture the break things, as I’ve come to find out, is not optional. To continue to improve you have to make mistakes from which you have to learn and my recent blunder is one of those mistakes. During the development of the new comment section a serious bug slipped through making it possible for attackers to hijack cookies from users. The bug was discovered and fixed under 2 hours and didn’t have any major repercussions apart from the inconvenience of clearing the session storage and having all users log in again.

I’ve learned a lot this month, some lessons where learned the hard way, from mistakes, but it’s been a fun ride so far and I can’t wait to see what the future will bring.

Some geeky statistics:

Git commits: 101
Git code line additions: 4.674 ++
Git code line deletions: 1.430 —
Coffee cups consumed: 82
Number of times I caused chaos: 1
Minutes wishing I was somewhere else: 0.5

Some of tasks accomplished so far:

Created Vagrant machine to use for local development.
Implemented a PayPal cash out option.
Designed and implemented a new comment section, replacing Disqus.
Created a Python script to generate a image sitemap.
Integrated internal notifications into Hipchat, our internal communication application.
Did various CSS and Javascript fixes.

Getting the IP address of a specific interface in puppet using facter

2013-10-06T22:13:00+02:00

One of the problems that I find annoying in puppet is that you can’t easily obtain the IP address of a specific network interface.

Let’s say that you want to define a class or type to which you wish to pass a interface name and use it’s IP address as a variable.

class dns ($interface) {
    $address = [ insert magic here ]

    $message = "So you want to know the IP address for ${interface} huh?"
    $message += "Well, it's ${address}"
    notify {  "${message}": }
}

class { 'dns': interface => 'eth0' }

We could use facters $::ipaddress, but what happens if the network interface is not our main network interface? What if we want to use more than one network interface? There’s $::ipaddress_eth0, but how do we construct or reference a variable in a dynamic way (construct a variable based on another variable) ?

Luckily there’s a nifty trick involving inline templates and lookupvar:

class dns ($interface) {
    $address = inline_template("<%= scope.lookupvar('::ipaddress_${interface}') -%>")

    $message = "So you want to know the IP address for ${interface} huh?"
    $message += "Well, it's ${address}"
    notify {  "${message}": }
}

class { 'dns': interface => 'eth0' }

Vagrant and SSH agent forwarding

2013-09-13T08:29:00+02:00

I haven’t even started yet and I can already hear you muttering over there, “What the hell is this Vagrant thing and why should I care?”. Well, Vagrant is a wrapper around VirtualBox, the virtualization software, that can create homogeneous development environments automatically without any effort from the developer. This means that we have a consistent development environment across out team with the same OS version, same package versions, same database, same settings. No more But it worked on MY machine excuses.

Lets say that inside this virtual machine you need to use your SSH key, maybe the key for your Github account to access your private or public git repositories or maybe the key to connect to a remote server. That can be a problem, you don’t want to distribute your SSH keys with the Vagrant box. Each SSH key should be tied to an individual developer account so to prevent SSH key sharing you would need to either distribute each SSH key with a new Vagrant box or copy it during provisioning. That’s not really homogeneous, you will end up with a Vagrant box for each developer or an inconvenient way of providing the SSH key depending on the developer’s OS.

Enter SSH agent forwarding. With SSH agent forwarding we can use the SSH key from our local machine inside the Vagrant box.

To enable agent forwarding for all ssh connections inside your Vagrant box you need to set the following in your Vagrant file inside the config section (Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|):

[...]
    config.ssh.private_key_path = "~/.ssh/id_rsa"
    config.ssh.forward_agent = true
[...]

Due to a Vagrant bug SSH Agent Forwarding not available during provisioning (see issue), to work around that we need to create a file in /etc/sudoers.d/ with the following contents:

Defaults env_keep += "SSH_AUTH_SOCK"

To create it automatically during provisioning we can add the following to our Vagrant file:

config.vm.provision :shell do |shell|
    shell.inline = "touch $1 && chmod 0440 $1 && echo $2 > $1"
    shell.args = %q{/etc/sudoers.d/root_ssh_agent "Defaults env_keep += \"SSH_AUTH_SOCK\""}
end

or in the shell script that you’re using for provisioning:

SSH_FIX_FILE="/etc/sudoers.d/root_ssh_agent"
if [ ! -f  $SSH_FIX_FILE ]
    then
    echo "Defaults env_keep += \"SSH_AUTH_SOCK\"" > $SSH_FIX_FILE
    chmod 0440 $SSH_FIX_FILE
fi

Copenhagen and my r’bad_luck’

2013-09-08T11:46:00+02:00

To tell you the story of my string of bad luck and my first week of internship in Copenhagen, I’ll first have to start with my last week. It was Saturday, the day before I had to leave for Copenhagen, I was there minding my own business, when my laptop suddenly decided that he is thirsty. Now, my laptop being a laptop and having no hands, couldn’t achieve this feat by itself.

Enter me and my clumsy hands. In my infinite star gazing mode, I knock down a glass of water down on my laptop, helping him quenching his thirst and in the same time helping him commit suicide.

Enter me in panic mode. There I was on a Saturday evening without a laptop and I would have to start work on Monday. Luckily I managed to borrow an older laptop from somebody I know but only on Sunday evening, leaving me with little time to swap it’s hard-drive with the hard-drive from my old laptop and to try to boot my old Linux installation from it. If I managed to get it to boot it I would at least have all my files and could work properly. But the laptop that I borrowed, being as stubborn as a wild stallion being caught and trained, decided to all of a sudden defy me and refuse to boot my Linux installation properly.

After a couple of hours of intense training I tamed him and manage to boot my Linux installation successfully. I was finally presented with a beautiful login screen with my name on it.

Fast forward Friday evening. The end of my first internship week. After an office party I head down to Copenhagen Central Station at around 2 AM to take the train back to Horsens to sleep at a friend for the weekend and to take my things back to Copenhagen. Having still a couple of hours till the train arrives I sit on a bench and put my backpack in between my legs. After a couple of minutes I start to doze off. Upon waking up I realize that my backpack with all my things is gone, somebody stole it in the meantime. I had my laptop with all my data, phone, id, credit card and my ticket back to Horsens in it.

Now, at this point, I have to mention that I also didn’t have a place to stay in Copenhagen. When leaving for Copenhagen I didn’t find anything to rent so I just lived in a hostel for the past 5 days with the plan of returning to Horsens for the weekend. I packed just enough clothes for the week and my intention was to go back to Horsens for the weekend and return on Sunday when the apartment that I finally manage to rent would be available.

So exit scene for now, it’s Friday night (3AM), I’m standing in the middle of Copenhagen Central Station, with my hands on top of my head, philosophizing on universe’s strange idea of humor. I am without a place to stay, without a way to contact somebody, without knowing anybody or anyplace and without a way to at least get back to Horsens. Everything I have on me are a couple of beer caps.

About stuff

2013-07-31T19:53:00+02:00

I moved to Denmark, from my home country Romania, exactly two years ago to the day with just a small suitcase and a backpack. In them I had a hand-full of clothes, a few personal belongings and my laptop.

During the past couple of years things started to accumulate, I started buying new things, some out of necessity others because I wanted them.

My luggage when moving to Horsens, Denmark

But fortunately because as a student I had limited space and resources I was forced in a certain mindset, making me think twice before buying something. For example when I needed something, I tried to buy the cheapest thing that would serve it’s purpose, I bought a computer chair and a table for 5$, a bed for 50$ and a lamp for 2$. Being cheap, when moving out instead of taking them with me I could just leave them behind,less thing to move, less hassle and effort spent on moving them out.

Being accepted for an internship in Copenhagen and facing the possibility of living in a hostel for an undetermined period of time made me decide to see how much I could slim down my list of possessions again. My objective was simple, everything that I own should fit in my backpack and my travel bag (or less if possible).

What did I end up with?

Everything I currently own

Clothes, nothing spectacular here.
Two pairs of shoes, I’ve always owned no more than 2-3 pair, never felt like I needed more.
Thinkpad T500 laptop + mouse.
Kindle which I find incredibly useful, it allows me to carry tons of books around with little physical space requirements. I would love to own physical books, but it feels like they would be too much of a hassle right now.
External hard-drive to back-up the data from my laptop.
HTC One S phone, it also acts as my camera, alarm clock and whatever other uses I can think of.
iPod Touch, I bought it to develop and test a iOS application. I’m considering selling it and buying a proper MP3 player.
Longboard, I’ve bought this one last year and I consider it incredibly fun.
Ogio backpack.

Not shown in the picture:

Charger and a docking station for the laptop
A pair of headphones,a usb cable for my phone/iPod/Kindle and one for my external hard-drive
A computer that I keep back home in Romania which is acting as an off-site back-up server for my files, every night the content of my laptop is saved on to it. In this way if anything happens to my laptop or the data on it, I would still have what’s most important, my files and data
A shoe box filled with personal memories.

Now I’m gonna leave you with a great comedy sketch by George Carlin and an awesome website on the subject.

Generating password hashes with puppet

2013-07-08T10:25:00+02:00

Puppet expects the user’s password to be encrypted in the format the local system expects, for most modern Unix-like systems (Linux, *BSD, Solaris, etc.) this format is a salted SHA1 password hash.

To generate a password hash to use with puppet manifest files you can use the mkpasswd utility (it’s available in the whois package):

$ mkpasswd -m sha-512
Password:
$6$qfPDlAej83p$cj2nc1NjbKjhL42Mo/3Uia4NqD4dIB3ouVeI/tSG92UqH5cMKOA/ihjmxAuRtKHzGED0EHmdM0iNxa/662NW//

You can then use the password hash in a puppet manifest file:

user { 'root':
    ensure   => 'present',
    password => '$6$qfPDlAej83p$cj2nc1NjbKjhL42Mo/3Uia4NqD4dIB3ouVeI/tSG92UqH5cMKOA/ihjmxAuRtKHzGED0EHmdM0iNxa/662NW//',
}

Don’t forget to put the password in quotes so that puppet does not interpret it as a variable if it contains the dollar sign ($).

If you want the passwords to be stored in plain text in the puppet manifest you can use puppet’s generate function to call mkpassword and return the generated the hash version of the password:

$password = 'your_plain_text_password'
user { 'root':
    ensure   => 'present',
    password => generate('/bin/sh', '-c', "mkpasswd -m sha-512 ${password} | tr -d '\n'"),
}

References:

Testing print stylesheets using Google Chrome

2013-05-14T21:46:00+02:00

Testing CSS print media queries is really difficult but a new featured was introduced in Google Chrome in December that allows you to apply different CSS media types like handheld, print, screen and so on.

Open Developer Tools in Google Chrome by clicking the menu icon at the top-right of your browser window, then select Tools → Developer tools.
Open up the Overrides menu in the Developer Tools by clicking the gear in the bottom right corner:

Enable “Emulate CSS media” and select the “print” media type option from the drop-down box:

You can find more information and nifty tricks in the official documentation.

Programing the STK600 AVR board on Linux using C

2012-09-27T10:51:00+02:00

Setting up the developing enviroment

Note: I’m assuming you have a .deb based distribution and a AVR STK600 board with a ATmega2560 microcontroller that you are programming using an USB cable.

Installing the software

We need to install the following programs and their dependencies:

gcc-avr: The GNU C compiler (cross compiler for avr)
gdb-avr: The GNU Debugger for avr
avr-libc: Standard C library for Atmel AVR development
avrdude: Tool to download the compiled code and data to the Atmel AVR microcontroller

Install them by running the following command:

$ sudo apt-get install avrdude avrdude-doc gcc-avr binutils-avr avr-libc simulavr gdb-avr

Setting up the permissions

In order to download the firmware on the device as a regular user we need to setup some udev rules for device permissions otherwise we get the following errors:

avrdude: usb_open(): cannot read serial number...
avrdude: usb_open(): cannot read product name...
avrdude: usbdev_open(): error setting configuration 1...
avrdude: usbdev_open(): did not find any USB device...

Create a new file under /etc/udev/rules.d/ named 41-atmega.rules with the following content:

SUBSYSTEM=="usb", ATTR{idVendor}=="03eb", ATTR{idProduct}=="product_id", OWNER="my_username", GROUP="my_group", MODE="0666"

Where you need to replace product_id, my_username and my_group with your own values.

my_username is the username that you login to the system, you can find it by running the following command:

$ id -gn

my_group is the primary group of your username, you can find it by running the following command:

$ id -un

product_id is the USB Product ID assigned by the manufacturer, you can find it by running the following command:

$ lsusb |grep Atmel | awk '{ print $6 }' | awk 'BEGIN { FS = ":" } ; { print $2 }'

Reload udev to apply our rules:

1	$ sudo /etc/init.d/udev reload

Finally, be sure to disconnect and reconnect the AVR board for the new rule to be applied.

Creating a new project

Create a new file named test.c with the following content:

/* Sample C program to turn on all leds on the AVR */

#include <avr/io.h>

int main(void) {
    DDRA = 0xFF;
    PORTA = ~0xFF;
    return 0;
}

Create a new file named Makefile with the following content:

CC=`which avr-gcc`
CFLAGS=-g -Os -Wall -mcall-prologues -std=c99 -mmcu=atmega2560
OBJ2HEX=`which avr-objcopy`
AVRD=`which avrdude`
# Modify TARGET to match your C static without the .c extension (it's test.c in this case)
TARGET=test

program : $(TARGET).hex
    $(AVRD) -p atmega2560 -c stk600 -P usb -v -v -U flash:w:$(TARGET).hex
%.obj : %.o
    $(CC) $(CFLAGS) $< -o $@

%.hex : %.obj
    $(OBJ2HEX) -R .eeprom -O ihex $< $@

clean :
    rm -f *.hex *.obj *.o

Now all you have to do is run make in the project folder to build the binary and upload it to the AVR Board:

$ make

Usefull linux commands/tricks

2012-05-14T23:00:00+02:00

These are some sort of notes to myself, this list will be constantly updated.

Sort directories by size (human readable size):

# du --max-depth=1 -h |sort -hr

Set tabsize to 4 in vim:

:set tabstop=4

Show failed login attempts on your machine:

1	# lastb -F

Print the most recent login of all users and of a given user:

# lastlog
lastlog -u USERNAME

$ curl -d "param1=value1&param2=value2" http://example.com/
$ curl -X POST -d @static http://example.com/

Return current SSID

1	iwconfig wlan0 \| awk -F':' '/ ESSID/ {print $2}' \| tr -d '"'

Force a reboot:

# echo 1 > /proc/sys/kernel/sysrq
# echo b > /proc/sysrq-trigger

Force a shutdown:

# echo 1 > /proc/sys/kernel/sysrq
# echo o > /proc/sysrq-trigger

find . -maxdepth 1 -type d -exec basename {} \;|xargs -L 1 -I '{}' chown {}:{} {}

Print text at LINE_NUMBER in a huge file

1	awk 'NR >= LINE_NUMBER && NR <= LINE_NUMBER' data.txt

How to fix GC overhead limit exceeded in Eclipse

2012-03-27T10:13:00+02:00

If you’re performing memory intensive operations such as building workspace on big projects, Eclipse will throw this error when it runs out of memory:

An internal error occurred during: “Building workspace”. GC overhead limit exceeded

In order to fix it, you need to increase the memory allocation for Eclipse. To do this, open eclipse.ini and increase the Xms (heap’s start memory) and Xmx (heap’s maximum memory) values to a value that you think is reasonable with your system and projects, for example:

-Xms512m
-Xmx1024m

Backup and restore installed packages on Debian/Ubuntu

2012-03-05T10:24:00+01:00

If you want to reinstall a server or a desktop and want to install the same packages after reinstalling the operating system you can easily do that with dpkg.

First we need to backup the list of packages to a file and after reinstalling the operating system, in this example will also backup the /etc/apt/sources.list file and the /etc/apt/sources.list.d directory as they contain the list of repositories from which you installed packages.

Backuping up the list of installed package and the repository list into a tar archive

# mkdir ~/packages; cp -R /etc/apt/sources.list /etc/apt/sources.list.d/ ~/packages/; dpkg --get-selections > ~/packages/package.list; tar cPvzf ~/packages-`date +%Y-%m-%d`.tar.gz ~/packages/; rm -rf ~/packages/

Now you should have a packages-DATE.tar.gz archive under your root user’s home folder (/root/) containing the list of packages and repositories.

Restore the list of installed package and the repository list from the tar archive:

After reinstalling upload the packages-DATE.tar.gz archive to the root’s user’s home directory (/root/) and run the following command:

# mkdir ~/packages; tar xPvzf ~/packages-*.tar.gz; cp ~/packages/sources.list /etc/apt/sources.list; cp ~/packages/sources.list.d/* /etc/apt/sources.list.d/;apt-get update; dpkg --set-selections < ~/packages/package.list; apt-get install --yes dselect; dselect update; apt-get dselect-upgrade

This will install all of the packages which you had at the time of the backup.

Lock screen when closing laptop lid in Ubuntu 11.10

2012-02-13T13:03:00+01:00

An useful thing to do is to have lock the screen when you close the laptop lid so you don’t have to wait for the timeout but in Ubuntu 11.10 this feature was disabled.

Note: This bug has been fixed in Ubuntu 12.04 Precise Pangolin.

To have your screen locked when you close your lid under Ubuntu 11.10 you need to edit /etc/acpi/lid.sh and just after the line

1	. /usr/share/acpi-support/screenblank

add the following line:

1	su $user -c '/usr/bin/gnome-screensaver-command -l'

So the result should look something like:

[...]
    for x in /tmp/.X11-unix/*; do
        displaynum=`echo $x | sed s#/tmp/.X11-unix/X##`
        getXuser;
        if [ x"$XAUTHORITY" != x"" ]; then
            export DISPLAY=":$displaynum"
            . /usr/share/acpi-support/screenblank
            su $user -c '/usr/bin/gnome-screensaver-command -l'
        fi
    done
[...]

Password protecting folders with nginx

2012-02-13T12:20:00+01:00

If you need to password protect a folder using nginx here’s how. In this example we’re gonna password protect the /admin folder, trying to access the admin directory will prompt the user with a basic authentication dialog, and will be challenged against the created password file.

Edit your site’s configuration file and add the following lines inside the server-block configuration:

location /admin {
   auth_basic            "Restricted";
   auth_basic_user_file  /etc/nginx/htpasswd;
}

Next we need to create a password file, the safest place to store a password file is outside of the web-accessible location so we’re gonna place it in /etc/nginx.

# cd /etc/nginx/
# wget http://trac.edgewall.org/export/10975/trunk/contrib/htpasswd.py
# chmod 755 htpasswd.py
# ./htpasswd.py -c -b htpasswd USERNAME PASSWORD

Don’t forget to reload nginx so that the configuration changes take effect:

1	# /etc/init.d/nginx reload

Installing and adding an external Tomcat server in Netbeans

2012-01-28T05:18:00+01:00

Netbeans no longer comes with Tomcat bundled so if you want to deploy your application to an Tomcat server you need to manually register an external server.

Note: This article is old no longer and this is not true anymore, Tomcat is now bundled with Netbeans so the steps in this tutorial are not needed anymore. You can download and install the bundle from the official page.

Pre-requisites:

I’m assuming you are using at least Ubuntu Linux 11.10 and you already have installed the the latest Netbeans 7.1 with the Java Web Applications plugin installed.

I also assume that you want to install and configure Tomcat from the repository, if you don’t care an easier method would be to just download apache-tomcat.zip from the Tomcat website, extract the archive to a folder under your home directory and just skip to step 4 of this tutorial where Under Server Location you would browse to where you extracted the archive and under Username and Password you would put the username and password you would have put in tomcat-users.xml.

1. Install Tomcat

1	$ sudo apt-get install tomcat7 tomcat7-common tomcat7-admin tomcat7-user

2. Add an administrative user to Tomcat

Tomcat does have any users defined that could be used for administrative purposes so we need to open tomcat-users.xml in a text editor and add a user with the proper roles assigned to it.

1	$ sudo gedit /etc/tomcat7/tomcat-users.xml

Replace its content with the following and make sure that you replace USERNAME and PASSWORD with the values you want:

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
    <role rolename="manager-script"/>
    <role rolename="admin"/>
    <user username="USERNAME" password="PASSWORD" roles="manager-script,admin"/>
</tomcat-users>

3. Some “fixes”

Unfortunately it seams that Netbeans doesn’t correctly detect the the Tomcat installation directory so we need to make some temporary changes till we add the server otherwise will get an error saying “The specified Server Location (Catalina Home) folder is not valid.”

$ cd /usr/share/tomcat7/lib
$ mv catalina.jar catalina.jar.bk
$ wget http://www.razius.com/files/catalina.jar

We need to make sure that the configuration files can be read from the installation directory otherwise will get an error saying “The <CATALINA_HOME>/conf/server.xml can’t be read.”

1	$ sudo ln -s /etc/tomcat7/ /usr/share/tomcat7/conf

We need to chown the Tomcat installation and configuration directory so we don’t have any permission problems. Replace USERNAME with your Ubuntu username:

$ sudo chown -R USERNAME:USERNAME /etc/tomcat7/
$ sudo chown -R USERNAME:USERNAME /usr/share/tomcat7/

Disable the upstart Tomcat service (Netbeans start’s it up for us automatically):

1	$ sudo update-rc.d tomcat7 disable 0123456

4. Adding the server in Netbeans

Now that you have Tomcat installed and configured we can open up Netbeans and under the Services tab right click on the Servers node and click on Add server.

In the current list select Apache Tomcat and click Next. Under Server Location enter your Tomcat installation directory , by default this should be /usr/share/tomcat7 and under Username and Password put in the username and password that we added in the tomcat-users.xml file.

Click Finish. You should now be able to see Tomcat as an added server under the Servers Node.

5. Revert changes

If we are installing from the repository we need to revert the changes done in step 3:

$ cd /usr/share/tomcat7/lib
$ rm catalina.jar
$ mv catalina.jar.bk catalina.jar

Wordpress reset password without email

2011-09-13T14:45:00+02:00

If you’re like me and you forgot your wordpress password and don’t have email setup on your system there are alternative ways to set a new password rather then sending and email.

Pick a particular method depending on what type of access you have to your website/hosting.

Through MySQL:

The easiest ways is through the MySQL command line but you need ssh access to your server.

Connect to MySQL:

1	# mysql -u root -p

Select the wordpress database (replace DATABASE with the wordpress database, if you don’t know it you can find it in wp-config.php under DB_NAME):

mysql> use DATABASE;

Update the password running the folowing query (replace NEWPASSWORD with your new password and USERNAME with the username you wish to change the password for):

mysql> UPDATE wp_users SET user_pass = MD5('NEWPASSWORD') WHERE user_login = 'USERNAME';

Through ftp/file edit

If you don’t have MySQL access you can also reset it by editing your theme’s functions.php file. Edit the file and insert after the first <?php the folowing(replace NEWPASSWORD with your new password):

1	wp_set_password('NEWPASSWORD',1);

After you changed your password make sure you delete the line from functions.php otherwise it will reset your password on every page load.

Hello world!

2011-08-15T09:52:00+02:00

After much slacking around and ignoring this blog I decided to give it another spin and bring it back to life.

It will be mostly technical articles about Linux, system administration and network administration.

root@crappy:/# echo 'Hello, World!'
Hello, World!