Backup manually

In this section I will describe how to perform manual backups and later one I'll provide some bash scripts so automation can take place with those.

Data volumes

Thats right. The only thing you want to backup is Docker container's data volumes. Except database containers where you can simply use mysqldump command or any other containers with custom implementation (i.e. GitLab).

To back up Docker container volumes (in this example container's name is proxy), first you have to find one (or more), simply by running the following command:

# docker inspect --format '{{range .Mounts}}{{.Source}}{{" -> "}}{{.Destination}}{{"\n"}}{{end}}' proxy
/var/lib/docker/volumes/6890bad0490303e738ecae184da43fe9860e17ceca474c0a74c2d5974976e179/_data -> /usr/share/nginx/html

Then we can simply tar whole data volume path:

tar zcfv backup.tar.gz /var/lib/docker/volumes/6890bad0490303e738ecae184da43fe9860e17ceca474c0a74c2d5974976e179/_data  

Untar backup with the following:

tar zxfv backup.tar.gz -C /  

This will automatically overwrite the contents of your data volume (data volume path must be the same that we've compressed).

Database containers

To backup and restore databases (MySQL / MariaDB) running in a container is very simple. You can do that using the same method described about data volumes, but you can also use mysqldump tool that's more versatile.

Backup with the following command:

  docker exec database sh -c 'exec mysqldump -uroot -p"$MYSQL_ROOT_PASSWORD" --all-databases' > /tmp/backup.sql

This will backup all database tables of database named container to /tmp/backup.sql using $MYSQL_ROOT_PASSWORD environment variable as root password within container.

Restore simply with the following:

docker exec -i database sh -c 'mysql -uroot -p"$MYSQL_ROOT_PASSWORD"' < /tmp/backup.sql

Backup to Dropbox

Have plenty of unused space in Dropbox? Why not to use it as a backup storage? I've found that Dropbox-Uploader fits my needs perfectly. Follow all instructions on that GitHub page and setup your Dropbox.

Then you can simply backup your .sql or .tar.gz files:

./dropbox_uploader.sh upload backup.tar.gz /location_in_dropbox/

Automating backups

I wrote a simple bash script that decides on what's the best way of backing up a container based on its image name, i.e. if it's MySQL or MariaDB container, mysqldump tool will be used. On all other occasions, data volumes will be compressed using tar command.

Additionally my script can use several Dropbox accounts. The usage of a script is fairly simple:

./backup.sh <container_name> [dropbox_account]

You can add this to crontab and have daily/weekly/monthly backups.

Restore from backup script

As a backup, restoration script acts similarly:

./restore.sh <container_name> <dropbox_account> <days_back>

i.e.

./restore.sh proxy EA -1

There's actually two ways of going with this:

• Installing packages separately and do a slight configurations (takes more time)
• Deploying a docker container (simple, yet still in development)

Packages

We're going to use Deluge WebUI for headless torrents box, flexget for RSS integration and minidlnad for DLNA server.

Install required epel-release and nux-dextop repos:

yum install -y epel-release  
wget http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm && rpm -ivh nux-dextop-release-0-5.el7.nux.noarch.rpm  

Install required packages:

yum install -y minidlna deluge-web  
easy_install flexget  

Configuration

Flexget

To integrate Flexget (RSS for torrents) into Deluge WebUI we'll need to setup authentication with it. First extract username and password:

cat /var/lib/deluge/.config/deluge/auth | awk -F: '{print $1}'  
cat /var/lib/deluge/.config/deluge/auth | awk -F: '{print $2}'  

You'll see something like this:

# cat /var/lib/deluge/.config/deluge/auth | awk -F: '{print $1}'
localclient  
# cat /var/lib/deluge/.config/deluge/auth | awk -F: '{print $2}'
c120ff48a3c6fdad072fc59aa2b67fd1e000e5b7  

If you cannot find auth file, it's alternative location can be /root/.config/deluge/auth.

Download config.yml from GitHub and place it in /root/.flexget/. Change rss: with tracker RSS link that you're using and other options with preference of yours. Change username_goes_here and password_goes_here with the details we've extracted before, in example from this:

username: username_goes_here  
password: password_goes_here  

To this:

username: localclient  
password: c120ff48a3c6fdad072fc59aa2b67fd1e000e5b7  

You can test if flexget communicates with Deluge WebUI and torrent tracker (finds your shows, etc.) with the following command:

flexget execute -v

Finally, add flexget to cron so that it will perdiocially search for entries in RSS. Add the following line in crontab (crontab -e):

@hourly /usr/bin/flexget execute -v >> /var/log/flexget.log 2>&1

MiniDLNA

Edit minidlna.conf that's located in /etc/ and define where downloaded media will be located. My example:

media_dir=A,/home/Music  
media_dir=V,/home/Videos  
media_dir=P,/home/Pictures  

Restart minidlnad to apply changes:

systemctl restart minidlnad

Deluge WebUI

Deluge web client can be accessed via browser on 8112 port, i.e. http://localhost:8112. First-time password is deluge and can be changed instantly. Configuration is pretty straightforward, just make sure to change download path to be the same as provided in minidlnad.conf file.

That's it! You've got a fully working headless torrent box with RSS and DLNA server.

Docker Container

I've made a docker container named edgaras/dlna-torrentbox (more info can be found on docker hub). Please keep in mind that it's still in development and RSS is not working at the moment.

Deployment

If you have docker packages installed, just deploy container with the following command:

docker run --name torrent-box -p 8112:8112 -P -v /home -d edgaras/dlna-torrentbox:latest  

Explanation on command-line options:

• --name torrent-box - specify name of Docker container. In this case it'll be torrent-box.

• -v /home - mount /home as data volume. By default this minidlna will take media from /home/Music, /home/Videos, /home/Pictures.

• -d run as a daemon.
• edgaras/dlna-torrentbox:latest pull and use this image.

Configuration

After container deployment is complete you should be able to reach Deluge WebUI via http://IP:8112 change IP to your Linux machine one of course.

You'll be prompted for a password. It'll be deluge and you'll be able to change it right away. After that change your torrent download location to /home/Videos so that minidlna would be able to pick it up and stream to other devices.

Software

Casual

• Lithuanian Keyboard - "Lietuvių (skaičių eilės)"
• Paragon NFTS - makes NTFS writable
• Skitch - awesome screenshoting tool
• iA Writer - best writing machine that supports markdown
• Amphetamine - does not let Mac to sleep
• Pixelmator - for photo editing
• Pocket - read it later service
• Daisy Disk - shows disk space usage
• Hotkey-EVE - helps to learn shortcuts
• Battery Monitor - displays detailed information on battery
• Tunnelblick - for VPN
• VLC - best video player
• Unclutter - a new handy place on your desktop for storing notes, files and pasteboard clips

DevOps

• Slack - chat app with lots of integrations
• iTerm2 with Solarized Dark theme
• Atom.io - One of the best IDEs available. It's free and has killer Git integration. Used with Solarized Dark theme, git-plus and language-docker plugins.
• Homebrew - missing package manager for OS X:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"  

• Git - best version control system:

sudo brew install git  

With the following setup:

git config --global user.name "Edgaras"  
git config --global user.email "email"  
git config --global core.editor vim  
git config --global help.autocorrect 1  
git config --global color.ui auto  
git config --global core.autocrlf input  

• Vagrant + VirtualBox for setting up lab environment locally:

sudo brew tap caskroom/cask  
sudo brew install brew-cask  
sudo brew cask install vagrant virtualbox  

• ChefDK (+ kitchen) - to test recipes locally or manage remote Chef servers:

sudo brew cask install chefdk  

• Chicken - my favourite VNC client app

OS X Tweaks

• Increase standby delay

sudo pmset -a standbydelay 43200  

Packages

First of all, you’ll need these packages to be installed on the Linux machine:

• autoconf
• automake
• make
• gcc
• wget
• unzip
• libtool
• pam-devel (for CentOS / RHEL)
• libpam0g-dev (for Debian)

yum install pam-devel gcc make autoconf automake wget unzip libtool or
apt-get install libpam0g-dev gcc make autoconf automake wget unzip libtool if you’re on Debian.

Then, download google-authenticator from it’s Github page via git clone or wget command. An example with wget:

wget https://github.com/google/google-authenticator/archive/master.zip
unzip master.zip

Compile the code

After the files are on the filesystem, we have to compile google-authenticator:

cd google-authenticator-master/libpam/
./bootstrap.sh
./configure
make
make install

After make install successful output will look like this:

# make install
cp pam_google_authenticator.so /lib64/security
cp google-authenticator /usr/local/bin

Set-up google-authenticator

Now we need to configure google-authenticator, just run it and answer the questions with y/n with your preferences. I’ve answered all to yes:

# google-authenticator

Do you want authentication tokens to be time-based (y/n) y
Your new secret key is: RSXXXXXXXXXXXXXX
Your verification code is 010101
Your emergency scratch codes are:
  XXXXXXXXX
  XXXXXXXXX
  XXXXXXXXX
  XXXXXXXXX
  XXXXXXXXX

Do you want me to update your „/root/.google_authenticator”     file (y/n) y

Do you want to disallow multiple uses of the same authentication token? This restricts you to one login about every 30s, but it increases your chances to notice or even prevent man-in-the-middle attacks (y/n) y

Do you want to disallow multiple uses of the same authentication token? This restricts you to one login about every 30s, but it increases your chances to notice or even prevent man-in-the-middle attacks (y/n) y

By default, tokens are good for 30 seconds and in order to compensate for possible time-skew between the client and the server, we allow an extra token before and after the current time. If you experience problems with poor time synchronization, you can increase the window from its default size of 1:30min to about 4min. Do you want to do so (y/n) y

If the computer that you are logging into isn’t hardened against brute-force login attempts, you can enable rate-limiting for the authentication module. By default, this limits attackers to no more than 3 login attempts every 30s. Do you want to enable rate-limiting (y/n) y

Mobile app configuration

Open up your Google Authenticator application (doesn't matter if you're on Android or iOS), hit the advanced button (3 dots in upper right corner) and select setup account.

Now you can choose to Scan a barcode or Enter key provided and just enter provided Secret key earlier.

System configuration

Now we need that the system would use google-authenticator during SSH login.

We’ll need to edit /etc/pam.d/sshd and /etc/ssh/sshd_config files.

In /etc/pam.d/sshd add the following line:

auth required pam_google_authenticator.so

In /etc/ssh/sshd_config file change

ChallengeResponseAuthentication no

To

ChallengeResponseAuthentication yes

Restart ssh daemon systemctl restart sshd or /etc/init.d/ssh restart if you’re on Debian.

Setup

First, install required packages:

yum install bridge-utils tunctl bridge-utils bind-utils tuned-utils-systemtap lvm virt-manager libvirt virt-install qemu-kvm xauth dejavu-lgc-sans-fonts

Enable IPv4 forwarding:

echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.d/99-sysctl.conf

reboot your system and check if you can run virsh command:

virsh -c qemu:///system list

Edit /etc/sysconfig/network-scripts/ifcfg-eno1 file so it would reflect the following:

TYPE="Ethernet"
NAME="eno1"
UUID="60093307-ad32-425c-b859-724fd9941ba7"
DEVICE="eno1"
ONBOOT="yes"
BRIDGE="br0"

Create br0 interface named ifcfg-br0 and paste the following:

DEVICE=br0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
NAME=br0

Enable and start libvirtd daemon:

systemctl start libvirtd && systemctl enable libvirtd

Stop/disable NetworkManager and restart network daemon:

systemctl stop NetworkManager; systemctl restart network

Provisioning

Run osinfo-query os | grep centos to get required guest OS short ID like centos7.0

Download OS base image to /tmp:

wget http://mirror.duomenucentras.lt/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1503-01.iso

Install guest OS via virt-install:

virt-install --connect qemu:///system -n [name] -r [ram_size_in_MiBs] --vcpus=[virtual_cpus] --disk path=/var/lib/libvirt/images/[name].img,size=[size_in_GiBs] --graphics inc,listen=0.0.0.0 --noautoconsole --os-type linux --os-variant [short_id] --accelerate --network=bridge:br0 --hvm --cdrom [path_to_iso.iso]

i.e.:

virt-install --connect qemu:///system -n anaconda -r 2048 --vcpus=2 --disk path=/var/lib/libvirt/images/anaconda.img,size=2 --graphics inc,listen=0.0.0.0 --noautoconsole --os-type linux --os-variant centos7.0 --accelerate --network=bridge:virbr0 --hvm --cdrom /tmp/CentOS-7-x86_64-Minimal-1503-01.iso

After successful provision, we need to find VNC port to connect to:

virsh vncdisplay [name]

:0 means 5900, :1 - 5901 and so on.

Accordingly open firewall port:

firewall-cmd --zone=public --add-port=5900/tcp --permanent

And reload configuration to apply changes:

firewall-cmd --reload

Connect to your guest via VNC client to finish installation.

Chicken example:

Deleting guest OS and image

Destroy VM:

virsh destroy [name]

Make sure VM is not running with virsh list and then undefine domain (name):

virsh undefine [name]

Finally, remove VM image:

rm -rfv /var/lib/libvirt/images/[name].img

Useful commands

virsh list - show all running VMs

virsh --connect qemu:///system autostart [name] - enable guest OS autostart

virsh start [name] - start VM.

virsh shutdown [name] - stop VM.

virsh destroy [name] - restart VM.

First, you have to install Vagrant and VirtualBox on your workstation (guide on how to do that easily — http://apsega.lt/getting-started-with-vagrant-and-virtualbox-on-os-x/, but you can skip adding boxes and initialising VM steps).

Then, download and install Chef Development Kit from chef.io, or use command line and install it via brew cask:

brew cask install chefdk

After Chef DK has been successfully installed, we can create our Kitchen working directory (i.e. mkdir chef) and generate first Cookbook by running the following command:

chef generate cookbook nginx (you can specify your own cookbook name other than nginx)

Change directory to newly created cookbook:

cd nginx

Needed Kitchen virtual machine configuration is in .kitchen.yml file, while Vagrant file is in .kitchen/kitchen-vagrant/kitchen-nginx-default-centos71/ location. Editing .kitchen.yml is suffice for our Kitchen environment (changing box names, etc.).

After we've configured our desired VM state, we can bring it up easily:

kitchen converge

This command is all-in-one package that actually initialises Vagrant box, installs chef-client into it and executes your recipes. To check your VM state, you can run:

kitchen list

To login to your guest box run:

kitchen login

And you can always destroy your box with the following:

kitchen destroy

Once you're done with you virtual machine setup, package your Vagrant box:

vagrant package --output centos.box

Then, you can add your packaged box to the vagrant:

vagrant box add centos.box --name centos

Just to double-check if the box has been added to vagrant, list all added boxes:

vagrant box list

Edit Vagrant file to add the following lines (this configuration is for 3 replicated servers):

def config_centos(config, node_number)
config.vm.define "zipset#{node_number}" do |replica|
    replica.vm.box = "centos"
    replica.vm.hostname = "centos#{node_number}"
    replica.vm.network "private_network", ip: "192.168.50.10#{node_number}"
    end

    for node_number in 1..3
        config_centos config, node_number
    end

Now we can start all three servers at once with the following Vagrant command:

vagrant up /centos*/

You can check all of your Vagrant machines statuses with vagrant global-status command.

To install Vagrant and VirtualBox on Mac OS X, simply start by installing brew and cask with these 3 commands:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew tap caskroom/cask

brew install brew-cask

Then, install Vagrant and VirtualBox via brew and cask:

brew cask install vagrant virtualbox

After that you can simply check installed versions:

vagrant -v && vboxmanage -v

Add CentOS image to Vagrant (you can choose other than CentOS, here's a list of others):

vagrant box add centos https://github.com/tommy-muehle/puppet-vagrant-boxes/releases/download/1.1.0/centos-7.0-x86_64.box

To start your virtual machine, create a directory, change to it and initialise Vagrant box:

mkdir centos && cd centos && vagrant init centos

After that, you can start your box and SSH into it:

vagrant up

vagrant ssh

That's it! You've got CentOS VM running with Vagrant! You can package your vagrant box using vagrant package --output centos.box command.

Other useful Vagrant and VirtualBox commands can be listed with vagrant -h and vboxmanage -h | less.