export IFS="                                                                                                                                          
"
for file in `find /home/claw/Musik -name *.flac` ; do echo "converting $file" && nice -19 sox $file `echo $file | sed 's/\.flac/\.mp3/g'`   ; done

You may have to install sox and libsox-fmt-mp3:

apt-get install sox libsox-fmt-mp3

You can either configure the DNS for each Client or let the Server push the configuration to its Clients. To do this add the following lines to the configuration of the VPN-Server:

push 'dhcp-option DOMAIN lan'
push 'dhcp-option DNS 192.168.1.1

Where 192.168.1.1 is the IP of the DNS-Servers and “lan” is the Domain.

Important!:
OpenWRT got a different Syntax for the config. For OpenWRT the lines in /etc/config/openvpn are:

list 'push' 'dhcp-option DOMAIN lan'
list 'push' 'dhcp-option DNS 192.168.1.1'

You will learn how to create such an encrypted container here. Some common scenarios could be to store the container on an cloud service such as (Dropbox, Google Drive, Own Cloud and co.) or to store sensible data own your own devices or at your workplace.

First of all you have to install cryptsetup.

apt-get install cryptsetup

1.Create an Container

At first we need to create an Container in which we will store our data. Using dd we can create a file containing random data

dd if=/dev/urandom of=crypt bs=1G count=60

‘bs’ is the block-size and ‘count’ the amount. That means: 1G*60=60G. The result would be an container of 60GB named ‘crypt’. It is possible to enlarge it later and it should be enough for demonstration purposes.

Next step is to mount the container to a not yet used loop device. To find one just use:

losetup -f

As Result we will get e.g. /dev/loop0. To mount crypt to any free device use:

losetup $(losetup -f) crypt

2. Encrypting the container

2.1 Using AES:

cryptsetup -c aes-xts-plain -y -s 512 luksFormat /dev/loop0

or:

2.2 Using Twofish:

cryptsetup -c twofish-xts-plain -y -s 512 luksFormat /dev/loop0

3.Mounting the Container

Next step we need to create a filesystem within the encrypted container in order to store file in it. To do so use the following command. The value is used here ( crypt0 ) can be chosen as you want and the filesystem that is used is ext4:

cryptsetup luksOpen /dev/loop0 crypt0
mkfs.ext4 /dev/mapper/crypt0

If every thing is done we can unmount the container again:

umount /dev/mapper/crypto0

Filemanagers such as Dolphin or Nautilus are able to handle such LUKS encrypted devices and should display them in the overview.

3. Mounting the container after boot

This part could be an security risk because the encrypted filesystem is not hidden any more. That does not mean that it is unencrypted after boot but an attacker will notice that you have some files encrypted without having to search them.

By adding following line to /etc/rc.local:

losetup $(losetup -f) <em>/path/to/the/container</em>

The encrypted filesystem will be mounted after boot. That way you can encrypt it in your filemanager every time.

I have developed a concept which reduces the power consumption and uptime of my network attached storage.

The system is therefore intelligent because it is recognizing that its needed or not on its own and sets to standby.There are many methods out there how to do that but none of them worked for me quiet well because the system turned off when i was coping files what was not really meant to be. For doing it my way i am using simple bash scripts which i would like to share with you and explain.

Configuring the Server/NAS:

Many services are providing commands to monitor their activities. E.g samba provides ‘smbstatus’ which prints active connections.

root@claw-storage:~# smbstatus -S

Service      pid     machine       Connected at
-------------------------------------------------------

In that case there are no active connection. If there where any they would be listed like that:

root@claw-storage:~# smbstatus -S

Service      pid     machine       Connected at
-------------------------------------------------------
raid1        17006   claw-t430     Tue Sep 11 13:50:29 2012
raid1        17005   claw-t430     Tue Sep 11 13:50:29 2012

So if there are no active connections smbstatus prints 4 lines. If there are more there must be more lines.
We can use that condition in an if-statement:

if [[ $( smbstatus -S | wc -l ) -lt 5 ]] 
then 
pm-suspend
fi

“If nobody is using samba right now run pm-suspend”

Lets do that endlessly using a while-loop:

while(true) 
do
if [[ $( smbstatus -S | wc -l ) -lt 5 ]] 
then 
pm-suspend
fi
done

Thats a solution. But not a good one because if nobody is logged in the server will be shut down immediately. We’ll need something like tolerance limit. We are going to specify a variable that looses value every time. If the value reaches 0 it is time to trigger the standby:

sleep 15m # Wait 15Minutes before starting the check                            
checks=15 # Number of Checks before taking action   
live=$checks

while(true)
do
sleep 1m
if [[ $( smbstatus -S | wc -l ) -lt 5 ]] 
then 
live=$(($live-1))
fi
if [[ $live -eq 0 ]]
then
live=$checks
pm-suspend
done

At the start of that script we are going to wait for 15 minutes because there is ofcourse a good reason why the system has been turned on. The number for checks till standby is set 15. Every check that ends positive ( nobody is logged in ) reduces the variable $live by 1. If $live=0 its raised to $checks ( meaning 15 ) again and the pm-suspend is gone to be executed.

Thats the principle. I divided it to a function and added checks for SSH, RSYNC and rdiff. The whole is this:

#!/bin/bash

sleep 15m # Wait 15Minutes before starting the check
checks=15 # Number of Checks before taking action

live=$checks

loggedin() #This function checks if ssh,rsync,samba or rdiff is running
           #if nothing of that is running system is going to suspend
{
    if [[ $( smbstatus -S | wc -l ) -lt 5 && $( w | grep pts | wc -l ) -lt 1 && $( ps aux | grep rsync | wc -l ) -lt 2 && $( ps aux | grep rdiff | wc -l ) -lt 3 ]] 
    then
        live=$(($live-1))
    else
        live=$checks
    fi

    if [[ $live -lt 1 ]]
    then
        live=$checks
        /usr/sbin/pm-suspend # IF no one is connected pm-suspend
    fi
}

# Durchlauf 
while(true)   # Endless while loop
do            #
    loggedin  # function
    sleep 1m  # time between checks
done

Copy the script to /usr/local/sbin/checkalive and make it executable:

chmod +x /usr/local/sbin/checkalive

and start it after boot  by running it from rc.local:

/etc/rc.local:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

bash /usr/local/sbin/checkalive
exit 0

Now we need a way to start the system remotely. Lets check if wake on lan is supported by the ethernet controller:

root@claw-storage:~# ethtool eth0 | grep -i wake Supports Wake-on: umbg Wake-on: g

If there is a line “Wake-on: g” the system can be waked by a magic packet. If not you have to add the following line to /etc/rc.local:

ethtool -s eth0 wol g

Waking the Server/NAS

In order to send such a magic packet to the server or the network attached storage we need the MAC of the ethernet controller to which we want to send it:

root@claw-storage:~# ifconfig 
eth0      Link encap:Ethernet  Hardware Adresse 00:1b:21:59:90:4f  
          inet Adresse:192.168.1.137  Bcast:192.168.1.255  Maske:255.255.255.0
          inet6-Adresse: fe80::21b:21ff:fe59:904f/64 Gültigkeitsbereich:Verbindung

The MAC of eth0 is 00:1b:21:59:90:4f

Now we can send the packet to the broadcast address of the subnet

root@claw-t430:~#wakeonlan -i 192.168.0.255 00:1b:21:59:90:4f

And viola the system wakes up!

You can add to whole to a script e.g /usr/local/bin/wake:

#!/bin/bash
wakeonlan -i 192.168.0.255 00:1b:21:59:90:4f
exit 0

and make it executeable:

chmod +x /usr/local/bin/wake

Now it should be enough to run

wake

for waking the system up.

df displayed the following:

# df -H
Dateisystem   Size   Used  Avail Use% Eingehängt auf
/dev/md0      6,0T   5,6T    13G 100% /mnt/raid1

Calculative there is something wrong because there are still 0,4T left. This is because ext reserves 5% of space per default because you would not be able to login in if there not enough space on the filesystem and you wont be able to enter the rescuing rm command. For my md device, which does not contain any systemfiles there is no need to reserve any space :

#tune2fs -m <em>0</em> /dev/md0

Sets the reserve to 0% and the 0,4T are free for usage.

The articles using the whole width of the browser what isn’t really nice because your eyes need to do much more motions that way. Furthermore for such kind of text serif fonts are recommended, because we can read them better.

The readability concept is use for tt-rss is the same i use for this blog. If you like you should continue reading.

Because pictures can say more than a thousand word here are some screenshots:

Only theory. Here the practice:

tt-rss allows you on the webinterface under settings to use a own stylesheet.
Add the following text there:

body#ttrssMain, body#ttrssPrefs, body#ttrssLogin, body {
    font-family : inherit;
}

div.cdmContentInner {
    font-family : inherit;
    font-size : 14px;
    font-weight: normal;
    text-align: justify;
    margin: 0 35% 0 8%;
    width: 80%;
    max-width: 700px;
}

div.cdmHeader > * {
    font-size : 18px;
    font-weight: bolder;
}

Because this blog is about teaching and learning:

• font-family : inherit; – Defines the font to “Inherit”
• font-size : 14px; - 14 Pixel font size
• font-weight : normal; - font weight ( could be bold, bolder, etc.)
• text-align: justify; - justify Layout as know from newspapers.
• width: 80%; - Text should use 80% of available space
• max-width: 700px;- But not more than 700 pixels ( what should be about A4 on common monitor resolutions.)

“div.cdmContentInner {” ist for the article text and “div.cdmHeader > * {”  the headline..

Such dead sectors are blocks on the surface of an hard drive which are not writable due to physical damages. Facing bad blocks can be the sentence of death  for a RAID Array.

But how can we be sure that an hard drive is healthy?

On the one hand there is SMART, which provides various quick checks for the drive but according to studies it is not that meaningful.

If you want to be confident you need to check block by block without leaving one out.

For doing so i am using ‘badblocks’ – Badblocks can check destructive (meaning all data will be lost) or non-destructive. The advantage over the non-destructive mode is that the check will be done ten times faster using the destructive mode. Besides you can also load the check result of badblocks into fsck, which will mark the block as faulty on the filesystem to prevent further writes on them.

Destructive Mode

Warning: Using that mode your data will be wiped!

badblocks -wsv -o foo.badblocks /dev/sdX

• This line starts an check for /dev/sdX.
• -w is the destructive Mode.
• -s shows an progressbar (e.g.: 12.16% done, 35:46 elapsed).
• -v like always this is for ‘verbose’ whereby found bad blocks will be print to stdin.
• -o provides the path for an output file (i used foo.badblock here). It can be loaded by fsck.

Because all data will be wiped by this mode its suitable for checking new hard drives or those where the data is not needed anymore.

Non-Destructive Mode

That mode is absolutely harmless and no data will be lost.

badblocks -nsv -o foo.badblocks /dev/sdX

• This line starts an check for /dev/sdX.
• -n is the non-destructive mode.
• -s shows an progressbar (e.g.: 12.16% done, 35:46 elapsed).
• -v like always this is for ‘verbose’ whereby found bad blocks will be print to stdin.
• -o provides the path for an output file (i used foo.badblock here). It can be loaded by fsck.

This mode is suitable when there is a suspicion for bad blocks (e.g because S.M.A.R.T reported them). The found dead sectors can be marked as faulty by fsck which will lead to an consistent device again. But keep in mind: bad blocks are spreading.

You can tell fsck to mark the found bad blocks as faulty by running:

fsck -l foo.badblocks /dev/sdX

The executing of badblocks can take a long time. The faster destructive mode took 48 hours for an 2TB drive during my investigations.

mdadm has a function that can ensure this even without having to rebuild the raid.

Before you can start you always have to make sure that everything is save for the present:

root@nas34 cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md42 : active raid5 sdd[0] sdb[4] sda[3] sde[1]
      4395411456 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]

If everything looks just fine, as above, we would not have to be concerned, if there would not be the risk of bad blocks because they have the characteristic that they become conspicuous only if there are read or write actions performed to them. For ensuring that the array act correct we are going to perform a manual check:

echo check >/sys/block/md42/md/sync_action

Now we have to check that the “check” we have just sent to /sys/block/md42/md/sync_action produces an effect:

cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md42 : active raid5 sdd[0] sdb[4] sda[3] sde[1]
      4395411456 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
      [>....................]  check = 0.0% (23564/1465137152) finish=500.0min speed=63564K/sec

Alright! The action has been started. If the Raid does not show any errors:

cat /sys/block/md42/md/mismatch_cnt
0

and does not suddenly start to rebuild we can grant that it is stable and start with the proper work, without being concerned that one of the disk could be damaged any way.

If you want to you can cancel the procedure by sending ‘idle’ to  /sys/block/md42/md/sync_action:

echo idle > /sys/block/md42/md/sync_action

auffallend

In /etc/vsftpd.conf the value of “use_localtime=” has to be set to “YES”. After changing the configuration you will have to restart ftp server:

invoke-rc.d vsftpd restart

For now on the timestamps should be set correctly.

The theory is that the more hard drives are in an array the more performance in reading and writing you will have. But having more hard drives will mean that the parities have to be distributed complexer what would reduce to write performance.

According to my benchmark these facts have been proven because i could notice an enhancement on reading and writing throughput. Have a look at the diagrams:

Per Char there are no logical changes. The fact that the proportion has turned might be an accident.

The bars rise sensible here

On Rewrite the factor x2 can be noticed and the Random Seeks are 40-50 more per second.

Altogether you can say that the performance of the RAID has really increased a lot after attaching one more hard drive. With an input of 225.288 K/s i have exceeded the theoretical speed of Gigabit Ethernet (120 MB/s) obviously.

Last week i posted an article about the application Preload and how you can use it to tune the startup of applications impressively.

Like Preload, Prelink is loading the files that are required for applications into RAM in order to load them faster than from a hard drive.

Unlike Preload Prelink has its focus on the required Libraries. The way files are cached is different too. But let me explain:

Libraries are cached to RAM and attached to an unique memory address. The effect on doing this is that the files are accessible much faster. This tunes the startup of application sensible ( i would say about 30% ) because the dynamic linker does not need to fetch the often used libraries from hard drive and can find them much faster.

This effect is most qualified for applications that need to access many libraries. Like KDE and Gnome too.

Apple named it Prebinding, Microsoft named it Prefetcher.

You can install Prelink on Debian and Co. by typing

apt-get install prelink

to prompt. After installation is finished you will have to enable prelinking by editing following file:

editor /etc/default/prelink

change the value of ‘PRELINKING’ to ‘yes’. Like that:

# Set this to no to disable prelinking altogether
# or to yes to enable it. Any other value will
# make prelink do nothing.
# (if you change this from yes to no prelink -ua
# will be run next night to undo prelinking)
PRELINKING=yes # Fill in 'yes' here

When this is done the prelink daemon will fetch informations about used libraries. What actually is cached can be shown by running:

prelink -p

The configuration is stored here:

/etc/prelink.conf

The files contains paths that are involved or not:

# This config file contains a list of directories both with binaries
# and libraries prelink should consider by default.
# If a directory name is prefixed with `-l ', the directory hierarchy
# will be walked as long as filesystem boundaries are not crossed.
# If a directory name is prefixed with `-h ', symbolic links in a
# directory hierarchy are followed.
# Directories or files with `-b ' prefix will be blacklisted.
-b *.la
-b *.png
-b *.py
-b *.pl
-b *.pm
-b *.sh
-b *.xml
-b *.xslt
-b *.a
-b *.js
-b /lib/modules
-b /usr/lib/locale
-l /usr/local/sbin
-l /sbin
-l /usr/sbin
-l /usr/local/bin
-l /bin
-l /usr/bin
-l /usr/X11R6/bin
-l /usr/games
-l /usr/local/lib
-l /lib
-l /usr/lib
-l /usr/X11R6/lib

This file wont need any modification, unless in some special cases e.g. if huge files will be cached or some impotent wont.

It would be great if you can give any feedback on how efficient Preload or Prelink was for you. Feel free to leave a comment.

This daemon creates statistics of your process and application usage. And that is all that is need for speeding up the start of applications by about 50%.

Preload is one of the applications i usually install right after setting up a new desktop distribution. Die effect on the response time of a system is clearly observable. Everything goes faster an makes much more fun :).

The clou is about the caching process, which loads everything that is need for your most used applications into the RAM. Thus the needed files can be loaded from RAM and not from hard drive.

Memory like DDR3-1333 RAM has a throughput of 84,8 Gbit/s while a hard drive has ~0,8 Gbit/s. Access times for RAM are in nanosecond range while hard drives are about 9 milliseconds.

The reason why this better for loading applications should be clear to everyone.

You can install preload on Debian and Co. by executing:

apt-get install preload

Further configuration is not needed but if want to have a look anyway refer

/etc/preload.conf

Be careful with the value of ‘cycle’. Do not set it too low.

preload will reduce the speed of your system on boot because the daemon has to be loaded and the cache needs to be collected. But the log in to your environment will be faster because it is already cached into RAM.

With default settings preload uses maximal 50% of your free memory. Therefore there will never be an overflow and the more RAM you have the better preload can cache. But in my experience 4 gigabyte of RAM will be more than enough.

Usage is easy. First you will need to install corksrew. Using distributions that are based on Debian (like Ubuntu) just run

apt-get install corkscrew

in commandline.
After the installation is done we have to add the proxy server to the ssh config – Start your editor of choice using:

editor /etc/ssh/ssh_config

and add a line which looks (in my case) like that:

ProxyCommand corkscrew <strong>localhost 8118</strong> %h %p

I am using Tor which runs on my local machine (therefore i used ‘localhost’) on port 8118.
You will have to modify those value fitting to your setup.

I use symbolic links to do that. Those links are just an reference to an existing files. That means that both files coexist.

• If you modify A you will modify B as well.

In order to give you an example i will list how you can sync the databases of Taskwarrior using Dropbox:

You will have to create a softlink of a file you want to be synced to another,

ln -s ~/.task ~/DropBox/Private/

this will create that link of “.task” to the private folder of Dropbox.

Now we need to link the file in the privat Dropbox folder to our local files on the second machine ( so just the other way around ):

ln -s ~/DropBox/Private/ ~/.task

Thats it. Linking files that way you are able to sync just everything and put it on its right place.

Try it and enjoy!

In order to solve that problem i had to think about a solution that will constantly remind me of my pending tasks and forces me to complete them.

I decided that the way to do is to call task every time i open a terminal (Konsole for KDE).

I added the following two lines to the .bashrc in my home directory:

alias todo='task calendar ; task ; task summary'
todo

The first line defines a new alias named ‘todo’. When ‘todo’ is executed it actually start ‘task calendar’, ‘task’ and ‘task summary’ in a row. The second line executes todo when starting a bash.

That could like this:

First you will need the application “screen”. It will emulate a terminal (just as konsole, gterminal und other do).

apt-get install screen

Modify the permissions and SUID Bit in order to make screen executable to any user.
It is /usr/bin/screen for Debian. If it is not for your distribution you can find the binary using:

whereis screen

chmod +s /usr/bin/screen
chmod 755 /var/run/screen

We need a local user who will be able to log in and work on our shell. For security reasons you better set it up with a restricted Bash in which he wont be able to leave his home directory.

adduser --system --shell /bin/rbash <em><strong>USERNAME</strong></em>

and add a password

passwd <em><strong>USERNAME</strong></em>

Now your friend is able to connect to your machine using SSH:

ssh <em><strong>USERNAME</strong></em>@<strong><em>YOUR-IP/HOSTNAME/DYNDNS</em></strong>

In order to allow him joining your screen session you will have to edit /etc/screenrc

editor /etc/screenrc

Add the following lines to the “SCREEN SETTINGS” block:

multiuser on
acladd <strong><em>USERNAME</em></strong>

acladd defines the user who will be able to join and type into your shell (So you are able to insert more users).

Start a screen session:

screen -S <strong><em>SESSION-NAME</em></strong>

Your friend can now join that session using

screen -x <strong><em>YOUR-USERNAME</em></strong>/<strong><em>SESSION-NAME</em></strong>

If this wont work for you – Feel free to ask

# This file is sourced by /etc/init.d/sabnzbdplus
#
# When SABnzbd+ is started using the init script, the
# --daemon option is always used, and the program is
# started under the account of $USER, as set below.
#
# Each setting is marked either "required" or "optional";
# leaving any required setting unconfigured will cause
# the service to not start.

# [required] user or uid of account to run the program as:
USER=claw #<--- Add username here

# [optional] full path to the configuration file of your choice;
#            otherwise, the default location (in $USER's home
#            directory) is used:
CONFIG=

# [optional] hostname/ip and port number to listen on:
HOST=
PORT=

# [optional] extra command line options, if any:
EXTRAOPTS=

Notice line 12
That is the user the sabnzbdplus progress will run under when booting next time. Who does not want to wait long could run:

/etc/init.d/sabnzbdplus start

instead.

For solving this problem i wrote a tiny bash script and i want to share it with you

#!/bin/bash
status() {
        if [[ $(grep data.location /home/$USER/.taskrc) == data.location=/home/$USER/.task/work ]]; then #Using work?
                echo "Task is using WORK"
        elif [[ $(grep data.location /home/$USER/.taskrc) == data.location=/home/$USER/.task/home ]]; then #Using home?
                echo "Task is using HOME"
        else
                echo "Error"
        fi
}

if [[ $1 == home ]]; then
        echo "HOME activated" #Activates home
        sed -i 's/^data.location=\/home\/'${USER}'\/.task\/work/data.location=\/home\/'${USER}'\/.task\/home/' /home/$USER/.taskrc
elif [[ $1 == work ]]; then
        echo "Work activated" #Activates work
        sed -i 's/^data.location=\/home\/'${USER}'\/.task\/home/data.location=\/home\/'${USER}'\/.task\/work/' /home/$USER/.taskrc
else
        status
fi
exit 0

Just put that script to /usr/local/bin and name it task-switch

editor /usr/local/bin/task-switch

And copy the script text to it – There is an button for source view when hovering it.
Make it executable using:

chmod a+x /usr/local/bin/task-switch

Those who are into bash will notice that the .taskrc located in you /home-folder is modified. In order to make that possible you will have to create two directories in .task like that:

cd
cd .task
mkdir home work
<em>mv *.data home-oder-work</em>

For the last line you will have to chose if you want to migrate your existing database to ‘work’ or to ‘home’.
Use:

mv *.data home # For home

or

mv *.data work # For work

Now you have to edit the .taskrc and change the value of data.location= like that:

data.location=/home/'${USER}'/.task

change it to:

data.location=/home/'${USER}'/.task/home

Now the active database is set to ‘home’.
When executing task-switch you should get similar output:

claw@claw-hp625:~$ task-switch
Task is using HOME

If you want to switch to another database execute task followed by the database name you want to use:

claw@claw-hp625:~$ task-switch work
Work activated

If this is not working well in anyway feel free to contact me. I will try to fix your problems as soon as i can

sed 's/'${var}'/'${foo}'/g'

This is how variables can be used with sed

Comment
Switch to the “visual block mode” using CTRL+V
and mark downwards every line you want to comment out. (Arrow down or j).
If you have marked every line switch to the “insert mode” using SHIFT+I.
Now add the command char (e.g. # in bash) to the beginning of the upper line and then hit ESC for escaping the “insert mode”.
Now there will be added # at the beginning of every marked line.

Un-Comment
Switch to the “visual block mode” using CTRL+V
and mark every number-sign you want to remove (Arrow down or j)
If you have marked all signs just hit ‘x’