MY Experiences

Find Size of all files with certain extensions

2020-07-08T16:17:00.003+02:00

You can find the size with this find command:

find . -iregex ".*\.$jpg\|gif\|png\|jpeg$" -exec du -ch {} + | grep total$

iregex means case-insensitive.

Use different backend based on HTTP Header in Nginx

2020-05-07T09:39:00.005+02:00

To do this, you can use this configuration:

upstream production {
     server 10.0.0.1:80;
}

upstream stage {
     server 10.0.0.2:80;
}


# map to different upstream backends based on header
map $http_x_which_deployment $pool {
     default "production";
     stage "stage";
}

server {
     listen 80;
     server_name _;
     location / {
            proxy_pass http://$pool;
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $http_true_client_ip;
            proxy_set_header Host $host;
            proxy_connect_timeout       600;
            proxy_send_timeout          600;
            proxy_read_timeout          600;
            send_timeout                600;     }
}

Then if you add "X-Which-Deployment" header and set it to "stage", you will be pointed to "Stage Server" (here 10.0.0.2).

Elasticsearch flood stage disk watermark error

2020-04-08T14:44:00.005+02:00

If your disk is full, you must first empty it and then run these two curl commands:

curl -XPUT -H "Content-Type: application/json" http: //localhost:9200/_cluster/settings -d '{ "transient": { "cluster.routing.allocation.disk.threshold_enabled": false } }'

curl -XPUT -H "Content-Type: application/json" http: //localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}

Debian 8 and Varnish 4 Problem

2015-11-18T10:40:00.001+01:00

The Varnish service was starting OK but it wasn't using any of the parameters, set in /etc/default/varnish.

For example, set the listening port to be 80, but Varnish would always start with port 6081.

The Varnish service (/lib/systemd/system/varnish.service) wasn't using any parameter files - instead it was hard coded to start with the default values only.

The temporary fix is to edit the varnish.service file and manually configure the parameters to match your needs.

reference: https://www.varnish-cache.org/forum/topic/5206

Disable MySQL Strict mode in Debian 8 permanently

2015-11-12T15:59:00.001+01:00

All blogs say:

for turning off strict mode you must add following line to /etc/mysql/my.cnf:

sql_mode=""

and restart the mysql server.

That was not working me.

I have found another setting file for mysql: /usr/my.cnf

and in this file exists sql_mode parameter too.

sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

this line must be commented out too:

#sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

Don't forget to restart the mysql server.

Checking SSL Certifcate on Plesk Mail Server

2015-07-23T15:15:00.004+02:00

Run this command (mail.DOMAIN.de is the MX-Record):

SMTP:
openssl s_client -starttls smtp -crlf -connect mail.DOMAIN.de:25

POP3:
openssl s_client -starttls pop3 -crlf -connect mail.DOMAIN.de:110

IMAP:
openssl s_client -starttls imap -crlf -connect mail.DOMAIN.de:143

As result you can see your certificate if it installed correctly:

Certificate chain

0 s:/description=FmJdZEx3lsXPoIXv/C=DE/CN=mail.DOMAIN.de/emailAddress=hostmaster@DOMAIN.de

i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA

1 s:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA

i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority

---

How to downgrade Openssl on CentOS?

2015-07-23T15:08:00.005+02:00

List the available versions by running this command:

yum --showduplicates list openssl

you should see these lines:

Installed package:
openssl.x86_641:1.0.1e42.el7.9

Available packages:
openssl.x86_64 - 1:1.0.1e-42.el7 - base
openssl.x86_64 - 1:1.0.1e-42.el7_1.5 - updates
openssl.x86_64 - 1:1.0.1e-42.el7.4 - updates
openssl.x86_64 - 1:1.0.1e-42.el7.6 - updates
openssl.x86_64 - 1:1.0.1e-42.el7.8 - updates
openssl.x86_64 - 1:1.0.1e-42.el7.9 - updates

then doing:

yum downgrade openssl-1.0.1e-42.el7_1.5

it will downgrade openssl to 7_1.5 version.

Rollback to previous package version after bad upgrade with Debian

2015-03-27T16:01:00.004+01:00

This will show all the versions of the package in the cache:
apt-cache show [package name]

This will install an older version of the package
apt-get install [package name]=[version]

RSYNC using filter to include some files

2014-11-11T17:43:00.003+01:00

use rsync with Filter option to include some files.

For example:

rsync -auvz -f'+ [Aa][Mm]*' -f'+ */' -f'- *' /SOURCE-Folder/ /TARGET-Folder/

It means: exclude all files and include only files beginning with A and M (insensitive).

Add line to the beginning of a file in Bash

2014-11-05T10:53:00.002+01:00

To add a line to the beginning of a file, use this command:

sed -i '1s/^/YOUR NEW LINE\n/' YOUR-FILE.TXT

Mysql 5.5 Innodb slow insert query issue

2014-10-30T11:40:00.000+01:00

An upgrade MySQL from version 5.1 to 5.5 has very slowed Typo3 instances.

The reason was very slow insert queries in InnoDB tables.

In order to save you some distress, for a possible solution, add following line to my.cnf and restart the mysql:

innodb_flush_log_at_trx_commit=0

Disable SSLv2 and SSLv3 in Apache

2014-10-15T15:35:00.003+02:00

The best way to disable both SSLv2 and SSLv3 and only enable TLS 1.0-1.2 is to use this configuration in your Apache configuration file:

SSLProtocol TLSv1

Example:

SSLEngine On
SSLProtocol TLSv1
SSLCertificateFile      domain.crt
SSLCertificateKeyFile   domain.key

Testing your SSL Version

2014-10-15T15:31:00.006+02:00

if you want to test your ssl version details of perticular host use the following command:

openssl s_client -connect YOURDOMAIN:443

You need to replace YOURDOMAIN to your website hostname Output looks like below:

Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol : TLSv1
    Cipher    : ....
    Session-ID: ....

Checking Varnish configuration syntax

2014-09-24T15:29:00.000+02:00

If you have updated your Varnish configuration and want to check it,
run this command:

varnishd -C -f /etc/varnish/default.vcl

if everything is correct, Varnish will dump out the genrated configuration, otherwise you will get an error message pointing you to a specific line number.

Benchmarking LAN Network Connectivity in Linux

2014-09-23T18:00:00.003+02:00

Using netcat:

on destination machine run this command:
(This tells netcat to open port 20000 and listen for connections.)

nc -l 20000 > /dev/null

on source machine execute this command:
(This command will transfer 30MB of data over the network (a blocksize of 1MB, and 30 blocks, is 30MB)

dd if=/dev/zero bs=1M count=30 | nc HERE.COMES.DEST.IP 20000

Once all data is transferred, you’ll get the output from the dd command, showing throughput:

30+0 records in
30+0 records out
31457280 bytes (31 MB) copied, 5.40742 s, 5.8 MB/s

source: http://techthrob.com/2014/01/benchmarking-lan-network-connectivity-in-linux/

TYPO3 extension manger won't work in XAMPP under Windows

2014-09-19T15:35:00.000+02:00

Add these lines to all three apache configuration files (http.conf, http-ssl.conf,http-xampp.conf):

<IfModule mpm_winnt_module>
ThreadStackSize 8388608
</IfModule>

then restart the apache.

Getting HTTP HEAD from command line

2014-09-18T10:51:00.000+02:00

Install libwww-perl (Debian) / perl-libwww-perl (CentOS):

Debian: apt-get install libwww-perl

CentOS: yum install perl-libwww-perl

Run this command:

GET -Used http://YOUR-WEBSITE.COM/

MySQL Database Benchmark

2014-09-15T16:41:00.003+02:00

Installing sysbench:

apt-get install sysbench

Preparing the Test Database:

sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=ADMIN-USER --mysql-password=ADMIN-PASSWORD prepare

Running Test:
sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=ADMIN-USER --mysql-password=ADMIN-PASSWORD prepare

Cleaning test Databse:
sysbench --test=oltp --mysql-db=test --mysql-user=ADMIN-USER --mysql-password= ADMIN-PASSWORDcleanup

Add locale to Ubuntu

2014-05-08T12:03:00.003+02:00

check which locales are supported:
less /usr/share/i18n/SUPPORTED

Add locale (for example German) to list of generated:

echo de_DE.UTF8 >> /var/lib/locales/supported.d/local

Regenerate list (it will invoke locale-gen...):

sudo dpkg-reconfigure locales

How to download entire Youtube channel using commands in ubuntu

2014-01-03T11:19:00.006+01:00

Use youtube-dl command to download videos straight off of YouTube.

Install it:

apt-get install youtube-dl

To download an entire channel

youtube-dl -citw ytuser:USER

To download all videos on a playlist:

youtube-dl -citw PLAYLIST-URL

QMail Changing send IP

2013-10-24T14:06:00.000+02:00

The default IP for outgoing connections can be set using the "ip" utility. Check how routing is configured on the server:

assume we have two network interfaces vnet0:0 (192.168.175.100) and vnet0:1 (192.168.175.200) and 192.168.175.1 is gateway:

[root@server]#/sbin/ip route

192.168.175.0/24 dev venet0  scope host
169.254.0.0/16 dev venet0  scope link
default via 199.168.175.1 dev venet0

If no "src" is listed in the "ip" output for the default route, the main IP on the interface is used for outgoing connections.

To change Plesk default Mail IP address, check ifconfig on the server first as :

lo        Link encap:Lokale Schleife
          inet Adresse:127.0.0.1 Maske:255.0.0.0
          inet6 Adresse: ::1/128 Gültigkeitsbereich:Maschine
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:898825 errors:0 dropped:0 overruns:0 frame:0
          TX packets:898825 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 Sendewarteschlangenlänge:0
          RX bytes:2333074597 (2224.9 Mb) TX bytes:2333074597 (2224.9 Mb)

venet0    Link encap:UNSPEC Hardware Adresse 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet Adresse:127.0.0.1 P-z-P:127.0.0.1 Bcast:0.0.0.0 Maske:255.255.255.255
          UP BROADCAST PUNKTZUPUNKT RUNNING NOARP MTU:1500 Metric:1
          RX packets:69429506 errors:0 dropped:0 overruns:0 frame:0
          TX packets:109230137 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 Sendewarteschlangenlänge:0
          RX bytes:19231956377 (18341.0 Mb) TX bytes:136258969119 (129946.6 Mb)

venet0:0 Link encap:UNSPEC Hardware Adresse 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet Adresse:192.168.175.100 P-z-P:92.168.175.100 Bcast:0.0.0.0 Maske:255.255.255.255
          UP BROADCAST PUNKTZUPUNKT RUNNING NOARP MTU:1500 Metric:1

venet0:1 Link encap:UNSPEC Hardware Adresse 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet Adresse:192.168.175.200 P-z-P:92.168.175.200 Bcast:0.0.0.0 Maske:255.255.255.255
          UP BROADCAST PUNKTZUPUNKT RUNNING NOARP MTU:1500 Metric:1

Change the mail IP address through following command as :

[root@server]#/sbin/ip route change default via 192.168.175.1 dev venet0:1 src 192.168.175.200

now verify the mail IP address :

[root@server]#/sbin/ip route

192.168.175.0/24 dev venet0  scope host
169.254.0.0/16 dev venet0  scope link
default via 192.168.175.1 dev venet0  src192.168.175.200

If "ip route" command says nothing about gateway or gateway is 0.0.0.0, for example like this one:

[root@server]#/sbin/ip route

127.0.0.0/8 dev lo scope link
default dev venet0 scope link

you can use this command:

[root@server]#/sbin/ip route change default via 192.168.175.100 dev venet0 src 192.168.175.200

Useful commands with ffmpeg

2013-09-05T15:28:00.002+02:00

Converting mp4 to flv:

ffmpeg -i INPUT.mp4 -vcodec copy -acodec aac -strict experimental -ac 2 -ar 44100 -ab 192k OUTPUT.flv

Deleting logo from video:

ffmpeg -sameq -i INPUT.flv -vf "delogo=x=10:y=10:w=80:h=60:band=80:show=0" -vcodec flv -acodec copy OUTPUT.flv

this command deletes the logo which exists at x=10 and y=10 (x=0 and y=0 is top left). h is height and w is width. You can use show=1 to see the location of deleted logo.

Crop video:
ffmpeg -sameq -i INPUT.flv -vf "crop=WIDTH:HEIGHT"
OUTPUT.flv

Amazon AWS - Changing dynamically Apache Log File name based on Hostname

2013-07-11T15:15:00.001+02:00

Change your vhost apache config as follow (the LogFormat writes correct client IPs to log):

LogFormat "\"%{X-Forwarded-For}i\" %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined_new

CustomLog "|cat >> /shared/log/apache2/`hostname`-access.log" combined_new

Mysqldump only tables with certain prefix / Mysqldump wildcards

2013-07-02T14:20:00.003+02:00

This commans will dump the tables with prefix PREFIX:

mysql DATABASE -u USERNAME -p -e 'show tables like "PREFIX%"' | grep -v Tables_in | xargs mysqldump DATABASE -u USERNAME -p > DUMP.sql

Install ia32-libs in Debian Wheezy

2013-06-17T17:06:00.002+02:00

Since Wheezy introduces multiarch, the ia32-libs package in now deprecated. It is now possible to install 32bit packages directly:

dpkg --add-architecture i386 # enable multi-arch
apt-get update
apt-get install libc6:i386 # install base 32bit libraries