Hello!

We threw together a quick & simple script to dynamically update your .htaccess files within apache to add your dynamic IP address to the allow / deny fields.

If you’re looking to password protect an admin area (for example) but your office only has a dynamic IP address, then this script might be handy for you.

Its an extremely simple script that polls your dynamic hostname (if you use no-ip.org or dyndns.org for example) every 15 minutes as a cron job and, if it has changed, updates the .htaccess file

Hopefully it will make your life just a little bit easier

Sample Cron entry :

*/15 * * * * /bin/sh /usr/local/bin/htaccessdynamic.sh yourhostname.dyndns.org /var/www/website.com/public_html/.htaccess > /dev/null 2>&1

And now the script :

#!/bin/bash
# Dynamic IP .htaccess file generator
# Written by Star Dot Hosting
# www.stardothosting.com

dynDomain="$1"
htaccessLoc="$2"

dynIP=$(/usr/bin/dig +short $dynDomain)

echo "dynip: $dynIP"
# verify dynIP resembles an IP
if ! echo -n $dynIP | grep -Eq "[0-9.]+"; then
    exit 1
fi

# if dynIP has changed
if ! cat $htaccessLoc | /bin/grep -q "$dynIP"; then

        # grab the old IP
        oldIP=`cat /usr/local/bin/htold-ip.txt`

        # output .htaccess file
        echo "order deny,allow" > $htaccessLoc 2>&1
        echo "allow from $dynIP" >> $htaccessLoc 2>&1
        echo "allow from x.x.x.x" >> $htaccessLoc 2>&1
        echo "deny from all" >> $htaccessLoc 2>&1

        # save the new ip to remove next time it changes, overwriting previous old IP
        echo $dynIP > /usr/local/bin/htold-ip.txt
fi

Hello there!

We have recently been implementing several different backup strategies for properties that reside on the Amazon cloud platform.

These strategies include scripts that incorporate s3sync and s3fs for offsite or redundant “limitless” backup storage capabilities. One of the more recent strategies we have implemented for several clients is an automated Amazon EBS volume snapshot script that only keeps 7 day retention on all snapshot backups.

The script itself is fairly straightforward, but took several dry-runs in order to fine tune it so that it would reliably create the snapshots, but more importantly would clear out old snapshots older than 7 days.

You can see the for loop for deleting older snapshots. This is done by parsing snapshot dates, converting the dates to a pure numeric value and comparing said numeric value to a “7 days ago” date variable.

Take a look at the script below, hopefully it will be useful to you! There could be more error checking, but that should be fairly easy to do.

#!/bin/bash
# EBS Snapshot volume script
# Written by Star Dot Hosting
# www.stardothosting.com

# Constants
ec2_bin="/opt/aws/bin"
my_cert="/opt/aws/cert.txt"
my_key="/opt/aws/key.txt"
instance_id=`wget -q -O- http://169.254.169.254/latest/meta-data/instance-id`

# Dates
datecheck_7d=`date +%Y-%m-%d --date '7 days ago'`
datecheck_s_7d=`date --date="$datecheck_7d" +%s`

# Get all volume info and copy to temp file
$ec2_bin/ec2-describe-volumes -C $my_cert -K $my_key  --filter "attachment.instance-id=$instance_id" > /tmp/volume_info.txt 2>&1

# Get all snapshot info
$ec2_bin/ec2-describe-snapshots -C $my_cert -K $my_key | grep "$instance_id" > /tmp/snap_info.txt 2>&1

# Loop to remove any snapshots older than 7 days
for obj0 in $(cat /tmp/snap_info.txt)
do

        snapshot_name=`cat /tmp/snap_info.txt | grep "$obj0" | awk '{print $2}'`
        datecheck_old=`cat /tmp/snap_info.txt | grep "$snapshot_name" | awk '{print $5}' | awk -F "T" '{printf "%s\n", $1}'`
        datecheck_s_old=`date --date=$datecheck_old" +%s`

#       echo "snapshot name: $snapshot_name"
#       echo "datecheck 7d : $datecheck_7d"
#       echo "datecheck 7d s : $datecheck_s_7d"
#       echo "datecheck old s: $datecheck_s_old"

        if (( $datecheck_s_old <= $datecheck_s_7d ));
        then
                echo "deleting snapshot $snapshot_name ..."
                $ec2_bin/ec2-delete-snapshot -C $my_cert -K $my_key $snapshot_name
        else
                echo "not deleting snapshot $snapshot_name ..."

        fi

done

# Create snapshot
for volume in $(cat /tmp/volume_info.txt | grep "VOLUME" | awk '{print $2}')
do
        description="`hostname`_backup-`date +%Y-%m-%d`"
        echo "Creating Snapshot for the volume: $volume with description: $description"
        $ec2_bin/ec2-create-snapshot -C $my_cert -K $my_key -d $description $volume
done

Greetings,

We are very proud to announce the full site redesign / relaunch of our toronto based web design partner Shift8 Web.

They have done a fantastic job at redesigning their corporate website and we want all of you to check it out!

Of course if you have a web design project and want a good, reliable and attentive service, drop them a line or request a free quotation

Hello,

I thought I’d update our blog to let everyone know that we offer our extensive managed services catalog on our VPS infrastructure!

If you are looking for high quality managed services from systems administrators with extensive experience, then contact our sales department for a consultation / quotation!

Find below just a SAMPLE of some of the managed services we provide :

Support Services
- 24/7/365 Technical Support (Ticketing system)
- 24/7/365 Alert monitoring on-call rotation pager system
- Online Customer Service Center

Linux Administration
- Installs, reinstalls and updates
- Trouble Shooting and configuration
- Apache configuration, optimization & Setup
- Linux X64 installation, optimization and maintenance
- Security optimizations
- Proactive and reactive maintenance
- Installs, reinstalls and updates
- Trouble Shooting and configuration

Managed MySQL and MSSQL
- MySQL and MS SQL first time install service
- MySQL maintenance and troubleshooting

24/7/365 System Availability
- Availability monitoring (Ping and Synthetic Transactions)
- Performance monitoring of key server parameters
- Graphing and trending

Security Services
- Routine Security Patching
- Routine Security scanning and auditing
- Penetration testing, SQL Injection
- Quarterly security audit reporting
- Dedicated hardware firewalls for all clients

Hardware Maintenance
- Hardware failure alerting and monitoring

Offsite Backups
- Remote off-site backups per 50gb nightly

Hello there,

Occasionally some of our managed services work has us dealing directly with other cloud providers such as Amazon. One of our clients set a requirement to migrate over 5,000 domain’s to Amazon’s Route53 DNS service.

There was little doubt that this could be automated, but since we have never done this massive of a deployment through Amazon’s API directly, we thought it might be interesting to post the process as well as the script through which we managed the import process.

Essentially the script utilizes a master domain name list file as its basis for looping through the import. The master list refers to the bind zone files and imports them into Amazon’s Route53 via the Cli53 tool package.

One final note, the script outputs all completed domain imports into a CSV file with the following format :

domain.com,ns1.nameserver.com,ns2.nameserver.com,ns3.nameserver.com,ns4.nameserver.com

This is because when facilitating the actual nameserver change request, all the nameservers assigned to domains when imported to Route53 are randomly generated, so the script has to keep track of these nameserver/domain associations.

The script isn’t perfect and could benefit from some optimizations and more error checking (it does a lot of error checking already, however), but here it is in its entirety. We hope you will have some use for it!

#!/bin/sh
# Import all zone files into amazon
# Star Dot Hosting 2012
# www.stardothosting.com

currentmonth=`date "+%Y-%m-%d"`

#sanitize input and verify input was given
command=`echo "$1" | sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/'`

if [ -z "$1" ];
then
        echo "AWS ZONE IMPORT"
        echo "---------------"
        echo ""
        echo "Usage : ./importzone.sh file.txt"
        echo ""
        exit 0
fi

echo "zone import log : $currentmonth" > /var/log/importzone.log 2>&1
echo " " >> /var/log/importzone.log 2>&1

for obj0 in $(cat $1);
do
        echo "checking if $obj0 was already migrated ..."
        ls -la /usr/local/zones/$1-zones/complete | grep -w $obj0 >> /dev/null 2>&1
        if [ "$?" -eq 1 ]
        then
        echo "importing $obj0 ..."

        #check if zone file has NS records
        cat /usr/local/zones/$1-zones/$obj0.txt | grep NS >> /dev/null 2>&1
        if [ "$?" -eq 0 ]
        then
                echo "Nameserver exists, continuing..."
        else
                echo "Adding nameserver to record..."
                echo "$obj0. 43201 IN NS ns1.nameserver.com." >> /usr/local/zones/$1-zones/$obj0.txt
        fi

        #check if zone exists
        /usr/local/zones/cli53/bin/cli53 info $obj0 >> /var/log/importzone.log 2>&1
        if [ "$?" -eq 0 ]
        then
                # grab NAMESERVERS
                nameservers=`/usr/local/zones/cli53/bin/cli53 rrlist $obj0 | grep "NS" | awk -F "NS\t" '{printf "%s\n", $2}' | sed 's/.$/g' | sed ':a;N;$!ba;s/\n/,/g'`
   # import zone file
                /usr/local/zones/cli53/bin/cli53 import $obj0 -r -f /usr/local/zones/$1-zones/$obj0.txt
                if [ "$?" -eq 0 ]
                then
                        #move to complete folder
                        mv /usr/local/zones/$1-zones/$obj0.txt /usr/local/zones/$1-zones/complete
                else
                        echo "There was an error in importing the zone file!" >> /var/log/importzone.log
                        exit 1
                fi
        else
                #create on route53
                /usr/local/zones/cli53/bin/cli53 create $obj0 >> /var/log/importzone.log 2>&1
                # grab NAMESERVERS
                nameservers=`/usr/local/zones/cli53/bin/cli53 rrlist $obj0 | grep "NS" | awk -F "NS\t" '{printf "%s\n", $2}' | sed 's/.$/g' | sed ':a;N;$!ba;s/\n/,/g'`
                # import zone file
                /usr/local/zones/cli53/bin/cli53 import $obj0 -r -f /usr/local/zones/$1-zones/$obj0.txt
                if [ "$?" -eq 0 ]
                then
                        #move to complete folder
                        mv /usr/local/zones/$1-zones/$obj0.txt /usr/local/zones/$1-zones/complete
                else
                        echo "There was an error in importing the zone file!" >> /var/log/importzone.log
                        exit 1
                fi
        fi

        # output domain + nameservers in a CSV with format : domain.com,ns1,ns2,ns3,ns4
        echo "$obj0,$nameservers" >> nameserver_registrar_request.txt 2&>1
        else
                echo "Domain already migrated .. !"
        fi
done

Hello!

MySQL replication has been known to easily break, as a result of a large multitude of potential causes.

Sometimes the replication can even break if an erroneous query is executed on the master server.

With all the potential issues that may break replication, we thought it prudent to write an automated check script that can run on a scheduled basis (i.e. every 10-15 minutes), check the Slave status, report on any errors if applicable and attempt to repair replication.

We have built this script to exit and send mail alerts if any step of the checking and repairing process fails or generates an error in itself.

The script also generates a lock file to ensure that no more than one check process can run at any given time. We feel this script could be best used for scenarios for remote MySQL slaves, for example. Adding this extra layer may ensure a more reliable replication.

The repair process is simply 3 MySQL Commands :

stop slave;
reset slave;
slave start;

The above directives assume that you have a master.info with the mysql master server information statically set. No CHANGE MASTER commands have to be executed as a result. Resetting the slave clears the error and resumes replication, and all the queries missed during the time it failed should be queued and applied after it starts again.

Here is the script :

#!/bin/sh
# Slave replication auto recovery and alert
# Star Dot Hosting 2012

currentmonth=`date "+%Y-%m-%d"`
lock_file=/tmp/slave_alert.lck

echo "MySQL Replication Check Script" > /var/log/replication_check.log 2>&1
echo "------------------------------" >> /var/log/replication_check.log 2>&1
echo "$currentmonth" >> /var/log/replication_check.log 2>&1
echo "" >> /var/log/replication_check.log 2>&1

# Check if lock file exists
if [ -f $lock_file ];
then
        echo "Lock file exists! Possible conflict!" >> /var/log/replication_check.log 2>&1
        mail_alert
        exit 1
else
        touch $lock_file
fi

# Fix slave
function fix_replication () {
        mysql -u root --password="XXXXX" -Bse "stop slave" >> /var/log/replication_check.log 2>&1
        if [ "$?" -eq 0 ];
        then
                echo "Stop slave succeeded..." >> /var/log/replication_check.log 2>&1
        else
                echo "Slave recover function failed" >> /var/log/replication_check.log 2>&1
                mail_alert
                exit 1
        fi
        mysql -u root --password="XXXXX" -Bse "reset slave" >> /var/log/replication_check.log 2>&1
        if [ "$?" -eq 0 ];
        then
                echo "Reset slave succeeded..." >> /var/log/replication_check.log 2>&1
        else
                echo "Slave recover function failed" >> /var/log/replication_check.log 2>&1
                mail_alert

                exit 1
        fi
        mysql -u root --password="XXXXX" -Bse "slave start" >> /var/log/replication_check.log 2>&1
        if [ "$?" -eq 0 ];
        then
                echo "Slave start succeeded." >> /var/log/replication_check.log 2>&1
        else
                echo "Slave recover function failed" >> /var/log/replication_check.log 2>&1
                mail_alert
                exit 1
        fi
}

# Alert function
function mail_alert () {
        cat /var/log/replication_check.log | mail -s "Replication check errors!" kkutzko@n49.com
}

# Check if Slave is running properly
Slave_IO_Running=`mysql -u root --password="XXXXX" -Bse "show slave status\G" | grep Slave_IO_Running | awk '{ print $2 }'`
Slave_SQL_Running=`mysql -u root --password="XXXXX" -Bse "show slave status\G" | grep Slave_SQL_Running | awk '{ print $2 }'`
Last_error=`mysql -u root --password="XXXXX" -Bse "show slave status\G" | grep Last_error | awk -F \: '{ print $2 }'`

# If no values are returned, slave is not running
if [ -z $Slave_IO_Running -o -z $Slave_SQL_Running ];
then
        echo "Replication is not configured or you do not have the required access to MySQL"
        exit 1
fi

# If everythings running, remove lockfile if it exists and exit
if [ $Slave_IO_Running == 'Yes' ] && [ $Slave_SQL_Running == 'Yes' ];
then
        rm $lock_file
        echo "Replication slave is running" >> /var/log/replication_check.log 2>&1
        echo "Removed Alert Lock" >> /var/log/replication_check.log 2>&1
elif [ $Slave_SQL_Running == 'No' ] || [ $Slave_IO_Running == 'No' ];
then
        echo "SQL thread not running on server `hostname -s`!" >> /var/log/replication_check.log 2>&1
        echo "Last Error:" $Last_error >> /var/log/replication_check.log 2>&1
        fix_replication
        mail_alert
        rm $lock_file
fi

echo "Script complete!" >> /var/log/replication_check.log 2>&1
exit 0

Hello There,

This update is to highlight the managed hosting services and systems administration services offered by Star Dot Hosting.

We are proud to offer some of the highest standards of managed hosting services in the industry. Here are some of the services we offer :

- 24x7x365 Monitoring, Alerting & Trending
- 24x7x365 ~15minute response time support ticketing system
- Quarterly Security Auditing & Reporting
- Patch management & Maintenance
- Administration, troubleshooting, deployment and code pushes
- Hardware upgrades, deployments, expansion
- Dedicated load balancing
- Dedicated firewalls
- Database replication & clustering
- Dedicated edge content caching

Those are just some of the services we would provide to you! Our quotation process involves rigorous consultations which includes technology assessment and especially a complete understanding of how your scenario can be designed and where improvements can be made, if any.

Feel free to click the link below to visit our managed hosting page, and fill out a quotation request form!

Greetings,

I thought I’d share a quick script in the scope of backing up GIT repositories for the purposes of encrypted and compressed off-site backups.

Unfortunately git does not have an equivalent of svnadmin dump or export, which can conveniently be piped to stdout.

What the above scenario would do is shorten the amount of commands a script would require in order to accomplish a similar task.

Find below a quick bash script that clones a repository, tar/gzip’s it, encrypts the archive and keeps 7 days worth of archive files :

#!/bin/sh
# GIT Backup script
# Written by Star Dot Hosting

todaysdate=`date "+%Y-%m-%d"`

#check command input
if [ -z "$1" ];
then
        echo "GIT BACKUP SCRIPT"
        echo "-----------------"
        echo ""
        echo "Usage : ./backup.sh reponame , i.e. yourdomain.git"
        echo ""
        exit 0
fi

echo "GIT Backup Log: " $currentmonth > /var/log/backup.log
echo -e "----------------------------------------" >> /var/log/backup.log
echo -e "" >> /var/log/backup.log

# Find and remove files older than 7 days
/usr/bin/find /data/git/git-backups -type f -mtime +7 -delete >> /var/log/backup.log 2>&1

# Begin creating working directory to clone into
/bin/mkdir /data/git/git-backup/working >> /var/log/backup.log 2>&1
/usr/bin/git clone /data/git/$1 /data/git/git-backup/working >> /var/log/backup.log 2>&1

# Archive working directory into repo name encrpyted tar file
/bin/tar -czvf - /data/git/git-backup/working | /usr/bin/openssl enc -aes-256-cbc -pass pass:abcABC123 -e | dd of=/data/git/git-backup/$1.tar.gz.enc >> /var/log/backup.log 2>&1

# Remove working directory
/bin/rm -rf /data/git/git-backup/working >> /var/log/backup.log 2>&1

FYI if you ever needed to decrypt the openssl encrypted backup archive, the command below should do the job :

openssl aes-256-cbc -d -pass pass:abcABC123 -in $1.tar.gz.enc -out decrypted.tar.gz

Hello!

We thought it would be important to share this information as it might be interesting to someone who wants to replicate the same VPS across many instances in order to create a farm of web servers (for example).

This uses very similar concepts to our LVM XEN backup post a while back.

Step 1: Take a snapshot of the current VPS

This is simple. Use the lvcreate command with the -s option to create a snapshot of the running VPS. We assume your VPS is 5GB in size, so just replace that with however large your VPS is :

lvcreate -s -L 5GB -n snapshot_name /dev/VolGroup00/running_vps_image

Step 2: Create your new VPS

This is important. You want to create a new vps, assign a MAC and IP address first and let the creation process fully finish. Then shut the VPS down.

Step 3: Copy the snapshot to the new VPS

All you have to do is use the dd command to transfer the snapshot image to the newly created VPS image :

dd if=/dev/VolGroup00/snapshot_name of=/dev/VolGroup00/new_vps_image

All done! Dont forget to remove the snapshot after your done with it :

lvremove -f /dev/VolGroup00/snapshot_name

Start up the new vps and you should have a carbon copy of the previous vps!

Hey there,

Before I update regarding my continued experiences with Xen, KVM, Varnish, FreeBSD and all the other (exciting) things that I do, I thought I’d remind anyone here who is looking for linux vps hosting, virtual hosting or specifically xen vps hosting or even cpanel vps hosting to check out our company website : www.stardothosting.com

If you’re looking for managed hosting services, dedicated hosting or anything along those lines, feel free to fill out our managed hosting quotation form here : www.stardothosting.com/managed-hosting

Thanks!