Unix Admin Corner

Got dual cores/cpu or more? or maybe just more than one disk.

noreply@blogger.com (jamesd_wi) — Tue, 13 Oct 2009 23:22:00 +0000

I came across Thomas Maier-Komor’s home page when I found this cool utility he wrote xjobs, it takes what xargs does and makes it multicore aware so that it starts x number of jobs to get things done faster, which is really cool to me. Before xjobs I found my self always writing little scripts like below, so I can use all my cores and may even take advantage data that is cached and have the operation complete faster.
for I in *.zip ; do unzip $i & done
But this doesn’t allow me to control how many processes get started off at any given time. This is where xjobs comes in. It basically does the same thing but starts the number of processes as the system has processors, but of course this is configurable. Since I didn’t want to repeatedly uncompress files to test this, I decided to throw the fact that my systems have multiple disks and file systems and look at how xjobs can help with IO spreading as well as just the CPU load.
The task that I am testing with is find all copies of “tar” I have in /usr and /opt, traditionally I just use find, but that really isn’t very efficient since it looks at one file system at a time. I ran all of these commands multiple times, to be sure the Solaris has cached all relative metadata so none of the commands should require physical IO to the disks. First on my Dual core AMD box, /opt is on my non-root pool composed of 4x 512GB sata disks, and /usr is on my root pool made of a single 250GB sata disk.
Xjobs arguments used –j sets the number of jobs to use to complete the task and –v 0 tells it to be quient about process starting and ending.
To make find, xjobs friendly I wrote a small wrapper script

qfind:
#!/usr/bin/bash /usr/bin/find $2 -name $1 2> /dev/null

The actual tests:
#xjobs using two processes. $ time echo -e "/usr \\n/opt \n" | xjobs -j 2 -v 0 ~jamesd/xjobs/qfind tar /opt/csw/bin/tar /usr/bin/tar /usr/gnu/bin/tar /usr/sbin/tar real 0m2.159s user 0m0.389s sys 0m2.838s #xjobs using just one process $ time echo -e "/usr \\n/opt \n" | xjobs -j 1 -v 0 ~jamesd/xjobs/qfind tar /usr/bin/tar /usr/gnu/bin/tar /usr/sbin/tar /opt/csw/bin/tar real 0m2.825s user 0m0.375s sys 0m2.443s #the old fashion way, using find $ time /usr/bin/find /usr /opt -name 'tar' 2> /dev/null /usr/bin/tar /usr/gnu/bin/tar /usr/sbin/tar /opt/csw/bin/tar real 0m3.006s user 0m0.365s sys 0m2.633s

Now what surprised me the most, is that old fashioned find was slower than even xjobs when I limited it to just one process. Even with the extra work to spawn the xjob with one thread is a lot greater because it has to echo the two directories into xjobs and has to setup a pipe as well, and then run a script two times that spawns two copies of bash where find is just one command doing what it was designed to do.

Okay now if you have only one cpu but multiple drives does his help you? Let’s see. For this test I use my Blade 1500, single cpu, and 1 ide drive formatted with UFS for /usr and /opt on a zpool raidz composed of 3x 18GB 10k rpm drives.


frankenstein:xjobs-20091012$ time /usr/bin/find /usr /opt -name 'tar' 2> /dev/null\                /usr/bin/tar
/usr/sbin/tar
/usr/sfw/share/doc/ant/manual/api/org/apache/tools/tar
/opt/csw/bin/tar

real    0m5.827s
user    0m0.729s
sys     0m3.823s

frankenstein:xjobs-20091012$ time echo -e "/usr\n/opt"  | /usr/local/bin/xjobs -v 0 -j 1  ./qfind tar
/usr/bin/tar
/usr/sbin/tar
/usr/sfw/share/doc/ant/manual/api/org/apache/tools/tar
/opt/csw/bin/tar

real    0m5.969s
user    0m0.736s
sys     0m4.034s

frankenstein:xjobs-20091012$ time echo -e "/usr\n/opt"  | /usr/local/bin/xjobs -v 0 -j 2  ./qfind tar
/opt/csw/bin/tar
/usr/bin/tar
/usr/sbin/tar
/usr/sfw/share/doc/ant/manual/api/org/apache/tools/tar

real    0m5.264s
user    0m0.726s
sys     0m3.604s

The results show that xjobs was about .6 second or about a 10% gain by using xjobs, Since we didn’t have to access physical disks for any of these tests I think we can assume the results would be more impressive if the cache was cold.

Check out Thomas’s home page he has some other cool utilities, like multi-threaded tar patch and sysstat.

bored?

noreply@blogger.com (jamesd_wi) — Mon, 12 Oct 2009 15:56:00 +0000

watch Oracle Open World live

ZFS ARC summary in web language

noreply@blogger.com (jamesd_wi) — Wed, 16 Sep 2009 22:05:00 +0000

Lately I have been working on system monitoring scripts and decided to take Ben Rockwood’s ZFS ARC summary script and convert it to output in html format. This is just the first step I will be working on a configuration file to use MRTG to track this information s well. You can get the current release of this script at zfs_status_web.txt

Sample output, the script looks nicer because it doesn't conflict with my blog's CSS.

System Memory
Physical RAM	6006 MB
Free Memory	1062 MB
LotsFree	93 MB

ARC Size
Current Size	2115 MB (arcsize)
Target Size (Adaptive)	3298 MB (c)
Min Size (Hard Limit)	622 MB (zfs_arc_min)
Max Size (Hard Limit:	4982 MB (zfs_arc_max)

ARC Size Breakdown
Most Recently Used Cache Size	57% 1910 MB (p)
Most Frequently Used Cache Size	42% 1387 MB (c-p)

Tracking down sunray's on ciscos

noreply@blogger.com (jamesd_wi) — Fri, 11 Sep 2009 09:29:00 +0000

After installing mrtg/rrdtool on OpenSolaris Tracking down I wanted to see how much bandwidth my Sun Rays were using so I needed to know which port the sunray were plugged into on my Ciscos

Normally the last 2 lines is enough but ping is necessary at least on my Cisco 2950's and sunray's


ping [ipaddress]
show arp | include [ipaddress]
show mac-address-table address [mac address]

based on information I found at: http://www.velocityreviews.com/forums/t37560-how-to-track-down-whos-on-what-port-on-an-ios-6509.html

and for those that are interested here is the bandwidth usage of the sunray, its sitting idle and it uses very little bandwidth.

Happy B-Day to me!

noreply@blogger.com (jamesd_wi) — Tue, 18 Aug 2009 00:09:00 +0000

It's my Birthday, wasn't it just my birthday 365 days ago? sure feels like it. It's going okay, got news that I have a 2nd interview lined up for a job I'm trying for. So seems like a good day so far.

Are you old enough to get the humor in this?

noreply@blogger.com (jamesd_wi) — Tue, 04 Aug 2009 15:57:00 +0000

My friend Melanie posted this really funny skit, on her blog check it out.

Another funny story you don't have to old to get it, just married.

Interesting Links and Tools

noreply@blogger.com (jamesd_wi) — Tue, 04 Aug 2009 14:28:00 +0000

Unix isn’t going away, and 3 that it may.
Found it really interesting that 1/3 of all server sales by dollar are Unix Servers.

Freebase Parallax
Cool search engine that changes the way you search from one topic to one page, a more to a more relationship type search. watch the 10 minute video on the home page.

VirtualBox 3.02 released

One of the best free virtualization tools, new version, supports SMP in virtuals now and many other features, Virtual box keeps getting better and better.

FoxTab
One of the coolest Firefox plugins gives you a 3D w to choose which tab you want, works great when you have browser windows open and each has more than 10 tabs. Gives you a way to show off that fancy 3D video card, with out loading a game.

Sync Back
Need to backup your home windows boxes? Syncback is a great way to do it, backs up to directories, windows shares, even my favorite FTP

WinDirStat
Gives you a way to find all those large that you know they are somewhere, but not sure where, and allows you to see what is really using all your hard disk space.

2 AM coding session almost goes bad

noreply@blogger.com (jamesd_wi) — Mon, 03 Aug 2009 17:03:00 +0000

Well it figures, 2am the most wondrous and dangerous time for coders and sys-administrators alike, it’s when we come up with the best ideas fix bugs we fail to notice at any other time, perhaps even bugs we introduced in 2 AM coding sessions, it’s also when we manage to do the most damage to our projects.

Back story:

I have a home intranet that pulls and displays RSS and Atom feeds as well from the internet, as well as prints lists of Hyperlinks these are all stored in MySQL tables and they are classified by content type, General, Weather, Geek, and Sports. A function generates the SQL query to display what each family member is interested in. When I started it all works beautifully.

Tonight:

I got tired of seeing the RSS feed from WhiteHouse.gov, I think I could delete it, but nah it would be better to add a way to simply disable it for now since I’m burned out on politics at the moment and perhaps re-enable it later when the next election comes around. Sounds easy enough, right? I fire up PhpMyAdmin and make the necessary changes to the tables, I add a field to each of the tables, called “enabled” set the default value to 1 being that there are only 3 tables that are needed and they are rather small tables, less than 300 rows it all goes smoothly.

Now on to changing the PHP code, I fire up NetBeans 6.5.1 on my desktop, that has my Intranet site in a project and go about making the simple change to make the query generation function only display enabled rows. As NetBeans now understands PHP code since somewhere after 6.5. I can make my change and have reasonable confidence that my change will run.

$output .= ‘and `enabled`= 1 and ( ‘ ;

I click the little green triangle (play button) on the toolbar that tells NetBeans to run the project, so first NetBeans uses ftp to upload the changed file up to my web server and starts Firefox to display the intranet. As expected it goes fine. So I commit my changes to the source repository (I use subversion but most of the other ones work as well), yes also handled painlessly from inside NetBeans.

Now here is where I should have gone to bed, Do I? Of course not, I think maybe at some point I will want to display disabled the records I should make it possible as well. The new code to do this looks simple as well, but did I mention that it’s around 2am?

$output .= "and `enabled`= $this->prv_enabled and ( " ;

Because I now have a variable that must be replaced in the string I need to change from ‘(tick) to quotation marks, which I remember to do, then add a small function to change the newly created private variable. I also modified the constructor function to set the new variable. Add comments and fix a few other small things I see along the way. Now time to test, click on run project, NetBeans once again uploads the new version of the file, and tells Firefox to display to my home page, to my dismay, this time Firefox Renders my code to a blank page.

I think it’s not the end of the world. I flip back to NetBeans and take another look at my code. NetBeans shows the code to be good. Still no hint to what has happened. Just before deciding to hit undo all my changes I made one by one, I remember that NetBeans has local reversion history built in, I click on the Versioning menu, and choose local history Local history and up comes a window with a side by side display of the difference in each code change look at the first change that made today and all is well, then I click on the second change still good. I continue through all the changes twice, then I notice a red dash (this mark indicates that there is an error further down in the file.) had appeared on the right hand of the display, further down the window in one of the middle change sets. I then find a change I don’t remember making and hadn’t noticed in my previous reviews, did I mention it was after 2am? So I scroll down to that change, and all the code in a few lines further down were all marked as errors so I knew I had found the bad line, now as it was after 2:30, I was thankful to see a red X by the change, clicking it worked as expected and reverted the change, I didn’t have to figure out what changed, a simple click and it reverted the line by back. And all the bad code notifications went away, since this worked so well. I thought okay let’s see if what the problem was, and went to the edit menu and clicked undo, and the change went back in and I studied the line in question and found that I somehow I ended up changing a ‘ (tick) to the a quotation in this part of the file as well, which is strange since I did the first change by hand, but NetBeans made it relatively easy to find even in my sleep deprived state.

An hour worth of work saved and multiple hours of debugging the issue of a random change averted thanks to NetBeans, I fear how much work it would have been to find this bug in vi.

Happy Fathers Day

noreply@blogger.com (jamesd_wi) — Sun, 21 Jun 2009 21:18:00 +0000

Happy Father day to all the Fathers out there. Have a great day.

serve weather

noreply@blogger.com (jamesd_wi) — Fri, 19 Jun 2009 06:11:00 +0000

its hard to sleep when there is this many bright colors on the weather map, and they are expecting hail.

those in california aren't used to seeing this many colors on the radar map all the colors are usually associated with loud noise from thunder, lightning, and possibly hail, which is expected in the next few minutes.

Roy Wood has a really interesting number

noreply@blogger.com (jamesd_wi) — Thu, 18 Jun 2009 20:42:00 +0000

Just read this really interesting blog entry about an interesting number he recived in an email.

Its really cool. Check it out.

SXCE b115, not ready for prime time

noreply@blogger.com (jamesd_wi) — Wed, 10 Jun 2009 05:57:00 +0000

Decided to upgrade my b86 SXCE box to something more recent. But what I got was lots of broken bits instead. Wanted to play with the latest bits, Cross bow, DTrace extensions, etc.

My AMD box was perfectly stable and well performing on b86, I expected a smooth upgrade as usual, but I ran into a lot of headaches.

After what seemed like a nice smooth install, I booted it for the first time, using xVM mode, and on its way to boot up it crashed. Okay i don't mind a few rough edges, and booted back into normal mode with no xVM, which proceeded as expected.

Next step was to do a zpool import and see if it could find my data zpool named "tank", the system found 3 of the 4 drives that make up tank, okay that is scary since it found 3 of the 4 drives, some of the drives were on the motherboard, it found 1 of them as well as the root drive that is in a seperate pool, and it found 2 drives on a pci sata controller, i reseated the power cable and sata cables and tried again, still no luck. I moved the drive that wasn't comping up to a different port on the mother board, still no luck, finally giving up i moved it to the add on card, which is on pci-32bit and it worked. Okay I can breath a bit easier now at least it seems all my data. About 1TB of data, that is mostly not backed up , how do home users backup 1TB of data? not easily.. you keep backups of the important stuff and keep snapshots.

Well I though okay my data is in tact now to get the machine back to doing its job.

Restore passwd shadow group named.conf /etc/bind files to the place and restart dns okay that was easy thanks to the copies of those files that sit on my home directory on the pool i just imported.

Now to get Sun Ray services back up and running. After installing and following the directions, i found that the SRSS still is not happy with the changes to the sockets made back before b107 Bug ID#6799655 even though the bug is labeled fix it still requires the work around and a reboot. So another hour of debugging down.

Now on to tarantella/SGD I re-installed it and run tarantella start and saw mysterious "Segment faults" in the start scripts, not sure what is going on haven't had time to debug them fully, but at least it runs somewhat.

Now to move on to coolstack, I reinstalled it even though all the binaries were imported with my data pool I did the reinstall because i realized that there is no way to reimport the SMF manifests for it, but still this shouldn't be a that big of an issue. That is what I thought anyway. After I did the install that appeared to go with out issues, I attempted to start up mysql, and it failed, I finally tracked down a log file with the error in logged in it, libssl.so.0.97 wasn't being found. Apparently /opt/coolstack/lib/libssl.* symlinked it from /usr/sfw/lib where openssl is no longer there. Okay I tried using some other copies of the libs i had from Blastwave.org still no go.

Webstack 1.4 is supposedly its replacement, I went and downloaded the packages and tried to run ./install, like the README said, it required me to figure out which modules I wanted, after much trial and error all I got was "WS015 This platform is not supported" after verifying I downloaded the correct architecture, and Solaris version I ended up installing the packages manually instead of the nice installer. After 6 hours of debugging, I'm still quite a bit of away from having the machine back to where it was this morning, still need to configure apache, and php.

What a pain what happened to binary compatibility, all this happened from the OS that is supposed to support binaries compiled 10 years ago, as of now it can't even handle binaries compiled a year ago. Hope they fix these issues.

update:
Given up on b115, because twice now while trying to get squid the included one to work I have gotten the system into a crash loop, activate it and it crashes the system and then i crashes the system as it comes back up, first I thought it was my fault in that i did something wrong, so I started over. But nope, twice now. So I'm currently downloading Sol10U7 and going to give it a try.

timf:
asked the question when was the last time xvm worked, and it worked fine in b86 that the system was running before i did this upgrade. Everything worked fine in that relaease. Zero issues, mostly just wanted to upgrade to keep current.

System Config
asus m2a-vm motherboard
amd x5200+ cpu ( dual core 2.5ghz 1MB l2 cache per core)
250 GB sata drive on board. (root pool)
4x 500GB sata drives (data pool) spread betwen the on board controller and a realtek add on board in AHCI mode.
6GB of ram

Gmail + Laptop Tip

noreply@blogger.com (jamesd_wi) — Fri, 05 Jun 2009 17:40:00 +0000

I am an owner of a 16” LCD laptop, while the screen is nice in most aspects, but they have a low resolution of 1366 by 768, which is low by today’s standards especially if you’re a geek like me and have at least a 19” screen on your desktop. The one application, well webapp, that I have a problem with on this system is GMAIL with the multiple inboxes (available from the Gmail lab) enabled. I have found a way to make it usable, everyone probably already knows it as I did, but just didn’t make the connection on how useful it would be. Press Ctrl-minus to reduce the size of the font used in the browser. And you can actually get enough text in each of the inboxes.

Two movies one day

noreply@blogger.com (jamesd_wi) — Wed, 03 Jun 2009 03:55:00 +0000

Saw “UP 3D” really good movie, not up to Pixar’s normal standard, but Still worth going too. It does suck that most theaters are charging an up charge despite their normal high prices, compounded with expensive food prices.
Hung around and saw Star Trek, really funny, and provided a history to the characters and the scenery didn’t seem too far from the original’s TV show’s décor which I was afraid of.

So two good movies to go and see, just remember always eat before the film, even if it means hitting McDonalds before hand, so you can get out of theater for less than $20 a person.

Google Wave

noreply@blogger.com (jamesd_wi) — Sun, 31 May 2009 03:08:00 +0000

checked out and signed up wave.google.com looks really cool can't wait to try it out my self

Technology older than I am can surf the Web

noreply@blogger.com (jamesd_wi) — Fri, 29 May 2009 18:09:00 +0000

Here is an interesting video and webpage on using a 1964 300 baud modem to get on the web.

http://hackaday.com/2009/05/27/1964-300baud-modem-surfs-the-web/#comments

Solve it all – during a BBQ

noreply@blogger.com (jamesd_wi) — Mon, 25 May 2009 18:14:00 +0000

Having spent the weekend with extended family, you always run into the questions and suggestion, you know the type, oh cousin blah needs someone to help with his company, he does websites, you do computer stuff, it should be a match made in heaven they all think. Then you get to break it to them sorry I don’t have any artistic ability so I won’t be any help in the cousins business. But he has a lot of work, he has 40 websites on his server, sure you can help. They just don’t get it; I’m the one that maintains 250 servers and designs future deployments that are then sliced and diced into servers/zones/containers/virtuals that are then sold to the hundreds or even thousands of smaller customers like the cousin’s web hosting business. Yes I can write programs in a dozen or more programming languages, but do I want to do it 40 hours a week, do I want to do it on a daily basis? Hell no. I struggled to create a few web pages for the home intranet that keeps a family shopping list and pulls in a couple RSS feeds. But that is the extent of my hobby, I’m not good enough to do it for a living, nor do I want to.

Of course this is followed up by requests for virus removals, solve disk space issues, what computer should I buy, should I retire my Intel P3 600 machine, and the list goes on, they all revolve around Windows. They all think you can solve everyone problems. They just don’t get it, yes I use and manage computers for a living, I know windows, and do I want to solve all their problems? Nope. At a position I held a couple years ago with a fortune 500 company, I didn’t even have Administrator rights on the company issued laptop, it was great, when the issues came up and they did I got to call the corp help desk and they solved them all. I could have had Administrator rights if I filled out one form, but I didn’t want them. I had root on 175 Solaris boxes and a dozen Red Hat boxes, that was enough.

For many of the problems I will be hitting Google. Then comes the others that think because I’m between contracts they are going help me out and actually pay. But again they don’t get it, as a UNIX admin taking money from them sets up a bad pattern that I just don’t want to start. It all starts out nice and innocently, I do some small job for them at what to them seems like decent money, but compared to what UNIX administrators earn, it’s nothing. I set up my life to be able to cope with the loss of my regular income for periods of time when I don’t have my typical paycheck. But the family just doesn’t get it so it leads into them wanting to help me and get there computer problems solved but once the pattern starts it leads to a few more jobs that pay next to nothing when they give my contact information to a few friends, etc all with good intentions of trying to help out. So you end up making next to nothing, doing work you don’t like and frankly are not very good at, and that is the best scenario by the end of the arrangement I end up putting more time into solving their problems than they or even I dreamed it would take, and I end up making even less money because how can I charge them for 100 hours for what they thought would take 2 hours, and I thought would take 5 hours max and quoted for 10 hours just to be on the safe side. In the past before I learned my lesson I ended up making less than a $1 an hour for hundreds of hours of work, sure it was supposed to be a stupid database application or automated form that I thought would take 5 or 6 hours that exploded into something way bigger with lots and lots of tiny details that required more and more time.

I’m being followed

noreply@blogger.com (jamesd_wi) — Mon, 25 May 2009 11:00:00 +0000

Yesterday I finally joined Twitter, yes I may be the last in the Social network age to join twitter, yes from time to time, I feel like I’m in information overload. But it may be a better way to follow friends. Facebook seems okay but I seem to be a constant flow or requests to try some strange application or your friend has gotten a new farm animal, find your redneck name, etc. perhaps twitter will be better just giving short snippets of what people are doing and searching for content.

Much to my surprise I already have 6 followers and it’s a holiday weekend.

If anyone else wants to follow me here are my "social network profiles, feel free to follow me or add me as a friend, and I will do the same. Perhaps later I will even get back to Solaris/UNIX related posts.

Facebook:

http://www.facebook.com/profile.php?id=557067672

LinkedIn:

http://www.linkedin.com/in/jamesdwisconsin

Twitter:

jamesdwi

Back Home - Avoid Thrifty Car Rentals.

noreply@blogger.com (jamesd_wi) — Fri, 22 May 2009 18:12:00 +0000

I got back yesterday, had a great trip, thanks to all those that sent condolences and prayers they were much appreciated.

Trip was great got to spend a lot of time with my daughter and connecting with my family.

The one bad spot was caused by Thrifty rent a car, at Orange County Airport. Avoid them at all costs they don't have a heart and refuse to work with people no wonder they have to charge less to get people to rent from them, I was trying to pick up the rent a car that was already paid for, and they ran a credit check but a glitch in there system caused the check to fail, we even had Expedia, who went above and beyond for us, called and they refused to pick up there phone, only when we gave them our cellphone would they talk to expedia. Still they refused to help us causing us a delay of a 1/2 an hour. We ended up going to Budget who ran the same credit check and rented us a car.

RIP Mom

noreply@blogger.com (jamesd_wi) — Fri, 15 May 2009 14:14:00 +0000

Well my Mom took a turn for the worse, and died at 3:43pm on Wednesday, while I was still in the Air thanks to my younger brother calling me on Mother's day from the hospital, I was able to talk to Mom and say "I Love you", so am I am okay with not being any to see her before she passed.

Heading to the OC

noreply@blogger.com (jamesd_wi) — Wed, 13 May 2009 14:25:00 +0000

Flying to Orange County, CA today, wish it was for a better reason, my Mom is in the hospital.

But is currently stable, but they are aren't expecting her to make it. :-(

Open Office 3.1 is released

noreply@blogger.com (jamesd_wi) — Thu, 07 May 2009 23:59:00 +0000

I have been using OpenOffice for home stuff for years and it keeps getting better, perhaps it will get even more use by others because of the economy to excellerate acceptance. I'm downloading the latest version now to give it a try it appears to have some nice new features.

Download it here

Simple tests of my home ZFS server

noreply@blogger.com (jamesd_wi) — Wed, 06 May 2009 15:25:00 +0000

Some simple tests on my home ZFS server, it's running an older version of SXCE, only b96, current is b113, But thought my readers would like to see what ZFS can do on poormans hardware, Dual core x5200+ with the 1MB per core l2 cache, asus m2a-vm motherboard 6GB of ram and 4x sata 500GB drives...

amd:test# psrinfo -v

Status of virtual processor 0 as of: 05/06/2009 10:28:00

on-line since 05/05/2009 15:10:02.

The i386 processor operates at 2600 MHz,

and has an i387 compatible floating point processor.

Status of virtual processor 1 as of: 05/06/2009 10:28:00

on-line since 05/05/2009 15:10:09.

The i386 processor operates at 2600 MHz,

and has an i387 compatible floating point processor.

pool: tank

state: ONLINE

scrub: none requested

config:

NAME STATE READ WRITE CKSUM

tank ONLINE 0 0 0

raidz1 ONLINE 0 0 0

c0t3d0 ONLINE 0 0 0

c3t3d0 ONLINE 0 0 0

c0t2d0 ONLINE 0 0 0

c3t0d0 ONLINE 0 0 0

time (mkfile 100g test ; rm test)

real 26m16.079s

user 0m1.678s

sys 1m47.347s

amd:test# zfs set compression=on tank/test ; time (mkfile 100g test ; rm test)

real 12m19.286s

user 0m1.136s

sys 0m57.637s

Automate your home server

noreply@blogger.com (jamesd_wi) — Tue, 05 May 2009 14:26:00 +0000

At work we all think nothing about automating tasks, want a daily report put in a crontab, backup some files create a netbackup job, remove old log files another script, sure this is all great, but when was the last time you did this at home?

Most of us at home we take backups of our home directories when we think about it, if we are really bored or make a large change to our database and we are worried about it we take a dump of our database. But what about the little changes we make all the time or a link to our home intranet sure its not important but do you really want to recreate it?

I just yesterday did an dist-upgrade to my Mythtv box, and it didn’t go well, it has only been 3 years since I installed it, and never thought of doing a dist-upgrade, but it wasn’t broken, but of course it broke but it’s really my fault, I booted up a live disk, and backed up my home directories and /etc , and proceeded to try to do an upgrade, but it failed, Of course the rebuild process went well. When I get around to restoring my intranet, I remember oops forgot to save a copy of the data files. Do I blame Linux? No It’s my fault, it told me to backup the files before trying to upgrade, and I was really just a bad hard drive away from failure anyway, since it’s a home box and /var was sitting on a non mirrored disk. Well I am now restoring the data from my last database dump. Oh well it was from February, probably a couple dozen links are missing and the current shopping list.

So the moral of our story, automate stuff like backing up our home directories to a second machine, we harddisk is cheap we all have at least 2 machines, even if we just backup the data to another desktop, it only takes a few commands in a script to copy them over, dumping Mysql databases to an SQL file and scp it to another machine is easy enough and put these tasks to your crontab.

I am now going to move the database to my Solaris/ZFS fileserver and putting the data on ZFS so I can take snapshots and survive the loss of a disk and setting up automated snapshots the question now is do I create a script to shutdown Mysql take a daily snapshot, and then restart it just to be paranoid about the data files being out of corrupted during a database writes.

Wonders if any of my readers have "Hamthrax"

noreply@blogger.com (jamesd_wi) — Sun, 03 May 2009 03:40:00 +0000

So far me and my family are all swine flu free... And yes we all still eat pork products, since its not how its spread. Comment if you or your family has it.