My SysAd Blog -- Unix

How to Exclude a Directory for TAR

2012-02-05T04:00:00.000-08:00

I am doing a few aesthetic upgrades, so naturally I want to backup some of my website files. However, I do not want everything, i.e. transient files such images. I tarred up the website's primary directory but excluded its images sub-directory.

Frankly speaking, I did not want to waste time and bandwidth downloading images. Here is the syntax to exclude a directory.

# tar cvfp mytarball.tar /mypath/Example.com_DIR --exclude=/mypath/Example.com_DIR/images

Tar everything in the current directory but exclude two files

# tar cvpf mytar.tar * --exclude=index.html --exclude=myimage.png

Restore MySQL Database via Command Line

2011-11-04T03:07:00.000-07:00

This is mostly a note to myself. It is not really a tip.

Restoring a mysql database from the command line.

[root@server]# ls -la

total 326940

drwx------ 3 user1 usergrp 4096 Oct 29 12:10 .

drwxr-xr-x 3 root root 4096 Nov 4 05:22 ..

-rw-r--r-- 1 user1 usergrp 1239136 Oct 29 12:10 mysql_db.sql

[root@server]# pwd

/backups/db/backup

[root@server]# mysql --user=mysql_user --password=somepasswd mysql_backup < mysql_db.sql

Sun StorageTek 6140 Cache

2011-10-29T08:27:00.001-07:00

We have a Sun StorageTek 6140 Array attached to a couple NAS heads. We share out its file systems from the NAS. But it seemed the NAS was creating bottlenecks and hanging NFS. As usual, it turned out to be something else.

We observed slowness during high or bursty I/O operations. Short story...It was the 6140's cache. Make sure your cache batteries are good to go and up to date. They have a 3 year shelf life. It will save you a lot of frustration in the long-run.

HTML Symbol Entities issue on Blogger

2011-06-17T19:26:00.000-07:00

I have received a few requests on how to deal with the HTML Symbol Entities issue on Blogger. There really isn't a problem because you can just click the box below for your post and your character(s) will display correctly.

ERROR

Your HTML cannot be accepted: Tag is broken: xxxxx

Stop showing HTML errors for the body of this post

However, here are some good references for HTML Symbol Entities

http://www.w3schools.com/tags/ref_symbols.asp

http://www.ascii.cl/htmlcodes.htm

http://www.webstandards.org/learn/reference/charts/entities/symbol_entities/

http://webdesignerwall.com/tutorials/using-html-symbol-entities

http://www.entitycode.com/

Log an FTP Session with a Filter

2011-06-17T18:05:00.001-07:00

A couple colleagues and I were looking for an effective way to tersely log the activities of an FTP effort. All we wanted was the file name and throughput rate. If you are Ftping hundreds or thousands of files, the first one (below) could be quite verbose, especially if you have a security banner notice. Here are a couple ideas that we considered. We went with the alternative (a lot less text-based overhead).

#!/bin/sh

ftp -nv some_host ›› /tmp/myftplog ‹‹EOFTP

user some_user some_password

cd /some_dir/some_dir1/some_dir2

binary

put some_image.jpg

EOFTP

and its log...

$ more /tmp/myftplog

Connected to some_host.

220 ProFTPD x.x.x Server ready.

331 Password required for some_user

230 User some_user logged in

250 CWD command successful

200 Type set to I

200 PORT command successful

150 Opening BINARY mode data connection for some_image.jpg

226 Transfer complete

ftp: 26753 bytes sent in 0.00Seconds 26753000.00Kbytes/sec.

221 Goodbye.

###ALTERNATIVE###

#!/bin/sh

ftp -n some_host ›› /tmp/myftplog ‹‹EOFTP

user some_user some_password

cd /some_dir/some_dir1/some_dir2

binary

verbose

put some_image.jpg

EOFTP

and its log...

$ more /tmp/myftplog

Verbose mode On .

200 PORT command successful

150 Opening BINARY mode data connection for some_image.jpg

226 Transfer complete

ftp: 26753 bytes sent in 0.00Seconds 26753000.00Kbytes/sec.

221 Goodbye.

Display it...

$ tail -f /tmp/myftplog

The Importance of Library Level Integration in Open Source Environment

2011-03-19T01:03:00.000-07:00

By Jonathan Flack

Open Systems Engineer, Likewise Software

As an Open Systems Engineer at Likewise Software I routinely encounter unique customer problems. Most of these problems require a clever solution which tends to be specific to the customer site.

Earlier today, however, I got a query from an Ubuntu user that really made me consider how important library level integration by application developers can be when administering an open source environment. A lot of vendors out there provide free solutions without being open source, and a lot of these solutions solve one problem or another in your environment, but it's rare to come across a piece of software that really cleanly integrates into your deployed OS in a way that provides a multitude of very clean solutions to everyday problems.

This user had a simple Linux application for which he needed to verify user authentication by simply passing the username and password to generate a pass/fail response. The application was neither pam/nsswitch integrated or GSS-enabled (built with GSSAPI support).

The user's Linux environment was already authenticating to Active Directory using Likewise Open, our open source agent which allows user authentication against Microsoft's Active Directory. As a result, I know the user would have access to a fully functional Kerberos KDC.

Anyone who has integrated Kerberos in a large environment before knows configuration can be a very painful experience. Likewise Open however, properly configures the client side kerberos and server side DNS automatically during domainjoin, so this little trip down nightmare alley is completely avoided. We are also assured that the configuration is correct.

So the only thing left to do is leverage kinit to authenticate the user against AD's KDC.

The specifics of the code in his application are not relevant, but you can see a simple example of this working using kinit from the command line:

Positive Authentication -

[root ~]# kinit joesmith@MYDOMAIN.COM

Password for joesmith@MYDOMAIN.COM: ************ <== Correct Password

[root ~]# echo $?

Negative Authentication -

[root ~]# kinit joesmith@MYDOMAIN.COM

Password for joesmith@MYDOMAIN.COM: *^^^***^*** <== Incorrect Password

[root ~]# echo $?

This shows how tools which properly integrate across multiple subsystems ease the workload on already overtasked admins. It also clarifies the importance of open-standards based integration. By integrating directly with pam, nsswitch and Kerberos, Likewise-Open allowed him to leverage existing machine configurations and his AD authentication to easily solve a problem unique to his custom application.

One of the great side benefits is knowing that any other kerberized application, like Firefox or OpenSSH, will authenticate using the user's AD credentials as well.

This kind of elegant solution has been rare in large heterogeneous environments, but more and more open source developers are striving to achieve a high level of standards based integration without forcing closed source components on their end users. This is one of the clear long-lasting benefits of the open source movement and I'm very pleased to see more companies making this as part of their products' value proposition.

The Khan Academy is Teaching the Online World

2010-09-30T07:30:00.000-07:00

Here is a great learning site to help brush up on your algebra, geometry, trigonometry, and calculus skills. You can also find courses for economics, finance, history and science. This growing site is called the Khan Academy. The instructor, Salman Khan, is helping current students and former students gain or regain their footing with these topics. He basically provides a 10 minute block of instruction (due to youtube’s time limit) on a given topic, which is short enough to keep your attention span. His site offers over 1800 different courses.

Salman gave up a successful career in finance to share his vast knowledge with the world for FREE. From what I can see, he’s a terrific math teacher.

The Khan Academy is a not-for-profit 501(c)(3) with the mission of providing a world-class education to anyone, anywhere.

Check out the Khan Academy and its amazing story.

And by the way, the Academy was recently selected by Google's project 10^100

Use Redirect Symbol to Zero File

2010-07-02T05:37:00.001-07:00

I have mentioned this topic in an earlier post - zero'ing logfiles with cat /dev/null > logfile. But there is another easy way to zero a logfile in Bourne or Korn shell. It probably works in Bash and Zsh, too. I have to check. Here is an example below.

Note: I used the gts to represent the "greater than" or redirect symbol because blogger complained about using that particular character. Update: themadengineer reminded me on the use of "greater than" html syntax. Thanks.

Bourne Shell at the command prompt

# > mylogfile

Korn Shell at the command prompt

# > mylogfile

Earlier post: Don't remove those log files - zero them

Run the Windows Explorer GUI

2010-06-28T05:44:00.000-07:00

Here are a few easy ways to invoke the Windows Explorer GUI, which displays certain parts of the directory/folder tree depending on given syntax. I also threw in a few other miscellaneous items.

1. Select Run
Then type "."
This action will invoke the C:\Documents and Settings\mysysad

2. Select Run
Then type ".."
This action will invoke a window that displays C:\Documents and Settings

3. Select Run
Then type "..."
This action will invoke a window that displays My Computer

Other misc Run exercises

Select Run
Type "explorer" - same as Type "My Documents"
Type "explorer c:"
Type "explorer desktop"
Type "explorer ."
Type "explorer .."

Select Run
Type "control panel"

Select Run
Type "recent"

Select Run
Type "calc"

Select Run
Type "notepad"

Select Run
Type "wordpad"

Select Run
Type "ping MySysAdDOTCOM"

Select Run
Type "c:" or "d:" or "a:"

Select Run
Type "freecell"
Type "sol"
Type "mshearts"
Type "spider"
Type "winmine"

Select Run
Type "mspaint"

Select Run
Type "nslookup"

Select Run
Type "cmd"

Retrieve Information on LDAP User

2010-05-06T07:35:00.000-07:00

Here are a couple easy methods to get information from ldap for a particular user. Friend and I were talking about it yesterday because we had a typo in someone's login directory string (missing a character). He showed me a fancy one-liner but all I needed was a username, login directory, and its corresponding UID.

# ldaplist -l password username

# getent passwd username

# logins -x | grep username

The getent command is also handy to retrieve host and group information. It consults what databases are listed in the /etc/nsswitch.conf file.

# getent hosts hostname

# getent group 100

$HOME/.TTauthority not writable, auth entry not added

2010-04-27T05:13:00.000-07:00

After being absent from the work site for an extended period, I had a little problem with the dt session. I was being blocked a dt session (automatically logged me out--tried 3x). I observed Tooltalk/dt session errors on the screen.

But I also observed others logging into the same box without incident. Intuitively, I suspected something was wrong with my environment, so I logged in via a remote terminal. Once in the box, I reviewed the /var/adm/messages file and noticed /usr/dt/bin/ttsession errors were associated with my username. The messages file also contained this telling message - $HOME/.TTauthority not writable, auth entry not added.

Hmmm…ownership was not mine. I changed the ownership to yours truly and everything worked like a champ.

Microsoft Security Advisory --- F1 Key on Web

2010-03-03T03:31:00.000-08:00

A friend of mine from Japan emailed me this interesting article and thought I would share it.

Microsoft has issued a security advisory warning users NOT to press the F1 key when prompted by a website. This vulnerability could allow a hacker to hijack a computer. This vulnerability affects all versions of Internet Explorer on computers running Windows 2000 and Windows XP operating systems. More information can be found at

http://www.computerworld.com/s/article/9164038/Microsoft_Don_t_press_F1_key_in_Windows_XP

DO NOT press the F1 key if prompted by a website. Doing so could allow a hacker to hijack your computer!

Import and Enable the NFS Service

2010-02-27T22:15:00.000-08:00

Last week we had a little issue with mounting directories from a new Solaris 10 server. It was noted the nfs service was not running on it. We had to import and then enable the service. After that, we were able to mount the required directories from the server.

# svcs -a | grep nfs
# cd /var/svc/manifest/network
# svccfg -v import nfs/server.xml
# svcadm enable nfs/server

or

# svccfg -v import /var/svc/manifest/network/nfs/server.xml
# svcadm enable nfs/server

If you want to disable and delete a service, see below.

# svcadm disable network/nfs/server
# svccfg delete network/nfs/server

Issues With an Orphaned .attribute^ File

2009-11-23T07:07:00.000-08:00

We had intriguing problem related to a Unix-based directory. This directory was mapped to Windows via Citrix.

On the Windows side, the initial thought was that the folder was simply "hidden" via the attributes, which would be normal. However, the attributes for the folder didn't show the hidden option selected.

Recently, we performed an account migration activity on the Unix side. The culprit turn out to be an orphaned .attribute^ file on the Unix side. Apparently, this file's username and group was deleted prior to migration, and its state unexpectedly caused the directory's contents to be invisible to Windows-based viewers.

At any rate, we finally noticed the file with the ls -al command. The .attribute^ file was removed from the Unix side and viewing was back to normal on the Windows side.

ftpconfig - Setup Anonymous FTP

2009-10-23T07:10:00.000-07:00

To setup an anonymous FTP account, you might consider using the ftpconfig script. It seems to be a lot easier to setup an anon FTP account with Solaris 10 than it was for Solaris 8. The anon users log in and then they are given restricted access to a mini-root filesystem.

Here is the syntax.

# ftpconfig someftpdir
or
# ftpconfig -d someftpdir

If you have problems, you may have to check files in the /etc/ftpd directory. Hint: check the ftpaccess file

No Space Left on Device!

2009-09-18T08:26:00.000-07:00

About two weeks ago, my colleague and I had an issue with the /var partition on a seldom used Sun Blade 2500 workstation. Frankly speaking, I thought a simple fsck would suffice while in single user mode. Not quite. Anyway, we executed fsck on the partition and it "fixed" whatever errors reported. And while in single user mode, we wrote test files to the partition without incident. However, once we booted to init 3 run level, we still could not write to the /var partition, not even as root. That was a bit strange because df -k reported only 20% of the partition was used. hmmm...inode problem?? Then we looked through the logs and noticed a message about not having enough space on the device – in short, we ran out of inodes. Almost immediately, we suspected a problem with mail. We checked the mqueue directory and it contained almost a 1,000,000 small extraneous/useless messages from cron. We deleted most of them (after a few hours) and it resolved the issue. And yes, we got a handle on the cron that was generating those messages.

Troubleshooting Domain Issues with My SysAd Blog - Again

2009-07-10T00:18:00.000-07:00

Late last night (KST), I was reviewing my free MyBlogLog stats and noticed 8 visitors had stopped by the blog. Usually, by that time, mysysad would have had several hundred visits. I surfed over to the domain and verified it was down (this blog is hosted on the blogger.com platform and it is pointing to a custom domain). However, the blog was working with the old address, http://esofthub.blogspot.com

I pinged the old IP address - no answer. Then I pinged ghs.google.com, and it responded with a different IP address. I used that new address to update my A-Records with my domain registrar -- yahoo.

After 20-30 minutes of propagation, the mysysad.com domain was back online.

By the way, I just pinged the old address twice...no response. Update: 7 hours later - pinged again - no answer

Here is another post I wrote a few months ago about a similar issue...

My SysAd Blog Temporarily Reverts to its former Blogspot URL

142857 is an Interesting Number

2009-06-18T03:45:00.000-07:00

A couple of days ago a colleague sent me this interesting number, 142857, via email. I performed a Google search on the number and I was surprised to see so many returns - 190,000. It even has its own wiki page. For additional information, here is the wiki page for this cyclic number - 142857.

Below is the email.

Subject: Interesting Numbers

Roy,

I want to let you know a weird number which is 142857.

Why?

142857 X 1 = 142857
142857 X 2 = 285714
142857 X 3 = 428571
142857 X 4 = 571428
142857 X 5 = 714285
142857 X 6 = 857142

See, the answers show all same numbers with different position.

142857 X 7 = 999999

142 + 857 = 999
14 + 28 + 57 = 99

142857 X 142857 = 20408122449
20408 + 122449 = 142857

Interesting, huh?

Pipe Symbols are Missing from Some Posts

2009-05-30T08:52:00.000-07:00

You might have already noticed some of my theme formatting attempts appeared to have removed essential characters - in particular, the pipe symbols. I will try to go back and fix them when I get some time. I just want you to be aware of the inconvenience. Thanks Mr. Mendoza.

Gilbert Mendoza said...
Hey there. Just as an FYI, I believe your theme formatting has removed essential characters from some of your posts. For example, none of the pipe symbols separating each of the stringed commands are showing.
4/25/09 1:15 PM

esofthub said...
You're right Gilbert Mendoza. I just check a few posts and the "|" are missing. I guess changing between themes deleted them.
4/25/09 3:15 PM

Field 'user_website' doesn't have a default value

2009-05-30T08:45:00.000-07:00

I received a few complaints about not being able to register an account on my phpbb forum. To confirm, I tried creating a test account.

Here were the results:

General Error

SQL ERROR [ mysql4 ]

Field 'user_website' doesn't have a default value [1364]

An SQL error occurred while fetching this page. Please contact the Board Administrator if this problem persists.

Please notify the board administrator or webmaster: some one @gmail.com

In my case, here was the fix. Of course, make sure you backup your database.

http://www.phpbb.com/kb/article/doesnt-have-a-default-value-errors/

I ran the commands via phpmyadmin. It works fine now.

Determine Java Runtime Environment Version

2009-04-25T04:26:00.000-07:00

A few days ago I was asked for a quick and easy way to determine what java runtime environment version was installed via the Unix command line. The requester was trying to determine possible compatibility issues for some recently purchased applications.

Bear in mind that you may have multiple JRE environments installed on your box so consider the PATH value -- meaning you might have to specify an absolute path. e.g. /usr/some_java_version/bin/java

Here is the Unix one-liner syntax using default path

% java -version
java version "1.x.x_xx)

...

By the way, it is the same syntax for Windows.

C:\Users\mysysad> java -version
java version "1.x.0_xx"
Java(TM) SE Runtime Environment (build 1.x.0_xx-xxx)
Java HotSpot(TM) Client VM (build xx.3-bxx, mixed mode, sharing)

Access a Tape Drive Device From a non-global Zone

2009-04-17T05:48:00.000-07:00

A couple days ago, I had a friend who wanted to know how to access a tape device from his Solaris 10 non-global zone.

Here is an example on how to add a device to an existing non-global zone. By the way, the commands below are ran from the global zone, and you will have to reboot the non-global zone.

# zoneadm list
mysysad-global

# zonecfg -z sysad
zonecfg:sysad> add device
zonecfg:sysad:device> set match=/dev/rmt/0n
zonecfg:sysad:device> end
zonecfg:sysad> verify
zonecfg:sysad> commit
zonecfg:sysad> exit

# zoneadm -z sysad reboot

After the reboot, you should be able to access the tape drive from the non-global zone.

Disable On-screen Keyboard on Startup

2009-03-24T06:00:00.000-07:00

I was playing around with a few of Vista's settings on my laptop. Unfortunately, one of those settings was the “On-Screen Keyboard.” I had the utility set to automatically start up each time I logged on; however, I did not want or need that convenience. After a few more logins, it got really annoying. It was time to unset the setting, but it took me awhile to rediscover the setting and uncheck it.

At any rate, here is the path to the setting via the control panel.

Control Panel=>Ease of Access Center=>Use the computer without a mouse or
keyboard=>On-screen keyboard (uncheck it and then Save)

Create Tarballs Greater Than 2GB

2009-02-04T01:07:00.000-08:00

I was tarring a Unix-based application today when I encountered tar's 2GB file size limit. Frankly speaking, I was a bit surprised the application was greater than 2GB because it was only few hundred megabytes a few years ago. Anyway, to get around this annoying limitation, I used the -E option, but it did not work. I realized that I was using a non-standard tar binary because of redefined paths -- per site policy. I specified the absolute path for the OS-based tar command and the tar activity worked fine.

Here is an example.

# /usr/bin/tar -cvfpE GreaterThan2GB.tar /opt/MYAPP

Identify Zone Processes via Global Zone

2009-01-29T06:24:00.000-08:00

I had a colleague of mine ask a question about how to identify processes that belong to a particular zone via the global zone because the global zone “sees” everything. Recently, I attended a class and it covered the "deltas" between Solaris 10 and Solaris 8 and this question surfaced there, too.

The quick answer is to use the "-Z" option, which is the uppercase form.

Here is an example from the global zone.

# /usr/bin/ps -efZ