SCons is a software construction tool (build tool, or make tool) implemented in Python, that uses Python scripts as “configuration files” for software builds. Based on the design that won the Software Carpentry build tool competition, SCons solves a number of problems associated with other build tools, especially including the classic and ubiquitous Make itself.

In its evolution, SCons will be more general than a make replacement, it will be a Gnu Build System replacement. The Gnu Build System (GBS) is also known as the set of Autotools (autoconf, automake, autoheader, etc…).

Quote From http://www.scons.org/wiki/TheBigPicture

So scons is just a makefile replacement? Which claims to be simple and feature rich? Let me construct a simple but sufficient scons configuration file (SConstruct), and show you how you can compile your c++ projects. Then you make the judgement whether to go for it or not.

#filename: SConstruct
env = Environment(
CXX = 'g++34',
CCFLAGS = ['-g','-O3','-Wall','-DPOSIX_C_SOURCE=199506L','-D_REENTRANT','-DLINT_ARGS','-DLINUXOS'],
LIBS = ['pthread', 'c','nsl','rt' ,'lbestd'],
LIBPATH = ['lib'],
CPPPATH = ['.','lib/inc'],
LINKFLAGS = ['-mt']
)

env.Program(
'bin/msgQ',
['Main.cc', 'MainState.cc','MainFunc.cc',
'MyLog.cc','MyQueue.cc','MyTcp.cc',
'TcpSvr.cc']
)

To compile the source codes, simply do this:

scons

To clear the objects and binaries, simply do this:

scons -c

Let me briefly explain the items in the SConstruct file.

By default, scons does define a simple compilation environment for you, if you write helloworld.cc, you just need to write a line in SConstruct.

Program('helloworld.cc')

But the case above, I overwrite some environment variable’s value for my needs to build a complex application.

CXX is the variable defines you c++ compiler. My case, I change it to g++34.

You can check the default value by just insert a python print statement in your SConstruct file.

env = Environment()
print "CXX = ", env['CXX']

CCFLAGS defines compilation flags.
LIBS defines library to link.
LIBPATH defines additional library paths besides the system default.
CPPPATH defines include file paths.
LINKFLAGS defines compiler linker flags

After constructed the environment, just compile your binary by calling Program function.

scons have more features compiles java codes too and importantly it is cross platform. To know more? Check out the scons man pages ans well as user manual.

python -m SimpleHTTPServer 9914

9914 is the port number I choose to host my web, assume my IP is 192.168.1.1, in firefox you can ask your friend to access this url: http://192.168.1.1:9914.

It is easy isn’t it? You do NOT even need to run it as root, just run it as normal user but make sure your firewall allow the chosen port number.

A complex combo, I have constructed one last time in post what is your 10 common linux commands?

history | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' |  \
grep -v "./" | \
column -c3 -s " " -t | \
sort -nr | nl |  head -n10

But sometimes we tend to misuse them, I found someone likes to do this:

cat foobar.txt | grep "barfoo"

Actually grep alone is enough to do the search, we do not need cat or meow for that. As I show him this:

grep "barfoo" foobar.txt

Another example is list the names of all files under a given directory that match a particular string. Let say I wanna obtain a list of files that have string “foobar”.

find /opt/src/kmess -type f | xargs grep -l "foobar"

Hey! find is cool but grep alone is enough to do the job.

grep -lr "foobar" /opt/src/kmess

Try to goes through the grep manual, you will find out that grep is more powerful than what you expect.

for i in range(0,10):
    print "0x0001 << %d = 0x%04X" % ( i , 0x1 << i)

The result is

0x0001 << 0 = 0x0001
0x0001 << 1 = 0x0002
0x0001 << 2 = 0x0004
0x0001 << 3 = 0x0008
0x0001 << 4 = 0x0010
0x0001 << 5 = 0x0020
0x0001 << 6 = 0x0040
0x0001 << 7 = 0x0080
0x0001 << 8 = 0x0100
0x0001 << 9 = 0x0200

The important part in the example is this :

0x1 << i

Let say I wanna shift the bits of a value rightwards 3 times, I will do this:

value = 102
print value >> 3

The result will be 12.

To know more about what is bit shifting, check out wikipedia.

Have Fun!

Well, how to do it?

The idea is simple, the first instance of the python script will open a file, and lock it. Therefore when the consequent instances try to lock the file, it will fails.

The code is simple too.

import fcntl
def lockFile(lockfile):
    fp = open(lockfile, 'w')
    try:
        fcntl.lockf(fp, fcntl.LOCK_EX | fcntl.LOCK_NB)
    except IOError:
        return False

    return True

The function will try to lock the file specified by me, if it success, return True, else return False.

From any point of my codes, I’ll do

if not lockFile(".lock.pod"):
        sys.exit(0)

Interesting?

For example, I have a module that extract raw values into logs based on the specified interval. The result will looks like this (test.dat):

##date time slot_1  slot_2  slot_3  slot_4  slot_5  slot_6  slot_7  slot_8  slot_9  slot_10
06/02/2009 09:16:49 16  6  15  5  14  4  17  3  14  4
06/02/2009 09:16:54 16  8  13  3  15  4  17  4  15  5
06/02/2009 09:16:59 18  8  11  5  15  6  14  3  16  5
06/02/2009 09:17:04 19  6  12  5  18  7  15  5  18  5
06/02/2009 09:17:09 19  7  14  6  16  6  16  6  18  4
06/02/2009 09:17:14 21  7  17  5  19  7  16  4  20  5
06/02/2009 09:17:19 20  5  19  6  17  5  15  3  17  3
06/02/2009 09:17:24 20  6  21  5  18  4  14  4  18  4
06/02/2009 09:17:29 17  5  19  4  16  4  15  4  17  2

gnuplot allows us to create a template describe how our graph outlook. To generate the graph, we just have to execute the template, just like executing bash script.

This is a sample of a gnuplot template that I use to construct a plot graph, (test1.pg)

#!/usr/bin/gnuplot
reset
set terminal png

set xdata time
set timefmt "%d/%m/%Y %H:%M:%S"
set format x "%H:%M"
set xlabel "time"

set ylabel "total actives"
set yrange [0:31]

set title "M7YC Performance per time"
set key reverse Left outside
set grid

set style data linespoints

plot "test.dat" using 1:3 title "slot 1", \
"" using 1:4 title "slot 2", \
"" using 1:5 title "slot 3", \
"" using 1:6 title "slot 4", \
"" using 1:7 title "slot 5", \
"" using 1:8 title "slot 6", \
"" using 1:9 title "slot 7", \
"" using 1:10 title "slot 8", \
"" using 1:11 title "slot 9", \
"" using 1:12 title "slot 10"
#

To allow the template to be executable, the first line of the template must add the SHABANG ‘#!/usr/bin/gnuplot’, similar to bash script. And also remember to perform chmod to your template.

chmod +x test1.pg

I would like to generate graph that shows the performance value over time, therefore I have to read datetime into my X axis and performance value into my Y axis. First two columns of my data file are datetime, I need to set the xdata as time, the define the time format in the raw data files and at last define the x format to appear in my graph.

set xdata time
set timefmt "%d/%m/%Y %H:%M:%S"
set format x "%H:%M"

I would like to display my keys outside the graphs as well as enable the grid to ease the reading.

set key reverse Left outside
set grid

The plot graph I intended to generate consist of lines and dots.

set style data linespoints

The important part is how to define the source of my raw data, how many items I wanted to plot into the graph, as well as giving each item a title. The plot format may vary based on different graph format, but for my case it is

plot [raw_data] using [x value's column in data file]:[y value's column in data files] title [item's name], ...

The plot portion can be shorten into this:

plot "test.dat" u 1:3 t "slot 1", \
"" u 1:4 t "slot 2", \
"" u 1:5 t "slot 3", \
"" u 1:6 t "slot 4", \
"" u 1:7 t "slot 5", \
"" u 1:8 t "slot 6", \
"" u 1:9 t "slot 7", \
"" u 1:10 t "slot 8", \
"" u 1:11 t "slot 9", \
"" u 1:12 t "slot 10"

Execute the template and redirect the binary stream to construct a png file.

./test1.pg > test1.png

test1.png

gnuplot is far more powerful than what I have illustrate here, you may want to checkout the official website for more info.

P.S. I am using gnuplot v4.2 while writing this.

str = "Hello, "
...
str = str + "my name is "
...
str = str + Name
print str

In python, we have string stream (StringIO) that will behave like file stream, you can construct your string like this:

str=StringIO()
...
str.write("Hello, ")
...
str.write("my name is ")
...
str.write(Name)
print str.getvalue()

The same way, you can construct your binary bytes with StringIO and write it into file once you are done.

bin=StringIO()
bin.write("/x5F/x5F%c" % 0xFF)
...
file = open ("my.bin","wb")
file.write(bin.getvalue())
file.close()

I have carry out some research on python into writing binary files. Either writing byte by byte manually, construct a binary stream then write it one short, or convert it from hex string, they are easy to write in python.

To open a file for binary writing is easy, it is the same way you do for reading, just change the mode into “wb”.

file = open("test.bin","wb")

But, how to write the binary byte into the file?
You may write it straight away with hex code like this:

file.write("\x5F\x9D\x3E")
file.close()

Now, check it out with hexedit,

hexedit test.bin

You will see this:

00000000   5F 9D 3E                                                                                                _.>
00000020
00000040

Now, open the file to append more bytes:

file = open("test.bin","ab")

What if I want to store by bin value into a stream and write it one short?

s ="\x45\xF3"
s = s + "%c%c" % (0x45,0xF3)
file.write(s)
file.close()

Any convenient ways if I can obtained a hex string, and want to convert it back to binary format?
Yes, you just need to import binascii

import binascii
hs="5B7F888489FEDA"
hb=binascii.a2b_hex(hs)
file.write(hb)
file.close()

Isn’t it easy? I love python. You may want to discover more functions of binascii with ipython.

I created a hex string splitter script for my friend who needs to decode the raw data that will be written in hex string such as “5F443D95FEA3D4787AEDC4″. Every time, he split them byte by byte manually into this form (“5F 44 3D 95 FE A3 D4 78 7A ED C4″) for better reading. He was complain that the manual process was always bothering him. I told him that can be done by writting a simple bash script.

My hexsplit.sh will do this:

./hexsplit.sh "5F443D95FEA3D4787AEDC4"

But what if he grep this hex string from other file? Or what if the hex string are stored in a file? I want my bash script to be able to process hex string from pipeline stream as well as file redirection too, like this:

 grep "hex" dummy.log | cut -d"]" -f2 | ./hexsplit.sh

or like this:

 ./hexsplit.sh < dummy.log

How to write this hexsplit.sh?

hexsplit.sh

#!/bin/bash

if [ -e $1 ] ;then read str; else str=$1;fi
len=`expr length $str`
for (( a=0; a<=$len; a=a+2 )); do echo -n ${str:a:2}" "; done
echo ""

The important line is the if statement. If there are no argument specified ( -e $1), I read from the stream, else i take it from $1 ( param 1). It is so simple isn’t it?

Have fun!

last
...
mysurface    tty7         :0               Mon Oct  6 20:07 - down   (00:00)
reboot   system boot  2.6.24.4-64.fc8  Mon Oct  6 20:06          (00:00)
mysurface    pts/8        10.168.28.44     Mon Oct  6 17:42 - down   (01:58)
mysurface    pts/7        :0.0             Mon Oct  6 17:41 - 19:40  (01:59)
mysurface    pts/6        :0.0             Mon Oct  6 17:27 - 19:40  (02:13)
mysurface    pts/5        :0.0             Mon Oct  6 17:27 - 19:40  (02:13)
mysurface    pts/5        :0.0             Mon Oct  6 15:52 - 15:59  (00:07)
...

If the command line history could provides the date time of the commands being executed, that may really narrow down the scope of the user actions that cause the server malfunction. By default, history do not append with timestamp, but it is easy to configure it to display timestamp, you just need to set one environment variable HISTTIMEFORMAT.

HISTTIMEFORMAT takes format string of strftime. Check out the strftime manual to choose and construct the timestamp that suit your taste. My favorite is “%F %T “.

export HISTTIMEFORMAT="%F %T "

Execute history again and you will see the effect on the spot, bare in mind that the timestamp for command lines that executed at previous sessions may not valid, as the time was not tracked.

...
  994  2008-10-16 02:27:40 exit
  995  2008-10-16 01:12:20 iptables -nL
  996  2008-10-16 01:47:46 vi .bash_profile
  997  2008-10-16 01:47:55 history
  998  2008-10-16 01:48:03 . .bash_profile
  999  2008-10-16 01:48:04 history
 1000  2008-10-16 01:48:09 exit
 1001  2008-10-16 02:27:43 history
...

I would suggest you to put the export into ~/.bash_profile as well as /root/.bash_profile. In case you do not have .bash_profile, you can choose to put into ~/.bashrc.

Don’t mess up my servers! Your actions will be track!