LCF blog

Altar for the most black priest

2010-02-08T21:30:00.002+01:00

Pgpool-II 2.3 comming soon...

2009-11-15T22:27:00.002+01:00

Today Tatsuo Ishii wrote interesting news at pgpool mailing list, version 2.3 coming this month.

What's new in 2.3:


- Adopt PostgreSQL 8.4 parser. On of the visible effects of this is,
 WITH clause can be load balanced.

- Allow to use INSERT/UPDATE including CURRENT_TIME_STAMP,
 CURRENT_DATE, now(). pgpool-II guarantees that each DB node will be
 populated exactly same value for these data type. For example
 consider following table:

 CREATE TABLE t1(id INTEGER PRIMARY KEY, regdate TIMESTAMP DEFAULT CURRENT_TIMESTAMP);

 Let's populate t1:

 INSERT INTO t1(id) VALUES(1);

 Actual query executed by pgpool-II is:

 INSERT INTO "t1"("id", "regdate") VALUES (1,'2009-11-15 21:35:01.783053+09');

 '2009-11-15 21:35:01.783053+09' is extraced from PostgreSQL by
 executing SELECT CURRENT_TIMESTAP. So t1 tables on all DB nodes has
 exactly same value.

- Add new directive log_per_node_statement.  If true, print all
 statements to the log. Similar to log_statement except that prints
 DB node id and backend process id info. Example:

 2009-11-15 21:34:12 LOG:   pid 22285: DB node id: 0 backend pid: 22301 statement: CREATE TABLE t1(id INTEGER PRIMARY KEY, regdate TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
 2009-11-15 21:34:12 LOG:   pid 22285: DB node id: 1 backend pid: 22300 statement: CREATE TABLE t1(id INTEGER PRIMARY KEY, regdate TIMESTAMP DEFAULT CURRENT_TIMESTAMP);

Google Checkout over stunnel

2009-11-13T16:33:00.004+01:00

Recently run HTTPS connection to website over stunnel + haproxy (not really important atm why such exotic idea). More important that hole operation goes smooth, quick and easy. Everything work until today when I read email from one of developers in over company.
He got problem with Google Checkout:


We encountered an error trying to access your server at https://some.website.at.web/GoogleCheckout/response -- 
the error we got is javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake

First I check connection with browser but everything looks ok as it should be. Start digg but google didn't help me alot. So I check Google Checkout documentation and find such thing:


There are also a number of measures you can take to keep your 
communications with Google Checkout secure:

    * Never share your Merchant Key with anyone.

    * Sign your shopping cart XML using HMAC SHA1 and your Merchant Key.
      Signing your shopping cart authenticates the cart you send and 
      verifies that your cart hasn't been tampered with during transmission.

    * Send order processing commands over a secure HTTPS connection.
      When sending order processing commands to Google, use an HTTPS connection
      secured by 128-bit SSL v3 or TLS connection (SSL v2 is not allowed). 
      Use your Merchant ID and Merchant Key as the username and password for 
      HTTP Basic Authentication.

    * Verify the authenticity of the server certificate presented to you.

    * Specify an HTTPS callback URL secured by SSL v3 or TLS using a valid
      certificate from a major Certifying Authority to receive Google notifications.
      Only accept messages authenticated by HTTP Basic Authentication, using 
      your Merchant ID and Merchant Key as the username and password. 
      Take a look at our list of accepted SSL certificates.

    * Validate messages sent to your callback URL before processing them.

Link

Looks still ok - SSL v3 or TLS . I recheck and SSL v3 is working. But Google Checkout doesn't like it.

Solution:

Change (/etc/stunnel/stunnel.conf)


; Protocol version (all, SSLv2, SSLv3, TLSv1)
sslVersion = SSLv3

to:


; Protocol version (all, SSLv2, SSLv3, TLSv1)
sslVersion = all

Because "SSLv3 or TLS" mean "SSLv3 and TLS" for google.

SSD #2

2009-08-31T20:17:00.005+02:00

So today I could finally compare SUN SSD results with my SSD experience. I check two configurations:

- Dell R610, 12GB RAM, 4x50GB SSD RAID10
- Dell R610, 12GB RAM, 6x50GB SSD RAID10

SSD: SAMSUNG MCCOE50G Rev: 3D3Q

At both run clean CentOS with vdbench 5.01. Use same test as befor.

Tests
Server	i/o rate	MB/sec 1024**2	bytes i/o	read pct	resp time	resp max	resp stddev	cpu% sys+usr	cpu% sys
Random Read (4K) threads=32
HP DL360 G6 8x146GB SAS 15k RAID 10, 12GB RAM	2097,13	8,19	4096	100,00	15,257	407,631	13,818	0,3	0,2
HP DL360 G6 8x146GB SAS 15k RAID 50, 12GB RAM	1933,43	7,55	4096	100,00	16,549	416,267	15,998	0,3	0,2
DELL M600 2x146GB SAS 15K RAID1, 8GB RAM	595,50	2,33	4096	100,00	53,735	1313,692	66,092	0,2	0,2
DELL R610 4x50GB RAID10 12GB RAM SSD	16830,99	65,75	4096	100,00	1,9	28,730	0,566	1,8	1,7
DELL R610 6x50GB RAID10 12GB RAM SSD	18234,40	71,23	4096	100,00	1,753	22,646	0,439	2,1	1,9
Random Write (4K) threads=32
HP DL360 G6 8x146GB SAS 15k RAID 10, 12GB RAM	1963,12	7,67	4096	0,00	16,298	1511,424	15,338	0,4	0,3
HP DL360 G6 8x146GB SAS 15k RAID 50, 12GB RAM	1124,42	4,39	4096	0,00	28,457	1229,856	16,496	0,2	0,2
DELL M600 2x146GB SAS 15K RAID1, 8GB RAM	247,19	0,97	4096	0,00	129,457	1520,834	114,036	0,2	0,2
DELL R610 4x50GB RAID10 12GB RAM SSD	982,59	3,84	4096	0,00	32,564	339,488	16,693	0,3	0,3
DELL R610 6x50GB RAID10 12GB RAM SSD	1459,31	5,70	4096	0,00	21,926	243,077	10,288	0,2	0,2
50-50 Read/Write (4K) threads=32
HP DL360 G6 8x146GB SAS 15k RAID 10, 12GB RAM	1998,44	7,81	4096	50,03	16,010	702,082	29,572	0,3	0,3
HP DL360 G6 8x146GB SAS 15k RAID 50, 12GB RAM	1388,33	5,42	4096	50,02	23,047	1259,130	44,446	0,2	0,2
DELL M600 2x146GB SAS 15K RAID1, 8GB RAM	327,78	1,28	4096	49,95	97,625	1301,350	81,706	0,3	0,3
DELL R610 4x50GB RAID10 12GB RAM SSD	1674,78	6,54	4096	50,00	19,105	264,783	27,222	0,2	0,1
DELL R610 6x50GB RAID10 12GB RAM SSD	2413,06	9,43	4096	50,00	13,259	250,832	20,313	0,2	0,2
Sequential Read (MB/sec) threads=32
HP DL360 G6 8x146GB SAS 15k RAID 10, 12GB RAM	766,06	383,03	524288	100,00	41,769	1216,561	178,083	0,5	0,5
HP DL360 G6 8x146GB SAS 15k RAID 50, 12GB RAM	489,26	244,63	524288	100,00	65,407	2437,031	342,871	0,3	0,3
DELL M600 2x146GB SAS 15K RAID1, 8GB RAM	142,88	71,44	524288	100,00	223,902	1970,552	467,248	0,2	0,1
DELL R610 4x50GB RAID10 12GB RAM SSD	314,19	157,1	524288	100,00	101,837	980,956	275,867	0,1	0,1
DELL R610 6x50GB RAID10 12GB RAM SSD	453,45	226,72	524288	100,00	70,553	765,786	194,196	0,3	0,3
Sequential Write (MB/sec) threads=32
HP DL360 G6 8x146GB SAS 15k RAID 10, 12GB RAM	439,24	219,62	524288	0,00	72,853	590,213	46,006	0,2	0,2
HP DL360 G6 8x146GB SAS 15k RAID 50, 12GB RAM	1194,07	597,03	524288	0,00	26,797	155,358	5,514	0,6	0,6
DELL M600 2x146GB SAS 15K RAID1, 8GB RAM	140,25	70,13	524288	0,00	228,155	397,324	30,052	0,1	0,1
DELL R610 4x50GB RAID10 12GB RAM SSD	247,72	123,86	524288	0,00	129,175	244,959	14,121	0,1	0,1
DELL R610 6x50GB RAID10 12GB RAM SSD	370,52	185,26	524288	0,00	86,363	213,793	9,973	0,2	0,1

I have to say that's not impressive at all. True, reads are nice, but come on, writes 12x times slower ?!? It's really far far from ok.

Maybe it's Dell firmware, but atm I got latest firmware every part.

Back from holidays can be scary ;-D

2009-08-17T22:40:00.003+02:00

Vdbench 5.01

2009-08-03T22:59:00.002+02:00

Just notice new version of Vdbench 5.01 has been released.

From release notes special makes me happy:

- 64-bit support for Solaris, Linux and zLinux.

Cooling doors

2009-07-30T22:21:00.006+02:00

Sun provides two types of cooling doors, which attach directly to the Sun Blade 6048 Chassis

Quite WoW :)

SAS vs SSD with vdbench

2009-07-28T10:14:00.010+02:00

Server	SAS vs SSD
Server	HP DL360 G6 8x146GB SAS 15k RAID 10	HP DL360 G6 8x146GB SAS 15k RAID 50	DELL M600 2x146GB SAS 15K RAID1	Sun X4450 4x32GB SSD	Sun X4140/X4440 4x32GB SSD	Sun T5240 4x32GB SSD
Random Read (4K)	2.1K IOPS	1.9K IOPS	0.6K IOPS	98.4K IOPS	33.8K IOPS	71.5K IOPS
Random Write (4K)	1.9K IOPS	1.1K IOPS	0.25K IOPS	31.8K IOPS	16.6K IOPS	14.4K IOPS
50-50 Read/Write (4K)	2.0K IOPS	1.4K IOPS	0.3K IOPS	14.9K IOPS	13.3K IOPS	15.7K IOPS
Sequential Read (MB/sec)	383 MB/sec	245 MB/sec	71 MB/sec	764 MB/sec	463 MB/sec	1012 MB/sec
Sequential Write (MB/sec)	219 MB/sec	597 MB/sec	70 MB/sec	376 MB/sec	470 MB/sec	531 MB/sec

SSD results get from here
Test file was:


sd=sd1,lun=/dev/cciss/c0d0p5,openflags=o_direct
*wd=wd1,sd=sd1,xfersize=4096,rdpct=100
*rd=run1,wd=wd1,iorate=100,elapsed=10,interval=1

wd=wm_80dr,sd=sd*,readpct=0,rhpct=0,seekpct=100
wd=ws_80dr,sd=sd*,readpct=0,rhpct=0,seekpct=0
wd=rm_80dr,sd=(sd1-sd80),readpct=100,rhpct=0,seekpct=100
wd=rs_80dr,sd=(sd1-sd80),readpct=100,rhpct=0,seekpct=0
wd=rwm_80dr,sd=sd*,readpct=50,rhpct=0,seekpct=100
rd=default
###Random Read and writes tests varying transfer size
rd=default,el=30m,in=6,forx=(4K),forth=(32),io=max,pause=20
rd=run1_rm_80dr,wd=rm_80dr
rd=run2_wm_80dr,wd=wm_80dr
rd=run3_rwm_80dr,wd=rwm_80dr
###Sequential read and Write tests varying transfer size
rd=default,el=30m,in=6,forx=(512k),forth=(32),io=max,pause=20
rd=run4_rs_80dr,wd=rs_80dr
rd=run5_ws_80dr,wd=ws_80dr

Ice Age: Dawn of the Dinosaurs - Sun movie ?

2009-07-06T20:22:00.003+02:00

Blue Sky Studios, a wholly-owned unit of Fox Filmed Entertainment, deployed Sun Blade servers and Sun Ultra workstations for the rendering of its new animated feature Ice Age: Dawn of the Dinosaurs, giving them four times better performance than their previous solution.

Blue Sky deployed a Sun Constellation System consisting of 10 Sun Blade 6048 chassis with a total of 480 Sun Blade X6250 server modules, each powered by two Intel Xeon processor 5400 series; 120 Sun Ultra 24 workstations powered by the Intel Core 2 processor; and Sun's Enterprise Installation Services. After full implementation, Blue Sky reaped performance that was four times faster than their previous solution, allowing them to render digital frames faster and create more frames in less time.

http://www.sun.com/aboutsun/pr/2009-07/sunflash.20090706.1.xml

FCoE

2009-05-09T14:51:00.001+02:00

Today find 2 nice videos about FCoE:

- Overview
- FCoE looks like

GMV in Gdańsk

2009-04-22T14:28:00.004+02:00

GMV Create First Real-Time Passenger Information System For Gdansk...

... Not always everything goes smooth :)

warning: peer certificate won't be verified in this SSL session

2009-03-23T10:31:00.001+01:00


# Remove warning: "warning: peer certificate won't be verified in this SSL session"
require 'openssl'
module OpenSSL
  module SSL
    remove_const :VERIFY_PEER
  end
end
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE

Nice and useful.

Sun Cloud Computing

2009-03-19T17:16:00.002+01:00

Sun Cloud Computing executives Dave Douglas and Lew Tucker preview the Sun Cloud - the industry's most flexible, open and portable public cloud platform.

Since yesterday I'm really excited about Sun Cloud. From what I've seen, Sun Open Cloud Computing is more powerfull solution then AWS Amazon. Check how simple you can drag&drop hole environment then select hole or part and create "clone" for HA - Impressive!

Supernap

2009-03-18T23:27:00.001+01:00

Just incredible.

[A-100] Firmware 01-17-090125-15-POP-402

2009-03-06T23:37:00.002+01:00

#v+
[...]
11. Fixed HDD spin-up during standby
[...]
#v-

w0w finally they fix that. My ears love them :)

Release notes

My recent tricks for postgresql

2009-03-04T22:55:00.004+01:00

1. echo deadline > /sys/block/{device}/queue/scheduler

2. change filesystem from ext3 into reiserfs

Really nice for hour work include synchronize nodes. Of course after find out what to do :)

check_pgpool - nagios plugin for pgpool-II

2008-12-24T16:28:00.005+01:00

check_pgpool.rb - simple nagios plugin to check pgpool nodes status with pcp_node_info. Usefull to monitor are all nodes conneted.

checkcommands.cfg:


define command {
        command_name    check_pgpool
        command_line    $USER1$/check_pgpool.rb $ARG1$
}

services.cfg:


define service {
        use                     generic-service
        host_name               pool.hostname
        service_description     PGPOOL1
        is_volatile             0
        check_period            24x7
        max_check_attempts      4
        normal_check_interval   5
        retry_check_interval    1
        contact_groups          admins
        notification_interval   960
        notification_period     24x7
        check_command           check_pgpool!1
}

PostgreSQL tricks with broken db

2008-12-17T10:28:00.003+01:00

Few PostgreSQL tricks for broken db:

case:


pg_dump: query returned 0 rows instead of one: SELECT typlen, typinput, typoutput, typreceive, typsend, typmodin, typmodout, typanalyze, typinput::pg_catalog.oid as typinputoid, typoutput::pg_catalog.oid as typoutputoid, typreceive::pg_catalog.oid as typreceiveoid, typsend::pg_catalog.oid as typsendoid, typmodin::pg_catalog.oid as typmodinoid, typmodout::pg_catalog.oid as typmodoutoid, typanalyze::pg_catalog.oid as typanalyzeoid, typdelim, typbyval, typalign, typstorage, pg_catalog.pg_get_expr(typdefaultbin, 'pg_catalog.pg_type'::pg_catalog.regclass) as typdefaultbin, typdefault FROM pg_catalog.pg_type WHERE oid = '1325300'::pg_catalog.oid

solution:


reindexdb -h localhost -U postgres db_name

case:


pg_dump: Error message from server: ERROR: catalog is missing 4 attribute(s) for relid 1325439

solution:


REINDEX TABLE pg_catalog.pg_attribute;

case:


pg_dump: failed sanity check, parent table OID 16401 of pg_rewrite entry OID 16403 not found

solution:


delete from pg_rewrite where oid = 16403;
REINDEX TABLE pg_catalog.pg_rewrite;

Amazon CloudFront in Ruby

2008-12-02T20:30:00.005+01:00

Last month Amazon launch new service - Amazon CloudFront. Amazon CloudFront delivers your content from Amazon S3 using a global network of edge locations. Sweet, really nice solution for quite expensive Akamai (yes, I know Akamai is cheaper at some level, but startups don't have such budget). At start they release only Amazon CloudFront Authentication Tool for Curl. So decide to wrote own tool in ruby (atm there is RightScale Ruby library but when I wrote that there wasn't any such tool).

First, I try use curb (Libcurl bindings for Ruby), but then notice that can be done more simple only with Net::HTTP

According to Developer Guide correct REST Requests needs URI indicates the particular resource you want to act on, also needs correct Request Headers (Authorization, Content-Length, Content-Type, Date, Host, x-amz-date).

Date/x-amz-date like at provided by Amazon curl script, get from Amazon server:


def self.server_date(url)
  server_date=''
  uri = URI.parse(url)
  http = Net::HTTP.new(uri.host,443)
  http.use_ssl = true
  http.verify_mode = OpenSSL::SSL::VERIFY_NONE
  response=http.request_get(uri.path)
  response['Date']
end

Authorization generate as Amazon documentation said:


def self.sign(date)
  digest = OpenSSL::Digest::Digest.new('sha1')
  hmacd=HMAC.new(AWS_SECRET_ACCESS_KEY, digest)
  hmacd.update(date)
  signature=Base64.encode64(hmacd.digest)
end

So simple list distribution can be done like that:


def ACFList(url)
  date=server_date(url)
  uri = URI.parse(url)
  http = Net::HTTP.new(uri.host,443)
  http.use_ssl = true
  http.verify_mode = OpenSSL::SSL::VERIFY_NONE
  headers = {
    'X-AMZ-Date' => date,
    'Authorization' => 'AWS ' + AWS_ACCESS_KEY_ID + ':' + sign(date).chop
  }
  response=http.request_get(uri.path,headers)
  response
end

Example:


./acf.rb list https://cloudfront.amazonaws.com/2008-06-30/distribution
<?xml version="1.0"?>
<DistributionList xmlns="http://cloudfront.amazonaws.com/doc/2008-06-30/"><Marker></Marker><MaxItems>100</MaxItems><IsTruncated>false</IsTruncated></DistributionList>

Hole code can be find here

Kreatywna matematyka według Tesco

2008-12-01T21:35:00.006+01:00

Dzisiaj w Tesco odkryłem nową, kreatywną wersje matematyki. Idąc między regałami zauważyłem:

Ser Philadelphia - 3,29zł (ta mała cena po lewej stronie). Promocja 2 sztuki za 6zł. Niby ok, jest promocja jest dobrze. Niepokoi mnie tylko informacja "Oszczędzasz 6zł"

WTF ?!?


2x3,29zł = 6,58zł 
         - 6,00zł
-----------------
           0,58zł (nijak 6zł nie wychodzi)

Idąc za ciosem zrobiłem jeszcze kilka zdjęć:

Less spam ? Yeah McColo's is down.

2008-11-13T20:08:00.004+01:00

Really interesting news read today. According to Boris Veldhuijzen van Zanten spam levels fell by 66% after Tuesday. That happened after Internet providers cut off McColo’s (Site is down) connectivity to the Internet.

Incredible isn't it ?

Dj Wonter - Wings - Official Music Video

2008-10-30T15:29:00.001+01:00

Wicked tune made by Wonter. Check it !

Squid with AWStats

2008-10-27T21:25:00.006+01:00

I've search a bit, but couldn't find any nice howto. All links told me to turn on "emulate_httpd_log" or change logformat in squid.conf. Cool, but I've got logs from last month with standard logformat - squid and like to know how much bandwidth at Amazon EC2 will cost.

Lets see, standard squid logs looks like that:


1224992424.377      0 127.0.0.1 TCP_MEM_HIT/200 1046 GET http://sitename/www/Etrade/js/translation.js? - NONE/- application/x-javascript

With little sed help, logs could looks like that:


1224992424 0 127.0.0.1 TCP_MEM_HIT 200 1046 GET /sitename/www/Etrade/js/translation.js? - NONE/- application/x-javascript

How ?


LogFile="cat /path/to/log/squid-access_log | sed 's/\.[0-9]\{3\}\s\{5\}//' | sed 's/\// /' | sed 's/http\:\///' |"

Now just few min generate report .. et voila ;)

Why EC2 command line API-tools sucks...

2008-09-25T20:22:00.003+02:00

Every time cron daemon run ruby scripts witch contain command line EC2 API-tools load grow up and take a lot resources. In last days decide to rewrite it. For some configure files need to check often what IP's got working www backends or database backends. Got for that class in ruby:

machine.rb:


require 'resolv'

class Machine

 IMAGES   = {
         'www'           => ['ami-WWWWWWWW'],
         'db'            => ['ami-DBDBDBDB'],
         'kolektor'      => ['ami-POOLPOOL'],
 }

 MACHINES = {
  'www'  => [],
  'db'  => [],
  'kolektor' => [],
 }

 def self.getips
    output=`ec2-describe-instances`
    output.split("\n").each do |line| 
      IMAGES.each do |type, amis|
  amis.each do |ami|
   if line.match('\s' + ami + '\s.*?running')
    ip = Resolv.getaddress(line.split[3])
    MACHINES[type] << ip 
   end
  end
      end
    end
    MACHINES
 end
end

Easy for use at other scripts like that:

getwww.rb:


#!/usr/bin/ruby

require 'machines.rb'

puts Machine.getips['www']

But that was slow and take too much resources. Discover project amazon-ec2.rubyforge.org and rewrite that class.

machines_new.rb:


require 'resolv'
require 'rubygems'
require 'EC2'

class Machine

 ACCESS_KEY_ID = 'your_access_key_id'
 SECRET_ACCESS_KEY = 'your_secret_access_key'


 IMAGES   = {
         'www'           => ['ami-WWWWWWWW'],
         'db'            => ['ami-DBDBDBDB'],
         'kolektor'      => ['ami-POOLPOOL'],
 }

 MACHINES = {
  'www'  => [],
  'db'  => [],
  'kolektor' => [],
 }

 def self.getips
 ec2 = EC2::Base.new(:access_key_id => ACCESS_KEY_ID, :secret_access_key => SECRET_ACCESS_KEY)
        ec2.describe_instances.reservationSet.item.each do |reservation|
    IMAGES.each do |type, amis|
  amis.each do |ami|
     reservation.instancesSet.item.each do |item| 
   if item.imageId.match(ami)
    ip = Resolv.getaddress(item.dnsName)
    MACHINES[type] << ip
   end
     end
  end
    end
 end
 MACHINES
 end
end

How fast could be use non-JAVA EC2 API-tools ?

With old class:


# time for i in `seq 1 20`; do ./getwww.rb ; done
real 3m58.211s
user 0m48.870s
sys 0m2.650s

With new class:


# time for i in `seq 1 20`; do ./getwww.rb ; done
real 0m20.234s
user 0m6.570s
sys 0m0.870s

Only 11.5x times faster ! And doesn't take 80mb mem, just 10mem for 1 sec.

Ksero...

2008-09-21T18:44:00.002+02:00

Nigdy nie pomyślałbym, że ksero może mnie potrzebować. Widać to moje przeznaczenie, albo matrix i jestem tylko kawałkiem softu :(