c0t0d0s0.org

Experiences with the A380

nospam@example.com (Joerg Moellenkamp) — Tue, 10 Nov 2009 20:48:02 +0100

I wrote not that much about aviation in the last few months, but writing about 787 delays got boring. However ... today i found an interesting article about the experiences of Airbus, airlines, pilots and engine manufacturers with the first two years of the A380 in service. The article "A380 In-service report" at flightglobal.com is really an interesting read.

2004 Subscribers

nospam@example.com (Joerg Moellenkamp) — Tue, 10 Nov 2009 14:13:32 +0100

Yet another coffee for Local .... c0t0d0s0.org broke through the 2000 subscribers line yesterday:

Support for SAM/QFS offlined files in the Samba version supported by Sun

nospam@example.com (Joerg Moellenkamp) — Tue, 10 Nov 2009 12:48:19 +0100

Hierarchical storage management is really useful to offload seldom used data to cheaper media like tape while keeping them in access by a filesystem. SAM/QFS is such a filesystem. Your normal disks are just a cache, the data resides on tapes, magneto-opticals or slow disk drives (Just in case, you want to try it: I wrote a tutorial about the basic installation last year)

But there is a problem: Some operating environments generate a thumbnail to give the user an preview of the file. So you touch each an every file in a directory to generate this little picture.On a normal disk, this isn't a problem, but now think about a hierarchical filesystem, where some data is placed on tapes for example. To access this files, the system does something called "staging":You have to put the tape into the drive, you have to wind, and then you can read the file, then eject it to read the next tape - and this takes time ... a lot of time.

Now imagine an File Management Tool like the Windows Explorer that access all files. You know what happens? Yes, the system tries to stage all files in a directory. We call this effect staging storm. Your autoloader do nothing else than swapping tapes to generate thumbnails.

But there is a less known feature of Windows CIFS. You can declare a file as "offline", thus it just accesses the file when you explicitly try to read it by clicking on it. And here the problem starts: Someone has to tell Windows that this file is offline. The factory-default Samba has no mechanism to do this. This flag isn't supported by generic Samba, thus you can't limit the staging to the file you want to to thumbnail generation. It isn't the fault of Windows, because Samba tells it, that the files on tape aren't offline. So it thinks, they are available like on a hard disk. A countermeasure for this problem has to teach Samba to check for the state of a file in a SAM/QFS filesystem.

My colleague Dirk Nitschke pointed me to an important part of a new patch to the Samba binaries provided by Sun: Patch 119757-17 for SPARC and 119758-17 for x86 doesn't only update the Sun supported version to Samba 3.0.37. It integrates the modification to Samba due to CR 6737005 and PSARC 2009/381, that supports this Offline flag. This flag is set in the communication with a windows client, when the the matching flags in SAM/QFS that designate, that the file was released from the cache and just resides on slower media, is set. When a file carries this flag, the system won't generate an thumbnail, thus there is no need to stage it. To notify the user of this situation, offlined files have a small clock at the left side of the icon (at least in XP)

How does it look in a Windows Explorer? Dirk provided me a screenshot of his SAMQFS/Samba installation. You have to click on it to get a larger version.

When a file is offlined, you will see it by a small clock at the icon for the file. Additional the attribute column shows an O for offline. When you look at the output of sls (the ls of SamFS enabled to show additional SamFS specific properties of a file) the file context.txt is in the mode "offline".

With this patch the combination of SAM/QFS and Samba is much more usable and you don't have to compile your own version from source with a source patch provided by your friendly Sun rep, which isn't supported as a part of the support for the Solaris Operating Environment.

Relevanz

nospam@example.com (Joerg Moellenkamp) — Tue, 10 Nov 2009 10:43:33 +0100

Ich habe da ein weiteres nettes Beispiel gefunden, das in der Wikipedia mit zweierlei Maß gemessen wird: Da werden Blogger mit einer gewissen Reichweite, aber zuweilen für unbedarfte Leser erklärungs- und inkontextsetzungsbedürftigen Meinung (deren Dokumentation mithin Aufgabe eines enzyklopädischen Werkes der Neuzeit sein könnte) nicht in die Wikipedia aufgenommen, auf der anderen Seite ist mir gestern aber aufgefallen, das jede Menge aktive Generäle bis auf Brigadeebene in der Wikipedia mit Lebenslauf zu finden sind, was ich wiederum höchstens genauso relevant finde, wie eben jenen Blogger.

Das ist nur eine persönliche Meinung, aber trotzdem: Strange ...

Oracle 11g R2 for Solaris SPARC

nospam@example.com (Joerg Moellenkamp) — Tue, 10 Nov 2009 10:05:54 +0100

Oracle Database 11g Release 2 (11.2.0.1.0) for Solaris SPARC 64 Bit is now available for download at the OTN.

European Commission objects to Sun Oracle merger

nospam@example.com (Joerg Moellenkamp) — Tue, 10 Nov 2009 07:45:38 +0100

The European Commission has expressed their objections about the Sun acquisition by Oracle. Sun notified the SEC with an 8K filing available at EDGAR

It's based on the opinion, that the purchase of Mysql would harm the database market. I've already expressed my strong feeling, that this is utter nonsense. I think it's unnecessary to repeat it. I don't have any understanding for this decision, as it doesn't help competition and customers. In my opinion, it just helps some interested parties.

Oracle answered with a press release to the filing of the objections:

The Commission's Statement of Objections reveals a profound misunderstanding of both database competition and open source dynamics. It is well understood by those knowledgeable about open source software that because MySQL is open source, it cannot be controlled by anyone. That is the whole point of open source.

Even the Department of Justice issued a press release regarding this EC decision:

"Several factors led the Division to conclude that the proposed transaction is unlikely to be anticompetitive. There are many open-source and proprietary database competitors. The Division concluded, based on the specific facts at issue in the transaction, that consumer harm is unlikely because customers would continue to have choices from a variety of well established and widely accepted database products. The Department also concluded that there is a large community of developers and users of Sun's open source database with significant expertise in maintaining and improving the software, and who could support a derivative version of it.

However the cards are on the table now and the EC should tell Oracle and Sun, what they want to solve those objections. So it's a good that they filed the objections now and not at the last possible moment.

ZFS send dedup integrated in Opensolaris

nospam@example.com (Joerg Moellenkamp) — Mon, 09 Nov 2009 20:37:51 +0100

A few days deduplication for ZFS pools found it's integration to Opensolaris. Now the in-stream deduplication feature for ZFS send streams has followed the on-disk variant. With this feature blocks are just transmitted once in a stream, thus the depending on the data you have to transmit less data, as the ZFS just sends a reference to an already transmitted block instead the full data. You will find further information in the case log for PSARC 2009/557.

Panic? Don't panic?

nospam@example.com (Joerg Moellenkamp) — Mon, 09 Nov 2009 14:33:23 +0100

In the last few days i thought a little bit about this swine flu panic. Let's put aside that the vaccination is an adjuvanated vaccination and that it contains questionable chemicals.

Of course i'm reading all the stuff about the swine flu as almost everybody, as i'm trying do decide if i get the additional flu shot. Due to visiting many customers a year, i decided to get the seasonal flu shot some years ago and it's a yearly practice to me.

But i'm not convinced, that this flu shot is necessary. I just thought, it's even contra productive on a global scale. The following article is just a public braindump of thoughts....
Continue reading "Panic? Don't panic?"

Lawsuit against Intel

nospam@example.com (Joerg Moellenkamp) — Sun, 08 Nov 2009 19:29:34 +0100

The New York times reports about an lawsuit filing against Intel in "2nd UPDATE: NY AG: Dell Got $6B Via Secret Intel Pact >DELL" :

Dell, the world's third-biggest computer maker based on shipments, was allegedly paid about $6 billion between February 2002 and January 2007, according to the lawsuit. In one fiscal quarter, the lawsuit says payments from Intel constituted 116% of Dell's reported net income.

Interesting news as this sheds a different light to the financial performance of Dell and you could just speculate about the margins of Intel that allow them to pay $6B and still making money.

You will find the complaint for further reading at the site of the Attorney General of the State New York. The pdf of the complaint is available here.

Better checksum error reporting in ZFS

nospam@example.com (Joerg Moellenkamp) — Sun, 08 Nov 2009 15:21:54 +0100

After all the discussion about filesystems and checking them after errors, i want to point to a small extension: Since the introduction of the changes proposed by "PSARC 2009/497: zfs checksum ereport payload additions", error messages of ZFS because of checksum errors are much more informative. The extended message to the fault management gives you much more information about the issue:

cksum_expected = 0x3ffa2307eb97 0x100b4f8dbe8a6cc8 0xee4673db5aa10430 0x6d4b7e76d616a47
cksum_actual = 0x3ffa2307eb94 0x100b4f8dbe88ecd0 0xee4673dafaa44424 0x6d4a7e80d5cea4f
cksum_algorithm = fletcher4
bad_ranges = 0x0 0x8 0x10 0x18
bad_ranges_min_gap = 0x8
bad_range_sets = 0x0 0x0
bad_range_clears = 0x1 0x1
bad_set_bits = 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
bad_cleared_bits = 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x2 0x0 0x0 0x0 0x0 0x0 0x0 0x0

With this kind of information, you can get much better image of the the problem than just from the message "checksum error". You will find a deeper look into this change in the description of the change request 6867188. An overview of the meaning of the new values in the error report is available in the PSARC case documentation.

Presentation at the Apache Hadoop Get Together Berlin

nospam@example.com (Joerg Moellenkamp) — Fri, 06 Nov 2009 22:09:09 +0100

On 16th December i will talk about Hadoop on Sun at the Apache Hadoop Get Together in Berlin. The talk will be divided in three parts:

Advantages of using Hadoop on CMT systems like the T5120
Using Hadoop in conjunction with Solaris Features like Container and Resource Management in non-dedicated architectures
Hadoop and Grid Engine

There will be short live demonstrations in the second and third part of the presentation.

My presentation will not be the only one. Richard Hutton from nugg.ad talks about data processing with hadoop in his presentation "Moving from five days to one hour." Nikolaus Pohle from Nurago talks about data analysis with Hadoop for online market research.

links for 2009-11-06

nospam@example.com (del.icio.us) — Fri, 06 Nov 2009 13:34:08 +0100

The Billion Dollar Gram | Information Is Beautiful

(tags: visualization economics politics)
Why are the Microsoft Office file formats so complicated? (And some workarounds) - Joel on Software

"....are the product of a demented Borg mind"

(tags: microsoft programming article development windows software office formats excel)

Mysql competition by Amazon

nospam@example.com (Joerg Moellenkamp) — Fri, 06 Nov 2009 09:00:46 +0100

I wrote at several occasions, that this "Oracle purchase of Sun will limit database competition" isn't true as you can easily fork mysql and do your own development. Matt Asay has the same opinion and points to recently launched service by Amazon to substantiate his opinion:

The European Commission must be feeling a bit silly right about now. Despite insisting that Oracle has not responded to its requests for comment and concessions in its planned acquisition of Sun Microsystems (and the open-source database MySQL), Amazon.com recently offered the EC all the proof it needs that MySQL competition remains alive and well.[....]Oracle hasn't even started with MySQL yet, and it already faces significant competition, not to mention the other MySQL forks (e.g., Drizzle).

I think the steps of Amazon are a nice proof for the successful forkability of Mysql. Take the stuff, make a business out of it and make money. The more i think about this while situation, the more i think it's just a interest of a single person that blocks a merger that is good for everyone in IT. Nobody can be interested in IBM getting stronger (okay, of course: beside IBM), due to the FUD surrounding the situation of the Sun/Oracle merger. But this is the consequence at the moment.

Phenoms and recent OpenSolaris developer builds - revisted

nospam@example.com (Joerg Moellenkamp) — Thu, 05 Nov 2009 22:14:23 +0100

Last month i wrote about my boot problems with OpenSolaris dev builds and that i've suspected bug 6844241 as the root cause for this situation. Well, the fix for that bug was commited for build 126, thus i've tried to update my system today. Well ... the fix has been delivered, system runs:

jmoekamp@hivemind:~$ uname -a SunOS hivemind 5.11 snv_126 i86pc i386 i86pc Solaris jmoekamp@hivemind:~$

Loadbalancing with Opensolaris - or: PSARC 2008/575

nospam@example.com (Joerg Moellenkamp) — Thu, 05 Nov 2009 20:26:59 +0100

An nice addition to the networking functionality has found its way into the Opensolaris codebase: PSARC 2008/575 ILB: Integrated L3/L4 Load balancer. This code enables Opensolaris to act as a simple load balancing device. It already contains features like acting as a loadbalancer in NAT or direct-server-return-mode as well as ping/udp/tcp and scripted health checks. You will find more information about this new feature in the case log of PSARC 2008/575.

The configuration is pretty straight-forward. To configure a loadbalancing on a group of four servers you have to use this commands:

ilbadm create-servergroup -s servers=webserv1,webserv2,webserv3 webgroup
ilbadm add-server -s servers=webserv4 webgroup
ilbadm create-rule -i port=80,vip=15.192.0.0,ipversion=IPv4 \ 
	-m lbalg=hash-IP-port,type=NAT \
	-o servergroup=webgroup webrule

I think as soon as this code is available in Opensolaris, i will try to marry it with the KSSL functionality. Would be nice, to use both functions to create an SSL-offloading loadbalancer

No, ZFS really doesn't need a fsck

nospam@example.com (Joerg Moellenkamp) — Wed, 04 Nov 2009 22:33:57 +0100

There is a discussion at osnews.com about a simple question: "Should ZFS Have a fsck Tool?". The answer is simple: No. I could stop now, as this answer is pretty obvious when you work a while with ZFS, but i want to explain my position. And i want to ask a different question at the end.

I wrote this explanation already in the comment section of osnews.com, so the following text will be a reworked version of my comment at that site.
Continue reading "No, ZFS really doesn't need a fsck"

Estimating the impact of dedup

nospam@example.com (Joerg Moellenkamp) — Tue, 03 Nov 2009 00:15:05 +0100

It would be nice to have a estimation what the usage of deduplication would yield: Victor Latushkin points in his mail on zfs-discuss to a nice feature of the zdb command to get such an estimation.

Jeff Bonwick about ZFS Dedup

nospam@example.com (Joerg Moellenkamp) — Mon, 02 Nov 2009 15:52:41 +0100

Nothing really totally new for the readers of my blog, but the information in this article are from the most authoritative source in regard of ZFS: Jeff Bonwick's Blog - ZFS Deduplication

PSARC 2009/479 zpool recovery support

nospam@example.com (Joerg Moellenkamp) — Mon, 02 Nov 2009 15:32:00 +0100

There was another important addition to the Opensolaris code base. PSARC 2009/479 zpool recovery support. Perhaps it isn't a feature for the front pages, because such a feature would be futile for ZFS in a world where all devices adhere to standards and definitions as they should.

The problem: Especially with cheap hardware you have a large amount of strange and interesting effects like weirdly reordered commands or commands that should only return after completion of a task, but return directly after accepting the command. Or as the PSARC states:

Uncooperative or deceptive hardware, combined with power failures or sudden lack of access to devices, can result in zpools without redundancy being non-importable.

The code resulting out this PSARC case implements automatic ways and mechanism to discard the last transactions to a zpool to get back to an uncorrupted, thus importable state. You were able to do similar things manually in the past, but that was a somewhat arcane science. Due to the nature of ZFS as a copy-on-write filesystem, there is a probability, that you can import the filesystem at a point in time slightly before the corrupt state. So executing this command should give you an output like this one:

# zpool clear -F data Pool data returned to its state as of Tue Sep 08 13:23:35 2009. Discarded approximately 29 seconds of transactions.

As your last backup is often much older than a few seconds, this is a nice way to handle unimportable zpools due to "uncooperative or deceptive hardware". As the complete structure of the file system can be validated by the checksums, you can even be sure that this recovered state is a consistent state.



NEOSUG
nospam@example.com (Joerg Moellenkamp) — Mon, 02 Nov 2009 15:12:00 +0100

    Gregor Longariva just announced the the existence of another OpenSolaris User Group in Germany in this tweet. The NEOSUG is the OSUG of Nuremberg and : Erstes #osug Erlangen Nürnberg Treffen am 12.11. im Anschluss an den SunDay am #RRZE. Einladungen folgen noch. #opensolaris #neosug 
    
     



Deduplication integrated to ZFS
nospam@example.com (Joerg Moellenkamp) — Mon, 02 Nov 2009 12:24:50 +0100

    I've reported about the Dedup PSARC cases a few days ago. Now the code for deduplication in ZFS has been integrated into Opensolaris as stated by this mail on onnv-notify:  "PSARC 2009/571 ZFS Deduplication Properties, 6677093 zfs should have dedup capability". 

 
    
     



German Opensolaris User Group
nospam@example.com (Joerg Moellenkamp) — Mon, 02 Nov 2009 12:06:02 +0100

    Ich hatte da heute morgen beim Teekochen eine Idee, die ich gerne zur Diskussion stellen möchte. Es ist erstmal eine lose Idee, ein Gedankenblitz: Es bilden sich jetzt ja ueberall in Deutschland OSUGs, ich denke wir haben da bald eine kritische Masse erreicht, die, die eine weitere Vernetzung untereinander geradezu aufdrängt. Wie wäre es in der zweiten Jahreshälfte 2010 eine gemeinsame Konferenz der OSUGs durchzuführen?  
    
     



links for 2009-11-02
nospam@example.com (del.icio.us) — Mon, 02 Nov 2009 12:00:44 +0100

    
                Schweinegrippe: HÃ¶chste Warnstufe
                
                (tags: health grippe pandemiepanikl)
             
    
     



Kein Schwein steckt mich an ....
nospam@example.com (Joerg Moellenkamp) — Sun, 01 Nov 2009 16:14:13 +0100

     
    
     



What a bad day ... - revisited
nospam@example.com (Joerg Moellenkamp) — Sun, 01 Nov 2009 15:49:13 +0100

    When you think, it can't be worse, it will get worse .... after  a ruined Friday, i've got a bad cold ... well, at least i don't have fever now ....  
    
     



What a bad day ...
nospam@example.com (Joerg Moellenkamp) — Sat, 31 Oct 2009 07:02:53 +0100

    Yesterday was really one of my worst days in the last months ... no ... in the last years. I forgot the rule "You should never go to a dentist on a Friday". My dentist made some new inlays (so i don't have a single amalgan inlay now, just composite(?, the white paste) ones) but a root canal treatment, too. And the later one gave me the mother of all toothaches over the day. It was that bad, that i had to visit the dentist emergency  service in the army hospital here (they are an regular hospital, too. At foremost they do the emergency service for dentistry in the evening) in Hamburg. This was the most brutal visit at a  dentist i ever had, but now the tooth doesn't hurt that much any longer. However i had still toothache after the treatment there, so i walked the 5km back home through the relatively cold weather. However when i arrived at home, the toothache was  acceptable low.



 When you really want to hear a scary dentist story i should report about a apicoectomy 16 years ago:

Joerg: "Mum, they told me, that they use a new method. They carve a bone platelet with a chisel out of the jaw, cut the root, and lay it back. Carve ... doesn't sound enjoyable ..." 

Mum: "Jörg, don't be such an coward. I assume that chisel  is just the name for an medical tool. They won't use a hammer and a chisel". Well, that was a misjudgment  This still gnaws at her conscience when i'm joking about dentists. 14 years later i've learned, that the "bone platelet method" was just a short fashion in dentistry.  
    
     



Strange things in Russia
nospam@example.com (Joerg Moellenkamp) — Fri, 30 Oct 2009 07:45:17 +0100

    The ft.com/alphaville-blog reports about strange news. According to a russian governmental web page, Oracle is said to have withdrawn a regulatory filing in regard of the Sun Oracle/Merger. The strange thing:29 октября 2009 г. в ФАС России поступила просьба компании «Оракл Корпорейшн» (Oracle Corporation) об отзыве ходатайства вх. № 33986 от 4 августа 2009 г. о получении согласия на приобретение прав, позволяющих определять условия осуществления предпринимательской деятельности ООО «Сан Майкросистемс СПБ», ЗАО «Сан Майкросистемс» и ООО «Май Эс Кью Эл» без вынесения решения по существу.

30 октября 2009 г. ФАС России данную просьбу удовлетворил. 
Or in English via Google Translate:October 29, 2009 FAS Russia received a request from the company Oracle Corporation »(Oracle Corporation) to withdraw the application Rin. Number 33986 dated 4 August 2009 for consent for the acquisition of rights allowing to determine the conditions of doing business OOO Sun Microsystems SPB, JSC "Sun Microsystems" and OOO "May Ess Que El" without ruling on the merits.

October 30, 2009, FAS Russia granted the request.The FAS website announced yesterday, that it had granted the request today? WTF? Anyway: I wouldn't draw any conclusions  in regard of the merger out of it. Perhaps Oracle thinks that the request is unnecessary or they want to file a new one. I will just wait for an official comment by Oracle. And by the way: Why should Oracle issue a new FAQ paper about future intentions with Sun products on October 28th when they want to withdraw the offer? Really strange .... 
    
     



links for 2009-10-29
nospam@example.com (del.icio.us) — Thu, 29 Oct 2009 12:00:37 +0100

    
                Schwarzenegger Gives California Legislature A Hidden Finger
                
                (tags: politics funny california government usa writing)
            

                Did Schwarzenegger drop 4-letter bomb in veto?
                
                (tags: politics california funny usa)
             
    
     



The Deliverer's Knowledge: What's the max. number of IP instances per NIC?
nospam@example.com (Heiko Stein) — Wed, 28 Oct 2009 21:42:00 +0100

    A foreword by Jörg: This is the first blog entry written by Heiko. "the deliverer's knowledge" is just a working title and it's referring to the fact, that Heiko delivers vast knowledge on a daily basis at various customer sites. I just add the c0t0d0s0.org uncoporate non-design. So this is Heiko direct and unfiltered. Please welcome him. PS: Another substantially longer article will be online soon.



Take a look at:# ndd -get /dev/ip ip_addrs_per_if

256

That's the default. You can set the count up to 8191. (Btw: That's already the max. number of local zones per global zone)

# ndd -set /dev/ip ip_addrs_per_if  8191
But keep in mind, the newly set value isn't boot persistent, so you should set it during the system startup. For example by creating a transient SMF service or an init.d-script 
    
     



Quotes from the new overview/FAQ document regarding the Oracle/Sun merger
nospam@example.com (Joerg Moellenkamp) — Wed, 28 Oct 2009 14:43:22 +0100

    There is a new overview/FAQ document regarding the planned  Oracle/Sun merger available at oracle.com. I will cite some key passages in this article.

 
Continue reading "Quotes from the new overview/FAQ document regarding the Oracle/Sun merger"