No milk in my coffee

Introducing Unit Testing in Legacy code base

2009-02-26T04:37:00.001-08:00

In big companies, developing a green field project is a rare privilege
... most of the time you will have to develop on a existing code base of
versatile quality - Not due to the skill of the developer more likely to
the history of the project, lack of code review and poor quality focus,
short term vs. long term risk -.

Anyway if you are not lucky you are facing a big spaghetti monster
calling other pasta entities in ways ranging from CORBA, RV, rmi, EJB,
ftp messaging ... Chances is that the code can't even run on your dev
machine, making development laborious and non productive.

There are 2 ways to deal with this problem

1 Status quo, self explanatory
2 Introducing unit test, mocking the internal and external monsters

1 : simple solution, not very rewarding but if the cost of solution 2
overcome the benefit that's the way to go
2 : long term benefit in development and quality. But costly

Though with a bit of method there are way to make 2 less costly, all you
will need is :

- A good refactoring dev environment, eclipse or idea
- Junit, XmlUnit
- XStream an xml serializer/desiralizer
- an UAT/Dev environment that runs.

1 - identify the part the function you need to test

you won't be able to test all the code, it is important to test a small
chain of the code lest that the mocking will be too complex.

It's likely that you can identify the function that is the entry and
exit point

Object f1(Object... args)

It is that function that we will try to unit test.

2 - record samples

your system is probably generating high complex object, based on high
complex entry. Trying to reproduce the graph manually is laborious and
sure to be incomplete. That's what we need to used XStream to record the
args and the output of the function in a uat environment, getting that
way real life object.



// the new method that will delegate to the old
// and dump in and out
Object f1(Object... args)  {
  dumpIn(args);

  Object o  = _f1(args);

  dumpOut(o);
}

// was the real
Object _f1(Object args) {
       ...
}

3 - integrate your sample in your unit test

Now that you have sample with your input parameters and your expected
result you can start writing your unit test.

4 - mock the external call

It is likely that once you integrate the sample in the unit test, the
code will need to code external or internal component. you will need to
refactor that code in specific method that you will overwrite for your
test.



class F {
       Object _f1(Object... args) {


               // used to be inline
               Object extInf = getExtInfo();
       }


       Object getExtInfo() {
        ....
       }
}

class FTest extends F {

       Object getExtInfo() {
        return new MockObject();
       }
}

If the object is to complex to Mock you can use the XStream to dump a
sample in uat environment.

5 - run the test !

6 - how to improve the solution

It is possible to create the list of test automatically by scanning the
samples directory e.g.. Making it easier to add sample and extend the
coverage of the test and behaviour.

This is an iterative process, it is not possible to make everything
testable in one go, take your time and surely step by step the quality
of your code will improve.

Dr. Horrible's Sing-Along Lyrics

2008-07-25T05:38:00.000-07:00

Here

The Java Spirit

2008-07-25T01:34:00.001-07:00

- Composition is better than inheritance
- Constructor are for initialization only
- destructor/finalizer are for debugging only
- Free resource explicitly
- Declare the interface not the type ie List myList; not ArrayList myList
- Optimize on metrics of a real system
- Use a proper development environment
- Don't believe the hype about closure, anonymous inner class cover 90% of the cases
- Properties are private or protected.
- Immutable is your friend.
- Don't pool object, we are in the 21st century
- Elegant code is code that is meaningful
- the most useful api are : Collections, common-lang, java.text

... To be continued

Too many open file descriptors, leak detection

2008-06-11T07:04:00.000-07:00

In Java it is good practice to manually free the resources such as Streams, Connection. This is the consequence of the non deterministic property of the Garbage Collector. You object that is not reference might wait minutes before being process - or might even never -, so you can rely and the destructor - the finalize method - to do that - the finalizer make the collection of the object really costly-.

So if you open a Stream you should ideally do that :


InputStream is = null
try {
  is = new FileInputStream(...);
  
  ...

} finally {
  if (is != null) {
    try { is.close(); ) catch(Exception e) { // IGNORE }
  }
}

That's a lot of boiler plate code, and it's probably a good idea to use a Template pattern for that - look af Spring JDBCTemplate for example -.

What happen if you don't do that? That depends on how many file you are gonna open. It's really likely that the object is gonna get GC and FileInputStream as a finalizer where it closed itself.

But if your application uses file intensively you might end up with "Too many open file descriptors" errors. If you're lucky you have few entry points where stream are open, so you can easily fix it. Otherwise you will need a way to take the part of the code that are erronous.

What I propose under is to use Javassist to instrument the FileInputStrean and FileOutputStream class.

First we need a class that we would inject into the FileXStream to capture the states and detect the leaks.


package io;

public class LeakDetector {
 
 private boolean closed;
 private Exception trace;
 
 
 public LeakDetector() {
  trace = new Exception();
 }
 
 public void close() {
  this.closed = true;
 }
 
 protected void finalize() {
  if (! closed) {
   printAlert();
  }
 }

 private void printAlert() {
  if (System.err != null) {
   System.err.println("LeakDetector detected a leak from the object open at the following trace :");
   trace.printStackTrace();
  }
 }
}

Then we need a class that instruments the FileXStreams


package io;

import java.io.IOException;
import java.util.concurrent.Callable;

import javassist.CannotCompileException;
import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtConstructor;
import javassist.CtField;
import javassist.CtMethod;
import javassist.NotFoundException;

public class Instrumenter implements Callable<Object> {

 private String[] clazzes;
 
 public Instrumenter(String[] args) {
  this.clazzes = args;
 }

 public static void main(String[] args) throws Exception {
  
  if (args == null || args.length == 0) {
   args = new String[] { "java.io.FileInputStream", 
     "java.io.FileOutputStream" };
  }
 
  new Instrumenter(args).call();
 }

 public Object call() throws NotFoundException, CannotCompileException, IOException {
  ClassPool pool = ClassPool.getDefault();
  for(String clazz : clazzes) {
   instrument(clazz, pool);
  }
  
  return null;
 }

 private void instrument(String clazz, ClassPool pool) throws NotFoundException, CannotCompileException, IOException {
  CtClass ctClass = pool.get(clazz);
  
  ctClass.addField(CtField.make("private io.LeakDetector __io_ld;", ctClass));
  
  for(CtConstructor cConst : ctClass.getConstructors()) {
   cConst.insertBefore("__io_ld = new io.LeakDetector();");
  }
  
  for(CtMethod cMethod : ctClass.getMethods()) {
   if (cMethod.getName().equals("close")) {
    cMethod.insertBefore("__io_ld.close();");
   } else if (cMethod.getName().equals("finalize")) {
    cMethod.insertBefore("__io_ld._finalize();");
   }
  }
  
  ctClass.writeFile("bin/java");
 }
}

This class write the new FileInputStream in the bin/java directory. We need do assemble the LeakDetector and the new FileInputStream, FileOutputStream under the same jar. all you need to next is to run your application with the "-Xbootclasspath/p:leakdetector.jar" on the command line.

Note: Applications that use this technique for the purpose of overriding a system class in rt.jar should not be deployed as doing so would contravene the Java 2 Runtime Environment binary code license.

If you run the following code, with the bootclasspath set :


import java.io.FileInputStream;
import java.io.FileOutputStream;


public class DodgyIOClass {

 public static void main(String[] args) throws Exception {
  
  
  new FileOutputStream("c:/test.txt");
  
  new FileInputStream("c:/test.txt");
  
  
  Thread.sleep(3000);
  System.gc();
  Thread.sleep(3000);
  
  
 }
}

It should prints :


LeakDetector detected a leak from the object open at the following trace :
java.lang.Exception
 at io.LeakDetector.<init>(LeakDetector.java:11)
 at java.io.FileOutputStream.<init>(FileOutputStream.java)
 at java.io.FileOutputStream.<init>(FileOutputStream.java:70)
 at DodgyIOClass.main(DodgyIOClass.java:10)
LeakDetector detected a leak from the object open at the following trace :
java.lang.Exception
 at io.LeakDetector.<init>(LeakDetector.java:11)
 at java.io.FileInputStream.<init>(FileInputStream.java)
 at java.io.FileInputStream.<init>(FileInputStream.java:66)
 at DodgyIOClass.main(DodgyIOClass.java:12)

JSR 203 NIO 2 in Java 7

2008-04-02T08:10:00.000-07:00

So it seems.

That would be probably one of the most awaited fix, a well design file system api.

no more


if (!file.delete())  {
 // delete did not work but I don't know why 
}

Java performance improvements touted

2008-03-28T09:22:00.000-07:00

Java is not the slowpoke of old days and performance now matches or exceeds applications developed in C, technologists stressed Wednesday during a presentation at TheServerSide Java Symposium in Las Vegas.

From : DZone

Nice to see that written again.

IE 6 display = 'none' not working

2008-03-14T03:01:00.000-07:00

see Here

Using


  element.style.display = ''

instead solve the problem

How to disable escaping in xslt

2008-03-14T02:48:00.001-07:00


 <xsl:text disable-output-escaping="yes"><!--</xsl:text>

NIO / IO debates

2008-03-05T08:53:00.000-08:00

More NIO / IO benchmark or evidence

Writing Java Multithreaded Servers - whats old is new (PDF Slides)

See Micro benchmark Channel vs InputStream / 2

String sorting tip

2008-01-24T01:25:00.001-08:00

Good tip here



 Collator esCollator =
   Collator.getInstance(new Locale("es"));
 Collections.sort(list, esCollator);

Image Resizer for widescreen picture frame and the main color problem

2008-01-05T04:13:00.000-08:00

-- UPDate
http://www.comesolvego.com/2008/04/29/resize-images-with-java-high-quality-and-working-solution/
--

For Xmas I receive a picture frame, unfortunately the image is expanded
to the size of the frame without respecting the ratio.

I try to find a batch solution to resize and fit the picture so that it
would not be deformed.

I did not find anything so I decided to try to write a small java
program to do it.
it would take the picture resize it so it fit in the frame and put it
center in a image the size of the frame.

It was relatively easy.


package org.srf;

import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.Rectangle;
import java.awt.RenderingHints;
import java.awt.image.BufferedImage;

public class FitImage {

 public BufferedImage  scaleImage(BufferedImage image, int width, int height) throws Exception {

  int originalHeight = image.getHeight();
  int originalWidth = image.getWidth();
  
  
  double scale = Math.min(
    height/ (double)originalHeight,
    originalWidth/ (double)originalWidth
  );
  
  int newWidth = (int)Math.round(scale * originalWidth);
  int newHeight = (int)Math.round(scale * originalHeight);
  
  
  // Draw the scaled image
  BufferedImage thumbImage = new BufferedImage(newWidth, newHeight, BufferedImage.TYPE_INT_RGB);
  
  Graphics2D graphics2D = thumbImage.createGraphics();
  graphics2D.setRenderingHint(RenderingHints.KEY_INTERPOLATION,
    RenderingHints.VALUE_INTERPOLATION_BILINEAR);
  
  graphics2D.drawImage(image, 0, 0, newWidth, newHeight,
    null);

  return thumbImage;

 }
 
 public BufferedImage frameImage(BufferedImage image, int width, int height, Color backgroundColor) throws Exception {
  
  int paddingX = (width - image.getWidth()) / 2;
  int paddingY = (height - image.getHeight()) / 2;

  
  // Draw the scaled image
  BufferedImage thumbImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
  
  
  Graphics2D graphics2D = thumbImage.createGraphics();
  
  graphics2D.setColor(backgroundColor);
  
  graphics2D.fill(new Rectangle(0, 0, width, height));
  
  graphics2D.drawImage(image, paddingX, paddingY, null);

  return thumbImage;

 }
}

Then what color do I want to draw the border. The average color of the
picture? does not seems right if you have a picture with blue and green
you might end up with a yellow ...

So what about the dominant colour ? the main colour of the picture ?
That quite a fusy concept like that. The color are distributed in a 4
dimension space (red, green, blue, alpha) what would that mean to be the
dominant color? What would be an efficient algorythm to determine it.

I came up with something that give relatively good result also not sure
about the validity of it.

It iteratively cut the spaces in 2^4 boxes and keep the one where there
is most color.
It does the same thing on this box reduce it a specific number of time.
As the color are define by a number between 0-256, each iteration
dividing the solution possibilite by to, in 8 iteration I arrived to a
result. What does it mean not sure... but on test I did it seems to work
fine.


package org.srf;

import java.awt.Color;
import java.awt.image.BufferedImage;

public class ColorHelper {
 private boolean discardPixelOutOfTheBox = true;
 private boolean shiftRange = false;
 
 
 public void setDiscardPixelOutOfTheBox(boolean discardPixelOutOfTheBox) {
  this.discardPixelOutOfTheBox = discardPixelOutOfTheBox;
 }


 public void setShiftRange(boolean shiftRange) {
  this.shiftRange = shiftRange;
 }


 public Color getMainColor(BufferedImage image, int refineTo) {
  Color c = null;
  
  int[] rgbs = image.getRGB(0, 0, image.getWidth(), image.getHeight(), null, 0, image.getWidth());
  
  
  int[][] colorRanges = new int[][]{ {0, 0xFF}, {0, 0xFF},{0, 0xFF}, {0, 0xFF}};
  
  for(int refine = 0; refine < refineTo ; refine++) {
   int[] colorSpace = new int[2 * 2 * 2 * 2];
   for(int rgb : rgbs) {
    int gi = calculateSpacePoint(colorRanges, rgb);
    if (gi != -1 ) {
     colorSpace[gi] ++;
    }
   }
   
   int ci = calculateSpacePointWinner(colorSpace);
   
   shrinkSpace(colorRanges, ci);
   
  }
  
  int[] crgbs = new int[4]; 
  // take middle of colorRange
  for(int i = 0; i < colorRanges.length; i++) {
   int[] range = colorRanges[i];
   crgbs[i] =  (range[0] +  range[1]) /2  ;
  }
  
  c = new Color(crgbs[2],crgbs[1],crgbs[0],   crgbs[3]);
  
  return c;
 }


 private int calculateSpacePointWinner(int[] colorSpace) {
  int ci = 0;
  int ciValue = 0;
  for(int i = 0; i < colorSpace.length; i++) {
   if (colorSpace[i] > ciValue) {
    ci = i;
    ciValue = colorSpace[i];
   }
  }
  return ci;
 }


 private int calculateSpacePoint(int[][] colorRanges, int rgb) {
  int gi = 0;
  for(int i = 0; gi != - 1 && i < 4; i++) {
   int v  = (rgb >> (i * 8)) & 0xFF;
   
   if (!discardPixelOutOfTheBox 
     || (v >= colorRanges[i][0] && v <= colorRanges[i][1])) {
    int rangeMiddle = (colorRanges[i][0] + colorRanges[i][1]) / 2;
    
    int vi = v < rangeMiddle ? 0:1;
    
    gi |=  vi << i;
   } else {
    gi = -1;
   }
   
  }
  return gi;
 }


 private void shrinkSpace(int[][] colorRanges, int ci) {
  // re-adjust colorRange
  for(int i = 0; i < colorRanges.length; i++) {
   int[] range = colorRanges[i];
   int cri = (ci >> i ) & 0x1;
   
   shrinkDimension(range, cri);
  }
 }


 private void shrinkDimension(int[] range, int cri) {
  int rangeShift = shiftRange  ? (range[1] - range[0]) /4 : 0;
  int middleRange = (range[0] +  range[1]) /2;
  if (cri == 0) {
   range[1] =  middleRange + rangeShift ;
  } else {
   range[0] = middleRange - rangeShift;
  }
 }

}

XML/Castor notes 1

2007-12-11T01:38:00.000-08:00

Marshal to a document object



 Document serializeObject(Object o) throws Exception {

  DocumentBuilder documentBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
  
  Document document = documentBuilder.newDocument();
  
  Marshaller marshaller = new Marshaller(d);
  
  synchronized (mapping) {
   marshaller.setMapping(mapping);
  }
  
  marshaller.marshal(o);
  
  return document;
 }

Write the Document to an xml file



 void write(Document doc, File f) throws Exception {
  Transformer trans = TransformerFactory.newInstance().newTransformer();
  
  Source source = new DOMSource(doc);
  Result result = new StreamResult(new FileOutputStream(f));
  
  trans.transform(source, result);
  
 }

Acegi 2

2007-12-10T04:34:00.001-08:00

does not kill fairy by murder anymore.

Map & Reduce explained for dummies

2007-12-10T03:00:00.000-08:00

Can Your Programming Language Do This?

Also I'm quite sure the cost of doing it in java is overrated - what about anonymous class, used in the common collection for example - ... This is a nice piece of explanation.

Interruptible Call

2007-12-06T07:34:00.001-08:00

In the not so new concurrent api, a call - Executor.call(new Call()) can be cancel by the Future object return.

Future.cancel(true) is gonna interrupt the thread. But does that mean that your code gonna stop automatically if it already started? Only if it's on a blocking call, which it's quite unlikely. So that your code is interruptible you need to test it the thread has been interrupted.


   while(test) {
      if (Thread.interrupted()) {
         throw new InterruptedException();
      }
   }

This way your cancel is more reactive.

Micro benchmark Channel vs InputStream / 2

2007-12-04T07:40:00.000-08:00

If you follow actually the samples like Grep. Channel are slower than the BufferedReader

On a directory with 16 files, with a total of 42 MB :
nio direct=Time Taken: 3703ms,
nio=Time Taken: 3781ms,
io=Time Taken: 3063ms

So ?


 public class NIOReader implements Reader {

  private ByteBuffer byteBuffer;
  
  public NIOReader(int bufferSize, boolean direct) {
   if (direct) {
    this.byteBuffer = ByteBuffer.allocateDirect(bufferSize);
     
   } else {
    this.byteBuffer = ByteBuffer.allocate(bufferSize);
   }
   
  }

  public String readFile(File f) throws IOException {
   FileInputStream fis = null;
   FileChannel channel = null;
   
   try {
    fis = new FileInputStream(f);
    
    channel = fis.getChannel();
    
    MappedByteBuffer buffer = channel.map(MapMode.READ_ONLY, 0, channel.size());
    
    CharBuffer charBuffer = Charset.defaultCharset().decode(buffer);

    return charBuffer.toString();
    
   } finally {
    if (fis != null)
     fis.close();
   }
   
   
  }

 }

Micro benchmark Channel vs InputStream

2007-12-04T02:27:00.000-08:00

I wrote a little non representative test to check is it was worth rewriting the IO, in an application that write and read a lot of data from the file system, using the not so new NIO api.

On a directory with containing 8 305 files all around 1 to 4 KB the result are around :
nio direct=Time Taken: 5610ms
nio=Time Taken: 5766ms
io=Time Taken: 5578ms

On a directory with 16 files, with a total of 42 MB :
nio direct=Time Taken: 875ms
nio=Time Taken: 906ms
io=Time Taken: 3079ms

So for big files it's definitly worth it, but not for small file.

It's tested under windows XP, would the result be the same under a Unix OS? Don't know ...



import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.util.HashMap;
import java.util.Map;

import junit.framework.TestCase;

import org.apache.log4j.Logger;

public class NIOTest  extends TestCase {
 
 private static Logger  logger= Logger.getLogger(NIOTest.class);
 public class NIOReader implements Reader {

  private ByteBuffer byteBuffer;
  
  public NIOReader(int bufferSize, boolean direct) {
   if (direct) {
    this.byteBuffer = ByteBuffer.allocateDirect(bufferSize);
     
   } else {
    this.byteBuffer = ByteBuffer.allocate(bufferSize);
   }
   
  }

  public String readFile(File f) throws IOException {
   FileInputStream fis = null;
   FileChannel channel = null;
   StringBuilder sb = new StringBuilder((int)f.length());
   
   try {
    fis = new FileInputStream(f);
    
    channel = fis.getChannel();
    
    
    while(channel.read(byteBuffer) != -1) {
     byteBuffer.flip();
     sb.append(byteBuffer.toString());
     byteBuffer.clear();
    }
    
   } finally {
    if (fis != null)
     fis.close();
   }
   
   
   return sb.toString();
  }

 }
 public class IOReader implements Reader {

  private byte[] buffer;

  public IOReader(int bufferSize) {
   this.buffer = new byte[bufferSize];
  }

  public String readFile(File f) throws IOException {
   FileInputStream fis = null;
   
   StringBuilder sb = new StringBuilder((int)f.length());
   
   try {
    fis = new FileInputStream(f);
    
    int c = -1;
    
    while( (c = fis.read(buffer)) != -1) {
     sb.append(new String(buffer, 0 , c));
    }
    
   } finally {
    if (fis != null)
     fis.close();
   }
   
   
   
   
   return sb.toString();
  }

 }

 File dir = new File ("C:\\my directory with files");
 
 int size = 3;
 int buffer = 4096;
 
 public void testIO() throws IOException {
  for (int i = 0 ; i < size; i++) {
   Map stopWatches = new HashMap();
   
   logger.info("test io");
   stopWatches.put("io", testLoop(size, new IOReader(buffer), dir));
   logger.info("test nio");
   stopWatches.put("nio", testLoop(size, new NIOReader(buffer, false), dir));
   logger.info("test nio direct");
   stopWatches.put("nio direct", testLoop(size, new NIOReader(buffer, true), dir));
   
   
   logger.info(stopWatches.toString());
  }
 }
 
 public StopWatch testLoop(int size, Reader reader, File dir) throws IOException {
  StopWatch stopWatch = new StopWatch();
  
  stopWatch.start();
  
  for(int i = 0; i < size; i ++) {
   test(reader, dir);
  }
  
  stopWatch.stop();
  
  logger.info("end loop " +stopWatch);
  return stopWatch;
  
 }
 
 public void test(Reader r, File dir) throws IOException {
  for(File f: dir.listFiles()) {
   if (f.isFile()) {
    String str = r.readFile(f);
    //logger.debug("read "+ str.length()+" in "+f);
   }
  }
 }
 
 interface Reader {
  String readFile(File f) throws IOException;
 }

}

7.4 Preventing Castor from checking for a default constructor (since 0.9.5)

2007-11-29T09:39:00.001-08:00

Sometimes it's useful to prevent Castor from checking for a default constructor, such as when trying to write a mapping for an interface or type-safe enum. You can use the "undocumented" verify-constructable="false" attribute on the element to prevent Castor from looking for the default constructor.

been looking for this one ...
See

Interesting post ab out Database Partitioning using Spring

2007-11-28T01:36:00.000-08:00

Horizontal Database Partitioning with Spring and Hibernate

Castor elusive StackOverflowError

2007-11-01T06:10:00.000-07:00

On one of my project, I'm using Casor - 1.1.2.1 - to serialize an object tree to xml. At the beginning of the week I had a weird StackOverflowError when writing a big set of objects. I did not pay much attention do it, was not a priority.

Today I had the same error, on a very reasonably size object tree. So I try to find where it was comming from - a cycling mapping maybe ... -. I putted my eclipse in debug mode, and add a breakpoint on the error. And then no more SOE ... but a error when setting the mapping in the Marshaller... weird ...

What can it be? that looks like a multi thread issue... But the Marshaler are both distinct, the only common thing is the Mapping. Could it be ?
So I replaced


 Marshaller marshaller = new Marshaller(serializer
   .asDocumentHandler());
 if (mapping != null) {
     marshaller.setMapping(mapping);
 }
 marshaller.marshal(o);

with


 Marshaller marshaller = new Marshaller(serializer
   .asDocumentHandler());
 if (mapping != null) {
  synchronized (mapping) {
   marshaller.setMapping(mapping);
  }
 }
 marshaller.marshal(o);

And it solve my problem.

Conclusion
Castor Mapping object is not Thread Safe...

How weird....

Comparing double with relative precision

2007-09-27T02:11:00.001-07:00


  private static final int DOUBLE_PRECISION = Math.getExponent(0.00000000000001);

 
  public static boolean equalsDouble(double d1, double d2) {
    double diff = Math.abs(d1 - d2);

    if (diff == 0.0) {
      return true;
    }
  
    int minExp = Math.min(Math.getExponent(d1), Math.getExponent(d2));
  
    int diffExp = Math.getExponent(diff);
  
    int relExp = diffExp - minExp;
  
    return  relExp <= DOUBLE_PRECISION;
 }

The test :

 
  public void testDoubleEquals() {
    assertTrue( Utils.equalsDouble(0.0, 0.0));
    assertTrue( Utils.equalsDouble(123.56, 123.56));
    assertTrue( Utils.equalsDouble(15698965.23659898, 15698965.23659898));

    assertTrue( Utils.equalsDouble(1.000000000000001, 1.000000000000000));
    assertTrue( Utils.equalsDouble(1000000000000001d, 1000000000000000d));
    assertTrue( Utils.equalsDouble(0.000000000000001000000000000001d, 0.000000000000001d));
  
    assertFalse( Utils.equalsDouble(1.00000000000011, 1.00000000000001));
    assertFalse( Utils.equalsDouble(100000000000011d, 100000000000001d));
    assertFalse( Utils.equalsDouble(0.00000000000000100000000000011d, 0.00000000000000100000000000001d));

    assertFalse( Utils.equalsDouble(15d, 1d));
    assertFalse( Utils.equalsDouble(15d, 15.6d));

 }

Cursors random walks

2007-09-20T02:24:00.000-07:00

I was working on an existing database with a simple table like this


CREATE TABLE T (
 id int,
 name varcher(255),
 job_id int
)

there is about 400 rows in this table, and the DB design is quite poor.

I decided to refactor a procedure that was going through the table T using a counter and assuming that it would match the id. It was working fine but it's kind of hard to imagine a weirder way of parcouring a table.

It did look like this


  select @insertRowCount = count(*) from T
    
  while @counter < @insertRowCount
  begin
   select @name = name from T where id = @counter
   
   ...
   
   set @counter = @counter+1
  end

The most obvious modification was using a cursor instead of this counter thing.


  declare Tcursor cursor for 
   select  name , id
   from T

  open jobInfo

  fetch jobInfo into  @name, @id

  while(@@sqlstatus=0)
  begin

   fetch jobInfo into  @name, @id
  end
  
  close jobInfo

Looks better, doesn't it?

except that the cursor did not return all the row ... Kind of weird.

After a bit of googling I find my answer, it need a unique index, which you've got normaly with a primary key...


  create unique index PK_T on T(id)

PS: I do know such database is badly design...

Micro benchmark - Collection sorting

2007-08-24T05:17:00.000-07:00

What is the most efficient



    List list = new ArrayList();
    
    for(Long l : array) {
     list.add(l);
    }
    
    Collections.sort(list);



    TreeSet list = new TreeSet();

    for(Long l : array) {
     list.add(l);
    }

After a micro benchmark, on 1000 random arrays of 100 000 element

	Total Time (ms)	Average Time (ms)	Min Time (ms)	Max Time (ms)
Collections.sort	81 893	81	62	125
TreeSet	187 578	187	140	281

In the Benchmark the Collections.sort takes 43 % of the time that it takes using the Tree.

Normalization under attack

2007-08-16T01:33:00.001-07:00

Data normalization, is it really that good?

Xalan-specific output properties nightmare

2007-07-02T06:48:00.000-07:00

At work we are using Xalan 2.7.0, it works fine except that for some reason we would like to modify a bit the html output method. This method strip the attribute value for readonly and disabled attribute. E.g :

is output as

that's fair enough but not the behaviour we need. So I start looking for a way to change that. I quicly found the way the method influence the Serializer. If you like in the output_html.properties you'll find

# Xalan-specific output properties. These can be overridden in the stylesheet
# assigning a xalan namespace. For example:
#
# xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
# xmlns:xalan="http://xml.apache.org/xalan">
#
# xalan:content-handler="MyContentHandler"/>;
# ...
# Note that the colon after the protocol needs to be escaped.

That was a great start. So how does the html content handler work ? Would it be possible to easily change its behaviour ? At first I thought so. The code interesting here is in the method protected void processAttribute(java.io.Writer writer, String name, String value, ElemDesc elemDesc) throws IOException

/**
   * Process an attribute.
   * @param   writer The writer to write the processed output to.
   * @param   name   The name of the attribute.
   * @param   value   The value of the attribute.
   * @param   elemDesc The description of the HTML element
   *           that has this attribute.
   *
   * @throws org.xml.sax.SAXException
   */
  protected void processAttribute(
      java.io.Writer writer,
      String name,
      String value,
      ElemDesc elemDesc)
      throws IOException
  {
      writer.write(' ');

      if (   ((value.length() == 0) || value.equalsIgnoreCase(name))
          && elemDesc != null
          && elemDesc.isAttrFlagSet(name, ElemDesc.ATTREMPTY))
      {
          writer.write(name);
      }
      else
      {
          // %REVIEW% %OPT%
          // Two calls to single-char write may NOT
          // be more efficient than one to string-write...
          writer.write(name);
          writer.write("=\"");
          if (   elemDesc != null
              && elemDesc.isAttrFlagSet(name, ElemDesc.ATTRURL))
              writeAttrURI(writer, value, m_specialEscapeURLs);
          else
              writeAttrString(writer, value, this.getEncoding());
          writer.write('"');

      }
  }

So when the ElemDesc which is defined in the ToHtmlStream got the attribute with the fllag ATTREMPTY it does that. Easy ! so we probably can extend this class and change the behaviour of processAttribute? No! Is there an easy way to change the ElemDesc? No! Maybe we can just copy and past the class? Not really without putting it in the jar, solution a bit to risky because the Xalan Jar is probably shared.

So the solution? using reflexion to do a deep copy of the specific not very friendly Trie object - we want that only on the specified page -, and create an Handler that gonna delegate to the classic ToHtmlStream to which we are gonna change the root of the Trie object to point to the copy with the modified ElemDesc ... It does work but it's a nightmare of 200 lines of reflexion that used the name of private field of private inner class ...


public class NewToHTMLStream implements SerializationHandler {

    ToHTMLStream stream = null;

    Object trieCopy = null;

    static final Set ATTRIBUTES = new HashSet(Arrays.asList(new String[] { "READONLY", "DISABLED" }));

    static final Object lock = new Object();

    private static Field fieldm_htmlInfo;

    private static Field field_trie_root;

    private static Field field_trie_node_nodes;

    private static Field field_trie_node_value;

    private static Field field_m_elementFlags;

    private static Field field_ElemDesc_m_attrs;

    private static Field field_ElemDesc_m_Flags;

    private static Method elementDesc_setAttr;

    private static Constructor eltDescConstructor;

    private static Constructor nodeConstructor;

    private static Constructor constructorTrie;

    private static Field fieldMMap;

    private static Field fieldMValues;

    private static Field fieldMFirstFree;

    public NewToHTMLStream() throws IllegalArgumentException, NoSuchFieldException, IllegalAccessException, SecurityException, NoSuchMethodException, InvocationTargetException, InstantiationException {

        stream = new ToHTMLStream();

        synchronized (lock) {
            if (trieCopy == null) {
                trieCopy = copyTrie();
            }
        }

        
        Object htmlInfo = fieldm_htmlInfo.get(stream);
        
        field_trie_root.set(htmlInfo, field_trie_root.get(trieCopy));

    }

    public static Object copyTrie() throws NoSuchFieldException, IllegalArgumentException, IllegalAccessException, SecurityException, NoSuchMethodException, InvocationTargetException, InstantiationException {
        initReflexion();

        Object m_elementFlags = field_m_elementFlags.get(ToHTMLStream.class);
        Object newTrie = constructorTrie.newInstance(new Object[] { m_elementFlags });

        Object root = (Object) field_trie_root.get(m_elementFlags);

        Object newRoot = copyNode(root, newTrie);

        field_trie_root.set(newTrie, newRoot);

        return newTrie;

    }

    private static Object copyNode(Object root, Object trie) throws IllegalArgumentException, InstantiationException, IllegalAccessException, InvocationTargetException {
        Object copy = nodeConstructor.newInstance(new Object[] { trie });

        field_trie_node_value.set(copy, copyElementDescription(field_trie_node_value.get(root)));

        Object[] originalNodes = (Object[]) field_trie_node_nodes.get(root);
        Object[] newNodes = (Object[]) field_trie_node_nodes.get(copy);

        for (int i = 0; i < originalNodes.length; i++) {
            Object n = originalNodes[i];
            if (n != null) {
                newNodes[i] = copyNode(n, trie);
            }
        }

        return copy;
    }

    private static ElemDesc copyElementDescription(Object o) throws IllegalArgumentException, InstantiationException, IllegalAccessException, InvocationTargetException {
        if (o == null)
            return null;

        ElemDesc desc = (ElemDesc) o;

        ElemDesc desc2 = eltDescConstructor.newInstance(new Object[] { field_ElemDesc_m_Flags.get(desc) });

        /*
         * m_attrs copy except ATTRIBUTES
         */

        StringToIntTable m_attrs = (StringToIntTable) field_ElemDesc_m_attrs.get(desc);

        if (m_attrs != null) {
            String[] m_map = (String[]) fieldMMap.get(m_attrs);
            int[] m_values = (int[]) fieldMValues.get(m_attrs);
            int m_firstFree = (Integer) fieldMFirstFree.get(m_attrs);

            for (int j = 0; j < m_firstFree; j++) {
                int flag = 0;
                String attributeName = m_map[j];
                if (!ATTRIBUTES.contains(attributeName)) {
                    flag = m_values[j];
                }

                elementDesc_setAttr.invoke(desc2, new Object[] { attributeName, flag });
            }
        }

        return desc2;
    }

    private static void initReflexion() throws NoSuchFieldException, NoSuchMethodException {
        fieldm_htmlInfo = setAccessible("m_htmlInfo", ToHTMLStream.class);

        field_m_elementFlags = setAccessible("m_elementFlags", ToHTMLStream.class);

        constructorTrie = field_m_elementFlags.getType().getDeclaredConstructor(new Class[] { field_m_elementFlags.getType() });
        constructorTrie.setAccessible(true);

        field_trie_root = setAccessible("m_Root", field_m_elementFlags.getType());
        field_trie_node_nodes = setAccessible("m_nextChar", field_trie_root.getType());
        field_trie_node_value = setAccessible("m_Value", field_trie_root.getType());

        nodeConstructor = field_trie_root.getType().getDeclaredConstructor(new Class[] { field_m_elementFlags.getType() });
        nodeConstructor.setAccessible(true);

        field_ElemDesc_m_attrs = setAccessible("m_attrs", ElemDesc.class);
        field_ElemDesc_m_Flags = setAccessible("m_flags", ElemDesc.class);
        elementDesc_setAttr = setAccessible("setAttr", new Class[] { String.class, Integer.TYPE }, ElemDesc.class);

        eltDescConstructor = ElemDesc.class.getDeclaredConstructor(new Class[] { Integer.TYPE });
        eltDescConstructor.setAccessible(true);

        fieldMMap = setAccessible("m_map", StringToIntTable.class);
        fieldMValues = setAccessible("m_values", StringToIntTable.class);
        fieldMFirstFree = setAccessible("m_firstFree", StringToIntTable.class);
    }

    private static Method setAccessible(String methodName, Class[] paramTypes, Class clazz) throws SecurityException, NoSuchMethodException {
        Method method = clazz.getDeclaredMethod(methodName, paramTypes);

        method.setAccessible(true);
        return method;
    }

    private static Field setAccessible(String fieldName, Class clazz) throws NoSuchFieldException {
        Field fieldMAttrs = clazz.getDeclaredField(fieldName);

        fieldMAttrs.setAccessible(true);
        return fieldMAttrs;
    }

    /**
     * DELEGATE to stream ...
     */