However, if you just want a dump of the current state of the heap – this is how you do it.

jps

Similar to ps. But shows only Java processes. Very nice! Use this to identify your target JVM pid. You can of course use the Unix ps command or the Windows tasklist command or the graphical TaskManager.

jmap -dump:file=heap.bin 924

That would dump the heap of the Java process with pid 924 into a file named heap.bin.

Simple enough, isn’t it? The tricky bit is to actually analyze a heap dump to extract information from all the data contained therein. I’ll write about analyzing heap dumps in a future post.

Got a quick tip with other useful ways to get heap dumps? Please leave a comment!

Static member classes – These will not hold on to a reference to the enclosing class. You do not need a reference to the enclosing class to instantiate this. This one is not really an inner class.

Non static member classes – These will hold on to a reference to the object of the enclosing class. Beware of leaks and concurrency issues. Always make it a static member class if you don’t use anything from the enclosing instance.

Anonymous classes – Good for one off use cases. Can reduce code readability if they get too large. Can be a bit more difficult to unit test.

Local classes – A class that is local to a method/block. Not used too often. Holds a reference to the enclosing instance.

But it is actually a form of something you use quite frequently – anonymous inner classes.

		
		// Normal list initialization
		List<String> days1 = new ArrayList<String>();
		days1.add("Sun");
		days1.add("Mon");
		days1.add("Tue");

		// Double braces initialization. Compact!
		List<String> days2 = new ArrayList<String>() {{
			add("Sun");
			add("Mon");
			add("Tue");
		}};

My suggestion is to NOT use it because it can cause serialization warnings and is just too unfamiliar for most Java developers. When in doubt, favour clarity!

Here is some CLI magic that helps you do so (sourced from CommandLineFu).

find -not -empty -type f -printf “%s\n” | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 –all-repeated=separate

The first column is the md5 hash and the second is the full path. The rows are grouped by files with similar content and separated by a line per group.

Oh, so beautiful! Use effectively to eliminate code and test smell from your project!

<!– standard.jar –>
<dependency>
<groupId>taglibs</groupId>
<artifactId>standard</artifactId>
<version>1.1.2</version>
</dependency>

<!– JSTL –>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>1.1.2</version>
</dependency>

Try restarting Eclipse if the error doesn’t go away after adding this.

Do drop me a comment if this or something else works for you!

[Esc]:set nowrap[Enter]

There, that will disable wrapping and your file will make sense once again.

I had a system with 6 gig of RAM. I allocated 500 meg for a RAM Disk. You can create one with one of the several RAMDisk programs out there. There are few which are freeware as well.

In MySql, you’ll need to edit your my.cnf to use data files located in a folder on the RAM Drive. Stop your DB, copy the existing DB files to the folder in the RAM Drive. The edited my.cnf should point to this folder. That’s all.

Note that you could also use a different storage engine – but that changes how the database works with respect to transactions and probably other things so I wouldn’t recommend this.

I managed to cut down DB build time from around 2 minutes to under 10 seconds!

I was also able to reveal a ConcurrentModificationException thanks to the DB/disk not being a bottleneck anymore when running some stress tests.

Go, give it a try today!

Sure, you could drop the jar into your localRepository while you work on it initially. But when you commit your code, the project repository still won’t have it. So others on your team would get a missing dependency error.

What you really need to do is to deploy the jar into your project repository. Here is what should work to do this -

mvn deploy:deploy-file -DgroupId=com.datacash -DartifactId=datacash-api -Dversion=2.1.1 -Dpackaging=jar -Dfile=DataCash-Java-2.1.1.jar -Durl=http://yourteamdevserver:8081/artifactory/libs-snapshots -DrepositoryId=snapshots

You can use any appropriate groupId and artifactId. Tools like Artifactory also have a web UI to install jars.

Do leave a comment if this or something else that you tried worked for you!

You may notice that if you look at the field in your database, you do see the time being persisted. So infact, it is something you are doing wrong when retrieving the field from the DB.

The reason why this problem occurs is in part because of poor API design.

/*
* What do you think this code does?
*/
Date date = resultSet.getDate("my_date_field");

What doesn’t strike most developers right away is that resultSet.getDate returns a java.sql.Date. And that java.sql.Date extends java.util.Date!

By definition, a java.sql.Date has only Date – no time! But it is assignable to a java.util.Date which is the norm in typical application code. That’s where your time part probably gets eaten up due to an incorrect getter for the JDBC resultSet.

Similarly, java.sql.Time is returned by resultSet.getTime. It can be assigned to a java.sql.Date. In this case, it is the date part that tends to mysteriously disappear.

If you want to do this right -

• Make sure you know what your DB uses for the column – a TIMESTAMP field or a DATETIME field or a DATE field.
• Use the right getter from the ResultSet class – Date, Time and Timestamp are all compatible with java.util.Date but only Timestamp contains the date as well as the time.
• Use the correct setter when saving data to your DB as well.

While we are on the topic of dealing with Date and Time in Java, have you considered ditching the primitive Java Date / Calendar API and adopting Joda-Time instead? If you are at an early stage in your project, that is something you definitely need to look at.

In Windows, doing the equivalent requires sending a Ctrl+Break to the input console which results in a SIGBREAK being sent to the process. But what if you do not have an input console? A process started with javaw.exe like what Eclipse does has no input console where you can type in the Ctrl+Break.

So, if you have a running java process without an input console, how do you make it do a thread dump?

The answer is simple – use a utility called SendSignal. Very cleverly written stuff. The author’s description of how he wrote that and some other utilities as part of his Windows programming interests is a nice read.

So, download SendSignal and keep it handy as part of your developer arsenal.

SendSignal.exe <pid> will send a ctrl+break to the process. This will cause a Java process to do a stack dump to stdout.

There are other ways too. Those are for another post.

// Unncessary code
if (data != null && data instanceof InterestingData) {

}

// Less code. Better!
if (data instanceof InterestingData) {

}

2. A null can be cast to anything without causing a NullPointerException.

// Not so good...
if (someReference == null) {
	return null;
}
else {
	return (SpecificReference) genericReference;
}

// Better.
return (SpecificReference) genericReference;

3. Autoboxing / Unboxing can cause a NullPointerException on lines that can leave you confounded. This generally happens when a Long (a wrapper type) is passed or received as an argument for a long (a primitive type) parameter. So watch out for these when trying to figure out a stack trace that does not seem to make sense.

4. a < b – This can result in a surprising NPE! This happens when wrapper types are being compared and one of them happens to be a null.

5. The equals() method should not be returning true when a null is passed to it. – The general contract of the equals method states that if null is passed to equals, false should be returned. Makes sure you do this in your code. It is easy to miss and cause a NPE instead.

6. While chained statements are nice to look at in the code, they are not NPE friendly. A single statement spead over several lines will give you the line number of the first line in the stacktrace regardless of where it occurs.

7. A reference to “this” can never be null. While it should be obvious, I have seen code that does a null check on this. Ack!

8. An awesome tip to avoid NPE is to return empty strings or empty collections rather than a null. Do this consistently across your app. You will note that a bucket load of null checks become unneeded if you do so.

9. You should put constraint checks at the beginning of your method so that the rest of your code does not have to deal with that possibility. So if someone passes in a null, things will break (IllegalArgumentException perhaps?) early in the stack rather than in some deeper location where the root problem will be difficult to identify. Aiming for fail fast behaviour is a good choice in most situations.

Or consider allowing a null pointer exception to happen if a null is passed in rather than check for a null and then doing nothing and returning. This is akin to swallowing an exception.

10. The next time someone says Java does not have pointers, ask them what causes a NullPointerException. Watch their facial expression change as they protest.

Ok, ok. I know, Java references and C++ pointers are different animals. Yes, I know about arithmetic on pointers from C / C++. Oh yes, I know about the Unsafe class from Sun Java too.

Go ahead, Null Ninja! Apply these tips and push that cyclomatic complexity down, down, down!

class Crash {

	public static void main(String[] args) {

		Object[] link = null;

		while (true) {
			link = new Object[] { link };
		}

	}
}

What has happened here is that the deep chain of objects we have allocated here causes a StackOverflow on what appears to be a recursive call in the garbage collector.

Usually such errors happen because of misbehaving drivers. Perhaps your app uses a finger print scannner or a printer or a scanner? Any JNI code going wrong causes this.

A common pattern when dealing with native code in Java is to avoid doing the calls in the main process of your application. Instead, have a different process that does the dirty work via JNI and communicate with that process via some mechanism – perhaps sockets. So if something goes wrong in JNI, only that process with gets killed rather than the entire app. Remember, with JNI exceptions, you get a dead JVM rather than an exception. This is because there is no telling what a misbehaving piece of native code may have done to the memory within a process.

My hs_err_pid####.log file -

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_STACK_OVERFLOW (0xc00000fd) at pc=0x6d8b99f0, pid=3780, tid=2232
#
# JRE version: 6.0_21-b07
# Java VM: Java HotSpot(TM) Client VM (17.0-b17 mixed mode, sharing windows-x86 )
# Problematic frame:
# V  [jvm.dll+0x99f0]
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x02b09c00):  VMThread [stack: 0x02bf0000,0x02c40000] [id=2232]

siginfo: ExceptionCode=0xc00000fd, ExceptionInformation=0x00000001 0x02bf2ffc 

Registers:
EAX=0x22e62fc4, EBX=0x22e62fc0, ECX=0x371bb3a0, EDX=0x00000001
ESP=0x02bf3000, EBP=0x02bf3014, ESI=0x22e62fc0, EDI=0x22e62fbc
EIP=0x6d8b99f0, EFLAGS=0x00010246

Top of Stack: (sp=0x02bf3000)
0x02bf3000:   6da2d162 22e62fc4 22e62fbc 22e62fc0
0x02bf3010:   22e62fc0 02bf302c 6da2d1de 22e62fc0
0x02bf3020:   22e62fac 22e62fb0 22e62fb0 02bf3044
0x02bf3030:   6da2d1de 00000003 22e62f9c 22e62fa0
0x02bf3040:   22e62fa0 02bf305c 6da2d1de 00000003
0x02bf3050:   22e62f8c 22e62f90 22e62f90 02bf3074
0x02bf3060:   6da2d1de 00000003 22e62f7c 22e62f80
0x02bf3070:   22e62f80 02bf308c 6da2d1de 00000003 

Instructions: (pc=0x6d8b99f0)
0x6d8b99e0:   5d c2 08 00 cc cc cc cc cc cc cc cc cc cc cc cc
0x6d8b99f0:   55 8b ec 8b 45 08 56 8b 30 85 f6 74 40 8b 0e 83 

Stack: [0x02bf0000,0x02c40000],  sp=0x02bf3000,  free space=c02bf2b74k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0x99f0]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
V  [jvm.dll+0x17d1de]
......

JavaThread 0x002b6400 (nid = 2948) was being processed
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
v  ~RuntimeStub::new_object_array Runtime1 stub
J  Crash.main([Ljava/lang/String;)V
v  ~StubRoutines::call_stub
VM_Operation (0x0095fa14): GenCollectForAllocation, mode: safepoint, requested by thread 0x002b6400

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x02b25000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=5292, stack(0x02dd0000,0x02e20000)]
  0x02b16400 JavaThread "CompilerThread0" daemon [_thread_blocked, id=4848, stack(0x02d80000,0x02dd0000)]
  0x02b14c00 JavaThread "Attach Listener" daemon [_thread_blocked, id=5384, stack(0x02d30000,0x02d80000)]
  0x02b13800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=528, stack(0x02ce0000,0x02d30000)]
  0x02b0fc00 JavaThread "Finalizer" daemon [_thread_blocked, id=324, stack(0x02c90000,0x02ce0000)]
  0x02b0b400 JavaThread "Reference Handler" daemon [_thread_blocked, id=5940, stack(0x02c40000,0x02c90000)]
  0x002b6400 JavaThread "main" [_thread_blocked, id=2948, stack(0x00910000,0x00960000)]

Other Threads:
=>0x02b09c00 VMThread [stack: 0x02bf0000,0x02c40000] [id=2232]
  0x02b38800 WatcherThread [stack: 0x02e20000,0x02e70000] [id=4228]

VM state:at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x002b55f0] Threads_lock - owner thread: 0x02b09c00
[0x002b5a00] Heap_lock - owner thread: 0x002b6400

Heap
 def new generation   total 4736K, used 512K [0x22990000, 0x22eb0000, 0x27c30000)
  eden space 4224K,   0% used [0x22990000, 0x22990000, 0x22db0000)
  from space 512K, 100% used [0x22e30000, 0x22eb0000, 0x22eb0000)
  to   space 512K,   0% used [0x22db0000, 0x22db0000, 0x22e30000)
 tenured generation   total 12160K, used 12091K [0x27c30000, 0x28810000, 0x32190000)
   the space 12160K,  99% used [0x27c30000, 0x287fed48, 0x287fee00, 0x28810000)
 compacting perm gen  total 12288K, used 25K [0x32190000, 0x32d90000, 0x36190000)
   the space 12288K,   0% used [0x32190000, 0x32196528, 0x32196600, 0x32d90000)
    ro space 10240K,  54% used [0x36190000, 0x3670c0f0, 0x3670c200, 0x36b90000)
    rw space 12288K,  55% used [0x36b90000, 0x3722fb80, 0x3722fc00, 0x37790000)

Dynamic libraries:
0x00400000 - 0x00424000 	E:\Java\jdk1.6.0_21\bin\java.exe
0x7c900000 - 0x7c9b2000 	D:\WINDOWS\system32\ntdll.dll
0x7c800000 - 0x7c8f6000 	D:\WINDOWS\system32\kernel32.dll
0x77dd0000 - 0x77e6b000 	D:\WINDOWS\system32\ADVAPI32.dll
0x77e70000 - 0x77f03000 	D:\WINDOWS\system32\RPCRT4.dll
0x77fe0000 - 0x77ff1000 	D:\WINDOWS\system32\Secur32.dll
0x7c340000 - 0x7c396000 	E:\Java\jdk1.6.0_21\jre\bin\msvcr71.dll
0x6d8b0000 - 0x6db57000 	E:\Java\jdk1.6.0_21\jre\bin\client\jvm.dll
0x7e410000 - 0x7e4a1000 	D:\WINDOWS\system32\USER32.dll
0x77f10000 - 0x77f59000 	D:\WINDOWS\system32\GDI32.dll
0x76b40000 - 0x76b6d000 	D:\WINDOWS\system32\WINMM.dll
0x76390000 - 0x763ad000 	D:\WINDOWS\system32\IMM32.DLL
0x6d860000 - 0x6d86c000 	E:\Java\jdk1.6.0_21\jre\bin\verify.dll
0x6d3e0000 - 0x6d3ff000 	E:\Java\jdk1.6.0_21\jre\bin\java.dll
0x6d340000 - 0x6d348000 	E:\Java\jdk1.6.0_21\jre\bin\hpi.dll
0x76bf0000 - 0x76bfb000 	D:\WINDOWS\system32\PSAPI.DLL
0x6d8a0000 - 0x6d8af000 	E:\Java\jdk1.6.0_21\jre\bin\zip.dll

VM Arguments:
java_command: Crash
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=E:\Java\jdk1.6.0_21
CLASSPATH=.;E:\Java\jre6\lib\ext\QTJava.zip
PATH=E:\Java\jdk1.6.0_21\bin;D:\WINDOWS\SYSTEM32;D:\WINDOWS;D:\WINDOWS\SYSTEM32\WBEM
USERNAME=Onkar Joshi
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 6 Model 10 Stepping 0, AuthenticAMD

---------------  S Y S T E M  ---------------

OS: Windows XP Build 2600 Service Pack 3

CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 10 stepping 0, cmov, cx8, fxsr, mmx, sse, mmxext, 3dnow, 3dnowext

Memory: 4k page, physical 1015152k(349348k free), swap 1974304k(600992k free)

vm_info: Java HotSpot(TM) Client VM (17.0-b17) for windows-x86 JRE (1.6.0_21-b07), built on Jul 17 2010 01:10:15 by "java_re" with MS VC++ 7.1 (VS2003)

time: Mon Mar 07 02:40:13 2011
elapsed time: 0 seconds

A problem that you can run into when attempting to do this is that since the code runs right in the beginning, the JVM may rush past it before you have had a chance to attach a debugger via Eclipse and set your break points.

The solution is to setup your JVM/debugging config so that it is set to suspend at startup.

A typical way to setup remote debugging is to use the following JVM argument in the code used to launch your application so that a debugger can later attach itself to port 8000.

-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000

Infact this is what Eclipse does when you launch an app in debug mode rather than normal mode. It adds this line in the JVM arguments used when launching your application.

Note the suspend=n. If that is set to y, the JVM will suspend on startup. It will wait for a debugger to attach to it and release it before it will proceed.

So using the suspend flag, our problem is solved. Happy debugging!

referenceToSomethingImportant = referenceToSomethingImportant;

You immediately delete it since assigning a variable to itself will never do any thing. Of course the compiler or the JIT machinery is always going to optimize it away anyway. Right?

Wrong.

Oh yes, there might be another thread you exclaim. But then doing something that has no effect will still have no effect when multiple threads are doing it, eh? Then what?

You are neglecting to consider the fact that read-write-assignment can do something other then memory operations to store data. Visibilty. That’s the key.

If “value” were declared to be volatile, the JVM now cannot optimize away everything from that line that you decided to remove. Because besides the data, there is now a memfence / mfence / membar / barrier / whatever-for-your-architecture instruction that is also being inserted as part of that bit of code when it is translated to assembly language for your machine.

This is to make the happens before guarantees associated with volatile reads and writes work correctly. Specifically, a happens before at that pont was caused by the volatile read / write. That caused changes up to that point by the thread to be flushed out of what could have been buffered in the SRAM (static RAM) on a specific core in the CPU causing it to be visible to other the reads.

There are situations where piggybacking the flush of such changes caused by operations on volatile variables can help in squeezing out the last bits of performance in concurrent code by avoiding synchronization which has the overhead of locking besides providing the same visibility guarantees as volatile does.

Highly unrecommended for anyone but authors of concurrency libraries!

The common, simplistic understanding of the volatile keyword in Java is that if a variable is being modified by multiple threads, it should be marked volatile so that changes can be seen by other threads. (There is more to it that I’ll leave for a future post.)

If you had something like a shared boolean primitive marked volatile, this would work fine. However, when an array is marked volatile, it is only the array reference that is marked volatile! The contents of array itself do not inherit the memory visibility semantics of the volatile keyword at all.

So what you have is not an array of volatile primitives. Instead, you have a volatile array of non-volatile primitives.

This means that even if you declare an array to be volatile, modifying the contents of  that array does not guarantee that those modifications will be seen by a different thread reading the contents of the array. However, if you update the reference to point to a new array object – this new array will correctly be seen by threads reading the reference. (No JVM guarantee about seeing changes to the contents of the new array though!)

So what is the correct way to share contents of a mutable array with multiple threads? Use the Atomic*Array classes in the java.util.concurrent.atomic package.

AtomicLongArray, AtomicIntegerArray, AtomicReferenceArray<V> are your friends.

“No AtomicFloatArray and AtomicDoubleArray!”, you exclaim. Well, recall that a float is 32 bits and so is an integer. A double is 64 bits and so is a long. Use the Double.doubleToLongBits and Float.floatToIntBits methods to your advantage. Simple, isn’t it?

An alternative way to get volatile semantics for an array of primitives is to wrap the primitive with a wrapper that declares the primitive volatile. Accessing the volatile field from the array of wrappers will give you the desired volatile semantics.

What about an array of AtomicReference<V> variables? (AtomicReference<V>[] array;) – Yes, this is going to work too. But you should probably use AtomicReferenceArray<V> instead. It uses less memory which can be significant if you have millions of values in the array.

There is more to volatile – but that’s for another post.

I also noticed that a pop-up window had been launched on my system. And that window had an interesting URL.

http://someSMSGateway.com/blah/foo.php?username=abc&pass=pqr&mobile=9890123456&message=theMessageIGotAsAnSMS – So, the brilliant programmers for that site decided to send their username, password for the SMS gateway to my browser and launched a request to send the SMS from my system via JavaScript!

Heh. I copied that off. Pasted it into my browser again, I got the SMS on my mobile again. Modified the message a bit. Got the modified message.

It is difficult to imagine what they thought when they decided to do this. Maybe some confusion over what happens on the server side and what happens at the client in PHP pages when using JavaScript? Or maybe someone decided to “optimize” things by offloading the task of sending messages to the client side. The URL itself was not easily visible when I used Internet Explorer rather than Firefox. Perhaps they didn’t even test in Firefox – never realizing that their account info with the SMS gateway was so clearly visible to their users.

I suppose it won’t be long before they go crying to the cops saying, “Hackers used our username, password to send out a million spam messages.”

The data is in the format (orderId, storeIdentifier, amountDue).

What optimization can you do here?

Note that the storeIdentifier is going to be repeated a lot. Every time you read a record and split it into a string and possibly store it into an in-memory data structure, you will be creating a new String object. So 100 million String objects will be created for the storeIdentifier. But you know that there are only (say) 100 stores in all! So there is a massive amount of wasted memory.

What you can do here is – Right after you have read the storeIdentifier string, do this -

storeIdentifier = storeIdentifier.intern();

That would put the store identifier into the String pool and keep the number of String instances with the same data minimal by returning the String from the pool once it has been put into it by the first invocation for the string.

Nuggets of wisdom

1. Use intern() only if you really need to use it. And only if you know the extra instances are going to be a problem. And only if you really understand how it works.
2. Older JVMs had a problem collecting interned strings. Newer JVMs handle this fine. Don’t worry about leaks due to a growing pool. If other references are gone, interned strings will be collected by the GC.
3. Interned strings go into the PermGen Space area of memory in some JVMs. This is not part of the normal heap. If you send too many strings here, an OutOfMemoryError will hit you even though your heap may have several GB available.
4. Interned strings can be compared with == rather than .equals(). This is a bit faster. But it is rarely worth the brittle code.
5. Calling String.intern() can be a performance hit. It takes CPU cycles to maintain the pool and do the comparisons. Are you sure you are saving enough memory to make it worth the CPU? Measure. Don’t guess.
6. Use String.intern() only if the set of possible Strings that will be interned has a bound tight enough such that the set of different strings is much smaller than the total number of strings that will be read.

The thread stack is used to push stacks frames in nested method calls. If the nesting is so deep that the thread runs out of space, the thread dies with a StackOverflowError.

The default thread stack size varies with JVM, OS and environment variables. A typical value is 512k. It is generally larger for 64bit JVMs because references are 8 bytes rather than 4 bytes in size. This means that if your app uses 150 threads, 75MB will be used for thread stacks. In some environments the defaults stack may be as large as 2MB. With a large number of threads, this can consume a significant amount of memory which could otherwise be used by your application or OS.

In most applications, 128k happens to be enough for the stack. What you really need to do is adjust and observe. If you don’t see your app running out of stack space, use the -Xss JVM parameter to specify a smaller stack (-Xss128k).

Note that it is entirely possible that your OS rounds up values for stack size specified by your -Xss parameter. Watch out for that.

Using the program below, you can see how stack space is used up by methods with varying number of arguments. You’ll get the StackOverflowError with fewer nested method calls for a smaller stack. You could try adjusting the number of arguments and even the type of arguments for different behaviour.

By adjusting the stack size, and keeping the code the same, you can see the JVM dying at different points in the recursive call.

Such methods should probably have been provided with the JDK in the first place. You can add such methods to a Util class in your application for each type of collection that you wish to use and save your self the pain of generic collection initialization throughout your code base.

The methods in the Util class below use the type inference feature in the JDK to infer the appropriate type at compile time.

Nice, eh? Go ahead and use it in your code!

You look at the File class for ways to do this. You notice the getFreeSpace() method. And you use it to do your checks for the disk space.

Voops! That may work. But it may not. Depends on your operating system and environment configuration.

You should actually be using the getUsableSpace() method. This method does a few additional checks, where possible, about write permissions and quota available to the user. Also, on some systems a certain percentage of total disk space is reserved for admin/root users. This can make your checks for space pass incorrectly if you happen to use the getFreeSpace() method. And you won’t even find out until it starts affecting your app in a different environment on your prod boxes.

Now that you’ve read this, don’t make that mistake. Use the getUsableSpace() method. And send your colleagues a link to this article to make sure they know to do so too.

Solution
You can easily get a ConcurrentHashSet with the following code -

Collections.newSetFromMap(new ConcurrentHashMap<Object,Boolean>())

Notes

• A Set lends itself to implementation via a Map if you think about it. So can actually just use a Map. But that may not fit in well with the context of your use.
• The HashSet class internally uses a HashMap.
• The ConcurrentHashSet obtained via the method inherits pretty much all the concurrency features of the underlying collection.
• See api-docs for newSetFromMap

Rather, they descend from VirtualMachineError – “Thrown to indicate that the Java Virtual Machine is broken or has run out of resources necessary for it to continue operating.”

Nothing terribly useful. But good to know nonetheless for a Java developer.

Transfer only the difference!

In almost all cases, the files that are changed between builds are very few.

In one of the cases where my team encountered this, they used to transfer 25MB EAR files over a very slow network to be deployed in a different location – this took them around 20 minutes each time. By transferring only the changes, we were able to bring down the data that needed to be transferred to only 1MB or so. That was a massive productivity boost for the team.

In another case, my team transferred build artifacts around 1 GB in size. After a few tests, I found that the difference generated for these between subsequent builds was only 50MB or so!

In more practical terms: Say an “oldFile” is available at both locations and “newFile” is available at one of them. You generate the “patchFile” where both newFile and oldFile are present. You transfer patchFile to where you have just oldfile. Apply patchFile to the oldFile. The newFile is now available to you here too!

You can compress the patch file too. Unless your tool already does so (a flag perhaps?), you should probably do so unless your patches are small enough anyway.

There are multiple utils available to this. They are generally referred to as binary diff or byte level differencing tools and use some very clever algorithms to do what they do.

xdelta - I used this one. Produced smaller patches for me. Free!

bsdiff / bspatch - Part of Unix/Linux type systems. Standard commands which are always available. Free!

RTPatch - Enterprise features. Costs moolah.

Go ahead, save a bunch of time for your team today!

Here is how you can set one. While you are at it, you probably want to define a few more shortcuts to functionality you use frequently.

In Eclipse, go to Window > Preferences > Type in ‘Keys’ in the search box > Select ‘Keys‘ to get the screen to edit the command bindings.

Search for the ‘Skip All Breakpoints’ command. Add your preferred shortcut at the Binding field. I use Ctrl+Alt+B.

If you toggle that bit of functionality as often as I do, you know this is gonna save you a bunch of time.

Any non-default bindings you use all the time? Lemme know!

/dev/random will block after the entropy pool is exhausted. It will remain blocked until additional data has been collected from the sources of entropy that are available. This can slow down random data generation.

/dev/urandom will not block. Instead it will reuse the internal pool to produce more pseudo-random bits.

/dev/urandom is best used when:

• You just want a large file with random data for some kind of testing.
• You are using the dd command to wipe data off a disk by replacing it with random data.
• Almost everywhere else where you don’t have a really good reason to use /dev/random instead.

/dev/random is likely to be the better choice when:

• Randomness is critical to the security of cryptography in your application – one-time pads, key generation.

org.hibernate.HibernateException – “A collection with cascade=”all-delete-orphan” was no longer referenced by the owning entity instance: com.onkarjoshi.hibernate.test.Person.children”

Your likely problem

Looks like you have a Parent entity and a Child entity. The Parent has a collection of Child entities with cascade=”all-delete-orphan”.

You are setting a new collection via the setter thus leaving the original collection unreferenced by the Parent entity. This doesn’t fly well with Hibernate and leaves it confused about what to do.

parent.setChilden(new HashSet<Child>()); // This won’t work. Could be an ArrayList too.

parent.getChildren().clear(); // There, fixed that!

So generally speaking, just clear out the old collection, rather than dereferencing it and creating a new one.

Strongly reachable

• An object is strongly reachable if it can be reached by some thread without traversing any reference objects. A newly-created object is strongly reachable by the thread that created it.

This is the reference you know about and use all the time. Any object that is reachable by a strong reference (from a live thread) will not be collected by the GC.

Softly reachable

• An object is softly reachable if it is not strongly reachable but can be reached by traversing a soft reference.

The garbage collector will collect softly reachable objects only if it actually requires more memory. Before it does a GC it is guaranteed to collect softly reachable objects.

So it is possible to write code on the lines of – “Hey, I do want this data to be kept in memory. But in case the JVM starts running out of memory, go ahead and collect it and null out the referent. I’ll handle it in my code.” The exact algorithm that is used to determine whether a SoftReference should be cleared out is JVM vendor dependent. It tends to be a function of how often and how long ago the reference was last used to reach the referent.

The semantics of a SoftReference make it possible to make a memory sensitive cache. The gotcha is that the exact behaviour will be JVM dependent. And as with anything else related to garbage collector behaviour, guarantees are few and are vendor dependent. For higher reliability (and many more features) most people tend to use something like Ehcache rather than roll out their own caching logic using SoftReference unless they have a very simple use case. Used in the right situation, they certainly can result in some elegant, simple code.

Weakly reachable

• An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by traversing a weak reference. When the weak references to a weakly-reachable object are cleared, the object becomes eligible for finalization.

In a nut-shell, a weak reference will not prevent its referent from being collected by the garbage collector. The most common use case of a WeakReference is via a WeakHashMap. It is an easy way to associate data with an object bound to the lifetime of the object. The bound data will stay in the map only while the key is not weakly reachable due to the key being strongly reachable because of reachability via a reference elsewhere in the application. Once all other references in the application are gone, the key becomes weakly reachable and is gone from the map without the user needing to explicitly remove the entry. An elegant way to prevent memory leaks in some cases!

Phantom reachable

• An object is phantom reachable if it is neither strongly, softly, nor weakly reachable, it has been finalized, and some phantom reference refers to it.

The referent of a PhantomReference can never be retrieved. The get() method on this reference always returns null! So what is the use?

All Reference types allow specifying a ReferenceQueue in the constructor. The semantics of how and when a PhantomReference is enqueued makes it possible to do finalization in a better, less fragile way. More on that in a future blog post perhaps.

Unreachable

• An object is unreachable, and therefore eligible for reclamation, when it is not reachable in any of the above ways.

See also: Package java.lang.ref

Task A – Starts at t = 0. Takes a read lock. Runs for 3 seconds. Scheduled to run every 5 seconds.

Task B – Starts at t = 2. Takes a read lock. Runs for 3 seconds. Scheduled to run every 5 seconds.

t ->
AAAXXAAAXXAAAXXAAAXXAAAXX...
XXBBBXXBBBXXBBBXXBBBXXBBB...

Task C – Starts at t = 10 (or rather tries to start at any time after t = 0) . Tries to take a write lock. Can never get it!

With the scheduling above, either A or B always have the read lock. A write lock can be acquired only if there are no read locks given out. So if Task C starts later and tries to get the write lock (which it gets only when no read locks have been granted) then Task C will get stuck for eternity.

One may think that this situation is rare. But it is not. With the common scenario of many readers and few or rare writers, it can happen quite often. While the lock throughput may be high, there are no guarantees about fairness.

To counter this situation, we introduce the concept of a fairness policy in granting locks. Depending on fairness, a thread requesting a read lock may block if a writer thread is already waiting for a write lock. This means that a read lock will not be granted even if no write lock has currently been granted! This is known as a write biased read write lock. The pitfall is that the fairness overhead can reduce throughput drastically for a highly concurrent application.

The recommendation is to use truly concurrent data structures like ConcurrentHashMap or ConcurrentLinkedQueue if possible rather than going for the cheap (in terms of development time) protection of your data structures via a ReadWriteLock. If it is a custom data structure, the RWL can certainly be an easy way of increasing concurrency rather than redesigning your stuff for concurrency with volatiles, reference variables, lock striping, atomic variable, CAS (compare-and swap), condition queues and other concurrent data structure design techniques.

In Java, the class ReentrantReadWriteLock class which implements the ReadWriteLock interface takes a boolean argument in a constructor to indicate fairness.

In .NET, the ReaderWriterLockSlim class provides similar functionality.

See this article from the Java Specialists newsletter for benchmarks and numbers comparing fair and non-fair locks and their behaviour in Java 5 and Java 6.

Nothing unremarkable about it except that it is probably the first IT company ad that I saw on Facebook.

Interesting. Many people fill in the names of current and past employers, how long they have been there and which city they are based in. If I were a recruiter at Infosys, Wipro, TCS etc., I would consider doing some targeted advertising. If there were a place to put in core tech skills – recruiters would be able to do some really targeted advertising like: “show this ad to all people with SAP in their core skills living in Pune and have been with current employer for over 2 years”.

Maybe Facebook could even scrape data off a LinkedIn profile if I place a link to it on my profile. (Probably won’t make LinkedIn happy if they do it without some $$$ changing hands.)

I think that the targeted advertising potential of Facebook (and other sites like Google/GMail/AdSense, LinkedIn) still remains under utilized. While a lot of people talk about privacy issues, I am not particularly bothered as far as there is no spam, phone calls or a human personally trying to get in touch with me or looking at my data. If I am going to see ads, I would willingly volunteer data on what I am interested in so that I see relevant ads.

Your thoughts?

MyBS.methodRequired has Required propagation.

MyBS.methodRequiresNew has Requires_New propagation.

… methodRequired (…) { … methodRequiresNew(…); … }

Do you get a new transaction if you call methodRequiresNew from methodRequired?

You don’t! Not if you are using the default proxy mode via AOP proxies through which transactional advice is applied. If you move the method with the new transaction to a different class and then call it – you do get a new transaction. What’s happening here?

When proxies are used to add transactional behaviour and you call a method directly within the class, there is no proxy in between. So no transactional behaviour can be added. So only calls from ‘outside’ will use the behaviour specified by the @Transactional annotation. For ‘internal’ calls, the @Transactional annotation is silently ignored!

The ways around this that I can think of (Comments welcome if you have more suggestions):

• Move such code to a new class. Then the calls become external. This may not be very neat depending on how your code is structured.
• Use BeanFactoryPostProcessor and put a reference to the proxied bean into the bean. A self referring bean. Instead of calling the method directly, use the injected reference to make the call.
• You could also just pull the proxied bean from the application context with the getBean method. But that’s Anti Dependency Injection. Not good.
• Use AspectJ mode for transactions instead of proxy mode.

Quote from the Spring API reference docs:

Note: In proxy mode (which is the default), only ‘external’ method calls coming in through the proxy will be intercepted. This means that ‘self-invocation’, i.e. a method within the target object calling some other method of the target object, won’t lead to an actual transaction at runtime even if the invoked method is marked with @Transactional!

Consider the use of AspectJ mode (see below) if you expect self-invocations to be wrapped with transactions as well. In this case, there won’t be a proxy in the first place; instead, the target class will be ‘weaved’ (i.e. its byte code will be modified) in order to turn @Transactional into runtime behavior on any kind of method.

The VM to be used can be specified by the -server and -client switches passed when starting the JVM with the java command. The default VM when no switch is specified is the Client VM.

The difference, in a nutshell, is that the Client VM is optimized for fast load times at the cost of runtime performance. The Server VM is built for aggressive optimization for runtime performance at the cost of slower load time.

The example below shows a simple instance of a clearly observable difference between the two. The code puts out the time taken to run a loop that counts down from 500 million to zero.

The results indicate that the Client VM took around 260 milliseconds each time. Amazingly, the Server VM takes ZERO milliseconds each time! This happens because of the more aggressive optimization. The compiler does some funky analysis and reaches the conclusion that the code in the loop does nothing and removing that code would not affect the flow of the program. So it just deletes that code from the compiled code that it produces. This results in the zero run time that we observe. Cool, eh?

Note that there is no need to recompile the code into new .class files between the tests with the two switches. The .class file fed to the JVM is the same in both cases. So the same byte code is initially read but the Server VM puts more effort into doing optimizations and doing JIT (Just In Time) compilation to produce faster code.

Go ahead and check right now if the startup script for your big server side Enterprise Java app has the -server switch. If not, perhaps you should seriously consider adding it in.

If you use an ArrayList for that list and no synchronization, you run the risk of a ConcurrentModificationException being thrown every hour when the list is modified by the Quartz job because the list may currently be in the middle of a traversal or iteration for a user visiting your homepage.

One way a person might try to fix it is to use a Vector or rather Collections.synchronizedList(List<T>). But this doesn’t help! While add(), remove(), get() will now be thread-safe on the list, Iterators will still throw ConcurrentModificationException! You will need to synchronize on the list object before iterating on it as well as in the Quartz job where you modify it. That’s just terrible from a performance and scalability point of view! Synchronization will be needed everywhere and all the time just because of an hourly modification.

Fortunately, there is a better solution. Use a CopyOnWriteArrayList.

Whenever a structural modification is made to this list, a new copy is created. This is not inefficient if modifications happen rarely. Iterators on this will not experience a ConcurrentModification because the list they started iterating on is never modified.

Another way one may try to avoid Concurrent Modification without adding synchronization code all over is to build a new List in their scheduled job and then switch out the reference which is used for iteration with the new list. Assigning a reference is an atomic operation in the JVM. This approach would fail if the code doing the iteration used an old style for loop - for (int i=0; i<list.size(); i++) { … list.get(i) …}. A change in the size of the switched list introduces a failure point. Worse, there are visibility problems here since the changes from a different thread may not be seen at all. Using the volatile keyword may help see the changes to the reference from a different thread, but the changed list size would still be a problem.

The memory consistency and happens-before guarantees of Copy On Write Array List make these problems disappear. Also, the code is simple since one doesn’t need to use volatile or synchronized. Less code, less bugs!

Another use case for CopyOnWriteArrayList is the case of the Observer design pattern. If event listeners are added and removed by multiple threads, it is quite possible that using a CopyOnWriteArrayList might be appropriate for correctness and simplicity.

Hope this helps. Remember to get your code reviewed by your nearest concurrency guru!

Use JConsole. It comes with the JDK. But out-of-the-box, JConsole won’t provide the info we need. You will need to use it with a plugin that gathers and displays CPU usage info.

The JTop plugin comes with the JDK with source as a demo plugin for JConsole. It shows cumulative CPU usage for each thread. So if a thread has high CPU usage it can be spotted easily. You can see the JTop tab in JConsole if your start it like this -

jconsole -pluginpath “%JAVA_HOME%\demo\management\JTop\JTop.jar”

JTop may be nice. But what you would rather be using is the Top Threads plugin written by Peter Doornbosch. It is free and is available from his blog. The TopThreads JConsole plugin will show you more info and includes usage history, current and average CPU usage. To get the TopThreads tab, start JConsole like this -

jconsole -pluginpath “c:\topthreads.jar”

JVisualVM also comes with the JDK and can help in such situations. The CPU profiling feature is especially handy considering the fact that it comes with the JDK and is free. Tools like JProfiler and YourKit are pretty costly unless you are really putting them to good use. Surprisingly few people know about this neat tool that comes bundled with the JDK. If your JDK bin directory is in your PATH, you can start JVisualVM from the console like this -

jvisualvm

Go ahead and give it a quick try right away. Start your Java app. Start JVisualVM. Attach JVisualVM to your app. Check out all the data the monitors and profilers provide.

Most of the time when you are debugging, you want to be looking at your own code. Not code from libraries or the JDK or even the ‘framework’ parts of your own code base.

You can configure the Eclipse Java debugger to not step into those bits of code by configuring a ‘Step Filter’.

1. Go to Windows -> Preferences -> Java -> Debug -> Step Filtering.
2. Check ‘Use Step Filters’.
3. Check the appropriate options on the screen. You can add parts that are relevant to your own codebase.
4. Click ‘Apply’.

Now the next time you press F5 when your breakpoint is on an orderList.add(newOrder) statement, you will not step into the code for the ArrayList.add method. Instead, the debugger will send you to the next appropriate line after the call. This makes sense since you are not likely to be looking for bugs in the JDK. Concentrating on the relevant bits helps maintain your train of thought in those intense debugging sessions, doesn’t it?

If you set a normal breakpoint, you’ll have to break and run repeatedly while manually verifying if the breakpoint is finally being hit for your item of interest.

A better approach is to set a conditional breakpoint. Right click on the breakpoint, select ‘Breakpoint properties…’, check ‘Enable condition’. You can now set a condition for the breakpoint to break. You can use the variables in the context of the breakpoint in this condition. Something like ‘orderNumber.equals(“RA54365″)’.

Now you’ll hit the breakpoint only for the item of interest. Can be quite a bit of a time saver when used in the right situation. Note that the icon for a conditional breakpoint shows up like a regular breakpoint icon with a question mark superimposed on it.

I have also used this at times when I know that something I am doing is resulting in bad data but I’m not sure what. I set a conditional breakpoint and keep trying out things. When I hit the right steps, the breakpoint suspends the thread and alerts me to it. Neat, isn’t it? You can do the same when developing with IDEA or NetBeans too.

Got any cool debugging tips? Drop me a comment!

I am not impressed with the software at all. The configuration UI takes way too long to load. There is no way to tell the software to completely unload itself from memory. The resource usage shoots up too much, too often. I disabled scheduled scans and disabled automatic installation of updates as well.

I noticed that their mcshield.exe process is set to a “high” base priority. It is this process that tends to eat 100% CPU too often. Because of the high base priority, all the other processes on my single threaded system become completely non responsive. I searched around and found no way to lower the base priority of this process. For a single core CPU, it does not seem like a good default to me to set a strongly CPU bound process to a “high” base priority which starves every other process of any CPU time at all. I did find a whole lot of other people complaining about the high CPU usage caused by this process. Even Google Suggest points to this being a problem for many.

I used AVG (free edition) earlier and that definitely seemed to perform better.

Some of my colleagues purchased the Kaspersky Internet Security Suite recently. They offer a really good deal – something like a 3-PC license for Rs.600 or so in India. And that’s what most non-corporate customers would actually be willing to pay for in India. My PC at work too has Kaspersky and I’m pretty happy with how it works there so far.

More on MSDN about how Windows schedules threads.

Of course, updates are important and disabling automatic updates would not be a wise thing to do.

The solution that works for me?

Start > Run > cmd

sc stop wuauserv

That stops the Windows Automatic Updates Service. That makes the updates reminder screen go away along with the system tray icon until you restart. Ah, peace. You could use the same command with the start switch as well. The service is started automatically when you restart.

I haven’t seen how Microsoft did this in Windows Vista or Window 7. I sure hope it is handled better.

Some notes for the Erlang layman-

• It is a functional programming language – that’s quite different from what a Java programmer is used too.
• Variables in Erlang don’t vary! Everything is immutable. (Wow, how does someone do any programming at all in Erlang?)
• The immutability takes away a lot of the problems of doing concurrency correctly.
• Erlang is designed to have hot swappable code – change running code without bringing down your systems!
• No sharing and no locking. This means that an entire class of problems related to synchronization, locks, memory visibility in languages like Java go away.
• A ‘process’ in Erlang is extremely lightweight and is analogous to what is a thread to a Java programmer (but there are differences if you go into details).
• Erlang aims to be scalable to hundreds of thousands or even millions of concurrent processes.
• Joe Armstrong reasons that the shared memory model of doing concurrency cannot scale like the pure message passing model of concurrency in Erlang can.

I looked around on a few job sites to see if investing time learning more about Erlang is likely to be worth it. Looks like there are very, very few jobs out there currently that quote Erlang as a required skill. Perhaps that will change in a few years when multi-core systems scale up to hundreds of cores. The design principles are still a valuable takeaway from the talks. At least I know something about another way to solve problems of scale, concurrency and fault tolerance. Perhaps I’ll choose Erlang when the right problem comes along.

For the time being, I’ll probably just try out running some basic Erlang examples and will keep an eye out for growth in its popularity.

Links to Joe’s presentations on Erlang on InfoQ
Functions + Messages + Concurrency = Erlang
Systems that Never Stop (and Erlang)
Message Passing Concurrency in Erlang
Erlang – Software for a concurrent world

One way to do this is –

Make the code changes in your Eclipse workspace (or whatever IDE you use). Use the ‘Export as Jar’ option for the changed source file(s) – export only the compiled class files. Eclipse will put them in the appropriate package structure. Add the the jar onto the beginning of your classpath – how to do this will depend on your application. The easiest way is to just drop it into the $JAVA_HOME\lib\ext folder – the CLASSPATH won’t matter then.

Or you could try something from this list that works for you -

• Modify a start script.
• Change an environment variable.
• Add to a properties file.
• Maybe change a Jar file manifest.

Of course, you’ll also need to restart your app. Hot-Swapping is also a possibility – that doesn’t need a restart to update classes in many cases. And Java Rebel brings to the table a much more powerful form of Hot-Swapping that requires far fewer restarts. But neither are recommended for production systems.

The class files within your patch jar should now be used rather than your original class files since the class loader will pick what it finds first. When you are done with your testing, you can simply remove the patch jar and restart again. You can even leave the configuration in place – a missing patch jar shouldn’t cause any trouble.

To do this, a Webmaster needs to register on Google Webmaster Tools. He is then given a code that he needs to place at a certain location on his website. Google will then send a crawler to that location to look for he code. If the correct code is found, the Webmaster has effectively proved that he has control over that domain.

Google will then show the webmaster various statistics related to search terms, crawlers and links on the claimed website.

Now for quite some time, I was somehow under the impression that free WordPress.com blogs (yourname.wordpress.com) could not prove ownership of their blog since there was no way to upload files to the domain or to add Meta tags to the blog code which is another way that Google allows you to add the code to your site.

Well, silly me. It turns out that even free WordPress.com blogs do have a way to add the required Meta tags. But in my defense, they have really put it in a place where it is hidden away quite well.

Click Tools under the Tools menu on the left hand side bar. Scroll to the lower part of the content section of the page. And here you see the “Webmaster Tools Verification” section. You can enter codes for Google Webmaster Tools, Yahoo! site Explorer and Bing Webmaster Center over here.

• A control channel (for FTP commands)
• A data channel

In an Active mode FTP session the data channel is established by the server initiating a connection to the client. This quite often turns out to be a problem for a client that is behind a corporate Firewall / NAT because connections coming into the network are blocked by default.

To get around this, FTP has a Passive mode. In this mode, the client tells the server to listen for a data connection initiated by the client. The server tells the client which IP, port it will listen on. So the firewall problem is solved since the control and the data channel will both be setup by outgoing connections that are likely to be safe from firewall interference.

Here are some notes you may find to be of use if you are dealing with this -

• Passive mode ensure that all connections are initiated by the client.
• The PASV command is used to enter Passive mode.
• The server replies to PASV with something like “227 Entering Passive Mode (192,168,10,25,10,20).” – This means that the server will wait for a data connection from the client on 192.168.10.25 on port 2580 (256 x 10 + 20).
• In case of an Active mode connection, the client tells the server that it will wait for a data connection by issuing the PORT command – “PORT 192,168,10,25,10,20″.
• If you are writing code to use FTP, SFTP or FTPS without using an FTP client and run into problems, test with a known FTP client – if it can transfer files successfully, surely your code should be able to do so as well.
• All popular FTP clients show a log of the commands they are executing. Look at these for hints on what could be going wrong.
• Make sure you are using the right type out of FTP, SFTP, FTPS or out of Active, Passive or out of Explicit, Implicit SSL.
• SFTP is different. It has no concept of Active or Passive mode or of separate connections being established for data and control channels.
• You can use TCPView to monitor what connections are being setup on your machine.

JScape

• I read about quite a few people recommending this as the quality solution.
• This isn’t free in terms of $$$.
• If file transfer is important to you and you are willing to pay for it, you may want to go with JScape.

Zehon

• The Zehon libraries have support for FTP, SFTP and FTPS.
• They offer a very simple interface in the form of a bunch of static methods to do actions like sending and receiving files.
• No source code is provided.
• Can be difficult to debug in case of problems.
• They do use Apache Commons Net as a base but some of the power is hidden by Zehon in exchange for simplicity.

Apache Commons Net

• Using this directly offers greater power and more complete control over the protocol features compared to Zehon.
• For a quick start, see sample code in “examples.FTPExample” provided with the Apache Comons Net source distribution.
• No support for SFTP.

JCraft JSch

• Widely used for SFTP.
• “JSch is a pure Java implementation of SSH2.”

Here are some issues you may run into -

• You may need to pick up some understanding of “Impicit SSL” and of the “PASV” (passive) mode to deal with firewall issues.
• For Zehon, I had to specify a keystore it could use. Apache picked up the default java certs keystore with its default configuration.
• I believe the core Apache Commons Net library (and hence Zehon too) has some kind of a caching/flushing bug. When trying to transfer very small files of size 1 byte to 175 bytes or so, the library reported the file to have been transferred successfully – but it actually resulted in a zero byte file being created at the target location. Beware! I’ll write about how I fixed this in a different post. I had to add a bit of code in the Apache Commons Net sources and use my own compiled jar for it to work well.
• If you are really doing security correctly – you will need to know how to use keytool and how to handle your certificates correctly.
• You can use the free FileZilla FTP Server to start an FTP server on your own machine for development and testing.

Few people have used the fsutil command in Windows. Here is how you can use it to create a file of any desired size -

fsutil file create myFile.dat 10240

This will create a file of size 10KB in the current directory.

There are other interesting things that fsutil is capable of.  You may want to check out the other options it shows in the command line help.

You can do it with the dd command. You can use /dev/random or /dev/zero as a data source for random bytes or null bytes.

bs = block size
count = number of blocks
if = input file
of = output file

The total size of the file created will be bs x count bytes.

dd if=/dev/random of=myFile.dat bs=1024 count=102400
dd if=/dev/random of=myFile.dat bs=1024 count=1024
dd if=/dev/random of=myFile.dat bs=100 count=1
dd if=/dev/random of=myFile.dat bs=1 count=1

The above commands will create files of sizes 100MB, 1MB, 100 bytes, 1 byte respectively with random data.

“Process is trying to inject into another process. This behaviour is typical of same malicious programs (Invader).“

Now that’s just silly. Kaspersky really ought to have white-listed jconsole.exe. It is perfectly safe and is designed to do what Kaspersky thinks is malicious behaviour.

My problem was compounded by the fact that I didn’t have admin rights to Kaspersky settings. It was maintained remotely by the admins in our corporate network. It had be restored by them every time Kaspersky quarantined the file and deleted it from its original location until I finally got them to change the setting for “Intrusion into process” under Proactive Defense settings to “Ask user” rather than “Quarantine” for me.

In my opinion Kaspersky is at fault. It’s signature DB should have had jconsole.exe white-listed.

Remember, just because two files are the same size, it doesn’t mean that they are the same. It is possible that your download program padded the file to the correct size but didn’t fill in all the data. Sometimes, stopping and resuming downloads mid way can leave a file corrupted. Memory and disk errors can silently cause corruption by bit rot.

So how do you check the integrity of the file?

Use a checksum like md5sum. It is available by default on all Linux / Unix / Solaris systems. Windows users can use the free utility WinMD5.

The md5 checksum is basically a digital fingerprint of the file. It works by processing the contents of the file via a mixing algorithm that produces output that is practically unique for unique input.

Check the screenshot below that I took on WinXP using WinMd5 sum (on Linux the command is ‘md5sum fileName’). I was comparing the checksum of an Ubuntu Linux VMWare virtual machine I downloaded to the checksum provided on the page from which I had downloaded the file. You can see that my md5 hash turned out to be the same as what was listed on the website (68b2773b177cff1ba71071f83707fba9). If even a single bit were different at my end, the fingerprint would have been completely different.

Next time you are wondering if a file is exactly the same as another, just compare the md5 checksums of the two files.

Have you noticed that on some sites, Firefox will never give you the option to save your password? Some places where this happens is on Yahoo and Hotmail login screens and on corporate email sites. Ever wondered why this happens?

This happens because some sites explicitly prevent that from happening by using an “autocomplete=off” attribute in the form element for that page.

To override this, you can modify a script in Firefox -

1. Close Firefox.
2. Open nsLoginManager.js from C:\Program Files\Mozilla Firefox\components in a text editor like Notepad or NoteTab++.
3. Search for “autocomplete” in this file and look for the code in the screen shot below.
4. Change “off” to “XXX” and save the file.
5. Done! Firefox will now ignore the autocomplete directive and you will be able to save passwords on more sites!

You may find other tips on the net which suggest commenting out the code in this method or to add code to return false from it. What is cool about my way is that the lines and offsets of all the code in that file stays the same. You just replace three characters with three different characters. This will make sure that if updates are applied to the file, there is no chance of it getting corrupted.

I tested this on Firefox 3.5 and it should work for sure on older 3.x versions as well.

What do you do if you want the browser to show the user the SaveAs dialog allowing the file to be saved to a location on the disk?

The browser recognizes only the file type from the well known Content-Type HTTP header. Whether to open or to do a ‘Save As’ and an appropriate file name can be indicated by the less known Content-Disposition header. So if you want your browser to show you a SaveAs dialog, make your HTTP headers look like this -

Content-Type: application/pdf
Content-Disposition: attachment; filename=Report-May-2010.pdf;

Usually, one would do this by using the response.setContentDisposition method or its equivalent in your programming language of choice.

Tip: If you are trying to check this or implement this, use FireBug to view the HTTP headers your app is sending.

That means that instead of doing…

java -cp lib\x.jar;lib\y.jar;lib\z.jar; com.company.MyProggy

…you could just do…

java -cp lib\*.jar; com.company.MyProggy

That generally turns out to be quite a time saver. Especially so if you are trying out something at a remote box and vi is your only editor.

However, if you use the wild card, there is no way to predict the order in which the jars will be used. In some cases, the order is important – and in those cases, you will need to add in the jars like one normally does by enumerating each one explicitly.

Also, don’t forget the semi colon at the end of the classpath. Countless hours have been lost over things not working due to that missing separator. It can be really frustrating!

They showed me the full database connection string with the DB server details, IP address and a username and a password. They even let me know where on their server the code for their website was present. Gives a whole new meaning to open source!

Note the following lines in the screen shot below with the stack trace they threw at me -

• “ConStr = “PROVIDER=SQLOLEDB; DATA SOURCE=X.X.X.X;UID=XXX;PWD=XXX;DATABASE=XXX;network library=dbmssocn;”
• “D:\Webchutney\AirtelWebsite\Conn.asp” - It seems WebChutney is a company that has Airtel as one of its clients.

Oh well, the IP was of a machine on their internal network so not like anyone could have gained access from outside the network with ease. But when this happens, hackers/crackers generally start sniffing around and probing your network for weak points and generally speaking, that can’t lead to much good.