Stas's blog

The magic of conditional operator

2011-12-05T09:20:00.001Z

Can you guess what is going to be the output of the following piece of code?

    Object obj = false ? new Long(1) : new Integer(1);
    System.out.println(obj.getClass());

The smart once can guess that it is going to be "class java.lang.Long", otherwise there won't be a question. Now can you answer why? To be honest, I was a surprised by such an outcome, but apparently, according to JLS that is correct behaviour. Here is quotation from "15.25 Conditional Operator ? :":

The type of a conditional expression is determined as follows:

If the second and third operands have the same type (which may be the null type), then that is the type of the conditional expression.

If one of the second and third operands is of type boolean and the type of the other is of type Boolean, then the type of the conditional expression is boolean.

If one of the second and third operands is of the null type and the type of the other is a reference type, then the type of the conditional expression is that reference type.

Otherwise, if the second and third operands have types that are convertible (§5.1.8) to numeric types, then there are several cases:

If one of the operands is of type byte or Byte and the other is of type short or Short, then the type of the conditional expression is short.

If one of the operands is of type T where T is byte, short, or char, and the other operand is a constant expression of type int whose value is representable in type T, then the type of the conditional expression is T.

If one of the operands is of type Byte and the other operand is a constant expression of type int whose value is representable in type byte, then the type of the conditional expression is byte.

If one of the operands is of type Short and the other operand is a constant expression of type int whose value is representable in type short, then the type of the conditional expression is short.

If one of the operands is of type; Character and the other operand is a constant expression of type int whose value is representable in type char, then the type of the conditional expression is char.

Otherwise, binary numeric promotion (§5.6.2) is applied to the operand types, and the type of the conditional expression is the promoted type of the second and third operands. Note that binary numeric promotion performs unboxing conversion (§5.1.8) and value set conversion (§5.1.13).

Otherwise, the second and third operands are of types S1 and S2 respectively. Let T1 be the type that results from applying boxing conversion to S1, and let T2 be the type that results from applying boxing conversion to S2. The type of the conditional expression is the result of applying capture conversion (§5.1.10) to lub(T1, T2) (§15.12.2.7).

In the other words, it says that if second and third operands are convertible to primitive numeric types, then the result type is based on numeric promotion of converted values. Here how it looks after applying these rules:

Object obj = Long.valueOf(false ? (new Long(1L)).longValue() : (new Integer(1)).intValue());

My opinion, it all looks like a magic, and at the end, that's the price Java paid for autoboxing. Well world is not perfect and that's it's beauty, isn't it? :)

Java file flushing performance

2011-11-09T22:29:00.000Z

There are many situations when it is required to ensure that data was written to the disk and write is also required to be fast. The most most common where it has to happen are databases, journalling, etc. Also, it is often required to update some random position in a file. I specifically what to place emphasis on random access here, as far as the rest will cover just cases where it is supported, i.e. I'm not going to mention OutputStream.flush() & related topics. Just haven't tried it, as far as that wasn't my case at the moment.

There are several way of flushing the data to disk in java. These options can be quite different in the way they implemented internally and in their performance. Here is the list of existing things you can do:

FileChannel.force()
'rws' or 'rwd' mode of RandomAccessFile, which 'works much like the force(boolean) method of the FileChannel class' (from javadoc).
MappedbyteBuffer.force()
RandomAccessFile.getFD().sync()
any close() method. Here I mean seek and close stream each time when access is required. Doing tests, I actually didn't seek, as far was updating data with zero offset.

Surprisingly (the only unsurprising exception is close()) all these methods gives very different performance and it varies almost randomly on different OSes and file systems. Worth noticing that hardware can also put it's correction on the performance of any of these methods. I have also a strong feeling that performance may vary even with minor change in OS or JVM version number. Here is the table with time it takes to flush 8bytes (keep in mind, that the real amount of flushed data depends on the size of caches and going to be much more that 8byes), just to give a flavour of how different is that:

	RandomAccessFile. getFD().sync()	RandomAccessFile, rwd mode	MappedbyteBuffer.force()	FileChannel.force()
Windows	0.2818ms	0.0125ms	0.007ms	0.139ms
Linux	0.5354ms	0.5144ms	0.4663ms	0.0093ms

Please, do not treat these numbers as any relevant result. They are here just to give an example how these things can vary.

So, what the conclusion? Conclusion is that if you would need to write high-performance application which does lots of IO, you really need to test different approached on different OSes, on different file systems and, preferably on different JVMs. Do not expect something to be fast on Linux (Solaris, AIX, etc) production box, when it is fast on your Windows (Linux, etc) workstation and vice versa. As can be seen, the difference can be in orders of magnitude.

Embedding Jetty7

2011-07-25T22:01:00.001+01:00

Jetty web container provides the way to build highly modularized web application. It is also as very fast a lightweight. Following example shows how to build simple web application using embedded version of Jetty7.

The main class:

package sample.jetty7;

import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.webapp.WebAppContext;

import java.net.URL;

public class SampleServer {
    public static void main(String args[]) throws Exception {
        Server server = new Server(8080);

        final URL warUrl = SampleServer.class.getClassLoader().getResource("webapp");
        final String warUrlString = warUrl.toExternalForm();
        server.setHandler(new WebAppContext(warUrlString, "/"));

        server.start();
        server.join();
    }
}

web.xml:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">
    <servlet>
        <servlet-name>appServlet</servlet-name>
        <servlet-class>sample.jetty7.SampleServlet</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>appServlet</servlet-name>
        <url-pattern>/<url-pattern>
    </servlet-mapping>
</web-app>

To keep a bit of mystery in the post, I will not provide source for the servlet. It is suggested that reader can find it out on his own.

Here is list of dependencies in ivy format. If required, it shouldn't be a problem to translate it into Maven format or just get jars from some public repository.

<!-- Jetty Core -->
<dependency org="org.eclipse.jetty" name="jetty-server" rev="7.4.4.v20110707"/>
<dependency org="org.eclipse.jetty" name="jetty-webapp" rev="7.4.4.v20110707"/>
<dependency org="org.eclipse.jetty" name="jetty-servlet" rev="7.4.4.v20110707"/>
<dependency org="org.eclipse.jetty" name="jetty-util" rev="7.4.4.v20110707"/>
<dependency org="org.eclipse.jetty" name="jetty-http" rev="7.4.4.v20110707"/>
<dependency org="org.eclipse.jetty" name="jetty-io" rev="7.4.4.v20110707"/>
<dependency org="org.eclipse.jetty" name="jetty-security" rev="7.4.4.v20110707"/>
<dependency org="org.eclipse.jetty" name="jetty-xml" rev="7.4.4.v20110707"/>
<dependency org="org.eclipse.jetty" name="jetty-continuation" rev="7.4.4.v20110707"/>
<!-- JSP Support -->
<dependency org="org.apache.tomcat" name="jasper" rev="6.0.32"/>
<dependency org="org.apache.tomcat" name="jasper-el" rev="6.0.32"/>
<dependency org="org.apache.tomcat" name="jasper-jdt" rev="6.0.29"/>
<dependency org="org.apache.tomcat" name="juli" rev="6.0.32"/>
<dependency org="javax.servlet.jsp" name="jsp-api" rev="2.1"/>
<dependency org="javax.servlet" name="servlet-api" rev="2.5"/>

The final step is building a jar file with following structure:

now just run the main class. It better to run and kill it several times to feel deeply that pleasure of fast startup and small footprint. When you enjoyed enought, take a final deep breath and go to "http://localhost:8080/sample.jsp" to check that it is actually working.

The most complete list of -XX options for Java 6 JVM

2011-07-20T14:23:00.007+01:00

There was a wonderful page with that list (http://www.md.pp.ru/~eu/jdk6options.html), but it seems it's gone now and not available any more. Luckily there is http://web.archive.org which allows to make time flowing backwards and recover things which have passed away long time back. So, I have just get that page out of grave and copy/pasted here to return that bit of information back to life.

product flags are always settable / visible
develop flags are settable / visible only during development and are constant in the PRODUCT version
notproduct flags are settable / visible only during development and are not declared in the PRODUCT version
diagnostic options not meant for VM tuning or for product modes. They are to be used for VM quality assurance or field diagnosis of VM bugs. They are hidden so that users will not be encouraged to try them as if they were VM ordinary execution options. However, they are available in the product version of the VM. Under instruction from support engineers, VM customers can turn them on to collect diagnostic information about VM problems. To use a VM diagnostic option, you must first specify +UnlockDiagnosticVMOptions. (This master switch also affects the behavior of -Xprintflags.)
manageable flags are writeable external product flags. They are dynamically writeable through the JDK management interface (com.sun.management.HotSpotDiagnosticMXBean API) and also through JConsole. These flags are external exported interface (see CCC). The list of manageable flags can be queried programmatically through the management interface.
experimental experimental options which become available just after XX:+UnlockExperimentalVMOptions flag is set.
product_rw flags are writeable internal product flags. They are like "manageable" flags but for internal/private use. The list of product_rw flags are internal/private flags which may be changed/removed in a future release. It can be set through the management interface to get/set value when the name of flag is supplied.
product_pd
develop_pd

Name	Description	Default	Type
product
UseMembar	(Unstable) Issues membars on thread state transitions	false	bool
PrintCommandLineFlags	Prints flags that appeared on the command line	false	bool
JavaMonitorsInStackTrace	Print info. about Java monitor locks when the stacks are dumped	true	bool
LargePageSizeInBytes	Large page size (0 to let VM choose the page size	0	uintx
LargePageHeapSizeThreshold	Use large pages if max heap is at least this big	128*M	uintx
ForceTimeHighResolution	Using high time resolution(For Win32 only)	false	bool
PrintVMQWaitTime	Prints out the waiting time in VM operation queue	false	bool
PrintJNIResolving	Used to implement -v:jni	false	bool
UseInlineCaches	Use Inline Caches for virtual calls	true	bool
UseCompilerSafepoints	Stop at safepoints in compiled code	true	bool
UseSplitVerifier	use split verifier with StackMapTable attributes	true	bool
FailOverToOldVerifier	fail over to old verifier when split verifier fails	true	bool
SuspendRetryCount	Maximum retry count for an external suspend request	50	intx
SuspendRetryDelay	Milliseconds to delay per retry (* current_retry_count)	5	intx
UseSuspendResumeThreadLists	Enable SuspendThreadList and ResumeThreadList	true	bool
MaxFDLimit	Bump the number of file descriptors to max in solaris.	true	bool
BytecodeVerificationRemote	Enables the Java bytecode verifier for remote classes	true	bool
BytecodeVerificationLocal	Enables the Java bytecode verifier for local classes	false	bool
PrintGCApplicationConcurrentTime	Print the time the application has been running	false	bool
PrintGCApplicationStoppedTime	Print the time the application has been stopped	false	bool
ShowMessageBoxOnError	Keep process alive on VM fatal error	false	bool
SuppressFatalErrorMessage	Do NO Fatal Error report [Avoid deadlock]	false	bool
OnError	Run user-defined commands on fatal error; see VMError.cpp for examples	""	ccstr
OnOutOfMemoryError	Run user-defined commands on first java.lang.OutOfMemoryError	""	ccstr
PrintCompilation	Print compilations	false	bool
StackTraceInThrowable	Collect backtrace in throwable when exception happens	true	bool
OmitStackTraceInFastThrow	Omit backtraces for some 'hot' exceptions in optimized code	true	bool
ProfilerPrintByteCodeStatistics	Prints byte code statictics when dumping profiler output	false	bool
ProfilerRecordPC	Collects tick for each 16 byte interval of compiled code	false	bool
ProfileVM	Profiles ticks that fall within VM (either in the VM Thread or VM code called through stubs)	false	bool
ProfileIntervals	Prints profiles for each interval (see ProfileIntervalsTicks)	false	bool
RegisterFinalizersAtInit	Register finalizable objects at end of Object. or after allocation.	true	bool
ClassUnloading	Do unloading of classes	true	bool
ConvertYieldToSleep	Converts yield to a sleep of MinSleepInterval to simulate Win32 behavior (SOLARIS only)	false	bool
UseBoundThreads	Bind user level threads to kernel threads (for SOLARIS only)	true	bool
UseLWPSynchronization	Use LWP-based instead of libthread-based synchronization (SPARC only)	true	bool
SyncKnobs	(Unstable) Various monitor synchronization tunables	""	ccstr
EmitSync	(Unsafe,Unstable) Controls emission of inline sync fast-path code	0	intx
AlwaysInflate	(Unstable) Force inflation	0	intx
Atomics	(Unsafe,Unstable) Diagnostic - Controls emission of atomics	0	intx
EmitLFence	(Unsafe,Unstable) Experimental	0	intx
AppendRatio	(Unstable) Monitor queue fairness" ) product(intx, SyncFlags, 0,(Unsafe,Unstable) Experimental Sync flags" ) product(intx, SyncVerbose, 0,(Unstable)" ) product(intx, ClearFPUAtPark, 0,(Unsafe,Unstable)" ) product(intx, hashCode, 0, (Unstable) select hashCode generation algorithm" ) product(intx, WorkAroundNPTLTimedWaitHang, 1, (Unstable, Linux-specific)" avoid NPTL-FUTEX hang pthread_cond_timedwait" ) product(bool, FilterSpuriousWakeups , true, Prevent spurious or premature wakeups from object.wait" (Solaris only)	11	intx
AdjustConcurrency	call thr_setconcurrency at thread create time to avoid LWP starvation on MP systems (For Solaris Only)	false	bool
ReduceSignalUsage	Reduce the use of OS signals in Java and/or the VM	false	bool
AllowUserSignalHandlers	Do not complain if the application installs signal handlers (Solaris & Linux only)	false	bool
UseSignalChaining	Use signal-chaining to invoke signal handlers installed by the application (Solaris & Linux only)	true	bool
UseAltSigs	Use alternate signals instead of SIGUSR1 & SIGUSR2 for VM internal signals. (Solaris only)	false	bool
UseSpinning	Use spinning in monitor inflation and before entry	false	bool
PreSpinYield	Yield before inner spinning loop	false	bool
PostSpinYield	Yield after inner spinning loop	true	bool
AllowJNIEnvProxy	Allow JNIEnv proxies for jdbx	false	bool
JNIDetachReleasesMonitors	JNI DetachCurrentThread releases monitors owned by thread	true	bool
RestoreMXCSROnJNICalls	Restore MXCSR when returning from JNI calls	false	bool
CheckJNICalls	Verify all arguments to JNI calls	false	bool
UseFastJNIAccessors	Use optimized versions of GetField	true	bool
EagerXrunInit	Eagerly initialize -Xrun libraries; allows startup profiling, but not all -Xrun libraries may support the state of the VM at this time	false	bool
PreserveAllAnnotations	Preserve RuntimeInvisibleAnnotations as well as RuntimeVisibleAnnotations	false	bool
LazyBootClassLoader	Enable/disable lazy opening of boot class path entries	true	bool
UseBiasedLocking	Enable biased locking in JVM	true	bool
BiasedLockingStartupDelay	Number of milliseconds to wait before enabling biased locking	4000	intx
BiasedLockingBulkRebiasThreshold	Threshold of number of revocations per type to try to rebias all objects in the heap of that type	20	intx
BiasedLockingBulkRevokeThreshold	Threshold of number of revocations per type to permanently revoke biases of all objects in the heap of that type	40	intx
BiasedLockingDecayTime	Decay time (in milliseconds) to re-enable bulk rebiasing of a type after previous bulk rebias	25000	intx
TraceJVMTI	Trace flags for JVMTI functions and events	""	ccstr
StressLdcRewrite	Force ldc -> ldc_w rewrite during RedefineClasses	false	bool
TraceRedefineClasses	Trace level for JVMTI RedefineClasses	0	intx
VerifyMergedCPBytecodes	Verify bytecodes after RedefineClasses constant pool merging	true	bool
HPILibPath	Specify alternate path to HPI library	""	ccstr
TraceClassResolution	Trace all constant pool resolutions (for debugging)	false	bool
TraceBiasedLocking	Trace biased locking in JVM	false	bool
TraceMonitorInflation	Trace monitor inflation in JVM	false	bool
Use486InstrsOnly	Use 80486 Compliant instruction subset	false	bool
UseSerialGC	Tells whether the VM should use serial garbage collector	false	bool
UseParallelGC	Use the Parallel Scavenge garbage collector	false	bool
UseParallelOldGC	Use the Parallel Old garbage collector	false	bool
UseParallelOldGCCompacting	In the Parallel Old garbage collector use parallel compaction	true	bool
UseParallelDensePrefixUpdate	In the Parallel Old garbage collector use parallel dense" prefix update	true	bool
HeapMaximumCompactionInterval	How often should we maximally compact the heap (not allowing any dead space)	20	uintx
HeapFirstMaximumCompactionCount	The collection count for the first maximum compaction	3	uintx
UseMaximumCompactionOnSystemGC	In the Parallel Old garbage collector maximum compaction for a system GC	true	bool
ParallelOldDeadWoodLimiterMean	The mean used by the par compact dead wood" limiter (a number between 0-100).	50	uintx
ParallelOldDeadWoodLimiterStdDev	The standard deviation used by the par compact dead wood" limiter (a number between 0-100).	80	uintx
UseParallelOldGCDensePrefix	Use a dense prefix with the Parallel Old garbage collector	true	bool
ParallelGCThreads	Number of parallel threads parallel gc will use	0	uintx
ParallelCMSThreads	Max number of threads CMS will use for concurrent work	0	uintx
YoungPLABSize	Size of young gen promotion labs (in HeapWords)	4096	uintx
OldPLABSize	Size of old gen promotion labs (in HeapWords). See good explanation about that parameter here.	1024	uintx
GCTaskTimeStampEntries	Number of time stamp entries per gc worker thread	200	uintx
AlwaysTenure	Always tenure objects in eden. (ParallelGC only)	false	bool
NeverTenure	Never tenure objects in eden, May tenure on overflow" (ParallelGC only)	false	bool
ScavengeBeforeFullGC	Scavenge youngest generation before each full GC," used with UseParallelGC	true	bool
UseConcMarkSweepGC	Use Concurrent Mark-Sweep GC in the old generation	false	bool
ExplicitGCInvokesConcurrent	A System.gc() request invokes a concurrent collection;" (effective only when UseConcMarkSweepGC)	false	bool
UseCMSBestFit	Use CMS best fit allocation strategy	true	bool
UseCMSCollectionPassing	Use passing of collection from background to foreground	true	bool
UseParNewGC	Use parallel threads in the new generation.	false	bool
ParallelGCVerbose	Verbose output for parallel GC.	false	bool
ParallelGCBufferWastePct	wasted fraction of parallel allocation buffer.	10	intx
ParallelGCRetainPLAB	Retain parallel allocation buffers across scavenges.	true	bool
TargetPLABWastePct	target wasted space in last buffer as pct of overall allocation	10	intx
PLABWeight	Percentage (0-100) used to weight the current sample when" computing exponentially decaying average for ResizePLAB.	75	uintx
ResizePLAB	Dynamically resize (survivor space) promotion labs	true	bool
PrintPLAB	Print (survivor space) promotion labs sizing decisions	false	bool
ParGCArrayScanChunk	Scan a subset and push remainder, if array is bigger than this	50	intx
ParGCDesiredObjsFromOverflowList	The desired number of objects to claim from the overflow list	20	intx
CMSParPromoteBlocksToClaim	Number of blocks to attempt to claim when refilling CMS LAB for parallel GC.	50	uintx
AlwaysPreTouch	It forces all freshly committed pages to be pre-touched.	false	bool
CMSUseOldDefaults	A flag temporarily introduced to allow reverting to some older" default settings; older as of 6.0	false	bool
CMSYoungGenPerWorker	The amount of young gen chosen by default per GC worker thread available	16*M	intx
CMSIncrementalMode	Whether CMS GC should operate in \"incremental\" mode	false	bool
CMSIncrementalDutyCycle	CMS incremental mode duty cycle (a percentage, 0-100). If" CMSIncrementalPacing is enabled, then this is just the initial" value	10	uintx
CMSIncrementalPacing	Whether the CMS incremental mode duty cycle should be automatically adjusted	true	bool
CMSIncrementalDutyCycleMin	Lower bound on the duty cycle when CMSIncrementalPacing is" enabled (a percentage, 0-100).	0	uintx
CMSIncrementalSafetyFactor	Percentage (0-100) used to add conservatism when computing the" duty cycle.	10	uintx
CMSIncrementalOffset	Percentage (0-100) by which the CMS incremental mode duty cycle" is shifted to the right within the period between young GCs	0	uintx
CMSExpAvgFactor	Percentage (0-100) used to weight the current sample when" computing exponential averages for CMS statistics.	25	uintx
CMS_FLSWeight	Percentage (0-100) used to weight the current sample when" computing exponentially decating averages for CMS FLS statistics.	50	uintx
CMS_FLSPadding	The multiple of deviation from mean to use for buffering" against volatility in free list demand.	2	uintx
FLSCoalescePolicy	CMS: Aggression level for coalescing, increasing from 0 to 4	2	uintx
CMS_SweepWeight	Percentage (0-100) used to weight the current sample when" computing exponentially decaying average for inter-sweep duration.	50	uintx
CMS_SweepPadding	The multiple of deviation from mean to use for buffering" against volatility in inter-sweep duration.	2	uintx
CMS_SweepTimerThresholdMillis	Skip block flux-rate sampling for an epoch unless inter-sweep duration exceeds this threhold in milliseconds	10	uintx
CMSClassUnloadingEnabled	Whether class unloading enabled when using CMS GC	false	bool
CMSCompactWhenClearAllSoftRefs	Compact when asked to collect CMS gen with clear_all_soft_refs	true	bool
UseCMSCompactAtFullCollection	Use mark sweep compact at full collections	true	bool
CMSFullGCsBeforeCompaction	Number of CMS full collection done before compaction if > 0	0	uintx
CMSIndexedFreeListReplenish	Replenish and indexed free list with this number of chunks	4	uintx
CMSLoopWarn	Warn in case of excessive CMS looping	false	bool
CMSMarkStackSize	Size of CMS marking stack	32*K	uintx
CMSMarkStackSizeMax	Max size of CMS marking stack	4*M	uintx
CMSMaxAbortablePrecleanLoops	(Temporary, subject to experimentation)" Maximum number of abortable preclean iterations, if > 0	0	uintx
CMSMaxAbortablePrecleanTime	(Temporary, subject to experimentation)" Maximum time in abortable preclean in ms	5000	intx
CMSAbortablePrecleanMinWorkPerIteration	(Temporary, subject to experimentation)" Nominal minimum work per abortable preclean iteration	100	uintx
CMSAbortablePrecleanWaitMillis	(Temporary, subject to experimentation)" Time that we sleep between iterations when not given" enough work per iteration	100	intx
CMSRescanMultiple	Size (in cards) of CMS parallel rescan task	32	uintx
CMSConcMarkMultiple	Size (in cards) of CMS concurrent MT marking task	32	uintx
CMSRevisitStackSize	Size of CMS KlassKlass revisit stack	1*M	uintx
CMSAbortSemantics	Whether abort-on-overflow semantics is implemented	false	bool
CMSParallelRemarkEnabled	Whether parallel remark enabled (only if ParNewGC)	true	bool
CMSParallelSurvivorRemarkEnabled	Whether parallel remark of survivor space" enabled (effective only if CMSParallelRemarkEnabled)	true	bool
CMSPLABRecordAlways	Whether to always record survivor space PLAB bdries" (effective only if CMSParallelSurvivorRemarkEnabled)	true	bool
CMSConcurrentMTEnabled	Whether multi-threaded concurrent work enabled (if ParNewGC)	true	bool
CMSPermGenPrecleaningEnabled	Whether concurrent precleaning enabled in perm gen" (effective only when CMSPrecleaningEnabled is true)	true	bool
CMSPermGenSweepingEnabled	Whether sweeping of perm gen is enabled	false	bool
CMSPrecleaningEnabled	Whether concurrent precleaning enabled	true	bool
CMSPrecleanIter	Maximum number of precleaning iteration passes	3	uintx
CMSPrecleanNumerator	CMSPrecleanNumerator:CMSPrecleanDenominator yields convergence" ratio	2	uintx
CMSPrecleanDenominator	CMSPrecleanNumerator:CMSPrecleanDenominator yields convergence" ratio	3	uintx
CMSPrecleanRefLists1	Preclean ref lists during (initial) preclean phase	true	bool
CMSPrecleanRefLists2	Preclean ref lists during abortable preclean phase	false	bool
CMSPrecleanSurvivors1	Preclean survivors during (initial) preclean phase	false	bool
CMSPrecleanSurvivors2	Preclean survivors during abortable preclean phase	true	bool
CMSPrecleanThreshold	Don't re-iterate if #dirty cards less than this	1000	uintx
CMSCleanOnEnter	Clean-on-enter optimization for reducing number of dirty cards	true	bool
CMSRemarkVerifyVariant	Choose variant (1,2) of verification following remark	1	uintx
CMSScheduleRemarkEdenSizeThreshold	If Eden used is below this value, don't try to schedule remark	2*M	uintx
CMSScheduleRemarkEdenPenetration	The Eden occupancy % at which to try and schedule remark pause	50	uintx
CMSScheduleRemarkSamplingRatio	Start sampling Eden top at least before yg occupancy reaches" 1/ of the size at which we plan to schedule remark	5	uintx
CMSSamplingGrain	The minimum distance between eden samples for CMS (see above)	16*K	uintx
CMSScavengeBeforeRemark	Attempt scavenge before the CMS remark step	false	bool
CMSWorkQueueDrainThreshold	Don't drain below this size per parallel worker/thief	10	uintx
CMSWaitDuration	Time in milliseconds that CMS thread waits for young GC	2000	intx
CMSYield	Yield between steps of concurrent mark & sweep	true	bool
CMSBitMapYieldQuantum	Bitmap operations should process at most this many bits" between yields	10*M	uintx
BlockOffsetArrayUseUnallocatedBlock	Maintain _unallocated_block in BlockOffsetArray" (currently applicable only to CMS collector)	trueInDebug	bool
RefDiscoveryPolicy	Whether reference-based(0) or referent-based(1)	0	intx
ParallelRefProcEnabled	Enable parallel reference processing whenever possible	false	bool
CMSTriggerRatio	Percentage of MinHeapFreeRatio in CMS generation that is allocated before a CMS collection cycle commences	80	intx
CMSBootstrapOccupancy	Percentage CMS generation occupancy at which to initiate CMS collection for bootstrapping collection stats	50	intx
CMSInitiatingOccupancyFraction	Percentage CMS generation occupancy to start a CMS collection cycle (A negative value means that CMSTirggerRatio is used). See good explanation about that parameter here.	-1	intx
UseCMSInitiatingOccupancyOnly	Only use occupancy as a crierion for starting a CMS collection. See good explanation about that parameter here.	false	bool
HandlePromotionFailure	The youngest generation collection does not require" a guarantee of full promotion of all live objects.	true	bool
PreserveMarkStackSize	Size for stack used in promotion failure handling	40	uintx
ZeroTLAB	Zero out the newly created TLAB	false	bool
PrintTLAB	Print various TLAB related information	false	bool
TLABStats	Print various TLAB related information	true	bool
AlwaysActAsServerClassMachine	Always act like a server-class machine	false	bool
DefaultMaxRAM	Maximum real memory size for setting server class heap size	G	uintx
DefaultMaxRAMFraction	Fraction (1/n) of real memory used for server class max heap	4	uintx
DefaultInitialRAMFraction	Fraction (1/n) of real memory used for server class initial heap	64	uintx
UseAutoGCSelectPolicy	Use automatic collection selection policy	false	bool
AutoGCSelectPauseMillis	Automatic GC selection pause threshhold in ms	5000	uintx
UseAdaptiveSizePolicy	Use adaptive generation sizing policies	true	bool
UsePSAdaptiveSurvivorSizePolicy	Use adaptive survivor sizing policies	true	bool
UseAdaptiveGenerationSizePolicyAtMinorCollection	Use adaptive young-old sizing policies at minor collections	true	bool
UseAdaptiveGenerationSizePolicyAtMajorCollection	Use adaptive young-old sizing policies at major collections	true	bool
UseAdaptiveSizePolicyWithSystemGC	Use statistics from System.GC for adaptive size policy	false	bool
UseAdaptiveGCBoundary	Allow young-old boundary to move	false	bool
AdaptiveSizeThroughPutPolicy	Policy for changeing generation size for throughput goals	0	uintx
AdaptiveSizePausePolicy	Policy for changing generation size for pause goals	0	uintx
AdaptiveSizePolicyInitializingSteps	Number of steps where heuristics is used before data is used	20	uintx
AdaptiveSizePolicyOutputInterval	Collecton interval for printing information, zero => never	0	uintx
UseAdaptiveSizePolicyFootprintGoal	Use adaptive minimum footprint as a goal	true	bool
AdaptiveSizePolicyWeight	Weight given to exponential resizing, between 0 and 100	10	uintx
AdaptiveTimeWeight	Weight given to time in adaptive policy, between 0 and 100	25	uintx
PausePadding	How much buffer to keep for pause time	1	uintx
PromotedPadding	How much buffer to keep for promotion failure	3	uintx
SurvivorPadding	How much buffer to keep for survivor overflow	3	uintx
AdaptivePermSizeWeight	Weight for perm gen exponential resizing, between 0 and 100	20	uintx
PermGenPadding	How much buffer to keep for perm gen sizing	3	uintx
ThresholdTolerance	Allowed collection cost difference between generations	10	uintx
AdaptiveSizePolicyCollectionCostMargin	If collection costs are within margin, reduce both by full delta	50	uintx
YoungGenerationSizeIncrement	Adaptive size percentage change in young generation	20	uintx
YoungGenerationSizeSupplement	Supplement to YoungedGenerationSizeIncrement used at startup	80	uintx
YoungGenerationSizeSupplementDecay	Decay factor to YoungedGenerationSizeSupplement	8	uintx
TenuredGenerationSizeIncrement	Adaptive size percentage change in tenured generation	20	uintx
TenuredGenerationSizeSupplement	Supplement to TenuredGenerationSizeIncrement used at startup	80	uintx
TenuredGenerationSizeSupplementDecay	Decay factor to TenuredGenerationSizeIncrement	2	uintx
MaxGCPauseMillis	Adaptive size policy maximum GC pause time goal in msec	max_uintx	uintx
MaxGCMinorPauseMillis	Adaptive size policy maximum GC minor pause time goal in msec	max_uintx	uintx
GCTimeRatio	Adaptive size policy application time to GC time ratio	99	uintx
AdaptiveSizeDecrementScaleFactor	Adaptive size scale down factor for shrinking	4	uintx
UseAdaptiveSizeDecayMajorGCCost	Adaptive size decays the major cost for long major intervals	true	bool
AdaptiveSizeMajorGCDecayTimeScale	Time scale over which major costs decay	10	uintx
MinSurvivorRatio	Minimum ratio of young generation/survivor space size	3	uintx
InitialSurvivorRatio	Initial ratio of eden/survivor space size	8	uintx
BaseFootPrintEstimate	Estimate of footprint other than Java Heap	256*M	uintx
UseGCOverheadLimit	Use policy to limit of proportion of time spent in GC before an OutOfMemory error is thrown	true	bool
GCTimeLimit	Limit of proportion of time spent in GC before an OutOfMemory" error is thrown (used with GCHeapFreeLimit)	98	uintx
GCHeapFreeLimit	Minimum percentage of free space after a full GC before an OutOfMemoryError is thrown (used with GCTimeLimit)	2	uintx
PrintAdaptiveSizePolicy	Print information about AdaptiveSizePolicy	false	bool
DisableExplicitGC	Tells whether calling System.gc() does a full GC	false	bool
CollectGen0First	Collect youngest generation before each full GC	false	bool
BindGCTaskThreadsToCPUs	Bind GCTaskThreads to CPUs if possible	false	bool
UseGCTaskAffinity	Use worker affinity when asking for GCTasks	false	bool
ProcessDistributionStride	Stride through processors when distributing processes	4	uintx
CMSCoordinatorYieldSleepCount	number of times the coordinator GC thread will sleep while yielding before giving up and resuming GC	10	uintx
CMSYieldSleepCount	number of times a GC thread (minus the coordinator) will sleep while yielding before giving up and resuming GC	0	uintx
PrintGCTaskTimeStamps	Print timestamps for individual gc worker thread tasks	false	bool
TraceClassLoadingPreorder	Trace all classes loaded in order referenced (not loaded)	false	bool
TraceGen0Time	Trace accumulated time for Gen 0 collection	false	bool
TraceGen1Time	Trace accumulated time for Gen 1 collection	false	bool
PrintTenuringDistribution	Print tenuring age information	false	bool
PrintHeapAtSIGBREAK	Print heap layout in response to SIGBREAK	true	bool
TraceParallelOldGCTasks	Trace multithreaded GC activity	false	bool
PrintParallelOldGCPhaseTimes	Print the time taken by each parallel old gc phase." PrintGCDetails must also be enabled.	false	bool
CITime	collect timing information for compilation	false	bool
Inline	enable inlining	true	bool
ClipInlining	clip inlining if aggregate method exceeds DesiredMethodLimit	true	bool
UseTypeProfile	Check interpreter profile for historically monomorphic calls	true	bool
TypeProfileMinimumRatio	Minimum ratio of profiled majority type to all minority types	9	intx
Tier1UpdateMethodData	Update methodDataOops in Tier1-generated code	false	bool
PrintVMOptions	print VM flag settings	trueInDebug	bool
ErrorFile	If an error occurs, save the error data to this file [default: ./hs_err_pid%p.log] (%p replaced with pid)	""	ccstr
DisplayVMOutputToStderr	If DisplayVMOutput is true, display all VM output to stderr	false	bool
DisplayVMOutputToStdout	If DisplayVMOutput is true, display all VM output to stdout	false	bool
UseHeavyMonitors	use heavyweight instead of lightweight Java monitors	false	bool
RangeCheckElimination	Split loop iterations to eliminate range checks	true	bool
SplitIfBlocks	Clone compares and control flow through merge points to fold some branches	true	bool
AggressiveOpts	Enable aggressive optimizations - see arguments.cpp	false	bool
PrintInterpreter	Prints the generated interpreter code	false	bool
UseInterpreter	Use interpreter for non-compiled methods	true	bool
UseNiagaraInstrs	Use Niagara-efficient instruction subset	false	bool
UseLoopCounter	Increment invocation counter on backward branch	true	bool
UseFastEmptyMethods	Use fast method entry code for empty methods	true	bool
UseFastAccessorMethods	Use fast method entry code for accessor methods	true	bool
EnableJVMPIInstructionStartEvent	Enable JVMPI_EVENT_INSTRUCTION_START events - slows down interpretation	false	bool
JVMPICheckGCCompatibility	If JVMPI is used, make sure that we are using a JVMPI-compatible garbage collector	true	bool
ProfileMaturityPercentage	number of method invocations/branches (expressed as % of CompileThreshold) before using the method's profile	20	intx
UseCompiler	use compilation	true	bool
UseCounterDecay	adjust recompilation counters	true	bool
AlwaysCompileLoopMethods	when using recompilation, never interpret methods containing loops	false	bool
DontCompileHugeMethods	don't compile methods > HugeMethodLimit	true	bool
EstimateArgEscape	Analyze bytecodes to estimate escape state of arguments	true	bool
BCEATraceLevel	How much tracing to do of bytecode escape analysis estimates	0	intx
MaxBCEAEstimateLevel	Maximum number of nested calls that are analyzed by BC EA.	5	intx
MaxBCEAEstimateSize	Maximum bytecode size of a method to be analyzed by BC EA.	150	intx
SelfDestructTimer	Will cause VM to terminate after a given time (in minutes) (0 means off)	0	intx
MaxJavaStackTraceDepth	Max. no. of lines in the stack trace for Java exceptions (0 means all)	1024	intx
NmethodSweepFraction	Number of invocations of sweeper to cover all nmethods	4	intx
MaxInlineSize	maximum bytecode size of a method to be inlined	35	intx
ProfileIntervalsTicks	# of ticks between printing of interval profile (+ProfileIntervals)	100	intx
EventLogLength	maximum nof events in event log	2000	intx
PerMethodRecompilationCutoff	After recompiling N times, stay in the interpreter (-1=>'Inf')	400	intx
PerBytecodeRecompilationCutoff	Per-BCI limit on repeated recompilation (-1=>'Inf')	100	intx
PerMethodTrapLimit	Limit on traps (of one kind) in a method (includes inlines)	100	intx
PerBytecodeTrapLimit	Limit on traps (of one kind) at a particular BCI	4	intx
AliasLevel	0 for no aliasing, 1 for oop/field/static/array split, 2 for best	2	intx
ReadSpinIterations	Number of read attempts before a yield (spin inner loop)	100	intx
PreBlockSpin	Number of times to spin in an inflated lock before going to an OS lock	10	intx
MaxHeapSize	Default maximum size for object heap (in bytes)	ScaleForWordSize (64*M)	uintx
MaxNewSize	Maximum size of new generation (in bytes)	max_uintx	uintx
PretenureSizeThreshold	Max size in bytes of objects allocated in DefNew generation	0	uintx
MinTLABSize	Minimum allowed TLAB size (in bytes)	2*K	uintx
TLABAllocationWeight	Allocation averaging weight	35	uintx
TLABWasteTargetPercent	Percentage of Eden that can be wasted	1	uintx
TLABRefillWasteFraction	Max TLAB waste at a refill (internal fragmentation)	64	uintx
TLABWasteIncrement	Increment allowed waste at slow allocation	4	uintx
MaxLiveObjectEvacuationRatio	Max percent of eden objects that will be live at scavenge	100	uintx
OldSize	Default size of tenured generation (in bytes)	ScaleForWordSize (4096*K)	uintx
MinHeapFreeRatio	Min percentage of heap free after GC to avoid expansion	40	uintx
MaxHeapFreeRatio	Max percentage of heap free after GC to avoid shrinking	70	uintx
SoftRefLRUPolicyMSPerMB	Number of milliseconds per MB of free space in the heap	1000	intx
MinHeapDeltaBytes	Min change in heap space due to GC (in bytes)	ScaleForWordSize (128*K)	uintx
MinPermHeapExpansion	Min expansion of permanent heap (in bytes)	ScaleForWordSize (256*K)	uintx
MaxPermHeapExpansion	Max expansion of permanent heap without full GC (in bytes)	ScaleForWordSize (4*M)	uintx
QueuedAllocationWarningCount	Number of times an allocation that queues behind a GC will retry before printing a warning	0	intx
MaxTenuringThreshold	Maximum value for tenuring threshold. See more info about that flag here.	15	intx
InitialTenuringThreshold	Initial value for tenuring threshold	7	intx
TargetSurvivorRatio	Desired percentage of survivor space used after scavenge	50	intx
MarkSweepDeadRatio	Percentage (0-100) of the old gen allowed as dead wood. "Serial mark sweep treats this as both the min and max value." CMS uses this value only if it falls back to mark sweep." Par compact uses a variable scale based on the density of the" generation and treats this as the max value when the heap is" either completely full or completely empty. Par compact also" has a smaller default value; see arguments.cpp.	5	intx
PermMarkSweepDeadRatio	Percentage (0-100) of the perm gen allowed as dead wood." See MarkSweepDeadRatio for collector-specific comments.	20	intx
MarkSweepAlwaysCompactCount	How often should we fully compact the heap (ignoring the dead space parameters)	4	intx
PrintCMSStatistics	Statistics for CMS	0	intx
PrintCMSInitiationStatistics	Statistics for initiating a CMS collection	false	bool
PrintFLSStatistics	Statistics for CMS' FreeListSpace	0	intx
PrintFLSCensus	Census for CMS' FreeListSpace	0	intx
DeferThrSuspendLoopCount	(Unstable) Number of times to iterate in safepoint loop before blocking VM threads	4000	intx
DeferPollingPageLoopCount	(Unsafe,Unstable) Number of iterations in safepoint loop before changing safepoint polling page to RO	-1	intx
SafepointSpinBeforeYield	(Unstable)	2000	intx
UseDepthFirstScavengeOrder	true: the scavenge order will be depth-first, false: the scavenge order will be breadth-first	true	bool
GCDrainStackTargetSize	how many entries we'll try to leave on the stack during parallel GC	64	uintx
ThreadSafetyMargin	Thread safety margin is used on fixed-stack LinuxThreads (on Linux/x86 only) to prevent heap-stack collision. Set to 0 to disable this feature	50*M	uintx
CodeCacheMinimumFreeSpace	When less than X space left, we stop compiling.	500*K	uintx
CompileOnly	List of methods (pkg/class.name) to restrict compilation to	""	ccstr
CompileCommandFile	Read compiler commands from this file [.hotspot_compiler]	""	ccstr
CompileCommand	Prepend to .hotspot_compiler; e.g. log,java/lang/String.	""	ccstr
CICompilerCountPerCPU	1 compiler thread for log(N CPUs)	false	bool
UseThreadPriorities	Use native thread priorities	true	bool
ThreadPriorityPolicy	0 : Normal. VM chooses priorities that are appropriate for normal applications. On Solaris NORM_PRIORITY and above are mapped to normal native priority. Java priorities below NORM_PRIORITY" map to lower native priority values. On Windows applications" are allowed to use higher native priorities. However, with ThreadPriorityPolicy=0, VM will not use the highest possible" native priority, THREAD_PRIORITY_TIME_CRITICAL, as it may interfere with system threads. On Linux thread priorities are ignored because the OS does not support static priority in SCHED_OTHER scheduling class which is the only choice for" non-root, non-realtime applications. 1 : Aggressive. Java thread priorities map over to the entire range of native thread priorities. Higher Java thread priorities map to higher native thread priorities. This policy should be used with care, as sometimes it can cause performance degradation in the application and/or the entire system. On Linux this policy requires root privilege.	0	intx
ThreadPriorityVerbose	print priority changes	false	bool
DefaultThreadPriority	what native priority threads run at if not specified elsewhere (-1 means no change)	-1	intx
CompilerThreadPriority	what priority should compiler threads run at (-1 means no change)	-1	intx
VMThreadPriority	what priority should VM threads run at (-1 means no change)	-1	intx
CompilerThreadHintNoPreempt	(Solaris only) Give compiler threads an extra quanta	true	bool
VMThreadHintNoPreempt	(Solaris only) Give VM thread an extra quanta	false	bool
JavaPriority1_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority2_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority3_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority4_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority5_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority6_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority7_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority8_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority9_To_OSPriority	Map Java priorities to OS priorities	-1	intx
JavaPriority10_To_OSPriority	Map Java priorities to OS priorities	-1	intx
StarvationMonitorInterval	Pause between each check in ms	200	intx
Tier1BytecodeLimit	Must have at least this many bytecodes before tier1" invocation counters are used	10	intx
StressTieredRuntime	Alternate client and server compiler on compile requests	false	bool
InterpreterProfilePercentage	number of method invocations/branches (expressed as % of CompileThreshold) before profiling in the interpreter	33	intx
MaxDirectMemorySize	Maximum total size of NIO direct-buffer allocations	-1	intx
UseUnsupportedDeprecatedJVMPI	Flag to temporarily re-enable the, soon to be removed, experimental interface JVMPI.	false	bool
UsePerfData	Flag to disable jvmstat instrumentation for performance testing" and problem isolation purposes.	true	bool
PerfDataSaveToFile	Save PerfData memory to hsperfdata_ file on exit	false	bool
PerfDataSamplingInterval	Data sampling interval in milliseconds	50 /ms/	intx
PerfDisableSharedMem	Store performance data in standard memory	false	bool
PerfDataMemorySize	Size of performance data memory region. Will be rounded up to a multiple of the native os page size.	32*K	intx
PerfMaxStringConstLength	Maximum PerfStringConstant string length before truncation	1024	intx
PerfAllowAtExitRegistration	Allow registration of atexit() methods	false	bool
PerfBypassFileSystemCheck	Bypass Win32 file system criteria checks (Windows Only)	false	bool
UnguardOnExecutionViolation	Unguard page and retry on no-execute fault (Win32 only)" 0=off, 1=conservative, 2=aggressive	0	intx
ManagementServer	Create JMX Management Server	false	bool
DisableAttachMechanism	Disable mechanism that allows tools to attach to this VM	false	bool
StartAttachListener	Always start Attach Listener at VM startup	false	bool
UseSharedSpaces	Use shared spaces in the permanent generation	true	bool
RequireSharedSpaces	Require shared spaces in the permanent generation	false	bool
ForceSharedSpaces	Require shared spaces in the permanent generation	false	bool
DumpSharedSpaces	Special mode: JVM reads a class list, loads classes, builds shared spaces, and dumps the shared spaces to a file to be used in future JVM runs.	false	bool
PrintSharedSpaces	Print usage of shared spaces	false	bool
SharedDummyBlockSize	Size of dummy block used to shift heap addresses (in bytes)	512*M	uintx
SharedReadWriteSize	Size of read-write space in permanent generation (in bytes)	12*M	uintx
SharedReadOnlySize	Size of read-only space in permanent generation (in bytes)	8*M	uintx
SharedMiscDataSize	Size of the shared data area adjacent to the heap (in bytes)	4*M	uintx
SharedMiscCodeSize	Size of the shared code area adjacent to the heap (in bytes)	4*M	uintx
TaggedStackInterpreter	Insert tags in interpreter execution stack for oopmap generaion	false	bool
ExtendedDTraceProbes	Enable performance-impacting dtrace probes	false	bool
DTraceMethodProbes	Enable dtrace probes for method-entry and method-exit	false	bool
DTraceAllocProbes	Enable dtrace probes for object allocation	false	bool
DTraceMonitorProbes	Enable dtrace probes for monitor events	false	bool
RelaxAccessControlCheck	Relax the access control checks in the verifier	false	bool
UseVMInterruptibleIO	(Unstable, Solaris-specific) Thread interrupt before or with EINTR for I/O operations results in OS_INTRPT	true	bool
AggressiveHeap	The option inspects the server resources (size of memory and number of processors), and attempts to set various parameters to be optimal for long-running, memory allocation-intensive jobs. The JVM team view AggressiveHeap as an anachronism and would like to see it go away. Instead, we'd prefer for you to determine which of the individual options that AggressiveHeap sets actually impact your app, and then set those on the command line directly. You can check the Open JDK source code to see what AggressiveHeap actually does (arguments.cpp)	false	bool
UseCompressedStrings	Use a byte[] for Strings which can be represented as pure ASCII. (Introduced in Java 6 Update 21 Performance Release)	false	bool
OptimizeStringConcat	Optimize String concatenation operations where possible. (Introduced in Java 6 Update 20)	false	bool
UseStringCache	Enables caching of commonly allocated strings.	false	bool
manageable
HeapDumpOnOutOfMemoryError	Dump heap to file when java.lang.OutOfMemoryError is thrown	false	bool
HeapDumpPath	When HeapDumpOnOutOfMemoryError is on, the path (filename or" directory) of the dump file (defaults to java_pid.hprof" in the working directory)	""	ccstr
PrintGC	Print message at garbage collect	false	bool
PrintGCDetails	Print more details at garbage collect	false	bool
PrintGCTimeStamps	Print timestamps at garbage collect	false	bool
PrintClassHistogram	Print a histogram of class instances	false	bool
PrintConcurrentLocks	Print java.util.concurrent locks in thread dump	false	bool
experimental
UseG1GC	Switch on G1 for Java6. G1 is default for Java7, so there is no such option there.	false	bool
product_rw
TraceClassLoading	Trace all classes loaded	false	bool
TraceClassUnloading	Trace unloading of classes	false	bool
TraceLoaderConstraints	Trace loader constraints	false	bool
PrintHeapAtGC	Print heap layout before and after each GC	false	bool
develop
TraceItables	Trace initialization and use of itables	false	bool
TracePcPatching	Trace usage of frame::patch_pc	false	bool
TraceJumps	Trace assembly jumps in thread ring buffer	false	bool
TraceRelocator	Trace the bytecode relocator	false	bool
TraceLongCompiles	Print out every time compilation is longer than a given threashold	false	bool
SafepointALot	Generates a lot of safepoints. Works with GuaranteedSafepointInterval	false	bool
BailoutToInterpreterForThrows	Compiled methods which throws/catches exceptions will be deopt and intp.	false	bool
NoYieldsInMicrolock	Disable yields in microlock	false	bool
TraceOopMapGeneration	Shows oopmap generation	false	bool
MethodFlushing	Reclamation of zombie and not-entrant methods	true	bool
VerifyStack	Verify stack of each thread when it is entering a runtime call	false	bool
TraceDerivedPointers	Trace traversal of derived pointers on stack	false	bool
InlineArrayCopy	inline arraycopy native that is known to be part of base library DLL	true	bool
InlineObjectHash	inline Object::hashCode() native that is known to be part of base library DLL	true	bool
InlineNatives	inline natives that are known to be part of base library DLL	true	bool
InlineMathNatives	inline SinD, CosD, etc.	true	bool
InlineClassNatives	inline Class.isInstance, etc	true	bool
InlineAtomicLong	inline sun.misc.AtomicLong	true	bool
InlineThreadNatives	inline Thread.currentThread, etc	true	bool
InlineReflectionGetCallerClass	inline sun.reflect.Reflection.getCallerClass(), known to be part of base library DLL	true	bool
InlineUnsafeOps	inline memory ops (native methods) from sun.misc.Unsafe	true	bool
ConvertCmpD2CmpF	Convert cmpD to cmpF when one input is constant in float range	true	bool
ConvertFloat2IntClipping	Convert float2int clipping idiom to integer clipping	true	bool
SpecialStringCompareTo	special version of string compareTo	true	bool
SpecialStringIndexOf	special version of string indexOf	true	bool
TraceCallFixup	traces all call fixups	false	bool
DeoptimizeALot	deoptimize at every exit from the runtime system	false	bool
DeoptimizeOnlyAt	a comma separated list of bcis to deoptimize at	""	ccstr
Debugging	set when executing debug methods in debug.ccp (to prevent triggering assertions)	false	bool
TraceHandleAllocation	Prints out warnings when suspicious many handles are allocated	false	bool
ShowSafepointMsgs	Show msg. about safepoint synch.	false	bool
SafepointTimeout	Time out and warn or fail after SafepointTimeoutDelay milliseconds if failed to reach safepoint	false	bool
DieOnSafepointTimeout	Die upon failure to reach safepoint (see SafepointTimeout)	false	bool
ForceFloatExceptions	Force exceptions on FP stack under/overflow	trueInDebug	bool
SoftMatchFailure	If the DFA fails to match a node, print a message and bail out	trueInProduct	bool
VerifyStackAtCalls	Verify that the stack pointer is unchanged after calls	false	bool
TraceJavaAssertions	Trace java language assertions	false	bool
ZapDeadCompiledLocals	Zap dead locals in compiler frames	false	bool
UseMallocOnly	use only malloc/free for allocation (no resource area/arena)	false	bool
PrintMalloc	print all malloc/free calls	false	bool
ZapResourceArea	Zap freed resource/arena space with 0xABABABAB	trueInDebug	bool
ZapJNIHandleArea	Zap freed JNI handle space with 0xFEFEFEFE	trueInDebug	bool
ZapUnusedHeapArea	Zap unused heap space with 0xBAADBABE	trueInDebug	bool
PrintVMMessages	Print vm messages on console	true	bool
Verbose	Prints additional debugging information from other modes	false	bool
PrintMiscellaneous	Prints uncategorized debugging information (requires +Verbose)	false	bool
WizardMode	Prints much more debugging information	false	bool
SegmentedHeapDumpThreshold	Generate a segmented heap dump (JAVA PROFILE 1.0.2 format) when the heap usage is larger than this	2*G	uintx
HeapDumpSegmentSize	Approximate segment size when generating a segmented heap dump	1*G	uintx
BreakAtWarning	Execute breakpoint upon encountering VM warning	false	bool
TraceVMOperation	Trace vm operations	false	bool
UseFakeTimers	Tells whether the VM should use system time or a fake timer	false	bool
PrintAssembly	Print assembly code	false	bool
PrintNMethods	Print assembly code for nmethods when generated	false	bool
PrintNativeNMethods	Print assembly code for native nmethods when generated	false	bool
PrintDebugInfo	Print debug information for all nmethods when generated	false	bool
PrintRelocations	Print relocation information for all nmethods when generated	false	bool
PrintDependencies	Print dependency information for all nmethods when generated	false	bool
PrintExceptionHandlers	Print exception handler tables for all nmethods when generated	false	bool
InterceptOSException	Starts debugger when an implicit OS (e.g., NULL) exception happens	false	bool
PrintCodeCache2	Print detailed info on the compiled_code cache when exiting	false	bool
PrintStubCode	Print generated stub code	false	bool
PrintJVMWarnings	Prints warnings for unimplemented JVM functions	false	bool
InitializeJavaLangSystem	Initialize java.lang.System - turn off for individual method debugging	true	bool
InitializeJavaLangString	Initialize java.lang.String - turn off for individual method debugging	true	bool
InitializeJavaLangExceptionsErrors	Initialize various error and exception classes - turn off for individual method debugging	true	bool
RegisterReferences	Tells whether the VM should register soft/weak/final/phantom references	true	bool
IgnoreRewrites	Supress rewrites of bytecodes in the oopmap generator. This is unsafe!	false	bool
PrintCodeCacheExtension	Print extension of code cache	false	bool
UsePrivilegedStack	Enable the security JVM functions	true	bool
IEEEPrecision	Enables IEEE precision (for INTEL only)	true	bool
ProtectionDomainVerification	Verifies protection domain before resolution in system dictionary	true	bool
DisableStartThread	Disable starting of additional Java threads (for debugging only)	false	bool
MemProfiling	Write memory usage profiling to log file	false	bool
UseDetachedThreads	Use detached threads that are recycled upon termination (for SOLARIS only)	true	bool
UsePthreads	Use pthread-based instead of libthread-based synchronization (SPARC only)	false	bool
UpdateHotSpotCompilerFileOnError	Should the system attempt to update the compiler file when an error occurs?	true	bool
LoadLineNumberTables	Tells whether the class file parser loads line number tables	true	bool
LoadLocalVariableTables	Tells whether the class file parser loads local variable tables	true	bool
LoadLocalVariableTypeTables	Tells whether the class file parser loads local variable type tables	true	bool
PreallocatedOutOfMemoryErrorCount	Number of OutOfMemoryErrors preallocated with backtrace	4	uintx
PrintBiasedLockingStatistics	Print statistics of biased locking in JVM	false	bool
TraceJVMPI	Trace JVMPI	false	bool
TraceJNICalls	Trace JNI calls	false	bool
TraceJNIHandleAllocation	Trace allocation/deallocation of JNI handle blocks	false	bool
TraceThreadEvents	Trace all thread events	false	bool
TraceBytecodes	Trace bytecode execution	false	bool
TraceClassInitialization	Trace class initialization	false	bool
TraceExceptions	Trace exceptions	false	bool
TraceICs	Trace inline cache changes	false	bool
TraceInlineCacheClearing	Trace clearing of inline caches in nmethods	false	bool
TraceDependencies	Trace dependencies	false	bool
VerifyDependencies	Exercise and verify the compilation dependency mechanism	trueInDebug	bool
TraceNewOopMapGeneration	Trace OopMapGeneration	false	bool
TraceNewOopMapGenerationDetailed	Trace OopMapGeneration: print detailed cell states	false	bool
TimeOopMap	Time calls to GenerateOopMap::compute_map() in sum	false	bool
TimeOopMap2	Time calls to GenerateOopMap::compute_map() individually	false	bool
TraceMonitorMismatch	Trace monitor matching failures during OopMapGeneration	false	bool
TraceOopMapRewrites	Trace rewritting of method oops during oop map generation	false	bool
TraceSafepoint	Trace safepoint operations	false	bool
TraceICBuffer	Trace usage of IC buffer	false	bool
TraceCompiledIC	Trace changes of compiled IC	false	bool
TraceStartupTime	Trace setup time	false	bool
TraceHPI	Trace Host Porting Interface (HPI)	false	bool
TraceProtectionDomainVerification	Trace protection domain verifcation	false	bool
TraceClearedExceptions	Prints when an exception is forcibly cleared	false	bool
UseParallelOldGCChunkPointerCalc	In the Parallel Old garbage collector use chucks to calculate" new object locations	true	bool
VerifyParallelOldWithMarkSweep	Use the MarkSweep code to verify phases of Parallel Old	false	bool
VerifyParallelOldWithMarkSweepInterval	Interval at which the MarkSweep code is used to verify phases of Parallel Old	1	uintx
ParallelOldMTUnsafeMarkBitMap	Use the Parallel Old MT unsafe in marking the bitmap	false	bool
ParallelOldMTUnsafeUpdateLiveData	Use the Parallel Old MT unsafe in update of live size	false	bool
TraceChunkTasksQueuing	Trace the queuing of the chunk tasks	false	bool
ScavengeWithObjectsInToSpace	Allow scavenges to occur when to_space contains objects.	false	bool
UseCMSAdaptiveFreeLists	Use Adaptive Free Lists in the CMS generation	true	bool
UseAsyncConcMarkSweepGC	Use Asynchronous Concurrent Mark-Sweep GC in the old generation	true	bool
RotateCMSCollectionTypes	Rotate the CMS collections among concurrent and STW	false	bool
CMSTraceIncrementalMode	Trace CMS incremental mode	false	bool
CMSTraceIncrementalPacing	Trace CMS incremental mode pacing computation	false	bool
CMSTraceThreadState	Trace the CMS thread state (enable the trace_state() method)	false	bool
CMSDictionaryChoice	Use BinaryTreeDictionary as default in the CMS generation	0	intx
CMSOverflowEarlyRestoration	Whether preserved marks should be restored early	false	bool
CMSTraceSweeper	Trace some actions of the CMS sweeper	false	bool
FLSVerifyDictionary	Do lots of (expensive) FLS dictionary verification	false	bool
VerifyBlockOffsetArray	Do (expensive!) block offset array verification	false	bool
TraceCMSState	Trace the state of the CMS collection	false	bool
CMSTestInFreeList	Check if the coalesced range is already in the free lists as claimed.	false	bool
CMSIgnoreResurrection	Ignore object resurrection during the verification.	true	bool
FullGCALot	Force full gc at every Nth exit from the runtime system (N=FullGCALotInterval)	false	bool
PromotionFailureALotCount	Number of promotion failures occurring at ParGCAllocBuffer" refill attempts (ParNew) or promotion attempts (other young collectors)	1000	uintx
PromotionFailureALotInterval	Total collections between promotion failures alot	5	uintx
WorkStealingSleepMillis	Sleep time when sleep is used for yields	1	intx
WorkStealingYieldsBeforeSleep	Number of yields before a sleep is done during workstealing	1000	uintx
TraceAdaptiveGCBoundary	Trace young-old boundary moves	false	bool
PSAdaptiveSizePolicyResizeVirtualSpaceAlot	Resize the virtual spaces of the young or old generations	-1	intx
PSAdjustTenuredGenForMinorPause	Adjust tenured generation to achive a minor pause goal	false	bool
PSAdjustYoungGenForMajorPause	Adjust young generation to achive a major pause goal	false	bool
AdaptiveSizePolicyReadyThreshold	Number of collections before the adaptive sizing is started	5	uintx
AdaptiveSizePolicyGCTimeLimitThreshold	Number of consecutive collections before gc time limit fires	5	uintx
UsePrefetchQueue	Use the prefetch queue during PS promotion	true	bool
ConcGCYieldTimeout	If non-zero, assert that GC threads yield within this # of ms.	0	intx
TraceReferenceGC	Trace handling of soft/weak/final/phantom references	false	bool
TraceFinalizerRegistration	Trace registration of final references	false	bool
TraceWorkGang	Trace activities of work gangs	false	bool
TraceBlockOffsetTable	Print BlockOffsetTable maps	false	bool
TraceCardTableModRefBS	Print CardTableModRefBS maps	false	bool
TraceGCTaskManager	Trace actions of the GC task manager	false	bool
TraceGCTaskQueue	Trace actions of the GC task queues	false	bool
TraceGCTaskThread	Trace actions of the GC task threads	false	bool
TraceParallelOldGCMarkingPhase	Trace parallel old gc marking phase	false	bool
TraceParallelOldGCSummaryPhase	Trace parallel old gc summary phase	false	bool
TraceParallelOldGCCompactionPhase	Trace parallel old gc compaction phase	false	bool
TraceParallelOldGCDensePrefix	Trace parallel old gc dense prefix computation	false	bool
IgnoreLibthreadGPFault	Suppress workaround for libthread GP fault	false	bool
CIPrintCompilerName	when CIPrint is active, print the name of the active compiler	false	bool
CIPrintCompileQueue	display the contents of the compile queue whenever a compilation is enqueued	false	bool
CIPrintRequests	display every request for compilation	false	bool
CITimeEach	display timing information after each successful compilation	false	bool
CICountOSR	use a separate counter when assigning ids to osr compilations	true	bool
CICompileNatives	compile native methods if supported by the compiler	true	bool
CIPrintMethodCodes	print method bytecodes of the compiled code	false	bool
CIPrintTypeFlow	print the results of ciTypeFlow analysis	false	bool
CITraceTypeFlow	detailed per-bytecode tracing of ciTypeFlow analysis	false	bool
CICloneLoopTestLimit	size limit for blocks heuristically cloned in ciTypeFlow	100	intx
UseStackBanging	use stack banging for stack overflow checks (required for proper StackOverflow handling; disable only to measure cost of stackbanging)	true	bool
Use24BitFPMode	Set 24-bit FPU mode on a per-compile basis	true	bool
Use24BitFP	use FP instructions that produce 24-bit precise results	true	bool
UseStrictFP	use strict fp if modifier strictfp is set	true	bool
GenerateSynchronizationCode	generate locking/unlocking code for synchronized methods and monitors	true	bool
GenerateCompilerNullChecks	Generate explicit null checks for loads/stores/calls	true	bool
GenerateRangeChecks	Generate range checks for array accesses	true	bool
PrintSafepointStatistics	print statistics about safepoint synchronization	false	bool
InlineAccessors	inline accessor methods (get/set)	true	bool
UseCHA	enable CHA	true	bool
PrintInlining	prints inlining optimizations	false	bool
EagerInitialization	Eagerly initialize classes if possible	false	bool
TraceMethodReplacement	Print when methods are replaced do to recompilation	false	bool
PrintMethodFlushing	print the nmethods being flushed	false	bool
UseRelocIndex	use an index to speed random access to relocations	false	bool
StressCodeBuffers	Exercise code buffer expansion and other rare state changes	false	bool
DebugVtables	add debugging code to vtable dispatch	false	bool
PrintVtables	print vtables when printing klass	false	bool
TraceCreateZombies	trace creation of zombie nmethods	false	bool
MonomorphicArrayCheck	Uncommon-trap array store checks that require full type check	true	bool
DelayCompilationDuringStartup	Delay invoking the compiler until main application class is loaded	true	bool
CompileTheWorld	Compile all methods in all classes in bootstrap class path (stress test)	false	bool
CompileTheWorldPreloadClasses	Preload all classes used by a class before start loading	true	bool
TraceIterativeGVN	Print progress during Iterative Global Value Numbering	false	bool
FillDelaySlots	Fill delay slots (on SPARC only)	true	bool
VerifyIterativeGVN	Verify Def-Use modifications during sparse Iterative Global Value Numbering	false	bool
TimeLivenessAnalysis	Time computation of bytecode liveness analysis	false	bool
TraceLivenessGen	Trace the generation of liveness analysis information	false	bool
PrintDominators	Print out dominator trees for GVN	false	bool
UseLoopSafepoints	Generate Safepoint nodes in every loop	true	bool
DeutschShiffmanExceptions	Fast check to find exception handler for precisely typed exceptions	true	bool
FastAllocateSizeLimit	Inline allocations larger than this in doublewords must go slow	100000	intx
UseVTune	enable support for Intel's VTune profiler	false	bool
CountCompiledCalls	counts method invocations	false	bool
CountJNICalls	counts jni method invocations	false	bool
ClearInterpreterLocals	Always clear local variables of interpreter activations upon entry	false	bool
UseFastSignatureHandlers	Use fast signature handlers for native calls	true	bool
UseV8InstrsOnly	Use SPARC-V8 Compliant instruction subset	false	bool
UseCASForSwap	Do not use swap instructions, but only CAS (in a loop) on SPARC	false	bool
PoisonOSREntry	Detect abnormal calls to OSR code	true	bool
CountBytecodes	Count number of bytecodes executed	false	bool
PrintBytecodeHistogram	Print histogram of the executed bytecodes	false	bool
PrintBytecodePairHistogram	Print histogram of the executed bytecode pairs	false	bool
PrintSignatureHandlers	Print code generated for native method signature handlers	false	bool
VerifyOops	Do plausibility checks for oops	false	bool
CheckUnhandledOops	Check for unhandled oops in VM code	false	bool
VerifyJNIFields	Verify jfieldIDs for instance fields	trueInDebug	bool
VerifyFPU	Verify FPU state (check for NaN's, etc.)	false	bool
VerifyThread	Watch the thread register for corruption (SPARC only)	false	bool
VerifyActivationFrameSize	Verify that activation frame didn't become smaller than its minimal size	false	bool
TraceFrequencyInlining	Trace frequency based inlining	false	bool
PrintMethodData	Print the results of +ProfileInterpreter at end of run	false	bool
VerifyDataPointer	Verify the method data pointer during interpreter profiling	trueInDebug	bool
TraceCompilationPolicy	Trace compilation policy	false	bool
TimeCompilationPolicy	Time the compilation policy	false	bool
CounterHalfLifeTime	half-life time of invocation counters (in secs)	30	intx
CounterDecayMinIntervalLength	Min. ms. between invocation of CounterDecay	500	intx
TraceDeoptimization	Trace deoptimization	false	bool
DebugDeoptimization	Tracing various information while debugging deoptimization	false	bool
GuaranteedSafepointInterval	Guarantee a safepoint (at least) every so many milliseconds (0 means none)	1000	intx
SafepointTimeoutDelay	Delay in milliseconds for option SafepointTimeout	10000	intx
MallocCatchPtr	Hit breakpoint when mallocing/freeing this pointer	-1	intx
TotalHandleAllocationLimit	Threshold for total handle allocation when +TraceHandleAllocation is used	1024	uintx
StackPrintLimit	number of stack frames to print in VM-level stack dump	100	intx
MaxInlineLevel	maximum number of nested calls that are inlined	9	intx
MaxRecursiveInlineLevel	maximum number of nested recursive calls that are inlined	1	intx
InlineSmallCode	Only inline already compiled methods if their code size is less than this	1000	intx
MaxTrivialSize	maximum bytecode size of a trivial method to be inlined	6	intx
MinInliningThreshold	min. invocation count a method needs to have to be inlined	250	intx
AlignEntryCode	aligns entry code to specified value (in bytes)	4	intx
MethodHistogramCutoff	cutoff value for method invoc. histogram (+CountCalls)	100	intx
ProfilerNumberOfInterpretedMethods	# of interpreted methods to show in profile	25	intx
ProfilerNumberOfCompiledMethods	# of compiled methods to show in profile	25	intx
ProfilerNumberOfStubMethods	# of stub methods to show in profile	25	intx
ProfilerNumberOfRuntimeStubNodes	# of runtime stub nodes to show in profile	25	intx
DontYieldALotInterval	Interval between which yields will be dropped (milliseconds)	10	intx
MinSleepInterval	Minimum sleep() interval (milliseconds) when ConvertSleepToYield is off (used for SOLARIS)	1	intx
ProfilerPCTickThreshold	Number of ticks in a PC buckets to be a hotspot	15	intx
StressNonEntrant	Mark nmethods non-entrant at registration	false	bool
TypeProfileWidth	number of receiver types to record in call profile	2	intx
BciProfileWidth	number of return bci's to record in ret profile	2	intx
FreqCountInvocations	Scaling factor for branch frequencies (deprecated)	1	intx
InlineFrequencyRatio	Ratio of call site execution to caller method invocation	20	intx
InlineThrowCount	Force inlining of interpreted methods that throw this often	50	intx
InlineThrowMaxSize	Force inlining of throwing methods smaller than this	200	intx
VerifyAliases	perform extra checks on the results of alias analysis	false	bool
ProfilerNodeSize	Size in K to allocate for the Profile Nodes of each thread	1024	intx
V8AtomicOperationUnderLockSpinCount	Number of times to spin wait on a v8 atomic operation lock	50	intx
ExitAfterGCNum	If non-zero, exit after this GC.	0	uintx
GCExpandToAllocateDelayMillis	Delay in ms between expansion and allocation	0	uintx
CodeCacheSegmentSize	Code cache segment size (in bytes) - smallest unit of allocation	64	uintx
BinarySwitchThreshold	Minimal number of lookupswitch entries for rewriting to binary switch	5	intx
StopInterpreterAt	Stops interpreter execution at specified bytecode number	0	intx
TraceBytecodesAt	Traces bytecodes starting with specified bytecode number	0	intx
CIStart	the id of the first compilation to permit	0	intx
CIStop	the id of the last compilation to permit	-1	intx
CIStartOSR	the id of the first osr compilation to permit (CICountOSR must be on)	0	intx
CIStopOSR	the id of the last osr compilation to permit (CICountOSR must be on)	-1	intx
CIBreakAtOSR	id of osr compilation to break at	-1	intx
CIBreakAt	id of compilation to break at	-1	intx
CIFireOOMAt	Fire OutOfMemoryErrors throughout CI for testing the compiler (non-negative value throws OOM after this many CI accesses in each compile)	-1	intx
CIFireOOMAtDelay	Wait for this many CI accesses to occur in all compiles before beginning to throw OutOfMemoryErrors in each compile	-1	intx
NewCodeParameter	Testing Only: Create a dedicated integer parameter before putback	0	intx
MinOopMapAllocation	Minimum number of OopMap entries in an OopMapSet	8	intx
LongCompileThreshold	Used with +TraceLongCompiles	50	intx
MaxRecompilationSearchLength	max. # frames to inspect searching for recompilee	10	intx
MaxInterpretedSearchLength	max. # interp. frames to skip when searching for recompilee	3	intx
DesiredMethodLimit	desired max. method size (in bytecodes) after inlining	8000	intx
HugeMethodLimit	don't compile methods larger than this if +DontCompileHugeMethods	8000	intx
UseNewReflection	Temporary flag for transition to reflection based on dynamic bytecode generation in 1.4; can no longer be turned off in 1.4 JDK, and is unneeded in 1.3 JDK, but marks most places VM changes were needed	true	bool
VerifyReflectionBytecodes	Force verification of 1.4 reflection bytecodes. Does not work in situations like that described in 4486457 or for constructors generated for serialization, so can not be enabled in product.	false	bool
FastSuperclassLimit	Depth of hardwired instanceof accelerator array	8	intx
PerfTraceDataCreation	Trace creation of Performance Data Entries	false	bool
PerfTraceMemOps	Trace PerfMemory create/attach/detach calls	false	bool
SharedOptimizeColdStartPolicy	Reordering policy for SharedOptimizeColdStart 0=favor classload-time locality, 1=balanced, 2=favor runtime locality	2	intx
product_pd
UseLargePages	Use large page memory		bool
UseSSE	0=fpu stack,1=SSE for floats,2=SSE/SSE2 for all (x86/amd only)		intx
UseISM	Use Intimate Shared Memory. [Not accepted for non-Solaris platforms.] For details, see Intimate Shared Memory.	true	bool
UseMPSS	Use Multiple Page Size Support w/4mb pages for the heap. Do not use with ISM as this replaces the need for ISM. (Introduced in 1.4.0 update 1, Relevant to Solaris 9 and newer.) [1.4.1 and earlier: false]	false	bool
BackgroundCompilation	A thread requesting compilation is not blocked during compilation		bool
UseVectoredExceptions	Temp Flag - Use Vectored Exceptions rather than SEH (Windows Only)		bool
DontYieldALot	Throw away obvious excess yield calls (for SOLARIS only)		bool
ConvertSleepToYield	Converts sleep(0) to thread yield (may be off for SOLARIS to improve GUI)		bool
UseTLAB	Use thread-local object allocation		bool
ResizeTLAB	Dynamically resize tlab size for threads		bool
NeverActAsServerClassMachine	Never act like a server-class machine		bool
PrefetchCopyIntervalInBytes	How far ahead to prefetch destination area (<= 0 means off)		intx
PrefetchScanIntervalInBytes	How far ahead to prefetch scan area (<= 0 means off)		intx
PrefetchFieldsAhead	How many fields ahead to prefetch in oop scan (<= 0 means off)		intx
CompilationPolicyChoice	which compilation policy		intx
RewriteBytecodes	Allow rewriting of bytecodes (bytecodes are not immutable)		bool
RewriteFrequentPairs	Rewrite frequently used bytecode pairs into a single bytecode		bool
UseOnStackReplacement	Use on stack replacement, calls runtime if invoc. counter overflows in loop		bool
PreferInterpreterNativeStubs	Use always interpreter stubs for native methods invoked via interpreter		bool
AllocatePrefetchStyle	0=no prefetch, 1=dead load, 2=prefetch instruction		intx
AllocatePrefetchDistance	Distance to prefetch ahead of allocation pointer		intx
FreqInlineSize	maximum bytecode size of a frequent method to be inlined		intx
PreInflateSpin	Number of times to spin wait before inflation		intx
NewSize	Default size of new generation (in bytes)		uintx
TLABSize	Default (or starting) size of TLAB (in bytes)		uintx
SurvivorRatio	Ratio of eden/survivor space size		intx
NewRatio	Ratio of new/old generation sizes		intx
NewSizeThreadIncrease	Additional size added to desired new generation size per non-daemon thread (in bytes)		uintx
PermSize	Default size of permanent generation (in bytes)		uintx
MaxPermSize	Maximum size of permanent generation (in bytes)		uintx
StackYellowPages	Number of yellow zone (recoverable overflows) pages		intx
StackRedPages	Number of red zone (unrecoverable overflows) pages		intx
StackShadowPages	Number of shadow zone (for overflow checking) pages" this should exceed the depth of the VM and native call stack		intx
ThreadStackSize	Thread Stack Size (in Kbytes)		intx
VMThreadStackSize	Non-Java Thread Stack Size (in Kbytes)		intx
CompilerThreadStackSize	Compiler Thread Stack Size (in Kbytes)		intx
InitialCodeCacheSize	Initial code cache size (in bytes)		uintx
ReservedCodeCacheSize	Reserved code cache size (in bytes) - maximum code cache size		uintx
CodeCacheExpansionSize	Code cache expansion size (in bytes)		uintx
CompileThreshold	number of method invocations/branches before (re-)compiling	10000	intx
Tier2CompileThreshold	threshold at which a tier 2 compilation is invoked		intx
Tier2BackEdgeThreshold	Back edge threshold at which a tier 2 compilation is invoked		intx
TieredCompilation	Enable two-tier compilation		bool
OnStackReplacePercentage	number of method invocations/branches (expressed as % of CompileThreshold) before (re-)compiling OSR code		intx
develop_pd
ShareVtableStubs	Share vtable stubs (smaller code but worse branch prediction		bool
CICompileOSR	compile on stack replacement methods if supported by the compiler		bool
ImplicitNullChecks	generate code for implicit null checks		bool
UncommonNullCast	Uncommon-trap NULLs passed to check cast		bool
InlineIntrinsics	Inline intrinsics that can be statically resolved		bool
ProfileInterpreter	Profile at the bytecode level during interpretation		bool
ProfileTraps	Profile deoptimization traps at the bytecode level		bool
InlineFrequencyCount	Count of call site execution necessary to trigger frequent inlining		intx
JVMInvokeMethodSlack	Stack space (bytes) required for JVM_InvokeMethod to complete		uintx
CodeEntryAlignment	Code entry alignment for generated code (in bytes)		intx
CodeCacheMinBlockLength	Minimum number of segments in a code cache block.		uintx
notproduct
StressDerivedPointers	Force scavenge when a derived pointers is detected on stack after rtm call	false	bool
TraceCodeBlobStacks	Trace stack-walk of codeblobs	false	bool
PrintRewrites	Print methods that are being rewritten	false	bool
DeoptimizeRandom	deoptimize random frames on random exit from the runtime system	false	bool
ZombieALot	creates zombies (non-entrant) at exit from the runt. system	false	bool
WalkStackALot	trace stack (no print) at every exit from the runtime system	false	bool
StrictSafepointChecks	Enable strict checks that safepoints cannot happen for threads that used No_Safepoint_Verifier	trueInDebug	bool
VerifyLastFrame	Verify oops on last frame on entry to VM	false	bool
LogEvents	Enable Event log	trueInDebug	bool
CheckAssertionStatusDirectives	temporary - see javaClasses.cpp	false	bool
PrintMallocFree	Trace calls to C heap malloc/free allocation	false	bool
PrintOopAddress	Always print the location of the oop	false	bool
VerifyCodeCacheOften	Verify compiled-code cache often	false	bool
ZapDeadLocalsOld	Zap dead locals (old version, zaps all frames when entering the VM	false	bool
CheckOopishValues	Warn if value contains oop ( requires ZapDeadLocals)	false	bool
ZapVMHandleArea	Zap freed VM handle space with 0xBCBCBCBC	trueInDebug	bool
PrintCompilation2	Print additional statistics per compilation	false	bool
PrintAdapterHandlers	Print code generated for i2c/c2i adapters	false	bool
PrintCodeCache	Print the compiled_code cache when exiting	false	bool
ProfilerCheckIntervals	Collect and print info on spacing of profiler ticks	false	bool
WarnOnStalledSpinLock	Prints warnings for stalled SpinLocks	0	uintx
PrintSystemDictionaryAtExit	Prints the system dictionary at exit	false	bool
ValidateMarkSweep	Do extra validation during MarkSweep collection	false	bool
RecordMarkSweepCompaction	Enable GC-to-GC recording and querying of compaction during MarkSweep	false	bool
TraceRuntimeCalls	Trace run-time calls	false	bool
TraceJVMCalls	Trace JVM calls	false	bool
TraceInvocationCounterOverflow	Trace method invocation counter overflow	false	bool
TraceZapDeadLocals	Trace zapping dead locals	false	bool
CMSMarkStackOverflowALot	Whether we should simulate frequent marking stack / work queue" overflow	false	bool
CMSMarkStackOverflowInterval	A per-thread `interval' counter that determines how frequently" we simulate overflow; a smaller number increases frequency	1000	intx
CMSVerifyReturnedBytes	Check that all the garbage collected was returned to the free lists.	false	bool
ScavengeALot	Force scavenge at every Nth exit from the runtime system (N=ScavengeALotInterval)	false	bool
GCALotAtAllSafepoints	Enforce ScavengeALot/GCALot at all potential safepoints	false	bool
PromotionFailureALot	Use promotion failure handling on every youngest generation collection	false	bool
CheckMemoryInitialization	Checks memory initialization	false	bool
TraceMarkSweep	Trace mark sweep	false	bool
PrintReferenceGC	Print times spent handling reference objects during GC (enabled only when PrintGCDetails)	false	bool
TraceScavenge	Trace scavenge	false	bool
TimeCompiler	time the compiler	false	bool
TimeCompiler2	detailed time the compiler (requires +TimeCompiler)	false	bool
LogMultipleMutexLocking	log locking and unlocking of mutexes (only if multiple locks are held)	false	bool
PrintSymbolTableSizeHistogram	print histogram of the symbol table	false	bool
ExitVMOnVerifyError	standard exit from VM if bytecode verify error (only in debug mode)	false	bool
AbortVMOnException	Call fatal if this exception is thrown. Example: java -XX:AbortVMOnException=java.lang.NullPointerException Foo	""	ccstr
PrintVtableStats	print vtables stats at end of run	false	bool
IgnoreLockingAssertions	disable locking assertions (for speed)	false	bool
VerifyLoopOptimizations	verify major loop optimizations	false	bool
CompileTheWorldIgnoreInitErrors	Compile all methods although class initializer failed	false	bool
TracePhaseCCP	Print progress during Conditional Constant Propagation	false	bool
TraceLivenessQuery	Trace queries of liveness analysis information	false	bool
CollectIndexSetStatistics	Collect information about IndexSets	false	bool
TraceCISCSpill	Trace allocators use of cisc spillable instructions	false	bool
TraceSpilling	Trace spilling	false	bool
CountVMLocks	counts VM internal lock attempts and contention	false	bool
CountRuntimeCalls	counts VM runtime calls	false	bool
CountJVMCalls	counts jvm method invocations	false	bool
CountRemovableExceptions	count exceptions that could be replaced by branches due to inlining	false	bool
ICMissHistogram	produce histogram of IC misses	false	bool
PrintClassStatistics	prints class statistics at end of run	false	bool
PrintMethodStatistics	prints method statistics at end of run	false	bool
TraceOnStackReplacement	Trace on stack replacement	false	bool
VerifyJNIEnvThread	Verify JNIEnv.thread == Thread::current() when entering VM from JNI	false	bool
TraceTypeProfile	Trace type profile	false	bool
MemProfilingInterval	Time between each invocation of the MemProfiler	500	intx
AssertRepeat	number of times to evaluate expression in assert (to estimate overhead); only works with -DUSE_REPEATED_ASSERTS	1	intx
SuppressErrorAt	List of assertions (file:line) to muzzle	""	ccstr
HandleAllocationLimit	Threshold for HandleMark allocation when +TraceHandleAllocation is used	1024	uintx
MaxElementPrintSize	maximum number of elements to print	256	intx
MaxSubklassPrintSize	maximum number of subklasses to print when printing klass	4	intx
ScavengeALotInterval	Interval between which scavenge will occur with +ScavengeALot	1	intx
FullGCALotInterval	Interval between which full gc will occur with +FullGCALot	1	intx
FullGCALotStart	For which invocation to start FullGCAlot	0	intx
FullGCALotDummies	Dummy object allocated with +FullGCALot, forcing all objects to move	32*K	intx
DeoptimizeALotInterval	Number of exits until DeoptimizeALot kicks in	5	intx
ZombieALotInterval	Number of exits until ZombieALot kicks in	5	intx
ExitOnFullCodeCache	Exit the VM if we fill the code cache.	false	bool
CompileTheWorldStartAt	First class to consider when using +CompileTheWorld	1	intx
CompileTheWorldStopAt	Last class to consider when using +CompileTheWorld	max_jint	intx
diagnostic
UnlockDiagnosticVMOptions	Enable processing of flags relating to field diagnostics	trueInDebug	bool
LogCompilation	Log compilation activity in detail to hotspot.log or LogFile	false	bool
UnsyncloadClass	Unstable: VM calls loadClass unsynchronized. Custom classloader must call VM synchronized for findClass & defineClass	false	bool
FLSVerifyAllHeapReferences	Verify that all refs across the FLS boundary are to valid objects	false	bool
FLSVerifyLists	Do lots of (expensive) FreeListSpace verification	false	bool
FLSVerifyIndexTable	Do lots of (expensive) FLS index table verification	false	bool
VerifyBeforeExit	Verify system before exiting	trueInDebug	bool
VerifyBeforeGC	Verify memory system before GC	false	bool
VerifyAfterGC	Verify memory system after GC	false	bool
VerifyDuringGC	Verify memory system during GC (between phases)	false	bool
VerifyRememberedSets	Verify GC remembered sets	false	bool
VerifyObjectStartArray	Verify GC object start array if verify before/after	true	bool
BindCMSThreadToCPU	Bind CMS Thread to CPU if possible	false	bool
CPUForCMSThread	When BindCMSThreadToCPU is true, the CPU to bind CMS thread to	0	uintx
TraceJVMTIObjectTagging	Trace JVMTI object tagging calls	false	bool
VerifyBeforeIteration	Verify memory system before JVMTI iteration	false	bool
DebugNonSafepoints	Generate extra debugging info for non-safepoints in nmethods	trueInDebug	bool
SerializeVMOutput	Use a mutex to serialize output to tty and hotspot.log	true	bool
DisplayVMOutput	Display all VM output on the tty, independently of LogVMOutput	true	bool
LogVMOutput	Save VM output to hotspot.log, or to LogFile	trueInDebug	bool
LogFile	If LogVMOutput is on, save VM output to this file [hotspot.log]	""	ccstr
MallocVerifyInterval	if non-zero, verify C heap after every N calls to malloc/realloc/free	0	intx
MallocVerifyStart	if non-zero, start verifying C heap after Nth call to malloc/realloc/free	0	intx
VerifyGCStartAt	GC invoke count where +VerifyBefore/AfterGC kicks in	0	uintx
VerifyGCLevel	Generation level at which to start +VerifyBefore/AfterGC	0	intx
UseNewCode	Testing Only: Use the new version while testing	false	bool
UseNewCode2	Testing Only: Use the new version while testing	false	bool
UseNewCode3	Testing Only: Use the new version while testing	false	bool
SharedOptimizeColdStart	At dump time, order shared objects to achieve better cold startup time.	true	bool
SharedSkipVerify	Skip assert() and verify() which page-in unwanted shared objects.	false	bool
PauseAtStartup	Causes the VM to pause at startup time and wait for the pause file to be removed (default: ./vm.paused.)	false	bool
PauseAtStartupFile	The file to create and for whose removal to await when pausing at startup. (default: ./vm.paused.)	""	ccstr

Glossary

TLAB

Thread-local allocation buffer. Used to allocate heap space quickly without synchronization. Compiled code has a "fast path" of a few instructions which tries to bump a high-water mark in the current thread's TLAB, successfully allocating an object if the bumped mark falls before a TLAB-specific limit address. Here is nice article with explanation about TLABs.

Integration of TeamCity 6.x and Fitnesse

2011-07-12T11:15:00.005+01:00

Team city is wonderful CI and build management platform, which combines great capabilities with really simple configuration. I seriously love it for the way it is designed, configured and managed and would recommend it to anybody who cares about quality and wants to have fast feedback on build/test/integration problems. FitNess is “The fully integrated standalone wiki, and acceptance testing framework”, which I, personally, do not like, but testers (at least ones I know) say it is good. I still do not believe them, but have to live with it :)

Integration problem is divided nicely into two pieces - one is fighting with FitNesse, another is integration of battle outcome into the TeamCity. So, lets start with FitNesse first.

FitNess is not easily-integratable framework for a few reasons. The first problem is test URL & Fixtures’ path usually are not configurable via parameters. Fortunately, there is quite good solution, which allows to pass variable to FitNesse using “-D” java argument. Another issue is that it runs just as HTTP server, there is no other way to execute tests, other from running the server. Luckily it can run it and after execution stop it and such behaviour can be managed via command line. And the last issue is report produced by Fitnesse. It has to be converted into something widely-acceptable, e.g. into JUnit report.

Now let sort all these issues out. The first step is updating test scripts to make them accept test URL and path to fixtures from command line. Assuming that there is just one property defined for URL and just one reference to fixture path, all which need to be done is Wiki page to contain something like this:

!define TEST_SERVER_URL {${testserver.host_port}}
!path ${testserver.fixture_jar}

Note another set of curly brackets around value of TEST_SERVER_URL variable, it shouldn’t be missed. After these values are defined, it is possible to pass them via command line:

$JAVA_HOME/bin/java -classpath <path_to_fixtures_jar> -Dtestserver.fixture_jar=<path_to_fixtures_jar> -DDtestserver.host_port=<test_server_url> fitnesseMain.FitNesseMain -p <fitnesse_port> -d <path_to_test_suite> -c <suite_url>\&format=xml > <report_temp_file>

where:
path_to_fixtures_jar - is the path to jar with fixtures. It is referenced twice, because it is assumed that it has both FitNesse distribution and fixtures inside it. For me, it just looks like the most convenient way to package it.
test_server_url - URL to the server being tested. That will be the value of TEST_SERVER_URL. Be careful here - as many other tools written in cowboy style dev technique, it hardly have any logs and when you will try to run test on bad (not connectable) URL or with invalid or unavailable port number, it will hang forever.
fitnesse_port - now this is weired, but is required by FitNesse. It requires port to run. Just choose something random, which is not likely to be used by something else. I have no idea why, but '0' port (which should just select first available port) didn’t work for me, FitNesse just hanged without any messages, as it loves to do.
path_to_test_suite - path to the folder with test suite.
suite_url - that’s the path of the suite without host and port. To identify it, run FitNesse, goto the page with Suite and hover mouse over “Suite” link on the left hand side of FitNesse page with the suite. Do not miss ‘format=xml’ suffix.
report_temp_file - is just a test file with report which will be generated by FitNesse.

Now when we have report, we need to convert it into the JUnit format to allow TeamCity to understand it. Fist, it has to be cleaned-up, because contains not just XML, but also some other irrelevant stuff. I was was lazy here and done it simply with following command:

grep ".*<.*>" $TEMP_FILE_TXT > $TEMP_FILE_XML"

then is more interesting part. That report has to be converted into JUnit XML and that is done by applying following XSLT:

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
  <xsl:element name="testsuite">
    <xsl:attribute name="tests">
      <xsl:value-of select="sum(testResults/finalCounts/*)" />
    </xsl:attribute>
    <xsl:attribute name="failures">
      <xsl:value-of select="testResults/finalCounts/wrong" />
    </xsl:attribute>
    <xsl:attribute name="disabled">
      <xsl:value-of select="testResults/finalCounts/ignores" />
    </xsl:attribute>
    <xsl:attribute name="errors">
      <xsl:value-of select="testResults/finalCounts/exceptions" />
    </xsl:attribute>
    <xsl:attribute name="name">AcceptanceTests</xsl:attribute>
  <xsl:for-each select="testResults/result">
    <xsl:element name="testcase">
      <xsl:attribute name="classname">
        <xsl:value-of select="/testResults/rootPath" />
      </xsl:attribute>
      <xsl:attribute name="name">
        <xsl:value-of select="relativePageName" />
      </xsl:attribute>
      <xsl:choose>
        <xsl:when test="counts/exceptions > 0">
          <xsl:element name="error">
            <xsl:attribute name="message">
              <xsl:value-of select="counts/exceptions" />
              <xsl:text> exceptions thrown</xsl:text>
              <xsl:if test="counts/wrong > 0">
                <xsl:text> and </xsl:text>
                <xsl:value-of select="counts/wrong" />
                <xsl:text> assertions failed</xsl:text>
              </xsl:if>
            </xsl:attribute>
          </xsl:element>
        </xsl:when>
        <xsl:when test="counts/wrong > 0">
          <xsl:element name="failure">
            <xsl:attribute name="message">
              <xsl:value-of select="counts/wrong" />
              <xsl:text> assertions failed</xsl:text>
            </xsl:attribute>
          </xsl:element>
        </xsl:when>
      </xsl:choose>
    </xsl:element>
  </xsl:for-each>
  </xsl:element>
</xsl:template>
</xsl:stylesheet>

to execute conversion we can use standard tool, provided by JDK. Surprisingly, found that not many people know about it, but it can be very handy. Here is snippet which allows to execute XSLT transformation in command line:

$JAVA_HOME/bin/java com.sun.org.apache.xalan.internal.xsltc.cmdline.Compile fitnesse2junit.xslt
$JAVA_HOME/bin/java com.sun.org.apache.xalan.internal.xsltc.cmdline.Transform <report_temp_file> fitnesse2junit > funct_test_res.xml

The result is going to be XML test report in JUnit format.

That’s all with FitNesse. Now it’s TeamCity’s turn. It is really up to developer how to do that, I will just give a brief overview and some flavour of what has to be done:

Create new build task for execution of the script which runs FitNesse. That script should be just a collection of command lines provided above.
FitNesse requires fixtures and tests, so these have to be retrieved from VCS or linked as artifact dependencies from the other build. Either way these files will available for script which runs FitNesse and can be added into classpath, etc.
I would recommend to put such properties like 'test_server_url' into 'Environment Variables' in 'Build Parameters'. Then they can be accessible by scripts.
To see test execution results, add a Feature which will analyse XML with JUnit report. That is easy and there is already build-in Feature type in TeamCity which supports JUnit. That functionality is available from “Build Step/Add build feature”.

EhCache replication: RMI vs JGroups.

2011-07-03T19:27:00.006+01:00

Recently, I was working on one product which required replicated caching. Caching provider was already decided - EhCache, and the remained was a question about transport. Which one is the best option? By the best option here I mean just the one which has better performance. The performance measurement was done just between two of available transports - JGroups and RMI, others were not considered, sorry.

Replication was tested between two nodes. The main goal was to understand how increase of message data size and total number of messages affects performance. Another goal was to find the point where replication performance getting really bad. Latter is not that easy, because test used limited amount of memory and non-leaner performance deterioration could be caused by exhaust of free heap space.

Below are memory size and software versions used to run the test:

All tests used 6GB of heap for all executions.
Tests were executed on the EhCache v.2.3.2
JVM is Sun java 1.6.0_21

The test itself is very simple. One node puts some number of elements with some size in the cache, other node reads all these elements. The test output is the the time required to read all elements. Timer starts just after first element is read.

The first test creates 10000 elements for each iteration. The variable is the message size, which increased twice on each iteration. On the first iteration size is 1280 bytes, on the last one - 327680 bytes (320 Kb). It means that final iteration with 10000 elements, where each size was 320 Kb will transfer approximate 3Gb of data. The tests have shown that EhCache copes very well with increasing size of the element and the slowdown was approximately proportional to the size of transferred data, which can be seen on the graph:

Here y-axis is time required for transfer in milliseconds and x-axis the the size of the element. No need to give much comments. RMI definitely looks better than JGroups.

In the seconds test, the variable was number of elements and the size of the element stayed constant and equal to 1280 bytes. As in previous test the number of messages was multiplied by two in each iteration and the amount of data transferred in final iteration was the same 3Gb. Graph below show how did it go:

As in previous graph, y-axis is the time require to transfer all elements in one iteration. X-axis is the number of elements. Again, it can be seen that RMI is the leader. I believe hat JGroups hit the heap at the latest iteration, that’s why it performed so bad. It means that JGroups has more memory overhead per element.

For the once, who do not trust (I woulnd’t ;) ) to my results and want to try it yourself, here are sources and configuration.

And, as conclusion... Well, RMI and JGroups both are acceptably fast. JGroups is definitely more memory consuming, which means one can hit a problem using it with big amounts of data. RMI, on the other hand uses TCP, instead of UDP, which, with big amount of nodes, may cause higher network load. Latter, unfortunately, is not covered by the test by any means and the real impact is not clear.

jakarta regexp vs java.util.regex

2011-01-15T15:08:00.005Z

Have never really been thinking about which library to use for regular expression, it always was java.util.regex by default. But I found that some people prefer to use Jakarta Regexp, and usually there is not clear reason for that choice. So, I decided to spent some time trying to find out what is better and probably write a couple of tests. After some time spent in the Internet looking for the examples of tests, it appeared that there was a good guy already who have done all that work. Here is the link to the page with results of his investigation:

http://tusker.org/regex/regex_benchmark.html

The conclusion is that Jakarta Regexp doesn't look good at all. Also you may notice that there are some other libraries who are doing regexps and some have very impressive performance and seems like worth trying.

All you need to know about QuickSort

2010-11-30T23:13:00.006Z

It would be true to say that Quicksort is one of the most popular sorting algorithms. You can find it implemented on the most of the languages and it is present in almost any core library. In Java and Go Quicksort is default sorting algorithm for some data types and it is used in in C++ STL (Introsoft which is used there begins with Quicksort). Such popularity can be explained by the fact that on average, Quicksort is one of the fastest known sorting algorithms. Interestingly that complexity of Quicksort is not less than it is for other algorithms like MergeSort or HeapSort. The best case performance is O(nlogn) and on the worst case it gives O(n^2). Latter, luckily, is exceptional case for the proper implementation. Quicksort performance is gained by the main loop which tends to make excellent usage of CPU caches. Another reason of popularity is that it doesn't need allocation of additional memory.

Personally for me Quicksort appeared as one of the most complex sorting algorithms. The basic idea is pretty simple and usually takes just a few minutes to implement. But that version, of course, if not practically usable. When it comes to details and to efficiency, it is getting more and more complicated.

Quicksort was first discovered by C.A.R. Hoare in 1962 (see "Quicksort," Computer Journal 5, 1, 1962) and in following years algorithm slightly mutated. The most known version is Three-way Quicksort. The most comprehensive of widely known ones is Dual-Pivot Quicksort. Both algorithms will be covered in that post.

The Java language was used to implement all algorithms. That post do not pretend to make adequate performance analysis. Test data used for performance comparison is incomplete and used just to show certain optimization techniques. Also, algorithm implementations are not necessary optimal. Just keep that in mind while you are reading.

Basics

The basic version of Quicksort is pretty simple and can be implemented just in few lines of code:


public static void basicQuickSort(long arr[], int beginIdx, int len) {
    if ( len <= 1 )
         return;
    
    final int endIdx = beginIdx+len-1;

    // Pivot selection
    final int pivotPos = beginIdx+len/2;
    final long pivot = arr[pivotPos];
    Utils.swap(arr, pivotPos, endIdx);

    // partitioning
    int p = beginIdx;
    for(int i = beginIdx; i != endIdx; ++i) {
         if ( arr[i] <= pivot ) {
             Utils.swap(arr, i, p++);
         }
     }
     Utils.swap(arr, p, endIdx);

     // recursive call
     basicQuickSort(arr, beginIdx, p-beginIdx);
     basicQuickSort(arr, p+1,  endIdx-p);
}

The code looks pretty simple and easily readable. Pivot selection is trivial and doesn't require any explanation. The partitioning process can be illustrated using following figure:

pointer "i" moves from the beginning to the end on array (note, that the last element of the array is skipped - we know that it the pivot). If i-th element is "<= pivot" then i-th and p-th elements are swapped and "p" pointer is moved to the next element. When partitioning is finished array will look like this:
Remember, that in the code, at the end of array there is element with pivot value, and that element is excluded from the pivoting loop. That element is put on p-th position, which makes p-th element included in "<= pivot" area. If you need more details, have a look at Wikipedia. There is pretty good explanation with lots of references. I would just emphasize your attention that algorithm consists of three main sections. These sections are pivot selection, partitioning and recursive call to sort partition. To make separation clearer the algorithm can be written down as:


public static void basicQuickSort(long arr[], int beginIdx, int len) {
    if ( len <= 1 )
        return;
 
    final int endIdx = beginIdx + len - 1;
    final int pivotIdx = getPivotIdx(arr, beginIdx, len);
    final long pivot = arr[pivotIdx];

    Utils.swap(arr, pivotIdx, endIdx);
    int p = partition(arr, beginIdx, len, pivot);
    Utils.swap(arr, p, endIdx);

    basicQuickSort(arr, beginIdx, p-beginIdx);
    basicQuickSort(arr, p+1,  endIdx-p); 
}  

public static int partition(long[] arr, int beginIdx, int len, long pivot) {
     final int endIdx = beginIdx + len - 1;
     int p = beginIdx;
     for(int i = beginIdx; i != endIdx; ++i) {
         if ( arr[i] <= pivot ) {
             Utils.swap(arr, i, p++);         
         }     
     }     
     return p;
}

public static int getPivotIdx(long arr[], int beginIdx, int len) {
     return beginIdx+len/2;
}

Now let's have a look how it performs vs Java 1.6 sort algorithm. For the test I will generate array using following loop:


static Random rnd = new Random();
private static long[] generateData() {
   long arr[] = new long[5000000];
   for(int i = 0; i != arr.length; ++i) {
       arr[i] = rnd.nextInt(arr.length);
   }
   return arr;
}

Then I run each JDK 6 Arrays.sort() and basicQuickSort() for 30 times and took the average run time as the result. New set of random data was generated for each run. The result if that exercise is this:

	arr[i]=rnd.nextInt(arr.length)
Java 6 Arrays.sort	1654ms
basicQuickSort	1431ms

Not that bad. Now look what would happen if input data has some more repeated elements. To generated that data, I just divided nextInt() argument by 100:

	arr[i]=rnd.nextInt(arr.length)	arr[i]=rnd.nextInt(arr.length/100)
Java 6 Arrays.sort	1654ms	935ms
basicQuickSort	1431ms	2570ms

Now that is very bad. Obviously that simple algorithm doesn't behave well in such cases. It can be assumed that the problem is in the quality of the pivot. The worst possible pivot is the biggest or the smallest element of the array. In that case, algorithm would has O(n^2) complexity. Ideally pivot should be chosen such as it splits an array into two parts with equal sizes. It means that ideal pivot is the median on all values of given array. Practically that is not good idea - too slow. Therefore, usually, implementation uses median of 3-5 elements. The decision on the number of elements used for pivot can be based on the size of partitioned array. The code for the pivot selection may look like this:


public static int getPivotIdx(long arr[], int beginIdx, int len) {
   if ( len <= 512 ) {
       int p1 = beginIdx;
       int p2 = beginIdx+(len>>>1);
       int p3 = beginIdx+len-1;

       if ( arr[p1] > arr[p2] ) { int tmp = p1; p1 = p2; p2 = tmp; }
       if ( arr[p2] > arr[p3] ) { p2 = p3; }
       if ( arr[p1] > arr[p2] ) { p2 = p1; }

       return p2;
   } else {
       int p1 = beginIdx+(len/4);
       int p2 = beginIdx+(len>>1);
       int p3 = beginIdx+(len-len/4);
       int p4 = beginIdx;
       int p5 = beginIdx+len-1;

       if ( arr[p1] > arr[p2] ) { int tmp = p1; p1 = p2; p2 = tmp; }
       if ( arr[p2] > arr[p3] ) { int tmp = p2; p2 = p3; p3 = tmp; }
       if ( arr[p1] > arr[p2] ) { int tmp = p1; p1 = p2; p2 = tmp; }
       if ( arr[p3] > arr[p4] ) { int tmp = p3; p3 = p4; p4 = tmp; }
       if ( arr[p2] > arr[p3] ) { int tmp = p2; p2 = p3; p3 = tmp; }
       if ( arr[p1] > arr[p2] ) { p2 = p1; }
       if ( arr[p4] > arr[p5] ) { p4 = p5; }
       if ( arr[p3] > arr[p4] ) { p3 = p4; }
       if ( arr[p2] > arr[p3] ) { p3 = p2; }
       return p3;
   }
}

Here are results after improvements in pivot selection strategy:

	arr[i]=rnd.nextInt(arr.length)	arr[i]=rnd.nextInt(arr.length/100)
Java 6 Arrays.sort	1654ms	935ms
basicQuickSort	1431ms	2570ms
basicQuickSort with 'better' pivot	1365ms	2482ms

Unfortunately, the improvement is almost nothing. It appeared that pivot selection is not the root cause of the problem. But still let's keep it, it doesn't harm, even helps a little bit. It also significantly reduce possibility of O(n^2) behaviour. Another suspect is the algorithm itself. It seems like it's not good enough. Obviously it doesn't perform well, when collection has repeated elements. Therefore something has to be changed.

Three-way partitioning

The way to get around that problem is three-way-partitioning. As a result of such partitioning, elements which are equal to the pivot are put in the middle of the array. Elements which are bigger than pivot are put in the right side of the array and ones which are smaller on the left side, appropriately.
Implementation of that partitioning method consists of two stages. In the first stage arrays is scanned by two pointers ("i" and "j") which are approaching in opposite directions. Elements which are equals to pivot are moved to the ends of array:

It can be seen that after the first stage elements which are equal to the pivot are located on the edges of the array. On the second stage these elements are moved to the middle. That is now their final position and they can be be excluded from the further sorting:

After implementation of such algorithm partitioning function is getting much more complicated. In that implementation the result of the partitioning is lengths of two bound partitions:


public static long partition(long[] arr, int beginIdx, int endIdx, long pivot) {
   int i = beginIdx-1;
   int l = i;
   int j = endIdx+1;
   int r = j;
   while ( true ) {
       while(arr[++i] <> pivot){}

       if ( i >= j )
           break;

       Utils.swap(arr, i, j);
       if ( arr[i] == pivot ) {
           Utils.swap(arr, i, ++l);
       }
       if ( arr[j] == pivot ) {
           Utils.swap(arr, j, --r);
       }
   }
   // if i == j then arr[i] == arr[j] == pivot
   if ( i == j ) {
       ++i;
       --j;
   }

   final int lLen = j-l;
   final int rLen = r-i;

   final int pLen = l-beginIdx;
   final int exchp = pLen > lLen ? lLen: pLen;
   int pidx = beginIdx;
   for(int s = 0; s <= exchp; ++s) {
         Utils.swap(arr, pidx++, j--);
   }
   final int qLen = endIdx-r;
   final int exchq = rLen > qLen ? qLen : rLen;
   int qidx = endIdx;
   for(int s = 0; s <= exchq; ++s) {
         Utils.swap(arr, qidx--, i++);
   }

   return (((long)lLen)<<32)|rlen;
}

The pivot selection has to be changed as well, but more just for convenience, the idea remains absolutely the same. Now it returns actual value of pivot, instead of index:


public static long getPivot(long arr[], int beginIdx, int len) {
   if ( len <= 512 ) {
       long p1 = arr[beginIdx];
       long p2 = arr[beginIdx+(len>>1)];
       long p3 = arr[beginIdx+len-1];

       return getMedian(p1, p2, p3);
   } else {
       long p1 = arr[beginIdx+(len/4)];
       long p2 = arr[beginIdx+(len>>1)];
       long p3 = arr[beginIdx+(len-len/4)];
       long p4 = arr[beginIdx];
       long p5 = arr[beginIdx+len-1];

       return getMedian(p1, p2, p3, p4, p5);
   }
}

And here is the main method, which is slightly changed as well:


public static void threeWayQuickSort(long[] arr, int beginIdx, int len) {
    if ( len < 2 )
        return;

    final int endIdx = beginIdx+len-1;
    final long pivot = getPivot(arr, beginIdx, len);
    final long lengths = threeWayPartitioning(arr, beginIdx, endIdx, pivot);

    final int lLen = (int)(lengths>>32);
    final int rLen = (int)lengths;

    threeWayQuickSort(arr, beginIdx, lLen);
    threeWayQuickSort(arr, endIdx-rLen+1, rLen);
}

now let's compare it with Java 6 sort:

	arr[i]=rnd.nextInt(arr.length)	arr[i]=rnd.nextInt(arr.length/100)
Java 6 Arrays.sort	1654ms	935ms
basicQuickSort	1431ms	2570ms
basicQuickSort with 'better' pivot	1365ms	2482ms
Three-way partitioning Quicksort	1330ms	829ms

Huh, impressive! It is faster than standard library, which, by he way, implements the same algorithm. To be honest I was surprised, when found that it is such an easy task to beat standard library.
But what about making it even faster? There is one trick which always helps and it works for all sorting algorithms which work with consecutive memory. That trick is Insertion sort. Although is has big chance of O(n^2), it appears to be very very effective on the small arrays and always gives some performance improvements. Especially that is noticeable when input data is not sorted and there are not many repeated elements. All you need to do is just add it at the beginning of sorting method:


public static void threeWayQuickSort(long[] arr, int beginIdx, int len) {
    if ( len < 2 )
        return;

    if ( len < 17 ) {
        InsertionSort.sort(arr, beginIdx, len);
        return;
    }

    final int endIdx = beginIdx+len-1;
    final long pivot = getPivot(arr, beginIdx, len);
    final long lengths = threeWayPartitioning(arr, beginIdx, endIdx, pivot);

    final int lLen = (int)(lengths>>32);
    final int rLen = (int)lengths;

   threeWayQuickSort(arr, beginIdx, lLen);
   threeWayQuickSort(arr, endIdx-rLen+1, rLen);
}

and run the test again:

	arr[i]=rnd.nextInt(arr.length)	arr[i]=rnd.nextInt(arr.length/100)
Java 6 Arrays.sort	1654ms	935ms
basicQuickSort	1431ms	2570ms
basicQuickSort with 'better' pivot	1365ms	2482ms
Three-way partitioning Quicksort	1330ms	829ms
Three-way partitioning Quicksort with Insertion sort	1155ms	818ms

now standard library looks just awful. It looks now that all is said and done. But it in reality that's not the end of the story and there is something else to talk about.

Dual-pivot Quicksort

Moving forward, I found that Java 7 is much more advanced and performs much faster than Java 6 version and outperforms all previous tests:

	arr[i]=rnd.nextInt(arr.length)	arr[i]=rnd.nextInt(arr.length/100)
Java 6 Arrays.sort	1654ms	935ms
Java 7 Arrays.sort	951ms	764ms
basicQuickSort	1431ms	2570ms
basicQuickSort with 'better' pivot	1365ms	2482ms
Three-way partitioning Quicksort	1330ms	829ms
Three-way partitioning Quicksort with Insertion sort	1155ms	818ms

After several seconds of very exciting research study it was found that Java 7 uses new version of Quicksort algorithm which was discovered just in 2009 by Vladimir Yaroslavskiy and named Dual-Pivot QuickSort. Interestingly that after some search in internet, I have found algorithm called "Multiple pivot sorting" which was published in 2007. It seems like generic case of "Dual-Pivot QuickSort" where is possible to have any number of pivots.
As you may notice from the name, the main difference of that algorithm is that it is using two pivots, instead of one. Coding now is getting even more complicated. The simplest version of that algorithm may look like this:


public static void dualPivotQuicksort(long arr[], int beginIdx, int len) {
    if ( len < 2 )
        return;

    final int endIdx = beginIdx+len-1;

    long pivot1 = arr[beginIdx];
    long pivot2 = arr[endIdx];

    if ( pivot1 == pivot2 ) {
        final long lengths = QuickSort.threeWayPartitioning(arr, beginIdx, endIdx, pivot1);
        final int lLen = (int)(lengths>>32);
        final int rLen = (int)lengths;

        dualPivotQuicksort3(arr, beginIdx, lLen);
        dualPivotQuicksort3(arr, endIdx-rLen+1, rLen);
    } else {
        if ( pivot1 > pivot2 ) {
            long tmp = pivot1;
            pivot1 = pivot2;
            pivot2 = tmp;
            Utils.swap(arr, beginIdx, endIdx);
        }

        int l = beginIdx;
        int r = endIdx;
        int p = beginIdx;

        while ( p <= r ) {
            if ( arr[p] < pivot1 ) {
                Utils.swap(arr, l++, p++);
            } else if ( arr[p] > pivot2 ) {
                while ( arr[r] > pivot2 && r > p ) {
                    --r;
                }
                Utils.swap(arr, r--, p);
            } else {
                ++p;
            }
        }
        if ( arr[l] == pivot1 ) ++l;
        if ( arr[r] == pivot2 ) --r;

        dualPivotQuicksort3(arr, beginIdx, l-beginIdx);
        dualPivotQuicksort3(arr, l, r-l+1);
        dualPivotQuicksort3(arr, r+1, endIdx-r);
    }
}

First code picks up two pivots. If pivots are the same, it means we have just one pivot and in that case we can used three-way method for partitioning. If pivots are different, then partitioning process will look like this:

Scanning pointer "p" is moving from the beginning of array. If current element is "<> pivot1", then r-th element is swapped with p-th and "r" pointer is moved to next element backwards. All stops when "p" becomes less than "r". After partitioning, array will look like this:

When partition is finished, algorithms is called recursively for each partition.

Reader shouldn't expect good performance from the provided code, it is not fast and performs even worse than Java 6 Arrays.sort. I was provided just to illustrate the concept.

To be honest I failed to make my implementation to perform any better than version from Java 7. I must admit, that Yaroslav made a very good job there. Therefore I do not think that there is any sense in discussing my implementation here in details.

But, if someone wants to challenge Java 7 version I can point to some direction for optimizations. Firstly, which is seems obvious? is pivot selection. Another easy improvement is Insertions sort at the beginning. Also, I have noticed that that algorithm is very sensitive to inlining, so there is sense to inline Utils.swap(). As other option, you can decided to go thought the middle partition and move elements equals to pivot1 or pivot2 to their final positions which will exclide them from the further sorting. I found that it is effective for relatively (<=512 elements) small arrays. You can also have a look at source from the Java 7 and try to implement some tricks from there. Be ready to spend a lot of time :) All in all, it can be seen that over the years sorting is getting better and better. And that statement doesn't only relate to Quicksort. Other sorting algorithms are improving as well. As examples can be considered Introsoft or Timsort. However, it would be true to say that nothing really new was discovered in that area since 1960s-1980s. Hopefully we will be lucky enough to see something completely new and radical in the future.

For ones who want to dig deeper, as the starting point, I would suggest to visit following links:

Quicksort Wikipedia article

Dual-Pivot QuickSort

Quicksort Is Optimal presentation by Robert Sedgewick & Jon Bentley

MIT lecture about quicksort

JCaptcha, SecureRandom & performance

2010-10-11T23:08:00.057+01:00

Personally, I'm not big fan of JCaptcha library and recently was lucky enough to find another problem there. The problem is related mostly to linux and it's implementation of java.security. SecureRandom, which is known to be slow and is locking on every call to it.

For some reason (I suspect that this was the reason) JCaptacha overuse java.security.SecureRandom when it generates background image using FunkyBackgroundGenerator. Number of calls to get next random float can easily reach something around 100 000 per one captcha image. It is basically called at least once for each pixel.

Lets run some quick tests to have a look how bad is that. I have wrote write simple captch engine:


public static class MyImageCaptchaEngine extends ListImageCaptchaEngine {
  protected void buildInitialFactories() {
      WordGenerator wgen = new RandomWordGenerator("ABCDEFGHIJKLMNOPQRSTUVWXYZ123456789");
      RandomRangeColorGenerator cgen = new RandomRangeColorGenerator(
         new int[] {0, 100},
         new int[] {0, 100},
         new int[] {0, 100});
      TextPaster textPaster = new RandomTextPaster(new Integer(7), new Integer(7), cgen, true);

      BackgroundGenerator backgroundGenerator = new FunkyBackgroundGenerator(new Integer(200), new Integer(100));

      Font[] fontsList = new Font[] {
         new Font("Arial", 0, 10),
         new Font("Tahoma", 0, 10),
         new Font("Verdana", 0, 10),
      };

      FontGenerator fontGenerator = new RandomFontGenerator(new Integer(20), new Integer(35), fontsList);

      WordToImage wordToImage = new ComposedWordToImage(fontGenerator, backgroundGenerator, textPaster);
         this.addFactory(new GimpyFactory(wgen, wordToImage));
  }
}

And the test itself:


long begin = System.currentTimeMillis();
for(int i = 0; i != 100; ++i) {
  engine.getNextCaptcha();
}
long end = System.currentTimeMillis();
System.out.println("Total time is [" + (end - begin) + "]");

now lets run it:

Total time is [10967]

Ok, so what we have now it 10967ms, which, I believe, is bad. It can be significantly improved. I'm not very big fan of high-quality random backgrounds, so I will replace SecureRandom class, used by parent of FunkyBackgroundGenerator with "pseudo" random Random class. Funs of high-quality random backgrounds can still use SecureRandom for seeding, through:


public static class MyFunkyBackgroundGenerator extends FunkyBackgroundGenerator {
  public MyFunkyBackgroundGenerator(Integer width, Integer height) {
     super(width, height);
     try {
         Field rndField = AbstractBackgroundGenerator.class.getDeclaredField("myRandom");
         rndField.setAccessible(true);
         rndField.set(this, new Random());
     }
     catch (Exception e) {
         e.printStackTrace();
     }
  }
}

I know, that is dirty hack, but as far "myRandom" is declared with default visibility, that's the shortest way to replace it for now . And what we have now is:

Total time is [1308]

Approximately 7 time quicker. Not that bad, specially for a sample case. In real world application improvement will be even more significant because some other processes may use '/dev/(u)random' or application itself can utilize SecureRandom for other purposes.

That's not it. There is another bottleneck, which is usage of Java2D for rendering captcha images. Java2D is well known for it's problems with multi-threading and the summary of these problems is that Java2D doesn't scale well. Some details can be found here. Possibly the way to fix that problem may be removing Java2D and using direct access to image via WritableRaster instead. However, it doesn't solve all problems, as far as Java2D is still used for drawing text.

ConcurrentHashMap revealed

2010-08-17T12:31:00.094+01:00

Java 1.5 introduced some new cool concurrency stuff which is located in java.util.concurrent package. There are lots of good things there including new ConcurrentHashMap class which I'm going to talk about. That class is targeted to be used in concurrent environment and provides significant performance benefits over synchronized version of HashMap. Javadoc doesn't really provide lots of details on how that class works and why can be better than synchronized version of HashMap. I think that understanding of these details is crucial for using that class in a right way, so I've made some research to undercover implementation details and mechanisms used to implement that class.

Map Regions

Firstly I will repeat what is already said in JavaDoc - that map implementation consists of regions. Each region is hash table - an array where each element is associated with some range of hash codes and contains linked list of entries. It means that structurally region is very similar to normal HashMap. All write operations have to acquire write lock on the whole region. All read operation on the region are performed without locking, apart from one small exception. That exception happens on attempt to read value of entry and that value is "null". In that case code suspects that compiler and could possibly assign entry to array element before calling entry's constructor (good example and explanation of that problem can be found here). In reality it would mean that the compiler has a bug, because such optimization contradicts Java Memory Model agreement. When that happens code tries to acquire write lock and perform reading inside that lock. Such strategy guarantees that initialization of the object is completed before the reference is assigned to the array element. As far as by contract ConcurrentHashMap doesn't allow neither key or value to be "null" and a possibility of compiler bug is extremely low, chance of "blocking read" is negligible.
Another interesting thing about region is that it has volatile counter which counts number of modifications of that region. That value is used in many methods of ConcurrentHashMap to identify that region’s state is not changed during execution of current method. Basically it is used as a means of solving ABA problem. It doesn't seem right to me that such strategy is applied even on read methods, the most of them are much slower than they can be without that check. There is also some locking (e.g. in size() operation) of individual segments which looks unnecessary as far as result is not going to be accurate anyway. But, on the other hand, that class was created by Doug Lea who is very respectable for his concurrency research, so I suspect I'm missing something...

Iteration through ConcurrentHashMap

Some other stuff, which may raise questions, is relationship between collection and enumeration objects returned by “keySet()”, “values()”, “entrySet()”, “keys()” and “elements()” and ConcurrentHashMap instance which have created them. Implementation of all these objects is based on internal class HashIterator and the noticeable thing here is that instance of that class has some information about the state of ConcurrentHashmap at the moment of it’s (HashIterator’s instance) creation. As an example, there is direct reference to the segment’s data array and on the moment when the HashIterator instance is actually used, given segment could have already decided to create a new array, because the old one appeared to be too small. It means that data returned by these collections and enumerations may not reflect changes in the instance of ConcurrentHashmap which happened after given collection or enumeration was created.
Performance considerations for these objects are exactly the same as for ConcurrentHashMap’s methods - read methods are non-blocking and for update methods lock is acquired on segment level. Interesting, but not really practically useful fact, is that HashIterator isn’t that smart as ConcurrentHashMap itself and doesn’t lock when entry’s value is null. So in theory, it is possible that iterator created by “elements()” method will return null value from “next()” or “nextElement()”.

Building the map

To build a map properly you have to understand what constructor’s arguments mean. Here is a brief explanation in the light of things which are written above:

Default initial capacity. That parameter is used to calculate initial capacity of segments. The capacity of each segment is nearest power of two bigger than provided initial capacity divided by number of regions (see Default concurrency level). E.g. initial capacity is 1000 and concurrency level is 3, then number of regions is 4 and capacity of each region is 256 (which is nearest power of two bigger than 1000/4). Default value of that argument is 16.
Default load factor. Is used to identify the moment when the region has to be resized (see section about memory consumption below). No magic, used as is. Default value of that argument is 0.75.
Default concurrency level. Defines number of regions which will be created. Exact number of regions is nearest power of two bigger than provided concurrency level. Concurrency level is always less than 2^16. E.g. if concurrency level is 18, number of regions will be 32. Default value of that argument is 16. Keep in mind, that more regions (better concurrency), means higher memory consumption.

And, probably, the last bit - memory consumption. In these terms ConcurrentHashMap is approximately the same as HashMap, but multiplied by number of regions. Region grows at the moment when number of elements appeared to be equal to loadFactor*currentSizeOfRegion value. New region size is always twice as big as previous one, maximum size of region is 2^30.

Migration to ConcurrentHashMap

Other possible question which can be raised is the migration from synchronized version of HashMap to CuncurrentHashMap. There are several things which has to be evaluated during the migration process:

Some methods can use synchronized map object for synchronization purposes. If someone accrue the lock on the object, all it’s methods are locked as well. That’s be biggest fun. There is no simple way to archive such behaviour with CuncurrentHashMap.
CuncurrentHashMap’s size method is slow. In the worst case it spins couple of time and then get the lock on all regions. This is much-much-much slower than HashMap implementation, which returns back just the value of the “size” field.
Iterator of CuncurrentHashMap does not throw ConcurrentModificationException. Do not see how this can cause problem, but just keep it is mind.
Containing the same amount of elements ConcurrentHashMap object requires more memory than HashMap object (see “Building the map” section).

All in all, ConcurrentHashMap is brilliant container. In multi-threaded environments it will have much bigger throughput than synchronized version of HashMap on read and write operations. Such improvement is result of absence of locking on read and using of region-based locking on write operations. There is sill small chance that segment will be locked for a very short time on read, but chance is so small that it can be just ignored. Write operations are locking just one region, other regions can be updated at the same time.

The price for all these improvements are increase in memory usage and some degradation of performance on read operations. More memory is used because each region wastes approximately the same amount of memory as one HashMap. Performance drop on accessing elements is because of ABA checks. Write operations do not seem to introduce any performance degradation, apart from use of locking.

Some hints for writing secure code

2010-05-25T15:40:00.063+01:00

Security and data protection are becoming now more and more popular topics. We are coming into the world where too much information is transfered/used/processed by computer systems and any leak of that information can cause a big trouble. Thus, it is very important for application to protect customer information as much as it can and do not allow it to spread out.

There are many aspects of application security and these cover processes, architecture, infrastructure, code, etc. The whole topic is extremely big and versatile and there are some books written to cover all its possible verges. I will touch just a small piece which is related to something which is in area of developer's responsibility - code and application architecture. Also, I assume that that reader mostly works on web applications implemented on Java or similar platform.

I disagree that creation of secure code is hard work; I would say it's just a question of some knowledge and discipline. Here are several guidelines which developer has to follow to cover the most of application security vulnerabilities. Of course list is not complete and mostly covers just protection of customer's data.

Always hash user passwords

The worst thing you can do is do not hash user passwords and store then as clear text. Less bad thing is to encode them, but it’s still naughty. You do not need to know your customers’ password, let you know, better you sleep.

I can't imagine scenario where application has to have an access to user passwords, but easily can imagine what would happen if someone will get an access to such treasure. Thus, user's password must be hashed, preferably using salted hash and good hash function like SHA-2.

And be always suspicious about websites, which are able to send you your password by mail.

Always encrypt sensitive data

If application operates with sensitive data, e.g. password for connecting to legacy backend systems or credit card numbers, that password mustn't be in clear text and must be encrypted. The most of the data has to be encrypted just in storage, but some also on runtime. There is very small chance that someone will get an access to your memory dump, that chance is really-really small, but for very critical data it shouldn't be ignored.

Of course, encryption has to be done with proper cipher, AES should be fine. The only problem is that to encrypt something you need to have a secret, which has to be unencrypted and if attacker will get it, the whole system will be compromised. So that secret has to be protected properly, e.g. with help of HSM, or at least it has to be protected on OS level with proper permissions.

Logging

Logging mustn’t have user-related or any other type of sensitive information. Examples can be credit card details, bank details, personal messages, etc. Be very careful with “toString” implementation of classes which contain sensitive information. Always keep in mind that logs can fall into somebody's dirty hands.

Always use strong ciphers and hashes

There is no much sense to use hash or cipher if it can be easily compromised. Thus, application has to use the best available set. Another thing developer has to remember is the fact, that even the best things will become trash in the face of tomorrow. It means that must be a way to change algorithm in future. For instance, hashed password may have prefix with algorithm name at the beginning, so as soon new algorithm is available it can be used. Here is an example of such hash:
{SHA-1}:pOtmGeFyIsThHT6LfNE846FJAWxjmLiR
Of course old passwords will remain the same, but it’s better than nothing. The similar principle applies to encrypted values with only difference that encrypted values can be recovered and re-encrypted.
Acceptable choice of algorithms at this moment (05.2010) is SHA-2 for hashing and AES for symmetric cryptography. There are not many choices of asymmetric algorithms, RSA is the best known and wide used example. Apart from algorithm have a thought about hash/key size - bigger is better for security.

Use Prepared Statement
That's just a must. Always use PreparedStatement or equivalent and never build SQL statement by concatenating arguments which with high probability will be cause of SQL injection problem. Let database driver think about that problem.

Hide implementation details from the user
It doesn't matter who are your users, it can be another system or real person, you have to tell them as less as you can about your system. That knowledge can help attacker to recognize products you use or give some information how you system is build. All it can give some idea how application can be hacked. For example, printing exception stack trace into browser window, which is happening on some websites, provides almost all information about libraries, used products, architecture, etc.

Validate user input
Anything which comes externally must be validated. Any value has to have boundaries. The most convenient way to archive that is using regular expressions. This protects against XSS and related problems.

Shield output
Do not trust to anything and your own system is not exception. Before output any data to customer, it has to be shielded to remove any invalid characters. And again it can be base for XSS and related attacks.

Re-create session after authentication
It is possible that session id was compromised or used in session fixation attack. Thus, using the same session id after authentication will open the system to attacker, so it has to be re-generated.

Use authorization
You should explicitly authorize uses on every layer; otherwise there is too much space for the holes in application. That is specifically important for web applications where sometimes all authorization is based on just URL principle, which is dangerous practice.

Use auditing
Would be funny if someone will attack your system and you will not have any tracks of it. Therefore, developer has to have some sort of log for recoding activities performed by the system. That logs shouldn’t be confused with logs used for debugging and troubleshooting, latter contain debug information which help to developer to look for and fix bugs (although they can contain relevant information as well and can be used for tracking attacker actions as well). Auditing logs, on other hand, contain events raised on performing certain activities, e.g. login attempts, payments, etc.

In conclusion, I would remind that it’s not complete list, there are much more things which you can do to protect your application. For ones who are interested in making their application more secure, I would recommend following links:

PCI DSS. This is more about payments and protecting card details, but do not forget that card details are just another piece of information.
Wikipedia application security page. Has lots of good links for further reading.
OWASP website. Lots of information about vulnerabilities and how to avoid them and about web application security in general.

Killing the private (and protected)

2010-04-05T20:17:00.044+01:00

Recently on one forum I've found topic which released from memory my C++ days. That was topic about using "private" modifier on methods. The question there was about cases where to use it and why. Everybody knows when you are programming on C++ (as the most glaring example) one of "good practices" is "less client can do - better". Of course that's not in sense of functionality provided to client, but in mind of something which is not explicitly provided. In C++, I believe, the main reason for that is fragility of runtime, which can be easily killed if someone accidentally will do something wrong, which can also cause physical damage to person who is responsible for doing that (worth mentioning, in Java situation is slightly different, it's hard (but still possible, of course) to kill the application by "accidental coding"). And a consequence of such “good practice” is the basic rule – “make private as much as you can”.

That rule makes a good job in protecting object’s invariant and supporting encapsulation, but it has some disadvantages as well. First, it indicates that author most probably haven’t ever unit-tested that method and, second, which is more important for the topic, client of such code loses freedom of modification of library if he needs to (btw, some other techniques, like final classes, static members, “default” access modifier are doing the same thing). In the most it affects developers who use languages similar to Java. The way how Java developer works assumes that he spends lots of time in libraries’ code – understanding how they work, looking possible ways of customization, etc. The most of IDE now days are brilliant and I feel no difference in browsing my own code or library code. Moreover, if I don’t have source code, IDE will kindly decompile class for me. As a consequence of such close interaction with the library code, the situation when you want to fix/change/hack something in library is not very rare. For me that happens at least several times per year. And the one of the worst things I can see in that case is method I want to “modify” is private and I can’t change its behaviour by inheritance or implementation of other strategy. In that situation author of library had decided what is safe for me and what is not. I appreciate that, but would prefer to make decision myself.

Some can say that with loss of “private” modifier we are loosing benefits of encapsulation. I wouldn't be so sure about it. The same effect can be easily archived using other methods, like combination if pure Interfaces and Factories. And it works even better – client would never know instance of which class he is using, all he knows is just an interface.

The real problem of removing “private” is that we need to use something instead. The closest candidate is “protected” (I will not cover “default”, etc modifiers for simplicity reasons). Problem of “protected” is that such method is considered as part of interface. Commonly it is used as technique for implementing Template Method pattern. But the same effect can be achieved using Strategy, which provides less coupling, is more flexible in implementation, etc (see here) and as a result you will have class just with “public” methods.

All in all, I suppose, that ideal approach is to build systems based on interfaces and appropriate patterns, which will exclude the most of need for “private” and “protected” method modifiers. In that case, actually, I even do not worry about them J But ideal case usually never happens. As a general approach for other cases, I would suggest to avoid private as much as you can and use appropriate patterns instead. Also, it’s good to avoid protected as well, but if you can’t then state clearly, that these methods are not part of the interface and if client is going to override them, he is doing it on his own risk.

Public key infrastructure

2010-03-31T22:37:00.002+01:00

Some time ago I was asked to create presentation for my colleagues which describes Public Key Infrastructure, its components, functions, how it generally works, etc. To create that presentation, I've collected some material on that topic and it would be just dissipation to throw it out. That presentation wasn’t technical at all, and that post is not going to be technical as well. It will give just a concept, high-level picture, which, I believe, can be a good base knowledge before start looking at details.

I will start with cryptography itself. Why do we need it? There are at least three reasons for that - Confidentiality, Authentication and Integrity. Confidentiality is the most obvious one. It's crystal clear that we need cryptography to hide information from others. Authentication confirms that message is send by subject which we can identify and our claims about it are true. And finally, Integrity ensures that message wasn't modified or corrupted during transfer process.

We may try to use Symmetric Cryptography to help us to achieve our aims. It uses just one shared key, which is also called secret. The secret is used for encryption and for decryption of data. Let’s have a look how it can help us to archive our aims. Does it encrypt messages? Yes. Well, Confidentiality is solved, as soon as nobody else, except communicating parties, knows the secret. Does it provide Authentication? Mmm… I would say, no. If there are just two parties in conversation, is seems ok, but if there are hundreds, then should be hundreds secrets, which is hard to manage and distribute. What about Integrity? Yes, it works fine – it’s very hard to modify encrypted message. As you can guess, symmetric cryptography has one big problem – and that problem is “shared secret”. These two words… they don't even fit one to other. If something is known by more that one person, it is not a secret any more. Moreover, to be shared, that secret somehow has to be transferred and during that process there are too many way for secret to be stolen. This means that such type of cryptography hardly solves our problems. But it is still in use and works quite well for its purposes. It's very fast and can be used for encryption/decryption of big amounts of data, e.g. you hard drive. Also, as far as it hundreds or even thousands times faster that asymmetric cryptography, it’s used in hybrid schemas (like TLS aka SSL), where asymmetric cryptography is used for just for transferring symmetric key and encryption/decryption is done by symmetric algorithm.

Let’s have a look at Asymmetric Cryptography. It was invented very recently about 40 years ago. The first paper (“New Directions in Cryptography”) was published in 1976 by Whitfield Diffie and Martin Hellman. Their work was influenced by Ralph Merkle, who believed to be the one who created the idea of Public Key Cryptography in 1974 (http://www.merkle.com/1974/) and suggested it as project to his mentor - Lance Hoffman, who rejected it. “New Directions in Cryptography” describes algorithm of key exchange known as “Diffie–Hellman key exchange”. Interesting fact that the same key exchange algorithm was invented earlier, in 1974 in Government Communication Headquarters, UK by Malcolm J. Williamson, but that information was classified and fact was disclosed just in 1997.

Asymmetric Cryptography uses pair of keys - one Private Key and one Public Key. Private Key has to be kept secret and not shared with anybody. Public Key can be available to public; it doesn’t need to be secret. Information encrypted with public key can be decrypted only with corresponding private key. As far as Private Key is not shared, there is no need to distribute it, and there is reasonably small chance that it will be compromised. So such way of exchanging information can solve Confidentiality problem. What about Authentication and Integrity? These problems are solvable as well and utilise mechanism called Digital Signature. The simplest variant if Digital Signature can use following scenario – subject creates a hash based on message, encrypt that hash with Private Key and attach it to message. Now if recipient wants to verify the subject who created a message, he will encrypt that hash using subject’s public key (that’s Authentication) and compare it with hash generated on recipient side (Integrity). In reality hash is not exactly encrypted, instead it used in special signing algorithm, but the overall concept is the same. It’s important to notice that in Asymmetric Cryptography each pair of keys serves just one purpose, e.g. if pair is used for signing, it can’t be used for encryption.

Digital Signature, also, is the base for Digital Certificate AKA Public Key Certificate. Certificate is pretty much the same as your passport. It has identity information, which is similar to name, date of birth, etc. in passport. Owner of certificate has to have Private Key which matches Public Key stored in certificate, similar passport has photo of the owner, which matches owner’s face. And, finally, certificate has a signature, and its meaning is the same, as meaning of stamp in passport. Signature proves that certificate was issued by organization which made that signature. In Public Key Infrastructure world such organizations are called Certificate Authorities. If one system discovers that Certificate is signed by “trusted” Certificate Authority, it means that system will trust to information in certificate.

Last paragraph may not be obvious, especially “trust” part of it. What does “trust” mean in that context? Let have a look at simple example. Every site on Web which makes a use of encrypted connection does it via TLS (SSL) protocol, which is based on Certificates. When you go to https://www.amazon.co.uk and it sends its certificate back to your browser. In that certificate there is information about website and reference to Certificate Authority who signed that certificate. First browser will look at the name in certificate – it has to be exactly the same as website domain name, in our case, that’s “www.amazon.co.uk”. Then browser will verify that certificate is signed by Trusted Certificate Authority, which is VeriSing in case of Amazon. You browser already has a list of Certificate Authorities (this is just a list of certificates with public keys) which are known as trusted ones, so it can verify that certificate is issued by one of them. There are some other verification steps, that these two are the most important ones. Assume in our case verification was successful (if it’s not browser will show is big red warning message, like that one) – certificate has proper name in it and was signed by Trusted Certificate Authority. What does it give to us? Just one thing – we know that we are on www.amazon.co.uk and the server behind that name is Amazon server, not some dodgy website, which just looks like Amazon. When we enter our credit card details and we can be relatively sure that they will be sent to Amazon, but not to hacker’s database. Our hope here based on assumption that such Certificate Authorities like VeriSign do not give dodgy certificates and Amazon server is not compromised. Well, better than nothing J

Another example are severs in organization, which use certificates to verify that they can trust one to other. The schema there is very similar to browser’s ones, except two differences:

Mutual authentication. Certificates are, usually, verified but both sides, not just by client. Client has to send his certificate to server.
Certificate Authority, is hosted inside the company.

When CA is inside the company we can be almost sure that certificates are going to be issued only to properly validated subjects. It gives some confidence that hacker can’t inject his server, even if he has access to network infrastructure. Attack is possible only if CA is compromised or some server’s Private Key is compromised.

We already know, Certificate Authority is the organization which issues certificate and in the Internet, an example of such organization is VeriSing. If certificate is created to be used just inside organization (intranet), it can be issued by Information Security Department which can act as Certificate Authority. When someone wants to have certificate, he has to send certificate request which is called Certificate Signing Request to Certificate Authority. That certificate consists of subject’s identity information, subject’s public key and signature, created by subject’s private key to ensure, that subject who sent request has appropriate private key. Before signing Certificate Authority passes that request to Registration Authority who verifies all details, ensures that proper process is followed, etc. It’s possible that Certificate Authority can also act as Registration Authority. After all, if everything is ok, Certificate Authority creates new certificate signed by its private key and send it back to subject which requested certificate.

I've already mentioned Certificate validation process. Here are some details of it; worth mentioning theirs details are still high-level. Validation consists of several steps which, broadly speaking can be described as:

Certificate data validation – validity date, presence of required fields, their values, etc.
Verify that certificate is issued by Trusted Certificate Authority. If you are browsing internet that list if already built-in in your browser. If that’s communication between two systems, each system has a list of trusted Certificate Authorities; usually that is just a file with certificates.
Certificate’s signature is valid and made by Certificate Authority who signed that certificate.
Verify that certificate is not revoked.
Key verification – proves that servers can decode messaged encrypted by certificate’s Public Key.

Mentioned above certificate revocation can happen because of many reasons – certificate could be compromised, or, in corporate world, employee, which owned certificate, left company, or sever which had certificate was decommissioned, etc. On order to verify certificate revocation, browser or any other piece of software, has to use one or both of following techniques:

Certificate Revocation List (CRL). That’s just a file, which can be hosted on http server. It contains list of revoked certificate IDs. That’s method is simple and straightforward, it doesn’t require lots of efforts for implementation, but has three disadvantages – that’s just a file, which means, that it’s not real-time, it can use significant network traffic and it’s not checked by default by the most of the browsers (I would even say by all browsers), even if certificate has a link to CRL.
Online Certificate Status Protocol (OCSP). That is preferable solution, which utilizes dedicated server, which implements protocol which will return back revocation status of certificate by its id. If browser (at least FireFox > v.3.0) will find link to that server in certificate, it will make a call to verify that certificate is not revoked. Only disadvantage is that OCSP server has to be very reliable and be able to answer on requests all the time.

In internet certificate usually contains links to CRL or OCSP inside it. When certificates are used in corporate network these links are usually known by all parties and there is no need to have them in certificate.

So, finally, what is Public Key Infrastructure? That’s infrastructure, which supports everything which was described above and generally consists of following elements:

Subscribers. Users of certificates. Clients and ones who owns certificates.
Certificates.
Certificate Authority and Registration Authority.
Certificate Revocation Infrastructure. Server with Certificate Revocation list or OCSP Server.
Certificate Policy and Practices documents. Describe format of certificate, format of certificate request, when certificated have to be revoked, etc. Basically all procedures related to infrastructure.
Hardware Security Modules, which are usually used to protect Root CA’s private key.

And that entire infrastructure support following functions, which we’ve just discussed:

Public Key Cryptography.
Certificate issuance.
Certificate validation.
Certificate revocation.

And that’s it. Appeared to be not such a big topic ;)

Mac in 1984

2010-03-23T22:27:00.004Z

Was looking for some presentations on Web and found one made by Steve Jobs in 1984. The reaction of people there is just amazing, I have never seen anything similar on IT event. I wish I would visit one which will be the same impressive :)

Redirecting or error output to variable in shell

2010-03-04T16:42:00.010Z

Spent couple of painful hours trying to do that. And eventually, here is the code which will output standard output into the file and error into the variable:


var=`(ls -l > ./file.txt) 2>&1`

bloody shell...

Java bridge methods explained

2010-03-01T09:53:00.230Z

Bridge methods in Java are synthetic methods, which are necessary to implement some of Java language features. The best known samples are covariant return type and a case in generics when erasure of base method's arguments differs from the actual method being invoked.

Have a look at following example:

public class SampleOne {
    public static class A<T> {
        public T getT() {
            return null;
        }
    }

    public static class  B extends A<String> {
        public String getT() {
            return null;
        }
    }
}

Which in reality is just an example of covariant return type and after erasure will look like following snippet:

public class SampleOne {
    public static class A {
        public Object getT() {
            return null;
        }
    }

    public static class  B extends A {
        public String getT() {
            return null;
        }
    }
}

And after the compilation decompiled result class "B" will be following:

public class SampleOne$B extends SampleOne$A {
public SampleOne$B();
...
public java.lang.String getT();
Code:
0:   aconst_null
1:   areturn
public java.lang.Object getT();
Code:
0:   aload_0
1:   invokevirtual   #2; // Call to Method getT:()Ljava/lang/String;
4:   areturn
}

Above you can see there is new synthetic method "java.lang.Object getT()" which is not present in source code. That method acts as bridge method and all is does is delegating invocation to "java.lang.String getT()". Compiler has to do that, because in JVM method return type is part of method's signature, and creation of bridge method here is just the way to implement covariant return type.

Now have a look at following example which is generics-specific:

public class SampleTwo {
    public static class A<T> {
        public T getT(T args) {
            return args;
        }
    }

    public static class B extends A<String> {
        public String getT(String args) {
            return args;
        }
    }
}

after compilation class "B" will be transformed into following:

public class SampleThree$B extends SampleThree$A{
public SampleThree$B();
...
public java.lang.String getT(java.lang.String);
Code:
0:   aload_1
1:   areturn

public java.lang.Object getT(java.lang.Object);
Code:
0:   aload_0
1:   aload_1
2:   checkcast       #2; //class java/lang/String
5:   invokevirtual   #3; //Method getT:(Ljava/lang/String;)Ljava/lang/String;
8:   areturn
}

here, the bridge method, which overrides method from base class "A", not just calling one with string argument (#3), but also performs type cast to "java.lang.String" (#2). It means, that if you will execute following code, ignoring compiler's "unchecked" warning, the result will be ClassCastException thrown from the bridge method:

A a = new B();
a.getT(new Object()));

These two examples are the best known cases where bridge methods are used, but there is, at least, one more, where bridge method is used to "change" visibility of base class's methods. Have a look at following sample and try to guess where compiler may need the bridge method to be created:

package samplefour;

public class SampleFour {
    static class A {
        public void foo() {
        }
    }
    public static class C extends A {

    }
    public static class D extends A {
        public void foo() {
        }
    }
}

If you will decompile class C, you will see method "foo" there, which overrides method from base class and delegates to it:

public class SampleFour$C extends SampleFour$A{
...
public void foo();
Code:
0:   aload_0
1:   invokespecial   #2; //Method SampleFour$A.foo:()V
4:   return

}

compiler needs that method, because class A is not public and can't be accessed outside it's package, but class C is public and all inherited method have to become visible outside the package as well. Note, that class D will not have bridge method, because it overrides "foo" and there is no need to "increase" visibility.
It looks like, that type of bridge method, was introduced after bug which was fixed in Java 6. It means that before Java 6 that type of bridge method is not generated and method "C#foo" can't be called from package other than it's own via reflection, so following snippet causes IllegalAccessException, in cases when compiled on Java version < 1.6:

package samplefive;
...
SampleFour.C.class.getMethod("foo").invoke(new SampleFour.C());
...

Normal invocation, without using reflection, will work fine.

Probably there are some other cases where bridge methods are used, but there is no source of information about it. Also, there is no definition of bridge method, although you can guess it easily, it's pretty obvious from examples above, but still would be nice to have something in spec which states it clearly. In spite of the fact that method Method#isBridge() is part of public reflection API since Java1.5 and bridge flag is part of class file format, JVM and JLS specifications do not have any information what exactly is that and do not provide any rules when and how it should be used by compiler. All I could find is just reference in "Discussion" area here.

Kill all child processes from shell script

2010-02-23T09:48:00.007Z

Small and simple script. Creates Ctrl-C trap and kills all it's child processes in it. Nice to have when your script has many child processes which execution have to be stopped when main script is interrupted by Ctrl-C or other signal.


kill_child_processes() {
    isTopmost=$1
    curPid=$2
    childPids=`ps -o pid --no-headers --ppid ${curPid}`
    for childPid in $childPids
    do
        kill_child_processes 0 $childPid
    done
    if [ $isTopmost -eq 0 ]; then
        kill -9 $curPid 2> /dev/null
    fi
}

# Ctrl-C trap. Catches INT signal
trap "kill_child_processes 1 $$; exit 0" INT

for ((  i = 0 ;  i <= 5;  i++  ))
do
  # do something...
  sleep 10 &
done

wait

Exporting keys from keystore

2010-02-08T13:40:00.015Z

Recently I had a similar feeling to one I had writing one of previous posts. It appeared that standard java tools do not have some basic functionality, which obviously (well, probably just for me :) ) should be there. Now I had to export key stored in keystore to share it with other department. It appeared, that keytool can't do that and you have to write tiny program by yourself. Not a big problem, really, it's even nice.

There are lots of posts in web describing how to solve that problem, and here is the best code example I found so far to solve that it.
And here is copy/paste of code snipped, just in case if original post will pass away.


File keystoreFile = new File("The filename of the keystore");
KeyStore ks = KeyStore.getInstance("JKS"); // or whatever type of keystore you have
char[] pw = "the keystore password".toCharArray();
InputStream in = new FileInputStream(keystoreFile);
ks.load(in, pw);
in.close();
for (Enumeration en = ks.aliases(); en.hasMoreElements();)
{
    String alias = en.nextElement();
    System.out.println("  Alias\t:" + alias);
    // If the key entry password is not the same a the keystore password then change this
    KeyStore.Entry entry = ks.getEntry(alias, new KeyStore.PasswordProtection(pw)); 
    if (entry instanceof KeyStore.SecretKeyEntry) {
        System.out.println("    SecretKey");
        KeyStore.SecretKeyEntry skEntry = (KeyStore.SecretKeyEntry) entry;
        SecretKey key = skEntry.getSecretKey();
        System.out.println("      alg\t: " + key.getAlgorithm());
    } else if (entry instanceof KeyStore.PrivateKeyEntry) {
        System.out.println("    PrivateKey");
        KeyStore.PrivateKeyEntry pkEntry = (KeyStore.PrivateKeyEntry) entry;
        PrivateKey key = pkEntry.getPrivateKey();
        System.out.println("      alg\t: " + key.getAlgorithm());
        java.security.cert.Certificate certificate = pkEntry.getCertificate();
        System.out.println("      Certificate type\t: " + certificate.getType());
        System.out.println("      Public key\t: " + certificate.getPublicKey().getAlgorithm());
    } else if (entry instanceof KeyStore.TrustedCertificateEntry) {
        System.out.println("    Certificate");
        KeyStore.TrustedCertificateEntry certEntry = (KeyStore.TrustedCertificateEntry) entry;
        java.security.cert.Certificate certificate = certEntry.getTrustedCertificate();
        System.out.println("      type\t: " + certificate.getType());
    }
}

If you need to send key to someone, it handy to make it base64 encoded:


byte[] keyData = key.getEncoded();
BASE64Encoder b64Encoder = new BASE64Encoder();
String b64 = b64Encoder.encode(keyData);
System.out.println("-----BEGIN KEY-----");
System.out.println(b64);
System.out.println("-----END KEY-----");

Compile recursively with javac

2010-01-21T11:20:00.011Z

To my shame I have never used javac directly for compiling source files, always it was something like "ant". And today it was the first time on my memory when I had to do that :) It was a simple application, just an example for presentation and I didn't want to add anything additional there.
Thing which looks like a trivial task appeared to be not so trivial, because javac doesn't recursively compile files in directories, you have to specify each directory separately or create file with list of files for compilation, something like that:

find ./src -name "*.java" > sources_list.txt
javac -classpath "${CLASSPATH}" @sources_list.txt

On Windows first line should be replaced with: dir .\src\*.java /s /B > sources_list.txt

List of Open source trading software

2009-12-20T12:02:00.048Z

For everybody who is interested in topic. List of links below should help to make initial evaluation of available open-source java trading software & related products and also provides some other interesting links. Note, that projects below are not ordered in any particular order.

Groups, forums, communities

Google Group, JavaTraders
http://groups.google.com/group/JavaTraders

Elite trader, the #1 community for active traders of Stocks, Futures, Options, and Currencies.
http://www.elitetrader.com/

Trading software

Marketcetera
Open source platform for strategy-driven trading, providing you with all the tools you need for strategy automation, integrated market data, multi-destination FIX routing, broker neutrality and more.
Looks like is the leader in that list - it's well supported, has lots of capabilities and is active project.
Latest version available at 23.12.2009: 1.5.0 (released 05.2009)
http://trac.marketcetera.org/
http://www.marketcetera.com/

EclipseTrade
EclipseTrader is an application focused to the building of an online stock trading system, featuring shares pricing watch, intraday and history charts with technical analysis indicators, level II/market depth view, news watching, and integrated trading. The standard Eclipse RCP plug-ins architecture allows third-party vendors to extend the functionality of the program to include custom indicators, views or access to subscription-based data feeds and order entry.
Latest version available at 23.12.2009: 0.30.0 (released 07.2009)
http://sourceforge.net/projects/eclipsetrader/
http://eclipsetrader.sourceforge.net/

JSystemTrader
JSystemTrader is a fully automated trading system (ATS) that can trade various types of market securities during the trading day without user monitoring. All aspects of trading, such as obtaining prices, analyzing price patterns, making trading decisions, placing orders, monitoring order executions, and controlling the risk are automated according to the user preferences. The central idea behind JSystemTrader is to completely remove the emotions from trading, so that the trading system can systematically and consistently follow a predefined set of rules.
Latest version available at 23.12.2009: 6.24 (released 09.2008)
http://groups.google.com/group/jsystemtrader

ActiveQuant
AQ is a framework or an API for automated trading, opportunity detection, financial engineering, research in finance, connecting to brokers, etc. - basically everything around trading, written in Java, using Spring. All is published under a usage friendly open source license.
Latest version available at 23.12.2009: ??? Failed to find any possibility to download it or get latest version number, all links to that information are broken.
http://www.activestocks.eu/?q=node/1
http://www.activestocks.eu/

AIOTrade
AIOTrade (former Humai Trader) is a free, open source (under the terms of BSD license) stock technical analysis platform with a pluggable architecture that is ideal for extensions such as indicators and charts. It's built on pure java.
Latest version available at 23.12.2009: 1.0.3a (released 02.2007)
http://sourceforge.net/projects/humaitrader
http://blogtrader.org/

JStock
JStock makes it easy to track your stock investment. It provides well organized stock market information, to help you decide your best investment strategy.
No automated trading support.
Latest version available at 29.12.2009: 1.0.5g (released 12.2009)
http://jstock.sourceforge.net
https://sourceforge.net/projects/jstock/

Merchant of Venice
Venice is a stock market trading programme that supports portfolio management, charting, technical analysis, paper trading and experimental methods like genetic programming. Venice runs in a graphical user interface with online help and has full documentation.
Latest version available at 23.12.2009: 0.7b (released 04.2006)
http://sourceforge.net/projects/mov
http://mov.sourceforge.net/

Market Analysis System
The Market Analysis System (MAS) is an open-source software application that provides tools for analysis of financial markets using technical analysis. MAS provides facilities for stock charting and futures charting, including price, volume, and a wide range of technical analysis indicators. MAS also allows automated processing of market data — applying technical analysis indicators with user-selected criteria to market data to automatically generate trading signals — and can be used as the main component of a sophisticated trading system.
Latest version available at 23.12.2009: 1.6.6 (released 07.2004)
http://sourceforge.net/projects/eiffel-mas
http://eiffel-mas.sourceforge.net/

Open Java Trading System
The Open Java Trading System (OJTS) is meant to be a common infrastructure to develop stock trading systems. The project's aim is to provide a self contained pure Java (platform independent) common infrastructure for developers of trading systems.
Latest version available at 23.12.2009: 0.13 (released 06.2005)
http://sourceforge.net/projects/ojts/
http://ojts.sourceforge.net/

Oropuro trading system
The software perform the technical analysis of stock or commodity for various markets, manage portfolio definitions and orders. It has the base characteristics of most populars technical analysis software.
The most of information about that project is in Italian language, so it's really hard to dive in it :(
Latest version available at 23.12.2009: 0.2.4 (released 11.2007)
http://sourceforge.net/projects/oropuro
http://www.oropuro.org

TrueTrade
TrueTrade is a framework for developing, testing and running automatic trading systems. It is intended to provide support for a wide range of orders, financial instruments and time scales. It provides tooling for backtesting the strategy against historical data, and a separate tool for running the strategies in live mode.
Latest version available at 23.12.2009: 0.5 (released 05.2007)
http://code.google.com/p/truetrade/
http://groups.google.com/group/TrueTrade-Gen
http://groups.google.com/group/TrueTrade-Dev

(j)robotrader
Robotrader is a simulation platform for automated stock exchange trading. It delivers statistics to analyse performance on historic data and allows comparison between trading strategies.
Latest version available at 23.12.2009: 0.2.7 (released 02.2006)
http://jrobotrader.atspace.com
http://sourceforge.net/projects/robotrader/

SFL Java Trading System Enviroment
At current moment project is inactive. Author suggests to use ActiveQuant instead.
http://sourceforge.net/projects/sfljtse
http://www.sflweb.org/index.php?blog=sfljtse

Related software

TA-Lib: Technical Analysis Library
TA-Lib is widely used by trading software developers requiring to perform technical analysis of financial market data.
* Includes 200 indicators such as ADX, MACD, RSI, Stochastic, Bollinger Bands etc...
* Candlestick pattern recognition
* Open-source API for C/C++, Java, Perl, Python and 100% Managed .NET
Latest version available at 23.12.2009: 0.4 (released 09.2007)
http://ta-lib.org/index.html

Tail - A java technical analysis lib
Technical Analysis studies forecasting future price trends with the objective of managea best moment to buy and sell shares. The Tail's target is to develop a Java Open-Source library that abstracts the basic components of Technical Analysis, supplying tools for creation, manipulation and evaluation of strategies to buy and sell.
Latest version available at 15.01.2010: 1.0 (released 12.2007)
http://tail.sourceforge.net/

JessX
JessX Project's main goal is to create a program allowing for the simulation of a financial market with realistic features (such as an order book and realistic orders). Researchers and teachers in Finance may find it helpful in their works.
Latest version available at 23.12.2009: 1.5 (released 05.2008)
http://jessx.ec-lille.fr/

QuickFIX/J
100% Java Open Source FIX (Financial Information eXchange protocol) Engine
Latest version available at 23.12.2009: 1.4 (released 02.2009)
http://www.quickfixj.org/

Auge
Auge is an easy-to-use and very simple financial portfolio management application. Auge will help you monitor and analyze your stock and mutual fund positions, providing powerful insight into your entire investment portfolio.
Latest version available at 23.12.2009: 0.2 (released 04.2007)
http://sourceforge.net/projects/auge
http://auge.sourceforge.net/

Matrex
Advanced spreadsheet.
Latest version available at 23.12.2009: 1.3.8 (released 10.2009)
http://sourceforge.net/projects/matrex/
http://matrex.sourceforge.net/

Data Visualizer
Data Visualizer displays text file stock market type data ("Date,Open,High,Low,Close,Volume,Adjusted Close Price") as Stock Charts, featuring a variation of Japanese "Candlesticks" chart elements.
Latest version available at 23.12.2009: 0.0.1 (released 03.2006)
http://sourceforge.net/projects/dataviews
http://dataviews.sourceforge.net/

Forex Optimizer
Absolutely new revolutionary trade platform, is intended both for beginners, and for the tempered traders of Forex. Beginners can study market Forex, using a simulator, not risking the capitals and not being connected to the Internet. For more skilled traders Forex Optimizer allows to create and optimize trade strategy, not having knowledge in programming to operate (to make trading operations) the real account of the broker. The platform can offer professionals greater functionality for application of the strategy and methods of trade in market Forex.
Latest version available at 08.12.2010: 2.7 (released ??)
http://www.gordago.com/opensource/forex-optimizer/

http://gordago.googlecode.com/svn/tags/2.7/

Hibernate + Spring in Standalone application

2009-10-21T15:11:00.077+01:00

Just a quick reminder howto use Hibernate + Spring in standalone application.
First comes hibernate session factory. I prefer to use separate file for hibernate, Instead of configuring it Spring's applicationContext.xml:


<bean name="hibernateSessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
    <property name="configLocation" value="hibernate.cfg.xml"/>
    <!-- Those package will be scanned for classes with persistence annotations -->
    <property name="packagesToScan" value="net.test.domain"/>
    <!-- Annotated package. Contains package-level configuration. -->
    <property name="annotatedPackages" value="net.test.domain"/>
</bean>

Transaction manager:


<bean id="hibernateTransactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
    <property name="sessionFactory" ref="hibernate.session.factory"/>
</bean>
<tx:annotation-driven transaction-manager="hibernateTransactionManager"/>

Next is hibernate session. Hibernate guys suggest to use HibernateDaoSupport class as base for your DAOs, but to be completely honest, I'm not really comfortable with it, because it adds really weird dependency on Spring in DAO classes. Instead, it looks more natural to use dependency injection, which is really Spring's approach. To archive that all you need to do is mark session definition as being scoped proxy, set scope to 'prototype' and define SessionFactoryUtils#getSession as factory method. In result, each call to "any_method" in hibernate session bean instance is converted to SessionFactoryUtils.getSession().any_method():


<bean name="hibernateSession" class="org.springframework.orm.hibernate3.SessionFactoryUtils" factory-method="getSession"
  scope="prototype">
    <constructor-arg index="0" ref="hibernateSessionFactory"/>
    <constructor-arg index="1" value="false"/>
    <aop:scoped-proxy/>
</bean>

And all together:


<?xml version="1.0" encoding="UTF-8"?>
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd">

    <bean name="hibernateSessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
        <property name="configLocation" value="hibernate.cfg.xml"/>
        <!-- Those package will be scanned for classes with persistence annotations ->
        <property name="packagesToScan" value="net.test.domain"/>
        <!-- Annotated package. Contains package-level configuration. -->
        <property name="annotatedPackages" value="net.test.domain"/>
    </bean>

    <bean id="hibernateTransactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
        <property name="sessionFactory" ref="hibernateSessionFactory"/>
    </bean>
    <tx:annotation-driven transaction-manager="hibernateTransactionManager"/>
    <bean name="hibernateSession" class="org.springframework.orm.hibernate3.SessionFactoryUtils" factory-method="getSession"
  scope="prototype">
        <constructor-arg index="0" ref="hibernateSessionFactory"/>
        <constructor-arg index="1" value="false"/>
        <aop:scoped-proxy/>
    </bean>

    <bean name="someDao" scope="singleton" class="net.test.TestDAO">
        <property name="session" ref="hibernateSession"/>
    </bean>
</beans>

In DAO class there is no need to worry about session management, "tx:annotation-driven" will do all work for you. The only thing, which developer has to think about is appropriate usage of transaction annotations.

How to make IntelliJ Idea usable on linux

2009-08-20T16:38:00.005+01:00

Which Idea on Linux there are two big problems - memory (which I failed to resolve :)) and overall performance. When I've just installed it, it wasn't usable on may box with four Xeon CPUs and four gigs of memory, and that fact surprised me a lot. After some time spent looking on Internet found several tips, which resolved most of my problems. First one is following java setting, which switches off storing images in pixelmaps:

-Dsun.java2d.pmoffscreen=false

Another thing is one of Linux file system settings, which disables update of accessed timestamp. I'm working with quite big projects, which have several thousand files and updating timestamp of each file access is unnecessary overhead. That feature is something which doesn't have lots of use and can be witched off, specially when you are running Linux as desktop. Just add following to your filesystem mount settings in fstab:

noatime,nodiratime

After that Idea became reasonably fast, but I must admit, on Win XP or Vista it still runs faster and uses less memory.

I'm running Fedora 10 with NVidia graphic drivers.

Udpate 2010.12: I've just installed Idea 10. It looks just awesome and it's much faster! Works brilliantly. Thanks to IntelliJ!

Memory mapped files in Java? Easy!?

2009-02-22T21:06:00.019Z

Heh. Not so easy :) In NIO there is ability to work with memory mapped files, which is provided via java.nio.MappedByteBuffer. I was really happy with idea using memory mapped files, which means icreasing of performace of working with with big files. But... well, it is not so good, as could be. Generally speaking, it works, but has some serious pitfalls, which are well-known and described (and discussed) here:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4724038
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6417205 (fixed in 1.6)

In two words, you can't unmap file from memory, which is causing all other issues. It is released only when GC collects that buffer. Such limitation doesn't allow you to use that feature in any application which need to release file or work with mappings really intensive (create lots of new ones).