Weblogic Server: Core issues

Ant Script to start server and deploy application

2010-02-22T05:20:00.000-08:00

Following ant script will: a) start the server (wlserver ant task) b) deploy application on the application from the server

Deployment Plan

2010-02-04T09:04:00.000-08:00

Overriding value using Deployment Plan

We can override the values of the deployement descriptors at the runtime using plan.xml. Following are the steps which can be followed to override context-root of the application specified in the weblogic.xml:

1) My weblogic.xml looks as follows:

I want to change the context-root from virtual to TestApp at runtime.

2) Create a deployment plan using weblogic.PlanGenerator as follows:

java weblogic.PlanGenerator -all -plan Plan.xml Virtual.war

This will generate Plan.xml outside the archived application.

3) Plan.xml contains the values which can be overridden in the deployment descriptor at runtime. We can override the value of context-root at runtime as follows:

4) Now deploy the application using weblogic.Deployer as follows:

java weblogic.Deployer -adminurl t3://localhost:7001 -user weblogic -password weblogic -redeploy -name Virtual -source Virtual.war -targets AdminServer -plan Plan.xml

5) Test the application using the following URL:

http://localhost:7001/TestApp

Default URL of application:
http://localhost:7001/Virtual

At runtime the value of context-root was overridden to TestApp.

Cluster Unicast messaging mode

2010-02-04T07:19:00.000-08:00

UNICAST messaging mode

WLS10.x onwards there are 2 messaging modes of cluster: Unicast and Multicast. According to the following document Unicast is the recommended messaging mode though there have been many issues reported.

http://download-llnw.oracle.com/docs/cd/E11035_01/wls100/cluster/features.html

a) In Unicast messaging mode, cluster members are split into groups and every group has a group leader. Cluster members communicate to the group leader when they need to send a broadcast message which is usually the heartbeat message. When the cluster members detect the failure of a group leader, the next oldest member becomes the group leader. All group leaders are connected to each other.

b) The frequency of communication in unicast mode is similar to the frequency of sending messages on multicast port.

Unicast messaging mode Model:

Unicast Configuration:

Default Network Channel:

While creating a cluster choose Unicast as the messaging mode. If we leave the ‘Unicast broadcast channel’ blank then default listening port of the server is used if no channel is configured.

Config.xml looks like follows when default network channel is in use:

In the standard out of managed servers we will get the following message:

This means that the default channel will listen all the protocols.

Custom Network Channel:

For customer Network channels to be used with Unicast we need to create Network Channel by the same name under each server in cluster from Protocol > Channels

We can create custom network channels for different protocols. Protocol used by unicast is cluster-broadcast for communication. Use this custom channel name in the Unicast broadcast channel.

NOTE: For example, If you are configuring Network channel for different protocols say http, t3,iiop etc then cluster-broadcast will be one of the common protocol which will be default be configured along with other protocols. Hence, custom channel created for http, t3, iiop can be used as unicat broadcast channel.

Config.xml will have the following information where Channel-0 is the custom Network channel.

Here I configured Channel-0 for http protocol which is used as Unicast broadcast channel and I get the following message in the standard out of managed server:

While configuring channels other things that have that have to be taken care:

1) Listen Address, Listen port: If this is left blank then the server’s listen address will be taken as default address and port
2) External Address, External port: This is the address of firewall or loadbalancer if there is any.
3) HTTP enabled for this protocol: If you want that all http requests should be listen by the protocol for which network channel is configured then we need to keep this checked
4) Outbound Enabled: If this channel is going to listen to outbound traffic then this has to be checked

Refer: http://download.oracle.com/docs/cd/E15051_01/wls/docs103/config_wls/network.html#wp1058979

Unicast Debug Flag:
-Dweblogic.debug.DebugUnicastMessaging

Testing Unicast connectivity

In Unicast, we do not have any utility check network issues. We can suggest the customer to change the messaging mode to Multicast and check for network issues if there are any using the multicast utility

Multicast or IP multicast is a simple broadcast technology that enables multiple applications to subscribe to a given IP address and port number and listen for messages. The IP/port combination is setup when the cluster is defined and server instances uses multicast for JNDI updates andcluster heartbeats. So using java utils.MulticastTest -n server1 -a 224.x.x.x –p 9001 We just check whether all the Nodes
available in the Cluster is able to communicate or not. It is actually testing at the N/W level.

Configuration: I-Planet as a proxy with Weblogic server

2010-01-27T14:17:00.000-08:00

Steps To Configure WLS Plug-in in IPlanet Web Server

1. Install SunOne Web-Server.
2. Install WLS with the Plugin Option enabled.This option installs WLS with all the plugins files required for supported web-servers.
3. Refer BEA_HOME\weblogic92\server\plugin for plugin files.

4. Navigate to appropriate OS and copy the IPlanet plug-in files.

5. Plug-in file for (IPlanet)Sun Server is proxy61.dll

6.Paste the proxy61.dll file in Sun Server’s plugin\lib folder(SUNHOME\WebServer6.1\plugins\lib).

7. The next is to edit the magnus.conf and obj.conf files of Sun Server and make entry for WLS plug-in(SUNHOME \WebServer6.1\https-beatest\config).

8. The entries that should be made in magnus.conf file are:

(1)A LoadModule entry in Loadmodule specification.

Init fn="load-modules" funcs="wl_proxy,wl_init" shlib="C:/Sun/WebServer6.1/plugins/lib/proxy61.dll"
Init fn="wl_init"

(2) Object specification in obj.conf for cluster environment.

Service fn="wl_proxy" WebLogicCluster="127.0.0.1:7003,127.0.0.1:7005" Debug="ALL" WLLogFile="proxy1.log"

127.0.0.1:7003,127.0.0.1:7005 is the cluster addresses to which the sun server redirects the request. The application is deployed on this cluster.

(3) Object specification in module configuration for single server.

Service fn="wl_proxy"
WebLogicHost=127.0.0.1
WebLogicPort=7001
WLLogFile="proxy1.log"

9. After completing these specification start the Sun server.

10. Start the WLS admin server and all managed servers participating in the cluster.

11. Open the browser and type the request URL to the application deployed in the cluster.

Some JRMC commands

2010-01-27T11:59:00.000-08:00

JRMC commands and their outputs:

Command:
jrcmd 864 print_class_summary

Output:
*****class java/security/cert/CertificateParsingException
****class java/security/SignatureException
****class java/security/DigestException
****class java/security/NoSuchProviderException
****class java/security/NoSuchAlgorithmException
***class javax/naming/NamingException
****class javax/naming/CommunicationException
****class javax/naming/NameAlreadyBoundException
****class javax/naming/NamingSecurityException
*****class javax/naming/NoPermissionException
****class javax/naming/NameNotFoundException
*****class weblogic/jndi/internal/AdminModeAccessException
****class javax/naming/ConfigurationException
****class javax/naming/InvalidNameException
***class java/lang/InterruptedException
*****class java/rmi/server/ExportException
****class javax/management/remote/JMXProviderException
****class java/nio/charset/CharacterCodingException
****class java/io/EOFException
****class java/util/zip/ZipException
****class java/io/FileNotFoundException
*****class weblogic/utils/jars/JarFileUtils$PathLengthException
****class java/net/UnknownHostException
****class java/net/MalformedURLException
****class java/io/UnsupportedEncodingException
***class java/lang/IllegalAccessException
***class java/lang/ClassNotFoundException
***class java/security/PrivilegedActionException
***class java/lang/NoSuchFieldException
***class java/lang/reflect/InvocationTargetException
***class java/lang/RuntimeException
****class org/apache/commons/beanutils/ConversionException
****class org/apache/beehive/netui/util/config/ConfigInitializationException
****class org/apache/commons/logging/LogConfigurationException
****class javax/xml/rpc/soap/SOAPFaultException
****class javax/xml/rpc/JAXRPCException
****class org/apache/commons/beanutils/ConversionException
****class org/apache/beehive/netui/util/config/ConfigInitializationException
****class java/lang/annotation/IncompleteAnnotationException
****class weblogic/xml/schema/binding/util/ClassUtil$ClassUtilException
****class com/sun/org/apache/xerces/internal/impl/xpath/regex/ParseException
****class org/apache/commons/logging/LogConfigurationException
****class javax/ejb/EJBException
****class org/apache/oro/text/MalformedCachePatternException
*****class org/apache/oro/text/perl/MalformedPerl5PatternException
****class weblogic/wsee/deploy/WSEEServletEndpointException
****class weblogic/cluster/migration/MigrationException
****class weblogic/rmi/extensions/NotImplementedException
****class sun/security/util/PendingException
****class com/bea/common/engine/SecurityServiceRuntimeException
*****class com/bea/common/engine/InvalidParameterException
*****class com/sun/org/apache/xerces/internal/xni/parser/XMLParseException
*****class com/sun/org/apache/xerces/internal/xni/parser/XMLConfigurationExcepti
on
****class com/bea/xml/XmlRuntimeException
*****class com/bea/xml/SchemaTypeLoaderException
****class com/bea/xbean/regex/ParseException
****class com/bea/xbean/values/XmlValueDisconnectedException
****class weblogic/utils/NestedRuntimeException
*****class weblogic/security/providers/authentication/LDAPAtnDelegateException
*****class weblogic/security/SecurityInitializationException
*****class weblogic/ldap/EmbeddedLDAPException
*****class weblogic/security/internal/encryption/EncryptionServiceException
***class javax/xml/parsers/FactoryFinder$ConfigurationError
***class weblogic/utils/NestedError
****class weblogic/management/configuration/ConfigurationError
****class weblogic/utils/AssertionError
***class sun/management/AgentConfigurationError
***class bea/jrockit/management/server/Main$AgentError
***class java/lang/AssertionError
***class java/lang/VirtualMachineError
****class java/lang/UnknownError
****class java/lang/InternalError
****class java/lang/OutOfMemoryError
****class java/lang/StackOverflowError
*class java/lang/Thread
**class weblogic/deploy/api/spi/deploy/WebLogicDeploymentManagerImpl$1
**class weblogic/platform/GCMonitorThread
**class com/octetstring/vde/DoSManager
**class com/octetstring/vde/frontend/LDAP
**class com/octetstring/vde/backend/standard/TransactionProcessor
**class com/octetstring/vde/util/TimedActivityThread
**class com/octetstring/vde/LDAPServer
**class weblogic/timers/internal/TimerThread$Thread
**class weblogic/t3/srvr/T3Srvr$2
**class weblogic/kernel/AuditableThread
***class weblogic/server/channels/ServerThread
***class weblogic/kernel/ExecuteThread
****class weblogic/kernel/ServerExecuteThread
***class weblogic/work/ExecuteThread
**class weblogic/t3/srvr/ShutdownOnExitThread
**class java/util/TimerThread
**class java/util/logging/LogManager$Cleaner
**class java/lang/ref/Reference$ReferenceHandler
*class jrockit/vm/RNI
*class jrockit/vm/Reflect
*class jrockit/vm/VM
**class jrockit/vm/Locks
*class jrockit/vm/AccessController
*class java/lang/reflect/AccessibleObject
**class java/lang/reflect/Constructor
**class java/lang/reflect/Field
**class java/lang/reflect/Method
*class java/lang/String
*class [Z
*class [S
*class [D
*class [F
*class [J
*class [I
*class [C
*class [B
*class java/lang/Class
^^^^^^ Class Summary Information ends here ^^^^^^

Command:

jrcmd 864 print_object_summary increaseonly=true
864:

Output
--------- Detailed Heap Statistics: ---------
36.4% 10151k 125390 +10151k [C
11.2% 3119k 133108 +3119k java/lang/String
4.7% 1304k 11928 +1304k java/lang/Class
4.4% 1215k 2722 +1215k [B
3.4% 946k 10371 +946k [Ljava/util/HashMap$Entry;
3.2% 889k 37944 +889k java/util/HashMap$Entry
2.5% 701k 9127 +701k [Ljava/lang/Object;
1.8% 512k 16401 +512k java/util/TreeMap$Entry
1.6% 439k 23655 +439k [Lcom/bea/staxb/buildtime/internal/bts/JavaTypeName;

1.5% 410k 13138 +410k java/util/LinkedHashMap$Entry
1.5% 409k 7479 +409k com/bea/staxb/buildtime/internal/bts/QNameProperty
1.4% 387k 16512 +387k java/util/Hashtable$Entry
1.4% 382k 8155 +382k java/util/HashMap
1.3% 367k 23536 +367k com/bea/staxb/buildtime/internal/bts/MethodName
1.2% 339k 14505 +339k javax/xml/namespace/QName
1.1% 308k 13173 +308k com/bea/staxb/buildtime/internal/bts/JavaTypeName
1.0% 283k 1197 +283k [I
0.9% 256k 2875 +256k [Ljava/util/Hashtable$Entry;
0.8% 211k 2704 +211k com/bea/staxb/runtime/internal/ByNameRuntimeBindingType$ElementQNameProperty
0.7% 206k 5539 +206k [Ljava/lang/String;
0.7% 194k 2492 +194k java/lang/reflect/Method
0.7% 183k 5860 +183k java/lang/ref/SoftReference
0.5% 144k 6171 +144k java/lang/ref/WeakReference
27856kB total ---

--------- End of Detailed Heap Statistics ---

Command:

jrcmd 864 print_threads

Output:

===== FULL THREAD DUMP ===============
Thu Jan 28 01:41:58 2010
BEA JRockit(R) R27.2.0-131-78843-1.5.0_10-20070320-1457-windows-ia32

"Main Thread" id=1 idx=0x4 tid=236 prio=5 alive, in native, waiting
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x0ABAF240[fat lock]
at jrockit/vm/Threads.waitForSignal(J)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:474)
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:730)
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x0ABAF240[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:380)
at weblogic/Server.main(Server.java:67)
at jrockit/vm/RNI.c2java(IIII)V(Native Method)
-- end of trace

"(Signal Handler)" id=2 idx=0x8 tid=1872 prio=5 alive, in native, daemon

"(Code Generation Thread 1)" id=3 idx=0xc tid=2956 prio=5 alive, in native, native
_waiting, daemon

"(Code Optimization Thread 1)" id=4 idx=0x10 tid=2880 prio=5 alive, in native, nat
ive_waiting, daemon

"(GC Main Thread)" id=5 idx=0x14 tid=2064 prio=5 alive, in native, daemon

"(GC Worker Thread 1)" id=? idx=0x18 tid=1136 prio=5 alive, in native, native_wait
ing, daemon

"(GC Worker Thread 2)" id=? idx=0x1c tid=2056 prio=5 alive, in native, native_wait
ing, daemon

"(VM Periodic Task)" id=6 idx=0x20 tid=3284 prio=10 alive, in native, daemon

"(Attach Listener)" id=7 idx=0x24 tid=2392 prio=5 alive, in native, daemon

"Finalizer" id=8 idx=0x28 tid=2368 prio=8 alive, in native, native_waiting, daemon

at jrockit/memory/Finalizer.getPendingFinalizee(Z)Ljava/lang/Object;(Native Me
thod)
at jrockit/memory/Finalizer.access$100(Z)Ljava/lang/Object;(Unknown Source)
at jrockit/memory/Finalizer$4.run()V(Unknown Source)
at java/lang/Thread.run(Thread.java:595)
at jrockit/vm/RNI.c2java(IIII)V(Native Method)
-- end of trace

"Reference Handler" id=9 idx=0x2c tid=2312 prio=10 alive, in native, native_waitin
g, daemon
at java/lang/ref/Reference.getPending()Ljava/lang/ref/Reference;(Native Method
)
at java/lang/ref/Reference.access$000()Ljava/lang/ref/Reference;(Unknown Sourc
e)
at java/lang/ref/Reference$ReferenceHandler.run()V(Unknown Source)
at jrockit/vm/RNI.c2java(IIII)V(Native Method)
-- end of trace

"(Sensor Event Thread)" id=10 idx=0x30 tid=2952 prio=5 alive, in native, daemon
----------

===== END OF THREAD DUMP ===============

Command:
>jrcmd 864 verbosity set=memory,memdbg,codegen,opt,sampling filename="output.log"
864:

Output
Current logstatus:
jrockit : level=WARN, decorations=101, sanity=NONE
memory (gc) : level=INFO, decorations=101, sanity=NONE
nursery (yc) : level=WARN, decorations=101, sanity=NONE
model : level=WARN, decorations=101, sanity=NONE
devirtual : level=WARN, decorations=101, sanity=NONE
codegen (code) : level=INFO, decorations=101, sanity=NONE
native (jni) : level=WARN, decorations=101, sanity=NONE
thread : level=WARN, decorations=101, sanity=NONE
opt : level=INFO, decorations=101, sanity=NONE
load (lib) : level=WARN, decorations=101, sanity=NONE
ssa : level=WARN, decorations=101, sanity=NONE
constprop : level=WARN, decorations=101, sanity=NONE
inline : level=WARN, decorations=101, sanity=NONE
preprocess : level=WARN, decorations=101, sanity=NONE
jvmpi : level=WARN, decorations=101, sanity=NONE
memdbg : level=INFO, decorations=101, sanity=NONE
jvmtidbg : level=WARN, decorations=101, sanity=NONE
sampling (hotspot) : level=INFO, decorations=101, sanity=NONE
cpuinfo : level=WARN, decorations=101, sanity=NONE
cpuid : level=WARN, decorations=101, sanity=NONE
sym : level=WARN, decorations=101, sanity=NONE
codegc : level=WARN, decorations=101, sanity=NONE
classgc : level=WARN, decorations=101, sanity=NONE
errorsim (esim) : level=WARN, decorations=101, sanity=NONE
memprof (mp) : level=WARN, decorations=101, sanity=NONE
jvmti : level=WARN, decorations=101, sanity=NONE
gcpause : level=WARN, decorations=101, sanity=NONE
gcreport : level=WARN, decorations=101, sanity=NONE
finalizer : level=WARN, decorations=101, sanity=NONE
jripc : level=WARN, decorations=101, sanity=NONE
memleak : level=WARN, decorations=101, sanity=NONE
compaction : level=WARN, decorations=101, sanity=NONE
dpgo : level=WARN, decorations=101, sanity=NONE
license : level=WARN, decorations=101, sanity=NONE
tasks : level=WARN, decorations=101, sanity=NONE
exceptions : level=WARN, decorations=141, sanity=NONE
stackoverflow : level=WARN, decorations=141, sanity=NONE
sysgc (systemgc) : level=WARN, decorations=101, sanity=NONE
locks : level=WARN, decorations=101, sanity=NONE
strcmp : level=WARN, decorations=101, sanity=NONE
bca : level=WARN, decorations=101, sanity=NONE
ir : level=WARN, decorations=101, sanity=NONE
starttime : level=WARN, decorations=101, sanity=NONE
largepages : level=WARN, decorations=101, sanity=NONE
mgmnt (management) : level=WARN, decorations=101, sanity=NONE
jmm : level=WARN, decorations=101, sanity=NONE
perf : level=WARN, decorations=101, sanity=NONE
attach : level=WARN, decorations=101, sanity=NONE
referents (verboserefs) : level=WARN, decorations=101, sanity=NONE
class : level=WARN, decorations=101, sanity=NONE
intrinsics (intrinsic) : level=WARN, decorations=101, sanity=NONE
verify (verifier) : level=WARN, decorations=101, sanity=NONE
jra : level=WARN, decorations=101, sanity=NONE
timing : level=WARN, decorations=101, sanity=NONE

Command:

>jrcmd 864 timestamp
864:

Output:
==== Timestamp ==== uptime: 0 days, 00:17:34 time: Thu Jan 28 01:46:22 2010

Command:

>jrcmd 864 print_memusage
864:
[JRockit] memtrace is collecting data...
[JRockit] *** 0th memory utilization report
(all numbers are in kbytes)
Total mapped ;;;;;;; 733268
; Total in-use ;;;;;; 415560
;; executable ;;;;; 11572
;;; java code ;;;; 5952; 51.4%
;;;; used ;;; 5626; 94.5%
;; shared modules (exec+ro+rw) ;;;;; 14168
;; guards ;;;;; 544
;; readonly ;;;;; 2536
;; rw-memory ;;;;; 398312
;;; Java-heap ;;;; 262144; 65.8%
;;; Stacks ;;;; 4008; 1.0%
;;; Native-memory ;;;; 132159; 33.2%
;;;; java-heap-overhead ;;; 10155
;;;; codegen memory ;;; 1088
;;;; classes ;;; 56320; 42.6%
;;;;; method bytecode ;; 6374
;;;;; method structs ;; 6889 (#146974)
;;;;; constantpool ;; 23334
;;;;; classblock ;; 2185
;;;;; class ;; 3703 (#11938)
;;;;; other classdata ;; 8443
;;;;; overhead ;; 442
;;;; threads ;;; 20; 0.0%
;;;; malloc:ed memory ;;; 29814; 22.6%
;;;;; codeinfo ;; 1306
;;;;; codeinfotrees ;; 492
;;;;; exceptiontables ;; 170
;;;;; metainfo/livemaptable ;; 5894
;;;;; codeblock structs ;; 2
;;;;; constants ;; 10
;;;;; livemap global tables ;; 1027
;;;;; callprof cache ;; 0
;;;;; paraminfo ;; 116 (#1622)
;;;;; strings ;; 10863 (#185955)
;;;;; strings(jstring) ;; 0
;;;;; typegraph ;; 2871
;;;;; interface implementor list ;; 283
;;;;; thread contexts ;; 25
;;;;; jar/zip memory ;; 9862
;;;;; native handle memory ;; 21
;;;; unaccounted for memory ;;; 34781; 26.3%;1.17
---------------------!!!

Command:
>jrcmd 864 heap_diagnostics

Output:

======== BEGIN OF HEAPDIAGNOSTIC =========================

Invoked from diagnosticcommand

Total memory in system: 8377249792 bytes
Available physical memory in system: 1293717504 bytes
-Xmx (maximal heap size) is 4294967296 bytes
Heapsize: 4294967296 bytes
Free heap-memory: 3712605608 bytes

mmStartCompaction = 0x2b1e042b0000, mmEndCompaction = 0x2b1e082b0018


Memory layout:
00010000-003d0000 rwxp 00010000 00:00 0
00400000-00412000 r-xp 00000000 fd:04 1287492                            /opt/jrmc-3.1.0-1.5.0/bin/java
00420000-004e0000 rwxp 00420000 00:00 0
00511000-00513000 rwxp 00011000 fd:04 1287492                            /opt/jrmc-3.1.0-1.5.0/bin/java
00520000-07d60000 rwxp 00520000 00:00 0
07d7a000-0f9a5000 rwxp 07d7a000 00:00 0
0f9b0000-15ff0000 rwxp 0f9b0000 00:00 0
4005b000-4005d000 rwxp 4005b000 00:00 0
4005d000-40060000 ---p 4005d000 00:00 0
40060000-4009f000 rwxp 40060000 00:00 0
4009f000-400a2000 ---p 4009f000 00:00 0

--------- Detailed Heap Statistics: ---------
21.5% 72399k   637351     +0k [C
8.6% 29000k   281013     +0k [Ljava/lang/Object;
6.5% 21797k   697514     +0k java/lang/String
5.1% 17145k    31500    -59k [B
4.1% 13697k    87524     +0k [Ljava/util/HashMap$Entry;
2.7% 9274k    65950     +0k java/lang/reflect/Method
2.4% 8101k   207404     +0k java/util/HashMap$Entry
2.0% 6778k   173542     +0k com/bea/common/security/xacml/attr/StringAttribute
2.0% 6629k    33943     +0k java/lang/Class


    337244kB total ---

--------- End of Detailed Heap Statistics ---
----- Reference Objects statistics separated per class -----
   Total Reach Act PrevAct Null
   ----- ----- --- ------- ----
Finalizers:
       5     2   3       0    0 Total for all Finalizers
       2     1   1       0    0 => java/net/SocksSocketImpl
       1     0   1       0    0 => weblogic/socket/WeblogicSocket
       1     1   0       0    0 => java/net/SocketInputStream
       1     0   1       0    0 => java/net/SocketOutputStream
----- End of Reference Objects statistics -----
------------------- Printing heap ---------------------
"o"/"p" = 1k normal/pinned objects
"O"/"P" = 50k normal/pinned objects
"."/"/" = <1k normal/pinned objects (fragmentation)
" "/"_" = 1k/50k free space
-------------------------------------------------------
//////o///o/p/oo/OOOoooooooooooooooooooooo/OOooooo      0x2b1dfe2f7c70
ooooooooo/OOOOOOooooooooooo/Oooooooooooooooooooooo      0x2b1dfe357b98
ooooooooooooo..        OOOOOOOoooooooooooooo./oooo      0x2b1dfe3b7ac0
oooooooooooooooooooooooo/OOooooooooooooooooooooooo      0x2b1dfe3dbce0
oooooooooooooooooooo/oooooooooooooooo/OOOOOooooooo      0x2b1dfe423d38
oooooooooooooooooooooooooooo.   OOOooooooooooooooo      0x2b1dfe453ec0
ooooooooooooooooooooooooooo//.....................      0x2b1dfe460210
..............................Ooooooooooo/OOOOoooo      0x2b1dfe4a8268





the available commands are:

kill_rmp_server
start_rmp_server
kill_management_server
start_management_server
checkjrarecording
stopjrarecording
startjrarecording
print_object_summary
memleakserver
print_class_summary
print_codegenlist
run_optfile
dump_codelayout
dump_codelist
dump_codemap
print_utf8pool
print_properties
print_threads
datadump_request
runsystemgc
runfinalization
heap_diagnostics
oom_diagnostics
print_exceptions
version
timestamp
command_line
sanity
verbosity
set_filename
help
print_memusage

Reference Links:
http://www.oracle.com/technology/products/jrockit/pdf/missioncontrol_whitepaper_june08.pdf
http://download.oracle.com/docs/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/ctrlbreakhndlr.html

python to start, stop, deploy and undeploy application

2009-12-17T20:39:00.001-08:00

import sys
import os
from java.lang import System

import getopt

def usage():
print "Usage:"
print "java weblogic.WLST app_ctl.py [-n] -u user -p password -a url -m moduleName -c command -d sourcedir -t targets -s stagemode"
try:
opts, args = getopt.getopt(sys.argv[1:], "nu:p:a:m:c:d:t:s:",
["user=", "password=", "url=",
"moduleName=", "command=", "soucedir", "targets", "stagemode"])
except getopt.GetoptError, err:
print str(err)
usage()
#sys.exit(2)

reallyDoIt = true
user=None
password=None
url=None
moduleName=None
command=None
sourcedir=None
targets=None
stagemode=None

for opt, arg in opts:
if opt == "-n":
reallyDoIt = false
elif opt == "-u":
user = arg
elif opt == "-p":
password = arg
elif opt == "-a":
url = arg
elif opt == "-m":
moduleName = arg
elif opt == "-c":
command = arg
elif opt == "-d":
sourcedir = arg
elif opt == "-t":
targets = arg
elif opt == "-s":
stagemode = arg

if not user:
print "Missing \"-u user\" parameter."
usage()
#sys.exit(2)
elif not password:
print "Missing \"-p password\" parameter."
usage()
#sys.exit(2)
elif not url:
print "Missing \"-a url\" parameter."
usage()
#sys.exit(2)
elif not moduleName:
print "Missing \"-m moduleName\" parameter."
usage()
#sys.exit(2)
elif not command:
print "Missing \"-c command\" parameter."
usage()
#sys.exit(2)

print "Got all the required parameters."

#######################
if command == 'stop':
connect(user, password, url)
print 'Starting stop of '+moduleName+'...'
progress='null'
try:
cd("/AppDeployments/"+moduleName)
progress=stopApplication(moduleName)
print 'Checking stop status...'
state = progress.getState()
if state == 'completed':
print 'OK: Stop of '+moduleName+' was successful...'
else:
print 'ERROR: Stop of '+moduleName+' was not successful...'
except :
print 'Application '+moduleName+' does not exist.'
elif command == 'start':
connect(user, password, url)
print 'Starting start of '+moduleName+'...'
progress='null'
try:
cd("/AppDeployments/"+moduleName)
progress=startApplication(moduleName)
print 'Checking start status...'
state = progress.getState()
if state == 'completed':
print 'OK: Start of '+moduleName+' was successful...'
else:
print 'ERROR: Start of '+moduleName+' was not successful...'
except :
print 'Application '+moduleName+' does not exist.'
elif command == 'deploy':
connect(user, password, url)
print 'Starting deploy of '+moduleName+'...'
progress='null'
try:
#cd("/AppDeployments/"+moduleName)
progress=deploy(appName=moduleName, path=sourcedir, targets=targets, stageMode=stagemode)
print 'Checking deploy status...'
state = progress.getState()
if state == 'completed':
print 'OK: Deploy of '+moduleName+' was successful...'
else:
print 'ERROR: Deploy of '+moduleName+' was not successful...'
except :
print 'Application '+moduleName+' does not exist.'
elif command == 'undeploy':
connect(user, password, url)
print 'Starting undeploy of '+moduleName+'...'
progress='null'
try:
#cd("/AppDeployments/"+moduleName)
progress=undeploy(moduleName)
print 'Checking undeploy status...'
state = progress.getState()
if state == 'completed':
print 'OK: Undeploy of '+moduleName+' was successful...'
else:
print 'ERROR: Undeploy of '+moduleName+' was not successful...'
except :
print 'Application '+moduleName+' does not exist.'

################################
#
# stopProgress='null'
# stopProgress=stopApplication("spd")
# print 'Checking stop status...'
# stopProgress.getState()
#
#if stopProgress.getState == 'completed':
# print "Stop successful..."
#else:
# print "ERROR: stop spd not successful..."
#
#elif action == "start":
#
# connect(uname, pwd, url)
# startProgress='null'
# startProgress=startApplication('spd')
# print 'Checking start status...'
# startProgress.getState()

Taking thread dumps for weblogic server

2009-12-15T06:28:00.000-08:00

A) On Windows machine: cltr + break
On Unix Machine:
get pid using ps -ef grep java
kill -3
B) When weblogic is running as windows service
For Out file add in JAVA_OPTIONS:
-Dweblogic.Stdout=
a) Using beasvc :
beasvc –dump – svcname:”bea Admin_server”
b) Using Weblogic Admin utility
C:\bea\user_projects\domains\rDomain>java weblogic.Admin -url t3://localhost:7001 -username weblogic -password weblogic THREAD_DUMP
Thread Dump has been printed to the servers standard out (by default, the shell in which the server is running).

C) We can also collect thread dumps from the console:
Server > Monitoring > Threads > Dump Thread Stacks

D) Using WLST:
1) java weblogic.WLST
2) connect("weblogic","weblogic","t3:\\localhost:7001")
3) threadDump()

Troubleshooting: HTTP Session Replication Failure Issues

2009-12-02T16:12:00.000-08:00

One of the major advantage of weblogic server is 'HTTP session replication'. Sometimes, HTTP session states are not replicated from the primary server to the secondary server. This is the most common issue which is observed.

Symptoms:

The application using HTTP session does not function as designed and you see a loss of session data.
You might be asked to re-log into the application even when the session has still not timed out.
You see errors and warnings related to HTTP session failures in the server's log file.
The request is not failed over to another server properly.

There are several reasons as to why session replication fails. We will look at ways to diagnose the issue, possible reasons for it, and the ways to address them.

Types of HTTP Session Replication
There are five different implementations of session persistence:

Memory (single-server, non-replicated): When you use memory-based storage, all session information is stored in memory and is lost when you stop and restart WebLogic Server.
File system persistence: Session information is stored in a file under the PersistentStoreDir specified.
JDBC persistence: Session information is stored in a database table.
Cookie-based session persistence: Session information is stored in a cookie.
In-memory replication (across a cluster): Session data is copied from one server instance to another into memory.

Usually observed causes session not getting replicated:

1) All application objects are not serialized
2) Session is not set using set.attribute()
3) Network issue causing the managed server to move out of the cluster.
4) Weblogic server hang resulting to non-responsiveness to replicated session
5) Issue at the webserver end.

Diagnosing the Issue
Consider a scenario with two servers (MyServer-1, MyServer-2) in a cluster. When you enable the debug flags (see the "Enable the Debug Flags to Track Session Replication Failures"section), you will see the messages below when a request from a client is sent the very first time.

On MyServer-1:

<000000>

<000000>

This logging message means that the primary server is MyServer-1 and a secondary has been created on MyServer-2. A message like the one shown below will be logged on MyServer-2 to confirm that.

ExecuteThread: '1' for queue: 'Replication'> <>
<000000>
####

<> <000000>

If you check the JSESSIONID it looks like:

JSESSIONID=1E9Xwn7nLYfOsc1obgRZIwW5s72an7HPPvSD7iaWHMXzpHga5cQj
!-1587343083!-1587348922

JSESSIONID is the default name of the cookie, which could be changed to anything in weblogic.xml. The format of JSESSIONID is

SessionId!PrimaryServer JVM Hash!SecondaryServer JVMHash

Every time data is changed (either set/get or removed) in the session you'll see the logging message.

On MyServer-1:

<000000>

On MyServer-2:

####
<> <000000>

If for any reason session replication fails, you will see the message below in the MyServer-1 log:

<000000>

<000000>

And, the JSESSIONID would appear as

JSESSIONID=1E9Xwn7nLYfOsc1obgRZIwW5s72an7HPPvSD7iaWHMXzpHga5cQj!-1587343083!NONE

The secondary server hash would become NONE

Enable the Debug Flags to Track Session Replication Failures
You can enable the flags DebugCluster, DebugClusterAnnouncements, DebugFailOver, DebugReplication, and DebugReplicationDetails.

Configuration: Diagnostics JDBC store Archive

2009-12-02T15:46:00.000-08:00

The Archive component of the WebLogic Diagnostic Framework (WLDF) captures and persists all data events, log records, and metrics collected by WLDF from server instances and applications running on them. You can access archived diagnostic data in online mode (that is, on a running server). You can also access archived data in off-line mode using the WebLogic Scripting Tool (WLST).

Here, I have given steps which have to be followed to configure JDBC store Diagnostics Archive. Database used is: Pointbase

1) Create a Datasource from weblogic console with the JNDI name say: DS1
2) Create the following tables in Database:

a) wls_hvst

DROP TABLE wls_hvst;
CREATE TABLE wls_hvst (
RECORDID INTEGER IDENTITY,
TIMESTAMP NUMERIC default NULL,
DOMAIN varchar(64) default NULL,
SERVER varchar(64) default NULL,
TYPE varchar(64) default NULL,
NAME varchar(250) default NULL,
ATTRNAME varchar(64) default NULL,
ATTRTYPE INTEGER default NULL,
ATTRVALUE VARCHAR(4000)
);
COMMIT;

b) wls_event

DROP TABLE wls_events;
CREATE TABLE wls_events (
RECORDID INTEGER IDENTITY,
TIMESTAMP NUMERIC default NULL,
CONTEXTID varchar(128) default NULL,
TXID varchar(32) default NULL,
USERID varchar(32) default NULL,
TYPE varchar(64) default NULL,
DOMAIN varchar(64) default NULL,
SERVER varchar(64) default NULL,
SCOPE varchar(64) default NULL,
MODULE varchar(64) default NULL,
MONITOR varchar(64) default NULL,
FILENAME varchar(64) default NULL,
LINENUM INTEGER default NULL,
CLASSNAME varchar(250) default NULL,
METHODNAME varchar(64) default NULL,
METHODDSC varchar(4000) default NULL,
DROP TABLE wls_events;
CREATE TABLE wls_events (
RECORDID INTEGER IDENTITY,
TIMESTAMP NUMERIC default NULL,
CONTEXTID varchar(128) default NULL,
TXID varchar(32) default NULL,
USERID varchar(32) default NULL,
TYPE varchar(64) default NULL,
DOMAIN varchar(64) default NULL,
SERVER varchar(64) default NULL,
SCOPE varchar(64) default NULL,
MODULE varchar(64) default NULL,
MONITOR varchar(64) default NULL,
FILENAME varchar(64) default NULL,
LINENUM INTEGER default NULL,
CLASSNAME varchar(250) default NULL,
METHODNAME varchar(64) default NULL,
METHODDSC varchar(4000) default NULL,

3) Create a WLDF Module from Diagnostics > WLDF Module and target to a server

4) For the diagnostics module create a Collection Metrics for JDBCDataSourceRuntimeMBean

5) Click on Diagnostics > Archives and select the server for which we need to collect data.

6) Change the archive type to JDBC instead of File store and select the datasource which has to be used for JDBC store

7) Deploy any application which used database connection as we have configured the diagnostics module for JDBCRuntimeMbean and access the application.

8) The data will be archived in the table wls_hvst :

References:

http://download.oracle.com/docs/cd/E11035_01/wls100/wldf_configuring/config_diag_archives.html

Use 81 style execute queues

2009-12-02T15:02:00.000-08:00

From Weblogic 9.x onwards work managers are used instead of execute queues. Oracle provides a flag that enables you to disable the self-tuning execute pool and provide backward compatibility for upgraded applications to continue to use user-defined execute queues. Following steps can be followed to use 81 execute queues with later weblogic versions

1) Manually add the following in the config.xml:

AdminServer

true

Or we can add the following in JAVA_OPTIONS :

-Dweblogic.Use81StyleExecuteQueues=true

2) Start the server

On adding this parameter we will get queue tab under server > configuration > Queue the console:

Create a new Queue with the name: weblogic.kernel.default

We can edit the queue length and other parameters as in WLS8.1

Reference link:

http://download.oracle.com/docs/cd/E11035_01/wls100/perform/appb_queues.html#wp1048423

Production redeployment of application/ versioning/ side by side deployment

2009-12-02T14:42:00.000-08:00

We need versioning for updating the application in production on runtime. The sessions which are already logged in will be using the previous version of the application and the new sessions created will use the new version of application.

1) In the application, Version-1 in MANIFEST.MF we need to add the following:

Manifest-Version: 1.0
Created-By: 1.5.0_04 (Sun Microsystems Inc.)
Weblogic-Application-Version: Version-1

Version-2 MENIFEST-MF
Manifest-Version: 1.0
Created-By: 1.5.0_04 (Sun Microsystems Inc.)
Weblogic-Application-Version: Version-1

2. Open a command window and browse to your domain bin directory.
3. start Admin server using Startweblogic.cmd.
4. Type setdomainenv.cmd to set your weblogic Domain.
5.Now to deploy the first version run the following command (please change as per your specifications):
C:\bea9.1\user_projects\domains\base_domain>java weblogic.Deployer -appversion 1 -adminurl http://127.0.0.1:7001 -username weblogic -password weblogic -deploy -name deployment_name -source c:/.../ -targets AdminServer
6. Now to Redeploy use the following command:
C:\bea9.1\user_projects\domains\base_domain>java weblogic.Deployer -appversion 2 -adminurl http://127.0.0.1:7001 -username weblogic -password weblogic -redeploy -name deployment_name -source c:/.../ -retiretimeout 120

7. We will get the version -1 application in retiring state in the console. The requests which were handled previously by this application will be handled by this version.
8. Version-2 application will have ACTIVE status and will be accepting new requests coming in to the application.

Reference:
http://download.oracle.com/otn_hosted_doc/wls/redeployment/wls-side-by-side-non-annotation_viewlet_swf.html