My Techniques

My Favorite Firefox Add-ons

2009-01-03T17:55:00.006+05:30

One of the reasons why I stick to Firefox instead of using Chrome or something else is Add-ons. These are my favorite add-ons.

Adblock Plus - If you ask me to choose one add-on without which I can't live, that would be Adblock Plus.
IE Tab - Embeds Internet Explorer within Firefox. Can't tell how much time it saves for me. (Unfortunately most of work day revolves around those IE only websites. Now I don't need to leave Firefox).
Secure Login - Automates login to websites.
Delicious Bookmarks - My favorite bookmarking tool.
Cookie Swap - Allows me to quickly switch between different accounts on a website.
DownThemAll - Very useful for batch downloading.

I also love these bookmarklets. They offer powerful functionality in a simple manner.

Effective Oracle by Design

2008-05-17T12:29:00.001+05:30

I just finished reading Effective Oracle by Design by Tom Kyte. It is such a wonderful book that I regret why I missed it all these days. This book helped me in asserting some of my previously held notions, in dispensing with some of my myths and introduced me to many new concepts and features. It will be a valuable addition to my tech library. Highly recommended for anyone with some knowledge in Oracle.

Mine Archived Logs

2008-05-16T13:33:00.001+05:30

Sometime back, we had a rather difficult problem. There was a master table and nobody is supposed to delete records from that table. But once in a while, we found some records missing. To investigate further, we suggested creating a trigger on that table or enabling audit trail, but dba didn't agree. Then we came across this neat feature in Oracle called mining archived logs. We used that technique and found the culprit. Every one is impressed by the ability to analyze the problem after the fact (contrast to trigger/audit trail approach, which requires to configure the solution first and then wait for the problem to occur).

Here is the script we used...

(by the way, we were using oracle 9i R2, if that matters)

#################################

#!/bin/ksh
#You can expect spikes in CPU usage while this script executes.
#So schedule this script for offpeak hours.
#Invoke this script with nohup

if [ ! -f "${ORACLE_HOME}/bin/sqlplus" ]
then
echo "Error: Oracle not setup"
exit 1
fi

THIS_PROC=`basename $0 .sh`
TODAY=`date +%Y%m%d_%H%M%S`
LOGFILE=/tmp/${THIS_PROC}.log
SPOOLFILE=/tmp/${THIS_PROC}_${TODAY}.html

echo "" >>${LOGFILE}
echo "************************************************" >>${LOGFILE}
echo "Started ${THIS_PROC} on `date`" >> ${LOGFILE}

${ORACLE_HOME}/bin/sqlplus -s ${USER_PASS} << _EOF_ >> ${LOGFILE} 2>&1
ALTER SESSION SET NLS_DATE_FORMAT='DD-MON-YYYY HH24:MI:SS';
SET SERVEROUTPUT ON FORMAT WRAPPED SIZE 1000000
SET PAGES 1000
SET NUMWIDTH 20
SET ECHO OFF
SET FEEDBACK ON
SET HEADING ON
SET TIMING ON

--This will throw ORA-00942 error if the table doesn't exist
--Ignore the error
--Is it ok to drop any old data in this table?
DROP TABLE LOGMNR_DATA;

--Just create table structure without any data
CREATE TABLE LOGMNR_DATA
AS SELECT *
FROM V\$LOGMNR_CONTENTS
WHERE 1 = 0;

BEGIN
--You might want to mine just yesterday's data
--Otherwise increase date range
FOR REC IN (SELECT NAME
              FROM V\$ARCHIVED_LOG
              WHERE COMPLETION_TIME >= TRUNC(SYSDATE-1,'DD')
                AND COMPLETION_TIME < TRUNC(SYSDATE,'DD')
             )
LOOP
    DBMS_OUTPUT.PUT_LINE(SYSDATE || ' : Mining ' || REC.NAME);

    --Mine one log at a time so as not to overload CPU
    SYS.DBMS_LOGMNR.ADD_LOGFILE(LOGFILENAME =>REC.NAME,
                                OPTIONS => SYS.DBMS_LOGMNR.NEW);
    SYS.DBMS_LOGMNR.START_LOGMNR(OPTIONS=> SYS.DBMS_LOGMNR.DICT_FROM_ONLINE_CATALOG);

    --Here you are trying to trace deletion of access_number 7739324493
    --You need to change this query as per your requirements
    --SQL_REDO column in V\$LOGMNR_CONTENTS will have the complete SQL statement
    INSERT INTO LOGMNR_DATA
    SELECT *
    FROM   V\$LOGMNR_CONTENTS
    WHERE OPERATION='DELETE'
      AND SQL_REDO LIKE '%7739324493%';

COMMIT;
SYS.DBMS_LOGMNR.END_LOGMNR;

DBMS_OUTPUT.PUT_LINE(SYSDATE || ' : Finished mining ' || REC.NAME);
END LOOP;
END;
/

SET MARKUP HTML ON SPOOL ON
SET TERM OFF
SPOOL ${SPOOLFILE}

TTITLE BOLD LEFT 'LOGMINER OUTPUT OF ACCESS_NUMBER 7739324493 DELETION (RETRIEVED FROM ARCHIVED LOGS)'
SELECT *
FROM LOGMNR_DATA;

SPOOL OFF
_EOF_

echo "" >>${LOGFILE}
echo "`date`: Sending the output to my.mailid@company.com" >> ${LOGFILE}

gzip ${SPOOLFILE}
uuencode ${SPOOLFILE}.gz ${SPOOLFILE}.gz >${SPOOLFILE}.gz.dat
mailx -s "Output of ${THIS_PROC} process" my.mailid@company.com < ${SPOOLFILE}.gz.dat
rm -f ${SPOOLFILE}.gz.dat

echo "" >>${LOGFILE}
echo "Completed ${THIS_PROC} on `date`" >> ${LOGFILE}
echo "************************************************" >>${LOGFILE}

#################################

Week Numbers

2008-05-16T12:26:00.005+05:30

In my previous project, one of the reporting dimensions was week number. There was a lot of churn around this dimension. This is the summary of our discussions.

Four different types of week numbers are being used.

Simple Week
ISO Week
North American Week
Financial Week

Note: We were using Oracle 9i R1.

1. Simple Week

Week 1 is from Jan 01 to Jan 07
Week 2 is from Jan 08 to Jan 14
Week 53 is from Dec 31 to Dec 31 (non-leap years)
Week 53 is from Dec 30 to Dec 31 (leap years)
Can be computed as to_char(date_date,’WW’)
This is the least used one. No issues found.

2. ISO Week

Week Starts with Monday
The first week of a year is the week that contains the first Thursday of a year
An ISO year has 52 or 53 full weeks
Can be computed as to_char(date_date,’IW’)
More details can be found at http://en.wikipedia.org/wiki/ISO_week_date

3. North American Week

Week Starts on Sunday
There might be partial weeks at beginning and ending of the year
The ending week of the year will be 53 or 54
No direct way to compute in Oracle

Can be computed as

ALTER SESSION SET NLS_TERRITORY='AMERICA';
Week Number = TRUNC(( TO_NUMBER(TO_CHAR(sysdate,'ddd'))- (7-TO_NUMBER(TO_CHAR(TRUNC(sysdate,'yyyy'),'d'))+1)-1)/7+2)
Replace sysdate with any valid date

4. Financial Week

Financial Calendar published by respective organizations
No pre-defined algorithm
Organizations need to define the week number for each calendar date

Lessons Learnt - I

2008-05-14T21:25:00.001+05:30

In Oracle 9i R2, I couldn't alter the table definition once the table is compressed. I used to get ORA-22856 error when I tried to add a new column to compressed table. Ultimately, I had to remove compression (by creating uncompressed table from compressed one and dropping the compressed table).
In Oracle 9i R2, it seems there is a limitation on using ANSI syntax. If the combined number of columns in the tables used in 'from' clause exceed some number like 1050, we get ORA-01445 error. I had to disable ANSI syntax in BOXI Universe Designer and rewrite the queries in traditional syntax. Oh! that was a tedious task.
Never use varchar2 or char fields to store date values. In my previous project, our clients used to store date keys as YYYYMMDD (varchar2(8)). When joined to other tables with these date keys, even for few months data, oracle used to opt for full table scans. But when joined with date data type, it worked fine. Tom Kyte talks about this in Chapter 7 of his book "Effective Oracle by Design" (pages 374-377).

Add-ons for Firefox 3 Beta

2008-02-06T21:40:00.001+05:30

Did you like Firefox 3 Beta? I do a lot! But the problem is that many of my existing add-ons are not compatible with this beta release. I am forced to disable them to use Firefox beta. Then I did some research and found a solution!

For example, you are using del.icio.us buttons in Firefox 2. When you upgrade to Firefox 3 beta, it says that the add-on is not compatible with this version and will be disabled. If you try to install the add-on again, it won't allow.

But there is a quick fix! Goto del.icio.us buttons page and right click on 'Install Now', select 'Save Link as' and save the .xpi file. Open this .xpi file with your zip utility (xpi is nothing but a zip file) and edit install.rdf file (The screenshot is from 7zip utility).

Change the value of maxversion parameter to 3.0.0.* from 2.0.0.*

Save the file and update the .xpi file. Now open this .xpi with Firefox 3 beta and the add-on will install without any problem.

This technique works for almost any add-on (unless it explicitly uses new features of Firefox 3 beta).

Oracle 9i TNS Listener Crashes

2008-02-05T07:20:00.001+05:30

I've been running Oracle 9i server on my workstation for quite sometime, without any issues. But when I replaced my machine with IBM Thinkpad T60 laptop, TNS listener never came up. The listener service status seems to up initially, but when I connect to the server using sqlplus or toad, I get "tnslsnr.exe Application Error. The instruction at xxxx referenced memory at xxxx. The memory could not be read" and the service used to abort. Then I came across this post, and saw that my laptop has Venturi Client installed by default. I uninstalled that software and now my TNS listener works just fine.

High Quality Free Software

2007-11-17T05:22:00.001+05:30

Recently I came across a project called OpenDisc. It is a collection of various high quality open source software. I am blogging about it here because that collection contains many of my favorite software. I am going to try out the other software also. You can see the complete list of programs here. Take a look. I bet you will be impressed.

Repetitive Strain Injury

2007-11-16T12:30:00.000+05:30

This term might sound strange to you. But if you are addicted to computer or forced to use computer (like me), then you need to know about it. As usual, complete details can be found in wikipedia, but few words here.

Repetitive Strain Injury (RSI), as the name suggests, is a problem caused by repeated movement of a particular part of the body (hand and neck in our case). I already started feeling its effects. So I decided to do something about it.

The first thing I realized is that I am giving very little exercise for my body. I know that I need to stretch regularly. But once I am into work, I forget about everything (I'm not kidding. One day, before investigating an issue, I noted down the time. When I noticed the time next time, I was shocked to see that I didn't move from my seat for 1 hour and 10 minutes). So I searched for a good reminder software and found Workrave (freeware). This small application is really cute. Try it out and you will love it. I configured it to give a 30 second micro-break for every 10 minutes and a 10 minute rest-break for every one hour and a 9 hour work limit per day. I am feeling a lot better since I started using this tool.

I would say that Workrave is a must for every software engineer.

Password Management

2007-11-16T09:24:00.000+05:30

With Dictionary attacks becoming more sophisticated day by day, it is important for us to make our passwords more difficult to break. The passwords I thought strong two years back are very weak by today's standards. So to adapt to today's demands, we need to shift our mindset from "password" to "passphrase". Instead of thinking of words, think of sentences.

Creating Passphrase

Take your favorite sentence (I take my favorite slokas)
Then take the first two or three letters from each word
Convert some of the letters to Capitals (I use my own algorithm)
Convert some of the letters to numbers or insert some numbers
Make sure that the resulting passpharse is not easy to guess (should not be found in dictionaries)
Check the strength of your password with this tool and make sure that it is strong.
Practise this passphrase at least ten times and make sure that you can recreate it every time without error.

This ensures that you have a strong password by today's standards.

Maintaining Passwords

Then how to maintain these gazillion passwords? Every website requires a password. Some won't accept special characters and for some it's exactly opposite. To add to that, my company has so many services and each of them has its own password and its own rules. Worse enough, some services assign a random password and won't allow me change it. I am fed up with all these and decided to use a password manager. I am skeptical about online services. So I did some research on desktop utilities and zeroed on KeePass.

Advantages of KeePass

Offers maximum security with strong encryption
No installation required
Lightweight
Non-intrusive to regular work
Offers all the features that I require (like auto-fill, password generator etc)

Since I started using KeePass, my life became so easy.

Backup of KeePass Database

But there is a problem with KeePass. The database (an encrypted file which contains all your passwords) is stored locally on your computer. Incase your drive crashes or need to check for a password while you are using another computer is problematic. To overcome this, I use a small utility called Blat to automatically mail my KeePass database to my email id.

I'll blog about Backup in detail later.

Some Tips

Even in KeePass, I won't save my password exactly as it is, at least for highly confidentail passwords (like bank passwords, email passwords etc). Somebody can crack KeePass two years down the line, we never know. So if my password is hrt8Utn, I will write it as hrt (enimidi) Utn in KeePass. Even if the hacker opens my KeePass database, he needs to know that enimidi is Telugu word for 8.

Get Rid of Telemarketing Calls

2007-11-16T08:03:00.000+05:30

Recently I started receiving lot of marketing calls on my cell phone. Annoyingly, many of them are pre-recorded messages (auto dialers). I got fedup and registered my number with Do Not Call Registry.

If you are also in the same situation and are in USA , you can register online in https://www.donotcall.gov/.
If you get marketing calls even after that, you can file a complaint online in http://www.fcc.gov/cgb/complaints.html

If you are in India, this might be helpful.

Returning a Workstation

2007-11-06T11:15:00.000+05:30

My job demands changing projects frequently, so as my workstations. I thought it might be useful to jot down what I am doing before leaving a workstation behind. If interested, read on!

Step 1: Backup

It is important that I take backup of all my work. "How" is not a problem for me (with all these memory drives). The problem is with "What". Identifying what all to backup is a herculean task for me. So I organized myself like this:

When I get new workstation, I create a folder called Ramana and organize everything inside it. So when it comes to backup & cleanup, it becomes very straightforward.
I create a folder called "Custom Program Files" and install all software that I need in that folder. It serves two purposes. Since my account doesn't have admin privileges, I am not allowed to install to regular Program Files folder. Secondly, when it comes to cleanup, I know exactly which software I installed.
I save all temp stuff (like photos, music, downloads etc) in My Documents Folders.
Now there is another set of files, over which I have no control. yes, my personal settings for various software. For them I create a briefcase in my work folder (In Windows XP, File -> New -> Briefcase). Here is a partial list of items I add to the briefcase (This list changes from project to project)

Outlook archive files
Lotus archive files
Yahoo messenger archive file
Skype archive file
Apache cgi-bin folder
Oracle tnsnames.ora
Toad Settings (.ini files found in User Files folder)

Before I take backup, I click on Update All in the briefcase and I am done.

Step 2: Cleanup

Once I made sure that I backed-up everything I need, I proceed to cleanup.

Delete all the folders that have my data. (using Normal Windows Delete Button)
Delete everything from My Documents folder.
Uninstall all the software that I installed.
From Internet explorer, Delete everything (cookies, history, form data etc)
From the briefcase (that I mentioned previously), delete the files from their original locations.
Delete all entries I added to Scheduled Tasks.
Now I need to delete system backups, if any. Goto Programs -> Accessories -> System Tools -> Backup and select Restore. Note down all the backup files and where they are located. Go to those folders and delete all backup files.
Need to check if my IT department is using any other backup policies. I found that they use Thinkvantage Rescue and Recovery. I open the application and see all backup locations. Delete all those backup files.
Finally, clear Recycle Bin.

Step 3: Wipe Out

Do you think I am done? No, not yet. You might be knowing that normal delete just removes the reference to the file from system tables and marks that location as free. Any simple Recovery software can bring all our files back. So I need to make sure that I completely wipe out all the traces of my files.

The following method is going to wipe out all the free space in the disk. It guarantees that the deleted files can not be recovered by any means while not touching existing data. This is useful since my work computers come pre-installed with OS and some standard software (like Microsoft office, Lotus Notes etc) and I am not supposed to delete them.

Install File Shredder (freeware) from http://fileshredder.org/
Shutdown all active applications and run the file shredder program.
Click on "Shred Free Disk Space" and select all disk drives and select algorithm as "Secure erasing algorithm with 7 passes". Then click Start.
Once it is done, click on "Add Folders" and select C:\Documents and Settings\my user id and click on "Shred Now" button.
There might be errors on deleting some files locked by active processes, but I ignore.

That's it. I may not be 100% clean (See this article ), but at least better than doing nothing!!!

Outstanding Questions
How to cleanup the backups that my IT department maintains outside my computer (like on network folder)