Simeon's TechNotes

Human versus Machine Intelligence

2009-11-20T16:14:00.002+11:00

I recently picked up a copy of Kevin Warwick's book titled "I, Cyborg". Kevin can boast of being the world's first truly voluntary cyborg (cybernetic organism, part human and part machine).

An excerpt from the book that I found compelling was facts regarding the comparison of human and machine intelligence. Here are some snippets:

Machines can communicate in parallel, over long distances with little or no error. Humans communicate in serial fashion, over short distances, with enormous errors in message passing.
Machine brains of the future appear to be unlimited in the number of cells and complexity of connections. Human brains will very slowly evolve. In the short term, the number of cells will remain roughly constant.
Machine intelligence is easily networked. The human brain is stand-alone.
Computers can process information in hundreds of dimensions. Humans can only conceive of space as consisting of four dimensions at the most (including time).
Computers can sense the world in a variety of ways. Human senses are relatively poor - there is a lot that goes on that we never know about; e.g.; sensing different spectra.
Computers are fast. The human brain is slow
Computers have good, long-term memory storage capabilities. The human ability to leverage memory capabilities is weak in comparison.

MOSS 2007 Enterprise Features Not Available in the Standard License

2009-11-10T11:05:00.000+11:00

ID	Category	Feature	Description
1	Enterprise Search	Business data search	Search data residing in your line-of-business applications using the Business Data Catalogue. Structured content sources and line-of-business application data and reports accessible through Web services or ADO.NET can be indexed and retrieved through the Business Data Catalogue as search results or into a SharePoint list.
2	Business Intelligence	Integrated, flexible spreadsheet publishing	Office Excel 2007 provides an integrated publishing experience that lets information workers easily choose what they want to share with others and determine how others can interact with published spreadsheets.
3	Business Intelligence	Share, manage, and control spreadsheets	Share business data broadly while maintaining control and helping to protect sensitive information. Provides access to spreadsheet data and analysis through server-calculated, interactive Office Excel spreadsheets from a Web browser. Use these spreadsheets to maintain and efficiently share one centralized version of the truth while helping to protect any sensitive or proprietary information embedded in documents, such as financial models, by limiting access to portions of the spreadsheet and auditing their usage.
4	Business Intelligence	Web-based business intelligence using Excel Services	Excel Services empowers spreadsheet authors to easily and broadly share spreadsheets that use the new business intelligence (BI) functionality through the browser. Fully interactive, data-bound spreadsheets including charts, tables, and PivotTable views can be created as part of a portal, dashboard, or business scorecard, without requiring any development.
5	Business Intelligence	Data Connection Libraries	Data Connection Libraries are centralized SharePoint document libraries that store Office Data Connections (ODC); ODCs describe connections to external data. Centralizing Office Data Connections makes it easier to share, manage, and discover data connections that can be used by any Microsoft Office program.
6	Business Intelligence	Business Data Catalog	The Business Data Catalogue (BDC) tightly integrates external data into the Office SharePoint Server 2007 user experience, providing access to external data residing within backed line-of-business applications, and enabling the display of and interaction with external data through a set of Business Data Web Parts
7	Business Intelligence	Business Data Web Parts	Use SharePoint Business Data Web Parts for viewing lists, entities, and related information retrieved through the Business Data Catalogue.
8	Business Intelligence	Business Data actions	With no custom coding, easily create actions that open Web pages, display the user interfaces of line-of-business applications, launch InfoPath forms, and perform other common tasks. Use the convenient links that appear beside business objects returned from the Business Data Catalogue.
9	Business Intelligence	Integrated business intelligence dashboards	Create rich, interactive BI dashboards that assemble and display business information from disparate sources by using built-in Web parts such as dynamic KPIs, Excel spreadsheets, Microsoft SQL Server Reporting Services reports, or a collection of business data connectivity Web Parts that can visualize information residing in back end line-of-business applications.
10	Business Intelligence	Report Centre	An out-of-the-box site optimized for report access and management, including a report library, data connection library, and a dashboard template. These sites, hosted by the new Report Centre, provide consistent management of reports, spreadsheets, and data connections
11	Business Intelligence	Key performance indicators	KPIs communicate goals and status to drive results. Using the KPI Web Part, a user can create a KPI list within a Web Part page, without writing code. The KPI Web Part can display KPIs from Microsoft SQL Server Analysis Services, Excel spreadsheets, SharePoint lists, or manually entered data.
12	Business Intelligence	Filter Web Parts	Filters enable dashboards to be personalized by communicating shared parameters among Web Parts on a dashboard. The parameters passed can be automatically applied based on user profiles, SharePoint lists, the BDC, manually entered information, and so forth.
13	Forms-driven business processes	Browser-based forms	Microsoft Office InfoPath Forms Services available in Office SharePoint Server 2007 and Microsoft Office Forms Server 2007 makes it possible to design Web-capable forms in Office InfoPath 2007 and distribute them on corporate intranets, extranets, or the Internet. Users can fill out forms in a browser or HTML-enabled mobile device with no download or client components needed.
14	Forms-driven business processes	Centralized forms management and control	Office InfoPath Forms Services provide a administrator controlled centralized form solution catalogue that makes it easy for users to find a form and minimize form solution downtime when upgrades are necessary. It also provides administrators with the tools to manage form solution security and accessibility.
15	Forms-driven business processes	“Design once” development model	Forms designers can design their forms once and deploy them for use both within the rich Office InfoPath 2007 client program and through a Web browser. Office Forms Server 2007 automatically converts the form into ASP.NET Web forms, with no additional work from the designer.
16	Forms-driven business processes	Form Import Wizard	The designer in Office InfoPath 2007 provides an easy way to convert forms designed in Office Excel and Office Word into rich Office InfoPath 2007 forms. The Form Import Wizard handles the conversion of form fields, repeating tables, rich text boxes, and other elements, dynamically generating the underlying XML structure for the new form.
17	Forms-driven business processes	Integrated deployment model for “no-code” forms	The Publish Wizard in Office InfoPath 2007 makes it easy to publish forms that do not have any managed code components to a Windows SharePoint Services library, while making the form available as a browser-based form at the same time.
18	Forms-driven business processes	Compatibility Checker	The Compatibility Checker helps forms designers validate those features that need to work across the broadest range of Web browsers.
19	Platform	Excel spreadsheet developer reuse and extensibility	Spreadsheets published to SharePoint sites are accessible more securely using Web services. This means that application developers can take advantage of the logic embedded in a spreadsheet by remotely accessing spreadsheets from any platform. Business analysts can modify or update the model without affecting the business application or having to involve an application developer in the process.

Google Wave Robot Development for Microsoft-oriented Developers

2009-10-09T21:05:00.006+11:00

Receiving a Google Wave invitation from Google almost 2 months ago provided me with a compelling reason to get started with Google Apps-based development.

If you are a Microsoft-oriented developer, and are attempting to get started with Google Wave development, the following tips may help;

Use Eclipse as your IDE and Java for your development. Integration with the AppEngine from Eclipse is baked in (once the Google Plugin for Eclipse is installed) and very cool.
I would stay away from the Python alternative as I found the AppEngine integration and deployment options relatively fractured and not as smooth. This could, of course, change as Google Wave itself and supporting tools are still in development.
Get started by writing robots for the Wave as this would immediately expose you to the Wave API event model. This helped me immensely.
If you are writing complicated Wave artefacts, keep an eye on memory consumption of the Chrome browser process (if you are using Google Chrome as your browser).

Below is a screenshot of my first Wave Robot that took me no longer than 30 minutes to build, deploy and test when leveraging Eclipse and Java. Exact instructions are available at http://code.google.com/apis/wave/extensions/robots/java-tutorial.html.

Memory Profiling: Determining Application Pool Process ID

2009-10-06T10:57:00.006+11:00

To track the memory usage of an Application Pool in IIS 6.0, we first have to determine the Process ID (PID). The PID can easily be determined using the following steps;

Open a command prompt
Browse to C:\Windows\System32
Run cscript iisapp.vbs

A list of application pools with their relevant process id's will now be displayed. Using Task Manager or Performance Monitor, the memory consumption of the process can now be tracked.

To track the memory usage of an Application Pool in IIS 7.0, we have to go through the same process of determining the Process ID using the following steps,

Open a command prompt
Browse to C:\Windows\System32\inetsrv
As shown below run appcmd list wps

A list of application pools with their relevant process id's will now be displayed in " ".

Comprehensive Linux Command List

2009-09-23T12:16:00.024+10:00

Please note that the following list of Linux commands has been copied from http://www.pixelbeat.org/cmdline.html.

I intend to keep adding useful commands to this list as I grow in my experience of using Ubuntu.

Command		Description
	apropos whatis	Show commands pertinent to string. See also threadsafe
	man -t man \| ps2pdf - > man.pdf	make a pdf of a manual page
	which command	Show full path name of command
	time command	See how long a command takes
	time cat	Start stopwatch. Ctrl-d to stop. See also sw
	nice info	Run a low priority command (The "info" reader in this case)
	renice 19 -p $$	Make shell (script) low priority. Use for non interactive tasks
Directory Navigation
	cd -	Go to previous directory
	cd	Go to $HOME directory
	(cd dir && command)	Go to dir, execute command and return to current dir
	pushd .	Put current dir on stack so you can popd back to it
File Searching
	alias l='ls -l --color=auto'	quick dir listing
	ls -lrt	List files by date. See also newest and find_mm_yyyy
	ls /usr/bin \| pr -T9 -W$COLUMNS	Print in 9 columns to width of terminal
	find -name '*.[ch]' \| xargs grep -E 'expr'	Search 'expr' in this dir and below. See also findrepo
	find -type f -print0 \| xargs -r0 grep -F 'example'	Search all regular files for 'example' in this dir and below
	find -maxdepth 1 -type f \| xargs grep -F 'example'	Search all regular files for 'example' in this dir
	find -maxdepth 1 -type d \| while read dir; do echo $dir; echo cmd2; done	Process each item with multiple commands (in while loop)
	find -type f ! -perm -444	Find files not readable by all (useful for web site)
	find -type d ! -perm -111	Find dirs not accessible by all (useful for web site)
	locate -r 'file[^/]*\.txt'	Search cached index for names. This re is like glob file.txt
	look reference	Quickly search (sorted) dictionary for prefix
	grep --color reference /usr/share/dict/words	Highlight occurances of regular expression in dictionary
Archives and Compression
	gpg -c file	Encrypt file
	gpg file.gpg	Decrypt file
	tar -c dir/ \| bzip2 > dir.tar.bz2	Make compressed archive of dir/
	bzip2 -dc dir.tar.bz2 \| tar -x	Extract archive (use gzip instead of bzip2 for tar.gz files)
	tar -c dir/ \| gzip \| gpg -c \| ssh user@remote 'dd of=dir.tar.gz.gpg'	Make encrypted archive of dir/ on remote machine
	find dir/ -name '*.txt' \| tar -c --files-from=- \| bzip2 > dir_txt.tar.bz2	Make archive of subset of dir/ and below
	find dir/ -name '*.txt' \| xargs cp -a --target-directory=dir_txt/ --parents	Make copy of subset of dir/ and below
	( tar -c /dir/to/copy ) \| ( cd /where/to/ && tar -x -p )	Copy (with permissions) copy/ dir to /where/to/ dir
	( cd /dir/to/copy && tar -c . ) \| ( cd /where/to/ && tar -x -p )	Copy (with permissions) contents of copy/ dir to /where/to/
	( tar -c /dir/to/copy ) \| ssh -C user@remote 'cd /where/to/ && tar -x -p'	Copy (with permissions) copy/ dir to remote:/where/to/ dir
	dd bs=1M if=/dev/sda \| gzip \| ssh user@remote 'dd of=sda.gz'	Backup harddisk to remote machine
rsync (Network efficient file copier: Use the --dry-run option for testing)
	rsync -P rsync://rsync.server.com/path/to/file file	Only get diffs. Do multiple times for troublesome downloads
	rsync --bwlimit=1000 fromfile tofile	Locally copy with rate limit. It's like nice for I/O
	rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html'	Mirror web site (using compression and encryption)
	rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/	Synchronize current directory with remote one
ssh (Secure SHell)
	ssh $USER@$HOST command	Run command on $HOST as $USER (default command=shell)
	ssh -f -Y $USER@$HOSTNAME xeyes	Run GUI command on $HOSTNAME as $USER
	scp -p -r $USER@$HOST: file dir/	Copy with permissions to $USER's home directory on $HOST
	ssh -g -L 8080:localhost:80 root@$HOST	Forward connections to $HOSTNAME:8080 out to $HOST:80
	ssh -R 1434:imap:143 root@$HOST	Forward connections from $HOST:1434 in to imap:143
wget (multi purpose download tool)
	(cd dir/ && wget -nd -pHEKk http://www.pixelbeat.org/cmdline.html)	Store local browsable version of a page to the current dir
	wget -c http://www.example.com/large.file	Continue downloading a partially downloaded file
	wget -r -nd -np -l1 -A '*.jpg' http://www.example.com/dir/	Download a set of files to the current directory
	wget ftp://remote/file[1-9].iso/	FTP supports globbing directly
	wget -q -O- http://www.pixelbeat.org/timeline.html \| grep 'a href' \| head	Process output directly
	echo 'wget url' \| at 01:00	Download url at 1AM to current dir
	wget --limit-rate=20k url	Do a low priority download (limit to 20KB/s in this case)
	wget -nv --spider --force-html -i bookmarks.html	Check links in a file
	wget --mirror http://www.example.com/	Efficiently update a local copy of a site (handy from cron)
Networking (Note ifconfig, route, mii-tool, nslookup commands are obsolete)
	ethtool eth0	Show status of ethernet interface eth0
	ethtool --change eth0 autoneg off speed 100 duplex full	Manually set ethernet interface speed
	iwconfig eth1	Show status of wireless interface eth1
	iwconfig eth1 rate 1Mb/s fixed	Manually set wireless interface speed
	iwlist scan	List wireless networks in range
	ip link show	List network interfaces
	ip link set dev eth0 name wan	Rename interface eth0 to wan
	ip link set dev eth0 up	Bring interface eth0 up (or down)
	ip addr show	List addresses for interfaces
	ip addr add 1.2.3.4/24 brd + dev eth0	Add (or del) ip and mask (255.255.255.0)
	ip route show	List routing table
	ip route add default via 1.2.3.254	Set default gateway to 1.2.3.254
	tc qdisc add dev lo root handle 1:0 netem delay 20msec	Add 20ms latency to loopback device (for testing)
	tc qdisc del dev lo root	Remove latency added above
	host pixelbeat.org	Lookup DNS ip address for name or vice versa
	hostname -i	Lookup local ip address (equivalent to host `hostname`)
	whois pixelbeat.org	Lookup whois info for hostname or ip address
	netstat -tupl	List internet services on a system
	netstat -tup	List active connections to/from system
Windows Networking (Note samba is the package that provides all this windows specific networking support)
	smbtree	Find windows machines. See also findsmb
	nmblookup -A 1.2.3.4	Find the windows (netbios) name associated with ip address
	smbclient -L windows_box	List shares on windows machine or samba server
	mount -t smbfs -o fmask=666,guest //windows_box/share /mnt/share	Mount a windows share
	echo 'message' \| smbclient -M windows_box	Send popup to windows machine (off by default in XP sp2)
Text Manipulation (Note sed uses stdin and stdout. Newer versions support inplace editing with the -i option)
	sed 's/string1/string2/g'	Replace string1 with string2
	sed 's/$.*$1/\12/g'	Modify anystring1 to anystring2
	sed '/ #/d; /^ $/d'	Remove comments and blank lines
	sed ':a; /\\$/N; s/\\\n//; ta'	Concatenate lines with trailing \
	sed 's/[ \t]*$//'	Remove trailing spaces from lines
	sed 's/$[`"$\]$/\\\1/g'	Escape shell metacharacters active within double quotes
	seq 10 \| sed "s/^/ /; s/ *$.\{7,\}$/\1/"	Right align numbers
	sed -n '1000{p;q}'	Print 1000th line
	sed -n '10,20p;20q'	Print lines 10 to 20
	sed -n 's/.*	Extract title from HTML web page
	sed -i 42d ~/.ssh/known_hosts	Delete a particular line
	sort -t. -k1,1n -k2,2n -k3,3n -k4,4n	Sort IPV4 ip addresses
	echo 'Test' \| tr '[:lower:]' '[:upper:]'	Case conversion
	tr -dc '[:print:]' < /dev/urandom	Filter non printable characters
	history \| wc -l	Count lines
Set Operations (Note you can export LANG=C for speed. Also these assume no duplicate lines within a file)
	sort file1 file2 \| uniq	Union of unsorted files
	sort file1 file2 \| uniq -d	Intersection of unsorted files
	sort file1 file1 file2 \| uniq -u	Difference of unsorted files
	sort file1 file2 \| uniq -u	Symmetric Difference of unsorted files
	join -t'\0' -a1 -a2 file1 file2	Union of sorted files
	join -t'\0' file1 file2	Intersection of sorted files
	join -t'\0' -v2 file1 file2	Difference of sorted files
	join -t'\0' -v1 -v2 file1 file2	Symmetric Difference of sorted files
Math
	echo '(1 + sqrt(5))/2' \| bc -l	Quick math (Calculate φ). See also bc
	echo 'pad=20; min=64; (10010^6)/((pad+min)8)' \| bc	More complex (int) e.g. This shows max FastE packet rate
	echo 'pad=20; min=64; print (100E6)/((pad+min)*8)' \| python	Python handles scientific notation
	echo 'pad=20; plot [64:1518] (100106)/((pad+x)8)' \| gnuplot -persist	Plot FastE packet rate vs packet size
	echo 'obase=16; ibase=10; 64206' \| bc	Base conversion (decimal to hexadecimal)
	echo $((0x2dec))	Base conversion (hex to dec) ((shell arithmetic expansion))
	units -t '100m/9.58s' 'miles/hour'	Unit conversion (metric to imperial)
	units -t '500GB' 'GiB'	Unit conversion (SI to IEC prefixes)
	units -t '1 googol'	Definition lookup
	seq 100 \| (tr '\n' +; echo 0) \| bc	Add a column of numbers. See also add and funcpy
Calendar
	cal -3	Display a calendar
	cal 9 1752	Display a calendar for a particular month year
	date -d fri	What date is it this friday. See also day
	[ $(date -d "tomorrow" +%d) = "01" ] \|\| exit	exit a script unless it's the last day of the month
	date --date='25 Dec' +%A	What day does xmas fall on, this year
	date --date='@2147483647'	Convert seconds since the epoch (1970-01-01 UTC) to date
	TZ=':America/Los_Angeles' date	What time is it on West coast of US (use tzselect to find TZ)
	echo "mail -s 'get the train' P@draigBrady.com < /dev/null" \| at 17:45	Email reminder
	echo "DISPLAY=$DISPLAY xmessage cooker" \| at "NOW + 30 minutes"	Popup reminder
Locales
	printf "%'d\n" 1234	Print number with thousands grouping appropriate to locale
	BLOCK_SIZE=\'1 ls -l	get ls to do thousands grouping appropriate to locale
	echo "I live in `locale territory`"	Extract info from locale database
	LANG=en_IE.utf8 locale int_prefix	Lookup locale info for specific country. See also ccodes
	locale \| cut -d= -f1 \| xargs locale -kc \| less	List fields available in locale database
recode (Obsoletes iconv, dos2unix, unix2dos)
	recode -l \| less	Show available conversions (aliases on each line)
	recode windows-1252.. file_to_change.txt	Windows "ansi" to local charset (auto does CRLF conversion)
	recode utf-8/CRLF.. file_to_change.txt	Windows utf8 to local charset
	recode iso-8859-15..utf8 file_to_change.txt	Latin9 (western europe) to utf8
	recode ../b64 < file.txt > file.b64	Base64 encode
	recode /qp.. < file.txt > file.qp	Quoted printable decode
	recode ..HTML < file.txt > file.html	Text to HTML
	recode -lf windows-1252 \| grep euro	Lookup table of characters
	echo -n 0x80 \| recode latin-9/x1..dump	Show what a code represents in latin-9 charmap
	echo -n 0x20AC \| recode ucs-2/x2..latin-9/x	Show latin-9 encoding
	echo -n 0x20AC \| recode ucs-2/x2..utf-8/x	Show utf-8 encoding
CDs
	gzip < /dev/cdrom > cdrom.iso.gz	Save copy of data cdrom
	mkisofs -V LABEL -r dir \| gzip > cdrom.iso.gz	Create cdrom image from contents of dir
	mount -o loop cdrom.iso /mnt/dir	Mount the cdrom image at /mnt/dir (read only)
	cdrecord -v dev=/dev/cdrom blank=fast	Clear a CDRW
	gzip -dc cdrom.iso.gz \| cdrecord -v dev=/dev/cdrom -	Burn cdrom image (use dev=ATAPI -scanbus to confirm dev)
	cdparanoia -B	Rip audio tracks from CD to wav files in current dir
	cdrecord -v dev=/dev/cdrom -audio *.wav	Make audio CD from all wavs in current dir (see also cdrdao)
	oggenc --tracknum='track' track.cdda.wav -o 'track.ogg'	Make ogg file from wav file
Disk Space (See also FSlint)
	ls -lSr	Show files by size, biggest last
	du -s * \| sort -k1,1rn \| head	Show top disk users in current dir. See also dutop
	df -h	Show free space on mounted filesystems
	df -i	Show free inodes on mounted filesystems
	fdisk -l	Show disks partitions sizes and types (run as root)
	rpm -q -a --qf '%10{SIZE}\t%{NAME}\n' \| sort -k1,1n	List all packages by installed size (Bytes) on rpm distros
	dpkg-query -W -f='${Installed-Size;10}\t${Package}\n' \| sort -k1,1n	List all packages by installed size (KBytes) on deb distros
	dd bs=1 seek=2TB if=/dev/null of=ext3.test	Create a large test file (taking no space). See also truncate
	> file	truncate data of file or create an empty file
Monitoring/Debugging
	tail -f /var/log/messages	Monitor messages in a log file
	strace -c ls >/dev/null	Summarise/profile system calls made by command
	strace -f -e open ls >/dev/null	List system calls made by command
	ltrace -f -e getenv ls >/dev/null	List library calls made by command
	lsof -p $$	List paths that process id has open
	lsof ~	List processes that have specified path open
	tcpdump not port 22	Show network traffic except ssh. See also tcpdump_not_me
	ps -e -o pid,args --forest	List processes in a hierarchy
	ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu \| sed '/^ 0.0 /d'	List processes by % cpu usage
	ps -e -orss=,args= \| sort -b -k1,1n \| pr -TW$COLUMNS	List processes by mem usage. See also ps_mem.py
	ps -C firefox-bin -L -o pid,tid,pcpu,state	List all threads for a particular process
	ps -p 1,2	List info for particular process IDs
	last reboot	Show system reboot history
	free -m	Show amount of (remaining) RAM (-m displays in MB)
	watch -n.1 'cat /proc/interrupts'	Watch changeable data continuously
System Information (see also sysinfo) ('#' means root access is required)
	uname -a	Show kernel version and system architecture
	head -n1 /etc/issue	Show name and version of distribution
	cat /proc/partitions	Show all partitions registered on the system
	grep MemTotal /proc/meminfo	Show RAM total seen by the system
	grep "model name" /proc/cpuinfo	Show CPU(s) info
	lspci -tv	Show PCI info
	lsusb -tv	Show USB info
	mount \| column -t	List mounted filesystems on the system (and align output)
	grep -F capacity: /proc/acpi/battery/BAT0/info	Show state of cells in laptop battery
#	dmidecode -q \| less	Display SMBIOS/DMI information
#	smartctl -A /dev/sda \| grep Power_On_Hours	How long has this disk (system) been powered on in total
#	hdparm -i /dev/sda	Show info about disk sda
#	hdparm -tT /dev/sda	Do a read speed test on disk sda
#	badblocks -s /dev/sda	Test for unreadable blocks on disk sda
Interactive (see also linux keyboard shortcuts)
	readline	Line editor used by bash, python, bc, gnuplot, ...
	screen	Virtual terminals with detach capability, ...
	mc	Powerful file manager that can browse rpm, tar, ftp, ssh, ...
	gnuplot	Interactive/scriptable graphing
	links	Web browser
	xdg-open .	open a file or url with the registered desktop application
Miscellaneous
	alias hd='od -Ax -tx1z -v'	Handy hexdump. (usage e.g.: • hd /proc/self/cmdline \| less)
	alias realpath='readlink -f'	Canonicalize path. (usage e.g.: • realpath ~/../$USER)
	set \| grep $USER	Search current environment
	touch -c -t 0304050607 file	Set file timestamp (YYMMDDhhmm)
	python -m SimpleHTTPServer	Serve current directory tree at http://$HOSTNAME:8000/

Reverting KDE to GDE in Ubuntu Jaunty

2009-09-22T21:21:00.003+10:00

Being a recent convert to Linux, I installed the KDE desktop on Ubuntu.

However, I did not like KDE much and then decided to revert back to the default Gnome Desktop Environment. To revert desktops, I found the easiest way to run the following command at console login;

> sudo dpkg-reconfigure gdm

The next step is to remove the complete K Desktop Environment. Because several other packages are installed as part of KDE, the following command removes all of them;

> sudo apt-get remove akregator amarok amarok-common apport-qt ark cdrdao dolphin dontzap dragonplayer exiv2 foomatic-db-gutenprint gdebi-kde gnupg-agent gtk2-engines-qtcurve gwenview hpijs-ppds ijsgutenprint install-package jockey-kde k3b k3b-data kaddressbook kamera kate kde-icons-oxygen kde-printer-applet kde-style-qtcurve kde-window-manager kde-zeroconf kdebase-bin kdebase-data kdebase-plasma kdebase-runtime kdebase-runtime-bin-kde4 kdebase-runtime-data kdebase-runtime-data-common kdebase-workspace-bin kdebase-workspace-data kdebase-workspace-libs4+5 kdebluetooth kdegraphics-strigi-plugins kdelibs-bin kdelibs-data kdelibs4c2a kdelibs5 kdelibs5-data kdemultimedia-kio-plugins kdepasswd kdepim-kresources kdepim-strigi-plugins kdepim-wizards kdepimlibs-data kdepimlibs5 kdeplasma-addons kdeplasma-addons-data kdesudo kdm kfind khelpcenter4 klipper kmag kmail kmix kmousetool knotes konqueror konqueror-nsplugins konqueror-plugin-searchbar konsole kontact kopete korganizer kpackagekit krdc krfb ksnapshot ksysguard ksysguardd ksystemlog ktimetracker ktorrent ktorrent-data kubuntu-artwork-usplash kubuntu-default-settings kubuntu-desktop kubuntu-docs kubuntu-konqueror-shortcuts kuser kvkbd kwalletmanager language-selector-qt libakonadiprivate1 libao2 libaudio2 libavahi-qt3-1 libboost-program-options1.35.0 libclucene0ldbl libdbus-qt-1-1c2 libeet1 libexiv2-5 libflac++6 libgeoip1 libk3b3 libk3b3-extracodecs libkcddb4 libkdecorations4 libkdepim4 libkexiv2-7 libkholidays4 libkipi6 libkleo4 libkonq5 libkonq5-templates libkpgp4 libksieve4 libkwineffects1 libloudmouth1-0 liblua50 liblualib50 libmad0 libmimelib4 libmodplug0c2 libmpcdec3 libmsn0.1 libmysqlclient15off libokularcore1 libpackagekit-glib11 libpackagekit-qt11 libphonon4 libplasma3 libpoppler-qt4-3 libpq5 libqca2 libqca2-plugin-ossl libqedje0 libqimageblitz4 libqt3-mt libqt4-assistant libqt4-core libqt4-dbus libqt4-designer libqt4-help libqt4-network libqt4-opengl libqt4-qt3support libqt4-script libqt4-sql libqt4-sql-mysql libqt4-sql-sqlite libqt4-svg libqt4-test libqt4-webkit libqt4-xml libqt4-xmlpatterns libqtcore4 libqtgui4 libqzion0 libraptor1 librasqal1 librdf0 libsearchclient0 libsoprano4 libstreamanalyzer0 libstreams0 libstrigihtmlgui0 libstrigiqtdbusclient0 libvncserver0 libxcb-shape0 libxcb-shm0 libxcb-xv0 libxine1 libxine1-bin libxine1-console libxine1-misc-plugins libxine1-x libzip1 mysql-common okular okular-extra-backends openoffice.org-kde openoffice.org-style-crystal oxygen-cursor-theme packagekit packagekit-backend-apt phonon phonon-backend-xine pinentry-gtk2 pinentry-qt4 plasma-widget-network-manager plasma-widget-quickaccess python-dev python-kde4 python-packagekit python-plasma python-qt4 python-qt4-common python-qt4-dbus python-sip4 python2.6-dev qt4-qtconfig quassel quassel-data raptor-utils redland-utils software-properties-kde soprano-daemon speedcrunch strigi-client strigi-daemon system-config-printer-kde systemsettings ttf-dejavu ttf-dejavu-extra update-manager-kde update-notifier-kde && sudo apt-get install ubuntu-desktop

Wolfram Alpha and Computational Search

2009-08-22T13:36:00.012+10:00

I was honestly more excited about the launch of Wolfram Alpha (http://www.wolframalpha.com/) than Microsoft Bing in May this year. It must be noted that Wolfram Alpha is a computational engine and the service that Wolfram provides is very different from Google or Microsoft search.

Stephen Wolfram, the founder of Mathematica led the effort to devise a strategy that enables the Wolfram Alpha engine to understand the language and interpretive intricacies of the web as compared to the more typical semantic tagging approach. Though not on the same scale, Wolfram's efforts in the present day are very similar to an ancient Sanskrit grammarian named Pāṇini (पाणिन) from the Indian subcontinent who lived around 27 B.C.E and produced 3,959 sutras or rules of the Sanskrit grammar. Pāṇini essentially came up with formal rules to describe Sanskrit grammar and used these rules to compute poetic forms in Sanskrit. Wolfram's task of attempting to interpret the "language of the web" and making all available data computable by leveraging models, is much more arduous in terms of scale and this is why I am fascinated by the efforts of Wolfram Research.

The best way to understand how the computational knowledge engine differs from existing search engines is to run a few queries . As an extremely trivial example, I typed in "Australia India" and the below results were returned in Wolfram Alpha. A similar query in either Google or Bing led to a whole bunch of search engine results that were not relevant (purely) in context of quantifying comparative (computational) statistics of the two countries.

Upgrading to SharePoint 2010

2009-07-11T07:58:00.017+10:00

I attended a very insightful event held by Quest Software last week in Sydney. The first two sessions in the event were hosted by a shining star in the global SharePoint community; Joel Oleson or SharePointJoel who used to work for MS but now works with Quest. The best take-away from the event was Joel's advice on preparation required to upgrade a current SharePoint 2007 deployment to the next version of the platform - SharePoint 2010.

I wanted to share my notes from the event, including comments of my own, as I believe this could be quite useful to members of the community who did not get a chance to attend. There are, of course, no warranties attached to this material. I will update this blog of any new learnings gained from the Tech Preview release in the upcoming weeks.

SharePoint 2010 H/W and S/W Requirements

Only runs on 64-bit H/W
Application server platform must be Windows Server 2008. Does not run on Windows Server 2003.
Database server platform must be SQL Server 2005 or 2008. Will not support SQL Server 2000.
Internet Explorer 6.0 browser client is not supported. Browser client must be IE 7+, FireFox 3.11+ and Safari (version compatibility unknown but will test and update).
Office client does not have to be upgraded.

I found the mandatory 64-bit hardware requirement an interesting one because of the obvious costs to the enterprise. Also, several organisations that I work with still use IE 6.0 on the desktop, so there obviously has to be an SOE upgrade within the enterprise if SharePoint 2010 is to be used (another good case for application virtualisation, maybe?).

Considerations with Planning an Upgrade to SharePoint 2010

Organisations who have architected their current SharePoint 2007 Site Collections intelligently by assigning each Site Collection to a separate content database will be able to shorten their upgrade window as parallel content database upgrades are now supported.
Any Site Collections with more than 1800 windows security principals should be reviewed and reduced prior to migration. Note that "Domain\User" and "Domain\Group" constitute one Security Principal each. Any number of AD users can exist in the one "Domain\Group" and only one Security Principal object is used.
Locked Sites must be reviewed and unlocked prior to the upgrade. As you are probably aware, a site is locked automatically when it reaches it's storage quota limit. Additionally, a SharePoint admin can also lock a site. Locked sites are read-only sites. It is a good idea to review Site Collection quotas at this stage as well.
SharePoint information architects will need to review any Lists that hold more than 5000 items in indexed columns.
stsadm's backup and restore tools work well for content databases at or below the 15GB limit. Use database detach/attach for any content database that surpasses this.

Recommendations to Prepare an Upgrade Plan:
After the considerations above have been addressed, I.T. teams may want to consider the following upgrade steps:

Document the current SharePoint 2007 environment
Review all hardware and software assets (including 3rd party) in the SharePoint Solution with an emphasis on cost (don't forget to review your current licensing model).
Identify all customizations made to the SharePoint 2007 solution if you don't use SharePoint out-of-the-box in the Commodity Hosting model. I do believe that upgrading customizations could potentially be the biggest "gotcha". Joel's recommendation is to work with the 3rd party vendors to identify upgrade releases for their products (if any). If your developers have been involved in customizations, then ensure there is a clear plan to have an early code and feature compatibility review with a partner who is on the Tech Preview programme.
The following tools may be used to identify customizations and issues affecting the upgrade - WinDiff, Beyond Compare, SP Diagnostics tool and PreUpgradeCheck.
Upgrade environment to SharePoint 2007 Service Pack 2 or later (the PreUpgradeCheck tool mentioned above needs SP2+ as a pre-requisite).
Review any deployment of the Fantastic Forty Administrator templates. These could cause an issue with the upgrade and will be rejected by the PreUpgradeCheck tool.

Developer Preparation

Create a Windows Server 2008, 64-bit version copy of your current SharePoint 2007 environment. Ensure that SharePoint 2007 SP2 is installed.
Run the PreUpgradeCheck tool on this environment.
Address issues outlined in the above two sections.
Test the upgrade when the Tech Preview is released with a trusted partner (only certain Microsoft Partners get access to the Tech Preview).
Close any gaps in entities that could not be upgraded and build a strategy around this.
If your management definitely wants to upgrade, ensure that the development team are aware of what "not to do" in the coming months while your current SharePoint 2007 installation evolves.

New Trend Pushing Further Decommodization of Business Intelligence

2009-06-24T10:17:00.017+10:00

For several months I've noticed a new trend in Business Intelligence that I was finding hard to concisely articulate. Some research on Google led me to the right buzzwords - "Information Post-Discovery". Information Post-Discovery is the most exciting shift at the forefront of the amalgamation of Online Analytical Processing (OLAP) and Enterprise Search.

"Pre-discovery" of information typically relates to the design phase of a BI project that might include extensive modelling of data structures, the design of data extract definitions and ETL data contracts. These design processes are employed by the architect to solely understand the scope of data to be stored in the OLAP system and ensure that a justifiable outcome is delivered. As most seasoned BI professionals are aware, the Planning stage in any BI project (in which the above design artefacts are delivered) is the most time consuming.

Enterprise Search products such as Microsoft FAST Search, Attivio (led by Ali Riaz former director of FAST Search) and Endeca offer a new storage structure that leverages inverted indexes a.k.a. n-gram indexes at the basic level. These products are designed to identify and present user-explorable relationships in structured and unstructured data that can be seamlessly fed to the engine via connectors. Information is therefore "post-discovered". The "Navigator" mechanism in FAST ESP is one such example where users are driven to explore relationships in information that they did not know existed.

While I do not believe there will be an immediate, major shift to post-discovery systems, these systems certainly present a very compelling proposition to I.T. decision makers who struggle to enable transparency across enterprise information silos.

Risks in Cloud Computing

2009-06-20T11:38:00.015+10:00

I've had several discussions with eager colleagues recently who've wanted to start leveraging cloud topologies in their enterprise designs. While I'm very interested in the underlying technology supporting cloud-based architectures like Apache Hadoop or Microsoft Dryad, I would personally think twice (as of this writing) before recommending a cloud-based solution for the enterprise. Here are some of my concerns:

Data Retention: I believe that at the present moment, more than ever, architects should not ignore the fiduciary needs of data storage. While legal data retention policies vary geographically, cloud architectures can be geographically dispersed and should therefore automate data retention by relevant locale-specific policy selection. The issue of data retention becomes even more interesting when an enterprise is dispersed across multiple jurisdictions like the multinationals and a single line of business system (whose data resides in the cloud) is used. There are sometimes paradoxical legislations that govern data storage and access like International Traffic in Arms Regulations (ITAR) in the United States that prevents some manufacturing firms from storing product data outside the country, while the USA Patriot Act may allow the U.S. government access to any information stored within its borders. The question is; if the cloud vendor is U.S.-based does this mean that an Australian (for example) company's data could potentially be exposed to the U.S. government based on local U.S. legislation?
Data Centre Evaluation: If I was having my company's data stored offsite, then I would want to personally evaluate the data centre used by the cloud vendor to identify proper SLA adherence. I have not heard of cloud computing vendors permitting regular customer visits to their premises to monitor adherence.
Complete (Meta)Data Ownership: A piece of data stored in an application hosted on the cloud, has, in most cases, several bits of meta data associated with it. These pre-requisite data elements (like primary keys identifiers for example) may be coupled only to the cloud-based architecture. In the case that an enterprise wants to migrate 'off' the cloud platform then the process of reconciling related metadata could be tedious and exhaustive. SLA's need to be reviewed thoroughly before committing to hosted applications in the cloud. Middleware cloud computing services like Elastra and RightScale offer possible options to manage this scenario.
Data Privacy: I belive this is the biggest obstacle to adoption of a cloud computing platform. I am not sure if the privacy issue is ever going to be definitively addressed.

Zenta's Brilliant Ant Inspired Hexapod Robot

2009-04-13T09:24:00.004+10:00

Ace roboticist Kare Halvorsen's (aka Zenta) new hexapod creation has created a stir in the hobbyist robotics community recently. The robot demonstrates super fluid motion with a 2 degrees of freedom tail, a 3 degrees of freedom head and 3 degrees of freedom attached to each of the hexapod's 6 legs.

Irrespective of the robot being remote controlled, the below video made my jaw drop.

Congratulations on a brilliant creation, Zenta!!!

Formula for Calculating Day of Week in MS Office

2009-03-08T20:54:00.005+11:00

This formula applies to SharePoint 2007 too. Remember to replace [DATE] below with your date point of reference. Also note that this formula applies to locales where Sunday is considered to be the first day of the week.


=CHOOSE(MOD([Date],7),"Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday")

Business Reporting Standardisation using XBRL

2009-03-07T08:56:00.010+11:00

Extensible Business Reporting Language (XBRL) is a XML specification that is being used (and continuously developed) by a non-profit consortium of roughly 450 major companies around the world for exchange of business reporting data. The reporting data can include, but is not limited to financial statements, financial information, general ledger transactions, regulatory filings (such as annual and quarterly reports). Non financial information that may have relevance to the context of financial data being described can also be included.

The clear value proposition of using XBRL is articulated on XBRL.org - "Companies can use XBRL to save costs and streamline their processes for collecting and reporting financial information. Consumers of financial data, including investors, analysts, financial institutions and regulators, can receive, find, compare and analyse data much more rapidly and efficiently if it is in XBRL format."

There is a strong drive in Australia to adopt XBRL schemas and taxonomies as standard means for business financial entity reporting.

The current version of the standard is available for review at http://www.xbrl.org/Specification/XBRL-RECOMMENDATION-2003-12-31+Corrected-Errata-2008-07-02.htm.

Queries to retrieve table and column definitions in SQL Server

2009-02-18T11:22:00.008+11:00

T-SQL script to dynamically retrieve column information in SQL Server. Replace 'YourTableName' below with the table name of interest (whose definition you wish to retrieve).


SELECT OBJECT_name(c.object_id) AS Table_Name, 
      c.name AS Column_Name,
      t.name AS Column_Data_Type,
      c.max_length AS Column_Data_Type_Length
FROM sys.columns AS c, sys.types AS t 
WHERE c.user_type_id=t.user_type_id
AND OBJECT_name(c.object_id) = 'YourTableName'
ORDER BY c.column_id

.... and in the event that you wish to retrieve a list of all tables in SQL Server;


–---- All tables
EXEC sp_tables 

–---- Alternatively, if you need to filter further
SELECT Owner = TABLE_SCHEMA, TableName = TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = ‘BASE TABLE’
AND OBJECTPROPERTY(OBJECT_ID(TABLE_NAME), ‘IsMsShipped’) = 0 ORDER BY TABLE_SCHEMA, TABLE_NAME

SharePoint 2007 - Custom Central Admin Sections and Site Settings Links

2009-01-25T18:49:00.024+11:00

It is very easy to tailor the SharePoint 2007 interface for custom development as shown below.

In this example I intend to add an administration section in Central Administration -> Application Management and a custom link to a site’s Site Settings ->Site Collection Administration sections.

Step 1: Create Central Administration Targeted Feature
I first created a directory at $\Program Files\Common Files\microsoft shared\Web Server Extensions\12\TEMPLATE\FEATURES\SimeonLoboAdminFeatures and added the below files to this directory:

elements.xml

feature.xml

Step 2: Create Site Settings Targeted Feature
I then created a directory at $\Program Files\Common Files\microsoft shared\Web Server Extensions\12\TEMPLATE\FEATURES\SimeonLoboSiteFeatures and added the below files to this directory:

elements.xml

feature.xml

Step 3: Run the following commands to install and activate the above features
C:\>path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\12\BIN"

C:\>STSADM.EXE -o installfeature -filename SimeonLoboAdminFeatures\feature.xml -force

C:\>STSADM.EXE -o activatefeature -filename SimeonLoboAdminFeatures\feature.xml -url http://YourTargetSite:YourTargetPort -force

C:\>STSADM.EXE -o installfeature -filename SimeonLoboSiteFeatures\feature.xml -force

C:\>STSADM.EXE -o activatefeature -filename SimeonLoboSiteFeatures\feature.xml -url http://YourTargetSite:YourTargetPort -force

C:\>iisreset

Step 4: Browse to Central Administration -> Application Management and verify that you have a section that appears as shown below

Step 5: Browse to the Site Settings section of your targeted site for SimeonLoboSiteFeatures and verify that you have a link that appears as shown below

The Location and GroupId attributes in the <CUSTOMACTION> node in elements.xml file allows you to target various functional entities in SharePoint. The following table lists the various targetings possible:

Area	Location	GroupID
Display form toolbar	DisplayFormToolbar	N/A
Edit form toolbar	EditFormToolbar	N/A
New form toolbar	NewFormToolbar	N/A
List view toolbar	ViewToolbar	N/A
List item menu	EditControlBlock	N/A
Library/List New menu	Microsoft.SharePoint.StandardMenu	NewMenu
Library/List Actions menu	Microsoft.SharePoint.StandardMenu	ActionsMenu
Library/List Settings menu	Microsoft.SharePoint.StandardMenu	SettingsMenu
Library Upload menu	Microsoft.SharePoint.StandardMenu	UploadMenu
Site Actions menu	Microsoft.SharePoint.StandardMenu	SiteActions
Site Settings page, Site collection administration links	Microsoft.SharePoint.SiteSettings	SiteCollectionAdmin
Site Settings page, Site administration links	Microsoft.SharePoint.SiteSettings	SiteAdministration
Site Settings page, Galleries links	Microsoft.SharePoint.SiteSettings	Galleries
Site Settings page, Look and feel links	Microsoft.SharePoint.SiteSettings	Customization
Site Settings page, Users and permissions links	Microsoft.SharePoint.SiteSettings	UsersAndPermissions
Site Actions menu for surveys	Microsoft.SharePoint.StandardMenu	ActionsMenuForSurvey
Site Settings page, links for surveys	Microsoft.SharePoint.SiteSettings	SettingsMenuForSurvey
Content type settings links	Microsoft.SharePoint.ContentTypeSettings	N/A
Central Admin Operations Page	Microsoft.SharePoint.Administration.Operations	N/A
Central Admin Application Management Page	Microsoft.SharePoint.Administration.ApplicationManagement	N/A

Risk Mitigation in I.T. Outsourcing Strategies

2009-01-21T05:36:00.007+11:00

The scandal enveloping Satyam, India’s 4th largest technology company has made several organisational I.T. policy makers queasy (to put it mildly). As I have been responsible for insourcing and outsourcing decisions in my career, the latest developments in India have forced me to contemplate the actual value proposition that outsourcing provides while taking into account contractual risk mitigation. While an organisation may secure its outsourcing contracts via back-to-back service level agreements, the very prospect of almost immediate, multiple project failures and internal/external client dissatisfaction is a decision maker’s worst nightmare. Proponents of outsourcing also have to battle with their conscience about the moral justifications of sending work overseas. Scandals such as these help reaffirm the conviction that maybe insourcing is a better model.

While browsing the topic, one restless night, I happened to find an article that provided some good advice regarding outsourcing. The advice below is from Fujitsu Network Communications SVP and CIO, Jeff Meier. From experience I found that many of Jeff’s pointers ring clear and true:

Perform due diligence on determining whether outsourcing is right for your company. Tie your sourcing strategy to the company's business strategy. Make sure your IT organization has a sourcing plan.
Fully understand how outsourcing will affect the current IT organization's employee morale and productivity. Also, will a potential outsourcing relationship impact current customer relationships?
Obtain upper management support by creating a decision-making framework based on your sourcing strategy. This is achieved by having an accurate understanding of your IT organization's total costs, your IT service levels that need to be attained, and what IT functional areas would be best considered for outsourcing.
Always conduct a full-scale RFP process in selecting an IT outsourcing provider. If your company has business units that provide IT services, stand firm on insisting that these business units participate in the RFP process just like an external organization.
Ensure that a clear, detailed understanding exists between your company and the IT sourcing provider on the total responsibilities of each party in the partnership. Ensure the responsibilities of each party are thoroughly defined in the forthcoming legal agreement governing the IT outsourcing relationship.
Maintain a line of strong internal IT leaders responsible for setting IT strategy and managing the day to day activities of the IT outsourcing relationships. The IT leaders should report directly into the CIO office.
If the deal involves the transfer of internal employees to the external IT outsourcing provider, identify and document in the contract agreement "key personnel" who will continuously remain active on the your account unless otherwise specified. If key personnel terminate employment with the IT outsourcing partner, clearly lay out the steps required to replace the departed key personnel with the IT outsourcing partner.
Clearly define and document your security and intellectual property protection requirements with the IT outsourcing partner.
Specify that the internal IT organization is responsible for the management of all services provided by the IT outsourcing partner. Your CIO should be the primary point of contact for all current or potential activities of the IT outsourcing provider.
Once an IT outsourcing partnership begins, be diligent in monitoring the performance of the IT outsourcing partner. Ensure that agreed-to service levels are being achieved and insist that the provider submits improvement plans for service levels not being met.

Automated Scanning of PC's Memory Configuration

2009-01-14T00:51:00.005+11:00

A free, downloadable self scanning utility from crucial.com allows you to scan your computer's memory from the desktop without having to look under the hood. A direct link to the tool can be found at http://www.crucial.com/systemscanner/index.aspx.

Apart from being able to provide the below (current) configuration information, the tool is also able to provide a recommendation in the event that you want to upgrade your memory.

Resulting Scan Information:

Maximum Memory Capacity
Currently Installed Memory
Available Memory Slots
Number of Banks
Dual Channel Support
CPU Manufacturer
CPU Family
CPU Speed

Lateral Thinking and Solution Architecture

2009-01-08T23:26:00.005+11:00

These Christmas holidays (December 2008) I decided to look up an old favourite subject of mine; lateral thinking. I was introduced to lateral thinking almost 3 years ago and found the concepts extremely helpful in my professional role as a Solutions Architect. I benefited most from using a lateral thinking approach while solving the more difficult, innovative architectural issues that frame an enterprise-level, distributed software solution.

So what exactly is lateral thinking and how can this particular thought process benefit software architects who need to be creative in their solution modelling and design? This post attempts to explain this concept.

Dr Edward de Bono who invented lateral thinking maintains that the brain is essentially an information handling system that has certain characteristic limitations. Lateral thinking is an attempt to compensate for these limitations while leveraging the advantages. The best way to explain lateral thinking is to describe how it differs from traditional “vertical thinking”. As children, most of us are taught to think vertically from a young age. The core concept of vertical thinking is that while solving a problem that involves multiple steps, we have to be right at each step to be able to move to the next logical one. While we were at school this was evident in solving mathematical problems, at the workplace (in software development, at least) best practice vertical thought is typically encapsulated in a software design pattern or baked in to an architect’s or developer’s integrated development environment. Best practice software development methodologies are established to help guide a project along the most optimal delivery path. How do we then respond to sudden change in a chaotic I.T. environment once we have fixed these response patterns in a solution being designed or implemented? Sadly, this has been the I.T. industry’s greatest challenge and our most evident shortcoming. The apparent lack of flexibility in reshaping a project’s output after the project commences has lead to several I.T. projects failing and/or requiring considerable re-work and cost to the client. This is where I believe lateral thinking from an architectural perspective helps the most because it helps architects challenge the status quo of existing patterns to invent more optimal ones.

Consider these important differences made by Dr de Bono between vertical and lateral thinking:

Vertical thinking is selective, lateral thinking is generative.
Vertical thinking moves only if there is a direction in which to move, lateral thinking moves in order to generate a direction.
Vertical thinking is analytical, lateral thinking is provocative.
Vertical thinking is sequential, lateral thinking can make jumps.
With vertical thinking one has to be correct at every step, with lateral thinking one does not have to be.
With vertical thinking one uses the negative to block off certain pathways, with lateral thinking there is no negative.
With vertical thinking one concentrates and excludes what is irrelevant, with lateral thinking one welcomes chance intrusions.
With vertical thinking categories, classifications and labels are fixed, with lateral thinking they are not.
Vertical thinking follows the most likely paths; lateral thinking explores the least likely.
Vertical thinking is a finite process; lateral thinking is a probabilistic one.

While thinking vertically an architect may stop looking for a solution after s/he has reached, what is believed to be the, “best fit” alternative.

While thinking laterally an architect explores all alternatives possible and documents them. If solution scope changes, then one of the other documented alternatives may provide the most optimal solution that best addresses the scope change. The manner in which alternatives are researched is vastly different in lateral thinking. Steps in the lateral thought process do not have to be sequential; one can jump ahead to a new point and then come back later to fill the gap. When one jumps right to the solution, then the soundness of that solution obviously cannotdepend on the soundness of the path by which it was reached. Nevertheless the solution may still make sense in its own right without having to depend on the pathway by which it was reached. As with trial-and-error a successful trial is still successful even if there was no good reason for trying it.

The diagram below is a graphical representation of the fundamental difference in approach of vertical and lateral solution thinking. For more information on the topic, do visit Dr de Bono’s web site at http://www.edwdebono.com/index.html.

New ASP.NET Charting Control

2008-12-27T00:05:00.011+11:00

Almost a year and half after Microsoft acquired Dundas data visualization technology they (Microsoft) have released a new ASP.NET charting control for .NET Framework 3.5 as shown below. This release is definitely exciting as it empowers seamless development of rich graphical interfaces that previously almost always carried a price tag.

The new charting controls can be downloaded from here. To enable the Visual Studio 2008 IDE to leverage these new charting controls, an add-on pack for Visual Studio 2008 can be downloaded from here.

Back to Basics: .NET 2.0 Document Streaming via WSI 1.1 Basic Profile Web Service

2008-12-26T14:50:00.022+11:00

This post is for a colleague who will be working on streaming documents through a web service in .NET Framework 2.0. The interface being developed is mandated not use WCF, SOAP attachments or web service extensions.

The below sample demonstrates the use of a base64BinaryByteArray to recompose a physical document from a stream of bytes.


using System;
using System.Web;
using System.Web.Services;
using System.Web.Services.Protocols;
using System.IO;
using System.Text;
using System.Drawing;

[WebService(Namespace = "http://simeonlobo.blogspot.com")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
public class Service : System.Web.Services.WebService
{
    public Service () {

        //Uncomment the following line if using designed components 
        //InitializeComponent(); 
    }

    [WebMethod]
    public void CreateRecordWithDocument(string filename, 
                                         byte[] base64BinaryByteArray,
                                         string documentID)
    {
        try
        {
            //Example only: Recompose the document from a stream on NTFS
            FileStream fs = new(FileStream filename, FileMode.OpenOrCreate, FileAccess.Write);
            BinaryWriter writer = new BinaryWriter(fs);
            fs.Write(base64BinaryByteArray, 0, base64BinaryByteArray.Length);
            fs.Close();
            
            //Example only: Delete temporary document after you are finished with it
            File.Delete("D:\\" + filename);
        }
        catch (Exception exc)
        {
            WriteToErrorLog("CreateRecordWithDocument", exc.Message.ToString(), "D:\\error.log");
        }
    }

    public void WriteToErrorLog(string errorNumber, string errorDescription, string logFilePhyPath)
    {
        // create a writer and open the file
        TextWriter tw = new StreamWriter(logFilePhyPath);

        // write a line of text to the file
        tw.WriteLine(DateTime.Now + ", " + errorNumber + ", " + errorDescription);

        // close the stream
        tw.Close();
    }

}

SharePoint 2007 “Save Site As Template” Hack

2008-12-04T17:49:00.003+11:00

This is a little hack that I found quite useful.

In the event that the Save Site As Template option is missing for a sub site under Site Settings -> Look and Feel, then the easiest way to access this functionality is by appending “_layouts/savetmpl.aspx” in the URL for the site.

As an example, if I can access a sub site from the URL:
http://TopLevelCollection/SubSite

Then I can save the template for the sub site by using the URL:
http://TopLevelCollection/SubSite/_layouts/savetmpl.aspx

SharePoint 2007 Forms Based Authentication and Check-Out

2008-12-01T16:27:00.001+11:00

If you ever have had a scenario where you had a SharePoint site working with the Windows Authentication provider and had to turn on Forms Based Authentication, then you may begin to experience issues with checking out documents via Office (Microsoft Word as an example). It is not a widely documented configuration setting but enabling client side integration will fix the issue of checking out documents (see setting below). Credit for finding the setting goes to my mate, Simon Tyrrell :)

After configuring this setting, the larger authentication issue for MS Office still remains; after checking out the document, Office is not able to authenticate successfully unless persistent cookies are enabled. As you may know persistent cookies are enabled when a user clicks on the "Sign me in automatically" checkbox when they log in to SharePoint and these are stored on the users hard disk until they time out or the user erases them. If the user does not click the "Sign me in automatically" checkbox while logging in, session cookies are used and this essentially means that other than the client browser no other application can read the session cookie. The important takeaway here is that Office applications can only read persistent cookies.
Though the default timeout for peristent cookies is 30 minutes, the web.config file can be configured to set this to a longer time period. The web.config setting below demonstrates a timeout of 120 minutes or 2 hours. Another important note is that the moment the cookie expires, client side integration stops working and the user is prompted to enter credentials again.

To solve the issue, Microsoft have released an unwarranted custom httphandler that converts the Forms-based authentication (FBA) prompt originating from a client-side application (like MS Word; when client integration and FBA are enabled) and translates this to a Basic Authentication prompt. This allows the user to re-authenticate into SharePoint. Click here to download the httphandler.

Client-side Integration Setting (click for larger picture)

Microsoft TownSquare

2008-11-30T22:22:00.002+11:00

While browsing the web over the weekend, I happened to stumble across information regarding Microsoft TownSquare (shown in picture); a Facebook-type social networking tool built on top of SharePoint 2007 that is being used within Microsoft by about 8000 employees since January 2008.

TownSquare was built by a prototyping group within Microsoft called Office Labs that drives innovation in some Office Products. Bram Paperman, Program Manager for Office Labs in the U.S. mentions that the idea for TownSquare was born out of the SharePoint Team at Microsoft wanting to examine what it would be like to be able to create a social networking news feed within the Enterprise. While SharePoint 2007 does provide RSS feed capability at a site level, there was no way to seamlessly surface new content from multiple sites.

TownSquare allowed Microsoft employees to toggle users in and out of the system; implying the process of aggregation was more focussed. After targeting other users, Microsoft employees could then track targeted user interaction with the SharePoint system - notifications would be sent out the moment a targeted user added or updated content.

I thought this was a really cool idea.

C# 4.0 “dynamic” versus “var”

2008-11-26T09:33:00.005+11:00

The C# language is evolving so fast that it does take effort to keep up with Anders Hejlsberg, C#’s designer at Microsoft :)

Talking about the next version of C# in this video, Anders describes implementing a runtime operation called “dynamic” that eliminates all type safety for a variable. This essentially means that until the code is run, there is no way to determine if operations on the variable will fail.

There seems to be confusion about dynamic and var, so here is my 2 cents worth:

An example of using "var":


//While using “var”, the compiler essentially figures out the
//type of the variable using type inferencing
var a = “G’Day Mate!!!”;

//The below operation can be validated for errors at compile time (as opposed
//to runtime in the case of using “dynamic”)
a = a.ToUpper();

An example of using "dynamic":


//Sample dynamic instantiation
dynamic dyn = GetObject();

//An attempt to run dynamic code 
//Only after we run this code, we will know whether we have errors 
dyn.Walk();

High Availability Theory and the 9’s Measurement

2008-11-19T09:51:00.000+11:00

High availability is an indicative measure of the “up time” of an application. The moment a host application/service stops servicing requests initiated by users or other application processes, the host application/service is termed to be unavailable and the “down time” clock starts ticking.

While negotiating availability with clients, non-functional requirement documents should clearly highlight the high availability quantifiers; Mean Time Between Failure (MTBF) and Mean Time To Recovery (MTTR).

Mean Time Between Failure (MTBF): Is calculated as (Hours / Failure Count) and is the average length of time the host application/service runs before failing.

Mean Time To Recovery (MTTR): Is calculated as (Repair Hours / Failure Count) and is the average length of time needed to repair and restore the host application/service after a failure.

Therefore, Availability = (MTBF / (MTBF + MTTR)) * 100

A popular way to describe high availability is by using the 9’s measurement:

Three nines (99.9 percent availability) represents about 8.5 hours of service outage in a single year.
Four nines (99.99 percent), represents about 1 hour of service outage in a year.
Five nines (99.999 percent) represents about 5 minutes of outage per year.