Delphi musings

Digitally signing Delphi executables

2012-03-30T04:32:00.003+01:00

Background

From Windows XP Service Pack 2 onwards, any application downloaded with a mainstream web browser (Internet Explorer, Firefox, Opera) will display a Security Warning dialog when executed.

Zone Identifier

The warning dialog is shown when an Authenticode-compliant* download tool writes an Alternative Data Stream (so the the NTFS file system must be in use) for a binary file using the IAttachmentExecute interface
(http://msdn.microsoft.com/en-us/library/windows/desktop/bb776297%28v=vs.85%29.aspx).

The Save method of this interface is used to create a stream called “Zone.Identifier” and writes to it the values:

[ZoneTransfer]
ZoneID=3

– “3” indicating an Internet download source.

References:

http://bit.ly/GUqsok
http://bit.ly/HaCR0Y
http://bit.ly/GROWtT

A digitally signed application will be identified by its “Publisher” – usually a company name – and this will be listed towards the top of the dialog.

*Microsoft's “Introduction to Code Signing”
http://msdn.microsoft.com/en-us/library/ms537361%28VS.85%29.aspx

In the case of an unsigned application, Windows will automatically apply the disconcerting label “The publisher could not be verified. Are you sure you want to run this software?”, and “Unknown Publisher” in place of a company name.

Executing an unsigned application

Running a signed application

From Windows Vista onwards (when User Account Control is enabled), an additional dialog is shown when Windows detects execution of an “Installer” under the following conditions:

Installer Detection only applies to:

32 bit executables

Applications without a requestedExecutionLevel

Interactive processes running as a Standard User with UAC enabled

Before a 32 bit process is created, the following attributes are checked to determine whether it is an installer:

Filename includes keywords such as "install," "setup," and "update."

Keywords in the following Versioning Resource fields: Vendor, Company Name, Product Name, File Description, Original Filename, Internal Name, and Export Name.

Keywords in the side-by-side application manifest embedded in the executable.

Keywords in specific StringTable entries linked in the executable.

Key attributes in the resource file data linked in the executable.

Targeted sequences of bytes within the executable.

Note: The keywords and sequences of bytes were derived from common characteristics observed from various installer technologies

Reference: http://msdn.microsoft.com/en-us/library/aa905330.aspx

Again, a signed “Installer” will be identified by a company name as the “Verified publisher” of the application, and the dialog colored with prosaic hues. An unsigned “Installer” will list the Publisher as “Unknown” and be decorated with amber “hazard” colouring.

Running an unsigned "installer"

Running a signed "installer"

This is all part of Microsoft's system for securing users from potentially malicious software where origin and integrity cannot be tested and confirmed (with a substantial recurring financial cost to software authors for participating in the system).

Newer versions of Visual Studio bundle all of the Microsoft tools necessary for digitally signing a binary file with that programming environment.

Delphi developers are required to source them elsewhere.

Getting the tools

For the purposes of this article, we're going to be generating our own private key and digital certificate for signing executables. Links at the end of the article expound upon these same ideas, detailing the steps involved in acquiring a certificate from a recognised Certificate Authority (CA).

There are limitations to a self-created certificate; as an individual you are not a part of the “chain of trust” that exists between the various issuing organisations, and as a result, any binary file signed with the certificate will not be validated by Windows (the “Unknown Publisher” message will still be shown).

However, it can be a useful for internal testing, or for preparing your build processes in anticipation of adding a CA-sourced certificate at some future date.

To generate our private key and signing certificate, four tools are needed:

makecert.exe

Cert2Spc.exe

pvkimprt.exe

signtool.exe

...and an additional DLL file, “capicom.dll”.

There are various ways of obtaining them (they used to be included in the .NET SDK, but this has since evolved into the “Windows SDK”), one option is to download the “Microsoft Windows SDK for Windows 7 and .NET Framework 4 (ISO)” - a 570MB package.

Another is the “Windows SDK for Windows Server 2008 and .NET Framework 3.5” - a whopping 1.3 GB download.

Windows SDK for Windows 7 and .NET Framework 4 (GRMSDK_EN_DVD.iso):

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=8442

Windows SDK for Windows Server 2008 and .NET Framework 3.5 (6.0.6001.18000.367-KRMSDK_EN.iso):

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=24826

In either case a tool capable of browsing ISOs – such as the indispensable freeware/open-source 7zip (http://www.7-zip.org/) - can be used to locate the files inside of them without any protracted CD-burning or other intermediate extraction.

If you opt for the former, open the ISO, then the Setup folder, then the WinSDKTools folder, and within that the cab1.cab file. Here you'll find 3 of the 4 tools above – extract these to a convenient location.

Within the cabinet the files are named:

WinSDK_makecert_exe_5D21BAF1_83A4_4E71_998E_FF39C36EA905_x86

WinSDK_Cert2Spc_Exe_910B78B5_FBE7_44CD_867D_0F90509859B0_x86

WinSDK_signtool_exe_B2E1011D_2F14_488D_A056_C5BD55106409_x86

Rename these to:

makecert.exe

Cert2Spc.Exe

signtool.exe

...respectively. Note, though, that the Windows 7/.NET 4 SDK does not include (as far as I can tell), the “capicom.dll” file, so this must be downloaded separately (via the webpage “Platform SDK Redistributable: CAPICOM”) – it's available (at slightly under 2MB) here:

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=25281

If you choose the later option, open the ISO with a browsing tool (such as 7zip), open the Setup folder, locate the WinSDKTools-WinSDKTools-common.0.cab file, open this, and extract the four files:

capicom_dll.970E4F94_546F_49F3_BF1F_18BE6B938B02

Cert2Spc_exe.D7AE6AF7_EC98_4D5C_97F8_562E6B8AF64F

makecert_exe.E4279728_DED0_47AC_8B96_F1269703DEFB

signtool_exe.B68FF751_0B1A_4F33_B044_1871CB4B13CC

...then rename them to:

capicom.dll

Cert2Spc.exe

makecert.exe

signtool.exe

pvkimprt.exe is not available in either distribution so, also, must be downloaded separately. It can be found via the webpage “Office 2000 Tool: PVK Digital Certificate Files Importer” (http://www.microsoft.com/download/en/details.aspx?displaylang=EN&id=6563).

It's a self-extracting zip file, but again, a tool like 7zip can browse its content without having to execute it and step through the extraction wizard. When opened, extract the file “pvkimprt.exe” to a convenient location. This file is, in turn, a self-extracting zip. Open it with 7zip, and extract “pvkimprt.exe”.

Generating a certificate

With all of the required tools available, we can now generate our private key and code signing certificate.

Open a command prompt (type cmd in the Run or “Start Search” box, or use Start->All Programs/Programs->Accessories->Command Prompt), navigate to the directory containing the tools, and execute them in the following sequence (with all spaces and other punctuation as shown):

makecert.exe MyCertificateFile.cer -r -n "CN= MyCompanyName " -$ individual -sv MyPrivateKeyFile.pkv -pe -eku 1.3.6.1.5.5.7.3.3

This first step will produce a “self-signed” certificate. Substitute “MyCertificateFile.cer” with the name you want to give to the generated certificate (the name you choose is just a label, the value of the text is inconsequential to the outcome of later stages – in all instances it makes sense to retain the suggested extensions, though, for clarity); “MyCompanyName” with an appropriate value for your “Publisher” identity; “MyPrivateKeyFile.pkv” with a name to use for your private key store file.

For further information about the meaning of the command line switches added here, use the in-built help “makecert.exe /?”.

The string after the “eku” switch indicates that the file to create will be a “code signing” certificate.

When you run the tool, you will be prompted to enter a “Private Key Password” - you will need to provide this again in step 3, and each time you sign a file, so make a note of the password you choose to use here.

cert2spc.exe MyCertificateFile.cer MyCertificateFile.spc

Replace “MyCertificateFile.cer” with the name you chose to give your generated self-signed certificate from step 1, and choose a matching name (again, just for clarity – it doesn't have to match) to replace “MyCertificateFile.spc”. This will create a “Software Publisher Certificate” (SPC) from the “MyCertificateFile.cer” file – a certificate equivalent to one that would be issued from a Certificate Authority.

pvkimprt.exe -pfx MyCertificateFile.spc MyPrivateKeyFile.pkv

Replace “MyCertificateFile.spc”with the name you chose for your Software Publisher Certificate (.spc file) in step 2, and “MyPrivateKeyFile.pkv” with a name for the file to be created by Pvkimprt – a “Personal Information Exchange” (PFX) file – this is what will be used to for code signing.

Pkimprt is a GUI wizard. When you run it you'll first be asked to enter your “Private Key Password” - the password you selected for step 1.

None of the default wizard values need to be changed, so click through each successive form by clicking Next (x3), then on the “Password” form, enter your Private Key Password twice more, and click Next again.

On the “File to Export” form, click the Browse button, navigate to the directory containing the signing tools, and type a name for the PFX file – e.g., “MyPFXFile.pfx”, then click Next.

On the final wizard form, “Completing the Certificate Export Wizard”, click the Finish button to complete the procedure.

As a result of these steps, you should now have four new files labelled something like:

MyPrivateKeyFile.pkv

MyCertificateFile.cer

MyCertificateFile.spc

MyPFXFile.pfx

Code signing

To use our newly generated certificate, we need signtool.exe and a target file. Suppose we have a binary executable named “MyProgram.exe” - the syntax for signing this file would be:

signtool.exe sign /f MyPFXFile.pfx /p MyPrivateKeyPassword /v /t http://timestamp.verisign.com/scripts/timstamp.dll MyProgram.exe

- where “MyPrivateKeyPassword” is the password you chose in step 1 (note also that “timstamp.dll” is not a misspelling).

Automating the process

To make things easier and avoid having to re-type long commands in the case of a syntax error, it makes sense to string the tool commands together in a batch file. See the listing below for two batch files based upon the text of the steps listed above.

Generating a private key, self-signed certificate, and PFX file:

MakeCert.bat

@ECHO OFF

makecert.exe MyCertificateFile.cer -r -n "CN= MyCompanyName " -$ individual -sv MyPrivateKeyFile.pkv -pe -eku 1.3.6.1.5.5.7.3.3

cert2spc.exe MyCertificateFile.cer MyCertificateFile.spc

pvkimprt.exe -pfx myCertificateFile.spc myPrivateKeyFile.pkv

Signing a binary file:

SignBinary.bat

@ECHO OFF

signtool.exe sign /f MyPFXFile.pfx /p MyPrivateKeyPassword /v /t http://timestamp.verisign.com/scripts/timstamp.dll %1

To use the second batch file with a named binary, launch it with a parameter – the name of the target binary file, e.g., “SignBinary.bat MyProgram.exe”.

References

Signtool:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa387764%28v=vs.85%29.aspx

Signtool example:
http://poweradmin.se/blog/2010/03/07/the-wizard-in-signtool-exe-digital-signing-for-dummies/

MakeCert documentation:
http://msdn.microsoft.com/en-us/library/bfsktky3%28v=vs.71%29.aspx

MakeCert example:
http://www.source-code.biz/snippets/vbasic/3.htm

Code Signing for Developers – a Delphi-orientated article covering the same sort of territory, but addressing (in addition) sourcing and using a Certificate Authority issued certificate:
http://www.wiscocomputing.com/articles/code-signing.htm

A Microsoft article explaining different methods for sourcing a code signing certificate: http://technet.microsoft.com/en-us/library/cc732597%28v=ws.10%29.aspx#BKMK_Anchor3

Using Makecert to Create Certificates for Development:
http://www.digitallycreated.net/Blog/38/using-makecert-to-create-certificates-for-development

How to create your own certificate for signing ClickOnce manifests and Strong Naming assemblies (highly recommended!):
http://www.uphillriver.com/CreateYourOwnCertificate.aspx

All About Authenticode (highly recommended!):
http://www.tech-pro.net/authenticode.html

Code Signing for Developers - An Authenticode How-To (highly recommended!):
http://www.tech-pro.net/code-signing-for-developers.html

Snippets for the code bank pt. 2

2012-03-08T04:32:00.001+00:00

A follow-up to last months post (“Snippets for the code bank” @ http://bit.ly/xwIV51), below is an assortment of functions and definitions from my Inno Setup/Rem Objects Pascal Script grab-bag. Some of them are inter-dependent, so it's important to be aware that because none are member methods, where Function A is used by Function B, Function A must be declared first.

Again, none of this is complicated or pioneering, but each function was designed to fill a specific need that existed at the time – one that isn't covered by the wide range of methods provided with the Pascal Script language library.

In all instances, where a call is made to a routine that is not part of the stock Pascal Script library, the listing for the routine can be found elsewhere in this or another of the "Code Bank" posts.



All references to "Common.iss" refer to the unit from which these functions are taken. Modify this in your own script to match the name of the unit in which you use it (e.g., for debugging purposes).

Stringlist routines

//------------------------------------------------------------------------------
// Delete duplicate strings from List
//
procedure RemoveDuplicates(const List: TStringList);
var
  TempList: TStringList;
  i, j: Integer;
begin
  if List = nil then
    Exit;

  TempList := TStringList.Create;
  try
    TempList.Sorted := True;
    i := 0;
    while i <= List.Count - 1 do begin
      if TempList.Find(List[i], j) then
        List.Delete(i)
      else begin
        TempList.Add(List[i]);
        Inc(i, 1);
      end;
    end;
  finally
    TempList.Free;
    TempList := nil;
  end;
end;

//------------------------------------------------------------------------------
// Explode a delimited string (S) into a list of strings using a separator (Sep)
// (n.b., Untested)
//
procedure StrToStrings(const S, Sep: String; const List: TStringList; const AllowBlank: Boolean);
var
  I, J, L, M: Integer;
  Left: String;
begin
  if List <> nil then
    List.Clear;

  if (List = nil) or (S = '') or (Sep = '') then
    Exit;

  L := Length(Sep);
  M := Length(S);
  I := 1;
  J := PosEx(Sep, S, 1);
  while (J > 0) do begin
    Left := Copy(S, I, J - I);
    if (Left <> '') or AllowBlank then
      List.Add(Left);
    I := J + L;
    J := PosEx(Sep, S, I);
  end;
  if I < M then begin
    Left := Copy(S, I, M - (I - 1));
    if (Left <> '') or AllowBlank then
      List.Add(Left);
  end;
end;

String search routines

//------------------------------------------------------------------------------
// Search for a String (Value) in an array of Strings (StrArray), return the
// array index if found, else -1
//
function StrIndex(const Value: String; const CaseSensitive: Boolean; const StrArray: array of String): Integer;
var
  i, Hi, Lo: Integer;
  Found: Boolean;
  SearchTerm: String;
begin
  Result := -1;
  Lo := 0;
  Hi := GetArrayLength(StrArray);
  Found := False;
  if not CaseSensitive then
    SearchTerm := LowerCase(Value)
  else
    SearchTerm := Value;

  for i := Lo to Hi - 1 do begin
    if CaseSensitive then
      Found := StrArray[i] = SearchTerm
    else
      Found := LowerCase(StrArray[i]) = SearchTerm;
    if Found then begin
      Result := i;
      Break;
    end;
  end;
end;

//------------------------------------------------------------------------------
// Get the index of SubStr in MainStr starting from a specified offset
//
function PosEx(const SubStr, MainStr: String; Offset: Integer): Integer;
var
  Loop, MStrLen, SStrLen: Integer;
  TmpStr: String;
begin
  Result := 0;
  MStrLen := Length(MainStr);
  SStrLen := Length(SubStr);

  if (SubStr = '') or (MainStr = '') or (Offset <= 0) or (Offset > (MStrLen - SStrLen)) then
    Exit;

  if Offset = 1 then
    Result := Pos(SubStr, MainStr)
  else begin
    Loop := Offset;
    while Loop <= MStrLen do begin
      if MainStr[Loop] = SubStr[1] then begin
        TmpStr := Copy(MainStr, Loop, SStrLen);
        if TmpStr = SubStr then begin
          Result := Loop;
          Break;
        end;
      end;
      Inc(Loop, 1);
    end;
  end;
end;

//------------------------------------------------------------------------------
// The same as Pos, but searching begins from the right of the string, rather
// than the left e.g., calling: RPos('C', 'ABCD') will return index 3, but the
// search begins at character 'D' in the string 'ABCD' and searches towards the
// beginning of the string.
//
function RPos(const Str1, Str2: String; const CaseInsensitive: Boolean): Integer;
var
  Str1Len, Str2Len: Integer;
  LStr1, LStr2: String;
begin
  Result := 0;
  Str1Len := Length(Str1);
  Str2Len := Length(Str2);
  if (Str1Len = 0) or (Str2Len = 0) or (Str1Len > Str2Len) then
    Exit;

  LStr1 := Str1;
  LStr2 := Str2;
  if CaseInsensitive then begin
    LStr1 := Lowercase(LStr1);
    LStr2 := Lowercase(LStr2);
  end;

  LStr1 := ReverseString(LStr1);
  LStr2 := ReverseString(LStr2);
  Result := Pos(LStr1, LStr2);
  if Result <> 0 then
    Result := Str2Len - (Result - 1) - (Str1Len - 1);
end;

var
  FTokenIndex: Integer;
  FTokenString: String;

//------------------------------------------------------------------------------
// Find the first character from offset 'Start' that is NOT a blank character
//
const
  TokenSeparator = ' ';

function FindNextNonBlank(const Value: String; const Start: Integer): Integer;
var
  i, ValLen: Integer;
begin
  Result := 0;
  ValLen := Length(Value);
  if (Value = '') or (Start <= 0) or (Start > ValLen) then
    Exit;

  i := Start;
  while (i <= ValLen) and (Value[i] = TokenSeparator) do
    Inc(i, 1);

  if (i = ValLen) and (Value[i] = TokenSeparator) then
    //
  else
    Result := i;
end;

//------------------------------------------------------------------------------
// Return a token from global variable 'FTokenString' - SetTokenString should be called ahead of
// this function to set the FTokenString value and reset the FTokenIndex variable.
// PeekToken does not update the FTokenIndex value
//
function PeekToken: String;
var
  i, j, k: Integer;
begin
  Result := '';
  if FTokenString = '' then
    Exit;

  // Find first non-space character
  i := FindNextNonBlank(FTokenString, 1);
  if i <= 0 then
    Exit;

  // If GetToken has been called previously, get index of last token
  if FTokenIndex > 0 then begin
    // Shouldn't need the 'against 0' test, but just in case user changed FTokenString...
    k := 0;
    while (i > 0) and (k < FTokenIndex) do begin
      Inc(k, 1);
      i := PosEx(TokenSeparator, FTokenString, i);
      if i > 0 then
        i := FindNextNonBlank(FTokenString, i + 1);
    end;

    // Something went wrong
    if k < FTokenIndex then
      Exit;
  end;

  j := i;
  if j > 0 then
    Inc(j, 1);
  j := PosEx(TokenSeparator, FTokenString, j);
  if (j <= 0) then begin
    if (i >= 1) and (i < Length(FTokenString)) then
      j := Length(FTokenString) + 1
    else
      Exit;
  end;

  Result := Copy(FTokenString, i, j - i);
end;

//------------------------------------------------------------------------------
// Return a token from FTokenString - SetTokenString should be called ahead of this function to
// set the FTokenString value and reset the FTokenIndex variable.
// GetToken can be called in a loop, e.g., until GetToken returns '' (blank). GetToken increments
// FTokenIndex by '1' after each successful call (so that it can 'carry on' from where it left
// off on a subsequent call)
//
function GetToken: String;
begin
  Result := PeekToken;
  if Result <> '' then
    Inc(FTokenIndex, 1);
end;

//------------------------------------------------------------------------------
procedure SetTokenString(const Value: String);
begin
  FTokenString := Value;
  FTokenIndex := 0;
end;

String sort routines

These first six functions are a collective; this is a Pascal Script translation of a Delphi
code sample originally provided at Experts-Exchange (http://bit.ly/yKBzd8), which
implements a pure-pascal algorithm for "natural order" string sorting (comparable
to the WinAPI function StrCmpLogicalW - available with Windows XP and later). See
function descriptions below for more details

//------------------------------------------------------------------------------
// See previous "Code Bank" posting for IsDigit
//
function NonNullChar(const AChar: Char): Boolean;
begin
  Result := not IsDigit(AChar) and (Ord(AChar) <> 0);
end;

//------------------------------------------------------------------------------
// Next three functions are for sorting Strings into 'natural' order, e.g.
// 1.txt, 2.txt, 3.txt, 123.txt, instead of
// 1.txt, 123.txt, 2.txt, 3.txt
//
function GetEndOfGroup(const Val: String; const Index: Integer): Integer;
var
  i: Integer;
begin
  Result := 0;
  i := Index;
  if (Length(Val) <= 0) or (i > Length(Val)) then
    Exit;

  if IsDigit(Val[i]) then begin
    while (i <= Length(Val)) and IsDigit(Val[i]) do
      Inc(i, 1);
  end else begin
    while (i <= Length(Val)) and (NonNullChar(Val[i])) do
      Inc(i, 1);
  end;

  Result := i;
end;

//------------------------------------------------------------------------------
function CompareGroups(var S1Index, S2Index: Integer; const E1, E2: Integer; const S1, S2: String): Integer;
var
  sTemp1, sTemp2: String;
begin
  sTemp1 := Copy(S1, S1Index, E1 - S1Index);
  sTemp2 := Copy(S2, S2Index, E2 - S2Index);
  Result := CompareText(sTemp1, sTemp2);
  S1Index := E1;
  S2Index := E2;
end;

const
  A1_BEFORE_A   = True;
  A01_BEFORE_A1 = True;
  SWAP          = 1;
  NO_SWAP       = -1;

//------------------------------------------------------------------------------
// See also, StrCmpLogicalW - WinXP+ only, but 'better' sorting, ref:
// http://www.experts-exchange.com/Programming/Languages/Pascal/Delphi/Q_23086281.html
//
function XPSortCompareStrings(const S1, S2: String): Integer;
var
  E1: Integer; // end of group in first string
  E2: Integer; // end of group in second string
  FallbackResult: Integer;
  i, j: Integer;
begin
  Result := 0;
  if (Length(S1) <= 0) or (Length(S2) <= 0) then
    Exit;
  i := 1;
  j := 1;

  repeat
    E1 := GetEndOfGroup(S1, i);
    E2 := GetEndOfGroup(S2, j);

    if (E1 = 0) then begin
      if (E2 = 0) then
        Break
      else begin
        if (A1_BEFORE_A) then
          Result := SWAP
        else
          Result := NO_SWAP;
      end;
    end else begin
      if (E2 = 0) then begin
        if (A1_BEFORE_A) then
          Result := NO_SWAP
        else
          Result := SWAP
      end else begin
        if (IsDigit(S1[i])) then
          if (IsDigit(S2[j])) then begin
            FallbackResult := (E2 - j) - (E1 - i);

            while (S1[i] = '0') do
              Inc(i, 1);

            while (S2[j] = '0') do
              Inc(j, 1);

            Result := (E1 - i) - (E2 - j);

            if (Result = 0) then
              Result := CompareGroups(i, j, E1, E2, S1, S2);

            if (Result = 0) then
              if (A01_BEFORE_A1) then
                Result := FallbackResult
              else
                Result := -FallbackResult;
          end else
            Result := NO_SWAP
        else begin
          if IsDigit(S2[i]) then
            Result := SWAP
          else
            Result := CompareGroups(i, j, E1, E2, S1, S2);
        end;
      end;
    end;
  until (Result <> 0);
end;

//------------------------------------------------------------------------------
// Next two functions are for sorting a TStringList into 'natural' order
//
procedure ExchangeItems(const Index1, Index2: Integer; const List: TStringList);
var
  sTemp: String;
  oTemp: TObject;
begin
  if not Assigned(List) or (Index1 < 0) or (Index1 > List.Count - 1) or (Index2 < 0) or (Index2 > List.Count - 1) then
    Exit;

  sTemp                := List[Index1];
  oTemp                := List.Objects[Index1];
  List[Index1]         := List[Index2];
  List.Objects[Index1] := List.Objects[Index2];
  List[Index2]         := sTemp;
  List.Objects[Index2] := oTemp;
end;

//------------------------------------------------------------------------------
procedure QuickSort(L, R: Integer; const List: TStringList);
var
  I, J, P: Integer;
begin
  if (List = nil) or (L < 0) or (R > List.Count - 1) or (L > R) then
    Exit;

  repeat
    I := L;
    J := R;
    P := (L + R) shr 1;
    repeat
      while XPSortCompareStrings(List[I], List[P]) < 0 do
        Inc(I, 1);
      while XPSortCompareStrings(List[J], List[P]) > 0 do
        Inc(J, -1);
      if I <= J then begin
        ExchangeItems(I, J, List);
        if P = I then
          P := J
        else if P = J then
          P := I;
        Inc(I, 1);
        Inc(J, -1);
      end;
    until I > J;
    if L < J then
      QuickSort(L, J, List);
    L := I;
  until I >= R;
end;

String test routines

//------------------------------------------------------------------------------
// Determines whether StrA occurs at the very end of StrB
function StrEndsWith(const StrA, StrB: String): Boolean;
var
  StringPos: Integer;
begin
  Result := False;
  StringPos := pos(StrA, StrB);
  if StringPos > 0 then
    Result := StringPos = Length(StrB) - (Length(StrA) - 1)
end;

//------------------------------------------------------------------------------
// Returns the length of a string less any line-break characters
function StringLength(const Value: String): Integer;
var
  CRLFLen, ValLen, i: Integer;
  CRLFStr: String;
begin
  Result := 0;
  if Value = '' then
    Exit;

  CRLFStr := constCRLF;
  CRLFLen := Length(constCRLF);
  ValLen := Length(Value);
  i := 1;
  Result := 0;
  while i < ValLen do begin
    if (Value[i] = CRLFStr[1]) and (Copy(Value, i, CRLFLen) = CRLFStr) then
      Inc(i, CRLFLen)
    else begin
      Inc(Result, 1);
      Inc(i, 1);
    end;
  end;

  Result := Max(Result, 1);
end;

String transformation routines

//------------------------------------------------------------------------------
// Flip the order of characters in a string
//
function ReverseString(const S: String): String;
var
  I, L: Integer;
begin
  L := Length(S);
  SetLength(Result, L);

  for I := 0 to L - 1 do
    Result[L - I] := S[I + 1];
end;

const
  constCRLF = #13#10;

//------------------------------------------------------------------------------
// Replace all occurrences of '%s' (string placeholder) with constCRLF
//
function CRLFFormat(const Value: String; const AddSpaces: Boolean): String;
var
  TmpStr: String;
  CRLFStr: String;
begin
  TmpStr := Value;
  CRLFStr := constCRLF;
  if AddSpaces then
    CRLFStr := constSP + CRLFStr + constSP;
  StringChangeEx(TmpStr, '%s', CRLFStr, False);
  Result := TmpStr;
end;

//-----------------------------------------------------------------------------------------------
// Append a String (StrToAdd) to another String (MainString) in a non-memory-fragmentary way (in theory)
// Note: LastRealIndex marks the last 'used' character in MainString, NOT the next free character
//
const
  constStrLenIncAmount = 500;
  
procedure EfficientAppend(var MainString: String; var LastRealIndex: Integer; const StrToAdd: String);
var
  i: Integer;
begin
  if Length(StrToAdd) <= 0 then
    Exit;
  if Length(MainString) < constStrLenIncAmount then
    SetLength(MainString, constStrLenIncAmount);
  if LastRealIndex > Length(MainString) then
    LastRealIndex := Length(MainString);

  if (LastRealIndex + Length(StrToAdd)) > Length(MainString) then begin
    if (LastRealIndex + Length(StrToAdd)) > (Length(MainString) + constStrLenIncAmount) then
      SetLength(MainString, Length(MainString) + Length(StrToAdd) + constStrLenIncAmount)
    else
      SetLength(MainString, Length(MainString) + constStrLenIncAmount);
  end;

  Inc(LastRealIndex, 1);
  for i := 1 to Length(StrToAdd) do begin
    MainString[LastRealIndex] := StrToAdd[i];
    if i < Length(StrToAdd) then
      Inc(LastRealIndex, 1);
  end;
end;

//-----------------------------------------------------------------------------------------------
// Concatonate a group of strings into a single string
//
function StringsToStr(const List: TStringList; const Sep: String): String;
var
  i, j: Integer;
begin
  Result := '';
  if (List = nil) or (List.Count < 0) or (Length(Sep) <= 0) then
    Exit;
    
  j := 0;
  for i := 0 to List.Count - 1 do begin
    if i < List.Count - 1 then
      EfficientAppend(Result, j, List[i] + Sep)
    else
      EfficientAppend(Result, j, List[i]);
  end;
    
  SetLength(Result, j);
end;

//-----------------------------------------------------------------------------------------------
// Format a string value so that text fits within the width of a Vista MessageDlg box (to prevent
// the OS from automatically line-breaking a string at awkward intervals). Note that any line-
// breaks already included in the text are recognised and retained, but must be separated from
// surrounding text with a space for and aft.
const
  MaxVistaMsgBoxChars = 60;

function FormatStringForVistaMsgBox(const Value: String): String;
var
  LValue, TempToken: String;
  LValLen, PeekTokenLen: Integer;
begin
  Result := '';
  if Value = '' then
    Exit;

  SetTokenString(Value);

  LValue := GetToken;
  while True do begin
    LValLen := StringLength(LValue);
    TempToken := PeekToken;
    PeekTokenLen := StringLength(TempToken);
    LValLen := LValLen + PeekTokenLen;
    if (LValLen > MaxVistaMsgBoxChars) or (TempToken = constCRLF) then begin
      Result := Result + LValue + constCRLF;
      if TempToken = constCRLF then
        GetToken;
      LValue := GetToken;
    end else begin
      if LValue <> constCRLF then
        LValue := LValue + constSP;
      LValue := LValue + GetToken;
    end;

    if PeekToken = '' then
      Break;
  end;

  Result := Result + LValue;
end;

2012-03-01T19:33:00.002+00:00

Contact! Populating the Outlook Address Book

A standalone tool for scanning Outlook emails and extracting names and addresses into the Address Book.

I recently formatted the hard disk of an ageing PC with an ancient Win-XP install, then re-added a fresh copy of Windows. While this put back much needed spring in the performance step of the machine, it was followed on by the tedious process of re-adding all third party applications and copying back user data.

Among these was Microsoft's venerable Office 2003 software suite, notable as the last in the series to feature the traditional tool and menu bars (rather than the Ribbon UI of Office 2007 and later). The user had never populated the Outlook address book; instead (if unknowingly) relying on the NickName list, a file automatically updated with names and addresses by Outlook from the To: and CC/BCC: fields of outgoing messages.

This file, labelled <Outlook Profile Name>.NK2, resides in the %appdata%\Microsoft\Outlook folder – on the XP machine this resolved to:

c:\documents and settings\UserName\Application Data\Microsoft\Outlook\User Name.NK2

The size of the file copied from the previous install was about 400KB, and the user of the machine made regular use of the AutoCompletion feature that the NickName list facilitates. Unfortunately, it didn't survive the transfer. It's still not clear what went wrong, but when the NickName file was copied to the new Windows install, Outlook ignored it (or wouldn't work with it). Stories abound about NK2 file corruption (http://bit.ly/yMdivj), so perhaps this is to blame, although it went unnoticed during the previous incarnation of Windows/Outlook on the same PC.

So the user was faced with the onerous prospect of starting again with an empty NickName list, and slowly re-growing it over time.

Delphi to the rescue

I've been peripherally aware of the Office controls under the “Servers” tab in Delphi for a while, but I've never had cause to investigate further. This seemed like a good opportunity to see what was possible.

My intention was to bypass the NickName file completely. It isn't something you can manually update, there are no user tools for manipulating it, and it doesn't appear to have a very good reputation for reliability. Instead I wanted the user to become familiar with the Outlook Address Book.

This is a slightly confusing area because there are some related terms that are often used interchangeably, but mean different things. For example, some people talk about the “Address Book”, while others call the same the “Contact List”. Microsoft provides this definition (http://bit.ly/x0nRa4):

Think of the Address Book as a container of individual address books or address lists, such as the following:

Contacts - The address book where you keep your personal addresses. In the Address Book dialog box, Contacts appear under Outlook Address Book. Contacts are stored in a .pst data file and support unicode.
Personal Address Book - The predecessor to Contacts in earlier versions of Outlook that uses older technology and doesn't support unicode. The Personal Address Book is stored in a .pab data file.
LDAP Internet directories - Think of LDAP as a White Pages for the Internet. lobal Address List (GAL) - If your organization uses Microsoft Exchange Server e-mail accounts, the GAL displays names of people in your organization.
Third-party address books.

So the “Address Book” is an over-arching container for other lists that contain details of people you know about – and one of these is the Contact list represented as a folder under the “Folder List” pane in Outlook.

Although importing the old NickName list failed, copying all Person Folder (.pst) files had worked without problems. So we had a store of all sent and received emails – the raw data for entries in our new Address Book if you like – but no simple way of getting this information into the Address Book short of copying and pasting it all in, one entry at a time.

My intention then, using Delphi and COM, was to create a tool which would iterate over every email in the Personal Folders store extracting names and email addresses as it went, then import each of these values into the Contacts list.

The tools

Because the need was pressing, I had very little time to decide how to go about creating the tool. There are some useful tutorials and code snippets scattered about the web, but nothing that dealt with my specific requirements. In the event I went for a quick 'n' dirty solution using Delphi MAPI (MAPI definition: http://bit.ly/gBAAaY) header translations from dimastr.com, home of Outlook Redemption (also written in Delphi) http://bit.ly/y8wbkw (see the “Extended MAPI headers for Borland Delphi” item).

As a result the tool eschews the Delphi-provided MS-Office Type Library files (Outlook2000.pas etc.), and instead uses OLEVariants and late-binding. At the time this was for no other reason than that in creating a few simple proof-of-concept apps, I found the late-binding technique quicker to get to grips with (perversely). But thinking further ahead, it would hopefully make the tool more portable to other Outlook versions.

There was another consideration that swayed the balance somewhat, which was that with the dimastr headers and late-binding, running the tool doesn't trigger Outlook's Security Alert (via the Object Model Guard) dialog (“Another process is trying to access e-mail addresses you have stored in Outlook”). Compiling against the Delphi Outlook Type Library files always did. I haven't looked at which classes or combinations of classes might be responsible for tripping the alert, so it may be that this is just as easy to mitigate in the latter instance.

Early-binding vs late-binding

http://bit.ly/zB0Z8P
http://bit.ly/A7G0O2

Exploring the UI

It's all very plain and utilitarian, but it does the job. The main form is divided into three vertically stacked components: a Listview at the top (labelled “1” in the image), which is used to display all names and email addresses found (plus an index, indicating the order in which they were discovered), a RichEdit in the middle (labelled “2” in the image) for giving feedback about any errors encountered during the scan, and at the bottom a panel with buttons (labelled “3”, “4”, and “5”) for controlling the application.

There's also a Statusbar at the foot of the main form, which gives a running count of the total number of names and addresses found, and the name of the folder currently being explored (when the application is in its “running” state).

Tool main form at the end of a scan (with email addresses blanked out)

The Listview can be sorted by Index, Sender name, or Sender email address (sorting is by Sender name in the screen shot).

Where a duplicate contact is found it is ignored and reported in the RichEdit (a “duplicate” is any contact with the same Sender name/Sender email address as a previously seen contact. Note that this means a contact with a different Sender name, but an email address found before – or visa versa - is not considered a duplicate).

Where an invalid email address is found (using a web procured algorithm that isn't quite RFC 5322 compliant http://tools.ietf.org/html/rfc5322 – but works in most instances – for example, it will reject email addresses with a '+' symbol in the “local” part of any address, the segment before the '@' sign, which is incorrect behaviour, but this can be a tricky problem: http://bit.ly/16XkKb), it is also reported to the RichEdit and not inserted into the Address Book.

There are two buttons in the panel to the lower-left of the main form labelled “3” and “4” in the first screen shot.

Button “3” has the caption “Excluded Contacts”. Clicking this button displays a modal child form with a Memo component and two buttons of its own - “OK” and “Cancel”.

By entering names or email addresses into the Memo, one name or address per line, it is possible to build up a “block list” of contacts that will not be added to the Address Book if found. Wild card symbols are supported (“?” as a place-holder for an individual character, or “*” as a place-holder for multiple characters), so a single entry can be used to cover multiple contact variations, e.g., “John D*” would block any contacts with names including “John Doe”, “John Dean”, “John Dillinger” etc.

Excluded contacts form with two entries

Here we see two added “block list” entries, “*@hotmail.com” and “Amazon*”, which would prevent any contacts with a hotmail.com email address, or Sender name beginning “Amazon” from being added to the Address Book.

Button “4” has the caption “Excluded Folders”. Clicking this button will also show a modal child form, and like the “Excluded Contacts” form, this contains a Memo and “OK” and “Cancel” buttons. This form implements another sort of “block list”, this time for folders in the Outlook “Folder List” pane. By adding entries to the Memo – again, one entry per line – folders enumerated during the scan with a name matching a “block list” entry will not be searched for emails.

As above, wild card symbols can be used, so adding a folder name such as:

Personal Folders.I?box

- would block email searching in folders named “Inbox” and “IMBox”.

Excluded folders form with 6 blocked items

Here we see six block list entries, preventing email searching in any of the folders “Deleted Items”, “Calendar”, “Contacts”, “Journal”, “Notes”, and “Tasks”. Folders must be “fully qualified”, that is, be written to include all ancestor folders in a pre-amble to the folder name, e.g., “Personal Folders.Delete Items”.

The block lists can be used in combination, to exclude entire folders and/or names and email addresses.

Button 5, with caption Populate Outlook Contacts, begins the scanning process. Below, two buttons with captions “Pause” and “Stop” are disabled when no scan is in progress, but become enabled once a scan is underway. The Pause button temporarily suspends a running scan (until it is pressed again); the Stop button halts a running scan.

Contacts are added to the Address Book as they are discovered, so Stop-ping a running scan will prevent additional contacts from being added, but any found prior to that event will already have been inserted and saved.

Code Listing

Event handler for the "Populate Outlook Contacts" button click event

procedure TfrmMain.btnAddAddressesClick(Sender: TObject);
var
  OutlookApp, OLFolderList, OLFolder: OleVariant;
  OLContactList: Variant;
  slDupList: TStringList;
  TotalAddresses: Integer;
begin
  redtStatusMsgs.Clear;
  TotalAddresses := 0;
  SetControlsEnabled(Self, True, []);
  SetControlsEnabled(Self, False, ['btnPause', 'btnStop']);
  slDupList := nil;
  OutlookApp := Unassigned;
  MAPIInitialize(nil);
  try
    try
      slDupList := TStringList.Create;
      slDupList.Sorted := True;
      slDupList.Duplicates := dupIgnore;

      OutlookApp := CreateOleObject('Outlook.Application');
      if VarIsEmpty(OutlookApp) then begin
        StatusMessage('Creating Outlook object: aborting.', tstError);
        Exit;
      end;

      // Get the Items collection from the Contacts folder
      OLContactList := OutlookApp.GetNameSpace('MAPI').Folders('Personal Folders').Folders('Contacts').Items;
      if VarIsEmpty(OLContactList) then begin
        StatusMessage('Getting Contacts object: aborting.', tstError);
        Exit;
      end;

      // Iterate folders
      OLFolderList := OutlookApp.GetNameSpace('MAPI').Folders;
      if VarIsEmpty(OLFolderList) then begin
        StatusMessage('Getting Folders object: aborting.', tstError);
        Exit;
      end;
      // Get list of folders below the "root" folder (i.e. "Personal Folders")
      OLFolder := OLFolderList.GetFirst;

      IterateFolders(OLFolder, OLContactList, slDupList, TotalAddresses);

      redtStatusMsgs.Lines.Add('');
      StatusMessage('Total contacts added: ' + IntToStr(slDupList.Count), tstStatus);
    except
      on E: SysUtils.Exception do
        StatusMessage('(' + E.ClassName + ') ' +  E.Message, tstError);
    end;
  finally
    MAPIUninitialize;
    FreeAndNil(slDupList);
    // Close Outlook
    OutlookApp := Unassigned;
    SetControlsEnabled(Self, False, []);
    SetControlsEnabled(Self, True, ['btnPause', 'btnStop']);
  end;
end;

The main application loop

function TfrmMain.IterateFolders(const Folder: OleVariant; const OLContactList: Variant; const slDupeList: TStringList; var TotalAddresses: Integer): Boolean;
var
  OLFolderList, OLFolder, OLFolder2: OleVariant;
  OLItemList, OLItem, OLContact: Variant;
  sSender, sAddress, sPath: String;
  i, j, k: Integer;
  bDupe, bBadAddress, bExcludedContact: Boolean;
  intrfTemp: IInterface;
  liEntry: TListItem;
begin
  Result := False;
  try
    // Get list of folders below Folder
    OLFolderList := Folder.Folders;

    for i := 1 to OLFolderList.Count do begin
      // Get a MAPIFolder
      OLFolder := OLFolderList.Item(i);
      if VarIsEmpty(OLFolder) then begin
        StatusMessage('Getting Folder (' + IntToStr(i) + ') object: aborting.', tstError);
        Exit;
      end;

      // Print description
      stat1.Panels[1].Text := OLFolder.Name;

      // Check whether this is an "excluded" folder
      sPath := OLFolder.Name;
      OLFolder2 := OLFolderList.Parent;
      while (Pointer(IDispatch(OLFolder2.Parent)) <> nil) do begin
        // MAPIFolder
        if Supports(OLFolder2, StringToGUID('{00063006-0000-0000-C000-000000000046}'), intrfTemp) then
          sPath := OLFolder2.Name + '.' + sPath;
        OLFolder2 := OLFolder2.Parent;
      end;
      if IsExcludedFolder(sPath) then begin
        StatusMessage('(IGNORED) excluded folder: ' + sPath, tstStatus);
        Continue;
      end;

      // Recursively iterate through sub-folders
      if OLFolder.Folders.Count > 0 then begin
        Result := IterateFolders(OLFolder, OLContactList, slDupeList, TotalAddresses);
        if not Result then
          Exit;
      end;
      
      // Iterate over folder items (i.e., emails)
      OLItemList := OLFolder.Items;

      for j := 1 to OLItemList.Count do begin
        // Running total
        stat1.Panels[3].Text := IntToStr(TotalAddresses);
        Inc(TotalAddresses);

        if j mod 50 = 0 then
          Application.ProcessMessages;

        while FPaused do begin
          Sleep(50);
          Application.ProcessMessages;
          if Application.Terminated or FStopped then
            Exit;
        end;

        if FStopped then
          Exit;

        // A _MailItem
        OLItem := OLItemList.Item(j);
        sSender := GetProperty(OLItem, PR_SENDER_NAME);
        sAddress := GetProperty(OLItem, PR_SENDER_EMAIL_ADDRESS);

        // Ignore duplicates, any messages without a valid email address, or any names or addresses in the "excluded" lists
        bBadAddress := False;
        bExcludedContact := False;
        bDupe := slDupeList.Find(LowerCase(sAddress) + '=' + LowerCase(sSender), k);
        if not bDupe then begin
          bExcludedContact := IsExcludedContact(sSender) or IsExcludedContact(sAddress);
          if not bExcludedContact then
            bBadAddress := not IsValidEmail(sAddress);
        end;

        if bDupe or bBadAddress or bExcludedContact then begin
          if bDupe then
            StatusMessage('(DUPLICATE) ' + sSender + ', ' + sAddress, tstStatus)
          else if bBadAddress then
            StatusMessage('(INVALID) ' + sSender + ', ' + sAddress, tstWarning)
          else
            StatusMessage('(EXCLUDED) ' + sSender + ', ' + sAddress, tstStatus);
          Continue;
        end;
        slDupeList.Add(LowerCase(sAddress) + '=' + LowerCase(sSender));

        // Print description
        liEntry := lv1.Items.Add;
        liEntry.Caption := IntToStr(lv1.Items.Count);
        liEntry.SubItems.Add(sSender);
        liEntry.SubItems.Add(sAddress);

        // The above presents a problem that can probably only be sorted out manually?
        // e.g., if two people with identical names are found and different email addresses
        // are they the same person with more than one address?
        // Or are they two separate individuals?
        //
        // If two different people share an email address?
        //
        // For simplicities sake, every unique name/address combo is considered a separate contact
        OLContact := OLContactList.Add;
        OLContact.FullName := sSender;
        OLContact.Email1Address := sAddress;

        // Save the new record.
        OLContact.Save;
      end;
    end;
  except
    on E: SysUtils.Exception do
      StatusMessage('(' + E.ClassName + ') ' +  E.Message, tstError);
  end;
  Result := True;
end;

Conclusion

That's about all there is to it. At the end of a scan – processing 5000 email addresses takes ~5 minutes on my 2.0GHz system, but involves a lot of disk activity, so this may be the chief bottleneck – the Contacts list of the Address Book will contain all names and email addresses found.

Be aware that Outlook will happily accept multiple identical Contact entries, and no provision has been made for this in the tool – we were starting from a “blank slate” so dealing with this wasn't a requirement; if you are updating an existing Contacts list, this may be something you need to consider. At present I leave this as an exercise for the reader.

Companion Files:
Tool source code and compiled file (419 KB):
http://www.mediafire.com/?62cmin1khrr2z9p

Snippets for the code bank

2012-01-20T01:49:00.000+00:00

Quick re-cap from previous posts: Inno Setup is an open-source, freeware “installation builder”. It's an application for creating “installers” for other applications. Though it can be just as usefully employed for adding files of any variety to a Windows PC.

Specifying which files or directories to copy to, update, or create on the target PC is as easy as listing the appropriate paths in an Inno Setup script. But if you want to define more detailed custom actions, eventually you'll want to take a look at the scripting capabilities that are built into the Inno Setup development environment.

If you recall, Inno Setup uses Pascal Script as its lingua franca for coding duties (also available for free with full source code as a stand-alone download from software developer Rem Objects). And also that Pascal Script is a fully-featured implementation of the Pascal programming language, with most of the classes and functions available from the world of its binary-compiled big brothers, like Delphi.

Inno Setup comes supplied with a host of code samples for common scripting tasks, and the Help file is another useful source of worked examples. You can also find Pascal Script documentation on the Rem Objects website, and there are several other enthusiast sites that carry extra useful code listings and libraries – most notably the InnoSetup Extensions Knowledgebase.

But I've amassed quite a collection of little functions and procedures of my own over time, and so I'll be adding some of them here and in subsequent posts as they come to light (I'll be dredging through past projects to haul them out for public consumption/ridicule).

Many of these are trivially simple, but you plug the holes where you finds 'em. Lots more to come.

In all instances, where a call is made to a routine that is not part of the stock Pascal Script library, the listing for the routine can be found elsewhere in this or another of the "Code Bank" posts.



All references to "Common.iss" refer to the unit from which these functions are taken. Modify this in your own script to match the name of the unit in which you use it (e.g., for debugging purposes).

Boolean routines

//------------------------------------------------------------------------------
// Convert a Boolean value to a String
//
function BoolToStr(const Value: Boolean): String;
begin
  Result := 'True';
  if not Value then
    Result := 'False';
end;

//------------------------------------------------------------------------------
// Convert a String to a Boolean (see previous Code Bank post for StrIndex)
//
function StrToBool(const Value: String): Boolean;
begin
  if StrIndex(Value, False, ['true', 't', '1', 'yes', 'y']) >= 0 then
    Result := True
  else if StrIndex(Value, False, ['false', 'f', '0', 'no', 'n']) >= 0 then
    Result := False
  else
    RaiseException('Invalid String/Boolean representation');
end;

Character test routines

//------------------------------------------------------------------------------
// Returns True if "AChar", a single character, is a number
//
function IsDigit(const AChar: Char): Boolean;
begin
  Result := (Ord(AChar) >= 48) and (Ord(AChar) <= 57);
end;

//------------------------------------------------------------------------------
// Returns True if "AChar", a single character, is a letter
// n.b., Roman/Latin alphabet only!
//
function IsAlpha(const AChar: Char): Boolean;
begin
  Result := ((Ord(AChar) >= 65) and (Ord(AChar) <= 90)) or ((Ord(AChar) >= 97) and (Ord(AChar) <= 122));
end;

Debug routines

//------------------------------------------------------------------------------
// Show a message if 'DebugMode' is active
//
procedure DebugMsg(const UnitName, MethodName, LineNo, Msg: String);
var
  MsgOut: String;
begin
  if FDebugMode then begin
    MsgOut := 'Unit: ' + UnitName + #13 + 'Method: ' + MethodName + #13 + 'Line number: ' + LineNo + #13 + 'Message: ' + Msg;
    MsgBox(MsgOut, mbInformation, MB_OK);
  end;
end;

var
  FDebugMode: Boolean;

//------------------------------------------------------------------------------
// Sets global variable "FDebugMode" to True if a parameter is passed to Setup
// from the command line in the forum "/debug=true"
//
procedure CheckForDebugMode;
var
  About: String;
begin
  // Show Debug messages?
  FDebugMode := ExpandConstant('{param:debug|false}') = 'true';
  if FDebugMode then
    DebugMsg('Common.iss', 'CheckForDebugMode', '-1', 'Debug mode enabled');
  else
    DebugMsg('Common.iss', 'CheckForDebugMode', '-1', 'Debug mode NOT enabled');
end;

Math routines

//------------------------------------------------------------------------------
// Silly-simple implementation of the Delphi 'Inc' function - just adds 'Amount'
// to a passed in value
//
procedure Inc(var Value: Integer; const Amount: Integer);
begin
  Value := Value + Amount;
end;

//------------------------------------------------------------------------------
// Compare two Integers, return the higher
//
function Max(const Value1, Value2: Integer): Integer;
begin
  Result := Value1;
  if Value2 > Value1 then
    Result := Value2;
end;

//------------------------------------------------------------------------------
// Compare two Integers, return the lower
//
function Min(const Value1, Value2: Integer): Integer;
begin
  Result := Value1;
  if Value2 < Value1 then
    Result := Value2;
end;

Registry routines

//------------------------------------------------------------------------------
// Extract a value from the Registry
// This function specifically deals with values stored under the "Uninstall" key
// entry created by an Inno Setup installer. This includes useful information
// such as the "DisplayName", "InstallDate" and "InstallLocation" - the folder to
// which the application was copied.
//
function GetRegistryKeyValue(const KeyValueName, AppName: String; var FoundValue: String): Boolean;
var
  RootKeyName: Integer;
  SubKeyName, ValueName: String;
begin
  Result := False;
  FoundValue := '';
  
  RootKeyName := HKEY_LOCAL_MACHINE;
  SubKeyName  := 'SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\' + AppName + '_is1';
  ValueName   := KeyValueName;
  try
    Result := RegQueryStringValue(RootKeyName, SubKeyName, ValueName, FoundValue);
  except
  end;
end;

ActiveX routines

type
  TMSXMLVersion = (msxOne, msxTwo, msxThree, msxFour, msxFive, msxSix);

const
  MSXMLOneProgID   = 'MSXML.DOMFreeThreadedDocument';
  MSXMLTwoProgID   = 'MSXML.DOMDocument';
  MSXMLThreeProgID = 'MSXML2.DOMDocument';
  MSXMLFourProgID  = 'Msxml2.DOMDocument.4.0';
  MSXMLFiveProgID  = 'Msxml2.DOMDocument.5.0';
  MSXMLSixProgID   = 'Msxml2.DOMDocument.6.0';

//------------------------------------------------------------------------------
// Generic method for assessing success in attempts to create a named OLE object class
function OLEObjectTest(const ClassName: String): Boolean;
begin
  Result := False;
  try
    CreateOLEObject(ClassName);
    Result := True;
  except
  end;
end;

//------------------------------------------------------------------------------
// Test whether a given MSXML version is installed: note, later versions are backward-compatible,
// so a later revision implies the availability of functionality in an earlier edition
function MSXMLVerInstalled(const Version: TMSXMLVersion): Boolean;
begin
  Result := OLEObjectTest(MSXMLSixProgID) and (Integer(Version) <= Integer(MemberToMSXMLVersion(msxSix)));
  if (Result) or (not Result and (Integer(Version) = Integer(MemberToMSXMLVersion(msxSix)))) then
    Exit;
  Result := OLEObjectTest(MSXMLFiveProgID) and (Integer(Version) <= Integer(MemberToMSXMLVersion(msxFive)));
  if (Result) or (not Result and (Integer(Version) = Integer(MemberToMSXMLVersion(msxFive)))) then
    Exit;
  Result := OLEObjectTest(MSXMLFourProgID) and (Integer(Version) <= Integer(MemberToMSXMLVersion(msxFour)));
  if (Result) or (not Result and (Integer(Version) = Integer(MemberToMSXMLVersion(msxFour)))) then
    Exit;
  Result := OLEObjectTest(MSXMLThreeProgID) and (Integer(Version) <= Integer(MemberToMSXMLVersion(msxThree)));
  if (Result) or (not Result and (Integer(Version) = Integer(MemberToMSXMLVersion(msxThree)))) then
    Exit;
  Result := OLEObjectTest(MSXMLTwoProgID) and (Integer(Version) <= Integer(MemberToMSXMLVersion(msxTwo)));
  if (Result) or (not Result and (Integer(Version) = Integer(MemberToMSXMLVersion(msxTwo)))) then
    Exit;
  Result := OLEObjectTest(MSXMLOneProgID) and (Integer(Version) <= Integer(MemberToMSXMLVersion(msxOne)));
end;

System routines

// These record types, the DLL function prototypes and the GetNativeSystemInfo,
// CharArrayToString and GetWindowsVersion functions are all related.
// The function of interest here is "GetWindowsVersion" (the others are all
// helpers); this is copied from the JEDI JCL JCLSysInto.pas unit, and modified
// for Pascal Script compatibility.
// When called it returns an enumerated value representing the version of Windows
// reported by the operating system - wvWin95 for Windows 95, wvWin2000 for Windows
// 2000 and so on.
//
type
  TOSVersionInfoExA = record
    dwOSVersionInfoSize: DWORD;
    dwMajorVersion: DWORD;
    dwMinorVersion: DWORD;
    dwBuildNumber: DWORD;
    dwPlatformId: DWORD;
    szCSDVersion: array [0..127] of Char;  
    wServicePackMajor: WORD;
    wServicePackMinor: WORD;
    wSuiteMask: WORD;
    wProductType: BYTE;
    wReserved: BYTE;
  end;

  TSystemInfo = record
    wProcessorArchitecture: Word;
    wReserved: Word;
    dwPageSize: DWORD;
    lpMinimumApplicationAddress: Integer;
    lpMaximumApplicationAddress: Integer;
    dwActiveProcessorMask: DWORD;
    dwNumberOfProcessors: DWORD;
    dwProcessorType: DWORD;
    dwAllocationGranularity: DWORD;
    wProcessorLevel: Word;
    wProcessorRevision: Word;
  end;

  TCharArray = array of Char;

  TWinVersion = (wvUnknown, wvWin95, wvWin95OSR2, wvWin98, wvWin98SE, wvWinME,
    wvWinNT31, wvWinNT35, wvWinNT351, wvWinNT4, wvWin2000, wvWinXP,
    wvWin2003, wvWinXP64, wvWin2003R2, wvWinVista, wvWinServer2008,
    wvWin7, wvWinServer2008R2);

function DLL_GetVersionEx(var VersionInformation: TOSVersionInfoExA): Integer;
external 'GetVersionExA@kernel32.dll stdcall';  

procedure DLL_GetNativeSystemInfo(var SystemInfo: TSystemInfo);
external 'GetNativeSystemInfo@kernel32.dll stdcall delayload';

procedure DLL_GetSystemInfo(var SystemInfo: TSystemInfo); 
external 'GetSystemInfo@kernel32.dll stdcall delayload';

function GetSystemMetrics(nIndex: Integer): Integer; 
external 'GetSystemMetrics@user32.dll stdcall';

//------------------------------------------------------------------------------
function GetNativeSystemInfo(var SystemInfo: TSystemInfo): Boolean;
var
  SysInfo: TSystemInfo;
begin
  Result := False;

  SysInfo.wProcessorArchitecture := 0;
  SysInfo.wReserved := 0;
  try    
    DLL_GetNativeSystemInfo(SysInfo);
  except
    // GetNativeSystemInfo is only available in Window XP and later - so trying to 
    // call the function will fail on earlier OS'.
    // Note that WE only use it to distinguish between XP and 2003, so this shouldn't 
    // be an issue, but if this function were ever needed elsewhere... 
    // Note also that GetSystemInfo is NOT available on Windows 95
    DLL_GetSystemInfo(SysInfo);
  end;
end;

//------------------------------------------------------------------------------
function CharArrayToString(const CharArray: TCharArray): String;
var
  i: Integer;
begin
  SetLength(Result, GetArrayLength(CharArray));
  for i := 1 to Length(Result) do
    Result[i] := CharArray[i - 1];
end;

//------------------------------------------------------------------------------
const
  PROCESSOR_ARCHITECTURE_INTEL = 0;
  VER_PLATFORM_WIN32_WINDOWS   = 1;
  VER_PLATFORM_WIN32_NT        = 2;
  SM_SERVERR2                  = 89;

function GetWindowsVersion: TWinVersion;
var
  TrimmedWin32CSDVersion: string;
  OSVersionInfoExA: TOSVersionInfoExA;
  SystemInfo: TSystemInfo;
  KernelVersionHi: Cardinal;
  KernelDLLPath: String;
  VersionMS, VersionLS: Cardinal;
begin
  Result := wvUnknown;
  OSVersionInfoExA.dwOSVersionInfoSize := SizeOf(OSVersionInfoExA);
  DLL_GetVersionEx(OSVersionInfoExA);
  TrimmedWin32CSDVersion := CharArrayToString(OSVersionInfoExA.szCSDVersion);
  TrimmedWin32CSDVersion := Trim(TrimmedWin32CSDVersion);
  KernelDLLPath := AddBackslash(GetSystemDir) + 'kernel32.dll';
  GetVersionNumbers(KernelDLLPath, VersionMS, VersionLS); 

  case OSVersionInfoExA.dwPlatformId of
    VER_PLATFORM_WIN32_WINDOWS:
      case OSVersionInfoExA.dwMinorVersion of
        0, 1, 2, 3, 4, 5, 6, 7, 8, 9:
          if (TrimmedWin32CSDVersion = 'B') or (TrimmedWin32CSDVersion = 'C') then
            Result := wvWin95OSR2
          else
            Result := wvWin95;
        10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 
        31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51,
        52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72,
        73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89:
          // On Windows ME Win32MinorVersion can be 10 (indicating Windows 98
          // under certain circumstances (image name is setup.exe). Checking
          // the kernel version is one way of working around that.
          if VersionMS = $0004005A then // 4.90.x.x
            Result := wvWinME
          else
          if (TrimmedWin32CSDVersion = 'A') or (TrimmedWin32CSDVersion = 'B') then
            Result := wvWin98SE
          else
            Result := wvWin98;
        90:
          Result := wvWinME;
      end;
    VER_PLATFORM_WIN32_NT:
      case OSVersionInfoExA.dwMajorVersion of
        3:
          case OSVersionInfoExA.dwMinorVersion of
            1:
              Result := wvWinNT31;
            5:
              Result := wvWinNT35;
            51:
              Result := wvWinNT351;
          end;
        4:
          Result := wvWinNT4;
        5:
          case OSVersionInfoExA.dwMinorVersion of
            0:
              Result := wvWin2000;
            1:
              Result := wvWinXP;
            2:
              begin
                OSVersionInfoExA.dwOSVersionInfoSize := SizeOf(OSVersionInfoExA);
                SystemInfo.wProcessorArchitecture := 0;
                SystemInfo.wReserved := 0;
                GetNativeSystemInfo(SystemInfo);
                if GetSystemMetrics(SM_SERVERR2) <> 0 then
                  Result := wvWin2003R2
                else if (SystemInfo.wProcessorArchitecture <> PROCESSOR_ARCHITECTURE_INTEL) and (DLL_GetVersionEx(OSVersionInfoExA) <> 0) and (OSVersionInfoExA.wProductType = VER_NT_WORKSTATION) then
                  Result := wvWinXP64
                else
                  Result := wvWin2003;
              end;
          end;
        6:
          case OSVersionInfoExA.dwMinorVersion of
            0:
              begin
                OSVersionInfoExA.dwOSVersionInfoSize := SizeOf(OSVersionInfoExA);
                if (DLL_GetVersionEx(OSVersionInfoExA) <> 0) and (OSVersionInfoExA.wProductType = VER_NT_WORKSTATION) then
                  Result := wvWinVista
                else
                  Result := wvWinServer2008;
              end;
            1:
              begin
                OSVersionInfoExA.dwOSVersionInfoSize := SizeOf(OSVersionInfoExA);
                if (DLL_GetVersionEx(OSVersionInfoExA) <> 0) and (OSVersionInfoExA.wProductType = VER_NT_WORKSTATION) then
                  Result := wvWin7
                else
                  Result := wvWinServer2008R2;
              end;
          end;
      end;
  end;
end;

//------------------------------------------------------------------------------
// Pascal Script doesn't like casting from an enumerated member to Integer, but will allow casting
// from an anonymous enumerated value to Integer
//
function MemberToMSXMLVersion(const Value: TMSXMLVersion): TMSXMLVersion;
begin
  Result := Value;
end;

//------------------------------------------------------------------------------
function MemberToWinVersion(const Value: TWinVersion): TWinVersion;
begin
  Result := Value;
end;

//------------------------------------------------------------------------------
// Returns TRUE if the version of Windows reported by the OS matches one of the
// values in the "WinVersionSet" set, e.g., if the installer is running on Windows
// 2000, and "WinVersionSet" contains [wvWin98, wvWin98SE, wvWinME, wvWin2000] then
// the function would return TRUE.
//
function IsGivenWindowsVersion(const WinVersionSet: TWinVersionSet): Boolean;
begin
  Result := GetWindowsVersion in WinVersionSet;
end;

//------------------------------------------------------------------------------
// Returns TRUE if the version of Windows reported by the OS is the same as or
// later (more recent) than the value passed in through "WinVersion".
// e.g., if the installer is running on Windows 2000, and "WinVersion" contains
// wvWinXP, the function would return TRUE.
//
function IsGivenWindowsVersionOrLater(const WinVersion: TWinVersion): Boolean;
var
  ActualOSVer, TargetOSVer: Integer;
begin
  ActualOSVer := Integer(GetWindowsVersion);
  TargetOSVer := Integer(MemberToWinVersion(WinVersion));
  Result := ActualOSVer >= TargetOSVer;
end;

//------------------------------------------------------------------------------
// Returns TRUE if the version of Windows reported by the OS is the same as or
// earlier than the value passed in through "WinVersion".
// e.g., if the installer is running on Windows 2000, and "WinVersion" contains
// wvWin98, the function would return TRUE.
//
function IsGivenWindowsVersionOrEarlier(const WinVersion: TWinVersion): Boolean;
var
  ActualOSVer, TargetOSVer: Integer;
begin
  ActualOSVer := Integer(GetWindowsVersion);
  TargetOSVer := Integer(MemberToWinVersion(WinVersion));
  Result := ActualOSVer <= TargetOSVer;
end;

//------------------------------------------------------------------------------
// Returns True if: 1) the operating system is Windows 95; 2) no web-browser is installed
//
function IsBrowserlessOS: Boolean;
var
  sVersion: String;
begin
  try
    sVersion := '';
    RegQueryStringValue(HKEY_LOCAL_MACHINE, 'SOFTWARE\Microsoft\Internet Explorer', 'Version', sVersion);
  except
  end;
  Result := (sVersion = '') and (GetWindowsVersion = wvWin95);
end;

//------------------------------------------------------------------------------
// Unregister a DLL file, displaying an error message if procedure fails
//
function UnregisterDLL(const FileName: String): Boolean;
var
  DLLLongName, DLLShortName: String;
  SUnRegOK, LUnRegOK: Boolean;
begin
  DLLLongName  := GetLongName(FileName);
  DLLShortName := GetShortNameEx(FileName);

  if not FileExists(FileName) then begin
    DebugMsg('Common.iss', 'UnregisterDLL', '-1', 'DLL file not found: ' + FileName);
    Exit;
  end;

  SUnRegOK := UnRegisterServer(Is64BitInstallMode, DLLShortName, False);
  if not SUnRegOK then
    LUnRegOK := UnRegisterServer(Is64BitInstallMode, DLLLongName, False);
  Result := SUnRegOK or LUnRegOK;
  if not Result then
    MsgBox(FormatStringForVistaMsgBox(CRLFFormat('{#AppUnregisterDLLError}', True)) + FileName, mbError, MB_OK);
end;

File routines

//------------------------------------------------------------------------------
// Remove the final file extension part from a file path
// e.g., "test.txt" becomes "test"
//
function TrimFileExt(const FileName: String): String;
var
  FileExt: String;
  i: Integer;
begin
  Result := FileName;
  if Length(FileName) > 0 then begin
    FileExt := ExtractFileExt(FileName);
    i := Length(FileExt);
    if i > 0 then begin
      if i < Length(FileName) then
        Result := Copy(FileName, 1, Length(FileName) - i)
      else
        Result := '';
    end;
  end;
end;

//------------------------------------------------------------------------------
// "ADir" is a file path - the Result is the parent directory of this path
// e.g., "C:\Temp\Test" becomes "C:\Temp"
//
function UpOneDirectory(const ADir: String): String;
var
  SlashPos: Integer;
begin
  Result := ADir;
  if (Result = '') or (IsDriveRoot(ADir)) then
    Exit;

  if Result[Length(Result)] = '\' then
    Result := RemoveBackslash(Result)
  else
    Result := Copy(Result, 1, RPos('\', Result, False) - 1);

  SlashPos := RPos('\', Result, False);
  Result := Copy(Result, 1, SlashPos);
end;

//------------------------------------------------------------------------------
// Returns True if "ADir" represents the "root" of a drive
// "C:" returns True
// "C:\" returns True
// "C:\test" returns False
//
function IsDriveRoot(const ADir: String): Boolean;
var
  Drive, DriveAndPath: String;
begin
  // UNC/drive letter root?
  Drive := ExtractFileDrive(ADir);    // Returns 'C:' or '\\server\share'
  DriveAndPath := Drive;
  if Drive <> '' then
    DriveAndPath := AddBackslash(Drive);

  Result := (ADir = Drive) or (ADir = DriveAndPath);
end;

//------------------------------------------------------------------------------
// Enumerates files in a named directory, and saves the results in a passed-in
// StringList
//
procedure EnumDirFiles(const FileList: TStringList; const Directory: String; const ClearList: Boolean);
var
  PathD: String;
  FileD: String;
  FindRec: TFindRec;
begin
  if (FileList <> nil) and (ClearList) then
    FileList.Clear;

  if (FileList = nil) or not DirExists(Directory) then
    Exit;

  PathD := AddBackslash(Directory)
  if FindFirst(PathD + '*.*', FindRec) then begin
    try
      repeat
        // Only add files, not directories
        if FindRec.Attributes and FILE_ATTRIBUTE_DIRECTORY = 0 then begin  
          // We found one file that matches our wildcards
          // FindFirst returns the name of the file with no path so must account for that
          FileD := PathD + FindRec.Name
          if FileExists(FileD) then
            FileList.Add(FileD);
        end;
      until not FindNext(FindRec);
    finally
      FindClose(FindRec);
    end;
  end;
end;

//------------------------------------------------------------------------------
// Enumerates all sub-directories in a named directory (StartDir), and all child
// directories of those directories - and so on - until all directories in the
// hierarchy added to DirList
//
procedure EnumDirectories(const DirList: TStringList; const StartDir: String; const ClearList: Boolean);
var
  PathD: String;
  FindRec: TFindRec;
  Index, Count: Integer;
begin
  if (DirList <> nil) and (ClearList) then
    DirList.Clear;

  if (DirList = nil) or not DirExists(StartDir) then
    Exit;

  DirList.Add(AddBackslash(StartDir));
  Index := 0;
  try
    repeat
      PathD := DirList[Index];
      Count := 0;
      if FindFirst(PathD + '*.*', FindRec) then begin
        repeat
          // Only add directories - and don't RE-add the name of the directory we're searching through (e.g., when Count = 0)
          if (FindRec.Attributes and FILE_ATTRIBUTE_DIRECTORY <> 0) and (Count > 0) and (FindRec.Name <> '.') and (FindRec.Name <> '..') then
            // Find* returns names without path parts so we must account for that
            DirList.Add(AddBackslash(PathD + FindRec.Name));
          Inc(Count, 1);
        until not FindNext(FindRec);
      end;
      Inc(Index, 1);
    until Index > DirList.Count - 1;
  finally
    FindClose(FindRec);
  end;
end;

//------------------------------------------------------------------------------
// Pascal Script version of JCLFileUtils function of same name - find a file name
// that does not correspond to any existing file.
// If 'Simple' is true, returns the first available free name in a numbered
// sequence, e.g., if:
// test0.txt, test1.txt, test3.txt exist, would return test2.txt
// If 'Simple' is false, returns the the next available free name beyond the last
// existing name in a numbered sequence, e.g., if:
// test0.txt, test1.txt, test3.txt exist, would return test4.txt
//
function FindUnusedFileName(const FileName, FileExt, Suffix: String; const Simple: Boolean): String;
var
  I: Integer;
  TempString, NoSuffix, sSuffix: String;
  iSuffix: Integer;
begin
  I := 0;
  if Simple then begin
    Result := FileName + '.' + FileExt;

    while FileExists(Result) do begin
      Result := FileName + Suffix + IntToStr(I) + '.' + FileExt;
      Inc(I, 1);
    end;
  end else begin
    TempString := GetMostRecentBackup(FileName + Suffix + '.' + FileExt);
    if CompareText(TempString, FileName + Suffix + '.' + FileExt) = 0 then
      Result := FileName + Suffix + IntToStr(I) + '.' + FileExt
    else begin
      NoSuffix := StripSuffix(TempString, sSuffix);
      iSuffix := StrToIntDef(sSuffix, -1);
      if iSuffix < 0 then begin
        if FDebugMode then
          Log('Common.iss->FindUnusedFileName->NonSimple->Error extracting suffix from filename: ' + TempString);
        iSuffix := 0;
      end;
      Inc(iSuffix, 1);
      Result := FileName + Suffix + IntToStr(iSuffix) + '.' + FileExt;
    end;
  end;
end;

//------------------------------------------------------------------------------
function GetAllUsersAppDataFolder(Param: String): String;
var
  WinVer: TWinVersion;
begin
  Result := '';
  WinVer := GetFriendlyWindowsVersion;

  // Windows 95: 
  // Windows 98/ME: C:\Program Files\Common Files
  // WindowsNT: C:\WINNT\Profiles\All Users\Application Data
  // Windows2000/XP: C:\Documents and Settings\All Users\Application Data
  case WinVer of
    wvWin95, wvWin95OSR2, wvWin98, wvWin98SE, wvWinME: Result := ExpandConstant('{cf}');
    wvWinNT31, wvWinNT35, wvWinNT351, wvWin2000, wvWinXP, wvWin2003, wvWinXP64, wvWin2003R2, wvWinVista, wvWinServer2008, wvWin7, wvWinServer2008R2: Result := ExpandConstant('{commonappdata}');
  end;

  DebugMsg('Common.iss', 'GetAllUsersAppDataFolder', '-1', 'All Users Application Data folder determined to be: ' + Result);
end;

//------------------------------------------------------------------------------
// Copied from JEDI JCLFileUtils.pas PathGetLongName2 (with some small modifications to make it
// PascalScript compatible)
//
function GetLongName(const Path: String): String;
var
  LPath: String;
  I: Integer;
  FindData: TFindRec;
  IsBackSlash: Boolean;
begin
  LPath := ExpandFileName(Path);
  Result := ExtractFileDrive(LPath);
  i := Length(Result);
  if Length(LPath) <= i then
    Exit;   // only drive
  if LPath[i + 1] = '\' then begin
    Result := AddBackslash(Result);
    Inc(i, 1);
  end;
  Delete(LPath, 1, i);
  repeat
    i := Pos('\', LPath);
    IsBackSlash := i > 0;
    if not IsBackSlash then
      i := Length(LPath) + 1;
    if FindFirst(Result + Copy(LPath, 1, i - 1), FindData) then begin
      try
        Result := Result + FindData.Name;
        if IsBackSlash then
          Result := AddBackslash(Result);
      finally
        FindClose(FindData);
      end;
    end else begin
      Result := Result + LPath;
      Break;
    end;
    Delete(LPath, 1, i);
  until Length(LPath) = 0;
end;

//------------------------------------------------------------------------------
function GetShortNameEx(const Path: String): String;
var
  TempFile: String;
  TempFileShort: String;
begin
  Result := Path;

  TempFile := AddBackslash(GetTempDir) + ExtractFileName(Path);
  if not FileExists(TempFile) then
    if not SaveStringToFile(TempFile, '', False) then
      Exit;

  TempFileShort := GetShortName(TempFile);
  Result := AddBackslash(GetShortName(ExtractFilePath(Path))) + ExtractFileName(TempFileShort);
  DeleteFile(TempFile);
end;

//------------------------------------------------------------------------------
// Retrieve the registry-stored location of the directory within which the name application resides
//
const
  scInstallLocation = 'InstallLocation';

function GetAppInstallDir(const AppName: String): String;
var
  AppPath: String;
  KeyName: String;
begin
  Result := '';
  KeyName := AppName;

  if not GetRegistryKeyValue(scInstallLocation, KeyName, AppPath) then
    RaiseException('Error retrieving ' + AppName + ' install location from registry: Common.iss->GetAppInstallDir');
  Result := AddBackslash(AppPath);
  DebugMsg('Common.iss', 'GetAppInstallDir', '-1', 'App install directory determined to be: ' + Result);
end;

//------------------------------------------------------------------------------
// Return an appropriate Help File name (personal convention means that a Help File is always
// labelled 'HelpFile' (to make its purpose obvious); where Internet Explorer is available on the
// client PC, add a 'CHM' extension. Where Internet Explorer is unavailable, fall-back to WinHelp
// and add a 'HLP' file extension
//
const
  scCHM          = 'chm';
  scHLP          = 'hlp';
  scHelpFileName = 'HelpFile';
  
function GetHelpFileName(Param: String): String;
var
  sVersion: String;
  RegLookupOK: Boolean;
begin
  // Default to Compiled HTML HelpFile name
  Result := scHelpFileName + '.' + scCHM;
  RegLookupOK := False;
  try
    sVersion := '';
    RegLookupOK := RegQueryStringValue(HKEY_LOCAL_MACHINE, 'SOFTWARE\Microsoft\Internet Explorer', 'Version', sVersion);
  except
  end;
  
  if (not RegLookupOK) or (RegLookupOK and (sVersion = '')) then begin
    if GetWindowsVersion = wvWin95 then
      Result := scHelpFileName + '.' + scHLP;
  end;
end;

Inno Setup: a better software installer

2012-01-17T01:33:00.000+00:00

So you've written an application, written supporting documentation, designed ini, XML files or a Registry key hierarchy to store your program settings.

How do you get it all out to your users?

That question is more practically split into two separate enquiries:

In what format do you supply your files so that they can be transferred for use on another PC
How do you transfer your files to another PC

The Linux world largely addressed these issues with a single combined solution, the Advanced Packaging Tool (APT), first released in 1998. APT operates according to the idea of a software “repository”, an indexed list of software available for download (over a network, the Internet, or on local media, like a CD-ROM or USB drive).

The user browses the repository to identify software that they wish to install, then provides APT with the application (or “package” to use the colloquial terminology) name.

APT acquires a copy of the package, and installs it on the system. That's it.

APT is a console tool, but there are graphical front-ends (such as the Synaptic Package Manager) to drive these processes.

Synaptic package manager in Ubuntu

It's all very simple and straight-forward:

Any application the user might wish to install is listed in a single central repository (Apple and Andriod users and developers will be familiar with this idea and its benefits from the Appstore and Andriod Market)
APT includes a very sophisticated “conflict resolution system” which takes care of transparently installing any resource dependencies listed by a package. If an application requires additional shared libraries that are provided by another vendor, APT will download these – and take care of figuring out which revision is most appropriate – before proceeding with the installation.

The Windows environment has had a more varied history. Until the advent of Office 2000 there was no operating system level support for managing software installs beyond components of the OS itself (adding or removing elements such as Windows Media Player, Notepad, Networking support etc.).

Publicly available from June 1999, Office 2000 was the first shipping product to make use of a new Microsoft technology called Microsoft Installer 1.0. This was pushed to the target PC first, and then assumed control of the installation procedure for the main office suite, remaining on the host machine once this had run to completion.

Version 1.1 followed in February 2000 as an integral part of Windows 2000. This minor version update was also compatible with the Windows 9x line.

More recently version 5.0 was delivered as a component of the Windows 7 and Windows Server 2008 R2 operating systems.

Windows Installer versions, courtesy of Wikipedia

Windows Installer is a framework for managing software on the Windows platform. It doesn't take the form of an application that, as a computer user, you launch and run independently. Rather it is a set of services that are triggered in response to user interaction with a Microsoft Installer file (a file with a .msi extension), e.g., a double-click in Explorer, or passed as an argument to the msiexec Windows Installer Tool.

A Microsoft Installer file, an “installation package”, uses COM Structured Storage: a technique for combining lots of separate files into one contiguous file, and the methods for reading and writing to them in an abstract way – without reference to the true nature of the package file, allowing contained files to be accessed as though they existed individually.

All application files, directories, Registry keys, shortcuts, and other data (“components” in the MSI vernacular) is bound up in an installation package file.

When the file is executed (internally it is passed off to the misexec tool), the user is presented with a graphical “wizard” style interface and steps through a series of consecutive forms of the options made available to them (e.g., destination directory, registration details, language options etc.).

msiexec creates an equivalent “undo” action for every change made to the target PC, so modifications can be “rolled back” in the event of an error, user cancellation, or upon program removal.

It also supports “patching” using .msp files that modify the state of an existing installation.

Note, though, that no facility exists to address the second of our two early questions. There is no unified solution to the problem of locating Windows desktop software to install. There is no single Internet hosted repository (or offline catalogue) that users can look to as a guide for identifying applications which fulfil a requirement.

There is no vetting system, no merit system, no robust guarantee of safety from malware – and since Microsoft, with all of the resources at its disposal, does not provide this service to third party developers, no other vendor, lesser enriched, is ever likely to do so (this situation may change over time for Windows 8 Metro applications, which will only be available through the Windows Store).

Users are left in the unenviable position of trying to guess which software might meet their needs. For this they might consult Google, a “ratings” website, or ask some of their friends.

But in an age of ranking wars, search-engine-optimised page results, crowd-sourced “user reviews” and user naivety of best-of-breed applications, these are unlikely to prove reliable measure of usefulness.

Even when a suitable software titled has been successfully isolated, many unscrupulous websites simply re-package popular existing software in adware-laden containers (viz. VLC http://bit.ly/ohpD4b), booby traps for the unwary, luring them away from canonical download sources.

Not to mention the confusing world of trialware, freeware, open source, for-personal-use-only or other licensing variations that can confuse the uninitiated.

In fact, we can abandon any idea of coming to a resolution for this problem – there simply isn't one – and instead focus entirely on ways of dealing with the first portion of our original question: the format in which application files are supplied to end users.

Leveraging the tools

So as of Windows 2000, the Microsoft Installer framework became an integral part of the operating system. However, by that time a host of bespoke and niche tools had sprung up to fill a gap that had existed since the genesis of the Windows shell.

The requirements were simple (consider, also, that this was in an age of narrow-band Internet access):

minimize the number of files to be distributed (irrespective of the number of files to be included in the distribution).
minimize the size of the file(s) to be distributed.
determine a way of incorporating programming logic to orchestrate directory creation, file copying, Registry updating, etc. at the user end.

Other requirements might also be desirable, a mechanism for automatically deleting all components of an installed application when a user decides to remove it, for example, but early efforts had more modest objectives.

Some of the tools that stepped in to try to satisfy these requirements have long since fallen out of favour. It wasn't uncommon – even until recently in the case of web site authoring app KompoZer, for example – to send out an “image” of the full application as a compressed ZIP file.

WinZip later began marketing a standalone tool for “self-extracting” ZIP files, and added the same as a reduced-capability feature to their flagship WinZip product.

Compressed archives were useful for minimizing file sizes and payload, but they offered no help in tackling the other major issue, embedding some kind of scripted control of installation activities on the end user PC. At this point the onus remained with users, other than directions from a help file, it was their responsibility to decide which directories to create, and files to extract. But this was clearly a job best left to the developer, familiar with the intricacies of their own software and the landscape of the operating systems they were targeting.

Several more advanced utilities followed in quick succession towards the end of the 90s, among them Wise, InstallShield, and the Nullsoft Scriptable Install System (from the WinAmp stable).

These were all independent projects, they had overlapping functionality, they were all completely self-contained installation “engines”. None of them used the Microsoft Installer API, they evolved before it had been created, but they all implemented a comparable set of features.

It was into this climate that Jordan Russell released the first iteration of Inno Setup in 1997. Inno Setup is written in Delphi, and in the great tradition of these things, created out of dissatisfaction with the status quo. In common with its contemporaries, Inno Setup is an autonomous installation engine, it has no external dependencies, it doesn't rely upon the presence of an existent set of operating system services, just stock, plain-vanilla WinAPI.

It has maintained this autonomy, building the code base to expand its repertoire of new and unique features (see below), throughout the course of its history: anything you might require of it, as an install builder, is natively supported or easily scriptable.

Contrast this with the approach taken by the two dominant commercial alternatives, Wise and InstallShield. Both moved away from in-house back-end architectures, and retooled their products around the Microsoft Installer framework.

Subsequent years saw a tit-for-tat legal battle between the two companies, with allegations of patent infringement and even espionage (http://bit.ly/xH2OOw and http://lat.ms/ysU7DU). Legal wrangles and buyouts of both companies (Wise by Altiris, and subsequently by Symantec, InstallShield by Macrovision, then Acresso Software Corporation, now called Flexera Software) went ill for Wise, Symantec end-of-lined all products by the close of 2011.

This convergence around a common, generic solution – out-sourcing back-end functionality to the Microsoft framework – saps competitive advantage and innovation.

Going Inno

The Microsoft Installer framework was designed to provide developers with a set of interfaces for publishing their software to Windows. But because it's baked into the OS, and it's seen several updates over time, many of the more advanced features have only become available in recent Windows OS'. The most recently supported MSI for the Win9x family is 2.0; the most recently supported version for Windows 2000 is 3.1.

This page - http://bit.ly/ziwvdS – shows a list of functions unavailable prior to revision 4.0, including the much touted “advertise” feature₁, the ability to install 32-bit applications on 64-bit Windows, access to the Restart Manager (which advises of those running applications which should be shut down prior to, or restarted following, updates or installs – as opposed to the entire OS), visual UAC cues, prompting for “files in use”, and a host of others (see also “What's new in Microsoft Installer 4.0” and “What's new in Microsoft Installer 4.5” http://bit.ly/wfzcSp, http://bit.ly/x7y7Nr and http://bit.ly/zv5m8A).

But more pragmatically, vendors with products built around the MSI framework are keeping parity with Microsoft's support schedule for its desktop operating systems, e.g., InstallShield's installation packages require target systems to be running Windows 2000 or later (http://bit.ly/wuzK0p).

Non-standard actions - “Custom Actions” - are complicated to add to a MSI package. MSDN gives a few examples:

Launching an executable during installation that is installed on the user's machine or that is being installed with the application
Call special functions during an installation that are defined in a dynamic-link library (DLL)
Use functions written in the development languages Microsoft Visual Basic Scripting Edition or Microsoft JScript literal script text during an installation
Defer the execution of some actions until the time when the installation script is being executed
Add time and progress information to a ProgressBar control and a TimeRemaining Text control

These, and other “custom” functionality, must be implemented in separate DLL files (http://bit.ly/AEfg1t).

Further, scripting is only available via VBScript or JScript; users more familiar with other common languages and dialects, the C#, C++, F#, Python, Lua, Delphi et al crowds, are limited to these choices, with as much re-learning of syntax and method libraries as that might entail.

With all of this in mind, lets consider why Inno Setup may be a better solution for you, particularly – but not exclusively! - as a Delphi developer.

An installer built with Inno Setup can be deployed to every 32 and 64 bit Windows version. An installer file will run equally well on everything from Windows 95A to Windows 7. It will run happily on unpatched Windows NT and Windows 2000 systems.

Inno Setup is available in ANSI and Unicode flavours. The edition you choose to install (they can be installed side-by-side) will determine which of these formats will be supported in the installation packages you build. The Unicode edition supports Unicode text throughout the IDE and on every form, control and dialog of your installation packages.

Inno Setup with Hebrew text

Scripting, which isn't required to build an installation package (a fully functional installer can be put together by filling in just a few fields of text), offers tremendous flexibility for customizing the behaviour of your installer. There are, for example, “events” that are fired at various stages of installation (and uninstallation), and you can use any of these to prompt the user (e.g., for registration details), validate user input, show custom forms, download files, show help, cancel or rollback the install, launch an installer for a dependant application (e.g., the .Net framework), register DLLs, or any other task that isn't supported as an option in the IDE or involves more complicated logic.

Inno Setup uses PascalScript (from RemObjects) for this purpose, which implements a large proportion of the functions and classes available in native Delphi, and shares identical syntax and the language's strong typing. In fact, you could conceivably write an entire “application” using only Inno Setup, with custom forms and PascalScript to deal with processing duties – something akin to a “Microsoft Access application”, but with a different emphasis.

Eighteen example script files are included with the distribution, and the Inno Setup Help file is very thorough, with lots of annotated code snippets alongside written explanations.

Inno Setup is both freeware (with no usage restrictions), and open source. The source code is available for Delphi versions 2 through to 2009.

When you start Inno Setup a dialog box appears with several pre-defined options, including a Wizard to help guide through the process of creating a basic installer.

This can be modified later (to flesh out any extra detail that you might want to add), or used as-is: just a few clicks and a handful of fields to fill out and you have a finished install package. The install package is a single .exe file containing everything that you want to include with your distribution. Inno Setup applies compression to all of the files that are bundled into the package with zip, bzip, lzma or lzma2.

Regular updates: Inno Setup is under constant development, Jordan provides access to the very latest sources from CVS and Git repositories, but the stable binary release is updated several times a year and supports the latest operating system features (e.g., Windows 7 taskbar buttons, code signing, UAC considerations).

Third party support: there are quite a few separate tools and add-ons that are available from links on the Inno Setup download page, including:

InnoIDE, a graphical alternative to the text editor provided with Inno Setup. A potential time saver in that it leads you through the data gathering process, like filling in a questionnaire, instead of having to read through the documentation to see what options are available. There are also forms for generating code that corresponds to the values set in visual controls: for example a “Message Box Expert” allows the user to define the appearance of a message dialog then provides the PascalScript to invoke a matching dialog at runtime.

InnoIDE Message Box Expert for dialog design
InnoScriptStudio, another graphical script building tool with very similar goals to InnoIDE.

InnoScriptStudio options form
ISSkin, a “skinning” add-on that can be used to change the visual style of your finished installer package. See also Graphical Installer, a commercial (“pay what you want”) product along the same lines.

One of the 16 or so skins available with the ISSkin extension

Skinning with Graphical Installer

...and half a dozen others that are less ambitious in scope.

Summary

I've been using Inno Setup for several years, and over that time I've made some pretty unusual demands of it but it's always been possible to achieve what I've wanted.

It strikes a perfect balance between ease of use and breadth of features; you can create a fully-functional installer in a couple of minutes, or get deep into the code and bolt on some really creative extensions.

It's a mature, stable, open-source product with regular updates, an active newsgroup and a lot of supporting websites.

If you're a Delphi user you'll be instantly at home with PascalScript, for other users the Pascal syntax is easy to pick up (it was invented as a teaching language, after all), or you can stick to one of the visual editing tools to generate some of your code.

Useful resources:

Inno Setup main page:

http://www.jrsoftware.org/isinfo.php

Inno Setup Extensions Knowledgebase:

http://www.vincenzo.net/isxkb/index.php?title=Main_Page

Graphical Installer:

http://www.graphical-installer.com/joomla/

Inno Setup Unpacker (a tool for extracting files from a compiled installer package): http://innounp.sourceforge.net/

₁ “Advertising” is a feature available in recent versions of the Microsoft Install that allows an application or feature to be made known to the user – e.g., with a Start Menu shortcut – which has not yet been installed. Clicking the link or activating the feature will cause an “on demand” installation of the item.

Batch file driven compilation

2012-01-11T00:20:00.000+00:00

Building a Delphi project with batch files (and a handful of command line tools)

The humble batch file. Ubiquitous since the days of MS-DOS, batch files offer a means of chaining together a list of commands to do work that requires the involvement of several executable files in a sequence.

Little changed in the era of Windows 7 & 8, batch files can be used to automate tedious and repetitive tasks in a way that can be replicated across every member of the Windows family.

This is precisely the objective I had in mind when it came to co-ordinating all of the tools involved in building a large Delphi project. For demonstration and “toy” projects, it's generally sufficient to compile an executable from the IDE, and then copy or run the resulting binary as-is.

But when you're putting together something bigger there are real benefits to a consistent script-based approach to directing all of the separate applications and actions that you might need to produce your final distributable.

There are a slew of commercial and free/open source visually orientated tools that can help you with this, but I didn't want to learn my way around another user interface and work flow when I already had a clear idea of how everything would fit together.

Automated Build, and Continuous Integration tools (http://bit.ly/zkGkdm)

If you prefer pointing-and-clicking to the utilitarianism of the command line interface, there are plenty of graphical tools that can help you take control of your application build process.

Some of the most frequently referenced include the open source Java-based CruiseControl and derivatives (http://bit.ly/qqPY), Team Foundation Server from Microsoft (http://bit.ly/QiewU), geared towards Visual Studio, and FinalBuilder, a commercial product written in Delphi (http://bit.ly/wIraXy).

It's very easy to log events as they happen using commands within the batch file, and conditional expressions allow you to branch the order of processing depending on system or “variable” states. In other words, you lose little in the way of functionality by choosing a command line build system.

And there's a certain satisfaction to the UNIX-like practise of teaming a collection of small, purpose specific tools to create more or less complex behaviours, and working directly against the command prompt.

Batch file fundamentals

Wikipedia is a useful starting point for details of some of the commands that can be invoked from a batch file, but I've also found the Google Groups archived Usenet posts an invaluable reference source:

http://en.wikipedia.org/wiki/List_of_DOS_commands

http://en.wikipedia.org/wiki/COMMAND.COM

http://en.wikipedia.org/wiki/CMD.EXE

https://groups.google.com/grphp?hl=en

http://ss64.com/nt/

http://bit.ly/3YZVaa

Rather than go through each command individually (both the Wikipedia List of DOS commands article and in-built help command issued from the command line provide all of this information. In addition help can be called with a parameter – the name of another command – for a more detailed explanation, e.g., help if), I want to highlight a couple of lesser known commands, and some of the “programming like” capabilities that they afford us.

The first of these is CALL. The Windows help for the CALL command says:

Calls one batch program from another.

This concise summary belies its enormous utility, particularly when used in conjunction with the second important command, or instruction, LABEL. The Windows help description for CALL continues:

CALL command now accepts labels as the target of the CALL. The syntax is:

CALL :label arguments

A new batch file context is created with the specified arguments and control is passed to the statement after the label specified.

It's non-obvious on first inspection but what this means is that you can create a library of batch file “sub routines” that can then be called from any other (or the same) batch file, in much the same way that you might use – say – a “Strings” library unit in Delphi.

The only complication here is that a batch file sub-routine can't return a result: it can only set a global “result” variable (or any number of global variables – none of these is actually a sub-routine “result”, but can be treated as such) that are visible to, and can be accessed by, the caller.

Lets have a look at a short example to see this in action. Here we're using two batch files – a.bat and b.bat. a.bat is our “main” file, b.bat is our library file. a.bat is going to read a list of names from a file and, by calling a routine in b.bat, determine the longest name in the file then print this on screen.

Coding conventions

"Constant" environment variables. Although these are indistinct from other variables, I've adopted a naming scheme that sets them apart from environment variables that may be updated later in time. For these constant variables, the first letter of the name will always be a lower-case 'c' (for “constant”); for variables that may change, the first letter of the name will always be a lower-case 'v' (for “variable”).

The only exceptions to this rule are for variables that are used “globally” (from any point within the batch file). The first letter of the name for these variables is always a lower-case 'g' (for “global”), and then followed by a second letter 'c' or 'v' depending upon whether or not it might be modified after its declaration.

a.bat script listing:

1:   @ECHO OFF  
2:    
3:   SET viLongestName=0  
4:   SET vsLongestName=  
5:   ECHO JOHN   >  c:\temp\names.txt  
6:   ECHO JIM    >>c:\temp\names.txt  
7:   ECHO THOMAS >>c:\temp\names.txt  
8:   ECHO MARK   >>c:\temp\names.txt  
9:    
10:  SETLOCAL ENABLEDELAYEDEXPANSION  
11:  FOR /F %%S IN (c:\temp\names.txt) DO (  
12:   CALL "b.bat" TRUE :strlen %%S  
13:   IF !gvStrLen! GTR !viLongestName! (  
14:    SET viLongestName=!gvStrLen!  
15:    SET vsLongestName=%%S  
16:   )  
17:  )  
18:    
19:  IF NOT "!vsLongestName!"=="" (  
20:   ECHO Longest name is !vsLongestName! with !viLongestName! characters  
21:  ) ELSE (  
22:   ECHO No names found in file  
23:  )  
24:  ENDLOCAL

Deconstructing this:

@ECHO OFF turns off console “echoing” for each statement evaluated by the interpreter; with this switched off, the batch file will not print anything to the console unless 1) an error occurs, or 2) It is explicitly ECHO(ed) to the console by the batch file.

Lines 3 and 4 set variables “iLongestName” (representing the length of the longest name found so far) and “sLongestName” (representing the text of the longest name found so far) to their starting values. All variables – more correctly called “local environment variables”₁ - are represented as strings.
But we can still store a numerical value in an environment variable because the comparison operators available:

EQU – for testing whether two values are identical

NEQ – for testing whether two values are non-identical

LSS – for testing whether one value is “less than” another value

LEQ – for testing whether one value is “less than OR equal to” another value

GTR – for testing whether one value is “greater than” another value

GEQ – for testing whether one value is “greater than OR equal to” another value

...can all be used with numbers stored as strings in the same way that:

var

aString, bString: String;

begin

aString := '1';

bString := '2';

if bString > aString then

ShowMessage(bString + ' is greater than ' + aString)

else

ShowMessage(aString + ' is greater than ' + bString);

end;

Will work as expected: the message: “2 is greater than 1” would be shown (bString evaluates to “greater then” aString).
Lines 5, 6, 7 and 8 write out some names to a text file, c:\temp\names.txt

Something to note here, the ECHO command usually prints a string to the console, but in this instance we use '>' to “redirect” the string to a file.

Using '>' overwrites the content of a file with a new string; using '>>' appends a string to the existing content of a file (or creates a new file and adds the string to it if no file of the given name currently exists).

The additional spacing on some of these lines (i.e., between the names and the '>' or '>>' characters) is added for visual clarity and has no bearing on the way that they are evaluated by the interpreter.
Line 10 sets ENABLEDELAYEDEXPANSION. This is required because we're updating variable values in a loop. Without ENABLEDELAYEDEXPANSION, variable values are fixed when the script is initially parsed (each variable keeps the first value it is assigned); by setting ENABLEDELAYEDEXPANSION variables are instead updated at runtime, so can be modified in the loop. Every declaration of ENABLEDELAYEDEXPANSION must be paired with a closing ENDLOCAL, which signifies the end of a delayed variable expansion block.
Line 11 sets up the loop which iterates over all of the lines in thenames.txt file.

The command line interpreter For command is very versatile, with the ability to walk directories and process all or a subset of the files found. Each file can be parsed line-by-line and tested, for example, for specific End Of Line characters, or broken up into individual tokens for further processing - including evaluating tokens at numbered indexes along each line.

In this example, For /F tells the interpreter that we're operating on one or more files, and %%S is the syntax for the loop variable that stores each line of each file as it is parsed out ('S' could be any character, I've chosen 'S' as shorthand for “String”).

IN precedes a parenthesis enclosed set of files (the wildcard character may be used here; individual file names within the parenthesis should be separated by a space) to loop over, and DO precedes a list of commands to perform on them – or in our case, because we're using the/F parameter - to perform on each line of thenames.txt file.
Line 12, the first to be processed within the scope of the FOR loop (which is delimited by the opening and closing parenthesis) uses theCALL command to switch processing to b.bat, and passes CALL three parameters, TRUE, :strlen and %%S. These three parameters are presented to the CALL(ed) batch file, which can retrieve them as command line parameters - not unlike the ParamStr(<n>) function in Delphi – and use them as arguments in the named sub-routine.

The parameters could, of course, be anything (any String values), but they have special meaning in the context of this example.

The first parameter, TRUE, is used to determine whether to set a FAIL errorcode if a problem occurs in the called batch file. The second parameter is a label that identifies the name of the sub-routine to execute in the called batch file. The third parameter contains a line of text from the names.txt file.
Line 13 is a conditional test, using IF, of the “result” global variable set by the sub-routine in b.bat. If the global variable evaluates to “greater than” (GTR) the value currently stored in iLongestName, then processing continues to the two lines below, which respectively assign this value to iLongestName, and the text stored in %%S to sLongestName (i. e ., keeping a running count of the longest name seen so far).
When the FOR loop terminates (line 17), execution continues to the test on line 19. sLongestName is compared against an empty string (if the names.txt file contained no entries, sLongestName would never have been modified from its starting value). If it has been assigned a name from the file, the name and its length are printed to the console.

The comparison is made using the == operator, but could alternatively have used EQU. The test employs a common convention when checking against an “empty” value. A variable cannot be tested against nothing (i.e., it might be assumed that leaving a space after the equality operator would be equivalent to testing against an empty String – but this is not so – a command must follow the equality test, and so a second test value must come ahead of this).

By surrounding the variable to test with quotes, and adding a pair of consecutive quotes on the other side of the operator, the interpreter will return TRUE if no value has been assigned to the variable (from the perspective of the interpreter the comparison would resolve to: “”==””).
A final point about this first code listing: variables referenced outside of a ENABLEDELAYEDEXPANSION/ENDLOCAL block are represented using the %VariableName% syntax (each variable must have a percent sign at the beginning and end); variables inside of a delayed expansion block that are to be updated must instead be represented using exclamation marks as variable name delimiters, e.g., !VariableName!

Now lets examine the code listing for b.bat

1:   @ECHO OFF  
2:    
3:   REM Contains functions used in more than one batch file  
4:   REM Prerequisities: None  
5:   REM NOTE: for all functions, first parameter must be a boolean that indicates whether to issue a 'fail' errorcode if a problem occurs  
6:   REM NOTE: for all functions, second parameter must be a label that indicates which function to call  
7:    
8:   SET cErrOnFail=%1  
9:   SET cFuncName=%2  
10:  SET cUnitTempFile=%TEMP%\Temp%RANDOM%.txt  
11:    
12:  FINDSTR.EXE /n ^%cFuncName% b.bat>%TempOutputFile%  
13:  SET /p cLblExists=<%TempOutputFile%  
14:    
15:  IF NOT DEFINED cLblExists GOTO error0  
16:  GOTO %cFuncName%  
17:    
18:  REM Return the length of a string passed in as the first command line parameter  
19:  REM -param1 = RESERVED  
20:  REM -param2 = RESERVED  
21:  REM -param3 = String to be measured (if more than one word, enclose in double-quotes)  
22:  :strlen  
23:  SET gvStrLen=0  
24:  IF "%~3"=="" GOTO :EOF  
25:    
26:  SET _FName=_%RANDOM%.txt  
27:  SET _FNameFull=%TEMP%\%_FName%  
28:    
29:  ECHO %~3 > %_FNameFull%  
30:  FOR /F "tokens=3" %%F IN ('DIR /-C "%_FnameFull%"^|"FIND.EXE" /I "%_FName%"') DO (  
31:   SET /a gvStrLen=%%F - 3  
32:  )  
33:  DEL %_FNameFull%>nul  
34:  IF ERRORLEVEL 1 GOTO error2

Many of the lines in this file are comments, ignored by the interpreter, and included to aid callers in understanding how to make use of it. These lines all begin with the REM(ark) statement.

The first line of interest is 8: lines 8, 9 and 10 all set “constant” environment variables that will not change for the duration of the execution of the batch file. The variables on lines 8 & 9 are both assigned values from the “parameters” passed in by the calling batch file, a.bat (which are stored in %1 and %2, though the series could run up to %n, where 'n' is the index of the last parameter).

cerrOnFail is set to TRUE (i.e., a String value containing the text “TRUE”, rather than a Boolean) and determines whether to return an error code if something goes wrong (the code listing for b.bat is part of a longer batch file, and cErrOnFail is not used in this abridged version).

cfuncName holds the value of the second parameter, which is the name of the sub-routine that the caller wants to execute in b.bat (in this case the :strlen routine).

cunitTempFile is assigned a file path in the TEMP directory (using the pre-defined %TEMP% environment variable – a complete list of pre-defined environment variables can be viewed by typing SET at the command prompt), a name that begins “Temp”, and is followed by a random number provided by the %RANDOM% environment variable, and “.txt” extension.

This doesn't guarantee uniqueness, but it's good enough for our purposes. This will be used as a “scratch” file, a temporary dumping place for data that will be used later.
Line 12 uses the FINDSTR command to parse b.bat for a token matching the name of the sub-routine we want to execute (“:strlen”). The /n switch causes FINDSTR to return the line number if a match is found, and this is written out to our scratch file using redirection '>'.
Line 13 calls SET with the /p switch, which reads in a String from a named file instead of accepting a value typed interactively at the command prompt. Here the “constant” variable cLblExists is assigned the line number written to the scratch file by FINDSTR.
Line 15 checks whether the call to SET was successful. If FINDSTR did not locate “:strlen” inside of b.bat, nothing would have been written to the scratch file and variable cLblExists would be unassigned. If cLblExists is unassigned, this test will fail and processing will jump to the position in b.bat indicated by the label error0 using the GOTO command (note that, as with GOTO in Delphi, processing does not return after the code following the label has been executed).
Line 16 is the last portion of the “lead in” part of b.bat, before work begins inside of the sub-routine specified by a.bat. The GOTO command causes processing to jump to the sub-routine name stored in cFuncName.
Line 22 is the label for the :strlen sub-routine (the “entry point” for the function, as it were).
Line 23 initializes a global variable (global because all batch file variables are, but also because it is used to store the routine “result”, and so will be referenced outside of “scope” of :strlen) named gvStrLen to its starting value of 0.
Line 24 is a test which evaluates whether the third parameter passed to b.bat (the String to have its length measured) is empty, and if it is, skips to the EndOfFile (:EOF – a pre-defined label), after which the interpreter returns execution to a.bat.
Lines 26 & 27 set two variables that store the name and fully qualified name of a scratch file to use for temporary storage.
Line 29 writes the passed-in String to the scratch file.

Line 30 is a compound statement. The FOR command is used with the /F switch. In this instance it's indicating that we're working with a String (which will be returned by FIND at runtime), although it can also be used to indicate operations on a set of files, as we saw in the FOR loop in a.bat.

In additional the “tokens” option is specified with a qualifier “3”. This tells FOR that we want it to search for the third token (note that, because we're not specifying the “delims” option, the default token separator - a space or tab - will be used) in the String that will be given to it by FIND.

The section of the statement following IN (inside of the parenthesis) is processed first. The DIR command is called with the /-C switch and the path to the scratch file as a parameter. When supplied with a file name as a parameter, DIR performs a FindFirstFile on the file path, rather than a directory enumeration. Calling DIR with /-C means the output from the DIR(ectory) listing will print:

1) the file creation date

2) the file creation time

3) the file size in bytes

4) the file name

But it will also print a summary “table” with volume and directory information. To remove this, the output from DIR is piped (using the “pipe” | character – which must be escaped in this statement to prevent it from being parsed early, by inserting a carat ^ character immediately in front of it) into FIND. FIND searches through the DIR output for a line containing the name of the scratch file, and returns this line alone, minus the volume/directory information table.

This line is then passed to the FOR loop, which extracts the third token from the line – the file size in bytes – and stores it in the gvStrLen variable.

It is this value that is equal to the length of the passed-in String. This works because the scratch file is otherwise empty, has no binary header or other fields, and so is comprised entirely from the bytes representing the text it contains (with each character one byte in length).
Line 33 deletes the temporary scratch file from disk (we clean up after ourselves). Notice that the output of the DEL command is directed to the special nul device, which essentially swallows it and prevents it from appearing in the console. Why do this? Deletion of the temporary file is not critical to the outcome of the sub-routine call so we don't worry if it fails to run successfully.
Finally, line 34 checks whether an error was signalled by DIR or FIND (i.e., if either command set the environment variable ERRORLEVEL by way of an exit code upon termination), and if so, skips to a label called error2, which is omitted from this listing, but would record the error to a log file before exiting b.bat.

Character escaping

In much the same way that a single quote character must be escaped in a Delphi String, or a backslash character in a C String, any percent character must be escaped inside of a batch file to distinguish it from a variable.

Practical application

So how would these ideas be applied in building a Delphi project? As a worked example I'll discuss the steps involved in building my own project using the batch files I created to automate each stage.

Coding convention

My batch files are divided into two categories, “work” files, and “library” files. A work batch file does a job – compiles a Delphi project, uploads a file to a FTP server, creates an installer – the library files are listings of independent sub-routines that are used by the work files.

To prevent a “work” batch file from being called more than once (if everything has been structured correctly this shouldn't happen anyway), each file is given a special global variable of the form:

BUILD_<FILENAME>

And this variable is assigned the String “ACTIVE” the first time the batch file is called. The variable is tested each time the file is invoked (using the DEFINED command, which tests whether a variable has been previously declared), and if it has, processing skips to the end of the file.

The first step involves setting environment variables containing paths to frequently accessed files and directories, including:

The path to the Build directory (where the batch scripts are located)
The path to the UNIX tools directory (more on this below)
The names of .inc files included (using the $I compiler directive) in more than one project
The path to the Windows System(32/64) directory
The paths to some scratch files for temporary String storage
The path to a log file for recording errors
A drive letter mapped to the directory containing all other project folders (using the SUBST command – all of the folders exist on a local hard drive, but mapping a drive letter with SUBST made it easier to work with them)
The path to the Delphi folder; the path to the Delphi Source folder

At this point all other batch file global variables/constants are also set to their default starting values. I've gathered this collection of variable assignment commands into a batch script called “Globals.bat” and this is always the first file to run. Every other batch file includes a check which forces Globals.bat to be called before executing any of its own commands if the Globals.bat special global variable is undefined (see box-out above).

This way we can be certain that all of the files and directories any of our other batch files might need to know about will have been already been set and are ready for use.

The second step is to prepare the way for building all of the application executables, which consist of the main program binary, and a collection of helper utilities and DLL files (all are separate Delphi projects).

This stage does things like:

Ensure each project has an associated DCC32.CFG file
Ensure the paths inside of each DCC32.CFG file point to the correct locations (to make it easier to move the build process from machine to machine, where the Delphi source files may reside in different directories)
Update configuration files for shared resources, e.g., I use FastMM4 as my drop-in Memory Manager replacement (available from SourceForge, but supplied as standard with Delphi 2006 and later), and I keep two separate configuration file “profiles” depending upon whether I'm debugging, or building a “release” binary

These various processes are all performed by a batch file called “FileUpdates.bat”.

The Delphi DCC32.CFG file

When you compile an application in the Delphi IDE, the compiler uses the Library paths variable (which is persistently stored in the Registry) to find the location of any unit files not contained within the project folder.

You can inspect the Library paths value by clicking Tools->Environment Options, selecting the Library tab and then clicking the browse button alongside the “Library paths” edit box.

When compiling from the command line with the DCC32 compiler, the Library paths variable is not used. Instead paths to folders containing all:

Delphi compiled unit files (.dcu)

Delphi source code files (.pas)

Resource files (.res)

Included files (.inc)

...used by the project that are outside of the project folder must either be passed as arguments to DCC32, or (more practically, because it's likely to be a long list) added to a file named DCC32.CFG which is stored in the project folder*.

Alternatively, the two methods can be mixed – you can enter a set of paths as command line arguments to DCC32 AND include additional paths in a DCC32.CFG file (though the paths passed directly from the command line have a higher precedence).

Any compiler directives must also be specified in this way (as command line arguments or in the DCC32.CFG file).

*It can also reside in the same directory as DCC32.EXE, but this can be tricky since each project is likely to have a different set of dependencies. Creating an all-encompassing DCC32.CFG file with paths to every Delphi unit on disk could be one way around that, if a rather unwieldy solution.

The third step is to build each of my projects using the Delphi command line compiler, DCC32.

While the final distributable - an installer application created using Inno Setup by Jordan Russell – is a single file, it bundles together all of the binaries, help files, license files, ini files and other resources that constitute “my application”. The sequence of commands involved in compiling any of these projects is much the same, so we'll limit our examination of them to the “main” application executable.

The batch file for compiling this executable can be roughly divided into eight sections. In order these are:

Setting unit-wide variable values for: the path to the folder containing the icon for the executable; the name for the compiled file (which is distinct from the project name); the path to a scratch file for compiler error logging; the path to an anti-tampering utility which marks the compiled file with a CRC stamp (more on this below).
Taking a backup of the DCC32.CFG file in the project folder, and replacing the original with an updated copy created during Step Two.
Regenerating the project .res file to include the most recent version of the application icon (and “labelling” the build in the Version Control System used for this application – more on this below).
Regenerating all other binary .res files from their counterpart .rc files for resources compiled into the executable using the $I compiler directive (these include additional icons, internationalized text, copies of ini files should the user want to restore default settings etc.).
Generating a new serial number for the “trial” version of the application (more on this below).
Invoking DCC32 to compile the project.
Modifying the compiled executable icon using a third party utility: prior to Delphi 2007, “Vista style” icons (up to 256x256 pixels, with PNG compression) were not supported via the IDE (though later versions are reported to have some problems, resolved finally in XE, see http://bit.ly/v1ozEs). There are several solutions to this problem (see http://bit.ly/uPjrDr), but I plumped for ReplaceVistaIcon from RealWorld Graphics (http://bit.ly/bjLlci).
Moving compiled executable and map files out of the project directory, renaming the executable to match its “market” name, and stamping the executable with a CRC value.

My Delphi project is a shareware application with two release versions, a “trial” edition, that expires after a fixed number of uses, and a “retail” version that has no usage limitations.

It's also a little unusual in that it has a “boot” loader (my term), a small standalone executable that's launched first. This checks whether it was started from a desktop short-cut or the console – the “main” executable is then started by the loader with a switch indicating which of these modes to adopt (it can run with or without a GUI).

As an attempt at a double line of defence, I integrated shareware protection components from two vendors, both open source: mxProtector from Max's Components for the loader, and OnGuard from TurboPower for the main executable. Each has a multitude of options for configuring the method of protection, and in either case I went for a “maximum uses” limit, but also enforced a 365-day cut-off (from the date of compilation, and regardless of total uses) with OnGuard for the main executable. Both components encode this information in a string (or “serial number”, if you will) that is stored in a separate text file.

OnGuard also includes a utility (StampEXE) that can take the CRC (a short value calculated from the content of a file - http://bit.ly/MTih2) of a compiled file, and then embed this value within the file. Each time the file is run it performs a self-test to check whether it has been modified in any way.

The hobbled-shareware model is a bit passé these days, but that's the topic for another discussion.

Because I planned to add incremental improvements over time, updating both trial and retail versions on a regular basis, one of my main motivations for scripting the build process was to make all of this a single-click process.

The 365-day cut-off that I'd built into the trail version using the OnGuard component would obviously be a problem for users who downloaded it towards the end of this time-frame. So I wanted to include updates to the serial numbers as a part of the overall build process. That way, with frequent (bi-monthly) builds and web uploads, users would never be exposed to a “stale” trial product that was nearing the end of its shelf life.

Version Control System

When it came to choosing a Version Control System for managing my source code and associated project files, I wanted something that had some integration with the Delphi IDE. There are countless VCS packages to choose from - Borland's StarTeam, Mercurial, CVS, Subversion, Bitkeeper and Git, to name a small number of the more well known examples – but I wanted to stick to the Delphi ecosystem so I settled on the JEDI VCS, hosted at SourceForge.

It doesn't support distributed revision control (http://bit.ly/dtDJ1), which stores a local copy of the repository for each client, allowing many users to work with the code base offline and tools for merging changes, but this was a single user project so that was never going to be an issue. Regardless, the more traditional approach of the JEDI VCS, with a single central database, and file check-in/check-out is notionally simpler, and it's proven to be a very reliable solution (the client is chock full of features, and has all sorts of useful built-in tools for doing things like side-by-side diffs).

One of the features I've come to appreciate most is the Label Manager. Quoting from the supplied Help File:

"Labeling modules [a JEDI VCS term for any file that can be stored in the repository] is recommended as a way to freeze a moment in the development cycle. With a label, you can easily find, get and work with a development state that has been identified as significant in the development cycle."

By creating a new Label and then “stamping” (associating it with) all files used in a build, I can create a “snapshot” of my project at the time of compilation. If a problem is later discovered with that executable, I can go back into the JEDI VCS and recreate the project exactly as it was when the executable was compiled, and verify whether or not the bug has been fixed in later releases.

The JEDI VCS client can be driven from the command line, so it was trivial to add a batch file command for doing this automatically for each build.

Step four creates the application Help files: these are supplied in two formats, Compiled HTML (.chm) for Windows 98 and later, and WinHelp (.hlp) for Windows 95. Again, there are dozens of different editors, commercial and freeware (http://bit.ly/kmjnc2 – several are Delphi applications, Help & Manual is one, Precision Helper is another, and freeware to boot), that can be used to write documentation, but I use Help & Manual from EC Software.

Help & Manual can be launched from the command line with switches for automating different tasks, such as “compiling” written content into one or more help file formats (HTML, PDF, CHM, etc.). I've included the batch file (HelpFile.bat) I use for this step in the companion files archive at the end of the article, but because of the diversity of help file authoring software, it's not likely to be of much use without the necessary modifications to support other clients (if these support command line compiling at all).

The fifth step pulls together all of the files distributed with the application into a single stand-alone executable. I use Jordan Russell's Inno Setup (http://bit.ly/17JDQN) to create an installer that can be downloaded from my website.

All of the logic for identifying which files to include, whether or not compression should be used on included files, which platforms to target and so on is contained within a handful of Inno Setup scripts that are parsed by the Inno compiler. Therefore the batch file (Installers.bat) which launches Inno Setup and directs it to compile these scripts is relatively short and simple. Because two separate installers are created, one for the shareware, one for the retail version, there is some special handling; a FOR loop calls Inno Setup once for each of these versions and updates a text file containing a “build number” (shown on the installer title bar) as it goes.

The sixth and final step takes the finished installers and uploads them by FTP to my website. This batch file, FTPUpload.bat:

Sets variables for the FTP host, FTP server path and local path to the installer executables
Uses a command line FTP client to upload the installers to the FTP server

Windows includes a command line FTP utility – called “ftp” - but repeated upload tests always resulted in corruption of any binary files sent to the server. A Delphi console application using the Indy socket library was created instead, and this is now used to transmit the installers to the server.

Step six rounds out the cycle. The entire process takes about 10 minutes, but once I've hit RETURN on the first batch file in the sequence, it all runs to completion unattended. I encountered some problems early on in testing that seemed to point to an obscure issue with the command line interpreter (a highly unlikely though not unprecedented scenario! http://bit.ly/eYx8pA), but lengthy investigation revealed my anti-virus software to be the culprit. It was interfering with Inno Setup during compilation (as a result of heuristic scanning of the installer executable as it was being assembled); disabling it during active builds prevented the error from re-occurring.

3rd Party Tools

Although there's a tremendous amount that you can do using only commands available from the interpreter (i.e., supplied with the Windows operating system), constructing sub-routines to perform, for example, complex file parsing can be extremely long-winded.

As a short-cut, or where some functionality beyond the scope of what's available with native commands is required, I've made use of a set of console-only tools that have been ported from the UNIX world.

There are several projects that have a similar ambition, Cygwin and Gow are very popular, but these are either more heavyweight than I needed, or include installers that make modifications to the Windows environment with a view to extending the POSIX capabilities of the platform.

Instead I opted for the SourceForge hosted UnxUtils package (http://bit.ly/aCjSI), a sub-1MB zip file containing a little over 30 common GNU utilities (things like sed, grep and tail).

Several of the sub-routines in my UtilityFunctions.bat file make use of these tools or other small, single-task console applications that I've written myself. These are all trivial, but included in the companion files archive at the end.

₁ Local Environment Variables are only valid within a single instance of the command shell. Once the shell is closed, they are disposed of. System Environment Variables are operating system wide, and common to all instances of the command shell.

Companion Files:

Example batch file scripts and additional command line utilities (1.06 MB):

http://www.mediafire.com/?e8a833tahsekhih