Random thoughts on coding & technology

DUnit and TCustomAttributes

noreply@blogger.com (Jordi Corbilla) — Thu, 05 Apr 2012 21:28:00 +0000

In this article I am playing with Unit tests and TCustomAttributes as I am working on different ideas to build up a lightweight testing framework. Basically I liked the idea of TestCase attribute from NUnit and I wanted to do something similar using TCustomAttributes and accessing those attributes using the Delphi RTTI library. To understand better my purposes have a look at the following example extracted from NUnit webpage:

[TestCase(12,3,4)]
[TestCase(12,2,6)]
[TestCase(12,4,3)]
public void DivideTest(int n, int d, int q)
{
  Assert.AreEqual( q, n / d );
}

This simple example will execute three times the method or test using the parameters defined on the test case attributes. What if we could perform something similar? Would not be cool?

Here is what I have done so far:

Creation of the Custom Attributes:

type
  TUserPasswordAttribute = class(TCustomAttribute)
  private
    FPassword: string;
    FUserName: string;
    Fresponse: Boolean;
    procedure SetPassword(const Value: string);
    procedure SetUserName(const Value: string);
    procedure Setresponse(const Value: Boolean);
  public
    constructor Create(aUserName: string; aPassword: string; aResponse : Boolean);
    property UserName: string read FUserName write SetUserName;
    property Password: string read FPassword write SetPassword;
    property response : Boolean read Fresponse write Setresponse;
  end;

  TUserAgeAttribute = class(TCustomAttribute)
  private
    FAge: integer;
    FUserName: String;
    Fresponse: Boolean;
    procedure SetAge(const Value: integer);
    procedure SetUserName(const Value: String);
    procedure Setresponse(const Value: Boolean);
  public
    property UserName : String read FUserName write SetUserName;
    property Age : integer read FAge write SetAge;
    property response : Boolean read Fresponse write Setresponse;
    constructor Create(aUserName : string; aAge : Integer; aResponse : Boolean);
  end;

{ TUserPasswordAttribute }

constructor TUserPasswordAttribute.Create(aUserName, aPassword: string; aResponse : Boolean);
begin
  SetUserName(aUserName);
  SetPassword(aPassword);
  Setresponse(aResponse);
end;

procedure TUserPasswordAttribute.SetPassword(const Value: string);
begin
  FPassword := Value;
end;

procedure TUserPasswordAttribute.Setresponse(const Value: Boolean);
begin
  Fresponse := Value;
end;

procedure TUserPasswordAttribute.SetUserName(const Value: string);
begin
  FUserName := Value;
end;

{ TUserAgeAttribute }

constructor TUserAgeAttribute.Create(aUserName: string; aAge: Integer; aResponse : Boolean);
begin
  SetUserName(aUserName);
  SetAge(aAge);
  Setresponse(aResponse);
end;

procedure TUserAgeAttribute.SetAge(const Value: integer);
begin
  FAge := Value;
end;

procedure TUserAgeAttribute.Setresponse(const Value: Boolean);
begin
  Fresponse := Value;
end;

procedure TUserAgeAttribute.SetUserName(const Value: String);
begin
  FUserName := Value;
end;

Those two custom attributes will serve as an example for what I intend to do. I need to test a login and some data from a current user and those bespoke attributes would be used by the test case.

The Framework:

type
  TAttributeProc = reference to procedure(CustomAttr: TCustomAttribute);

  TFrameworkTestCase = class(TTestCase)
  public
    procedure TestAttributesMethod(CustomProc: TAttributeProc);
  end;

implementation

{ TFrameworkTestCase }

procedure TFrameworkTestCase.TestAttributesMethod(CustomProc: TAttributeProc);
var
  ContextRtti: TRttiContext;
  RttiType: TRttiType;
  RttiMethod: TRttiMethod;
  CustomAttr: TCustomAttribute;
begin
  ContextRtti := TRttiContext.Create;
  try
    RttiType := ContextRtti.GetType(Self.ClassType);
    for RttiMethod in RttiType.GetMethods do
      for CustomAttr in RttiMethod.GetAttributes do
        CustomProc(CustomAttr);
  finally
    ContextRtti.Free;
  end;
end;

As you can see TFrameworkTestCase inherits from TTestCase and it adds the magic of reading the custom attributes and invoking the delegate as many times needed.

Using the small framework:

unit TestUnit1;
{

  Delphi DUnit Test Case
  ----------------------
  This unit contains a skeleton test case class generated by the Test Case Wizard.
  Modify the generated code to correctly setup and call the methods from the unit
  being tested.

}

interface

uses
  TestFramework, Windows, Forms, Dialogs, Controls, Classes, RTTI, SysUtils, Variants,
  Graphics, Messages, Unit1, StdCtrls;

type
  TestTLoginCase = class(TFrameworkTestCase)
  strict private
    FLogin: TLogin;
  public
    procedure SetUp; override;
    procedure TearDown; override;
  published
    [TUserPasswordAttribute('User1', 'Password1', True)]
    [TUserPasswordAttribute('User2', 'Password2', True)]
    [TUserPasswordAttribute('User3', 'Password3', True)]
    [TUserPasswordAttribute('User3', '', False)]
    procedure TestUserLogin;
    [TUserAgeAttribute('User1', 26, True)]
    [TUserAgeAttribute('User2', 27, True)]
    [TUserAgeAttribute('User3', 28, False)]
    procedure TestUserAge;
  end;

implementation

procedure TestTLoginCase.SetUp;
begin
  FLogin := TLogin.Create;
end;

procedure TestTLoginCase.TearDown;
begin
  FLogin.Free;
  FLogin := nil;
end;

procedure TestTLoginCase.TestUserAge;
var
  aAge: integer;
  aUserName: string;
  aResponse : boolean;
begin
  TestAttributesMethod(procedure (CustomAttr : TCustomAttribute)
    begin
        if CustomAttr is TUserAgeAttribute then
        begin
          aUserName := TUserAgeAttribute(CustomAttr).UserName;
          aAge := TUserAgeAttribute(CustomAttr).Age;
          aResponse := TUserAgeAttribute(CustomAttr).Response;
          Assert(aResponse=(FLogin.fetchDatauser(aUserName)=aAge), 'Incorrect value ' + aUserName);
        end;
    end);
end;

procedure TestTLoginCase.TestUserLogin;
var
  aPassword: string;
  aUserName: string;
  aResponse : boolean;
begin
  TestAttributesMethod(procedure (CustomAttr : TCustomAttribute)
    begin
        if CustomAttr is TUserPasswordAttribute then
        begin
          aUserName := TUserPasswordAttribute(CustomAttr).UserName;
          aPassword := TUserPasswordAttribute(CustomAttr).Password;
          aResponse := TUserPasswordAttribute(CustomAttr).Response;
          Assert(FLogin.UserLogin(aUserName, aPassword)=aResponse, 'Incorrect user ' + aUserName);
        end;
    end);
end;

initialization
RegisterTest(TestTLoginCase.Suite);

end.

Notice that every test case contains a set of Custom attributes and they will be executed by using a delegate. I have included the Result parameter in the attribute so the test can know the result straight away and inform about it.

Related links:

Fluent Interfaces example using Delphi part II

noreply@blogger.com (Jordi Corbilla) — Sun, 25 Mar 2012 21:20:00 +0000

Here is the second part of this interesting topic. As I'm still trying to redeem myself from my non popular first example, I'm sure this one will reach the expectations. For this example I'm trying to mimic the way LINQ works, using generics and delegates and I have adapted my solution using fluent Interfaces as well. This solution presents a IQueryList which contains a TList<T> which can be queried like we were using SQL. So, we can select certain values and apply a where clause to filter the final list. This example will give you a hint on how to correctly implement fluent interfaces and how to extend this functionality for your applications.

Delegates:

uses
  Generics.Collections, Generics.Defaults;

type
  TProc<T> = procedure (n : T) of Object;
  TProcList<T> = procedure (n : TList<T>) of Object;
  TFunc<T> = reference to function() : T;
  TFuncParam<T, TResult> = reference to function(param : T) : TResult;
  TFuncList<T, TResult> = reference to function(n : TList<T>) : TResult;
  TFuncListSelect<T, TResult> = reference to function(n : TList<T>) : TList<T>;

IQueryList:

IQueryList<T, TResult> = interface
    function Where(const param : TFuncParam<T, TResult>) : IQueryList<T, TResult>;
    function OrderBy(const AComparer: IComparer<T>) : IQueryList<T, TResult>;
    function Select(const param : TFuncListSelect<T, TResult>) : IQueryList<T, TResult>;
    function FillList(const param : TFuncList<T, TResult>) : IQueryList<T, TResult>;
    function Distinct() : IQueryList<T, TResult>;
    function List() : TList<T>;
  end;

  TQueryList<T, TResult> = class(TInterfacedObject, IQueryList<T, TResult>)
  private
    FList : TList<T>;
  protected
    function Where(const param : TFuncParam<T, TResult>) : IQueryList<T, TResult>;
    function FillList(const param : TFuncList<T, TResult>) : IQueryList<T, TResult>;
    function Select(const param : TFuncListSelect<T, TResult>) : IQueryList<T, TResult>;
    function Distinct() : IQueryList<T, TResult>;
    function List() : TList<T>;
    function OrderBy(const AComparer: IComparer<T>) : IQueryList<T, TResult>;
  public
    constructor Create();
    destructor Destroy(); override;
    class function New: IQueryList<T, TResult>;
  end;

constructor TQueryList<T, TResult>.Create();
begin
  FList := TList<T>.Create;
end;

destructor TQueryList<T, TResult>.Destroy;
begin
  if Assigned(FList) then
    FList.Free;
  inherited;
end;

function TQueryList<T, TResult>.Distinct: IQueryList<T, TResult>;
var
  list : TList<T>;
  i : integer;
begin
  list := TList<T>.Create();
  for i := 0 to FList.Count-1 do
  begin
    if not list.Contains(FList[i]) then
      list.Add(FList[i]);
  end;
  FList.Free;
  FList := list;
  result := Self;
end;

function TQueryList<T, TResult>.FillList(const param : TFuncList<T, TResult>): IQueryList<T, TResult>;
begin
  param(FList);
  Result := Self;
end;

function TQueryList<T, TResult>.List: TList<T>;
begin
  result := FList;
end;

class function TQueryList<T, TResult>.New: IQueryList<T, TResult>;
begin
  result := Create;
end;

function TQueryList<T, TResult>.OrderBy(const AComparer: IComparer<T>): IQueryList<T, TResult>;
begin
  FList.Sort(AComparer);
  result := Self;
end;

function TQueryList<T, TResult>.Select(const param: TFuncListSelect<T, TResult>): IQueryList<T, TResult>;
begin
  FList := param(FList);
  result := Self;
end;

function TQueryList<T, TResult>.Where(const param: TFuncParam<T, TResult>): IQueryList<T, TResult>;
var
  list : TList<T>;
  i: Integer;
  Comparer: IEqualityComparer<TResult>;
begin
  list := TList<T>.Create();
  for i := 0 to FList.Count-1 do
  begin
    Comparer := TEqualityComparer<TResult>.Default;
    if not Comparer.Equals(Default(TResult), param(FList[i])) then
      list.Add(FList[i]);
  end;
  FList.Free;
  FList := list;
  result := Self;
end;

Example Implementation <Integer, Boolean>:

procedure DisplayList();
var
  IqueryList : IQueryList<Integer, Boolean>;
  item : integer;
begin
  //Create the list and fill it up with random values
  IqueryList := TQueryList<Integer, Boolean>
    .New()
    .FillList(function ( list : TList<Integer> ) : Boolean
              var k : integer;
              begin
                for k := 0 to 100 do
                  list.Add(Random(100));
                result := true;
              end);

  //Display filtered values
  for item in IqueryList
    .Select(function ( list : TList<Integer> ) : TList<Integer>
              var
                k : integer;
                selectList : TList<Integer>;
              begin
                selectList := TList<Integer>.Create;
                for k := 0 to list.Count-1 do
                begin
                  if Abs(list.items[k]) > 0 then
                    selectList.Add(list.items[k]);
                end;
                list.Free;
                result := selectList;
              end)
    .Where(function ( i : integer) : Boolean
          begin
            result := (i > 50);
          end)
    .Where(function ( i : integer) : Boolean
          begin
            result := (i < 75);
          end)
    .OrderBy(TComparer<integer>.Construct(
         function (const L, R: integer): integer
         begin
           result := L - R; //Ascending
         end
     )).Distinct.List do
          WriteLn(IntToStr(item));
end;

This example fills up an initial list with 100 random numbers and then I query the list to give me all the values from the list which absolute value is greater than 0 and the values are between 50 and 75. From this list I want all the values ordered by value and I do not want repeated numbers (distinct method).

Example Implementation <String, String>:

procedure DisplayStrings();
const
   Chars = '1234567890ABCDEFGHJKLMNPQRSTUVWXYZ!';
var
  S: string;
  IqueryList : IQueryList<String, String>;
  item : String;
begin
  //Fill up the list with random strings
  IqueryList := TQueryList<String, String>
    .New
    .FillList(function ( list : TList<String> ) : String
              var
                k : integer;
                l : Integer;
              begin
                Randomize;
                for k := 0 to 100 do
                begin
                  S := '';
                  for l := 1 to 8 do
                    S := S + Chars[(Random(Length(Chars)) + 1)];
                  list.Add(S);
                end;
                result := '';
              end);
  //Query the list and retrieve all items which contains 'A'
  for item in IqueryList.Where(function ( i : string) : string
          begin
            if AnsiPos('A', i) > 0 then
              result := i;
          end).List do
    WriteLn(item);
end;

The string example is quite similar, it fills up a list with 100 random string values and then it filters the list displaying only items which contains the character "A".

Everything is based on interfaces so the garbage collector can step in and avoid memory leaks and you can find a sound widespread of generics, delegates and chaining methods all in once. This solution gives you control on the way data is treated and it can work with any type as it is using generics.

Enjoy it!.

I look forward to your comments.

Jordi

Related Links:

Monitoring Global Atom Table part III

noreply@blogger.com (Jordi Corbilla) — Sat, 25 Feb 2012 22:37:00 +0000

New version v1.4 has been released as there were few bugs detected. This version also includes a new and very interesting feature, inspecting atoms from windows services. "A Windows Service applications run in a different window station than the interactive station of the logged-on user. A window station is a secure object that contains a Clipboard, a set of global atoms, and a group of desktop objects. Because the station of the Windows service is not an interactive station, dialog boxes raised from within a Windows service application will not be seen and may cause your program to stop responding. Similarly, error messages should be logged in the Windows event log rather than raised in the user interface".

Source : Microsoft.

This actually means that a running service is using a different set of global atoms than the current user. To display those atoms, atom table monitor v1.4 includes an Atom scanner service which uses the same core engine than Atom monitor and retrieves the list of Global atoms and RWM atoms from the system under the window station.

Current version contains: Atom Table monitor v1.4.

- Atom monitor win32 stand-alone tool.

- List of common patterns.

- Atom scanner win32 service.

- Install / Unninstall service batch files.

Session selection screen:

If the service is up and running, we can select the option to display the atoms from the service session. If the service is not detected the monitor will stop itself.

Service session monitoring RWM atoms:

This screen is displaying the amount of atoms which are being monitored by the service session. You can play with that by creating a small tool to leak atoms and use different configurations from the service. Have a look at my previous post How to run an application under active session account from a windows service.

User session monitoring RWM atoms:

Check out the amount of patterns which match an specific subset of atom strings. This will help you to rapidly identify which atoms are being created and which is the source.

Installing the service:

Use the batch files to install / uninstall ATOMScannerService.exe. Once installed, run it under local account.

Once up and running, select "Monitor Atoms from service session" on Option's tab and press scan atom table.

Related links:

How to run an application under active session account from a Windows service using Delphi

noreply@blogger.com (Jordi Corbilla) — Sat, 25 Feb 2012 17:22:00 +0000

To run an application from a service impersonating an user account, first we need to install "JEDI API Library & Security Code Library" as it contains different interesting OS callings which are really useful in order to achieve our purposes. Once the library has been unzipped, create your own service where the library is located and add the following paths to your search path in your project options:

.\JEDI API 2.3 and JEDI WSCL 0.9.3\jwa\branches\2.3\Common
.\JEDI API 2.3 and JEDI WSCL 0.9.3\jwa\branches\2.3\Win32API

Then instead of using CreateProcess function to execute an application, we need to use CreateProcessAsUser function. The new process runs in the security context of the user represented by the specified token. The service must be run by the LocalSystem account which is a predefined local account used by the service control manager. To be able to use the function from jedi-apilib which retrieves the token from the current user we need to use WTSQueryUserToken ( WtsGetActiveConsoleSessionID, hToken ) function. This function will only work under LocalSystem account which has SE_TCB_NAME property enabled, otherwise the query will be false.

Use the following example within your service:

uses
  ComObj, ActiveX, JwaWinbase, JwaWtsApi32;

{$R *.DFM}

procedure ServiceController(CtrlCode: DWORD); stdcall;
begin
  ServiceExample.Controller(CtrlCode);
end;

function TServiceExample.GetServiceController: TServiceController;
begin
  Result := ServiceController;
end;

procedure TServiceExample.ServiceExecute(Sender: TService);
begin
  Timer1.Enabled := True;
  while not Terminated do
    ServiceThread.ProcessRequests(True); // wait for termination
  Timer1.Enabled := False;
end;

procedure TServiceExample.Timer1Timer(Sender: TObject);
const
  ProgramName = 'C:\myTestproject1.exe';
var
  hToken: THandle;
  StartupInfo: Windows.TStartupInfo;
  ProcessInfo: Windows.TProcessInformation;
  res: boolean;
begin
  Windows.GetStartupInfo(StartupInfo);
  if WTSQueryUserToken(WtsGetActiveConsoleSessionID, hToken) then
  begin
    res := Windows.CreateProcessAsUser(hToken, ProgramName, nil, nil, nil, False, CREATE_NEW_CONSOLE, nil, nil, StartupInfo, ProcessInfo);
    if res then
      WaitForSingleObject(ProcessInfo.hProcess,INFINITE);
  end;
end;

Related links:

Monitoring Global Atom Table part II

noreply@blogger.com (Jordi Corbilla) — Mon, 20 Feb 2012 22:25:00 +0000

Atom table monitor is now available on google code. Atom table is an important resource to take into account when developing win32/win64 applications as if the table gets depleted a "System Error. Code: 8. Not enough storage is available to process this command" would be returned leaving the system in an unresponsive / unstable state. This issue only happens under Windows Vista / 7 / Server 2008, so it is crucial to avoid leaking atoms. There is a very interesting article (Identifying atom leaks) from Microsoft debug team which actually shows the way to monitor the atom entries by debugging the kernel. In the first part of this article (Monitoring Global Atom Table part I) I had delved into detail by debugging the kernel and explaining how to display different atom entries. In that post I released alpha version of "Atom table monitor" which I have seen it is wrong as it shares the same memory area for global atom table and registerwindowmessage table. In this new version (v1.2) both tables are displayed into separate memory grids:

Global atom table:

RegisterWindowMessage table:

Display list of entries:

Matching string patterns:

Counters:

Testing screen:

Enjoy the tool!.

Related articles:

Use ADPlus to troubleshoot hangs and crashes in our Delphi applications

noreply@blogger.com (Jordi Corbilla) — Sat, 18 Feb 2012 00:53:00 +0000

It's again debugging time and in this post I'm going to put forward how to use ADPlus to troubleshoot hangs or crashes of our Delphi applications. As I'm sure you know, applications are getting more and more complicated using different libraries and third party tools and sometimes it is quite difficult to find out where the current problem is located. It has happened to me quite often that a tool or app blows app without reason, without a proper exception (even though all exception handling mechanisms are there) so the final user never gets any indication about what's wrong. With few simple steps from this article you will be able to set up ADPlus correctly, attach it to your running process and then create the crash dump for its further analysis. Let's start with debugging time!.

Installing Debugging tools for windows:

This is the Swiss knife for any good developer. Bring it always with you as it is really helpful. You can download the library from here. Just install it and save it to and USB to be portable. Then spend time using it, not only using ADPlus but also with Windbg, as it is crucial that you know about how to analyse the crash dump.

Setting up our Delphi Applications:

To be sure our Delphi app is correctly identified, we need to generate the map file and then use an external tool to convert all that information to symbols.

Edit in the project options to generate debug information and a detailed map file. Then using map2dbg we will transform the map file into debug symbols (.dbg files).

Once your project is correctly set up, build it and you will get the map file. Download the latest version of map2dbg v1.3 and copy map2dbg.exe where your project is located and run the following command line:

C:\testAdPlus>map2dbg.exe Project1.exe
Converted 6882 symbols.

You will now see a .dbg file with all necessary information for ADPlus.

Setting up the Symbols:

Once again, if we run ADPlus without setting up the symbols, we will only see address of memory without descriptions:

Call stack below ---
*** ERROR: Module load completed but symbols could not be loaded for C:\testAdPlus\Project1.exe
 # ChildEBP RetAddr  Args to Child              
WARNING: Stack unwind information not available. Following frames may be wrong.
00 0012f550 004832bd 01aba8c0 0045f20b 0012f700 Project1+0xb3295
01 0012f698 00487615 0012f91c 01aba8c0 0012f9a8 Project1+0x832bd
02 0012f6e4 0045eed1 01aba8c0 0012f91c 01aba8c0 Project1+0x87615
03 0012f710 00487768 00050980 0012f9a8 01acb650 Project1+0x5eed1
04 0012f86c 00487615 00000111 0012f91c 09010401 Project1+0x87768
05 0012f8b8 004a7c25 0012f9a8 00000111 01acb650 Project1+0x87615
06 0012f8e4 00486cb3 0012f8f8 00486ccb 0012f914 Project1+0xa7c25
07 0012f914 0043fd4a 00000111 00000980 00050980 Project1+0x86cb3
08 0012f92c 762cfd72 000209ae 00000111 00000980 Project1+0x3fd4a

Setting up _NT_SYMBOL_PATH environment variable:
Create a new general environment variable with the following name: _NT_SYMBOL_PATH and the following value:
symsrv*symsrv.dll*C:\Symbols*http://msdl.microsoft.com/download/symbols.
Where C:\Symbols is the path to your symbols. This is the same parameter set in WinDbg which I explained in my previous post (monitoring atom table part I).

Setting up ADPlus:

We can either run ADPlus using a simple configuration or use the configuration file which is much more complete. In any case, the fastest way is using the simple configuration which will help us to get the expected outcome.

C:\Program Files\Debugging Tools for Windows (x86)>adplus -crash -pmn Project1.exe -o C:\Adplus -mss c:\symbols

This configuration will run ADPlus looking for crashes, waiting for Project1.exe and it will output the results in C:\ADPlus and it will use the symbols from C:\Symbols.

Sample test:
This small piece of code will help me to generate a small access violation and then ADPlus will catch the crashing and it will generate the crash dump.

procedure TForm1.Button1Click(Sender: TObject);
begin
  try
    SimulateSystemException;
  finally
    Close;
  end;
end;

procedure TForm1.SimulateSystemException;
var
  p: PChar;
begin
  p := PChar(5);
  p^ := #9; //Access Violation here
end;

Once ADPlus is running, run your project, in my case Project1.exe and wait until it crashes (in my case it is just simulated so, the crash dump is instantly generated).

You will get a new line in the cmd window telling you:
Attaching to 4812 - Project1 in crash mode 02/18/2012 00:16:00

Once it crashes, go to C:\ADPlus folder and you will get a new folder with current date and the crash dump in it.

Analysing crash dump with Windbg:
Just open Windbg, load the first crash dump (FirstChance_Process_Shut_Down) and use the following commands:

0:000> !sym noisy
noisy mode - symbol prompts on
0:000> .reload Project1.exe
0:000> !analyze -v
ERROR: FindPlugIns 8007007b
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************

DBGHELP: kernel32 - public symbols  c:\symbols\kernel32.pdb\FCCF6FAC09804D49A4BB256A77F519572\kernel32.pdb
DBGHELP: Project1.exe is stripped.  Searching for dbg file
SYMSRV:  c:\symbols\Project1.dbg\4F3D9543457000\Project1.dbg not found
SYMSRV:  http://msdl.microsoft.com/download/symbols/Project1.dbg/4F3D9543457000/Project1.dbg not found
DBGHELP: .\Project1.dbg - file not found
DBGHELP: .\exe\Project1.dbg - path not found
DBGHELP: .\symbols\exe\Project1.dbg - path not found
DBGHELP: Project1.exe missing debug info.  Searching for pdb anyway
DBGHELP: Can't use symbol server for Project1.pdb - no header information available
DBGHELP: Project1.pdb - file not found
*** WARNING: Unable to verify checksum for Project1.exe
*** ERROR: Module load completed but symbols could not be loaded for Project1.exe
DBGHELP: Project1 - no symbols loaded
DBGHELP: user32 - public symbols c:\symbols\user32.pdb\CFD2C4C8EB9C406D8B6DC29512EB176A2\user32.pdb
DBGHELP: ole32 - public symbols c:\symbols\ole32.pdb\EDE30219D57144FAAC83675A6573D1982\ole32.pdb

Once processed, we can actually spot that the symbols are missing. This is because we need to place the symbols at the location defined by Windbg -> c:\symbols\Project1.dbg\4F3D9543457000\Project1.dbg. Just copy your dbg file into defined location and try again reloading Project1.exe from Windbg command line.

Now we can analyse again the crash dump and get all the information needed about the crash. This time with the correct symbols:

Call stack below ---
*** WARNING: Unable to verify checksum for C:\Users\jordi coll\Desktop\testAdPlus\Project1.exe
 # ChildEBP RetAddr  Args to Child              
00 0012f550 00473c9d 008fa8c0 00455747 0012f700 Project1!Unit1.TForm1.SimulateSystemException+0x5
01 0012f698 00477ff5 0012f91c 008fa8c0 0012f9a8 Project1!Controls.TControl.Click+0x75
02 0012f6e4 0045540d 008fa8c0 0012f91c 008fa8c0 Project1!Controls.TWinControl.WndProc+0x56d
03 0012f710 00478148 00010d2e 0012f9a8 0090b650 Project1!StdCtrls.TButtonControl.WndProc+0x71
04 0012f86c 00477ff5 00000111 0012f91c 09010401 Project1!Controls.DoControlMsg+0x28
05 0012f8b8 004984b1 0012f9a8 00000111 0090b650 Project1!Controls.TWinControl.WndProc+0x56d
06 0012f8e4 00477693 0012f8f8 004776ab 0012f914 Project1!Forms.TCustomForm.WndProc+0x599
07 0012f914 0043c146 00000111 00000d2e 00010d2e Project1!Controls.TWinControl.MainWndProc+0x2f
08 0012f92c 762cfd72 00010d26 00000111 00000d2e Project1!Classes.StdWndProc+0x16
09 0012f958 762cfe4a 00380fc8 00010d26 00000111 USER32!InternalCallWinProc+0x23
0a 0012f9d0 762d0943 00000000 00380fc8 00010d26 USER32!UserCallWinProcCheckWow+0x14b (FPO: [Non-Fpo])
0b 0012fa10 762d0b36 00fd2450 00fd23e8 00000d2e USER32!SendMessageWorker+0x4b7 (FPO: [Non-Fpo])
0c 0012fa30 74d1b4ba 00010d26 00000111 00000d2e USER32!SendMessageW+0x7c (FPO: [Non-Fpo])
0d 0012fa50 74d1b51c 00a54518 00000000 00080032 comctl32!Button_NotifyParent+0x3d (FPO: [Non-Fpo])
0e 0012fa6c 74d1b627 54010001 00000001 0012fb48 comctl32!Button_ReleaseCapture+0x112 (FPO: [Non-Fpo])
0f 0012facc 762cfd72 00010d2e 00000202 00000000 comctl32!Button_WndProc+0xa98 (FPO: [Non-Fpo])
10 0012faf8 762cfe4a 74cb70f8 00010d2e 00000202 USER32!InternalCallWinProc+0x23
11 0012fb70 762d09d3 00000000 74cb70f8 00010d2e USER32!UserCallWinProcCheckWow+0x14b (FPO: [Non-Fpo])
12 0012fba0 762d0979 74cb70f8 00010d2e 00000202 USER32!CallWindowProcAorW+0x97 (FPO: [Non-Fpo])
13 0012fbc0 004780f5 74cb70f8 00010d2e 00000202 USER32!CallWindowProcW+0x1b (FPO: [Non-Fpo])
14 0012fd44 00477ff5 00000202 008fa8c0 0012fd80 Project1!Controls.TWinControl.DefaultHandler+0xdd
15 0012fd90 0045540d 0012fe64 00000202 008fa8c0 Project1!Controls.TWinControl.WndProc+0x56d
16 0012fdd0 0043c146 00000202 00000000 00080032 Project1!StdCtrls.TButtonControl.WndProc+0x71
17 0012fde8 762cfd72 00010d2e 00000202 00000000 Project1!Classes.StdWndProc+0x16
18 0012fe14 762cfe4a 00380fbb 00010d2e 00000202 USER32!InternalCallWinProc+0x23
19 0012fe8c 762d018d 00000000 00380fbb 00010d2e USER32!UserCallWinProcCheckWow+0x14b (FPO: [Non-Fpo])
1a 0012fef0 762d022b 00380fbb 00000000 00010d2e USER32!DispatchMessageWorker+0x322 (FPO: [Non-Fpo])
1b 0012ff00 004a15d2 0012ff24 00010001 0012ff70 USER32!DispatchMessageW+0xf (FPO: [Non-Fpo])
1c 0012ff1c 004a1617 00010d2e 00000202 00000000 Project1!Forms.TApplication.ProcessMessage+0x122
1d 0012ff40 004a1942 0012ff54 004a194c 0012ff70 Project1!Forms.TApplication.HandleMessage+0xf
1e 0012ff70 004a9c96 0012ffc4 00405a14 0012ff88 Project1!Forms.TApplication.Run+0xce
1f 0012ff88 7610d0e9 7ffdd000 0012ffd4 77711603 Project1!Project1.Project1+0x4e
20 0012ff94 77711603 7ffdd000 7736d39e 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])
21 0012ffd4 777115d6 004a9c48 7ffdd000 00000000 ntdll!__RtlUserThreadStart+0x23 (FPO: [Non-Fpo])
22 0012ffec 00000000 004a9c48 7ffdd000 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

0:000> kvn
 # ChildEBP RetAddr  Args to Child              
00 0012ff30 774a5370 7747b148 ffffffff 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
01 0012ff34 7747b148 ffffffff 00000000 0012ff58 ntdll!ZwTerminateProcess+0xc (FPO: [2,0,0])
02 0012ff44 76f241ec 00000000 77e8f3b0 ffffffff ntdll!RtlExitUserProcess+0x7a (FPO: [Non-Fpo])
03 0012ff58 00405f73 00000000 0012ff88 00000000 kernel32!ExitProcess+0x12 (FPO: [1,0,0])
04 0012ff70 004a9c9b 0012ffc4 00405a14 0012ff88 Project1!System.Halt0+0xf3
05 0012ff88 76f2d0e9 7ffd3000 0012ffd4 77481603 Project1!Project1.Project1+0x53
06 0012ff94 77481603 7ffd3000 77aa9471 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])
07 0012ffd4 774815d6 004a9c48 7ffd3000 00000000 ntdll!__RtlUserThreadStart+0x23 (FPO: [Non-Fpo])
08 0012ffec 00000000 004a9c48 7ffd3000 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

The only remaining thing is to analyse the crash dump trying to identify which are the regions involved in the crash and tackle them!.

Happy ninja debugging!.

Related links:

Monitoring Global Atom Table part I

noreply@blogger.com (Jordi Corbilla) — Wed, 08 Feb 2012 23:34:00 +0000

The aim of this article is to give a sound understanding about Atoms, how to monitor them and check whether we have or have not any process that is potentially leaking atoms. As Microsoft very well defines:
"An atom table is a system-defined table that stores strings and corresponding identifiers. An application places a string in an atom table and receives a 16-bit integer, called an atom, that can be used to access the string. A string that has been placed in an atom table is called an atom name.
The system provides a number of atom tables. Each atom table serves a different purpose. Applications can use local atom tables to store their own item-name associations.
The system uses atom tables that are not directly accessible to applications. However, the application uses these atoms when calling a variety of functions. For example, registered clipboard formats are stored in an internal atom table used by the system. An application adds atoms to this atom table using the RegisterClipboardFormat function. Also, registered classes are stored in an internal atom table used by the system. An application adds atoms to this atom table using the RegisterClass or RegisterClassEx function."

source: Microsoft.

Atoms are stored as two-byte integers (uint16) and there can be 0xFFFF-0xC000=0x4000 (16384) entries maximum. If 0xFFFF is reached ERROR 8 is returned ("System Error. Code: 8. Not enough storage is available to process this command")

Discovering Atom table:
To display atom table we need to use Debugging tools for Windows: Windbg and debug the kernel to dump al registers.

First of all:

Install all necessary tools and then run Windbg and configure all symbol parameters to correctly use kernel debug mode(for windows 7/Server 2008 debug mode needs to be enforced using bcdedit /debug on ). The most important module here is win32k.sys.

Set Symbol file path (Menu File -> Symbol file path):

"symsrv*symsrv.dll*C:\Symbols*http://msdl.microsoft.com/download/symbols"

As we need win32k.sys, we need to generate the pdb files with symbols information using symchk tool from debugging directory:

C:\Program Files\Debugging Tools for Windows (x86)>symchk C:\temp\bb\win32k.sys /v
[SYMCHK] Searching for symbols to C:\temp\bb\win32k.sys in path symsrv*symsrv.dll*C:\Symbols*http://msdl.microsoft.com/download/symbols
DBGHELP: Symbol Search Path: symsrv*symsrv.dll*C:\Symbols*http://msdl.microsoft.com/download/symbols
[SYMCHK] Using search path "symsrv*symsrv.dll*C:\Symbols*http://msdl.microsoft.com/download/symbols"
DBGHELP: No header for C:\temp\bb\win32k.sys.  Searching for image on disk
DBGHELP: C:\temp\bb\win32k.sys - OK
SYMSRV:  win32k.pdb from http://msdl.microsoft.com/download/symbols: 1170455 bytes - copied
DBGHELP: win32k - public symbols
         C:\Symbols\win32k.pdb\D8788E4736B34ED5B8516DBB9D45E9942\win32k.pdb
[SYMCHK] MODULE64 Info ----------------------
[SYMCHK] Struct size: 1680 bytes
[SYMCHK] Base: 0xBF800000
[SYMCHK] Image size: 1839616 bytes
[SYMCHK] Date: 0x43446a58
[SYMCHK] Checksum: 0x001ca511
[SYMCHK] NumSyms: 0
[SYMCHK] SymType: SymPDB
[SYMCHK] ModName: win32k
[SYMCHK] ImageName: C:\temp\bb\win32k.sys
[SYMCHK] LoadedImage: C:\temp\bb\win32k.sys
[SYMCHK] PDB: "C:\Symbols\win32k.pdb\D8788E4736B34ED5B8516DBB9D45E9942\win32k.pdb"
[SYMCHK] CV: RSDS
[SYMCHK] CV DWORD: 0x53445352
[SYMCHK] CV Data:  win32k.pdb
[SYMCHK] PDB Sig:  0
[SYMCHK] PDB7 Sig: {D8788E47-36B3-4ED5-B851-6DBB9D45E994}
[SYMCHK] Age: 2
[SYMCHK] PDB Matched:  TRUE
[SYMCHK] DBG Matched:  TRUE
[SYMCHK] Line nubmers: FALSE
[SYMCHK] Global syms:  FALSE
[SYMCHK] Type Info:    TRUE
[SYMCHK] ------------------------------------
SymbolCheckVersion  0x00000002
Result              0x00130001
DbgFilename
DbgTimeDateStamp    0x43446a58
DbgSizeOfImage      0x001c1200
DbgChecksum         0x001ca511
PdbFilename         C:\Symbols\win32k.pdb\D8788E4736B34ED5B8516DBB9D45E9942\win32k.pdb
PdbSignature        {D8788E47-36B3-4ED5-B851-6DBB9D45E994}
PdbDbiAge           0x00000002
[SYMCHK] [ 0x00000000 - 0x00130001 ] Checked "C:\temp\bb\win32k.sys"

SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 1

Once done, you will see the pdb file generated in C:\Symbols folder and you would be able to load it from Windbg command line using the WinDbg commands http://windbg.info/doc/1-common-cmds.html.

kd> !sym noisy
noisy mode - symbol prompts on
kd> .reload win32k.sys
kd> lm
start    end        module name
804d7000 806cdc00   nt         (export symbols)       ntkrnlpa.exe
bf800000 bf9c1200   win32k     (deferred)

Now we know that win32k.sys it's loaded in address bf800000 and the referenced memory can be displayed by using dq command (using a 64-bit pointer) and only the first dword (L1):

kd> dq win32k!UserAtomTableHandle L1
DBGHELP: win32k - public symbols  
         c:\symbols\win32k.pdb\D8788E4736B34ED5B8516DBB9D45E9942\win32k.pdb
bf9a7c18  81b4a270`e1c47230

kd> dq 81b4a270`e1c47230+10
e1c47240  e1a841c0`e1ba3008 e15fb8f8`e19fae00
e1c47250  e1ba3080`e1700738 e1546428`e1708f88
e1c47260  e15f4e28`e15f4c78 e19e9058`e1a04278
e1c47270  e15f9d20`e15fb878 e170eda8`e1704d90
e1c47280  e1a0bea8`e1f71658 e1a841f0`e1a61550
e1c47290  e15464a0`e1091388 e1b16bc8`e1542440
e1c472a0  e1001d00`e1a014c0 e181efe0`e1c42118
e1c472b0  e1bb9178`e16f33e0 e15fa858`e19e9078
kd> dq
e170076a  04080000`000081cc 04016d53`6d4d0001
e170077a  00006156`4d430c07 00260012`6b76002c
e170078a  0001007c`fec80000 7250ffff`00010000
e170079a  6f724773`7365636f 7963696c`6f507075
e17007aa  0407ffff`ffffffff d4987346`744e0c05
e17007ba  00000000`0000e16e 4fcf0000`00000000
e17007ca  29880003`00000000 04050000`0000e170
e17007da  0401e24e`4d430001 f6886944`624f0c02

Showing the content of the memory:

Now that we now where Global Atom table is allocated, we can start inspecting its elements by displaying Unicode characters from memory by using du command:

kd> du e1ba3080`e1700738+C
e1700744  "DDEMLUnicodeClient"
kd> du e1546428`e1708f88+C
e1708f94  "ACTIVATESHELLWINDOW"

We can dump the memory using 32-bit pointers (dd command) and get the same results:

kd> dd e1c47240
e1c47240  e1ba3008 e1a841c0 e19fae00 e15fb8f8
e1c47250  e1700738 e1ba3080 e1708f88 e1546428
e1c47260  e15f4c78 e15f4e28 e1a04278 e19e9058
e1c47270  e15fb878 e15f9d20 e1704d90 e170eda8
e1c47280  e1f71658 e1a0bea8 e1a61550 e1a841f0
e1c47290  e1091388 e15464a0 e1542440 e1b16bc8
e1c472a0  e1a014c0 e1001d00 e1c42118 e181efe0
e1c472b0  e16f33e0 e1bb9178 e19e9078 e15fa858
kd> dd e1c472b0
e1c472b0  e16f33e0 e1bb9178 e19e9078 e15fa858
e1c472c0  e1a84198 e1a61500 e195d558 e15ff980
e1c472d0  e15424a0 00000000 00000000 00000000
e1c472e0  00000000 00000000 00000000 00000000
e1c472f0  00000000 00000000 00000000 00000000
e1c47300  00000000 00000000 00000000 00000000
e1c47310  00000000 00000000 00000000 00000000
e1c47320  00000000 00000000 00000000 00000000
kd> du e1ba3008+c
e1ba3014  "Native"
kd> du e15fa858+c
e15fa864  "OleClipboardPersistOnFlush"

To display the whole content we can use the following function to loop through all buckets, listing all atoms created:

kd> r $t0=poi(poi(win32k!UserAtomTableHandle)+C)
kd> .for(r $t1=0; @$t1<@$t0; r $t1=@$t1+1) {du poi(poi(win32k!UserAtomTableHandle)+10+(@$t1*4))+C}
e1ba3014  "Native"
e1a841cc  "ObjectLink"
e19fae0c  "6.0.2600.2180!tooltips_class32"
e15fb904  "Static"
e1700744  "DDEMLUnicodeClient"
e1ba308c  "DataObject"
e1708f94  "ACTIVATESHELLWINDOW"
e1546434  "FlashWState"
e15f4c84  "SysCH"
e15f4e34  "PBrush"
e1a04284  "6.0.2600.2180!msctls_progress32"
e19e9064  "SysIC"
e15fb884  "DDEMLEvent"
e15f9d2c  "SHELLHOOK"
e1704d9c  "Custom Link Source"
e170edb4  "ControlOfsM*EVJK"
e1f71664  "DesktopSFTBarHost"
e1a0beb4  "SysDT"
e1a6155c  "Link Source"
e1a841fc  "FileName"
e1091394  "ReBarWindow32"
e15464ac  "SysWNDO"
e154244c  "DDEMLAnsiServer"
e1b16bd4  "SysLink"
e1a014cc  "NetworkName"
e1001d0c  "USER32"
e1c42124  "OleDraw"
e181efec  "FileNameW"
e16f33ec  "MoreOlePrivateData"
e1bb9184  "Edit"
e19e9084  "Binary"
e15fa864  "OleClipboardPersistOnFlush"
e1a841a4  "OwnerLink"
e1a6150c  "ListBox"
e195d564  "Embed Source"
e15ff98c  "SysIMEL"
e15424ac  "ComboLBox"

(Source code from http://blog.lordjeb.com/tag/debugging/)
Another way of displaying the content of Global Atom Table is by using !gatom command from user session on Windbg. (http://msdn.microsoft.com/en-us/library/ff563166(v=vs.85).aspx)

0:001> !gatom
Global atom table c00c( 1) = Protocols (18) pinned
c00d( 1) = Topics (12) pinned
c00e( 1) = Formats (14) pinned
c01f(36) = OleEndPointID (26)
c026( 1) = AppProperties (26)
c024( 1) = Delphi00000DB8 (28)
c01e(94) = UxSubclassInfo (28)
c010( 1) = EditEnvItems (24) pinned
c022( 3) = CAddressComboEx_This (40)
c01d( 6) = OleDropTargetMarshalHwnd (48)
c028( 1) = Delphi000009C0 (28)
c008( 1) = StdDoVerbItem (26) pinned
c01c( 6) = OleDropTargetInterface (44)
c02c( 1) = WndProcPtr0040000000000DC4 (52)
c02b( 1) = ControlOfs0040000000000DC4 (52)
c014( 1) = Save (8) pinned
c016( 1) = MSDraw (12) pinned
c02a( 1) = WndProcPtr0040000000000940 (52)
c024( 1) = Delphi00000DB8 (28)
c01e(94) = UxSubclassInfo (28)
c023( 3) = CAutoComplete_This (36)
c007( 1) = StdShowItem (22) pinned
c011( 1) = True (8) pinned
c010( 1) = EditEnvItems (24) pinned
c019( 1) = PROGMAN (14)
c012( 1) = False (10) pinned
c015( 1) = Close (10) pinned
c022( 3) = CAddressComboEx_This (40)
c004( 1) = StdEditDocument (30) pinned
c01d( 6) = OleDropTargetMarshalHwnd (48)
c028( 1) = Delphi000009C0 (28)
c008( 1) = StdDoVerbItem (26) pinned
c01c( 6) = OleDropTargetInterface (44)
c02c( 1) = WndProcPtr0040000000000DC4 (52)
c005( 1) = StdNewfromTemplate (36) pinned
c02b( 1) = ControlOfs0040000000000DC4 (52)
c014( 1) = Save (8) pinned
c016( 1) = MSDraw (12) pinned
c02a( 1) = WndProcPtr0040000000000940 (52)
c002( 1) = StdNewDocument (28) pinned
c027( 1) = ddeconv (14)
c029( 1) = ControlOfs0040000000000940 (52)
c00f( 1) = Status (12) pinned
c017(27) = ThemePropScrollBarCtl (42)
c009( 1) = System (12) pinned
c01b( 3) = AnimationID (22)
c025( 1) = Folders (14)
c00b( 1) = StdDocumentName (30) pinned
c013( 1) = Change (12) pinned
c001( 1) = StdExit (14) pinned
c006( 1) = StdCloseDocument (32) pinned
c01a( 6) = CAddressBand_This (34)
c00a( 1) = OLEsystem (18) pinned
c018( 3) = CC32SubclassInfo (32)

But As we can see, we are not displaying all table completely as it is quite difficult to find the correct entry as the table contains atoms, registered clipboard formats, classes, etc. If we follow the steps from Microsoft debug team, the job is far way easy:

kd> dq win32k!UserAtomTableHandle l1
bf9a7c18  81c58ce0`e1aa1da8
lkd> dq 81c58ce0`e1aa1da8+10 l1
e1aa1db8  e1929b00`e1423c58
kd> dt nt!_HANDLE_TABLE e1929b00`e1423c58
   +0x000 TableCode        : 0xe19012b8
   +0x004 QuotaProcess     : 0xc0040004 _EPROCESS
   +0x008 UniqueProcessId  : 0x06010001 Void
   +0x00c HandleTableLock  : [4] _EX_PUSH_LOCK
   +0x01c HandleTableList  : _LIST_ENTRY [ 0x4d4d49 - 0xc060405 ]
   +0x024 HandleContentionEvent : _EX_PUSH_LOCK
   +0x028 DebugInfo        : (null) 
   +0x02c ExtraInfoPages   : 0n4390977
   +0x030 FirstFree        : 0x490050
   +0x034 LastFree         : 0x50005c
   +0x038 NextHandleNeedingPool : 0x50004e
   +0x03c HandleCount      : 0n3473456
   +0x040 Flags            : 0x310030
   +0x040 StrictFIFO       : 0y0
kd> !list "-t nt!_RTL_ATOM_TABLE_ENTRY.HashLink -e -x \"du @$extret+C\" e1929b00`e1423c58"
du @$extret+C 
e1423c64  "Native"
du @$extret+C 
e19012c4  "Object Descriptor"
du @$extret+C 
e1905f14  "Link Source Descriptor"
du @$extret+C 
e17e3fec  "Button"
du @$extret+C 
e108d014  "msctls_updown32"
du @$extret+C 
e108e224  "6.0.2600.2180!Static"
du @$extret+C 
e1c016f4  "MSIMEMouseOperation"
du @$extret+C 
e112b32c  "WorkerW"
du @$extret+C 
e167218c  "MSUIM.Msg.StubCleanUp"
du @$extret+C 
e1e6f12c  "Desktop More Programs Pane"
du @$extret+C 
e1e949fc  "FileContents"
du @$extret+C 
e1be4c1c  "CMBExecute"
du @$extret+C 
e2218d8c  "C:\WINDOWS\system32\xpsp2res.dll"
e2218dcc  ""
du @$extret+C 
e222a80c  "C:\WINDOWS\WinSxS\x86_Microsoft."
e222a84c  "Windows.Common-Controls_6595b641"
e222a88c  "44ccf1df_6.0.2600.2180_x-ww_a84f"
e222a8cc  "1ff9\comctl32.dll"
du @$extret+C 
e16524dc  "ControlOfs0040000000000DC4"
du @$extret+C 
e1971964  "ControlOfs0040000000000940"
du @$extret+C 
e214b4c4  "ControlOfs0040000000000EC4"
du @$extret+C 
e162340c  "ControlOfs0040000000000274"
du @$extret+C 
e1e7b7fc  "application/x-compressed"

Using Atom table monitor v1.0:
I have developed a small tool to visually monitor all atoms and look for different patterns on it using regular expressions and display the match with a different colour.

This small app will use GlobalGetAtomName (to get all atoms that have been added using GlobalAddAtom) and GetClipboardFormatName (to get all atoms that have been added using RegisterWindowMessage) functions to get all atoms and display them into a 128x128 memory grid using Delphi XE. It also keeps track of the amount of atoms through time plotting the results in a chart.

procedure ScanAtoms;
var
  index: WORD;
  arrAtom, arrClipboard: array [0 .. 1024] of char;
  AtomName, RWMName: string;
  lenAtom, lenClipboard: integer;
begin
  for index := $C000 to $FFFF do
  begin
    lenAtom := GlobalGetAtomName(index, arrAtom, 1024);
    lenClipboard := GetClipboardFormatName(index, arrClipboard, 1024);
    if lenAtom > 0 then
      FATomTable[index - $C000].atom := StrPas(arrAtom)
    else if lenClipboard > 0 then
      FATomTable[index - $C000].atom := StrPas(arrClipboard);
  end;
end;

Features:

Display global atom table from 0xC000 to 0xFFFF.
Display atom string from table.
Look for patterns and match them with a specific colour
Counters detailing the matches.
Plotting amount of atoms.

Testing:
Tool can be tested by using GlobalAddAtom or RegisterWindowMessages functions and check out the monitor.

Download:

Atom table monitor v1.0.

Jordi Corbilla

Related links:

Projecting 3D points to a 2D screen coordinate system

noreply@blogger.com (Jordi Corbilla) — Tue, 31 Jan 2012 22:17:00 +0000

In this article I will put forward a small function to project 3D points to a 2D coordinate system. I have used VLO framework to display all the points after the calculation of different common functions. The most difficult part is to correctly project 3D points into a 2D perspective. Basically the way to display the points is by using a viewport and setting it up correctly. Once all the points are calculated, we need to apply the transformations matrix to position the viewport and then draw the 3D object projected into a 2D canvas from the viewport perspective. The following function will use rotation matrix to correctly position the points.

Sample code:

procedure T3DMesh.CalcPoints();
function d3Dto2D (viewport: T3DViewPort; point : T3DPoint; centre : T2DPoint; zoom : double) : T2DPoint;
var
  p : T3DPoint;
  t : T3DPoint;
  d2d : T2DPoint;
begin
  p.x := viewport.x + point.x;
  p.y := viewport.y + point.y;
  p.z := viewport.z + point.z;

  t.x := (x * cos(viewport.roll)) - (z * sin(viewport.roll));
  t.z := (x * sin(viewport.roll)) + (z * cos(viewport.roll));
  t.y := (y * cos(viewport.pitch)) - (t.z * sin(viewport.pitch));
  z := (t.y * cos(viewport.pitch)) - (t.z * sin(viewport.pitch));
  x := (t.x * cos(viewport.yaw)) - (t.y * sin(viewport.yaw));
  y := (t.x * sin(viewport.yaw)) + (t.y * cos(viewport.yaw));
  d2d := nil;
  if z > 0 then
  begin
      d2d := T2DPoint.Create(((x / z) * zoom) + centre.x, ((y / z) * zoom) + centre.y);
  end;
  result := d2d;
end ;

var
  listPoint : TObjectList;
  i: Integer;
  j: Integer;
  x, y, z: Double;
  d2dpoint : T2DPoint;
begin
  listPoint := TObjectList.Create;
  listPoint2 := TObjectList.Create;

  x :=-2.0;
  y := -2.0;
  z := 0.0;
  for i := 0 to 40 do
  begin
    y := -2.0;
    for j := 0 to 40 do
    begin
      z := cos((x * x + y * y) * 2) * exp(-1 * ((x * x) + (y * y)));
      listPoint.Add(T3DPoint.Create(x,y,z));
      y := y + 0.1;
    end;
    x := x + 0.1;
  end;

  for i := 0 to listPoint.count - 1 do
  begin
    d2dpoint := d3Dto2D(viewport, T3DPoint(listPoint[i]), centre, zoom);
    if Assigned(d2dpoint) then
      listPoint2.Add(d2dpoint);
  end;
  listPoint.Free;
end;

Examples:

z = sin(x) * cos (y):

z = cos((x^2+y^2)*2) * exp-(((x^2)+(y^2))):

z = x^2*exp(-x*x)*y^2*exp(-y*y):

With parameters:

Related links:

Scripting Language with Thundax P-Zaggy

noreply@blogger.com (Jordi Corbilla) — Sun, 15 Jan 2012 01:09:00 +0000

In this article I will show you the approach taken to solve a common problem: Creating your own Scripting Language using a compiler. Sometimes we need to create our own language and to achieve that we either can create from scratch a lexical analyser in a high level language or use an open source automatic generator of lexical analysers. As I do not want to spent too much time in this task, I obviously chose the second one using an easy approach using JFLEX and CUP. There are many to choose from, e.g. Antlr, JavaCC, SableCC, Coco/R, BYacc/J, Beaver, etc. but I like the way Cup/Jflex work together. Jflex will automatically generate the finite automata of the lexical analyser through the regular expressions that define the tokens from a language. CUP is a system written in Java used to generate LALR syntax analysers. Cup file will contain the syntax definition of the language (Grammar) using a notation similar to BNF (Backus-Naur).

This approach will allow me to create a small compiler which will contain a lexical analyser, a syntax analyser and a semantic analyser without having to add extra workload to my simple Language builder. Cup/Jflex compiler will handle whether the language is well defined or not and in my Delphi project I only have a dummy function that will run if the previous compiler returns 0 errors.

I have also taken advantage of the SynEdit multi-line edit control to enhance the visualization of the simple Scripting Language. The exposed example is pretty basic and it will help to keep the grammar out of the Delphi code. I have tried other Delphi alternatives like GOLD parser and Coco/R but I did not get the expected results. So, If you have any better idea, please do let me know!.

This example will allow the user to define the graph by calling the layout object and connecting one node from another using a numeric description. In left side figure you can actually see the basics of the language and its definition. It invokes the layout object and then it generates a connection from the node(1) to node(2) and it ends the sentence with ";" character. There is no need to define the nodes as if a node does not exist it will be automatically created. The compiler created will manage all kind of errors and warnings and it will inform the user about them. Check that it will inform whether you are committing a lexical error (non recognized symbols, etc) and syntax error (bad composed expressions) through an Error or a semantic error (duplicated lines) through a warning.

A most developed example is shown in the following figure:

Once the language is free of errors, the generator is enabled and P-Zaggy can build the graph using the instructions previously defined and analysed by the external compiler. Note that to be able to check the language you must have installed Java.

The following graph have been created using the basic language:

As you can see it is faster than placing all the objects with the mouse and we can use external tools to generate the script and then check them with the compiler.

Here you can get the latest version of the app:

Thundax P-Zaggy v1.2.0 build 278.

Jflex regular expressions:

<YYINITIAL>layout {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.layout, new sToken(pos - yytext().length(), yyline, yytext()));
}
<YYINITIAL>node  {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.node, new sToken(pos - yytext().length(), yyline, yytext()));
}
<YYINITIAL>\( {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.OpenBracket, new sToken(pos - yytext().length(), yyline, yytext()));
}
<YYINITIAL>\)  {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.CloseBracket, new sToken(pos - yytext().length(), yyline, yytext()));
}
<YYINITIAL>tonode {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.tonode, new sToken(pos - yytext().length(), yyline, yytext()));
}
<YYINITIAL>parameters  {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.parameters, new sToken(pos - yytext().length(), yyline, yytext()));
}
<YYINITIAL>\. {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.dot, new sToken(pos - yytext().length(), yyline, yytext()));
}
<YYINITIAL>\; {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.endSentence, new sToken(pos - yytext().length(), yyline, yytext()));
}
<YYINITIAL>[0-9]+ {
  if (CUP$parser$actions.verbose)
    System.out.println(yytext());
  pos += yytext().length();
  return new Symbol(sym.INTEGER, new sToken(pos - yytext().length(), yyline, yytext()));
}

[ \t\r\n]+  { pos = 1; }
[a-z]+          {
  System.out.println("ERROR at line "+(Integer.valueOf(yyline)+1) + ": '" +yytext() + "' not recognized");
  pos += yytext().length();
  return new Symbol(sym.ff, new sToken(pos - yytext().length(), yyline, yytext())); }
.   {
  System.out.println("ERROR at line "+(Integer.valueOf(yyline)+1) + ": '" +yytext() + "' not recognized");
  pos += yytext().length();
  return new Symbol(sym.ff, new sToken(pos - yytext().length(), yyline, yytext()));
}

Cup grammar:

terminal        sToken  layout, node, OpenBracket, CloseBracket, tonode, parameters, dot, endSentence, ff;

terminal  sToken          INTEGER, ITEM, QUOTE;

non terminal  scriptClass graphLine;
non terminal    sToken          itemBracket, itemReturned;

graphLine ::=  layout : t1
   dot : t2
   node : t3
   itemBracket : e1
   dot : t6
   tonode : t7
   itemBracket : e2
          endSentence : t10
          graphLine
                        {:
    scriptLines.AddTokens(e1, e2);
    RESULT = scriptLines; :}
          | error;

itemBracket ::= OpenBracket : t4
   itemReturned : e1
   CloseBracket : t5 {: RESULT = (sToken)e1; :}
                        | error;

itemReturned ::= INTEGER : e1 {: RESULT = (sToken)e1; :}
                 | error;

Related links:

The rules of a creator’s life (the secret formula)

noreply@blogger.com (Jordi Corbilla) — Mon, 02 Jan 2012 11:44:00 +0000

(source creativesomething.net). Rules created by Tanner Christensen of Creative Something showcases a recipe for success.

Finite automata to Regular Grammar with Thundax P-Zaggy

noreply@blogger.com (Jordi Corbilla) — Fri, 30 Dec 2011 18:00:00 +0000

Moving ahead with FA, now we can generate the right-linear grammar that corresponds to the finite automata generated. The system is pretty simple and basically what the algorithm does is to go recursively through every state and to store the production (A → bC) using the info from the transition and both states.

P-Zaggy will use description text to assign unique variable names to each state. Each transition in the graph matches a production in the right-linear grammar. For example (right side image), the transition from “A” to “B” along the transition “a” matches the production “A → aB”. This production will be automatically added into the production list in the "Finite Automaton (simulation)" tab. To generate the full grammar, just open your finite automata and press "Get grammar" button to get the linear grammar. Now as any final state carries the production to λ, λ symbol has been added to the system as empty string.

In the following image you will see a general example with the automatically generated grammar:

Example with λ string:

Demo video:

Get the latest version from here:

Thundax P-Zaggy v1.20 build 208. (Executable)

Looking back/ahead:

This would be the last post of the year 2011 and I'm taking this moment to review all the good stuff I have been publishing in my blog, from the physics engine to different code snippets and all of them with the help of a big an bright Delphi community. I always find difficult to get round to writing more often but I always have my notebook with me and every time and idea pops out I am there to catch it. I have an endless list of really interesting stuff I would like to tackle and I hope next year would be more fruitful as there are less than 50 articles for this year (even though the visits have been increased to almost 6K per month wow!, I am grateful for that!). I am also blissfully happy for putting grain of salt in the community as I have widely used the knowledge of others and it is great to pay back.

Next year I will be implementing different algorithms and I will be focusing on grammars, compilers and AI as I am very keen on those subjects and as you already know I like a lot visual stuff (And everything or almost everything with my loved Delphi). I'm preparing a roadmap about different utilities I have in mind and stay tuned for new and very interesting stuff!.

Happy new year 2012!.

Jordi Corbilla

Related links:

Finite Automata with Thundax P-Zaggy part II

noreply@blogger.com (Jordi Corbilla) — Thu, 29 Dec 2011 18:53:00 +0000

Going on with the further enhancement of P-Zaggy (part I) to use its graph creation capability to run and simulate DFA, I have released a new version (v1.2.0 build 180) which allows you to check a bunch of strings in batch. One the DFA is defined, we can easily test our input strings by adding them into the list and the application will check every item and it will populate the status with "accepted/rejected" message. This version still only works with one char regular expressions (^[A-Z]$) so more complex expressions (^[A-Z]+$) will be used but not correctly adapted.

Examples:

Related links:

Regular expressions.

Finite Automata with Thundax P-Zaggy part I

noreply@blogger.com (Jordi Corbilla) — Thu, 29 Dec 2011 02:25:00 +0000

I have moved forward the development of Thundax P-Zaggy with the addition of a Finite Automata builder and simulator. "A deterministic finite automaton (DFA) also known as deterministic finite state machine is a finite state machine that accepts/rejects finite strings of symbols and only produces a unique computation (or run) of the automaton for each input string. (wikipedia)". Building the automata with P-Zaggy is quite simple, so let's start by building a DFA for the following language:

L = {ba | abⁿa, n > 0}.

As it is shown in the previous image, the DFA would be as follows:

With the tool we would be able to create our diagrams placing the states and transitions. Once the diagram is set up, all transitions need to have the transition string to change the ongoing state and initial and final states need to be selected (using right click and selecting "Mark as Start Node" for Initial state and "Mark as End Node" for Final state).

We can start the simulation with very simple steps. Firstly, select the tab called "Finite Automaton (simulation)" and enter the test input string to test. Then press start button to internally initialize the graph as a DFA and press step to process every character of the input string while the diagram is changing states and showing you which is the actual state and whether the string is being accepted or rejected. To set up a single loop transition, select the node and with right click select "Set connector loop" and add text for the transition.

In this beta version, we can also use Regular Expressions in our transitions thanks to the widespread use of regular expressions inside Delphi XE.

From the same screen we can test our regular expressions and check whether if they match or not. There are still some issues to tackle like "empty string" and other considerations that I would be delving into detail in the following days. I am also working on a Layered-Tree-Draw Algorithm using Reingold-Tilford Algorithm and the enhancement of the better recognition of more complex regular expressions.

Have a look at this preview of the functioning of the graph, it is really worth it:

Stay tuned and give P-Zaggy a try from here:

Thundax P-Zaggy v1.2.0 build 155. (there are two examples included).

Related links:

Install multiple instances of the same Delphi application service

noreply@blogger.com (Jordi Corbilla) — Sat, 10 Dec 2011 00:46:00 +0000

In this article you will find an example on How to install multiple instances of the same Delphi application service under Windows. This topic was raised a few years ago in StackOverflow and I have decided to give more details about how to use multiple instances of the same service.

The service needs to be unique in name and the best way to sort this out is using a parameter to describe the service in a unique way. The service itself will use this parameter to compose its name even though it is using the same executable. In this example I will identify the services when installing the service using the command line with the following instruction: "myService /install Param1". Param1 will be used to compose the internal service name as myServiceParam1. If we want to use another instance, just install a second service with a different param name as: myService /install Param2.

To achieve this is pretty simple, just create the service using your Delphi XE and then create the methods ServiceBeforeUninstall and ServiceCreate:

unit uService;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, SvcMgr, Dialogs, WinSvc;

type
  TServiceExample = class(TService)
    procedure ServiceCreate(Sender: TObject);
    procedure ServiceBeforeUninstall(Sender: TService);
  private
    FDescription: String;
    FPreviousName: String;
    FPreviusDisplayName: string;
  public
    function GetServiceController: TServiceController; override;
    { Public declarations }
  end;

var
  ServiceExample: TServiceExample;

implementation

uses
  ComObj;

{$R *.DFM}

procedure ServiceController(CtrlCode: DWORD); stdcall;
begin
  ServiceExample.Controller(CtrlCode);
end;

function TServiceExample.GetServiceController: TServiceController;
begin
  Result := ServiceController;
end;

procedure TServiceExample.ServiceBeforeUninstall(Sender: TService);
begin
  Name := FPreviousName + FDescription;
  DisplayName := FPreviusDisplayName + FDescription;
end;

procedure TServiceExample.ServiceCreate(Sender: TObject);
begin
  FDescription := System.ParamStr(2);
  FPreviousName := Name;
  FPreviusDisplayName := DisplayName;
  Name := Name + FDescription;
  DisplayName := DisplayName + FDescription;
end;

end.

Notice the System.ParamStr(2) that will use the second parameter which was input from the command line. The first parameter is the /install command needed to install the service.

After the execution of the following commands:

myService /install Param1
myService /install Param2

You will see two installed services using the same executable:

To uninstall them, use the opposite command:

myService /uninstall Param1

myService /uninstall Param2

But this is not ending here. Now the services are installed, but neither the service name is set nor the description. To achieve this, we need to use the ChangeServiceConfig functions from WinSvc - Service Control Manager unit.

Have a look at the complete source code:

unit uService;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, SvcMgr, Dialogs, WinSvc;

type
  TServiceExample = class(TService)
    procedure ServiceCreate(Sender: TObject);
    procedure ServiceBeforeUninstall(Sender: TService);
    procedure ServiceAfterInstall(Sender: TService);
  private
    FDescription: String;
    FPreviousName: String;
    FPreviusDisplayName: string;
  public
    function GetServiceController: TServiceController; override;
    { Public declarations }
  end;

type
  SERVICE_DESCRIPTION = packed record
    lpDescription: PChar;
  end;

  PSERVICE_DESCRIPTION = ^SERVICE_DESCRIPTION;

var
  ServiceExample: TServiceExample;

const
  SERVICE_CONFIG_DESCRIPTION = 1;

function ChangeServiceConfig2(hService: SC_HANDLE; dwInfoLevel: DWORD; lpInfo: Pointer): BOOL; stdcall; external 'advapi32.dll' name 'ChangeServiceConfig2W';

implementation

uses
  ComObj;
{$R *.DFM}

procedure ServiceController(CtrlCode: DWORD); stdcall;
begin
  ServiceExample.Controller(CtrlCode);
end;

function TServiceExample.GetServiceController: TServiceController;
begin
  Result := ServiceController;
end;

procedure TServiceExample.ServiceAfterInstall(Sender: TService);
var
  SvcMgr, Svc: SC_HANDLE;
  desc: SERVICE_DESCRIPTION;
begin
  SvcMgr := OpenSCManager(nil, nil, SC_MANAGER_ALL_ACCESS);
  if SvcMgr = 0 then
    Exit;
  try
    Svc := OpenService(SvcMgr, PChar(Name), SERVICE_ALL_ACCESS);
    if Svc = 0 then
      RaiseLastOSError;
    try
      desc.lpDescription := PWideChar(ParamStr(0) + ' ' + FDescription);
      ChangeServiceConfig( Svc, SERVICE_NO_CHANGE, SERVICE_NO_CHANGE,SERVICE_NO_CHANGE, desc.lpDescription, nil, nil, nil, nil, nil, nil);
      desc.lpDescription := PWideChar('Service Example with Params = ' + FDescription);
      ChangeServiceConfig2(Svc, SERVICE_CONFIG_DESCRIPTION, @desc.lpDescription);
    finally
      CloseServiceHandle(Svc);
    end;
  finally
    CloseServiceHandle(SvcMgr);
  end;
end;

procedure TServiceExample.ServiceBeforeUninstall(Sender: TService);
begin
  Name := FPreviousName + FDescription;
  DisplayName := FPreviusDisplayName + FDescription;
end;

procedure TServiceExample.ServiceCreate(Sender: TObject);
begin
  FDescription := System.ParamStr(2);
  FPreviousName := Name;
  FPreviusDisplayName := DisplayName;
  Name := Name + FDescription;
  DisplayName := DisplayName + FDescription;
end;

end.

This solution is using the example provided by koochangmin on his Delphi blog. If you have a closer look at the ServiceAfterInstall method, you will find the composition of the name and description which will be used to populate the values in the service:

That is the way the service needs to be created in order to let the application use the parameters in runtime.

If you have any problem uninstalling the services, just use the SC command to delete the service:

sc delete ServiceExampleParam1

Related links:

Google and Facebook "Secrets" unveiled

noreply@blogger.com (Jordi Corbilla) — Mon, 07 Nov 2011 18:38:00 +0000

Here you can see an interesting video from one of the TED talks this year in which we can see how the information is filtered and presented according to our surfing (using algorithmic and personalized filters). It gives interesting reflections.

Enjoy the video!.

Monitoring Desktop Heap Memory and troubleshooting issues (part II)

noreply@blogger.com (Jordi Corbilla) — Mon, 17 Oct 2011 19:07:00 +0000

Now that we know what Desktop Heap Memory is and How works (Monitoring Desktop Heap Memory and troubleshooting issues (part I)), we can extend this functionality and retrieve the Heap size from our Delphi app and even better, create a new desktop with a specified heap size.

To achieve this I am going to use the public API functions introduced in Windows Vista: CreateDesktopEx, which allows the caller to specify the size of desktop heap.And, GetUserObjectInformation that includes a flag for retrieving the desktop heap size (UOI_HEAPSIZE).

Get Heap Size:

To get the heap size, we just need to invoke the GetUserObjectInformation using the UOI_HEAPSIZE flag:

procedure GetHeapSizeClick();
var
  HDesktop: HDESK;
  UHeapSize: ULong;
  tempDword: DWORD;
begin
  HDesktop := OpenInputDesktop(0, False, DESKTOP_CREATEMENU or
    DESKTOP_CREATEWINDOW or DESKTOP_ENUMERATE or DESKTOP_HOOKCONTROL or
    DESKTOP_WRITEOBJECTS or DESKTOP_READOBJECTS or DESKTOP_SWITCHDESKTOP or
    GENERIC_WRITE);

  GetUserObjectInformation(HDesktop, UOI_HEAPSIZE, @UHeapSize, SizeOf(UHeapSize), tempDword);
  OutputDebugString(PChar('UOI_HEAPSIZE ' + Inttostr(UHeapSize)));
  CloseDesktop(HDesktop);
end;

The output of the string is as follows:
Debug Output: UOI_HEAPSIZE 12288 Process Project1.exe (4224)

Where the value highlighted in bold, is the predefined size of your heap in the SubSystems\Windows registry key.

Create a new desktop with a specific heap size:

Using the CreateDesktopEx or CreateDesktopExW (Unicode):

var
  HDesktopglobal: HDESK;

procedure CreateDesktop();
var
  UHeapSize: ULong;
  tempDword: DWORD;
begin
  try
    HDesktopglobal := CreateDesktopExW('Z', nil, nil, DF_ALLOWOTHERACCOUNTHOOK, GENERIC_ALL, nil, 3052, nil);
  except
    on e: exception do
      ShowMessage(SysErrorMessage(GetLastError));
  end;
  if HDesktopglobal <> 0 then
  begin
    GetUserObjectInformation(HDesktopglobal, UOI_HEAPSIZE, @UHeapSize, SizeOf(UHeapSize), tempDword);
    OutputDebugString(PChar('UOI_HEAPSIZE ' + Inttostr(UHeapSize)));
  end;
end;

procedure CloseDesk();
begin
  CloseDesktop(HDesktopglobal);
end;

We can check the result with dheapmon tool:

With more details using dheapmon -v and the looking for 'Z' desktop.

The output of the string is as follows:

Debug Output: UOI_HEAPSIZE 3052 Process Project1.exe (364)

Remember that the Desktop needs to be closed.

Related links:

Open a list of URLs

noreply@blogger.com (Jordi Corbilla) — Sun, 16 Oct 2011 01:05:00 +0000

Going on with my previous entries, in this post I am releasing a little tool to open a web browser (in this case google chrome) and populate it with a list of stored URLs. This tool would be the culmination of the articles: My first Google Chrome extension (Get all URLs from Google Chrome tabs) and Get Chrome active tab URL using Delphi. And as you know, I love automating and when I am surfing the internet I like keeping track of my URLs and being able to retrieving and saving them easily.

You can download the tool from here. The tool only needs the path to your browser (in my case google chrome) and a list of pre-saved urls (for example the output list from my chrome extension).

The tool looks like this:

And it uses the ShellExecute function from ShellAPI.

Enjoy it!.

Related Links:

Open browser.

Monitoring Desktop Heap Memory and troubleshooting issues (part I)

noreply@blogger.com (Jordi Corbilla) — Sun, 16 Oct 2011 00:48:00 +0000

This article wants to put together a series of interesting articles to monitor the desktop heap memory and finally put forward a little in-house tool to monitor the output of Desktop Heap Information Monitor Tool v8.1 from AireSoft using the algorithm from "capturing output capture" from one of my previous posts.

Installing "Desktop Heap Information Monitor Tool v8.1:
To correctly install the tool (dheapdmp.zip), go to AireSoft web page and follow the installation steps from the web. If an error pops up, follow the guidelines from the following links:
- dheapInst - driver installation error occurred (2).
- dheapinst - Driver Installation error occured (124).

If it helps, I got the error "Driver Installation error occured (124)" and I fixed it by changing the compatibility of dheapinst to Windows Server 2003 SP1.

Once installed, if we run dheapmon -l to install the monitor and then again dheapmon, we will see the heap memory information:

More about "Desktop heap memory":

The Desktop Heap Monitor is a tool that examines usage of desktop heap.

WIN32 subsystem has internal heap area known as "desktop heap." When you run a large number of Windows-based programs, "Out Of Memory" error messages appear when you attempt to start new programs or try to use programs that are already running, even though you still have plenty of physical and pagefile memory available.

This behavior can occur if the desktop heap in the WIN32 subsystem is depleted. To fix this problem follow the next workaround :
- http://support.microsoft.com/kb/126962/

source: Microsoft.

Increasing "Desktop heap memory":

To increase desktop heap memory, we can use the tool "Microsoft Fix It 50496" that will increase the parameters automatically. If you want to fix it manually, you just need to go to the following registry path and tweak it:
- HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\SubSystems

Where the default values for my windows vista are:

%SystemRoot%\system32\csrss.exe

ObjectDirectory=\Windows

SharedSection=1024,12288,512

Windows=On

SubSystemType=Windows

ServerDll=basesrv,1

ServerDll=winsrv:UserServerDllInitialization,3

ServerDll=winsrv:ConServerDllInitialization,2

ProfileControl=Off

MaxRequestThreads=16

The first SharedSection value (1024) defines the heap size common to all desktops. This includes the global handle table.

The second SharedSection value (12288) controls the size of the desktop heap that is associated with an interactive window station (used for Windows objects). This static value is used to prevent ill- behaved applications from consuming too many resources. Because the desktop heap is mapped into each process' address space, this value should not be set to an arbitrarily high value (as it would decrease performance), but should only be increased sufficiently to allow all the desired applications to run.

The third SharedSection value (512) controls the size of the desktop heap for each desktop that is associated with a "non-interactive" window station.

source : microsoft.

Monitoring Desktop Heap Memory:
As it is quite difficult to monitor, I have developed a little tool that you can download from here, to log every change on the table result.

Just point the tool to the dheapmon executable and press start. Every 10s the output will be scanned and displayed into the component, taking into account the previous value, and if a change occurs it will be displayed in another colour.

Final notes:
Desktop Heap memory is related to "Out of Memory" issues and this article will help you to work with this parameter and how to deal with it and monitor it.

Stay tuned for Part 2, where I will go further with Desktop heap memory using Delphi.

Jordi Corbilla.

Related links:

My tribute to a man who was a unique genius, Steve Jobs

noreply@blogger.com (Jordi Corbilla) — Sat, 08 Oct 2011 21:59:00 +0000

Steve Jobs was a unique creative genius whose life was far too brief. Source: macworld.co.uk.

Get Chrome active tab URL using Delphi

noreply@blogger.com (Jordi Corbilla) — Sun, 18 Sep 2011 14:17:00 +0000

After developing my first Chrome extension to retrieve all URLs from Google chrome, I have been trying to reproduce the same behaviour using Delphi, but it was impossible. Chrome is still adding some new features and extending their extensions for developers. For example, others navigators such as Internet Explorer and Firefox they have a DDE extension to retrieve some of the properties but not available for Chrome.

The best way I found was using FindWindow and SendMessage windows functions to get the text from the tab and get tue URL. It's not a win solution but will help you to retrieve the URL of the active page without copy-paste action, just by switching to the next tab and retrieving data from the tool.

The basic code to achieve this is the following one:

function GetChromeActiveTabURL(Wnd: HWnd; Param: LParam): Bool; stdcall;
var
  urls: TStrings;
  hWndMainWindow, hWndTab: HWND;
  Buffer : array[0..255] of Char;
  res : boolean;
begin
  res := true;
  urls := TStrings(Param);
  SendMessage(Wnd, WM_GETTEXT, Length(Buffer), integer(@Buffer[0]));
  hWndMainWindow := FindWindow('Chrome_WidgetWin_0', Buffer);
  application.ProcessMessages;
  if hWndMainWindow <> 0 then
  begin
    hWndTab := FindWindowEx(hWndMainWindow, 0, 'Chrome_AutocompleteEditView', nil);
    if hWndTab <> 0 then
    begin
      SendMessage(hWndTab, WM_GETTEXT, Length(Buffer), integer(@Buffer));
      urls.Add(Buffer);
      res := false;
    end;
  end;
  Result := res;
end;

procedure TUrlChrome.GetUrl(Sender: TObject);
var
  Urls: TStringList;
begin
  Urls := TStringList.Create;
  try
    EnumWindows(@GetChromeActiveTabURL, LParam(Urls));
    Memo1.Lines.AddStrings(Urls);
  finally
    FreeAndNil(Urls);
  end;
end;

To get the class name window, we can use Winspector to inspect chrome and get the names for the main window and for the tabs:

Chrome_WidgetWin_0:

Chrome_AutocompleteEditView:

And you can get the tool from here: ThundaxChromeURL.

I'm not proud of this solution, but at least it will work for current versions of Chrome. I also recommend to give a go to my chrome extension (It's not been published into Chrome market, it turns out that you have to pay a $5.00 fee), that is much better than the tool as it's able to get all urls.

Related links:

My first Google Chrome extension (Get all URLs from Google Chrome tabs)

noreply@blogger.com (Jordi Corbilla) — Sat, 17 Sep 2011 19:45:00 +0000

As a good internet researcher I always have hundreds of tabs opened in my Chrome navigator and sometimes I find a bit of chore to copy and paste all the URL that I have done a research on and that any other day I would be using again. So, that's my extension about, to display in a pop up window all the url's from all tabs to then copy them without problems (the correct name would be [search purposes] -> Get all URLs from Google Chrome tabs).

I have been struggling to write a little tool using Delphi, but it only works with Internet explorer and Mozilla firefox, so I thought it could be great to give chrome extensions a try.

It is quite easy to develop and debug using all the development tools available from the navigator itself. As in the following example, I have opened 3 new tabs with the www.google.es URL, and if we click the extension button, will get a pop up list with the URLs from all tabs:

The source code is as follows:

manifest.json:

{
  "name": "Get All URL's Extension by Jordi Corbilla",
  "version": "1.0",
  "description": "Extension to display all the URL's from all TABs.",
  "browser_action": {
    "default_icon": "icon.ico",
    "popup": "urlList.html"
  },
  "permissions": [
    "tabs"
  ],
  "icons" : {

    "48" : "icon.ico",


    "128" : "icon.ico"


  }
}

urlList.html:

<!DOCTYPE>
<!--
 * Copyright (c) 2011 Jordi Corbilla. All rights reserved.  
-->
<html>
  <head>
    <script>
      chrome.tabs.getAllInWindow(null, function(tabs) {
        tabs.forEach(function(tab){
          myFunction(tab.url);	
        });
      });

      function myFunction(tablink) {
        console.log(tablink);
        var oNewNode = document.createElement("LI");
        urlList.appendChild(oNewNode);
        oNewNode.innerText=tablink;  
      }
</script>
</head>
  <body>
    <div>URL List</div>
    <ul ID = urlList>
    </ul>
  </body>
</html>

You can download the extension from here: URLJCExtension.zip. To install, you only need to unzip the file and then go to chrome://extensions/, expand developer mode and load my extension (from the unzipped folder):

As the extension uses the console log method, you can see all the links in the console (just for debug purposes, but it's really good to have a go on it).

Just right click on the extension icon and then "Inspect Pop-up" and you'll see the following window with the current list of urls:

Enjoy the extension!.

Related links:

Fluent Interfaces example using Delphi part I

noreply@blogger.com (Jordi Corbilla) — Fri, 26 Aug 2011 16:14:00 +0000

I was introduced to Fluent interfaces when I was developing with Java using Dependency Injection and somebody told me that it could be really interesting if I post an example using Delphi and chaining method, and here I am!. I have developed a simple example that uses chaining methods to populate a THouse object with TDoors and TWindows objects with the hope it will make it easy to understand the concept. There are several examples on the net, but I couldn't find any interesting enough. So this simple example will try to introduce you to "how to develop fluent interfaces using method chaining" with the aim of writing more readable code. The overview of the architecture is the following one, using UML notation:

The Fluent Interface unit example is as follows (uFI.pas) using generics and interfaces:

// ***********************************************
// @author  jordi corbilla
// @comment Fluent interface composition example
//
// ***********************************************

unit uFI;

interface

uses
    Graphics, Contnrs, Generics.Collections, Classes;

type
    TWindow = class;

    IWindow = interface
        function SetColor(color: TColor): TWindow;
        function ToString(): string;
    end;

    TWindow = class(TInterfacedObject, IWindow)
    private
        FColor: TColor;
    public
        function SetColor(color: TColor): TWindow;
        function ToString(): string; override;
        class function New: TWindow;
    end;

    TDoor = class;

    IDoor = interface
        function SetColor(color: TColor): TDoor;
        function ToString(): string;
    end;

    TDoor = class(TInterfacedObject, IDoor)
    private
        FColor: TColor;
    public
        function SetColor(color: TColor): TDoor;
        function ToString(): string; override;
        class function New: TDoor;

    end;

    IHouse = interface
        function AddWindow(window: TWindow): IHouse;
        function AddDoor(door: TDoor): IHouse;
        function ToString(): String;
    end;

    THouseItemsList<T> = class(TObjectList)
        function ToString(): string; override;
    end;

    TDoorList = class(THouseItemsList<IDoor>)
        function ToString(): string; override;
    public

    end;

    TWindowList = class(THouseItemsList<IWindow>)
        function ToString(): string; override;
    end;

    THouse = class(TInterfacedObject, IHouse)
    private
        FDoorList: TDoorList;
        FWindowList: TWindowList;
    protected
        function AddWindow(window: TWindow): IHouse;
        function AddDoor(door: TDoor): IHouse;
    public
        property WindowList: TWindowList read FWindowList;
        property DoorList: TDoorList read FDoorList;
        class function New: IHouse;
        function ToString(): String; override;
        constructor Create();
        destructor Destroy(); override;
    end;

implementation

uses
    SysUtils;

{ THouse }

function THouse.AddDoor(door: TDoor): IHouse;
begin
    Self.FDoorList.Add(door);
    result := Self;
end;

function THouse.AddWindow(window: TWindow): IHouse;
begin
    Self.FWindowList.Add(window);
    result := Self;
end;

constructor THouse.Create;
begin
    FDoorList := TDoorList.Create;
    FWindowList := TWindowList.Create;
end;

destructor THouse.Destroy;
begin
    FreeAndNil(FDoorList);
    FreeAndNil(FWindowList);
    inherited;
end;

class function THouse.New: IHouse;
begin
    result := Create;
end;

function THouse.ToString: String;
var
    description: TStringList;
begin
    description := TStringList.Create;
    try
        description.Add(Self.FDoorList.ToString);
        description.Add(Self.FWindowList.ToString);
    finally
        result := description.Text;
        description.Free;
    end;
end;

{ TWindow }

class function TWindow.New: TWindow;
begin
    result := Create;
end;

function TWindow.SetColor(color: TColor): TWindow;
begin
    Self.FColor := color;
    result := Self;
end;

function TWindow.ToString: string;
begin
    result := 'Window color ' + ColorToString(FColor) + sLineBreak;
end;

{ TDoor }

class function TDoor.New: TDoor;
begin
    result := Create;
end;

function TDoor.SetColor(color: TColor): TDoor;
begin
    Self.FColor := color;
    result := Self;
end;

function TDoor.ToString: string;
begin
    result := 'Door color ' + ColorToString(FColor) + sLineBreak;
end;

{ THouseItemsList }

function THouseItemsList<T>.ToString: string;
var
    i: Integer;
    description: String;
begin
    for i := 0 to Self.count - 1 do
        description := description + ' ' + Self[i].ToString;
    result := description;
end;

{ TDoorList }

function TDoorList.ToString: string;
var
    description: TStringList;
begin
    description := TStringList.Create;
    try
        description.Add('Number of doors: ' + inttostr(Self.count));
        description.Add('Descriptions: ');
        description.Add( inherited ToString);
    finally
        result := description.Text;
        FreeAndNil(description);
    end;
end;

{ TWindowList }

function TWindowList.ToString: string;
var
    description: TStringList;
begin
    description := TStringList.Create;
    try
        description.Add('Number of Windows: ' + inttostr(Self.count));
        description.Add('Descriptions: ');
        description.Add( inherited ToString);
    finally
        result := description.Text;
        FreeAndNil(description);
    end;
end;

end.

Now it comes when we can use the object using FI notation in a more readable way:

var
    House: IHouse;
begin
    House := THouse.New.AddWindow(TWindow.New.SetColor(clFuchsia)).AddDoor(TDoor.New.SetColor(clWhite))
        .AddWindow(TWindow.New.SetColor(clRed)).AddDoor(TDoor.New.SetColor(clGreen));
    Memo1.Lines.Add(House.ToString);
    House := nil;

    House := THouse.New;
    House.AddWindow(TWindow.New.SetColor(clRed)).AddWindow(TWindow.New.SetColor(clBlue)).AddWindow(TWindow.New.SetColor(clGreen));
    House.AddDoor(TDoor.New.SetColor(clRed)).AddDoor(TDoor.New.SetColor(clBlue)).AddDoor(TDoor.New.SetColor(clGreen));
    Memo1.Lines.Add(House.ToString);
    House := nil;

Notice that now, we can chain our objects, composing the structure of the House by using the AddWindow and AddDoor method. To display the result I'm using the ToString method that will do the job for my purpose.

And the result of the previous execution:

Number of doors: 2
Descriptions:
Door color clWhite
Door color clGreen

Number of Windows: 2
Descriptions:
Window color clFuchsia
Window color clRed

Number of doors: 3
Descriptions:
Door color clRed
Door color clBlue
Door color clGreen

Number of Windows: 3
Descriptions:
Window color clRed
Window color clBlue
Window color clGreen

Just play with it and you'll experience the powerfulness of this method. I hope you like it.

I'm looking forward to your comments.

Jordi

Building my own Delphi Physics Engine part IX

noreply@blogger.com (Jordi Corbilla) — Mon, 22 Aug 2011 20:08:00 +0000

I'm quite familiar with GDI, the graphics provided under Microsoft and which is widely used in Windows applications. TDPE is now ready to use Direct2D as now it's fully supported by Delphi 2010 and XE. According to MSDN, Direct2D is: a hardware-accelerated, immediate-mode, 2-D graphics API that provides high performance and high quality rendering for 2-D geometry, bitmaps, and text. The Direct2D API is designed to interoperate well with GDI, GDI+, and Direct3D.

As I'm testing this under Windows Vista, I've had to upgrade my windows version to Service Pack 2 and then I've installed the following update (KB971512 - The Windows Graphics, Imaging, and XPS Library contain the latest advancements in modern graphics technologies for gaming, multimedia, imaging and printing applications.). Direct2D is already installed in Windows 7.

We can use the same methods from TCanvas for our TDirect2DCanvas as they derive from the same ancestor TCustomCanvas. But you need to check the documentation for differences in the way they work.

A simple way to use Direct2D is as follows:

uses
    Direct2D, D2D1;

procedure Paint();
begin
    if TDirect2DCanvas.Supported then
    begin
        d2dCanvas := TDirect2DCanvas.Create(Canvas, ClientRect);
        try
            d2dCanvas.RenderTarget.beginDraw;
            d2dCanvas.RenderTarget.Clear(D2D1ColorF(clBlack));
            d2dCanvas.RenderTarget.SetAntialiasMode(D2D1_ANTIALIAS_MODE_PER_PRIMITIVE);
            // drawing goes here
            Paint(d2dCanvas);
            d2dCanvas.RenderTarget.EndDraw;
        finally
            FreeAndNil(d2dCanvas);
        end;
    end;
end;

Here is the result of the Physics Engine using a Direct2D Render and a GDI Render:

You will see the difference when switching from GDI to Direct2D as the performance is quite different. When it comes to Direct2D, my computer is getting quite slower as my graphic card is not powerful at all and we can see that it takes a while to render the image.

GDI uses Pixel graphics but Direct2D can supports vector graphics as well, in which mathematical formulas are used to draw the lines and curves. Vector graphics provides high quality rendering independent of resolution of the device, while the pixelated graphics has dependency with resolution which may results in choppy graphics.

Most of the GDI APIs are not using anti-aliasing and transparency. Ofcrouse there are functions to do so but always there’s programming cost for taking advantage of these features. Also if we apply transparency and anti-aliasing, the computations are done using CPU. Direct2D can take advantage of graphics hardware and delegate the computationally intensive tasks to GPU.

Source : Programming Direct2D.

Get the latest version here: ThundaxBallDemo v1.584.
Interaction:
'd' will drop a box with a random colour and a constant angular velocity.
'f' will drop a circle with a random colour and a constant angular velocity.
'mouse interaction' mouse is able to grab items.
'q' Enable/Disable GDI or Direct2D rendering.

Note: You need to have installed Direct2D to run the app using Direct2D, otherwise GDI will be used.

Related links:

Building my own Delphi Physics Engine part VIII

noreply@blogger.com (Jordi Corbilla) — Wed, 10 Aug 2011 21:04:00 +0000

TDPE is almost ready and it will not take long to release the first stable library!. I am eager to continue working on the project trying to sort out different physic scenarios to make my dynamic engine even more realistic. Do not hesitate to give it a try as you will find the experience quite interesting. I am still working on the n-edge object and the cutting tool and trying to refactor all the code to cope with a layer architecture. Using this concept, I want to structure my system in a way that every component works using a certain level of abstraction. As you see in the next figure, every object is allocated in the screen while there is another layer which is responsible for the collision detection and another one for the forces interaction. You will notice in the video that there are still little issues to tackle regarding the object interaction as sometimes we can experience overlapping.

In the figure above, you can see that now the items are not rigid and they interact with each other as they were real items. Notice that the square item will go through the round shape of the circle.

Enjoy the video!.

Get the latest executable here: ThundaxBallDemo v1.504.
Interaction:
'd' will drop a box with a random color and a constant angular velocity
'f' will drop a circle with a random color and a constant angular velocity
'mouse interaction' mouse is able to grab items

Related links:

Continuous Integration for your Delphi projects using Jenkins CI

noreply@blogger.com (Jordi Corbilla) — Sun, 10 Jul 2011 20:07:00 +0000

Going on with the utilization of Msbuild, I'm putting forward this post using Jenkins CI (Continuous Integration) previously known as Hudson, for your Delphi projects under Windows Vista. In a nutshell, Jenkins provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases the productivity. (Source: What is Jenkins?). With these simple steps I'll lead you throughout the whole installation and configuration of your first Delphi built project using Jenkins CI and MsBuild.

1 . Download the latest version of Jenkins CI.

Use the following link to download the latest windows installer, in my case v1.419. The default installation path is C:\Program Files\Jenkins and there you'll find the service wrapper jenkins.exe and the Java web archive (jenkins.war) that is very useful to start the Jenkins if you have difficulties installing it with a container. It's very easy to run Jenkins by itself using the command: java -jar jenkins.war (which uses Winstone as a container).

2. Setting up environment variables.

To make sure that Jenkins runs perfectly, we need to set up the JENKINS_HOME environment variable:

3. Adding administrator privileges to Jenkins windows service.

We need to be sure that the service has enough permissions to be running. To do this, go to console and type down "services.msc". Then look for Jenkins service and add the log on information using an administrator account.

I had a lot of troubles trying to execute MSBuild from Jenkins until I realised that the service didn't have enough permissions.

4. Installing Jenkins CI plug-ins.

Once everything is done, we can go on with Jenkins and open a web explorer and type http://localhost:8080. Now we need to add the following plug-ins: Jenkins MSBuild Plugin and Hudson Setenv Plugin. To install them go to "Manage Jenkins"-> "Manage Pluggins" -> Available and look for them. Then, save the changes at the end of the page and reload Jenkins.

Once installed, you'll find them under the Installed section:

5. Configure MSBuild.

Now the plug-in is ready, and we need to supply the path of our MSBuild. Go to "Manage Jenkins" -> "Configure Jenkins" and add the Msbuild information in the MSbuild section:

6. Adding environment variables.

As we are trying to build Delphi projects we need to set up Jenkins with the same environment variables included in rsvars.bat batch file. Go to "Manage Jenkins" -> "Configure Jenkins" and add the following environment variables according to your Delphi version:

7. Setting up your project.

Start a new job with your project description and choose the "Build a free-style software project" type. And add the following MSbuild information to build your project (dproj file):

8. Build your Project.

Everything is ready to do the build. Now execute the "Build now" option and you'll see your project being compiled and built using MSBuild.

Now, you can add your Delphi projects and improve the quality of your software.

Related Links: