Davis's tech blog

Office where Mac computers couldn't browse HTTPS sites

2011-11-15T23:20:00.009+02:00

Recently I was invited to troubleshoot network problems in an office where all Mac computers virtually couldn't browse any HTTPS web sites. They reported that opening online banking site tool about 4 minutes on Mac computer and only a couple of seconds on Windows PC. Meanwhile the same Mac computer was able to open the same online banking site in seconds when connected to internet at another office.
My first suspicion was lowered (less that 1500) path MTU and ICMPs blocked by ISP firewall, however this turned out to be false (by running mturoute from a Windows machine). Also running ping with 1472 bytes of ICMP data (and DF flag set) to some hosts on the Internet verified that MTU was 1500 and showed no packet loss.
Running Wireshark on one of affected Mac computer while opening a HTTPS website and analysis of gathered dump showed a few percent packet loss in TCP connection to HTTPS web site. In some cases (I assume, when TCP was in slow-start phase after retransmissions) these lost packets led to state when web server didn't send any more data for several seconds (after server continued sending data, a packet or two later TCP fast retransmissions were triggered and normal TCP operation resumed). After some more investigation it turned out that some ISPs configure traffic shaping with very small queues and burst limits that can be exceeded even in some moments of web page loading (by traffic created by web page download and other internet usage at those moments) causing packet loss in TCP connections. I wasn't provided with packed dump from Windows computer, so I couldn't investigate why that delay didn't happen on Windows computers.
The most notable difference between TCP connections initiated by Mac and Windows computers was TCP timestamps (MacOS by default sends them, while Windows doesn't). When TCP timestamps (as well as TCP window scaling) were temporary (until reboot) turned off in Mac computer by invoking:
sudo sysctl -w net.inet.tcp.rfc1323=0
HTTPS web pages opened in it as fast as in Windows computers. Afterwards the changes were made permanent by invoking:
sudo sh -c 'echo "net.inet.tcp.rfc1323=0" >> /etc/sysctl.conf'

Problems with old PSAPI.DLL and Internet Explorer 7 on Windows XP

2009-10-24T19:42:00.006+02:00

Recently I faced a computer with Windows XP Service Pack 3 and Internet Explorer 8 Beta. Internet Explorer 8 Beta didn't work properly with computer's owner favorite social portal site, so we decided to downgrade IE.
After uninstall of IE 8 we had some minor problems with IE 6, so I decided to install Internet Explorer 7. Real problems started with IE 7. I wasn't able to launch it normally - double clicking desktop icon created a shortcut to Internet Explorer on desktop. Somehow default action (Open Home Page) and also Start Without Add-ons were missing from IE desktop icon and the first action has become Create Shortcut. After typing HTTP URL in Windows Explorer address bar it showed error The specified procedure could not be found. and applications trying to initialize IE components displayed error message with Entry Point Not Found in title and text The procedure entry point GetProcessImageFileNameW could not be located in the dynamic link library PSAPI.DLL.
Cause and resolution of this problem were quite simple - there was an older version of PSAPI.DLL in %SystemRoot%\system32 folder. Copying this file from another Windows XP SP3 machine resolved the problem (this file also could be extracted from Service Pack 3 installation or Windows XP SP3 install CD). The file PSAPI.DLL is opened by different Windows components and some other software, so the old file can't just be deleted or overwritten - it has to be renamed. If PSAPI.DLL is extracted from Windows installation CD or Service Pack setup, it must be extracted from psapi.dl_ by invoking expand psapi.dl_ %SystemRoot%\system32\psapi.dll in Command Prompt.

Problems installing Apache HTTP Server

2009-03-10T20:45:00.006+02:00

A few days ago I wanted to install Apache web server on a computer running Windows XP (development system), but .msi installer of current version (2.2.11) didn't initialize properly - after some seconds message consisting of Installation Wizard Interrupted and The Installation Wizard was interrupted before Apache HTTP Server 2.2.11 could be completely installed. appeared on installation wizard and the only enabled button was Finish.
I created Windows Installer log file by invoking MSIEXEC /I apache_2.2.11-win32-x86-openssl-0.9.8i.msi /L*v LogFileName.txt. In log file I immediately noticed these two lines:

Action ended 16:50:09: ResolveServerName. Return value 3.
MSI (c) (D0:18) [16:50:09:437]: Doing action: SetupCompleteError

I tried to enumerate areas where this particular computer was different from an average workstation/server and tried to find out what exactly could cause such error. I paid special attention to network configuration, so one of first things that came into my mind was seven network interfaces (some physical, some virtual) of this workstation. About five network interfaces were online (Connected), so I disabled (simply right clicked on these network interfaces in Control Panel - Network Connections and selected Disable) some of them, leaving only two NICs online. This solved my problem - Apache web server installation wizard initialized properly and I had no further problems installing Apache HTTPD. After installation I, of course, re-enabled those network interfaces.

Finding broken video card cooler by analyzing crash dump

2008-11-03T20:16:00.002+02:00

A few days ago one of my workmates had strange problems with his workstation, including several Blue Screens per day.
I opened two crash dumps with Debugging Tools for Windows and both had video card driver on the stack. After telling this to my workmate he correctly suspected that video card cooler has failed. So we fixed his computer by replacing video card.
With this post I wanted to accent that also hardware problems can be noticed by looking at crash dumps. For those who yet don't know how simple it is to open a crash dump I can recommend this article.

Problems with Visual Studio 2008 database projects

2008-10-05T19:09:00.004+02:00

Recently I faced a situation where a software developer using Visual Studio 2008 wasn't able to do virtually anything in database projects.
It was possible to create a new database project, but on many operations, including Import Database Schema and New Schema Comparison an error message stating Object reference not set to an instance of an object was shown.
I was told that reinstall of Visual Studio doesn't help. Also System Restore made situation only worse.
Accidentally it was discovered that this is a per-user problem (only developers account was affected). After some research I discovered that most likely some of Visual Studio components have had some problems previously so they were skipped from loading.
So a simple "%ProgramFiles%\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe" /ResetSkipPkgs in Command Prompt of developers account solved this problem.
Most likely this method (resetting skipped packages) can also solve some other "cryptic" per-user problems of Visual Studio 2008.

Security of Windows (PPTP based) VPNs

2008-02-02T14:35:00.000+02:00

Recently I have faced two interesting articles about security of PPTP (most common VPN type natively supported by Windows). PPTP requires virtually no configuration on client side (only server name/IP address, username and password are required to connect), so it is easy to set up, deploy and use. I have always had several questions about security of PPTP and they are answered here:

Removing Direct3D updates

2007-10-21T15:28:00.000+02:00

A few days ago I installed Direct3D updates for DirectX 9.0c (required by a game) on one laptop computer with Windows XP SP2.
I was pretty scared, because after installing these updates any fullscreen Direct3D application (e.g. most games) crashed notebook's video driver in a few seconds (sometimes with a bluescreen, but more often with Windows - Display Driver Stopped Responding error message). As far as I know Microsoft doesn't provide users with any opportunity to uninstall DirectX (and it's updates). There are some third party DirectX removal tools, but I wasn't willing to risk to use them, so I made an experiment on virtual machine.
It was a nice surprise, that Direct3D updates (at least those I had installed) don't replace any files, neither they create any noticeable registry entries. So I simply deleted Direct3D update files from the laptop by invoking ren %SystemRoot%\system32\d3dx9_*.dll *.dll.bak in Command Prompt. After deleting these files Direct3D started again to work normally.
I assume that similar methods may be used to remove updates for other DirectX components as well.

Strange memory/registry problems on Terminal Server

2007-08-25T20:30:00.000+03:00

Recently I was asked to help troubleshoot a terminal server that sometimes couldn't load registry hives of users logging on. The server was running Windows 2000 Server and there were lots of users connecting to it.
When hive for Administrator account wasn't loaded these three errors were logged to event log:

RegLoadKey failed. Return value Insufficient system resources exist to complete the requested service. for C:\Documents and Settings\Administrator\ntuser.dat.

Windows cannot load your profile. Please increase the registry size and restart the computer.

Windows cannot load the user's profile but has logged you on with the default profile for the system.
DETAIL - Insufficient system resources exist to complete the requested service.

Also some virtual memory related errors were logged. Partially working solution (increasing registry size limit using System Properties control panel applet and increasing PagedPoolSize registry value) was found in some forum before I started my research.
At first I looked at pool usage (Kernel Memory in Performance tab of Task Manager) and value for paged pool was pretty big (over 150 MB). Next I enabled pool tagging and used Poolmon.exe to see what consumes most of paged pool. Name of the tag that used about 128 MB of paged pool was CM (referred as Configuration Manager (registry) in pooltag.txt), so it probably was actually related to registry. Examining size of main registry hive files (located at %SystemRoot%\system32\config) revealed that hive named default (this hive stores HKEY_USERS\.DEFAULT registry key) was about 100 MB in size, so I tried to use NTREGOPT to compact that hive.
Hive was compacted to about 85 MB that is far beyond its size on average systems. While examining contents of that hive (exported as usual .reg file) I noticed that most of .reg file is consumed by HKEY_USERS\.DEFAULT\Printers\DevModes2 key. This key was trashed by lots of printers connected via terminal services clients. I added a Scheduled Task that invokes regedit /s cleanprinters.reg every night (when no users should be connected) and created cleanprinters.reg with the following contents:

REGEDIT4

[-HKEY_USERS\.DEFAULT\Printers\DevModes2]
[HKEY_USERS\.DEFAULT\Printers\DevModes2]

After running cleanup task the first time and once more compacting registry with NTREGOPT hive named default and used paged pool sizes decreased almost to normal.

P.S. After all troubleshooting I found that this problem is also described in MS KB906952, but anyway I like my own solution.
P.S.2 There were also trashed some other (printer dependant) registry keys (under HKEY_USERS\.DEFAULT\Software and per-user HKEY_USERS\SID-of-user\Software) increasing registry size for about 14 MB for default user and 7 MB for each regular user.

Manual Folder Redirection

2007-05-21T17:47:00.000+03:00

Some days ago I faced folder redirection problems for some users who's My Documents folders were redirected to share where they had Change (not Full Control) access (additionally restrictive NTFS permissions were used). These users were also configured to use roaming profiles.
The problem was that My Documents folder wasn't redirected for some (not all) users, instead it pointed to local copy of My Documents folder from roaming profile (in other words My Documents path for those users was "%userprofile%\My Documents" instead of "\\file-server\users\%username%\My Documents").
Also some .tmp files were created in "\\file-server\users\%username%\My Documents" and an error similar to this was logged in event log:

Failed to perform redirection of folder My Documents. The files for the redirected folder could not be moved to the new location. The folder is configured to be redirected to <\\file-server\users\%username%\My Documents>. Files were being moved from <C:\Documents and Settings\Davis\My Documents> to <\\file-server\users\Davis\My Documents>. The following error occurred while copying <C:\Documents and Settings\Davis\My Documents\My Music> to <\\file-server\users\Davis\My Documents\My Music>:
The security descriptor structure is invalid.

Although this was not tested, I suppose that folder redirection needs Full Control access to redirection destination to copy files. I didn't want to give users ability to change access permissions on their folders, so I didn't consider Full Control access as a good solution. Also just setting registry paths via logon script didn't seem to be a good idea.
After some search on internet I didn't found any suitable solution (this is the main reason why I have published this).
The main idea is that folder redirection will work fine if there will be no folders/files to move. So I simply wrote a script that moved all My Documents folders from roaming profiles to folder redirection targets (actually that script was a bit different but this doesn't matter).
Basically the solution is to manually (with help of a script) move folders that should be redirected to their redirection targets. When there will be no folders to move, folder redirection will simply change registry paths instead of complaining about permissions.
Moving folders is very simple with roaming profiles and I think it can be easily implemented in logon scripts for automated processing of local profiles as well.

Auditing object access to trace malware

2007-04-17T07:20:00.000+02:00

Windows has built-in object access auditing that can be used to record changes on NTFS volumes.
Setting auditing of successful file creations in folders most commonly affected by malware, can
facilitate removal of such software.

I think good settings would be auditing of successful file creations in %SystemRoot%, %SystemRoot%\system32 including subfolders and %ProgramFiles% including subfolders.
Auditing can be configured in folders Properties - Advanced Security Settings - Auditing tab.
Additionally Audit object access must be enabled in Security Policy of computer.
With such settings all new file creations in abovementioned folders will be logged to Security event log. Also size and retention of security event log should be adjusted to prevent repletion.

Icon editor

2007-02-21T23:08:00.000+02:00

As a software developer I have always wanted a good and free icon editing software.
Recently I have found one - IcoFX. It has all major features I need for icon creation and it is freeware.
http://icofx.xhost.ro/ - IcoFX website

UTF-8 search with MySQL and PHP

2007-01-29T00:24:00.000+02:00

A few minutes ago I helped a friend of mine to troubleshoot Unicode search in his web page. He was using a TEXT type field with utf8 encoding and search wasn't working properly. Also encoding of web page was UTF-8.
After a little research connection encoding turned out to be latin1. So a simple mysql_query("SET NAMES utf8"); before actual database queries solved his problem.