I frequently receive requests to automate the retrieval of user IDs for a list of arbitrary names, such as from Active Directory. The name list largely consists of last names followed by first names, but some of the names are reversed. Some of the names are nicknames or initials.

In our example case, the names in Active Directory aren’t properly normalized either. Merging AD Domains from multiple, older networks often results in non-normalized data (or more specifically, differently normalized data). Names may be entered as proper names or nicknames, and nicknames may be notated in ad-hoc formats (e.g., in parentheses, etc). Names may be initialed—especially middle names—and abbreviations may or may not have periods. Last names may have suffixes (Jr, III, etc.) in a variety of formats.

Because the names in AD are not normalized, plus the arbitrary state of the names list we have to match, there is no way that I know of for a script to reliably normalize the list or the database, let alone compare them both. We could attempt to build-in dictionaries of names and compare them, but we would still fail on corner cases.

For example, if I tell you a full name is some arrangement of ‘Ernie’, ‘Ford’, and ‘Tennessee’, how can you guarantee what order the partial names are? While you may be able to make an educated guess, how do you write a script that guesses correctly? What about ‘John Olivia Newton’? Even if you built a dictionary of English names, what if I gave you ‘Ali Muhammad’ with no specified order?

No matter how we attempt to automate matching these names to the data, there will still have to be a manual reckoning. The goal for the script is to make that manual process as simple and automatic as possible so that it can be done quickly, efficiently, and—most importantly—by somebody else. The goal isn’t to get 100% accuracy—80% would be nice with some lists. On the other hand, manually resolving 20% of a list of thousands of names is still a big job, but the 80% saving is huge.

If you have such a file, all you actually have to do is drop it onto this script, and the script will create an output file in the same folder. You can also pass the file name as a command-line parameter. The script doesn’t take extra parameters—the output file will be the input filename with ‘.out.tab’ tacked onto it, etc.

Name dumps typically mix up first and last names, so this script tries to match names in both directions. The exception to that is the simplest translation. If ‘Smith, Andrew’ is in the file, and matches to ‘Smith, Andrew’ in AD, there is a strong indication (the comma) that the names are in the same order. That’s considered a quick match and warrants no further resolution.

If the simple translation fails, the script begins to run queries against AD, successively truncating the names (attempting to remove anything like initials, suffixes, etc.), and using them as wildcards until something is found, and eventually trying pieces of the names if necessary. The output includes profile information, as available, to help identify users.

Caveats

• Quick matches run through a fast translation (the AD NameTranslate object built into XP). There is some room for error; if the names have duplicates, for example (note Gene Cox in the list below). If this assumption of accuracy doesn’t work for your situation, you can disable that function (set it to just return an empty string).
• There is no realistic way for a script to know that ‘Bob Smith’ could be listed as ‘Robert Smith’, so it keeps broadening the net until something becomes plausible. If there is already a ‘B. Smith’ of some other name, broader scans will not run, so Robert may never show in the list.
• Some queries (such as Bob Brown or Will Smith) may return too many names to be useful. There is a configurable threshold for that eventuality.

You will notice that the output file is a .tab (which is probably an unrecognized type on your system) and that it looks oddly formatted. The file is designed so that you can run through the output fairly rapidly in a text editor and delete extraneous names, leaving the correct user IDs. Open the remainder in Excel (just right-click the file, and open it with Excel, in most cases), and all of the user IDs are in the third column—so even though editing is a manual process, it’s still relatively quick and easy.

Attached is a sample of the output, with mostly trouble names included.

Wesley Walker	=	OurCorp\WWalker
Betn Gotham	=	OurCorp\BGotham
MacLarsen, William C. (Carlson) = ...
      {searching for 'MacLarsen' & 'William C. (Carlson)'}
      Carlson MacLarsen (Carlson MacLarsen Project Office) (MAC Services - Office) =	CMacLars
Ferrous, B D = ...
      {searching for 'Ferrous' & 'B D'}
      Ferrous, Brandon (Desktop Ops) =	BFerrou
      Ferrous, BD (Corporate Services) =	BDFerro
O'Malley, Katherine = ...
      {searching for 'O''Malley' & 'Katherine'}
      Katherine O'Malley (Company User Administration) (Company Administration) =	KOMalle
      Katherine O'Malley (Network & Server) =	KOMalle
Thistle, Bob = ...
      {searching for 'Thistle' & 'Bob'}
      did not find 'B* Thistle' or 'B* Bob'
      Broadening Search...
      {searching for 'Thistle'}
      Robert Thistle (Network & Server Group) =	RThistle
      Thistle, Alex (Michael Robert) =	AThistle
      {searching for 'Bob'}
      did not find '* Bob'
White, Clarence = ...
      {searching for 'White' & 'Clarence'}
      Found 57 of 'C* White' or 'B* Clarence'. 30 is too many.
Garofolo, Gerald = ...
      {searching for 'Garofolo' & 'Gerald'}
      Jerry Garofolo (Project Coordinator) (Business Support) =	JGarofol
Gene Cox      =     DOMAIN\ADMIN
Priestley Jr, Carl = ...
      {searching for 'Priestley Jr' & 'Carl'}
      did not find 'C* Priestley Jr' or 'C* Carl'
      Broadening Search...
      {searching for 'Priestley Jr'}
      did not find '* Priestley Jr'
      Trying just the first part (Priestley)
      {searching for 'Priestley*'}
      Nick Priestley =	NPriestle
      Priestley, Carl (Legacy Support)  =	CPriestle
      {searching for 'Carl'}
      Kelly Carl (HR) =	KCarl
      John Carl (HR) =	JCarl

Below is the script:

Option Explicit

Const LDAPOU = "dc=domain,dc=Company,dc=NET"
Const TOOMANYCHOICES=30

Const ForReading = 1
Const ForWriting = 2

dim objArgs
Set objArgs = WScript.Arguments
If objArgs.count < 1 Then
	wscript.echo "Usage: userstocomputers.vbs input.csv"
	wscript.quit
End If

Dim strInputFileName, strOutputFileName

strInputFileName = objArgs(0)
strOutputFileName=strInputFileName & ".tab"

Dim objFSO, objInputFile, objOutputFile, szScratch, szScratch1
Set objFSO = CREATEOBJECT("Scripting.FileSystemObject")
Set objInputFile = objFSO.OpenTextFile(strInputFileName,ForReading,false)
Set objOutputFile = objFSO.OpenTextFile(strOutputFileName,ForWriting,true)

Dim strLineText, objSystemSet, objSystem
While not objInputFile.AtEndOfStream

	strLineText = split(Trim(objInputFile.Readline), ",")
	if ubound(strLineText) > 0 then
		strLineText(0)=trim(strLineText(0))
		strLineText(1)=trim(strLineText(1))
		szScratch= strLineText(1) & " " & strLineText(0)
		szScratch1=TranslateADName(4, 3, szScratch)
		if len(szScratch1) > 0 then
			objOutputFile.WriteLine szScratch & vbtab & "=" & vbtab & szScratch1
		else
			szScratch= strLineText(0) & ", " & strLineText(1)
			szScratch1=TranslateADName(4, 3, szScratch)
			if len(szScratch1) > 0 then
				objOutputFile.WriteLine szScratch & vbtab & "=" & vbtab & szScratch1
			else
				objOutputFile.WriteLine szScratch & " = ..."

				if LookupADName( strLineText(0), strLineText(1) )=0 then
					objOutputFile.WriteLine vbtab & "Broadening Search..."
					if LookupADName(strLineText(0), "")=0 then
						if instr(strLineText(0), " ")>0 then
							objOutputFile.WriteLine vbtab & "Trying just the first part (" & left(strLineText(0),instr(strLineText(0), " ")-1) & ")"
							LookupADName left(strLineText(0),instr(strLineText(0), " ")-1)&"*", ""
						end if
					end if
					if LookupADName(strLineText(1), "")=0 then
						if instr(strLineText(1), " ")>0 then
							objOutputFile.WriteLine vbtab & "Trying just the first part (" & left(strLineText(1),instr(strLineText(1), " ")-1) & ")"
							LookupADName left(strLineText(1),instr(strLineText(1), " ")-1)&"*", ""
						end if
					end if
				end if
			end if
		end if
	elseif ubound(strLineText) = 0 then
		strLineText(0)=trim(strLineText(0))
		szScratch1=strLineText(0)  & vbtab & "=" & vbtab & TranslateADName(4, 3, strLineText(0) )
			if len(szScratch1) > 0 then
				objOutputFile.WriteLine szScratch & vbtab & "=" & vbtab & szScratch1
			else
				objOutputFile.WriteLine szScratch & " = ..."
				LookupADName strLineText(0), ""
			end if
	end if
Wend
Wscript.Echo "Done"

Function TranslateADName(FromType, ToType, FromName)
	Dim objTranslator
	Const ADS_NAME_INITTYPE_GC					= 3

	Const   ADS_NAME_TYPE_1779					= 1
	Const   ADS_NAME_TYPE_CANONICAL				= 2
	Const   ADS_NAME_TYPE_NT4					= 3
	Const   ADS_NAME_TYPE_DISPLAY					= 4
	Const   ADS_NAME_TYPE_DOMAIN_SIMPLE			= 5
	Const   ADS_NAME_TYPE_ENTERPRISE_SIMPLE			= 6
	Const   ADS_NAME_TYPE_GUID					= 7
	Const   ADS_NAME_TYPE_UNKNOWN				= 8
	Const   ADS_NAME_TYPE_USER_PRINCIPAL_NAME		= 9
	Const   ADS_NAME_TYPE_CANONICAL_EX				= 10
	Const   ADS_NAME_TYPE_SERVICE_PRINCIPAL_NAME		= 11
	Const   ADS_NAME_TYPE_SID_OR_SID_HISTORY_NAME	= 12 

	Set objTranslator = CreateObject("NameTranslate")

	on error resume next
	objTranslator.Init ADS_NAME_INITTYPE_GC, ""
	objTranslator.Set FromType, FromName

	TranslateADName = objTranslator.Get(ToType)
	on error goto 0
End Function

Function LookupADName(fname, lname) ' Returns number of records
	dim objConnection, objCommand, objRecordSet, searchscope, strItem
	On Error Resume Next
	Const ADS_SCOPE_SUBTREE = 2

	fname=trim(replace(fname, "'", "''"))
	lname=trim(replace(lname, "'", "''"))
	if len(fname)=0 and len(lname)=0 then exit function

	Set objConnection = CreateObject("ADODB.Connection")
	Set objCommand =   CreateObject("ADODB.Command")
	objConnection.Provider = "ADsDSOObject"
	objConnection.Open "Active Directory Provider"
	Set objCommand.ActiveConnection = objConnection

	objCommand.Properties("Page Size") = 1000
	objCommand.Properties("Searchscope") = ADS_SCOPE_SUBTREE
	if len(fname)=0 or len(lname)=0 then
		lname = fname & lname
		if lname = "*" then
			objOutputFile.WriteLine vbtab & "Skipping '*'"
			exit function
		end if
		objOutputFile.WriteLine vbtab & "{searching for '" & lname & "'}"
		objCommand.CommandText = "SELECT sAMAccountName, displayName, department, description FROM 'LDAP://" & LDAPOU & "' WHERE objectCategory='user' AND " & _
			"sn='" & lname & "'"
		searchscope="'* " & lname & "'"
	else
		objOutputFile.WriteLine vbtab & "{searching for '" & fname & "' & '" & lname & "'}"
		objCommand.CommandText = "SELECT sAMAccountName, displayName, department, description FROM 'LDAP://" & LDAPOU & "' WHERE objectCategory='user' AND " & _
			"( (givenName='" & left(lname,1) & "*' AND sn='" & fname & "') OR " & _
			"(givenName='" & left(fname,1) & "*' AND sn='" & lname & "') )"
		searchscope="'" & left(lname,1) & "* " & fname & "' or '" & left(fname,1) & "* " & lname & "'"
	end if
	Set objRecordSet = objCommand.Execute
	objRecordSet.MoveFirst
	LookupADName = objRecordSet.RecordCount
	if objRecordSet.RecordCount>=TOOMANYCHOICES then
		objOutputFile.WriteLine vbtab & "Found " & objRecordSet.RecordCount & " of " & searchscope & ". " & TOOMANYCHOICES & " is too many."
	elseif objRecordSet.RecordCount=0 then
		objOutputFile.WriteLine vbtab & "did not find " & searchscope
	else
		Do Until objRecordSet.EOF
			objOutputFile.Write vbtab & objRecordSet.Fields("displayName").Value
			if not isnull(objRecordSet.Fields("description").Value) then
				For each strItem in objRecordSet.Fields("description").Value
					if len (strItem)>0 then objOutputFile.Write  " (" & strItem & ")"
				Next
			end if
			if len(objRecordSet.Fields("department").Value)>0 then objOutputFile.Write  " (" & objRecordSet.Fields("department").Value & ")"
			objOutputFile.WriteLine  " ="& vbtab & objRecordSet.Fields("sAMAccountName").Value
			objRecordSet.MoveNext
		Loop
	end if
	on error goto 0
End Function

1. We receive surveys/orders/documents that have been filled out by various people as Excel spreadsheets
2. We need to put the information into a database
3. We want you to write a script to automate the imports.

My thoughts as a scripter:

1. This is an excellent way to keep a contractor in a job for a long time.
2. You will never get everybody to input the data the same way into Excel. Nobody ever has.
   • People will put things on different lines, because it is Excel, and they can.
   • People will never enter data the same way. If you give them a yes/no column in a spreadsheet, some time you will get “yes/no”, sometimes “y/n”, “t,f”, “X, ”, “☑,☒”, “☺,☹”, “+,-”. Every time somebody creates a new item, the script breaks, and has to be modified to compensate, or you have to send the form back or correct it. With a web form, you just give them a yes/no box, or a check box, and they can’t put in different values (it’s self-correcting).
   • Whenever the user changes the values, if you haven’t predicted what they will enter, you will have to manually redo those files, because they will break the script. This always happens. Then you have to go back and fix the script.
3. If you want to run the script on a server, you either have to work out how to dissect XLS files, or install Excel on the server. The first is a royal pain, and the second shouldn’t be allowed to happen.

The best way to deal with this is to avoid it altogether:

1. Get your database in a SQL database back end (and don’t confuse Excel with a database.)
2. Set up a web form that looks like the spreadsheet, to do data input straight into the database
3. Set up list boxes, etc, so that the data is picked, and not created on the fly.
4. Instead of filling out the spreadsheet, users just fill out a web page.
5. If you really want, you can send out Access-via-Outlook forms to anybody with Outlook. They are like Access forms, except they get emailed, and they send the data straight into a database.

Miscellaneous advantages to web form:

1. It takes almost all of the work out
2. You don’t have to correct data for people. If there’s a problem, it can let them know as they enter it, and they can correct it on the spot.
3. You don’t have to worry about different versions of Excel and interoperability.
4. You don’t even have to worry if people have Excel.
5. People can fill in a web form from any machine–PC, Macintosh, Blackberry.
6. If you change things, the web form changes along with the back end, and anybody going to the form automatically gets the new version–with Excel, you get back what you sent out any time ago.

Doing this as a script will cost you a lot of time and effort (which probably translates into money), and will never work right (which will cost you a lot more money). Doing this as a web form will be relatively easy (and cheap), and the data will be self-correcting. If you want to do Access-via-Outlook forms, you will have plenty of money left over, in case you need to hire somebody to create that (it shouldn’t be hard, nor take long to make.) Besides, starting with a website, and just having an input form added to it is barely more work than the website by itself–and whoever builds the website will probably build the form as they build the site, anyway, just to do test data (I always do, when I build databases.)

If you still insist on doing this as a script, do it in Perl. Perl has modules to parse XLS files (so it doesn’t need Excel) and is built to parse native language text–which it will have to do for anything people can type in wrong–and can compile into an EXE, if you need to run it on a server without installing a new language. If you don’t need to run it on a server, Perl is still the best language to do it in. It’d still be an expensive and troublesome pain, but not as much as other scripting languages.

This kind of project is a knock-off for a web-developer. It’s a career for a scripter.

This thread script:

1. takes a given PC name, pings it and reverse-resolves it.
2. logs it as responsive or unresponsive (if unresponsive, it quits)
3. maps a drive to the PC, with the same letter as the thread
4. runs arbitrary code against the pc
5. unwinds
6. deletes itself

The drive mapping can take a list of username/password combinations, starting with the user who runs it.

@ECHO OFF
(set ip=%3)
(set pcname=%3)
(set drive=%2)
(set num=00000%1)
(set logfile=logs\%num%.txt)
(set outputname=thread)
(Set outputfile=.\log.csv)

:::::::::::::::::::::::::::::::::::::::::::::::
:: 1) Set variables above. This file called as;
:: newletter.bat    count newletter LineFromCSV
:: parameters are     %1      %2      %3,%4...
::
:: 2) Write a batch file that scans what you
:: need locally, and uses the variables above.
::
:: 3) Update the ":::Variable Part:::" section
:: with your batch file.
:::::::::::::::::::::::::::::::::::::::::::::::

ECHO. %*

set num=%num:~-5%
(SET pingnum=1)
set pingtarget=%pcname%
:PINGSTAT
@for /F "delims=" %%A in ('ping -n %pingnum% -a %pingtarget%') do (
 call :log xxxLx "%%A"
 @for /F "tokens=1,3,11" %%B in ("%%A") do (
  @IF "%%B"=="Pinging" (SET resolvedip=%%C)
  @IF "%%B"=="Packets:" (SET percentloss=%%D)
))
if DEFINED percentloss (SET percentloss=%percentloss:~1,-1%)
if DEFINED resolvedip (SET ip=%resolvedip:~1,-1%)

IF NOT "%percentloss%"=="100" GOTO :RESPONSIVE
:: quarter ping failed. Check for partial connectivity.
:: if full ping failed, exit.
call :log xxxLx "%pcname% Failed ping [*%pingnum%]"
@if "%pingnum%"=="1" (
 (SET pingnum=4)
 call :log xxxLx "retrying with 4 pings . . ."
 GOTO :PINGSTAT
)
call :log xxULC "%num%,%pcname%,%ip%,Unavailable. Lost %percentloss% percent of %pingnum% pings"
GOTO :END

:Responsive
call :log xxxLx "%percentloss% percent Loss"
IF ""=="%percentloss%" (
call :log xxULC "%num%,%pcname%,%ip%,Not Found"
GOTO :END
)

:: At this point, the machine is online and available

::::::::::::: connect drive :::::::::::::
if NOT exist %drive%:\nul GOTO ADDDRIVE
:DRIVEEXISTS
call :log xxxLx "%drive% exists . . ."
@for /F "delims=" %%A in ('net use %drive%: /delete /y') do (call :log xxxLx "%%A")
if exist %drive%:\nul (
 ping -n 3 localhost >null
 goto :DRIVEEXISTS
)
call :log xxxLx "%drive%:\nul doesn't exist"

:ADDDRIVE
call :log xxxLx "attaching \\%pcname%\c$ to %drive%:"

FOR %%A in ("%USERID% %PASSWORD%","%pcname%\Administrator Password") do (
  For /F "tokens=1,2" %%B in (%%A) do (
  call :connect %%B %%C
))
IF "%ERR%"=="0" goto :MAIN
call :log xxxLC "%num%,%pcname%,%IP%,Can't Connect"
Goto :End
::::::::::::: /connect drive :::::::::::::

:MAIN
call :log xxxLx "Entering Main . . . "
del netuse%drive%.txt -y

:::::::::::::::::: Variable part ::::::::::::::::::

:: Log success
call :log SxxLC "%num%,%pcname%,%ip%

:::::::::::::::::: /Variable part ::::::::::::::::::
call :log xxxLx "Disconnecting . . . "
net use %drive%: /delete /y

GOTO :END

:log
:: SFUL (Success Failure Unavailable LogDetails CommonOutput) "Message"
(set log=%1)
echo %~2
if "%log:~0,1%"=="S" ECHO.%~2>>%outputname%.Updated.log
if "%log:~1,1%"=="F" ECHO.%~2>>%outputname%.Noupdate.log
if "%log:~2,1%"=="U" ECHO.%~2>>%outputname%.Unavail.log
if "%log:~3,1%"=="L" if defined logfile ECHO.%~2>>%logfile%
if "%log:~4,1%"=="C" if defined outputfile ECHO.%~2>>%outputfile%
Goto :EOF

:connect
if EXIST %drive%:\nul Goto :EOF
call Set USER=%1
call Set PWD=%2
call :log xxxLx "connecting as %USER%"
if "%USER%"=="" (
 net use %drive%: \\%pcname%\c$ > netuse%drive%.txt 2>&1
) ELSE (
 net use %drive%: \\%pcname%\c$ /USER:%USER% %PWD% > netuse%drive%.txt 2>&1
)
(set ERR=%ERRORLEVEL%)
call :log xxxLx "netuse returns '%ERR%'"
type netuse%drive%.txt
if defined logfile type netuse%drive%.txt>>%logfile%
IF "%ERR%"=="0" (
 call :log xxxLx "Connected as %USER%."
 del netuse%drive%.txt -y
 goto :EOF
)
FOR /F "tokens=3 skip=1" %%p IN ('Find /I "System Error" netuse%drive%.txt') DO (SET NETUSEERR=%%p)
call :log xxxLx "NETUSEERR = %NETUSEERR%"
FOR /F "delims==" %%p IN ('net helpmsg %NETUSEERR%') DO (
SET NETUSEDESC=%%p
)
(SET NETUSEERR=[%NETUSEERR%] %NETUSEDESC:,=%)
del netuse%drive%.txt -y
call :log xxULx "%num%,%pcname%,%ip%,Doesn't Connect as %USER%. '%NETUSEERR%'"
Goto :EOF

:END
call :log xxxLx "Done."
del %0

download thread2.bat

This code does nothing to the target machines; it assumes that if it’s made it as far as a mapped drive, it’s finished. You can actually run just about anything against the machines: write a batch file to do *stuff* and use the variables listed in the block at the beginning of this script. When it works, plug it into this script, in the section marked

:::::::::::::::::: Variable part ::::::::::::::::::

if you need to execute a process on the target machines, you can do so with psexec or beyondexec. You can with rclient, also, but it’s turning into a security liability, at this age.

What makes multithreading worthwhile is lag and the fact that “all computers wait at the same speed”. Without some sort of wait, threading can actually slow a system down due to overhead. Think of it like this: the pony express moved mail at the maximum speed a horse could travel. If they had implemented it with two or more horses, running parallel, the mail wouldn’t have gotten there any faster. two horses (ignoring load, so carriages don’t enter into this) don’t run faster than one horse. On the other hand, if running parallel horses doesn’t increase your speed, but you add the overhead of having to split the mailbags (ie, take the time to count or weigh the mail, so that the horses carry the same amount) then you are running at a net loss.

The pony express, however, was a system that was running at near maximum efficiency from our point of view. Adding more horses into the mix could increase complexity and overhead, but not speed up the system. It’s a different story for pizza delivery boys, or jugglers. Jugglers spend most of their time waiting for the ball to drop–literally. They throw a ball into the air, and it flies up, stops, falls back down. It isn’t until the ball comes back down that the juggler has any more input into the process. A juggler can therefore achieve higher efficiency by launching multiple asynchonous balls. They’re not parallel, per se, but they are fired off one after another, and function on their own. In this case, the balls can slightly change the ‘running’ of each other. To increase the numbers of balls in the air, it becomes necessary to throw them higher, requiring that their ‘runtime’ be slightly longer. In general, though, you can keep adding balls until you are running your juggler at maximum speed before you have to lengthen the ‘run’ of the balls. Once your juggler is maxed out, it will affect the speed of the system, but you’ve still drastically increased the amount of stuff that’s going on by utilizing the juggler’s idle time.

Say a busy pizza company guarantees delivery within x minutes. In this case, using one delivery boy could get very expensive–any delivery x/2 minutes away takes up the delivery boy’s whole transit time (x/2 minutes each way). Any order placed while the delivery boy is out will be delayed while he is out, and could easily be delayed beyond the time limit. The company could try to maximize their efficiency by organizing deliveries together on a trip, and hiring the fastest possible driver to minimize the transit times–but the customer is a wildcard. Nobody knows how long it’s going to take a customer to check their order, look for their checkbook, find out checks aren’t accepted, look for cash, and work out a tip–and all pizza boys wait at the same speed. What they need to do is hire several delivery boys, have each one take one existing order and leave (in parallel) and as soon as they are back, take the next order and run (asynchronously). In the pizza business, they probably want to grab the pizza that needs to be delivered furthest, but they could always settle on the first-come, first-served model.

Multithreading requires a process to run, and a control system to keep it from running amok. Below we have the control system for a simple multithread emulator. It takes a pool of necessary tasks and keeps a list of available threads that it can use to process those tasks. Our model is most like the pizza company. We have arbitrary length processes, and are going to fire off as many as we can at once, and as each finishes, it grabs the next and runs. We don’t prioritize the machines, so they’ll just be first-come, first-serve.

Our ‘thread’ process will need do several things, in order to function as an efficient system.

1. it will quarter-ping the target, to make sure it is online
2. if the quarter ping succeeds, excellent, but on failure the thread will perform a full ping, to account for network connectivity issues
3. if the machine is found online, the thread will map a drive to that machine’s default admin shares
4. the tread will perform an arbitrary process against the target

The first three actions all require waiting for the network or the target machine to respond. While waiting, the process will use around 0% of the CPU. This makes it a candidate for threading. If processing a single target machine took 100% of the CPU, then threading would take more resources than it returned. In our case, we can take advantage of unused cycles.

• We’re going to emulate the task pool with a list of machines that need the same task run on each of them.
• We’ll also emulate the available threads with a list of available network drives. In a full application, we’ll be mapping a drive to each of the PCs, so using the available drives gives us both a control (we only run as many threads as we have drives, so we can change the number of threads we wish to run) and a mechanism to identify what is connected where. This has the side-effect of making this file specific to a computer. To make it universal, we would want to have it generate it’s own list based on what drives are not mapped in the system.

@ECHO OFF
(set drivestring=J K L)&(set childtemplate=thread.bat)&(set inputfile=computers.csv)
Set outputfile=.\log.csv
Echo.number,machine,ip> %outputfile%
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Set variables above.
:: Make sure drivestrings reflect executing machine's available drives.
:: If common outputfile is used, make sure it matches in %childtemplate%
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

set /P USERID=Enter your domain\userid:
set /P PASSWORD=Enter your password:
cls

@ECHO OFF
MD logs
set strlen=0
for %%A in (%drivestring%) do SET /A strlen+=1

SET count=0

FOR /F "delims=" %%A IN (%inputfile%) DO (
 call SET /A count+=1
 echo.
 call set comline=%%A
 call :processmachine %%A
)
pause
goto :EOF

:processmachine
  echo Processing #%count%: %comline%
  set newletter=
  @for %%D in (%drivestring%) do (
   @if NOT exist %%D.bat (
    @if NOT exist %%D:\nul (call set newletter=%%D)
    @if exist %%D:\nul Echo %%D: already connected. Skipping.
   )
  )
  if NOT DEFINED newletter (
   nul
   goto processmachine
  ) ELSE (
   @echo new letter = '%newletter%'
   type %childtemplate% > %newletter%.bat
   START "%count% -- %newletter%.bat" %comspec% /c newletter%.bat %count% %newletter% %comline%
 )

download master.bat
The master takes the list of available thread slots, and for each one, it makes a copy of the thread template named x.bat (where x=slot name), and launches that bat file. When it runs out of slots, it sleeps briefly, and looks through the list, to see if any of the bat files are missing–thereby using the bat file itself as a crude lock. As long as the file is running, and the file is there, that process ‘owns’ that slot (and the corresponding drive mapping). The last thing a thread needs to do before it exits, is to delete itself.

This is weak as threaded systems go; it relies on the individual threads to clean up after themselves. If a thread batch dies, it leaves behind an orphaned lock file, and that thread doesn’t get reused–resulting in a resource leak. A better alternative would be to check the running process list for each thread-bat rather than relying on the existence of the lock.

All of this should fall well short of the mark for qualifying as a true ‘multithreaded’ system; for the purpose of this discussion, I’m using the term ‘multithreading’ loosely, vaguely, and expansively. I believe each command prompt executes in its’ own processor thread (although that would be worth looking into); but that shouldn’t be enough to classify it as true multithreading. “Asynchronously executing multiple simultaneous batch processes” is probably a better description. This is, however, sufficient for quickly assembling network compliance scanners, application fixes, and tools of that nature.

The master runs against a list of PCs; for each machine, launching a thread, and passing it some pertinent information. To that end, we need a list. The master.bat is set up to read from a CSV file named computers.csv. For our purposes, all it needs to be is a list of PC Names, one per row:

PCName1
PCName2
PCName3
PCName4
PCName5

download computers.csv
Below, you will find a simple thread template. This one is really just an illustration, so it’s short; in part two, we will look at a significantly more intricate version.

@echo off
(set ip=%3)
(set pcname=%3)
(set drive=%2)
(set num=00000%1)

Title Thread %~n0 handling %PCNAME%

<nul (set/p z=Running)
echo.

    for /l %%A in (%1,1,10) do (
        <nul (set/p z=%%A )
        >nul ping 127.0.0.1 -n 2
    )
echo.

Del %0

download thread.bat
This bat simply puts up a progress bar, then exits. Each bat runs for one second less than the previous one, beginning with 10 seconds. You can run the bat, if you like–it makes no changes on the machine–but realize that the last thing it does is delete itself. (Nothing will get you in the habit of backing up your work faster than working on self-deleting scripts.)

If you copy these three files down to a common directory and run them, it should all work. They’re set up to run five machines on three available threads, so you can get a good idea of how it works. When run, the master.bat will prompt for a username/password combination. This isn’t actually used in this iteration (it’s in there for Multithreading in Batch Script, Part 2: Running Code). You can just enter through the prompts (Multithreading in Batch Script, Part 2: Running Code is written so that if you enter through the prompts, it will use your current credentials.)

Recursion is a simple concept once it gets into your head, and there are any number of treatises about it on the web–including Recursion in Batch, Part 1: An Example–so go read that if you aren’t comfortable with it.

To create a tree routine without recursion, you’d have to create large dynamic arrays, or something worse–I’ve had to do that in Wise (which has no array functionality) by appending folderspecs to a string, and keeping an eye on the length of the string, so I can shove overflow into another . . . it ain’t worth it.

@ECHO OFF
IF [%1]==[/?] (
 Echo Graphically displays the folder structure of a drive or path.
 Echo Usage: %~nx0 [/dasf] [drive:][path] [ [/dasf] [drive:][path] ...]
 Echo   /d:  show File Dates
 Echo   /a:  Show File Attributes
 Echo   /s:  Show FileSizes
 Echo   /f:  Show Files in Directories
 Echo   /t:  Use Tree.com format
 Echo   /c:  Use compact format - negates /t
 Echo.
 GOTO :EOF
)

(SET Hasfiles=) & (SET Root=)
Set Params=/
Goto :GotSwitch

:ParseParams
SHIFT
SET PARAMS=%~0
If "%PARAMS%"=="" (IF "%ROOT%"=="" (SET PARAMS=.) ELSE (GOTO :DONE))
IF "%PARAMS:~0,1%"=="/" GOTO :GOTSWITCH
IF "%PARAMS:~0,1%"=="-" GOTO :GOTSWITCH
SET Root=%PARAMS%
CALL :Root "%ROOT%"
GOTO :ParseParams
:GOTSWITCH
(SET COMPACT=) & (SET SHOWDATE=) & (SET SHOWATTRIB=) & (SET SHOWSIZES=) & (SET SHOWFILES=) & (SET TREESTYLE=)
:NEXTSWITCH
SET PARAM=%PARAMS:~0,1%
SET PARAMS=%PARAMS:~1%
IF /I [%PARAM%]==[d] SET SHOWDATE=True
IF /I [%PARAM%]==[a] SET SHOWATTRIB=True
IF /I [%PARAM%]==[s] SET SHOWSIZES=True
IF /I [%PARAM%]==[f] SET SHOWFILES=True
IF /I [%PARAM%]==[t] SET TREESTYLE=True
IF /I [%PARAM%]==[c] SET COMPACT=True
IF NOT [%PARAMS%]==[] GOTO :NEXTSWITCH
IF /I [%COMPACT%]==[True] (SET TREESTYLE=)
GOTO :ParseParams

:Done
pause
GOTO :EOF
:ROOT
ECHO.
REM ~ ECHO Showdate=%Showdate%
REM ~ ECHO SHOWATTRIB=%SHOWATTRIB%
REM ~ ECHO SHOWSIZES=%SHOWSIZES%
REM ~ ECHO SHOWFILES=%SHOWFILES%
REM ~ ECHO FileSing=%FileSing%
REM ~ ECHO FileMult=%FileMult%
REM ~ ECHO DirSing=%DirSing%
REM ~ ECHO DirMult=%DirMult%
REM ~ goto :EOF

ECHO %~f1
:WalkTree
SETLOCAL
:: Set up Remainder of switches -- ME DirSpec |[|...] FileSpec
REM ~ Echo Params: %*
IF NOT [%1]==[] PUSHD %1
SET RECURPIPE=%~2

(SET LASTDIR=) & (SET LASTFILE=)
For /D %%D in (*.*) DO SET LASTDIR=%%D
FOR %%F in (*.*) DO SET LASTFILE=%%F
REM ~ Echo LastFile=%LASTFILE%
REM ~ Echo LastDir =%LASTDIR%
REM ~ ECHO RecursivePipe %RECURPIPE% Multi %FILEMULT% single %FILESING%
IF /I [%TREESTYLE%]==[True] (
 (Set VPAD=True) & (SET PADDING=) & (SET SPC=   ) & (SET FileSing=) & (SET FileMult=) & (SET DirSing=ÀÄÄÄ) & (SET DirMult=ÃÄÄÄ) & (SET DIRPAD=®DIR¯   )
) ELSE (
 (Set VPAD=True) & (SET PADDING= ) & (SET SPC=  ) & (SET FileSing=À) & (SET FileMult=Ã) & (SET DirSing=Ô) & (SET DirMult=Æ) & (SET PipeCHR=³) & (SET DIRPAD=®DIR¯   )
)
IF /I [%COMPACT%]==[True] (Set VPAD=) & (SET PADDING=) & (SET SPC=) & (SET FileSing=À) & (SET FileMult=Ã) & (SET DirSing=Ô) & (SET DirMult=Æ) & (SET PipeCHR=³) & (SET DIRPAD=ÍÍÍÍÍ   )
IF /I NOT [%SHOWFILES%]==[True] (Set VPAD=)

IF "%TREESTYLE%"=="True" (
 IF "%LASTDIR%"=="" (
  (SET Pipe=%RECURPIPE%%FILEMULT%%SPC%) & (SET Pipend=%RECURPIPE%%FILESING%%SPC%) & (SET VPADDING=%RECURPIPE%)
 ) ELSE (
  (SET Pipe=%RECURPIPE%%PIPECHR%%SPC%) & (SET Pipend=%RECURPIPE%%PIPECHR%%SPC%) & (SET VPADDING=%RECURPIPE%%PIPECHR%)
 )
) ELSE (
 IF "%LASTDIR%"=="" (
  (SET Pipe=%RECURPIPE%%FILEMULT%) & (SET Pipend=%RECURPIPE%%FILESING%) & (SET VPADDING=%RECURPIPE%)
 ) ELSE (
  (SET Pipe=%RECURPIPE%%FILEMULT%) & (SET Pipend=%RECURPIPE%%FILEMULT%) & (SET VPADDING=%RECURPIPE%%PIPECHR%)
 )
)
IF [%3]==[] (SET Pipend=%PIPE%)

IF /I [%SHOWFILES%]==[True] FOR %%F in (*.*) DO (
 IF "%LASTFILE%"=="%%F" (CALL :PrintFile "%PIPEND%" "%%F") ELSE (CALL :PrintFile "%PIPE%" "%%F")
)

IF NOT "%LASTFILE%"=="" IF /I [%VPAD%]==[True] ECHO.%VPADDING%

(SET Pipe=%RECURPIPE%%DIRMULT%) & (Set Pipend=%RECURPIPE%%DIRSING%)
(SET Pipecarry= ) & (SET LASTFILE=)

For /D %%D in (*.*) DO if "%LASTDIR%"=="%%D" (
 CALL :PrintFile "%PIPEND%" "%%D"
 CALL :WalkTree "%%D" "%RECURPIPE%%PIPECARRY%%SPC%" "%LASTDIR%"
) ELSE (
 CALL :PrintFile "%PIPE%" "%%D"
 CALL :WalkTree "%%D" "%RECURPIPE%%PIPECHR%%SPC%" "%LASTDIR%"
)

POPD
ENDLOCAL
GOTO :EOF

:PrintFile
(SET FILESIZE=) & (Set ATTRIBS=)
(Set ATTRIBS=%~a2 )
IF [%SHOWDATE%]==[True] (Set DATTIME=%~t2 )
(Set FILESIZE=%~z2       )
SET FILESIZE=%FILESIZE:~0,7%
IF 999999 LSS %~z2 (Set FILESIZE=%~z2)
(Set FILESIZE=%FILESIZE% )
IF /I "%ATTRIBS:~0,1%"=="d" (SET FILESIZE=%DIRPAD%)
IF NOT [%SHOWSIZES%]==[True] (Set FILESIZE=)
IF NOT [%SHOWATTRIB%]==[True] (Set ATTRIBS=)
:: Files with illegal characters need to stay quoted
SET FNAME=%2
SET FNAME=%FNAME:&=%
:: Filenames with parentheses break IF blocks
IF [%FNAME%]==[%2] GOTO :DropQuotes
SET FNAME=%2
Goto :Print
:DropQuotes
SET FNAME=%~2
:Print
ECHO %~1%PADDING%%DATTIME%%ATTRIBS%%FILESIZE%%FNAME%

download filetree.bat
Originally written July 4th, 2005
Picture courtesy of The Motivator

Recursions can do their work inbound or outbound (there’s probably a term for it, but I don’t know what it is). In a similar way, if I want to know how far it is to your house, I can drive there, and clock the miles, or drive there then clock the miles on the return trip. The decision is made on details such as whether or not I know where your house is.

@ECHO OFF
IF "%1"=="" (
 ECHO Usage: ! [1-12]
 GOTO :EOF
)

:: For any given recursion, you have to have an end,
:: where it begins to return. This is the 'bottom' of
:: the recursion (similar to a do-while-loop)
IF "%1"=="0" (
 SET TOTAL=1
 Echo. %INDENT% ==========
 ECHO. %INDENT%  seed = 1
 Goto :EOF
)

:: Descending. Some recursions work on the way down,
:: some on the way up. This one only counts, on the way
:: down - partly because it's easier to seed The total
:: with 1 at that point, than try to create a mechanism to
:: store the number you enter, but not store any of the
:: other numbers on the way down. From the side, a
:: recursion is similar to dropping objects as you walk
:: through a maze, and picking them back up as you
:: retrace your steps.

:: Make this visual
Echo. %INDENT%Ú Count = %1
:: /a = do arithmetic
Set /a COUNT=%1-1

:: Build connecting lines, as we go
(SET INDENT=%~2³)

:: Recurse - the function calls itself for the next step.
CALL !.bat %COUNT% "%INDENT%"

:: Ascending. As each function call returns, it daisy-chains
:: back up the way it descended, doing the last half of
:: each function call as it goes.

:: This is the single functional line of the function.
Set /a TOTAL=%TOTAL% * %1

:: indent=Mid$(indent,2)
SET INDENT=%INDENT:~1%
:: Make it visual
Echo. %INDENT%À * %1 = %TOTAL%

download !.bat
!.bat calculates factorials. If you’ve forgotten, a factorial is a number multiplied by every positive integer smaller than itself: factorial X=1*2*…*X so factorial 3 (or 3!) = 3 * 2 * 1. This is an ideal case for recursive functions, because a) the series clearly ends, and b) for every value of x, that part of the function is x*x-1 or f(x)=x*f(x-1); so, if x=3, then 3! = (3 * 2!) where 2! = 2 * 1! and 1!=1. In other words, we don’t have to know anything more about the values other than for any x, x!=x*(x-1)!, and that 1!=1 (ie, that’s where we stop going deeper).

Run !.bat, and give it a number (ie, ! 3) . . . but 12 is it’s limit, because it’s doing 16-bit integer math. Beyond 65535 it rolls over like a pinball machine, and the math is just wrong. The script is simple, and you should get the hang of the code pretty quick. Heck, it’s only 40 lines, and most of them are comments, or are there to show the nesting as obviously as possible. In a real language, a lot of recursions are one or two lines long.

In the code, I embedded Unicode characters into the bat file. They look different in most other code pages, so you will probably see the following characters:

• Ú = ┌
• ³ = │
• À = └

As you may guess, those three characters form the basis for the tree structure.

As a sidebar; this little app has an overflow condition, as mentioned above. It does 16-bit math, so as the total grows, it approaches a limit where it is no longer valid. The function grows (logarithmically? it’s not exponentially) faster as it goes, but if we used a more linear function (say, f(x)=x+1 ) we would still have constraints we need to worry about. Every time you call a bat file, the system opens a new shell space for it, with its’ own environment full of it’s own copy of the existing environment variables. This environment is between 160 bytes and 32Kb, depending on how your system is set up, and all of the running shells run in the same 640Kb section of memory reserved for DOS. If you run too many shells, they’re going to fill up the reserved space, have no where to put their children, and die of overpopulation (insert planned parenthood reference, here.) In higher level (or lower level) languages, this is analogous to ‘bashing the stack‘. Because of this, it’s important to evaluate things like how big it’s going to get. Our batch file can’t process numbers larger than 12, so it’s constraint keeps it fairly small; however your implementation of a google-bottles-of-beer-on-the-wall probably wouldn’t get very far. In a case like that, putting your lyric into a function and calling it from a For loop would get you farther, because the function would be called one-at-a-time, instead of one-from-the-previous. In other words, each function call would end (and release its’ environment/stack space) before the next one was called.

The second app in the series is Recursion in Batch, Part 2: a tree command.

Originally written July 4th, 2005

They asked me if there’s a way to close all of the connected files in the application shares on those servers. I put together this little batch script to do just that.

@echo off
(set share=d:\share\)
REM !! IMPORTANT set length in 4th from last line !!

FOR /f "Skip=4 tokens=1,2" %%i in ('NET FILES') do (
 IF /I NOT "%%i"=="The" (
  call :checkpath %%i "%%j"
 )
)
::pause
GOTO :EOF

:checkpath
 set id=%1
 set folder=%~2\
 :: !! The last number in the next line needs to
 :: be the length of the SHARE string
 SET root=%FOLDER:~0,9%
 IF /I NOT "%ROOT%"=="%SHARE%" GOTO :EOF
 echo %ID% is open on %FOLDER%. Closing...
 NET FILE %id% /CLOSE

This was run directly from the package that pushes the installer, but it could just as easily be pushed in any other manner. You could also use it on a schedule to drop open locks. You just need to uncomment the pause if you want to see the results. (the :: is the equivalent of REM in batch, except it actually runs faster.)

control.exe timedate.cpl,,/Z Eastern Standard Time

The time zone string is exactly the same as the one listed in the timezone applet. You can see a list of them in the registry under this key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones

Each timezone listed has a pair of subkeys names Dlt and Std, which are the strings for Standard time and Daylight Savings Time in that zone.