Tech Efficiency

Bash on Windows10

2016-12-25T01:55:00.003-08:00

In other posts I talk about how how using basic bash utilities will help you be more efficient when doing most tasks at work. Here I talk about how you can set this up.

The steps below explain how to enable Bash for Windows 10. I should note that this will require admin privileges. In most Enterprise environments you won't have admin rights and this may require special access to do.

Intalling Windows Subsystem for Linux

Press the Windows Key
Type Features
Click "Turn Windows Features on or off"
Enter root password if prompted
In the dialog that appears click "Windows Subsystem for Linux (Beta)"
Click Ok

Using Bash for the First Time

Once the application is installed you can start using bash (bourne again shell). Bash is the shell, or command language interpreter, for the GNU operating system and is extremely versatile and powerful. If you're new to this (which you likely are if you are following this guide) then the power of bash may not be immediately obvious to you. I'll discuss this in detail in later posts.

To start bash you can do either of the following

Search for "bash" after hitting the Windows key
Hitting Windows+R to open the Run menu and typing "bash"

A comment about interpreters

By default the interpreter you will run "C:\Windows\System32\bash.exe" inside of is trash. It feels very similar to the command prompt interpreter. I immediately recommend you go and install ConEmu.

The wrong tools

2016-12-16T00:37:00.000-08:00

I preface this post by saying that I have spent my career working in Operations. As such, I spend a lot of time thinking about the tools I use and how to be efficient. True efficiency necessitates that your workstation act as an extension of yourself. This means that your ability to execute actions should be able to closely track your train of thought. More succinctly, the latency between the idea "I should grab the logs from this device" and the moment you are executing that action should be brief.

Ask yourself, how long does it take you to open a terminal? The answer to that question should be less than 1 second. How quickly can you be at the prompt of any remote device (router, VM, etc)? The correct answer here is no longer than the amount of time it would take you type the hostname.

The Wrong Tools

In my experience network engineers working in larger institutions (generally internet service providers or big enterprise corporations) use primarily Microsoft Windows for their workstation or laptop. As a consequence of this, these engineers are primarily using tools like

PuTTY for SSH or TELNET (ugh)
PuTTY Connection Manager (if you want to be "efficient")
SecureCRT (if you're wanting to go even fancier)
Some crappy third party FTP server and FTP client
Some crappy third party SCP client (if you're lucky, your crappy third party FTP client does this for you!)
WinRAR and WinZIP (I guess Windows can do this for you nowadays, space age technology, I tell ya)
Microsoft Excel (it's already installed on your work computer, so why not use it for everything, right?)

It's my contention that using any of the above software applications in your daily work as a network engineer is making you an inefficient engineer. Let's set aside the fact that having to download third party applications in order to do things as simple as SSH and SCP is crazy. The important point is that these functions are part of your day to day job. They should require zero thought to use and should be inter-operable.

How much of your time would it take backup configs from a single endpoint. How about 100 endpoints? The answer shouldn't differ greatly. And if you thinking that you'll need to find an scp client that can consume a list of targets from a Excel .xlsx file, you're on the wrong track.

The Right Tools

Largely the answer here is use a linux distribution as your primary operating system to perform your work. If your employer does not let you do this, kick and scream until they do. Other options include

Mac OS (be careful of overly glossy buttons)
Windows 10 with Linux Bash Shell (is this a thing yet?)
Cygwin

Once you're sitting within the warm glow of a linux shell, acquaint yourself with the below tools. All of these tools are older than you are and have withstood the test of time.

awk
sed
grep
tr
scp
tar

Learn how to use command line editors like VI or EMACS. Then learn enough bash scripting so that you can chain all of these tools together without needing to read MAN pages. And lastly learn to not overdo it. That is, don't write bash scripts to when you're dealing with a problem that is sufficiently complex that you should be using a proper programming language.

Excel User Defined Function for String Split - Split an IP Address by Octet

2012-03-25T01:15:00.001-07:00

I routinely find myself in a situation in Excel where I am having to use annoying string manipulation functions in order to extract a desired string from rows of text. Having some experience in Python and VBA the one function I end up missing the most is the split function. Simply put, this function breaks up a string by a specified delimiter and returns an array of the results. This proves to be very useful.

For example, say you want to break out the octets of multiple IP addreses. In order to accomplish this task in Excel you would have to use a combination of find() and mid() functions. Although possible, this is time consuming and often requires a bit of tuning before you get exactly what you want. Instead, try this UDF.
'bcell : A reference to a cell
'delimiter : a string containing the character(s) you would like to use to split the input string by
'index : which item from the resulting array you would like to return
Function stringsplit(bcell As Range, delimiter As String, _
index As Integer) As String
On Error GoTo errHandler
Dim splitStr As Variant
'We want index to start from 1, not 0
'This is a more user friendly approach
If index = 0 Then
GoTo errHandler
End If
index = index - 1
varSplit = Split(bcell.Value, delimiter)
StringSplit = varSplit(index)
Exit Function
errHandler:
StringSplit = "#N/A"
End Function

Here are some examples how we can use this in Excel. Let's assume A1 has the value "192.168.1.254" and we want to break out individual octets of this IP address.

=stringsplit( A1, ".", 1 )
=192

=stringsplit( A1, ".", 2 )
=168

If you're looking for something that is more specialized for handling IP addresses you may be interested in the below function. Usage is simply, =octet(A1,2) returns the second octet. The findIP() support function is used in case the text you are trying to parse contains other characters.

'This function returns the value of a specified octet in an IP address
'address : IP address stored as a string e.g. "192.168.1.1"
'index : integer value ranging from 1 to 4, specifying the octet to be returned
Function octet(address As String, index As Integer) As String
Dim addSplit As Variant
'Catch some typical error cases
If index = 0 Or index > 4 Or FindIP(address) = "" Then
GoTo errHandler
End If
'Use this offset to index the array
index = index - 1
addSplit = Split(FindIP(address), ".")
Octet = Int(addSplit(index))
Exit Function
errHandler:
Octet = "#N/A"
End Function

'A support function that parses input strTest through a regular
'expression to find an IP Address
'A blank value is returned if no IP address can be found
Function FindIP(strTest As String) As String
Dim RegEx As Object
Dim valid As Boolean
Dim Matches As Object
Dim i As Integer
Set RegEx = CreateObject("VBScript.RegExp")
RegEx.Pattern = "\b(?:\d{1,3}\.){3}\d{1,3}\b"
valid = RegEx.test(strTest)
If valid Then
Set Matches = RegEx.Execute(strTest)
FindIP = Matches(0)
Else
FindIP = ""
End If
End Function

Using TFTP to copy a new IOS Image

2011-10-26T11:26:00.000-07:00

Copying a new IOS version to a cisco device can done in a variety of ways, the below shows one such method

Connect the port Fa0/2 of the device to your computer with an ethernet cable
Change your computer IPv4 settings

192.168.1.1/24 instead of automatically detect

Turn off any firewall or policy settings on your computer that may prevent your computer being a TFTP server
Run SolarWinds TFTP Server (or any TFTP server application of your choice)
Start the TFTP Service
On the device use the below configuration

en
conf t
vlan 2
exit
int vlan 2
ip address 192.168.1.2 255.255.255.0
no shut
exit
int fa0/2
switchport access vlan 2
end
wr me

Do a test ping between the computer and the cisco device to test connectivity
Perform a TFTP copy command on the device

copy tftp flash
192.168.1.1
c2960-lanbasek9-mz.122-58.SE2.bin
c2960-lanbasek9-mz.122-58.SE2.bin

Do a show flash if you want to verify the file was copied
To boot from the new flash you would use the below configuration commands

conf t
boot-start-marker
boot system flash:c2960-lanbasek9-mz.122-58.SE2.bin
boot-end-marker
end
wr mem
reload

Query Access Database (MDB) from Python

2011-09-22T16:09:00.000-07:00

Ever wish that you could quickly run a query against an Access database (MDB) from a command line? If you're already working within a command line this can be very efficient to do a quick lookup. Here's how this can be accomplished.

Install pyodbc
Copy the lookup.py code into a file and edit the MDB and SQL variables to suit your needs
Copy the lookup.bat file into a file (you may need to edit the path to point to lookup.py)
You should save lookup.bat to a folder that is within your Windows Path

lookup.py

import sys, csv
import pyodbc

# Handle command line arguments
if len(sys.argv) == 1:
    print "Enter usage notes here"
    print "(use % for wildcard)"
    sys.exit()
print "Searching for '%s'" % sys.argv[1]
print ""

# Query the CLLI MDB
MDB = "PATH TO MDB"
DRV = "{Microsoft Access Driver (*.mdb)}"
conn = pyodbc.connect("DRIVER=%s;DBQ=%s" % (DRV,MDB))
curs = conn.cursor()
SQL = "SELECT * FROM table WHERE col='%s'" % sys.argv[1]
curs.execute(SQL)
rows = curs.fetchall()
curs.close()
conn.close()

# Print out results
i = 1
for row in rows:
    print "Result #%s" % i
    print "--------------------------"
    for col in row:
        print col
    print ""
    i = i + 1

lookup.bat

@call python.exe lookup.py %1

Excel User Defined Function for Ping

2011-09-20T15:46:00.000-07:00

Here's another Excel UDF that I'm sharing with everyone. This User Defined Function allows a user to execute a Windows ping command from an Excel cell. Currently this code is written to parse a string for an IP Address and ping that address (that's the desired effect I was looking for). If you wish to ping hostnames as well, you can make the following subsitution

Old: retVal = sPing(FindIP(sHost), 1, 200)

New: retVal = sPing(sHost, 1, 200)

Here are some examples of how to use the UDF and the code itself.

Ex. =ping("192.168.1.0") 'returns average ms response time

Ex. =ping("192.168.1.0") 'returns received ICMP packets

'--------------------------------------------

'Ping UDF

'--------------------------------------------

'A ping function formatted for use as an Excel cell formula

'sHost - the host to be pinged

'selectVal - optional parameter that defines what should be returned

' - refer to sPing return type for clarification of what

' value to be used

'          - the default return is the millisecond average ping time
Function Ping(sHost As String, Optional selectVal As Integer) As String
Dim retVal As Variant
retVal = sPing(FindIP(sHost), 1, 200)
If retVal(1) <> 0 Then
    If selectVal = 0 Then
        Ping = retVal(4)
    Else
        Ping = retVal(selectVal)
    End If
Else
    Ping = "#N/A"
End If

End Function

'--------------------------------------------

'Find IP Address

'--------------------------------------------

'A support function that parses input strTest through a regular

'expression to find an IP Address

'A blank value is returned if no IP address can be found
Function FindIP(strTest As String) As String
    Dim RegEx As Object
    Dim valid As Boolean
    Dim Matches As Object
    Dim i As Integer
    Set RegEx = CreateObject("VBScript.RegExp")

    RegEx.Pattern = "\b(?:\d{1,3}\.){3}\d{1,3}\b"
    valid = RegEx.test(strTest)
    If valid Then
        Set Matches = RegEx.Execute(strTest)
        FindIP = Matches(0)
    Else
        FindIP = ""
    End If
End Function

'--------------------------------------------

'Ping Support Function

'--------------------------------------------

'This function runs a cmd ping request

'sHost - the host to be pinged

'numPings - the number of ICMP ping packets to send

'msDelay - the ping timeout delay

'The return value from this function is an array with the following

'values defined

'sPing(0) - Transmitted ICMP Packets

'sPing(1) - Received ICMP Packets

'sPing(2) - Lost ICMP Packets

'sPing(3) - Maximum Round Trip Time

'sPing(4) - Average Round Trip Time

'sPing(5) - Comments
Function sPing(sHost As String, numPings As Integer, msDelay As Integer) As Variant
    Dim oFSO As Object, oShell As Object, oTempFile As Object
    Dim sLine As String, sFilename As String
    Dim retVal(5) As String

    Set oFSO = CreateObject("Scripting.FileSystemObject")
    Set oShell = CreateObject("Wscript.Shell")


    sFilename = oFSO.GetTempName
    oShell.Run "cmd /c ping -n " & numPings & " -w " & msDelay & " " & sHost & " > " & sFilename, 0, True
    Set oTempFile = oFSO.OpenTextFile(sFilename, 1)
    Do While oTempFile.AtEndOfStream <> True
    sLine = oTempFile.Readline
    cmdStr = cmdStr & Trim(sLine)
    Loop
    oTempFile.Close
    oFSO.DeleteFile (sFilename)

    If cmdStr = "IP address must be specified." Then
        retVal(5) = "IP address must be specified"
        sPing = retVal
        Exit Function
    End If

    If InStr(1, cmdStr, "Ping request could not find host", vbTextCompare) > 0 Then
        retVal(6) = "Could not find host"
        sPing = retVal
        Exit Function
    End If

    txLoc = InStr(1, cmdStr, "Sent = ", vbTextCompare)
    If txLoc > 0 Then
        txLoc = txLoc + 7
        txLoc2 = InStr(txLoc, cmdStr, ",", vbTextCompare)
        retVal(0) = Trim(Mid(cmdStr, txLoc, txLoc2 - txLoc))
    End If

    rxLoc = InStr(1, cmdStr, "Received = ", vbTextCompare)
    If rxLoc > 0 Then
        rxLoc = rxLoc + 11
        rxLoc2 = InStr(rxLoc, cmdStr, ",", vbTextCompare)
        retVal(1) = Trim(Mid(cmdStr, rxLoc, rxLoc2 - rxLoc))
    End If

    ltLoc = InStr(1, cmdStr, "Lost = ", vbTextCompare)
    If ltLoc > 0 Then
        ltLoc = ltLoc + 7
        ltLoc2 = InStr(ltLoc, cmdStr, "(", vbTextCompare)
        retVal(2) = Trim(Mid(cmdStr, ltLoc, ltLoc2 - ltLoc))
    End If

    maxLoc = InStr(1, cmdStr, "Maximum = ", vbTextCompare)
    If maxLoc > 0 Then
        maxLoc = maxLoc + 10
        maxLoc2 = InStr(maxLoc, cmdStr, "ms", vbTextCompare)
        retVal(3) = Trim(Mid(cmdStr, maxLoc, maxLoc2 - maxLoc))
    End If

    aveLoc = InStr(1, cmdStr, "Average = ", vbTextCompare)
    If aveLoc > 0 Then
        aveLoc = aveLoc + 10
        aveLoc2 = InStr(aveLoc, cmdStr, "ms", vbTextCompare)
        retVal(4) = Trim(Mid(cmdStr, aveLoc, aveLoc2 - aveLoc))
    End If
    sPing = retVal
End Function

Excel User Defined Function for CIDR Subnet Mask Notation

2011-09-19T14:10:00.000-07:00

Here's another one of the Excel UDF's I have prepared for performing network operations in Excel. I find it specifically useful when dealing with large data sets. This function converts a subnet mask to CIDR notation. It is implemented with statically defining a dictionary object. It may not be an elegant solution, but it is works.

Ex. CIDR("255.128.0.0")=9

'--------------------------------------------
'CIDR (Classless Inter-Domain Routing)
'--------------------------------------------
'Given an inputStr value containing a subnet mask in an IP 

'format this functio returns the corresponding subnet mask 

'in CIDR Notation
Function cidr(inputStr As String)
    Dim maskDict As Dictionary
    Set maskDict = New Dictionary
    maskDict.Add "128.0.0.0", 1
    maskDict.Add "192.0.0.0", 2
    maskDict.Add "224.0.0.0", 3
    maskDict.Add "240.0.0.0", 4
    maskDict.Add "248.0.0.0", 5
    maskDict.Add "252.0.0.0", 6
    maskDict.Add "254.0.0.0", 7
    maskDict.Add "255.0.0.0", 8
    maskDict.Add "255.128.0.0", 9
    maskDict.Add "255.192.0.0", 10
    maskDict.Add "255.224.0.0", 11
    maskDict.Add "255.240.0.0", 12
    maskDict.Add "255.248.0.0", 13
    maskDict.Add "255.252.0.0", 14
    maskDict.Add "255.254.0.0", 15
    maskDict.Add "255.255.0.0", 16
    maskDict.Add "255.255.128.0", 17
    maskDict.Add "255.255.192.0", 18
    maskDict.Add "255.255.224.0", 19
    maskDict.Add "255.255.240.0", 20
    maskDict.Add "255.255.248.0", 21
    maskDict.Add "255.255.252.0", 22
    maskDict.Add "255.255.254.0", 23
    maskDict.Add "255.255.255.0", 24
    maskDict.Add "255.255.255.128", 25
    maskDict.Add "255.255.255.192", 26
    maskDict.Add "255.255.255.224", 27
    maskDict.Add "255.255.255.240", 28
    maskDict.Add "255.255.255.248", 29
    maskDict.Add "255.255.255.252", 30
    maskDict.Add "255.255.255.254", 31
    maskDict.Add "255.255.255.255", 32
    cidr = maskDict.Item(FindIP(inputStr))
End Function

'--------------------------------------------

'Find IP Address

'--------------------------------------------

'A support function that parses input strTest through a regular

'expression to find an IP Address

Excel User Defined Function - nslookup

2011-08-01T23:36:00.000-07:00

Recently I have found myself in a position where I needed to run basic networking commands on numerous remote devices. At the end of the day I often found myself with several command prompts or cygwin bash shells open where I had to run umpteen nslookup commands.

It didn't take long for me to find this whole process entirely unsatisfactory. I wanted to a way to run these commands very quickly and have the results returned in a place that I could refer to afterwards. I turned to the internet and ultimately did not find a simple, free, trustworthy utility that I could use to serve this purpose.

I turned to Excel and decided to write a user defined function that completes an nslookup command. The function has two parameters as described below:

nslookup( lookupVal, [addressOpt] )

lookupVal: represents the hostname or ip address that you wish to lookup

addressOpt: An integer that can have one of the values below

<blank>	Auto Detect
0	Auto Detect
1	Forced Address Lookup
2	Forced Name Lookup

Here's some examples of how to run the code

=nslookup("hostname")
=nslookup("148.175.254.250")
=nslookup("www.google.com",1)
=nslookup("149.175.254.240",2)
=nslookup(B1)

The full Excel UDF is shown below. In order to get this function working properly you should save copy this code into a module, save the Excel File as an .xlam file, and add the file as an Add-in to Excel. A subsequent blog post will detail this process.

Public Function NSLookup(lookupVal As String, Optional addressOpt As Integer) As String
   Const ADDRESS_LOOKUP = 1
   Const NAME_LOOKUP = 2
   Const AUTO_DETECT = 0
   
   'Skip everything if the field is blank
   If lookupVal <> "" Then
        Dim oFSO As Object, oShell As Object, oTempFile As Object
        Dim sLine As String, sFilename As String
        Dim intFound As Integer
        Set oFSO = CreateObject("Scripting.FileSystemObject")
        Set oShell = CreateObject("Wscript.Shell")
        
        'Handle the addresOpt operand
        'Regular Expressions are used to complete a substring match for an IP Address
        'If an IP Address is found, a DNS Name Lookup will be forced
        If addressOpt = AUTO_DETECT Then
            ipLookup = FindIP(lookupVal)
            If ipLookup = "" Then
                addressOpt = ADDRESS_LOOKUP
            Else
                addressOpt = NAME_LOOKUP
                lookupVal = ipLookup
            End If
        'Do a regular expression substring match for an IP Address
        ElseIf addressOpt = NAME_LOOKUP Then
            lookupVal = FindIP(lookupVal)
        End If
        
        'Run the nslookup command
        sFilename = oFSO.GetTempName
        oShell.Run "cmd /c nslookup " & lookupVal & " > " & sFilename, 0, True
        Set oTempFile = oFSO.OpenTextFile(sFilename, 1)
        Do While oTempFile.AtEndOfStream <> True
            sLine = oTempFile.Readline
            cmdStr = cmdStr & Trim(sLine) & vbCrLf
        Loop
        oTempFile.Close
        oFSO.DeleteFile (sFilename)
        
        'Process the result
        intFound = InStr(1, cmdStr, "Name:", vbTextCompare)
        If intFound = 0 Then
            NSLookup = "NotFound"
            Exit Function
        ElseIf intFound > 0 Then
            'TODO: Cleanup with RegEx
            If addressOpt = ADDRESS_LOOKUP Then
                loc1 = InStr(intFound, cmdStr, "Address:", vbTextCompare) + InStr(intFound, cmdStr, "Addresses:", vbTextCompare)
                loc2 = InStr(loc1, cmdStr, vbCrLf, vbTextCompare)
                nameStr = Trim(Mid(cmdStr, loc1 + 8, loc2 - loc1 - 8))
            ElseIf addressOpt = NAME_LOOKUP Then
                loc1 = InStr(intFound, cmdStr, "Name:", vbTextCompare)
                loc2 = InStr(loc1, cmdStr, vbCrLf, vbTextCompare)
                nameStr = Trim(Mid(cmdStr, loc1 + 5, loc2 - loc1 - 5))
            End If
        End If
        NSLookup = nameStr
    Else
        NSLookup = "N/A"
    End If
End Function

Function FindIP(strTest As String) As String
    Dim RegEx As Object
    Dim valid As Boolean
    Dim Matches As Object
    Dim i As Integer
    Set RegEx = CreateObject("VBScript.RegExp")
    
    RegEx.Pattern = "\b(?:\d{1,3}\.){3}\d{1,3}\b"
    valid = RegEx.test(strTest)
    If valid Then
        Set Matches = RegEx.Execute(strTest)
        FindIP = Matches(0)
    Else
        FindIP = ""
    End If
End Function