Virtualize the World!

Exchange 2010 UM Old Voicemails

2015-08-10T10:57:00.001-05:00

I ran into an issue in applying Exchange 2010 Service Pack 3 on an Unified Messaging server - the Prerequisite Check came back with the following error:

Unified Messaging Role Prerequisites
Failed

Error:
The Unified Messaging voice mail folder 'C:\Program Files\Microsoft\Exchange Server\V14\unifiedmessaging\voicemail' isn't empty. This folder must be empty before upgrade can proceed.
Click here for help... http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.141).aspx?v=14.3.123.3&e=ms.exch.err.Ex28883C&l=0&cl=cp

I went and looked, and indeed this folder was not empty. Looking at the dates on the voicemails, it was obvious that they were all old, and had been there for some time (the most recent was 6 months old). I believe there had been some previous delivery issues that had prevented them from being delivered to the proper mailbox. Since they were all old, I merely made a copy of them, and then removed them from this folder. Once the voicmail folder was empty, SP3 was able to continue installing without issue.

------
Dustin Shaw
VCP

Dump SMTP Relay and Connection Info from IIS on 2003 via VBS

2015-07-23T11:44:00.000-05:00

I had the need to pull all the SMTP related information from an old 2003 IIS server setup to do relaying. The specific information I was looking for was:

IPs allowed to Relay
IPs allowed to Connect
Domains and if they had any SmartHost setup

Obviously, pulling this information via the GUI was not practical - you can only view 5 Relay IPs at a time, 2 Connection IPs at a time, and you have to manually check each domain's properties to verify SmartHost information.

I was also unable to pull the information straight out of the Metabase Explorer, as I would still have to go to each domain separately, and then convert the Relay and Connection IPs from Hex.

I looked around, but was unable to locate a ready-to-use VBScript that gave me what I wanted. I did find a script here that dumped the IIS SMTP Relay IPs, so I started there and adapted to also get the Connection IPs (listed as IPSecurity). Then I found this site that detailed how to get the Domains and their settings. I added this to the script, and voila, I had what I wanted in a nice CSV file.

To run the script, copy the below into notepad, save as ExportIISSMTPSettings.vbs and run with the following command:
cscript ExportIISSMTPSettings.vbs > IISSMTPServerSettings.csv

'#####================================================================================
'## Title: ExportIISSMTPSettings.vbs
'##     '##
'#####================================================================================

Set objSMTP = GetObject("IIS://localhost/smtpsvc/1") 'Connect to the IIS Namespace, You can change the "smtpsvc/1" to fit your needs.
Set objRelayIpList = objSMTP.Get("RelayIpList") 'Get the RelayIPListObject
Set objIPSecurity = objSMTP.Get("IPSecurity") 'Get the IPSecurityObject

' *** Get Relay List
' GrantByDefault returns 0 when "only the list below" is set (false) and -1 when all except the list below is set(true)
Wscript.echo "Results will be display based on the Relay Restrictions Radio Buttion Selection"
Wscript.echo " o Only the list below"
Wscript.echo " o All Except the list below"
Wscript.echo "-------------"
If objRelayIpList.GrantByDefault = true Then
    Wscript.Echo "All except the list below :"
    Wscript.echo "-------------"
    objCurrentList = objRelayIpList.IPDeny
Else
    Wscript.Echo "Only the list below :"
    Wscript.echo "-------------"
    objCurrentList = objRelayIpList.IPGrant
End If
    count = 0
For Each objIP in objCurrentList
    Wscript.Echo objIP
    count = count + 1
Next
If count = 0 Then
    Wscript.Echo "There were no IP Addresses Found"
End If
' *** Get Connection Control List
Wscript.echo "Results will be display based on the Connection Control Radio Buttion Selection"
Wscript.echo " o Only the list below"
Wscript.echo " o All Except the list below"
Wscript.echo "-------------"
If objIPSecurity.GrantByDefault = true Then
    Wscript.Echo "All except the list below :"
    Wscript.echo "-------------"
    objCurrentList = objIPSecurity.IPDeny
Else
    Wscript.Echo "Only the list below :"
    Wscript.echo "-------------"
    objCurrentList = objIPSecurity.IPGrant
End If
    count = 0
For Each objIP in objCurrentList
    Wscript.Echo objIP
    count = count + 1
Next
If count = 0 Then
    Wscript.Echo "There were no IP Addresses Found"
End If
Wscript.echo ""
' *** Get Domains and settings
Wscript.echo "Displaying list of Domains and settings"
Wscript.echo "-------------"
Wscript.echo "Route Actions:"
Wscript.echo "2: Use DNS to route to this domain"
Wscript.echo "4098: Forward all mail to smart host"
strComputer = "."
Set objWMIService = GetObject _
    ("winmgmts:{authenticationLevel=pktPrivacy}\\" _
        & strComputer & "\root\microsoftiisv2")
Set colItems = objWMIService.ExecQuery _
    ("Select * from IIsSmtpDomainSetting")

For Each objItem in colItems
    Wscript.echo ""
    For Each strTurn in objItem.AuthTurnList
        Wscript.Echo "Authentication Turn List: " & strTurn
    Next
    Wscript.Echo "CSide Etrn Domains: " & objItem.CSideEtrnDomains
    Wscript.Echo "Name: " & objItem.Name
    Wscript.Echo "Relay For Authentication: " & objItem.RelayForAuth
    Wscript.Echo "Relay IP List: " & objItem.RelayIpList
    Wscript.Echo "Route Action: " & objItem.RouteAction
    Wscript.Echo "Route Action String: " & objItem.RouteActionString
    Wscript.Echo "Route Password: " & objItem.RoutePassword
    Wscript.Echo "Route User Name: " & objItem.RouteUserName
Next

------
Dustin Shaw
VCP

EventID 9385 on Exchange 2010 After Demoting DC

2015-06-02T09:00:00.005-05:00

I recently demoted an old Domain Controller in an effort to move forward in my domain - it was a 32-bit 2008 Server, and all the rest of the DCs are 2008R2 or 2012R2. I don't have any needs today to move up AD functionality today (we are already on 2008 Forest and Domain Functionality), but it never hurts to be ready.

After demoting an old Domain Controller, I recently started receiving Error 9385 on one of my Exchange 2010 Mailbox servers:

Log Name: Application
Source: MSExchangeSA
Date: 6/2/2015 8:43:34 AM
Event ID: 9385
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: MailboxSVR.internal.domain
Description:
Microsoft Exchange System Attendant failed to read the membership of the universal security group '/dc=domain/dc=internal/ou=Microsoft Exchange Security Groups/cn=Exchange Servers'; the error code was '8007203a'. The problem might be that the Microsoft Exchange System Attendant does not have permission to read the membership of the group.

If this computer is not a member of the group '/dc=domain/dc=internal/ou=Microsoft Exchange Security Groups/cn=Exchange Servers', you should manually stop all Microsoft Exchange services, run the task 'add-ExchangeServerGroupMember,' and then restart all Microsoft Exchange services.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchangeSA" />
<EventID Qualifiers="49152">9385</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-06-02T13:43:34.000000000Z" />
<EventRecordID>2199039</EventRecordID>
<Channel>Application</Channel>
<Computer>MailboxSVR.internal.domain</Computer>
<Security />
</System>
<EventData>
<Data>/dc=com/dc=wmfingrp/dc=internal/ou=Microsoft Exchange Security Groups/cn=Exchange Servers</Data>
<Data>8007203a</Data>
</EventData>
</Event>

After doing some research, most articles said that you need to make sure that it's a member of the group, etc, but all of that was correct. There weren't any references to the old DC in anything I checked (DNS was pointed elsewhere, Domain controllers and Global catalog servers that this Exchange server used were pointed elsewhere, etc). But, I knew it had to do with my demoted DC. For some reason this particular Exchange server was really hoping that the DC would answer his requests. I didn't notice any other performance or user issues during this time, so it looks like the Exchange server was able to get his answer elsewhere after checking here.

Once I was able to take a maintenance window, I rebooted the affected Exchange server, and all was well. It just needed to clear it's head after loosing it's good friend.

------
Dustin Shaw
VCP

FortiGate Extractors for Graylog 1.0

2015-05-06T08:45:00.002-05:00

Graylog is a nice opensource alternative to Splunk and other SIEM tools. I've been using it for several years, and continue to make tweaks to improve its usefulness in my environment. I'm excited now that it is on version 1.0 (and was renamed Graylog instead of Graylog2), and is a lot more stable.

One of the tweaks I made a while back on a previous version was to create create a DRL extractor for FortiGate (a firewall made by FortiNet). I've now updated this extractor so that you can import it using the new JSON format directly into the web interface (instead of having to create the DRL file, etc).

To apply the extractors on Graylog, go to your FortiGate Input, and Import Extractors. The details on how to do that can be found on Graylog's site here.

Here's the JSON script for the extractors:

{
"extractors": [
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sdevname=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "source",
"title": "FGTsource"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\saction=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "action",
"title": "FGTaction"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sapp=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "app",
"title": "FGTapp"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sappact=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "appact",
"title": "FGTappact"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sappcat=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "appcat",
"title": "FGTappcat"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sapplist=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "applist",
"title": "FGTapplist"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sattack=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "attack",
"title": "FGTattack"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sdevid=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "devid",
"title": "FGTdevid"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sdir=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "dir",
"title": "FGTdir"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sdstcountry=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "dstcountry",
"title": "FGTdstcountry"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sdstintf=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "dstintf",
"title": "FGTdstintf"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sdstip=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "dstip",
"title": "FGTdstip"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sdstport=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "dstport",
"title": "FGTdstport"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sdtype=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "dtype",
"title": "FGTdtype"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sduration=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "duration",
"title": "FGTduration"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\serror_reason=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "error_reason",
"title": "FGTerror_reason"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\seventtype=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "eventtype",
"title": "FGTeventtype"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sfile=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "file",
"title": "FGTfile"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sgroup=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "group",
"title": "FGTgroup"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\shostname=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "hostname",
"title": "FGThostname"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sidentidx=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "identidx",
"title": "FGTidentidx"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sinit=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "init",
"title": "FGTinit"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\slocip=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "locip",
"title": "FGTlocip"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\slocport=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "locport",
"title": "FGTlocport"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\slogid=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "logid",
"title": "FGTlogid"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\smode=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "mode",
"title": "FGTmode"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\smsg=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "msg",
"title": "FGTmsg"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\soutintf=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "outintf",
"title": "FGToutintf"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\speer_notif=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "peer_notif",
"title": "FGTpeer_notif"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\spolicyid=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "policyid",
"title": "FGTpolicyid"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sprofile=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "profile",
"title": "FGTprofile"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sprofiletype=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "profiletype",
"title": "FGTprofiletype"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sproto=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "proto",
"title": "FGTproto"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\squarskip=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "quarskip",
"title": "FGTquarskip"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\srcvdbyte=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "rcvdbyte",
"title": "FGTrcvdbyte"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\srcvdpkt=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "rcvdpkt",
"title": "FGTrcvdpkt"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sref=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "ref",
"title": "FGTref"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sremip=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "remip",
"title": "FGTremip"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sremport=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "remport",
"title": "FGTremport"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sresult=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "result",
"title": "FGTresult"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\srole=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "role",
"title": "FGTrole"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\ssentbyte=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "sentbyte",
"title": "FGTsentbyte"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\ssentpkt=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "sentpkt",
"title": "FGTsentpkt"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sservice=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "service",
"title": "FGTservice"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sservice=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "service",
"title": "FGTservice"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\ssrccountry=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "srccountry",
"title": "FGTsrccountry"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\ssrcintf=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "srcintf",
"title": "FGTsrcintf"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\ssrcip=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "srcip",
"title": "FGTsrcip"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\ssrcport=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "srcport",
"title": "FGTsrcport"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sstage=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "stage",
"title": "FGTstage"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sstatus=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "status",
"title": "FGTstatus"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sstatus=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "status",
"title": "FGTstatus"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\ssubtype=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "subtype",
"title": "FGTsubtype"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\stransport=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "transport",
"title": "FGTtransport"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\stype=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "type",
"title": "FGTtype"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\strandisp=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "trandisp",
"title": "FGTtrandisp"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\stransip=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "transip",
"title": "FGTtransip"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\suser=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "user",
"title": "FGTuser"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sutmaction=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "utmaction",
"title": "FGTutmaction"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sutmevent=(\\S+)\\s"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "utmevent",
"title": "FGTutmevent"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\svd=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "vd",
"title": "FGTvd"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": ".+\\svirus=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "virus",
"title": "FGTvirus"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\svpntunnel=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "vpntunnel",
"title": "FGTvpntunnel"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sxauthgroup=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "xauthgroup",
"title": "FGTxauthgroup"
},
{
"condition_type": "none",
"condition_value": "",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "^.+\\sxauthuser=\\\"(.+?)\\\""
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "xauthuser",
"title": "FGTxauthuser"
}
],
"version": "1.0.0"
}

------
Dustin Shaw
VCP

Outlook Sync Issues

2015-03-31T08:35:00.000-05:00

Synchronization issues are something that seem to plague any Exchange Admin. They come up for no reason, and go away for no reason. There have been a number of posts on the issue over the years, most telling MS to go fix their issues.

One particular issue that I've seen recently is when moving a database from one Exchange 2010 DAG Member to another, the Sync Issues folder starts filling up with the following Synchronization Log messages:

7:57:57 Synchronizer Version 15.0.4701

7:57:57 Synchronizing Mailbox 'User Name'

7:57:57 Synchronizing Hierarchy

7:57:58 Synchronizing server changes in folder 'Inbox'

7:57:58 Downloading from server 'Client-Access-Server'

7:57:58 1 item(s) added to offline folder

7:57:58 Error synchronizing folder

7:57:58 [8004010F-501-8004010F-0]

7:57:58 The client operation failed.

7:57:58 Microsoft Exchange Information Store

7:57:58 For more information on this failure, click the URL below:

7:57:58 http://www.microsoft.com/support/prodredirect/outlook2000_us.asp?err=8004010f-501-8004010f-0

7:57:58 Canceled

Checking all the normal locations, there is nothing wrong with the synchronization, yet every 5 minutes Outlook reports an error synchronizing. The above example is with Outlook 2013 on Exchange 2010 SP3 RU6, but I've seen it through several Service Packs and Rollups. The issue also shows up on Outlook 2010.

Looking through all the settings on the Exchange DAG Members, and the Client Access Server in use (which doesn't change during the failover), there is nothing out of the ordinary that might keep it from syncing.

If I find a solution to this particular issue, I'll post it back here for posterity. Until then, this is merely another "Microsoft, please fix you're sync problems" blog.

------
Dustin Shaw
VCP

Brother ControlCenter4 Scanning Permissions

2015-03-18T11:23:00.000-05:00

Brother ControlCenter4 Scanning to File requires certain permissions to write files to the folder. These permissions use the current logged in user permissions. To test that the write permissions are accessible, ControlCenter4 appears to write a .tmp file and create a temp folder. I say appears because I have not found any documentation on the subject.

I ran across these required permissions because a scanner I support was trying to write to a network share that allowed Full Access MINUS the ability to create folders. Without the ability to create folders, it would drop a .tmp file in the folder on scan, and pop up an Unable to write the file to "Destination Folder". error like the below:

Once I assigned the user the ability to create folders in this directory (only), ControlCenter4 was able to scan properly.

------
Dustin Shaw
VCP

Install ElasticSearch 1.3 on CentOS 6 for Graylog2 1.0

2015-02-18T09:15:00.000-06:00

Graylog2 1.0 requires ElasticSearch 1.3 (or up). To fulfill this requirement, you will need to install ElasticSearch - I recommend on a separate server for environment growth. IOPS on your hard disk matters here.

First, make sure your CentOS 6 is fully patched:
#yum update

Then install Java 1.7:
#yum install java
Make sure that it prompts you to install Java 1.7 (look for the below text):
Installing:
java-1.7.0-openjdk

Set up the ElasticSearch Repositories and install Elasticsearch 1.3:

Import they key:
#rpm --import https://packages.elasticsearch.org/GPG-KEY-elasticsearch

Create a new repository file in /etc/yum.repos.d/ named elasticsearch.repo with the following contents:
[elasticsearch-1.3]
name=Elasticsearch repository for 1.3.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/1.3/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1

Install ElasticSearch:
#yum install elasticsearch

Add ElasticSearch to boot process:
#chkconfig --add elasticsearch

Stop the elasticsearch service so that we can update the cluster name:
#service elasticsearch stop

Edit the /etc/elasticsearch/elasticsearch.yml file to update your cluster.name variable. Ex:
cluster.name: graylog2_production

Update any additional settings needed and save the file. I recommend updating the path.data and path.logs to custom directories.

Start the elasticsearch service and set it to run on startup:
#service elasticsearch start
#chkconfig elasticsearch on

Check your logs to make sure that it started properly and joined the cluster (if there is an existing one).

For Graylog2, the recommended settings are also to increase the open file limit to at least 64000 as seen in the Configuring and tuning Elasticsearch documentation. I did this by increasing the max number of ulimit open file below.

Edit /etc/sysctl.conf and add the following line at the end:
fs.file-max = 65536

Save the file. Next edit /etc/security/limits.conf and add the following lines:
*               soft    nproc           65535
*               hard    nproc           65535
*               soft    nofile          65535
*               hard    nofile          65535

Save the file and restart the server.
#shutdown -r now

Once restarted, verify that the max open file ulimit has been increased.
# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30507
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 65535
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Additional recommended settings are to increase the ES_HEAP_SIZE. I did this by editing /etc/init.d/elasticsearch and adding the following line after checkJava under start():
ES_HEAP_SIZE=2g

They recommend that you leave 50% of your memory for other system functions, and I had 4 Gig of RAM, hence the 2g setting.

------
Dustin Shaw
VCP

Deploying vCOPS 5.x in vSphere Essentials Plus

2014-10-10T07:11:00.000-05:00

VMware vCenter Operations Manager 5.x has a requirement for DRS (Distributed Resource Schedule) to be able to be deployed in a cluster - this is because it is deployed as a vApp, and vApps require DRS.

If you are attempting to install vCOPS on vSphere Essentials, Essentials Plus, or Standard, you will discover that you don't have license for DRS. VMware released this article as the resolution to the problem. Here are the steps that they detailed:

---

To deploy vCenter Operations Manager in a vCenter Server environment with an Essentials Plus or Standard cluster of three ESX hosts:

Remove one of the ESX hosts from the cluster so that it resides directly under the parent datacenter.
On that ESX host, deploy the vCenter Operations Manager vApp, specifying static IP addresses.
Power on the vApp before moving the host back into the cluser to ensure IP settings are picked up.
License the solution in vCenter.
Move the ESX host with vCenter Operations back into the cluster.

Note: Static IP addresses are required.

Warning: The steps above dissolve the vApp container and an error is displayed. Disregard the error message and continue moving the host into the cluster.

Moving the ESX host with the vCenter Operations vApp back into the cluster results in the addition of the two virtual machines (the UI virtual machine and the Analytics virtual machine) to the cluster, without the vApp container. vCenter Operations Manager 5.x continues to function normally when this happens.
---

Also, in case you need it, the default logins for vCOPS are:
Default Administration:
admin
admin

Default Root Login:
root
vmware

------
Dustin Shaw
VCP

User exceeded the maximum of 250 objects of type "objtMessage".

2014-09-08T09:47:00.000-05:00

I have users that are regularly exceeding 250 objtMessage objects in Exchange 2010. What this really means is that the user has over 250 Messages open at once. While this seems high, when you consider third party tools and other such things, 250 is really not that much. I actually have a couple of users that regularly spike over 500 on a daily basis.

Whenever this behavior exhibits itself, it's in the form of ERROR in the Application log with EventID 9646. The text reads:

Mapi session "64807ed0-b1b3-4938-92e3-b9dbd9cfe67b: /o=Company/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=User Name" exceeded the maximum of 250 objects of type "objtMessage".

The default setting for this (and other open item limits in Exchange 2010) is detailed in Microsoft's Exchange Store Limits article.

To change from the default settings, open the registry on your Mailbox Server, and navigate to the following key:
HKLM\SYSTEM\ConcurrentControlSet\services\MSExchangeIS\ParametersSystem\MaxObjsPerMapiSession

Update/create the DWORD named objtMessage and give it the decimal value that you need (up it to 500).

Don't forget if you are in a DAG to repeat the registry entries on all Mailbox Servers.

------
Dustin Shaw
VCP

The symbolic link cannot be followed

2014-07-30T12:46:00.000-05:00

When you setup symbolic links on a server that point to another server, you will by default run into the inability for a client computer to follow the links with the following error:

The symbolic link cannot be followed because its type is disabled.

This is because the ability to traverse from one remote system to another across the symbolic link is disabled by default. You can see what is disabled and what is enabled on a computer by running the fsutil command:

>fsutil behavior query eymlinkevaluation
Local to local symbolic links are enabled.
Local to remote symbolic links are enabled.
Remote to local symbolic links are disabled.
Remote to remote symbolic links are disabled.

You have two methods to enable this - enable it locally on each machine, or enable it via Group Policy.

Local

The downsides to enabling it locally are obvious, but sometimes you just need it on one stubborn computer *right now* and can't wait for GP. To enable Remote to Remote symbolic links, run the following command:
fsutil behavior set symlinkevaluation R2R:1

Similarly, you can change the settings for Local to Local (L2L), Local to Remote (L2R), and Remote to Local (R2L) by using 1 for enabled and 0 for disabled.

Group Policy

To enable (or disable) Remote to Remote symbolic links in Group Policy, create a new GPO Policy (or edit a current one), and edit it. Navigate to:
Computer Configuration -> Administrative Templates -> System -> Filesystem
You can then set the settings how you want in Selectively allow the evaluation of a symbolic link

Once you've created your new GPO, test it and validate that it is successfully applied using gpresult /R and rsop.

How to use Group Policy to allow the users to chose any screensaver except (None)

2014-07-28T11:43:00.000-05:00

I just found one of the most beautiful Group Policies that I've ever come across:

How to use Group Policy to allow the users to chose any screensaver except (None)

This post is from Group Policy Central, and is 4 years old, but I've verified that it works properly with Windows 7 and 8, and is just a beautifully done Group Policy. Thanks Kevin for creating it and thank Alan for sharing.

The below is excerpts from the posting:

Step 1. Edit a Group Policy Object (GPO) that is targeted to the users accounts you wan to apply this policy
Step 2. Navigate to User Configuration > Preferences > Windows Settings > Registry then from the menu click on Action > New > Registry Item

Step 3. Select “Update” from the Action then type “Control Panel\Desktop” in the Key Path: text field then type “SCRNSAVE.EXE” in the Value Name text field and “C:\Windows\System32\scrnsave.scr” in the Value data: text field.

Step 4. Click on the Common tab and then tick “Item-level targeting” and then click the “Targeting…” button.

Now we will target the screen saver to apply only when the “HKCU\Control Panel\Desktop\SCRNSAVE.EXE” registry key does NOT exist as this means the screen saver has been configured to “(None)”.
Step 5. Click on “New Item” then the “Registry Match” option.

Step 6. Select the “Value exists” Match type” then type “Control Panel\Desktop” in the key path field and then type “SCRNSAVE.EXE” in the value name field

Step 7. Click back on the targeting setting in the top pane and press “F8” which changes the option to “does not exist” then click OK and OK.

This policy will now apply the blank screen saver on the next group policy refresh to all targeted users whenever they select the “(None)”.

Installing Exchange Server 2010 SP3 Rollup 6

2014-07-19T01:18:00.000-05:00

To get the permissions correct for installing Rollups on Exchange 2010 SP3, you will need to either disable UAC (not recommended) or you will need to launch the Rollup installer from an elevated command prompt (Right click and Run as Administrator) with the following command:
msiexec /update Exchange2010-KB2936871-x64-en.msp

This will allow the rollup to install properly. Other words, it will Roll Back and say that it Ended Prematurely.

Another note on Rollup 6 for SP3 is that it takes (at least in my environment) an extremely long time to generate native images for .NET assemblies. One of my servers took 45 minutes for this process. Wait it out and you'll be able to get it installed, just plan your windows accordingly.

------
Dustin Shaw
VCP

The wizard was interrupted before VMware Tools could be completely installed.

2014-05-15T08:21:00.000-05:00

Upon attempting to install VMware Tools on a Windows Server 2008R2 server, I received an error stating "The wizard was interrupted before VMware Tools could be completely installed."

After looking around on the internet, the closest thing I could find was this post by David Homer, detailing a similar issue with VMware Tools on VMware Workstation.

I tried his fixes (remove VMware Tools registry key, remove the vmtools service), but none of them allowed me to get past the problem.

What eventually worked was doing a search in the registry for all references to "vmware" and removing all keys having to do with VMware Tools - make sure you don't remove any of the keys referring to your SCSI devices/other hardware; just the VMware Tools entries.

I believe it was most likely the Installer registry entries that were mucking it up.

After a quick reboot to refresh the registry, VMware Tools installed with no issues.

------
Dustin Shaw
VCP

Graylog2 Version Check Errors

2014-04-30T07:09:00.000-05:00

I noticed periodic warnings in my Graylog2 0.20.1 instance saying:
WARN : org.graylog2.periodical.VersionCheckThread - Could not perform version check

The fix for this (as detailed here in Google Groups by Lennart Koopmann) is to set an undocumented flag in your /etc/graylog2.conf as follows:
versionchecks = false

------
Dustin Shaw
VCP

Graylog2 Extractors for FortiGate

2014-04-29T14:00:00.000-05:00

UPDATE: I've created the JSON version of this for Graylog 1.0 in a new post here.

Graylog2 is a really powerful log monitor, but it needs some customization when it comes to specific devices.

Pointing a Fortinet FortiGate firewall to Graylog2 results in mediocre usage, due to the syslog field not getting extracted properly. Essentially you end up with all of the data just dumped into the message field.

To point the FortiGate to Graylog2, open the FortiGate console and config the syslog settings:
config log syslogd setting

Then enable and point the logging to your server:
set status enable
set server 192.168.40.50
set port 514
set facility syslog

Once Graylog2 is successfully receiving the logs from the FortiGate, you will need to use extractors for custom processing of the specific syslog messages using the Drools Rule File in Graylog2. You can read more in Graylog2's help file "Custom message rewriting/processing."

To implement the drl file, edit your graylog2.conf file and uncomment the following line, making sure that it points to your drl file:rules_file = /etc/graylog2.drl

Then create your drl file with the rules that you need. Below are the extractors that I use for FortiGate. These are most of the fields that I've run across in the messages that seemed to be of value to me. If there are others, then follow the template and add them. Please notice that some of the fields (like action) don't have quotes around the variable, but others (like app) do, and there are different patterns to match them. There are also a couple of exceptionones (service, status, and vd) that sometimes have quotes and sometimes don't have quotes, so I've got rules in there to take both into account.

The only one that I can't seem to get working correctly is msg one that is supposed to replace the "message" field with the contents of "msg." I'll update here if I can get it working.

To implement the drl file, restart your graylog2-server service, and watch the logfile to make sure that there are no errors on start up.

# Graylog2 Extractors
import org.graylog2.plugin.Message
import java.util.regex.Matcher
import java.util.regex.Pattern
import java.text.DateFormat
import java.text.ParseException

# Fortigate
rule "Fortigate source rewrite"
when
    m : Message ( message matches ".+devname=.+\\sdevid=.+" )
then
    Matcher matcher = Pattern.compile("^.+\\sdevname=(\\S+)\\s").matcher(m.getMessage());
    if (matcher.find()) {
      m.addField("source", matcher.group(1));
    }

    Matcher action = Pattern.compile("^.+\\saction=(\\S+)\\s").matcher(m.getMessage());
    if (action.find()) {
      m.addField("action", action.group(1));
    }

    Matcher app = Pattern.compile("^.+\\sapp=\"(\\S+)\"").matcher(m.getMessage());
    if (app.find()) {
      m.addField("app", app.group(1));
    }

    Matcher appact = Pattern.compile("^.+\\sappact=(\\S+)\\s").matcher(m.getMessage());
    if (appact.find()) {
      m.addField("appact", appact.group(1));
    }

    Matcher appcat = Pattern.compile("^.+\\sappcat=\"(\\S+)\"").matcher(m.getMessage());
    if (appcat.find()) {
      m.addField("appcat", appcat.group(1));
    }

    Matcher applist = Pattern.compile("^.+\\sapplist=\"(\\S+)\"").matcher(m.getMessage());
    if (applist.find()) {
      m.addField("applist", applist.group(1));
    }

    Matcher attack = Pattern.compile("^.+\\sattack=\"(\\S+)\"").matcher(m.getMessage());
    if (attack.find()) {
      m.addField("attack", attack.group(1));
    }

    Matcher devid = Pattern.compile("^.+\\sdevid=(\\S+)\\s").matcher(m.getMessage());
    if (devid.find()) {
      m.addField("devid", devid.group(1));
    }

    Matcher dir = Pattern.compile("^.+\\sdir=(\\S+)\\s").matcher(m.getMessage());
    if (dir.find()) {
      m.addField("dir", dir.group(1));
    }

    Matcher dstcountry = Pattern.compile("^.+\\sdstcountry=\"(\\S+)\"").matcher(m.getMessage());
    if (dstcountry.find()) {
      m.addField("dstcountry", dstcountry.group(1));
    }

    Matcher dstintf = Pattern.compile("^.+\\sdstintf=\"(\\S+)\"").matcher(m.getMessage());
    if (dstintf.find()) {
      m.addField("dstintf", dstintf.group(1));
    }

    Matcher dstip = Pattern.compile("^.+\\sdstip=(\\S+)\\s").matcher(m.getMessage());
    if (dstip.find()) {
      m.addField("dstip", dstip.group(1));
    }

     Matcher dtype = Pattern.compile("^.+\\sdtype=\"(\\S+)\"").matcher(m.getMessage());
    if (dtype.find()) {
      m.addField("dtype", dtype.group(1));
    }

    Matcher duration = Pattern.compile("^.+\\sduration=(\\S+)\\s").matcher(m.getMessage());
    if (duration.find()) {
      m.addField("duration", duration.group(1));
    }

    Matcher error_reason = Pattern.compile("^.+\\serror_reason=\"(\\S+)\"").matcher(m.getMessage());
    if (error_reason.find()) {
      m.addField("error_reason", error_reason.group(1));
    }

    Matcher eventtype = Pattern.compile("^.+\\seventtype=(\\S+)\\s").matcher(m.getMessage());
    if (eventtype.find()) {
      m.addField("eventtype", eventtype.group(1));
    }

     Matcher file = Pattern.compile("^.+\\sfile=\"(\\S+)\"").matcher(m.getMessage());
    if (file.find()) {
      m.addField("file", file.group(1));
    }

    Matcher group = Pattern.compile("^.+\\sgroup=\"(\\S+)\"").matcher(m.getMessage());
    if (group.find()) {
      m.addField("group", group.group(1));
    }

    Matcher hostname = Pattern.compile("^.+\\shostname=\"(\\S+)\"").matcher(m.getMessage());
    if (hostname.find()) {
      m.addField("hostname", hostname.group(1));
    }

    Matcher identidx = Pattern.compile("^.+\\sidentidx=(\\S+)\\s").matcher(m.getMessage());
    if (identidx.find()) {
      m.addField("identidx", identidx.group(1));
    }

    Matcher init = Pattern.compile("^.+\\sinit=(\\S+)\\s").matcher(m.getMessage());
    if (init.find()) {
      m.addField("init", init.group(1));
    }

    Matcher locip = Pattern.compile("^.+\\slocip=(\\S+)\\s").matcher(m.getMessage());
    if (locip.find()) {
      m.addField("locip", locip.group(1));
    }

    Matcher locport = Pattern.compile("^.+\\slocport=(\\S+)\\s").matcher(m.getMessage());
    if (locport.find()) {
      m.addField("locport", locport.group(1));
    }

    Matcher logid = Pattern.compile("^.+\\slogid=(\\S+)\\s").matcher(m.getMessage());
    if (logid.find()) {
      m.addField("logid", logid.group(1));
    }

    Matcher mode = Pattern.compile("^.+\\smode=(\\S+)\\s").matcher(m.getMessage());
    if (mode.find()) {
      m.addField("mode", mode.group(1));
    }

    Matcher msg = Pattern.compile("^.+\\smsg=\"(\\S+)\"").matcher(m.getMessage());
    if (msg.find()) {
      m.addField("message", msg.group(1));
    }

    Matcher outintf = Pattern.compile("^.+\\soutintf=\"(\\S+)\"").matcher(m.getMessage());
    if (outintf.find()) {
      m.addField("outintf", outintf.group(1));
    }

    Matcher peer_notif = Pattern.compile("^.+\\speer_notif=\"(\\S+)\"").matcher(m.getMessage());
    if (peer_notif.find()) {
      m.addField("peer_notif", peer_notif.group(1));
    }

    Matcher policyid = Pattern.compile("^.+\\spolicyid=(\\S+)\\s").matcher(m.getMessage());
    if (policyid.find()) {
      m.addField("policyid", policyid.group(1));
    }

    Matcher profile = Pattern.compile("^.+\\sprofile=\"(\\S+)\"").matcher(m.getMessage());
    if (profile.find()) {
      m.addField("profile", profile.group(1));
    }

    Matcher profiletype = Pattern.compile("^.+\\sprofiletype=\"(\\S+)\"").matcher(m.getMessage());
    if (profiletype.find()) {
      m.addField("profiletype", profiletype.group(1));
    }

    Matcher proto = Pattern.compile("^.+\\sproto=(\\S+)\\s").matcher(m.getMessage());
    if (proto.find()) {
      m.addField("proto", proto.group(1));
    }

     Matcher quarskip = Pattern.compile("^.+\\squarskip=\"(\\S+)\"").matcher(m.getMessage());
    if (quarskip.find()) {
      m.addField("quarskip", quarskip.group(1));
    }

    Matcher rcvdbyte = Pattern.compile("^.+\\srcvdbyte=(\\S+)\\s").matcher(m.getMessage());
    if (rcvdbyte.find()) {
      m.addField("rcvdbyte", rcvdbyte.group(1));
    }

    Matcher rcvdpkt = Pattern.compile("^.+\\srcvdpkt=(\\S+)\\s").matcher(m.getMessage());
    if (rcvdpkt.find()) {
      m.addField("rcvdpkt", rcvdpkt.group(1));
    }

    Matcher ref = Pattern.compile("^.+\\sref=\"(\\S+)\"").matcher(m.getMessage());
    if (ref.find()) {
      m.addField("ref", ref.group(1));
    }

    Matcher remip = Pattern.compile("^.+\\sremip=(\\S+)\\s").matcher(m.getMessage());
    if (remip.find()) {
      m.addField("remip", remip.group(1));
    }

    Matcher remport = Pattern.compile("^.+\\sremport=(\\S+)\\s").matcher(m.getMessage());
    if (remport.find()) {
      m.addField("remport", remport.group(1));
    }

    Matcher result = Pattern.compile("^.+\\sresult=(\\S+)\\s").matcher(m.getMessage());
    if (result.find()) {
      m.addField("result", result.group(1));
    }

    Matcher role = Pattern.compile("^.+\\srole=(\\S+)\\s").matcher(m.getMessage());
    if (role.find()) {
      m.addField("role", role.group(1));
    }

    Matcher sentbyte = Pattern.compile("^.+\\ssentbyte=(\\S+)\\s").matcher(m.getMessage());
    if (sentbyte.find()) {
      m.addField("sentbyte", sentbyte.group(1));
    }

    Matcher sentpkt = Pattern.compile("^.+\\ssentpkt=(\\S+)\\s").matcher(m.getMessage());
    if (sentpkt.find()) {
      m.addField("sentpkt", sentpkt.group(1));
    }

    Matcher service = Pattern.compile("^.+\\sservice=\"(\\S+)\"").matcher(m.getMessage());
    if (service.find()) {
      m.addField("service", service.group(1));
    } else {
      Matcher servicenq = Pattern.compile("^.+\\sservice=(\\S+)\\s").matcher(m.getMessage());
      if (servicenq.find()) {
        m.addField("service", servicenq.group(1));
      }
    }

    Matcher srccountry = Pattern.compile("^.+\\ssrccountry=\"(\\S+)\"").matcher(m.getMessage());
    if (srccountry.find()) {
      m.addField("srccountry", srccountry.group(1));
    }

    Matcher srcintf = Pattern.compile("^.+\\ssrcintf=\"(\\S+)\"").matcher(m.getMessage());
    if (srcintf.find()) {
      m.addField("srcintf", srcintf.group(1));
    }

    Matcher srcip = Pattern.compile("^.+\\ssrcip=(\\S+)\\s").matcher(m.getMessage());
    if (srcip.find()) {
      m.addField("srcip", srcip.group(1));
    }

    Matcher srcport = Pattern.compile("^.+\\ssrcport=(\\S+)\\s").matcher(m.getMessage());
    if (srcport.find()) {
      m.addField("srcport", srcport.group(1));
    }

    Matcher stage = Pattern.compile("^.+\\sstage=(\\S+)\\s").matcher(m.getMessage());
    if (stage.find()) {
      m.addField("stage", stage.group(1));
    }

    Matcher status = Pattern.compile("^.+\\sstatus=\"(\\S+)\"").matcher(m.getMessage());
    if (status.find()) {
      m.addField("status", status.group(1));
    } else {
      Matcher statusnq = Pattern.compile("^.+\\sstatus=(\\S+)\\s").matcher(m.getMessage());
      if (statusnq.find()) {
        m.addField("status", statusnq.group(1));
      }
    }

    Matcher subtype = Pattern.compile("^.+\\ssubtype=(\\S+)\\s").matcher(m.getMessage());
    if (subtype.find()) {
      m.addField("subtype", subtype.group(1));
    }

    Matcher transport = Pattern.compile("^.+\\stransport=(\\S+)\\s").matcher(m.getMessage());
    if (transport.find()) {
      m.addField("transport", transport.group(1));
    }

    Matcher type = Pattern.compile("^.+\\stype=(\\S+)\\s").matcher(m.getMessage());
    if (type.find()) {
      m.addField("type", type.group(1));
    }

    Matcher trandisp = Pattern.compile("^.+\\strandisp=(\\S+)\\s").matcher(m.getMessage());
    if (trandisp.find()) {
      m.addField("trandisp", trandisp.group(1));
    }

    Matcher transip = Pattern.compile("^.+\\stransip=(\\S+)\\s").matcher(m.getMessage());
    if (transip.find()) {
      m.addField("transip", transip.group(1));
    }

    Matcher user = Pattern.compile("^.+\\suser=\"(\\S+)\"").matcher(m.getMessage());
    if (user.find()) {
      m.addField("user", user.group(1));
    }

    Matcher utmaction = Pattern.compile("^.+\\sutmaction=(\\S+)\\s").matcher(m.getMessage());
    if (utmaction.find()) {
      m.addField("utmaction", utmaction.group(1));
    }

    Matcher utmevent = Pattern.compile("^.+\\sutmevent=(\\S+)\\s").matcher(m.getMessage());
    if (utmevent.find()) {
      m.addField("utmevent", utmevent.group(1));
    }

    Matcher vd = Pattern.compile("^.+\\svd=\"(\\S+)\"").matcher(m.getMessage());
    if (vd.find()) {
      m.addField("vd", vd.group(1));
    } else {
      Matcher vdnq = Pattern.compile("^.+\\svd=(\\S+)\\s").matcher(m.getMessage());
      if (vdnq.find()) {
        m.addField("vd", vdnq.group(1));
      }
    }

    Matcher virus = Pattern.compile("^.+\\svirus=\"(\\S+)\"").matcher(m.getMessage());
    if (virus.find()) {
      m.addField("virus", virus.group(1));
    }

    Matcher vpntunnel = Pattern.compile("^.+\\svpntunnel=\"(\\S+)\"").matcher(m.getMessage());
    if (vpntunnel.find()) {
      m.addField("vpntunnel", vpntunnel.group(1));
    }

    Matcher xauthgroup = Pattern.compile("^.+\\sxauthgroup=\"(\\S+)\"").matcher(m.getMessage());
    if (xauthgroup.find()) {
      m.addField("xauthgroup", xauthgroup.group(1));
    }

    Matcher xauthuser = Pattern.compile("^.+\\sxauthuser=\"(\\S+)\"").matcher(m.getMessage());
    if (xauthuser.find()) {
      m.addField("xauthuser", xauthuser.group(1));
    }

end

------
Dustin Shaw
VCP

Install ElasticSearch 0.90.10 on CentOS 6

2014-04-23T16:01:00.003-05:00

Graylog2 0.20.x requires ElasticSearch 0.90.10. To fullfil this requirement, you will need to manually download and install the RPM for ElasticSearch.

Download ElasticSearch 0.90.10 from the ElasticSearch Downloads page here.

Save the file and upload it to your CentOS 6 server.

Install Java 1.7:
#yum install java-1.7.0-openjdk.x86_64

Install the RPM:
#rpm -ivh elasticsearch-0.90.10.noarch.rpm

Stop the elasticsearch service so that we can update the cluster name:
#service elasticsearch stop

Edit the /etc/elasticsearch/elasticsearch.yml file to update your cluster.name variable. Ex:
cluster.name: graylog2_production

Update any additional settings needed and save the file. I recommend updating the path.data and path.logs to custom directories.

Start the elasticsearch service and set it to run on startup:
#service elasticsearch start
#chkconfig elasticsearch on

Check your logs to make sure that it started properly and joined the cluster (if there is an existing one).

For Graylog2, the recommended settings are also to increase the open file limit to at least 64000 as seen in the Configuring and tuning ElasticSearch for Graylog2 >v0.20.0 documentation. I did this by increasing the max number of ulimit open file below.

Edit /etc/sysctl.conf and add the following line at the end:
fs.file-max = 65536

Save the file. Next edit /etc/security/limits.conf and add the following lines:
*               soft    nproc           65535
*               hard    nproc           65535
*               soft    nofile          65535
*               hard    nofile          65535

Save the file and restart the server.
#shutdown -r now

Once restarted, verify that the max open file ulimit has been increased.
# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30507
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 65535
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Additional recommended settings are to increase the ES_HEAP_SIZE. I did this by editing /etc/init.d/elasticsearch and adding the following line after checkJava under start():
ES_HEAP_SIZE=2g

They recommend that you leave 50% of your memory for other system functions, and I had 4 Gig of RAM, hence the 2g setting.

------
Dustin Shaw
VCP

Ship RHEL 6 or CentOS6 syslogs

2014-04-23T14:50:00.001-05:00

Reference more for myself than for y'all, but you get the shared benefit.

To ship the logs from RHEL6 or Centos 6 to a remote syslog server, edit the following file: /etc/rsyslog.conf

At the bottom of the file, remove the comment from the remote-host entry, and update with your server name or IP. Example:
*.* @@192.168.40.15:514

Restart the rsyslog service:
service rsyslog restart

Make sure you are receiving the logs at your syslog server.

------
Dustin Shaw
VCP

Install VMware Tools on RHEL 6 or CentOS 6

2014-04-23T13:31:00.002-05:00

Below is the process to install VMware Tools on RHEL 6 or CentOS 6. This guide is more here for me than anyone else, but I hope that you can benefit from it.

Install the Pre-Requisites:
yum install make gcc kernel-devel kernel-headers glibc-headers perl

Start the VMware Tools installation process on your VM:

Mount the VMware Tools installation media:
mkdir /mnt/cd
mount /dev/cdrom /mnt/cd
Expected warning:
mount: block device /dev/sr0 is write-protected, mounting read-only

Extract the installer:
cp /mnt/cd/VMwareTools-9.0.10-1481436.tar.gz /tmp/
umount /mnt/cd
cd /tmp
tar xvf VMwareTools-9.0.10-1481436.tar.gz
cd vmware-tools-distrib/

Install tools (accepting all defaults):
sudo ./vmware-install.pl -d

Reboot the VM to verify that the service starts up automatically as expected.
shutdown -r now

------
Dustin Shaw
VCP

Extend a volume in RedHat 6 or CentOS 6

2013-09-30T09:57:00.003-05:00

To extend a partition on a VM on CentOS 6 or RHEL 6, do the following:

Extend the Partition in vSphere to the size that you need:

in the VM, refresh the Partition Tables by the following command:

echo 1 > /sys/block/sda/device/rescan
Run fdisk to create a new primary partition:

fdisk /dev/sda

Press p to print the number of partitions
Press n to create a new primary patition
Press p for primary
The partition number should automatically be selected, and press enter twice to accept the beginning and ending blocks on the free space.
Press w to write the changes.

Reboot the system: shutdown -r now

Verify that the new partition exists:
fdisk -l

Convert the new partition to a physical volume:
pvcreate /dev/sda3
Extend the Physical Volume (run vgdisplay to obtain the name of the Volume Group; default is VolGroup00):
vgextend VolGroup00 /dev/sda3

Extend the Logical Volume (run vgdisplay to obtain the free space and replace # below, and lvdisplay to obtain the name of the Logical Volume; default is /dev/VolGroup00/LogVol00)
lvextend -L+#G /dev/VolGroup00/LogVol00

Expand the ext3 online using the following command (substituting your proper Volume Group and Logical Volume names):
resize2fs /dev/VolGroup00/LogVol00

Confirm that your new space is available:
df -h

This was written using VMware's KB article on the subject as a reference.

------
Dustin Shaw
VCP

vRanger 5 and ESXi Performance Settings

2012-07-19T08:27:00.002-05:00

Everyone should know that the previous recommendations from Quest were to increase the CPU Reservations to 1500 on ESX servers for the best performance from vRanger.

I just recently ran across an article (here) from Sept 2011 detailing that this is NOT the case for ESXi. There is no performance gain on ESXi by increasing the CPU reservations from the default 293.

------
Dustin Shaw
VCP

vRanger error "There is an error in XML document"

2012-07-19T08:10:00.000-05:00

I ran into an issue with one of our repositories - every job in the particular repository was failing with the error:

An internal error occurred during execution, please contact Quest support if the error persists. Error Message: There is an error in XML document (823, 223).

This error is commonly caused by a corruption or issue with the manifest file as detailed here.

To fix it, go into the manifest file for the repository (GlobalManifest.metadata), make sure that all trailing brackets are closed, and add </vRangerGlobalManifestFile> at the end of the document.

Save and test.

------

Dustin Shaw
VCP

vCenter Server 5 Service Fails

2012-05-24T07:02:00.000-05:00

We had an issue with the vCenter Server 5 Service failing recently. Basically what happened was the VMware VirtualCenter Server service failed (out of the blue) with the following Informational Event ID 1000 logged in the Application log:

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Starting VMware VirtualCenter 5.0.0 build-623373

the message resource is present but the message is not found in the string/message table

Followed by another Info Event 1000:

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Log directory: C:\ProgramData\VMware\VMware VirtualCenter\Logs.

the message resource is present but the message is not found in the string/message table

And followed by an Error Event 1000 (upon it attempting to auto-restart the service):

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Failed to intialize VMware VirtualCenter. Shutting down...

the message resource is present but the message is not found in the string/message table

First try was to restart all the vCenter services - only the VMware VirtualCenter Server service was offline; all other services (like VMware VirtualCenter Management Webservices) were still online. This obviously failed with the same error as before.

All of the SQL services were verified online (these are kept on a separate server due to the size), and accessible. I ran across a similar Discussion thread in VMware Communities, which pointed to issues inside of the SQL Database. Based off of this, I decided the first order was to look at my SQL server to see what was going on there. I was able to log in, and verified that everything was up. Then I looked at my logs, and saw Event ID 17053:

C:\VCDB.ldf: Operating system error 112(There is not enough space on the disk.) encountered.

And instantly I knew my problem. Some yahoo (namely the yahoo writing this article) must not've been paying attention when installing the VCDB database, and stuck it in the root of the C drive. Naturally, that yahoo had to fix his own problem...

So I went into Microsoft SQL Server Management Studio, and ran the following command:

alter database VCDB modify file ( name = vcdb , filename = 'E:\SQL\MDF\VCDB.mdf' )
alter database VCDB modify file ( name = vcdb_log , filename = 'E:\SQL\LDF\VCDB.ldf')
go

Then I Offlined the database (Right click, Tasks, Take Offline), and moved the files to their new homes. Then I Onlined the database, and verified that the path was correct for it. You can see the full process of how to move a SQL database here.

Once this was done, I went back to my vCenter Server, and was able to bring all my services online without incident, and was able to again go in and manage my vCenter 5 Server via vSphere Client.

------
Dustin Shaw
VCP

WNLB and VMware

2012-03-13T06:22:00.000-05:00

So I believe I've found an (known) issue with 2003 Windows Network Load Balancing and VMware.
VMware reports that WNLB on Windows 2003 Servers does not behave as expected here. Basically the article says the the NLB will point to one of the servers, not all of them, when running in unicast. They give you two fixes: use multicast; or reconfigure your Port Groups (or vSwitches) to prevent RARP Packet Transmissions. Interesting thing, with the current environment these servers are hosted in, I am unable to either run multicast or disable Switch Notify. I'll have to take that up with the Network Team, or perhaps investigate some NLB hardware.

So here's where I'm left with - these servers are not supposed to go down (hence the NLB), but they are everyday at 4AM (fun call). This is when backups are running, so I'm adjusting the time to see if the issue follows.

What appears to be happening is that when snapshots are taken for backups, the NLB seems to freak out at the one dropped ping. Currently the backups all run at once, which makes them all hiccup at the same time, killing the NLB for a good reported 45 minutes (no idea why so long). If the issue follows the backups, perhaps staggering the backups might solve the problem (let the NLB roll from one server to another).

I'll keep you posted on what I find.
------
Dustin Shaw
VCP

Host update fails after updating to 4.1u2

2012-02-23T08:39:00.000-06:00

After updating vCenter Update Manager (right after updating my vCenter Server) from 4.1u1 to 4.1u2, I received the following error when trying to update on of my hosts:

Remediation did not succeed for esxihost: SingleHostRemediate: esxupdate error, version: 1.30, operation: 7: ('http://esxihost:9084/vci/hostupdates/hostupdate/vmw/vibs/cross_oem-vmware-esx-drivers-net-vxge_400.2.0.28.21239-1OEM.vib','/var/tmp/cache/-1699692350','[Errno 14] HTTP Error 404: Not Found')

After doing some research, I discovered what happened is that Update Manager 4.1u2 is case sensitive, whereas 4.1u1 was not. Any patches that were downloaded previous to the 4.1u2 update that contain upper case letters will fail with this error. VMware has a KB article about it here.

The patches affected are the ones below. I've put the correct capitalization on them for you. If you go to your patch repository, you can rename the files that you have to the below, and you should then be able to update your hosts. The repositories are located here:

Windows2008: C:\ProgramData\VMware\VMware Update Manager\Data\Hostupdate\vmw\vib\
Windows2003: C:\Documents and Settings\All Users\Application Data\VMware\VMware Update Manager\Data\Hostupdate\vmw\vib

bind-libs-9.3.6-4.P1.el5_5.3.i386.vib
bind-libs-9.3.6-4.P1.el5_5.3.x86_64.vib
bind-utils-9.3.6-4.P1.el5_5.3.x86_64.vib
bind-libs-9.3.6-4.P1.el5_5.3.i386.vib
cross_oem-vmware-esx-drivers-net-vxge_400.2.0.28.21239-1OEM.vib
cross_oem-vmware-esx-drivers-scsi-3w-9xxx_400.2.26.08.036vm40-1OEM.vib
vmware-esx_swMgmt_provider-4x.1.0.1-1.4.348481.vib

------
Dustin Shaw
VCP

Syslog not configured on ESXi 4.1u2

2012-02-22T11:27:00.000-06:00

When I updated my ESXi host from 4.1u1 to 4.1u2, I got the following error message:

Configuration Issues
Issue detected on esxihost in datacenter: Warning: Syslog not configured. Please check Syslog options under Configuration.Software.Advanced Settings in vSphere client.

I thought it was odd since the host previously never complained about the Syslog before. I compared the settings between it and the other 4.1u2 hosts that I had, and indeed, it was missing the Syslog.Local.DatastorePath setting.

The setting on my hosts was:
[] /scratch/log/messages

Once I copied this into my Syslog.Local.DatastorePath setting on the server, it was happy. I went ahead and copied the setting to my remaining 4.1u1 servers so that they will be happy when updated as well.

So apparently ESXi 4.1u2 has issues with the Syslog running on ramdisk, but ESXi 4.1u1 doesn't.

I found the following VMware KB Article that explains why it complains about it:
Syslog not configured messages on ESXi host console or in logs

------
Dustin Shaw
VCP