T3CHNOT3S

How I Used AI to Audit My pfSense HA Firewall Pair – Part 2

2026-06-12T12:47:09.703-07:00

How I Used AI to Audit My pfSense HA Firewall Pair – Part 2

The Follow-Up Session: Syslog Crashes, ARP Conflicts, and a Mystery Device

Session Dates: May 30 – June 2, 2026

Introduction

In Part 1, I connected Claude Code to my pfSense Community Edition high-availability firewall pair and used it to perform a full infrastructure audit. The session uncovered stale firewall rules, an Unbound DNS deadlock, and a hidden DNS dependency between HA nodes. It also answered a question I had going into the project: could an AI assistant perform a meaningful infrastructure review? The answer was yes.

After publishing Part 1, I kept using the same Claude Code + MCP workflow. What followed was not another audit. Instead, it became a series of investigations into issues hiding in plain sight. Some had existed quietly for years. Others pointed in one direction while the root cause lived somewhere completely different.

This article covers several findings from those follow-up sessions, including a syslog daemon crash loop, a mystery device appearing in ARP tables, and an IP conflict involving a consumer appliance impersonating a production server.

Part 1 vs Part 2

Part 1 focused on auditing configuration.

Part 2 focuses on understanding behavior.

Audits answer questions like:

Is this rule still needed?
Why is this interface configured this way?
Are these settings still appropriate?

Investigations answer different questions:

Why does this service keep crashing?
Why does this system occasionally disappear?
Why does everything look healthy even when it isn’t?

As it turns out, those questions led to some interesting places.

Finding #1 – The Syslog Crash Nobody Noticed

The first clue came from centralized logging.

Both firewall nodes appeared healthy. CARP was stable. DNS was stable. Firewall processing was normal. Nothing obvious pointed toward a problem.

Yet syslog delivery would occasionally stop after boot.

At first glance, the issue looked like a Graylog problem. Logs would stop flowing, eventually resume, and then behave normally until the next reboot. Nothing in the firewall dashboard suggested a failure.

A closer look revealed something unexpected.

The syslog daemon (syslogd) on both pfSense nodes was crashing approximately two minutes after every reboot.

Even more interesting, it eventually recovered on its own.

Because recovery occurred automatically, the issue never stayed visible long enough to attract attention. The environment continued operating normally most of the time, masking the underlying problem.

Root Cause

Claude Code began reviewing startup behavior and syslog configuration.

The culprit turned out to be an obscure pfSense logging option called logall.

When enabled, pfSense inserts a catch-all forwarding rule in addition to individual facility-based forwarding rules already configured for remote logging.

Effectively, every log message was forwarded twice.

Under normal conditions, the duplication wasn’t obvious.

Under heavy pfBlockerNG DNS logging volume, the doubled UDP stream combined with pfSense’s boot-time service restart sequence created a race condition which consistently crashed syslogd.

The result:

Duplicate log traffic
Unnecessary network load
A predictable service crash after every reboot

The fix was straightforward:

Disable logall
Re-enable explicit facility forwarding
Validate remote logging

Immediately afterward, duplicate logging disappeared and syslogd remained stable through multiple reboots.

The central log server saw roughly a fifty percent reduction in incoming UDP traffic.

Sometimes the hardest problems are not caused by broken components.

Sometimes a component does exactly what it was configured to do.

Finding #2 – The Mystery Device

While reviewing logs and network activity, Claude Code identified an unknown device appearing periodically in ARP tables.

The device had:

No DHCP lease
Minimal network activity
No obvious hostname
No open ports worth investigating

At first glance, it looked like an abandoned static-IP device quietly sitting on the network.

OUI analysis told a different story.

The hardware vendor belonged to a major consumer smart-home appliance manufacturer.

Interesting, but not particularly useful.

Then the IP address entered the picture.

The device was using the same IP assigned to the centralized syslog server.

Things got much more interesting.

Finding #3 – The ARP Conflict

Further investigation revealed a surprisingly simple problem.

Every time the appliance booted, it announced ownership of the syslog server address through gratuitous ARP updates.

Those announcements occasionally replaced the legitimate ARP entry maintained by the firewall.

For brief periods, traffic intended for the syslog server was redirected toward the appliance instead.

The symptom appeared as intermittent logging failures.

The root cause turned out to be an IP conflict involving a device most people would never suspect.

Once the connection became clear, the fix was straightforward.

Resolution

Two complementary controls were implemented:

Static DHCP mapping for the appliance
Static ARP enforcement for both the appliance and the syslog server

Together, those changes permanently anchored each device to the correct address.

ARP instability disappeared immediately.

This investigation was particularly satisfying because the original symptom involved logging.

The root cause involved a consumer appliance quietly impersonating a production server.

Without tracing relationships between systems, the connection would have been easy to miss.

Finding #4 – A Filesystem Check Years Overdue

One item carried forward from the previous session was a full offline filesystem check.

This wasn’t exciting work.

No outage.

No mysterious error.

No dramatic root cause.

Just maintenance.

Both nodes were rebooted into single-user mode and subjected to a complete fsck validation.

The result was exactly what I hoped to find:

No filesystem corruption
No pending repairs
No inconsistencies

Sometimes confirmation is the outcome.

Not every investigation needs to discover something broken.

A Story for Another Day

Not every investigation from this session made it into this article.

One ongoing effort began with a routine health check on downstream infrastructure and eventually led to discovery of an undocumented routing dependency involving the pfSense HA pair.

The findings deserve their own write-up.

I’ll cover this story separately once the work is complete.

What Claude Code Did Not Do

One topic generated a fair amount of discussion after Part 1 was published.

How much of this work was actually performed by Claude Code?

The answer is both more and less than some people expect.

Human Approval Required – Claude Code did not autonomously change anything. Every finding came with a recommendation. Every recommendation waited for explicit approval before implementation. The session was collaborative by design: Claude Code as analyst, operator as decision-maker.

No Understanding of Business Intent – Claude Code did not understand business intent behind every rule. It could identify technically redundant rules, questionable configurations, or opportunities for cleanup. It could not determine whether a rule existed because of a contractual requirement, a legacy integration, or a deliberate architectural decision. Technical analysis and business context are not the same thing.

Validation and Testing Still Matter – Claude Code did not replace validation or testing. Changes were reviewed before implementation. CARP health was checked afterward. pfsync health was checked afterward. Services were validated afterward. Finding a problem and safely correcting a problem remain two different skills.

Architectural Context Required Human Knowledge – Claude Code did not identify architectural context without guidance. Several findings required institutional knowledge unavailable to the AI. Understanding why something exists often matters more than understanding how it works.

Deferred Items Still Required Operator Judgment – Several items remain open. Not because they were missed. Not because the AI failed to identify them. Because the correct solution requires planning, scheduling, additional data, or a maintenance window. Knowing when not to make a change is just as important as knowing what to change.

Takeaways

Part 1 proved AI could perform a meaningful infrastructure audit.

Part 2 demonstrated something different.

AI can also be an effective investigation partner.

The most valuable discoveries from this session were not firewall rules or package updates. They were hidden relationships between systems:

A logging option causing service crashes
A consumer appliance interfering with a server
Infrastructure components quietly compensating for one another
Operational issues hidden because everything appeared healthy

What impressed me most was not Claude Code’s ability to suggest fixes. It was its ability to keep asking questions.

Why is this service crashing?
Why does this device exist?
Why does this behavior occur?

All of this is where I continue to find value in AI-assisted operations - by not replacing expertise, not replacing judgment. Not replacing accountability. Instead, accelerating investigation and surfacing relationships humans often overlook when systems appear healthy.

The firewall pair is healthier than it was at the start of the session.

More importantly, I understand it better than before.

Generated with Claude Code (Anthropic) — June 2026
Reviewed and approved by Dean Flaming – June 2026

How I Used AI to Audit My pfSense HA Firewall Pair - Part 1

2026-05-26T16:28:34.836-07:00

How I Used AI to Audit My pfSense HA Firewall Pair

A Real-World Claude Code + MCP Integration Story

Session Date: May 25–26, 2026

Published by Dean Flaming

Introduction

I’ve been running a pfSense CE high-availability firewall pair at home for years. When Anthropic released Claude Code with MCP (Model Context Protocol) support, I wondered: could I point an AI assistant directly at my firewalls and have it conduct a real security audit? The answer turned out to be yes — and the process surfaced issues I hadn’t noticed in years of hands-on management, plus two unexpected infrastructure crises which made the session much more interesting than anticipated.

This post walks through the full story: setting up the integration, running the audit, and everything going sideways along the way. All environment-specific values (IPs, hostnames, domain names) have been replaced with generic examples so you can adapt this to your own setup.

Notes

One housekeeping note before diving in: everything in this post is real — the commands, the findings, the detours, the fixes. What isn't real are the specifics. All IP addresses, hostnames, domain names, and usernames have been replaced with generic examples to keep the actual environment private. The story is accurate; the “names” (and IPs) have been changed “to protect the innocent.” 😉

Important: This session was performed against a home lab environment with full backups and HA failover available. Direct AI access to infrastructure should always begin with read-only permissions and staged validation before allowing configuration changes in production environments.

Part 1: Wiring Claude Code to pfSense

Before any analysis could happen, I needed to give Claude Code a way to reach the firewalls. The integration uses two channels: SSH for interactive shell access and raw diagnostics, and the pfSense REST API for structured read/write operations. An MCP server running locally on my Mac bridges Claude Code to both.

Integration Architecture

Claude Code runs on macOS and communicates with pfSense through two channels:

SSH (via MCP SSH server) — interactive shell access, pfctl commands, log inspection, low-level diagnostics

pfSense REST API (pfSense-pkg-RESTAPI) — structured read/write access to firewall rules, aliases, NAT, interfaces, and packages

The MCP server runs as a local process on the Mac and exposes SSH operations as typed tools Claude Code can invoke during a session. No agent or daemon is required on pfSense itself.

Step 1 — macOS Prerequisites

The MCP SSH server bridging Claude Code to pfSense is a Node.js package — Node.js is the runtime it needs to execute. This walkthrough is written from a macOS + Terminal perspective, but the same integration works on other platforms with minor path differences. On Windows, npm installs globals to a path under %APPDATA% rather than /opt/homebrew/bin, and you’d use PowerShell or WSL instead of Terminal. On Linux, the paths follow your distro’s package manager conventions. The commands themselves are the same; only the install locations differ.

1a. Node.js and npm

The MCP SSH server requires Node.js. Verify both are present:

node --version

npm --version

If not installed, use Homebrew:

brew install node

Note: This setup uses Apple Silicon (M-series Mac). The Homebrew prefix is /opt/homebrew rather than /usr/local. All paths below reflect this.

1b. Install the MCP SSH Manager

npm install -g @iflow-mcp/mcp-ssh-manager

On Apple Silicon the binary installs to /opt/homebrew/bin/ as mcp-ssh-manager. Confirm:

npm root -g

ls /opt/homebrew/bin/ | grep ssh

1c. Create the Project Directory

All session files live in a single project directory. Claude Code auto-loads .mcp.json and .claude/settings.local.json when launched from this directory:

cd "~/projects/pfsense-tuneup"

Note: Finder blocks creation of dot-prefixed folders (e.g. .claude) with an error. Use Terminal instead: mkdir -p .claude

Step 2 — Project Configuration Files

Claude Code is project-aware. When you launch it from a directory, it automatically reads a small set of configuration files from the directory if they exist. These files are how you tell Claude Code which tools to connect to, which operations are permitted, and what context to carry into the session. Without them, Claude Code starts with no knowledge of your environment and no way to reach pfSense. With them, every session picks up right where you left off — no re-explaining, no re-wiring.

Three files are involved: .mcp.json tells Claude Code which external servers (tools) to load. settings.local.json defines a permission boundary — what Claude Code is and isn’t allowed to do with your system. CLAUDE.md is a plain text file giving Claude Code persistent context about your environment: node roles, credentials, IP addresses, rules of engagement. Think of CLAUDE.md as a briefing document Claude reads at the start of every session.

2a. .mcp.json

Registers the pfSense SSH MCP server. Auto-loaded by Claude Code on startup:

{

"mcpServers": {

"pfsense-ssh": {

"command": "/opt/homebrew/bin/mcp-ssh-manager",

"env": {

"SSH_CONFIG_PATH": "/Users/localuser/.claude/pfsense-hosts.toml"

}

2b. .claude/settings.local.json

Defines allowed and denied Bash operations. The CARP VIP is explicitly blocked for SSH writes to prevent split-brain if the VIP floats mid-session:

{

"permissions": {

"allow": [

"Bash(ssh * fw1.example.com:*)",

"Bash(ssh * 10.0.1.1:*)",

"Bash(ssh * fw2.example.com:*)",

"Bash(ssh * 10.0.1.2:*)",

"Bash(diff:*)", "Bash(cat:*)", "Bash(grep:*)"

"deny": [

"Bash(ssh * fw.example.com:*)",

"Bash(ssh * 10.0.1.254:*)",

"Bash(rm:*)", "Bash(curl:*)", "Bash(wget:*)"

]

}

2c. ~/.claude/pfsense-hosts.toml

SSH host definitions for the MCP server. Stored in the home .claude/ directory so the MCP daemon finds it regardless of working directory:

[hosts.pf1]

host = "fw1.example.com"

user = "pfadmin"

key = "/Users/localuser/.ssh/pfsense_ed25519"

alias = "pf1"

notes = "Primary / CARP master"

[hosts.pf2]

host = "fw2.example.com"

user = "pfadmin"

key = "/Users/localuser/.ssh/pfsense_ed25519"

alias = "pf2"

notes = "Secondary / CARP backup"

Step 3 — SSH Key Setup

The MCP server runs unattended — when Claude Code needs to run a command on pfSense, it connects automatically in the background without any human at the keyboard. A password-based connection requires interactive input every time, which breaks automation entirely. An SSH key pair solves this: the private key stays on your Mac and authenticates silently, while the public key lives on pfSense and validates the connection.

Key-based auth is also more secure than passwords for remote access. Once you’ve confirmed keys work, you can disable password auth on pfSense entirely — at which point there’s nothing for a brute-force attack to guess.

The ~/.ssh/config file avoids specifying the key path and connection options on every ssh command. It also handles a pfSense-specific quirk: pfSense 2.8.1 ships with an older OpenSSH advertising key exchange algorithms with an @libssh.org suffix newer macOS SSH clients don’t automatically negotiate. Without the config entry, every connection attempt throws a "no matching key exchange method found" error.

3a. Generate a Passphrase-Free Key

A passphrase-free ed25519 key is required for unattended MCP automation:

ssh-keygen -t ed25519 -C "claude-code-pfsense" -f /Users/localuser/.ssh/pfsense_ed25519 -N ""

Note: The -N "" flag sets an empty passphrase. Without this, the MCP server cannot authenticate unattended. If you prefer a passphrase, use ssh-add --apple-use-keychain on macOS and add UseKeychain yes to ~/.ssh/config.

3b. Add the Public Key to pfSense

Navigate to System → User Manager → [your SSH user] → Edit on each node directly (not via the VIP) and paste the contents of:

cat /Users/localuser/.ssh/pfsense_ed25519.pub

into the Authorized SSH Keys field. Save on each node.

Note: HA config sync (XMLRPC) does not replicate SSH authorized keys. The key must be added manually on both nodes.

3c. Configure macOS SSH Client

Create ~/.ssh/config if it does not exist (touch ~/.ssh/config && chmod 600 ~/.ssh/config), then add:

Host fw1.example.com fw2.example.com

User pfadmin

IdentityFile /Users/localuser/.ssh/pfsense_ed25519

KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256

Note: The @libssh.org suffix on curve25519-sha256 is required. pfSense 2.8.1 uses the suffixed variant. Without it you will get: no matching key exchange method found.

3d. Enable SSH on pfSense

On each node: System → Advanced → Admin Access → Secure Shell:

Enable Secure Shell: checked

SSHd Key Only: Public Key Only (set after confirming key auth works)

SSH Port: 22

Verify no inbound WAN firewall rule exists for port 22 before enabling. pfSense does not create one by default, but confirm under Firewall → Rules → WAN.

3e. Required User Privilege

The SSH user account requires the User — System: Shell account access privilege. Without it, pfSense drops the connection immediately after key authentication. Verify under System → User Manager → [user] → Edit → Effective Privileges.

3f. Login Protection Settings

Tightened on both nodes (System → Advanced → Admin Access → Login Protection):

Threshold	20 (block after 2 failed attempts; most attacks score 10 each)
Blocktime	300 seconds
Detection time	3600 seconds
Pass list	10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24

Note: Internal subnets on the pass list bypass login protection. This prevents Claude Code’s MCP automation from triggering brute-force blocking during rapid-fire SSH operations.

Step 4 — pfSense REST API Package

pfSense does not ship with a REST API. Out of the box, the only programmatic interfaces are the web UI (requiring screen-scraping) and the SSH shell. SSH alone is workable — you can run pfctl commands, read config.xml, and parse output — but it’s fragile. Shell output changes format, XML parsing is error-prone, and there’s no transactional safety net if something goes wrong mid-operation.

It should be noted, the pfSense-pkg-RESTAPI package is a community-maintained, open-source extension and is not part of the default Netgate-supported pfSense CE installation. Use of it adds a proper REST and GraphQL API with over 200 endpoints covering every aspect of pfSense configuration. This gives Claude Code structured, typed access to firewall rules, aliases, NAT entries, interfaces, and packages — the difference between asking "what does this output mean?" and making a direct API call returning clean JSON. It also supports a dry_run parameter, letting Claude Code propose changes and show you exactly what would be modified before anything is applied.

Install on each node independently via SSH. This package is not replicated by XMLRPC sync:

ssh fw1.example.com

pkg-static add https://github.com/pfrest/pfSense-pkg-RESTAPI/releases/latest/download/pfSense-2.8.1-pkg-RESTAPI.pkg

ssh fw2.example.com

pkg-static add https://github.com/pfrest/pfSense-pkg-RESTAPI/releases/latest/download/pfSense-2.8.1-pkg-RESTAPI.pkg

After install: System → REST API for configuration, and:

Swagger docs: https://fw1.example.com:4444/api/v2/docs

REST API base: https://fw1.example.com:4444/api/v2

Step 5 — MCP Server Validation

The MCP server is a local process on your Mac Claude Code spawns at startup. If .mcp.json is misconfigured, the binary path is wrong, or the hosts TOML file can’t be found, the server either fails silently or starts but can’t reach pfSense. Claude Code will still work — it just won’t have the pfSense tools available, and you’d get confusing "I don’t have a tool for this" responses rather than a clear error. Validating upfront with /mcp confirms the full chain is working: Claude Code loaded the server, the server started, and the SSH tools are registered and ready.

Launch Claude Code from the project directory and verify MCP connectivity:

cd "~/projects/pfsense-tuneup"

claude

Inside the session, run /mcp. Expected output:

pfsense-ssh · ✓ connected · 37 tools

With all of this confirmed, I was ready to start the actual audit.

Part 2: The Environment

My home lab runs a pfSense CE 2.8.1 HA pair. The setup is more complex than a typical home lab — multiple internal subnets for different AD domains, a DMZ for externally-accessible servers, and 16 CARP VIPs spread across all interfaces. Here’s the layout:

Nodes

Primary (CARP Master)	fw1.example.com — 10.0.1.1
Secondary (CARP Backup)	fw2.example.com — 10.0.1.2
Virtual IP (Load Balanced)	fw.example.com — 10.0.1.254
Platform	pfSense Community Edition 2.8.1 (amd64)
HA Sync	pfsync + XMLRPC over dedicated virtual interface (172.16.0.0/24)

Network Interfaces

WAN (em0)	INTERNET — 203.0.113.16/28 (static block) – not my actual IP addresses
LAN (em1)	INSIDE — 10.0.1.0/24 (primary trusted network)
opt1 (em2)	CORP1_NET — 10.0.2.0/24 (AD domain: CORP1.LOCAL)
opt2 (em3)	CORP2_NET — 10.0.3.0/24 (AD domain: CORP2.LOCAL)
opt3 (em4)	HA — 172.16.0.0/24 (dedicated pfsync/XMLRPC link)
opt4 (em5)	GUEST — 10.0.10.0/24 (guest WiFi/captive portal)
opt5 (em6)	DMZ — 10.0.50.0/24 (externally-facing servers)
opt6	TMobile — DHCP (secondary ISP, outbound only)
opt7	IOT — unconfigured (intentionally deferred)

Installed Packages

pfBlockerNG-devel 3.2.14_1 — DNSBL + IP geo-blocking

HAProxy 0.63_11 — load balancing

ntopng 6.2.0 — traffic analytics

System_Patches 2.2.27 (upgraded to 2.2.29 this session)

Avahi, Backup, bandwidthd, Open-VM-Tools

CARP Configuration

16 CARP VIPs spread across WAN, LAN, CORP1_NET, CORP2_NET, GUEST, DMZ

preempt=1 on both nodes (primary re-asserts master after recovery)

pfsync syncok=1 on both nodes throughout session

XMLRPC config sync: primary pushes all changes to secondary automatically

Part 3: Setting Up pfBlockerNG DNSBL

Before the main audit, I had Claude Code configure pfBlockerNG DNSBL — the DNS-based blocklist engine intercepting every DNS query and checking it against threat feeds. This turned out to be its own mini-adventure.

DNSBL Architecture

pfBlockerNG DNSBL works by injecting a Python module (pfb_unbound.py) into Unbound DNS resolver. Every DNS query passes through the Python module, which checks against downloaded blocklists. Blocked domains are redirected to a loopback VIP where a lighttpd instance serves a block page.

Loopback VIP

The DNSBL block page server requires a dedicated IP alias on the loopback interface (lo0). IP: 10.200.200.1/8. The pfSense REST API does not accept "lo0" as a valid interface name for Virtual IP creation — this must be created manually via the Web UI: Firewall → Virtual IPs → Type=IP Alias, Interface=Localhost.

Feed Cleanup

Several feed sources were dead or incorrect:

RansomwareTracker (abuse.ch) — feed discontinued in 2019, removed

feodotracker.abuse.ch/downloads/domainblocklist.txt — does not exist; Feodo is IP-based only

sslbl.abuse.ch/blacklist/dstdomains.txt — does not exist; SSLBL is certificate/IP-based

Replacement feeds added:

URLhaus (abuse.ch) — real-time malware URL feed (typically 0–50 entries; aggressive removal is expected behavior, not a config error)

Hagezi Threat Intelligence Feeds — 1.4 million entries

Phishing Army — 78,000+ phishing domains

EasyList and EasyPrivacy — ad/tracker domains

Adaway — mobile ad domains

DNS Redirect NAT Rules

Five NAT port forward rules were created to intercept all DNS queries and redirect them through pfSense’s own Unbound resolver, ensuring every client — regardless of hardcoded DNS settings — passes through DNSBL:

INSIDE (LAN)	Redirect port 53 → 10.0.1.1
CORP1_NET (opt1)	Redirect port 53 → [opt1 CARP VIP]
CORP2_NET (opt2)	Redirect port 53 → [opt2 CARP VIP]
GUEST (opt4)	Redirect port 53 → [guest CARP VIP]
DMZ (opt5)	Redirect port 53 → [DMZ CARP VIP]

Each rule uses a negated destination (!pfSense_IP) so queries already directed at pfSense pass through unaffected.

Part 4: The Audit — What Claude Found

With connectivity established, I asked Claude Code to run a full audit: examine firewall rules, NAT rules, packages, CARP status, and DNS configuration on both nodes. Here’s what it surfaced, prioritized by severity:

Priority 1 — Critical

System_Patches update available: 2.2.27 → 2.2.29 on both nodes

DNS resolver cross-dependency: each node used the other as primary DNS, creating a hidden single point of failure

Priority 2 — High Security

Rules 48/49: Dead WAN rules for DC DNS traffic on the wrong interface, mislabeled as “DHCP” when port was 53

Rules 73/74: Duplicate HZN HTTP redirect WAN rules, one always shadowed

Rules 70/72: Production UAG/Horizon WAN rules with logging disabled

Rule 137: HA Sync interface running any-to-any with inet46

Priority 3 — Medium

Rule 109: Internal network group rule using inet46 despite no IPv6 deployment

Rule 89: LAN rule labeled “Allow STEAM Traffic” but actually opening IM ports

Rule 139: GUEST DNS rule allowing any destination on port 53, undermining DNS redirect enforcement

Priority 4 — Housekeeping

Six disabled “ALLOW ALL TRAFFIC - DO NOT USE” test rules across all interfaces

Disabled GUEST to LAN captive portal rule, superseded by newer rules

WAN Horizon rule with dst=any (deferred pending log research)

NAT port forward duplicates for Horizon (noted for future review)

Part 5: The Saga — A “Simple” Package Update Goes Wrong

What should have been a two-minute package upgrade turned into a two-hour debugging session, revealing a configuration flaw silently lurking for years. This was the most interesting part of the session. And, dare I say, one of the more frustrating parts of the audit.

Attempt 1: Package Manager UI

Clicked Confirm on the System_Patches upgrade screen. The page appeared to hang. SSH investigation showed upgrade processes running in the background, but the Unix socket streaming progress to the browser was never created, leaving the UI frozen. After several minutes with no progress, PF1 was rebooted.

PF2 seamlessly assumed CARP master status for all 16 VIPs during the reboot — confirming HA was functioning correctly. PF1 recovered and resumed master.

Attempt 2: Diagnostics > Command Prompt

Ran pkg upgrade -y pfSense-pkg-System_Patches directly from Diagnostics → Command Prompt. The browser loading bar partially advanced and stalled indefinitely. The command eventually errored out.

Root Cause: Unbound DNS Deadlock

SSH investigation revealed the true cause: Unbound DNS resolver on PF1 was running — process visible, all ports bound and listening — but not answering any queries. The pkg tool, which needs to resolve pkg.pfsense.org to download packages, was hanging indefinitely waiting for DNS never coming.

The culprit: pfBlockerNG injects a Python module (pfb_unbound.py) into Unbound’s query processing pipeline. After the reboot, the module deadlocked during initialization, leaving Unbound’s process alive but unable to process queries.

The Hidden Time Bomb: Cross-Referenced DNS

This is where it got interesting. Comparing the DNS configuration on both nodes revealed something silently present for years:

PF1's /etc/resolv.conf: first nameserver = 10.0.1.2 (PF2's physical LAN IP)

PF2's /etc/resolv.conf: first nameserver = 10.0.1.1 (PF1's physical LAN IP)

Each node was using the other as its primary DNS server. This design, likely intended for redundancy, actually created a cascading failure: when PF1’s Unbound deadlocked, it took PF1’s own DNS resolution down. PF2 appeared healthy only because it was outsourcing DNS to PF1.

On closer inspection, PF2’s Unbound was also deadlocked — it just hadn’t been noticed because PF2 was leaning on PF1. The only reason anything had been working at all was a prior-running Unbound instance on PF1’s CARP VIP.

Claude Code caught this. I hadn’t.

Resolution

Services → DNS Resolver → Restart DNS Resolver on both nodes — cleared the Python module deadlock

System → General Setup → DNS Servers — PF1 now uses 10.0.1.1 first; PF2 uses 10.0.1.2 first

After these changes, both nodes resolved DNS independently. The System_Patches upgrade completed on both nodes without issue.

Long-Term Consideration

The pfBlockerNG Python module deadlock appears to occur when Unbound is restarted while the module is mid-initialization — most commonly when loading a large blocklist like the 1.4 million-entry Hagezi TIF feed. The restart fix is straightforward and reliable, but the deeper issue is detectability.

As for the detectability, a deadlocked Unbound looks healthy from the outside: the process is running, ports are bound, and nothing in the logs indicates a problem. Standard process-based monitoring will miss it entirely. What's needed is a query-level health check — something actively testing whether Unbound is answering DNS, not just whether it's running.

As a practical runbook item: if pkg operations hang at the download stage, or browser-based commands in the pfSense web UI stall with no progress, the first step is always to restart DNS Resolver before assuming anything else is wrong.

Part 6: The Rule Cleanup

With DNS stable and packages current on both nodes, the session moved into the firewall rule findings. Claude Code worked through each item methodically — explaining the issue, the risk, and the recommended fix before making any change. Eight rules were modified or removed in total, ranging from dead rules on the wrong interface to logging gaps on production inbound traffic.

1. Rules 48 & 49 Deleted — Dead DC DNS Rules (154→152)

Two rules from 2019 on the WAN interface matching Domain Controller source addresses with destination port 53. Problems: DC traffic enters on opt1/opt2, never on WAN so these rules could never match. Both were mislabeled “Outbound DHCP Requests” but port was 53 (DNS, not DHCP 68). Additionally redundant since DNS redirect NAT rules already intercept DC DNS on the correct interfaces.

2. Rules 71 & 72 Deleted — Duplicate HZN HTTP Redirect (152→150)

Two WAN pass rules both matching tcp any → 10.0.50.24:80 with slightly different descriptions, indicating one was a copy of the other. Doubly redundant: a broader Horizon WAN rule already covered port 80 as part of the Horizon_Ports alias.

3. Logging Enabled on UAG WAN Rules 68 & 70

Two production WAN pass rules handling inbound external traffic to the UAG reverse proxy (10.0.50.27) had logging disabled. These rules handle external user access to enterprise remote access infrastructure. Logging is essential for incident detection on production inbound rules.

4. HA Sync Rule 133 Tightened

The HA sync interface rule was running as any/any with inet46. The HA interface is a virtual point-to-point link (172.16.0.0/24) existing only inside the hypervisor. Changes: inet46 → inet (no IPv6 on this interface), source/destination scoped to 172.16.0.0/24. pfsync and XMLRPC sync verified healthy immediately after.

5. Rule 105 — Internal Group inet46 → inet

The pass-all rule on the trusted internal interface group was configured for inet46 — both IPv4 and IPv6 — despite no IPv6 deployment in the environment. This matters because even without a routable IPv6 address, every modern device auto-assigns a link-local address in the fe80::/10 range. With inet46 in place, those addresses flow through pfSense under the same pass-all policy, bypassing pfBlockerNG geo-IP enforcement entirely since it operates on IPv4 only. Changed to inet. No functional impact to users or services, but a meaningful reduction in policy bypass surface.

6. Rule 85 — Mislabeled STEAM Rule Corrected

A LAN rule labeled “Allow STEAM Traffic to Internet” was actually using the IM_Ports alias (5190/AIM, 5222/XMPP, 6667/IRC, 5050/Yahoo). The correct Steam rule was a separate entry using the Steam_Services alias. Description corrected to “Allow Instant Messaging Traffic to Internet.”

7. Rule 135 — GUEST DNS Tightened

GUEST DNS pass rule had dst=any on port 53. The DNS redirect NAT rule already forces all GUEST DNS to pfSense’s resolver, but dst=any on the firewall rule meant the NAT could be bypassed without the firewall noticing. Changed dst=any → dst=10.0.10.X (pfSense resolver VIP). The NAT and firewall rule now independently enforce the same policy — defence in depth.

8. Rule 97 Deleted — Superseded Captive Portal Rule (150→149)

A disabled LAN rule allowing GUEST traffic to the UniFi controller at 10.0.1.50:8880. Placed on the wrong interface (LAN instead of GUEST). Two active, correctly-configured replacements on the GUEST interface already covered this. Rule deleted.

Part 7: Deferred Items

A good audit produces two lists: what got fixed and what needs more thought before touching. The items below fall into the second category — either because the right fix requires log research first, or because the scope is larger than a single session should bite off.

WAN Horizon Rule — Destination Tightening

WAN pass rule covering Horizon Ports (443, 8443, 22443, 4172, 9427, 32111, 50002) has dst=any. Correct destination is 10.0.50.24. Before tightening: pull firewall logs for this rule’s tracker to confirm all legitimate destination IPs, then scope to an alias. Do not change dst=any blindly.

NAT Port Forward Duplicates for Horizon

Apparent duplicate NAT rules for Horizon traffic (two rules for each of several port forwards). Needs log research to identify which set is actively used before removing.

IOT Interface

opt7 has no IP or rules and is intentionally deferred. Planned design: no internet access, no DMZ access, INSIDE may reach IOT devices.

pfBlockerNG Unbound Deadlock Monitoring

Add monitoring check for Unbound query responsiveness (not just process existence)

Check if a pfBlockerNG-devel update addresses the initialization deadlock

Runbook: if pkg or browser commands hang, restart DNS Resolver first

What Claude Code Did Not Do

Human Approval Required - Claude Code did not autonomously change anything. Every finding came with a recommendation, and every recommendation waited for explicit approval before anything was applied. The session was collaborative by design — Claude Code as analyst, operator as decision-maker.

Limited Understanding of Business Intent – Claude Code did not understand business intent behind every rule. It could identify a rule as technically redundant or incorrectly scoped, but it had no way of knowing whether a rule existed for a contractual reason, a legacy integration, or a deliberate architectural decision. Several items were flagged as candidates for removal and then left in place after context was provided. For example, having what appeared to be redundant rules were potentially different rules – one firewall rule allowing traffic from one subnet to another and another firewall rule allowing traffic to/from the Internet. Providing added context allowed for Claude Code to better understand intent of why these existed and were not merely duplicates.

Validation and Testing STILL Matter - It did not replace validation or testing. Changes were reviewed before being applied, and CARP and pfsync health were checked after each one. The audit surfaced the findings; verifying nothing broke was still an operator responsibility.

Architectural Context Requires Human Knowledge - Claude Code did not identify architectural context without some level of human guidance. The IOT interface, the BIG-IP NAT rules, the WatchGuard entry — Claude Code flagged these as worth reviewing, but understanding why they exist and whether they should stay required institutional knowledge the AI simply doesn't have access to. I will say, once Claude Code was provided some business use-case context, it immediately updated recommendations in a number of places and linked the added context to multiple rules and configurations without issue.

Deferred Items Still Require Operator Judgment – Claude Code still required operator judgment for the deferred items. The Horizon destination tightening, the NAT duplicate cleanup, the Unbound monitoring gap — none of these were left open because Claude Code missed them. They were left open because the right fix requires additional data, extended planning, or a dedicated change window. Knowing when not to change something is just as important as knowing what to change.

Results

The session surfaced more than expected — not just stale rules and housekeeping items, but two infrastructure issues hiding in plain sight. What follows is the complete list of changes made, in the order they were addressed.

Packages

✓ System_Patches upgraded: 2.2.27 → 2.2.29 on both nodes

Configuration

✓ DNS Resolver restarted on both nodes to clear Python module deadlock

✓ Both nodes now use own IP as first DNS server (eliminated cross-dependency)

✓ 5 DNS redirect NAT rules created across all internal interfaces

✓ pfBlockerNG DNSBL feeds updated: dead feeds removed, URLhaus/Hagezi/Phishing Army added

Firewall Rules

✓ Rules 48/49 deleted: dead DC DNS rules on wrong interface (154→152)

✓ Rules 71/72 deleted: duplicate HZN HTTP redirect WAN rules (152→150)

✓ Rules 68/70 patched: logging enabled on UAG/Horizon WAN rules

✓ Rule 133 patched: HA Sync scoped to HA subnet, inet46→inet

✓ Rule 105 patched: Internal group inet46→inet

✓ Rule 85 patched: description corrected to match actual alias (IM_Ports, not Steam)

✓ Rule 135 patched: GUEST DNS dst=any→dst=pfSense resolver VIP

✓ Rule 97 deleted: superseded captive portal rule (150→149)

Final State

Total firewall rules	154 → 149 (5 removed, 0 added)
Packages	All current, no updates pending
Notices	0 on both nodes
CARP	16 VIPs MASTER on primary, 16 BACKUP on secondary
pfsync	syncok=1 on both nodes
XMLRPC	Config identical on both nodes

Takeaways

A few things stood out from this session:

The cross-referenced DNS configuration is the kind of design decision which looks reasonable at the time — each node pointing to the other for redundancy. However, this hides a potential cascading failure with DNS. It took an unrelated Unbound deadlock to surface it, and an audit with no prior assumptions to flag it.

The inet46 rules were a real finding. With pfBlockerNG geo-IP operating on IPv4 only, leaving inet46 on internal trust rules creates a bypass path for any device auto-configuring a link-local IPv6 address. This is easy to miss.

The mislabeled rule (STEAM vs IM_Ports) had presumably been wrong for years. No one catches these in day-to-day operations. Automated review with no prior assumptions found it immediately.

MCP + Claude Code as an audit platform works well. The 37 SSH tools gave Claude Code full access to pfctl output, config.xml, log data, and package state — enough to conduct a genuine infrastructure review, not just a surface-level scan.

The full setup — from installing the MCP SSH server on macOS through to validating the /mcp connection in Claude Code — took about 45 minutes. The audit session itself ran several hours, mostly because of the DNS deadlock detour, with some breaks to let Claude Code run.

Running this against a live firewall — even a home lab one — does require some trust in the process. I was pleasantly surprised at how much structure Claude Code brought to each issue: what was found, why it mattered, and why the recommendation was the right call. All-in-all, an excellent experience and a great way to use AI in a supplemental capacity to achieve a real goal.

I'll be making this a regular cadence — at least quarterly or semi-annually — to ensure unused rules and configurations are disabled or removed completely..

Update: Downstream Consequences Worth Noting

Shortly after publishing this post, one of my four Horizon VDI servers went offline. No amount of rebooting or service restarts brought it back. Two things stood out immediately: the certificate loaded on the host had expired, and the host was reporting no Internet connectivity

The certificate expiration explained why Horizon services weren’t functioning on the host — a renewal missed during maintenance the week prior. Replacing the certificate and restarting services, however, still didn’t bring it back online.

The Internet connectivity issue traced back to DNS. The specific subnet’s AD DNS was configured to route outbound resolution through the public-facing side of the pfSense firewalls rather than through pfSense’s own DNS resolver. This was one of the misconfigurations corrected during the audit session — because in its original form, it allowed a complete bypass of pfBlockerNG DNSBL outbound. Not a catastrophic security gap, but not intentional either.

The DNS redirect NAT rules put in place during the session enforced resolver routing properly, and the host in question was the first to feel it. The other Horizon hosts hadn’t propagated the change yet — but it was only a matter of time.

The fix was straightforward once the cause was clear. The audit created the condition; understanding the audit made diagnosing it fast.

Home Labs are Fun! 😂

See Part 2 for the continuation of the story!

Generated with Claude Code (Anthropic) — May 2026
Reviewed and approved by Dean Flaming – May 2026

macOS Terminal Commands to Add Spaces to the Dock

2025-09-21T13:39:00.000-07:00

Adding Spaces to macOS Dock

Have you ever wanted to have spaces on your macOS Dock? I know, some would argue, "You're using it wrong!" Quite simply, that's not how tech folks should think. Remember, users should be able to make their computer conform to the their own habits - not the user conforming to the computer (more specifically, the software engineers who don't ever think about the end user experience or their solution's usability).

And, if you're like me, I detest LaunchPad, Widgets on my desktop (in Tahoe or newer), Stage Manager, and the new "Applications" menu in macOS Tahoe. I find them useless while I know others love them

Additionally, while everyone wants you to "buy my cheap tool/use my free tool" (which, let's face it, needs far too many permissions and probably opens too many security holes), there is an easier, free way built right into macOS.

So how do we add spaces to the Dock?

Types of Spaces on macOS Dock

First, we need to understand, there are two main sides to the dock. You can easily see this with the vertical separator bar between static apps and running apps (if you don't minimize apps to their icon).

This means, when adding a space, you need to define which side of this vertical separator bar you wish for the space to be one.

Prerequisites:

One thing to note up front:

Be semi-comfortable in macOS Terminal

Adding a Space to the Left Side of the macOS Dock

Open Terminal
Execute the following command:

defaults write com.apple.dock persistent-apps -array-add '{tile-data={}; tile-type="spacer-tile";}' && killall Dock

NOTE: You do not need to execute this as "sudo" (Super User 'do'). Additionally, you will see the Dock disappear and reappear with the space added just to the left of the vertical separator bar.

Once the space is added, you can click and hold on the space area to drag it around or remove it if desired.

Adding a Space to the Right Side of the macOS Dock

Open Terminal
Execute the following command:

defaults write com.apple.dock persistent-others -array-add '{tile-data={}; tile-type="spacer-tile";}' && killall Dock

NOTE: You do not need to execute this as "sudo" (Super User 'do'). Additionally, you will see the Dock disappear and reappear with the space added just to the right of the vertical separator bar.

Once the space is added, you can click and hold on the space area to drag it around or remove it if desired.

Done!

Honeywell Wi-Fi Thermostat or RedLINK Gateway and Outbound Internet Connectivity Failures

2025-07-05T09:31:00.000-07:00

Problem:

The Honeywell Wi-Fi Thermostats and RedLINK Gateways fail to route outbound, even when they have an internally resolvable IP address provided by DHCP, when the router/gateway does not also handle DNS requests on the same IP. This is due to (poor) internal coding of the Honeywell devices ignoring all DNS and other secondary DHCP provided resources except the gateway IP.

Symptoms:

The Honeywell Wi-Fi thermostat or RedLINK gateway may indicate it does not have the ability to contact the Internet even though the Internet connection is up, the Honeywell device has a DHCP assigned IP address (for Honeywell Wi-Fi thermostats, see the Wi-Fi screen), and other devices can communicate outbound and to the device.
The mytotalconnectcomfort.com website shows an error that it has lost connectivity with the device.

For Honeywell Wi-Fi thermostats, they show error such as “Internet Connection Lost” or “Waiting for Updates”.

Environment:

In my case, I have two pfSense Firewalls/Gateways operating in an HA configuration.

Primary pfSense Firewall IP: 192.168.1.252 (example)
Secondary pfSense Firewall IP: 192.168.1.253 (example)
CARP Virtual IP (VIP): 192.168.1.254 (example address - used as default gateway and DNS server for clients)
DHCP assigned address provides:

Gateway
DNS
NTP

Honeywell Wi-Fi Thermostats and/or RedLINK Gateway requires outbound DNS resolution to: mytotalconnectcomfort.com

Solution:

Assuming the Honeywell device is actually connecting to Wi-Fi and getting a DHCP provided IP address and other devices on that same network subnet can communicate (i.e. Ping) the Honeywell device… AND other devices on that same network subnet can browse outbound to the mytotalconnectcomfort.com website, then the issue is likely to be DNS resolution. Allow DNS resolution on internal Gateway address.

pfSense with HA Specific Solution

In my case, I have two HA configured pfSense routers/firewalls which I had forgotten to bind DNS Resolver to the CARP (LB VIP) address.

Configure Unbound (DNS Resolver) to Listen on the CARP VIP

Login to pfSense Web GUI (typically at 192.168.1.252 for the primary pfSense and 253 for the secondary pfSense)
Navigate to Services → DNS Resolver
In the Network Interfaces section:

Select the following interfaces:

LAN (includes .252 and .253)
192.168.1.254 (CARP VIP)
Localhost (optional)

This ensures that Unbound listens on:

192.168.1.252 (Primary)
192.168.1.253 (Secondary, when in MASTER state)
192.168.1.254 (CARP VIP, used by clients and devices like RedLINK)

Save and Apply Changes

NOTE: You may need to “reboot” the thermostat after you apply any changes to network resolution and routing settings. The Honeywell Wi-Fi thermostat and RedLINK devices aren’t exactly smart and have a limited code configuration so restarting them may be the quickest option to allow testing.

Results:

DNS requests to 192.168.1.254 (CARP VIP) are answered by Unbound on the MASTER firewall.
Honeywell Wi-Fi or RedLINK Gateway successfully resolves mytotalconnectcomfort.com and establishes outbound communication.
Failover between .252 and .253 does not interrupt DNS functionality, as only the MASTER firewall owns .254.

References:

Reddit Thread: https://www.reddit.com/r/Ubiquiti/comments/d93e9z/honeywell_thermostat_connection_issue/

Setting Up the Broadlink RM4 Mini with Home Assistant (Without the Broadlink App)

2025-06-08T12:26:00.000-07:00

Introduction:

This guide outlines the current (as of writing) process for setting up a new Broadlink RM4 Mini with Home Assistant—without relying on Broadlink’s mobile apps or cloud services.

References:

Everything Smart Home YouTube Video - Control IR Devices with Home Assistant EASILY - Broadlink RM4 Pro
MJG59 Python Broadlink - Git Repository

Notes:

Double quotes are used below for readability—do not include them when entering Python commands.
Single quotes will likely NOT copy correctly (as the correct ASCII input) so please replace all single quotes by manually typing them on the command line if using copy/paste.
Some commands may differ depending on your system or environment—refer to the GitHub repository for alternatives.
This guide is based on macOS, but most steps work on Windows with minor adjustments:

Use `python`/`pip` instead of `python3`/`pip3` may be required.
`broadlink.xdiscover()` may not work on Windows due to firewall/network configurations. Use `broadlink.hello(‘RM4_IP_ADDRESS')` instead.

Calling “broadlink.hello(‘RM4_IP_ADDRESS')” or “broadlink.discovery()” or any of the python commands from the Git repository directly without piping it to a variable (i.e. not using “variable = “ in front of the command) will port the output to screen instead of a variable.

Preparations:

Download and install Python if necessary

Install Python if you need to.

macOS: Use Brew to install Python (‘brew install python’).
Windows: Download python from Python.org and install.

After Python is installed, install Python-Broadlink from Github

Follow the instructions on the Github repository to install.
Use “pip3 install broadlink” or “pip install broadlink” commands based on your OS configurations.

Instructions:

Discovering, Connecting, Configuring The RM4 Mini:

After unboxing, connect the RM4 Mini to power.

It should blink rapidly 4 times…then pause. Then repeat.

This means the RM4 Mini is in AP mode.

If it is not blinking rapidly 4 times then pausing, it is not in AP Mode.

There are several ways to get it into AP mode, the easiest being long-press for 10 seconds and it should become available to join via Wi-Fi.

NOTE Long pressing 3-4 seconds just resets it. Continuing to hold for 10 seconds forces it into AP mode.

From your computer where Python and the Broadlink GIT repository are installed, join the Broadlink wifi (something like “Broadlink_Wifi_device” or similar).

NOTE: It helps to have both wired and wireless network connections so you can scan your network to see when it joins wifi (assuming wifi and wired are on the same subnet or reachable from each other if on different subnets).
NOTE: There is no password.

Launch Python

macOS Terminal: python3
NOTE: As stated under initial notes, this guide is based on use of macOS Terminal.

For Windows, refer to the notes section above for use of Python and Pip on Windows.

Import Broadlink libraries

Python Command “import broadlink"

Check your Wi-Fi IP address.

It should be a “192.168.10.100” and the “router” should be “192.168.10.1”
The “router” address is the IP of the RM4 Mini you just plugged in.
NOTE: This is what I was seeing on my RM4 minis. They may change based on environment or software updates.

From the Python command line, initiate a discover command.

See the GitHub page for various options here.
I had to define my local IP address in the command.

Command “devices = broadlink.discover(local_ip_address=‘COMPUTER_IP_ADDRESS’)”.

Scan the network (Optional)

If you can scan the network you should only see the following:

Your computer’s Wi-Fi IP of 192.168.10.100
The RM4 Mini’s IP of 192.168.10.1

Send the Wifi Configuration Command (YES, you send the wifi command before connecting to configure other settings)

Use the following command to setup wifi on the RM4 Mini

Command: “broadlink.setup(‘SSID’, ‘PASSWORD’, 3, ip_address=‘BROADCAST_IP_ADDRESS’)” (i.e. the broadcast address of the Broadlink wifi AP).
NOTE: See the Git page for details.
I had to use the broadcast address to configure the device.

If correctly done, the device will configure itself and reboot and your computer will drop off the wifi.

Your RM4 mini should have rebooted and joined the wifi which was previously defined.

If not, do a long reset and reconfigure.

Initiate another device discovery but use your network’s broadcast address.

Command: “devices = broadlink.discover(discover_ip_address=‘BROADCAST_IP_ADDRESS’)” where the broadcast address is now the broadcast address of the current network - assuming your RM4 mini is rebooted and joined your wifi network.

Conduct a search for all devices by initiating the following Python Commands

Commands:

“for device in broadlink.xdiscover():”
“ print(device)”
NOTE: The print command is an example action. Do whatever you want or issue all of the commands up front for the discovered device(s).

NOTE: When you type out a “for” command and hit enter after the first line, it will take you to a “. . . “ line.

Press ENTER at the end of the first line to go to the second line.
Press TAB and type the second line.
Press ENTER at the end of the second line.
Press ENTER one more time.

You will see a list of all RM4 mini devices on your network.

Conduct a “Hello” to link to the specific one in question.

Send the command “device = broadlink.hello(’RM4_IP_ADDRESS’)” to define the RM4 mini address.

NOTE the single quotes may not copy correctly. Retype if needed.

Authenticate to the device.

Use “device.auth()” to authenticate to the device in question.

Set the device name.

Command: “device.set_name(‘Master Bedroom Remote’)
NOTE: This is ONLY done so when you add more devices via this process, you know which ones are already configured.
NOTE: These names mean nothing to Home Assistant as it stores names within itself.

Unlock the device.

Command: “device.set_lock(False)

Check device lock state.

Command: “device.is_locked”
NOTE: This should return false. If set to true, ensure you are authenticated (rerun “device.auth()” if necessary) then rerun “device.set_lock(False)”

You are now ready to join it to Home Assistant.

Adding the RM4 Mini to Home Assistant:

If you have troubles here, please check out Everything Smart Home's YouTube video linked at the top.

Open Home Assistant and browse to SETTINGS > DEVICES AND SERVICES.
Within there a new device should appear.

Click ADD.

Set the Home Assistant Name for the device.

NOTE: I suggest making it the same as the name set above on the Broadlink configurations. This way it is easier to reference later.

Define the Home Assistant location.
Turn on the RM4 Mini device in Home Assistant.

Go to the RM4 mini device in Home Assistant (SETTINGS > DEVICES AND SERVICES > BROADLINK) and select the RM4 mini device and within its controls, click the switch to turn it on in Home Assistant.

The device is now added to Home Assistant.

You are ready to start “learning” IR and RF codes.

NOTE: If I’m importing learned codes from another RM4 Mini, I like to learn one code first to autogenerate the file(s) necessary in Home Assistant. This is super beneficial for multiple of the same device, such as multiple mini-split AC units with the same remote and commands, but each in different areas of the house. See the “Modifying the Broadlink Remote Codes Files” section below.

Learning Codes in Home Assistant for a Device:

Within Home Assistant, browse to DEVELOPER TOOLS > ACTIONS
Under Actions, select REMOTE: LEARN COMMAND
Click ENTITY and select the RM4 MINI you just configured.
For DEVICE, check the box and type in the simple name of the device you are “learning” commands for.

NOTE: If you are learning multiple commands for a device, ensure you are using the same name for each command.

For COMMAND, check the box and type in a name for the command to be learned (e.g. “power” or “power_on”).
For COMMAND TYPE, check the box and ensure the correct type is set (i.e. IR or RF).
If using a toggle button which has multiple commands (i.e. Power button has one command for power on and another command for power off), then you can check the box and ENABLE the switch.

This is optional

Timeout is optional
Click the PERFORM ACTION button and the light on the RM4 Mini should light up.
Press the specific button (perform the action on the remote) which you want to record.

The PERFORM ACTION will go green with a check mark if successful in recording the command.

Testing Learned Codes in Home Assistant:

Within Home Assistant, browse to DEVELOPER TOOLS > ACTIONS
Under Actions, select REMOTE: SEND COMMAND
Click ENTITY and select the RM4 MINI you just configured.
For DEVICE, check the box and type in the simple name of the device you previously “learned” commands for.
For COMMAND, check the box and type in a name for the command which was learned (e.g. “power” or “power_on”).
For COMMAND TYPE, check the box and ensure the correct type is set (i.e. IR or RF).
Repeats, Delays, and Hold are optional.
Click the PERFORM ACTION button and see if the device responds accordingly.

Modifying the Broadlink Remote Codes Files:

NOTE: If you modify the “/config/.storage/broadlink_remote_XXXXXXXXXX_codes” file for the RM4 mini device by pasting in additional commands/codes, make sure you do the following:

Save the file and close it.
Reload the device in Home Assistant SETTINGS > DEVICES & SERVICES > BROADLINK page by clicking the 3 vertical dots and selecting RELOAD.

This will reload the commands file.

Conclusion:

You’re now fully set up to control IR/RF devices with your Broadlink RM4 Mini in Home Assistant. Enjoy building your smart home automations!

Fix the OBS "Could Not Find the Virtual Camera System Process" Error on macOS Without Logging Out

2024-12-09T12:52:00.006-08:00

If you run OBS Studio on macOS as a virtual camera, sooner or later you will hit this error at the worst possible time:
Could not find the virtual camera system process. This may be resolved by logging out of and back into your macOS user account, or by restarting your computer.
The dialog is technically correct. A logout or restart clears it. Both also kill every open app and session, usually five minutes before a meeting starts. The good news: you can reset the virtual camera in place, no logout required.

Why It Happens

Modern OBS builds run the macOS virtual camera as a CoreMedia I/O (CMIO) camera extension managed by the operating system. The extension process occasionally dies or fails to re-register after an OBS or macOS update, and OBS loses track of it. Extensions normally register at session start, which is why the dialog points you toward a logout. Toggling the extension in System Settings forces an immediate re-registration instead.

The Fix

The order of operations matters here, so follow the sequence exactly.
1.     Quit OBS: Fully quit with Cmd+Q. The reset will not take effect while OBS is running.
2.     Open Camera Extensions: Go to System Settings > General > Login Items & Extensions and scroll down to the Camera Extensions section.
3.     Toggle the OBS extension off: Uncheck the toggle for the OBS camera extension. This unloads the stuck instance from your session.
4.     Open OBS, then quit it again: Launch OBS briefly with the extension disabled, then quit. This clears its handle to the old extension.
5.     Toggle the extension back on: Recheck the toggle. macOS re-registers the extension immediately.
6.     Start the virtual camera: Open OBS and click Start Virtual Camera. The error should be gone.

Two Follow-Up Notes

• App refresh: If Zoom, Teams, or a browser was already running and no longer lists OBS Virtual Camera in its device picker, restart the app so it re-enumerates cameras.
• Legacy plugin conflict: If the toggle sequence still fails, look in /Library/CoreMediaIO/Plug-Ins/DAL/ for an old DAL-style plugin left over from a legacy OBS install. Remove it, since it conflicts with the modern camera extension.

Credit

Credit where due: Roman Zipp documented this toggle sequence on his blog, and it has held up across multiple OBS releases. His post saved me a logout cycle, and hopefully this one saves you the same.

Workspace ONE Access and Office 365 Integration with Azure AD and Tear Down Notes

2021-02-24T14:54:00.005-08:00

INTRODUCTION:

This is a simple whitepaper and procedures on integration of Workspace ONE Access with MS Office 365 (and Azure AD) as well "tear down" steps for a lab environment. It is designed to help someone go through all of the major steps with integration of a VMware Workspace ONE Access tenant with a Microsoft Azure & Office 365 tenant in order to fully test out the integration process as well as being able to tear everything back down and reset both tenants in order to repeat the process.

This is just one of the ways in which integration CAN work but not the only way. Each customer environment is very unique and will be different in some or many aspects. These procedures mainly focus on use of Workspace ONE Access to provide Single Sign-On into O365 resources and NOT provide provisioning (a.k.a. user account creation) in O365 nor automatically define license levels.

For additional details and resources see Steve the Identity Guy's blog on O365 integration.

PREREQUISITES:

Workspace ONE Access tenant (either on-premises or cloud will work)

Service FQDN (i.e. <tenant>.vmwareidentity.com)
Administrative Login Info
Globally Unique Issuer Name (e.g. <tenant>.vmwareidentity.com or <tenant>.workspaceoneaccess.com or "<tenant>.company.com").
UPN and ImmutableID User Attributes (MS ImmutableID = Source Anchor and is either objectGUID or mS-DS-ConsistencyGuid)

NOTE: Within newer Workspace ONE Access tenants, there is a "sourceAnchor" user attribute which can be used to define whether "objectGUID" or "mS-DS-ConsistencyGuid" is the mapped value for easy changing of Microsoft's ImmutableID (a.k.a. Source Anchor). In the case this doesn't exist, create this within the Admin UI under Identity & Access Management > Setup > User Attributes prior to creating the Directory and then ensure when setting up the tenant Directory (to an on-premises Active Directory via AD-IWA), ensure the desired User Attribute mappings are defined properly.
A.D. Synchronized – Leave UPN as $(user.userPrincipalName)
Workspace ONE Access Connector installed and activated on Domain Member
Domain Integration with defined Source Anchor AND userPrincipalName

$(user.sourceAnchor) = objectGUID or mS-DS-ConsistencyGuid
NOTE: SEE SETTING UP SOURCE ANCHOR IN WORKSPACE ONE ACCESS SECTION

O365/Azure tenant

Customer must have O365 Premium Business account
Customer O365 Registered Domain (e.g. company.com)
Customer O365 Admin credentials
Issuer Uri (e.g. vidm.company.com or whatever is set in O365 VMware Identity Manager/Workspace ONE Access Apps)
Customer’s desired Federation Brand Name (e.g. “VMware, Inc.”)
Passive Logon Uri (i.e. “https://<IDM_FQDN>/SAAS/API/1.0/POST/sso”)
Active Logon Uri (i.e. “https://<IDM_FQDN>/SAAS/auth/wsfed/activelogon”)
Logoff Uri (i.e. “https://login.microsoftonline.com/logout.srf”)
Metadata Exchange Uri (i.e. “https://<IDM_FQDN>/SAAS/auth/wsfed/services/mex”)
SAML Signing Certificate (without spaces, carriage returns, “-----BEGIN CERTIFICATE-----”,or “-----END CERTIFICATE-----”)

Licensing

MS Office 365 Business Premium licenses (even just one will do for testing)
VMware Workspace ONE Access (SaaS or On-Premises) will work without a license

WORKSPACE ONE UEM NOTES:

For Workspace ONE UEM, so long as Workspace ONE Access is defined for Enrollment Authentication at the Company OG, Workspace ONE UEM does NOT need to know anything about mS-DS-ConsistencyGuid or sourceAnchor. Knowledge of these attributes can be handled solely within Workspace ONE Access. Enrollment Authentication is set within Workspace ONE UEM Admin Settings > Devices & Users > General > Enrollment under the Authentication tab on the Customer OG.
For Hub Services within Workspace ONE to work properly, both Workspace ONE Access and Workspace ONE UEM do need to know about objectGUID and its specific mappings. This means, if using sourceAnchor in Workspace ONE Access, one must ALSO set "objectGUID" to a mapping of "objectGUID" in the Workspace ONE Access global User Attributes and in Directory settings under Mapped Attributes for synching into Workspace ONE Access (See screenshots within the SETTING UP SOURCE ANCHOR IN WORKSPACE ONE ACCESS procedure further below) as well as also having "Object Identifier" within the Workspace ONE UEM Settings > System > Enterprise Integrations > Directory Services section of the Customer OG where "Object Identifier" is mapped to "objectGUID".

NOTE: It should again be noted, having objectGUID mapped into both Workspace ONE Access and Workspace ONE UEM are NOT required for O365 integrations unless specifically using the older objectGUID mappings for O365. Having objectGUID mappings in both Access and UEM ARE REQUIRED for Workspace ONE Hub Services to properly function. This is different than what is needed for O365 functionality.

STAND-UP (INTEGRATION) STEPS FOR A LAB:

1. Use O365 or Azure Domain Admin Console to enable Active Directory sync

Office 365 Admin Console: https://portal.office.com or https://login.microsoft.com

Azure Admin Console: https://manage.windowsazure.com

2. Use O365 or Azure Domain Admin Console to create Azure Domain within Azure Tenant

3. Activate Sync in O365 or Azure Domain Admin console

4. Install Azure AD Connect tools on a Domain Member server

5. Walk through initial configurations of AAD Connect and conduct sync of On-premises Active Directory to Azure AD

NOTE: This is a wizard procedure within AAD Connect tools

6. Open PowerShell command prompt on member server where Azure AD Connect tools are installed and connect to Azure or O365 tenant

Command: Connect-MsolService

7. In PowerShell, list the Azure Domains

Command: Get-MsolDomain

8. Now it is time to convert the Azure Domain if it is not already converted

NOTE: When Using Azure Admin Console and wizard options, the domain is already converted to Federated (Skip step 8. and 8.a. and go to step 9.)

8.a. To convert an Azure Domain created through the O365 Admin Console, use the following command within PowerShell on the AD Domain Member Server where AAD Connect tools are installed.

Command Template: Set-MsolDomainAuthentication -DomainName <domain> -Authentication Federated -IssuerUri <"Outlook Tenant Issuer"> -FederationBrandName <"My Company, Inc."> -PassiveLogOnUri "https://<Identity Manager Tenant FQDN>/SAAS/API/1.0/POST/sso" -ActiveLogOnUri "https://<Identity Manager Tenant FQDN>/SAAS/auth/wsfed/active/logon" -LogOffUri "https://login.microsoftonline.com/logout.srf" -MetadataExchangeUri "https://<Identity Manager Tenant FQDN>/SAAS/auth/wsfed/services/mex" -SigningCertificate <Get this from VMware Identity Manager / Workspace ONE Access admin UI>

Command Example: Set-MsolDomainAuthentication -DomainName test.com -Authentication Federated -IssuerUri "test.vmwareidentity.com" -FederationBrandName "Test.com" -PassiveLogOnUri "https://test.vmwareidentity.com/SAAS/API/1.0/POST/sso" -ActiveLogOnUri "https://test.vmwareidentity.com/SAAS/auth/wsfed/active/logon" -LogOffUri "https://login.microsoftonline.com/logout.srf" -MetadataExchangeUri "https://test.vmwareidentity.com/SAAS/auth/wsfed/services/mex" -SigningCertificate MIIDHTCCAgWgAwIBAgIHRzGnzeX6CTANBgkqhkiG9w0BAQUFADBGMSAwHgYDVQQDDBdWTXdhcmUgSWRlbnRpdHkgTWFuYWdlcjEVMBMGA1UECgwMRkxBTUlOR1RFU1QxMQswCQYDVQQGEwJVUzAeFw0xNTA2MDIyMDE1MjhaFw0yNTA1MzAyMDE1MjhaMEYxIDAeBgNVBAMMF1ZNd2FyZSBJZGVudGl0eSBNYW5hZ2VyMRUwEwYDVQQKDKxGTEFNSU5HVEVTVDExCzAJBgNVBAYTAlVTMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvPTLdMe5VlWVmZq783oAAJcMipdJ5I1qw9k08sdy9kBTDj0zxGX9e3Pjokrk4ySB0F9gjDl5eb/DpOoyvMtmH8vEQe/w35pfxURJVnvT/qW0QqLiPllk1AyR0+j/1O0jO0IAZ4zVmsUCjtWt0FUTEHrBMEVE273bve7KExfLuuwxXDtaPwJ5cNFeB6f1SeZp+esEbP42MM7dQRsVwF+sUIgE5iU6jj0bMxM1Cp+cpusDwQoM9R+CfrcqWOZcN0SD+XhMtleCWKbRLYuqHgk1eVEHcR2oZCEj2dAHxT67T1U94nbue901brLN75Uw3rCH89K2flHDmoRj3ZwFymifZQIDAQABoxAwDjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4IBAQCNLzLgkISAroUFZc0pXb0E7YPPMLcVDXAqdiF+v1j9nnvay4EhZgGFGzZgkAl0hEstMhQTVz5WL+0PiiXma+jxpAdnulwWIHz8rWs20fyT76LQpw1BWFyX3/EdIFPLB3Lt2Ji9tkfwjLoOZm7xQoBQWds2ABriS9r7nbDcY6fcDknxZyziwusSpwcbvaeL534WNJKo4flTMTiPhSR2xfz8WfgZlTICnWoeYLzT18uXZ9qbkCWiiOrchZK6VBL7b66E0Dm/YAyjie3DLbR8r965qmaxgNDFMakASHd/rIo5FL0jna94tGvICAQo8zrxKNcdVc04MykSgssGERPw5VzU

9. To verify a domain created and converted to "federated" through the Azure Admin console, use the following verification commands in PowerShell on the AD Domain Member Server where AAD Connect is installed and configured.

NOTE: THE FOLLOWING POWERSHELL COMMANDS ARE USED TO VERIFY THE DOMAIN CREATED THROUGH THE AZURE ADMIN CONSOLE ONLY. If you completed step 8. and 8a. then skip step 9.a. and 9.b. and go to step 10.

9.a. Get the DNS (or TXT) domain record verifications for the actual owned domain (the one matching the Azure/O365 domain name created in the MS Azure tenant).

Command: Get-MsolDomainVerificationDns -Domainname [domain] [-mode ,domainverificationmode.] [-tenantid ,guid.] [,commonparameters.]

Command Example: Get-MsolDomainVerificationDns -Domainname test.com -Mode dnstxtrecord

9.b. Using the PowerShell Command Prompt on the AD Member Server where the AAD Connect tools are installed, run the following command to confirm the federation of the Azure Domain to the VMware Identity Manager / Workspace ONE Access Identity Provider (IdP).

NOTE: In order to confirm ownership of the actual domain, a custom TXT or MX DNS record must be added to the root domain entry. This would have been accomplished within the Azure Admin Console.

Command: Confirm-MsolDomain -DomainName <string> [-ActiveLogOnUri <string>] [-DefaultInteractiveAuthenticationMethod <string>] [-FederationBrandName <string>] [-IssuerUri <string>] [-LogOffUri <string>] [-MetadataExchangeUri <string>] [-NextSigningCertificate <string>] [-OpenIDConnectDisoveryEndpoint <string>] [-PassiveLogOnUri <string>] [-SigningCertificate <string>] [-SupportsMFA <boolean>] [-TenantId <Guid>] [<CommonParameters>]

Command Example: Confirm-MsolDomain -DomainName test.com -ActiveLogOnUri "https://test.vmwareidentity.com/SAAS/auth/wsfed/active/logon" -DefaultInteractiveAuthenticationMethod Federated -FederationBrandName "Test.com" -IssuerUri test.vmwareidentity.com -LogOffUri "https://login.microsoftonline.com/logout.srf" -MetadataExchangeUri "https://test.vmwareidentity.com/SAAS/auth/wsfed/services/mex" -PassiveLogOnUri "https://test.vmwareidentity.com/SAAS/API/1.0/POST/sso" -SigningCertificate MIIDHTCCAgWgAwIBAgIHRzGnzeX6CTANBgkqhkiG9w0BAQUFADBGMSAwHgYDVQQDDBdWTXdhcmUgSWRlbnRpdHkgTWFuYWdlcjEVMBMGA1UECgwMRkxBTUlOR1RFU1QxMQswCQYDVQQGEwJVUzAeFw0xNTA2MDIyMDE1MjhaFw0yNTA1MzAyMDE1MjhaMEYxIDAeBgNVBAMMF1ZNd2FyZSBJZGVudGl0eSBNYW5hZ2VyMRUwEwYDVQQKDKxGTEFNSU5HVEVTVDExCzAJBgNVBAYTAlVTMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvPTLdMe5VlWVmZq783oAAJcMipdJ5I1qw9k08sdy9kBTDj0zxGX9e3Pjokrk4ySB0F9gjDl5eb/DpOoyvMtmH8vEQe/w35pfxURJVnvT/qW0QqLiPllk1AyR0+j/1O0jO0IAZ4zVmsUCjtWt0FUTEHrBMEVE273bve7KExfLuuwxXDtaPwJ5cNFeB6f1SeZp+esEbP42MM7dQRsVwF+sUIgE5iU6jj0bMxM1Cp+cpusDwQoM9R+CfrcqWOZcN0SD+XhMtleCWKbRLYuqHgk1eVEHcR2oZCEj2dAHxT67T1U94nbue901brLN75Uw3rCH89K2flHDmoRj3ZwFymifZQIDAQABoxAwDjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4IBAQCNLzLgkISAroUFZc0pXb0E7YPPMLcVDXAqdiF+v1j9nnvay4EhZgGFGzZgkAl0hEstMhQTVz5WL+0PiiXma+jxpAdnulwWIHz8rWs20fyT76LQpw1BWFyX3/EdIFPLB3Lt2Ji9tkfwjLoOZm7xQoBQWds2ABriS9r7nbDcY6fcDknxZyziwusSpwcbvaeL534WNJKo4flTMTiPhSR2xfz8WfgZlTICnWoeYLzT18uXZ9qbkCWiiOrchZK6VBL7b66E0Dm/YAyjie3DLbR8r965qmaxgNDFMakASHd/rIo5FL0jna94tGvICAQo8zrxKNcdVc04MykSgssGERPw5VzU

NOTE: The command looks very similar to that of the image shown in step 8.a.

10. Within VMware Identity Manager / Workspace ONE Access admin UI, create the O365 resources using the "Office365 Portal" from the VMware Catalog (i.e. type "Office365 Portal" in the Search field when creating a new SaaS Application in the VMware Identity Manager/Workspace ONE Access Catalog).

11. DONE.

TEARDOWN STEPS FOR A LAB:

1. First we must connect to Azure AD within PowerShell. Using the PowerShell Command Prompt on the AD Member Server where the AAD Connect tools are installed, run the following commands to connect to the Azure tenant

Command: Connect-MsolService

1.a. To list the Azure Domains within PowerShell use the following command:

Command: Get-MsolDomain

2. Now we must convert the existing FEDERATED Azure Domain to a SYNC domain. Using the PowerShell Command Prompt on the AD Member Server where the AAD Connect tools are installed, run the following command to convert the federated domain back to a sync domain.

Command: Convert-MSOLDomainToStandard –DomainName <federated domain name> -SkipUserConversion $false -PasswordFile <file>

Command Example: Convert-MsolDomainToStandard -DomainName test.com -SkipUserConversion $false -PasswordFile "C:\userpasswords.txt"

NOTES: Operation cannot overwrite existing file. Additionally, the users must be converted during the domain conversion in order for the user accounts to be deleted (i.e. Use "-SkipUserConversion $false" to ensure users are converted).

https://msdn.microsoft.com/en-us/library/dn194122.aspx

3. Assuming we are completely tearing down the Azure Domain Integration with an IDP and the Azure Domain Federation, After converting the Azure Domain to SYNC, we can now deactivate the Azure Domain with Active Directory sync using one of the two following selections:

3.a. VIA MS CONSOLES

O365 ADMIN - Click USERS, SET UP next to ACTIVE DIRECTORY, and disable synchronization.

AZURE ADMIN - Select your tenant, Click on DIRECTORY INTEGRATION, click DEACTIVATE.

AZURE PREVIEW ADMIN - Select INTEGRATION, click DEPLOY DIRECTORY SYNC, then proceed to next step to deactivate.

3. b. POWERSHELL

Essentially we use the below command to set directory sync to false.

Command: Set-MsolDirSyncEnabled -EnableDirSync $false

To check this occurred, use the following PowerShell command to validate

Command: Get-MsolCompanyInformation

NOTE: Within the results, look to the value for DirectorySynchronizationEnabled - it should be false.

Reference: https://msdn.microsoft.com/en-us/library/azure/dn144760.aspx

4. Again, assuming we are completely tearing down the Azure Domain Integration, we can now delete all users via one of the Azure or O365 Admin Consoles.

WARNING: DO NOT DELETE ADMIN ACCOUNTS!

NOTE: This may take multiple attempts to delete all users. Additionally, the AD Sync account may need to have the "User name" renamed from "Sync_<TENANT_ID>" to something like "Disabled" before it can be deleted.

5. Again, assuming we are completely tearing down the Azure Domain Integration, we can now delete the Azure/O365 Domain via one of the Azure or O365 Admin Consoles or via PowerShell on the AD Domain Member where AAD Connect tools are installed. (see below for powershell)

Command: Remove-MsolFederatedDomain -DomainName <string> [-SupportMulitpleDomain <Boolean>] [-Confirm] [-WhatIf] [<CommonParameters>]

6. Again, assuming we are completely tearing down the Azure Domain Integration, you may now uninstall the Azure AD Connect tools from the Activce Directory Domain member where they were installed. This is commonly listed as "Microsoft Azure AD Connect" in the Windows Add/Remove Programs wizard/applet.

NOTE: If the uninstall occurs correctly, this will remove all Azure tools and Azure PowerShell tools.

Once done, also remove any cached data within the registry of the AD member server where AAD Connect tools were installed (in the event you wish to redo the test from scratch).

6.a. Delete "%LOCALAPPDATA%\AADCONNECT".

6.b. Delete registry settings

"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ADHealthAgent"

"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Azure AD Connect"

7. At any point during this whole deactivation process, we can also delete any/all O365 apps from VMware Identity Manager / Workspace ONE Access which were initially created during this test process.

8. We must now wait for the Azure Domain deactivation to complete. This can take up to 24hrs but can be as quickly as an hour.

9. Again, assuming we are completely tearing down the Azure Domain Integration, we can also remove any TXT or MX records within the actual domain root DNS records. This is subject to Public DNS synchronization and DNS TTLs.

10. DONE

ADDITIONAL NOTES AND OPTIONS:

VMware Docs:

TechZone - https://techzone.vmware.com/understanding-microsoft-office-365-integration

O365 SAML - https://www.vmware.com/pdf/vidm-office365-saml.pdf

SAML Integrations Docs - https://www.vmware.com/support/pubs/vidm_webapp_sso.html

VMware Personal Blogs on Azure<>O365:

Workspace ONE and Azure AD: Part 1 - https://blog.virtualprivateer.com/workspace-one-and-azure-ad-1/

Workspace ONE and Azure AD: Part 2 - https://blog.virtualprivateer.com/workspace-one-and-azure-ad-2/

Workspace ONE and Azure AD: Part 3 - https://blog.virtualprivateer.com/workspace-one-and-azure-ad-3/

Azure AD Connect: Design Concepts (Using objectGUID or ms-DS-ConsistencyGuid and when to use one vs. the other and how to switch):

https://docs.microsoft.com/en-us/azure/active-directory/hybrid/plan-connect-design-concepts

NOTE: When installing Azure AD Connect with Express mode, the Azure AD Connect wizard automatically determines the most appropriate AD attribute to use as the sourceAnchor attribute

How to Bulk Convert Users if Keeping Azure Domain:

http://blog.bittitan.com/2012/01/24/office-365-powershell-tip-bulk-change-upns/

Get-MsolUser -all | Where { -Not $_.UserPrincipalName.ToLower().StartsWith(“admin@”) } | ForEach { Set-MsolUserPrincipalName -ObjectId $_.ObjectId -NewUserPrincipalName ($_.UserPrincipalName.Split(“@”)[0] + “@example.com”) }

MS Technet - DirSync: How To Switch From Single Sign-On To Password Sync:

http://social.technet.microsoft.com/wiki/contents/articles/17857.dirsync-how-to-switch-from-single-sign-on-to-password-sync.aspx#sample-script-to-manually-convert-all-users-in-a-domain

Azure Active Directory:

- https://www.microsoft.com/en-us/download/details.aspx?id=55271

- https://msdn.microsoft.com/library/azure/jj151815.aspx

SETTING UP SOURCE ANCHOR IN WORKSPACE ONE ACCESS:

When using sourceAnchor instead of objectGUID, prior to the directory in WS1 Access being configured, the User Attributes under Identity & Access Management > Setup > User Attributes need to have sourceAnchor defined and mapped to the correct Source Anchor / ImuutableID (i.e. objectGUID or mS-DS-ConsistencyGuid).

1. Within the Workspace ONE Access Admin Console, browse to Identity & Access Management > Setup > User Attributes and ensure "sourceAnchor" is defined there.

NOTE: It is possible "sourceAnchor" may not be there in some versions of Workspace ONE Access or VMware Identity Manager. If not, it can be manually added by clicking one of the green "+" signs.

2. Now, when creating the Directory within Workspace ONE Access, ensure the mapped attribute for "sourceAnchor" is defined to either "objectGUID" or "mS-DS-ConsistencyGuid". Which is used will be based upon the on-premises Active Directory settings and Azure AD Connect synchronization settings defined.

WARNING: It is recommended to do User Attribute changes within Workspace ONE Access PRIOR to creating a Workspace ONE Access Directory linked to Active Directory as making any attribute a REQUIRED attribute or removing ANY attribute will not be an option until the Workspace ONE Access Directory is deleted (and subsequently recreated after the User Attribute changes take place)!

a. Browse to Identity & Access Management > Manage > Directory and if no directory exists, create a new directory (ADD DIRECTORY) and ensure on the User Attributes/Mapped Attributes section has the following attributes mapped.

1. userPrincipalName -> userPrincipalName

NOTE: This is needed by O365/Azure domains along with the Source Anchor (below).

2. sourceAnchor -> (either objectGUID or ms-DS-ConsistencyGuid)

NOTE: Most of the Identity Industry defines this as Source Anchor while Microsoft defines this as ImmutableID. They are the same thing.

2. Finish any additional mapping settings to create or update the directory and click SAVE/NEXT/SYNC on whichever wizard in order to finish confirmation of the settings to add in the above desired attributes.

3. DONE

MICROSOFT DOCUMENTED SMART LINKS / DEEP LINKS:

Once an anchor O365 application is in place within Workspace ONE Access (for SSO), one can then create smart links or deep links to specific resources such as PowerPoint, Word, Excel, or other resources. This is done in order to simply provide an easy link for users which conducts Single Sign-on directly to the resource the user needs once authenticated to Workspace ONE Access rather than logging into Workspace ONE Access, then selecting O365, gaining access to the initial O365 panel of resources, and from there clicking on the specific app.

How this is done within Workspace ONE Access is simply by browsing to the Web Apps section of the Catalog, selecting the NEW button to create a new web app, typing in a name (no need to search - just type in name), uploading an icon (click NEXT), and then selecting "Web Application Link" for the Authentication Type and pasting in one of the below Microsoft Smart/Deep Link URLs for the specific resource. Once done, don't forget to set any categories and assign users/groups within the WS1 Access Administrator console.

Here are the Microsoft Documented Smart/Deep Links at last update of this document.

Azure Admin Center:

https://login.microsoftonline.com/login.srf?wa=wsignin1%2E0&rver=6%2E1%2E6206%2E0&wreply=https%3A%2F%2Fportal.azure.com%2Fsignin%2Findex&whr={your_domain_here.com}

Exchange Admin Center:

https://outlook.office365.com/ecp/?rfr=Admin_o365&exsvurl=1&mkt=en-US&Realm={your_domain_here.com}

Excel:

https://login.microsoftonline.com/login.srf?wa=wsignin1%2E0&rver=6%2E1%2E6206%2E0&wreply=https%3A%2F%2Foffice.live.com%2Fstart%2FExcel.aspx%3Fauth%3D2&whr={your_domain_here.com}

OneDrive:

https://login.microsoftonline.com/login.srf?wa=wsignin1%2E0&rver=6%2E1%2E6206%2E0&wreply=https%3A%2F%2F{your_tenant_here}-my.sharepoint.com%2F&whr={your_domain_here.com}

Outlook Web Access:

https://outlook.com/owa/{your_domain_here.com}

PowerPoint:

https://login.microsoftonline.com/login.srf?wa=wsignin1%2E0&rver=6%2E1%2E6206%2E0&wreply=https%3A%2F%2Foffice.live.com%2Fstart%2FPowerPoint.aspx%3Fauth%3D2&whr={your_domain_here.com}

SharePoint:

https://login.microsoftonline.com/login.srf?wa=wsignin1%2E0&rver=6%2E1%2E6206%2E0&wreply=https%3A%2F%2F{your_tenant_here}.sharepoint.com%2F&whr={your_domain_here.com}

Teams:

https://login.microsoftonline.com/common/oauth2/authorize?client_id=cc15fd57-2c6c-4117-a88c-83b1d56b4bbe&response_mode=form_post&response_type=code+id_token&scope=openid+profile&redirect_uri=https%3a%2f%2fteams.microsoft.com%2f&domain_hint={your_domain_here.com}

Word:

https://login.microsoftonline.com/login.srf?wa=wsignin1%2E0&rver=6%2E1%2E6206%2E0&wreply=https%3A%2F%2Foffice.live.com%2Fstart%2FWord.aspx%3Fauth%3D2&whr={your_domain_here.com}

Office 365 Admin Center:

https://login.microsoftonline.com/common/oauth2/authorize?client_id=00000006-0000-0ff1-ce00-000000000000&response_mode=form_post&response_type=code+id_token&scope=openid+profile&redirect_uri=https%3a%2f%2fadmin.microsoft.com%2f&domain_hint={your_domain_here.com}

Power BI:

https://app.powerbi.com/?noSignUpCheck=1&domain_hint={your_domain_here.com}

MyApps Service Provider Example:

https://myapps.microsoft.com/{your_domain_here.com}/signin/{Service}%20{Name}%20{Here}/{Application_Guid_Goes_Here}

NOTE: Big Thanks to Matt Williams over at VirtualPrivateer for posting these all in one place.

Work Around MS Teams Lack of AGC on macOS

2020-10-09T11:55:00.005-07:00

Intro:

For those of us working from macOS and being required to use Microsoft Teams, you may have noticed that Microsoft Teams will automatically adjust the microphone to "make things easier" for those users who don't have any audio equipment or knowledge of how to use attached (or even built-in) audio equipment by automatically adjusting the Automatic Gain Control on the microphone within the Mac's System Audio Settings.

For anyone who uses attached audio equipment or has custom audio settings and has preferred microphone input levels, MS Teams will ignore this and override all manual gain control settings on macOS and take control of input and use Automatic Gain Controls. This can severely diminish the experience of MS Teams and potentially prevent the use of MS Teams on a macOS system where custom audio levels are required.

Workaround:

While I cannot take credit for this information (I found this posted as a comment by "chornsec" on a Microsoft Tech Community post), below is a viable workaround for making MS Teams work on macOS without implementing it's own Automatic Gain Controls (AGC).

Procedure:

1. Open 'Audio MIDI Setup' application. You can use the macOS Search feature to find the app.

2. In the left pane, click + symbol in the lower left corner and choose Create Aggregate Device.

3. Ensure that the new Aggregate Device is selected in the left pane.

4. In the main window area, select the 'Use' checkbox for your real input device. ONLY SELECT ONE SOURCE for input as with aggregate devices, the other sources will be relegated to other audio channels.

5. Select the real input device from the left pane and set either the Master Stream input or the individual channel gains correctly.

NOTE: On my system, this was a USB Sound Device (and note that I selected the "2 ins" and not the "2 outs"). Additionally, I always set my inputs to 50%. On my system, I cannot use "Master" as the channels are set individually - but this is not a big deal.

6. Go back to your new Aggregate Device and rename it by clicking its entry in the left pane.
In mine, I renamed it to MXL 990.

7. Quit & restart the MS Teams application if it was open during this process.

8. In MS Teams, choose the newly created aggregate device as the microphone device.

Benefits:

Since the macOS Aggregate Device does not have a gain setting, MS Teams cannot use AGC to adjust the Microphone input.

Rebuilding VMware Identity Manager or VMware Workspace ONE Access On-Premises Appliance vApp Application Properties

2020-07-30T19:40:00.011-07:00

Rebuilding Workspace ONE Access vApp Option Settings

There has been a few times which I or a customer has needed to rebuild the vApp Options Properties for an On-Premises deployment of VMware Workspace ONE Access (formerly known as VMware Identity Manager). Here are the steps to rebuild the necessary properties within the on-premises deployment to make things work again.

Why would someone need to do this?

Well, let us say your environment unfortunately lost its vCenter server due to an upgrade issue or some other unexpected failure. That's a horrible thought, I know...but it's not catastrophic. You can really either restore vCenter from backup, or build a new vCenter.

I know what you're thinking... "I have backups of my vCenter configuration!" That's great...really! But have you tested them? Do they work during a restore? Mine didn't.... <---Insert frowny face here.

This meant my only option left was to build a new vCenter from scratch, import my vSphere hosts, and reconfigure the lost settings. Sounds easy, yes? Well, mostly, it was. Except when I checked in on my VMware Workspace ONE Access on-premises tenant. It stores all of its major settings in vCenter within each of the VM's vApp Options Properties values. Yeah....major frowny face.

What does this last bit really mean? It means VMware Workspace ONE Access - when installed on-premises - does not know how to boot properly when vCenter no longer exists (It can run without vCenter on VMware Workstation or VMware Fusion, but it is coded to detect this from install and it makes the necessary accommodations during install to allow for running without vCenter - but when vCenter existed initially and no longer exists, all settings are lost).

What settings am I really talking about? These (below)...

So how do we get these settings back without having to completely reinstall and reconfigure VMware Workspace ONE Access?

Well, I'm glad you asked. And it is actually not terribly hard to do. First though, while I'm not going to go through the step-by-step (that's not really necessary), there are a few things you need to do.

If this is production, schedule an outage for all of Workspace ONE. Give yourself a few hours.
I'm also assuming you have vCenter completely restored and working properly (and making backups which work).
And, I'm going to assume you know your way around vCenter and how to make VM Snapshots (you'll need it if you toast one of your on-premises Workspace ONE Access SLES appliances).
With that out of the way, it is now time to start editing each of your on-premises SLES appliances.

NOTE: From here on out, I'm assuming you are using at least vCenter 6.7 or higher and navigating through the HTML5 admin UI.

Enable vApp Options

Within the vCenter Admin Console, go ahead and browse to the VM, select the Configure tab at the top of the VM page, and select vApp Options under the Settings menu on the left side of the VM settings page. From there, you can select EDIT (on the right side of the window) and enable vApp Options for each of the SLES VMs.

There are a couple of specific settings needed here.

Check the box to Enable vApp options.
Under the IP Allocation tab ensure Authoring IP Protocol is set to Both, Authoring IP allocation scheme has both DHCP and OVF environment unchecked, and Deployment IP Protocol is set to IPv4 and Deployment IP Allocation is set to Static - Manual.
Under OVF Details tab ensure ISO Image is unchecked, VMware Tools is checked, and Installation boot is unchecked.
NOTE: DO NOT CLICK OK YET!
Under the Details tab ensure Name is set to "IdentityManager" (no space and case sensitive) and Vendor is set to "VMware, Inc."
Once you have the above set, click OK to save and then the rest is just inputting the properties key and values.

NOTE: This is entered under AUTHORING>PROPERTIES section. It then builds the Application and Network Properties sections at the top of the vApp Properties under "Application properties - IdentityManager”. Once the below is entered, its fields must be edited in the “Application properties - IdentityManager” section prior to booting up.

Make sure to refer to the image at the top for what this should look like once entered.

NOTE: Slide the graph to the right to see all remaining fields.

Key ID

Label

Category

Type

Key class ID

Key Instance ID

Description

Static Property Type

Static Property

Length

Static Property

Choice List

Static Property

Default Value

Static Property

User Configurable

Dynamic property

Macro

Dynamic property

Network

vmname

String

0 - 65535

IdentityManager

[]

ceip.enabled

Join the VMware Customer Experience Improvement Program.

Application

boolean

Send anonymous customer experience data to VMware.

Boolean

True

[x]

vamitimezone

Timezone setting

Application

String

Sets the selected timezone.

String choice

"Pacific/Samoa", "US/Hawaii", "US/Alaska", "US/Pacific", "US/Mountain", "US/Central", "US/Eastern", "America/Caracas", "America/Argentina/Buenos_Aires", "America/Recife", "Etc/GMT-1", "Etc/UTC", "Europe/London", "Europe/Paris", "Africa/Cairo", "Europe/Moscow", "Asia/Baku", "Asia/Karachi", "Asia/Calcutta", "Asia/Dacca", "Asia/Bangkok", "Asia/Hong_Kong", "Asia/Tokyo", "Australia/Sydney", "Pacific/Noumea", "Pacific/Fiji"

Etc/UTC

[x]

DNS

Networking Properties

String

vami

IdentityManager

The domain name servers for this VM (comma separated). Leave blank if DHCP is desired. All fields but hostname are required for static IP.

String

0 - 65535

[x]

gateway

Default Gateway

Networking Properties

String

vami

IdentityManager

The default gateway address for this VM. Leave blank if DHCP is desired. All fields but hostname are required for static IP.

String

0 - 65535

[x]

domain

Domain Name

Networking Properties

String

vami

IdentityManager

The domain name of this VM. Leave blank if DHCP is desired.

String

0 - 65535

[x]

searchpath

Domain Search Path

Networking Properties

String

vami

IdentityManager

The domain search path (comma or space separated domain names) for this VM. Leave blank if DHCP is desired.

String

0 - 65535

[x]

vami.hostname

Host Name (FQDN)

Networking Properties

String

The FQDN name for this VM. Leave blank for DHCP or reverse DNS to be used to lookup hostname.

String

0 - 65535

[x]

ip0

IP Address

Networking Properties

String

vami

IdentityManager

The IP address for this interface. Leave blank if DHCP is desired. All fields but hostname are required for static IP.

String

0 - 65535

[x]

netmask0

Netmask

Networking Properties

String

vami

IdentityManager

The netmask or prefix for this interface. Leave blank if DHCP is desired. All fields but hostname are required for static IP.

String

0 - 65535

[x]

Apple Script/App for Managing Server Fans via IPMI

2020-07-30T17:11:00.001-07:00

Intro:

Below is an Apple Script I wrote up and compiled to control the fans on my servers via the Dell’s IPMI (iDRAC) interface. This uses HomeBrew for macOS as well as the “ipmitool” cask. Instructions are embedded in the script if you do not have both HomeBrew and the “ipmitool” cask installed.

Prerequisite:

In order to use the script, you need to enable IPMI over LAN on the iDRAC network setting.

Installation:

Simply copy down the SCPT file attached here and either paste or open in Apple Script Editor.
Reset the SERVER NAMES AND IP ADDRESSES BEFORE EXECUTING! If you remove one or two servers, don’t forget to comment out or modify the any lines where all three servers are referenced.
The first time you run it, it will prompt to install HomeBrew and the “ipmitool” cask if you don’t already have those on your system.

Installing HomeBrew and ipmitool:

You may manually install HomeBrew and the “ipmitool” cask by using these instructions.
Launch Terminal
Install HomeBrew
1. HomeBrew Install Command:
  
  /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Install ipmitool
1. ipmitool install:
  
  brew install ipmitool

This script will:

Check for HomeBrew and “ipmitool” cask and redirect you to the correct installation instructions if you do not have these.
Allow you to choose between pulling the stats of all servers defined (I currently have it coded for 3 hosts but it is a simple copy/paste to add more hosts).
Allow you to set fan speeds automatically by choosing LOW or HIGH fan speed offsets…
…or Allow you to manually set fan speeds to any percentage of your choosing (between 20% and 100%).

Script UI Screenshots:

Compiling:

If you wish to compile it, simply drop it into Apple Script Editor and export as an application (select “Application” in the File Format) and any other options you desire and hit SAVE.

Script Options:

The script (in it’s current state) will prompt for user credentials each and every time you run it.
If you wish to set your IPMI Admin and Password into the script, the variables are already defined (as IPMIAdmin and IPMIPwd) and set to blank.
If you wish to add more servers, simple add in additional SERVER#IP and SERVER#NAME variables and duplicate the commands

The Apple Script:

ScreenCast Link

VMware Workspace ONE Access User Sync Configurations and Exclusions

2020-07-30T15:24:00.014-07:00

There are a number of reasons which may cause users to not be able to authenticate into Workspace ONE Access (formerly known as "VMware Identity Manager"). Of the varying possibilities, it is important to understand not only the various ways a user (or group) may be synched from Active Directory, it is also important to understand how to exclude specific users to maintain an automated sync and a solid working Workspace ONE Access tenant. Below are points to help ensure you don’t fall prey to the most common reasons.

User Synchronization Misconfigurations:

1. Base DNs not defined/not properly defined - When base DNs for user synch are set too deep in the Sync Settings [within the User tab] or when just synching specific user accounts (e.g. "CN=John Doe,CN=Users,DC=Corp,DC=local”). It should be noted, base DNs do not necessarily have to be set, but it is a good practice to do so, as some accounts may not be synched otherwise. Use of filters on this one is recommended to remove any red synch errors.

2. Using Group Membership and Nested Group Membership to Sync Users - I’ve also seen this when not synching users directly but rather when using groups to sync users. Specifically, when using the “Sync nested group members” option (If this is not set, users will not be synched). It should be noted using this configuration solely for synching users is not the best option as it is more common that users will be missed than it is for users to “accidentally” gain access to something. Therefore it is recommended to use this setting in combination with defining base DNs to synch users from.

3. Horizon Provisioning To Sync Users - This option was only available in older version of VMware Identity Manager (prior to 19.03). It is similar to the second configuration option - in that when defining no users or groups to synch within Identity Manager Directory configurations and just using the “Perform Directory Sync” Horizon setting (this must be enabled) means any users who do not have a Horizon resource provisioned will not be synched into Workspace ONE Access. It should be noted using this configuration solely for synching users is not the best option as it is more common that users will be missed than it is for users to “accidentally” gain access to something. Therefore it is recommended to use this setting in combination with defining base DNs to synch users from.

4. Only User Accounts which are Provisioned Resources are Synched - This is a feature of Workspace ONE Access and VMware Identity Manager 3.X and higher which is to ensure initial A.D. synch did not overwhelm production Active Directory servers or take excessively long to synch users. Ultimately this means any user who does not have a Workspace ONE Access resource actually provisioned - whether it be SaaS, Web, ThinApp, Horizon, or Citrix - will not be synched into Workspace ONE Access until they have a resource assigned to them. To determine if this is the case, the easiest way to work around this is to utilize group assignment and define a resource to either an Active Directory group which the user is a member of AND that has also been synched into Workspace ONE Access OR use the Workspace ONE Access built-in ALL USERS group (a word of caution as doing so may add many more users to Workspace ONE Access).

NOTE: Users who may have had a resource provisioned to them at one point but have all provisions removed will not automatically be removed from Workspace ONE Access.

Preventing Sync Errors Through User Exclusions:

After defining User DNs for synching users, and once synchronization is successful, you’ll likely start to notice sync alerts in red showing either within the test run or during the actual directory sync. During the test run, these will show within a red box using black text and indicate why these are occurring (e.g. the account doesn’t have a valid email address, a valid first name or last name, etc.).

After the actual sync, within the Sync Log tab, sync alerts will show as red text within the Sync Alerts log, and again will state the reason why. It should be noted, a sync may have alerts regardless if it successfully completed the sync or not.

These sync alerts must be resolved in order for automated, periodic directory syncs to continue working. The reason why is, after accumulating 1000 sync alert errors, automated syncing will stop until directory settings are re-saved. Therefore, it is imperative to alleviate these sync alerts - and the only means for doing so is by using the user exclusions.

It is best practice to add in filters as broadly as possible in order to easily capture as many accounts which should not be synchronized into Workspace ONE Access as possible.. However, take care not to catch any valid accounts with your exclusions as this will result in REMOVING these user accounts from Workspace ONE Access and prevent them from logging in.

In the example above, all accounts in my test domain which should not be synchronized did not have a valid email address, therefore I set the “Mail…does not contain…’@‘“ rule so that any account it found (i.e. a Windows admin or service account) which does not need user interactive login to Workspace ONE Access is prevented from synchronizing.

Conclusion:

While it is necessary to check users are properly synching, it is also required that accounts which do not need to be synched (e.g. service accounts, etc.) are properly excluded...and that sync alerts are not causing automated sync failures and additional problems down the road.

Resolving the macOS Catalina Software Update Flag Always On Issue

2020-05-12T12:02:00.000-07:00

Issue:

I just recently updated my iMac to Catalina (finally) as I finally eliminated or upgraded all of my 32-bit apps. After upgrading, my macOS decided it would continue showing that I needed upgrades, even though I had the latest updates applied.

Even going into Terminal and running softwareupdate --list didn't show any additional updates.

Resolution:

To fix this, I did the following to reset the Software Update tag when Catalina gets stuck thinking it needs to update after it has already been updated to the latest..

Quit System Preferences if open.
Set macOS Catalina as an ignored update.

Within Terminal, run this command with quotes. You will need to type your system administrator password.

sudo softwareupdate --ignore "macOS Catalina"

Clear out the last major OS release seen by the system.

Within Terminal, run this command.

defaults delete com.apple.preferences.softwareupdate LatestMajorOSSeenByUserBundleIdentifier

Tell macOS to rescan for updates.

Within Terminal, run this command. You will notice the badges disappear at this point, but continue on.

softwareupdate --list

Clear out all ignored updates (if you only ever did this within this procedure then you’re basically reversing step 1.).

Within Terminal, run this command. You may need to type in your system administrator password.

sudo softwareupdate --reset-ignored

Tell macOS to rescan for updates.

Within Terminal, run this command. You will notice the badges disappear at this point, but continue on.

softwareupdate --list

You should now be able to open System Preferences and re-enable software updates and not have it re-flag the Software Updates icon.

Credits:

Thanks to bernuli on the MacRumours forum for the bulk of this.

Workspace ONE Access User Authentication Flows - Configuring Password (Cloud Deployment) From Start to Finish

2020-05-07T21:10:00.000-07:00

Introduction:

This is a quick and rough guide to enabling “Password (cloud deployment)” in Workspace ONE Access.

Outcome:

This procedure will allow a customer admin to change from direct Password authentication via Workspace ONE Access Connector(s) to indirect Password (Cloud Deployment) authentication via Workspace ONE Access Connector(s).

Assumptions:

Workspace ONE Access tenant exists
Workspace ONE Access admin login capable

Logic Flow:

Within the Workspace ONE Access admin console, navigate to Identity & Access Management > (Manage) > Identity Providers. Does Workspace ONE Access Built-in Identity Provider exist?

Yes, continue on to major step 2.0.
No.

Create a new “Built-in” IDP by clicking the ADD IDENTITY PROVIDER button in the upper right. SEE THE NOTES ON CREATING OR RECREATING TEH BUILT-IN IDP at the bottom of this blog post.

RETURN HERE WHEN DONE.

Is there a Connector assigned to the Built-in IDP?

Figure 1: Is there a Connector Assigned to the Built-in IDP?

Yes, continue on to major step 3.0.
No

Go into Identity & Access Management > Identity Providers > Built-in (or whatever it might be renamed to). You should see a box in the Connectors section to select (and then add) your connector(s).
After selecting your connector, click Add Connector.
Repeat these three steps for each connector you wish to add.

Does the Password (Cloud Deployment) box now appear under Connector Authentication Methods within the Built-in IDP (Refer to Figure 2 below as a reference)?

Yes, continue on to major step 4.0.
No, troubleshoot major Step 2.0 above.

NOTE: You may need to click the SAVE button and return back to the Built-in IDP screen in order to see a box. If one does not appear, ensure your connector(s) are properly added and show with a red X (which allows for deletion).
Open a support ticket with VMware Support if necessary.

Is the Password (Cloud Deployment) box checked?

Figure 2: Is the Password (Cloud Deployment) box checked?

Yes, continue on to major step 5.0.
No, check the Password (Cloud Deployment) box.
Click the SAVE button at the bottom of the Built-in IDP screen.

NOTE: From this step, you may use the VMware documentation which exists to walk through enabling the Workspace ONE Access Policies to utilize Password (Cloud Control).

Within the Workspace ONE Access admin console, navigate to Identity & Access Management > (Manage) > Policies and open the default_access_policy_set.
Edit the default_access_policy_set and navigate to step "2 Configuration” within the EDIT POLICY wizard.
Open each policy rule one at a time.

Does the policy rule show “Password” as the authentication type in the “then the user may authenticate using” drop-down box?

If NO, then continue on to the next policy rule.
If YES, then modify the policy rule to replace Password with Password (cloud deployment) and click SAVE.
If no more policy rules, click NEXT and SAVE on the EDIT POLICY wizard.

Figure 3: Edit each Policy Rule and replace PASSWORD with PASSWORD (CLOUD DEPLOYMENT)

Do the Password (Cloud Deployment) modifications now work locally and remotely?

Yes. You are finished.
No. Troubleshoot policy modifications in major steps 6 and 7 above. If necessary open a support ticket with VMware Support.

Resources:

https://docs.vmware.com/en/VMware-Workspace-ONE-Access/20.01/ws1_access_authentication/GUID-E4582D91-506B-4AA6-9F1F-EC9F7775DA49.html

Notes on Creating (or recreating) the Built-in IdP:

Assuming the default “Built-in” IdP was deleted or not created from the start. one can just create a new “Built-in” IDP.

IF YOU ALREADY HAVE A BUILT-IN IDP, skip to step 4.
Navigate to Identity & Access Management > Identity Providers and click the ADD IDENTITY PROVIDER button in the upper right of the page and then selecting the option in the menu to create a built-in IdP. *
Name it (e.g. "Built-in"), set the Users Directory (e.g. your Active Directory integration - DO NOT select System Directory), select the Network(s) such as ALL RANGES, select the Authentication Methods if needed, and click ADD.
Now go back into Identity & Access Management > Identity Providers > Built-in (or whatever it might be renamed to). You should see a box in the Connectors section to select (and then add) your connector(s).
Once you finish adding your connector(s), you will see “Password (cloud deployment)” show in the Connectors Authentication Methods section. Check it and click SAVE.

*NOTE: Other than "System Directory", for each Directory you add in Identity & Access Management > Manage > Directories, you should also have a "Built-in" identity provider specific to that Directory. Here's the catch... they cannot all be named "Built-in", so name them " Built-in" or similar.

Cross Organization Identity Brokering Using VMware Workspace ONE Access 3rd Party IdP Integration for Multi-Organization Trust & SSO

2020-04-24T18:43:00.000-07:00

Intro...

As a Customer Success Architect focused on Workspace ONE, I have often been asked intriguing questions such as, can one use two different Identity Providers which trust each other to take the place of some sort of trust or connection. My longtime coworker, Peter Bjork, wrote about this in regards to replacing Active Directory trusts over on VMware's TechZone awhile back from a conceptual standpoint...but I thought I would go into the architecture design and considerations aspects and what this looks like from a VMware Customer perspective as I (and some of my coworkers) have helped some customers actually deploy this in production.

Problem (Business Use Case):

More and more customers are joining forces to work together either through mergers and acquisitions or as partnerships. With these mergers and partnerships, customers are needing to share resources. This now must be done through single sign-on mechanisms as industry security brokers are changing from on-premises (active directory) to cloud-based solutions. More importantly, customers are not wanting to go the legacy route of providing complex, 2-way active directory trusts, but rather wish to use cloud based, open standard, secure solutions which exist in the marketplace today.

Executive Summary:

Utilizing a Cross Organization Identity Brokering solution allows for disparate companies to become technology partners by providing a secure identity management solution between trusted partners which replicates only the necessary attributes of a user’s identity throughout in order to not only provide access to services, applications, and desktops from partner companies, but allow for the user to have a fully seamless, secure, single sign-on experience via their own VMware Identity Manager portal to all resources across partnered companies.

In addition, this also provides the partnered companies to ensure their own environments remain protected to the necessary levels as well as an easy way to separate linked resources, should the partnership dissolve.

High Level Design:

Benefits

So what does this all mean? Well, I'll let you read through all of the major benefits below in detail, but in short, the above diagram shows a user from either org being able to login to their own org Workspace ONE Intelligent Hub App Catalog and have a resource from a partner org delivered into their portal. They don't need to know anything more, anything new, or do anything different. They login, get their apps (resources), and use them just the same - from any device they are authorized to access these resources from, anywhere they are authorized to access these from, regardless of where the resources are.

Here is a bulleted list of benefits.

Allows for users in either company (entity) to access applications, desktops, and services offered or made available by the other company / entity while still maintaining Single Sign-On functionality through their own company’s tenant.
Allows for individual companies to securely provide access to resources they own or control to users in other “trusted” partner companies without creating and maintaining an Active Directory or other security broker trust relationship.
Allows for automation of Identity synchronization through use of 3rd Party identity management solutions.
Allows for designation of multiple security checkpoints within both companies to ensure a user is properly identified, authenticated, and authorized while maintaining security and SSO functionalities.
In addition to SSO functionalities for all users across multiple entities, allows for a single, self-service portal to present all resources available to the user from across multiple tenants and providers.

One other thing to note... should there be any other additional Identity Providers (IdPs) in the mix (and Workspace ONE Access is just the Universal Services Broker), those IdPs and the resources they protect can also be accommodated in the above design.

Sure, it looks a bit complicated...but it can just work.

Challenges, Concerns, Blockers, and Requirements:

Speaking of it looking complicated, let us dig into the challenges, concerns, blockers, and requirements.

Challenges:

First off, it is more complex. We are not talking about one organization's environment here, but two organization environments! At very least, that is double the complexity... but have no fear. There are smart people to help with this. And a large part of what we will get into below regarding Concerns, Blockers, and Requirements are mostly People/Process challenges - something that - with open communications - can be solved.

The second challenge is education - education on how the above solution works, what pieces and parts are needed, and what things need to be agreed upon.

Design Data: The design flow above has two major parts - and they are not what you might think (I'm guessing you may be thinking ORG1 and ORG2). The two major parts of the design are actually the top half and bottom half! Just split the diagram horizontally across the middle.

Top Half: The top half is, quite simply a standard 3rd Party Identity Provider trust configuration. We go into 3rd Party IdPs in detail over on my VMware Community Blog post here. The only difference is, we have two Active Directory or LDAP (or security broker database) configurations. Quite simply, this means any resource within the Workspace ONE Access catalog in Org 1 may be shared with any user who is properly replicated from Org 2 (into the Org 1 trusted secondary domain). This includes any SAAS, Web, or Virtual resource such as a Horizon app or VDI Desktop. This also includes any MFA requirements either organization may have for users from either organization.

NOTE: Workspace ONE UEM resources may NOT be shared across orgs and remain org specific only! This is specifically due to device enrollment - and is very likely not something you or any other customer would want to share with a partner org.

Bottom Half: The bottom half is simply user replication (what we call true "Identity Management") between various environments. Identity Replication is where the real challenge is...because not only is it technically challenging, it is also legally challenging when the two organizations are completely separate entities.

Identity Replication:

The way we solve the technical challenge of Identity Replication in order to assist and support the legal concerns with Identity Replication is shown in the diagram above. Basically, in order to replicate identities properly, we utilize a Virtual Directory Structure (VDS) which is put in place to properly synchronize identities from certain sources to other certain destinations. And in conjunction with this, we don't just want to allow the VDS to create a replicated identity in other organization's primary security broker (e.g. Active Directory, OAM, etc.). Doing so without providing some checks and balances leaves the entire system ripe for exploit. Therefore, rather than create the "replicated" identity into the partner organization's primary domain (or security database), we use a standalone secondary domain with a (usually filtered) two-way trust in place between the organization's primary and secondary domains. Assuming this will be completely 2-way across both organizations, this configuration allows for both partner organizations to ensure there are ways to a.) block users from accessing primary domain resources they should not be allowed to access, and b.) turn off all access quickly and efficiently at any step.

Let us assume the trusts are setup and step through Identity Replication flows here. This means the following assumptions exist and/or have been completed:

Primary ORG1 > VDS > Standalone ORG2 automated replication is working.
Primary ORG2 > VDS > Standalone ORG1 automated replication is working.
A SAML IDP to SP Trust exists between ORG 1 Workspace ONE Access and ORG 2 Workspace ONE Access.
A SAML IDP to SP Trust exists between ORG 2 Workspace ONE Access and ORG 2 Workspace ONE Access.

NOTE: SAML IDP<>IDP trusts are one way and therefore two one-way trusts must exist for flows to work from either organization.
Both ORGS have added their respective secondary (or standalone) domains to the their respective Workspace ONE Access Directory configurations for user sync and authentication.
Any shared resources within each respective organization are automatically mapped across to the other organization's Workspace ONE Access environment.

ORG 1 and ORG 2 IT Departments have previously agreed and stood up settings to allow for automated account creation within their respective Standalone (or secondary) domains.
Optionally, ORG 1 and ORG 2 IT Departments have previously agreed and configured Group synchronizations.
In my logic flow walk-through below, I am assuming both organizations have on-premises Active Directory. That is not a requirement, however, as most any LDAP connection would work, so long as the supporting resources behind the Identity Providers support the provisioning via LDAP or otherwise.

NOTE: VMware Horizon only supports provisioning through on-premises Active Directory at the time of writing this blog.

Starting with ORG 1 Primary Domain environment in the top left, let us assume we have a user (Jane) which needs access to a Horizon resource in ORG 2.

ORG 1 IT Dept tags/flags Jane's Active Directory account with a setting to ensure VDS sees the account as one which should now be replicated.
VDS automatically captures Jane's Identity in ORG 1 and replicates it to the ORG 2 standalone (or secondary) domain.
ORG 2 Workspace ONE Access sees the new user on the next automated (or manual) directory sync and creates the new user in ORG 2 Workspace ONE Access Directory.
Application Provisioning: Either ORG 2 manually provisions the Horizon resource to Jane (either to her user account or by adding her user account to a group in ORG 2 environment) or ORG 1 and ORG 2 optionally stood up group synchronization as well to allow for automated provisioning.
After ORG 2 provisioning sync has occurred between ORG 2 Horizon and ORG 2 Workspace ONE Access, the Horizon resource is authorized to Jane's ORG 2 account in ORG 2 Workspace ONE Access catalog.
Application Provisioning: ORG 1 manually (or automatically) provisions the ORG 1 mapping of the ORG 2 Horizon resource to Jane's ORG 1 account within ORG 1 Workspace ONE Access.
After ORG 1 provisioning sync has occurred, Jane now has the ORG 2 Horizon resource within her ORG 1 Workspace ONE Access catalog.
When Jane launches the ORG 2 Horizon resource in her ORG 1 Workspace ONE Access catalog, SSO kicks in because she has an Identity in ORG 2 with a mapped attribute and both Workspace ONE Access tenants trust each other, allowing the resource to be immediately accessed.

Concerns (Blockers) and Necessities (Requirements):

Here I have listed out the many concerns and potential blockers as well as requirements here to help organizations work through these.

Some sort of VDS solution should be used to automate the synchronization of selected identities and groups and their defined attributes within each source AD. In some cases, we have seen this manually done, but when enterprise organizations of significant size start counting through the amount of users and multiply each user by the number of attributes being synched as well as groups and group member info, this can very quickly get overwhelming for a manual process. And a manual process is certainly very open and prone to errors. Using an automated process such as Microsoft Identity Manager, Radiant Logic, or other automated Identity Lifecycle Management tools from IBM or others (or even some self created scripted solutions) will be much better than a manual process.

This combined set of users and potentially groups will become the Virtual Directory Structure (VDS).

Users within VDS will have a defined set of agreed upon attributes between both organizations.

First name
Last name
Email
UPN
objectGUID
NT Account Name (sAMAccountName)? - Some older applications need to utilize these for authentication into the app.
Password??? - This might not be needed as 3rd party IdP trust will not use this but trusting services might still require it. Radiant Logic VDS can sync password but this means password policies will need to be an agreed upon standard between entities.

Note: Horizon TrueSSO would eliminate the need for password sync if both sides used VMW Horizon.

Source entity (ORG1 or ORG2 - can be automatically derived from email or UPN)
Member of (or Group Membership).
Etc. (phone, office location, etc.)

VDS will be synced into local Active Directory domain in each entity environment.

Example: VDS.local
VDS to AD domain sync may need to be selective per entity in order to avoid duplicate entries.

Example: ORG1 users synced into VDS from ORG1 domain, do not need to be synced into VDS.local AD domain as those won't be utilized anyways, and may potentially cause duplicate email and UPN entries between VDS.local and org1.local.

Local 2-way trust between VDS.local and entity primary domain will exist to allow trusted services to perform Windows authentication (i.e. Horizon or Citrix session authentication).
A manual 2-way domain trust with SID filtering may be needed to ensure user accounts from other entities do not have any additional access to other resources.
Citrix will not fully support TrueSSO-like functionality at this time and will prompt for a user password for current and immediate logins to Citrix via Workspace ONE.

A password will only be prompted once for each ws1 login. If a user logs in to ws1 only once per day then a password prompt will only happen once per day if timeout settings are longer than that.

Both Identity Manager tenants will need to trust each other. This is a standard SP to IdP trust between each Identity Manager.
Admins in both entities will need to have deep links (i.e. Relay state links) of 3rd party IdP protected apps in order to properly define the same app in both.

Example: App1 is horizon app in ORG2 domain. In order to properly display and allow for SSO for users from ORG1, App1 must be manually created in ORG1 Identity Manager as a 3rd party app and properly assigned.
Note: Assignment of apps can be done either by dynamic groups or by synced groups. Further discovery on this may be needed but shouldn't be an issue.

All customers replicating into VDS and utilizing replicated user accounts for SSO should have independent security audits to ensure all best practices and requirements are adhered to.

Technical Details:

Domain Trust and SID Filtering:

Ensure we are not sharing SIDs across domains. A potential malicious administrator could get additional access to the other domain. This eliminates lateral movement of an exploited system. SID filter quarantining for external trusts.
Implement selective authentication across the trusts.

Authentication Setting	Interforest Trust Type	Description
Domain-wide Authentication	External	Permits unrestricted access by any users in the trusted domain to all available shared resources located in the trusting domain. This is the default authentication setting for external trusts.
Forest-wide Authentication	Forest	Permits unrestricted access by any users in the trusted forest to all available shared resources located in any of the domains in the trusting forest. This is the default authentication setting for forest trusts.
Selective Authentication	External and Forest	Restricts access over an external or forest trust to only those users in a trusted domain or forest who have been explicitly given authentication permissions to computer objects (resource computers) residing in the trusting domain or forest. This authentication setting must be manually enabled.

Outcomes:

So, again, what does this all mean? In a nutshell, employees who are properly authorized will be able to login to their own organization tenant and maintain Single Sign-On access to protected resources across both environments...AND... IT organizations get to sleep well at night knowing they are providing something which is complex in design but easy for users to securely access as Workspace ONE helps ensure all of this just works.

Is this easy to do? Maybe not... but when you make it easy for your employees to access the resources they need from the locations they need to work from while maintaining proper security, you know your users are happily using the solution without trying to circumvent it to make their job easier.

Resources:

The below is a video I created of this a couple of years ago demo'ing the whole process.

Using VMware Identity Manager to transform users between Active Directory domains - https://blogs.vmware.com/horizontech/2017/05/using-vmware-identity-manager-transform-users-active-directory-domains.html
Setting up a 3rd Party IdP in VMware Identity Manager (Concepts and Steps) - https://communities.vmware.com/docs/DOC-34295

Resetting On-Premises Workspace ONE Access / VMware Identity Manager ROOT and SSHUSER Password Expirations and History

2020-04-24T16:50:00.001-07:00

The Why:

Like any good IT person who must both maintain existing production environments while working to expand and/or enhance said environment's capabilities, having both a Dev/Test environment and a User Acceptance Test (UAT) environment for allowing your fellow employees to test some determined settings for User Experience (UX) (in conjunction with an actual Production environment) is certainly prudent. And having the Dev/Test environment to not only stand-up and play with, but break in a safe manner, is a very good thing. Having such playground often mean you not only get to - but often need to - dig into the underbelly of said environments to learn a few things.

To that end, there are a number of situations where you need to either reset the root or sshuser password expirations or reset the password history of an on-premises tenant of VMware Identity Manager - or what is now known as Workspace ONE Access - in order to accomplish a specific test.

Here is how to do both...

The Procedure (Resetting PWD Expiration and History):

If you can SSH in to the appliance(s), then doing so will certainly help with this procedure. However, if you need to reset the root & admin passwords, see this document as this applies to all Horizon App Manager, Horizon Workspace, Workspace Portal, VMware Identity Manager, and Workspace ONE Access SLES appliances.

NOTES:

All commands must be done from root session or console access (ssh and then su to root or login as root to console).
Below shows editing of files using VI ("vi") from the console.

Modifying the Password History Settings

Edit the /etc/pam.d/common-password file and change the REQUIRED parameter to OPTIONAL and remove "enforce_for_root remember=x" (where "x" is a number).

Clear Existing Password History

Edit the /etc/security/opasswd file and delete all of the contents. Save and exit.

Reset Password Expirations for root and sshuser Accounts

Run "chage" for both root and sshuser accounts to reset password expiration.

The Conclusion:

You should now be able to access your test lab on-premises VMware Identity Manager / Workspace ONE Access appliance(s) with those accounts without worrying about a forced password change.

Using PostMan to Update Workspace ONE / VMware Identity Manager Allowed Redirects

2020-01-14T14:07:00.003-08:00

On some occasions, customers may need to have a resource send a command to VMware Identity Manager but then redirect back. By default, VMware Identity Manager does not do redirects (this is by design), however, it can support them with a slight modification to trusts. The changes are a one-time change per VMware Identity Manager tenant and remain from that point forward unless removed by an administrator.

The following are the instructions for grabbing the bearer token from the HZN cookie and then applying using Postman to access the APIs to update the settings for allowing redirects.

Getting Bearer Token from HZN cookie in browser

NOTE: Most modern browsers have an inspection mode. Usually this is accessed by right-clicking on the web page and selecting INSPECT.

Open a browser such as Chrome and authenticate as admin account to your Workspace ONE tenant (Note: Make sure you have the option to view the Administrator console).
Open INSPECT mode in the browser by right-clicking on the page and selecting INSPECT from the context menu.
Select APPLICATION from the inspection window.
Select your Workspace ONE / VMware Identity Manager portal under cookies.
Find HZN under the name column and copy the value data to your clipboard.
In step 5.) of the below instructions, paste the token in for authentication as a Bearer Token.

Using PostMan to access Workspace ONE APIs to GET values

When you define the parameters in PostMan, make sure the following are set.

1. Start with GET command to test URL.
URL to the API with correct FQDN of VMware Identity Manager tenant. In this case, to read the “allowedredirects” value from the Workspace ONE tenant, use the below URL.
https:///SAAS/jersey/manager/api/authsettings/allowedredirects
Open Authorization tab…
Set authentication type to BEARER TOKEN.
Paste in token from HZN cookie from browser (see above procedure to get token from HZN cookie).
When you press SEND, the Body value should appear as shown in the image below.

Using PostMan to access Workspace ONE APIs to POST values

When you define the parameters in PostMan, make sure the following are set.

Start with POST command to test URL.
URL to the API with correct FQDN of VMware Identity Manager tenant. In this case, to update the “allowedredirects” value from the Workspace ONE tenant, use the below URL.
https:///SAAS/jersey/manager/api/authsettings/allowedredirects
To note: allowed redirects are per tenant (not per node within a tenant) so they only need to be set using the LB FQDN (e.g. portal.flaming.ws or workspace.mydomain.com).
Open Authorization tab…
Set authentication type to BEARER TOKEN.
Paste in token from HZN cookie from browser (see above procedure to get token from HZN cookie).
Switch to the HEADERS tab in Postman.
For updating the allowedredirects value we need to add in the “Accept” header with the following value:
application/vnd.vmware.horizon.manager.authsettings.allowedredirects+json
For updating the allowedredirects value we need to add in the “Content-Type” header with the following value:
application/vnd.vmware.horizon.manager.authsettings.allowedredirects+json

NOTE: If you do not see the above image, you are likely in Bulk Edit mode. If you’d like, you can just paste in the text per the next image, or shift to the Key-Value Edit mode in postman to see the above screen.
Within Postman, switch to the Body tab.
Select “raw” for the entry type. This allows you to type in whatever code you wish.
Select the code type. This is optional as TEXT or TEXT PLAIN will also work. The code we are entering is JSON so either TEXT/TEXT PLAIN or JSON will work.
Enter the following code. PUT YOUR WEB APP LOGOUT/LOGIN PAGE AND ANY OTHER PAGES IN COMMA SEPARATED FORMAT BETWEEN THE DOUBLE QUOTES. ENSURE YOU ADD THE ASTERISK AT THE END OF EACH TO BE ABLE TO POINT TO ANY SUBPAGE.

SINGLE SITE CODE EXAMPLE:

{
"allowedRedirects": [“https://www.flaming.ws*"]
}

MULTI-SITE CODE EXAMPLE:

{
"allowedRedirects": ["https://www.google.com*,https://www.flaming.ws*"]
}

When you press SEND, the pages you define will show up under “allowedRedirects” within the Body outcome.

If you make an error just correct it and repost the update. If you wish to remove all redirects, simply delete all web sites between the quotes.

At this point, redirections are now set and you should be able to make an Oauth or SAML call to force logout with a redirection back to your application.

REMOVE ALL REDIRECTS CODE EXAMPLE:

{

"allowedRedirects": [“"]

}

Cleaning up ElasticSearch Unassigned Shards in On-Premises VMware Identity Manager (Workspace ONE Access)

2019-12-23T14:10:00.001-08:00

Background Info:

On occasion, my on-premises VMware Identity Manager cluster of appliances will get an elasticsearch error due to unassigned shards. If you ever see an error in your VMware Identity Manager (vIDM) or Workspace ONE Access appliance dashboard regarding the analytics service, it usually means this issue exists for you as well.

The Solution:

There are a couple of options for resolving this. One is via slightly unhelpful documentation on the VMware website here. As for some more helpful command line options, I've posted some below. Basically with option 1, you manually go through each node looking for and removing each unassigned shard using the curl XGET and XDELETE options. With option 2, you go through the command line to remove the shards via standard OS file commands.

Option 1:

1. SSH to the appliance or one of your appliances in the cluster. Login as root.
2. Run the following command to determine if you have unassigned shards
curl http://localhost:9200/_cluster/health?pretty

NOTE: The output should look something like the below:

{

"cluster_name" : "horizon",

"status" : "green",

"timed_out" : false,

"number_of_nodes" : 3,

"number_of_data_nodes" : 3,

"active_primary_shards" : 0,

"active_shards" : 0,

"relocating_shards" : 0,

"initializing_shards" : 0,

"unassigned_shards" : 0,

"delayed_unassigned_shards" : 0,

"number_of_pending_tasks" : 0,

"number_of_in_flight_fetch" : 0,

"task_max_waiting_in_queue_millis" : 0,

"active_shards_percent_as_number" : 100.0

}

3. Run the following command to view the unassigned shards.

curl -XGET ‘http://localhost:9200/_cat/shards?h=index,shard,prirep,state,unassigned.reason’ | grep UNASSIGNED

4. Delete the unassigned shards by running the following command
curl -XDELETE ‘http://localhost:9200/v3_YYYY-MM-DD/

Once all the unassigned shards have been deleted refresh the dashboard and Analytics connection should be successful.

And running the curl http://localhost:9200/_cluster/health?pretty should report 0 unassigned shards and status: green.

Option 2:

The other option for cleaning shards is on the console through the file system. This will have to be done manually on all nodes:

Find out what indices have hung shards.

curl -XGET 'http://localhost:9200/_cat/shards?h=index,shard,prirep,state,unassigned.reason'|grep UNASSIGNED
DO THE ABOVE COMMAND ON ALL NODES BEFORE PROCEEDING TO STEP 2!

Make sure elasticsearch is not running on all of the nodes:

STOP ELASTICSEARCH ON ALL NODES FIRST
service elasticsearch stop

Remove the index's files from the disk on each node. Replace and with the proper values (i.e Node 0 or 1 and the Index Name from the output of the curl command in step 1):

rm -rf /db/elasticsearch/horizon/nodes//indices/

Restart elasticsearch on each node ONLY after doing the above steps (2 and 3) on each node.

service elasticsearch start

Option 2-ALTERNATE:

WARNING: DO NOT DO THIS on a production environment!!

The lazy way to do this is dump the node’s ‘indices’ folder. The downside to doing this is you lose all other historical data. If you’re fine with that, then certainly this is an option. The command is simply “rm -rf /db/elasticsearch/horizon/nodes//indices/“.

NOTE: If you dump the Indices folder, the default settings are shown below.

drwxr-xr-x 30 elasticsearch users 4096 Sep 12 10:35 indices

To recreate the folder, stop elasticsearch, create the folder, set the user and group ownership, and then set the permissions, then restart elasticsearch.

service elasticsearch stop
md indices
chown elasticsearch:users indices/
chmod 755 indices/
Do an ls -l -a rom the numbered folder (typically “0”) to see if permissions are correctly set.
service elasticsearch start

The commands for the above are shown here.

md /db/elasticsearch/horizon/nodes/0/indices

chown elasticsearch:users /db/elasticsearch/horizon/nodes/0/indices/

chmod 755 /db/elasticsearch/horizon/nodes/0/indices/

Once this is completed, you may wish to recheck the folder is created with the correct permissions. Simply run the “ls -l -a” command on the node folder.

ls -l -a /db/elasticsearch/horizon/nodes/0/

From that point, restart the elasticsearch service on each node one at a time.

Combined Commands for copy/paste into SSH.

NODE 0:

NOTE: STOP ELASTICSEARCH ON ALL NODES FIRST

service elasticsearch stop

rm -rf /db/elasticsearch/horizon/nodes/0/indices/

md /db/elasticsearch/horizon/nodes/0/indices

chown elasticsearch:users /db/elasticsearch/horizon/nodes/0/indices/

chmod 755 /db/elasticsearch/horizon/nodes/0/indices/

ls -l -a /db/elasticsearch/horizon/nodes/0/

service elasticsearch start

NODE 1:

NOTE: STOP ELASTICSEARCH ON ALL NODES FIRST

service elasticsearch stop

rm -rf /db/elasticsearch/horizon/nodes/1/indices/

md /db/elasticsearch/horizon/nodes/1/indices

chown elasticsearch:users /db/elasticsearch/horizon/nodes/1/indices/

chmod 755 /db/elasticsearch/horizon/nodes/1/indices/

ls -l -a /db/elasticsearch/horizon/nodes/1/

service elasticsearch start

BOTH NODE’S 1 AND 2:

NOTE: STOP ELASTICSEARCH ON ALL NODES FIRST

service elasticsearch stop

rm -rf /db/elasticsearch/horizon/nodes/0/indices/

md /db/elasticsearch/horizon/nodes/0/indices

chown elasticsearch:users /db/elasticsearch/horizon/nodes/0/indices/

chmod 755 /db/elasticsearch/horizon/nodes/0/indices/

rm -rf /db/elasticsearch/horizon/nodes/1/indices/

md /db/elasticsearch/horizon/nodes/1/indices

chown elasticsearch:users /db/elasticsearch/horizon/nodes/1/indices/

chmod 755 /db/elasticsearch/horizon/nodes/1/indices/

ls -l -a /db/elasticsearch/horizon/nodes/0/

service elasticsearch start

Pushing iTunes Track Info to NiceCast with AppleScript

2014-04-18T12:52:00.001-07:00

Background Info:

On occasion, I like to do podcasts and, thus, have built a system which has significant audio capabilities. Specifically, my Mac has multiple mixers and amps connected such as (from the small end) a StarTech USB Surround Sound input/output device, M-Audio FastTrack USB mixer, a Tascam US-144 MKII USB mixer, a Xenyx 502, and a Focusrite Saffire Pro 40. I also have things like an MXL 990 and an Epiphan VGA2USB converter as well.

The Problem:

I have various applications such as AirFoil and NiceCast from Rogue Amoeba Software which are great at what they do, but I would rather have my audio come out of an app like iTunes and into the mixers and amps for processing, then back into my Mac for transmission. NiceCast does a great job of directly capturing song info from iTunes and transmitting down the line with the music, but, like I said, I want to mix and blend the audio beforehand. NiceCast doesn't directly accommodate this but it does have a manual input option via a NowPlaying.txt file which allows pulling in track title, artist, album, and time. The trick is, how do I get that info from the source app - in this case, iTunes - to Nicecast via this NowPlaying.txt file in the format necessary?

The Solution:

The resolution, use AppleScript with iTunes to extract this data.

Below is my AppleScript which accommodates this need. To make this work, I simply compiled into an OS X Application and saved it to the "~/Library/iTunes/Scripts" folder. Then, when playing my music, I simple use the iTunes Script Menu which appears when a script is available to send the track info to the NowPlaying.txt file for Nicecast to read when broadcasting.

AdminScriptEditor Script Conversion

global lasttrack

On run
   -- iTunes Setup for launch of the code
   tell application "iTunes"
      Set line1 To "Title: " & (Get name of current track)
      Set line2 To "Artist: " & (Get artist of current track)
      Set line3 To "Album: " & (Get album of current track)
      Set line4 To "Time: " & (Get Time of current track)
      Set lasttrack To name of current track
   End tell
   Set Track_Info To line1 & "
" & line2 & "
" & line3 & "
" & line4
   Set this_file To "Macintosh HD:Users:username:Library:Application Support:Nicecast:NowPlaying.txt"
   my write_to_file(Track_Info, this_file, False)
End run

--Now comes the idle checks
On idle
   tell application "iTunes"
      Set currentTrack To name of current track
   End tell
   if currentTrack Is Not lasttrack Then
      tell application "iTunes"
         Set line1 To "Title: " & (Get name of current track)
         Set line2 To "Artist: " & (Get artist of current track)
         Set line3 To "Album: " & (Get album of current track)
         Set line4 To "Time: " & (Get Time of current track)
         Set lasttrack To line1
      End tell
      Set Track_Info To line1 & "
" & line2 & "
" & line3 & "
" & line4
      Set this_file To "Macintosh HD:Users:username:Library:Application Support:Nicecast:NowPlaying.txt"
      my write_to_file(Track_Info, this_file, False)
   end if
   return 10
End idle


On write_to_file(this_data, target_file, append_data)
   try
      Set the target_file To the target_file As String
      Set the open_target_file To open for access file target_file with write permission
      if append_data Is False Then Set eof of the open_target_file To 0
      write this_data To the open_target_file starting at eof
      close access the open_target_file
      return True
   On Error
      try
         close access file target_file
      End try
      return False
   End try
End write_to_file

Disabling the Built in Keyboard and Trackpad on a MacBook Pro

2013-12-19T12:05:00.001-08:00

If, like me, you use your MacBook Pro in a multi-screen scenario with an external keyboard and mouse, then probably like me, you've wondered how to disable the built in trackpad and keyboard so inadvertent key presses don't mess things up (i.e. the cat).

Here's how.

Notes:

A couple of things to note when doing this.

Disabling the built-in keyboard must be done manually on each reboot. This isn't a huge deal since we're not rebooting our Mac's very much (hopefully).

WARNING: Before disabling the built-in keyboard, make darn sure you have an external keyboard connected!!

WARNING: Before disabling the built-in trackpad, make sure you have an external mouse connected!

NOTICE: In following these procedures, you take full responsibility for you messing up your system if something doesn't go right!

How to disable the keyboard:

Source: http://forums.macrumors.com/showpost.php?p=9536110&postcount=4

Open Terminal and type in the following command.

sudo kextunload /System/Library/Extensions/AppleUSBTopCase.kext/Contents/PlugIns/AppleUSBTCKeyboard.kext/

How to enable the keyboard:

Source: http://forums.macrumors.com/showpost.php?p=9536110&postcount=4

Open Terminal and type in the following command.

sudo kextload /System/Library/Extensions/AppleUSBTopCase.kext/Contents/PlugIns/AppleUSBTCKeyboard.kext/

How to disable the built in Trackpad:

Simply open OS X Settings, browse to Accessibility, select Mouse & Trackpad, and check the "Ignore built-in trackpad when mouse or wireless trackpad is present" option.

How to enable the built in Trackpad:

Simply open OS X Settings, browse to Accessibility, select Mouse & Trackpad, and uncheck the "Ignore built-in trackpad when mouse or wireless trackpad is present" option.

Conclusion:

Now my cat can sit on my computer without causing issue.

5 Steps to Capturing an App or Plugin for use with IE

2013-03-23T09:48:00.001-07:00

Thought I'd share this as I still get this question quite often.

Question:
How do I capture just an Internet Explorer plugin?
How do I capture an application to use the native Internet Explorer through the ThinApp package?

Step 1. ThinApp Pre-Installation Setup Capture on Win XP SP3.
Step 2. Install the application or IE6 Plugin.
Step 3. Click the INTERNET EXPLORER button in ThinApp.
Step 4. Select "Include an Entry Point for the system-installed Internet Explorer".
Step 5. Click OK and POSTSCAN to continue capturing and building your application with ThinApp.

Conclusion:
This will enable the IEXPLORE.EXE entry point in the ThinApp package for launching the natively installed Internet Explorer through the ThinApp package with the captured application or plugin.

- Posted using BlogPress from my iPad

How to Fix Skype Audio Selection on Mountain Lion

2012-08-11T11:50:00.001-07:00

My 2 cents…

Apparently Skype's (now Microsoft) claim of Skype 5.8.0.1027 being Mountain Lion compatible is only partially true at best.

What's the issue?

Whenever starting Skype 5.8.0.1027 on Mountain Lion, one gets varying results of audio selection issues. Usually the issue is the phrase "No Value" shows for some or all audio device selections in the Skype Audio/Video config selections.

How do I resolve this?

The only workaround I have found at this time is to go into "Activity Monitor" and kill the CoreAudioD service (Note: Use the search box in the upper-right).

Once done, go back into Skype Audio/Video settings and you should see your correct audio devices for selection. On occasion I have to retry this a couple of times in order for it to work.

The other option is from within Terminal, execute the below command:

sudo killall coreaudiod

Thanks:

Thanks to the folks over on the Apple Communities for posting this info.

Astaro Security Gateways, High Availability, and vSphere Virtual Switches

2012-05-21T17:05:00.001-07:00

Preamble:

So, if you've not figured it out by now, I'm a computer geek. :-) And since I primarily work from my home office, one of the things I love to do is run things locally in my own labs. But to be cost efficient - both in hardware and in space and power - I run everything on two VMware vSphere 5 hosts.

I probably could get away with using more of the technology within my hosts (you'll see why I say this in a bit) but I like to tweak things and make changes to match a customer network so as to see as real-to-life results as possible.

Environment:

To give you some background, as mentioned I have two vSphere 5 hosts. I also run different labs on each and they are not identical hardware (not a big deal with regards to this issue). And, I also run to virtual Astaro Security Gateway firewalls (one virtual appliance on each host) in active-passive mode. This is so I can reboot one host at a time and not lose my Internet connection (I use VoIP and remote connectivity for a ton of things).

I do not use virtual distributed switches. Rather, I use simple virtual switches with VLANs (I know, I probably should use distributed switches but to me this seems overkill for what I have - and don't forget, my hosts are not on identical hardware so I have a different number of NICs in each). And, I don't think having distributed virtual switches would have alleviated the problem - in fact it probably would have compounded the issue.

So...My issue which resulted in many, many hours of troubleshooting was this.

Issue:

When setting two (virtual) Astaro Security Gateways to High Availability Mode (we'll call these Firewall 1 and Firewall 2), and the Slave unit was up and running in passive mode, with one Astaro VM on each host (we'll call these vSphere Host 1 and 2), and both were connected via VLANs, Guest VMs on vSphere Host 1 could not reach the Internet when Firewall 2 on vSphere Host 2 was set to Master and vice versa (i.e. Guest VMs on vSphere Host 2 could not reach the Internet when Firewall 1 on vSphere Host 1 was set to Master and Firewall 2 on vSphere Host 2 was set to Slave).

Symptoms:

I actually tested a number of different things, but the short version is, whenever I disconnected a VM network NIC (DMZ or Local Network) from the Slave firewall, all Guest VMs on the same host could suddenly reach the Internet through the Master firewall on the other host.

Additionally, none of the Guest VMs on host with the Slave firewall could ping the Master firewall - but they could ping all other VMs and addresses on the subnet (physical or virtual).

At first I thought it was an issue with the original VM I saw this symptom on, but it turns out the VM was on the host which was running the Slave firewall. My next thought was it was a firewall issue (I was half-right here) but I didn't have all the pieces to draw the correct conclusion so I abandoned this early on. Then I went on to look at the virtual switches as there was a lot of information around this type of issue and virtual switches needing to be in promiscuous mode - which turns out was NOT the case here.

Resolution:

What actually turned out to be the issue was the virtual switches do not like it when two devices show up with the same Mac address - something which the Astaro Security Gateways do when put into High Availability Mode (everything is synchronized, right down to the MAC addresses on all network connections!).

So how do I make this work? Well, I had two options. The first being configure the virtual switches so they could handle the multiple devices with the same MAC addresses. The second option being, disable the Astaro Security Gateways from synchronizing the MAC addresses when in High Availability Mode.

I chose the second option - disabling the MAC synching as this was much easier than tweaking the virtual switches via command line to get them to work.

Modifying the Astaro to Not Sync the MAC in HA Mode:

Boot both appliances and let them sync.
Open the consoles on both and login.
From the console on each, run the below command:

cc set ha advanced virtual_mac 0

Reboot the firewall which is currently in Slave mode (You'll see this denoted by the "S" in console window prior to the command line (the Master will have an "M").

NOTE: While the Slave unit is down, you may be able to ping the master from any of the VMs on the vSphere host where the Slave is running.

When the Slave firewall comes back online, you should be able to ping the Master from any of the VMs on the vSphere host where the Slave is running.

Warnings:

The downside is, anything locked to the MAC of the master Astaro will need to reinitiate the connection on the slave Astaro on failover - as the Slave will become Master - and the "New Master" MAC addresses will not be identical to the "Old Master" MAC addresses. In short, there will be a hiccup in network communications while the switch is occurring….and in most cases this isn't a huge deal.

Chrome Print Preview - How to Disable

2012-04-23T11:54:00.001-07:00

Updated!

It looks like Google Chrome 20 removed the below flag for disabling Print Preview, so a work around for both operating systems is as follows:

MAC: Open System Preferences, browse to the Keyboard applet, the select the Keyboard Shortcuts tab/button. Select "Application Shortcuts" in the left side and in the right, add in the following Keyboard Shortcuts for printing.

WIN: Open the shortcut placed on the desktop and edit it to add --disable-print-preview after the end-quote for Chrome.EXE.

I used to be a Google Chrome fan but lately some of what Google has done with Chrome has not made me the happiest of campers. Chrome's Print Preview being forced on is one of those things. And, unfortunate for those who're on the latest versions of Chrome, we cannot do the below. Hopefully you still can…

When I go to print, this only serves to slow me down or hinder my workflow. Why? It has no programmable key sequences and takes over a few of the system keys (such as COMMAND + P on a Mac).

NOTE: Click on images for larger versions.

Here's how to turn older versions off both in Windows and on a Mac.

In Chrome, type about:flags in the address bar and hit ENTER.

Find the Disable Print Preview section and click ENABLE.

Click the RELAUNCH NOW button at the bottom of the screen.

When Chrome relaunches, you'll see when you go to print, the normal system print dialog appears.

Happy Printing!

YouTube Phishing Emails

2012-04-07T15:15:00.001-07:00

I've noted a couple of these "Phishing" emails coming into my junk folder lately and thought I would point out a few reasons why these are phishing (that's "fishing" spelled the hacker way with a "ph") emails.

What is a "Phishing" email?

Well, Wikipedia has some very extensive and accurate explanations of what a "Phishing" scam really is. In short, it is an attempt to trick you into sending your personal information to an unknown source.

Uncool…

So how do we know this is a scam email? It looks like it came from a valid email address. Well, never forget email addresses, just like caller IDs on a phone, can be faked.

So let's dissect this email…

My first clue was this ended up in my "Junk" folder. But this doesn't mean it is junk email per se. It just means my email program thought it was junk from the message (turns out it was right). To note though, I do get a lot of valid emails in my junk folder only because they are mainly emails I don't care to read most of the time so I've taught my email program those emails are junk (usually hitting the "Junk" or "Spam" button on an email is all you need to do now a-days).

My second clue? The format looked off to me. How so? Well, look at the below image when we expand out the window a bit. What looks off to you?

NOTE: For any picture, click on the picture to load the image in a separate page.

To me, the paragraph formatting looks off. And if I resize the window to smaller than what the text is, it really looks off (see below).

But does an improperly formatted email mean it is a fake? No, not on by itself. It should be noted the vast majority of scam emails come out of Russia, China, and Africa. And those are all countries where English is not usually a first language - so formatting and grammar mistakes are often a sign of a scam. But over the years, scammers have been getting better and better at disguising their scam as legitimate. Just look at Bernie Madoff.

But back to this email. What was my biggest clue?? Well, when I mouse over any of the links in the email, you can see from the above screenshot the displayed link (i.e. the "YouTube" link) and the actual link (i.e the "ag-socrat.spb.ru" link) are not the same. This should be your biggest tip - NEVER click a link in an email unless you know where it will take you!! It is similar to receiving a call from your (supposed) credit card company or bank asking you to verify your personal data. You don't answer the question directly, you call them back on their publicly listed number and have them ask to verify you so you can then address the issue. Same goes for any email with a link like this one.

And, if you look a bit closer, you'll see the TLD (Top Level Domain) is "RU" - meaning this is a link to some Internet server in Russia.

And while I strongly recommend against doing so on your own system, when I click on any of these links, Google Chrome (my default browser) warns me this is a scam.

So what can I do to protect myself?

Great question! If you have Windows, a good Antivirus and Malware system from someone such as Trend Micro or McAfee are highly recommended. They also have stuff for a Mac as well if you'd like to have the extra protection. NOTE: Remember, a Mac OS operates very differently from a Windows system. In order for a Mac to become infected with malware, you must absolutely provide credentials to the malware in order for it to gain access to your system.

But this wasn't malware, it was a "Phishing" scam. How else can I be protected?

Even better question! Use of an "intelligent" firewall such as the Sophos Astaro Security Gateway (Free for home users) is a great solution for those who are more technically inclined. They even have a Virtual Appliance if you have your own VMware vSphere (ESXi) servers (also free)!

Below I show a screenshot from the Sophos Astaro Security Gateway Firewall settings which allows country blocking.

Aside from Country Blocking, the Astaro has built in support for all sorts of email filtering, security, and protection - including Antivirus, Antispam, and other settings to help block "Phishing" emails such as the one above.

What if I am not a Computer Geek or do not have the resources to setup such a firewall?

Rest assured, the previously mentioned sites (Trend Micro, McAfee, etc.) and many, many others all have great solutions. Even Google's GMail has many Anti-Spam and Anti-Phishing options built into it. The bottom line is you must educate yourself in protecting yourself.

And above all, do not trust any email unless you know it comes from a trusted source. Your diligence in this is your own greatest protection.

How to enable VNC access to VMs on vSphere 5 (ESXi)

2012-03-17T17:51:00.001-07:00

I'm one of those people who use a simple vSphere (ESX for you "Old School" types) setup. I just need to run a whole bunch of VMs but don't need to sit in the vSphere Infrastructure Client (VIC) to do so. Partly because I don't run Windows as a primary system (I use a Mac) and there isn't a VIC for the Mac nor is there a Web Console for the Mac (yet…at the time of this posting at least) and partly because I just want to view the console as a boot occurs on my VMs (Windows and non-Windows alike).

So, how do you make this happen? Here's the procedure.

Login to the vSphere host either at the console or remotely.
NOTE: On the console, this requires logging into the ALT+F2 screen as root or other admin with permissions, going to TROUBLESHOOTING OPTIONS, and enabling either ESXi Shell (console shell access) or SSH (remote shell access).

Change owner on firewall folder and services.xml file
\etc\vmware\firewall\services.xml

CHMOD help
CHOWN help

I did the following (as this was my own lab in a protected environment):

Edit the SERVICE.XML file to create a VNC service.
NOTE 1: Use VI or another text editor to add the below text at bottom of the file (assuming it is still in a default format).
NOTE 2: If you have modified your firewall then your service id may be different! This should be a sequential number which you can get from the previous listed service. By default it should be approximately "0038".

AdminScriptEditor Script Conversion

  <!-- VNC -->
  <service id="0038">
    <id>VNC</id>
    <rule id='0000'>
        <direction>outbound</direction>
        <protocol>tcp</protocol>
        <porttype>dst</porttype>
        <port>
           <begin>5800</begin>
           <end>5999</end>
        </port>
     </rule>
     <rule id='0001'>
        <direction>inbound</direction>
        <protocol>tcp</protocol>
        <porttype>dst</porttype>
        <port>
           <begin>5800</begin>
           <end>5999</end>
        </port>
     </rule>
     <enabled>true</enabled>
     <required>false</required>
  </service>

NOTE 3: Use "i" (lower case) to insert text in VI.

NOTE 4: Use [ESC] to exit insert mode within VI.

NOTE 5: Use ":x" (i.e. colon then lower case "x") to save and write changes.

NOTE 6: You will notice in the above settings I am opening all ports between 5800 and 5999. You may not wish to do this!!

Save Services.xml file
NOTE: Any errors at this point will be file/folder permissions related.

Reset file permissions on SERVICE.XML.

#chmod 444 /etc/vmware/firewall/service.xml

Refresh firewall rules

Reference: VirtuallyGhetto has a great ESX Firewall reference.

Commands to refresh the ESX firewall:

esxcli network firewall refresh

esxcli network firewall ruleset list

Power off and edit VM per VMware KB 1246.
Using a VNC Client to connect to virtual machines

NOTE: Use specific unique ports per VM to allow connection.

General Info:

Now that you've finished modifying your vSphere server, just go grab your favorite VNC client and connect. Since I use a Mac, I like Chicken of the VNC (how can you NOT like that name?) and you cannot beat the price (free).

It should be noted, this will not be a good solution for BC/DR situations. See "Cloud-Buddy"'s article "VNC and vSphere 5 for Mac users" which discusses this solution and the pros and cons.

Additional information can be found on VMware's Knowledge Base. Specifically see the article, "Creating Custom Firewall Rules in VMware ESXi 5.x".