<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://t3hcr.github.io/feed.xml" rel="self" type="application/atom+xml" /><link href="https://t3hcr.github.io/" rel="alternate" type="text/html" /><updated>2026-03-19T13:07:37+00:00</updated><id>https://t3hcr.github.io/feed.xml</id><title type="html">t3hcr</title><subtitle>Stuff I want to share with you</subtitle><author><name>t3hcr</name></author><entry><title type="html">Veeam 13 Upgrade Troubleshooting</title><link href="https://t3hcr.github.io/Veeam-13-upgrade-troubleshooting/" rel="alternate" type="text/html" title="Veeam 13 Upgrade Troubleshooting" /><published>2025-12-30T00:00:00+00:00</published><updated>2025-12-30T00:00:00+00:00</updated><id>https://t3hcr.github.io/Veeam-13-upgrade-troubleshooting</id><content type="html" xml:base="https://t3hcr.github.io/Veeam-13-upgrade-troubleshooting/"><![CDATA[<p><em>Author’s note: This post is being made to help others and is the result of my interactions with AI and should be considered as such. ALWAYS backup and test before implementing any fixes. If you’re unsure or uncomfortable making changes (or fixing them) contact Veeam support!</em></p>

<h2 id="overview">Overview</h2>

<p>Upgrading <strong>Veeam Backup &amp; Replication</strong> from <strong>v12 to v13</strong> is usually painless — until the installer hard-stops with this message:</p>

<blockquote>
  <p><strong>[Error] Outdated Veeam Agents</strong><br />
Please upgrade or remove the following Veeam agents from the product configuration:<br />
<strong>OLD-AGENT-HOST (Veeam Agent for Windows 3.0.2.1170)</strong></p>
</blockquote>

<p>The problem?<br />
<code class="language-plaintext highlighter-rouge">OLD-AGENT-HOST</code> <strong>no longer existed anywhere</strong> in the environment.</p>

<ul>
  <li>No backups</li>
  <li>No protection groups</li>
  <li>No inventory objects</li>
  <li>Nothing visible in the UI</li>
</ul>

<p>And yet the v13 installer refused to proceed.</p>

<p>This post walks through <strong>why this happens</strong>, <strong>why the usual cleanup methods fail</strong>, and <strong>how to fix it correctly</strong> when your Veeam configuration database is running on <strong>PostgreSQL</strong>.</p>

<hr />

<h2 id="whats-actually-going-on">What’s Actually Going On</h2>

<p>Veeam v13 introduced <strong>stricter prerequisite checks</strong> than v12. One of those checks validates <strong>agent inventory state</strong>, not just backup jobs or objects.</p>

<p>That inventory data lives in Veeam’s <strong>Enterprise Plug-in (EP)</strong> tables and can persist even after:</p>

<ul>
  <li>a machine is decommissioned</li>
  <li>the agent is uninstalled</li>
  <li>the backup object is deleted</li>
</ul>

<p>Veeam v12 tolerates this inconsistency.<br />
Veeam v13 does <strong>not</strong>.</p>

<p>There is no “ignore” or “continue anyway” option. The stale record must be removed.</p>

<hr />

<h2 id="why-the-ui-and-powershell-show-nothing">Why the UI and PowerShell Show Nothing</h2>

<p>Every obvious place was checked:</p>

<ul>
  <li><strong>Home → Backups → Agent Backups</strong></li>
  <li><strong>Inventory → Physical Infrastructure</strong></li>
  <li><strong>Protection Groups</strong></li>
</ul>

<p>Nothing referenced <code class="language-plaintext highlighter-rouge">OLD-AGENT-HOST</code>.</p>

<p>PowerShell wasn’t helpful either — even when the Veeam cmdlets are available, these EP inventory records often don’t surface in a way that allows removal.</p>

<p>At this point, the <strong>configuration database is the only source of truth</strong>.</p>

<hr />

<h2 id="safety-first">Safety First</h2>

<p>Before touching the database:</p>

<ul>
  <li>Take a <strong>Veeam Configuration Backup</strong></li>
  <li>Preferably stop write-heavy services (at minimum <strong>Veeam Backup Service</strong>)</li>
  <li>Use transactions so you can safely roll back if needed</li>
</ul>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">BEGIN</span><span class="p">;</span>
<span class="c1">-- changes</span>
<span class="k">COMMIT</span><span class="p">;</span>
<span class="c1">-- or ROLLBACK;</span>
</code></pre></div></div>

<hr />

<h2 id="step-1--identify-the-blocking-agent-version">Step 1 — Identify the Blocking Agent Version</h2>

<p>The installer helpfully tells you the agent version it’s unhappy with.</p>

<p>Query the EP agent inventory:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span>
    <span class="n">host_id</span><span class="p">,</span>
    <span class="n">bobject_id</span><span class="p">,</span>
    <span class="n">agent_version</span><span class="p">,</span>
    <span class="n">is_upgrade_required</span><span class="p">,</span>
    <span class="n">agent_installation_status</span>
<span class="k">FROM</span> <span class="nv">"backup.model.epagents"</span>
<span class="k">WHERE</span> <span class="n">agent_version</span> <span class="o">=</span> <span class="s1">'3.0.2.1170'</span>
   <span class="k">OR</span> <span class="n">agent_version</span> <span class="k">ILIKE</span> <span class="s1">'3.0.2.%'</span><span class="p">;</span>
</code></pre></div></div>

<p>Example Result:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>host_id	agent_version
&lt;EP_HOST_UUID&gt;	3.0.2.1170
</code></pre></div></div>

<p>This confirms the installer is blocking on a legacy Agent 3.x record.</p>

<hr />

<h2 id="step-2--map-the-agent-record-to-a-host">Step 2 — Map the Agent Record to a Host</h2>

<p>The epagents table does not store hostnames directly.
Host details are stored in:</p>

<p>backup.model.ephosts</p>

<p>Inspect the table (optional but recommended):</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">\</span><span class="n">d</span> <span class="nv">"backup.model.ephosts"</span>
</code></pre></div></div>

<p>Now retrieve the EP host record:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span>
    <span class="n">id</span><span class="p">,</span>
    <span class="n">connection_point</span><span class="p">,</span>
    <span class="n">display_name</span><span class="p">,</span>
    <span class="n">os_version</span><span class="p">,</span>
    <span class="n">host_state</span><span class="p">,</span>
    <span class="n">host_aux_data</span>
<span class="k">FROM</span> <span class="nv">"backup.model.ephosts"</span>
<span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="s1">'&lt;EP_HOST_UUID&gt;'</span><span class="p">::</span><span class="n">uuid</span><span class="p">;</span>
</code></pre></div></div>

<h3 id="what-youll-see">What You’ll See</h3>

<p>If this is the ghost entry, fields like these will jump out immediately:</p>
<ul>
  <li>connection_point: OLD-AGENT-HOST</li>
  <li>display_name: OLD-AGENT-HOST</li>
</ul>

<p>At this point, you’ve proven exactly what’s blocking the upgrade.</p>

<hr />

<h2 id="why-deleting-the-ep-host-is-the-correct-fix">Why Deleting the EP Host Is the Correct Fix</h2>

<p>This is the critical relationship:</p>

<pre><code class="language-pgsql">backup.model.epagents.host_id
  → references backup.model.ephosts(id)
  → ON DELETE CASCADE
</code></pre>

<p>Deleting the EP host automatically removes:</p>
<ul>
  <li>the EP agent inventory record</li>
  <li>EP agent membership rows</li>
  <li>EP agent backup statistics</li>
  <li>related EP metadata</li>
</ul>

<p>This is the clean, supported way to resolve the condition.</p>

<hr />

<h2 id="step-3--delete-the-orphaned-ep-host">Step 3 — Delete the Orphaned EP Host</h2>

<p>Delete the EP host inside a transaction:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">BEGIN</span><span class="p">;</span>

<span class="k">DELETE</span> <span class="k">FROM</span> <span class="nv">"backup.model.ephosts"</span>
<span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="s1">'&lt;EP_HOST_UUID&gt;'</span><span class="p">::</span><span class="n">uuid</span><span class="p">;</span>

<span class="k">COMMIT</span><span class="p">;</span>
</code></pre></div></div>

<p>Verify Cleanup</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="o">*</span>
<span class="k">FROM</span> <span class="nv">"backup.model.epagents"</span>
<span class="k">WHERE</span> <span class="n">host_id</span> <span class="o">=</span> <span class="s1">'&lt;EP_HOST_UUID&gt;'</span><span class="p">::</span><span class="n">uuid</span><span class="p">;</span>
</code></pre></div></div>

<p>Expected result:</p>

<p>0 rows</p>

<h2 id="step-4--clear-cached-installer-state">Step 4 — Clear Cached Installer State</h2>

<p>After database changes, don’t reuse an already-running installer window.</p>

<p>On the Veeam server, restart the following services:</p>
<ul>
  <li>Veeam Backup Service</li>
  <li>Veeam Installer Service</li>
  <li>Veeam Broker Service (if present)</li>
</ul>

<p>Then launch the v13 installer fresh.</p>

<p>The “Outdated Veeam Agents” check should now be gone.</p>

<h2 id="common-gotchas">Common Gotchas</h2>
<p>PostgreSQL case sensitivity</p>
<ul>
  <li>Unquoted identifiers fold to lowercase</li>
  <li>Quoted identifiers are case-sensitive</li>
</ul>

<p>If a table is named bobjects, “BObjects” will not work.</p>

<p>The hostname may not be stored as name</p>

<p>You may need to correlate using:</p>
<ul>
  <li>connection_point</li>
  <li>display_name</li>
  <li>agent_version</li>
  <li>host_id</li>
</ul>

<h1 id="deleting-bobjects-is-not-enough">Deleting bobjects is not enough</h1>

<p>The v13 installer checks EP agent inventory, not just backup objects.</p>

<h2 id="lessons-learned">Lessons Learned</h2>

<ol>
  <li>Veeam v13 validates agent inventory, not just backups</li>
  <li>Orphaned EP records can block upgrades silently</li>
  <li>Removing the EP host is the correct cleanup method</li>
  <li>Restart services to avoid cached prerequisite results</li>
</ol>

<h2 id="final-advice">Final Advice</h2>

<ul>
  <li>Always take a configuration backup before DB work</li>
  <li>Delete parent EP host records, not child agent rows</li>
  <li>Trust what the installer tells you — it’s reading real data</li>
</ul>

<p>This is one of those cases where the UI shows nothing, PowerShell can’t help, and the database is the only place the truth lives.</p>]]></content><author><name>t3hcr</name></author><summary type="html"><![CDATA[Author’s note: This post is being made to help others and is the result of my interactions with AI and should be considered as such. ALWAYS backup and test before implementing any fixes. If you’re unsure or uncomfortable making changes (or fixing them) contact Veeam support!]]></summary></entry><entry><title type="html">Culture Torii</title><link href="https://t3hcr.github.io/Culture-Torii/" rel="alternate" type="text/html" title="Culture Torii" /><published>2020-03-20T00:00:00+00:00</published><updated>2020-03-20T00:00:00+00:00</updated><id>https://t3hcr.github.io/Culture-Torii</id><content type="html" xml:base="https://t3hcr.github.io/Culture-Torii/"><![CDATA[<p>This is something new I learned about recently.  While reading some blog posts, I noticed a SaaS product called Torii.</p>

<p>Being a good netizen, I did a quick websearch for Torii to see if I could dig a bit more into it.  Turns out, I got side tracked for a bit of culture. (one of my favorite kinds of distractions!) And now, I’m sharing this with you.</p>

<p>Torii, the SaaS product: <a href="https://toriihq.com/platform/">https://toriihq.com/platform/</a></p>

<p>Torii, the traditional gateway : <a href="https://en.wikipedia.org/wiki/Torii">https://en.wikipedia.org/wiki/Torii</a></p>

<p>You’ve likely seen a real torii before if you’ve viewed photos from Asia – especially Japan, if I understand correctly.  I never knew that’s what those structures were named, nor their purpose.</p>

<p>Now I know, and so do you. 😁</p>]]></content><author><name>t3hcr</name></author><summary type="html"><![CDATA[This is something new I learned about recently. While reading some blog posts, I noticed a SaaS product called Torii.]]></summary></entry><entry><title type="html">Playing With Chocolatey</title><link href="https://t3hcr.github.io/playing-with-chocolatey/" rel="alternate" type="text/html" title="Playing With Chocolatey" /><published>2019-11-19T00:00:00+00:00</published><updated>2019-11-19T00:00:00+00:00</updated><id>https://t3hcr.github.io/playing-with-chocolatey</id><content type="html" xml:base="https://t3hcr.github.io/playing-with-chocolatey/"><![CDATA[<h1 id="playing-with-chocolatey">Playing with Chocolatey</h1>

<p>I started playing with <a href="https://www.chocolatey.org">Chocolatey</a> today. I’m planning to use it to install and update software used by myself and fellow administrators on various Windows servers.  The nice thing about it is there’s no “click, click, next, finish” to it.  It’s repeatable and reproducible and makes admin life easier.</p>

<p>To top it off, when used in combination with PowerShell you can remotely install software.  As an example:</p>

<pre><code class="language-PowerShell">Invoke-Command -ComputerName HOST1,HOST2,HOST3 {
    choco install git.install -y;
    choco install sysinternals -y;
    choco install vscode -y;
    choco install vscode-powershell -y
} -Credential (Get-Credential) -Verbose
</code></pre>

<p>I’m just getting started with it and am curious to see how upgrading software works.</p>

<p>Are you using Chocolatey? What are you using it for? I’d be curious to hear from others to learn from you. 😊</p>]]></content><author><name>t3hcr</name></author><summary type="html"><![CDATA[Playing with Chocolatey]]></summary></entry><entry><title type="html">Broken Activation In Windows Server 2019</title><link href="https://t3hcr.github.io/broken-activation-in-Windows-Server-2019/" rel="alternate" type="text/html" title="Broken Activation In Windows Server 2019" /><published>2019-09-23T00:00:00+00:00</published><updated>2019-09-23T00:00:00+00:00</updated><id>https://t3hcr.github.io/broken-activation-in-Windows-Server-2019</id><content type="html" xml:base="https://t3hcr.github.io/broken-activation-in-Windows-Server-2019/"><![CDATA[<p><em>Posting this here, just incase anyone else is trying to move the future along and dealing with this bug.</em></p>

<h1 id="caseissue">Case/issue:</h1>
<p>When activating Windows Server 2019 via the GUI and with a MAK key, you receive the following message: “The product key you entered didn’t work. Check the product key and try again, or enter a different one.”  (in my case it listed error code 0x80070490 - your error may vary)</p>

<h1 id="how-i-resolved-the-issue">How I resolved the issue:</h1>
<p>Via Admin-elevated Command Prompt, I entered the following: <code class="language-plaintext highlighter-rouge">slmgr /ipk &lt;INSERT-YOUR-MAK-KEY-HERE&gt;</code></p>

<h1 id="scratching-my-head">Scratching my head</h1>
<p>I’m a bit confused as to why something like this exists in a modern server OS, but hey - at least it’s not a huge roadblock.  While I prefer to keep activations with MAK keys to a minimum (and use KMS), in this instance a MAK was necessary.  (a non-domain joined host for a specific purpose)</p>

<p>Hope this helps someone today or further into the future.</p>]]></content><author><name>t3hcr</name></author><summary type="html"><![CDATA[Posting this here, just incase anyone else is trying to move the future along and dealing with this bug.]]></summary></entry><entry><title type="html">Adding Comments</title><link href="https://t3hcr.github.io/adding-comments/" rel="alternate" type="text/html" title="Adding Comments" /><published>2019-07-13T00:00:00+00:00</published><updated>2019-07-13T00:00:00+00:00</updated><id>https://t3hcr.github.io/adding-comments</id><content type="html" xml:base="https://t3hcr.github.io/adding-comments/"><![CDATA[<p>Coming back to whittle on this little side project to get a small blog setup for myself.</p>

<p>Today, read through commenting options for Github pages/blogs.  I’m leaning towards open source situations - and desiring to keep content (including comments) as close to my virtual home as possible.</p>

<p>The key source of help was this <a href="https://lazywinadmin.com/2019/04/moving_blog_comments.html">blog article by François-Xavier Cat</a>. (thanks François!)</p>

<p><a href="https://utteranc.es">Utterances</a> will allow for me to keep comments within the blog itself. (Saved as GitHub issues on the blog repo)</p>

<p>A bit more info can be seen at Utterance’s site at: <a href="https://utteranc.es/?installation_id=1271653&amp;setup_action=install">https://utteranc.es/?installation_id=1271653&amp;setup_action=install</a></p>

<p>Have fun out there!</p>

<p>-=t3hcr=-</p>]]></content><author><name>t3hcr</name></author><summary type="html"><![CDATA[Coming back to whittle on this little side project to get a small blog setup for myself.]]></summary></entry><entry><title type="html">Well Its A Start</title><link href="https://t3hcr.github.io/well-its-a-start/" rel="alternate" type="text/html" title="Well Its A Start" /><published>2019-05-12T00:00:00+00:00</published><updated>2019-05-12T00:00:00+00:00</updated><id>https://t3hcr.github.io/well-its-a-start</id><content type="html" xml:base="https://t3hcr.github.io/well-its-a-start/"><![CDATA[<p>Tranquility base here.</p>

<p>I’ll be back with more content as I get this figured out.</p>

<p>Until next time… learn something you can share with those around you.  I’ll be doing my part to do this for you and others too!</p>

<p>t3hcr</p>]]></content><author><name>t3hcr</name></author><summary type="html"><![CDATA[Tranquility base here.]]></summary></entry></feed>