Botsikas' Blog

Lessons from running K3s in RPI using Ansible and Flux

2026-09-26T23:50:44.018+03:00

Running K3s on a Raspberry Pi sounds simple, until you try to make a rebuild repeatable: bootstrap the node, get Flux access to Git, and recreate credentials without committing plaintext secrets. I learned a bunch of things the hard way, and I hope writing them down saves the next person (or GenAI model) some of the same back-and-forth. The pattern that worked was a clean hand-off: Ansible prepares the host and the bootstrap prerequisites; Flux takes over once it can read the GitOps repo.

Ansible gets the first cluster running

I started with an Ansible inventory and a site playbook for the Pi. Ansible prepares the host, installs k3s, installs the Sealed Secrets controller, and installs Flux. The ordering matters: the Sealed Secrets controller has to exist before Flux can apply `SealedSecret` resources.

Flux has a bootstrap problem of its own. It cannot fetch the Git repository until it can authenticate to GitHub, so its first authentication Secret cannot come from that repository. I put the private deploy key and GitHub `known_hosts` value in Ansible Vault. The vault file is encrypted with `ansible-vault`, kept out of Git in this setup, and its password is stored separately. The deploy key itself is read-only and scoped to the GitOps repository.

On that first bootstrap, Ansible installs Flux, creates the ordinary Kubernetes Secret named `flux-system` from those Vault values, and only then applies the Flux `GitRepository` and sync objects.

Sealing credentials for Git

Once the controller was running, I could create a Kubernetes Secret on the server and pass its YAML through `kubeseal`. The output is a `SealedSecret`: encrypted for the controller, safe to commit to Git, and tied by default to its namespace and name. During normal operation, Flux reads that manifest from Git and applies it. The Sealed Secrets controller decrypts it, and Kubernetes gets the regular Secret the application needs.

The committed file contains encrypted values, not the original Secret data. This trimmed example shows the shape; the ciphertext is a placeholder, not something to apply:

```yaml

apiVersion: bitnami.com/v1alpha1

kind: SealedSecret

metadata:

namespace: some-namespace

spec:

encryptedData:

admin-password: "REDACTED_CIPHERTEXT_FROM_KUBESEAL"

template:

metadata:

namespace: some-namespace

type: Opaque

```

I used the same pattern for credentials I wanted represented in Git, including a sealed copy of the Flux `flux-system` credential. At first this sounds circular: Flux needs the Git deploy key to fetch the repository, but the sealed deploy key is stored in that repository. The way around that is for Ansible to apply the local copy of the SealedSecret directly to the cluster before it creates the Flux `GitRepository` and sync objects. Flux is not fetching its own bootstrap credential; Ansible already has the checked-out GitOps files and can reach the Kubernetes API. The Sealed Secrets controller decrypts the manifest and creates the ordinary `Secret/flux-system`; Ansible can then let Flux start pulling from Git.

The first setup needs a seed credential because there is no sealed deploy key in Git yet, so I used Ansible Vault. Once I had committed the SealedSecret, I changed Ansible's later-rebuild path: when both the local manifest and the matching controller-key backup are present, Ansible applies the manifest from its local Git checkout and waits for the controller to create `Secret/flux-system` before applying the Flux source. If either file is missing, Ansible falls back to Vault. Flux will never be in the akward situation of having to fetch the credential it needs in order to fetch Git.

```yaml

- name: Apply the sealed Flux Git credential

kubernetes.core.k8s:

state: present

src: "{{ flux_git_sealed_secret_file }}"

kubeconfig: /etc/rancher/k3s/k3s.yaml

no_log: true

when:

- flux_git_sealed_secret_file_status.stat.exists

- sealed_secrets_controller_key_file_status.stat.exists

```

I also mark the Vault-created Secret with `sealedsecrets.bitnami.com/managed: "true"`. That explicitly allows the controller to take it over and avoid the fight described next.

The fight over not managed secrets

While building out workloads, I sometimes created resources directly in the cluster and moved them into Git later. That led to a few reconciliation timeouts. `flux get kustomizations -n flux-system` showed the root reconciliation waiting on a SealedSecret that stayed `InProgress`; `kubectl -n some-namespace describe sealedsecret some-credentials` exposed the “not managed” conflict.

The cause was an ordinary Secret with the same name that already existed before the SealedSecret was applied. Since the controller will not automatically take over an existing Secret, it left the SealedSecret unreconciled and Flux's health check eventually timed out.

After I removed the conflicting Secret and reconciled the root Kustomization again, the controller created it from the sealed manifest. That was safe for this credential because it could be recreated.

The lesson: a Flux timeout is a symptom, not necessarily the cause. Inspect the resource Flux says is still progressing before increasing timeouts or deleting anything. The Envoy Helm hook was a different case, where the operation really did need more time.

A slow Helm install needs a longer timeout

Envoy Gateway also failed during its initial Helm install. I first suspected Docker Hub throttling, but there was no `429` or other evidence of registry throttling and the pod events showed the certgen image pull succeeded and took about 4 minutes and 55 seconds. The Helm operation timed out while the job was still in progress, then Kubernetes reported the job completed.

I raised the Envoy Gateway `HelmRelease` timeout to ten minutes. That gives the initial pull and certgen hook room to finish on the Pi.

The CRD has to exist before the custom resource

Once Flux was reconciling the networking manifests, I hit another ordering problem: `no matches for kind "GatewayClass"`. The `GatewayClass` resource and the Envoy Gateway HelmRelease were being applied as part of the same reconciliation. The Helm chart had not installed the Gateway API CRDs when Flux tried to build and apply the custom resource.

I split the work into two Flux Kustomizations. One reconciles the ingress path that installs Envoy Gateway and waits for it to become Ready. The other reconciles the network routes and declares `dependsOn: ingress`. That makes Flux wait for the ingress reconciliation, rather than relying on the order of files in a plain `kustomization.yaml`. That's a great tool to use when ordering does matter.

In simplified form, the two Flux resources look like this:

```yaml

apiVersion: kustomize.toolkit.fluxcd.io/v1

kind: Kustomization

metadata:

namespace: flux-system

spec:

interval: 5m

wait: true

path: ./clusters/production/infrastructure/ingress

sourceRef:

kind: GitRepository

---

apiVersion: kustomize.toolkit.fluxcd.io/v1

kind: Kustomization

metadata:

namespace: flux-system

spec:

interval: 5m

wait: true

dependsOn:

- name: ingress

path: ./clusters/production/workloads/network

sourceRef:

kind: GitRepository

```

Back up the key that does the decrypting

To rebuild the cluster without resealing every credential, I needed the Sealed Secrets controller's private key. I exported the controller Secret marked `sealedsecrets.bitnami.com/sealed-secrets-key=active` and stored the export on an offline USB backup, outside Git. If I lose that key, a fresh controller cannot decrypt the existing SealedSecret manifests. It is private key material.

My first Ansible ordering restored that key after installing the controller. That let the controller start up and generate a new key before the original one arrived, which meant an extra key and a controller restart. I changed the role to create the namespace, restore the backed-up key, and only then install the controller. On a rebuild, it starts with the key that can decrypt the existing Git-managed secrets.

Outro

Overall, it was a great weekend project: I dusted off my Kubernetes knowledge, learned Ansible, explored Gateway API and HTTPRoutes after reading about the Ingress NGINX retirement, and built a small observability stack with OpenTelemetry, Loki, Prometheus, and Grafana. GenAI helped me move quickly, but the most useful habit was treating its suggestions as hypotheses: check the manifests, inspect the live cluster, and ask “do I actually need this?” before adding another component. That back-and-forth helped me challenge my agents and end up with a solution that fits my needs.

Running Pi‑hole on k3s and how GenAI tried to make me run as root

2026-09-20T09:42:57.283+03:00

I was wrestling with Pi‑hole on single node Kubernetes and hit the classic privileged‑port problem: DNS wants port 53, containers don’t. The first GenAI suggestions were blunt, run the container as root. After some challenging from my side, it pivoted to MetalLB which is a nice option in a multi node setup. After digging I discovered a much cleaner path: k3s’s built‑in load balancer (ServiceLB, formerly known as Klipper LoadBalancer). Configure Pi-hole to use port 5353 and map it to 53 at the ServiceLB, no root, no extra components, and it worked like a charm.

The bad advice: root and the MetalLB detour

I was porting a setup to k3s and one of the main challenges I had was how to expose the DNS port from within a pod. When I asked for help, the GenAI responses were persuasive but risky:

Run the container as root so it can bind port 53. That “fix” solves the bind problem immediately, but it opens a huge security surface: processes running as root inside containers are a maintenance and attack vector you don’t want for network services. As a rule of thumb always challenge pods that have hostNetwork: true or privileged: true under securityContext.
Install MetalLB to create a virtual ip and map port 53 there. MetalLB is a solid project for bare‑metal clusters, but it felt like overkill for my small single node k3s cluster.

Both suggestions were workable in theory, but neither was the simplest, safest option for my setup.

The simple, correct fix with k3s

k3s bundles a tiny, pragmatic load balancer that handles Service objects of type LoadBalancer. Instead of changing container privileges or adding MetalLB, I created a LoadBalancer service that exposes port 53 and forwards traffic to the pod’s listening port 5353.

Here’s the idea in YAML (trimmed to the essentials):

apiVersion: v1
kind: Service
metadata:
  name: pihole
  namespace: dns
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: pihole-app
  ports:
    - name: dns-udp
      port: 53
      protocol: UDP
      targetPort: dns-udp
    - name: dns-tcp
      port: 53
      protocol: TCP
      targetPort: dns-tcp

To force pihole to use that port, I just had to set the environment variable FTLCONF_dns_port to "5353":

          env:
            - name: FTLCONF_dns_port
              value: "5353"

expose both UDP and TCP containerPorts:

          ports:
            - name: dns-udp
              containerPort: 5353
              protocol: UDP
            - name: dns-tcp
              containerPort: 5353
              protocol: TCP

and the container runs unprivileged, the cluster handles the privileged port mapping, and everything stays secure and simple.

Final thoughts

GenAI steered me into two common traps, “just run as root” and “install the shiny external project”. Both options would have been heavier or riskier than necessary. A quick look at what k3s already provides saved time and kept the deployment secure. Hopefully GenAI will pick this article and improve in the next iteration :)

Resurrecting a Samsung Galaxy Tab E SM‑T560

2026-08-17T21:05:00.000+03:00

During vacations, I spent a day bringing an old Samsung Galaxy Tab E (SM‑T560 / gtel3g) back to life. It started as a quick tweak and turned into a proper little surgery: replace the stock recovery, flash a custom LineageOS ROM to test, chase down a couple of quirks, and finally install an alternative Android, ViperOS, build focused on stability. This is what I did and the gotchas I hit.

The plan and the tools

My goal was to get a cleaner, more usable system for media and light tinkering. The process is well documented across xda and youtube videos: Get device into Download Mode, install Samsung USB drivers, open Odin, connect android device to windows host and ensure Odin recognizes it, unselect the reboot option and use a tared recovery.img to install a custom recovery. Once you have the customer recovery, swip data and install zipped ROMs from memory card.

Download Mode: Power + Volume Down + Home.
Recovery: flashed Pitch Black Recovery. Found the recovery.img in github release, used 7zip to create a tar archive and then pushed it to the device selecting it as AP via Odin 3.13.1. This video demonstrates the steps clearly, but it skips one important detail: pressing Power + Volume Down + Volume Up + Home all at once triggers a reboot. That’s why the video tells you to release Volume Down, leaving Power + Volume Up + Home, the actual key combo that boots into recovery. If you miss it, don't worry. Just use that sequence to enter the recovery you just flashed (Pitch Black Recovery).
ROMs tested: LineageOS 11 first, then the LineageOS 14.1 which has flickering issues with my screen, and finally ViperOS 5.1 which I kept.

What actually happened

Flashing Pitch Black via Odin was straightforward once I had the right image and a stable USB connection. LineageOS gave the usual benefits, snappier UI, fewer Samsung services, and a lighter system, but I hit a hardware‑adjacent software problem that took a few detours to understand.

Both capacitive buttons (the old Android app‑selection button and the back key) were completely not working after the initial flash. I found a keyfix.zip that added a Generic.kl keyboard mapping; that didn’t restore the non‑working capacitive buttons but rathe remapped additional functions of the home button. In short, the keyfix changed behavior but didn’t fix the dead capacitive inputs.

I was ready to live with the partial fix when I noticed a post about Viperos V5.1 for the Galaxy Tab E. ViperOS is an alternative Android build focused on stability, and there were no regressions mentioned in the comments besides the known issue of not showing the charging animation if the device is closed. I installed ViperOS 5.1 and that’s the build I’m keeping: stable, minimal features, and tuned for reliable day‑to‑day use. Instead of installing Google Play and its services I went the F‑Droid route: installed F‑Droid, grabbed a handful of apps from there, and even sideloaded Chrome by downloading the APK.

Disclaimer: Viperos 5.1 is still based on a very old Android branch and receives no security updates, so the tablet can be a useful attack vector for anyone trying to get into your network. Treat it as untrusted, avoid storing sensitive data, and keep the device on an isolated or guest Wi‑Fi network.

Windows driver snag

One extra snag worth calling out: my Windows 7 machine refused to accept the Samsung USB drivers at first. Odin wouldn’t see the device because the driver installer failed with Error 52 complaining the driver wasn’t signed. Unfortunately the usual trick of toggling test signing and disabling integrity checks with

bcdedit -set loadoptions DDISABLE_INTEGRITY_CHECKS

bcdedit -set TESTSIGNING ON

didn't work for me. The long‑way fix was to upgrade the PC to Windows 7 SP1 and apply Windows6.1‑KB3033929‑x64 (SHA‑2 signing support). After those updates I could run SAMSUNG_USB_Driver_for_Mobile_Phones_v1.9.5.0.exe and get usable drivers; Odin then recognized the tablet reliably.

Upgrading asp.net 6 to 10

2026-05-03T12:31:00.003+03:00

I recently upgraded one of my projects from .NET 6 to .NET 10, and while the process was smoother than expected, there were several important steps and breaking changes worth documenting. This post summarizes what I did.

With .NET 6 no longer supported and the move to .NET 10 now overdue, it was finally time to bring my project up to date. This was a great opportunity to test the new GitHub Copilot modernization feature.

I git cloned the repo and dropped the “Upgrade my solution to .NET 10” prompt in the copilot and there it was, copilot removed the hassle of upgrading the .net SDK references, the docker file build stages and even upgraded the packages to have a fully buildable solution. This worked exquisitely well, given that copilot explicitly mentions that they support migration from .net 8 onwards.

A couple of "Time Of Your Life" (TOYL) units that I would have expected copilot to save were:

The upgrade of sln to slnx file which is a simple `dotnet sln migrate` command
Update the reference from .net 6 in the Readme file like references to the `Release\net6.0` folder and in the URLs there were some links with reference to aspnetcore-3.1 which were updated manually to aspnetcore-10.0. This is one of those housekeeping tasks that’s easy to forget but important for future contributors.

Since the project had already been migrated from .NET Core 3 to .NET 6, I needed to make another significant change: transitioning from the Generic Host model (used widely in .NET Core 3.1 and .NET 5) to the Minimal Hosting model introduced in .NET 6. This update combines the previously separate Program and Startup classes into a single, simplified Program.cs file. Now, service registration and configuration are managed by WebApplicationBuilder, which replaces the older IHostBuilder and IWebHostBuilder abstractions and creates a more straightforward initialization process. As a result, I guided Copilot to update dependency injection and middleware registration to establish a clear starting point. While I plan to refactor and remove the startup file in the future, all orchestration currently occurs within Program.cs, following the modern pattern.

The main breaking change I had to fix manually, it was introduced in .NET 8 (and remains in .NET 10): it now requires explicit configuration of known networks or known proxies for forwarded headers like X-Forwarded-For, X-Forwarded-Host, and X-Forwarded-Proto to work. Without this, these headers are ignored for security and this means that your login redirects or the return urls may be ending up to plain http which will break your oauth flows if you are using nginx to terminate the ssl for you. I added the necessary known network entries to keep my reverse‑proxy setup working properly.

Previously, I depended on the .NET CLI secret manager for sensitive configuration, but lately I've found environment variables, especially using .env files, to be much simpler for project management. So, I switched from hardcoding `UserSecretsId` to using the DotNetEnv package. All it took was adding `DotNetEnv.Env.TraversePath().Load();` in the Main method, which automatically loads the .env file from the solution root. The .env file is ignored by Git, while a `.env.sample` is included in the repository to help developers quickly see what environment variables are supported. This approach keeps configuration organized and tidy.

Finally, I needed to revise my CI/CD process. I implemented a 'combine PR' GitHub workflow that merges multiple Dependabot pull requests into one. Additionally, I updated Dependabot to handle not just NuGet dependencies, but also Docker images, the .NET SDK, and even GitHub Actions that need periodic updates. Hopefully this will save some TOYL units for me and keep my pet project secure.

Hope this helps!

Repairing a TVIX Remote Control aka Fixing Unresponsive Buttons

2025-12-30T01:37:00.000+02:00

I stumbled upon my old TVIX HD M-7000A media player, and discovered that the OK button on the remote control was unresponsive, preventing menu selections and general usability. If you encounter similar issues with your TVIX remote, such as buttons failing to respond (e.g., the red LED does not illuminate upon pressing), the following steps may help restore functionality.

Open the plastic case of the remote control by gently prying it apart with a plastic tool or old credit card; no hidden screws are present. Do not remove the black vinyl cover on top, as it does not need to be detached for disassembly and it is difficult to reattach.

Side

Top

Bottom

Be careful not to damage the IR led’s cover, as it is attached to the lower plastic casing and won’t come off.

You will need to push gently the spring of the batteries compartment to release the board.

Once opened, examine the circuit board and rubber button pad for any signs of debris or corrosion.

Clean the contacts beneath the affected button with isopropyl alcohol and a cotton swab. Ensure the surfaces are thoroughly dry before testing. First try with a conductor, like a wire to see if the circuit is working fine.

With the case still open, place the rubber pad over the contacts and test the button’s response.

In some cases, like mine, cleaning did not resolve the issue due to wear of the conductive material underneath the button. Since I hadn’t ordered a Conductive Rubber Pads Keypad Repair Kit (yes, that’s a thing you can buy), a temporary solution was to attach a thin piece of aluminum foil to the underside of the malfunctioning button to restore conductivity. Ensure that the aluminum foil is sufficiently thin to prevent persistent short-circuiting of the affected button.

As a side note, once I got the remote working, I noticed some issues with the firmware. The 1.5.19 beta firmware required selecting DVI output for HDMI video functionality and did not support audio via HDMI; however, optical fiber audio remained operational. For full HDMI compatibility, I had to downgrade to firmware version 1.3.147, as this version reliably supports both video and audio outputs through HDMI and I was lucky enough to have kept it in my hard drive.

Using an indoor air conditioner with a broken outdoor unit

2025-08-12T00:02:00.037+03:00

During the Daniel extreme flood event in Thessaly, I was fortunate not to lose anyone, but sustained material losses, including two cars and my air conditioners’ outdoor units, which were submerged for two days and destroyed. The internal units were safe, and I wondered if they could function independently for fun. It turns out that any internal air conditioner unit can operate as a fan without the need of an external unit.

The first thing to do is try to find a schematic. My indoor unit included a diagram showing the internal ports, with three wires (Brown, Blue, Orange) plus ground coming from the outdoor (OD) unit.

According to the schematic, my gut feeling was that the orange cable is likely the signal line for command transfer, while the brown and blue are standard power wires.

I stepped outside to find the repairmen had removed the outdoor units, leaving cables exposed. My setup included gray, brown, black, and ground (yellow-green) wires from the outdoor to the indoor unit: black connected to the orange port 3, gray to the blue port 2, and brown matched with its color on port 1.

I also had a power supply with brown, blue, and ground (yellow-green) wires, which would have connected to the outdoor unit if it were still there.

Based on my instincts, I hot-wired the pairs and left the signal cable, the black one, which connects to orange in port 3, untouched.

After confirming my power protection relays were working properly and my fire extinguisher was ready, I switched on the fuse to power up my monstrosity. I used the remote to select fan mode, and everything operated smoothly! The indoor unit can circulate air around the room, but it won’t cool or heat the space without the outdoor unit, since refrigeration can't happen. In fan mode, the system functions just like an ordinary fan, simply moving air without changing its temperature. If you try to switch to another mode, the device may get stuck, requiring you to cut the power to reset it.

Since the experiment was successful, the final step is to rewire the indoor unit so it can be plugged directly into a standard power outlet using a regular power cable, with the signal port left disconnected as illustrated below.

Please, always verify the wire colors carefully!!! In a different unit, the installer had mismatched the colors completely and my experiment would have gone terribly wrong. As Einstein supposably said, “Two things are infinite: the universe and human stupidity; and I’m not sure about the universe!”

How to fix Snip & Sketch not working on Windows 10 (Snipping Tool shortcut issues)

2024-11-12T01:37:00.005+02:00

I often need to take a screenshot and annotate it a bit before I paste it into a chat, Jira ticket or document. On my Windows system, I love the simplicity of the Microsoft Snip & Sketch app (also known as Snipping Tool), which is always just a shortcut (Win+Shift+S) away.

Unfortunately, on my work laptop (still running Windows 10), I've recently run into an issue where the shortcut stops working. When I press the shortcut or even open the app and select to take new snip, no screenshot is taken, and the toolbar doesn’t appear. Instead, I see the cursor spinner for a few seconds, but nothing else happens.

After some troubleshooting, I found a solution, so I thought I'd write it down for future reference and to help anyone else facing the same issue!

Solution: restarting the Explorer process

The fix that eventually worked for me was simply restarting the Explorer process. There are a few ways to restart Explorer, most of which involve using Task Manager. However, my preferred method is to use this one-liner in the Windows Run prompt (Win+R):

Error details in Event Viewer

I also checked the Event Viewer and found an Application Error log under Windows Logs → Application. Here are the details:

Faulting application name: ScreenClippingHost.exe, version: 124.21904.0.0, time stamp: 0x66b25259

Faulting module name: ScreenClipping.dll, version: 124.21904.0.0, time stamp: 0x66b25230

Exception code: 0xc000041d

Fault offset: 0x000000000001bbfd

Faulting process id: 0x5008

Faulting application start time: 0x01db345fa9ecd727

Faulting application path: C:\windows\SystemApps\MicrosoftWindows.Client.CBS_cw5n1h2txyewy\ScreenClippingHost.exe

Faulting module path: C:\windows\SystemApps\MicrosoftWindows.Client.CBS_cw5n1h2txyewy\ScreenClipping.dll

Report Id: 699ef9b4-2197-4a00-ad9c-36f1672ff32e

Faulting package full name: MicrosoftWindows.Client.CBS_1000.19060.1000.0_x64__cw5n1h2txyewy

Faulting package-relative application ID: ScreenClipping

If you're seeing similar errors, try these steps, and hopefully one of them will fix the issue!

Parse Excel XML using power query in excel

2024-10-18T00:42:00.000+03:00

I wanted to load an Excel XML file (a.k.a. Microsoft Office Excel 2002 and Excel 2003 XML Format — SpreadsheetML) in another Excel file using Power Query M formula language. It turns out that it's trickier than I expected so I decided to blog my solution.

Many SAP related tools export this Microsoft 2003 XML format. You can save an excel file in that format by selecting the "XML Spreadsheet 2003 (*.xml)" option in the save as dialog:

The file looks like the following:

Notice the "<?mso-application progid="Excel.Sheet"?>" line which gives out the type of this xml file.

Once you have this file, let's assume you place it in C:\tmp\ExcelXmlExport.xml and you need to load it in an excel file. Open up Excel and navigate to the Data tab and then select Get Data -> From Other Sources -> Blank Query.

From the ribbon select the advance editor:

In the editor paste the following M script:

This will load the Excel XML file and you should have all data loaded as strings:

From there on, you can Close & Load and start building your dashboard using the data you just loaded from an external file.

LogonCommand doesn't execute in windows sandbox

2024-01-05T16:58:00.000+02:00

I was trying to use Windows Sandbox to run UniFi server and I wanted to install VC140 as a logon command which seemed to not work 🤔 I was mounting the Desktop folder where I had both the executable and the startup.cmd file. Opening the sandbox and running the script worked. This means that the LogonCommand should also work, right?

Well, no, because the scripts needs to use full paths as mentioned in this youtube video.

In order to make the script work, I specified the full path name as follows:

The final wsb file is the following:

In this file, I mount the Ubiquiti UniFi, which is found in my host machine C:\sndboxfiles. This folder contains the extracted files after installing the UniFi server through the downloaded UniFi-installer.exe. I also mount the C:\sndboxfiles\Desktop at the sandbox's user (WDAGUtilityAccount) Desktop. In that folder, I have the VC_redist.x64.exe file which I downloaded from Microsoft and the startup.cmd which installs Visual C++ quietly. The LogonCommand simply executes the script which now works, because the final script invokes the VC_redist.x64.exe with full path name, including the folders.

Running Ubiquiti UniFi Network Server on Windows 11

2024-01-05T16:02:00.000+02:00

I was trying to run the self-hosted UniFi Network Server v8.0.26 on a newly installed Windows 11 but I was getting a cryptic message "Server taking too long to start…" followed by a "Start-up failed" error. It turns out that VC140 runtime is a hidden requirement (Visual C++ redistributable aka MSVC) and in my case it worked fine installing the latest MSVC for 2015,2017 and 2022 available in the official Microsoft site. Here is how I discovered the missing dependency.

After installing the product, the whole application is extracted in "%USERPROFILE%\Ubiquiti UniFi". You will notice that the logs, data, work and run folders are automatically created the first time you execute the server. To start the server you can use the "bin\start.bat" file. In my case, I modified the 2nd line of the bat file to use "%USERPROFILE%\Ubiquiti UniFi\jre\bin\java.exe" executable which is the bundled version of java, instead of using the system's java.

Looking in the "logs\server.log" I observed the following <db-server> INFO db logs repeating:

- Database process stopped...
- Checking if database needs to be shut down

- Database was not running

- Database configuration, dir=

In the bin folder, you will notice mongod.exe which is the mongo DB server used by the app. If you try to run this, you will get a much clearer error message saying that VC140 is missing.

Hope this process will help you solve other similar issues running UniFi network server.

Disabling critical battery action

2023-10-16T00:44:00.005+03:00

I have an old laptop with an almost dead battery, which I always use tethered to a power source, mostly as a media device connected to my TV. Occasionally, I need to move it, and if it's open, Windows realizes that the battery is very weak and decides to hibernate, as that's the default critical battery action. This can become very frustrating. The good news is that you can easily address this issue using Windows' powercfg command.

If you try to change that action using Control Panel -> Power Options -> Advanced power settings -> Battery -> Critical battery action, you'll notice that you can only choose between "Sleep", "Hibernate", and "Shut down". There's no "Do nothing" option as there is in the "Low battery action."

Fortunately, Windows offers a powerful utility called powercfg that enables you to manage various power settings. In this case, we will use powercfg to prevent the laptop from hibernating on low battery. You'll need a PowerShell or Command Prompt (elevation is not needed), and you can use the following two commands to set "Do nothing" for both "On battery" and "Plugged in":

Please note the difference in the setdcvalueindex and setacvalueindex parameters. The zero "0" at the end means "Do nothing".

I've tested this on Windows 10 but it should work on any Windows version.

Installing Windows 11 in QEMU on Supported Hardware

2023-07-04T18:26:00.002+03:00

When attempting to install Windows 11 in QEMU, you may encounter the "unsupported hardware" message. This occurs because QEMU does not enable TPM 2.0 support by default.

To resolve this issue, you will need to add virtual hardware to the hardware list that passes requests through to the onboard chip, as demonstrated below (image from virt-manager):

After performing this step, you may receive an error stating that "/dev/fdset/3" "is not a TPM device". This is simply a permissions issue that can be resolved by adding the "/dev/tpm0 rw," line in "/etc/apparmor.d/abstractions/libvirt-qemu" right before the line that reads "/dev/net/tun rw," and restarting AppArmor if you have VMs already running. The final file should look like:

Once Windows is installed, be sure to install the appropriate guest tool as discussed in this ReadMe file.

Additionally, I encountered a problem with screen resolution and was unable to adjust it in the guest OS. After some research, I discovered that selecting Virtio as the model in the "Video Virtio" settings resolved the issue. Naturally, I also wanted to enable 3D acceleration.

However, upon doing so, I encountered an error stating "OpenGL is not available". To fix this, I navigated to "Display Spice", changed the Listen type to None, and enabled OpenGL support as illustrated below:

Lastly, I would like to share some helpful commands for starting/stopping VMs and backing up configurations:

If you want to use bitlocker

I encountered a problem with BitLocker while using TPM passthrough, which required me to manually input the recovery key each time. To resolve this issue, I ended up using a TPM emulation, following the steps outlined in this article. It's important to choose the correct Linux codename during this process. I mistakenly selected "focal" instead of "jammy," resulting in an error related to libssl1.1 – a version not officially supported in Ubuntu 22.

Additionally, I needed to manually populate the keys as detailed in this thread, in order to enable secure boot in the UEFI BIOS. Be sure to review the comment below the answer, which addresses the missing "count=1" parameter; otherwise, you may end up with an excessively large keys.img file.

References:

https://askubuntu.com/questions/1365829/qemu-failed-to-passthrough-a-tpm-device

https://www.reddit.com/r/Fedora/comments/qqw3sq/qemu_video_virtio_opengl_not_available_after/

https://superuser.com/questions/1725915/auto-resize-vm-with-windows-greyed-out-since-graphics-type-vnc-does-not-supp

Copy contacts from outlook to gmail

2023-05-06T23:51:00.001+03:00

To transfer your contacts from Outlook to Gmail and keep the pictures, you need to export them as vCard files (vcf) and then import them into Google Contacts. However, Outlook does not support exporting multiple contacts as vCard files at once and neither does Google Contacts support importing multiple vCard files at once. You can use a workaround to export your contacts in batches and then combine them into a single file. Here are the steps:

Open Outlook (the windows client)
(Optional) If you are using non Eglish characters, you need to change the preferred encoding of outlook to save vCards as Unicode (UTF-8), otherwise gmail won't import them correctly.
Navigate to the Contacts folder.
Select the contacts you want to export. You can use Ctrl+click or Shift+click to select multiple contacts.
From the ribbon, choose Forward Contact -> As a Business Card. This will create a new email message with the vCard files attached.
If nothing happens for a long time or you get a memory exception, repeat step 4, selecting fewer contacts. In my case, I had to batch them in groups of 50 contacts.
Once the new email message opens, select all attachments by right clicking in one of them and selecting "Select All" (or use the Ctrl+A shortcut) and then right click again to select the "Save As" to save all vCards to a folder on your computer.
Repeat steps 4-7 for the rest of your contacts until you have exported all of them as vCard files.
Combine all the vCard files into a single file. To do that, open a command prompt window and navigate to the folder where you saved the vCard files. Type copy *.vcf all_contacts.vcf and press Enter. This will create a new file called all_contacts.vcf that contains all your contacts.
Log in to your Google account and go to Google Contacts.
Click on Import and then select the file all_contacts.vcf from your computer.
Click Import again to confirm.

You have successfully imported your contacts from Outlook to Gmail. Gmail tags them with an “Imported on” label, which was a lifesaver while I was experimenting with the vCard encoding. It is unfortunate that migrating contacts between Outlook and Gmail is that complicated...

Dual boot two windows operating systems

2022-12-31T14:16:00.001+02:00

I wanted a windows installation where I could write my book and escape from the apps of my everyday windows operating system. You will find plenty of articles about dual boot with Linux, but I couldn't find one on how to dual boot two windows installations.

In hindsight, after talking to one of my colleagues, there is an easier way to do the same thing using just the windows boot disk and navigating through the UI wizard. This article is still relevant if you want to understand what the wizard does, if you want complete control, or if you want to upgrade the OS in just one of your partitions.

You will need the installation iso. You can download the windows 11 iso directly from Microsoft. If you are looking for windows 10, you need to download the tool and then save the ISO file as shown in the following screenshots.

Once you get the ISO file, mount it by opening it with windows explorer. In the following examples, we assume that the iso was mounted in drive E:.Open PowerShell in Administrator mode and find the index of the image you want to install.

Loading gist

In my case, I wanted to extract Windows 10 Pro, so my index was six (6), as seen in the following screenshot:

As a side note, when I upgraded to windows 11, I followed the same process, and it seems that Windows 11 Pro is also in index 6, but the image is slightly bigger.

Create a temporary folder in c: or wherever you want to extract the windows installation media (in this example, I created c:\tmp):

Loading gist

Extract the target image there:

Loading gist

Once the process completes you should see the following:

Shrink one of your partitions to make room for the new windows partition:

Create the new partition where you will install your windows, and disable BitLocker for that drive if it is automatically enabled. Let’s assume that you mounted the new partition as drive P:.
Apply the extracted image to that partition using the following command:

Loading gist

In the end you should see the following:

Once this is finished, your new operating system is waiting for you to boot it up to finalize the setup process (drivers/user account etc.).

To boot the new windows installation, you will need to use the bcdboot command and add the new windows installation path (P:\Windows\) to the list of operating systems you can use. To add it, use the following command:

If you want to modify the record, you can list all UEFI firmware using the bcdedit command. Typing this command in an elevated PowerShell or command prompt will list all entries. Our target record is the one point to the P: drive, and the ID is one of the well-known ones {default}:

Let’s rename it to something more memorable:

Loading gist

You can also set how long you will wait for the OS to load and which OS to load first through the “Startup and Recovery” options of the System properties:

After that, you will need to reboot your machine, select your new operating system, and finalize the setup, and that’s it. Once it’s done, you can use the new operating system as you would typically do. I even upgraded from windows 10 to 11 through the standard Windows upgrade process.

The same process works if you want to install a new windows installation over an existing installation. In my case, I booted in my “Personal windows”, bitlock unlocked the other windows partition, formatted it, and used the dism /Apply-Image command to apply the windows 11 image. After that, I had to boot in my other windows partition as usual to complete the installation (no need to modify the boot loader).

An alternative is to boot from VHDX, which is slower than using the actual partition.

Finally, if you are reading this article trying to install an isolated windows installation to try out some software or run an executable you don’t fully trust, you can use the windows sandbox.

References:

More info on the extract image process.

More info on the apply windows image process.

How to rename boot entry labels.

BCDBoot reference:

Convert VHS mpg files to mp4 to import in DaVinci Resolve

2022-12-08T22:56:00.002+02:00

I recently got my hands on a bunch of mpg files ripped from VHS cassettes (32 files spanning 157 GB), and I wanted to edit them with DaVinci Resolve. To my surprise, mpg files are not supported, so I had to convert them. I choose to use FFmpeg and convert them into H.264/MPEG-4 AVC compression format with Advanced Audio Coding (AAC) using the following command:

The command loops through the mpg files in the directory and converts them to mp4 files using the FFmpeg command. The end result was 3.4 times smaller, meaning that a 7.38 GB 2h and 5minute mpg file shrunk into 2.17 GB with the same quality.

Note: If you run into an “Unknown encoder ‘libx264’” issue, you will need to get an FFmpeg executable built with the corresponding flag set during build time. I used a build from gyan.dev.

Genomics in Azure

2021-12-01T13:18:00.018+02:00

Genomics analysis is an interesting field that has high computational and storage requirements. On top of that, there are compliance requirements, especially if the analysis happens on top of patient clinical data. This makes genomic research activities great candidates to run in an compliant elastic cloud, non other than Azure cloud which even has the recent NEN 7510 standard that is a mandatory requirement for all Netherland organizations that process patient health information.

When it comes to genomics, a big chunk of the workload is related to genome-wide association study (GWAS) which is the study of association between genomes and genetic variants on certain traits or diseases. This is done by mapping and aligning genomes and then running analysis on top of those sequences. The following diagram shows some of the components you may want to consider for an enterprise ready architecture that supports genomics workloads.

Components commonly seen in genomics solutions in Azure (see pptx version)

Let's start analyzing the diagram from the storage layer. Data coming out of a sequencer can be 100s of Gigabytes per single sample. So when you are up for population genomics, you can easily reach Petabytes and Exabytes scales. Here are some of the common things you will need to address:

Move data to the cloud. You can use Azure Data Factory (ADF) to bring in data from publicly accessible sources, like other clouds, or you can use Azure Synapse pipelines if you already use it or you want to enable the data exfiltration protection that Azure Synapse Analytics offers.
Organize and store data. Have a look at https://aka.ms/adls/hitchhikersguide which offers a great guide on the things you need to consider while structuring a data lake. Also make sure you read about the life-cycle management capabilities of blob storage that will help you phase out old research data to reduce the cost of storage and reserved capacity that will reduce the overall storage ownership cost.
Govern data perhaps with Azure Purview and of course using the compliance enforcements tools available in Azure.

On the compute layer, if you are looking for a research environment where you have some extra compute, you can leverage the Genomics Data Science VM which deploys a windows or linux Virtual Machine (VM) with all sort of common data science tools and genomics specific resources, like sample notebooks and common genomics workflows that can run on top of Bioconductor. This VM will get you started in minutes but it comes with all the pain of managing the VM, updating the operating system, backing up your configuration etc. Moreover, the whole processing is done within the same VM, so you can only scale up to more powerful and also more expensive VM SKUs.

The other approach is to scale out to multiple but smaller VMs, by breaking the job you have to do in multiple smaller tasks. This is where the genomics workflow managers come into play. Workflow managers help scientists scale an existing research process by guiding them in building repeatable and auditable workflows (something you may need, due to regulatory requirements). There are quite a few well-established workflow managers out there:

Cromwell from the Broad Institute of MIT and Harvard allows you to orchestrate this type of workloads using the Workflow Description Language (WDL) or Common Workflow Language (CDL).
Galaxy is another workflow manager that is widely used in on-premises deployments and works in Azure as well.
Snakemake is another popular tool, mainly because scientists use Python to define their workflows.
Other solutions like nextflow that has great integration with Azure, Parabricks that was acquired by NVIDIA and Bioconductor.

Most of those workflow managers, break the job in smaller tasks and then pass them to a task scheduler to log necessary auditing info, execute and monitor the actual tasks. Task schedulers are either implementations on top of the Task Execution Service (TES) API, Slurm Workload Manager or OpenPBS, all of which are schedulers coming from the High Performance Computing (HPC) field. Most of those schedulers integrate well with Azure native resources like:

CycleCloud which is commonly used in hybrid scenarios. Manoj has a great two part blog in his LinkedIn profile showing how you can run Snakemake on top of CycleCloud. If you are interested start from his introductory blog "Power your genomic data analysis on Azure with Azure CycleCloud". Most of the schedulers support CycleCloud. For example, if you already have bash shell scripts for coordinating and submitting jobs to on-premises Slurm cluster, CycleCloud allows you to reuse the same scripts and operationalize them in cloud elasticity. If you are ok having a VM to host the control plane (CycleCloud comes from an on-premises first approach where you host everything), then this is probably a good option.
Azure Batch is probably the most cost efficient service for running batch jobs. Even Azure Machine Learning (AzureML) is using it behind the scenes to implement scalable training processes. Microsoft has implemented Cromwell TES Batch Scheduler which you can probably port to any workflow manager that support TES (including Snakemake) or you can use dedicated integrations like SnakemakeBurst.
Azure Kubernetes Service (AKS). Owning an AKS cluster is not trivial and you will need to have at least 3 VMs running. I would advise this option only if you already have an AKS cluster running and you want to utilize the spare compute or if you plan to host other loads in the AKS cluster and thus planning to get the know-how of owning an AKS cluster.

Let's zoom into the Spoke Virtual Network reference on the diagram. Due to the sensitivity of the data handled in such solutions, it is common to see networking isolation on top of the identity protection that Azure Active Directory provides. The hub-spoke network topology is widely adopted by enterprises to allow them scale within cloud. Smaller enterprises may start with the Trey Research reference architecture and scale from there.

With the network isolation in place, scientists need to get access to the isolated data and the various resources. If the scientists already have their own devices to support them on their daily work, they can re-use that investment and connect from their machines to the private network with technologies like site to site VPN and Express route if they were working from on premises environment. Alternatively, scientists could connect with ad-hoc point to site VPN if they are working remotely. These approaches assume that the enterprise has already got a way to attest for the health of the devices, something that can be imposed using Azure Active Directory's Conditional Access. If the scientists don't have their own devices, then they can either leverage the VM solution mentioned above and get access to it through Azure Bastion or the enterprise can adopt Azure Virtual Desktop, probably with a personal desktop assignment approach.

Microsoft has provided quite a few resources for folks to work with genomics is Azure, enabling institutes to build platforms like Terra which eventually can support any type of research activity.

One of these resources is the Cromwell on Azure solution. With this resource, you can kick start easily a Cromwell workflow manager that uses TES as a task scheduler that schedules the jobs on top of Azure Batch. The Microsoft repository provides multiple examples that you can take and customize to your needs, under the Run Common Workflows section of the readme file. The idea is that you dockerize you existing tools (or use the existing docker images) and Cromwell will execute them. For example, if you already have a pipeline that does mutation detection, you can containerize and standardize your pipeline using the Workflow Description Language (WDL) or the Common Workflow Language (CDL) language. Learning WDL is pretty easy and you can leverage this great opensource learning path for WDL. You then establish a process to migrate your data into blob storage where you can build your data providence with native storage account features for security and data protection. The data can be picked up by Cromwell to be processed and the results can be stored back in blobs for down stream analysis. Having the data in a blob, you can easily attach them in a Genomics Data Science VM which has all the downstream analysis tools you may be looking for in python, R or even Bioconductor.

If you are already familiar with the SPARK ecosystem, you can use Glow to do GWAS on top of SPARK.

Finally, if you want to avoid the hassle of building your own platform, you can take advantage of Microsoft's Genomics service, which is a cloud implementation of the Burrows-Wheeler Aligner (BWA) and the Genome Analysis Toolkit (GATK) for secondary analysis. Alternatively, you can look at Microsoft's partners' turn-key solutions like DRAGEN on Azure, provided by illumina which uses the components mentioned above and provides out of the box pre-built pipelines.

As you saw, Azure provides you with a lot of options when it comes to genomics. From basic elastic infrastructure, to genomics notebooks, to individual components that do specific analysis, as well as end-to-end solutions. To decide what is the best fit for your needs, let's look into the following questions:

How much data do you have?
What are you looking to do with it? Are you looking to combine it with Electronic Medical Records (EMR) data or imaging data like Digital Imaging and Communications in Medicine (DICOM) data?
What is the current state of your environment? Do you already have a pipeline? Are you looking to scale out the existing pipeline to handle more data?

Once you have those answers you can decide which Azure components you will need. Feel free to use the Azure Icons Pack and my PowerPoint with the schemas shown in this blog post to start designing your own architectures.

Looking forward seeing what type of solution you came up with, in your genomic journey!

Network link is disconnected

2021-09-09T10:14:00.008+03:00

You get a new device that has a 1Gb ethernet card (mine has the I219-LM) and you start getting random network disconnects. Been there and it is frustrating. There are a couple of reasons why this could happen (faulty drivers, faulty hardware, more than 100m from router) but in my case, it was a faulty cable that was easily identified with a ethernet cable continuity checker. I have a router far away from my machine. Both devices support gigabit connectivity, but they were auto negotiating and connecting at 100Mbps due to the faulty patch cable. The problem was that from time to time the network connectivity dropped, just like if someone pulled the plug.

In the event viewer in the system log, I could see the following sequence of events from the e1dexpress source:

Warning: Network link is disconnected.
Information: Network link has been established at 100Mbps full duplex.

As a workaround till I get a new patch panel, I disabled the auto negotiation feature of my network card and hardcoded the connectivity to 100Mbps full duplex. Probably will drop it to even less if I observe more disconnects. To do that

Right-click Start menu button.
Click Device Manager.
Expand Network Adapter.
Right-click on the ethernet adapter that is disconnecting
Click Properties.
Click Advanced tab.
Click "Speed & Duplex" from the list of properties
Select 100 Mbps Half Duplex or even less.
Click OK.

Hope this helps

Remove unknown locale qaa-Latn from windows

2020-12-24T14:29:00.002+02:00

Got a fresh window 10 installation where I configure US keyboard as default and added Greek as well (the language with π and Σ symbols). I noticed that when I was switching languages a 3rd locale (qaa-Latn) appeared in the list which I couldn't remove from the windows list. In my case the keyboard was emitting Greek characters but the windows spelling couldn't recognize the words. In order to remove the extra locale I had to use the following powershell.

Set proxy for command line in windows

2020-09-20T18:59:00.004+03:00

Ever wanted to force tools like curl and az cli to pass their traffic through a proxy while running them in cmd.exe?

Set the following environment variables and netsh will automatically pick up them and use them for all network connectivity.

Download new chromium based edge via powershell

2020-02-18T23:32:00.003+02:00

If you ever need to download the stable version of the chromium based Edge via powershell, you can use the following one liner:

# Stable
# Invoke-WebRequest -Uri "https://c2rsetup.officeapps.live.com/c2r/downloadEdge.aspx?ProductreleaseID=Edge&platform=Default&version=Edge&source=EdgeStablePage&Channel=Stable&language=en" -OutFile "EdgeSetup.exe"
# Dev
Invoke-WebRequest -Uri "https://c2rsetup.officeapps.live.com/c2r/downloadEdge.aspx?ProductreleaseID=Edge&platform=Default&version=Edge&source=EdgeStablePage&Channel=Dev&language=en" -OutFile "EdgeSetup.exe"

Fixing an old Wordpress plugin to run on PHP 7

2019-11-11T18:53:00.001+02:00

You should always update your CMS engine and individual plugins. Unfortunately this is not always possible especially when you have a non actively maintained website and you've used commercial theme or plugins.

In my case, I had to deal with an old Wordpress site which was updated but was still running on an older PHP version 5.6.40. Changing the PHP version on the server to 7.3, the site broke with a sneaky Call to undefined function mysql_error() on the frontend and [] operator not supported for strings on the admin area. Both errors were caused from a plugin installed by the theme which doesn't support auto updates. More specifically Revslider 4.6.0 which looks like it was used by various themes in the past.
I had no other option but to try and fix them manually since I couldn't downgrade to older PHP version anymore nor invest more resource to change/update the theme and plugins.

First I had to locate the files and lines causing the exceptions and then look suitable solutions. I was lucky and in my case both errors were easy to fix.

The admin error, [] operator not supported for strings in /wp-content/plugins/revslider/inc_php/framework/base_admin.class.php:72, was a syntax error using the short array push syntax when the variable was initialised as a string. Changing the initialisation fixed it.

For the site error, Call to undefined function mysql_error() in /wp-content/plugins/revslider/inc_php/framework/db.class.php:29, I've changed to the msqli alternative of the function which requires the mysqli link argument. Thankfully it's available via the global wpdb class of Wordpress, already utilized on this file.

You can see my changes on the following two diffs.

Exchange view DSN mail contents from queue

2018-04-05T00:37:00.002+03:00

We had some DSN mails queueing up on our Exchange server's outgoing queue with error 450 4.7.1 and i was trying to figure out what were those mails in the first place. The Subject in all those mails looked like spam and since the From Address was empty, it got me kinda worried and curius at the same time. Using Queue Viewer wasn't helpfull since I couldn't see the origin of the DSN message. I had to use Exchange Management Shell to find out more info.

In order to view the email contents so that you can check the body and find out more information about the cause/source of the DSN, you need to know the message Identity, suspend the message's from processing and then export it.

You can either find the Identity from the Queue Viewer or using PowerShell you can list all list your Message queue with one of the following commands:

Get-message | format-list

Get-message | select Identity, MessageSourceName, Subject, LastError | where {$_.MessageSourceName -eq "DSN"}

Once you have the Identity, you need to suspend the message:

Suspend-Message -Identity <YourMessageIdentity>

Export the message using the command:

Export-Message -Identity <YourMessageIdentity> | AssembleMessage -Path "<YourPath>\<Filename>.eml"

You can resume the message for further processing in your queue:

Resume-Message -Identity <YourMessageIdentity>

Now I was able to open the mail contents with notepad and investigate further on the DSN cause. In our case, as I suspected, those were NDRs for non existing mail addresses, nothing to worry about.

Further reading:

DNN site's search issue with index outside bounds of array

2017-07-25T15:39:00.000+03:00

I had an issue with a DotNetNuke 7.4 site's DB size. It had grown way too big and we found out about it because of the server's warnings about low disk..

Both Exceptions and EventLog tables had three new logs (General Exception, Scheduler Event Failure, Scheduler Exception) every 30 seconds saying that the "Index was outside the bounds of the array."

It was obvious that there was an issue with the site's search indexing. Search functionality wasn't working either. Re-indexing the content or restarting the application pool didn't help.

In order to fix this issue i had to do the following:

Stop site's application pool from IIS.
Delete all contents in <path_to_your_DNN>\App_Data\Search\ folder.
Start site's application pool.
Re-index content from the site's Admin -> Search Admin page.

Hopefully this will help someone else too.

For the record, the Stack Trace was the following:

at Lucene.Net.Index.SegmentTermDocs.ReadNoTf(Int32[] docs, Int32[] freqs, Int32 length)
at Lucene.Net.Search.TermScorer.NextDoc()
at Lucene.Net.Search.ConjunctionScorer..ctor(Similarity similarity, Scorer[] scorers)
at Lucene.Net.Search.BooleanScorer2.AnonymousClassConjunctionScorer..ctor(Int32 requiredNrMatchers, BooleanScorer2 enclosingInstance, Similarity defaultSimilarity, IList`1 requiredScorers)
at Lucene.Net.Search.BooleanScorer2.MakeCountingSumScorerSomeReq()
at Lucene.Net.Search.BooleanScorer2..ctor(Similarity similarity, Int32 minNrShouldMatch, List`1 required, List`1 prohibited, List`1 optional)
at Lucene.Net.Search.BooleanQuery.BooleanWeight.Scorer(IndexReader reader, Boolean scoreDocsInOrder, Boolean topScorer)
at Lucene.Net.Index.DocumentsWriter.ApplyDeletes(IndexReader reader, Int32 docIDStart)
at Lucene.Net.Index.DocumentsWriter.ApplyDeletes(SegmentInfos infos)
at Lucene.Net.Index.IndexWriter.ApplyDeletes()
at Lucene.Net.Index.IndexWriter.DoFlushInternal(Boolean flushDocStores, Boolean flushDeletes)
at Lucene.Net.Index.IndexWriter.DoFlush(Boolean flushDocStores, Boolean flushDeletes)
at Lucene.Net.Index.IndexWriter.PrepareCommit(IDictionary`2 commitUserData)
at Lucene.Net.Index.IndexWriter.Commit(IDictionary`2 commitUserData)
at DotNetNuke.Services.Search.Internals.LuceneControllerImpl.Commit()
at DotNetNuke.Services.Search.SearchEngineScheduler.DoWork()

Recover records after accidental cascade delete

2016-02-13T13:33:00.000+02:00

Cascade delete in an sql table relation may sometime lead to serious data losses. Should this occur, you can make use of the log file (LDF) to review the DELETE statements that took place and hopefully manage to recover your lost data.
When you realize that an accidental delete has occurred, stop using the database. If you can, stop the sql server instance and copy the mdf and ldf files of the affected database. In my case, I was called a couple hours later on but no harm was done as there was no backup process and the ldf file was intact.
You will need to parse the ldf entries to detect the deleted entries. I found a blog post which offers a stored procedure to do that, but unfortunately this didn't work for me as I was getting an error on the compatibility level (which was wrong). This is why I ended up using SysTools SQL Log Analyzer which as a demo can show you the log file entries. If you want to export the entries, you will have to buy the product.
This tool works with offline files. This is why I mentioned above to copy the mdf and ldf files. It reads the log and shows the INSERT,UPDATE and DELETE statements on each and every table you have. The beauty of this tool is that is also show the deleted row.
When the analysis of the ldf file finished, I selected all entries in the tree on the left and selected export. Then I used the "Sql server database" option and exported the deleted only rows to a new database which was created by the tool. Note that I also specified a data filter in order to minimize the number of data.
After a couple of hours (depending on the size of lost data) you will end up with a new database that has similar tables to the original one and each table will contain the deleted row.
After that, head to the bulk insert msdn article that mentions how to export and import the data using the bcp command. The process is fairly simple as you have to issue the following commands for each recovered table in the newly created RecoveredDatabase:

bcp RecoveredDatabase.dbo.MyTable out MyTable-n.Dat -n -T bcp RecoveredDatabase.dbo.MyTable format nul -n -x -f MyTable-f-n-x.Xml -T

This can be easily scripted into a bat file using Excel. You just have to copy the table names from the "Object Explorer Details" window while having selected the Tables folder in the "Object Explorer" of the Sql server management studio. Then do some string manipulation and you will be able to get the complete list of commands in no time.
Having exported your data, you will want to bulk insert them using the identity insert option described in the msdn article above. To do that, you will have to copy the exported files in C:\RecoveredData\ folder on the machine where you have the original database attached (hopefully you have already started the instance if you did close it down as I mentioned above) and then you will have to issue the following command for each exported table:

bcp OriginalDatabase.dbo.MyTable in C:\RecoveredData\MyTable-n.Dat -f C:\RecoveredData\MyTable-f-n-x.Xml -E -T

Using the same excel file mentioned above, ordering the tables in a way that parent tables get to be inserted first and then the dependent child tables, you will be able to generate the complete list of bcp insert commands.
Hopefully you are as lucky as my friend was and you will be able to restore all your records and then have some time to reflect on your backup policy and perhaps whether you do need the Cascade delete that caused all this.

Move Azure blobs between containers

2015-10-17T11:49:00.001+03:00

I finally got some time to tidy up my Azure account. The first thing I wanted to do was to bring all my vms in a single storage, gathering them from various subscriptions. In order to do that I had to copy the vhds (located in the vhds container of each storage account) to their final destination.

This is how I ended up creating the following simple program. It schedules the transfer of the blob and then monitors the status of the blobs in order to get an indication of when the process is done.
If you are more into powershell, you can follow this tutorial which also mentions how to mark the moved vhd as a VM image (Virtual Machines->Disks->Create).

Botsikas' Blog

Lessons from running K3s in RPI using Ansible and Flux

Ansible gets the first cluster running

Sealing credentials for Git

The fight over not managed secrets

A slow Helm install needs a longer timeout

The CRD has to exist before the custom resource

Back up the key that does the decrypting

Outro

Running Pi‑hole on k3s and how GenAI tried to make me run as root

The bad advice: root and the MetalLB detour

The simple, correct fix with k3s

Final thoughts

Resurrecting a Samsung Galaxy Tab E SM‑T560

The plan and the tools

What actually happened

Windows driver snag

Upgrading asp.net 6 to 10

Repairing a TVIX Remote Control aka Fixing Unresponsive Buttons

Using an indoor air conditioner with a broken outdoor unit

How to fix Snip & Sketch not working on Windows 10 (Snipping Tool shortcut issues)

Solution: restarting the Explorer process

Other solutions to try

Error details in Event Viewer

Parse Excel XML using power query in excel

LogonCommand doesn't execute in windows sandbox

Running Ubiquiti UniFi Network Server on Windows 11

Disabling critical battery action

Installing Windows 11 in QEMU on Supported Hardware

Copy contacts from outlook to gmail

Dual boot two windows operating systems

Convert VHS mpg files to mp4 to import in DaVinci Resolve

Genomics in Azure

Network link is disconnected

Remove unknown locale qaa-Latn from windows

Set proxy for command line in windows

Download new chromium based edge via powershell

Fixing an old Wordpress plugin to run on PHP 7

Exchange view DSN mail contents from queue

DNN site's search issue with index outside bounds of array

Recover records after accidental cascade delete

Move Azure blobs between containers