| Exit | the | Fast | Lane |

New PC Build for 2019

noreply@blogger.com (Peter) — Thu, 01 Aug 2019 03:33:00 +0000

Well 2 years have passed so the time has come again for a PC upgrade! My goals for this year are simple: build smaller and break the 2-year cycle. :-) Certainly from a CPU/ memory perspective there is diminishing value upgrading this often. The biggest changes in the consumer PC space right now are around storage and graphics, which should last 4-6 years before driving the next upgrade I estimate. CPU performance just isn't that dramatically different from prior generations to necessitate upgrades this often, never mind the fun factor of doing a new build of course.
This was a tough year to do a build I felt, which is always complicated a bit by building Small Form Factor (SFF), but part selection generally seemed more difficult than previously. It's easy to build a maximum high-end PC with parts clearly designated to that category, but value for money is tough to justify unless you really need the horsepower. I'll explain more below in the part selection sections below.

Check out a few of my previous builds: 2017, 2015, 2013

This year's final build, read on to see what's inside!

Motherboard - ASRock Phantom ITX

I typically build mATX but had to do an honest assessment if that was still necessary now. In prior builds, DIMM sizes dictated 4 slots to hit 32GB affordably, plus I wanted another PCIe slot should I decide to run SLI. The truth is, I will never run SLI, I just won't and now 16GB DIMMs are much more affordable so I can get by with 2 to net my desired 32GB. My need to build mATX just isn't there anymore so I decided to look at mini-ITX. While Asus is my usual mobo brand of choice I found their ITX space not only too expensive with low supply but also several reliability concerns reported in the community on the Z390 parts. The one model that kept coming up consistently as high-quality and reliable was the Phantom Gaming-ITX/ac by ASRock. Particularly well-reviewed on this board is the power design which includes high quality capacitors and power chokes. Overclocking, dedicated AIO liquid cooler pump header, dual M.2 slots with a full heat spreader on the front, Thunderbolt 3 USB-C, plus 802.11ac if you need it. This board includes everything I need, in a very small footprint.

A Note on Chipsets

The latest high-end Intel chipset offering is the Z390 which only provides minimal improvement from the previous Z370. Unless you very specifically need one of these new features, the Z370 will suit you fine and will be cheaper. If the price is close between variants, get the Z390 so you're future-proofed.

The key changes on the Z390 vs the Z370 are:

Gen2 USB 3.1 support
ME 12 firmware
Native 802.11ac + Bluetooth 5
Wireless-AC adapter

Here are the block diagrams of both chipsets with the changes outlined in red on the Z390.

CPU - i7-9700K

Really tough choice picking a CPU this year! If you go by performance alone, the i9 reins supreme, but barely, compared to the 9700K and in some tests only by 1-2 tenths. Anand Tech has a good CPU comparison worth the time investment. I struggled between the 8th gen i7-8700K which has excellent OC potential performing within tenths of the 9th gen parts, the 9700K which has 8 cores (C) but no hyperthreading (HT) and the top dog: the i9 with 8C + HT. HT is now a premium feature on the 9th gen parts, only available on the i9. The 9700K gets 2 additional cores bringing its total to 8 vs 6 while HT is present on the 8th gen 8700K. Clocks between parts are within a few hundred MHz of each other with the blistering stock 5GHz mantle going to the i9 part alone.

One of the most interesting changes on the 9th gen parts is that the heat spreader (lid) is now soldered to the silicon providing superior thermal transfer. 8700K owners serious about OC will delid the part to achieve better cooling. This is much more difficult on the 9th gen parts (due to the solder) and provides arguably less benefit.

I came very close to just pulling the lever for the i9 but decided against it for a couple of reasons. First is the cost, at $525 at the time of build this is way too high at +$100 over the nearest contender and that's if you can even find the part in stock. The i9 parts run HOT, even at 95W stock, these chips can suck a lot more power and create a lot more heat under load. Liquid cooling is highly recommended if you go this route. 8C HT is nice but practically there are very few of us actually able to thread through all that potential given a standard mix of common apps and games. This presents the same problem for the acclaimed AMD Threadripper: lots of potential but nothing really to take advantage of it. 4C is probably plenty for most, 8C (non HT) is more than enough. There were also supply issues where I found the part difficult to procure so I ultimately decided to save $100 and went with the 9700K, which is still fairly expensive at $420. The 9700K proves to be a very capable OC part running a stable 5.4GHz vs the i9 running too hot under OC at the time of build. So you need to decide if you want to save or spend a hundred bucks between options, need HT or not and if you'll OC or not.

So much of this decision depends how you will use your rig. For me, being completely honest, it's probably 70% office/web stuff + 30% gaming as this is my main machine for all things. If all you intend to do is gaming, then you don't need a top bin part, you just don't. The GPU is much more important in that equation so you can get by with a lower bin CPU and save some scrills.

My gaming only recommendation: If you plan to OC, then get the 9600K, if no OC, then get the 8700 (non K). Yes, it's a generation older, but strongly holds its own in the performance charts vs the new gen parts. Hard to not make this about value for the performance provided. You might be willing to spend $500+ on the i9 part, but it's difficult to justify 2x the cost vs the 9600K given the i9 doesn't net 2x the performance gains in any test I've seen. Put the money saved into a better GPU!

My mixed use recommendation: Regardless of if you intend to OC, the 9700K is a solid option for mixed use cases and performs near the top of all charts right now. You will net solid office app + gaming performance nearly regardless of title.
That said, if you have the money to burn and want the best there is period, regardless of performance to dollar margins, the i9 is it.

i9-9900K: 8C HT, 5GHz, 16MB L3, 95W - $525
i7-9700K: 8C no HT, 4.9GHz, 12MB L3, 95W - $420
i7-8700K: 6C HT, 4.7GHz, 12MB L3, 95W - $380
i7-8700: 6C HT, 4.6GHz, 12MB L3, 65W - $310
i5-9600K: 6C no HT, 4.6GHz, 9MB L3, 95W - $270

CPU Cooler - Corsair H115i RGB Platinum 280mm AIO

While I don't need liquid cooling for my setup per se, space is limited in the Cerberus (keep reading) so an All-In-One (AIO) cooler becomes a great option to save space, maximize cooling and keep things quiet. It also looks awesome especially if you make use of the RGB options. This will provide more than enough cooling to OC should I decide to do that later. 280mm is the largest radiator I can fit into this case which will net plenty of cooling. Building a custom loop is of course another option but adds a ton of cost and for what I'm doing, I don't think it's necessary.

I also considered the NZXT X62 which has a great looking pump with the infinity mirror but there are a lot of concerns online with their CAM software and your data which they collect and upload into the cloud with no way to opt out. I decided to stay away for this reason. For pure cooling performance, you really cant go wrong either way here, both are excellent options.

When installing the 115i, once your put the Intel bracket on the back of the mobo + standoffs on the front, there is a bit of play once finger tightened. This goes away completely once the pump is installed and tightened down so don't fret. Do yourself a favor and connect the micro-USB to the pump BEFORE you install onto the mobo!! Clearance to the next door heatsink may be tight and require you to remove the pump to attach the USB, requiring redo of your thermal paste.

Memory - Corsair Vengeance RGB 2 x 16GB DDR4 @ 3200MHZ

Memory can be a touchy subject with people loyal to certain brands due to perception or experience. I've had great success with Corsair, G.Skill and Crucial across many builds over several decades. I do have to say though that the only problems I've had have been with Corsair DIMMS for some reason. One of the largest money saving vectors right now is deciding whether or not you want RGB (technically no one needs it, right?). If you don't, you can save some coin. I decided I did, so I splurged for the pricier modules. High clock speeds remain a marketing gimmick designed to bilk you of your hard earned cash. RAM modules are sold with a base clock but provide differing XMP profiles to boost them to advertised speeds in the BIOS. Depending on your motherboard this may or may not even be possible. From my research I've concluded that there is very little benefit in buying modules over 3000MHz. I bought the 3200MHz modules because they were the same price as the 3000MHz counterparts. There are several good articles written on this topic showing negligible performance difference between low to high clocked modules. If you find a good deal on a higher clocked part, go for it, otherwise, save your money. Another important consideration if you plan to build ITX is the width of the heat spreaders on the DIMMs. My G.Skill modules from 2 years ago are too wide to fit in my ASRock board while the new Corsair modules are slender and fit just fine.

GPU - Asus Strix 2080 Ti OC

GPU is another fun choice this year with lots of great options in both the NVIDIA and AMD camps. Coming from a 1070 in my previous build, I could have gone a number of directions: 1080 Ti. 2070, 2080, 2080 Ti or the higher end Radeon boards. But then you need to decide: Founders or AIB, 1, 2, 3 fans or liquid and which model to pick within a category. I found that AIB (Add In Board) partners make it deliberately difficult to compare cards, within the SAME BRAND!! Although this appears to have gotten a bit better, you will have to do some manual comparisons.

Founders cards are from NVIDIA directly and are for people who need the new toy on day 1 or prefer the "pure" experience, but these cards tend to be limited from a performance and cooling perspective. AIB suppliers like Asus, Zotac, MSI etc buy the GPU chips from NVIDIA then build their own products, often times based closely to the reference designs but charge a premium over Founders. The big difference with AIB parts is that clocks are typically higher, power and cooling designs are better and you can get options like more fans and RGB. I nearly bought the 2080 part but decided against it due to reviews, price points and performance so instead splurged and got the 2080 Ti. Zotac makes an excellent card as does Asus who have a really good VRM design in the 20 series.

The 2080 TI is an absolute beast of a GPU, 4352 CUDA cores and 11GB GDDR6 RAM. Early on, the AIBs used Micron memory chips which had a high number of failures so have since switched to Samsung as the supplier. While all 2080 TI's will have the same number of cores and RAM, the boost clocks and power demands will be different. All of these cards will have a base clock of 1350MHz but can boost up to different frequencies. Going from a boost clock of 1560MHz to 1665MHZ may not seem like much but keep in mind that this is a net of 105MHz across 4352 cores! That's a BIG difference, so higher clocks = higher cost. What do you really need here? Depends of course, get the highest clock for the best price you can find. If you can get an OC card within a few dollars of a base model, go for it. There are many good GPU performance reviews plus SFF builds on youtube. Considering I plan to keep this GPU for at least 6 years and don't want to mess with SLI, I felt it worth the additional investment to get the 2080 TI.

For SFF builders, card length and slot demands are the most important considerations. The Cerberus can support a max GPU length of 320mm which rules out a few GPUs if you want a 3-fan design. Most of these cards also require 2.7 slots, which really means 3. This Asus card measures 304mm which is just fine for my build. You also need to consider things like case fans and radiators, if you do a vertical mount radiator in the front of the case, for example, this will detract from usable GPU length.

A couple of build notes... This card is big and HEAVY, even with the reinforced PCIe slot on the mobo it sagged a bit, so I put a rubber stopper on the far end on top of the corner of the radiator to prop up the card to level. Probably not absolutely necessary but this bugged me so I fixed it. Experienced builders know this, but make sure to run 2 discrete power rails to your GPU, don't use a single rail with daisy-chained 8-pin connectors. Give the card the power it needs with no restrictions!

PSU - Corsair SF750

The one singular part in your build that will most likely drive PSU size will be the GPU. For the 2080 TI the recommended output is 650W. I chose to go a step higher to add a bit of buffer at 750W. The Corsair SF line is particularly good and well reviewed everywhere I looked. Incredibly small, incredibly powerful, clean, modular and the cables even come individually sleeved. This unit will set you back a few bucks, but it is best in class without question. Not much else to say here.

Media - Samsung NVMe + SANDisk SSD

The 960 Pro and EVO from my previous build still have plenty of life in them so I'm reusing them here. 512GB 960 Pro for the OS, 500GB 960 EVO for apps and a 2TB SANDisk SATA III for data storage, which is new for this build. Because I'm mounting my radiator to the bottom of this case, I don't have any room for a 3.5" HDD. But, I can mount 2 x SSDs to the back panel of the case and 2TB is really fine for my purposes, plus I can add another later if need be. Also, 4TB parts are on the horizon so that will be an option as well. There is next to no performance difference between SSDs anymore, so pick a reputable brand, choose the size you need and you'll be good. NVMe is definitely the way to go for the OS or if you just want to install a single module. A 1TB 970 EVO Plus is less than $250 right now so very affordable for top of the line performance and large capacity. There are cheaper options as well. Look for drive endurance which will be reported as a total TB written value and warranty. For reference, in 2 years I wrote 9TB on my 960 Pro housing the OS and the drive is capable of 400TBW. Lots of life left.
One thing to keep in mind when using NVMe is that if you populate the secondary rear slot (on the back of the mobo), it will disable a SATA port on the motherboard. On the Phantom ITX SATA3_1 is disabled but since I need only 1 SATA port for the SSD, not a big deal.

Case - Sliger Cerberus

I spent a LOT of time pouring over case options, comparing dimensions, supported mobo and GPU form factors, features and cooling. So I decided to build ITX which opened a lot of very small case options previously unavailable. The biggest downside to building SFF is cost, cases will be of higher quality materials, but since the sales volume is lower they cost a lot more. The other thing to consider is whether you really need 3.5" HDDs or if you can get by with 2.5" SSDs or simply NVMe M.2s. This is dictated partially by the cooling solution you choose, air cooling will net more options since you won't have a radiator consuming space.

My top contenders were:

Ncase M1 by SFFLab - sub 13L volume, 297mm - 317mm GPU length with vertical mount option, 240mm radiator support for liquid cooling - Starting at $195 but stock seems to be a constant problem.

Ghost S1 MkII by Louqe - Sub 20L volume with "top hat" options to increase capacity for cooling, 305mm 2-slot GPU length - Starting at €269 with varied stock. I decided I didn't like the top hat look plus others reported gaps on the edges. Pass.

Cerberus by Sliger - sub 20L volume supporting ITX or mATX, 330mm GPU length, 280mm radiator or 148mm air coolers, plus a ton of customization options - Starting at $225.

So I ultimately chose the Cerberus with liquid cooling and am very happy despite the $320 price tag as configured. I chose white with a black vented top, left side window, vented panel on the right side, a top mount bracket and aluminum feet. Build quality is excellent in this completely tool-less case that sports easy pop-off panels. The order in which you install things matter in SFF but I really enjoyed building in this case.

Displays - Alienware AW3418DW + Dell S2716DGR

If building a gaming rig, don't overlook one of the most important elements: the monitor. This is like having a high performance car and putting crappy tires on it. There is a ton of info on the web about frame rates vs resolution, so if you're intent on a 4K monitor, consider that this will likely be rendered at 60 FPS (unless a higher frequency option was released following the posting of this article). Linus Tech Tips did a good piece on this with a blind Pepsi challenge to see what gamers on his staff naturally preferred. Frame rate won the day, even at a lower resolution. This is why I bought the 120Hz (overclocked), curved 34-inch, 1440P (3440 x 1440) Alienware monitor with an IPS panel. The monitor I really wanted doesn't exist: 34-inch curved, G-Sync, 1440P @ 144Hz. 4K monitors will get there, but at the time of this build, 60 FPS was the best you could get. G-sync capable monitors will run more expensive but is a premium feature I feel worthwhile as an NVIDIA gamer. Typically this is an expensive monitor but there are deals to be had, plus Dell will price match and ship free if you buy direct. Widescreen gaming is an amazing experience and now that I've seen the light, I can never go back. I was worried based on the marketing photos that this monitor might be too cartoony or in your face "commercial gamer style" in the way that only Alienware can do it. Happy to report that the monitor looks very tasteful in person and feels of high quality. There are some subtle RGB touches built in but these can be customized or disabled completely if you prefer. Facing front, these lights are completely indistinguishable.
My secondary monitor is also G-Sync capable, technically a 1440P (2560 x 1440) gaming monitor but at 27-inch it compliments the Alienware beautifully. An unnecessary splurge in this use case perhaps, but I like it.

There are a couple of configuration steps required to get the desired frame rate:

Turn on OC on the monitor, set at 120
In the NVIDIA control panel make sure G-Sync is enabled

In the NVIDIA Control panel turn Vertical sync On (good article about these settings)

From Settings, click display, choose the target monitor, scroll to the bottom and select Advanced Display Properties, on next page choose Display adapter properties at the bottom

Select Monitor tab and set refresh rate at 120 (see image below).
If you fail to do this, you will NOT see 120 FPS!
If you have multiple G-Sync monitors connected, you need to do this on ALL or flickering will occur.

In game set the resolution, frame limit of 120 and DISABLE V-sync (if you have G-Sync enabled)
Some games, like Borderlands 2, are very sensitive to the polling rate of the mouse. If you get stuttering or frame drops while looking around, try lowering your polling rate.

This applies to non-gaming life in Windows too, especially if you have a Steel Series mouse! Drop the poll rate to the minimum!

Custom Cables - CableMod

This step is completely optional, of course, but since I've gone to the trouble of installing a view window with premium parts in a premium case, I want it to look pretty as well. The cables that the SF750 ships with are very nice, individually sleeved too, but black and way too long for SFF. The cable bundle was just too large and didn't look good. There are a number of custom cable makers out there, some specializing in SFF (PSlate Customs), but many get overloaded with orders and take long breaks before opening a new order window. CableMod is a subsidiary of the Chinese company TL Sourcing LTD but deliver high quality custom cables with no ordering black outs, so I went with them. I opted to do a tuxedo scheme so everything is white except for the black panel on the top of my case. I think it looks amazing. These cables are pricey and will take a few weeks to build and ship from China.

I bought 3 custom cables sleeved in durable nylon ModMesh in the following configurations that worked out really well (the ATX cable is tailored per your particular PSU):

24-pin ATX, 300mm with aluminum pro combs
Bridged 8+8-pin PCIe, 250mm with aluminum pro combs
Triple SATA power, 100mm spaced between each connector

Build Notes

I had a very good experience building in this case, but like all things SFF, the order in which you do things matters. My build order went like this: SSD, PSU, mobo + CPU + NVMEs (pre-installed), radiator, power rails, CPU cooling pump + power connectors, GPU, RAM, then case fans. That fan on top just barely clears the memory release clips but this is why I installed it last. This method worked well for me although the GPU has to be rotated diagonally a bit to clear the radiator tubes and anything plugged into the PSU. Connecting the power cables to the PSU last is certainly an option, but a pretty tight squeeze to get your hand in for the plugs on the bottom. You could also let the PSU dangle outside of the case and bolt it up last.

The red DIMMs shown in the pictures below were my old TridentZ modules as I was waiting for my Corsair RGBs. As I said before, the width of the heat spreaders matters a LOT on this board as the DIMM slots are very close. The Corsairs are very skinny and slot perfectly on this board.

If you plan to use an AIO liquid cooler with the radiator mounted at the bottom, the outer edge bolts will interfere with the case feet. I just left these off using only the center bolts which works fine, especially considering gravity's role here, I'm not worried. If you use the Demci filter on the bottom it will have to run over the bolts, which works, but it's not a completely clean seal.

Filters are optional and sold separately, of course, but you should note that this case in stock form has zero air filtration. Unless you're good with regularly blowing dust out of your case, $65 extra for pre-cut fitted Cerberus-specific filters is a sound investment, albeit expensive. There are a few non-magnetic panels on this case so you have to use the included sticky magnet brackets to give the filters something to attach to. It works fine.

The power cables shown below were also temporary while I waited for my custom CableMod order.

Benchmarks

Passmark9 shows some improvements in CPU and GPU as expected but still some strange comparisons when looking at global results. Why did the 9900K in this baseline score so low? Also curious that identically spec'd 32GB memory modules performed better. Take these for what they're worth.

Cinebench R20 definitely pushed the CPU harder by indication of peak CPU temps under load, but this is clearly a bench generally favoring high core counts as indicated by the top 3 scoring parts.

Final Thoughts

I've very happy with this powerful and nearly silently build that serves across a wide range of use cases. If you plan to use RGB-capable parts, consider how many different brands you are putting into your build. Each brand has it's own software load, so ideally you want to run as few of these as possible. I tried to keep everything Corsair, except for the GPU of course, which I ended up disabling RGB entirely via the on-board toggle.

Gaming performance is excellent as is web and office work. The 2080 Ti is more than capable of running current titles at ultra settings stable at 120 FPS linked lock-step to my G-Sync Alienware monitor. This is the biggest upgrade from my last build, no doubt. CPU and RAM really didn't make too big of a difference and my NVMe performance is the same as before: awesome. What has changed a bit are the thermals in this new build with the CPU running very cool nearly at all times.

Idle temps are very good with the CPU clocking a chilly 34C, as expected, thanks to liquid cooling. The mobo runs a bit hotter at all times:

Under load things get a bit warmer but more so on the mobo itself. The GPU definitely generates heat but low 70's C is the highest I've seen it go and this will vary based on the title in play.

Costs

i7-9700K - $410
Asrock Phantom ITX - $180
Corsair RAM - $260
Corsair AIO cooler - $170
Cerberus case - $320
Alienware monitor - $800
Asus 2080 Ti - $1280
Corsair PSU - $195
2TB SSD - $250
Demci Filters - $65
CableMod cables - $170
Case fans - $45
Total - $4,145

Failed Storage Spaces Disk Recovery

noreply@blogger.com (Peter) — Tue, 05 Feb 2019 06:09:00 +0000

This wasn't a fun one to write considering the circumstances. Data protection tech is great when everything is set up fresh and working properly. Everything is just fine, until it isn't and Storage Spaces failed me here. This post applies to the Windows 10 variation specifically which shares some of the same general principles with larger enterprise deployments in Server 2016/ 2019.

So, what happened?

My setup:

Asus Mobo, Intel Core i7-4790K, 32GB RAM
Windows 10 Enterprise, 1809
120GB SSD (Boot)
3 x WD Red 3TB HDDs

1 storage pool: 8TB
2 x mirrored vDisks

D:\ (Docs) - 2TB
M:\ (Media) - 2TB

The 3 x 3TB WD Red HDDs in my HTPC are going on 5 years old and were running fine, literally until I placed the order for new drives to replace proactively. Murphy. On the day I receive the drives, I see resiliency warnings in Spaces on one HDD and both vDisks. I also see in the System logs: "The device, \Device\Harddisk2\DR2, has a bad block." Fun. Below is the what Spaces told me which seems to lead to a pretty obvious next step right? Remove the drive labeled "Old2" from the pool then replace it. Right? WRONG.

The other bit of good news I uncovered was that Windows no longer recognized my Media volume, which instead of reporting ReFS was marked RAW. At this point my M drive was basically dead and I was unable to browse in File Explorer.

Here's where things go from bad to worse and in hindsight I can confidently say, DO NOT TRUST THE SPACES UI! At this stage I'm still hopeful that the repair process will remedy this. I click remove on the Old2 HDD which according to the UI, the process failed.

What I didn't know yet what that this drive marked as warning by the UI was not actually the drive having problems. Even better, this drive that supposedly "failed" to be removed from the pool was actually marked as retired which triggered repair and regeneration jobs for each vDisk. The UI later caught up and flagged Old2 as "preparing for removal. So, I dutifully complied, shut the box down and added a fresh WD Red HDD. Below is the Spaces UI before I added the new drive. Something to note here, if you interrupt these repair jobs via reboots, they will start over at 0%.

What I also didn't fully realize at this point was how Spaces actually behaves when you elect to voluntarily remove a drive from a pool. To see what's really going on, you have to use PowerShell, which is what I should have done from the start and ignored the UI entirely. The command Get-StorageJob will give you insight into the repair process. As you can see below I now have 4 jobs running, 2 per vDisk.
What is essentially happening here is that the disk I elected to remove from the pool is being indexed of all files and slowly drained from the pool as the files it hosts are being regenerated somewhere else in the pool from the 2nd copy in the mirror. Because I have 3 disks in my pool, no single disk has a complete copy of all files in a mirror set for a particular vDisk. This disk I'm removing has 633GB of data from the Media vDisk and 197GB of data from the Docs vDisk. There also appears to be no way to stop or cancel this job once it has begun.

Now we wait.

My WD Reds are 5400RPM SATA drives, so the slowest of the slow. With 197GB marked for recovery on my first vDisk, my repair operation averaged 2GB processed per hour. Molasses. The net here is that for Spaces to recover 197GB on a 2TB mirror space took ~43 hours. Ugh. Patience is very important as this is not a speedy process. The 633GB for the Media vDisk took even longer. What complicated the recovery process was that I was removing a perfectly good drive from the pool, because the UI, leaving the problem drive having to work with bad blocks to recover data. How long would this have taken if the bad drive was removed instead of a perfectly good one? No idea.

CrystalDiskInfo was useful here to shed some light on which drive was actually bad and why by reporting SMART data. There are a few interesting things happening in the image below that I want to call your attention to.

First, the Spaces UI would periodically change from a Blue information banner to the red Error banner you see below flagging my M vDisk as having no resiliency. This error would clear on its own after a few minutes. Checking in PowerShell shows everything as "OK" but the disk Crystal flagged as bad has high read errors and pending sector counts. So far this jives with was is being reported in the System event log.

Windows originally flagged a warning on the HDD listed on the bottom "Old2". There is no clear indication why although I have a suspicion a bit further down. Checking in Crystal, Old2 reads as good, echoed by PowerShell. Notice that Old2 is marked at "Retired" in PowerShell. The drive that actually has a problem, Old1, shows OK in the Spaces UI + PowerShell, but reporting bad blocks in the event log. Why doesn't Spaces see this drive has a problem?!

The other interesting thing here is that although my new HDD, serial ending 68E3, is contributing capacity to the pool, you'll notice that no data is actually being stored on it according to the UI. This is after 8 days of data recovery!! I would have thought Spaces would make use of all storage resources in the pool when performing a repair/ regeneration activity. Apparently not! Guessing the process rebuilds using existing media then I'll have to manually run an "optimize drive usage" job to re-balance the pool when the repair is complete. Geez, wtf.

As the HDD Old2 was very slowly being prepared to be removed, the % used as reported continuously dropped as the bytes were processed.

While I waited and since there is no way to stop the jobs running, I dug more into the physical infrastructure of the pool. Get-PhysicalDisk will show the specifics of all physical media in the system, including the device ID. If you recall back to my System log entry, disk 2 was reporting a bad block which coincides with what Crystal is saying.
Piping Get-PhysicalDisk to Get-VirtualDisk will also show some useful information as you can see what exactly each disk is doing in regards to each vDisk. I did this below for each disk respectively and you can see that the Media vDisk is flagged as Degraded on Old1. Because no single disk, has a full copy of any mirror data set, disks Old2 and Old3 are flagged as no redundancy, as at this point there is none. This output also gives a potential clue to as to why Old2 was flagged by Spaces to begin with. Why does Old2 not contribute to the Docs vDisk? Could this be the cause of the original warning? Highly suspect that this was happening at the same time Old1 is reporting a bad block in the event log.

Unfortunately the majority of Spaces-related event logs are near useless and there doesn't appear to be a way to enable verbose logging for troubleshooting purposes. Four separate Spaces-related event logs filled with useful detail like this:

So how did it all end? In failure. Now to be fair the Docs vDisk was recovered successfully but I never lost the file system on this volume either. I was so excited, after 14+ long days I got so close to the end. .03% to be drained from Old2 but it never finished. It got into this weird loop of small 256MB and 512MB repair jobs that just didn't seem to complete, over and over. I even let it sit for a full 24 hours just to see if it would work itself out. Nope. So here I am, a Spaces recovery job that won't finish and a RAW volume that I can't browse.

Great, what now?

Luckily I have a good backup, so as much as I hate that its come to this, I'll be rebuilding and restoring from backup. If you aren't as fortunate, don't fret, you have options. I looked at a few different tools that are capable of restoring from ReFS RAW disasters. The way these tools work is that they offer a free demo build that will allow you to scan your disks to identify what exactly can be recovered. If you actually want to restore real files then you have to pay.

ReclaiMe File Recovery is a tool that specifically calls out support for ReFS as well as NAS systems. There is another variant I found on Google specifically for Storage Spaces data recovery but the logos while similar are different, the docs are old and aside from a link to ReclaiMe on the page, I'm not completely sure it's the same company. File Recovery should be all you need here anyway as you are recovering whatever you can from bare drives. If you get to the point where you just need to save SOMETHING, for $79 ReclaiMe might be able to help.
EaseUS Data Recovery is another option and a name you might have seen before. They boast ReFS support as well as RAW partition recovery. $70 gets you an unlimited data recovery license for a single PC.
R-Studio I had never heard of before but they offer a really impressive list of capabilities and supported file/operating systems. If you search there are some people in the forums reporting using this tool and having successful recoveries. $60 gets you a R-Studio NTFS license which also supports ReFS.

If your drive has bad blocks you may have data loss. If you're recovering from a 2-way mirror in Storage Spaces, one of your drives from the pool should have the files you hope to recover.

Lessons Learned

First and foremost, I no longer trust ReFS. Search Google for "ReFS RAW" and look at how many people have experienced lost file systems turning RAW on their ReFS volumes. Completely unacceptable. If you value your data, stick to NTFS.

The Storage Spaces UI in Windows 10 is super buggy at best and completely useless at worst. Check in PowerShell and a third-party SMART tool like Crystal to see what's really going on before taking any action. Don't believe the UI if it flags a particular drive as having problems!

Always, always, ALWAYS add a new disk to you Spaces pool first, then re-balance before you elect to remove anything! Would that have saved me here? Considering that I got screwed by ReFS leaving my Media vDisk in a Raw state, I doubt it.

Initially, this experience had me rethinking my 3-disk mirror setup. At the end of the day what I have is 1 column with 2 data copies spread across 3 disks. I've proven previously that performance doesn't change unless you're running 2 columns, which requires 4 disks minimum, so that 3rd disk really only contributes storage here. I can still only tolerate a single disk failure but what I've done is spread my failure domain to a slightly larger surface. None of my 3 drives should ever have a full mirror replica of any vDisk volume. So in the event of a failure, what should happen is that I have less data to rebuild once the failed drive is replaced, thus speeding my recovery time. I'll stick with my 3-disk pool for now but will rebuild using NTFS. If I get burned again...

RAID and software-defined data protection are not means to replace backups! You still need to backup your data or roll the dice with the recovery tools should a disaster strike.

Jobsite Table Saw Router Extension

noreply@blogger.com (Peter) — Thu, 24 Jan 2019 03:08:00 +0000

The same basic principles outlined here can be applied to a cabinet saw as well, but this post focuses on the portable jobsite variety. That tenet, portability, is largely why a table saw router insert makes so much sense here. I really don’t want to take up even more precious space with a dedicated router table, plus this way I can easily move and deploy two important tool stations with one action. Ah, the life of the driveway wood worker!
My table saw and router are both Bosch models which happen to be best-in-class contenders for these particular tools. Just about everything else in my tool chest is Dewalt but these Bosch models were very deliberate purchases based on overwhelmingly positive community feedback. I can personally attest, the hype is real.

Core Components

The jobsite saw I’ll be modifying here is the Bosch 4100-10 (10” Gravity-Rise Worksite Table Saw). My plan is to remove the right side extension table and replace with a custom router insert: Link

The only non-standard feature I added to this saw initially is the Bosch Outfeed Support Assembly which doesn’t interfere at all with my router extension insert. There is also a left side outfeed assembly which I may add at some point later: Link. My general feedback on the outfeed quality and functionality is very good, although you waste a bit of the shipped components since Bosch packs parts for two different saws: Link

My router is the Bosch 1617EVS which is a variable speed model generating 2.25 HP @ 12 amps. I purchased as a kit that includes both a plunge and a fixed base, along with collets for both 1/4” and 1/2” bits. The fixed base is what I used to attach the router to the insert: Link

Here is the end result, want to see how I built it? Keep reading:

Parts List

3/4” MDF half sheet: Link
Rockler Pro Plate (Model A): Link
Amana 5/8” straight cut bit: Link
PowerTec T-Track (2 x 24”): Link
PowerTec 5/16" T-Bolt kit: Link
4 x 5/16" zinc carriage bolts + washers
2 x 5/16" 3-prong T nuts (for plate)
12 x stainless flat head screws
1 x 1/8" x 2" steel bar
Minwax fast-drying oil-based poly: Link
Minwax Past Wax: Link
3M Rugged Comfort Quick Latch Respirator 6503QL: Link
3M 2097 P100 Particulate Filter Pair: Link
Titebond III wood glue: Link
Dasco Pro Cold Chisel kit: Link

Cuts and Prep

Why MDF you may be asking? A couple of reasons, first it's relatively cheap, it's super easy to work with having no grain pattern as it can be cut in any direction, it glues up strong very nicely without need for fasteners and lastly it's dead flat which is important for this particular use case. The image below shows the 5 parts for the fence which will attach to the stock Bosch table fence directly. Plus the triangular supports and the support plate to round out the cut list. The general idea here was to create an insert that sits between the table slide rails and a fence slightly larger than the width of the table. This is 100% custom and can be applied to any other similar jobsite saw.

One quick note on working with MDF. Unless you have a stellar air filtration system you really need to wear a respirator. The MDF fiber particles are extremely fine, much like powder and will get everywhere, especially into your nose and lungs. I linked the 3M gear I use above which works incredibly well for me on all woodworking projects.

The cut image for the router plate got corrupted so sorry that's missing, but you'll be able to see in later photos what I did there. The length of the piece is from table rail to rail and I made it wide enough to house the plate, a handle cutout and a T-track rail.I used 6 support pieces on the underside which I'll show a bit later.

Router Plate Insert

I chose the Rockler Pro Plate as its aluminum which is a strength upgrade from many of the molded phenolic you will find for sale. I read several anecdotal accounts that these molded plates will sag and warp over time due to the weight of the router. No thanks, do it right with a metal plate!

Once the basic shape of the insert is cut, I created a simple template to cut the indention for the Pro Plate. Find the location on the board you want to the plate to sit, then surround with identical height scrap, hold down with double-sided tape. You want these pieces tight as possible to the edges of the plate. In hindsight using MDF around all 4 edges would have been better but what I did worked out ok.

Remove the plate and insert a center square piece which will be used by the router to rest on as it completes its task. Some people leave only a very tiny lip around the edge when doing these. A bigger lip equals more strength but also you really only need enough clearance for the router and base itself. I'll explain further just ahead. Set your plunge router at the exact depth of with width of your plate and route out the lip. Once the lip has been created, remove the center square and cut out the center piece of the board with a jig saw.

For the front T-track slot I used the plunge base with the Bosch Deluxe Router Guide. I HIGHLY recommend this part as a worthy addition to the tool box. The edge the guide glides on needs to be straight and true, if it isn't, your cuts will be mirrored imperfections. For this step, I marked the depth I needed for my rails on the side of the work piece, then plunged the bit real-time to the proper depth.

I needed to do a little flattening of my lip to get the plate to sit perfectly flush. I really don't want to mess with leveling screws in the corners but that's ok if you want to. I found that hand scraping with a chisel was a good way to remove material. Sanding just doesn't work very well with MDF and takes way too long. Check your level by using an MDF scrap sliding from the board surface over the router plate. If the plate is too high the scrap piece will knock on the edges. Keep scraping until perfectly flush.

So far so good.

Next I cut out the handle and pop-up notch to match the original extension wing that I removed from the table. Mark carefully, then cut out with the jig saw. I then softened the edges a bit with the rotary sander. This is similar to factory finish on the metal table edges.

Moving on to the underside... Next, I drilled holes in the side support pieces that will ultimately mate up with the bars to slide into the table rails. I don't have a drill press yet so used a block of maple and walnut with a true 90 junction that I used to drill straight holes. In lieu of a proper drill press this works well.

Here is a look at the underside and how I did the support pieces. 2 for each of the long runs, 1 on each end drilled for the metal bar that will slide into the table rails with a bit of overlapping between the ends and side pieces. I used no fasteners here at all, just glue and clamps which worked extremely well with Titebond 3. One thing to keep in mind here is that MDF is highly absorbent, especially on the ends, so any glue cleanup that you would normally do with a wood glue up won't work as well here. Some of the glue is going to get absorbed and won't wipe up well. I had no issues with adhesion on the ends either. You will read that some people go to careful lengths via multiple glue coats to maximize adhesion when attaching MDF end pieces. I really had no need to do that.

Using my trusty router guide, I cut a clean edge on the 2 sides to match the cuts in the metal table top. Now the tricky part, lining up the bars to attach this thing to the table. With the insert in place and leveled, I slid the bars into the table rails and marked the holes from the underside for drilling. I also marked the bar length for cutting at this step which I did with my jig saw and a metal cutting blade.

With the holes marked I used my "drill press" and a drop of oil for the bit to drill holes. Make sure your bits can be used to drill metal!

Because I intend to attach the bar to the insert using carriage bolts, I need the holes in the bar to be square so they will sit flush. This part is rather crude but it worked. Using a cold chisel, I flared the corners on each side of the holes, then with the bar securely clamped in my vise I knocked each side flat with a 3lb hammer. The best way to do this is to line up the edge you want to remove flat just above the vise jaw. Good work out on this step. Be careful here because the cold chisel will put gouges in your vise!

Ok so I'm finally done making changes to the insert piece and can start on finishing. Using a 50/50 mix of mineral spirits and oil based poly, I saturated the entire piece as much as I could using an old sock, 7 or 8 coats at least. This thinning will cause the poly to soak deep into the fibers adding strength and resiliency. After this dries a few hours I'll add several more coats then finish with a full strength coat a day later. Make sure to do some light sanding between coat sessions, 400 grit will do.

If you have the Bosch 1617EVS like me and want to use the Rockler Pro Plate, one small adjustment needs to be made to the plate. One of the cool things about this router is that the static base includes a height adjustment capability, built in. No need to buy one of the expensive (but awesome) JessEm router lift kits. But, unfortunately, the Pro plate doesn't have a hole for the adjustment key, so you will need to drill one. Luckily this is aluminum which is easy as butter to work with.
Here's what I did to figure out where to drill the adjustment screw hole. I inserted screws into the 3 holes of the static base that will ultimately attach to the Pro plate. I then found a random screw of similar height for the adjustment screw and inserted that into the adjustment hole. Lining up the 3 pre-drilled holes on the plate, I was able to mark where the adjustment hole needed to go. Pencil around the screw head is all that's needed here.

Mark the center of the hole with an auto center punch and drill the hole using a bit of the same diameter as the height adjustment tool that came with the Bosch kit. Aluminum is so soft you don't even need to use oil to drill, but it won't hurt anything if you do. That small lone hole below is mine.

Check for clearance, perfect.

Next I secured the Pro Plate to the insert using my pronged T nuts. Mark the location with the plate in the insert and drill to match the diameter of the nut. Insert from the bottom.

The plate will come with the right size screws in black for the corners. Screwing these in will tighten the pronged T nut from the bottom. I also cut to the T-track to length and screwed down with stainless flat head screws. I cut the track in a way to ensure the screw holes were as close to the edges as possible to secure a tight fit. Amazon Prime boxes make great work surfaces. :-)

With the plate securely in place, I screwed on the slide bars to each end of the insert. You don't want this fully tight yet, slide the insert on to the table first and square the rails.

Now, tighten the bolts of the new insert to the table rails. You can see that my insert deepened in color a bit after the poly coats. I also finished the piece with a coat of paste wax at the very end. What do you think?

Router Fence

On to the router fence, for which the pieces I already cut out. There are a lot of variations on this piece but what I wanted for sure was something adjustable that I could later add a feather board via T-track. The first step is to mock up the assembly and plan the slots that allow the front faces to slide. This needs to be done in conjunction with where you intend to place the supports, since your knobs that hold the front faces will come in contact with the supports. Find 2 spots on the front faces to drill holes, then with bolts inserted, plan how you want the slots to lay out and mark. Here I cut the slots already but hopefully you can see what I mean.

Once the slots are cut and support locations planned, countersink the bolt holes of the front faces so they won't interfere with any future work pieces.

Next I dealt with how to attach the custom router fence to the Bosch saw fence. Unfortunately the side slot on the Bosch fence is not a standard T track slot and I couldn't be bothered to run to the blue/orange store, so I cut two of my T track bolts down to size. Dremel + cut off wheel to the rescue.

With the head squared a bit I was able to fit into the side slot. From here you just need to decide where to punch the holes in the MDF piece that will sit adjacent. I centered my piece and marked for holes near the quarter-mid of each side. 2 bolts is good enough for this.

Due to the height of the slot in the Bosch fence in relation to the table, I really had no choice but to cut a notch to clear the adjustment knobs. I got the smallest I could find so to limit the depth of the notch I'd have to cut. A few bangs of the chisel and presto.

With slots cut and holes drilled I felt good enough for another glue up. Both of the vertical pieces sit on top of the bottom piece which will ultimately sit flat on the saw table. You can also see how I marked the holes to center the bolts from the Bosch fence to my rear piece.

While I glued on the fence face T-track carrier (upper portion of front fence face) I measured and cut supports. I wanted the triangular parts of these as flat as possible for the middle supports just in case I ever want to add a dust collection port.

With the supports cut, time for more glue, more clamps.

I let the glue harden overnight then moved on to another 50/50 poly/mineral spirits soaking. I honestly lost count of how many coats I did, I just kept soaking until my mix was gone, hitting the exposed edges especially hard.

I let this cure overnight again then did one last pure strength poly coat following a light sanding. After one last overnight curing I bolted on the front faces and screwed on the upper T-track.

Here's a look at the "knob chamber" so you can see each in relation to the slots and the supports. It's a little tight but plenty of room to move things around without collision.

There you have it! The project before the "project". Let me know what you think or if I can answer any questions. Thanks for stopping by!

Inspiration

BethePro MDF Router insert
Benchtop Router Table

The Future of Windows Server Management

noreply@blogger.com (Peter) — Fri, 13 Apr 2018 21:33:00 +0000

Say hello to what will very likely be the Server Manager replacement in Windows Server 2019: Windows Admin Center (WAC). Formerly Project Honolulu, WAC has officially gone GA with build 1804. Microsoft clearly has big plans for this tool which has the potential to replace everything and anything currently based on the well aged Microsoft Management Console (MMC). It will probably be the release that follows Server 2019 before we see any possible real deprecation of MMC.

Installation

Because WAC is a web service that manages individual servers, it must be installed on every node you intend to manage, for Server 2016/ 2012R2. Installation is a pain-free affair requiring only permission to modify a particular server’s trusted hosts settings, a designated web port for management and either a 60-day self-signed or other generated SSL cert.

If managing a cluster or larger environment, whichever node you intend to use as a gateway for management access must have visibility to all other nodes in the cluster or environment. Here I’m using S2D1 as my gateway and have manually added all relevant hosts and cluster objects. You can optionally use any Server 2016 or Win10 PC as a gateway to provide management access to all other servers in your environment. From what I can tell, the only thing that truly designates a node as a “gateway” is by adding other objects to it, per below.

The designated gateway itself can then authenticate assigned users or administrators based on local or Azure AD.

Upon installation, WAC will enable a few additional solutions depending on the features installed on your server. For my scenario WAC provided 4 distinct management panes: Server Manager, Computer Mgmt, Failover Cluster Manager and a brand new Hyper-Converged Cluster Manager.

Additional modules and configuration pages will be installable as extensions within WAC. All of the configuration pages in the sections that follow are essentially extensions so the extensibility of this tool is really only limited by the extensions available. Read: this tool has been designed for big things!

Computer Management

A notable change possibly coming here is that Computer Management will be for Win10 PCs only at some point. Right now you can use Server or Computer Management on servers, as neither tool is capable of completely replacing the other. “Computer” in this context will soon mean “PC”.

Server Manager

This is where the majority of new functionality exists currently. To a large extent, this is a combination of the legacy Server Manager tool with a few of the MMC-based tools and a nice coat of webby modern polish. Not all functionality is present here and for many things you will still need to access the MMC tool counterpart. A lot the items available are self explanatory but I’ll showcase a few of the interesting additions below.

Real-time system performance charts are visible on the Overview page with an option to enable disk metrics separately. The warning that follows suggests that these metrics consume a great deal of overhead so to only use when you must and then disable when you are finished. Notice that only my C drive is visible here, no Cluster Shared Volumes in use by S2D running on this same host. This appears to be a Server 2019 feature only.

Some areas are surprisingly good with new concise takes on existing models, such as the Firewall configuration. I can very simply see all incoming/ outgoing rules and easily create a new rule without having to use a series of wizard-driven questionnaires or multi-tab MMC dialogs. This is a fantastic example of what this tool should endeavor to change in future versions of Windows. If more granular manipulation is required, the settings dialog of any rule can be accessed to change things further.

A largely-functional network management stack is a welcomed addition here, no longer requiring ncpa.cpl to alter IP or DNS settings.

Another nice addition here is a fully functional and integrated Shell window. I can do whatever I need to within PowerShell right from WAC.

The registry editor makes an appearance here, no longer requiring regedit.exe to make any needed changes.

The Remote Desktop page provides an embedded frame for direct RDP connection to the server you are connected to. No need to launch a separate RDP client.

The Updates page in WAC appears to be a bit better than the Windows Update tool in the newer Settings UI. Windows Update says I only need a SQL SP2 update:

WAC shows me this same SQL update but a slew of others too, including Intel hardware driver updates. Awesome!

Hyper-V can be managed from WAC as well providing useful event and resource usage data including the ability to manipulate the VMs directly. One thing I don’t like about the current toolset is that when attempting to connect to a VM I am greeted with a popup to save a .rdp file, no direct/ protocol-less console. Hopefully this will get better.

Virtual Machine Inventory shows a summary view of all VMs on this particular host along with real-time resource utilization and clustered status. Important to note that VMs running as clustered on other nodes are not shown here.

The new VM dialog is simple and consolidated enabling quick deployment via minimal hoop jumping.

The full VM settings dialog is also available here for granular settings changes. This may actually be good enough to almost completely deprecate Hyper-V Manager for day-to-day use. Awesome!

Hyper-Converged Cluster Manager

To fully manage S2D clusters with Admin Center you need to be running a preview build of Server 2019, which I don’t have yet. Even though the error dialog below says an April patch is required for Server 2016, per this article, that’s not accurate. This gives a sense of the things the UI will let you do here however. It’s unclear if Admin Center will facilitate S2D setup or not, or if PowerShell will still be required for this in Server 2019.

Failover Cluster Manager

While Hyper-Converged Cluster Manager does not work for Server 2016, Failover Cluster Manager does, to some extent. If you have a S2D cluster some of this will not work, namely the details section on the Disks page. Anything touching S2D directly will likely fail and as a result may require a restart of the WAC service if the UI becomes unresponsive.

The dashboard page shows general status of the clustered components including options to start, stop, remove and simulate failures.

Many of the page tabs are fairly basic at the moment, so WAC doesn’t yet replace any MMC counterpart for full functionality.

Similar to the Virtual Machine page in the new Server Manager, the dialog here should be familiar although now we see which host server is running a particular VM in a clustered state. There is still this split brain problem in Hyper-V clusters where VMs are ‘optionally’ clustered or left local if you want. If you are running a cluster your VMs should be part of the cluster, storage resiliency is only possible if the VM lives on clustered storage. In the split brain model I now have 2 classifications of VMs, clustered or not and still require 2 distinct tools to manage each: Failover Cluster Manager for clustered VMs, Server Manager for non clustered VMs. I doubt this will change any time soon.

Other Items of Note

No love for IE. I got this somewhat hilarious error suggesting that I install Chrome or Edge to use Admin Center on the server itself, considering Server 2016 ships with IE only by default. Most will manage remotely but it would sure be nice to be able to do SOMETHING locally if I needed to without having to install another browser! Just another nail in the coffin. RIP IE.

The need to re-authenticate is definitely excessive, over and over and over again. Every piece part that you connect to, even sub-elements on particular pages sometimes prompted additional challenges for authentication. Even after entering explicitly defined admin credentials for “all connections” I was still prompted continuously to authenticate. In a large environment I would be tearing my hair out. This must be resolved!

Watch this space, managing Windows Server is about to change in a big way!

Minecraft LAN Gaming on Windows10

noreply@blogger.com (Peter) — Sun, 25 Feb 2018 20:12:00 +0000

Everybody loves Minecraft, especially your kids. They can play together easily using split screen on the PS4 but what if your kids are like mine and prefer to play on the PC? They can always find each other on a public server like Hypixel with other people on the internet, but sometimes it’s more fun to play together locally on the private LAN. This guide will provide the steps necessary to safely enable Minecraft LAN gaming without having to disable their firewalls. This has come up several times for me personally so time to document the steps.
So how does this work and what are the problems in native mode? Minecraft LAN mode takes a world from one hosting PC and makes it accessible to other Minecraft clients on the local network. As you have probably discovered, this doesn’t work at all if you just open up a game and start a LAN world. The joining PC you hope to share with just can’t see the world to join, resulting in a screen similar to this:

The problem here is that the hosting PC is successfully sharing the game but the joining PC can’t see it because the inbound traffic is being blocked by the firewall on the joining PC.

Network Profiles

The Windows 10 firewall works by shifting rule sets tied to specific network profiles, Public, Private and Domain. These profiles will take affect and do different things depending on the network you are attached to. By default, Win10 is configured in “shields up mode” with the network adapter set to the “public” firewall profile which is the most restrictive. While this is appropriate for public places or other people’s houses, I recommend changing this to the “private” profile, especially if on your trusted home LAN. This step is optional but makes general consumption of resources on your home network much easier and is relevant to the firewall rules we will enable in the next step. This step can be performed on all PCs on your local network.
First, while logged in with an administrator account, click the Wifi icon in the system tray next to the clock, then click the Properties link below your active connection.

This will open the Win10 settings dialog with more options, click the connection itself beneath the Wi-Fi toggle.

On the next page, click the “Private” radial button to switch the network profile. Important to note that if you access this page while running as a non-administrative user, you will not see this option.

Firewall Rules for Minecraft

Now that we have changed the firewall profile, we need to enable some special firewall rules on each joining PC. Important to note that any PC that wants to be a joining PC will need these rules in place. The key to enabling LAN gaming is by enabling a very specific Java executable through the firewall on the joining PC(s), as the PC hosting the game is initiating an outbound connection which will be allowed out by default. First, find the directory path that houses “javaw.exe”, this will vary depending on the JRE version you have installed but notice that this executable lives within the Minecraft program file space directly. The full path in this example is: C:\program files (x86)\minecraft\runtime\jre-x64\1.8.0_25\bin\javaw.exe.

Next, open the “Windows Defender Firewall with Advanced Security” app either as an administrator or right-click and choose “run as administrator”. Click the “Inbound Rules” element then in the center pane, look for 4 Java(TM)… rules that should have been created when the game was installed. You should see 2 rules for Public and 2 rules for Private profiles. Each rule set contains a rule to allow TCP connections and another for UDP connections solely for Javaw.exe. By default these rules are created and enabled but the connection is specifically blocked, hence the no symbol icon next to each.

To enable the connections, open each rule individually as appropriate for the active profile you have in place. If you followed the steps above, you will need to change the Private profile rules. On the General tab of the rule, select the “Allow the connection” radial.

If you click over to the Programs and Services tab, you will notice that the specific program specified in these rules is javaw.exe. Click OK to save the rule.

Once the rules are properly modified, you should see a green check icon next to each in the Inbound Rules list. This shows that the connection is allowed and that both rules are enabled for the Private profile. If you have the need to enable these rules for the Public profile, repeat these steps.

If you do not see these rules in your list by default, not to worry, we will create a new rule to cover the same actions. Right-click the Inbound Rules element in the left pane and select “New Rule”. Choose “Program” on the rule type page, then browse to Javaw.exe in the directory identified previously for the program path.

Select to allow the connection on the next page, then select the network profiles to which the rule should apply. If you or your kiddo will go to other untrusted networks (friends houses) to play Minecraft, you may want to enable the public profile here as well. Finally, give the new rule a name and click Finish to enable the rule.

Start the LAN Gaming

To begin, from the hosting PC start a Singleplayer game and load the world you wish to share, in this example we’ve selected “#Building World”:

Once the map loads, press ESC to bring up the game menu and select “Open to LAN”. Next, choose the game mode for the session and whether to allow cheats, click “Start LAN World” when ready.

From the joining PC, start Minecraft using the “Multiplayer” option. You should now see any public servers your have configured as well as the game hosted on the LAN. Join the server and PROFIT.

Resources

They Talk too Much!

Resource Sharing in Server2016 RDSH

noreply@blogger.com (Peter) — Thu, 15 Feb 2018 20:19:00 +0000

I first explored this topic several years ago for Server 2012 which remained applicable through the R2 lifecycle. Now with Server 2016 RDS alive and well, let’s see if anything has changed. What was true then is also true now and way back before when this was known as Terminal Services: a host’s physical or virtual resources are shared to several simultaneously connected users, each of which executing applications installed locally on the server. Remote Desktop Session Host, or RDSH, is what we call “session based” virtualization meaning that each user is essentially provided apps or a full desktop experience, but each user is simply a remote session, all existing within a single Windows Server OS. An RDSH host can be physical or virtual, virtual being more common now as you get the additional benefits of virtualization including clustering and HA. This is in stark contrast to desktop virtualization, aka VDI, in which each user is assigned a pooled or persistent desktop VM that has dedicated hardware resources provisioned from a server hypervisor. RDSH works great for application virtualization the likes of office and web apps. In short, RDSH = a single Server OS that users connect to, all CPU and RAM shared amongst all. Desktop virtualization = several desktop VMs each with their own assigned CPU and RAM.

What comes with Windows Server?

Starting in Server 2008R2, a feature called Dynamic Fair Share Scheduler was introduced which aimed to proactively and dynamically distribute CPU time based on the number of active sessions and load on each. After all, CPU is the resource consistently hit hardest in this use case. Additional drivers were added in Server 2012 to fairly share network and disk utilization, but not memory. This all remains true in Server 2016 with no apparent changes to the base mechanisms. Ultimately if memory sharing between user sessions in RDSH presents a challenge for you, based on user or application behavior, adding third-party tools or Hyper-V + desktop VMs may be a better option. That said, since your RDSH environment will likely be virtual, these elements are controlled on a per RDSH server VM basis. So balancing user load between these RDSH instances is the more important concern as no single server instance will be allowed to overwhelm the host server.
The CPU scheduler can be controlled via group policy, which is enabled by default, but not disk or network, also enabled by default.

This GPO element toggles the switch found in the registry at \HKLM\System\CurrentControlSet\Control\Session Manager\Quota system:

The other Fair Share components, can be found in the registry as well in a different location, each provide a binary on/off switch.

These elements can also be controlled via PowerShell through the command (gwmi win32_terminalservicesetting -N "root\cimv2\terminalservices"). Here I have DFSS disabled but disk and network enabled. Not sure why network shows no value here:

Test Scenario: CPU Fair Share

The output below was generated using the following configuration. Please note this does not represent an optimal or recommended RDSH configuration, it is intentionally low spec to easily showcase this scenario:

Host: Dell EMC PowerEdge R630
Hypervisor: Server 2016 Hyper-V Datacenter
RDSH: 1 x Server 2016 VMs configured with 1 vCPU/ 4GB RAM
Users: 3 total sessions represented by RDP-TCP, ICA-CGP/ pfine/b/c in the graph
Workload: Prime95 blend option
Monitoring: Perfmon captures from RDSH1 VM

This run demonstrates three user sessions executing Prime95, I spaced launches a few seconds apart, ending with all three competing for maximum CPU. As you can see, DFSS does its thing to limit CPU, 50% at two users, 33% once all three are active.

Network and disk IO fair share will work similarly should contention arise in those resources. So in a very basic form, as is the case with most Windows-included services, Microsoft gives you the bare functional minimum, which indeed works as intended. To take this functionality any further, you have to look to the partner ecosystem.

What about Citrix?

Citrix offers a number of products and services famously targeted at the EUC space, most notably XenApp and XenDesktop. Speaking of RDSH and user session/ application virtualization specifically, XenApp essentially encompasses the connection of your RDSH VMs running the Citrix VDA (Virtual Delivery Agent) to the Citrix management infrastructure. “XenApp” is the larger holistic solution which includes connection brokering, policies and management. For app and session delivery alone, you still need the RDSH role installed on Windows Server but run the VDA in addition on each instance. Previously resource management was enabled via policies in Studio but this has since migrated to Citrix Workspace Environment Management (WEM), which is the result of the recent Norskale acquisition and now represents the primary Citrix UEM tool. WEM does a whole lot more than resource management for RDSH, but for the context of this post, that’s where we’ll focus. WEM is available to XenApp & XenDesktop Enterprise or Platinum customers which you can read more about here.
WEM requires a number of components on its own, including Infrastructure Services (SQL DB, SyncFx), an agent for each RDSH instance and a management console. WEM includes a utility to create the database, another console for the infrastructure service configuration and finally another console to manage WEM as applied to the XenApp infrastructure. Once everything is created, talking and operational, WEM will require access to a valid Citrix license server before you can use it. Needless to say, WEM is usable only within a Citrix infrastructure.
To get started, make sure your agents are talking to your broker, which is set via GPO on the computer objects of your RDSH VMs. The requisite ADMX file is included with the installation media. Verify in the WEM management console under the Administration tab: Agents. If you don’t see your RDSH instances here, WEM is not working properly.

WEM provides a number of ways to control CPU, memory and IO utilization controlled by configuration sets. Servers are placed into configuration sets which tells the installed WEM agents which policies to apply, an agent can only belong to one set. If you want a certain policy to apply to XenApp hosts and another to apply for XenDesktop hosts, then you would create a configuration set for each housing the applicable policies. For this example, I created a configuration set for my RDSH VMs which will focus on resource controls.
The System Optimization tab houses the resource control options, starting with CPU, provides high level to more granular controls. At a very basic level, CPU spike protection is intended to ensure that no process, unless excluded, will exceed the % limits you specify. Intelligent CPU and IO optimization, highlighted below, causes the agent to keep a list of processes that trigger spike protection. Repeat offenders are assigned a lower CPU priority at process launch. That is the key part of what this solution does: offending processes will have their base priority adjusted so other process can use CPU.

To show this in action, I ran Prime95 in one of my XenApp sessions. As you can see below the Prime95 process is running with a normal base priority and happily taking 100% of available CPU.

Once CPU Spike protection is enabled, notice the prime95 process has been changed to a Low priority but it isn’t actually limited to 25% according to my settings. What this appears to do in reality, is make problem processes less important so that others can take CPU time if needed, but no hard limit seems to be actually enforced. I tried a number of scenarios here including process clamping, varying %, optimized vs non, all in attempts to force Prime95 to a specific % of CPU utilization. I was unsuccessful. CPU utilization never drops, other processes are allowed to execute and share the CPU, but this would happen normally with DFSS.

I also tried using the CPU Eater process Citrix uses in their videos thinking maybe Prime95 was just problematic, but if you notice even in their videos, it does not appear that the policy ever really limits CPU utilization. The base priority definitely changes so that part works, but “spike optimization” does not appear to work at all. I’d be happy to be proven wrong on this if anyone has this working correctly.

For memory optimization, WEM offers “working set optimization” which will reduce the memory working set of particular processes once the specified idle time has elapsed. Following the Citrix recommendation, I set the following policy of 5 minutes/ 5%.

To test this, I opened a PDF in Chrome and waited for optimization to kick in. It works! The first image shows the Chrome process with the PDF first opened, the second image shows the process after optimization has worked its magic. This is a 99% working set memory reduction, which is pretty cool.

The IO Management feature allows the priorities of specific processes to be adjusted as desired, but not specifically limited by IO.

What about Citrix XenApp in Azure?

XenApp can also be run in Microsoft Azure via the Citrix XenApp Essentials offer. After the requisite Azure and Citrix Cloud subscriptions are established, the necessary XenApp infrastructure is easily created from the Compute catalog directly in Azure. The minimum number of users is 25 but your existing RDS CALs can be reused and Citrix billing is consolidated on your Azure bill for ease. Azure provides a number of global methods to create, scale, automate and monitor your environment. But at the end of the day, we are still dealing with Windows Server VMs, RDSH and Citrix Software. All the methods and rules discussed previously still apply, with one important exclusion: Citrix WEM is not available in Azure/ Citrix Cloud, so DFSS is your best bet unless you turn to supported third-party softs.

Once the XenApp infrastructure is deployed within Azure, the Citrix Cloud portal is then used to manage catalogs, resource publishing and entitlements.

What about VMware?

VMware also provides a number of methods to deploy and manage applications, namely: ThinApp, AirWatch and App Volumes while providing full integration support for Microsoft RDSH via Horizon or Horizon Apps. For those keeping count, that’s FOUR distinct methods to deploy applications within Horizon. Similar to Citrix, VMware has their own UEM tool called… VMware UEM. But unlike Citrix, the VMware UEM tool can only be used for user and application management, not resource management.

While Horizon doesn’t provide any RDSH resource-optimizing capabilities directly, there are a few settings to help reduce resource intensive activities such as Adobe Flash throttling which will become less compelling as the world slowly rids itself of Flash entirely.

Something else VMware uses on RDSH hosts within a Horizon environment is memory and CPU loading scripts. These VBscripts live within C:\Program Files\VMware\VMware View\Agent\scripts and are used to generate Load Preference values that get communicated back to the Connection Servers. The reported value is then used to determine which RDSH hosts will be used for new session connections. If CPU or memory utilization is already high on a given RDSH host, VCS will send a new connection to a host with a lower reported load.

What about VMware Horizon in Azure?

VMware Horizon is also available in Azure via Horizon Cloud and has a similar basic architecture to the Citrix offer. Horizon Cloud acts as the control plane and provides the ability to automate the creation and management of resources within Azure. VMware has gone out of their way to make this as seamless as possible, providing the ability to integrate multi-cloud and local deployments all within the Horizon Cloud control plane, thus minimizing your need to touch external cloud services. From within this construct you can deploy applications, RDSH sessions or desktop VMs.

Horizon Cloud, once linked with Azure, provides full customized deployment options for RDSH farms that support delivery of applications or full session desktops. Notice that when creating a new farm, you are provided an option to limit the number of sessions per RDSH server. This is about it from a resource control perspective for this offer.

What about third party options?

Ivanti has become the primary consolidation point for predominant EUC-related tooling, most recently and namely, RES Software and AppSense. The Ivanti Performance Manager (IPM) tool (formerly AppSense Performance Manager) appears to be the most robust resource manager on the market today and has been for a very long time. IPM can be used not only for granular CPU/ memory throttling on RDSH but any platform including physical PCs or VDI desktops. I didn’t have access to IPM to demo here, so the images below are included from Ivanti.com for reference.

IPM provides a number of controls for CPU, memory and disk that can be applied to applications or groups, including those that spawn multiple chained processes with a number of options to dial in the desired end state. Performance Manager has been a favorite for RDSH and XenApp environments for the better part of a decade and for good reason it seems.

Resources

Citrix WEM product documentation: https://docs.citrix.com/en-us/workspace-environment-management/current-release.html
A year of field experience with Citrix WEM: https://youtu.be/tFoacrvKOw8
Citrix XenApp Essentials start to finish: https://youtu.be/5R-yZcDJztc
Horizon Cloud & Azure: https://youtu.be/fuyzBuzNWnQ
Horizon Cloud Requirements: http://www.vmware.com/info?id=1434
Ivanti Performance Manager product guide: https://help.ivanti.com/ap/help/en_US/pm/10.1/Ivanti%20Performance%20Manager%2010.1%20FR4%20Product%20Guide.pdf

Google Wifi Behind a Firewall

noreply@blogger.com (Peter) — Fri, 15 Sep 2017 19:25:00 +0000

More specifically behind a FIOS firewall but really much of the information below applies to any firewall you might put Google Wifi behind. I’ll dig into what you need to know and how to make some key services work while buried in a double-NAT, double firewall deployment.
Google Wifi (gWifi) is a relatively new mesh networking solution from Google that has a simple goal: Provide stupid easy, crazy fast, blanketed wifi coverage for your entire house regardless of size. This is done via very small (4.17” diameter) but very aesthetically pleasing Wifi “Points”, as Google calls them, that you place in key areas of your home to provide coverage. One Wifi Point will act as primary and connect via a BaseT Ethernet cable to your existing modem or router. The other Points will connect to the primary wirelessly and expand the signal of a single Wireless LAN (WLAN). The gWifi solution operates on both the 2.4GHz and 5GHz bands but unlike most other wireless routers, gWifi enables seamless and automatic switching between the bands as required, using a single SSID. Every client available today is supported here: 802.11a/b/g/n/ac. No need to set up and manually switch between discrete SSIDs for each band. gWifi will place your device on the best possible band given its capabilities and proximity to the gWifi Points.

Easily one the prettiest networking devices around

With the primary router cabled to the FIOS router, there is one additional switched Ethernet port available on that device and two ports available in each secondary mesh Point for wired Ethernet clients. Need more coverage? Add more gWifi Points. Unlike most routers there is no web interface here to access for management, you must use the gWifi app installed on your mobile device. The available features are intentionally limited to keep ease of use high, so those looking for advanced features like VPN may not find what they need here. Luckily, everything I need appears to be intact: custom DNS, DHCP reservations and port forwarding. Another important callout is that you cannot currently change the IP range that gWifi uses for DHCP assignment, it is and always will be 192.168.86.0/24. If this range is already in use on your network for some reason, gWifi will shift to 192.168.85.0/24. **Update - Google has now made it possible to change both the router LAN address as well as the DHCP pool range, so .86 is no longer required. If you don't care or have a good reason to change, honestly, just leave the default.

General setup is fairly simple: Connect an Ethernet cable to the WAN port of your primary gWifi Point from the switch side of your modem or firewall and power on. Download the gWifi app to your phone or tablet, find the device and follow the prompts to configure. Additional Points are easily added through the setup process which will help to ensure ideal placement for maximum coverage. Connect your wireless clients to the new network like normal and you’re done. The rest of this post will dig into more advanced topics around presenting services connected to the gWifi network out to the internet, or to clients connected to the FIOS network.

View of my gWifi mesh network

Network Architecture

In Google’s Wifi documentation, the clear preferred deployment model is the Google router connected directly to a broadband modem. This enables Google Wifi to be the primary router, AP and firewall of the house brokering all connections to the internet. Well, what if you don’t have a cable modem because you have FIOS or similar service? Or don’t really want gWifi as your barrier to the big bad web? In my particular case I use FIOS for internet access only, my TV service is provided via PlayStation Vue so I have no Set Top Boxes (STB) and my phone service is provided by Ooma. For those of us with FIOS, we have ONT boxes in our garages that connect to a router inside the house. If you have speeds >100Mbps this connection is made using a Cat5e cable instead of coax for lower speeds. I’ve read that while it appears possible to successfully replace the FIOS router with gWifi, there are some mixed results doing this. Really, this should only be considered if you are an internet-only customer in my view. If you have STBs, this won’t work as you need a coax connection between them and the router to complete the MoCa network. The easiest thing to do is simply leave the FIOS router in place as is and install the gWifi pieces behind it.
What this will create, is 2 stateful firewalls, directly connected LAN to WAN, each with their own separate subnets, with the Google device having a leg in both networks. Leave DHCP enabled on the FIOS router to serve the clients directly connected via the Ethernet switch, including the WAN port of the Google router. There is no way currently to manually specify a static IP address for the gWifi router, so it must receive via DHCP. **Update - you can now set a static IP for the WAN side of the primary gWifi router if you wish!
Important to note that the gWifi router can be put into bridge mode, thus making your FIOS router ‘primary’ but doing so will disable the mesh capabilities of the gWifi system. Only do this if you have a single gWifi router or knowingly want to disable mesh (why buy gWifi??).

The diagram below displays my home network and how I installed the gWifi system. The FIOS router is still my primary firewall and gateway to the internet, everything ultimately connects behind this device. I have a few devices that connect directly to the FIOS network and a 1Gb run that goes between floors that feeds the primary gWifi router from the FIOS router. The only device I have currently hard wired to the gWifi network is my media server (Plex/ NAS). This gives me 1Gb wired end to end from my PC to my NAS downstairs. All other devices like my PS4, Smart TVs, kid and Chromecast devices connect via wifi. The gWifi devices are depicted as G1, G2 and G3 below.

You’ll notice that the G1 router depicted above, has two IP addresses, 192.168.1.22 on the WAN side and 192.168.86.1 on the LAN side. The .22 address was assigned via DHCP from the FIOS router which I reserved so the G1 device will always receive this IP. This is important as we will configure an upstream static routing rule later that will point to this address, so we need it to not change. The gWifi routers will assign all IPs to all clients they service directly, wired or wireless. The G2 and G3 routers simply serve as extensions of G1 and will serve any clients that connect via proximity. By default, any device attached to the FIOS network (192.168.1.0) will have no knowledge of the .86 network, nor how to get to anything that lives there. So we have to tell the FIOS router how to find the .86 network, if I ever want my PC to connect to file shares hosted my my NAS, for example.

The main configuration activities I’ll cover here are:

Reserving IP addresses
Routing between networks
Port Forwarding key services
Configuring the Windows Firewall
Plex remote access

Reserving IP Addresses

In FIOS

First, find the IP assigned to the WAN side of the Google router via the gWifi app under Network Device Settings: Settings > Network & general > wifi points > [primary device] :

Login to the FIOS router and navigate to: Advanced > IP Address Distribution, then click the “Connection List” button. Find the gWifi WAN IP assignment in the connection list, click the pencil to edit and check the box to set the “Static Lease type”. This will ensure that the Google WAN port will always receive this IP.

In gWifi

Port forwarding rules can only be applied to reserved IP addresses, so lock in any PCs or Servers you intend to be configured with rules. To reserve an IP address assigned to any client on the gWifi network, open the app and navigate to: Network & general > Advanced networking > DHCP IP reservations and tap the green + button in the lower corner. Select the chosen device in the list and tap next. The MAC address and type of device is displayed, along with the gWifi Point to which the device is attached. Accept the current IP assignment or change it to suit your needs and tap next to save.

Routing

As the network exists right now, my PC can’t reach my media server on the .86 network as it doesn’t know how to get there. I could add a persistent route on my PC which would solve the problem for my PC, but a better option would be to configure a global routing rule on the FIOS router. This will enable any client on the FIOS network to be able to connect to the hosts or clients on the gWifi network. Let the router do its job: route traffic. The WAN port of the gWifi router is the gateway for any traffic destined to the .86 network. All traffic in or out of the .86 network will pass through this port. So my routing rule needs to send all traffic from the 192.168.1.0/24 network destined for the 192.168.86.0/24 network to 192.168.1.22. To illustrate this further, the image below depicts the connection from my PC to the the media server and associated hops along the way:

On the FIOS router, navigate to Advanced > Routing and click the “New Route” link in red. Enter the pertinent details and click apply. Traffic is now flowing in the right directions.

Port Forwarding

All network-accessible services, running on a PC or server, listen for connections via network ports, opened within those applications. TCP and UDP protocols with specific port numbers are assigned to applications and made available for access, such as HTTP (web server) listening on TCP 80. For my scenario, I have several ports that I need to open from my media server to make accessible to my PC. Per the diagram below, TCP 445 = SMB (file services), TCP 3389 = RDP and TCP 32400 = Plex server. Anything on the gWifi network that you wish to expose to the internet will need to be port forwarded on both the gWifi and FIOS routers, unless it advertises itself as a UPnP (Universal Plug and Play) capable service that can set its own rule on the router, which gWifi will allow. Plex happens to be one of those services, so TCP 32400 is automatically opened through the gWifi router. Currently there is no way to view or control UPnP-configured rules in the gWifi app, so you could have a slew of ports opened and not even know it. The only option at the moment is to disable UPnP entirely within the gWifi app. Hopefully Google will fix this in the near future.

The other two ports to be opened are done so with manual port forwarding rules in the gWifi app. Navigate to: Network & general > Advanced networking > Port forwarding and tap the green + button. Choose a device to forward a port from, which will only be possible if you reserved the IP address on that device. Select TCP, UDP or both as well as the internal and external ports. Both are required here. Tapping done will create the rule and set it as active.

After this step all required rules are in place on both the FIOS and gWifi router and the media server is accessible from the FIOS network. The image below shows everything configured so far along with the traffic flow.

Windows Firewall

Because my media server is a Windows box, if you need to access resources connected to the gWifi network such as NAS or Plex, you must allow these connections through the Windows firewall. By default, the Windows Firewall blocks all inbound connections unless a specific rule exists otherwise. Create a new custom firewall rule on the PC or server allowing traffic from the gateway of the FIOS network: 192.168.1.1/24. This will treat all traffic sourced from the FIOS network as trusted. Remember this entire network segment is already behind a firewall to the internet, the FIOS router. So you can safely port forward hosts on the gWifi network to the inside of the FIOS network. The new rule should be custom allowing all programs and all ports when applied to the remote IP address of the FIOS router. If you want to get more granular and only allow access to the specific ports you are exposing, that is an option as well.

Plex Remote Access

One additional step is required if you want to configure Plex with remote access. An additional port must be forwarded out from the FIOS router to allow an internal connection to TCP 32400. Log into the FIOS router and navigate to Firewall Settings > Port Forwarding. Add a new rule and make sure to click the Advanced button. Enter the IP addresses, ports and dropdowns per the screenshot below. Choose specify IP, enter the IP address of the Plex server, forward to port 32400. For the destination port, which is what will be exposed to the internet, you can either custom create the number or match the random 5-digit port generated in the Plex settings. Either way, these port numbers must match between the Plex server and the FIOS router. It’s probably best to not expose TCP 32400 directly to the internet. I don’t know of a Plex specific exploit but the service behind this port is well known so best not to advertise what it is.

Once the rule is in place and active, the Plex service should report as fully accessible. If the following dialog reports any issues, try specifying a manual public port and recreate the port forwarding rule on the FIOS router.

The diagram below depicts the remote access connection originating on the public internet, forwarding to the Plex service inside through both routers. Now your Plex server is accessible from anywhere on the internet. Remember that the public port can be any number you want.

Final Thoughts

Overall my experience with the gWifi system has been quite good. Elegant, unobtrusive, high performance and so simple. There’s a lot of high-end competition in the mesh wifi business right now and Google does a really good job, assuming you don’t require too many advanced features. Google also includes some decent performance testing tools in the gWifi app so you can gauge the performance of the speed to the internet, the wifi clients and the mesh itself as well as figure out where any problems might lie.

You can view a device’s network consumption real time and shut down any client you choose or give that device network priority for up to 4 hours. Being able to group and schedule internet access for particular devices is especially useful as a parent. If you have Sonos in the house, make sure to setup all devices using Standard mode connected to the new gWifi network. Boost mode will still work if required, but make sure to keep all devices on the same network for ease of interoperability. It appears problematic to have a controller on the other side of a firewall with all the Sonos devices behind it, due to the number of ports and connections required.

VMware vSphere HA Mode for vGPU

noreply@blogger.com (Peter) — Thu, 29 Jun 2017 18:55:00 +0000

The following post was written and scenario tested by my colleagues and friends Keith Keogh and Andrew Breedy of the Limerick, Ireland CCC engineering team. I helped with editing and art. Enjoy.

VMWare has added High Availability (HA) for virtual machines (VM’s) with an NVIDIA GRID vGPU shared pass-through device in vSphere 6.5. This feature allows any vGPU VM to automatically restart on another available ESXi host with an identical NVIDIA GRID vGPU profile if the VM’s hosting server has failed.
To better understand how this feature would work in practice we decided to create some failure scenarios. We tested how HA for vGPU virtual machines performed in the following situations:
1) Network outage: Network cables get disconnected from the virtual machine host.
2) Graceful shutdown: A host has a controlled shut down via the ESXi console.
3) Power outage: A host has an uncontrolled shutdown, i.e. the server loses AC power.
4) Maintenance mode: A host is put into maintenance mode.

The architecture of the scenario used for testing was laid out as follows:

Viewed another way, we have 3 hosts, 2 with GPUs, configured in an HA VSAN cluster with a pool of Instant Clones configured in Horizon.

1) For this scenario, we used three Dell vSan Ready Node R730 servers configured per our C7 spec, with the ESXi 6.5 hypervisor installed, clustered and managed through a VMWare vCenter 6.5 appliance.
2) VMWare VSAN 6.5 was configured and enabled to provide shared storage.
3) Two of the three servers in the cluster had an NVIDIA M60 GPU card and the appropriate drivers installed in these hosts. The third compute host did not have a GPU card.
4) 16 vGPU virtual machines were placed on one host. The vGPU VM’s were a pool of linked clones created using VMWare Horizon 7.1. The virtual machine Windows 10 master image had the appropriate NVIDIA drivers installed and an NVIDIA GRID vGPU shared PCI pass through device added.
5) All 4GB of allocated memory was reserved on the VM’s and the M60-1Q profile assigned which has a maximum of 8 vGPU’s per physical GPU. Two vCPU’s were assigned to each VM.
6) High Availability was enabled on the vSphere cluster.

Note that we intentionally didn’t install an M60 GPU card in the third server of our cluster. This allowed us to test the mechanism that allows the vGPU VM’s to only restart on a host with the correct GPU hardware installed and not attempt to restart the VM’s on a host without a GPU. In all cases this worked flawlessly, the VM’s always restarted on the correct backup host with the M60 GPU installed.

Scenario One

To test the first scenario of a network outage we simply unplugged the network cables from the vGPU VM’s host server. The host showed as not responding in the vSphere client and the VM’s showed as disconnected within a minute of unplugging the cables. After approximately another 30 seconds all the VM’s came back online and restarted normally on the other host in the cluster that contained the M60 GPU card.

Scenario Two

A controlled shutdown down of the host through ESXi worked in a similar manner. The host showed as not responding in the vSphere client and the VM’s showed as disconnected as the ESXi host began shutting down the VSAN services as part of ESXi’s shutdown procedure. Once the host had shut down, the VM’s restarted quite quickly on the backup host. In total, the time from starting the shutdown of the host until the VM’s restarted on the backup host was approximately a minute and a half.

Scenario Three

To simulate a power outage, we simply pulled the power cord of the vGPU VM’s host. After approximately one minute the host showed as not responding and the VM’s showed as disconnected in the vSphere client. Roughly 10 seconds later the vGPU VM’s had all restarted on the back up host.

Scenario Four

Placing the host in maintenance mode worked as expected in that any powered on VMs must first be powered off on that host before completion can occur. Once powered off, the VM’s were then moved to the backup host but not powered on (the ‘Move powered-off and suspended virtual machines to other hosts in the cluster’ option was left ticked when entering maintenance mode). The vGPU-enabled VM’s were moved to the correct host each time we tested this: the host with the M60 GPU card.

In conclusion, the HA failover of vGPU virtual machines worked flawlessly in all cases we tested. In the context of speed of recovery, we made the following observations with regard to the scenarios tested. The unplanned power outage test recovered the quickest at 1 minute 10 seconds followed by the controlled shutdown, with the network outage taking slightly longer again to recover.

Failure Scenario	Approximate Recovery Time
Network Outage	1 min, 30 sec
Controlled Shutdown	1 min, 30 sec
Power Outage	1 min, 10 sec

In a real world scenario the HA failover for vGPU VM’s would of course require an unused GPU card in a backup server for it to work. In most vGPU environments you would probably want to fill the GPU to its maximum capacity of vGPU based VM’s to gain maximum value from your hardware. In order to provide a backup for a fully populated vGPU host, it would mean having a fully unused server and GPU card lying idle until circumstances required it. It is also possible to split the vGPU VM’s between two GPU hosts, placing, for example 50% of the VM’s on each host. In the event of a failure the VM’s will failover to the other host. In this scenario though, each GPU would be used to only 50% of its capacity.

An unused back-up GPU host may be ok in larger or mission critical deployments but may not be very cost effective for smaller deployments. For smaller deployments it would be recommended to leave a resource buffer on each GPU server so that if there was a host failure, there is room on the GPU enabled server to facilitate these vGPU Enabled VM’s.

The Native MSFT Stack: S2D & RDS – Part 3

noreply@blogger.com (Peter) — Fri, 07 Apr 2017 04:12:00 +0000

Part 1: Intro and Architecture
Part 2: Prep & Configuration
Part 3: Performance & Troubleshooting (you are here)
Make sure to check out my series on RDS in Server 2016 for more detailed information on designing and deploying RDSH or RDVH.

Performance

This section will give an idea of disk activity and performance given this specific configuration as it relates to S2D and RDS.
Here is the 3-way mirror in action during the collection provisioning activity, 3 data copies, 1 per host, all hosts are active, as expected:

Real-time disk and RDMA stats during collection provisioning:

RDS provisioning isn’t the speediest solution in the world by default as it creates VMs one by one. But it’s fairly predictable at 1 VM per minute, so plan accordingly.

Optionally, you can adjust the concurrency level to adjust the number of VMs RDS can create in parallel by using the Set-RDVirtualDesktopConcurrency CMDlet. Server 2012R2 supported a max of 20 concurrent operations but make sure the infrastructure can handle whatever you set here.

Set-RDVirtualDesktopConcurrency –ConnectionBroker “RDCB name” –ConcurrencyFactor 20

To illustrate the capacity impact of 3-way mirroring, take my lab configuration which provides my 3-node cluster with 14TB of total capacity, each node contributing 4.6TB. I have 3 volumes fixed provisioned totaling 4TB with just under 2TB remaining. The math here is that of my raw capacity, 30% is usable, as expected. This equation gets better at larger scales, starting with 4 nodes, so keep this in mind when building your deployment.

Here is the disk performance view from one of the Win10 VMs within the pool (VDI-23) running Crystal DiskMark, which runs Diskspd commands behind the scenes. Read IOPS are respectable at an observed max of 25K during the sequential test.

Write IOPS are also respectable at a max observed of nearly 18K during the Sequential test.

Writing to the same volume, running the same benchmark but from the host itself, I saw some wildly differing results. Similar bandwidth numbers but poorer showing on most tests for some reason. One run yielded some staggering numbers on the random 4K tests which I was unable to reproduce reliably.

Running Diskspd tells a slightly different story with more real-world inputs specific to VDI: write intensive. Here given the punishing workload, we see impressive disk performance numbers and almost 0 latency. The following results were generated from a run generating a 500MB file, 4K blocks weighted at 70% random writes using 4 threads with cache enabled.

Diskspd.exe -b4k -d60 -L -o2 -t4 -r -w70 -c500M c:\ClusterStorage\Volume1\RDSH\io.dat

Just for comparison, here is another run with cache disabled. As expected, not so impressive.

Troubleshooting

Here are a few things to watch out for along the way.
When deploying your desktop collection, if you get an error about a node not having a virtual switch configured:

One possible cause is that the default SETSwitch was not configured for RDMA on that particular host. Confirm and then enable per the steps above using Enable-NetAdapterRDMA.

On the topic of VM creation within the cluster, here is something kooky to watch out for. If working on one of your cluster nodes locally and you try to add a new VM to be hosted on a different node, you may have trouble. When you get to the OS install portion where the UI offers to attach an ISO to boot from. If you point to an ISO sitting on a network share, you may get the following error: Failing to add the DVD device due to lack of permissions to open attachment.

You will also be denied if you try to access any of the local paths such as desktop or downloads.

The reason for this is that the dialog is using the remote browser for file access on the server you are creating the VM. Counter-intuitive perhaps but the work around is to either log into the host where the VM is to be created directly or copy your ISO local to that host.

Watch this space, more to come…

Part 1: Intro and Architecture
Part 2: Prep & Configuration
Part 3: Performance & Troubleshooting (you are here)

Resources

S2D in Server 2016
S2D Overview
Working with volumes in S2D
Cache in S2D

The Native MSFT Stack: S2D & RDS – Part 2

noreply@blogger.com (Peter) — Fri, 07 Apr 2017 04:05:00 +0000

Part 1: Intro and Architecture
Part 2: Prep & Configuration (you are here)
Part 3: Performance & Troubleshooting
Make sure to check out my series on RDS in Server 2016 for more detailed information on designing and deploying RDSH or RDVH.

Prep

The number 1 rule of clustering with Microsoft is homogeneity. All nodes within a cluster must have identical hardware and patch levels. This is very important. The first step is to check all disks installed to participate in the storage pool, bring all online and initialize. This can be done via PoSH or UI.

Once initialized, confirm that all disks can pool:

Get-PhysicalDisk -CanPool $true | sort model

Install Hyper-V and Failover Clustering on all nodes:

Install-WindowsFeature -name Hyper-V, Failover-Clustering -IncludeManagementTools -ComputerName InsertName -Restart

Run cluster validation, note this command does not exist until the Failover Clustering feature has been installed. Replace portions in red with your custom inputs. Hardware configuration and patch level should be identical between nodes or you will be warned in this report.

Test-Cluster -Node node1, node2, etc -include "Storage Spaces Direct", Inventory, Network, "System Configuration"

The report will be stored in c:\users\<username>\AppData\Local\Temp

Networking & Cluster Configuration

We'll be running the new Switch Embedded Team (SET) vSwitch feature, new for Hyper-V in Server 2016.

Repeat the following steps on all nodes in your cluster. First, I recommend renaming the interfaces you plan to use for S2D to something easy and meaningful. I'm running Mellanox cards so called mine Mell1 and Mell2. There should be no NIC teaming applied at this point!

Configure the QoS policy, first enable the datacenter bridging (DCB) feature which will allow us to prioritize certain services the traverse the Ethernet.

Install-WindowsFeature -Name Data-Center-Bridging

Create a new policy for SMB Direct with a priority value of 3 which marks this service as “critical” per the 802.1p standard:

New-NetQosPolicy "SMBDirect" -NetDirectPortMatchCondition 445 -PriorityValue8021Action 3

Allocate at least 30% of the available bandwidth to SMB for the S2D solution:

New-NetQosTrafficClass "SMBDirect" -Priority 3 -BandwidthPercentage 30 -Algorithm ETS

Create the SET switch using the adapter names you specified previously. Items in red should match your choices or naming standards. Note that this SETSwitch is ultimately a vNIC and can receive an IP address itself:

New-VMSwitch -Name SETSwitch -NetAdapterName "Mell1","Mell2" -EnableEmbeddedTeaming $true

Add host vNICs to the SET switch you just created, these will be used by the management OS. Items in red should match your choices or naming standards. Assign static IPs to these vNICs as required as these interfaces are where RDMA will be enabled.

Add-VMNetworkAdapter –SwitchName SETSwitch –name SMB_1 –managementOS
Add-VMNetworkAdapter –SwitchName SETSwitch –name SMB_2 –managementOS

Once created, the new virtual interface will be visible in the network adapter list by running get-netadapter.

Optional: Configure VLANs for the new vNICs which can be the same or different but IP them uniquely. If you don’t intend to tag VLANs in your cluster or have a flat network with one subnet, skip this step.

Set-VMNetworkAdapterVLAN –VMNetworkAdapterName “SMB_1” –VlanId 00 –Access –ManagementOS
Set-VMNetworkAdapterVLAN –VMNetworkAdapterName “SMB_2” –VlanId 00 –Access –ManagementOS

Verify your VLANs and vNICs are correct. Notice mine are all untagged since this demo is in a flat network.

Get-VMNetworkAdapterVlan –ManagementOS

Restart each vNIC to activate the VLAN assignment.

Restart-NetAdapter “vEthernet (SMB_1)”
Restart-NetAdapter “vEthernet (SMB_2)”

Enable RDMA on each vNIC.

Enable-NetAdapterRDMA “vEthernet (SMB_1)”, “vEthernet (SMB_2)”

Next each vNIC should be tied directly to a preferential physical interface within the SET switch. In this example we have 2 vNICs and 2 physical NICs for a 1:1 mapping. Important to note: Although this operation essentially designates a vNIC assignment to a preferential pNIC, should the assigned pNIC fail, the SET switch will still load balance vNIC traffic across the surviving pNICs. It may not be immediately obvious that this is the resulting and expected behavior.

Set-VMNetworkAdapterTeamMapping -VMNetworkAdapterName "SMB_1" -ManagementOS –PhysicalNetAdapterName “Mell1”
Set-VMNetworkAdapterTeamMapping -VMNetworkAdapterName "SMB_2" -ManagementOS –PhysicalNetAdapterName “Mell2”

To quickly prove this, I have my vNIC “SMB_1” preferentially tied to the pNIC “Mell1”. SMB_1 has the IP address 10.50.88.82

Notice that even though I have manually disabled Mell1, the IP still responds to a ping from another host as SMB_1’s traffic is temporarily traversing Mell2:

Verify the RDMA capabilities of the new vNICs and associated physical interfaces. RDMA Capable should read true.

Get-SMBClientNetworkInterface

Build the cluster with a dedicated IP and slash subnet mask. This command will default to /24 but still might fail unless explicitly specified.

New-Cluster -name "Cluster Name" -Node Node1, Node2, etc -StaticAddress 0.0.0.0/24

Checking the report output stored in c:\windows\cluster\reports\, it flagged not having a suitable disk witness. This will auto-resolve later once the cluster is up.

The cluster will come up with no disks claimed and if there is any prior formatting, they must first be wiped and prepared. In PowerShell ISE, run the following script, enter your cluster name in the red text.

icm (Get-Cluster -Name S2DCluster | Get-ClusterNode) {
Update-StorageProviderCache


Get-StoragePool | ? IsPrimordial -eq $false | Set-StoragePool -IsReadOnly:$false -ErrorAction SilentlyContinue


Get-StoragePool | ? IsPrimordial -eq $false | Get-VirtualDisk | Remove-VirtualDisk -Confirm:$false -ErrorAction SilentlyContinue


Get-StoragePool | ? IsPrimordial -eq $false | Remove-StoragePool -Confirm:$false -ErrorAction SilentlyContinue


Get-PhysicalDisk | Reset-PhysicalDisk -ErrorAction SilentlyContinue


Get-Disk | ? Number -ne $null | ? IsBoot -ne $true | ? IsSystem -ne $true | ? PartitionStyle -ne RAW | % {


$_ | Set-Disk -isoffline:$false


$_ | Set-Disk -isreadonly:$false


$_ | Clear-Disk -RemoveData -RemoveOEM -Confirm:$false


$_ | Set-Disk -isreadonly:$true


$_ | Set-Disk -isoffline:$true


}


Get-Disk |? Number -ne $null |? IsBoot -ne $true |? IsSystem -ne $true |? PartitionStyle -eq RAW | Group -NoElement -Property FriendlyName


} | Sort -Property PsComputerName,Count

Once successfully completed, you will see an output with all nodes and all disk types accounted for.

Finally, it’s time to enable S2D. Run the following command and select “yes to all” when prompted. Make sure to use your cluster name in red.

Enable-ClusterStorageSpacesDirect –CimSession S2DCluster

Storage Configuration

Once S2D is successfully enabled, there will be a new storage pool created and visible within Failover Cluster Manager. The next step is to create volumes within this pool. S2D will make some resiliency choices for you depending on how many nodes are in your cluster. 2 nodes = 2-way mirroring, 3 nodes = 3-way mirroring, if you have 4 or more nodes you can specify mirror or parity. When using a hybrid configuration of 2 disk types (HDD and SSD), the volumes reside within the HDDs as the SSDs simply provide caching for reads and writes. In an all-flash configuration only the writes are cached. Cache drive bindings are automatic and will adjust based on the number of each disk type in place. In my case, I have 4 SSDs + 5 HDDs per host. This will net a 1:1 cache:capacity map for 3 pairs of disks and a 1:2 ratio for the last 3. Microsoft’s recommendation is to make the number of cache drives a multiple of the number of capacity drives, for simple symmetry. If a host experiences a cache drive failure, the cache to capacity mapping will readjust to heal. This is why a minimum of 2 cache drives per host are recommended.

Volumes can be created using PowerShell or Failover Cluster Manager by selecting the “new disk” option. This one simple PowerShell command does three things: creates the virtual disk, places a new volume on it and makes it a cluster shared volume. For PowerShell the syntax is as follows:

New-Volume –FriendlyName “Volume Name” –FileSystem CSVFS_ReFS –StoragePoolFriendlyName S2D* –size xTB

CSVFS_ReFS is recommended but CSVFS_NTFS may also be used. Once created these disks will be visible under the Disks selection within Failover Cluster Manager. Disk creation within the GUI is a much longer process but gives meaningful information along the way, such as showing that these disks are created using the capacity media (HDD) and that the 3-server default 3-way mirror is being used.

The UI also shows us the remaining pool capacity and that storage tiering is enabled.

Once the virtual disk is created, next we need to create a volume within using another wizard. Select the vDisk created in the last step:

Storage tiering dictates that the volume size must match the size of the vDisk:

Skip the assignment of a drive letter, assign a label if you desire, confirm and create.

The final step is to add this new disk to a Cluster Shared Volume via Failover Cluster Manager. You’ll notice that the owner node will be automatically assigned:

Repeat this process to create as many volumes as required.

RDS Deployment

Install the RDS roles and features required and build a collection. See this post for more information on RDS installation and configuration. There is nothing terribly special that changes this process for a S2D cluster. As far as RDS is concerned, this is an ordinary Failover Cluster with Cluster Shared Volumes. The fact that RDS is running on S2D and “HCI” is truly inconsequential.
As long as the RDVH or RDSH roles are properly installed and enabled within the RDS deployment, the RDCB will be able to deploy and broker connections to them. One of the most important configuration items is ensuring that all servers, virtual and physical, that participate in your RDS deployment, are listed in the Servers tab. RDS is reliant on Server Manager and Server Manager has to know who the players are. This is how you tell it.

To use the S2D volumes for RDS, point your VM deployments at the proper CSVs within the C:\ClusterStorage paths. When building your collection, make sure to have the target folder already created within the CSV or collection creation will fail. Per the example below, the folder “Pooled1” needed to be manually created prior to collection creation.

During provisioning, you can select a specific balance of VMs to be deployed the RDVH-enabled hosts you select, as can be seen below.

RDS is cluster aware in that the VMs it creates can be created as HA within the cluster and the RD Connection Broker (RDCB) remains aware as to which host is running which VMs should they move. In the event of a failure, VMs will be moved to a surviving host by the cluster service and the RDCB will keep track.

Part 1: Intro and Architecture
Part 2: Prep & Configuration (you are here)
Part 3: Performance & Troubleshooting

Resources

S2D in Server 2016
S2D Overview
Working with volumes in S2D
Cache in S2D

The Native MSFT Stack: S2D & RDS – Part 1

noreply@blogger.com (Peter) — Fri, 07 Apr 2017 04:03:00 +0000

Part 1: Intro and Architecture (you are here)
Part 2: Prep & Configuration
Part 3: Performance & Troubleshooting
Make sure to check out my series on RDS in Server 2016 for more detailed information on designing and deploying RDSH or RDVH.

For Microsoft purists looking to deploy VDI or another workload on the native Microsoft stack, this is the solution for you! Storage Spaces Direct (S2D) is Microsoft’s new kernel-mode software-defined-storage model exclusive to Server 2016. In the last incarnation of Storage Spaces on Server 2012R2, JBODs were required with SAS cables running to each participating node that could then host SMB namespaces. No longer! Server 2016 Datacenter Edition, local disks and Ethernet (10Gb + RDMA recommended) are all you need now. The setup is executed almost exclusively in PowerShell which can be scripted, of course. Microsoft has positioned their first true HCI stack targeting the smaller side of SMB with a 16 node maximum scale. The S2D architecture runs natively on a minimum of 2 nodes configured in a Failover Cluster with 2-way mirroring.
NVMe, SSD or HDDs installed in each host are contributed to a 2-tier shared storage architecture using clustered mirror or parity resiliency models. S2D supports hybrid or all-flash models with usable disk capacity calculated using only the capacity tier. SSD or NVMe can be used for caching and SSD or HDD can be used for capacity. Or a mix of all three can be used with NVMe used for cache, SSD and HDD providing capacity. Each disk tier is extensible and can be modified up or down at any time. Dynamic cache bindings ensure that the proper ratio of cache:capacity disks remain consistent for any configuration regardless of whether cache or capacity disks are added or removed. The same is true in the case of drive failure in which case S2D self heals to ensure a proper balance. Similar to VMware VSAN, the storage architecture changes a bit between hybrid or all-flash. In the S2D hybrid model, the cache tier is used for both writes and reads. When the all-flash model is in play, the cache tier is used only for writes. Cluster Shared Volumes are created within the S2D storage pool and shared across all nodes in the cluster. Consumed capacity in the cluster is determined by provisioned volume sizes, not actual storage usage within a given volume.

A new Switch Embedded Teaming (SET) model brings NIC teaming integration to Hyper-V in a way that provides easy pNIC service protection and vNIC scale out for distributed management architectures. By contrast, in 2012R2 the teaming mechanism exists at the Windows OS layer with a few load balancing modes included. For Hyper-V in 2016 SET is the model you want to use. Important to note that this should not be combined with any other NIC teaming option, you need to use one or the other. vNICs are created to connect VMs to the SET switch or used by the Management OS for clustering or other services. If you want to use different VLANs for specific services, a vNIC per VLAN will be required. At a minimum, you will want three vNICs, one for mgmt and two for storage IO. Things like CSV, cluster heartbeating and Live Migration can easily make use of the RDMA/ SMB vNICs.

NTFS is available but ReFS is the recommended file system for most implementations which currently doesn’t support deduplication. Storage capacity efficiency is directly related to the resiliency model you choose to implement for your volumes. Mirroring does as it suggests by creating 2 or 3 copies of data on discrete hosts. Parity, also known as Erasure Coding, is similar to a RAID model which uses less storage for resiliency but at the cost of disk performance. New to Server 2016 is a mixed parity mode that allows a volume to exist as part mirror and part parity. Writes would first go to the mirror portion then later be de-staged to the parity portion. A desired level of fault tolerance will affect the minimum number of nodes required per a given configuration, as outlined in the table below.

Resiliency	Fault Tolerance	Storage Efficiency	Minimum # hosts
2-way mirror	1	50%	2
3-way mirror	2	33%	3
Dual parity	2	50-80%	4
Mixed	2	33-80%	4

Much more information on S2D resiliency and examples here. As a workload, RDS layers on top of S2D as it would with any other Failover Cluster configuration. Check out my previous series for a deeper dive into the native RDS architecture.

Lab Environment
For this exercise I’m running a 3-node cluster on Dell R630’s with the following specs per node:

Dual E5-2698v4
256GB RAM
HBA330
4 x 400GB SAS SSD
5 x 1TB SAS HDD
Dual Mellanox ConnectX-3 NICs RDMA Over Converged Ethernet (ROCE)
Windows 2016 Datacenter (1607)

The basic high-level architecture of this deployment:

Part 1: Intro and Architecture (you are here)
Part 2: Prep & Configuration
Part 3: Performance & Troubleshooting

Resources

S2D in Server 2016
S2D Overview
Working with volumes in S2D
Cache in S2D

New PC Build for 2017

noreply@blogger.com (Peter) — Wed, 01 Feb 2017 22:27:00 +0000

Credit: BitFenix

I can't believe it's time already, 2 years has flown by! I've managed to stick to a 2-year upgrade cadence for the last 8 years, performing meaningful upgrades at each interval. This year I debated skipping but I pressed forward as there are a few key technologies I want to enable: Pascal GPU, M.2 NVMe, DDR4 RAM and the Z270 chipset. 2 years is also a great time frame for reasonable cost reclamation on my used parts. The new Kaby Lake CPU upgrade doesn't look to be a massive jump from the Haswell chip I ran for the past 2 years (4790K) and an even smaller step from Skylake. In case you want to check out my previous builds they can be found here and here.
I plan to reuse my BitFenix Phenom M Micro-ATX case which I absolutely love, my amazing Noctua NH-U12S CPU cooler + case fans and my Silverstone ST55F PSU, but those will be the only holdovers.

Motherboard

Micro-ATX is still the smallest form factor that works for me given what I want to achieve and I endeavor to go as small as possible. The Asus ROG Strix Z270i came very close to pushing me into Mini-ITX. For this build I need 2 x M.2 slots and like the idea of having a 2nd PCIe slot should I want to do SLI at some point. 2 x DIMM slots is doable to run the 32GB RAM I need but again, no expansion while reusing my existing investment. So I stayed with mATX and went with the Asus Prime Z270M-Plus which will suit my needs without a lot of extra pizazz. There are a number of considerations in the Asus mobo lineup as well as in the Intel chipsets. The graphic below highlights the key features of my new board. Most notably, RAM clock support up to 3866MHz, a USB Type-C connector in the IO panel, dual x4 M.2 slots positioned optimally below the PCIe slots, Intel Gb NICs and SafeSlot Core which provides PCIe slot strength to hold those heavy GPUs.

Credit: Asus

Asus put together a great blog post that details all their current motherboard product lines and how they are differentiated. In short the Prime line is the all-around mainstream offer without a ton of extravagances, TUF is the military spec with drop resistance, Strix is geared at the gaming crowd with 5 levels (low to high) and supports higher RAM clocks, ROG is aimed at people who consider themselves enthusiasts with extra lighting support, overclocking options and the highest available RAM clocks. Having bought ROG boards in the past and not used a fraction of the extra stuff they come with, I took a serious feature evaluation on exactly what I need in a motherboard. Prime suits my needs just fine and the saved $ is welcomed.

Chipset

The 270 is the new chipset for the 7th gen Intel proc and comes in “Z” or “H” variations. Z for consumers includes features like overclocking while H is geared more for corporate builds with less features. New in the 200 series chipset is support for Intel’s Optane NVMe and dual m.2 slots with dedicated PCIe lanes. Both Skylake and Kaby Lake CPUs use the same 1151 socket and are interchangeable on either 100 or 200 series boards. Below is a PCH feature comparison of the 4 most relevant configurations at the moment. You notice the biggest difference between the 170 and 270 are more IO/PCIe lanes. Also on the 170 chipset, a M.2 device cost 2 x SATA ports since these share PCIe lanes (same on the Z97 boards), this tax has been removed on the 270.

Chipset	Intel Z270	Intel H270	Intel Z170	Intel H170
SKU Focus Segment	Consumer	Consumer / Corporate	Consumer	Consumer / Corporate
CPU Support	Kaby Lake-S / Skylake-S	Kaby Lake-S / Skylake-S	Kaby Lake-S / Skylake-S	Kaby Lake-S / Skylake-S
CPI PCI-e Configuration	1 x 16 or 2 x 8 or 1 x 8 + 2 x 4	1 x 16	1 x 16 or 2 x 8 or 1 x 8 + 2 x 4	1 x 16
Independent DisplayPort	3	3	3	3
Memory DIMMs	4	4	4	4
Overclocking	Yes	No	Yes	No
Intel SmartSound Technology	Yes	Yes	Yes	Yes
Intel Optane Technology	Yes	Yes	No	No
Intel Rapid Storage Technology	15	15	14	14
Intel Rapid Storage Technology From PCIe Storage Drive Support	Yes	Yes	Yes	Yes
RAID 0,1,5,10	Yes	Yes	Yes	Yes
Intel Smart Response Technology	Yes	Yes	Yes	Yes
I/O Port Flexibility	Yes	Yes	Yes	Yes
Maximum High Speed I/O Lanes	30	30	26	22
Total USB Ports (Max USB 3.0)	14 (10)	14 (8)	14 (10)	14 (8)
Max SATA 6 Gbps Ports	6	6	6	6
Max PCI-E 3.0 Lanes	24	20	20	16
Max Intel RST for PCIe Storage Ports (x2 M.2 or x4 M.2)	3	2	3	2

CPU

Intel is making it less difficult to choose a desktop CPU these days with far fewer overall options if you want a higher-end unlocked CPU under $500 and aren’t upgrading from a recent gen. If you want the best 8 core desktop CPU Intel has to offer and have no budget, look no further than the i7-6900K. If you’re upgrading from a 4th or earlier gen i7 the choice is simple: i7-7700K (14nm). If you have a Skylake CPU already (i7-6700K, also 14nm) there is no need to upgrade as any performance gain will be minimal. If you’re like me coming from Haswell (i7-4790K), the performance gain will be at best in the 10% range and may not be worth it, unless also like me, you want to upgrade to enable other specific features. It’s odd to put this in print, but for this build the CPU is actually the least exciting thing I’m adding here! It also appears that this trend will continue for the next few generations of Intel CPUs, so I might not be upgrading again in 2 years. Between the 6th and 7th generation CPUs mentioned above, the only improvements on the Kaby Lake are: +200MHz base clock (times 4 cores), +300MHz to Turbo, increased DDR4 speeds, and new embedded graphics gen, same socket, same 91W TDP. There just isn’t much here to warrant an upgrade from 6th to 7th gen sadly.

Another alternative if you’re tired of the stale and uninspiring Intel lineup is the AMD Ryzen 7 coming in a few weeks. Early benchmarks show Ryzen beating the pants off of the highest end Intel chips. Promising to ship for a fraction of the cost ($329) and boasting 8 cores with a higher base clock but lower TDP, Ryzen should get serious consideration. This will be ultimately a good thing for Intel and push them to respond in kind upping their game to compete. Who knows, in 2 years I may be rebuilding with AMD.

Credit: AMD

CPU Cooler

Since the mounting of Intel CPUs hasn’t changed enough from the 4th gen CPUs, I was able to reuse my Noctua NH-U12S tower cooler which is designed to support many different mounting requirements. This all copper 6 heat piper cooler has 45MM fins so will not encroach on any memory heat spreader, regardless of height. With the NF-F12 fan attached to the cooler, cold air is pushed across the fins, carrying hot air straight out the back of my case. This remains and excellent part and excellent choice for a SFF Kaby Lake PC build.

Credit: Noctua

Memory

Another reason I wanted to do this build was for the DDR4 upgrade as my Haswell build was DDR3. The memory market is a bit more complicated from the last time I looked at it, with most higher-end vendors playing the same marketing game. Most of the DIMMs available in this space are really 2133MHz DIMMs, but using XMP2 profiles enabled in the BIOS, the modules clock much higher. These modules are marketed using this higher frequency, which is not the default. This is the difference between the 1.2v SPD value of the module and the “tested” value enabled via higher voltage in XMP. What you pay for ultimately is this higher tested speed, a pretty heat spreader and a lifetime warranty. After reading several articles comparing various DDR4 clock speeds across various benchmarks and applications, what is glaringly apparent is that the usefulness of faster memory fits into a very limited use case. In other words, almost no one will truly benefit by paying too much for high clocked RAM. My research showed that 3000MHz is about the sweet spot for DDR4 right now, anything faster is just more expensive with unlikely-to-achieve returns.
Now that said, I was initially sold on a 16GB x 2 kit from the Corsair Vengeance LPX line marketed at 3200MHz. I went with these because Corsair is a well regarded brand and they were priced like a lower clocked pair. I was unable to ever achieve the marketed speed of 3200MHz, the closet I got, stable, was 3100MHz. But to get there I had to overclock the CPU as well. With only XMP enabled on these modules, I was never able to even POST, so I sent them back. There were several other reviews stating the same problem, which I should have heeded. Read the negative reviews and don’t settle for sub-par quality!
My second attempt was a 32GB 3000MHz TridentZ kit from G.Skill, F4-3000C15D-32GTZ. These modules are amazing, not only did the XMP profile take immediately as expected with no CPU OC required, they are quite possibly the most beautiful DIMMs I’ve ever laid eyes on. Pictures really don’t do them justice.

Credit: G.Skill

You’ll notice in the G.Skill spec table below the difference between the SPD and Tested values, these are sold with the tested speeds as you can see on the DIMM label above. Nevertheless, I REALLY like this memory!

XMP2 profile applied, RAM performing as tested, as expected.

One note on the capacity, how much RAM do you really need? Personally, I leave a ton of stuff open all the time on my PC. 50+ tabs in chrome, word docs, Visio diagrams, RDP sessions etc. I like not having to ever worry about not having enough memory, no matter the workload I need to run. Dishonored 2 itself can use upwards of 8GB all on its own! The other thing I like is not having to use a pagefile which saves wear and tear on my SSDs but also guarantees high performance for all applications. To pull this off, watch the committed value on the memory tab in Task Manager. If this grows anywhere near the amount of physical RAM you have, the pagefile should probably be left enabled. If not, you could disable the pagefile and ease the burden on your SSD.

Data

One the largest motivators for my upgrade this year was moving to Non-Volative Memory Express (NVMe), specifically for the really exciting modules in the new Samsung 960 line. I’ll continue my three-tier model for OS, apps and data, using NVMe for the first two and spinning media for the third. Eventually I’d like to replace the data tier with an SSD, when the time is right. For this build I have a 512GB Samsung 960 Pro for my C drive, a 500GB Samsung 960 Evo for the app drive and a 4TB Seagate BarraCuda 3.5” SATA disk for archive data. These new Samsung NVMe modules are outperforming anything else in this space right now with the following factory performance profiles.

Module	Capacity	Sequential Reads	Sequential Writes
960 Pro	512GB	3500MB/s	2100MB/s
960 Evo	500GB	3200MB/s	1900MB/s

These optimistic factory numbers are several times faster than any SSD from any previous build. Crazy! As of this writing, the 512GB Pro module is $330 with the 500GB Evo at $250. Definitely a healthy cost differential for what you may argue is a near negligible performance delta. I wanted, what was perceived, as the fastest NVMe on the market so it seemed worth the price. Capacities on the Pro module go up to 2TB and 1TB on the Evo. The other big difference between these is the Pro modules come with a 5 year warranty vs 3 years on the Evo. These are x4 PCIe 3.0 modules using Samsung’s new famed Polaris controller and 48-layer V-NAND memory.

Credit: Samsung

For local archive data I’m running the 4TB Seagate BarraCuda which is 6Gb SATA and 3.5”. Not a lot to say about this guy, who is a dinosaur in the making.

Credit: Seagate

My basic data scheme goes like this: C: OS only, D: Apps, games and a copy of working data (Dropbox), X: maintains a copy of working data from D and anything else too large or unimportant enough to replicate. Important data is then replicated to a 3rd target: NAS.

Graphics

Upgrading my graphics capability was easily the most noticeable upgrade from my prior build. My previous MSI GTX970 was so solid, I went that way again with a MSI GTX1070 Gaming X 8G. I’m a huge fan of these cards. This one in particular comes with a back plate, 8GB GDDR5 memory @ 8108MHz, a tunable GPU clock and fans that only run when temps exceed 60C (which isn’t very often). The 1080 model gets you higher GPU and memory clocks with more CUDA cores but looking at the benchmarks it barely outperforms its lesser sibling the 1070. I haven’t yet been able to throw anything that this card couldn’t handle at ultra resolution settings, so far Dishonored2, Mafia3 and Dues Ex: Mankind Divided.

Credit: MSI

PSU

Reusing now for its third build in a row, my trusty Silverstone ST55F. Full modularity providing up to 600w max output with a 140mm footprint at 80Plus Gold efficiency… Hard to beat this one, unless your case can’t accept the depth. One step above is the newer ST55F with the platinum efficiency rating. The PP05 short cable kit worked well again and kept cable clutter to a minimum, although I don’t require many cables anymore. And now with the NVMe’s I used a full cable less no longer needing to power 2 extra SATA devices. As always, make sure to check JonnyGuru.com first for thorough and detailed PSU reviews, the ST55F scores very high there.

Credit: Silverstone

Fans

High quality SSO-bearing driven cooling with a low decibel rating and zero vibration is worth the small upcharge. Noctua makes the best fans on the market, so outfitted the entire case. Dual NF-F12 PWM 120mm at the top for downward intake and 1 x NF-A14 140mm fan at the rear for exhaust, in addition to the 120mm NF-F12 on the CPU cooler. I can’t recommend these highly enough.

Credit: Noctua

Case and Complete Build

Once again I built into my BitFenix Phenom M which remains my favorite as easily the best case I’ve ever built with. If you go this route I highly recommend you flip the side panels putting the panel with the buttons and USB ports on the left side of the case instead of the default right side. Otherwise, any time you need to open the case you will have to contend with cords and possibly unplugging cables from a power switch or USB header.

Credit: BitFenix

The Phenom is a mix of steel and soft plastic with the same basic air flow design as the SG09: top down cooling, rear and optional bottom exhaust. This goes against the natural order of things (heat rising) but works very well with 2 x 120mm NF-F12 fans blowing cool air down into the case, directly into the GPU, down into the CPU cooler’s fan blowing across the tower fins and 140mm at the back to exhaust. The GPU breaths cool air, blows it across the PCB and out the rear slot. The rear fan can be either 120mm or 140mm, I opted for the later which is audible at times briefly under load but provides extremely effective cooling. Otherwise this build is silent! The black removable dust filter along the top of the case is the only filtered air input aside from the mesh in the front cover used by the solely PSU, which exhausts downward below the case. The case supports a 120mm radiator, such as the Corsair H80i, if desired. The base section can support dual 120mm fans, a single 200mm fan, or 2 x 3.5” HDDs. The case also comes with a removable heat shield underneath to prevent exhaust heat from the PSU getting sucked back into the case. Lots of different configurations are possible with this case which is fantastic!
The image below shows my completed build and illustrates the airflow scheme per my design selections. Ultimately this case works extremely well and keeps all components well cooled. There aren’t a lot of nooks to tuck cables into in this case so bundling is about as good as it gets which is complicated when using shorter cables. The key is to not block any airflow with the bundle.

CPU Performance

The tools I used to benchmark CPU are PassMark PerformanceTest 9 and Cinebench. The CPU Mark results with the 2 reference parts tells the tale, there is only a very small incremental gain over my previous 4790K. The gain is even smaller from the reference 6700K Skylake part, reinforcing the fact that if you own Skylake now, there is really no point at all upgrading to Kaby Lake. My results for all components ranked within the 99th percentile according to PassMark.

Similar story on Cinebench but as is the case with all the benchmarks, there’s a ton of variance and a wide spread in the scores.

Memory Performance

For the Memory Mark I included a few other high-end PC builds with varying memory speeds. Not sure how useful this bench is honestly due to that top result on slower DDR3, shown below. Generally not a lot of variance here further suggesting that paying for higher RAM clocks is probably not worth it.

Graphics Performance

At the time I bought my new MSI GTX 1070 Gaming X board, the benchmarks showed this card barely below the top dog 1080, but for a significant cash savings. Coming from the 970, the 1070 is improved in every way with more memory, more cores, higher clocks and higher supported resolutions (4K). There was nothing my prior 970 couldn’t handle, as long as I had my games properly optimized, which meant a setting quality, at best, to “high”. With the 1070 I can run very intensive titles like Dishonored 2 at Ultra settings with no trouble at all.

+3500 points in the 3D graphics mark test:

Old vs new on the Unigine Heaven bench, pretty significant difference even running the 1070 a step higher at Ultra:

960 Pro vs 960 Evo

Ok prepared to be amazed, here are the 960’s in action. This test consists of running a simple Crystal Disk Mark bench while monitoring IOPS with Perfmon. The 960 Pro generates nearly 200,000 read IOPS while clocking a staggering 3255 MB/s, which as amazing as that is, is actually much lower than what this part should be capable of!

Writes clock in around 140,000 IOPS with 1870MB/s on the 32 queues sequential test.

The highest numbers I observed in these tests were 3422MB/s reads and 1990MB/s writes. Impressive, but still below the advertised 3500/ 2100MB/s and the performance of this module is not nearly as consistent as it should be.The 960 Evo is no slouch either and actually outperforms the Pro in these tests!! This definitely should not be the case.

Running the same bench, the 960 Evo generated 203,000 IOPS while pushing 3335 MB/s. This is higher than the Pro module in both areas.

Reads are also impressive here and while the Evo doesn’t beat the Pro in MB/s, it does clock in 13,000 IOPS higher.

Running the benchmark within Samsung Magician draws an even starker picture. The Evo absolutely kills the Pro in read performance but the Pro wins the write bench by a small margin. These results should not be.

In light of this, the Pro module is going back, I’ll update this post with new numbers once I receive my replacement part. If the replacement doesn’t perform, it too will be going back to be replaced by another Evo module instead. As is stands, the Evo’s performance is very predictable, uncomfortably close to the Pro and for $65 less it may be the better buy, if you don’t mind a shorter 3 year warranty.

Just for comparison, here is the DiskMark for my older Samsung 840 Pro which is a solidly performing SSD:

Temps & Power Consumption

Even with my case’s upside-down airflow design, it does a good job keeping things cool. This is also in no small part due to the incredible Noctua fans which are literally the best you can buy and well worth it. I run my PC using the default Balanced power plan which allows the CPU to throttle down during idle or low activity periods. At idle with fans running at low speeds and the core voltage at .7v, the CPU chirps along at a cozy 38C with the fans barely spinning.

Under benchmark load with the CPU pegged at 100%, temps shift between low 50s to low 60’s with fans shifting their RPMs upward in kind. I touched 66 a couple of times but couldn’t capture the screenshot. 63 degrees under load I’d say is very good for this CPU! Once the Arctic Silver 5 thermal compound breaks in completely this should get a touch better.

During the PassMark bench I recorded the temps, voltage and frequency as shown below. Here you can see the full affect of the Kaby Lake stock Turbo in play with the CPU running its max clock of 4.5GHz. 55 degrees recorded with the fans at less than 50%. You can also see the XMP2 profile in affect with the RAM clock at 3000MHz.

I recorded power draw at idle and under load with a heavy graphics bench, where the PC will draw its highest wattage with the GPU thirstily slurping juice. At its lowest observed reading with absolutely nothing happening, I recorded 41W at the wall. Very efficient.

Under the heavy graphics bench, the highest reading I observed was 244w at the wall. As it stands, given my components, their power draw and my hungriest graphical workload, my 550W PSU is more than sufficient for this build.

Build Summary

All in all this is a pretty exciting year to build a high-end PC with loads of improvements available in graphics and disk.

Motherboard	Asus Z270M-Plus	$150
CPU	Intel Core i7-7700K	$379
CPU Cooler	Noctua NH-U12S	(reuse)
Memory	G.Skill TridentZ	$200
OS NVME Apps NVMe Data HDD	Samsung 960 Pro Samsung 960 Evo Seagate Baracuda	$330 $250 $130
Graphics	MSI GTX 1070 Gaming X	$399
PSU	Silverstone SST-ST55F-G	(reuse)
Fans	Noctua NF-F12 PWM NF-A14 PWM	(reuse)
Case	BitFenix Phenom M	(reuse)
OS	Window10 Pro
Monitors	2 x Dell U2415b	(reuse)
Mouse & Keyboard	Corsair M65/ K70	(reuse)
Total invested		$1838

Dell EMC ScaleIO - Scale and Performance for Citrix XenServer

noreply@blogger.com (Peter) — Fri, 23 Dec 2016 18:22:00 +0000

You'll be hearing a lot more about Dell EMC ScaleIO in the coming months and years. ScaleIo is a Software Defined Storage solution from Dell EMC that boasts massive flexibility, scalability and performance potential. It can run on nearly any hardware configuration from any vendor, all-flash or spinning disk, as well as supports XenServer, Linux and VMware vSphere. Like most other SDS solutions, ScaleIO requires a minimum of three nodes to build a cluster but unlike most others, ScaleIO can scale to over 1000 nodes in a single contiguous cluster. ScaleIO uses a parallel IO technology that makes active use of all disks in the cluster at all times to process IO. This performance potential increases as you scale and add nodes with additional disks to the cluster. This translates to every VM within the cluster being able to leverage the entire available performance profile of every disk within the cluster. Limits can be imposed, of course, to prevent any single consumer from draining all available IO but the greater potential is immense.

Key Features

Tiering - Unlike most SDS offers available, ScaleIO requires no tiering. For an all-flash configuration, every SSD in every node is completely usable. DAS Cache is required on hybrid models for write-back caching. It should also be noted that for hybrid configurations, each disk type must be assigned to separate volumes. There is no mechanism to move data from SSD to HDD automatically.
Scalability - 3 nodes are required to get started and the cluster can scale to over 1000 nodes.
Flexibility - Software solution that supports all hardware, all major hypervisors. Able to be deployed hyper-converged or storage only.
Data protection - two-copy mirror mesh architecture eliminates single points of failure.
Enterprise features - QOS, thin provisioning, snapshotting.
Performance - Parallel IO architecture eliminates disk bottlenecks by using every disk available in the cluster 100% of the time. The bigger you scale, the more disks you have contributing capacity and parallel IO. This is the ScaleIO killer feature.

ScaleIO Architecture

ScaleIO can be configured in an all-combined Hyper-converged Infrastructure (HCI) model or in a traditional distributed storage model. The HCI variant installs all component roles on every node in the cluster with any being able to host VMs. The distributed storage model installs the server pieces on dedicated infrastructure utilized for presenting storage only. The client consumer pieces are then installed on separate compute-only infrastructure. This model offers the ability to scale storage and compute completely separately if desired. For this post I'll be focusing on the the HCI model using Citrix XenServer. XenServer is a free to use open source hypervisor with a paid support option. It doesn't get much easier than this with XenServer running on each node and XenCenter running on the admin device of your choosing. There is no additional management infrastructure required! ScaleIO is licensed via a simple $/TB model but too can be used in trial mode.

The following are the primary components that comprise the ScaleIO architecture:

ScaleIO Data Client (SDC) - Block device driver that runs on the same server instance as the consuming application, in this case VMs. In the HCI model all nodes will run the SDC.
ScaleIO Data Server (SDS) - Not to be confused with the general industry term SDS, "SDS" in the ScaleIO context is a server role installed on nodes that contribute storage to the cluster. The SDS performs IOs at the request of the SDCs. In the HCI model all nodes will run the SDS role.
Metadata Manager (MDM) - Very important role! The MDM manages the device mappings, volumes, snapshots, storage capacity, errors and failures. MDM also monitors the state of the storage system and initiates rebalances or rebuilds as required.
- The MDM communicates asynchronously with the SDC and SDS services via a separate data path so to not affect their performance.
- ScaleIO requires at least 3 instances of the MDM: Master, Slave and Tie-Breaker.
- A maximum of 5 MDM roles can be installed within a single cluster: 3 x MDMs + 2 x tie-breakers.
- Only one MDM Master is active at any time within a cluster. The other roles are passive unless a failure occurs.
ScaleIO Gateway - This role includes Installation Manager for initial setup as well as the REST Gateway and SNMP trap sender. The Gateway can be installed on the ScaleIO cluster nodes or an external management node.
- If the gateway is installed on a Linux server, it can only be used to deploy ScaleIO to Linux hosts.
- If the gateway is installed on a Windows server, it can be used to deploy ScaleIO to Linux or Windows hosts.
- XenServer, based on CentOS, fully qualifies as a "Linux host".

Once the ScaleIO cluster is configured, all disks on each host are assigned to the SDS local to that host. Finally volumes are created and mounted as consumable to applications within the cluster.

The diagram below illustrates the relationship of the SDC and SDS roles in the HCI configuration:

My Lab Environment:

3 x Dell PowerEdge R630
- Dual Intel E5-2698v4
- 512GB RAM
- 480GB Boot SSD
- 4 x 400GB SSDs
- 5 x 1TB (7.2K RPM)
- 2 x 10Gb NICs
XenServer 6.5
ScaleIO 2.0
XenDesktop 7.11
I also used a separate Windows server on an old R610 for my ScaleIO Gateway/ Installation Manager

Here is the high-level architecture of my deployment. Note that there is a single compute volume which is mounted locally on each host via the XenServer Pool, so depicted below as logical on each:

Prep

As of this writing, XenServer 7.0 is shipping but 6.5 is the currently supported version by ScaleIO. The deployment steps for 7.0 should be very similar once officially supported. First install XenServer on each node, install XenCenter on your PC or management server, create a new pool, add all nodes to XenCenter, create a pool and fully patch each node with the XenCenter integrated utility. If the disks installed in your nodes have any prior formatting this needs to be removed in the PERC BIOS or via the fdisk utility within XenServer.

Now would be a good time to increase the memory available to Dom0 to the maximum 4GB, especially if you plan to run more than 50 VMs per node. From an SSH session or local command shell, execute:

/opt/xensource/libexec/xen-cmdline --set-xen dom0_mem=4096M,max:4096M

Install the packages required for ScaleIO on each node: numactl and libaio. OpenSSL needs to be updated to 1.0.1 using the XenCenter update utility via Hotfix XS65ESP1022.

Libaio should already be present but before numactl can be added, the repositories will need to be edited. Open the base repository configuration file:

vi /etc/yum.repos.d/CentOS-Base.repo

Enable the Base and released updates repositories changing "enabled=0" to "enabled=1". Save the file, :wq

Next install numactl, libaio should report as already installed, nothing to do. Repeat this on each node.

yum install numactl

ScaleIO Installation

Download all the required ScaleIO files: Gateway for Windows or Linux, as well as all the installation packages for ScaleIO (SDC, SDS, MDM, xcache, and lia). Install the ScaleIO Gateway files, either for Windows to do an external remote deployment, or for Linux to install the gateway on one of the XenServer nodes. For this installation I used an external Windows server to conduct my deployment. The ScaleIO Gateway installation for Windows has two prerequisites which must be installed first:

Java JRE
Microsoft Visual C++ 2010 x64 Redist

Next run the gateway MSI which will create a local web instance used for the remainder of the setup process. Once complete, in the local browser connect to https://localhost/login.jsp, and login using admin plus the password you specified during the Gateway setup.

Once logged in, browse to and upload the XenServer installation packages to the Installation Manager (installed with the Gateway).

Here you can see that I have the ScaleIO installation packages for Xen 6.5 specifically.

Once ready to install, you will be presented with options to either upload an installation CSV file, easier for large deployments, or if just getting started you can select the installation wizard for 3 or 5-node clusters. I will be selecting the wizard for 3-nodes.

Specify the passwords for MDM and LIA, accept the EULA, then enter IP addresses and passwords for the mandatory three MDM instances at the bottom. The IP addresses at the bottom should be those of your physical XenServer nodes. Once all information is properly entered, the Start Installation button will become clickable. If you did not install the 1.0.1 OpenSSL patch earlier, this step will FAIL.

Several phases will follow (query, upload, install, configure) which should be initiated and monitored from the Monitor tab. You will be prompted to start the following phase assuming there were no failures during the current phase. Once all steps complete successfully, the operation will report as successful and can be marked as complete.

ScaleIO Configuration

Next install then open the ScaleIO GUI on your mgmt server or workstation and connect to master MDM node configured previously. The rest of the configuration steps will be carried out here.

First thing, from the Backend tab, rename your default system and Protection Domain names to something of your choosing. Then create a new storage pool or rename the default pool.

I'm naming my pools based on the disk media, flash and spinning. Create the pool within the Protection Domain, give it a name and select the caching options as appropriate.

Before we can build volumes we need to assign each disk to each SDS instance running on each node. First identify the disk device names on each host via SSH or local command shell by running:

fdisk -l

Each device will be listed as /dev/sdx. Right-click the first SDS on the first node and select Add Device. In the dialog that follows, add each local disk on this node, assign it to the appropriate pool and name it something meaningfully unique. If your disk add operation fails here, you probably have previous formatting on your disks which needs to be removed first using fdisk or the PERC BIOS! You can add both SSD and HDD devices as part of this operation.

Once this has been successfully completed for each node, you will see all disks assigned to each SDS in the GUI along with the total capacity contributed by each. You'll notice that the circular disk icons next to each line are empty, because so are the disks at the moment.

Now keep in mind that ScaleIO is a mirror mesh architecture, so only half of that 4.4TB capacity is usable. From the Frontend tab, right-click the Storage Pool and create a new volume with this in mind. I'm using thick provisioning here for the sake of simplicity.

Once the volume is created, map it to each SDC which will make it available as a new disk device on each node. Right-click the volume you just created and select Map Volumes. Select all hosts then map volumes.

If you run fdisk -l now, you will see a new device called /dev/scinia. The final step required before we can use our new storage is mounting this new device on each host which only needs to be done once if your hosts are configured in a Pool. By default the XenServer LVM is filtering so does not see our ScaleIO devices called "scini". Edit the lvm configuration file and add this device type as highlighted below. Pay particular care to the text formatting here or LVM will continue to ignore your new volumes.

vi /etc/lvm/lvm.conf

Next confirm that LVM can see the new ScaleIO device on each node, run:

lvmdiskscan

Now it's time to create the XenServer Storage Repository (SR). Identify the UUIDs of the ScaleIO volumes presented to a host and identify the UUID of the host itself, just pick a host from the pool to work with. You will need the output of both of these commands when we create the SR.

ls -l /dev/disk/by-id | scini

xe host-list

This next part is critical! Because ScaleIO presents storage as a block device, specifically as an LVM, all thin provisioning support will be absent. This means that all VMs deployed in this environment will be full thick clones only. You can thin provision on the ScaleIO backend but all blocks must be allocated to your provisioned VMs. This may or may not work for your scenario but for those of you paying attention, this knowledge is your reward. :-)

Another very important consideration is that you need to change the "name-label" value for every SR you create. XenServer will allow you to create duplicate entries leaving you to guess which is which later! Change the portions in red below to match your environment.

xe sr-create content-type="ScaleIO" host-uuid=8ce515b8-bd42-4cac-9f76-b6456501ad12 type=LVM device-config:device=/dev/disk/by-id/scsi-emc-vol-6f560f045964776a-d35d155b00000000 shared=true name-label="SIO_FlashVol"

This command will give no response if successful, only an error if it wasn't. Verify that the volumes are now present by looking at the Pool storage tab in XenCenter. Creating this SR on one host will enable access to all within the pool.

Now the disk resources are usable and we can confirm on the ScaleIO side of things by going back to the dashboard in the ScaleIO GUI. Here we can see the total amount of raw storage in this cluster, 15.3TB, the spare amount is listed in blue (4.1TB), the thin yellow band is the volume capacity with the thicker green band behind it showing the protected capacity. Because I'm using thick volumes here, I only have 1.4TB unused. The rest of the information displayed here should be self-explanatory.

ScaleIO & XenDesktop

Since we're talking about Citrix here, I would be remiss if I didn't mention how XenDesktop works in a ScaleIO environment. The setup is fairly straight-forward and since we're using XenServer as the hypervisor, there is no middle management layer like vCenter or SCVMM. XenDesktop talks directly to XenServer for provisioning, which is awesome.

If running a hybrid configuration, you can separate OS and Temporary files from PvDs, if desired. Here I've placed my PvD's on my less expensive spinning volume. It's important to note that Intellicache will not work with this architecture!

Select the VM network which should be a bond of at least 2 NICs that you created within your XenServer Pool. If you haven't done this yet now would be a good time.

Finish the setup process then build a machine catalog using a gold master VM that you configured and shut down previously. I've configured this catalog for 100 VMs with 2GB RAM and 10GB disk cache. The gold master VM has a disk size of 24GB. Here you can see my catalog being built and VMs are being created on each host in my pool.

This is where things get a bit...sticky. Because ScaleIO is a block device, XenServer does not support the use of thin provisioning, as mentioned earlier. What this means is that the disk saving benefits of non-persistent VDI will be absent as well. Each disk image, PvD and temporary storage allocated to each VM in your catalog will consume the full allotment of its assignment. This may or may not be a deal breaker for you. The only way to get thin provisioning within XenServer & XenDesktop is by using shared storage presented as NFS or volumes presented as EXT3, which in this mix of ingredients, applies to local disk only. In short, if you choose to deploy VDI on ScaleIO using XenServer, you will have to use thick clones for all VMs.

Performance

Lastly, a quick note on performance, which I tested using the Diskspd utility from Microsoft. You can get very granular with diskspd to model your workload, if you know the characteristics. I ran the following command to model 4K blocks weighted at 70% random writes using 4 threads, cache enabled, with latency captured.

Diskspd.exe -b4k -d60 -L -o2 -t4 -r -w70 -c500M c:\io.dat

Here's the output of that command real time from the ScaleIO viewpoint to illustrate performance against the SSD tier. Keep in mind this is all being generated from a single VM, all disks in the entire cluster are active. You might notice the capacity layout is different in the screenshot below, I reconfigured my protection domain to use thin provisioning vs thick when I ran this test.

Here is a look at the ScaleIO management component resource consumption during a diskspd test against the spinning tier generated from a single VM. Clearly the SDS role is the busiest and this will be the case across all nodes in the cluster. The SDC doesn't generate enough load to make it into Top plus it changes its PID every second.

Closing

There are a lot of awesome HCI options in the market right now, many from Dell EMC. The questions you should be asking are: Does the HCI solution support the hypervisor I want to run? Does the HCI solution provide flexibility of hardware I want to run? And, is the HCI solution cost effective? ScaleIO might be perfect for your use case or maybe one of our solutions based on Nutanix or VSAN might be better suited. ScaleIO and XenServer can provide a massively scalable, massively flexible solution at a massively competitive price point. Keep in mind the rules on usable disk and thin provisioning when you go through your sizing exercise.

Resources

XenServer Hotfixes

Configure Dom0 memory

ScaleIO Installation Guide

ScaleIO User Guide

Exposing WiFi Passwords in Windows10

noreply@blogger.com (Peter) — Sat, 26 Nov 2016 20:20:00 +0000

With the holidays here and folks traveling around, this tip might prove useful. Your mother-in-law can't find the custom WiFi password you set on her router, so she calls you for help as she wants to get her friend connected to her network. You can easily expose the password of the network you are currently connected to via the Network Connections GUI. Hit Start--> type ncpa.cpl and hit enter. This will open the Network Connections dialog. Right-click the active WiFi connection and click Status. Next click the Wireless Properties button. Within the window that appears next, click the Security tab, then click "Show characters" as highlighted below. This will display the password of the network you are currently connected to.

Great, that reveals my current network but what about my mother-in-law's network at her house? Since I have connected to her network previously on this same PC, I can reveal the password using CMD or PowerShell. Open either a CMD or PowerShell window in Admin mode (right-click the start button, look about half way down). Enter the following command first to reveal the SSID names of all saved networks on this PC.

 netsh wlan show profiles

In this particular example I'm interested in exposing the password for the network called "house" as highlighted.

Next, run the following command to reveal the saved password for this network.

netsh wlan show profile name="house" key=clear

This will expose all settings for this wireless network including the password in clear text as highlighted below.

Deploying VMware VSAN 6.2 with vCenter Integrated - Part 4

noreply@blogger.com (Peter) — Fri, 07 Oct 2016 19:53:00 +0000

Part 1 - Architecture, Prep & Cluster Deployment

Part 2 - vCenter Deployment and Configuration

Part 3 - Network Configuration

Part 4 - Troubleshooting: things that can go wrong (you are here)

I love technology and HCI is awesome but things that can still go wrong. Here are a few items that you hopefully won't run into, but if you do here's how to deal with them. I broke and rebuilt this environment a couple of times to see where some of the pitfalls are.

Disk Group Trouble

I had a small bit of disk weirdness with my original host reporting that one of my disk groups was unhealthy with 6 absent disks, immediately following the build of my HA cluster. Interestingly, the SSD in this DG was recognized and online but all the other disks were phantoms created somehow when I moved this host into the cluster. The two real HDDs were unassigned and waiting idly (disks in use 10 of 12). Notice the second disk group is healthy and really underscores the value of running at least two DGs per host which I highly recommend as a minimum. Don't allow a disk or DG failure to generate a node failure event! The neat thing about VSAN is that you can add multiple disk groups which will increase capacity, increase performance and spread your failure domain wider.

This should hopefully be a rare occurrence but to resolve, I first vMotion'd my VCSA to another node. Enable vMotion on your existing vmk or create another, if you haven't already. Then select an absent disk and remove it from the cluster. At this point I had no data to migrate so had no need for evacuation.

Once I removed the last absent disk, vSphere removed the disk group which now only had a single SSD. Start the Create Disk Group dialog and add the second DG with all appropriate disks now accounted for.

Now VSAN is happy with all DGs healthy on all hosts and I had zero service disruption to make this adjustment.

I rebuilt this environment to try and recreate this problem but did not see this issue the second time through.

VCSA Stuck on VSS after vDS Migration

Attempting a migration of hosts + VCSA to vDS at the same time left VCSA stuck on VSS with no uplinks. I tried this one just to see if it would work, it didn't. The VCSA migration method I laid out in part 2 of this series is the way to go. In this scenario the VCSA stayed on the local VSS because it had a vNIC attached to the local port group. Below you can see that the VSS no longer had any vmnics which of course took down vCenter.

To remedy this you need to first manually remove a vmnic from the new vDS on the VCSA's host. First identify the port ID of the vmnic you would like to move.

esxcfg-vswitch -l

Remove that vmnic from the vDS.

esxcfg-vswitch -Q vmnic -V 17 dvSwitch

This vmnic should now be unassigned from the vDS and assignable to the local VSS. The vmnic can be claimed via the web host client or the desktop client.

Before the VSS would pass traffic I had to remove the vmnic and then add it back to the host (yep, 2 times). Now your VSS should have an uplink again which means vCenter should be accessible.

Change the vCenter vNIC to connect to a DPortGroup on the new vDS. Once this is complete you can safely migrate the host's vmnic back over to the vDS again.

Storage Providers out of Sync

If you run into an issue where you are unable to move a host into the VSAN cluster, your storage providers may be out of sync. This is indicated by a UUID mismatch error and failed node move operation.

To resolve, from the HA cluster object click Related Objects, Datastores, then right click the vsanDatastore and select Manage Storage Providers.

Select the icon with the red circular arrows to get all nodes in sync and retry your node add operation:

Restoring ESXi Hosts so you can Rebuild VSAN

Things went sideways for you for whatever reason and you need to go back to square 1.

You can do this (a) the careful way or (b) the launch the nukes way

(a) remove a vmnic from the host using the esxcfg-vswitch command, create a new VSS, add the vmnic, add the VMK to the VSS with a good IP. Repeat on each host.

(b) just delete the vDS which will destroy all your networking, VMKs included and will shut down all VMs. Log into iDRAC, launch the console, F2 for settings, Network Restore Options, Restore Network Settings. This will rebuild your VSS and put your mgmt VMK there with the last known IP config. Overall this way will get you on the path quicker.

Delete the VCSA VM and vsanDatastore, start over.

Part 1 - Architecture, Prep & Cluster Deployment

Part 2 - vCenter Deployment and Configuration

Part 3 - Network Configuration

Part 4 - Troubleshooting: things that can go wrong (you are here)

Resources:

Bootstrap vCenter for VSAN 5.5 (Virtually Ghetto)

Enable SSD option on disks not detected as SSDs

ESXCLI VSAN commands

VCSA Resource Requirements

Change vSphere Web Client session timeout

VMware Compatibility Guide

vSwitch command line configuration

Deploying VMware VSAN 6.2 with vCenter Integrated - Part 3

noreply@blogger.com (Peter) — Wed, 05 Oct 2016 23:04:00 +0000

Part 1 - Architecture, Prep & Cluster Deployment

Part 2 - vCenter Deployment and Configuration

Part 3 - Network Configuration (you are here)

Part 4 - Troubleshooting: things that can go wrong

Network Configuration

My new three node VSAN cluster is humming along just fine on the default VSS but to take this to the next level, I'm going to migrate the environment to a Virtual Distributed Switch (vDS). The desired end state architecture should look like the image below once complete.

First step create and name the new DSwitch.

Select the number of uplinks required. For my purposes I'll be using two uplinks with all vmnics balanced between them to provide redundancy across both ports.

Build Distributed Port Groups (DPG) for each network or service grouping you require, including for the purpose of assigning specific IPs in different VLANs to VMKs.

Add hosts to the DSwitch and migrate the networking, this can be done one by one or as a group by using template mode. Be very careful automating any kind of VCSA migration at this stage!

Select which tasks you want to complete as part of this operation. Do NOT migrate the VCSA as part of this task, strongly recommend you do that via a separate step.

Assign physical NICs (vmnics) to the uplink port group DVUplinks. Make sure that your vmnics are balanced across the DVUplinks. Notice below that I'm migrating vmnic0 which is currently assigned to the local VSS.

Migrate the VMkernel adapters to the new DSwitch. Here I'll be moving VMK0 on the local VSS to the new DPG DPortGroupMgmt.

Evaluate the impact and execute the operation. Once complete we can see that VMK0 has successfully migrated along with the host's vmnics, to the new DSwitch topology.

Repeat this step on the remaining hosts that are not hosting the VCSA! For the host running the VCSA, migrate only one vmnic to the DSwitch, leaving the other vmnic active on the local VSS. This is very important, if you deprive the VCSA of network connectivity, bad things happen. Here I'll be assigning only the vmnic currently unused by the host, vmnic0 attached to vSwitch0 must remain intact right now.

As an additional precaution, don't migrate VMK0 of this host yet, we need to first configure the VCSA with a vNIC attached to the DSwitch. As long as the IP and VLAN will be the same you can make this change hot with no disruption.

Now it's safe to migrate vmk0 and the remaining vmnic attached to the VSS vSwitch0 to the new DSwitch.

Here is the updated topology view with all host vmnics, VMKs and the VCSA migrated to the new DSwitch.

Next create additional VMKs on DPortGroups to isolate services like vMotion and VSAN if desired which can be done at the host level or via the DSwitch host manage dialogs.

If you will be turning up new VMKs for VSAN, make sure to have the new VMKs on all hosts configured and communicating before disabling this service on the old VMKs!

Part 1 - Architecture, Prep & Cluster Deployment

Part 2 - vCenter Deployment and Configuration

Part 3 - Network Configuration (you are here)

Part 4 - Troubleshooting: things that can go wrong

Resources:

Bootstrap vCenter for VSAN 5.5 (Virtually Ghetto)

Enable SSD option on disks not detected as SSDs

ESXCLI VSAN commands

VCSA Resource Requirements

Change vSphere Web Client session timeout

VMware Compatibility Guide

vSwitch command line configuration

Deploying VMware VSAN 6.2 with vCenter Integrated - Part 2

noreply@blogger.com (Peter) — Wed, 05 Oct 2016 16:45:00 +0000

Part 1 - Architecture, Prep & Cluster Deployment

Part 2 - vCenter Deployment and Configuration (you are here)

Part 3 - Network Configuration

Part 4 - Troubleshooting: things that can go wrong

Deploy vCenter

In case you hadn't seen it yet, deploying vCenter has changed a bit in 6.x. The Windows/ SQL version of vCenter is still available but under most circumstances it is cheaper and much easier to deploy the vCenter Server Appliance (VCSA). To get started, mount the ISO, install the client integration plugin then launch vcsa-setup.html.

Follow the prompts and deploy your VCSA on the first VSAN host just created. For this deployment I'm using the embedded Platform Services Controller and creating a new SSO domain. Size your VCSA based on the number of anticipated hosts and VMs to be supported in this cluster. As before, you can opt to use the internal PostgreSQL database or connect to Oracle externally. If you need to use SQL for the database, you will need to deploy the Windows version of vCenter.

Enabled Thin Disk Mode on the VSAN datastore if you're concerted about VCSA space consumption and select the VSAN datastore as the location for the VCSA deployment.

Select a VM network available on the ESXi host which will be a VMware Standard Switch (VSS) at this point, assign a static IP to the VCSA and complete the deployment. The installer will download and deploy the appliance.

WARNING - Make sure to manually create a DNS A record for your VCSA or you will get a first boot error when the appliance is unable to resolve its hostname. If this happens you will need to fix the DNS issue then redeploy the VCSA!! You can also avoid this by using the static IP in the system name field instead of a FQDN.

Once the deployment is complete, login to vCenter using the web client with the SSO account you setup previously: administrator@pfine.local. Create a new Datacenter grouping, add your ESXi hosts to vCenter, add your license keys then assign them to your vCenter and ESXi instances.

Active Directory Integration

If desired, join your VCSA to your AD domain for single sign-on. In the web client from Home, navigate to Administration --> System Configuration. Select the Nodes link, click your VCSA instance, click the Manage tab, click Active Directory under settings then click the Join button and enter the relevant details. Reboot the VCSA to enact the changes.

Login again to the Web Client and navigate to the Configuration page, choose the Identity Sources tab and add AD with your domain as the source type.

Finally add AD users or groups and assign them to vCenter roles. You can now log into vCenter with your domain credentials.

Build the HA Cluster

Before we build a new HA Cluster object, we first need to enable VSAN on the default management VMK0 which lives on the default VSS. Repeat this on each host that will join the cluster.

Right-click the datacenter object and invoke the New Cluster dialog. Give it a name, enable DRS, HA and VSAN. Make sure to leave the VSAN disk mode to automatic here. If you have a single caching SSD VSAN will build one disk group (DG), if you have two caching SSDs VSAN will build two DGs and spread the available capacity disks evenly amongst them. Select your desired automation and admission control levels.

Very important: Add the host running the VCSA to the new cluster first so it becomes the master host contributing its existing VSAN datastore! If you add one of the other hosts first, then a net new VSAN datastore would be created and make it difficult to add the pre-existing VSAN datastore later. Add each remaining host to the new HA cluster by dragging them in.

Assign a VSAN license to your cluster and double-check that all disks in the cluster are running the same On-disk format version. Found on the HA Cluster object/ Manage/ Virtual SAN/ General, update any that are outdated.

As an optional but recommended step, turn on the VSAN performance service from the Health and Performance page of the same tab. This will give you more historical performance information for the cluster.

Make sure to also update the VSAN HCL database and test your deployment for issues. This can be found on the Monitor tab under Virtual SAN on the Health page. You'll notice below that I have a few yellow warnings relating to my H310 storage controller. In this particular instance, I'm using a newer driver than what is actually certified for the HCL (06.805.56.00 vs 06.803.73.00). Very VERY important that whatever you deploy is fully certified via the VMware Compatibility Guide.

Now the VSAN cluster should be happy and humming along with all nodes contributed storage. Here you can see the dual disk group hybrid configuration of two of my hosts. Note that one of my caching devices failed on .93 so I had to replace it with a smaller 200GB SSD. Just an interesting thing to note here since I have two DGs on one host with slightly differing configurations. VSAN is happy despite.

On the VSAN monitoring tab we can see how the physical disks have been allocated and clicking around reveals which VM objects are stored on which disks including current consumption.

Here is a look at the capacity reporting of the cluster currently only hosting my VCSA with thick disks. Notice that dedupe and compression are unavailable as this is a hybrid configuration.

In the next part I'll walk through converting this environment from a Virtual Standard Switch to a Distributed vSwitch.

Part 1 - Architecture, Prep & Cluster Deployment

Part 2 - vCenter Deployment and Configuration (you are here)

Part 3 - Network Configuration

Part 4 - Troubleshooting: things that can go wrong

Resources:

Bootstrap vCenter for VSAN 5.5 (Virtually Ghetto)

Enable SSD option on disks not detected as SSDs

ESXCLI VSAN commands

VCSA Resource Requirements

Change vSphere Web Client session timeout

VMware Compatibility Guide

vSwitch command line configuration

Deploying VMware VSAN 6.2 with vCenter Integrated - Part 1

noreply@blogger.com (Peter) — Wed, 05 Oct 2016 16:29:00 +0000

Part 1 - Architecture, Prep & Cluster Deployment (you are here)

Part 2 - vCenter Deployment and Configuration

Part 3 - Network Configuration

Part 4 - Troubleshooting: things that can go wrong

Deploying VMware Virtual SAN (VSAN) into a greenfield environment can be done a couple of ways. The easiest of which would be to deploy a vCenter Server first on separate infrastructure, deploy the ESXi hosts and then build the cluster. But what if you want to deploy vCenter such that is resides on the shared datastore you intend to create with VSAN and live within the supporting hosts? This is called bootstrapping vCenter within VSAN and was previously covered by William Lam for a single node deployment on vSphere 5.5. The concept is similar here but I'll be deploying a full 3-node cluster, using vSphere 6.2 and configuring a two disk group hybrid config. VSAN being a kernel-level service within ESXi can be configured without vCenter on a single node. vCenter of course is required for multi-node clustering, licensing and management of the HA cluster, but the value here is that we can deploy VSAN first, then add vCenter to the newly created VSAN datastore without having to move things around after the fact.

Architecture

The basic tenets of the VSAN architecture are relatively simple: an ESXi kernel-level service enabled via VMK port, managed by vCenter, running on each node in a cluster whom contribute local disks to form a distributed datastore, accessible entirely via the network that connects them. VSAN uses the concept of Disk Groups (DG) to organize storage which are a collection of cache and capacity devices that can be all flash or a mix of flash and spinning disk (hybrid). One cache device is allowed per DG and I strongly recommend using at least 2 x DGs for resiliency as well as increased performance in all configurations. Caching behavior differs depending on the model deployed, hybrid uses 30% of the cache device for writes while all flash dedicates 100% of the cache device to writes. The basic rule of VSAN sizing is that cache should be sized based on 10% of anticipated consumed capacity (in VMDKs) before failures tolerated are considered. In other words, make sure your cache SSD is big enough to handle the capacity disks you put behind it, by at least 10%, per disk group. 10Gb networking is recommended and required for all flash configurations.

Policy plays an important role in VSAN which provides a great deal of configurability but also dictates the most important single policy element: Failures To Tolerate (FTT). FTT defaults to a value of 1 which means that every VM will have one replica of its data across the cluster. The maximum value is 3 but each replica created has an impact on available usable disk capacity, plan accordingly.

For more in-depth info and some light bedtime reading, check out the very good Virtual SAN 6.2 Design Guide.

My Environment:

3 x PowerEdge R720xd

2 x E5-2650v2 CPUs
384GB RAM
2 x 160GB SSDs (Boot)
2 x 400GB SSDs (Caching)
4 x 1TB HDDs (Capacity)
2 x 10Gb NICs
vSphere 6 Update 2

ESXi 6.0.0, 3620759
vCenter Server 6.0 Update 2 Appliance

Here is the architecture of the cluster I will deploy via this exercise. Even though I'm using 12G PowerEdge servers here, these steps should be very similar on 13G platforms.

Prep

Very important, make sure all applicable hardware components are on the VMware VSAN Certified List! First make sure that all the disks to be used in the VSAN cluster are in non-RAID pass-through mode, assuming your storage controller supports this. If using a supported Dell PERC controller, this should be the default. Conversion of each disk may be necessary if rebuilding from a previous configuration which is performed on the PD Mgmt tab.

If you don't see the option to "convert to non-RAID", first select the "Factory Default" option on the Ctrl Mgmt tab. You should then be able to convert all disks to non-RAID if required or they will default to this. Repeat this process on all hosts.

Install ESXi 6 on each node and enable the ESXi Shell or SSH, whichever you prefer, via the Troubleshooting Options menu of the Direct Console. Enter Alt+F1 at the home screen of the Direct Console UI to log into the ESXi Shell, to exit the ESXi Shell, press Alt+F2.

Verify that the disks intended to be used for VSAN are visible to the host and take note of the device names (naa.xx) as you will need these in a moment to build the DG. Below you can see the devices from the host client as well as within ESXi Shell running the command:

esxcli storage core device list

If using pass-through disks, the disks should be properly identified as SSD or HDD with a slew of additional information available. If using RAID0 disks, much less information will be visible here.

By default the VSAN policy should be set at a host Failure To Tolerate (FTT) of 1 for all classes with force provisioning set on vswap and vmem. This policy is present on a fresh ESXi host with no vCenter management. Force provisioning allows VSAN to violate the FTT policy which we need when building out this initial cluster on a single node, so we need to add this policy value to vdisk and vmnamespace policy classes.

Verify the VSAN policy defaults:

esxcli vsan policy getdefault

Enable force provisioning for vdisk and vmnamespace. Take note of the case sensitivity here, these commands will fail silently if case is incorrect.

esxcli vsan policy setdefault -c vdisk -p "((\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i1))"
esxcli vsan policy setdefault -c vmnamespace -p "((\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i1))"

Recheck the policy to ensure proper adhesion.

Create the VSAN Cluster

VSAN being a kernel-level service can be created without vCenter even being present. Within the ESXi Shell of your first host, run the following command to create the VSAN cluster:

esxcli vsan cluster new

Verify the details of the new cluster. Note that this host is now the Master node for the VSAN cluster:

esxci vsan cluster get

Once the cluster is created, add each disk to the cluster, note that any capacity disks you add here will go into the same disk group, 1 x SSD per DG. If you intend to create multiple disk groups, only add the disks you want present in the first disk group at this stage. -s signifies SSD and -d signifies HDD. Use multiple -s or -d statements within the command to add multiple disks. For my first disk group I'll be adding 1 x SSD (372GB) and 2 x HDDs (931GB)

esxcli vsan storage add -s naa.xx -d naa.xy -d naa.xz

Once complete, run the following to verify that the disks were added properly and assigned to the correct tier. All disks are designated as capacity or not, cache tier SSDs should report false:

esxcli vsan storage list

If you connect to this host using the vSphere desktop client you will see the new datastore listed under storage, it will not be visible in the web host client. Notice that the reported VSAN datastore capacity is based on the capacity tier disks only and represents a raw value (2 x 931GB = 1.8TB).

So at this point we have a fully functional baby VSAN deployment running on one node with a three disk hybrid configuration. In the next part we'll look at deploying and configuring vCenter to take this to the next level.

Part 1 - Architecture, Prep & Cluster Deployment (you are here)

Part 2 - vCenter Deployment and Configuration

Part 3 - Network Configuration

Part 4 - Troubleshooting: things that can go wrong

Resources:

Bootstrap vCenter for VSAN 5.5 (Virtually Ghetto)

Enable SSD option on disks not detected as SSDs

ESXCLI VSAN commands

VCSA Resource Requirements

Change vSphere Web Client session timeout

VMware Compatibility Guide

vSwitch command line configuration

Omega Calibre 9605

noreply@blogger.com (Peter) — Tue, 09 Aug 2016 22:09:00 +0000

I was fortunate enough to invest in a new Omega Seamaster Aqua Terra 150M Co-Axial Chronograph GMT recently and found that there was very little online information not only on this stunning timepiece, but the fantastic Omega in-house created movement, the Calibre 9605, living within. This post intends to shed a little more light on the capabilities of this movement and the Seamaster 231 reference models.

(Image courtesy of Omega)

A Brief History of Time (Keeping)

Omega is one of the most renown luxury watch makers in the world with a deep and storied history beginning in the 1840s. The famed Co-Axial movement, present in all Omega automatic watches, was brought to market in 1999 when Omega worked with genius watch-maker George Daniels who created the first new mechanical escapement the world had seen in 250 years. Many mechanical watches up to that point, and still to this day (Rolex), use the traditional lever escapement which make use of a lever with two ruby pallet stones pushed by the escape wheel. The escapement in a mechanical watch is what allows the power of the mainspring to be consumed in controlled manner while keeping near perfect time.

In a very simplified example, a tooth of the escape wheel is locked against a pallet stone that has a precise angle cut at the top, slowing the watch's gear train, until pushed to an unlocked position. This action of unlocking an escape wheel tooth sends an energy impulse back to the balance wheel causing it to swing back and forth with its own balance spring. The balance assembly in these movements are referred to as the "regulating organ" and are akin to a swinging pendulum in a pendulum clock. This motion causes the lever to pivot and lock the escape wheel using the opposite stone until pushed open by the next tooth of the escape wheel. This pushing of the balance wheel back and forth results in the frequency of the watch and determines its accuracy, typically 18,000 beats per hour, and is what produces the watch's ticking sound. The traditional lever escapement method requires lubrication of the pallet stones as friction is created when the escape wheel pushes against the angled pallet stones, 18,000 times per hour.

Daniels' design was different in that he used a modified lever fork with three pallet jewels driven by a modified double escape wheel fixed to the same axis (hence "co-axial") that separates the locking and impulse activities (rubies on fork: 1 impulse, 2 locking). This design also uses a modified balance with two additional impulse jewels, one for each direction of rotation. Most of the operation is handled by the larger lower wheel with the upper wheel receiving motion from the torque wheel but also sending a balance impulse during the anti-clockwise balance rotation. As the balance rotates clockwise, the entry pallet (on the far left of the fork) moves to an unlocked position and a tooth on the bottom wheel pushes the impulse jewel attached directly to the balance. When the balance completes its clockwise rotation, it swings anti-clockwise and the exit pallet (on the far right of the fork) moves to an unlocked position and a tooth from the top wheel pushes the jewel in the center of the fork. This pushes the lower impulse jewel attached directly to the balance by the end of the fork causing the balance to complete its anti-clockwise motion. The impulses to the balance wheel regulate the frequency of the watch and occur from both the upper and lower wheels depending on the rotational direction of the balance.

The expected result of the co-axial movement is an effective elimination of the sliding friction of the pallet stones over the escape wheel with what should ultimately result in less wear and lengthened service intervals. There is also a reduced need for pallet stone lubrication. Neat design but definitely a good deal more complex than the traditional lever escapement with several additional parts. Time will tell if the co-axial escapement is truly that much better but for now the experts agree that Omega has created something special and innovatively differentiated. Here's a great video from Omega detailing exactly how their Co-Axial movement functions: https://www.youtube.com/watch?v=2ID9bGj_gtY

(base image courtesy of Omega - I added labels)

Calibre 9605 Movement

Omega's Co-Axial calibre has evolved from what began as the 2500 to a fully in-house developed premium movement with several models and complications. The 9300 was Omega's first fully in-house made movement that would change the brand forever. Proving you can do it all under one roof is a big deal for a watch maker and as a consumer looking to invest in a premium timepiece. Omega boasts a few additional innovations which separate their watches from the competition. Omega features a free sprung balance that uses silicon for the balance wheel and balance spring which as a material is incredibly durable and resistant to magnetism. Magnets are the enemy of a mechanical watch and Omega has taken steps to reduce their effects, even more so with the new Master Co-Axial designation via the Swiss METAS certification. The 9605 is a COSC certified chronometer which is a label given to less than 3% of Swiss watches designating that a timepiece is made of the highest quality components and rigorously precision tested. Below is an image of the silicon balance wheel (Si14), balance spring and co-axial escapement.

This movement uses a twin barrel mainspring with a slipping clutch in the second barrel to prevent over-winding and is able to provide a 60 hour power reserve drawing from both barrels. This design aspect is also intended to prevent loss of accuracy over time due to any decreased torque when the watch is operating on its power reserves. The 9605 is also outfitted with a traditional column wheel for chronographic functions (start, stop, reset). The weighted rotor on the outermost portion of the movement drives the automatic winding mechanism and can wind the mainspring turning in either direction. The image below is the back of my Seamaster AT showing the balance bridge, 2nd barrel, column wheel and weighted rotor of the 9605 movement, stunningly finished in rhodium plating styled with "Geneva waves in arabesque." Fantastic doesn't begin to describe this remarkable piece.

Calibre 9605 Functions

The 9605 movement has a fancier twin called the 9615. The movements are identical in function but the 9615 uses gold for some pieces of the movement visible through the transparent case back. The 9605 features four complications in addition to the basic time function: chronograph, date, GMT, 12 hour and 60 minute recorders. This is an incredible tool for traveling! The GMT hand lives with the small seconds hand at 9:00. This small seconds hand is the primary seconds hand for the watch and will always be spinning when the watch is wound. For small seconds the 24 marker of the dial = 60 and the 12 marker = 30. The The sweep seconds hand on the main dial is controlled by the upper chronograph pusher and can run all the time if desired or function as a stopwatch.

The 12 hour and 60 minutes recorder hands at the 3:00 subdial is also controlled by the chronograph function via the upper chronograph pusher. This 12-hour subdial at 3:00 can be used to track a specific number of hours or minutes but it can also be used when travelling to another time zone. Simply start the chronograph function when the local time is noon or midnight. The local time will now be tracked via this subdial with the main dial representing the time back home. This is the easiest way to track two time zones while travelling. Alternatively the time on the main dial can be adjusted to reflect the local time. The lower pusher is used to reset the sweep seconds and 3:00 subdial only once the chronograph function has been stopped via the upper pusher.

Another useful feature when considering different time zones is the GMT hand in the subdial at 9:00. This hand should always point to GMT/ UTC and is used to calculate the time in another time zone. Simply add or subtract hours based on the offset of the other time zone in question (example, GMT-6). The date is fairly self explanatory but should be noted that Omega uses a gradual change mechanism that will slowly shift the date displayed between the hours of 9pm and 2am each day. Make sure to never manually adjust the date during these hours as damage could occur.

The crown has three positions once unscrewed, pulled directly outward.

The first position is to manually wind the mainspring for extra power. If the mainspring completely unwinds due to lack of use, you will use this position to restart the watch, ~30 turns of the crown. You can also manually wind if you haven't worn the watch for a few days and want to tighten the mainspring. Never wind the watch while on your wrist!
The second position is used to adjust the hour hand on the main dial. When this hand crosses midnight it will change the date.
The third position is used to adjust the minute hand on the main dial. Turning this position will also adjust the GMT hand in the 9:00 subdial. Make sure you are on the 'correct' side of midnight when adjusting the GMT hand.

Resources

Omega Operating Instructions

Omega Co-Axial in action

History of Omega

Native RDS in Server2016 - Part 4 - Scaling & HA

noreply@blogger.com (Peter) — Mon, 04 Jul 2016 02:41:00 +0000

Part 1: The Basics
Part 2: RDVH
Part 3: RDSH
Part 4: Scaling and HA (you are here)

Most environments that I've run across using native RDS for VDI (RDVH) tend to be fairly small, <500 seats but I have seen some larger footprints built around RDSH. The single largest problem for the native RDS solution is management of the environment. This tends to get pretty unwieldy from a management perspective around the 500 user mark using native tools. PowerShell can and should be used in larger environments. The RD Connection Broker (RDCB) itself is capable of 10K concurrent connections so clearly supports scale in and of itself, but it's really unfortunate that the surrounding native management tool stack isn't up to the task and there really isn't much to enable it either. Unidesk can be leveraged to extend the native RDS story (currently Server 2012 R2) providing much better manageability by integrating directly with the RDCB to create desktops and collections. Unidesk provides a good solution that fundamentally alters the deployed architecture using a highly scalable and robust layering model.
The other big consideration with scaling the native RDS stack is MAC address management in Hyper-V. This one is important, especially with compute host densities ever climbing as the semiconductors pump out increasingly core-dense CPUs. By default, Hyper-V supports 256 unique MACs per host. Every Hyper-V host in the world has a 3-octet prefix of 00-15-5D, the next two octets are unique to each host and derived from the IP address assignment, the last octet is auto-generated between 00-FF. The last octet alone is an 8-bit value so represents 256 possible MAC address. You can modify the 4th or 5th octets to increase the pool on a per host basis but be very careful that you don't accidentally assign an overlapping value. In other words, don't mess with this unless you really know what you're doing. Another scenario to avoid is MAC address pool conflicts, which would potentially happen if you deploy a Hyper-V host with a dynamic IP that could be leased to another new Hyper-V server at some point. Very important lesson here is to use static IPs for your Hyper-V hosts.

What about SCVMM?

This question usually comes up in relation to RDS, do you need System Center Virtual Machine Manager (SCVMM), can you use SCVMM, how does it integrate? The Citrix XenDesktop solution requires SCVMM as a component of that architecture for VM provisioning but not so for RDS. In the case of RDS, VMM not only is not required at all but there really isn't a very direct integration path between the products. SCVMM here should be seen as an external management tool to compliment the base set of tools used to manage your Hyper-V Failover Clusters, storage and VMs. So what can you do with VMM in an RDS deployment?
SCVMM can be used as a basic deployment enabler of the environment or a provisioning/ mgmt tool for unmanaged collections, but does not integrate directly with the RDS farm or the RDCB. This means that SCVMM cannot be used to provision any VM intended to exist within a managed pool owned by the RDCB. You can use SCVMM to create VMs for an unmanaged collection or deploy your RDSH VMs while also benefitted from using a much larger pool to manage assignable MAC addresses without worry of conflict or shortage.
To fully appreciate what is possible here it is important to understand the concept of unmanaged and managed collections in RDS. Managed collections are pools that the RDCB creates and maintains using a template VM, including the ability to recreate VMs as needed. Unmanaged collections are pools to which the RDCB brokers connections, but there is no template VM therefore you have to create and manage the pool manually. Everything I've shown so far in this series has been "managed" which is the most common deployment style due to ease of ongoing maintenance. If you want to use SCVMM to manage your desktop pool VMs and take advantage of features like Intelligent Placement and a massive MAC address pool, then you will need to use an unmanaged collection. This model is best suited for a 1:1 persistent desktop deployment and as you can see below, can still make use of UPDs.
For example, in this deployment I have SCVMM 2016 running SQL Server 2014 on a dedicated Server 2016 VM. I wish to deploy and manage a large RDS pool of persistent desktops using SCVMM. The first step is to create an unmanaged collection. This is specified during the creation of the collection by unchecking the "Automatically create and manage virtual desktops" option. Select any additionally desired options and deploy.

Once the collection is created, clone however many VMs required using SCVMM via PowerShell and spread them across the cluster using SCVMM’s Intelligent Placement feature. There is no way in the SCVMM GUI to clone multiple VMs so this operation is scripted, see Resources at the bottom. This method eliminates the concerns about too few or overlapping MAC addresses and balances the VMs across the cluster automatically based upon available capacity. Once the VMs are created, they then need to be manually added to the new unmanaged collection. This can be done using PowerShell or Server Manager. Once this has been done users will be able to see the collection in RD Web Access and the RDCB will be able to broker user connections to the pool. Thousands of VMs could be deployed this way and brokered using RDCB.

Add-RDVirtualDesktopToCollection -CollectionName Name -VirtualDesktopName Clone1 -ConnectionBroker RDCB.domain.com

Alternatively, VMs can be added to the unmanaged pool using Server Manager.

But wait, can't I manually add existing VMs to a managed collection too? Nope! You can add additional VMs to a managed collection but they must be based on the template already assigned to the collection thus ensuring consistency.

Service Templates
The other use case for SCVMM in the RDS context is for deployment and scaling of the surrounding infrastructure using Service Templates. Within SCVMM, one can create a Service Template to deploy entire or individual pieces of an RDS deployment. The Service Template element within SCVMM provides a visual method to build a master script that is used to provision management server VMs using a specific hardware configuration, with specific applications installed, in a specified order of execution. The possibilities here are nearly limitless as you can have at your disposal the full capability of any application, PowerShell or script. Lock down your Service Templates and you could build, rebuild or expand any deployment with the push of a button.

Scaling for Compute

I’ll talk about HA next which inherently brings scalability to the management stack but first, consider compute as part of the architecture. Compute in this context refers to the physical Hyper-V hosts that provide resources for desktop or RDSH VMs, exclusively. The limitations of the compute layer will almost always be based on CPU. It is the one finitely exhaustible resource not easily expanded unless you upgrade the parts. Adjusting resources to provide additional IO, memory or network throughput is a straight-forward process linearly scalable via the capabilities of the server platform. To get the best bang for the buck, most customers seek to deploy the highest reasonable number of users per compute host possible. Hyper-V provides a great deal of CPU efficiency at the expense of slightly higher IO. Depending on the workload and VM profile, one could expect to see 5-10+ VMs per core in an RDS deployment. Compute hosts used for RDSH VMs will require few total VMs per physical host but have the potential to host a much larger number of total users. NUMA architecture alignment is important to ensure maximum performance in these scenarios. A higher number of cores per CPU is generally more important than the clock speed. Considering that it is easy to achieve 256 VMs on a single compute host (default MAC address limit provided by Hyper-V), the appropriate host hardware mix should be selected to ensure the maximum performance and end user experience. Compute hosts can be added to a deployment in block fashion to satisfy a total desired number of users. Keep in mind the nuances of managing a native RDS stack at scale and whether or not it may make sense to invest in 3rd party solutions to bolster your deployment.

Solution High Availability

High-availability can be configured for this solution in a number of different areas. The general principles of N+1 apply at all service levels including physical components. The following guidance will provide a fully redundant RDS infrastructure:

Add Top of Rack switching infrastructure for physical port redundancy
Add Hyper-V compute and mgmt hosts for failover
- Hyper-V hosts configured in a failover cluster to protect physical compute resources also using Cluster Shared Volumes to protect storage (ideally cluster mgmt and compute separately)
Add load balancers to manage SSL offload and HTTPS connections from clients for RD Gateways and RD Web Access servers
Add additional RD Gateway and RD Web Access servers to provide resiliency and redundancy
Add additional RDCB servers configured to connect to a clustered SQL Server instance
Add a 2nd license server VM configured with temporary licenses, deploy both via GPO but list the primary instance first. Should the primary fail, the secondary will serve the environment using temporary entitlements until the primary is restored.
Cluster your file server or add a clustered NAS head back-ended by redundant shared storage for UPDs and user data

Here is another look at the larger architecture but within the context of providing HA:

RDCB HA

The RD Broker itself is the most single important role that needs special consideration and configuration to make it HA. Configuring HA for the RDCB creates a cluster with a DNS name assigned for load balancing that keeps track of the location and assignments of user sessions and desktops. First, create a new database on your SQL server with the RDCB server configured to have dbcreator permissions.

With SQL setup, install the SQL Native Client on all RDCB servers and launch the Configure High Availability wizard from the Deployment Overview.

Choose shared SQL mode, name the clustered RDCB instance and provide the SQL connection string in the following format.

DRIVER=SQL Server Native Client 11.0;SERVER=VMM16.dvs.com; Trusted_Connection=Yes;APP=Remote Desktop Services Connection Broker;DATABASE=RDSQL;

Once successfully completed, the RDCB will show as HA mode in the Deployment Overview and additional brokers can be added using the same dialog.

RDSH HA

An RDSH collection can be scaled or made HA by adding additional RDSH VMs. Once your new RDSH VMs are created and have the appropriate applications installed, they must be added to the "All Servers" management pool within Server Manager.

Once all hosts or VMs are added to the server management pool, you can add the new VMs to your existing RDS deployment.

After additional RDSH servers are added to the overall RDS deployment, they can then be added to a specific session collection from the Host Servers dialog of the collection management page.

Once successfully added to the collection, all added server instances will be visible and available to accept connections.

Load balancing between multiple RDSH instances can be configured from within the collection properties dialog to ensure equal distribution or bias, if desired. Making a server "heavier" via relative weight will cause the Connection Broker to push more connections to it accordingly.

RDWA HA

Additional RD Web Access servers can be added at any time from the Deployment Overview or Deployment Servers dialogs. Select the new server instance you wish to add, confirm and deploy. As always, make sure this server instance is added to the “All Servers” management pool. Behind the scenes IIS is configured on each selected instance to host the website and feed for RDweb.

Once deployed, you should see the new RDWA instance(s) in the RDS Deployment Properties accessible from the RDS Overview page.

Any Collection made to be visible in RD Web Access will be accessible from any RDWA instance. RDWA instances can be accessed directly via URL, load balanced with DNS or put behind a hardware or software load balancer (F5/ Netscaler).

RD Licensing

RD Licensing is one of the trickier roles to make HA as there is no easily exploitable native method to accomplish this. This is generally true regardless of broker solution selected in this space. There are a couple viable methods that require manual intervention that can be used to protect the RD License role. The first method requires two VMs each configured with the RD Licensing role, hosted on separate physical hosts. The first instance has the purchased licenses installed and validated by the Microsoft Clearing House. The second VM is configured with temporary licenses. Both instances are configured via GPO for users but the server with the validated licenses is on the top of the list. Should the primary fail, users can still connect to the environment using temporary licenses until the primary is back online.
The other method also involves two VMs. The primary VM is configured with purchased licenses installed and validated by the Microsoft Clearing House. The VM is cloned, shut down and moved to a separate physical host. Should the primary instance fail for whatever reason, the cold standby can then be powered on to resume the role of the primary. The caveat to this method is that if anything changes from a licensing perspective, the copy to cold stand-by clone process needs to be repeated.

RD Gateway

To optimize and secure connections to the RDS farm from untrusted locations (the interwebs), RDGW can be used and made HA. RDGW terminates SSL for remotely connecting clients, one tunnel for incoming data one for outgoing. UDP can also be utilized for optimized transport of data over WANs using the HTTP transport. RDGW is installed like any other RDS role and includes IIS as a requisite part of the install. RD Gateway Manager is used to manage the configuration and policies of the gateway including SSL certs and transport settings that provide the ability to change HTTP/UDP listeners. RDGW can also use RD Connection Authorization Policies (RD-CAP) which can be stored locally on the RDGW installed or managed centrally on an NPS server. RDGW can be load balanced as a regular HTTP web service including the offloading of SSL termination. DNS Round Robin is not supported and cannot be used in this scenario.

Failover Clustering and RDS

Lastly, a quick point on the role of Failover Clustering in RDS environments. Failover Clustering is recommended to provide HA for the Hyper-V environment and component level protection of the RDS deployment. Should a node fail or require maintenance in a Failover Cluster, it’s VMs will be restarted or evacuated to another node with available capacity. RDS is cluster aware in that it remains aware of the location of VMs within a Failover Cluster, including if they move around, but it does not integrate directly nor make direct use of the Failover Cluster. In this context the resources for the VMs themselves can be protected giving the management VMs a place to migrate or restart should a failure occur. Any storage added directly to the cluster should be converted to a Cluster Shared Volume enabling multiple simultaneous writers to each volume. RDS itself doesn’t care what the underlying storage is nor whether the environment is clustered or not. Remember that any provisioning activities you perform will address RDVH hosts directly with the RDCB providing the ability to select the number of VMs deployed on each host.

Part 1: The Basics
Part 2: RDVH
Part 3: RDSH
Part 4: Scaling and HA (you are here)

Resources

Creating Service Templates for RDS in SCVMM
Deploying multiple VMs from a template in SCVMM (PoSH)
Hyper-V Dynamic MAC addressing for RDS

Native RDS in Server2016 - Part 3 - RDSH

noreply@blogger.com (Peter) — Tue, 07 Jun 2016 22:33:00 +0000

Part 1: The Basics

Part 2: RDVH

Part 3: RDSH (you are here)

Part 4: Scaling and HA

In the first two parts of this series, we explored the basic constructs and architecture of RDS in Server 2016, how to create a farm and how to deploy virtual desktops. In this part we'll look at integrating RDSH into this environment to provide published applications and shared sessions which compliment a virtual desktop deployment. Many actually start with RDSH or published apps as step 1 then move into virtual desktops to provide greater control or performance.

Looking at the overview page of your RDS deployment, you'll notice the session host options are greyed out, this is because the deployment (farm) does not currently have these features installed.

To add RDSH hosts to an existing RDS deployment, you have to first modify the RDS deployment to enable session-based services. Once your RDSH VMs are created and running within Hyper-V, run the Add Roles and Features Wizard and choose session based deployment. You're going to have deja vu here but remember, this is all to modify the existing deployment and this is the last time you should need to run this wizard. The wizard will check for the existence of a broker and web access roles which should already exist. Once you get to the RD Session Host part of the dialog, here is where you add your session hosts to the deployment and install the RDSH role on each. Select your target VMs, confirm and deploy. The VMs receiving the RDSH role will be restarted as part of this process.

Via PowerShell (change to your server names):

New-RDSessionDeployment -ConnectionBroker CBname.domain.com -WebAccessServer WAname.domain.com -SessionHost RDSHname.domain.com

Make sure the new servers are added to the Servers pool within Server Manager. At this point you will be able to create a new session collection from the overview page in Server Manager which will no long be greyed out. Give the collection a name, specify the server(s) you want to host the collection and add the user groups you wish to entitle.

Via PowerShell (change to your server names):

New-RDSessionCollection -CollectionName RDSH -SessionHost RDSH.domain.com -ConnectionBroker CBname.domain.com

If you want to use UPDs to persist user settings between sessions, configure the location and size of the UPDs, confirm you selections and deploy.

Keep in mind that UPD's are collection-specific, they cannot be used in more than one collection.

Via PowerShell (change to your server names):

Set-RDSessionCollectionConfiguration -CollectionName RDSH -ConnectionBroker CBname.domain.com -DiksPath C:\ClusterStorage\Volume1\RDSH_UPD -EnableUserProfileDisk -MaxUserProfileDiskSizeGB 1

RD Licensing

By default, RDS will offer a 120-day grace period for licensing, after which you will need to use an RD License server instance with a proper license to suit your licensing mode. The license server activity is a light weight role and can be safely combined with the RD Connection Broker role. From the Deployment Overview page on the main RDS screen, select Add RD Licensing Server. Choose your broker VM and confirm.

Edit the deployment properties of your RDS Deployment and choose the RD Licensing section. Select your desired licensing mode and RD License server.

To manage your RDS Cals, launch the RD Licensing Manager from the Server Manager Tools menu. Connect to your license server, right-click and select Activate. Complete the information forms and start the Install Licenses Wizard. The wizard will contact the Microsoft ClearingHouse real-time to authorize your server. Microsoft takes licensing of RDSH servers very seriously as there is a tremendous opportunity for software license abuse. Enter the particulars of your license agreement to activate your deployment.

Another useful tool is the RD Licensing Diagnoser which can help identify problems as well as help achieve compliance. This tool can be launched from the Tools/ Remote Desktop Services menu within Server Manager.

Via PowerShell (change to your server names):

Add-RDServer -Server RDLicName.domain.com -Role RDS-Licensing -ConnectionBroker CBname.domain.com

Set-RDLicenseConfiguration -ConnectionBroker CBname.domain.com -LicenseServer RDLicName.domain.com -Mode PerUser

Managing the Collection

Once the collection is created it will become available for management in Server Manager. Entitlements, RD Web visibility, client settings and UPDs can be manipulated via the collection properties. Idle sessions can be force logged off while active sessions can be shadowed, disconnected, messaged or logged off. RemoteApp Programs provides application virtualization for programs running within the RDSH servers of the pool to RD Web Access. Session collections can offer full session desktops or published applications, but not both. Virtual GPU is not supported for session collections, you'll need to look to Citrix XenApp for that.

Once the collection is created you can either publish specific applications or full shared sessions themselves. Individual applications will launch seamlessly and can even be tied to specific file extensions. If you wish to publish a full shared session desktop, this is very easy to do. Simply edit the properties of the collection via the Tasks dropdown on the collection page and ensure that the box is checked to "Show the session collection in RD Web Access." This will provide an icon in RD Web that users can click to launch a full desktop experience on the RDSH server.

To publish specific applications, select "Publish RemoteApp Programs" from the tasks dropdown in the middle of the page. RDS will poll the RDSH servers currently part of your collection for available apps to publish. As the note at the bottom of the initial dialog says, make sure than any program you wish to publish is installed on all RDSH servers within the collection. Confirm your selections and publish.

Via PowerShell (use $variables and script for multiple apps or repeat command for each):

New-RDRemoteApp -CollectionName RDSH -DisplayName "OpenOffice Calc" -FilePath "%SYSTEMDRIVE%\Program Files (x86)\OpenOffice...scalc.exe" -ShowInWebAccess 1

Connecting to Resources

At this point, anything you have published and entitled will be available in RD Web Access. By default this URL as created within IIS will be https://<RD Web Access name>/RDWeb and should be secured using a proper certificate.You can see below that I now have a mix of RDSH published apps, a RemoteApp Program from a desktop collection, a session collection, as well as a desktop collection. What users will see will vary based on their entitlements. To enable direct connection to a collection using the Remote Desktop Connection (RDC), a collection within RD Web can be right clicked which will trigger a download of the .rdp file. This file can be published to users and used to connect directly without having to log into RD Web first.

Editing the .rdp file in a text editor will reveal the embedded characteristics of the connection which include the client settings, connection broker, gateway and most importantly the collection ID which in this case is "RDSH2".

RemoteApp Programs can also be delivered directly to a users' Start menu by connecting the client session to the feed of the RD Web Access Server. This can be configured individually or delivered via GPO. A proper security certificate is required.

When a user initiates a connection to a resource with an unknown publisher for the first time they will be greeted with the following dialog. The resources allowed by the remote host can be restricted via deployment properties or GPOs. The RemoteApp will launch within a seamless session window and to the user will appear to be running locally.

At this point the connection broker does as the name implies and connects users to resources within the deployment while maintaining where VMs are running and which VMs users are logged in to. In the next section we'll take a look at scaling and HA.

Part 1: The Basics

Part 2: RDVH

Part 3: RDSH (you are here)

Part 4: Scaling and HA

Resources:

RDS Cmdlets for Server 2016

Native RDS in Server2016 - Part 2 - RDVH

noreply@blogger.com (Peter) — Thu, 02 Jun 2016 04:04:00 +0000

Part 1: The Basics
Part 2: RDVH (you are here)
Part 3: RDSH
Part 4: Scaling and HA

In part 1 of this series we took a look at the over all architecture of RDS in Server 2016 along with the required components contrasting the function performed by each. If you're not new to RDS, things really haven't changed a great deal from Server 2012R2 from a basic architecture perspective. In this chapter we'll take a look at the RDVH role, what it does, how to get going and how to manage it.

Test Environment

Here is a quick rundown of my environment for this effort:

Servers - 2 x Dell PowerEdge R610's, dual X5670 (2.93GHz) CPUs, 96GB RAM, 6 x SAS HDDs in RAID5 each
Storage - EqualLogic PS6100E
Network - 1Gb LAN, 1Gb iSCSI physically separated
Software - Windows Server 2016 TP5 build 14300.rs1
Features - Hyper-V, RDS, Failover Clustering

Installation

The first step is to create an RDS deployment within your environment. Consider this construct to exist as a farm that you will be able to install server roles and resources within. Once the initial RDS deployment is created, you can create and expand collections of resources. An RDS deployment is tied to RD Connection Broker(s) which ultimately constitute the farm and how it is managed. The farm itself does not exist as an explicitly addressable entity. My hosts are configured in a Failover Cluster which is somewhat inconsequential for this part of the series. I'll explain the role clustering plays in part 4 but the primary benefits are being able to use Cluster Shared Volumes, Live Migration and provide service HA.
On one of your management hosts that already has Hyper-V enabled, fire up the Add Roles and Features Wizard, select Remote Desktop Services installation and choose your deployment model. "Standard" would be what is chosen most often here unless doing a POC then "Quick Start" may be more appropriate. MultiPoint is something entirely different which carries a different set of requirements. You don't have to use this wizard but it is an easy way to get going. I'll explain another way in a moment.

Next choose whether you'll be deploying desktop VMs or sessions. Desktops require the RDVH role on the parent, sessions require RDSH and can be enabled within Server VMs. For this portion we'll be focusing on RDVH.

Next select the hosts or VMs to install the RD Connection Broker and Web Access roles. For POCs everything on one host is ok, for production it is recommended to install these RDS roles onto dedicated Server VMs. You'll notice that I pointed the wizard at my Failover Cluster which ultimately resolved to a single host (PFine16A).

The third piece in this wizard is identifying the host(s) to assume the RDVH role. Remember that RDVH goes on the physical compute host with Hyper-V. Confirm your selections and deploy.

The installation will complete and affected hosts or VMs will be restarted. In order to manage your deployment, all Hyper-V hosts and server VM roles that exist within your deployment must be added to the Servers tab within Server Manager. This can be done at the very beginning as well, just make sure it gets done or you will have problems.

At this point your RDS deployment should be available for further configuration with the option to create additional roles or a desktop collection. Unless you are building a new RDS deployment, when adding additional server roles, it is much easier to select the role-based installation method from the Add Roles and Features Wizard and choose the piece-parts that you need. This is especially true if adding singular roles to dedicated server VMs. There is no need to rerun the "Remote Desktop Services Installation" wizard, in fact it may confuse things.

Desktop VM Templates and Collections

Before we create a new collection, we need to have a template VM prepared. This can be a desktop or server OS but the latter requires some additional configuration. By default, the connection broker looks for a desktop OS SKU when allowing a template VM to be provisioned. If you try to use a Server OS as your template you will receive the following error:

To get around this, you must add the following registry key to any host that sources or owns the master template. Reboot and you will be allowed to use a Server OS as a collection template.

HKLM\System\CurrentControlSet\Services\VMHostAgent\Parameters\SkipVmSkuValidation   REG_DWORD     1

Your template should be a normal VM (Gen 1 or 2), fully patched, desired software installed and manually sysprepped. That's right, RDS does not integrate the ability to auto-clone a template VM to new unique VMs, you need to do this manually as part of your collection prep.

Make sure whatever template you intend to provision in your desktop collection is already added to your RDVH host or cluster and powered off.

Launch the Create Collection wizard within Server Manager, give the collection a name and select pooled or personal desktops.

Select the template VM to use to create the collection, if using a Server OS make sure you completed the steps above to make the template VM visible and selectable in the RD Collection wizard. Keep in mind that a template VM can only be used in a single collection, no sharing allowed. Provide an answer file if you have one or use the wizard to generate the settings for time zone and AD OU.

Specify the users to be entitled to this collection and the number of desktops you wish to deploy along with the desired naming convention. For allocation, you can now decide how many VMs of the total pool to provision to each available physical host. Take note of this, VMM is not here to manage Intelligent Placement for you, you can accept the defaults or decide how many VMs to provision to a host.

Select where to store the VMs: this can be a local or shared drive letter, SMB share or CSV. The ability to use CSVs is one of the benefits of using Failover Clustering for RDS deployments. Next, if you intend to use UPDs for storing user settings, specify the path and size you want to allocate to each user. Again, CSVs work fine here. Confirm and deploy.

Behind the scenes, RDS is making a replica of your template VM's VHDX stored in the volume you specified, which it will then clone to the collection as new VMs.

The pooled VMs themselves are thin provisioned, snapped (checkpointed) and only consume a fraction of the storage consumed by the original template. When reverting back to a pristine state at log off, the checkpoint created during provisioning is applied and any changes to the desktop OS are discarded.

UPDs are ultimately VHDX files that get created for each user that connects to the collection. By default all user profile data is stored within a UPD. This can become unwieldy if your users make heavy use of the Desktop, Documents, Music, etc folders. You can optionally restrict which pieces should be stored within the UPD and use folder redirection for the others if desired.

When UPD is enabled for a collection, there is a template UPD created within the volume specified then each user gets a dedicated UPD with the SID of the user as the file name. Selected settings are saved within and follow the user to any session or desktop they log into within the collection. This file will grow up to the maximum allotted.

Managing the Collection

Depending on the speed of your hardware, your pooled VMs will deploy one by one and register with the broker. Once the collection is deployed it will become available for management in Server Manager. Entitlements, RD Web visibility, client settings and UPDs can be manipulated via the collection properties. Desktop VMs can be manipulated with a basic set of controls for power management, deletion and recomposition. RemoteApp Programs provides a way to publish applications running within the desktop pool to RD Web Access without requiring RDSH or RD Lics which may be useful in some scenarios. Virtual GPU can also be enabled within a desktop collection if the host is equipped with a supported graphics cards. Server 2016 adds huge improvements to the Virtual GPU stack.

Notice here that RDS is aware of which VMs are deployed to which hosts within my cluster. Should a VM move between physical hosts, RDS will see that too. When removing VMs from a collection, always do this from the Collections dialog within Server Manager as it's a cleaner process. This will turn off the VM and remove it from Hyper-V. If you delete a VM from Hyper-V manager, the RDS broker will report that VM as unknown in Server Manager. You will then need to delete it again from within Server Manager.

If you have an AD object that already exists with the same machine name, that VM will fail to provision. Make sure that if you redeploy any collection using the same names that AD is also cleaned up before you do so.

As is always the case when working with VMs in Hyper-V, manual file system cleanup is required. Deleting a VM from Hyper-V or RDS in Server Manager will not remove the files associated with the VM. You could always add them back if you made a mistake but otherwise you will need to manually delete or script an operation to do this for you.

RemoteApp Programs can be used to publish applications for users of a desktop collection. This does not require RDSH but it will only allow a single user to connect to a published resource, just like a desktop VM. If you use RemoteApp Programs within a desktop collection, you will be unable to publish the desktop collection itself. For example, I have Google Chrome published from one desktop within my collection, this prevents the entire collection from being presented in RD Web Access only allowing the published app. To make this collection visible in RD Web Access again, I have to unpublish all RemoteApp Programs from the collection then select to show the collection in RD Web Access. The same is true of session collections. You can publish apps or you can publish sessions but not both simultaneously within a single collection.

Connecting to Resources

Logging into RD Web Access is the easiest way to access published resources. By default this URL as created within IIS will be https://<RD Web Access name>/RDWeb and can be secured using a proper certificate. Depending on user entitlements, users will see published apps, sessions or desktops available for launching. To enable direct connection to a collection using the Remote Desktop Connection (RDC), a collection within RD Web can be right clicked which will trigger a download of the .rdp file. This file can be published to users and used to connect directly without having to log into RD Web first.

Editing the .rdp file in a text editor will reveal the embedded characteristics of the connection which include the client settings, connection broker, gateway and most importantly the collection ID which in this case is "RDC".

At this point the connection broker does as the name implies and connects users to resources within the deployment while maintaining where VMs are running and which VMs users are logged in to. In the next section we'll take a look at integrating RDSH into this environment.

Part 1: The Basics
Part 2: RDVH (you are here)
Part 3: RDSH
Part 4: Scaling and HA

Native RDS in Server2016 - Part 1 - The Basics

noreply@blogger.com (Peter) — Wed, 01 Jun 2016 02:28:00 +0000

Part 1: The Basics (you are here)
Part 2: RDVH
Part 3: RDSH
Part 4: Scaling and HA

As is the case with a number of Microsoft workload technologies, particularly in the End User Compute (EUC) space, Microsoft provides you with the basic tools to fulfill a particular requirement or use case. If you want to get more granular with management or fancier with features, your next step would be to invest into their partner ecosystem to expand upon the base offering. Such is and always has been the case with Remote Desktop Services (RDS). Remote Desktop Virtualization Host (RDVH) and Remote Desktop Session Host provide a very functional, but basic solution for virtual desktops or shared sessions. If you want greater granularity with an increased feature set while wanting to stay on Hyper-V, you can upgrade into the Citrix XenDesktop or XenApp product space. If you want to use vSphere as your hypervisor you could run RDSH VMs natively or add greater management granularity and features by buying into the VMware Horizon product space. At the end of the day, no matter which way you go, if you intend to deploy shared sessions or shared virtualized applications, as most environments do initially, Microsoft RDSH is the technology underneath it all. This series will cover the what, why and how of the native Microsoft RDS stack for Server 2016.

Use Cases

The first thing to decide is whether you need shared sessions/ published apps (most common) or dedicated pooled or personal virtual desktops assigned 1:1 to each user. RDSH is the most widely deployed technology in this space and where most environments initially invest to virtualize published applications or deploy a shared hosted workspace. If the shared session/ app solution falls short in any way or if users need greater dedicated performance, the next step is to isolate users via pooled or personal virtual desktops. Pooled desktops can be refreshed for each new user (non-persistent) while personal desktops are dedicated to a user (persistent). Pooled desktops or shared sessions can make use of User Profile Disks (UPD) which store user settings and folders in a central location. A collection can make use of both folder redirection as well as UPDs.
Virtual desktops are completely complimentary to published applications which can reduce the complexity of the template used for a collection while lowering the resource consumption of VMs running within. This is done by offloading the running applications to RDSH hosts keeping the desktop VMs leaner thereby yielding greater density per compute host. Multipoint is now an installable role in Server 2016 RDS which is useful for single-server multi-user scenarios not requiring RDP across the network. For Server 2016 all uses cases can be built on-premise, in the cloud, or a hybrid of the two. This blog series will focus on the on-premise variety which is also most common.

What's new in RDS 2016

Personal Session Desktops - A new type of session collection that allows users to be assigned to a dedicated RDSH VM (Azure deployments)
Gen2 VMs - Full support for Gen2 VM templates for pooled/ personal VM collections and personal session desktop collections
Pen Remoting - Surface Pro stylus devices available for use in RDS
Edge browser support
RemoteFX Virtual GPU - OpenGL, OpenCL, DX12, CUDA, 4K resolution, Server OS VMs, 1:1 Direct Device Assignment
Windows Multipoint Services - Now a part of RDS as an installable role for super low cost single server solutions, RDSH required
RDP10 - h.264/ AVC 444 codec, open remoting
Scale - RD Connection Broker capable of supporting 10K concurrent connection requests

Solution Components

At a very basic level, if you want shared sessions you need the RDSH role enabled on physical hosts or Server VMs. If you want virtual desktops you need Hyper-V and the RDVH role enabled on the physical host/ parent partition. Best practices suggest that only Hyper-V and RDVH roles be enabled on physical hosts, all other roles should exist within VMs to enable better scale, HA and portability. It is important to note that neither SQL Server nor System Center Virtual Machine Manager (SCVMM) are required for a basic RDS deployment.
The RDS management infrastructure for Server 2016 has two deployment scenario roles and four Services roles which can be deployed within a single VM for POCs or very small environments.

Hosts

Compute hosts - Hyper-V hosts used for the sole purpose of running RDSH VMs or pooled and personal desktop VMs.
Management hosts - Hyper-V hosts used for the sole purpose of running RDS infrastructure components such as Connection Brokers and Web Access servers.

Deployment Roles

RDVH - Virtualization Host role provides the ability to deploy pooled or personal desktops within Hyper-V, organized via collections. Enabled within parent partition of Hyper-V enabled host.
RDSH - Session Host role provides the ability to deploy servers or VMs hosting shared sessions or published applications, organized via collections. Can be enabled within physical or dedicated RDSH Server VMs.

Services Roles

RD Connection Broker - The broker is the heart of the operation and connects users to their desktops, sessions or apps while load balancing amongst RDSH and RDVH hosts. This role is mandatory.
RD Gateway - The gateway role is used to securely connect internet-based users to their session, desktop or app resources on the corporate network. This role is optional.
RD Web Access - Enables users to access RemoteApp and Desktop Collections via the Start menu or web browser. This role is mandatory.
RD Licensing - Manages the licenses required for users to connect to RDSH sessions, virtual desktops or apps. This role is mandatory after 120 days.

Collection Deployment Options

Sessions/ apps - Users connect via RDP to a RDSH host or VM to run a full desktop experience or a published app. All users connected to a single RDSH host share the physical resources of that host. In 2016 users can be configured to connect to a dedicated RDSH host (useful for Azure and desktop licensing rules).
Pooled desktops - Non-persistent desktop VMs created within a collection that have power state and automatic rollback capabilities to create fresh VMs for each user login. User settings can be stored on a UPD.
Personal desktops - Persistent desktop VMs created within a collection that have power state managed and are permanently assigned to a user of the desktop. This collection type does not require nor support the use of UPDs.
User Profile Disks - UPDs can be added to any session or pooled desktop collection to persist user profile data between session or desktop connection activities. One UPD per user.

View of the various RDS infrastructure roles as portrayed within Server Manager:

Architecture

Here are all those pieces showcased in a basic distributed architecture, separating resources based on compute and management roles as I like to do in our Dell solutions. You'll notice that there are a couple of ways to connect to the environment depending on where the user is physically as well as which resource they need to connect to. SQL Server is listed for completeness but is only required if you intend to add HA to your RDS deployment. Pooled (non-persistent) and Personal (persistent) desktops are functionally similar in that they are both based on a template VM to deploy the collection but only sessions and pooled collections can make use of a UPD. The storage hosting the UPDs is illustrated below as external to the management hosts but it could also be a file server VM that exists within. This RDS architecture can be deployed using traditional shared storage, software defined, or local disk.

Part 1: The Basics (you are here)
Part 2: RDVH
Part 3: RDSH
Part 4: Scaling and HA

Resources

Technet

Hayward AQR15 Install

noreply@blogger.com (Peter) — Mon, 02 May 2016 21:39:00 +0000

So I've decided to leave the tech business and start a pool services company, this is my first post of several in a series of related topics. Just kidding, but I did install a new salt water chlorine generator (SWCG) for my pool and here's what was involved. I've had an ailing Pentair IC40 for five years now that worked ok the first year I had it, but then it was nothing but trouble thereafter. Three flow switch replacements at $75+ each, constantly pouring in bleach to augment and this year, yet again, after removing the cell for the winter, when I reinstall it comes up red, system off, again. This means one of four things: water temp too low, salt too low, bad flow switch or bad cell. I'm betting the last as these things aren't really expected to survive more than 2-3 years, so my cell probably died over a year ago. Sadly, Pentair only offers an abysmal 1 year warranty on these units (60 days if you install yourself), which at the price of admission, is completely unacceptable. I set out to replace my bad Pentair solution with something better.

Disclaimer: As is always the case, anything presented here is for informational purposes only.

Anything you choose to replicate is at your risk or peril, not mine. If you have no experience

with plumbing or electrical work this is probably not the project for you!

So, why salt water?

Many confuse why people convert their pools to salt to begin with and what that really means. There's a couple of very good reasons: easier maintenance and easier on your body. Salt pools still require Chlorine (CL) but instead of using feeders, pucks or liquid CL, the sodium chloride in the water (NaCL) is converted to sodium hypochlorite aka Chlorine (NACIO) with the help of an electric salt cell as the water passes through it. The CL does its job destroying organics and returns to the water as salt when exhausted where the process repeats. The process is simple, renewable and cost effective. Salt pools require much lower CL levels overall (3-5ppm) and leave your skin feeling soft without the eye burn that comes with traditional CL pools. Your water chemistry still requires balance of course with a special consideration paid to PH, which will steadily creep upwards if unchecked, and calcium hardness. Calcium scaling is the enemy of salt cells which reduces the cell's effectiveness and life over time. You'll often see thin slivers of calcium on the pool steps or floor as the salt cell performs its "reversing" procedure to clean itself of scale buildup. There are of course a number of options that can be used to sanitize your pool, each with its pluses and minuses. Here is a short list of options from my research and practical experience from maintaining a pool for many years as well as why I settled on a replacement salt system.

Chlorine Tablets (Pucks) - This is the most common method used via floating dispensers or automatic in-line feeders. While it's easy to stack a few pucks in one of these delivery products and forget about it, the problem with this method is that these pucks contain additional chemicals besides CL, namely cyanuric acid (CYA) aka stabilizer. CYA is important to have in the pool to protect the CL from being destroyed by sunlight, but there is a specific balance to maintain. Pucks keep feeding CYA which keeps building in the water, finally to the point that the CL can no longer do its job. Algae starts building and you wonder why. The only way to get rid of CYA is to drain and refill.

Liquid Chlorine - Most commonly known as bleach, liquid CL is the purest and most beneficial sanitization method available for the pool. Varying concentrations can be found from 8.25% to 10+% but it's all the same stuff: bleach. The only problem with this method is having to constantly stock, store and add bleach to the pool. Slip up because your stock is low and you could be fighting an algae outbreak in no time. Contrary to what the pool store will tell you, you don't need to buy "special" chlorine nor shock for your pool. Bleach is all that you need.

Liquidator - Pouring bottles of bleach in the pool, while effective, is a hassle, enter the Liquidator (LQ). The LQ is a simple device that is used to passively add liquid CL to the pool using gravity and the suction side of the plumbing. Liquid CL is mixed with pool water in an 8 gallon tub that causes the heavier CL to sink to the bottom below the pool water. When the pump runs the liquid CL is pumped into the pool and chemical odors are kept at bay. The LQ is fairly cheap at $200 for the equipment plus the cost of bleach to maintain. The problem with this method is fine tune control of how much CL you add to the pool and keeping the tank filled with the CL kept cool. I've read recently of people having trouble with mineral build up in the LQ equipment causing them to abandon this solution completely.

Stenner - Stenner pumps have been around for ages and provide another method to get liquid CL into the pool but via an active pump, so there are no issues with mineral build up on the stenner side. Most use a 15 gallon tank, filled with bleach, and a 100PSI pump set to run on a timer to add CL to the pool when the main pool pump runs. Most are very happy with their Stenners but the issues with this solution are getting the right parts to build the pump and tank, storing CL which degrades when it gets hot and the cost/ hassle of keeping the tank filled. While the Stenner may be cheaper to build initially (~$400), the annual cost of bleach puts this as a potentially expensive solution depending on the number of active months in your season. The average I read was consuming ~15 gallons per month during the summer, which at $4+ per gallon adds up year after year.

SWCG - So here we are, back at the SWCG. I outlined the benefits in the first paragraph which center around ease of maintenance and body friendly properties. Buying into the SWCG solution is one of the more expensive options and will set you back at least $1K plus salt which is $5-10 per bag. After the initial investment, if all goes well, you'll be on easy street only having to adjust your salt after heavy rains or being closed for the winter. But, assuming this solution lasts for a minimum of 3 years, which is the warranty period of the Hayward AQR15, it is still cheaper than the Liquid CL methods in the long run. Problems can and will occur, however. The most likely issues will be failing flow switches, bad control boards and salt cells that require replacing, all of which should be covered during the warranty period if any of that were to happen. Once the cell has lived its useful life, the replacement for this particular model is ~$450, half of the initial investment, which would last another 3+ years.

More information on all these options including proper water chemistry and other pool basics can be found at my favorite online pool resource: troublefreepool.com (TFP).

Installing the Aqua Rite 15

The AQR15 is a fan favorite at TFP with many claiming years of faithful service well past the warranty period. Having done the math on the other options and ruling out the LQ and Stenner due to cost, I finally decided the AQR15 was the right move. After having such a rough go with the Pentair SWCG I was extremely hesitant to invest in a new setup which is not a cheap entry to market. $1000 will get you a salt cell and a control box/ transformer. The general rule of thumb for pool equipment is to "go bigger" than you need to. This mainly applies to filters and SWCGs as it is possible to go too big on the pump depending on the size of your pool. The AQR15 is a kit that includes: a flow switch mounted to a 2" slip-fit T union, a T-15 cell (chlorinates up to 40K gallons) and the control center that monitors each of these components while being able to provide useful metrics such as water temperature, water salinity and cell voltage/ amperage.

Here is the desired before and after design. The Pentair IC40 is a simpler product with the flow switch integrated into the cell housing using a single cable to connect to the transformer. The AQR has an external flow switch which requires 12-inches of straight pipe upstream from the switch but placed after all other pool components (filter, heater etc). If placing the flow switch after the T-15 cell, the cell itself serves as the required 12-inches. Both the flow switch and the cell must be wired to the control box individually.

First step is to modify the plumbing to accept the new Hayward components, which requires me to elongate my existing return loop. You always want to try and leave 2-3" of pipe, minimum, in your plumbing for future modifications. You'll see there are a few spots where I am now out of room. Any future updates will require me to rebuild larger downstream valve junctions and these Jandy valves are $50 each. I made cuts with my reciprocating saw to remove the existing Pentair couplings which, unfortunately, did not match the thread pattern of the new AQR and had to be replaced. I glued in the new flow switch and built the downstream side of the cell couplings as you can see below. Pay special attention to the direction of the flow switch whose arrows are on the top of the unit itself.

Next I built the other side of the cell coupling trying to elongate this section as little as possible. Any length I add here will have to be replicated to the upper section of the loop. After installing the cell I was able to cut the upper portion of the loop and elongate using a single 2" coupler. I got lucky. I was prepared to have to add a new section of pipe here with 2 couplers but that proved unnecessary. The cell can be installed in either direction since the flow switch is external to the cell. Don't forget to lube the O-rings of the cell couplings. Here is the new cell plumbed in (with my helper working on the bonding wire):

While the glue sets on the plumbing, on to the electrical portion of the install. The Hayward control box is much bigger than Pentair's, so while I was able to reuse 2 of the existing mounting screws, I needed to drill more. Drilling through brick sucks, it just does. I have my SWCG control box mounted above my timer and already have weatherproof conduit in place. Remove a knock out from the bottom corner of the new control box and shorten the wires to the appropriate length. Secure the conduit to the new box. The control box comes preconfigured for 240V setups which will be most common in larger in-ground pools. 2 x 120v hots plus a ground with the 2 middle connections of the box jumpered. Pretty simple. Make sure the breaker is off before you start fiddling with wires, 240v is not something to play with. Because my Pentair VS pump has a built-in timer, I have to use a separate timer to run my SQCG and heater or invest in a very costly automation system. Have I mentioned that I don't like Pentair much anymore?

With the power supplied to the control box, all that remains is connecting the bonding wire, flow switch and salt cell. The previous idiot that installed my Pentair system used the wrong kind of wire for bonding which isn't safe or to code. Code calls for 8 AWG copper and should connect to the exterior of all pool equipment that carries an electrical charge. The bonding wires of all components connect together and then run into an earth ground. This is to eliminate the risk of shock should any of these pieces become electrified. Here my helper is connecting our new bonding wire to the junction next to the pump.

The bonding wire connects to the exterior of the control box via screw mount and the flow switch connects using an RJ11 connector pre-assembled.

Before you connect the salt cell to the front of the control box, the proper model number needs to be entered in the control box first via the diagnostics menu. There are stickers everywhere warning you of this so best to take heed. By default the largest cell available, the T-15, is programmed. If you have a smaller model, change this first before supplying any power to the new cell. Once everything is wired, glue dry and proper cell configured, the system can be tested assuming you already have the proper salt level in your pool. If you do not have any salt yet, add this first and wait 24 hours before powering the new cell. The proper amount of salt plus other pool chemicals can be determined by using the pool calculator. Otherwise, set the desired % of CL output, set the cell to Auto and watch for green lights and power and generating. Expect a period of 'dialing in' while you figure out what output percentage and run times are necessary. Success!

Here is everything installed. So far very pleased with the setup.

Key Takeaways

Go bigger than you think you need to with pool equipment. A bigger SWCG means that it will do the job at a lower percentage of output which is great if you really need to crank things higher for short periods of time.
Plan your plumbing. A SWCG can be easily plumbed into a new or existing setup. Keep in mind that the Hayward AQR has an external flow switch that requires 12-inches of straight pipe in front of it, placed after all other pool equipment on the return side of the plumbing.
Use clear PVC primer for a cleaner finish. Nothing wrong with the purple stuff but clear is cleaner.
The Hayward T-15 cell can be mounted in either direction since the flow switch is external. Hayward encourages changing the direction after 3-month inspections or cleanings.
Bond your equipment with the proper 8AWG copper wire.
Make sure the control box is programmed with your particular cell before you supply power to the cell!
Time your cell to operate while the main pump is running only via timer box. Do not use the operation of the main pump to control the timing of the SWCG operation.

Resources

http://www.troublefreepool.com/

Pool School

The pool calculator

Aqua Rite Operation and Installation Manual

vSphere Design for NUMA Architecture and Alignment

noreply@blogger.com (Peter) — Fri, 01 Apr 2016 21:17:00 +0000

I’m publishing this design topic separately for both Hyper-V and vSphere, assuming that the audience will differ between the two. Some of this content will be repeated between posts but arranged to focus on the hypervisor each prospective reader will likely care most about.
Be sure to also check out: Hyper-V Design for NUMA Architecture and Alignment: Link

Non-Uniform Memory Access (NUMA) has been with us for awhile now and was created to overcome the scalability limits of the Symmetric Multi-Processing (SMP) CPU architecture. In SMP, all memory access was tied to a singular shared physical bus. There are obvious limitations in this design, especially with higher CPU counts and increased traffic on the bus.

NUMA was created to limit the number of CPUs tied to a single memory bus and as a result defines what is construed as a NUMA node. A high performing NUMA node is defined by the memory directly attached to a physical CPU socket in turn accessed directly by consuming applications and services. The memory controller is on-die and the memory DIMMs connect directly to the CPU socket. The total number of logical cores available to a NUMA node is double if Hyperthreading is enabled. Per the example below, I have a dual socket system with 6 physical cores per CPU, including hyperthreading there are 12 logical cores per physical NUMA node, netting a total of 24 logical cores in the system.

NUMA spanning occurs when memory or CPU cores are accessed remote to the NUMA node an application, service or VM is connected. NUMA spanning means your application or service won’t run out of memory or CPU cores within its NUMA node as it can access the remote DIMMs or cores of an adjacent CPU. The problem with NUMA spanning is that your application or service has to cross the lower performing processor interconnects to store its pages in the remote DIMMs or access remote cores. The maximum vCPU value assignable to a VM is dictated by the total number of logical cores available in all NUMA nodes.

What is NUMA alignment?

In the case of virtualization, VMs created on a physical server receive their vCPU entitlements from the physical CPU cores of their hosts (um, obviously). There are many tricks used by hypervisors to oversubscribe CPU and run as many VMs on a host as possible while ensuring good performance. Better utilization of physical resources = better return on investment and use of the company dollar. To design your virtual environment to be NUMA aligned means ensuring that your VMs receive vCPUs and RAM tied to a single physical CPU (pCPU), thus ensuring the memory and pCPU cores they access are directly connected and not accessed via traversal of a CPU interconnect (QPI). As fast as interconnects are these days, they are still not as fast as the connection between a DIMM and pCPU. Ensuing that your virtual environments are NUMA aligned means ensuring the highest possible performance.

Running many smaller 1 or 2 vCPU VMs with lower amounts of RAM, such as virtual desktops, aren’t as big of a concern and we generally see very good oversubscription at high performance levels with less need to span NUMA. In the testing my team does at Dell for VDI architectures, we see somewhere between 9-12 vCPUs per pCPU core depending on the workload and VM profile. For larger server-based VMs, such as those used for RDSH sessions, this ratio is smaller and more impactful because there are fewer overall VMs consuming a greater amount of resources. In short, NUMA alignment is much more important when you have fewer total VMs with larger vCPU and RAM allocations.

To achieve sustained high performance for a dense server workload, it is important to consider:

The number of vCPUs assigned to each VM within a NUMA node to avoid contention.
The amount RAM assigned to each VM to ensure that all memory access remains local within a physical NUMA node.

The easiest method to follow is to ensure that your VMs receive vCPUs in an even multiple of the total logical cores available per NUMA node. The following is an example of a balanced configuration based on a dual 6-core, 96GB server used for RDSH. The server hosts 6 high performance VMs that are configured within 2 NUMA node boundaries. Given the number of vCPUs and RAM assigned to each VM, each NUMA node is oversubscribed 2x from a pCPU core perspective (including hyperthreading) and does not exceed the 48GB of RAM available to each pCPU socket. This ensures that NUMA spanning is not required and minimizes potential contention amongst the VMs. Your oversubscription ratio may vary depending on workload or SLAs. For RDSH specifically, the best results are achieved using 1.5x or 2x oversubscription. Here is an example of a NUMA aligned configuration with 4 VMs:

vNUMA in vSphere

In vSphere 5.0 and later, the virtual NUMA topology can be exposed to the guest OS as long as it is hardware version 8 or later. vNUMA is enabled by default if a VM is configured with more than 8 total vCPUs (or 4 vCPU cores + 2 cores per NUMA node) and will match the host where it was powered on until the VM is restarted. This minimum number can be adjusted if desired as shown later.
When the NUMA topology is exposed to a guest VM, one of two things happen via the ESXi NUMA scheduler:

If a VM is assigned a vCPU value less than or equal to the number of total logical cores in a physical NUMA node, then this VM will be assigned to a single physical NUMA node and consume memory/ cores local to that node (ideal). This remains true until that VM is configured to access more memory than what is available in the single physical NUMA node, which would result in the VM spanning to the remote NUMA node (less than ideal).
If a VM is assigned more vCPUs than cores that exist in a single physical NUMA node, it will be assigned to at least 2 physical NUMA nodes. This will satisfy the resource requirements of the VM but will come with a performance penalty as the VM spans local and remote NUMA nodes (less than ideal).

Some conditions will cause VMs to not be managed by the ESXi NUMA scheduler, such as enabling CPU Hot Add or manual specification of a VM's CPU or memory affinity. This effectively disables vNUMA and carries some obvious inherent risks.
The NUMA scheduler assigns each VM to a “home node” at boot time which is a defined physical NUMA node within the server. ESXi tracks NUMA nodes within the System Resource Allocation Table (SRAT). When calculating a VM's NUMA home node, hyperthreading is ignored. So if your VM is configured with more vCPUs that a single physical CPU has cores, it will be assigned to 2 or more NUMA home nodes. ESXi will ensure VM memory locality by locking a VM to run from the same home node that contains its assigned memory. If system load changes such that a VM’s home node is no longer optimal, ESXi can dynamically migrate a VM to a new home node. Part of this migration could also include a migration of the VM’s memory assignment to prevent spanning NUMA if possible. Important to note that if you set CPU affinity for a VM, this essentially disables the NUMA scheduler for that VM.
There are 2 ways you can verify how many NUMA nodes are present according to what ESXi sees, via esxtop or esxcli. From the console of one of your hosts run esxtop. Once running, press "m" to enable memory stats. Later, to see the NUMA stats as they apply to VMs, from esxtop press "f" to change the fields displayed, select NUMA Stats with "g" and hit any key to exit.

As you can now see, ESXi is aware of 2 physical NUMA nodes in my test system each with 49GB memory directly attached to each node.

The second method is the "easy" way but displays less information overall. Run the following esxcli command which also reveals that this host has 2 NUMA nodes:

esxcli hardware memory get | grep NUMA

Further, just to completely remove any doubt about the number of logical cores available and to which physical NUMA node they belong, consider the following output from esxcli. 12 logical CPUs assigned to each physical node.

esxcli hardware cpu list | grep Node

NUMA Alignment in vSphere

The basic principles of NUMA are the same when dealing with vSphere or Hyper-v however executed slightly differently from a VM perspective. My vSphere cluster shares the same specs of my Windows cluster:

2 x Dell PowerEdge R610’s with dual X5670 6-core CPUs and 96GB RAM running vSphere 5.5 U2 in an HA Cluster. 24 total logical hyperthreaded processors as reported by ESXi.

vNUMA can be altered on a VM by VM basis through vCPU assignments or via advanced host settings if global tweaks need to be made. It’s worth noting here that although the vSphere Client will allow you to assign CPU cores as well as “cores per socket” to your VMs, this is generally not required. VMware added this option in previous versions to provide the ability to control the number of sockets seen by the guest OS as a way to get around certain software restrictions and limitations that likely no longer apply. In vSphere a vCPU is a vCPU and will be scheduled according to how it fits into the exposed vNUMA topology. You can specify the cores per socket which will affect the NUMA topology presented to the VM, but also realize that this could have negative impacts as to how well that VM plays within the presented topology. Unless you have a good reason to change this, leave the default value of 1 core per socket when configuring VMs in ESXi. In my test servers a single VM can be configured with up to 24 vCPUs and if it's a Windows server OS, up to 24 sockets, which constitutes the maximum logical configuration of all available cores on this host. Take for example a Server 2012 R2 VM called "X2" configured with 12 vCPUs using the default of 1 core per socket.

Using the Sysinternals Coreinfo tool we can view the virtual NUMA topology presented to the VM, but keep in mind this is not always indicative of the physical NUMA node assignment. With 12 vCPUs assigned, at 1 core per socket (12 sockets), ESXi exposes the VM to 2 vNUMA nodes. Note that this topology is virtual but at 12 vCPUs fits within a single physical NUMA node:

Because the number of vCPUs is >6, ESXi assigns this VM to both NUMA home nodes, shown here in esxtop via the NUMA Home Node (NHN) value. Remember that the NUMA Scheduler ignores the hyperthreaded cores when making this calculation:

Even though at 12 vCPUs this VM fits within the logical boundaries of a single physical NUMA node, this VM is assigned to more vCPUs than the 6 physical cores that exist in a single NUMA node, thus enabling ESXi to reserve the right to schedule this VM's CPU consumption on either NUMA node. If this VM were to be assigned 13 vCPUs, not only would it be assigned to both NUMA home nodes as we see above but it would span NUMA to fulfill it's operational requirements. Per the diagram below you can see that two similarly configured VMs fit within the constraints of each NUMA node from an assigned memory perspective, but because X2 is configured with more vCPUs than total logical cores that exist on one NUMA node, it must cross the QPIs to fully satisfy its entitlement.

Another example of vNUMA in action is the assignment of 24 vCPUs to this VM using an arbitrary 3 cores per socket. vNUMA creates 8 NUMA nodes (sockets) with 3 processors assigned to each.

The other example that would cause a VM to span NUMA would be if it were assigned to more memory than what exists within a single physical NUMA node. For this example I've reconfigured my X2 VM with 6 vCPUs to ensure CPU locality but assigned and reserved more memory than what exists within the NUMA home node to which it is assigned.

For NUMA spanning to actually take place the VM now has to actually consume more memory than what exists in the NUMA home node it was assigned. To do this I used a memory allocation tool and consumed 48GB of active memory. Hopping back over to esxtop we can see exactly the amount of memory being used locally (NLMEM) vs memory spanning to the remote NUMA node (NRMEM).

To illustrate what is happening here, X2 is utilizing memory from its assigned NUMA home node but is also having to access the remote NUMA node to store its memory pages. If the cluster didn't have enough resources available for the configuration of the VM, it would fail to power on.

There are a number of additional tweaks available in the Advanced System Settings for a given host or at the individual VM level. Be extremely careful here and only change something if you have education and purpose.

To allow a VM with fewer than 8 vCPUs to be managed by vNUMA, a configuration parameter entry must be added to its .vmx file. Open the settings of the VM to be configured, under the VM Options tab, expand Advanced and click Edit Configuration beside Configuration Parameters. Look for the entry “numa.vcpu.min” and edit its value, if it’s not there add this entry and specify the number of vCPUs you need in the value. This would be useful in scenarios where you have fewer VMs with larger memory assignments.

The numa.autosize.vcpu.maxPerVirtualNode value assigned from within a VM's configuration parameters, can be used to control the vNUMA topology presented to a VM. By default, based on the physical characteristics of my test servers, the value is 6 vCPUs per node as presented to the VM. This can be adjusted up or down if required.

Key Takeaways

As a matter of general best practice, assign vCPU values to VMs less than or equal to the number of physical cores on a single NUMA node.
Designing your virtual infrastructure to work in concert with the physical NUMA boundaries of your servers will net optimal performance and minimize contention.
A single VM consuming more vCPUs than a single NUMA node contains, will be scheduled across multiple NUMA nodes causing possible CPU contention.
A single or multiple VMs consuming more RAM than a single NUMA node contains will span NUMA and store a percentage of their pages in the the remote NUMA node causing reduced performance.
In vSphere vNUMA is enabled by default for VMs with 8+ vCPUs (adjustable) but enabling "hot add CPU" or configuring CPU affinity will essentially disable this feature. Be careful if you do this.
When the NUMA Scheduler calculates which NUMA home node to assign a VM to, hyperthreading is ignored. This calculation is based on the physical cores of the CPUs.

Resources:

http://lse.sourceforge.net/numa/faq/
NUMA in ESXi
http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.5.pdf
https://blogs.vmware.com/vsphere/2013/10/does-corespersocket-affect-performance.html
http://pubs.vmware.com/vsphere-55/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-55-resource-management-guide.pdf
https://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-60-resource-management-guide.pdf