a wandersick's Tech Notes

Quick Ansible Overview — Remediate Host Configuration Drift (Infrastructure-as-Code)

noreply@blogger.com (wandersick) — Tue, 22 Sep 2020 04:23:00 +0000

This introductory article is about Infrastructure-as-Code (IaC) and a major player of which, Ansible, its concepts — control node, managed host, inventory, playbook and modules — along with a quick example using an ansible command to remediate configuration drift of managed computers, so that a specified service on them is running in its desired state.

Infrastructure-as-Code

DevOps is not a Goal, but a never-ending process of continual improvement. — Jez Humble

It is the era of DevOps, where development (Dev) and operation (Ops) are converged and streamlined for the benefit of the business. On the infrastructure side, it is also desired that managing hosts (mostly servers and workstations) be done in a modern way to fit in the DevOps mindset where everything is defined as code to allow for agility which enables continual improvement.

Comes Infrastructure-as-Code automation engines, available as command-line tools which can be leveraged for infrastructure compliance remediation. Such things might sound difficult to be done in the past, but thanks to the advance in technology, the complexity is hidden by automation engines such as Ansible.

Ansible

Automation is good, so long as you know exactly where to put the machine. — Eliyahu Goldratt

Ansible is an open-source agentless configuration management tool from Red Hat which treats infrastructure as code. It can prevent configuration drift by maintaining the desired state, e.g. ensuring operating system firewall is running, by remediating security and compliance issues according to Ansible playbooks of rules specified against an Ansible inventory file of computers.

How to Get Ansible to Work

Operating Ansible commonly involves developing Ansible playbooks in YAML format, as well as writing inventory files in INI format. They are simpler than other syntax file formats such as JSON (which is also supported by Ansible), and are (arguably) more friendly to the less technically-inclined. These will be used in our quick example.

Some Ansible Terminology

Inventory — an INI file (e.g. inventory.ini) where a list of managed hosts (also known as managed nodes) are defined. These may be computers a system administrator would like to manage
Playbook — a YAML file (or JSON) where rules are defined, e.g. rule-book.yml

A computer with Ansible installed, the control node, can then execute the ansible-playbook command, specifying the inventory and playbook via its parameters, e.g. ansible-playbook -i inventory.ini rule-book.yml to check or remediate any compliance drift found in the managed hosts.

Ansible Host Inventory — inventory.ini

This is an example of an inventory which shows two Linux servers.

Specify two Linux servers named LNXSVR01 and LNXSVR02 (which are used below as an example of managed hosts) in inventory.ini:

LNXSVR01
LNXSVR02

Ansible Playbook — rule-book.yml

This is an example of a playbook which shows firewall must be started.

Perform compliance checking to ensure firewall service is running on all hosts defined in the inventory by creating the playbook, e.g. rule-book.yml as below:

---
- hosts: all # Apply to all hosts in the inventory file
  remote_user: root
  tasks:
    - name: ensure firewall service is running
      service:
        name: '{{ item }}'
        state: started
        enabled: yes
      with_items:
        - firewalld # Add more services to this list as required

Although the above example merely shows checking or remediating a single item, when rules like this are combined together according to system hardening guides such as the ones published by Center for Internet Security (CIS), they ensure computers are compliant with industry standards.

Ansible Command

Putting it together, run Ansible playbook rule-book.yml against computer inventory inventory.ini with superuser privilege via --sudo (or --become) parameter:

$ ansible-playbook -i inventory.ini rule-book.yml --sudo

The command can be fired up manually or via a scheduled cron job.

Ansible Checking and Remediation Results

LNXSVR01, whose firewall was not running, becomes compliant after the Ansible playbook execution. On the other hand, LNXSVR02, whose firewall was already running, was idempotent*:

PLAY [all] ***************************************************
TASK [Gathering Facts] ***************************************
ok: [LNXSVR02]
ok: [LNXSVR01]
TASK [Ensure firewall service is running] ***
ok: [LNXSVR02] => (item=firewalld)
changed: [LNXSVR01] => (item=firewalld)

PLAY RECAP ***************************************************
LNXSVR01      : ok=4    changed=1    unreachable=0    failed=0
LNXSVR02      : ok=4    changed=0    unreachable=0    failed=0

*Idempotent in this context means only execute when configuration has drifted; otherwise, no action would be performed no matter how many times the command has been run.

Extending Capability with Community Modules

Ansible, being the most popular configuration management project on GitHub and being in the top-10 chart there for many years, has an active community developing modules to extend capabilities of it.

For example, due to Windows previously lacking a native software package manager (e.g. yum on Red Hat or apt-get on Ubuntu Linux), Chocolatey, the most popular third-party Windows package manager, was born and can be used via win_chocolatey module in Ansible, which is one of the modules maintained by the community.

Taking AeroZoom as an example of Windows software on Chocolatey, which is a keyboard-free screen magnifier freeware application created and submitted by me to the Chocolatey community repository. Below, AeroZoom acts as a required item for compliance checking.

A playbook which performs compliance checking to ensure AeroZoom is installed would be like this:

---
  tasks:
    - name: ensure AeroZoom Windows application is installed
      win_chocolatey:
        name: AeroZoom
        state: present

Further from Here

Ansible is truly a powerful configuration management tool used by DevOps engineers. Being a simple command-line tool, it does not come with a graphical user interface. For those who need it, a separate web-based application, Red Hat Ansible Towers (or AWX, the open-source upstream variant of which), can be deployed.

Ansible can manage both Linux and Windows machines. However, the control node of Ansible is so far only available on Unix-esque platforms such as Linux, which is less approachable to IT professionals who may be more used to the Windows platform. Some time ago, Microsoft has included Windows Subsystem for Linux (WSL) in Windows 10 which can install and run Ansible. There are limitations last time I checked (such as not supporting cron which many people use to trigger Ansible), but once it matures, who knows? It could someday totally replace the need for a separate virtual machine or docker instance merely to run Ansible on Windows.

All in all, Ansible has high potential being a great lightweight (agentless) tool which is handy e.g. for sysadmins to ensure system compliance, DevOps engineers to configure operating systems after they are provisioned, and so much more!

If this article helps you get started with Ansible, feel free to let me know. In future, I may explore more aspects of Ansible where I find useful or interesting. Stay tuned! 😊

Folder Template Provisioner

noreply@blogger.com (wandersick) — Sun, 13 Sep 2020 12:48:00 +0000

For file server admins, this Windows batch script provisions (copies) new folders with exact permissions and content from a specified existing folder (template) based on the information inputted by a user via its command-line interface. It hopes to reduce the burden of Windows admins by handling over the task of folder provisioning to users.

It uses robocopy /MIR /COPYALL /ZB to solve the problem where folders copied using Windows Explorer (a.k.a. File Explorer) may not retain unique permissions and inherit permissions from parent folder.

Go to Download

Features

Accept input from user
- First name and last name (feel free to modify them for other use cases where appropriate)
Copy a new folder using specified template folder retaining exact permissions and content using robocopy /MIR /COPYALL named in LASTNAME, First Name format according to user input
Support of network-shared folder in UNC form (\\...) is available
- In other words, for drives mapped using a drive letters, they are supported by specifying the UNC path within this script. See How to Set up the Scripts section below
- To test or use this script for non-network cases (local drives) from v1.2 and on, specify \\127.0.0.1\... or \\localhost\... where required
Support of admin users with UAC enabled
- Automatically elevate the script to run it as admin if UAC (User Account Control) is enabled but admin rights are unavailable. This requires an optional script in the same folder (_elevate.vbs)
Partial support of non-admin users using runas /savecred (built-in)
- If users do not have admin rights, _runasAdmin.bat (included optional script) can be edited to leverage runas /savecred to run FolderTemplateProvisioner.bat (main script) as admin without entering admin credentials
- Also refer to Security Notes on the risk involved naturally with runas /savecred
Error checking and messages available
- Windows Script Host is unavailable (for the UAC elevation VBScript)
- Admin rights are unavailable
- Invalid first name / last name
- File copy (folder template provisioning) failure and the robocopy return code
Confirmation prompt
- Names inputted by user are shown again for review and confirmation, requesting user to confirm there is no existing files
Logging
- A robocopy log file is saved and appended to %Temp%\FolderCreator.log. As this is not meant to be permanent, it can get deleted by system shortly afterwards

Requirements

Windows OS with robocopy
admin rights (with or without UAC)
Non-admin rights (partially supported with an optional setup on _runasAdmin.bat)

Script Filenames

FolderTemplateProvisioner.bat (main script)
_elevate.vbs (optional, for admin users with UAC turned on, trigger UAC elevation prompt)
_runasAdmin.bat (optional, for non-admin users to leverage runas /savecred to run as admin)

Folder Hierarchy

│ FolderTemplateProvisioner.bat // main script
│ _elevate.vbs // For admin users with UAC enabled to auto-trigger UAC prompt (optional)
│ _runasAdmin.bat // For non-admins to run as admin without admin credentials (optional)
│
├───ZZ IT_do not use\01 Template Folder // Template folder (this can be a sub-folder)
│                                       // with folders/files of set permissions to copy
│
├───A // Last names beginning with an "A"
│   │───LASTNAME, Firstname 1
│   │───LASTNAME, Firstname 2
│   │                       .  // Each copied folder is named in this way according
│   │                       .  // to names inputted by user with sub-folders and files
│   │                       .  // retaining specific permissions
│   └───LASTNAME, Firstname n
│
├───B
├───C
│   .
│   .   // Created folders are placed in folders according to initials from user input
│   .
└───Z

Step by Step - How to Set up the Scripts

Step 1. Skip this step and go straight to step 2 if users have admin rights. This is for cases where users executing the script do not have admin rights.

Edit the UNC path in _runasAdmin.bat setting it to the the script path, e.g. \\serverName\d$\Human Resources\01_Personnel-Files\FolderTemplateProvisioner.bat

It must be a UNC path starting with "\" instead of a drive letter

Edit /u:DOMAIN\username by changing DOMAIN to the Active Directory domain name (NETBIOS name), and the username as the admin user. For non-domain (workgroup) scenarios, DOMAIN can be skipped by removing DOMAIN\

If a non-domain account is specified, runas command still requires the PC to be connected to domain. This appears to be a limitation of runas
User Account Control (UAC) has to be disabled, i.e. setting EnableLUA to 0 under Windows registry HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System, or the target runas user is the built-in domain or local Administrator account which is unaffected by UAC.
- Kindly ensure the group policy User Account Control: Use Admin Approval Mode for the built-in Administrator account is undefined or disabled; otherwise, it may prevent the built-in administrator account from bypassing UAC limitations (in case this approach is chosen instead of disabling UAC)

Before _runasAdmin.bat (or the runas /savecred ... command) can be run by non-admin users, it needs to be run once on PCs of users who need to use Folder Template Provisioner without admin credentials. (The first run involves prompting for admin credentials where an admin needs to be there to input admin password once)

Step 2. Edit encPath variable at the upper area of FolderTemplateProvisioner.bat script by setting it to the network folder containing the script, e.g. \\serverName\d$\Human Resources\01_Personnel-Files

Step 3. Edit templateName variable at the upper area of FolderTemplateProvisioner.bat script by setting it to the folder acting as the template, e.g. ZZ IT_do not use\01 Template Folder, with required files and permissions inside

Step 4. Place all scripts (FolderTemplateProvisioner.bat, optionally _elevate.vbs and _runasAdmin.bat) inside a folder containing A,B...Z sub-folders, sitting aside. The A-Z folders contains the template folder and provisioned folders named LASTNAME, Firstname copied by the script from the template folder

Step by Step - How to Provision a New Folder

Step 1. Double-click FolderTemplateProvisioner.bat and follow on-screen instructions

For non-admin users, they should run _runasAdmin.bat instead

Step 2. Input last name and first name

Step 3. Review the input

Step 4. Wait for robocopy file copy (folder template provisioning)

Step 5. Verify the created folder

Screenshots - Provisioning a New Folder with the Script

A. Run FolderTemplateProvisioner.bat

Note: non-admins should run _runasAdmin.bat instead, available since v1.2

For admins with UAC enabled, a UAC elevation prompt (like the one below) is displayed automatically, provided that an optional _elevate.vbs exists in the same folder. (_elevate.vbs should not be run directly. Instead, run FolderTemplateProvisioner.bat)

B. Input first name and last name

C. Review and confirm input

Note: Since v1.2, users would receive a message requesting them to ensure no folder of the same name exists in destination before continuing; otherwise, content would be overwritten.

D. Robocopy provisions new folder from folder template with exact content and permissions

Note: Since v1.2, users would receive a message at the end recommending them to verify the created folder at target location.

Screenshots - Error Messages

A. Name inputted by user is invalid

B. Robocopy error ends with a return code presented to user

C. No admin rights are detected, while UAC is off and/or the optional _elevate.vbs is not in the same folder

Other error messages include:

Windows Script Host is unavailable (for the UAC elevation VBScript)

Security Note

This part can be skipped if you don't let users use this set of scripts without admin rights.

As with most technology that enables convenience, kindly beware of the natural security risk involved with _runasAdmin.bat, i.e. runas /savecred. In this case, a non-admin user would be able to use the runas /savecred command to run any command as an admin without entering a password. Please do understand such risk involved if you still would like to go ahead with the non-admin approach to using this script.

Another Possible Approach for Non-Admin

This part can be skipped. It is only for enthusiasts wishing to modify this script to run it as non-admin without even using _runasAdmin.bat or runas /savecred.

Admins rights or the _runasAdmin.bat (runas /savecred) workaround for non-admin are already supported by this script. The below may not be suitable to all cases and hence, it is left as an idea for enthusiasts to attempt should it be desired.

Several prerequisites and/or adjustments would be required (experimental):

Remove /ZB parameter from the robocopy command in the script (note: /ZB uses backup operator rights to copy if permission is unavailable. This requires admin rights)
Change /COPYALL to /COPY:DATSO (this removes the 'manage auditing' part which requires admin rights)
The template folder should include the desired non-admin user as owner with full permission in the underlying folders
Remove lines in this script between attrib >nul 2>&1 and :skipAdminCheckSubsequent

Note: The above is approach is experimental, hence unsupported.

Download

⭐ Download Folder Template Provisioner on GitHub .

Release History

Ver	Date	Changes
1.2	20200730	- Support of network-shared folder in UNC form (`\\...`) is available - Support of non-admin users is provided by `runas /savecred` - Remove support of UNC-unsupported error-checking cases
1.1	20200719	Folder template consolidated as single folder instead of multiple ones within sub-folders
1.0	20200714	First release where template folders are located in each sub-folders

Skills to Get Equipped Career-wise amid COVID-19

noreply@blogger.com (wandersick) — Tue, 02 Jun 2020 13:17:00 +0000

Recently, I received an email invitation about writing an article on working from home (WFH) amid coronavirus outbreak. As an IT professional who still does much work in a traditional face-to-face way, that led to my interest in the topic, resulting in a presentation I have done privately and finally this post – this article discusses a way the pandemic has changed our lives and how I personally cope with it career-wise by getting equipped with desired skills on remote-work marketplaces.

Problem – Coping with Remote Work, the New Normal

A wise person once said the only constant is change – have you noticed what has changed during the pandemic?

Lately, people in certain parts of the world are joking weather and safety leave* will no longer be a thing due to the boom of communication technologies such as Slack, Zoom and Microsoft Teams, which turned remote work (telecommuting) into the new normal – you and I may still need to work like usual as long as the Internet is not broken.

*While natural disaster can be devastating and affect safety of people, there are also people who are less affected and they may still be entitled to weather and safety leave where they can take a rest.

Coincidentally, before the pandemic, Satya Nadella, the current CEO of Microsoft, had a news article of him discussing about an alternative view on work-life balance – work and life should be harmonized, coining the phrase "work-life harmony", where work and life are more like a mesh instead of balance. Interestingly, this will likely turn out to be true too in the post-pandemic era, as the pandemic has opened people's eyes to what is possible with remote work even after the pandemic passes.

However, people may not be familiar with remote work – not many industries or companies adopted the remote-work model before the pandemic. As remote work gets more and more popular, in the post-coronavirus era, people will be able to obtain remote jobs more easily than before, only if they are equipped with remote-work skills. So how should one cope with that? Here is a way I recently experienced due to my desire to explore and improve my remote-working skills.

Solution – Remote Work Marketplaces

Have you heard of remote work marketplaces like Fiverr, Upwork, Freelancer.com, etc.? These are online freelancing platforms which provide the service of matching freelancers with businesses. They became popular amid COVID-19 due to the lockdown and some companies laying off employees for cost-cutting outsourcing options. Many of these platforms saw a surge in traffic during the pandemic, as reflected in their stock price.

(Personally, I tried all three platforms and had more success with Upwork than the other ones, so Upwork is the one I mainly talk about in this article. Your mileage may vary and other platforms may be more suitable to you or your industry.)

Take Upwork as an example. Founded in 1999, Upwork is one of the top marketplaces for remote work. It matches businesses and talents from 180 countries. Clients include Microsoft and more than 30% of Fortune 500 companies. Freelancers using these platforms for freelancing usually get to enjoy payment protection via their escrow service. In turn, companies using them to hire freelancers can also more easily look for the right candidate by filtering or preferring freelancers that meet their specific criteria. In the case of Upwork, it could be meeting location and language requirements, or different verification options specific to Upwork, e.g. Top-Rated and Rising Talent. As a middle entity, these platforms can settle dispute or provide assistance in case either side is in trouble, letting both sides focus on what matters to them – the business itself.

Upwork has awarded me with Rising Talent status thanks to having a good start and meeting several other criteria. (Tips on how to achieve a great start will be provided in the Tips section futher down below.)

Benefits – How Remote Work Marketplaces Improve One’s Skills

1. Opportunities to experience unfamiliar territories

(As a person who loves to explore, the first two points are the most important factors for me!)

While earning to make a living is important, being a technology enthusiast myself, having a platform where I can learn or practice desired skills in a real work environment with tech companies is also a huge motivator for me personally – there aren't really that many opportunities in real life as compared to remote work marketplaces with a large number of clients. In the case of Upwork, some of their high-profile clients are Airbnb and Microsoft.

2. Engage in tasks that prepare one for a target career

Extending the above point, with the large variety of opportunities available on remote work marketplaces, people can sharpen their desired skills with an aim of getting equipped with skills related to their dream jobs by first acquiring virtual work experience related to their jobs, even before they are hired in real life.

Once in a while, there are also clients that are job interview exam providers. They may offer special tasks which involve creating, reviewing or test-driving job interview questions (e.g. PowerShell coding test in the case of an Microsoft DevOps job interview). In case that meets one's dream job criteria, he/she would be able to experience a similar job interview process even before being admitted to the actual job interview.

Client looking for freelancers to test drive screening exercises used in hiring processes, e.g. software engineering

3. Get good with proposal writing, contract bidding

Through a platform like Upwork (Fiverr works differently though), freelancers get to sharpen their proposal-writing and contract-bidding skills through negotiation with a large varieties of clients. (These are soft skills that are invaluable in real life too!) Once we master the art of them, we can more easily attract clients, win contracts and shine on the platforms.

4. Familiarize with a pure remote work environment

Finally, this point brings the focus back to the main theme of this post - due to COVID-19, remote work has become a norm during and even after the pandemic, with which employees from traditional industries may not yet be familiar. In order to survive and thrive, one has to get familiar with the tools involved in a usual remote work setup, such as Zoom, Slack, Microsoft Teams, as well as the soft skills involved. On remote work marketplaces, one gets to experience a pure remote work environment to get prepared for the WFH future.

Tips – Shine on Remote Work Marketplaces

1. First few contracts are key

In order to have a good career on the platforms, it is important to pick clients who give 5-star ratings and pick jobs with which we are confident to overdeliver. Great ratings and comments from the first few jobs make future ones easier. For example, there are clients on Upwork who would prefer freelancers with at least 90% of success rate.

Having a good work history of successful jobs, five stars and good comments unlock future opportunities for new freelancers

2. Craft client-attracting profile and contract-winning proposals

Take time to research on how to write a client-attracting profile and contract-winning proposals for your specific industry, in order to truly shine on the platforms. I found the YouTube videos from Josh Burn Tech to be a big help to get me started. (Thanks!) I believe writing a good profile also helped me pass the newcomer screening of Upwork and reduced the number of Upwork credits I wasted during proposal submission.

You’re so welcome. Great work! 👍
— Josh Burns Tech (@JoshBurnsTech) June 6, 2020

3. Empower your screen-sharing demo on Zoom with AeroZoom

There are many times where I would have to remote-control a computer or perform a live demo on Zoom (or Cisco WebEx, Microsoft Teams, etc.) during remote work. I found my free Windows wheel-zooming application, AeroZoom magnifier 🔎 (which you may have been on Lifehacker and other media), helpful in enlarging small texts and elements on my computer screen conveniently with just a single mouse using Left mouse button + Wheel-up/down (or a Mac-like hotkey e.g. Alt key + Wheel-up/down), so that viewers and I can view more clearly and focus on specific area of screen zoomed in.

Zoom in and out one-handedly with AeroZoom screen magnifier

Conclusion – 3 Takeaways

Although country borders are closing and traditional opportunities are shrinking, we can still leverage opportunities on remote work marketplaces.
Owing to the shift from face-to-face work to remote work, we have to invest in skills and learn about tools that enable or supplement the new way of work. We can practice the use of them on remote work marketplaces.
If we lack opportunities to get good with desired skills required by our next stage of career in real life, we can get good at them first on remote work marketplaces, due to the large variety of jobs available on those platforms.

I hope you find this article helpful regarding my recent experience. I would be delighted if it gives you some new ideas on how to cope with the current lockdown or even the remote-work future. Finally, I hope we can be better together. ❤️️

Stay safe!

Feel free to visit my IT freelancing profiles:

Fiverr: https://fiverr.com/wandersick
Upwork: (Available upon request)

100 Tips for AeroZoom

noreply@blogger.com (wandersick) — Sun, 12 Apr 2020 15:30:00 +0000

If you are a user of AeroZoom, the magnifier-enhancing application for Windows, you might have noticed there is an option to display a random tip during startup of AeroZoom. To make it easier to refer to the tips, this post lists all of the 100 tips stored in AeroZoom for its 'Startup Tips' functionality.

So, here it goes!

New to AeroZoom? To get started, hold [Left] mouse button and wheel up to zoom. While zoomed, press and hold the middle button to preview full screen or reset zoom level. Press [Left] and [Middle] to reset zoom. Press [Left] and [Right] to show/hide AeroZoom Panel. Read 'Quick Instructions' via the help (?) menu for more controls.
Mac users are used to zoom with the hotkeys Command+[+] and Command+[-]. AeroZoom makes such combination possible on Windows. Press [Left] and [Right] to launch AeroZoom Panel and select one of the other modifiers (e.g. Alt, Ctrl) and see which one works best for you!
For some users, 'Alt' seems to work better than other key modifiers. How about you? Let us know about your experience with AeroZoom by completing our survery at "? > Survey".
The fastest way to visit the official AeroZoom web page (e.g. to leave a comment or report a bug) is within the application at '? > AeroZoom Web'.

If you always forget what to press for the hotkeys, just remember one: Win+Alt+Q. Press it and the Quick Instructions (a list of hotkeys) will be shown to guide you.
The option "Workaround: Prefer &NumpadAdd/Sub to +/-" in "Tool > Preferences" menu enables AeroZoom to use an alternative keyboard shortcut to zoom in/out with the Windows Magnifier. Trying a different combination of this setting might solve problems such as +/- characters being generated during zoom in/out. By default NumpadAdd is enabled while NumpadSub is disabled. This combination works best normally for Ctrl modifier. You may finetune this setting if it does not work best for you. If you mess it up, simply restore AeroZoom to the default settings.
Some modifiers may work better than others. For example, in one case, Shift modifier generates +/- characters while Ctrl does not. Please examine which modifier works best for you.
Have another utility that already uses Left+Right / Right+Left hotkeys? (e.g. Firefox add-on: MouseZoom) To disable AeroZoom panel mouse hotkeys, we can make use of the Custom Hotkeys feature in "Tool > Custom Hotkey > Settings > Enable Left/Right" where the settings of Left+Right / Right+Left actions can be changed from Show Panel to None. To enable this feature, check "Tool > Custom Hotkeys > Enable Left/Right".
Did you know empty areas on the AeroZoom Panel can be dragged? Let's call AeroZoom Panel by Left+Right mouse buttons and try it! (Now, dragging the AeroZoom Panel does not trigger AeroShake like older versions did.)
Did you know double-clicking empty area on AeroZoom Panel switches between ZoomIt Panel/AeroZoom Panel? Just like clicking 'zoom' button on the Panel.
Did you know single-clicking the tray icon easily disables/enables AeroZoom temporarily? (Just like the 'off' button.) Useful when the app you're going to launch is incompatible with AeroZoom.
Not sure what a function on AeroZoom Panel is for? Hover the mouse cursor over any buttons or items and a short explanation will (usually) show.
Left-handed users may wish to try the new right mouse button modifier by clicking 'Right' radio button at the bottom of AeroZoom Panel. It has replaced the separate program (AeroZoom_LeftHand.exe) in older versions.
Do you use Windows Media Center? There's a Windows bug that hides the cursor when both Windows Magnifier and Windows Media Center are running and in full screen. AeroZoom provides a workaround - the 'Kill magnifier' hotkey Win+Shift+K to end the Magnifier process so that the cursor shows in such case. If mouse is preferred over keyboard, we may also call the AeroZoom Panel then press 'Kill'. An easier way of doing it is customize a hotkey action, e.g. the middle mouse button, so that we can just hold it to kill magnifier. To do that, go to 'Tool > Custom Hotkeys > Middle' and set its action to 'Kill magnifier'.
Do you use WizMouse? WizMouse is incompatible with AeroZoom. If you use both programs, go to WizMouse's Settings and check 'Left click tray icon to enable/disable', so that left-clicking WizMouse's tray icon easily disables WizMouse and enables AeroZoom, or vice versa.
Did you know that AeroZoom Panel can be triggered by holding the Left mouse button then press Right, as well as first holding the Right then press Left? All modifiers supports this method.
Zoom Speed Slider is useful for adjusting how far to zoom on each scroll. Aside from the default keyboard shortcuts: Win+Alt+F1 to F6, custom hotkeys can be defined to 'zoom faster' and 'zoom slower'. As you press the hotkey, the change (new level) is reflected on the slider immediately.
Sometimes you might want to find out how far you've zoomed or how much more you can zoom. Open the AeroZoom panel and switch to Magnification slider from the 'Az' menu. As you zoom in/out, the slider updates in real-time to indicate which zoom level you are at.
If you start AeroZoom frequently, you may enable 'Az > Run on Startup' to improve your experience. This is different from simply moving a shortcut of AeroZoom to the 'Startup' folder. Because AeroZoom runs better with admin rights, the shortcut method will present a UAC prompt to ask for elevation, while the built-in 'Run on Startup' function does not. It makes things easier.
The middle mouse button modifier is capable of zooming with just one button, without the need of holding any modifier in advance (such as Left, Ctrl) before scrolling, because the middle button is already a modifier itself. Try clicking 'Middle' radio button at the bottom of AeroZoom Panel to enable this feature. (The hotkeys are slightly different though. Be sure to take a look at the bottom of 'Tool > Quick Instructions'.)
Another quick zoom option is called Elastic Zoom which is a feature of AeroZoom 3. Elastic Zoom allows us to quickly zoom in at something and zoom out just by holding and releasing the same button. That means 'zoom out' is done automatically after the button is released. This is useful for a quick look at something closely. To zoom in 'elastically', just hold Ctrl then hit CapsLock once for live zoom, or hold Shift then hit CapsLock once for still zoom (the latter requires Sysinternals ZoomIt). And when you're done, just release the Ctrl/Shift buttons to zoom out. That's Elastic Zoom!
Since AeroZoom 3, for cosmetic and performance reasons, the borders of the Misclick-Preventing Pad has been removed and it's set to totally transparent. If you prefer to show it, you may go to 'Tool > Preferences > Advanced Options' to increase its transparency value and to set 'Borders' to 'Yes'.
To quickly restart AeroZoom in case it behaves badly to a bug (please report!) or whatever the reason is, simply [1] click any modifier radio button on the panel, or [2] select Az > Restart on the panel, or [3] right click tray icon and select 'Restart', or [4] use this secret hotkey Win+Alt+Shift+R. Also see next tip which is related.
To restore AeroZoom to default settings in case it behaves weirdly due to bad configuration, [1] perform a reset in 'Tool > Preferences > Advanced Options', or [2] use this secret hotkey Win+Shift+Ctrl+Alt+R.
AeroZoom can be an assistive tool for the disabled. For example, If a person prefers an color-inverted screen as soon as the computer finishes booting, first invert the color in AeroZoom, then select 'Az > Run on Startup'. Then that PC will always starts with inverted screen color. By the way, pressing Win+Alt+I toggles color inversion. The color inversion function of Windows Magnifier performs better than the high contast themes that comes with Windows 7.
Did you know AeroZoom can be a handy screen-capturing tool? If you're using Vista/7 Home Premium or above, a new capture can be triggered in many ways, such as by holding the middle mouse button, or clicking the Snip button on AeroZoom Panel, thanks to AeroSnip (Snipping Tool enhancements). AeroSnip (beta) allows us to select any region of the screen for capturing and annotation as well. (Mac users who loves Command+Shift+4 or Linux Compiz users who love Winkey+dragging to capture regions of screen should like this feature.)
Did you know both Print Screen button and Snipping Tool have been enhanced to Save Captures automatically? (Desktop by default.) To enable the feature, click 'Tool > Save Captures'.
Mac users may know this hotkey: Command+Option+Control+8 to make screen color negative, while Linux (Compiz) users may know Shift+M/N. The AeroZoom-equivalent is Win+Alt+I.
'Preview Full Screen' is useful in case you've zoomed in somewhere and want to jump to another part quickly without zooming out first. AeroZoom lets you triggers a full screen preview by holding the Middle mouse button for half a second. What if we want it to come up sooner/later than 0.5s? What if we don't have an Middle button? We can go to 'Tool > Custom Hotkeys > Settings' to shorten/delay it as well as choosing another Hotkey for 'preview full screen'. (We may also use the default hotkey provided by Windows Magnifier: Ctrl+Alt+Space.)
Holding the middle button triggers 2 different actions dynamically. While zoomed in, 'Preview Full Screen' is triggered to preview other areas of the screen; while zoomed out, a new snip is triggered. To customize the action, go to 'Tool > Perferences > Custom Hotkeys > Middle'.
Since AeroZoom 3, 'ZoomIt auto switching' is disabled by default. If you prefer the old behaviour, it can be enabled in 'Tool > Custom Hotkeys > Middle', so that holding the middle button triggers the 'still zoom' of 'Sysinternals ZoomIt' when 'Tool > Use ZoomIt as Magnifier' is enabled.
In older operating systems or system without Aero, AeroZoom can add wheel-zoom support to Sysinternals ZoomIt. To enable the feature (if it is not automatically adjusted), click 'Tool > Wheel with ZoomIt (Live) or (Still)'.
Do you always unintendedly trigger the snip/preview function of the middle mouse button? You may increase the threshold in 'Tool > Preferences > Advanced Options' to delay it.
If, for some reasons, you don't want AeroZoom to trigger snip/preview while holding the middle mouse button triggers, you can easily disable it in 'Tool > Custom Hotkeys > Enable Holding Middle'.
Do you have any issues while using AeroZoom? Please report at https://tech.wandersick.com/p/aerozoom-for-windows-7-magnifier.html. If it is a known issue with workarounds, read the [Compatiblity] and [Limitation] section of the Readme at '? > About > Readme' or 'AeroZoom Web'.
Love AeroZoom? Want to recommend it to a friend? Why not install it to your/his local computer? Just click 'Az > Install as Current User'. The installation is per-user and does not require administrator rights.
AeroZoom supports customization of 30+ hotkeys as well as including 30+ built-in functions such as these less-known ones: Speak, Google, Eject Disc, Monitor Off. Also, any external command or custom apps can be specified for doing anything. Check out the feature at 'Tool > Custom Hotkeys'
If you have a mouse with Back/Forward buttons, you may use either mouse button as a modifier (i.e. as the button to hold for scrolling) by clicking [B] or [F] at the bottom of AeroZoom Panel. AeroZoom 3 also optionally makes more hotkeys out of the buttons. Please go to 'Tool > Custom Hotkeys' to find out what AeroZoom 3 is capable of.
In older versions of AeroZoom, we clicked [Show] on AeroZoom Panel to bring up Windows Magnifier, but [Mag] has superseded the [Show] button in later versions as it was improved to dynamically 'show' and 'hide' Magnifier, so it was renamed to reflect the change.
Besides clicking [Mag], try pressing Win+Shift+` (the key below Esc) to show or hide the magnifier.
For performance reasons, it is suggested not exiting (kill) Magnifier manually. Instead, use the 'Mag' or 'Reset' buttons on AeroZoom Panel whenever possible.
Use of the [Kill] button is unrecommended unless Lens or Docked modes are used often, because 1) to stop a magnifier in action, [Reset] performs better; 2) To hide the floating magnifier window and icon, just click [Mag] again. [Mag] can be used to show Magnifier as well as hiding it.
There are many ways to show and hide AeroZoom Panel: 1) Double-clicking on AeroZoom's tray icon. 2) Pressing Win+Shift+ESC. 3) Left+Right mouse buttons. 4) Any hotkeys customized in 'Tool > Custom Hotkeys'.
Timer and stopwatch are useful additions to presentations. We have three third-party choices with AeroZoom: (1) Break Timer of Sysinternals ZoomIt, which can be activated by pressing 'Timer' on ZoomIt Panel. (2) Timer Tab and (3) Aero Timer can be activated at 'Tool > Misc Tools'. Moreover, they can all be activated by a custom hotkey defined in 'Tool > Custom Hotkeys > Settings'.
Enabling 'Tool > Misc Tools > Legacy: Click-n-Go Buttons' makes clicking any button on AeroZoom Panel hide AeroZoom Panel after the action is complete. This was actually the behaviour in older AeroZoom versions but is then turned off by default in order to make AeroZoom easier for beginners. After you've got used to using AeroZoom you may wish to turn it on to maximize productivity.
ZoomIt Tip: To add or decrease time for the break timer of Sysinternals ZoomIt, use the mouse wheel or arrow keys.
ZoomIt Tip: Did you know you can draw or type while zooming with Sysinternals ZoomIt? Just left-click anywhere to draw, press T to type! You want a black board? Press K. Need a white one? Press W. Undo an edit? Press Ctrl+Z. Erase everything? Press E. To save a screen shot to clipboard? Press Ctrl+C. To save as PNG? Press Ctrl+S. For more functions, see '? > Quick Instructions > ZoomIt'
ZoomIt Tip: Did you know the pen color of Sysinternals ZoomIt can be changed not only via the ZoomIt Options? While drawing/zooming using ZoomIt, just press of the following keys R,G,B,O,P,Y. Yes, you guessed it. They stand for Red, Green, Blue, Orange, Pink, Yellow.
ZoomIt Tip: Did you know the font size of Sysinternals ZoomIt can be changed not only via the ZoomIt Options? While in text mode, scroll using the mouse wheel or use the arrow keys.
ZoomIt Tip: Never press Alt+F4 to leave the active window of ZoomIt; otherwise ZoomIt.exe might hang leaving a phantom process. Instead, press Esc key or right click.
Miss the classic mini AeroZoom interface? Switch to it by right-clicking the panel or at 'Az > Go to Mini View'.
'Always on top' is applied to AeroZoom Panel as well as buttons - [Calc], [Type] and [Paint]. Turning off 'Always on Top' from the 'Tool' menu immediately disable the 'Always on Top' flag on any existing Calculator/Notepad/Wordpad/Paint windows; turning it on does not make them 'always on top' again however, unless you run those apps from the buttons again.
Keyboard shortcuts are supported for lots of tasks, press [Alt] on AeroZoom Panel to see them. For global shortcuts, press Win+Alt+Q (equal to clicking menu '? > Quick Instructions').
Sometimes it is convenient to temporarily disable mouse/all hotkeys. There are 3 modes: 1) Click 'off' once on AeroZoom Panel or the tray icon to turn off mouse zoom hotkeys (equal to pressing Win+Alt+H). 2) Click it twice to turn off all hotkeys including keyboard shortcuts. 3) Click it again to go back to normal. Alternatively, users can right-click on the tray icon to access the function.
Misclick-Preventing Pad may not be accurate for some types of window. That's why you may want to fine-tune it in 'Tool > Preferences > Advanced Options'.
Usually unnecessary, but Misclick-Preventing Pad can stay a little longer by increasing the time in 'Tool > Preferences > Advanced Options'. (If you do so you will also find that the whole pad can be dragged, not just the title bar.)
Besides presentations, a less known AeroZoom usage is capturing screen videos for tutorial purpose. Where normally we need a lot of work to zoom in using professional video editors such as Adobe Premiere. With AeroZoom, simply zoom in with the magnifier hidden during recording, and it's done. No post-production required.
The main differences between running Calulator, Paint, Notepad/Wordpad from AeroZoom Panel buttons ([Calc], [Paint], [Type]) and running them from the menu 'Tool > Windows Tools' are: 1. Texts and actions of the former can be customized in Advanced Options. 2. The former are affected by the 'Always on Top' setting. 3. If an existing window is found, the former will switch to it instead.
Do you find Misclick-Preventing Pad slow for your computer? You may decrease the pad size to speed it up in 'Tool > Preferences > Advanced Options', or increase it if more coverage is wanted. If it's really too slow, consider disabling it via 'Tool > Misclick-Preventing Pad'.
When 'Follow Keyboard' is on, Misclick-Preventing Pad may shift the screen wrongly. An workaround is to use 'Follow Text' instead.
Although unrecommended, operations (e.g. clicking buttons on AeroZoom Panel) can be sped up by setting 'Operation delay' in 'Tool > Preferences > Advanced Options' to a lower value. If it's too low, some things may break; tweaking is required to find out the best value for your system.
Pressing buttons on the AeroZoom panel has no effect at times? You may have to set 'Operation delay' in 'Tool > Perferences > Advanced Options' to a higher value for your system.
As you've used AeroZoom for a while, you may have tweaked the best settings. You can export the configurations at 'Az > Configuration File > Export' so that you won't lost the settings next time the computer is reinstalled.
AeroZoom settings are stored in the registry for both the portable and installed versions instead of .ini files. If you bring AeroZoom to another computer, the settings will be lost. This is by design because each PC has their own optimal settings. (Read next tips for solution.)
While the program of AeroZoom is portable, its settings are not. They are stored locally in the registry. This design ensures AeroZoom won't be affected by inconsistency across different PCs. (Read next tip for solution.)
How to make AeroZoom's settings portable? Since AeroZoom 3, when the current user is an admin, AeroZoom backs up its configurations automatically in AeroZoom\Data\ConfigBackup\ on a normal exit. (Exiting from the tray icon does not back up, however.) Then, on another PC, the backup can be imported via menu 'Az > Configuration File > Import'. Note: Starting with a fresh config file is still recommended as settings may be incompatible across PCs, although AeroZoom tries its best to avoid problems. (If you find any, please report.)
Limitation: Misclick-Preventing Pad may be inaccurate for some types of window. You may fine-tuning the offset in Tool > Preferences > Advanced Options'.
Limitation: Zooming may be jerky when both misclick-preventing pad and 'Follow keyboard' are on. (Use 'Follow text' as a workaround.)
Limitation: Turning ZoomIt or its panel on and off too often may leave lots of phantom tray icons.
Limitation: The Magnification slider gets jerky or does not work in rare situations. A workaround to restore it to normal is to use keyboard shortcut (Win+[+]) to zoom in once.
Limitation: 'Az > Run on Startup' fails to automatically run magnifier at startup if there are UAC prompts caused by other programs during boot.
Limitation: During use of AeroZoom, Magnifier cannot be run separately. This is normal as AeroZoom hides Magnifier to improve user experience. To show it anytime, just click [Mag] from AeroZoom Panel (eq to Win+Shift+`). If always showing is preferred, go to 'Tool > Preferences > Advanced Options', change 'Magnifier' to 'Show'
Limitation: Pressing buttons on the AeroZoom panel has no effect at times? You may have to set 'Operation delay' in 'Tool > Preferences > Advanced Options' to a higher value for your system.
System requirement: AeroZoom runs best on Windows 7 Home Premium/Professional/Ultimate/Enterprise. Although it also supports Vista and XP, some functions will be limited.
System requirement: Aero (theme) is a must. Windows 7 Starter/Home Basic cannot enable Aero at all, so Lens and Full Screen zoom cannot be used. In such cases, we can check 'Tool > Use ZoomIt as Magnifier' so that AeroZoom adds Wheel-Zoom support to Sysinternals ZoomIt.
Special Installation: AeroZoom by default installs into Local App Data, To install to All Users or perform unattended installation, see Setup.exe /?. Unless installing to Program Files, the installation is per-user and does not require admin rights.
For an introductory video of AeroZoom recorded in 2009 (in robotic English), visit https://youtu.be/_Ovr40yXAzk
If you're not new to AeroZoom anymore, you may get tired of receiving message boxes at first run of each functions after reinstallations. You may check '? > Disable First-Use Guide' to disable them.
When a new version of AeroZoom is installed, any found old copy is upgraded. This is usually fine except new default settings won't apply as old settings are respected. So it's recommended to do a reset in 'Tool > Preferences > Advanced Options > Reset' anyway.
Why is there just a button for quitting on the panel but not one for restarting AeroZoom? That's because clicking any of the Ctrl/Shift/Alt/Left/Right/Middle/F/B modifier radio buttons restarts AeroZoom.
Prefer to use another hotkey to show AeroZoom Panel? Override the defaults in Tool > Custom Hotkeys > Settings > Left/Right
As an Windows admin, if you want your standard users to enjoy the full functionality of AeroZoom (e.g. Color Inversion), there's no need to give them admin rights; you can switch off UAC instead (if UAC is unimportant for you).
Once an version of AeroZoom had [Paint] button on the panel but since then it was replaced by [Kill] to aid HTPC users to work around a Windows Media Centre bug. In case you prefer [Paint], go to 'Tool > Misc Tools > Legacy: Change Kill to Paint'.
Using a mouse which cannot perform Left & Right clicks at the same time to bring up AeroZoom Panel? (e.g. Microsoft Wedge Mouse) Use the keyboard shortcut Win+Shift+ESC or the tray icon. The Microsoft Wedge Mouse has been tested to work under Windows 10.
The "Reset" button in Advanced Options will also remove zoomit.exe, nircmd.exe (if available) and restores Snipping Tool settings besides AeroZoom settings. If this is too strong for you, use the Restore Default Settings function in the tray menu. This will only restore AeroZoom settings.
Define your own set of AeroZoom settings in different Quick Profiles and switch to them whenever you need, under 'Az > Quick Profile Switch'.
Help improve the next version of AeroZoom by filling in the AeroZoom User Experience Survey Questionnaire at https://tech.wandersick.com/p/aerozoom-user-experience-survey.html
AeroZoom Quick Profiles record settings of AeroZoom and (but not Windows Magnifier, Snipping Tool, ZoomIt) and provide switching between different sets of settings quickly.nnProfiles are automatically saved when AeroZoom exits, restarts or switches to another profile.
Startup Tips, First-Use Guides and warning messages are not available when Quick Profiles are used.
Leverage the middle button of a mouse in the Holding Middle function to trigger "ZoomIt: Draw". This allows drawing with a mouse single-handed while zooming at the same time
To avoid misfire of left-clicks during PowerPoint presentations, leverage the per-slide setting of PowerPoint at "Transition > On Mouse Click" to disable left clicks to advance slides (Note: AeroZoom 5.0 has an improved mechanism that handles this built in. There should not be a need to do this unless you use mouse buttons other than Left or Right to zoom)
Limitation: Sysinternals ZoomIt supports customizing its hotkeys, but AeroZoom only supports its default hotkey configuration, i.e. Ctrl+1, Ctrl+2, Ctrl+3 and Ctrl+4
Limitation: Smart Click Disabler (available since v5.0) only supports the below modifiers: Left Mouse Button, Right Mouse Button, X1 Mouse Button (Browser Back), X2 Mouse Button (Browser Forward)
Limitation: AeroSnip (based on Snipping Tool) may not be further developed as the feature is deprecated by Microsoft (in Windows 10 1809) and may be removed in future versions of Windows
AeroZoom supports both Windows Magnifier and Sysinternals ZoomIt. For Magnification, Windows Magnifier is recommended being more smooth, but ZoomIt can be leveraged for other features that it provides such as on-screen drawing, still zoom, and timer
AeroZoom zooms all the monitors by default, not just the active one. To change this (i.e., configure AeroZoom to zoom only the active monitor). select the "Wheel with ZoomIt (Live)" option within the AeroZoom panel under the "Tool" section. This setting ensures that zooming actions are confined to the monitor currently in focus, preventing unintended zooming on other displays.
To be continued 😅

Do you have any tips to share? Feel free to leave them in a comment down below.

AeroZoom 5.0 New Features – Beta Testers Needed (UPDATE: Beta Program Closed)

noreply@blogger.com (wandersick) — Sat, 04 Jan 2020 17:23:00 +0000

The next version of AeroZoom (5.0) will be available after the testing phase, which involves my testing as well as yours – help improve quality by becoming a beta tester. Email wandersick@gmail.com if interested. (Thanks!)

Welcome to support this project by buying a cup of coffee if my work is useful to you. That would be much appreciated. Thank you!

Last updated: 24 Sep 2019

AeroZoom 5.0 Features (TBC)

Hide Magnifier Glass
- The ability to completely hide Magnifier glass (i.e. Windows Magnifier) is back for Windows 10 starting from 1607 and up (Note: This feature was available for Windows 7 only)
Smart Click Disabler
- Optimized single-handed mouse zooming, thanks to an improved mechanism to prevent misfire of left/right/forward/back clicks while zooming, available as a tool under AeroZoom Panel menu as Smart Click Disabler, which is better than the existing Misclick-Preventing Pad
- Enables use of AeroZoom during full-screen PowerPoint presentation, etc.
User-Contributed Settings
- Users can now share their mouse-optimized AeroZoom settings to benefit others. For example, SwiftPoint GT mouse-optimized settings are included in AeroZoom 5.0
  - User-Contributed Settings (Az > Quick Profile Switch) now accepts contributions from AeroZoom users who can share their AeroZoom profiles with other users
    - Simply export your config (Az > Config File > Export Settings) and email wandersick@gmail.com with a description. Your contribution will be considered for inclusion in the next version
  - In AeroZoom 5.0, User-Contributed Settings are included for the SwiftPoint GT mouse, which reserves Left+Right for its own trigger of middle button, as it does not have a standalone middle button.
    - With the feature, SwiftPoint GT users can now trigger AeroZoom Panel by pressing Left+Right. Enable User-Contributed Settings at "Az > Quick Profile Switch > User-Contributed Settings > 2. SwiftPoint GT" which enables the below settings:
      - Pick an action: Show/hide panel
      - How long to hold the middle button (in ms): 0
      - [Check] Disable Full Screen Preview auto switching
      - [Check] Disable ZoomIt auto switching
      - [Check] Disable Zoom Reset hotkey
    - Users with a normal mouse (i.e. with a middle button) may consider configuring Holding Middle function to trigger "ZoomIt: Draw" (a useful pen for drawing on screen) or other features instead
Run Prompt
- The command AeroZoom is now available in Run prompt if AeroZoom has been installed to the system
Center Zoom
- The Center Zoom feature allows zooming to start from the center of the screen, is now out of 'beta' and moved to a more accessible menu location (Tool > Center Zoom)
Support for Windows 10 1909 Update
- Rename tools in 'Windows Tools' to reflect recent Windows changes
Updates to AeroZoom Dependencies
- Base AutoHotKey updated to v1.1.30.01 (November 11, 2018)
- Update wget.exe to fix zoomit.exe download issue while retaining Windows XP support
More Deployment Options
- New unattended installer is available for download. While existing Setup.exe also support silent installation, it is a separate file next to AeroZoom program files, unlike the new installer which contains the AeroZoom program files. See https://github.com/wandersick/az-autohotkey-silent-setup
- Availability on Chocolatey community repository, https://chocolatey.org/packages/aerozoom, installable by command choco install aerozoom
UX Improvement
- Holding middle button now defaults to New Snip (of Snipping Tool) or Win+Shift+S if on a supported Windows 10 release
- Right-clicking AeroZoom panel edges no longer triggers a sudden confusing switch between mini and full user interface
Deprecation of AeroSnip
- AeroSnip feature is now deprecated (remains supporting Vista and 7) due to certain issues it has with Snipping Tool of recent Windows releases, as well as its uncertain future having been deprecated by Microsoft in late 2018
Integration with other wandersick utilities
- Add options for ChMac and Enhanced Command Prompt Portable (ECPP)
Bug Fixes
- AeroZoom Setup.exe, while uninstalls successfully, fails to delete AeroZoom_*x64.exe files due to being in use
- AeroZoom Setup.exe does not uninstall itself cleanly, leaving Setup.exe in the installation directory
- Download failure of zoomit.exe
- Update checker inaccurately reports an update exists
- Holding middle button triggers new snip despite being configured not to in the last release
- Hotkeys e.g. 'Left+Right' are not completely disabled even when set to 'None' via 'Custom Hotkey' options

How-to: Create a Silent Installer with AutoHotkey and Publish it on Chocolatey

noreply@blogger.com (wandersick) — Sun, 14 Jul 2019 11:44:00 +0000

Supposed there is a portable Windows application without an installer and uninstaller, how to create them back? In today's post, we will explore one way to build a Setup.exe using AutoHotkey (AHK), with additional compression of 7-Zip applied to the Setup.exe and remaining files of the portable application for maximum compression, and then wrap it with an outer unattended installer, turning a portable application into an installable one while being suitable also for silent deployment.

The application example, i.e. the application for which a setup is created is AeroZoom. While some terminologies are specific to AeroZoom, the general concepts should apply to other software.

// ⭐1️⃣ to be built: outer unattended installer written in AHK

AeroZoom_Unattended_Installer.exe
    │
    │   // 2️⃣ to be built: 7-Zip self-extracting archive
    │
    └───AeroZoom_7-Zip_SFX.exe
          │
          ├───AeroZoom // portable app example
          │
          │      AeroZoom.exe
          │      ...
          │
          │      // ⭐3️⃣ to be built: inner setup written in AHK
          │
          │      Setup.exe
          │
          └───Data

AeroZoom_Unattended_Installer.exe will be the outer unattended installer we will write in AutoHotkey (AHK) language, which contains 7-Zip SFX AeroZoom_7-Zip_SFX.exe and is responsible for extraction as well as calling an inner Setup.exe to install AeroZoom for all users silently
The second-level AeroZoom_7-Zip_SFX.exe will be our 7-Zip self-extracting archive (SFX). It contains the portable application files of AeroZoom and the inner Setup.exe

Use of a 7-Zip SFX automatically provides silent extraction parameters for the outer AeroZoom_Unattended_Installer.exe to leverage
The SFX will be zipped with the ultra compression option to effectively reduce the size of AeroZoom from 32MB to 2MB in the case of AeroZoom which contains multiple similar executables

The third-level Setup.exe, the inner setup, is the second AutoHotkey program we will write which provides installations parameters for the outer AeroZoom_Unattended_Installer.exe which can be leveraged e.g. by choco install command of Chocolatey, uninstallation parameters for choco uninstall, as well as logic to determine several things such as whether AeroZoom has already been installed, which will be detailed in the last section

After walking through how to create AeroZoom_7-Zip_SFX.exe and AeroZoom_Unattended_Installer.exe (goal #1), we will take a detour to go through how to push this unattended installer to the community repository of Chocolatey (goal #2), the package manager for Windows, before going back to detailing how to create the inner Setup.exe (goal #3).

This how-to will be centered around our code repository on GitHub:

https://github.com/wandersick/az-autohotkey-silent-setup

Table of Content

Let's go and create all those exe files above!

Some Trivia of AeroZoom (before We Begin)

As you may have seen on Lifehacker and other media, AeroZoom magnifier is a Windows application I created to enhance Windows Magnifier and optionally Sysinternals ZoomIt to enable screen magnification by mouse-wheeling, as well as turning any mouse into a Home-Theater PC/presentation mouse, where zooming and positioning becomes a breeze without a keyboard

Originally, AeroZoom was built as a portable application. Its Setup.exe was introduced in a later version, v2.0, and the unattended setup AeroZoom_Unattended_Installer.exe, the first outcome of this article, was introduced in v4.0.

1. Step-by-Step: Building Outer Unattended Installer Using AutoHotkey

This first section walks through how to create the outer unattended installer, AeroZoom_Unattended_Installer.exe, written in AutoHotkey:

In Command Prompt, download or git clone our code repository to a desired directory e.g. c:\az-autohotkey-silent-setup

cd /d c:
git clone https://github.com/wandersick/az-autohotkey-silent-setup.git

So that the following folder structure is acquired

C:\az-autohotkey-silent-setup
│   ...
│
│   // AHK code of the outer unattended installer to be customized
│
│   AeroZoom_Unattended_Installer.ahk
│
│   // optional: for use with Compile_AHK II (alternative compiler)
│
└───AeroZoom_Unattended_Installer.ahk.ini

Download AeroZoom and extract it to a desired directory, e.g. C:\AeroZoom

C:\AeroZoom
│   AeroZoom.exe
│   Readme.txt
│   Setup.exe
│
└───Data

Build an inner Setup.exe using AutoHotkey following these instructions in the third and final section to learn the process, or leave it as is for now and learn how to build it later (recommended for now)
- Ensure Setup.exe is next to AeroZoom
```
C:\AeroZoom
│   AeroZoom.exe
│   Readme.txt
│
│   // if you manually build this Setup.exe, replace existing one
│   // otherwise, no action is required
│
│   Setup.exe
│
└───Data
```
Package (compress) the above C:\AeroZoom application directory in a 7-Zip SFX (self-extracting archive)
- Optionally, this step can be skipped by leveraging the AeroZoom download, which already comes with an SFX AeroZoom_v4.0.0.7_beta_2.exe after extraction
- Put the SFX file there and rename it as AeroZoom_7-Zip_SFX.exe
```
C:\az-autohotkey-silent-setup
│   AeroZoom_Unattended_Installer.ahk
│   AeroZoom_Unattended_Installer.ahk.ini
│   README.md
│   ...
│
│   // place SFX (containing C:\AeroZoom) built using 7-Zip here
│
└───AeroZoom_7-Zip_SFX.exe
```

Place an icon named AeroZoom_Setup.ico there (optional)

C:\az-autohotkey-silent-setup
│   AeroZoom_7-Zip_SFX.exe
│   AeroZoom_Unattended_Installer.ahk
│   AeroZoom_Unattended_Installer.ahk.ini
│   README.md
│   ...
│
│   // optionally, place an icon here
│
└───AeroZoom_Setup.ico

Edit AeroZoom_Unattended_Installer.ahk and change below C:\az-autohotkey-silent-setup\AeroZoom_7-Zip_SFX.exe to a desired location (no change if directory is the same as the example)

As shown below, the AutoHotkey source code of AeroZoom_Unattended_Installer.exe is relatively simple. It only contains three lines (excluding comments):

; Package an application (e.g. AeroZoom) in 7-Zip SFX
; (FYI: the AeroZoom download already comes with an SFX)
; Place it in the location specified below
; e.g. C:\az-autohotkey-silent-setup\AeroZoom_7-Zip_SFX.exe

FileInstall, C:\az-autohotkey-silent-setup\AeroZoom_7-Zip_SFX.exe, %A_ScriptDir%\AeroZoom_7-Zip_SFX.exe, 1

; Silently extract AeroZoom from SFX into the current folder

RunWait, %A_ScriptDir%\AeroZoom_7-Zip_SFX.exe -o"%A_ScriptDir%" -y

; Run silent setup command: Setup.exe /programfiles /unattendaz=1
; For AeroZoom, thse parameters will perform installation:
; - silently (/unattendedaz=1)
; - to All Users (/programfiles)
; Or uninstall in case AeroZoom is found in the target folder
; (built into the logic of Setup.exe of AeroZoom)

RunWait, %A_ScriptDir%\AeroZoom\Setup.exe /programfiles /unattendaz=1

Download and install AutoHotKey (specify 32-bit in the setup wizard to maximize compatibility of the compiled executables)
To compile the AutoHotkey script as an executable, download and install Compile_AHK II.
- While under repository directory (e.g. C:\az-autohotkey-silent-setup), right-click AeroZoom_Unattended_Installer.ahk and select Compile with Options which would parse parameters from AeroZoom_Unattended_Installer.ahk.ini
- The ini file has an option named Execution_Level=4 which means requireAdministrator (run as administrator). This is required for the installer to carry out installation successfully.
- While Compile_AHK II comes with compression feature, this article uses 7-Zip as 7-Zip reduces the file size much better (from 32MB to 2MB) in our case.
Alternatively, the default compiler which comes with AutoHotkey can be used instead of Compile_AHK II if preferred.
- While under repository directory (e.g. C:\az-autohotkey-silent-setup), compile AeroZoom_Unattended_Installer.ahk using the bundled AHk2Exe utility, usually located under C:\Program Files\AutoHotkey\Compiler as so:
- "C:\Program Files\AutoHotkey\Compiler\Ahk2Exe.exe" /in "AeroZoom_Unattended_Installer.ahk" /icon "AeroZoom_Setup.ico"
  - Icon parameter is optional: /icon "AeroZoom_Setup.ico"
- For me, I prefer Compile_AHK II than the official compiler in our case where the installer requires “Run as Administrator” rights. Compile_AHK II has built in support of Execution_Levels while the official compiler has other means to achieve it.
To each their own, choose a method with which you are comfortable. If you know of a better way, feel free to let me know!
Done. Now executing AeroZoom_Unattended_Installer.exe would silently trigger an extraction of 7-Zip SFX AeroZoom_7-Zip_SFX.exe and calls the inner AeroZoom Setup.exe to install AeroZoom for all users with its unattended installation parameter /programfiles /unattendAZ=1 (which will be further explained in the third and final section)

Goal #1: The end result, AeroZoom_Unattended_Installer.exe, is available for download here (as AeroZoom_v4.0.0.7_beta_2_silent_installer.exe).

About False-Positive Notices from Anti-virus Software

Like certain AutoHotkey applications, some anti-virus vendors may falsely identify the unattended installer created as undesirable. It is understandable considering the nature of this application is to perform installation and uninstallation in a silent way, which involves seemingly risky tasks such as modifying uninstallation keys in the registry.

On the other hand, I found that compiling using Compile_AHK II and/or an icon helps lower the detection rates a little bit (from 12 down to fewer than 10 out of 65 anti-virus applications on VirusTotal).

Still, this false-positive detection issue may not be completely eliminated and can be safely ignored. It is recommended to report your file to your anti-virus vendor as clean if possible.

2. Pushing Unattended Installer to Chocolatey

Thanks to this Chocolatey how-to article by Coffmans and guidance from Chocolatey, I was able to figure out how to push the outer unattended installer to the Chocolatey community repository here.

The Chocolatey package specification files customized for AeroZoom have been included in the git repository downloaded in step #1 of the above section which was initially created with the below `choco new ` command after Chocolatey has been downloaded and installed on a supported Windows machine.

Let's walk through the folder structure:

C:\az-autohotkey-silent-setup\Chocolatey\AeroZoom
│   ReadMe.md
│   _TODO.txt
│   ...
│
│   // 1️⃣ package metadata
│
│   aerozoom.nuspec
│
└───tools

        // 2️⃣ tasks run before [un]install scripts

        chocolateybeforemodify.ps1

        // 3️⃣ edited Chocolatey install script
        //    (for use with 'choco install')

        chocolateyinstall.ps1

        // 4️⃣ edited Chocolatey uninstall script
        //    (for use with: 'choco uninstall')

        chocolateyuninstall.ps1

aerozoom.nuspec contains the below self-describing package metadata (click the hyperlink to see the AeroZoom example with comments)
- id
- version
- packageSourceUrl
- owners
- title
- authors
- projectUrl
- iconUrl
- licenseUrl
- requireLicenseAcceptance
- projectSourceUrl
- docsUrl
- mailingListUrl
- bugTrackerUrl
- tags
- summary
- description
- releaseNotes
chocolateybeforemodify.ps1 performs pre-checks prior to installation and uninstallation. For example, to prevent running processes from conflicting with setup, we can use PowerShell cmdlet Stop-Process to forcefully terminate processes

I learned it is important to add -ErrorAction SilentlyContinue so that users do not see error texts when the command runs without any process to stop
```
Stop-Process -ProcessName aerozoom* -Force -ErrorAction SilentlyContinue
```

chocolateyinstall.ps1 performs silent installation by downloading and using an unattended installer, which is the outcome from the first section of this article. It should be hosted on a publicly accessible web server in order for this Chocolatey installation script to leverage
```
$url = 'https://github.com/wandersick/aerozoom/releases/download/4.0.2/AeroZoom_v4.0.0.7_beta_2_silent_installer.exe'
```
chocolateyuninstall.ps1 performs silent uninstallation by using a parameter specific to the application installer, which is /programfiles /unattendAZ=1 in the case of AeroZoom. (The parameters will be further explained in the third and final section)
```
silentArgs    = "/programfiles /unattendAZ=1"
```

One AeroZoom-specific side-note:

Due to the way Setup.exe of AeroZoom works, for chocolateyuninstall.ps1, I have added Substring (from PowerShell) to remove /programfiles /unattendAZ=2 from UninstallString (in Windows registry), replacing it with /programfiles /unattendAZ=1 from $silentArgs (in chocolateyuninstall.ps1). Otherwise, uninstallation would not be performed in an unattended way
```
$packageArgs['file'] = "$($_.UninstallString)".Substring(0, $_.UninstallString.IndexOf(' /'))
```

Once the PowerShell scripts have been edited, below are the commands for packaging, testing (installing and uninstalling), as well as pushing to the Chocolatey community repository

# create NuGet package (aerozoom.4.0.0.7.nupkg)
# ensure current directory is correct

cd az-autohotkey-silent-setup\Chocolatey\AeroZoom
choco pack

# install aerozoom.4.0.0.7.nupkg by running:
# - chocolateybeforemodify.ps1
# - chocolateyinstall.ps1

choco install aerozoom.4.0.0.7.nupkg

# uninstall aerozoom by running:
# - chocolateybeforemodify.ps1
# - chocolateyuninstall.ps1

choco uninstall aerozoom

# push to community repo (--api-key required)

choco push aerozoom.4.0.0.7.nupkg

Goal #2: Having gone through the moderation review process, AeroZoom is available on Chocolatey community repository now.

AeroZoom can now be installed via Chocolatey repository:

"choco install aerozoom" https://t.co/ai8s3WvJM7 pic.twitter.com/9b3ui770ZL
— wandersick (@wandersick) July 26, 2019

3. Step-by-Step: Building Inner Setup Using AutoHotkey

This final section walks through how to create the inner Setup.exe acquired after 7-Zip extraction.

The steps for building the Setup.exe would be:

First, let's delete Setup.exe in the AeroZoom folder
If not already done so, the files below can be downloaded here and extracted to a desired directory, e.g. C:\AeroZoom
```
C:\AeroZoom
│   AeroZoom.exe
│   Readme.txt
│   ...
│
│   // Setup.exe will be built using below method here
│
└───Data
```
Acquire the source code of Setup.exe, i.e. Setup.ahk
If not already done so, open Command Prompt, download or git clone our code repository to a desired directory e.g. c:\az-autohotkey-silent-setup
```
cd /d c:
git clone https://github.com/wandersick/az-autohotkey-silent-setup.git
```
So that the following folder structure is acquired
```
C:\az-autohotkey-silent-setup
│   Setup.ahk.ini
│   ...
│
│   // source code of Setup.exe of AeroZoom
│
└───Setup.ahk
```
If not already done so, download and install AutoHotKey (specify 32-bit in the setup wizard to maximize compatibility of the compiled executables)
To compile the AutoHotkey script into an executable, download and install Compile_AHK II.
- While under repository directory (e.g. C:\az-autohotkey-silent-setup), right-click Setup.ahk and select Compile with Options, which would parse parameters from Setup.ahk.ini
- The ini file contains Execution_Level=3 which means highestAvailable (run with highest privileges available) instead of requireAdministrator (run as administrator) as the Setup.ahk supports dynamically installing into current user profile (when no administrator rights are available) or installing into system directory e.g. C:\Program Files (x86) (when administrator rights are available and Setup.exe /programfiles is specified)
Done

Goal #3: The end result, Setup.exe, can be used to install and uninstall AeroZoom, provided that it is located in the below directory, and that suitable unattended installation parameters are passed (see Setup.exe /?), or the installation is performed in an normal attended way (i.e. double-clicking the Setup.exe and follow the on-screen installation instructions).

C:\AeroZoom
│   AeroZoom.exe
│   Readme.txt
│   ...
│
│   // Place the newly created Setup.exe in this folder
│   // It can install and uninstall AeroZoom here
│   Setup.exe
│
└───Data

If you would like, you may also continue from step 3 of the first section to leverage this newly created Setup.exe to create the outer unattended installer again, which enables a totally silent installation experience of AeroZoom without the need to supply any parameters.

How It Works

Instead of using a popular approach such as Windows Installer, AeroZoom implements Setup.exe from the ground up, which has its own parameters:

/programfiles for installing into C:\Program Files (x86) (or C:\Program Files for 32-bit OS) instead of current user profile, i.e. %localappdata%, usually C:\Users\{username}\AppData\Local
/unattendAZ=1 for an unattended installation

If it detects AeroZoom is already installed, the setup will perform an uninstallation instead

If /unattendAZ=2 or any other values, an uninstallation dialog box will also be prompted

targetDir=%localappdata%
If %1% {
  
  ; For each parameter

  Loop, %0%
  {
    ; Retrieve A_Index variable contents

    param := %A_Index%
    If (param="/unattendAZ=1")
      unattendAZ=1
    Else if (param="/unattendAZ=2")
      unattendAZ=2
    Else if (param="/programfiles")
    {
      targetDir=%programfiles%
      setupAllUsers=1
    }
    Else
    {
      Msgbox, 262192, AeroZoom Setup, Supported parameters:`n`n - Unattended setup : /unattendAZ=1`n - Install for all users : /programfiles`n`nFor example: Setup.exe /programfiles /unattendaz=1`n`nNote:`n - If setup finds a copy in the target location, uninstallation will be carried out instead.`n - If you install into Program Files folder, be sure you're running it with administrator rights.
      ExitApp, 5
    }
  }
}

During installation or uninstallation, Setup.exe would first check a few prerequisites, such as whether files it depends on exist or not, as well as terminating any executables that could be running.

Once it is OK to proceed with installation, Setup.exe would create shortcuts:

IfExist, %targetDir%\wandersick\AeroZoom\AeroZoom.exe
{
  ; Create shortcut to Start Menu (All Users)

  If setupAllUsers
  {
    FileCreateShortcut, %targetDir%\wandersick\AeroZoom\AeroZoom.exe, %A_ProgramsCommon%\AeroZoom.lnk, %targetDir%\wandersick\AeroZoom\,, AeroZoom`, the smooth wheel-zooming and snipping mouse-enhancing panel,,
    FileCreateShortcut, %targetDir%\wandersick\AeroZoom\AeroZoom.exe, %A_DesktopCommon%\AeroZoom.lnk, %targetDir%\wandersick\AeroZoom\,, AeroZoom`, the smooth wheel-zooming and snipping mouse-enhancing panel,,
  }
  ; Create shortcut to Start Menu (Current User)

  Else
  {
    FileCreateShortcut, %targetDir%\wandersick\AeroZoom\AeroZoom.exe, %A_Programs%\AeroZoom.lnk, %targetDir%\wandersick\AeroZoom\,, AeroZoom`, the smooth wheel-zooming and snipping mouse-enhancing panel,,
    FileCreateShortcut, %targetDir%\wandersick\AeroZoom\AeroZoom.exe, %A_Desktop%\AeroZoom.lnk, %targetDir%\wandersick\AeroZoom\,, AeroZoom`, the smooth wheel-zooming and snipping mouse-enhancing panel,,
  }
}

Next, Setup.exe would write uninstallation information such as UninstallString to the Windows Registry:

If setupAllUsers
{
  ; If AeroZoom was installed for all users
  
  RegWrite, REG_SZ, HKEY_CURRENT_USER, %regKey%, UninstallString, %targetDir%\wandersick\AeroZoom\Setup.exe /unattendAZ=2 /programfiles
}
Else
{
  ; If AeroZoom was installed for current user

  RegWrite, REG_SZ, HKEY_CURRENT_USER, %regKey%, UninstallString, %targetDir%\wandersick\AeroZoom\Setup.exe /unattendAZ=2
}

Other information it writes there are:

Publisher
DisplayVersion
InstallLocation
URLInfoAbout
HelpLink
InstallDate
DisplayName
DisplayIcon
EstimatedSize

Regarding uninstallation, it is simply the above process in reverse, i.e. deleting shortcuts and removing the registry entries.

One thing worth mentioning is previous version of AeroZoom installer (v4.0 and before) would leave Setup.exe under the installation directory. Since v5.0, this has been solved by calling a minimized cmd.exe /c to ping localhost followed by the self-destruction, in order for Setup.exe to delete itself.

; Remove directory content including Setup.exe that is running
; after 3 seconds using ping in a minimized Command Prompt

Run, cmd /c start /min ping 127.0.0.1 -n 3 >nul & rd /s /q "%targetDir%\wandersick\AeroZoom"

That's it.

For simplicity, only the essential parts of Setup.exe are described above. Things that are specific to AeroZoom (e.g. removing scheduled tasks that AeroZoom may have created if specified by user) are not mentioned. For the rest, please refer to the comments in the source code file Setup.ahk and the comments inside.

It would be my pleasure if you find this article informative. Welcome to comment if you have any question or suggestion. Have a nice day!

Tip-Packed Gem from Microsoft Ignite 2018 – Windows Life Hacks

noreply@blogger.com (wandersick) — Sat, 06 Jul 2019 05:34:00 +0000

I came across this top session, 45 Life Hacks in 45 Minutes, from Microsoft Ignite 2018 which I consider to be a gem to Windows enthusiasts. It is packed with useful Windows techniques, many of which are advanced while practical.

This blog post merely summarizes the interesting tips from the session as an entry point – to experience the amazingness of the live demos, please go watch the recorded session on YouTube and look for other works by Sami Laiho from whom I learned a lot.

1. Exit Explorer

Windows 10/2016

Right-click taskbar > Exit Explorer

Windows 8/2012

Press Alt+F4 to bring up shutdown dialog. Next, while holding Ctrl+Shift+Alt, click Cancel

2. Violate a Group Policy which e.g. only allows running IE, but not Command Prompt or others

In Task Manager > Run New Task
It always open a Command Prompt

3. Redo old commands from history without typing again

F7 hotkey
History of Command Prompt or PowerShell

4. Tab auto completion in Command Prompt

*.xlsx then tab in Command Prompt only tab-completes files with *.xlsx extension. Works with any extensions

5. Copy errors in dialog boxes

Copy error messages from error dialog by Ctrl+C

6. Perform OCR (Optical Character Recognition) with OneNote

Perform a capture in OneNote
Right-click pictures and select copy text

7. Figure out error code customer provided without error message

Decimal

net helpmsg 4006

winrm helpmsg 0x65

8. Client-server signaling before attempting a next step

WAITFOR [/T timeout] signal
Description

This tool sends, or waits for, a signal on a system. When /S is not specified, the signal will be broadcasted to all the systems in a domain. If /S is specified, then the signal will be sent only to the specified system.

9. Send command output to clipboard

Command Prompt

ipconfig | clip

PowerShell

Get-Service | Set-Clipboard

10. PSExec can encrypt credentials over the wire

Joke

Who uses PSExec now that we have PowerShell?
Mark Russinovich: "How can you remotely enable PowerShell remoting?"

11. Make fun by remotely executing some app under 'local system' user

It will be hard for the remote user to get rid of the app (strip off the frame)
psexec -sid \\student25 "c:\program files (x86)\Internet Explorer\iexplore.exe" -K http://www.bing.com

12. SYSTEM account is comparable to root of Linux

13. Run multiple registry editors

regedit /m

14. Run regedit under SYSTEM account to view secrets at HKLM\SECURITY\Policy\Secrets

15. Cool registry location but too long to remember and type

Add to favorites

16. HiveList

Registry location that indicates where the physical files of registry are
HKLM\SYSTEM\CurrentControlSet\Control\hivelist
e.g. For SAM, i.e. \REGISTRY\MACHINE\SAM

\Device\HarddiskVolume5\Windows\System32\config\SAM

17. Process Hacker: Impersonate as any logged-on users under another user session

Under Process Hacker, right-click explorer.exe > Miscellaneous > Run as this user...
If the other logged user is a domain admin, now you are the domain admin
Good for service troubleshooting. Impersonate as local system

18. Autoruns: Fix unbootable issue due to driver after Windows 10 upgrade

Analyze offline system with Sysinternals Autoruns under Windows PE
Boot from Windows installation media (or Windows PE)
Uncheck driver

19. Resource Hacker: Change string of Windows 10 new user message ("How are you doing?" "This won't take long", e.g. to fun messages)

cd C:\Windows\System32\oobe\en-US\
Edit msoobeFirstLogonAnim.dll.mui with Resource Hacker

20. Sysinternals Strings: Use strings.exe to go thru whole binary and it shows binaries that can be used

cd C:\Windows\System32\oobe\
Strings.exe FirstLogonAnim.exe > temp.txt
Change font and speed with FirstLogonAnim.html

21. Run notepad.exe but starts calc.exe instead

HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options

What gets executed when you type something

Create a new key of an application, e.g. "notepad.exe"
Create a new REG_SZ type string of value "Debugger" and data "calc.exe"

22. Remove Windows 10 New User Logon Animation

Continued from above, create a new key of "FirstLogonAnim.exe"
Create a new REG_SZ type string of value "Debugger" and data "NULL"

23. Leverage FirstLogonAnim.exe to display some other strings to block user from doing anything during any tasks, e.g. some SCCM task sequence

ChMac – Windows Command to Change MAC Addresses of Network Adapters

noreply@blogger.com (wandersick) — Sun, 30 Jun 2019 12:31:00 +0000

A quick Windows batch CLI tool to change or randomize network adapter MAC address for security, or to work around usage limit of public Wi-Fi hotspots, either auto or manually. The command – chmac – changes MAC address in an easy-to-use interactive console alongside CLI parameters

Named after getmac and chmod, chmac is a command-line-interface (CLI) tool for Windows that changes or randomizes MAC addresses of specified network adapters, e.g. for a client device to reuse public Wi-Fi hotspot that has exceeded usage limit for the day (e.g. hotel, restaurants), or to enhance security.

An easy-to-use interactive console is available, alongside command-line parameters, e.g. for scheduling jobs with Task Scheduler. ChMac also has built-in support for recurrence.

Go to Download

Also on Chocolatey: choco install chmac / winget: winget install chmac

For a quick start, refer to Examples section below; for screenshots, refer to Screenshots section.

List of Features

Change MAC addresses on Windows automatically or manually
- Automatically change MAC addresses on set intervals
  - Useful to reconnect to free public Wi-Fi hotspots that impose a time limit by recognizing MAC addresses to prevent the same device from reconnecting
Randomize MAC addresses for better security using public Wi-Fi
- Generate new MAC addresses randomly based on a customizable list of organizationally unique identifiers (OUI)
Optionally leverage DevCon.exe to simply the process by automatically disabling and re-enabling network interface card (NIC)
- ChMac also works without DevCon by showing the Network Connections folder when finished, so that users can manually disable and re-enable NIC for new settings to take effect
- On first launch, users are guided to download DevCon with convenient automatic and manual options
Error checking and rich return codes for scripting and other possibilities
Free and open-source software written in Windows batch language
Portable by default. Optional installer enables chmac command globally (in Command Prompt and Run prompt. For PowerShell, the command would be chmac.bat)
Supports Windows 2000/XP/Vista/8/10 and Server 2003/2008/2012/2016/2019
Easy-to-use interactive console as well as accepting command-line parameters

Just follow instructions on screen for using the interactive console. For command-line mode, see the following

How It Works

ChMac works by querying a list of network adapters from Windows registry under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E972-E325-11CE-BFC1-08002BE10318}.

Based on user selection, ChMac updates the registry key responsible for the MAC address, `NetworkAddress`, of the corresponding adapter with an inputted or randomized MAC address from customizable OUI lists, either ad-hoc or automatically with Task Scheduler or its built-in auto-changing mechanism.

After MAC address is updated, the network adapter will be automatically disabled and re-enabled if an OS-specific DevCon.exe (a separate download from Microsoft) is available in the OS (the environmental variable, `PATH`) or under ChMac dependency directory, `ChMac\Data\3rdparty`. For more details, refer to Dependencies section.

If DevCon is unavailable, the Network Connections window will be presented to user where network adapter can be disabled and re-enabled manually.

Syntax

chmac.bat [/d dir][/l][/m address][/n id][/r][/help][/?]

Parameters

/d dir        :: working directory - maybe required
                 (MUST be specified before other parameters)
/l            :: list network adapters and their IDs
/m            :: new mac address to be applied.
/n            :: adapter to be applied new mac address if /m is unspecified
                 new mac address will be randomized (OUI kept) and automatically filled in
/r            :: restore to original MAC address
/a            :: auto-change interval
                 suffix may be s for seconds, m for minutes, h for hours or d for days
                 e.g. enter '20m' to recur every 20 mins. [X] to reset or exit

When no parameter is specified, interactive mode is entered on launch.

MAC Address Format

The MAC address format can be any of the following:

AB-CD-EF-12-34-56
AB:CD:EF:12:34:56
AB.CD.EF.12.34.56
AB CD EF 12 34 56            :: without any separation in between at all

Note: To increase success rate of changing MAC addresses in Windows Vista and later OS, the OUI lists included in ChMac contains MAC addresses which starts with 02 only

Examples

Command-Line Mode

chmac /l                     :: list available adapter IDs
chmac /n 1                   :: update network adapter #1 with randomized mac address numbers
chmac /m 00301812AB01 /n 2   :: update network adapter #2 with the new mac address: 00-30-18-12-AB-01
chmac /n 3 /r                :: restore adapter 3 to its original MAC
chmac /n 4 /a 20m            :: auto-change MAC per 20 minute
chmac /?                     :: shows short help. [/help] for long

Interactive Mode

Simply run chmac.bat and follow on-screen instructions.

File Structure

ChMac                      :: Main directory
│   chmac.bat              :: Main script
│   Setup.exe              :: Optional installer enabling chmac command globally (in Command Prompt and Run prompt)
│   Readme.txt             :: Documentation
│
└───Data
    │   OUI_NT5.txt        :: Customizable list of organizationally unique identifiers for Windows XP/2003 and earlier
    │   OUI_NT6.txt        :: Customizable list of organizationally unique identifiers for Windows Vista and later
    │   _choiceMulti.bat   :: Sub-script by wandersick for providing a fallback to systems without choice.exe (XP/2003)
    │   _choiceYN.bat      :: Sub-script by wandersick for providing a fallback to systems without choice.exe (XP/2003)
    │
    └───3rdparty           :: Dependencies put to the top of PATH (higher priority than system PATH entries)
        │   grep.exe       :: GNU Linux utility included
        │   libeay32.dll   :: GNU Linux library included
        │   libiconv2.dll  :: GNU Linux library included
        │   libintl3.dll   :: GNU Linux library included
        │   libssl32.dll   :: GNU Linux library included
        │   sed.exe        :: GNU Linux utility included
        │   sleep.exe      :: GNU Linux utility included
        │   tr.exe         :: GNU Linux utility included
        │   wc.exe         :: GNU Linux utility included
        │   wget.exe       :: GNU Linux utility included
        │   which.exe      :: GNU Linux utility included
        │   devcon.exe     :: Optional: place OS-specific devcon.exe from Microsoft here
        │
        └───LP             :: Dependencies put to the bottom of PATH (lower priority than system PATH entries)
                Readme.txt :: Reminders for LP directory
                choice.exe :: Optional: already included in OS since Windows Vista
                getmac.exe :: Optional: already included in OS since Windows XP/2003
                reg.exe    :: Optional: already included in OS since Windows XP/2003

OUI Files

OUI (organizationally unique identifier) files are available in ChMac. Add desired OUIs there for randomization while using the [G] command to generate a new MAC address.

Not all OUIs work - begin with 02 for a higher success rate on Windows Vista or later.
- For technical details, see: http://blog.technitium.com/2011/05/tmac-issue-with-wireless-network.html

Filenames:

OUI_NT5.txt - for Windows 2000, XP and 2003
OUI_NT6.txt - for Windows Vista and later

File Content (Default):

Vendor=OUI
3Com=020475
Atheros=021374
Broadcom=020AF7
Cisco=020D28
Intel=020CF1
Linksys=020E08
Bigfoot=021903
Marvell=025043
NVIDIA=02044B
Realtek=02E04C
VIA=02127B

Step-by-Step Screenshots

Command-Line Mode

Display short help - chmac /?

1: List available network adapters - chmac /l

2: Change MAC address of adapter #6 with a random MAC address - chmac /n 6

3: In case DevCon.exe is unavailable, Network Connections window will automatically be presented where user can manually disable and re-enable adapter

The above is only needed if DevCon.exe unavailable. Refer to Dependencies section below for more information.

4: MAC address successfully changed

Interactive Mode

5: Interactive mode (with an additional option for auto-changing) - chmac.bat

6: Interactively configure auto-changing interval with a 20-min interval - 20m

7: Afterwards, select adapter #6 so that its MAC address is auto-changed every 20 mins - 6

8: MAC address successfully changed and waiting for next auto-changing cycle

9 (Bonus): Press R after choosing an adapter (while not in auto-changing mode) to restore original MAC address - R

Likewise, if required, press O to edit OUI lists using system-default text editor.

Requirements

All Windows operating systems from Windows 2000 and up (Windows 10 1809 at the moment) are supported.

To use this in Windows 2000 or some minimal Windows PE, place these files from another machine which runs English Windows XP or 2003: msvcp60.dll as well as getmac.exe and reg.exe into ChMac\Data\3rdparty\LP.

Admin Rights

Admin rights are required for editing MAC addresses, disabling and re-enabling network adapters.

ChMac does not automatically elevate itself if there is no admin rights. Although there is error checking mechanism for being non-admin, it would be better to make sure admin rights are available before executing ChMac

Dependencies

ChMac wraps around DevCon (optional), Windows-native and GNU Linux utilities

All of the below dependencies are either optional or do not require attention, as most of them are included already.

Optionally enhances ChMac using DevCon.exe, Microsoft Windows Device Console which can be downloaded during first launch of the script
- Beware of the version to download as there are lots of DevCon versions for different OS. See https://superuser.com/a/1099688/112570
Unix utilities leveraged by ChMac are included already: tr.exe, sed.exe, which.exe, sleep.exe, wc.exe, wget.exe, grep.exe
Other OS built-in dependencies natively in Windows since Windows XP: getmac.exe, reg.exe, msvcp60.dll
Optional OS built-in dependency natively in Windows since Windows Vista: choice.exe, automatically falling back to set /p if unavailable or when there are more than 8 network adapters, thru a sub-script, _choiceMulti.bat - https://tech.wandersick.com/2010/01/ws-choice-batch-script-auto-fallback-to.html
- choice.exe may be put here to enhance experiences in OS earlier than Windows Vista or Server 2003
- Do not copy choice.exe from Windows Vista or later but the one in Windows Server 2003 or Windows Resource Kit (9x/NT)

Notes

getmac.exe and reg.exe should be put inside ChMac\Data\3rdparty\LP (low priority) so that they do not override the version in the OS when a better version is available natively thru the OS
Other executables should be put inside ChMac\Data\3rdparty (high priority) so that they override the version in the OS in case one is available natively thru the OS

Return Codes / Error Messages

Return code: 0

Cause
- Success

Return code: 1

Cause
- Failure

Return code: 3

Cause
- NIC error
Messages
- Error: No network adapter found
- Error: Adapter in a nonoperational state
- Error: Adapter unplugged. Please connect first

Return code: 4

Cause
- Bad syntax
Messages (and related parameters)
- Syntax Error: Working directory (/d)
- Syntax Error: Wrong address length (/m)
- Syntax Error: Not hexadecimal (/m)
- Syntax Error: Not digit (/n)
- Syntax Error: Not digit or smhd (/a)
- Syntax Error: Working directory (/d)
- Syntax Error: Missing parameter /n (/m)
- Syntax Error: Please use "/" instead of '-'
- Syntax Error: Unknown parameter
- Syntax Error: Adapter ID unavailable (/n)

Return code: 5

Cause
- No exe is found
Messages
- Error: No reg.exe. Place one from Windows in "ChMac\Data\3rdparty\LP"
- Error: No getmac.exe. Place one from Windows in "ChMac\Data\3rdparty\LP"

Return code: 7

Cause
- No admin rights
Messages
- Error: No admin right. Please run as admin

Limitations

Some virtual adapters may be unsupported
Randomization logic randomizes numbers (0-9) instead of hexadecimal (0-9, A-F)

Download / Installation Options

⭐ Download ChMac or browse its source code on GitHub .

After downloading and extracting ChMac.zip, ChMac.bat portable application can be used immediately. Alternatively, Setup.exe (optional) provides installation into Current User or All Users location. For enterprise deployment, a silent installer is also available on GitHub.

For users of Chocolatey and winget repositories who prefers unattended installation into All Users location:

choco install chmac
winget install chmac

*Note for winget users: Please ignore the warning from SmartScreen, where you may safely click ‘more info’ to complete installation.

Word of Mouth

Below is one of the kind comments left by ChMac users. (Thanks!)
"Will keep this one handy!"

Release History

Ver	Date	Update Description
2.0	20200623	- Add setup.exe for an installation option alongside existing portable mode, adding to Run prompt and PATH environmental variable to enable the 'ChMac' command anywhere for ease of use - Fix (remove) EnableLUA debug message that is displayed during UAC elevation - Wrap exclamation marks around ChMacDir variable to handle parentheses - Fix a bug where ChMac (Command Prompt) does not totally exit at the end - Improve UAC elevation mechanism so that it supports Run prompt
1.3	20200614	- Elevate as administrator as needed when User Account Control (UAC) is enabled in the system
1.2	20190629	- Improve compatibility on Windows 10 by changing binaries of GNU Linux utilities - Handle cases when the system PATH variable contains parenthesis
1.1	20120916	Interim release
1.0	20100124	First released in 2010

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

Automate Windows Group Member Update via CSV File or LDAP Query with PowerShell

noreply@blogger.com (wandersick) — Sun, 12 May 2019 00:41:00 +0000

Designed for Active Directory (AD) admins, this PowerShell script ws-dynamic-group automates the addition or removal of group members in a Windows local or AD domain group based on content in a CSV file or an LDAP query, turning a static group into a dynamic one.

Preventive features such as setting a threshold to avoid deletion of a massive number of group members by mistake, logging and mail alerts are also available.

Go to Download

List of Features

Keeping track of a specified CSV file for adding or removing group members in a local/domain group by comparing the group members of the file with group members on a system. (For scheduling, add it to a task using the built-in Windows Task Scheduler)
- If group members exist in CSV file but not in system, add the usernames
- If group members exist in system but not in CSV file, delete the usernames
Specify usernames of group members dynamically or statically
- Dynamic mode: the CSV input file can be created dynamically according to an LDAP query (specified in the script) from a live Active Directory domain
- Static mode: the input CSV file can also be pre-created (at a location and file name defined in the script)
Settings can be supplied via command-line switches or by editing the script
- For example, the group name can be specified via customGroupName in the upper variable section within the script or by running the script with -groupName command-line switch as in ws-dynamic-group.ps1 -groupName managers
Apply the modifications (addition/deletion of group member) to the system either locally (workgroup) or on a domain controller (Active Directory)
- There is support for both local (workgroup) and domain (Active Directory) environments. (Refer to Script Settings section)
Skip group member deletion if there is a mistake on the source side (e.g. content in CSV file is wrong) when a configurable threshold is reached
- Email alerts can be additionally configured to report on the event of skipping group member deletion due to reaching a specified threshold, so that admins can take action accordingly
There are 3 types of logging supported
- Lists of group members "before" and "after" running the script are saved separately for each run under the directory 03_Done
  - The script backs up all users beforehand and afterwards to facilitate troubleshooting and comparison in times of need
- Console output logging (everything displayed during the session) is stored in a named file using the date and time of each execution under the script log directory: _log\Executions\console_log-<current_date_time>.log using Start-Transcript of PowerShell
- Action logging (group member additions and deletions), appending to the same CSV file under script log folder: _log\action_log-<group_name>.csv
  - Six columns are available: date time directory_mode group action account

Refer to Flow section for more details.

Quick Start

Store the script with its complete directory structure in a location specified in the script (Default: C:\ws-dynamic-group)
Create '01_Incoming\entrant.csv' according to the section CSV File Schema.
In Windows Task Scheduler, create a task that runs ws-dynamic-group.ps1

Command Example

ws-dynamic-group.ps1 -csvPath "C:\Folder\File.csv" -groupName "managers" -force $true

Refer to Script Settings section for details about the syntax and in-script settings.

Prerequisites

A machine of Windows or Windows Server which can run PowerShell from the required modules, where:
- PowerShell module, Microsoft.Powershell.LocalAccounts, is required for "local" mode
- PowerShell module, ActiveDirectory, is required for "domain" mode (and only domain controllers are supported)
The group must be pre-created in the target system
- If the specified group is not created, the script cannot continue
Administrator rights
- The script must be executed with a user account in the AD group 'Domain Admins' for the 'Domain' mode or 'Administrators' group for the 'Local' mode, in order to add or delete group members successfully
Notes for 'static' input mode
- incoming.csv must be a complete file containing all the users (usernames) of the specified groups which must be processed.
  - For example, if there are only a few group members in the CSV when the group in the live system actually has a lot of members, this will be regarded as an intentional deletion of all the group members from the group except those in the CSV file. (Mistakes may be prevented by configuring userDeletionThreshold threshold)
Refer to the notes under the section CSV File Schema

CSV File Schema

Sample of CSV file (01_Incoming\incoming.csv) for domain mode:

example_manager01
example_manager02
example_user01
example_user02

Sample of CSV file (01_Incoming\incoming.csv) for local mode (where computer name has to be specified):

HOSTNAME\example_manager01
HOSTNAME\example_manager02
HOSTNAME\example_user01
HOSTNAME\example_user02

Note:

The group name can be either specified in the upper variable section of the script with the variable customGroupName or via command-line switch -groupName as in ws-dynamic-group.ps1 -groupName managers
- Only one group name is supported per script. Multiple instances can be achieved if required

Log File Schema

date,time,directory_mode,group,action,account
20190805,015512AM,Domain,students,Deletion,example_user01
20190806,035503PM,Local,managers,Addition,example_manager01

Where the date and time of all records of each execution are equal to the time of execution of the script.

Directory Structure

The zip file can be extracted to the directory below, so that the following directory structure is created:

C:\ws-dynamic-group
│
├───_log
│   │   action_log-<group_name>.log // Editable action log file of addition and deletion actions in CSV (one file per group)
│   │
│   └───Executions
│           console_log-<date_time>.log // console output logging of everything displayed (one file per run)
│
├───01_Incoming
│       incoming.csv // set manually (CSV "static" input mode) or generated in real time (dynamic "LDAP" input mode)
│
├───02_Processing
│
├───03_Done
│   │
│   └───<date_time> // e.g. 20190312_030304PM
│           Completed // indication of completion of script
│           Completed_Backup_(Before) // indication of the end of the script section: Backup (before)
│           Completed_Main_Logic // indication of the completion of the script section: Primary logic
│           Completed_Main_Logic-User_Addition // indication of completion of script section: Main Function - User Addition
│           Completed_Main_Logic-User_Deletion // indication of completion of script section: Main Function - User Deletion
│           Completed_Main_Logic-SKIPPED_User_Deletion // indication of skipping of script section: Main Function - User Deletion
│           Completed_Backup_(After) // indication of completion of script section: Backup (After)
│           incoming.csv
│           Domain|LocalGroupMemberAfter_full-time students.csv // optional: if enabled ($backupAfter is $true)
│           Domain|LocalGroupMemberAfter_managers.csv // optional: as above
│           Domain|LocalGroupMemberBefore_full-time students.csv // optional: if enabled ($backupBefore is $true)
│           Domain|LocalGroupMemberBefore_managers.csv // optional: as above
│
└───Scripts
        ws-dynamic-group.ps1

Flow of the Script

This section describes the directory structure and the main actions performed by each execution of the script.

Create a unique folder for each execution (to improve organization and avoid conflict)
- Randomize a unique value made up of day time in milliseconds
Backup Before – back up existing group members to a file
- [Local|Domain]GroupMemberBefore_(groupName).csv
Acquiring input
- Acquiring usernames
  - For 'dynamic' LDAP input mode, leverage specified LDAP query in the script to dynamically generate 'incoming.csv' in '01_Incoming' folder
  - For 'static' CSV input mode, acquire 'incoming.csv' from '01_Incoming' folder (manually created by user) and move it to '02_Processing' folder for processing
- Acquiring group name
  - A group name per script is specified at the top variable section of the script with the customGroupName variable
Main Function – perform action (adding/removing users from groups) specified in CSV
- Add users from CSV to local or AD group, if the users are in CSV but not in system
- Remove users from existing groups, if the users are not in CSV but in system
Move completed folders to '03_Done' folder
Backup After – Back up final group members to a file
- [Local|Domain]GroupMemberAfter_(groupName).csv
Write a file named 'Completed' to '03_Done' when each section of the script ends

Cases Handled by the Script

Below are the cases handled by the script:

Case	CSV	System	Action
1	CSV has the user	System has the user	No change
2	CSV has the user	System has no such user	Add the user to system
3	CSV has no such user	System has the user	Remove the user from system
4	CSV has no such user	System has no such user	No change

Script Section for Case 2

Case	CSV	System	Action
2	CSV has the user	System has no such user	Add the user to system

# Take action on users who only exist in the CSV
if ($directoryMode -ieq "Local") {
    $beingAddedUsers = Compare-Object (Get-Content "$scriptDir\02_Processing\$currentDateTime\LocalGroupMemberBefore_$csvGroupname.csv") (Get-Content "$scriptDir\02_Processing\$currentDateTime\$csvFile") | Where-Object {$_.SideIndicator -eq "=>"} | Select-Object -ExpandProperty inputObject
} elseif ($directoryMode -ieq "Domain") {
    $beingAddedUsers = Compare-Object (Get-Content "$scriptDir\02_Processing\$currentDateTime\DomainGroupMemberBefore_$csvGroupname.csv") (Get-Content "$scriptDir\02_Processing\$currentDateTime\$csvFile") | Where-Object {$_.SideIndicator -eq "=>"} | Select-Object -ExpandProperty inputObject
}

ForEach ($beingAddedUser in $beingAddedUsers) {
    # Add user, who are only found in CSV but not in system, to the group '$customGroupName' defined at the top variable of this script
    if ($directoryMode -ieq "Local") {
        net localgroup `"$customGroupName`" `"$beingAddedUser`" /add
    } elseif ($directoryMode -ieq "Domain") {
        net group `"$customGroupName`" `"$beingAddedUser`" /add
    }
}

Script Section for Case 3

Case	CSV	System	Action
3	CSV has no such user	System has the user	Remove the user from system

# Take action on users who only exist in the system but not in CSV
if ($directoryMode -ieq "Local") {
    $beingDeletedUsers = Compare-Object (Get-Content "$scriptDir\02_Processing\$currentDateTime\LocalGroupMemberBefore_$csvGroupname.csv") (Get-Content "$scriptDir\02_Processing\$currentDateTime\$csvFile") | Where-Object {$_.SideIndicator -eq "<="} | Select-Object -ExpandProperty inputObject
} elseif ($directoryMode -ieq "Domain") {
    $beingDeletedUsers = Compare-Object (Get-Content "$scriptDir\02_Processing\$currentDateTime\DomainGroupMemberBefore_$csvGroupname.csv") (Get-Content "$scriptDir\02_Processing\$currentDateTime\$csvFile") | Where-Object {$_.SideIndicator -eq "<="} | Select-Object -ExpandProperty inputObject
}

ForEach ($beingDeletedUser in $beingDeletedUsers) {
    # Add user, who are only found in system but not in CSV, to the group '$customGroupName' defined at the top variable of this script
    if ($directoryMode -ieq "Local") {
        net localgroup `"$customGroupName`" `"$beingDeletedUser`" /del
    } elseif ($directoryMode -ieq "Domain") {
        net group `"$customGroupName`" `"$beingDeletedUser`" /del
    }
}

Tips

Parallel instances can be achieved by creating multiple copies of the script in different locations
- For example, the below sets of the script (scriptDir) can be created, each scheduled to run as required
  - C:\ws-dynamic-group\1, C:\ws-dynamic-group\2, C:\ws-dynamic-group\3...
The script can be scheduled using built-in Task Scheduler of Windows or any other preferred auto-startup method
in PowerShell prompt, use <tab> key to auto-complete the command-line parameters without typing them in full
If Backup (Before) and Backup (After) are needed but the sysadmin finds it undesirable to execute them along with the Main Function (e.g. due to an increase of processing time), a separate script may be used and scheduled solely for the Backup sections to run separately

Current Limitations

Use of PowerShell equivalent for net group and net localgroup
- Currently, for 'domain' mode, the script must be run on a domain controller (due to the use of 'net group' commands) where the use of Remote Server Administration Tools (RSAT) would be unsupported
In local mode (workgroup), Get-LocalGroup and/or Get-LocalGroupMember returns usernames in NETBIOS format HOSTNAME\Username instead of Username; therefore, the schema of the input CSV file must be adjusted accordingly
- Two different input CSV files, one for domain and one for local, are included in this script as an example
Instead of a try-catch block, @ (Get-Content file | Select-Object) is used to work around a problem in which Compare-Object generates errors when the number of objects to compare is zero

Script Settings

Command-Line Parameters

The essential parameters below can also be specified via the command line, replacing the equivalent ones in the script.

-csvPath <path to source file>
- -csvPath C:\Folder\File.csv in command line overrides $csvFile = "incoming.csv" in script
-groupName <group name>
- -groupName "managers" in command line equals $customGroupName = $true in script
-force $true
- -force $true in command line equals $forceUserDeletion = $true in script (Note: -force $falsedoes not override the script equivalent)

Param(
    # Source file path
    # - Syntax:   -csvPath <source file in absolute or relative path>
    # - Example:  ws-dynamic-group.ps1 -csvPath "C:\Folder\File.csv"
    # - Note:     Overrides $csvFile specified inside script
    #             (No need to name the file as "incoming.csv")
    [string]$csvPath,
    # Group name
    # - Syntax:   -groupName <group name of AD domain or local workgroup>
    # - Example:  ws-dynamic-group.ps1 -groupName "managers"
    # - Note:     Overrides $customGroupName specified inside script
    [string]$groupName,
    # Force user deletion (even if userDeletionThreshold is reached)
    # - Syntax:   -force $true
    # - Example:  ws-dynamic-group.ps1 -force $true
    # - Note:     Overrides $forceUserDeletion inside script
    #             ('-force $false' would not override the script equivalent)
    [boolean]$force
)

Internal Parameters

More settings can be modified at Editable Settings section in 'Scripts\ws-dynamic-group.ps1':

# [Editable Settings]

# ------- General Settings -------

# Script directory
# - Determine the working directory of the script and the relative path
# - Tip: If multiple instances are required, create copies of the script folder structure and set each scriptDir to be:
#   - Example 1: "c:\ws-dynamic-group\1" (where this script can be located at c:\ws-dynamic-group\Scripts\1\ws-dynamic-group.ps1)
#   - Example 2: "c:\ws-dynamic-group\2" (and so on)
$scriptDir = "c:\ws-dynamic-group"

# Format of date and time 
# - Used for uniquely naming and creating a new directory on each run by randomizing a value made up of day time
# - Example: Get-Date -format "yyyyMMdd_hhmmsstt" (e.g. "20190227_095047AM")
$currentDateTime = Get-Date -format "yyyyMMdd_hhmmsstt"

# ------- Section(s) of Script to Run -------
# 
$mainFunction = $true # Perform the main function of the script
    $userAddition = $true # Perform user addition for each user that is in CSV but not in system
    $userDeletion = $true # Perform user deletion for each user that is in system but not in CSV
$backupAfter = $true # Record final group members to a log file (Output File: GroupMemberAfter_GroupName.csv)

# ------- Settings - Input Source -------

# Input mode
# - Either case-insensitive 'static' (CSV) or 'dynamic' (LDAP)
#   - Static: Acquire CSV file pre-created and pre-inputted by user in 01_Incoming directory
#   - Dynamic: Also ends up being a CSV file, but generated live via a LDAP filter from current Active Directory domain
# - Note: dynamic LDAP input mode automatically assumes 'Domain' and overrides 'Local' directory Mode
# - Example 1: $inputMode = "Dynamic"
# - Example 2: $inputMode = "Static"
$inputMode = "Static"

    # Directory mode (for both input modes)
    # - Determine whether to interact with local (workgroup) or domain (AD) authentication provider
    # - Either case-insensitive 'Local' or 'Domain'
    #   - 'Local' (workgroup) directory mode where local group would be enumerated,
    #   - 'Domain' directory mode which is only supported to be run on a domain controller
    # - Note: If input mode is set to dynamic (LDAP), this has NO EFFECT and is automatically assumed to be "Domain"
    # - Example 1: $directoryMode = "Local"
    # - Example 2: $directoryMode = "Domain"
    $directoryMode = "Domain"

    # ------- Settings - Dynamic Input Mode -------

    # LDAP filter
    # - Acquire a list of users from AD domain to generate a CSV file for further processing (used by dynamic LDAP input mode)
    # - Example 1 (AND):  (&(otherFacsimileTelephoneNumber=1)(otherPager=1))
    # - Example 2 (OR):   (|(otherFacsimileTelephoneNumber=1)(otherPager=1))
    # - Example 3 (STAR): (samAccountName=s9999*)
    # - Example 4 (MORE): ((mailNickname=id*)(whenChanged>=20180101000000.0Z))(|(userAccountControl=514))(|(memberof=CN=VIP,OU=Org,DC=test,DC=com)))
    $ldapFilter = "(|(otherFacsimileTelephoneNumber=1)(otherPager=1))"

    # ------- Settings - Static Input Mode -------

    # NOTE: The below has NO EFFECT when source file path is specified via command line: e.g. ws-dynamic-group.ps1 -csvPath "C:\Folder\file.csv"

    # CSV filename
    # - For processing inside 01_Incoming folder (used by static CSV input mode)
    # - Example: $csvFile = "incoming.csv"
    $csvFile = "incoming.csv"

# ------- Settings - Group Name -------

# NOTE: The below has NO EFFECT when group name parameter is specified via command line: e.g. ws-dynamic-group.ps1 -groupName "managers"

# Custom group name
# - One group name can be specified here in each script
# Example: $customGroupName = "managers"
$customGroupName = "managers"

# ------- Settings - User Deletion Threshold -------

# Skip user deletion when the below threshold is reached
# Set it to a desired value e.g. 0.8 (80%)
# Example: $userDeletionThreshold = 0.8
$userDeletionThreshold = 0.8

# NOTE: The below has NO EFFECT when force parameter is specified via command line: e.g. ws-dynamic-group.ps1 -force $true

# Force user deletion even when threshold is reached
# Example: $forceUserDeletion = $true
$forceUserDeletion = $false

# ------- Settings - Email -------

# Send mail alerts to report skipping of user deletion due to reaching threshold
$mailSender = "Sender <sender@wandersick.com>"
$mailRecipient = "Recipient A <recipienta@wandersick.com>", "Recipient B <recipientb@wandersick.com>"
$mailSubjectFailure = "WS-Dynamic-Group Member Deletion Skipped - " + $currentDateTime
$mailBodyFailure = "This is an automated message after WS-Dynamic-Group script skips user deletion due to the number of users being deleted reaches defined threshold:`n`n - userDeletionThreshold of $userDeletionThreshold`n`nFor details, please check the folder where the WS-Dynamic-Group script is executed, which is named using the same timestamp in this email subject at:`n`n - $scriptDir\03_Done\$currentDateTime`n"
$mailServer = "10.123.123.123"

Download

⭐ The scripts and source code can be downloaded or browsed on GitHub .

Release History

Version 3.0 - 20200613
- Non-functional improvements
Version 2.3 - 20190808
- Fix the display bug of the currentAction variable in the CSV log file
- Fix the deletion action not being logged
Version 2.2 - 20190806
- Add console output log file under script log directory: _log\Executions\console_log-<current_date_time>.log achieved using Start-Transcript PowerShell cmdlet
- Add a (modifiable) action log file in CSV format under script log directory: _log\action_log-<group_name>.csv with actions "Addition" and "Deletion" recorded
  - 6 columns are in the action log file (CSV): date time directory_mode group action account
- Set $PSScriptRoot as the script root (where the script can be placed and executed anywhere)
Version 2.1 - 20190604
- Perform pre-checking prior to deleting user to avoid error of deleting a large number of users
  - A userDeletionThreshold will be checked before user deletion
    - When threshold userDeletionThreshold is reached (e.g. 0.8 meaning if over 80% of users will be deleted, skip the operation)
  - Once the user deletion operation has been skipped, an email indicating which execution has failed will be sent to alert the system administrator
  - Force (forceUserDeletion) parameter is supported when system administrator decides to continue even if userDeletionThreshold is reached
- Accept several essential named parameters via the command line, replacing the equivalent ones in the script
  - -csvPath <path to source file>
  - -groupName <group name>
  - -force $true
Version 2.0 - 20190528
- Minimize the iterations of for-loops (ForEach-Object) for performance enhancement by:
  - Replacing the comparisons in Main Function with Compare-Object
  - Eliminate the use of 'ForEach-Object' for replacing characters with 'Select-Object -ExpandProperty'
- It is no longer necessary to specify Username as the header in a CSV file
- The backupBefore parameter is no longer available for customization because backing up group members beforehand is now a hard requirement being a comparison source for Compare-Object cmdlet
  - The output file of backupBefore has been reformatted to a single column in order to fit the comparison. (The backupAfter output file has also been reformatted to align with the change, although this is not technically required, it still makes comparison easier in case of troubleshooting.)
Version 1.3 - 20190414
- Maintenance release – no new feature. (Therefore, keep using the last version would be OK)
- Update README document and refactor code for the purpose of an easier reading
  - Remove deprecated feature – only one group name can be specified, at the top variable section of the script (specifying multiple group names in CSV was deprecated in the last version)
Version 1.2 - 20190409
- Performance enhancement
  - Rough calculation of an execution of 10,000 objects is 3 times faster than last time
- Reduces the number of for-loops and/or nested for-loops
- Script has been separated into three sections which can be optionally enabled: Backup (Before), Main Function and Backup (After)
  - For performance reasons, only Main Function is enabled by default
- Within Main Function, more granularity is achieved by further separating it into User Addition and User Deletion. Each of them can be enabled and run sequentially or in parallel as required
  - For example, two sets of the script with similar settings, with the only difference being set A having userAddition enabled and set B having userDeletion disabled, may be run in parallel to speed up user addition and deletion
- Group name has to be specified at the top of script
  - Specifying it in CSV is no longer supported
  - Only one group name can be specified
Version 1.1 - 20190315
- In addition to the already supported group member updating method of statically creating CSV input file, dynamic LDAP input mode is now supported, where incoming.csv is generated according to an LDAP query specified in the script, live from current Active Directory domain
Version 1.0 - 20190312
- Monitor a CSV file for additions or deletions of users in one or more groups (as a scheduled task)
- Apply the modifications to the system accordingly, locally or on a domain controller (Active Directory)
- Support for local (workgroup) and domain (Active Directory) environments

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

Storage Failover and Failback PowerShell Scripts for Failover Cluster (e.g. Hyper-V) with an Interactive Console Menu

noreply@blogger.com (wandersick) — Wed, 30 May 2018 13:25:00 +0000

Note: This is a template to ease development. The storage-vendor-specific part of the scripts have to be coded by yourself. An alternative way is to engage professionals to develop the script on one of the freelancing platforms.

There sometimes comes a need to simplify complex operations, in this case failover and failback operations of SAN storage replication between sites (e.g. production and DR), for reasons such as letting operators or the less technically-confident colleagues to more easily perform the operations in case of disasters or drill tests. To achieve that, this template has been created.

Written primarily in PowerShell, this package contains a set of SAN storage failover and failback scripts for Microsoft Failover Cluster (including Hyper-V cluster) and vendor-neutral pseudo code for SAN storage (for further modification to support different SAN vendors). Not only does it perform storage failover and failback, services running on top of it such as databases and virtual machines can also be catered.

Moreover, it features a user-friendly interactive console menu, where complex operations are handled by the scripts in the backend.

Go to Download

Served as a general template for implementation, one has to fill in the blanks – SAN-vendor-specific commands – to fit for his/her own use. Although the Failover Cluster parts and the interactive console menu are already there, further coding and testing cannot be avoided.

The scripts were designed to be as reusable (e.g. parameterized) as possible, in order to allow me to more easily adapt it to fit different projects. It has also been released as an open-source project on the GitHub repository ws-storage-failover which encourages IT pros to fork it to for their own SAN storage systems, and if possible, contribute their modified bits in returns, assisting others who might want to implement it for their own choice of SAN storage systems.

Features

Script failover and failback between replicated SAN storage with Microsoft Failover Cluster (Hyper-V and others)
Operator-friendly interactive console menus through which failover, failback and validation, status reporting can be performed

Steps can be performed individually or all at once

Scripted in PowerShell (Windows operations) and pseudo code (SAN-specific operations) as a template to be modified or customized

Define variables or fill in parameters in SAN-Parameters.xml and SAN-Parameters.ps1
Pseudo code of SAN operations is provided inside <# #> inline comment blocks to be replaced as required per command-line reference of SAN storage
Speed up development of automated/manual SAN failover/failback with this set of scripts
Leverage the command-line interface usually provided by SAN storage systems over SSH which makes it possible to use plink.exe (from PuTTY) to command SAN storage systems in scripts

Default scenario is for storage systems located in two sites with SAN-level replication
Console outputs are logged under Logs folder, with error messages separately stored
Open-source project on GitHub, encouraging forking

Requirements

SAN storage systems with SAN replication enabled (2 sites – production and DR sites assumed) with hosts running Failover Cluster (e.g. Hyper-V) in each site
A Windows client that runs the console menu with PowerShell 3.0 or above (comes with Windows 8.1 and Windows Server 2012 R2)
PuTTY should be installed under a location specified in SAN-Parameters.ps1
A one-time authentication may be required by connecting to SAN storage systems in both sites via SSH (with PuTTY or plink.exe) in order to cache the host key in registry
Run PowerShell as Administrator prior to running the Console Menu script

Script File Structure

¦   SAN-Console_Menu.ps1 // Menu for selecting among operations (failover, failback, validation, etc.)
¦
+---Logs // Console output and errors are recorded separately here
¦       ...
¦
+---Parameters
¦       SAN-Parameters.ps1 // Script options can be specified here (PowerShell)
¦       SAN-Parameters.xml // SAN replication options specified here (XML)
¦
+---Scripts
        SAN-Failback.ps1 // Failback operation subscript called by Console Menu script
        SAN-Failover.ps1 // Failover operation subscript called by Console Menu script
        SAN-GetStatus.ps1 // Status querying subscript called by Console Menu script
        SAN-Plink.ps1 // Functions which command SAN using plink.exe from PuTTY
        SAN-Variables.ps1 // Variables from XML, PS1 parameter files and SAN are further processed here
        SAN-Validate.ps1 // Validation subscript called by Console Menu script to confirm settings are valid

Getting Started

Edit SAN-Parameters.ps1 and .xml files for options such as changing the user account for SSH communication with your SAN.
All pseudo code (SAN-specific operations) inside <# #> should be replaced with the implementation of your SAN vendor. For example, change Echo <# display LUN #> to the actual command lsvdisk (IBM/Lenovo Storwize), lun show (NetApp), volcoll (HPE Nimble), etc.
Perform further development or customization according to your needs. For example:
- Follow the comments in the script
- Names of functions, variables and echo messages are self-explanatory

Core Functions

A. Failover from Site 1 to Site 2 (e.g. Production to DR)

1. Hyper-V Cluster

End Replication - from Production Site to DR Site
Create Clone from Replicated LUN in DR Site
Present Cloned LUN to DR Site Host, Take Disk Online, Run Replication from Production Site to DR Site
Import VM in DR Site
Disconnect VM Network Adapter in DR Site

2. General Failover Cluster

End Replication - from Production Site to DR Site
Create Clone from Replicated LUN in DR Site
Present Cloned LUN to DR Site Host, Take Disk Online, Run Replication from Production Site to DR Site

B. Failback from Site 2 to Site 1 (e.g. DR to Production)

1. Hyper-V Cluster

Stop and Delete Replication Group - from Production Site to DR Site
Stop and Delete Replication Group - from DR Site to Production Site
Stop VM in Production Site
Stop CSV in Production Site
Delete VM in Production Site
Unpresent LUN of CSV from Production Site Hosts
Delete LUN of CSV in Production Site
Create LUN of CSV in Production Site
Create Replication Group - from DR Site to Production Site
Add LUN to Replication Group - from DR Site to Production Site
Run Replication Group - from DR Site to Production Site
Stop VM in DR Site
Take Disk Offline in DR Site Host
Unpresent Cloned LUN from DR Site Host
Stop and Delete Replication Group - from DR Site to Production Site
Present LUN of Replicated CSV to Production Site Hosts
Run CSV in Production Site
Import VM in Production Site
Disconnect VM Network Adapters in Production Site
Run VM in Production Site
Create Replication Group - from Production Site to DR Site
Add LUN to Replication Group - from Production Site to DR Site
Run Replication Group - from Production Site to DR Site

2. General Failover Cluster

Stop and Delete Replication Group - from Production Site to DR Site
Stop and Delete Replication Group - from DR Site to Production Site
Stop CSV in Production Site
Unpresent LUN of CSV from Production Site Hosts
Delete LUN of CSV in Production Site
Create LUN of CSV in Production Site
Create Replication Group - from DR Site to Production Site
Add LUN to Replication Group - from DR Site to Production Site
Run Replication Group - from DR Site to Production Site
Take Disk Offline in DR Site Host
Unpresent Cloned LUN from DR Site Host
Stop and Delete Replication Group - from DR Site to Production Site
Present LUN of Replicated CSV to Production Site Hosts
Run CSV in Production Site
Create Replication Group - from Production Site to DR Site
Add LUN to Replication Group - from Production Site to DR Site
Run Replication Group - from Production Site to DR Site

Limitations

There is no one-size-fits-all solution – modification is inevitable
Not all error messages are separately recorded in error log file; some errors only exist in the main log. Outputs and errors encountered in the menu are not logged
SAN storage credentials are stored in the ps1 configuration file in clear text (protect the file properly)

Download

⭐ The scripts and source code can be downloaded at GitHub .

Release History

Ver	Date	Update Description
1.1	20190302	Improved delimiter choice in Scripts\SAN-Variables.ps1
1.0a	20141231	First released in December 2014

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

Exchange 2016 Migration Checklist (with PowerShell Examples)

noreply@blogger.com (wandersick) — Sun, 20 May 2018 14:06:00 +0000

Recently, I have been working on Exchange migration projects and the “70-345 – Designing and Deploying Microsoft Exchange Server 2016” exam. (Yes. I have passed the exam, thankfully, and I am now a certified MCSE of the Productivity track.) I have found video lessons on Pluralsight to be of great help, such as this one. Based on studies, a checklist including PowerShell commands has been crafted in the hopes of easily keeping track of milestones throughout similar projects. Except where noted, the example is for non-HA migration scenario from Exchange 2010 and 2013 to 2016. (This document is also available on GitHub as “exchange-2016-migration-checklist.md”).

Need a Pluralsight referral code? Here's my referral URL: http://referral.pluralsight.com/mQgdSmZ for a discount (up to 50%) in Pluralsight registration

(Last updated: 16 April 2020)

Inventorying Existing Environment

List existing Exchange servers in the environment

Get-ExchangeServer | ft Name, Edition, AdminDisplayVersion, ProductId, IsExchangeTrialEdition | ft -autosize -wrap
Get-Command ExSetup | ForEach {$_.FileVersionInfo}

Have an estimation of how many mailboxes on each existing Exchange server
- Get-Mailbox | Group-Object -Property:Database | Select Name,Count | ft -auto
Collect AD forest functional level info
- Get-ADForest
Collect domain controller version in AD
- Get-ADDomainController | Select Name, OperatingSystem
Client Access Namespace (used by client to connect to Exchange)
- Inventory PowerShell – Collect internal and external domain names for each of the below
  - Autodiscover (SCP)
    - Get-ClientAccessServer | Select Identity,AutoDiscoverServiceInternalUri
  - Outlook Anywhere (RPC over HTTPS)
    - Get-OutlookAnywhere -ADPropertiesOnly | Select Server,Internalhostname,Externalhostname
    - Get-OutlookAnywhere -ADPropertiesOnly | Select Identity, *Auth* | fl
  - OWA
    - Get-OWAVirtualDirectory -ADPropertiesOnly | Select Server,InternalURL,ExternalURL
    - Get-OWAVirtualDirectory -ADPropertiesOnly | Select Identity,*Auth* | fl
  - ECP
    - Get-ECPVirtualDirectory -ADPropertiesOnly | Select Server,InternalURL,ExternalURL
    - Get-ECPVirtualDirectory -ADPropertiesOnly | Select Identity,*Auth* | fl
  - Offline Address Book (OAB)
    - Get-OABVirtualDirectory -ADPropertiesOnly | Select Server,InternalURL,ExternalURL
    - Get-OABVirtualDirectory -ADPropertiesOnly | Select Identity,*Auth* | fl
  - Exchange Web Services (EWS)
    - Get-WebServicesVirtualDirectory -ADPropertiesOnly | Select Server,InternalURL,ExternalURL
    - Get-WebServicesVirtualDirectory -ADPropertiesOnly | Select Identity,*Auth* | fl
  - MAPI/HTTP
    - Get-MAPIVirtualDirectory -ADPropertiesOnly | Select Server,InternalURL,ExternalURL
    - Get-MAPIVirtualDirectory -ADPropertiesOnly | Select Identity,*Auth* | fl
  - ActiveSync
    - Get-ActiveSyncVirtualDirectory -ADPropertiesOnly | Select Server,InternalURL,ExternalURL
    - Get-ActiveSyncVirtualDirectory -ADPropertiesOnly | Select Identity,*Auth* | fl

SSL Certificates
- For Exchange ActiveSync, Outlook Anywhere, Outlook Web App, etc.
- Requirements
  - Matches the server name to which clients connect
  - Still within validity period
  - Issued by a trusted certificate authority
- Reusing existing certificates for new Exchange server is acceptable
  - Provided that client connect to an alias address instead of addresses specifying server hostnames (e.g. mail.company.com instead of EXCH13.company.com)
- Inventory PowerShell
  - Take note of output with W which are certificates for IIS (not I which are for IMAP)
    - Get-ExchangeCertificate
  - Take note of CertificateDomains which are covered by the domain, and the validity status
    - Get-ExchangeCertificate -Thumprint <From_Above_Command> | fl
Mailbox Storage Quotas
- Beware of default mailbox quota – For Exchange 2016, it is 2GB by default
  - Mailbox migration fails if size exceeds target database
  - New databases should be configured with same or larger quotas
- Inventory PowerShell (second command supports querying Exchange 2010, if exists, from newer Exchange Management Shell)
  - Get-MailboxDatabase | Select Name,*Quota*
  - Get-MailboxDatabase -IncludePreExchange2013| Select Name,*Quota*
Email Routing Topology and Transport
- Internal mail flow between supported Exchange servers (Exchange 2010 <> 2013 <> 2016) is automatic (no further configuration required)
- Inbound mail flow from the Internet
  - Acquire internal DNS MX record:
    - Resolve-DnsName -Type MX -Name company.com -Server <internal_DNS>
    - Resolve-DnsName -Type A -Name mail.company.com -Server <internal_DNS>
  - Alternatively
    - nslookup -> server internal_DNS_Server -> set type=MX -> company.com
    - nslookup -> server internal_DNS_Server -> set type=A -> mail.company.com
  - Acquire external DNS MX and A record (or using www.mxtoolbox.com):
    - Resolve-DnsName -Type MX -Name company.com -Server 8.8.8.8
    - Resolve-DnsName -Type A -Name mail.company.com -Server 8.8.8.8
  - Alternatively
    - nslookup -> server 8.8.8.8 -> set type=MX -> company.com
    - nslookup -> server 8.8.8.8 -> set type=A -> mail.company.com
- Outbound mail flow to the Internet
  - Acquire info of Send Connectors (for Outbound email)
    - Get-SendConnector
    - Get-SendConnector | fl
      - Watch out for SmartHosts and SmartHostsString to check whether there is any use of a smart host (it should be empty if not)
      - Watch out for SourceTransportServer to check whether the source transport servers that are currently in use (i.e. existing Exchange servers)
Involves any non-Exchange servers? Internal application may send email via relay connectors
- Check devices, services and applications which use SMTP services
  - Backup applications which send notifications (e.g. Veeam, Backup Exec)
  - Anti-malware or anti-spam (e.g. Symantec Endpoint Protection Manager)
  - Virtualization (e.g. vCenter, System Center Virtual Machine Manager)
  - Voice or telephony
  - SMS or fax gateways
- Acquire info of Receive Connectors (for Relay connectors)
  - Get-ReceiveConnector -Server <Server_Name>
    - Look for anything other than the below, which probably states "Relay" or something with an address binding or {0.0.0.0:25}
    - For Exchange 2010, ignore "Default <Server_Name>" and "Client <Server_Name>"
    - For Exchange 2013, ignore "Default <Server_Name>", "Client Proxy <Server_Name>", "Default Frontend <Server_Name>", "Outbound Proxy Frontend <Server_Name>", "Client Frontend <Server_Name>"
- Once found out, look for the remote IP Ranges of it (i.e. IP addresses which are allowed to relay email via that Exchange server
  - Get-ReceiveConnector "Server_Name\Receive_Connector_Name" | Select RemoteIPRanges
- Once an IP address is acquired, perform PTR DNS lookup using nslookup to find out its hostname/FQDN
  - nslookup <IP_address>

Maximum allowed message size of Receive Connector

Get-ReceiveConnector | Select Identity,MaxMessageSize

Public Folders
- Inventorying existing public folders and existing Exchange Server
  - Get-PublicFolder -Recurse | Export-Clixml C:\PFMigration\Legacy_PFStructure.xml
- Acquire existing pubic folder statistics
  - Get-PublicFolderStatistics | Export-Clixml C:\PFMigration\Legacy_PFStatistics.xml
- Acquire public folder permissions
  - Get-PublicFolder -Recurse | Get-PublicFolderClientPermission | Select Identity, User -ExpandProperty AccessRights | Export-Clixml C:\PFMigration\Legacy_PFperms.xml
- Check if existing public folder names have backslash character which is invalid
  - Get-PublicFolderStatistics -ResultSize Unlimited | Where {$_.Name -Like "*\*"} } | fl name,identity
- Watch out for existing pubic folder migration job
  - Get-OrganizationConfig | fl PublicFoldersLockedforMigration,PublicFolderMigrationComplete
Other

Collect information on arbitration mailboxes

Get-Mailbox -Arbitration | Select name,database

Check whether Outlook Anywhere is enabled (on all servers)

Get-ClientAccessServer | Select Name,OutlookAnywhereEnabled

Check organization config e.g. whether MAPI over HTTP is enabled per Microsoft Docs

Get-OrganizationConfig
Get-OrganizationConfig | fl *mapi*

Check CAS mailbox e.g. whether OWA, ActiveSync, POP3, IMAP, MAPI over HTTP, etc. is enabled per mailbox

Get-CasMailbox
Get-CasMailbox | ft name, *mapi*

Basic Health Checking

Check service health

Test-ServiceHealth

Get a count of current email messages in mail queue on Hub Transport servers

Get-Queue

Check MAPI Connectivity

Test-MAPIConnectivity

Validates that the RPC/HTTP endpoint is able to receive traffic on the Mailbox server

Test-OutlookConnectivity -ProbeIdentity "OutlookRpcSelfTestProbe"

Validates that the MAPI/HTTP endpoint is able to receive traffic on the Mailbox server

Test-OutlookConnectivity -ProbeIdentity "OutlookMapiHttpSelfTestProbe"

Check replication status of all mailbox databases (for non-standalone, DAG scenario)

Get-MailboxDatabaseCopyStatus *\*

Check whether replication status is healthy

Test-ReplicationHealth

Check server component state

Get-ExchangeServer | Get-ServerComponentState

Run additional third-party health checking scripts as required, for example

Test-ExchangeServerHealth.ps1 -ReportMode -Log

Exchange 2016 Changes

Deprecation of Outlook Anywhere (RPC-over-HTTP) with MAPI-over-HTTP (enabled by default – introduced since Exchange 2013 SP1)
Only Mailbox server role and Edge Transport server role (Exchange 2013 additionally includes Client Access Server, while Exchange 2010 includes Hub Transport and Client Access Server)
Co-existence: Exchange 2010, 2013 and 2016 can proxy for one another (For Exchange 2010, Outlook Anywhere has to be used)
Exchange requirements
- Exchange 2010 SP3 with RU11 or later
- Exchange 2013 CU10 or later
- AD requirements
  - All domain controllers in the forest must be Windows Server 2008 or later
  - Forest functional level of Windows 2008 or higher (2008 R2 or higher if any Windows Server 2016 Domain Controller is in the environment)
- Outlook client requirements (Latest available service packs and updates recommended)
  - Windows: Outlook 2016, 2013, and 2010
  - Mac OS X: Outlook for Mac for Office 365 / Outlook for Mac 2011
Introduction of Public Folder mailboxes – no more legacy Public Folders

Installing Exchange

Note: For this section, it is recommended to also check Microsoft Docs for the latest prerequisites.

Privileges of account used during setup
- Domain Admin, Enterprise Admin and Schema Admin
Confirm PowerShell v4.0 is available
Confirm .NET Framework 4.8 is available
Confirm Windows components and Unified Communications Managed API 4.0 Core Runtime 64-bit are installed
- Acquire the PowerShell command (Install-WindowsFeature…) to install Windows components from https://technet.microsoft.com/en-us/library/bb691354%28v=exchg.160%29.aspx
Schema extensions
Prepare Active Directory
Prepare domains
Install product key

Avoiding Possible Impacts

End user
- Offline Address Book
  - Confirm existing mailbox databases have an Offline Address Book configured instead of leaving it blank
    - Each mailbox database in existing Exchange servers should have an office address book (e.g. the default one) assigned; otherwise, users may face a problem of unknowingly downloading the new OAB in Exchange 2016 which could take a lot of bandwidth
Client connectivity
- Autodiscover Service Connection Point (SCP)
  - Newly installed Exchange server has a default SCP URI of the server's FQDN (e.g. EXCH16.company.com), which generates certificate errors
  - Fix:
    - Get-ClientAccessServer -Identity EXCH16 | Select Name,AutoDiscoverServiceInternalUri
    - Set-ClientAccessServer -Identity EXCH16 -AutoDiscoverServiceInternalUri https://mail.company.com/Autodiscover/Autodiscover.xml
  - Ignore the AutoDiscover virtual directory setting which is not used for AutoDiscover – only SCP is used for AutoDiscover in an internal environment
Mail flow
- Internal to Internal Exchange servers
- Inbound from the Internet
- Outbound to the Internet
- SMTP relay connectors

Exchange Server 2016 Co-existence Tasks - Migrating Client Access

Note
- Manage object with the matching version of Exchange management tools
- Client connectivity must go to the highest version of Exchange (except for 2013/2016 coexistence)
- Email can route in or out of any version of Exchange
- Internal Exchange to Exchange mail flow is automatic for Exchange 2010, 2013 and 2016 (Outlook Anywhere is leveraged)

Import the SSL certificate
- Import from existing server to new server
- Enable them for SMTP and IIS
- Enable them for IMAP and POP3
Configure the client access namespaces
- Configure each HTTPS service with the same namespace as existing servers (virtual directories)
  - Defining variables
    - $InternalHostname = "mail.company.com"
    - $ExternalHostname = "mail.company.com"
    - $Server = "EXCH16"
  - Outlook Anywhere
    - Get-OutlookAnywhere -Server $Server | Set-OutlookAnywhere -ExternalHostname $ExternalHostname -InternalHostname $InternalHostname -ExternalClientsRequiresSsl $True -InternalClientsRequireSSL $true -DefaultAuthenticationMethod NTLM
  - OWA Virtual Directory
    - Get-OWAVirtualDirectory -Server $Server | Set-OWAVirtualDirectory -ExternalUrl https://$ExternalHostname/owa -InternalUrl https://$InternalHostname/owa
  - ECP Virtual Directory
    - Get-EcpVirtualDirectory -Server $Server | Set-EcpVirtualDirectory -ExternalUrl https://$ExternalHostname/ecp -InternalUrl https://$InternalHostname/ecp
  - ActiveSync Virtual Directory
    - Get-ActiveSyncVirtualDirectory -Server $Server | Set-ActiveSyncVirtualDirectory -ExternalUrl https://$ExternalHostname/Microsoft-Server-ActiveSync -InternalUrl https://$InternalHostname/Microsoft-Server-ActiveSync
  - EWS (Exchange Web Services) Virtual Directory
    - Get-WebServcesVirtualDirectory -Server $Server | Set-WebServicesVirtualDirectory -ExternalUrl https://$ExternalHostname/EWS/Exchange.asmx -InternalUrl https://$InternalHostname/EWS/Exchange.asmx
  - OAB (Offline Address Book) Virtual Directory
    - Get-OabVirtualDirectory -Server $Server | Set-OabVirtualDirectory -ExternalUrl https://$ExternalHostname/OAB -InternalUrl https://$InternalHostname/OAB
  - MAPI Virtual Directory
    - Get-MapiVirtualDirectory -Server $Server | Set-MapiVirtualDirectory -ExternalUrl https://$ExternalHostname/mapi -InternalUrl https://$InternalHostname/mapi
  - Note: These changes do not make client connect to new Exchange server immediately; client will still connect to where DNS is resolving the namespace
- Ensure new Exchange server uses existing authentication (e.g. form-based authentication with the same logon format)
- Special concerns for Exchange 2010
  - Outlook Anywhere must be enabled
  - Check whether Outlook Anywhere is enabled
    - Get-ExchangeServer | Where {($_.AdminDisplayVersion -Like "Version 14*") -And ($_.ServerRole -Like "*ClientAccess*")} | Get-ClientAccessServer | Select Name,OutlookAnywhereEnabled
- IIS authentication must be configured for co-existence
  - Enable Outlook Anywhere and configure IIS authentication
    - Get-ExchangeServer | Where {($_.AdminDisplayVersion -Like "Version 14*") -And ($_.ServerRole -Like "*ClientAccess*")} | Get-ClientAccessServer | Where {$_.OutlookAnywhereEnabled -Eq $False} | Enable-OutlookAnywhere -ClientAuthenticationMethod Basic -SSLOffloading $False - ExternalHostName $hostname -IISAuthenticationMethods NTLM, Basic
Test the namespaces
- Before DNS change (risky), use a hosts file for testing with a pilot group
  - Content of hosts file:
    - IP_Address_of_Exchange_2016 mail.company.com
Cutover namespaces to Exchange 2016
- Lower DNS TTL to 1 minute (done earlier than what the TTL specifies)
- Make DNS change for internal client (e.g. AD DNS server)
- Make firewall change for external client (NAT settings)
New Exchange server is in production for client connectivity now
Test using exrca > Exchange Server > Microsoft Outlook Connectivity Tests > Outlook Connectivity
- Usual to see a few failed Autodiscover items, as long as the overall Autodiscover is successful

Migrating Mail Flow

Internal mail flow between Exchange servers
- Change required: no need further setup (automatically established between installed Exchange servers)
- Common issue: maximum allowed message size setting differs among Exchange servers
- Compare using PowerShell the MaxMessageSize values among different servers
  - Get-ReceiveConnector | Select Identity,MaxMessageSize
- Fix using Set-ReceiveConnector
  - Get-ReceiveConnector -Server EXCH16 | Set-ReceiveConnector -MaxMessageSize 45MB
- Test mail flow by sending an email from one internal mailbox to another
- Outlook client: "Request a Delivery Receipt"
Inbound mail flow from the Internet
- Changes required
  - Firewall – by modifying NAT setting from the existing Exchange server to the new one, or
  - Email appliance or load balancer – by removing existing Exchange server and adding new one
- Test mail flow using exrca > Exchange Server > Internet Email Tests > Inbound SMTP Email
- Analyze mail routing by pasting email header into exrca > Message Analyzer
Outbound mail flow to the Internet
- Change required: EAC > Mail Flow > Edit Send Connector > Source Server: Remove existing Exchange server; ensure only new Exchange server is in the list
- Analyze mail routing by pasting email header into exrca > Message Analyzer
Devices and applications that use SMTP relay
- Change required
- EAC > Mail Flow > New Receive Connector
  - General tab
    - Name: Provide a name for a new relay connector, e.g. Relay EXCH16
    - Role: Frontend Transport
    - Type: Custom
    - Edit remote network settings by removing default 0.0.0.0-255.255.255.255 and adding IP addresses of devices and applications which send via this relay connector
  - Security tab
    - Tick "Anonymous users" under permission groups (leave all else unchecked except the first checkbox of TLS authentication)
- Allow Exchange relay connector to relay email sent to external recipient
  - Get-ReceiveConnector "EXCH16\Relay EXCH16" | Add-ADPermission -User 'NT Authority\Anonymous Logon' -ExtendedRights MS-Exch-SMTP-Accept-Any-Recipient
- Fix existing misconfiguration on devices and applications (if any)
- DNS alias should exist for SMTP, e.g. smtp.company.com
- Devices and applications should be using DNS alias instead of the hostname or IP address of the server
- Take the chance of migration to fix any nonoptimal settings

Migrating Public Folder

Note
- Public folder mailbox is introduced in Exchange 2016 – no more legacy public folders
- Exchange allows public folder mailboxes up to 100GB in size
- Public folders may be migrated ahead of other mailboxes if they can sometimes be very large in size
- For large environments, public folders may take few hours take time to finalizing cutover
- One-way migration process (can roll back but lose changes since migration)
- Follow Public Folder Batch Migration Guidance on TechNet
  - https://technet.microsoft.com/en-us/library/dn912663(v=exchg.150).aspx

Download Public Folders migration scripts
- https://www.microsoft.com/en-us/download/details.aspx?id=38407
- Use these scripts to migrate public folders from Exchange 2007 or Exchange 2010 to Office 365 and Exchange Online, Exchange 2013, or later.
Extract scripts to C:\PFMigration
- Create-PublicFolderMailboxesForMigration.ps1
- CreatePublicFolderMailboxesForMigration.strings.psd1
- Export-PublicFolderStatistics.ps1
- Export-PublicFolderStatistics.strings.psd1
- PublicFolderToMailboxMapGenerator.ps1
- PublicFolderToMailboxMapGenerator.strings.psd1
Inventory existing public folders
- Inventory existing public folders on existing Exchange Server
  - Get-PublicFolder -Recurse | Export-Clixml C:\PFMigration\Legacy_PFStructure.xml
- Acquire existing pubic folder statistics
  - Get-PublicFolderStatistics | Export-Clixml C:\PFMigration\Legacy_PFStatistics.xml
- Acquire public folder permissions
  - Get-PublicFolder -Recurse | Get-PublicFolderClientPermission | Select Identity, User -ExpandProperty AccessRights | Export-Clixml C:\PFMigration\Legacy_PFperms.xml
- Check if existing public folder names have backslash character which is invalid
  - Get-PublicFolderStatistics -ResultSize Unlimited | Where {$_.Name -Like "*\*"} } | fl name,identity
- Watch out for existing pubic folder migration job
  - Get-OrganizationConfig | fl PublicFoldersLockedforMigration,PublicFolderMigrationComplete
- Inventorying existing public folders and migration jobs on new Exchange Server
  - Get-PublicFolderMigrationRequest
  - Get-MigrationBatch | Where {$_.MigrationType.ToString() -eq "PublicFolder"}
  - Get-Mailbox -PublicFolder
Generate CSV files on existing Exchange server
- Export-PublicFolderStatistics.ps1
  - Run PowerShell commands to export public folder size statistics to CSV
    - cd C:\PFMigration
    - Export-PublicFolderStatistics.ps1 C:\PFMigration\PFSizeMap.csv Old_Exch_Server
  - Open CSV output with Excel and review the size. Exchange allows public folder mailboxes up to 100GB in size
- PublicFolderToMailboxMapGenerator.ps1
  - Run PowerShell commands to export public folder map to CSV according to size requirements
  - e.g. 10GB max for each mailbox; if larger than 10GB, additional public folder mailbox names will be generated
    - ps1 10000000 C:\PFMigration\PFSizemap.csv C:\PFMigration\PFMailboxMap.csv
  - Open CSV and review. The CSV will be used for migration by New-MigrationBatch.
Create public folder mailboxes on new Exchange server
- New-Mailbox -PublicFolder Mailbox1 -HoldForMigration:$true
Begin migration on new Exchange server
- Creating a new migration batch using the generated PFMailboxMap.CSV
  - New-MigrationBatch -Name PFMigration -SourcePublicFolderDatabase (Get-PublicFolderDatabase -Server Old_Exch_Server) -CSVData (Get-Content C:\PFMigration\PFMailboxMap.csv -Encoding Byte) -NotificationEmails recipient@company.com
- Start the migration batch
  - Start-MigrationBatch PFMigration
- Acquire migration status
  - Get-MigrationUser -BatchId PFMigration
- Optional: check migration status reporting email in the specified notification recipient mailbox
Lock public folders (outage required) on existing Exchange server
- Set-OrganizationConfig -PublicFoldersLockedForMigration:$true
  - Note
    - It can take up to several hours for some environments
    - Effect of the -PublicFoldersLockedForMigration:$true parameter is pubic folders cannot be accessed (Error: Cannot expand the folder. Outlook cannot access this Public Folder right now. Please try again later.)
Finalize the migration (outage required) on new Exchange server
- Enable public folders as remote
  - Set-OrganizationConfig -PublicFoldersEnabled Remote
- Complete migration batch
  - Complete-MigrationBatch PFMigration
- Get migration batch status (The command may be run a few times to keep tracking the statuses)
  - Get-MigrationBatch PFMigration
Test and unlock public folders on new Exchange server
- After the below command is entered, the specified user can check his/her mailbox to see if public folders are accessible
  - Set-Mailbox username -DefaultPublicFolderMailbox PF_Mailbox_Name
- Enable public folders as local
  - Set-OrganizationConfig -PublicFoldersEnabed Local

Migrating Mailbox

Create a new or configure existing Exchange 2016 Mailbox Database
- Reminder
  - Up to 5 DBs for Standard edition; 100 for Enterprise edition
- Best practice
  - Separate transaction logs and DV to different storage volume (especially for non-HA implementations)
    - Examine the paths (EdbFilePath, LogFolderPath)
    - Get-MailboxDatabase | fl *path*
    - Change path
    - Move-DatabasePath -Identity DB_Name -EdbFilePath "Target_Path_1" -LogFolderPath "Target_Path_2"
  - Other
    - Multiple small DBs is better than fewer large DBs (faster backup/restore/repair itmes)
      - Keep name short and identifiable "DB01 vs Mailbox Database 1"
      - Names must be unique
    - Rename DB
    - Set-MailboxDatabase "Source DB Name" -Name "Target Name"
- Common Issue: Mailbox quota differs among different databases on existing and new servers (default: 2GB)
  - Acquire the existing quota information
    - Get-MailboxDatabase -IncludePreExchange2013 | Select Name,IssueWarningQuota,ProhibitSendQuota,ProhibitSendReceiveQuota
  - Fix by matching the quota of new server with that of the existing one
    - Get-MailboxDatabase -Server EXCH16 | Set-MailboxDatabase -IssueWarningQuota 5GB -ProhibitSendQuota 6GB -ProhibitSendReceiveQuota 10GB
- Common Issue: Outlook Address Book not configured on existing Exchange servers

Prepare to migrate mailbox

Exclude mailbox databases from provisioning (mailbox provisioning load balancer)
- If no target DB is selected while creation of the migration batch, Exchange automatically distributes mailboxes across available mailbox databases
- To Exclude a DB from provisioning
  - Set-MailboxDatabase "DB name" -IsExcludedFromProvisioning $true
Schedule backup of the new server (make sure it is successful to backup and restore)
- Massive amount of transaction logs could be generated during migration on the destination server which could take significant amount of disk space
- May assume 1GB logs per 1GB mailbox data
- Keep monitoring disk space usage on target server
Migrate arbitration mailboxes first
- Check
  - Get-Mailbox -Arbitration | Select name,database
- Do
  - Get-Mailbox -Arbitration | New-MoveRequest
  - Get-MoveRequest | Get-MoveRequestStatistics
- Verify
  - Check report "Report: Download the report for this user". Look for start time and end time, error messages (if failed), etc.
Move shared mailboxes and delegates altogether
- Chief Executive Officer + his/her assistant
- Specific team + specific shared mailbox
- Equipment mailboxes + equipment manager
Migration speed depends on size of mailboxes, performance of source and destination servers
Online migration: migration can run while users are accessing their mailboxes
Cutover: 95% of mailbox is migrated, then either auto-suspend, require manual completion or auto-complete
- Completion stage involves disconnecting end user and displaying a message asking users to restart Outlook
- Configure backup to truncate transaction log (which could outgrow available target storage)

Decommission Servers

Sanity Check
- Client Access namespaces migrated?
  - Check DNS entries for Client Access namespaces
  - Check load balancer configuration
  - Review IIS logs (look for any user activity – see action list below)
- Mail flow migrated?
- Mailboxes migrated?
- Public folders migrated?
Action List
- Confirm server no longer responsible for send connectors (Outbound email to the Internet)
  - Ensure only new Exchange is in source server (EAC > Mail Flow > Edit Send Connector > Source Server)
- Analyse usage via transport logs to confirm no SMTP traffic hitting a relay receive connector
  - Enable verbose protocol logging for a few hours or days to look for activity
    - Get-ReceiveConnector -Server Old_Server | Select Identity,Enabled,ProtocolLogging
  - Should be enabled and verbose. If not, configure them so
    - Set-ReceiveConnector "Old_Connector" -ProtocolLoggingLevel Verbose
  - Check logs under C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\FrontEnd\ProtocolLog\SmtpReceive\RECV*.LOG
  - Afterwards, disable verbose logging
    - Get-ReceiveConnector -Server Old_Server | Where ($_.ProtocolLoggingLevel -eq "Verbose") | Set-ReceiveConnector -ProtocolLoggingLevel None
- Confirm no more mail flow appearing in message tracking logs
  - Get-MessageTrackingLogs -Start (Get-Date).AddDates(-3)
  - $messages | Get-MessageTrackingLogs -Start (Get-Date).AddDates(-5) -ResultSize unlimited
  - $messages | Group-Object -Property:Sender | Select Name,Count | ft -auto
- Confirm no more client access traffic in IIS logs
  - Path: C:\inetpub\logs\LogFiles
  - Filter for username by importing to Excel, Log Parser or Log Parser Studio
Remove mailbox databases and public folders
- Get-Mailbox -Database DB
- Remove-MailboxDatabase DB
- Remove-PublicFolderDatabase PF_Name
Verify OAB of new Exchange server is configured to existing OAB
- Get-OfflineAddressBook
Uninstall Exchange from Control Panel

Techniques

Start PowerShell Logging (append mode)

Start-Transcript transcript.log -Append

Stop PowerShell Logging

Stop-Transcript

Connect to Other Exchange Servers

Connect-ExchangeServer MBX01 -ClientApplication:ManagementShell

Invoke standard PowerShell commands on remote computers using PowerShell Remoting

Invoke-Command -ComputerName MBX01,MBX02 -ScriptBlock {Restart-Service -ServiceName MSExchangeIS}

Quick Windows Hardening with Infrastructure-as-Code – Chef and Inspec

noreply@blogger.com (wandersick) — Tue, 24 Apr 2018 14:08:00 +0000

CIS, Center for Internet Security, publishes prescriptive system hardening documents which provide guidance for establishing a secure system configuration on platforms such as Windows. Usually, their Windows hardening documents are over a hundred pages long and would take a long time to perform hardening manually by one person. Thankfully, there is an Infrastructure-as-Code configuration management approach, e.g. the one introduced below leveraging Chef and Inspec, to achieve automation of the hardening process and validating the results.

Figure 1. Content of harden_winrm.rb, with references from CIS sections as an example of Chef recipes. (This one is from MattTunny/windows_hardening GitHub repository)

Instead of demonstrating the power of infrastructure-as-code fully, this quick post only aims at introducing the concept by leveraging Chef hardening recipes found on the Internet, showing the steps to perform Windows hardening on a single local machine quickly, which may suit one-time use cases. We will install Chef Development Kit to use chef-apply for applying Chef recipes to harden a local machine and inspec for verifying the hardening.

Figure 2. Installing Chef Development Kit

Tools Required

Chef cookbook or recipes for Windows OS hardening (e.g. ones from my GitHub repository, wandersick/windows_hardening, originally forked from MattTunny/windows_hardening GitHub repository, adjusted to fit the examples using the chef-apply approach in this article. You may use others found on the Internet, or craft ones that are catered to your needs)

Download Here: https://github.com/wandersick/windows_hardening
Chef recipes crafted according to CIS-published guidance
The example is for Windows Server 2012 and 2012 R2, which is similar for Windows Server 2016 and 2019
You should customize. Here are some ideas:
1. You may not want to run some of the recipes which break functionalities such as harden_winrm.rb (WinRM)
2. In core_hardening.rb, you may want UAC to be disabled (EnableLUA not set to 1), the warning text displayed at logon to be different (legalnoticecaption and legalnoticetext edited as desired). For domain controllers, you may want EnableAuthEpResolution to be removed in order to enable certain cross-forest/domain scenarios. For older printers/scanners/copiers not supporting NTLMv2 for scanning to shared folders, or you are encountering an issue in which Remote Desktop client keeps saying ‘the logon attempt failed’, you may want LmCompatibilityLevel to be removed. Similarly for RestrictNTLMInDomain if it is a domain controller.
3. In localComputer.inf, you may prefer NewAdministratorName and NewGuestName are not set to _administrator and _guest, the maximum password age (MaximumPasswordAge) not set to 42 days, and the minimum password length (MinimumPasswordLength) not set to 14
(All of the above are the default settings from the recipes)
Some recipes require specific variables to be defined such as cwd for the path to some configuration files. Be sure to refer to the README.md and comments within each recipe file.

RegShot 2.0.1.70 (optional)

Download Here: https://www.wincert.net/forum/index.php?/files/file/16-regshot2-unicode/ (This site requires account registration and login prior to download)
This utility compares registry changes before and after hardening, provides a report of which, allows backup of registry in .reg and .hive files, as well as producing UNDO and REDO reg files.

Chef Development Kit installed on a supported Windows machine

Download Here: https://downloads.chef.io/chefdk#windows
Although the Chef Development Kit provides a lot of features, we will only use chef-apply for applying recipes (.rb files) to a local machine and inspec for verification
The version used for this article is 4.4.27-1

Windows Command Prompt (run as administrator)

Commands are highlighted and shown in this font: Courier New

Figure 3. Ensure Command Prompt is run as administrator; otherwise, chef-apply would not run correctly later

Pre-Hardening Configuration

Install Chef Development Kit (on the target system to be hardened) with default settings
Add C:\opscode\chefdk\bin\ to the environment variable – %PATH% – if not already added
Store downloaded windows_hardening directory like our working directory in this example, c:\temp\windows_hardening instead of c:\temp\windows_hardening-master (or main) which may be inaccurately shown in a few screenshots

Pre-Hardening Verification and Backup

For safety, back up the OS drive or at least the system state (most of the settings are registry settings)
Perform 1^st shot with Regshot 2.0

Save as .reg and .hive files (for an additional backup)
Do not close the Regshot application yet (a 2^nd shot will be run at the end of hardening as a comparison)

Verify with inspec

cd C:\temp\windows_hardening\
inspec exec test/integration/default/default_spec.rb
See a report of what have been hardened and what have not, for comparison later

Figure 4. Performing inspec checking before hardening reports a lower number of successful items

Performing Hardening

Friendly reminder: Ensure this is not done to a system without thoughtfulness — it is normal for issues to emerge after hardening. Think twice. Still insist? OK. Here you go!

First, ensure we are at the directory containing recipies.

cd C:\temp\windows_hardening\recipes
dir

Run the listed OS hardening recipes as needed with one of the chef-apply methods below.

Either apply recipes one by one using chef-apply, e.g. on the main recipe file containing most of the settings by replacing <file_name>.rb with core_hardening.rb:
chef-apply <file_name>.rb

Or apply multiple recipes in a go using chef-apply with a for loop:
for /f "usebackq tokens=* delims=" %i in (`dir /b`) do chef-apply "%i"

Figure 5. Applying OS hardening Chef recipe with chef-apply. (Note: some errors may be inevitable)

Tips

For the for-loop technique, please be reminded to carefully examine what will execute as all contents from all recipes will be applied. Some recipes may have a special purpose which are not meant to be run this way.
Most hardening rules should work out of the box and do not require further modification if you choose the compatible Chef recipe named core_hardening.rb (where most of the hardening rules reside).

Side Talk – Ideas for Fixing Recipes Incompatible with chef-apply

Let's take a detour before verifying the results.
The use of chef-apply in this post (treating Chef recipes as scripts), while straight-forward for a quick demonstration or use, is not how Chef is run in production. If you use recipes not from my GitHub repository, you might encounter errors for those incompatible with chef-apply. As an example, in this repository, a majority of errors encountered in the incompatible recipes were eliminated by manually modifying .rb files.
For example, for the harden_winrm.rb recipe in Figure 1 (the top of this article), remove the first line after comment, and if … end outer wrapping from each block, leaving only inner registry_key … end parts.

Figure 6. An edited version of harden_winrm.rb (lower) in comparison with before hardening, i.e. Figure 1 (upper), where the outer if … end has been stripped off

Figure 7. After the edit, 'chef-apply harden_winrm.rb' command runs correctly

Post-Hardening Verification and Backup

Verify with inspec again after applying (Observe the increase in the count of passed items and the decrease in failed items.

cd ..
inspec exec test/integration/default/default_spec.rb

Figure 8. Running the inspec command – 1^st run vs 2^nd run (on an unsupported Windows 10 1709 testing machine for showing the difference only) where the second run has a lower number of failed items

The test run results in Figure 9 are available are available for viewing here

Figure 9. 213 successful and 0 failed items are reported in a case of mine on Windows Server 2016, where my chef-apply-adapted fork of the hardening recipes were used

Perform 2^nd shot with Regshot 2.0 (to compare registry changes)

Save reg and hive files (for backup)
Back up C:\HIVE folder (the Regshot working folder) which contains UNDO and REDO reg files which are useful for rollback and for understanding what has changed. (Note that some keys are unrelated due to other concurrent Windows activities)

Figure 10. Regshot 2.0 produces RedoReg.txt and UndoReg.txt

For failed items reported by the second inspec run which should be much lower now (e.g. 25 failed items in a case of mine on Windows Server 2016, where some of the recipes were left out intentionally; 0 failed items in another case. In case of any issues, you can also always fix them according to the official CIS Benchmarks documents. For user rights and privileges, it would be useful to refer to Microsoft documentation as well.

Is It Good to Go?

Although this is a Windows hardening method that is not officially supported (use it at your own risk), when tested and implemented correctly by a DevOps engineer familiar with Infrastructure-as-Code (here are freelancing marketplaces to hire one), it is an elegant way to improve productivity and make sure security settings are correctly applied in a way that is less prone to human errors. For home servers or labs that do not require as much consideration, this is a godsend. Thanks!

DIY Your Own Easy-to-Use Encrypted Portable Disk – Simplify VeraCrypt with vCrypt2Go

noreply@blogger.com (wandersick) — Sun, 25 Feb 2018 18:27:00 +0000

Are you a security-aware person? Have you got a USB thumb drive or external hard drive you wish to use that does not have any built-in encryption features? You could turn it into one that does, with the cross-platform open-source alternative to BitLocker – TrueCrypt, succeeded by VeraCrypt – but it is not as easy or intuitive to unlock or use as a purchased encrypted external storage solution in the market, because of its feature-rich nature; unlocking is done via an advanced user interface that is more meant for the technically inclined.

Solution – vCrypt2Go and tCrypt2Go, Lock-and-Unlock Utilities, to Simplify and Speed Up Drive Unlocking

Named after VeraCrypt and 'BitLocker To Go', vCrypt2Go – a set of open-source and cross-platform lock-and-unlock utilities on top of VeraCrypt enabling users to DIY their own encrypted portable storage which is simple to use as a purchased solution in the market – simplifies the user experience of VeraCrypt Portable on removable media/external hard disks by hiding the complexities of VeraCrypt and its options from users, making it easy to lock and unlock an encrypted USB thumb drive or hard disk with as few steps as possible. (Likewise, the same applies for the predeccessor of VeraCrypt – TrueCrypt – after which tCrypt2Go is named.)

Go to Download

List of Features

Optimize use of portable drives encrypted by TrueCrypt and VeraCrypt
Easily unlock and lock portable drives with a simplified process, hiding complexities from users
- To unlock, double-click 'unlock', enter password in a simple user interface, then the unlocked drive is presented to the user; to lock, double-click 'lock', no further action required
- Easily identify executable icons which look like a lock (unlocked or locked)
A set of step-by-step installation guides available for encrypting different types of portable drives
Supports both encrypted partitions and containers (virtual disk files)
Handles different scenarios – admin or standard user account; with or without TrueCrypt/VeraCrypt preinstalled
Portable and cross-platform – Windows and Mac OS X (beta)
Automatic User Account Control elevation to highest privileges available on Windows Vista and later

(Refer to Technical Overview article for limitations.)

Seeing It in Action

As demonstrated, the goal of tCrypt2Go/vCrypt2Go is to make it seamless and convenient for users to secure and use their portable drive with the provided set of lock-and-unlock utilities. There will be no dialog shown to users other than entering the password for unlocking the protected area (which is a virtual disk file, or container, in the case of a USB thumb drive of removable media type as demonstrated below, or a partition in the case of an external hard disk of a fixed disk type). It works on Windows (stable) and Mac (beta) depending on which file system is chosen (FAT32, NTFS, exFAT) for formatting the protected partition. (For further details, read the Technical Overview article.)

Requirements

Hardware: Any unused portable storage (e.g. USB thumb drives, memory cards, external hard disks…)
Operating System: Any Windows and Mac operating systems supported by TrueCrypt and VeraCrypt, including Windows XP and later, Mac OS 10.6.5 (10.7 for VeraCrypt) and later

tCrypt2Go and vCrypt2Go on Windows are stable, but for Mac OS X they are currently in beta as they have yet to be tested in newer OS X versions

Getting Started

If you are a user who has been offered a USB drive with tCrypt2Go or vCrypt2Go utilities, read the User Guide below to understand how to use it. Put simply, double-click 'unlock', enter password, done; to lock, double-click 'lock', done.
If you are looking for how to set up a drive for use with tCrypt2Go or vCrypt2Go, read ones of the Installation Guides below. There are two options depending on whether you have a thumb drive or a hard disk.

To learn more about the technical details behind, read the Technical Overview article. Otherwise, feel free to skip to the Installation Guides to implement immediately.

Download

⭐ vCrypt2Go and tCrypt2Go can be downloaded at GitHub Releases .

List of Articles on tCrypt2Go/vCrypt2Go

Open Source

Source Code of tCrypt2Go and vCrypt2Go are available on the main page of the GitHub repository.

Release History

Ver	Date	Update Description
2.1	20180302	Corrected 64-bit executables to 32-bit for better compatibility
2.0	20180225	Added support for VeraCrypt. Change UAC elevation to HighestAvailable. First public release (Mac support is beta)
1.8	20130809	Improved error messages
1.7	20130624	Fixed a use for standard users under XP while TrueCrypt driver is installed - a bug introduced by new code
1.6	20130509	Added more meaningful error messages
1.5	20130320	Fixed an issue for non-admin users where TrueCrypt device driver is installed and UAC is enabled, a UAC prompt would be triggered but users cannot authenticate as they are not admins. They are now redirected to TrueCrypt installation prompt
1.4	20130118	Improved Mac and Windows executables. Checks for admin right and whether TrueCrypt is installed. Prompts user for TrueCrypt setup if required
1.3	20130117	Added hidden file support of Mac OS X
1.2	20121211	New version which uses AutoHotkey (for Windows)
1.1	20121105	Adds Mac OS X support using AppleScript and shell script
1.0	20121105	First private release using Windows batch script

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

Technical Overview – How tCrypt2Go and vCrypt2Go work

noreply@blogger.com (wandersick) — Sun, 25 Feb 2018 18:26:00 +0000

This is an article for enthusiasts detailing the technical design of tCrypt2Go for TrueCrypt (also applicable to vCrypt2Go for VeraCrypt unless otherwise specified).

(This is a sub-article – click here to view to the list of articles or the main article of tCrypt2Go and vCrypt2Go)

Topics Covered in this Article

1. Container for Thumb Drive (Removable Media) vs Partition for External Hard Disk (Fixed Disk)

In the high level, there are two modes from which a user can choose to encrypt their portable drives:

Encrypted Partition Mode (Entire Partition Encrypted) for a Hard Drive Setup
Encrypted File Container Mode (a Virtual Encrypted Disk within a File) for a Thumb Drive Setup or a Hard Drive Setup

Below comparison picture illustrates the difference between them:

※ 1 : For hard disks only, this would be the first unencrypted partition where the tCrypt2Go utilities are stored and accessed. This place should be as widely accessible as possible. Using exFAT would limit Windows XP from accessing this area. FAT would be most compatible so it is preferred. (Use NTFS if you agree FAT is more prone to file system corruption, however.)

※ 2 : For both thumb drives and hard disks, this is the encrypted area which must be formatted using TrueCrypt Format tool. However, for TrueCrypt, exFAT is not supported by the TrueCrypt formatter; we need to use command:

format {drive letter}: /fs:exfat /q /y (Command available in Windows Vista or later)

Encrypted Partition Mode

This mode requires two separate partitions – a smaller partition for unprotected area for storing lock-and-unlock utilities and a larger partition for protected encrypted data
This mode does NOT support devices of a Removable Media type, such as thumb drive or memory card
This mode supports devices of a Fixed Disk type, such as a USB external hard disk

Encrypted File Container Mode

This mode requires a single partition for both unprotected and protected areas
This mode supports both Removable Media and Fixed Disk device types
It involves much fewer configuration steps and has the benefit of unlocking (mounting virtual disk) without admin privileges on certain platforms

In Other Words…

Characteristics of a Thumb Drive Setup

One unencrypted partition (for storing tCrypt2Go utilities and TrueCrypt as well as encrypted container file)
TrueCrypt creates an encrypted container file that fills up the whole partition except tCrypt2Go utilities and TrueCrypt
tCrypt2Go utilities and TrueCrypt are stored in the same partition for unlocking (mounting) the encrypted file

Characteristics of a Hard Disk Setup

Two partitions: first partition is unencrypted and small in size (for storing tCrypt2Go utilities and TrueCrypt); second partition is encrypted and large in size (for storing user files)
TrueCrypt encrypts the second partition completely
tCrypt2Go utilities and TrueCrypt are stored in the unencrypted first partition for unlocking (mounting) the encrypted partition

Summary

If you only have a thumb drive (not an external hard drive), container must be used
If you have both types of devices
- If you are technical (or not afraid of challenges), choose separate modes for different device types – partition for USB hard disks and container for thumb drives
- If you are non-technical (or have no time for learning), choose container for everything

Files Stored in the Unprotected Partition

The below are files extracted from "tCrypt2Go and vCrypt2Go.zip" [Download] and copied to the root of the UNPROTECTED drive.

.DO_NOT_DELETE (hidden from Windows with a hidden attribute)
- TrueCrypt/VeraCrypt for Mac
  - README.txt
  - TrueCrypt.dmg/VeraCrypt.dmg (optional: store the original Mac installer here)
- TrueCrypt/VeraCrypt for Windows
  - README.txt
  - TrueCrypt Setup.exe/VeraCrypt Setup.exe (optional: store the original Windows installer here)
- Lock_Mac.command
- Unlock_Mac.command
- tc.exe/vc.exe
- TrueCrypt.sys/VeraCrypt.sys
- TrueCrypt.sys/VeraCrypt-x64.sys
- tc-container.tc/vc-container.vc (the encrypted container file built with ‘TrueCrypt Format.exe’ or ‘VeraCrypt Format.exe’ for Removable Media device type only)

AUTORUN.INF (to prevent older Windows versions from executing malware triggered via Autorun mechanism; hidden from Windows with a hidden attribute)
.Unlock.exe (unlock utility for Windows; hidden from Mac with a preceding dot)

.Lock.exe (lock utility for Windows; hidden from Mac with a preceding dot)

Lock.app (lock utility for Mac; hidden from Windows with a hidden attribute)

Unlock.app (unlock utility for Mac; hidden from Windows with a hidden attribute)

._Lock.app (specifies an icon for Mac; hidden from Windows with a hidden attribute)

._Unlock.app (specifies an icon for Mac; hidden from Windows with a hidden attribute)

Files with a Hidden Attribute Versus Those with a Preceding Dot

Selected files above have a hidden attribute (+h) set so that they are hidden in Windows and newer version of Mac OS X.

Hidden attribute vs dot (.)
- Some folders and files such as .Unlock.exe and .Lock.exe (Windows applications) are named with preceding dot so that they are hidden in older versions of Mac OS X. (The Finder application from newer versions of OS X can hide files and folders with a hidden attribute set, but older OS X versions only support hiding files with a dot.)
- Hidden attributes are not set on Unlock.app and Lock.app (Mac applications) because doing so would make them invisible in newer versions of Mac OS X. However, that makes them visible in Windows at all times. If Mac OS X support is not required, you may remove them to avoid causing confusion to users.
- ._Unlock.app and ._Lock.app are for Mac OS X. They contain the applicatin icon and metadata (generated by Finder); whereas on Windows, the icons are embedded in the executables.
AUTORUN.INF (with read-only and hidden attributes) is there to protect the drive from being infected with XP-era autorun viruses.

2. Choosing a File System for the Encrypted Partition

The below flowchart helps deciding which file system format to use (FAT32, exFAT or NTFS).

※ 1 : exFAT is supported after Windows Vista SP1 and Snow Leopard 10.6.5. For Windows XP to support exFAT, install this update (KB955704): http://support.microsoft.com/kb/955704

※ 2 : For USB thumb drives larger than 4GB and formatted in FAT32, the maximum size of TrueCrypt container still can only be 4GB or less. That means the rest of storage space will be wasted, unencrypted and insecure.

Decide in Another Way – Answering the Question: On Which Operating System Do You Plan to Use the Portable Drive?

Windows-only

Suggested file system: NTFS
For standard users, TrueCrypt preinstallation is required. If TrueCrypt is not detected, users will be prompted to install it. Corporate users will need to acquire approval from their IT administrators to input admin credentials to continue the installation.
For users with administrator rights, TrueCrypt is executed automatically in portable mode. There is no need for a TrueCrypt installation.

Windows and Mac OS X (Mac OS X 10.6.5 and later/Windows Vista SP1 and later)

Suggested file system: exFAT
For standard users, TrueCrypt preinstallation is required. If TrueCrypt is not detected, users will be prompted to install it. Corporate users need to acquire approval from their IT administrators to input admin credentials to continue the installation.
For users with administrator rights, preinstalling TrueCrypt is still suggested to minimize nuisance. (If TrueCrypt is not detected, users will be prompted to install TrueCrypt on first use. There is no need to acquire approval from their IT administrators as their accounts have enough rights to install software.)
An one-time visudo command setting is required for standard users to mount partitions.
For administrators, there is no need for this as their accounts have enough rights to run applications as administrator via the sudo command.

Linux

Unsupported by tCrypt2Go lock-and-unlock utilities, although TrueCrypt supports Linux. Linux users should have the skills to mount TrueCrypt volumes/partitions without the help of tCrypt2Go utilities. :)

3. Pros and Cons of Each File System (NTFS, exFAT and FAT32)

NTFS

Compatibility: Windows only
- Read-only (no write support by default) on Mac OS X. Methods such as NTFS-3G and OSXFuse are required for writing to NTFS, but performance could be poor. Use exFAT instead if Mac support is required.
Max volume size: 16TB minus 4KB (for 4KB cluster size)
Max per-file size: 2^64 bytes (16 ExaBytes) minus 1KB

exFAT

Compatibility: Cross-platform (Windows and Mac)
Supported OS versions:
- Mac OS X 10.6.5 or later
- Windows Server 2008/Vista SP1 or later
- Windows XP and Server 2003 with an update: http://support.microsoft.com/kb/955704
- By default, TrueCrypt format tool cannot format external drives to exFAT. It is required to the command format {drive letter} /fs:exfat /q /y or use VeraCrypt format tool
Max volume size: 128PB
Max per-file size: 16EB

FAT32

Compatibility: Cross-platform (Windows and Mac)
Supported OS versions: all
Max volume size: mostly 32GB (varies among applications and operating systems)
Max per-file size: 4GB minus 2 bytes

Summary

Use exFAT if you need NATIVE cross-platform (Windows and Mac) support and do not need support of Windows XP or Server 2003 or earlier and Mac OS X versions earlier than 10.6.5
Use NTFS if you do not need Mac OS X support or do not mind installing a few packages to get write-support on Mac OS X
Use FAT32 if your drive has a small capacity which is equal to or under the supported maximum size of FAT32

For users who have chosen partition mode (not container mode): the above are considerations only for the PROTECTED partition containing the encrypted data. For the very small UNPROTECTED partition containing the tCrypt2Go/vCrypt2Go lock-and-unlock utilities (used for unlocking the PROTECTED partition), always use FAT32.

After you have made a choice of file system, continue with the Installation Guide (step-by-step instructions) to set up your portable drive for use with tCrypt2Go and vCrypt2Go.

4. The Unlocking Flowchart

Below flowchart illustrates how the tCrypt2Go unlocker decides how TrueCrypt is run to unlock the protected drive.

※ 1 : User will be presented a screen with TrueCrypt setup file and a README.
※ 2 : User Account Control (UAC) has to be disabled for TrueCrypt device driver to work, or install TrueCrypt to keep UAC on.
※ 3 : During the unlocking (mounting) of TrueCrypt-encrypted partitions under Mac OS, TrueCrypt will sudo (prompt users for admin password). While it is fine for admin users, standard users cannot answer the sudo prompt successfully due to their lack of admin rights. Additional settings are required for standard users to unlock the encrypted partition.

Note: TrueCrypt-encrypted containers do not require admin rights, but partitions do.

5. Commands for Locking and Unlocking

The below commands could be all that you need if you merely need to make your own scripts!

Windows - Lock

TrueCrypt: TrueCrypt.exe /d /f /q
VeraCrypt: VeraCrypt.exe /d /f /q

Windows - Unlock - Removable Media

TrueCrypt: TrueCrypt.exe /w /c n /v container.tc
VeraCrypt: VeraCrypt.exe /w /c n /v container.vc

Windows - Unlock - Fixed Disk

TrueCrypt: TrueCrypt.exe /w /c n /a devices /q /e
VeraCrypt: VeraCrypt.exe /w /c n /a devices /q /e

Mac OS X - Lock

TrueCrypt: /Applications/TrueCrypt.app/Contents/MacOS/TrueCrypt -d –force
VeraCrypt: /Applications/VeraCrypt.app/Contents/MacOS/VeraCrypt -d –force

Mac OS X - Unlock - Removable Media

TrueCrypt: TrueCrypt.app/Contents/MacOS/TrueCrypt container.tc /Volumes/TrueCrypt –force
VeraCrypt: VeraCrypt.app/Contents/MacOS/VeraCrypt container.vc /Volumes/VeraCrypt –force

Mac OS - Unlock - Fixed Disk

TrueCrypt: /Applications/TrueCrypt.app/Contents/MacOS/TrueCrypt --auto-mount=devices --force
VeraCrypt: /Applications/VeraCrypt.app/Contents/MacOS/VeraCrypt --auto-mount=devices --force

List of Articles on tCrypt2Go/vCrypt2Go

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

Installation Guide – Setting up tCrypt2Go and vCrypt2Go to Encrypt Portable Hard Disks

noreply@blogger.com (wandersick) — Sun, 25 Feb 2018 18:25:00 +0000

Warning: This guide is for system administrators or experienced enthusiasts. It may involve a steep learning curve if you are a novice, and risk breaking systems or losing data if the involved low-level system tools (such as diskpart) are not used carefully. There is no guarantee. One way to avoid risk is to engage professionals to perform the process on one of the freelancing platforms.

This is an article for users illustrating how to encrypt a portable hard disk (click here if you have a thumb drive or memory card instead) as a partition for use with tCrypt2Go lock-and-unlock utilities for TrueCrypt Portable (also applicable to vCrypt2Go for VeraCrypt unless otherwise specified).

(This is a sub-article – click here to view to the list of articles or the main article of tCrypt2Go and vCrypt2Go)

List of Steps

Note: This method is slightly more difficult than setting up tCrypto2Go/vCrypt2Go for devices of the Removable Media type (such as a thumb drive or memory card). For a simpler approach (if partitions are not a must for you), you may read the Installation Guide for USB Thumb Drives (Removable Media) instead which leverages TrueCrypt/VeraCrypt container instead of partition. Containers (encrypted virtual disks) are supported by every device type (including USB hard disks and thumb drives), while what is described below is supported only by devices of the Fixed Disk type (such as a USB hard disk).

Required Files

tCrypt2Go and vCrypt2Go.zip [Download]
TrueCrypt [Download] or VeraCrypt [Download]

Supported Hardware

A portable drive of Fixed Disk type (such as a USB hard disk, NOT thumb drive or memory card.

1. Creating 2 Partitions and Copy TrueCrypt/VeraCrypt Files to the Partition Labelled UNPROTECTED

To encrypt portable drives of a Fixed Disk type (where more than one partition is supported), there will be two partitions. We need to partition both using the Disk Management utility in Windows.

a. Connect a portable drive to be encrypted to a PC running Windows 7 or above. (required for an option in newer diskpart)

b. Run Disk Management (press "Win+R" hotkey, enter "diskmgmt.msc")

c. Create two partitions. smaller one: 20MB. larger one: All available space. The smaller partition should be formatted as FAT32, while the larger one can be anything at this stage.
Label the smaller partition "UNPROTECTED" (to make it clear to the user that it is the gate to the PROTECTED drive to be unlocked using TrueCrypt/VeraCrypt via vCrypt2Go lock-and-unlock utilities.)

(Also, identify and write down the disk number and partition number of the encrypted partition on this screen. It will be required in section 3 while using diskpart. In this case, disk number is 1, encrypted partition number is 2)

Download and extract "tCrypt2Go and vCrypt2Go.zip" and copy all files under "tCrypt2Go\Hard Disk" or “vCrypt2Go\Hard Disk” folder to the root of the UNPROTECTED drive:

.DO_NOT_DELETE (hidden from Windows with a hidden attribute)
- TrueCrypt/VeraCrypt for Mac
  - README.txt
  - TrueCrypt.dmg/VeraCrypt.dmg (optional: download TrueCrypt or VeraCrypt and store the original Mac installer here)
- TrueCrypt/VeraCrypt for Windows
  - README.txt
  - TrueCrypt Setup.exe/VeraCrypt Setup.exe (optional: download TrueCrypt or VeraCrypt and store the original Windows installer here)
- Lock_Mac.command
- Unlock_Mac.command
- tc.exe/vc.exe (REQUIRED: download TrueCrypt or VeraCrypt and perform installation or extraction, then copy TrueCrypt.exe or VeraCrypt.exe here and rename them as tc.exe or vc.exe)
- TrueCrypt.sys/VeraCrypt.sys (REQUIRED: as above, copy TrueCrypt.sys or VeraCrypt.sys here)
- truecrypt-x64.sys/VeraCrypt-x64.sys (REQUIRED: as above, copy TrueCrypt-x64.sys or VeraCrypt-x64.sys here)
AUTORUN.INF (to prevent older Windows versions from executing malware triggered via Autorun mechanism; hidden from Windows with a hidden attribute)
.Unlock.exe (unlock utility for Windows; hidden from Mac with a preceding dot)

.Lock.exe (lock utility for Windows; hidden from Mac with a preceding dot)

Lock.app (lock utility for Mac; hidden from Windows with a hidden attribute)

Unlock.app (unlock utility for Mac; hidden from Windows with a hidden attribute)

._Lock.app (specifies an icon for Mac; hidden from Windows with a hidden attribute)

._Unlock.app (specifies an icon for Mac; hidden from Windows with a hidden attribute)

e. Ensure only the folders highlighted below are configured with hidden attributes

f. Right-click "DO_NOT_DELETE" folder, choose "Properties"

Configure the hidden attributes accordingly. Also, ensure "Read-only" is not set on all folders except AUTORUN.INF; otherwise, the drive cannot be written.

2. Eliminating Free Space on UNPROTECTED Partition (Preventing Users from Saving Files into Unencrypted Area)

After copying the files, the disk will still have a few megabytes of free space:

Record the red-circled area above, it will be used in step 2b. (In above example, it is 3510272)

We need to eliminate that so that user will not be able to put any file unencrypted at all.

a. Run Command Prompt as Administrator
b. Create a dummy file by entering fsutil file createnew DUMMY_FILE {input here the free space shown above without commas}

c. Finally, copy DUMMY_FILE into "UNPROTECTED\DO_NOT_DELETE"

d. The end result shows that there is no free space left.

3. Encrypting Second Partition with TrueCrypt/VeraCrypt

Note: the screenshots below are from TrueCrypt only, as there is not much difference between TrueCrypt and VeraCrypt.

a. Download and run TrueCrypt Setup/VeraCrypt Setup (see step 1c above for the location of this file) and install the program, or specify to extract the files to a location.
b. In extracted or installed directory, run "TrueCrypt Format.exe/VeraCrypt Format.exe"

c. Select "Encrypt a non-system partition/drive".

d. Select "Standard TrueCrypt/VeraCrypt" volume.

e. Choose "Select Device".

f. Highlight the partition to be encrypted.

g. Choose "Next".

h. Create encrypted volume and format it.

i. For "Encryption Algorithm" and "Hash Algorithm", choose any settings as desired.

j. Enter the maximum available size to be encrypted.

k. Enter a secure password twice.

l. "Yes" for Large Files.

m. Choose any desired file system (according to the explanation at the top of this guide). In this example, the below are selected:

Filesystem: NTFS; Cluster: Default; Quick Format: Checked

It is required to keep moving the cursor around the window for few seconds, then start formatting by clicking "Format".

(Note for exFAT and TrueCrypt users: Unless you use VeraCrypt, this TrueCrypt format tool cannot format a drive as exFAT. TrueCrypt users need to perform it in the Command Prompt. See Optional section for how.)

n. Answer "Yes" if the below warning is prompted.

o. Answer "OK" if the below warning is prompted.

p. Click OK and Exit to complete the procedure.

4. Unlocking Encrypted Partition

a. Unlock by double-clicking "UNPROTECTED\.Unlock.exe".

(Note: If you are mindful, you may have noticed in the example above, there are no preceding dot characters and the Mac-related files are removed. You may do so only if you do not require Mac support.)

b. Enter password.

c. Then the encrypted partition will be unlocked as below (E:\). Right now it is empty.

5. Labeling Encrypted Partition as 'PROTECTED'

a. Open "Computer". Label the encrypted partition 'PROTECTED'.

b. Run "diskmgmt.msc", confirm the partition layout and labels are as follows:

(Note: the volume labels may not always show on Windows, but it does on Mac. It is recommended to name them so for easier identification.)

6. Copying TrueCrypt/VeraCrypt Files to Encrypted Partition

Now, we will copy a few files there so that users can easily lock it within the encrypted partition.

a. Copy the a few files from step 1d here (so that users can conveniently lock the drive after use)

.DO_NOT_DELETE (hidden from Windows)
AUTORUN.INF (hidden from Windows)
Lock.app
._Lock.app (hidden from Windows)
.Lock.exe

b. Ensure only the highlighted folders/files below are configured with a hidden attribute.

c. Double-click ".Lock.exe" to test. If the encrypted partition letter (e.g. E:) becomes unmapped (dismounted), it works!

7. Suppressing "Format Disk" Prompt with Diskpart

There could be an annoying imperfection – whenever the drive is connected, Windows will asks whether you want to format it or not.

To work around the issue, use the command line disk partitioning tool "diskpart" to modify the partition type to 0x64. (Please make sure the below procedure is performed on Windows 7 or later only.

a. Run Command Prompt as Administrator
b. Enter diskpart

c. We will now use what was written down in section 1c. (disk number and partition number)
For this example, enter select disk 1

d. Enter select partition 2

e. Enter set id=64

Finished.

List of Articles on tCrypt2Go/vCrypt2Go

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

Installation Guide – Setting up tCrypt2Go and vCrypt2Go to Encrypt USB Thumb Drives

noreply@blogger.com (wandersick) — Sun, 25 Feb 2018 18:24:00 +0000

Warning: This guide is for system administrators or experienced enthusiasts. It may involve a steep learning curve if you are a novice, and risk breaking systems or losing data if the involved low-level system tools (such as diskpart) are not used carefully. There is no guarantee. One way to avoid risk is to engage professionals to perform the process on one of the freelancing platforms.

This is an article for users illustrating how to encrypt a USB thumb drive or memory card (click here if you have a portable hard disk instead) as a container (virtual disk file) for use with tCrypt2Go lock-and-unlock utilities for TrueCrypt Portable (also applicable to vCrypt2Go for VeraCrypt unless otherwise specified).

(This is a sub-article – click here to view to the list of articles or the main article of tCrypt2Go and vCrypt2Go)

List of Steps

Required Files

"tCrypt2Go and vCrypt2Go.zip" [Download]
TrueCrypt [Download] or VeraCrypt [Download]

Supported Hardware

A portable drive of Removable Media type (such as a thumb drive or memory card)
A portable drive of Fixed Disk type (such as an external hard disk)
- If you have this type of device, you may also the Installation Guide for Portable Hard Disk instead

1. Formatting Thumb Drive and Copy TrueCrypt Files

For USB thumb drives, We need to format it using Windows Explorer (or File Explorer).

a. Connect the drive to be encrypted to a PC running Windows XP or above. (Note: TrueCrypt discontinued support for Windows 2000 although it could still work.)
b. Run My Computer (press "Win+E" hotkey). Right-click the drive, select "Format"

c. Volume label: "UNPROTECTED"; Format options: "Quick Format"

d. Download and extract "tCrypt2Go and vCrypt2Go.zip" and copy all files in "tCrypt2Go\Thumb Drive" or "vCrypt2Go\Thumb Drive" folder to the root of the UNPROTECTED drive:

.DO_NOT_DELETE (hidden from Windows with a hidden attribute)
- TrueCrypt/VeraCrypt for Mac
  - README.txt
  - TrueCrypt.dmg/VeraCrypt.dmg (optional: download TrueCrypt or VeraCrypt and store the original Mac installer here)
- TrueCrypt/VeraCrypt for Windows
  - README.txt
  - TrueCrypt Setup.exe/VeraCrypt Setup.exe (optional: download TrueCrypt or VeraCrypt and store the original Windows installer here)
- Lock_Mac.command
- Unlock_Mac.command
- tc-container.tc/vc-container.vc (REQUIRED: the encrypted container file built with ‘TrueCrypt Format.exe’ or ‘VeraCrypt Format.exe’ for Removable Media device type)
AUTORUN.INF (to prevent older Windows versions from executing malware triggered via Autorun mechanism; hidden from Windows with a hidden attribute)
.Unlock.exe (unlock utility for Windows; hidden from Mac with a preceding dot)
.Lock.exe (lock utility for Windows; hidden from Mac with a preceding dot)
Lock.app (lock utility for Mac; hidden from Windows with a hidden attribute)
Unlock.app (unlock utility for Mac; hidden from Windows with a hidden attribute)
._Lock.app (specifies an icon for Mac; hidden from Windows with a hidden attribute)
._Unlock.app (specifies an icon for Mac; hidden from Windows with a hidden attribute)

(Note: If you are mindful, you may have noticed in the example above, there are no preceding dot characters and the Mac-related files are removed. You may do so only if you do not require Mac support.)

e. Ensure only the folders highlighted below are configured with hidden attributes

f. Right-click "DO_NOT_DELETE" folder, choose "Properties"
Configure the hidden attributes accordingly. Also, ensure "Read-only" is not set on all folders except AUTORUN.INF; otherwise, the drive cannot be written.

For "AUTORUN.INF", set it to "Read-only" and "Hidden"

2. Creating a 10 MB Dummy File as Buffer (Optional)

Note: this step is optional, but is recommended in preparation for a possible future situation in which more storage space is required for the executables or related resources required to unlock and lock the protected area.

a. Run Command Prompt as Administrator
b. Create a dummy file by entering fsutil file createnew DUMMY_FILE 10000000
c. Finally, copy DUMMY_FILE into "UNPROTECTED\DO_NOT_DELETE"

3. Creating Encrypted Container with TrueCrypt

a. Download and run TrueCrypt Setup/VeraCrypt Setup (see step 1d above for the location of this file) and install the program, or specify to extract the files to a location.
b. In extracted or installed directory, run "TrueCrypt Format.exe" (for TrueCrypt) or "VeraCrypt Format.exe" (for VeraCrypt).

c. Select "Create an encrypted file container".

d. "Select File"

e. Browse "DO_NOT_DELETE" folder.

f. Enter file name: tc-container.tc

g. Confirm path, which is "G:\DO_NOT_DELETE\tc-container.tc" for TrueCrypt in this example (or "vc-container.vc" for VeraCrypt). Then click "Next".

h. For "Encryption Algorithm" and "Hash Algorithm", choose any settings as desired.

i. We need to find out how much free space there is for encryption.

In "My Computer" (press "Win+E" hotkey), right-click the drive, select "Properties"

j. Copy the free space, e.g. 3981664256 (bytes)

k. Convert bytes to megabytes by inputting on Google: 3981664256 / 1024 / 1024

l. Use the rounded result "3796" here. Select "MB". We want to use as much free space as possible for encryption.

m. Enter a secure password.

n. Choose any desired file system (according to the explanation at the top of this guide). In this example, the below are selected:

Filesystem: FAT; Cluster: Default

It is required to keep moving the cursor around the window for few seconds, then start formatting by clicking "Format".

(Note for exFAT and TrueCrypt users: Unless you use VeraCrypt, this TrueCrypt format tool cannot format a drive as exFAT. TrueCrypt users need to perform it in the Command Prompt. See Optional section or how.)

o. Wait until it is done. It will take a long time for large or slower USB 2.0 flash drives.

4. Unlocking Encrypted Partition and Copy TrueCrypt Files There

Now, we will test the encrypted partition by unlocking it and copy a few files there so that users can easily lock it back.

a. Unlock by double-clicking "UNPROTECTED\.Unlock.exe".

b. Enter password.

c. Then the encrypted partition will be unlocked as below. Right now it is empty.

d. Optionally, copy "Lock.exe", the hidden "DO_NOT_DELETE" and "AUTORUN.INF" folders from step 1d here (so that users can conveniently lock the drive after use, and prevent the drive from executing Autorun malware under XP.)

e. Double-click "Lock.exe" to test. If the encrypted partition letter (e.g. E:) is unmapped (dismounted), it works!

List of Articles on tCrypt2Go/vCrypt2Go

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

User Guide – How to Unlock and Lock Encrypted Drives using tCrypt2Go or vCrypt2Go

noreply@blogger.com (wandersick) — Sun, 25 Feb 2018 18:23:00 +0000

This is an article for users illustrating how to unlock and lock drives encrypted for use with tCrypt2Go for TrueCrypt Portable (also applicable to vCrypt2Go for VeraCrypt unless otherwise specified) on PC and Mac.

(This is a sub-article – click here to view to the list of articles or the main article of tCrypt2Go and vCrypt2Go)

Topics Covered in this Article

1. Unlocking and Locking Encrypted Drives with tCrypt2Go – Admin User Guide – PC

In this section, the method for unlocking encrypted portable drive with administrator privileges under Windows is introduced.

a. Insert portable drive (encrypted according to tCrypt2Go/vCrypt2Go configuration guide)

Then go to [Start] > Computer.

b. Double-click drive with the label UNPROTECTED

× Reminder: It may not always be named UNPROTECTED; it can also be “Local Disk” or anything.

(This is unencrypted area, hence available space is 0 byte. Do not attempt to store anything.)

c. Double-click “.Unlock.exe” to unlock protected area.

d. Enter TrueCrypt/VeraCrypt password to unlock protected area.

e. If successful, an additional drive, usually named PROTECTED, will be mounted and prompted.

After use, double-click “.Lock.exe” to lock the drive.

2. Unlocking and Locking Encrypted Drives with tCrypt2Go – Admin User Guide – Mac

See Section 4, i.e. the Standard User Guide for Mac.

3. Unlocking and Locking Encrypted Drives with tCrypt2Go – Standard User Guide – PC

In this section, the method for unlocking encrypted portable drive under Windows is introduced.

a. Insert portable drive (encrypted according to tCrypt2Go/vCrypt2Go configuration guide).
Then go to [Start] > Computer.

b. Double-click drive with the label UNPROTECTED

× Reminder: It may not always be named UNPROTECTED; it can also be “Local Disk” or anything.

(This is unencrypted area, hence available space is 0 byte. Do not attempt to store anything.)

c. Execute “.Unlock.exe”

d. If there are no admin rights and on first-use, user will be prompted to install TrueCrypt/VeraCrypt

Press OK and continue with TrueCrypt installation. (If required, acquire approval from IT.)

Note: If TrueCrypt or VeraCrypt is already installed/admin rights are available (no need for installation)

If not first-use, please skip the next few steps.

e. Installation method of TrueCrypt (similarly for VeraCrypt) will be introduced as follows.

After clicking OK, a File Explorer window will be prompted, in which “TrueCrypt Setup.exe” or “VeraCrypt Setup.exe” will be presented for installation. Double-click it.

Note: In addition, TrueCrypt and VeraCrypt installers can be acquired respectively from:
- http://www.truecrypt71a.com and http://www.veracrypt.fr

f. Enter admin credentials

× Reminder: The below window may not be presented in some Windows systems, such as Windows XP or later Windows versions with UAC disabled.

g. Check “I accept the license term” to accept end-user licensing agreement. Click Next.

h. Select Install and click Next to begin installation on the current computer.

i. Click Install.

j. When installation is complete, click Finish.

k. Click No to skip tutorial (if prompted).

l. Now, with TrueCrypt/VeraCrypt installed, we can unlock protected area by double-clicking “.Unlock.exe”

m. Enter TrueCrypt/VeraCrypt password to unlock protected area

n. If successful, an additional drive, usually named PROTECTED, will be mounted and prompted.

Now the encrypted data in the protected area can now be accessed. Feel free to read or write there and your data will be encrypted.

o. After use, double-click “.Lock.exe” to lock the drive.

p. To safely remove the USB device, go to the tray area and perform “Safely Remove Hardware…”.

4. Unlocking and Locking Encrypted Drives with tCrypt2Go – Standard User Guide – Mac

In this section, the method for unlocking encrypted portable drive under Mac OS X is introduced.

× Note: For Mac OS X, it is optimal if TrueCrypt is installed. (For TrueCrypt encrypted partition, this is a must; for TrueCrypt container, it is still recommended to install TrueCrypt.)
× Reminder: Under default configuration, Mac OS X can only read but not write to NTFS file systems; if write is required, please format your portable drive as FAT32 or exFAT according to configuration guide.

a. Insert portable drive (encrypted according to tCrypt2Go/vCrypt2Go configuration guide). Run Finder.
b. Click the volume on the left pane (usually named UNPROTECTED) [1]; execute “Unlock” on the right. [2]

c. On first use, a message like the below will be prompted. Click OK to install TrueCrypt or VeraCrypt. (If required, acquire approval from IT.)

Note: Mac OS X behaves differently from Windows, where TrueCrypt must be installed in order to mount encrypted partitions (not encrypted containers) even if admin privileges are available

× If this is not the first time of use, or TrueCrypt/VeraCrypt is already installed, skip to Step 11

d. Installation method of TrueCrypt (similarly for VeraCrypt) will be introduced as follows.

After clicking OK, another Finder window will be prompted, presenting the installer. Double-click the installer.

× In addition, TrueCrypt and VeraCrypt installers can be acquired respectively from:
- http://www.truecrypt71a.com and http://www.veracrypt.fr

e. Click Accept to accept the end-user licensing agreement.
f. Double-click the installer presented.

g. Click Continue to continue installation.

h. Click Install to begin installation.

i. Enter admin credentials.

j. When done successfully, click Close to exit.

k. Now, with TrueCrypt/VeraCrypt installed, we can unlock protected area by double-clicking “Unlock”.

In Finder, click UNPROTECTED on the left pane [1]; click Unlock on the right. [2]

l. Enter TrueCrypt/VeraCrypt password to unlock protected area.

m. If the below “administrator privileges required” dialog is prompted, enter the admin password.

n. In case TrueCrypt/VeraCrypt fails to obtain administrator privileges, you may (ask an IT admin to) refer to “Granting Non-Admin Mac Users TrueCrypt/VeraCrypt Right (by Editing /etc/sudoer)” section in the configuration guide.

o. Now, try accessing data in the encrypted area.

Execute Finder again. There will be an additional partition on the left (usually named PROTECTED). Double-click the drive. Feel free to read or write there and your data will be encrypted.

× Reminder: Under default configuration, Mac OS X can only read but not write to NTFS file systems; if write is required, please format your portable drive as FAT32 or exFAT according to configuration guide.
p. After use, double-click “Lock” to lock the protected area. Before unplugging device, click the icon on the left.

q. At times, the below warning is prompted after unplugging device. Click Ignore.

List of Articles on tCrypt2Go/vCrypt2Go

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

Enhancing and Troubleshooting Common Issues with tCrypt2Go or vCrypt2Go

noreply@blogger.com (wandersick) — Sun, 25 Feb 2018 18:22:00 +0000

This is an article illustrating how to add miscellaneous enhancements or troubleshoot common issues with tCrypt2Go for TrueCrypt Portable (also applicable to vCrypt2Go for VeraCrypt unless otherwise specified).

(This is a sub-article – click here to view to the list of articles or the main article of tCrypt2Go and vCrypt2Go)

Topics Covered in this Article

1. Formatting Encrypted Partition as exFAT in Command Prompt for Cross-Platform Support (i.e. Write Support in Mac OS X) – exFAT and TrueCrypt Only

Skip this if you accept the limitations of NTFS – you don't require write support under Mac OS X
Skip this if you use VeraCrypt, the GUI format tool of which already include exFAT option

TrueCrypt format tool cannot format a drive as exFAT, but we may perform it using Command Prompt on Windows Vista SP1 or above:

a. Open Command Prompt as Administrator
b. Enter format {drive letter}: /fs:exfat /q /y

Note: exFAT is supported after Windows Vista SP1 and Mac OS X 10.6.5.

For Windows XP and Server 2003, it is required to install this update to enable exFAT support: http://support.microsoft.com/kb/955704

2. Granting Non-Admin Mac Users TrueCrypt/VeraCrypt Right (by Editing /etc/sudoer) – Mac-Only

Skip this step if only thumb drives (container mode) will be used. That is because only using TrueCrypt/VeraCrypt partitions (USB hard disks) requires admin rights under OS X, whereas TrueCrypt/VeraCrypt container (thumb drives) doesn't.
Skip this step if only Windows support is required.

After installing TrueCrypt/VeraCrypt on OS X, you may encounter the following issue if you use TrueCrypt/VeraCrypt on user accounts that have only standard user privileges.

The instructions below are meant to make it possible for non-admin Mac users to use TrueCrypt/VeraCrypt. It fixes the 'administrator privileges' prompt which pop ups while unlocking TrueCrypt/VeraCrypt partitions, as standard users do not have sudo rights at all.
Note:

These settings are required for non-admin Mac OS users to avoid failure during unlocking (mounting) TrueCrypt-encrypted partitions as detailed in the Unlocking Flowchart from the Technical Overview article.
The screenshots below focus on TrueCrypt only. You may need to adjust a bit for VeraCrypt.
This needs to be set on user's Mac machine because it is a per-machine setting.

a. First, ensure TrueCrypt/VeraCrypt for Mac is installed.

b. At the top right corner of OS X (Spotlight), input 'term', then click 'Terminal'

c. Before moving on, we need to find the real username of the user account, because the username may be different from a user's display name:

To make sure, we run some commands to find out.
Method a. If current-logged-on user account is the user's:

whoami

From this example, the username is 'wanderer'

Method b. If current user account is root/administrator

dscacheutil -q user | grep -A 3 -B 2 -e uid:\ 5'[0-9][0-9]'

From this example, we only have one user on this Mac and its true user name is 'user' too. (Here, the difference between the display name 'User' and the real account name 'user' is that 'U' is a different case.)

d. After confirming the username, enter: sudo visudo

e. Press [i] key to switch to -- INSERT -- mode. Add this line at the end of the file

{username} ALL = NOPASSWD: /Applications/TrueCrypt/VeraCrypt.app/Contents/MacOS/TrueCrypt/VeraCrypt --core-service

Change {username} to the user's real account name. In this example, it is 'user'

("%users" may not always work and should not be used.)

f. When all is OK, press [Esc] key to switch back to command mode, then input :wq! to save the file and exit.

g. Done! The setting should take effect immediately. Now log on to the user's account and try unlocking any TrueCrypt/VeraCrypt partitions without administrator privileges.
Reminder: Again, this is a per-machine setting that needs to be set on each Mac computer which requires the use of TrueCrypt/VeraCrypt.

3. Avoiding User Account Control (UAC) Prompts – Optional

During use of the encrypted storage device on Windows, clicking '.Unlock.exe' may activate a User Account Control dialog box.

Please check if the computer has either TrueCrypt 7.1[a]/VeraCrypt 1.21 and later (for TrueCrypt, installing a special driver package also counts).

If user has TrueCrypt/VeraCrypt, it would not trigger UAC prompts.
If user has the TrueCrypt driver package, it would not trigger UAC prompts only when User Account Control is disabled.

It is suggested that UAC be enabled for better security, but in case you need to disable UAC, run Command Prompt as administrator, then enter the below command and restart PC:

reg add HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System /f /v EnableLUA /t REG_SZ /d 0x0

To re-enable UAC, use this instead and restart PC:

reg add HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System /f /v EnableLUA /t REG_SZ /d 0x1

4. Suppress "Format Disk" Prompt with Diskpart

There could be an annoying imperfection – whenever the drive is connected, Windows will asks whether you want to format it or not.

To work around the issue, use the command line disk partitioning tool "diskpart" to modify the partition type to 0x64. (Please make sure the below procedure is performed on Windows 7 or later only.)

a. Run Command Prompt as Administrator
b. Enter diskpart

c. We will now use what was written down in section 1c. (disk number and partition number)
For this example, enter select disk 1

d. Enter select partition 2

e. Enter set id=64

Finished.

List of Articles on tCrypt2Go/vCrypt2Go

Welcome to support this project by buying a cup of coffee ☕ if this tool is useful to you. 😊 Thanks!

SharePoint for Project Management (2016)—Presentation Video & Slides for Download

noreply@blogger.com (wandersick) — Sat, 22 Apr 2017 10:27:00 +0000

Microsoft SharePoint, the widely leveraged document management and collaboration web application, is an established product with millions of users that has been growing strong since 2001, from on-premises to cloud as part of Office 365. Using SharePoint with Microsoft Office is a seamless experience.

I had the opportunity to present SharePoint publicly four times in 2016. Two of them were for work and two of them for school (one being in English). As posted below, you may find the slides (in PowerPoint format) on SlideShare and partial video of the English presentation on YouTube.

Academic Presentation 1 - Available on SlideShare

Download PPT slides on SlideShare for "SharePoint for Project Management (2016)"

An introduction to SharePoint Online of Office 365 with a focus on project management.

It demonstrates SharePoint basics, creating sites, managing projects using SharePoint and misc. features. It is dedicated to my teacher, Mr. Leung, who encouraged me to present SharePoint to inspire fellow classmates on how to better manage their projects.

Main introduced features

SharePoint Basics
Creating a site (under OneDrive for Business)*
Managing projects

*Note: Sites created this way are traditional Sites instead of the new style of Sites created by Office 365 Groups

Other introduced features

Co-Authoring
Checking Out/In
Version History
Alerting
Syncing with OneDrive for Business
Site Mailbox (Discontinued. Available in Office 365 Groups at the time of writing)
OneNote
Modern UI and Classic UI
Mobile Access
Customizing Site Design
User Management
Sharing a SharePoint Site
Wiki Page
All Site Contents

Academic Presentation 2 - Available on YouTube

A quick overview of misc. features offered by SharePoint. This video is an excerpt from my academic presentation in November 2016. Original audience: English teacher without an IT background and fellow classmates.

Introduced features

Version History
Co-Authoring
Checking Out/In
Alerting
Customizing Site Design
User Management
Sharing a SharePoint Site
Wiki Page
All Site Contents

Captions (for YouTube Video)

We've talked a lot about the high-level details of SharePoint. Let me dig deeper to show you a little more about the underlying workings of SharePoint.

Let's take a look at the SharePoint features one by one — Version History, Co-Authoring, Check Out, Alerting, Mobile Apps, User Management, and more. Let's go through each of them.

For Version History, as our teammate just said, the existing approach of using a file server would cause a burden to the IT team because users cannot restore files on their own. But they can, with this feature of SharePoint—Version History enables recovery of every previous version. That's very cool.

Next, we have Co-Authoring. As you can see from this animation, many users are editing the same document at the same time. You cannot achieve this with a file server. You can only do this on SharePoint.

And, in contrast, we have a feature called "Check Out". This is to prevent others from editing the same document, because sometimes you just want yourself to edit something; you don't want others to disturb you. So, you can try to "check out" your document. When you're done editing the document, you can "check in", so that others can use the document again.

When you "check in", you can input some comments. The comments will be displayed in the last feature called Version History I just talked about. That would be very useful.

Alerting. If you love updates you would love this feature, because you can set updates on any file you want to track, so that you can proactively receive updates whenever there is some change by your teammates. You don't need to go to SharePoint every time when you want to look for an update. That'd be very troublesome and you'll be very nervous. You don't have to do that; you just wait for an email. That's all you need to do when you set an alert on something you want to track.

And, with SharePoint, you don't need to be a great designer. You can just use the built-in templates, and your SharePoint site will look great.

These are the mobile apps for SharePoint. There are a lot. Some from Microsoft; some from third-parties, for Android and iOS.

User Management is very flexible on SharePoint. As you can see, there are a lot of permission levels. We have users and groups where we can assign permissions. However, that would be very troublesome. So, there's a simpler way. You just share your SharePoint site, and when you share, permissions will be granted automatically, so that you don't need to handle all the advanced stuff. Keep it simple.

For "Wiki Page", have you heard of Wikipedia? That's famous, Wikipedia. If you've heard of it, you would love this feature because SharePoint is a "Wikipedia"—every single page—you can edit it using Wiki syntax. Yeah, I know some of you might not like the wiki syntax. That is so hard to learn, so there is a What-You-See-Is-What-You-Get experience on SharePoint. If you use Microsoft Word, you will know how to use this wiki editor. It is the same experience. Very cool.

Ah, so we have an "(All) Site Content" button as well. Sometimes SharePoint has a lot of features and you might get lost. When you are lost, just click this "(All) Site Content" button on the left-hand side and you will find everything right there.

Note: Screenshots are from SharePoint Online of Office 365 (end of 2016)

As hinted in the video, there is an existing file server which would be replaced with SharePoint. A more detailed comparison between SharePoint and file server is available in Excel format here.

I hope the shared information is informative to you! Feel free to leave comments if you have any suggestions or questions.

Customizing Packages in HPE SPP ISO and Offline Firmware Update in ProLiant Servers

noreply@blogger.com (wandersick) — Wed, 31 Aug 2016 13:45:00 +0000

This is a quick guide showing how to include/slipstream custom packages (rpm, exe, etc.) in HP/HPE SPP (Service Pack for ProLiant) ISO images for updating of system software or firmware of components within HPE/HP servers. Next, the process of offline updating will be shown. Finally, a few handy tips and reference URLs will be provided before conclusion.

The Need for SPP Customization

Below are a few of the reasons:

SPP is released several times per year by HPE. Component updates released in between may not be covered and would require a separate updating process had SPP not been customized.
Although SPP supports online firmware updating within a suitable OS, in limited environments where online updating is impossible, offline updating, which often require SPP, might be the only way.
The current SPP ISO is over the size of a single-layer writable DVD disc. However, not all firmware files are required for everyone; it may be desirable to only select desired firmware files to make the ISO smaller and the whole patching process less time-consuming.

Preparation

Below were used for this guide:

Latest HPE SPP ISO (2016.04 as of this moment – Filename: 864794_001_spp-2016.04.0-SPP2016040.2016_0317.20.iso for this example.)
A Windows PC where SPP (HP SUM – Smart Update Manager) is customized. (A supported Linux OS may also be used.)
Packages of firmware component updates acquired from HPE web site.

Walkthrough – SPP Customization

Step 1. For this example, we will store the update packages downloaded from HPE under x:\custom_rpm. Any directory can be chosen.
Note: As we will only demonstrate offline updating, only .rpm packages are actually only used by the process. The relevant .exe packages (for online updating on Windows) are only included as a comparison.

Step 2. Extract content of SPP ISO to a folder without any space in path, e.g. x:\hp_spp and run x:\hp_spp\launch_hpsum.bat. (To do this on a supported Linux OS, execute the .sh version.)

Step 3. On the user interface launched within a browser, click Baseline Library.

Step 4. Click Add Baseline.

Step 5. Click Browse and select a directory path, e.g. X:\custom_rpm. Click Add when done.

Step 6. On the Components tab, confirm the list of detected components are complete (i.e. the same as the content of X:\custom_rpm in Step 1):

X:\custom_rpm\cp029738.exe
- Firmware of Smart Array v4.02
- Online Updating on Windows
X:\custom_rpm\cp030507.exe
- DL360/380 Gen9 UEFI/BIOS v2.22
- Online Updating on Windows
X:\custom_rpm\hp-firmware-smartarray-ea3138d8e8-4.02-2.1.x86_64.rpm
- Firmware of Smart Array v4.02 - Online Updating on Linux or Offline Updating
X:\custom_rpm\hp-firmware-system-p89-2.22_2016_07_18-1.1.i386.rpm
- DL360/380 Gen9 UEFI/BIOS v2.22
- Online Updating on Linux or Offline Updating

In comparison, the firmware and versions of the first two .exe packages are the same as the last two .rpm packages. (Again, as we will demonstrate offline updating by booting the customized SPP ISO via iLO Virtual Media, only the .rpm packages will be used.)

Step 7. Click Create custom.

Step 8. Complete the below information / select the relevant options. Below are specified for this example:

Description: Custom SPP (any name to describe the customization)
Version: 2016-08-30 00 (or leave default)
Output Location: X:\custom_spp (this should be a newly created directory where the customized ISO will be saved)
Make Bootable ISO File: checked
Extracted Source ISO Location: X:\hp_spp (this is where the SPP ISO was extracted)
Select HP SUM Source Location: Current Working Directory (leave default)
HP SUM Path: X:\hp_spp\hp\swpackages (leave default)

Step 9. Scroll down to Step 1 – Baseline Sources, select Additional Package where location is X:\custom_rpm.
The original baseline “Service Pack for Proliant” may also be selected, but it is deselected below as we would like to make the ISO smaller by only including the (additional) packages we need.

Step 10. Scroll down to Step 3 – Review, click Apply Filters. (Select nothing in Step 2 – Filters.)
At the bottom, click Create ISO and save Baseline.

Step 11. The customized SPP ISO will be created at the specified newly created directory (X:\custom_spp in this example).

Offline Firmware Updating with Customized SPP

Documented below is the process of offline firmware updating using the customized SPP ISO, where a HP server was booted to the ISO file mounted via iLO Virtual Media.

An interactive (manual) updating approach is demonstrated below where each step is reviewed before proceeding to the next. For updating of many servers, automatic (unattended) can be selected for effectiveness reasons. (If no key was pressed before timeout after booting system from the disc, the server would be updated in an unattended way and be automatically rebooted at the end.)

In Step 1 Inventory, an inventory is performed to compare the baseline on the SPP ISO with the firmware versions on the server to determine if there is any need to update firmware or system software on the server.

As shown in Step 2 Review below, only a .rpm package was selected although our SPP ISO contained two .rpm and two .exe packages.

It is because the .exe packages are for online updating on Windows, while the other .rpm package we included in earlier steps is not suitable to the server as it already had a firmware version newer than the one offered by that package.

In Step 3 Deployment, the update began.

The update was a success with a reboot required message shown on the right. Click View log to view more details.

The log showed an exit status of 1, where a reboot was needed to activate the new firmware image.

Additional Tips

Tip 1. Mount customized ISO with multiple options

There are many handy options to mount the SPP ISO for updating, such as:

Virtually insert the ISO using iLO Virtual Media (via Remote Console or HTTP server).
Physically insert a burned DVD disc.
Physically insert USB flash disk (with HPE USB Key Utility) for automatic or manual system updating.

Tip 2. Leverage One-Time Boot in iLO to make the system boot only once from the ISO

It is possible to dismount the SPP ISO automatically once HP SUM has finished updating and rebooted the server. Below One-Time Boot option can be configured via iLO (as long as the server is not in POST).

For legacy BIOS system, select CD/DVD as first option.
For UEFI system, select Generic USB Boot as first option.

(Be reminded to virtual-mount the ISO file.)

Tip 3. Select Red Hat OS instead of ESXi if you are downloading firmware for an ESXi OS

To acquire .rpm package for offline updating on a server with ESXi OS, do not select ESXi OS on the HPE web site, as it may only offer .exe (for online updating on Windows) instead of .rpm (for online updating on Linux/offline updating). Therefore, it may be better to select Red Hat as the OS.

HPE web site offers .exe package with VMware vSphere 6.0 selected as the operating system.

If we change the operating system to Red Hat Enterprise Linux 6 Server (x86-64)…

.rpm package will be offered instead.

Tip 4. Ensure a supported browser is set as the default browser in order to customize HP SPP (SUM) successfully

Minimum browsers recommended by HPE:

Chrome 24
Firefox 18
Internet Explorer 9

Tip 5. Intelligent Provisioning cannot be integrated into HP SPP
As of this moment, HPE Intelligent Provisioning (v2.40B) has its own ISO image for offline updating. Acquire it separately from HPE.

References

1. Building and deploying a customized SPP firmware ISO image

https://www.hpe.com/h20195/v2/getpdf.aspx/4AA5-6400ENW.pdf?ver=1.0

2. HP Service Pack for ProLiant Quick Start Guide

http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=c03190790

3. Service Pack for ProLiant – Official Download

http://www8.hp.com/us/en/products/server-software/product-detail.html?oid=5104018

4. HPE Intelligent Provisioning User Guide for HPE ProLiant Gen9 Servers

http://www.hp.com/ctg/Manual/c04419967.pdf

Finally

As someone who does not work in HPE, the information above was uneasy for me to locate. Hopefully this consolidated guide documenting SPP customization, offline updating process and related tips will help you next time you update firmware or system software of HPE servers.
Feel free to leave a comment if you have questions or tips to share regarding HPE server updating.

DIY Your Own Auto-Posting Service across Twitter, Facebook and Blogger

noreply@blogger.com (wandersick) — Sun, 21 Aug 2016 06:37:00 +0000

I enjoy reading quotes, concise and thoughtful ones, as IT workaholics like me are busy and quotes are easy to be consumed; they get me inspired quickly or start off the day with clarity.

To pay my respect and contribute, I wrote a few Windows and Linux scripts (documented below) and established these quote delivery services on Twitter and Facebook in February 2011:

Twitter: twitter.com/english_quotes & twitter.com/chinese_quotes
Facebook: facebook.com/quotes.english & facebook.com/chinese.quotes
Blogger: english-quotes.wandersick.com & chinese-quotes.wandersick.com

Follow @english_quotes Follow @chinese_quotes

English Quotes

This blog post describes the previous (v1, v2) and current (v3) designs of the services as well as sharing the scripts written to implement them. Unlike paid social media automation tools or services that are more user-friendly, this post demonstrates a more advanced technique; it is completely free of charge, but could be more friendly to system admins, IT pros or experienced tech enthusiasts than general users.

Flow – How It Works

Every day, a quote-sending script is run on a schedule using Task Scheduler (Windows) or cron (Linux) to compose a tweet (or a blog post for the earlier version) containing a quote taken from a text file (database) which in turn gets synchronized to multiple social media platforms using various free third-party web services.

Synchronization with Twitter, Facebook, and Blogger

Thanks to free and readily available cloud-based/on-premises social media automation tools, there is no need to code much. The v1 implementation made use of GoogleCL, an open-source cross-platform command-line utility which supported posting to Google services such as Blogger, while the final destinations, Twitter and Facebook, were synchronized with Blogger posts with help from other services such as FeedBurner Socialize and the built-in Facebook synchronization support of Twitter. By doing so, posting once on Blogger automatically posted on Twitter and Facebook, achieving automated synchronization across the three services.

Flow – v1 Implementation

Script (GoogleCL) > Blogger > FeedBurner > Twitter > Facebook sync of Twitter > Facebook

Script Download – v1 Implementation (Windows)

Script Download – v1 Implementation (Linux)

However, things in the cloud change too fast. Some time ago, GoogleCL, the base of the script, stopped working due to a change in the Open Authentication (OAuth) mechanism from Google's side.

#GoogleCL is broken, so @chinese_quotes and @english_quotes are not working. https://t.co/XbTlEtl1zt
— a wandersick (@wandersick) June 7, 2015

After the change, the v2 implementation make use of twt.exe, a Windows-based command-line tweeting utility to tweet first (thankfully a lot of good quotes are within the character limit imposed by Twitter) and then synchronize content with Blogger and Facebook using an IFTTT recipe called “New Tweet to Blogger” and the native Facebook synchronization of Twitter respectively.

Flow – v2 Implementation

Script (twt.exe) > Tweet > IFTTT recipe: “New Tweet to Blogger” > Blogger
Script (twt.exe) > Tweet > Facebook sync of Twitter > Facebook

2018 Update: Since August 2018, the automatic tweet sharing from twitter to facebook stopped working, as announced by Twitter.

"Twitter Announcement: An update to the Facebook Platform Policies ended the ability to automatically post Tweets to your Facebook profile or page."

Therefore, the flow has been slightly changed. Instead of sharing directly from Twitter to Facebook, now it shares from Twitter to Blogger (as before) and from Blogger to Facebook using IFTTT (recipe name: "from blogger to facebook pages").

Script Download – v2 Implementation (Windows)

2019 Update: Due to strengthened TLS requirements imposed by Twitter in 2019, this IFTTT mail-to-tweet recipe along with a PowerShell email client has replaced the command-line Twitter client, twt.exe (lacking support of TLS 1.2), where described in the v2 implementation.

It was due to a strengthened security requirement imposed by Twitter in 2019, that required some web services like this one to reprogram their services to correct errors caused by it 🧐 https://t.co/uSXg8LzkPM
— wandersick (@wandersick) March 12, 2020

Flow – v3 Implementation

The current implementation flow (v3) is as follows:

Two scripts are run to craft an email containing the latest quote
Run IFTTT applet 1: From email to Twitter
Run IFTTT applet 2: From Twitter to Blogger
Run IFTTT applet 3: From Blogger to Facebook pages

Script Download – v3 Implementation (Windows)

The involved scripts, quotes_en.bat and send-mail-en.ps1, can be found on GitHub at https://github.com/wandersick/ws-quotes

Note: From May 30, 2022, Google no longer supports signing in to Google Account using only your username and password. Two-step verification has to be enabled and an app password have to be generated for the PowerShell email script behind IFTTT applet 1 as I use a Google account for SMTP myself.

Conclusion

While there are GoogleCL alternatives which allowed the quote services to work towards this day, GoogleCL was irreplaceable in that it was more feature-rich. For example, it supported tags (turned into hashtags on Twitter and Facebook) which allow more granular categorization of posts, which is unsupported by the other approaches.

If you love quotes like I do, you may now build your own service with quotes you love with the above script and approach. Consider turning on Twitter mobile notification to have the quotes instantly pushed to your devices to (hopefully) get you inspired. Feel free to share your ideas with me by leaving a comment.

Exchange 2013/16 Journal Mailbox Archive-to-PST PowerShell Script Automation with Reporting

noreply@blogger.com (wandersick) — Sun, 03 Jul 2016 15:26:00 +0000

Recent Exchange versions have built-in support of journaling for recording all inbound and outbound email messages for backup or compliance reasons. Overtime, the journal mailbox grows so large and needs to be trimmed or pruned. This script was written to automate the process.

This article documents a PowerShell maintenance script I have written for reporting and automating a monthly archive-to-PST process of the Exchange 2013 journaling mailbox. (Based on reader feedback, it also suits Exchange 2016 Standard.)

Go to Download

Archiving Concept

This script uses the PowerShell cmdlet New-MailboxExportRequest -Mailbox to export Exchange Journaling Mailbox of previous month (e.g. 2016-01-01 to 2016-01-31) as a standard PST file (e.g. archive 2016_01_31.pst) to specified locations (up to two locations) and then uses Search-Mailbox -DeleteContent to delete email messages within the date range if successful. It is designed to be run at the beginning of each month (e.g. 2/Feb/16) using Windows Task Scheduler.

Email Alerting, Reporting and Logging

A log file is created under a specified directory during each execution. An email is sent to specified email addresses with the log file when done. The email message subject indicates COMPLETE/FAILURE.

In addition, a list of email messages returned from search and deleted by Search-Mailbox cmdlet will be available as zip(csv) attachments in another specified mailbox and folder as mentioned in the message body of above email.

Safety Measure to Prevent Accidental Email Deletion

In case mail archiving has failed, script will send an alert mail and exit so that no mail deletion will occur. In technical terms, when status of Get-MailboxExportRequest is not "Completed", script keeps on waiting (looping) until it is complete. If the loop is broken or script receives any status other than "Completed", execution will be terminated and failure will be reported by email.

Assumptions and Requirements

PowerShell 3.0 or above
Execute this script on Exchange server where (preferably):
- System Locale is English (United States)
- Short date format (for en-US) is MM/dd/yyyy
Sufficient disk space for storing PST files in the specified location(s)

Note: This script was designed and tested for email messages of previous month, and has not been tested in other configurations but could still work with some tweaking. (In fact, it does not only work for journal mailbox.)

Getting Started – Script Usage and Instructions

Edit variable parameters as required (See sections 1-4 in the script below.)
Save the script in a local folder e.g. C:\scripts\ws_email_archiving_script.ps1
To export email of last month, the script should be scheduled to run any day within the current month.

For example, to export Jan 2016 email, running the script any day within Feb 2016 would do; the beginning of month is recommended (e.g. 2/Feb/16)

Setting up Windows Task Scheduler

Program/script:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Add arguments:

-PSConsoleFile "C:\Program Files\Microsoft\Exchange Server\V15\Bin\exshell.psc1" -command ". 'C:\Program Files\Microsoft\Exchange Server\V15\Bin\Exchange.ps1'; &'C:\scripts\ws_email_archiving_script.ps1'"

Start in:

C:\Scripts

Alternatively, start the .ps1 script using a .bat script:

"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -PSConsoleFile "C:\Program Files\Microsoft\Exchange Server\V15\Bin\exshell.psc1" -command ". 'C:\Program Files\Microsoft\Exchange Server\V15\Bin\Exchange.ps1'; &'C:\scripts\ws_email_archiving_script.ps1'"

PowerShell Script

Download

⭐ The script can be downloaded at GitHub

Word of Mouth

Below are some of the kind comments left by users of this script. (Thanks!)

"We have some very large journaling mailboxes that we've been racking our brains trying to figure out what to do with to stop the sprawl. We're on Exchange 2016 Standard, so we're limited on our database count. This script has made it really easy for us to dig out emails by month and save them off to PST that we can store out on second tier storage for archive. Thanks for this!"
"I've used your script for a couple of years. It's GREAT and do a GREAT job. Thank you very much!"

Release History

Ver	Date	Update
1.3	20200614	- Add missing bcc parameters in the first two lines of Send-MailMessage - Change default setting of archiveEnable2ndCopy to false as it should not be commonly enabled - Add a tip in the comment section of the script at part 3 to make it easier to configure the parameters
1.2	20171022	- Turn journal mailbox name into a variable - Further ensure New/Get-MailboxExportRequest job name uniqueness by naming it with current date time (in yyyyMMdd_hhmmsstt) - Remove unused variables, scriptDir, startDateShort and endDateShort - Improve script comments
1.1	20160712	First public release
1.0	20160314	First internal release

Welcome to support this project by buying a cup of coffee if this tool is useful to you. 😊 Thanks! If you would love the script to be further developed, you may engage professional services on one of the freelancing platforms.

References

How can I move Exchange items to a PST using PowerShell?
http://serverfault.com/questions/511610/how-can-i-move-exchange-items-to-a-pst-using-powershell
MailboxExportRequest ContentFilter is “Received -ne $null” when querying by date
http://serverfault.com/questions/591798/mailboxexportrequest-contentfilter-is-received-ne-null-when-querying-by-date
Cannot get Exchange PowerShell script to run under Scheduled Tasks
http://serverfault.com/questions/576302/cannot-get-exchange-powershell-script-to-run-under-scheduled-tasks

Resolving 'Root Partition Is Filling Up' Issue on Sophos UTM Firewall

noreply@blogger.com (wandersick) — Sat, 18 Jun 2016 16:26:00 +0000

This is a quick guide on how to resolve an occasional problem that could happen to neglected Sophos UTM firewall with default settings, where updates are automatically downloaded to device, filling up storage after some time.

An email alert similar to the below is sent to administrator when it happens:

From: Firewall Notification System [do-not-reply@fw-notify.net]
Sent: 18 June 2016 11:58 PM
To: Administrator [admin@wandersick.ws]
Subject: [SOPHOS][INFO-150] Root partition is filling up - please check
Root partition is filling up - please check. Current usage: 86%
--
System Uptime      : 362 days 1 hour 2 minutes
System Load        : 0.05
System Version     : Sophos UTM 9.210-20
Please refer to the manual for detailed instructions.

To solve it, one way is to

SSH into Sophos UTM firewall using root login
Remove content under Sophos update directory (Main focus of this blog post)
Configure firmware download interval for System Up2Date packages from Automatic to Manual

1. SSH into Sophos UTM firewall using root with public key authentication

In order for root login to be made possible, it is required to first generate a private key and set up public key authentication. You may refer to any guide found on the Internet such as this one.

After enabling root login, we can then authenticate using the private key file using SCP/SSH clients such as WinSCP, PuTTy by specifying private key file as shown below.

a. Specifying private key file in WinSCP SCP client:

b. Specifying private key file in PuTTy SSH client:

2. Remove content under Sophos update directory

Commands for determining storage utilization on directory /var
cd /var

du -shx *

After root SSH login, executing the above directory size checking commands revealed the /var/up2date folder (containing downloaded firmware updates) took 2GB of space on the device.

Mitigate the situation by removing the update files under /var/up2date/sys. Below screenshot shows a way to do it using WinSCP. (Backing up the files is recommended prior to deletion.)

Command for listing space utilization statistics on each file system mounted on device
df -h

Before removing the files, the disk utilization was 86%:

After removing the files, the disk utilization lowered to 46%:

3. Configure download interval for System Up2Date packages as Manual

To prevent this problem from recurring, firmware download interval can be set to Manual on the firewall. After configuring the setting, firewall updates will not be automatically downloaded, hence device storage won’t be filled up as easily.

Tip: Be sure to do this quickly after moving/deleting the files. Otherwise, the device storage will be filled up again by the update download next time

Hope this post provides a quick way to resolve this problem for those who encounter it.

Update in April 2017: Fix it as early as we can! It was found that if we leave the problem unresolved until the firewall disk partition becomes 100% full, web administration console (httpd) will fail with the error below, and we must recover the firewall via console logon (or using Windows PE/Linux boot disk).

Internal Server Error

The server encounted an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at admin to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.

Troubleshooting firewall 100% disk full – can't believe we can boot up #firewall with Windows PE or Linux boot disc (USB) like a PC #Sophos pic.twitter.com/AhGFOzeMqj
— a wandersick (@wandersick) April 5, 2017

Reference

Root partition filling up…
https://community.sophos.com/products/unified-threat-management/f/52/t/30237

let's find where the problem is, from the command line as root:
cd /
du -shx *
And then go into the largest in the list and do du -shx * again.

Up2Date SSH
https://community.sophos.com/products/unified-threat-management/f/52/t/74866

It looks like you're now on 9.318, so I would set the Download interval to "Manual" and delete everything in /var/up2date/sys

10+ Smart-Working Enterprise IT Work Rules

noreply@blogger.com (wandersick) — Wed, 15 Jun 2016 15:14:00 +0000

Do you wonder what the work principles in IT (information technology) are like in an enterprise? As a young IT professional in a large enterprise, there were rules I had to learn in order not to go out of line — it was 2011, the first time I ever stepped into the world of IT support, where I learned the essentials from my supervisor and mentor who always told me not to work hard but work smart. This is a list of IT work ethic reminders I crafted for myself thanks to him, hoping it would be inspiring to everyone, working in the tech industry or not.

Image courtesy of Justin Baeder

Most of the time, IT support is a standby job — it may be peaceful but we should get prepared for the worst to come anytime.
Underpromise and overdeliver. We should leave buffer time when asked to provide with an estimated time required to fulfill a request. (If done early, users or requesters would be more satisfied.)
We would not be much different from workers in the industrial age if we don’t think and just do as we are told.When assigned a task, we should be curious. We should ask. We should seek every bit of detail. We should then planand state clearly any risks to avoid taking responsibility in case of failure.
When answering the management, do not rush to say yes or boldly suggest when we aren’t sure. Just chill and say we don’t know or we need to look into it first.
To maintain IT operation of an enterprise, sometimes doing nothing is better for users and the company in long term. For example, working around a slow 10-year-old PC to mitigate compatibility issues is bad. Why not use it as a reason to support buying a new PC that makes the user more productive and happy?
Get to know one or two colleagues well in each department. That will help make our work smooth, e.g. conducting a pilot test would encounter less reluctance from users we know better.
Avoid writing long emails which are often ignored or skim-read by users. Make emails concise and in point form so that users easily grasp them and do what we would like them to do.
Email is official (and cold). Never take sending emails as the sole action. Carbon-copying (cc) to lots of people or users’ department head could be invasive. Always casually talk to the user first, then email.
Don’t be too nice to users. In a big enterprise, our role is like the police, while users are like citizen. Rules are rules. We do not teach users how to violate rules; they could, however, figure out certain harmless shortcuts by themselves, as no policeman or policewoman would instruct pedestrians they can cross a car-free road when the pedestrian traffic light is red.
We should not let users feel we are soft in saying no. We should be firm in enforcing policies.
If we really would like users to obey IT rules and respect one another, whether it is email or talk, be firm, concise and to the point.
Being strict first, loose later would be better regarded by users than being loose first and strict later.
Think in the mind of challengers (e.g. auditor, the management). Anything that is emailed, logged, signed can be later challenged. Do not give in to laziness or convenience and let users bypass process, such as not requiring approval for installing high-risk software. We don’t want to cause trouble to our supervisors or managers later.
Because of our role of helping people, users and even the management can be very nice to us, but we should maintain our professionalism, which is what they expect from us. Stay humble, sincere, diligent and be courteous to everyone at all times.
IT professionalism also includes not peeking at users’ emails and files. Do not abuse our privileges (in front of users or behind them) without their consent.

I hope these rules will be of use to anyone new to the world of enterprise IT support and similar disciplines.

Script for Booting to Windows PE using HPE iLO PowerShell Cmdlets

noreply@blogger.com (wandersick) — Thu, 09 Jun 2016 01:29:00 +0000

This blog post documents the HP/HPE iLO PowerShell commands I used for scripting the process of booting to Windows PE for purposes such as using Microsoft SCCM (System Center Configuration Manager) and/or SCO (System Center Orchestrator) runbooks for automated operating system deployment.

Thanks to Kyle Murley's blog post which led me to this.

Prerequisites

HP iLO Cmdlets for Windows PowerShell (x86/x64) - http://www.hp.com/go/powershell (v1.3 as of this moment).
HP iLO Server with Advanced iLO License. (Already required by the first Mount-HPiLOVirtualMedia command) - This post is based on iLO4, but it might also work for iLO3 or others. Kindly let me know if it does for you.
Web server (e.g. Microsoft IIS) for hosting the Windows PE ISO file (as required by ImageURL parameter).
[Optional] Sysinternals Process Monitor for monitoring the download of Windows PE ISO file from the web server.
[Optional] HP Scripting Tools for Windows PowerShell User Guide (Also, I found the built-in command help is very useful, e.g. Get-Help Set-HPiLOVMStatus -full)

Not Covered by This Post

As the focus is on HP iLO Cmdlets, this post won't cover the below:

Embedding drivers into a Windows PE image.
Converting a boot.WIM Windows PE image to .ISO format (without "Press any key to boot from..." message). However, reference URLs are included at the bottom of this post.
Hosting a Windows PE ISO image on a web server for iLO to serve.

The Script (UEFI)

Web Server Recommendations

As the Windows PE ISO is quite large (the x64 version is about 250MB) for transferring over a WAN connection, it is recommended to use a web server which resides on the same site link as the iLO server does.
To increase security, consider use of HTTP over SSL (HTTPS) and/or non-common port.
To avoid complexity during initial testing, IP address is recommended instead of hostname/FQDN.

Additional Information

I noticed our new HP servers come with Generic USB Device as the first boot option by default. However, it is nice to specify it to ensure it is by the above command.
For UEFI, booting to Generic USB Device (Boot0008) is preferred to iLO Virtual USB CDROM (Boot000D) as the latter is not selectable after mounting the ISO with Mount-HPiLOVirtualMedia until a subsequent reboot past POST.
Persistent boot order is preferred to one-time boot order as the PowerShell results are more consistent. See below sections for further explanations on this.

The Script (Legacy BIOS)

There are two variants of the script. One is based on UEFI boot mode, while the other is based on Legacy BIOS. The above script is for UEFI; for Legacy BIOS, simply change lines beginning with UEFI to these

Running the Script

Note: The screenshot below is based on a previous version of the script.

Running the PowerShell commands in one go.

The server would then be rebooted.

When the Windows logo is shown, the ISO image is being transferred. When it has finished the transfer, a spinning circle would be shown under the logo indicating it is loading the Windows PE OS.

We can track the download of Windows PE ISO using netstat and Sysinternals Process Monitor as follows.

Netstat

netstat -n 2 | find /i "HP_iLO_IP_addr"

Process Monitor

Filter set to: "Path", "Contains", "your_winpe_filename.iso"

The Windows PE is successfully booted up.

Tips and Tricks

The PowerShell command Get-HPiLOPersistenBootOrder below reveals the boot code of each UEFI target in one go

Get-HPiLOPersistentBootOrder -Server $iloIP -user $iloUserName -pass $iloPassword | select -ExpandProperty device

Each of them corresponds to the item in the boot order list on iLO web interface, e.g. the first code Boot000D is "iLO Virtual USB 2 : HP iLO Virtual USB CD/DVD ROM" while the third code Boot0008 is "Generic USB Boot".

Alternatively, on the boot menu screen (F11) shown during POST, pressing F1 (Help) on each item also reveals the boot item code.

Boot0008 (Generic USB Boot) is recommended over Boot000D (iLO Virtual USB 2 : HP iLO Virtual USB CD/DVD ROM) as the latter is dynamic. It only appears after specifying mounting of the ISO, rebooting the server and getting past POST. When it does not appear, it would return "ERROR: No UEFI Target boot device with the specified BootXXXX name is available."

Persistent boot order is preferred to one-time boot order as the PowerShell results are more consistent. Setting one-time boot order via PowerShell may generate “There is no such EV.” error while host is powered off; setting it via web interface is OK though. Perhaps you might wonder whether it will boot to the ISO every time the server boots? Fortunately no, because we make use of the BOOT_ONCE option of Set-HPiLOVMStatus command to achieve one-time boot (which has nothing to do with boot order). This option mounts the ISO on next boot only and dismounts it automatically on subsequent boots. The VM in the command stands for Virtual Media which corresponds to the below part of the iLO web interface.

As shown, the "Connected" checkbox is only checked on next boot. It is automatically unchecked after subsequent reboots.

Verify the effect of Mount-HPiLOVirtualMedia and Set-HPiLOVMStatus commands using Get-HPiLOVMStatus

Get-HPiLOVMStatus -Server $iloIP -user $iloUserName -pass $iloPassword -Device CDROM

Where BOOT_OPTION of -VMBootOption parameter of Set-HPiLOVMStatus can be one of the below:

BOOT_ONCE (This should be the case if -VMBootOption is BOOT_ONCE which is option used above in the script.)
BOOT_ALWAYS (This should be the case -VMBootOption is CONNECT)
NO_BOOT (This should be the case if the Set-HPiLOVMStatus command has not been run, or if -VMBootOption is BOOT_ONCE and the server has already rebooted more than once.)

The VM_APPLET parameter above corresponds to "Image File (CD-ROM/DVD)" checkbox under Virtual Drives menu of the iLO Integrated Remote Console. This option is not available in the iLO PowerShell Cmdlets. Only ImageURL (requires a web server) can be specified.

References

1. Using HP iLO Scripting Tools for Windows

https://kylemurley.wordpress.com/2014/03/04/using-hp-ilo-scripting-tools-for-windows-powershell-mt-getscripting-podcast-powerscripting/

2. HPE Scripting Tools for Windows PowerShell User Guide - iLO Cmdlets [PDF]

http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=c03958206

3. HPE iLO 4 Scripting and Command Line Guide [PDF]

http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=c03334058

4. How to create an ISO image for UEFI platforms for a Windows PE CD-ROM for Windows Server 2008

https://support.microsoft.com/en-us/kb/947024

5. Creating an .iso file from a .wim file

http://www.acronis.com/en-us/support/documentation/ATI2016/index.html#26977.html
http://dl2.acronis.com/u/pdf/ATI2016_userguide_en-US.pdf (Updated)

"To create a bootable media by using a .wim file, you need to convert it to an .iso file first.
To create a PE image (.iso file) from the resulting .wim file:
Select Microsoft Windows AIK ® Windows PE Tools Command Prompt from the Start menu.
Users of Windows 7 should select Microsoft Windows AIK ® Deployment Tools Command Prompt from the Start menu.
If you create WinPE 4.0 ISO, select Windows Kits ® Windows ADK ® Deployment and Imaging Tools Environment from the Start menu.

Run the copype.cmd script to create a folder with Windows PE files. For example, from a command prompt, type:
copype amd64 c:\winpe_x64

Replace the default boot.wim file in your Windows PE folder with the newly created .wim file (for example, AcronisMedia.wim). If the AcronisMedia.wim file is located on c:\, then:
For WinPE 3.0, type:
copy c:\AcronisMedia.wim c:\winpe_x64\ISO\sources\boot.wim
For WinPE 4.0 or WinPE 5.0, type:
copy "c:\AcronisMedia.wim" c:\winpe_x64\media\sources\boot.wim

Use the Oscdimg tool. To create an .iso file, type:
oscdimg -n –bc:\winpe_x64\etfsboot.com c:\winpe_x64\ISO c:\winpe_x64\winpe_x64.iso

Alternatively, to make the media bootable on both BIOS and UEFI computers, type:

oscdimg -m -o -u2 -udfver102 -bootdata:2#p0,e,bc:\winpe_x64\etfsboot.com#pEF,e,bc:\winpe_x64\efisys.bin c:\winpe_x64\ISO c:\winpe_x64\winpe_x64.iso

Burn the .iso file to a CD using a third-party tool (for example, Nero) and you will have a bootable Windows PE disc with Acronis True Image 2016."

6. Windows boot iso file without press any key

http://serverfault.com/questions/353826/windows-boot-iso-file-without-press-any-key

"It depends on the BIOS. If it is a traditional PC BIOS then the ISO uses the El Torito boot code (etfsboot.com). In this case deleting \boot\bootfix.bin will work.
If it is an EFI BIOS then the ISO should use EFI boot code (efisys.bin). There is also a efisys_noprompt.bin boot code file that can be used when creating the ISO for EFI systems. Use that file instead if you want to eliminate the prompt."

You may bookmark or share this article using the following:

a wandersick's Tech Notes

Quick Ansible Overview — Remediate Host Configuration Drift (Infrastructure-as-Code)

Infrastructure-as-Code

Ansible

How to Get Ansible to Work

Some Ansible Terminology

Ansible Host Inventory — inventory.ini

Ansible Playbook — rule-book.yml

Ansible Command

Ansible Checking and Remediation Results

Extending Capability with Community Modules

Further from Here

Folder Template Provisioner

Features

Requirements

Script Filenames

Folder Hierarchy

Step by Step - How to Set up the Scripts

Step by Step - How to Provision a New Folder

Screenshots - Provisioning a New Folder with the Script

Screenshots - Error Messages

Security Note

Another Possible Approach for Non-Admin

Download

Release History

Skills to Get Equipped Career-wise amid COVID-19

Problem – Coping with Remote Work, the New Normal

Solution – Remote Work Marketplaces

Benefits – How Remote Work Marketplaces Improve One’s Skills

1. Opportunities to experience unfamiliar territories

2. Engage in tasks that prepare one for a target career

3. Get good with proposal writing, contract bidding

4. Familiarize with a pure remote work environment

Tips – Shine on Remote Work Marketplaces

1. First few contracts are key

2. Craft client-attracting profile and contract-winning proposals

3. Empower your screen-sharing demo on Zoom with AeroZoom

Conclusion – 3 Takeaways

100 Tips for AeroZoom

AeroZoom 5.0 New Features – Beta Testers Needed (UPDATE: Beta Program Closed)

How-to: Create a Silent Installer with AutoHotkey and Publish it on Chocolatey

This how-to will be centered around our code repository on GitHub: https://github.com/wandersick/az-autohotkey-silent-setup

Table of Content

Some Trivia of AeroZoom (before We Begin)

1. Step-by-Step: Building Outer Unattended Installer Using AutoHotkey

About False-Positive Notices from Anti-virus Software

2. Pushing Unattended Installer to Chocolatey

3. Step-by-Step: Building Inner Setup Using AutoHotkey

How It Works

Tip-Packed Gem from Microsoft Ignite 2018 – Windows Life Hacks

ChMac – Windows Command to Change MAC Addresses of Network Adapters

List of Features

How It Works

Syntax

Parameters

MAC Address Format

Examples

Command-Line Mode

Interactive Mode

File Structure

OUI Files

Step-by-Step Screenshots

Command-Line Mode

Interactive Mode

Requirements

Admin Rights

Dependencies

Return Codes / Error Messages

Limitations

Download / Installation Options

*Note for winget users: Please ignore the warning from SmartScreen, where you may safely click ‘more info’ to complete installation.

Word of Mouth

Below is one of the kind comments left by ChMac users. (Thanks!)"Will keep this one handy!"

Release History

Automate Windows Group Member Update via CSV File or LDAP Query with PowerShell

List of Features

Quick Start

Command Example

Prerequisites

CSV File Schema

This how-to will be centered around our code repository on GitHub:

https://github.com/wandersick/az-autohotkey-silent-setup

Below is one of the kind comments left by ChMac users. (Thanks!)
"Will keep this one handy!"