The first step is to gain SSH Access to the vCenter server. The steps are outlined in this Knowledge Base article on troubleshooting. Once you’ve managed to get a shell open on your vCenter server, the following commands can be used to retrieve the root password to gain access to the Supervisor.

Command> shell
Shell access is granted to root

root@sfo-w01-vc01 [ ~ ]# cd /usr/lib/vmware-wcp/
root@sfo-w01-vc01 [ /usr/lib/vmware-wcp ]# ./decryptK8Pwd.py
Read key from file
Connected to PSQL
Cluster: domain-c9:09ddf26c-f7e2-4333-b023-aa4a36024b79
IP: 192.168.1.200
PWD: AbCdEfGhIj
------------------------------------------------------------
root@sfo-w01-vc01 [ /usr/lib/vmware-wcp ]# sshpass -p 'AbCdEfGhIj' ssh -o \
stricthostkeychecking=no root@192.168.1.200
Last login: Fri Sep 18 10:45:31 2026 from 192.168.1.130
===============================================================================
EXERCISE EXTREME CAUTION when accessing a vSphere Supervisor control plane VM.
When running commands on a Supervisor control plane VM, you are acting with
elevated permissions that BYPASS PROTECTIONS. You may cause DATA LOSS, disrupt
running workloads, or PERMANENTLY DAMAGE the Supervisor, Supervisor Services
including the vSphere Kubernetes Service (VKS), or VKS Clusters.
Graceful recovery from changes made with these elevated permissions may not
be possible; it may be necessary to REDEPLOY THE ENTIRE SUPERVISOR, recreate
all VKS clusters, and redeploy all workloads.
===============================================================================
08:09:45 up 12 days, 22:03,  0 users,  load average: 7.72, 6.12, 5.87
tdnf update info not available yet!
root@423789b3f055e8e9961d56e456f323d6 [ ~ ]#

Once successfully logged onto the Supervisor, query the Data Service Manager deployment. In VCFA 9.1.1, the DSM is deploying using the VM Service feature. The follow commands can be used to check the name of the appliance and the opened ports on it. Note that on the Control Plane VM (Supervisor), the kubectl command is aliased to a simple ‘k’.

root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# k get vm -A | grep dsm
svc-dsm-provider-operator-r77bq   dsm-appliance                     PoweredOn     10d


root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# k get vmservice -n svc-dsm-provider-operator-r77bq
NAME                          TYPE           AGE
dsm-appliance-internal        ClusterIP      10d
dsm-appliance-lb              LoadBalancer   10d


root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# k get vmservice -n svc-dsm-provider-operator-r77bq \
dsm-appliance-lb -o yaml
apiVersion: vmoperator.vmware.com/v1alpha5
kind: VirtualMachineService
metadata:
  creationTimestamp: "2026-09-10T13:25:35Z"
  finalizers:
  - vmoperator.vmware.com/virtualmachineservice
  generation: 1
  name: dsm-appliance-lb
  namespace: svc-dsm-provider-operator-r77bq
  ownerReferences:
  - apiVersion: provider.dataservices.vmware.com/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: DSMAppliance
    name: dsm-appliance
    uid: 14347c54-628d-4a35-ad8d-802d02ea23bc
  resourceVersion: "2065849"
  uid: a25607df-b720-4e58-bb86-cee6525ea006
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    dsm.vmware.com/appliance-name: dsm-appliance
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 192.168.41.72

By default, the DSM appliance is only servicing the https port, 443. This allows administrator to connect to the DSM UI via a browser using the loadBalancer IP address, but no other ports are accessible via that address. To allow SSH access (port 22), create a new YAML manifest similar to the above for an additional VM Service on the DSM Appliance. A sample manifest will look something similar to the following:

apiVersion: vmoperator.vmware.com/v1alpha5
kind: VirtualMachineService
metadata:
  name: dsm-appliance-lb-additional
  namespace: svc-dsm-provider-operator-r77bq
spec:
  ports:
  - name: ssh
    port: 22
    protocol: TCP
    targetPort: 22
  selector:
    dsm.vmware.com/appliance-name: dsm-appliance
  type: LoadBalancer

Apply the manifest. This creates a new VM Service. Note that the SSH port 22 is opened on a different IP address compared to the IP address associated with the https port above. However both redirect to the same DSM appliance:

root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# k apply -f dsm-ssh.yaml
virtualmachineservice.vmoperator.vmware.com/dsm-appliance-lb-additional created


root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# k get vmservice -n svc-dsm-provider-operator-r77bq
NAME                          TYPE            AGE
dsm-appliance-internal        ClusterIP       10d
dsm-appliance-lb              LoadBalancer    10d
dsm-appliance-lb-additional   LoadBalancer    14s


root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# k get vmservice -n svc-dsm-provider-operator-r77bq dsm-appliance-lb-additional -o yaml
apiVersion: vmoperator.vmware.com/v1alpha5
kind: VirtualMachineService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"vmoperator.vmware.com/v1alpha5","kind":"VirtualMachineService","metadata":{"annotations":{},"name":"dsm-appliance-lb-additional","namespace":"svc-dsm-provider-operator-r77bq"},"spec":{"ports":[{"name":"ssh","port":22,"protocol":"TCP","targetPort":22}],"selector":{"dsm.vmware.com/appliance-name":"dsm-appliance"},"type":"LoadBalancer"}}
  creationTimestamp: "2026-09-21T08:27:34Z"
  finalizers:
  - vmoperator.vmware.com/virtualmachineservice
  generation: 1
  name: dsm-appliance-lb-additional
  namespace: svc-dsm-provider-operator-r77bq
  resourceVersion: "16164207"
  uid: ae4a417c-bedc-4d08-bd6b-b11264b47751
spec:
  ports:
  - name: ssh
    port: 22
    protocol: TCP
    targetPort: 22
  selector:
    dsm.vmware.com/appliance-name: dsm-appliance
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 192.168.41.73

At this point, SSH is opened to the DSM appliance via a new loadBalancer IP address. This means that we should be able to SSH to it. However, in order to be able to login as the root user, the root password must also be retrieved. This is help in a secret called dsm-appliance-root-cred in the same namespace. Here is just one example of how to retrieve it.

root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# k get secrets -n svc-dsm-provider-operator-r77bq
NAME                                 TYPE                             DATA   AGE
dsm-appliance-root-cred              Opaque                           1      10d
dsm-po-values                        Opaque                           1      10d
dsm-provider-operator-pull-regcred   kubernetes.io/dockerconfigjson   1      10d


root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# k get secrets -n svc-dsm-provider-operator-r77bq \
dsm-appliance-root-cred --template='{{.data.root_password}}' | base64 -d; echo
9L0b^9Lu%V@8mXl
root@423789b3f055e8e9961d56e456f323d6 [ ~ ]#

Now we have SSH opened, we have the IP address to reach the DSM appliance and we have the root password. Let’s see if we can successfully access it.

root@423789b3f055e8e9961d56e456f323d6 [ ~ ]# ssh root@192.168.41.73
Welcome to Photon 5.0 (\m) - Kernel \r (\l)
(root@192.168.41.73) Password: 9L0b^9Lu%V@8mXl

root@photon-6ca473c9efbb [ ~ ]# docker ps
CONTAINER ID   IMAGE                                                  COMMAND                  CREATED       STATUS       PORTS                      NAMES
dc797efaaa57   dsm-tsql-provisioner:9.1.1.0.25641961                  "/provisioner --conf…"   5 days ago    Up 5 days                               dsm-tsql-provisioner-service
77121de7aacd   vc-session-manager:9.1.1.0.25641961                    "/vc-session-manager…"   10 days ago   Up 10 days                              vc-session-manager-service
e155538f05d9   telegraf:v1.38.4_vmware.1                              "/bin/sh -c 'telegra…"   10 days ago   Up 10 days                              dsm-telegraf-service
c50cdede22b4   provider-update-service:9.1.1.0.25641960               "/bin/sh -c '/usr/ja…"   10 days ago   Up 10 days                              provider-update-service
8ecc8312b858   provider-monitoring-service:9.1.1.0.25641960           "/bin/sh -c '/usr/ja…"   10 days ago   Up 10 days                              monitoring-service
2ddc5884de06   registry:3                                             "/entrypoint.sh /etc…"   10 days ago   Up 10 days                              docker-registry
1297b75df54e   simplified-gateway:9.1.1.0.25641961                    "/gateway --v=5 --ds…"   10 days ago   Up 10 days   127.0.0.1:5443->6443/tcp   sgw-service
554efa07f087   dsm/cluster-api-controller:1.13.2_vmware.2             "/manager --leader-e…"   10 days ago   Up 10 days                              kubernetes-service-capi-1
aee34cb51ecf   dsm/kubeadm-bootstrap-controller:1.13.2_vmware.2       "/manager --leader-e…"   10 days ago   Up 10 days                              kubernetes-service-kadm-1
234b8a29a113   dsm/cluster-api-ipam-provider-in-cluster:1.1.0         "/manager --leader-e…"   10 days ago   Up 10 days                              kubernetes-service-caip-1
3d0d49c989fe   dsm/kubeadm-control-plane-controller:1.13.2_vmware.2   "/manager --leader-e…"   10 days ago   Up 10 days                              kubernetes-service-kcp-1
1718cea3e5e0   dsm/cluster-api-vsphere-controller:1.16.1_vmware.2     "/manager --leader-e…"   10 days ago   Up 10 days                              kubernetes-service-capv-1
cf767b7526af   kubernetes-service:9.1.1.0.25641963                    "/opt/moneta/kuberne…"   10 days ago   Up 10 days   0.0.0.0:6443->6443/tcp     kubernetes-service
a2b5b3972550   provider-service:9.1.1.0.25641960                      "/usr/bin/bash -c 'v…"   10 days ago   Up 10 days                              provider-service
3f4d4399ee9d   bitnami-influxdb:1.12.4-photon-5                       "/bin/sh -c 'influxd…"   10 days ago   Up 10 days                              influxdb-service
85df57a66487   provider-ui:9.1.1.0.25641967                           "/bin/sh -c '/usr/sb…"   10 days ago   Up 10 days   0.0.0.0:443->8443/tcp      provider-ui


root@photon-6ca473c9efbb [ ~ ]# kg get postgresclusters -A
NAMESPACE                   NAME                 STATUS   STORAGE   VERSION                AGE
7b6916-tenant-03-ns-g8krt   tenant-03-pg-db-01   Ready    20Gi      18.4+vmware.v9.1.1.0   10d

Everything looks to be working as expected. Remember that once you are finished with your troubleshooting on the DSM appliance, remove the VM Service that was created earlier. This will ensure that SSH access to the DSM appliance is once again removed. Retain the YAML manifest though, as you can re-apply it when you need to open SSH access once again.

The post How to enable SSH on DSM Appliance when deployed via VCFA 9.1.1 appeared first on CormacHogan.com.

Let’s use an LDAP user. In this case, my LDAP user is called tenant01. At the organisation level, this user is an org user with object store privileges. At the project level, this user has been given the role of Project Administrator in two projects in my organisation. The projects are called default-project and shared-project. This user also has a bucket in each project, built on the same object storage, proj-admin-bucket-01-def-proj (default-project) and proj-admin-bucket-02-shared-proj (shared-project). I am going to use the aws cli to demonstrate how it is not possible to copy from one bucket to another bucket, and then configure Native Object Storage bucket policies to show how it can be achieved. I will show you where to retrieve the account id shortly. I will also show you how to create the profiles for the user so that they can be used with the aws cli.

Initial setup

If using the aws cli, some configuration and credential files need to be created first. I am using an Ubuntu Linux jump-box, and have install the aws cli found here. I created a .aws/config and a .aws/credentials for my LDAP user tenant01. Note the relationship between the profile name used in the .aws/config and the entries in the credentials file. It is important to make these match. In the config file, the ca_bundle is optional. You can use –no-verify-ssl on the command line if you prefer. The CA file is the same CA that was used to sign the certificate for the object storage wildcard certificate when it was created in VCFA. The endpoint_url points to my object store. In the .aws/credentials, the key and secret can be obtain from the Native Object Storage UI in VCF Automation.

.aws/config

[profile tenant01-default-project]
region = cork-region
ca_bundle = /home/cormac/s3/ca-cert.pem
ignore_configure_endpoints_urls = true
endpoint_url = https://storage-tenant-01-obj-store-01.rainpole.io/

[profile tenant01-shared-project]
region = cork-region
ca_bundle = /home/cormac/s3/ca-cert.pem
ignore_configure_endpoints_urls = true
endpoint_url = https://storage-tenant-01-obj-store-01.rainpole.io/

.aws/credentials

[tenant01-default-project]
aws_access_key_id = TWM9F73VC7HDK9VUJ0ND
aws_secret_access_key = IXE9vVPHzzaqGXMvh2yOPLDBDsc33heHBPfwjYdy

[tenant01-shared-project]
aws_access_key_id = 5XYCB3GMJIPPBYEI3KDB
aws_secret_access_key = hWzoJV5GaQ3W1ClWJQctsjCnYcTSc8hB68iX2CVN

Test bucket access

We can now try to simple commands to list the contents of the buckets. We can start with the tenant01 LDAP user added to the default-project, and see if this user can use their default-project credentials to list the contents of their bucket in the default-project and then in the shared-project.

$ aws s3 ls s3://proj-admin-bucket-01-def-proj --summarize --human-readable --recursive \
--profile tenant01-default-project
2026-09-14 09:47:53 132 Bytes credentials_admin_20260910_151358.csv
2026-09-14 09:47:53 132 Bytes credentials_admin_20260910_152130.csv

Total Objects: 2
Total Size: 264 Bytes

$ aws s3 ls s3://proj-admin-bucket-02-shared-proj --summarize --human-readable --recursive \
--profile tenant01-default-project

aws: [ERROR]: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access denied

As expected, they can only list bucket contents in the. default-project as that is the project that the credentials are associated with. Let’s try this the other way around, and check if the LDAP user tenant01, who is also a project admin in the shared-project, can access the bucket contents in the default-project.

$ aws s3 ls s3://proj-admin-bucket-02-shared-proj --summarize --human-readable --recursive \
--profile tenant01-shared-project
2026-09-14 14:42:21 26.4 KiB supervisor-service-contour-legacy-v1.33.1_vmware.2-25276585.yml

Total Objects: 1
Total Size: 26.4 KiB

$ aws s3 ls s3://proj-admin-bucket-01-def-proj --summarize --human-readable --recursive \
--profile tenant01-shared-project

aws: [ERROR]: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access denied

So as expected, the same issue occurs. The tenant01 user who has credentials on the default-project cannot list objects in the shared-project since privileges on object storage are scoped at the project level. And vice-versa. The tenant01 user who has credentials on the shared-project cannot list objects in the default-project. Let’s see how we can build a policy that will allow the default-project user tenant01 access the proj-admin-bucket-02-shared-project bucket created in shared-project.

Build a bucket policy

To allow users from a remote project to access a bucket in a local bucket, a bucket policy must be created to allow it. The first thing you will need to retrieve before building a bucket policy is the account id of the project. We captured this in the table earlier. You can get this from the VCFA UI, as it is shown next to the Project name under the Native Object Storage view.

The default-project id is 38925014951642977:

The shared-project id is 67271507534621802:

You can use the aws cli to check that there is no existing bucket policy in place, making sure you use the correct credentials to do the query.

$ aws s3api get-bucket-policy –bucket proj-admin-bucket-02-shared-proj \
–profile tenant01-shared-project
aws: [ERROR]: An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist

We now want to set a bucket policy that allows users from the default-project ( account id beginning 389 ) to access a bucket (proj-admin-bucket-02-shared-project)in the shared-project. We can do this via the API /CLI or via Object Storage UI in VCF Automation. We want users from the remote project to be able to access the bucket read-only, so we are only allowing actions to list the bucket, get objects from the bucket and get the tags on these objects. We are not letting the remote users write anything to the bucket. The bucket policy, in JSON format, will look something like this:

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "AllowListBucket",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::38925014951642977:root"
        },
        "Action": "s3:ListBucket",
        "Resource": "arn:aws:s3:::proj-admin-bucket-02-shared-proj"
    }, {
        "Sid": "AllowGetObjects",
        "Effect": "Allow",
        "Principal": {
            "AWS": "38925014951642977"
        },
        "Action": ["s3:GetObject"],
        "Resource": "arn:aws:s3:::proj-admin-bucket-02-shared-proj/*"
    }, {
        "Sid": "AllowGetTags",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::38925014951642977:root"
        },
        "Action": ["s3:GetObjectTagging"],
        "Resource": "arn:aws:s3:::proj-admin-bucket-02-shared-proj"
    }]
}

Most of it is probably easy to understand, but the Principal might need a bit more explaining. Let’s break down the ARN in more detail:

• arn:aws: Amazon Resource Name (usually in the public AWS cloud).

• iam: The AWS service involved. In this case, Identity and Access Management.

• 38925014951642977: The Account ID. (Note: Standard AWS account IDs are 12 digits long but ours are 17 digits).

• root This does not literally mean the root user. Instead, in a policy context, it acts as a placeholder for the entire account. It basically delegates permission management to the administrator of that target account. It means: “I trust this account. The administrator of that account can now use their own IAM policies to grant their specific IAM users / roles to my S3 bucket.”

To create this policy via the API, run the following aws cli commands, where –policy file:// points to the JSON file above:

$ aws s3api put-bucket-policy --bucket proj-admin-bucket-02-shared-proj \
--policy file://enable-cross-bucket-iam-from-acc-38925014951642977-tenant01.json \
--profile tenant01-shared-project

If no error is returned, the policy has successfully taken effect. If you want to do this via the VCFA UI, login to the Organization where the Object Store has been created. Go to Build & Deploy > Object Storage. Select the project where you wish to apply the bucket policy, e.g., shared-project.  Click on the bucket to which the policy is to be applied. In this case, it is tenant-admin-bucket-02-shared-proj. Select Policy, and the Click Here to apply a bucket policy. Click JSON, and paste the bucket policy content created earlier.

Apply the bucket policy via the UI by clicking on Save. After saving, click Expand All to see the policy details. Note that this view will be available either way – if you applied the policy via the API or UI, it will appear here:

Now use the aws cli once more to see if it the bucket policy has taken effect.

$ aws s3api get-bucket-policy --bucket proj-admin-bucket-02-shared-proj \
--profile tenant01-shared-project 
{
"Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"AllowListBucket\",
\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::38925014951642977:root\"},
\"Action\":\"s3:ListBucket\",\"Resource\":\"arn:aws:s3:::proj-admin-bucket-02-shared-proj\"},
{\"Sid\":\"AllowGetObjects\",\"Effect\":\"Allow\","Principal\":{\"AWS\":\"arn:aws:iam::38925014951642977:root\"},
\"Action\":[\"s3:GetObject\"],\"Resource\":\"arn:aws:s3:::proj-admin-bucket-02-shared-proj/*\"},
{\"Sid\":\"AllowGetObjectTags\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::38925014951642977:root\"},
\"Action\":[\"s3:GetObjectTagging\"],\"Resource\":\"arn:aws:s3:::proj-admin-bucket-02-shared-proj/*\"}]}"
}

This looks good. Now see if the LDAP user tenant01, i.e., the project admin user from the default-project, can list the contents of the bucket that has been created on the same object storage but in a different project.

$ aws s3 ls s3://proj-admin-bucket-02-shared-proj --summarize \
--human-readable --recursive --profile tenant01-default-project
2026-09-14 14:42:21 26.4 KiB supervisor-service-contour-legacy-v1.33.1_vmware.2-25276585.yml

Total Objects: 1
Total Size: 26.4 KiB

Success! The user from the default-project can list the bucket contents of the shared-project bucket which resides in a different project, something this user could not do before the bucket policy was applied. Let’s now try to copy the contents of the bucket in the shared-project to a bucket in the default-project.

$ aws s3 cp s3://proj-admin-bucket-02-shared-proj s3://proj-admin-bucket-01-def-proj \
--recursive --profile tenant01-default-project
copy: s3://proj-admin-bucket-02-shared-proj/supervisor-service-contour-legacy-v1.33.1_vmware.2-25276585.yml to s3://proj-admin-bucket-01-def-proj/supervisor-service-contour-legacy-v1.33.1_vmware.2-25276585.yml

Let’s check if the file made it over to the default-project bucket:

$ aws s3 ls s3://proj-admin-bucket-01-def-proj --summarize --human-readable \
--recursive --profile tenant01-default-project
2026-09-14 09:47:53 132 Bytes credentials_admin_20260910_151358.csv
2026-09-14 09:47:53 132 Bytes credentials_admin_20260910_152130.csv
2026-09-16 09:06:38 26.4 KiB supervisor-service-contour-legacy-v1.33.1_vmware.2-25276585.yml

Total Objects: 3
Total Size: 26.7 KiB

Once again, success! If you have some issues, you can run some commands to check that access to the local bucket is working as expected. One useful command is checking if it is possible to upload a simple local file to the destination bucket from the jump-box using your locla project credentials. This will verify that the user has credentials to write to the local bucket.

$ aws s3 ls s3://proj-admin-bucket-01-def-proj --profile tenant01-default-project 
2026-09-14 09:47:53 132 Bytes credentials_admin_20260910_151358.csv
2026-09-14 09:47:53 132 Bytes credentials_admin_20260910_152130.csv
2026-09-16 09:06:38 26.4 KiB supervisor-service-contour-legacy-v1.33.1_vmware.2-25276585.yml

Total Objects: 3
Total Size: 26.7 KiB


$ echo "test" > test.txt

$ aws s3 cp test.txt s3://proj-admin-bucket-01-def-proj/ --profile tenant01-default-project
upload: ./test.txt to s3://proj-admin-bucket-01-def-proj/test3.txt

$ aws s3 ls s3://proj-admin-bucket-01-def-proj --profile tenant01-default-project
2026-09-14 09:47:53 132 Bytes credentials_admin_20260910_151358.csv
2026-09-14 09:47:53 132 Bytes credentials_admin_20260910_152130.csv
2026-09-16 09:06:38 26.4 KiB supervisor-service-contour-legacy-v1.33.1_vmware.2-25276585.yml
2026-09-16 09:11:41 5 Bytes test.txt

Total Objects: 4
Total Size: 26.7 KiB

Summary

Although still a tech preview in VMware Cloud Foundation version 9.1.1, Native Object Store has many production-ready features, especially around multi-tenancy. Whilst Object Storage is enabled at the scope of a Region, meaning that it is available to all organizations in a given region, the object stores themselves are organisation specific. Within an organisation, projects share the same underlying object store, but the access controls are set at the project level. As we have seen, an LDAP user who has been added to two projects may be allowed or denied access to the object store on a per project basis. Project Administrators decide which users are allowed to create, delete and access buckets, and the actions they can carry out on a bucket through the access control settings. Cross-Project bucket access can be also configured by a Project Admin through bucket policies, as seen above, to allow or deny remote access to buckets. Lots of nice granular controls in Native Object Storage.

The post Bucket Policies in VCFA 9.1.1 for Native Object Storage appeared first on CormacHogan.com.

1. As the Provider Admin, configure LDAP as an Identity Provider
2. As an Provider Admin, add the ‘Provider’ configured LDAP to an Organization
3. As an Org Admin, add LDAP user as an organization user
4. As an Org Admin, assign user a Project Admin role
5. As a Project Admin, observe Object Storage behaviour
6. As an Org Admin, create a new organization role to include additional object store privileges
7. As an Org Admin, change role of organization user (Project Admin)
8. As a Project Admin, login to organization and observe change in Object Storage behaviour
9. As an Org Admin, add a new LDAP user as organization user with additional object store privileges
10. As an Org Admin, assign Project Advanced User role to new LDAP user
11. As a Project Admin, assign Object Storage Access Controls to LDAP user
12. As an Project Advanced User, login to organization and observe Object Storage behaviour

At the end of this post, we should have a pretty good idea about how Native Object Storage access controls work for the different VCFA personas.

Step 1: Configure LDAP/Active Directory  on Provider

Begin by configuring LDAP as an Identity Provider for VCFA as the Provider Admin. You may get a ‘trust certificate’ popup similar to what I see here during testing. Once configured, it is worthwhile doing a quick test on some of the users to ensure you can query your LDAP Server / Active Directory successfully.

To do a user test, click on the Test icon, provide the password of the user used to configure LDAP in the first place, and then add the LDAP/AD user that you want to test. This will return a list of attribute and whether or not the are configured. If you are happy that LDAP is working, we can proceed to the Organization where Object Storage is configured. Below is an example of a Test output, displaying which user attributes are configured and which are not.

Step 2: Configure LDAP on Organization

Login to your Organization as the Provider Admin. Select the Administer tab, and then select Identity Providers. Next, select LDAP and then the Configure button, as shown below.

In the LDAP source selection list, choose VCF Automation system LDAP service, which is what was configured in step 1. Add a distinguished name for the Org. Save the settings.

Step 3: Import LDAP user as Organization User

Let’s import an LDAP user into our organization. You can do this as a Provider Admin, or you can log out and log in as the Org Admin. Return to the Administer section of the Organization but now select Access Control from the left hand menu list. At present there is only a single user, the tenant-03-admin. This is the first user that was added during the creation of the organization. Click on the Import Users button to add a new users from LDAP.

The Org Admin adds LDAP user ‘sadaf’. The Org Admin is also going to assign a role of “Organization User”. Click Import.

Step 4: Add LDAP user to Project as Project Admin

Now navigate to the Manage & Govern tab in the Organization. Select Projects, then click on the Project (e.g., default-project) that you wish to assign users to.

Click on Users, then Add Users. In the Users list, type in the LDAP User that was previously added to the organization, i.e., “sadaf”. Assign a Project Role. In this case, “sadaf” is assigned the role of Project Administrator. Click Add, and then make sure you click on the Save button located at the bottom of the window. I have missed this on a number of occasions, so be sure to do that step.

Check the users on the project and ensure that user ‘sadaf’ is listed as a Project Admin.

Looks good. Let’s proceed to trying to manage Native Object Storage as a Project Admin.

Step 5: Check Project Admin access to Native Object Storage

Log out of the organization as the provider admin / organization admin, and log back into this organization as ‘sadaf’, the Project Admin. From the Build & Deploy tab, select Object Storage. You should now observe something identical to this “Unable to Load Object Stores” message:

The reason for this is because, in tech preview in version 9.1.1, the “Organization User” role does not have sufficient privileges to either View or Manage object storage. We are going to have to create a new organization role with additional privileges to access the object storage. Let’s do that next.

Step 6: Create New Org Role with Object Storage Privileges

Log out of the tenant organization as the Project Admin and log back in as the Org Admin. Now, let’s take a look at the privileges associated with the Organization User. In the Organization, navigate to the Administer tab (which is only visible to Org Admins), and select Access Control once more from the left hand menu. Select Roles, then click on the Organization User to see the privileges/rights associated with that role. As you can see below there are no Object Store or vSAN Data Services privileges selected. Those are the ones needed for Object Storage.

We can now go ahead and clone the organization user role to a new role where we can edit the privileges. In the role listing, click on the 3 dots in front of the Organization User roles and select Clone.

Edit the new role, giving it a recognisable name, but more importantly, ensuring that the necessary Object Store and vSAN Data Service privileges are enabled. Note that in this case, I am selecting both management and view permissions. You may not want to do this, and instead only select View permissions for your Project Admins and Users. If you grant this role both sets of permissions, then anyone with this role will also be able to delete Object Stores in the Organisation, something you may not want to provide. Once the permsisions are selected, save this new Role.

Step 7: Assign new role to LDAP User

Staying in Access Control, select the Users view. Click the three dots before user ‘sadaf’ and select Edit. In the Assigned Roles section, remove the Organization User Role and add the new role with the object store privileges that you just cloned, in this case “Organization User with Object Store. Save the User settings.

Now log back into the Organization as the Project Admin ‘sadaf’ and see if the behaviour has changed with respect to Object Storage.

Step 8: Observe change in Object Storage behaviour

Now when Project Admin ‘sadaf’ logs into the organisation, navigates to Build & Deploy, and then select Object Storage from the left hand menu, she is immediately offered an access key and secret. This is much better than previous behaviour. Keep in mind that this is because ‘sadaf’ has already been added as a Project Admin to the default-project by the Org Admin, but now also has permissions to access to the object store. If multiple projects existed, and a different project other than default-project was selected, ‘sadaf’ would not have access to the object storage from that project unless the Org Admin adds ‘sadaf’ as a user to the project. ‘sadaf’ has only been given access to the default-project and no other projects. Access to the object store is scoped at the project level in VCF Automation.

After saving the key and secret somewhere safe, and closing the welcome popup, the Project Admin can proceed with other activities, such as bucket creation and assigning additional project users access to the object storage.

A Project Admin has access to Security Credentials, Access Control and Service Accounts in this Project. As a Project Admin, this user ‘sadaf’ can now control access to the Object Storage on behalf of other Project Users. Let’s look at that next.

Step 9: Import a new LDAP User as Project Advanced User

We have already seen how to do this. This will have to be done as the Org Admin (or Provider Admin). It cannot be done by a Project Admin, as the Project Admin does not have access to the Administer tab (as you can see in the previous screenshot in step 8). This time, when importing the user ‘paudie’ into the organization, ensure that the role is the org user + object storage privileges that we created earlier.

Step 10: Add new LDAP User as Project Advanced User

Next, add user “paudie” to the Project (default-project) as a Project Advanced User. This can be done by the Organization admin or as the Project admin.

Let’s login to the organisation and see if our newly added Project Advanced User “paudie” has access to the object store. The answer is no. They currently only have access to the Project, but not to Object Storage within the Project. Note that you are redirected to the Project Admin to resolve this access issue.

Step 11: Assign Object Storage Access Controls

Log out as Project Advanced User ‘paudie’, and login as Project Admin ‘sadaf’. Once logged in as a Project Admin, check the Access Controls associated with the Object Store. All policies and their users are displayed. As we can see, only the Org Admin (the first Org user created with the organisation) and user ”sadaf’ (Project Admin) have policies designed to give access to the object store in this project. Project Advanced User ‘paudie’ does not have any access to the object store.

Project Admin ‘sadaf’ can now go ahead and create some access rules for Project Advanced User ‘paudie’. We are not going to deal with granular rules here, but of course there are a range of actions that can be controlled, where we can grant or deny access to particular actions and to particular buckets. To keep things, simple, we are going to give the user ‘paudie’ access to all actions on all buckets on this object store. Give the policy a name, select the user to which it applies, and then setup your statement to allow or deny actions on buckets. Then click Create. Below is an example of an access control policy where ‘paudie’ is given access to all actions on all buckets on the object store in this project.

The policy will now be visible and can be viewed under Access Control. Note that this is only on the default-project only. This access control policy does not apply to other projects consuming the same object store.

Step 12: Observe change in Object Storage behaviour

Let’s log in to the organization once again as user ‘paudie’ and see if access to the Object Storage has changed now that this user has been granted access control via a new policy.

Success. Project Advanced User ‘paudie’ can now access the object store, has access to buckets and can manage his own Security Credentials, i.e., create additional credentials. However, since this user is not a Project Admin, ‘paudie’ does not have the ability to manage or change the Access Controls, nor does he have access to Service Accounts. These controls are only available to the Project Admin. Thus, user ‘paudie’ is not able to onboard any more project users to enable them to access Object Storage. This has to be done by ‘sadaf’ who has Project Admin privileges (or by the Organization admin).

Summary

That completes the post. I hope it has gone some way to explain how VCF Automation roles and Project roles are used to implement Access Controls on Native Object Storage in the VCF 9.1.1 Technical Preview. This table should provide a useful recap. Note that many of the Org Admin tasks listed below could also be done by the Provider Admin:

The post Identity Providers and Access Controls in VCFA 9.1.1 for Native Object Storage appeared first on CormacHogan.com.

Other enhancements include setting an AD user as the  SQL Server admin rather than a SQL user, and support for gMSA – Group Managed Service Accounts – to perform group refreshes and group membership of users. The latter feature is interesting as gMSAs improves security. gMSAs automatically generating complex, 128-character passwords for the account and rotate it automatically every 30 days (by default). Using a gMSA in place of a normal AD account also get automatically granted permissions to manage its own Service Principal Names (SPNs). When the SQL Server engine starts, a gMSA can automatically register and unregister its SPN.

gMSA Support

Let’s look at how to set up a gMSA account during the creation of a SQL Server Instance through VCFA in more detail. This first screenshot shows where to create the SQL Server instances in VCF Automation. When you are logged into VCFA as a Provider Admin, navigate to the VCF Services > Data Services Manager in the navigation toolbar on the left hand side. As you can see, the user experience has changed quite a bit from the previous release, so take some time to familiarise yourself with it.

During the creation of a new SQL Server instance, you can now select to have a SQL user or an Active Directory user as the SQL Server administrator. If you choose a Windows Principal (AD user), provide the domain name as well as the user name, as shown below.

In the next section on Active Directory, a gMSA may be selected in place of an AD user account to handle database access. If gMSA is selected, then the AD Account Username must be populated with the gMSA ‘SamAccountName’. In many case, the SamAccountName has a trailing ‘$’ sign, and this must be included. gMSAs automatically have privileges to Write Service Principal Names so this field is automatically set to on as is not configurable. You can still use the account to Write DNS Names, but you must ensure that your privileged used added to the gMSA has this privilege in Active Directory.

If you want to test gMSA, here are the steps that I followed to create such as account in my active directory. These were run from a PowerShell window which I had opened with ‘Run as Administrator’ privileges:

1. Check that a Key Distribution Service (KDS) Root Key exists

Active Directory requires a Key Distribution Service (KDS) Root Key to generate encrypted gMSA passwords. Check to see if one exists. If it does not, you will have to create one.

PS C:\Users\Administrator> Get-KdsRootKey

AttributeOfWrongFormat :
KeyValue               : {74, 163, 251, 83...}
EffectiveTime          : 3/13/2026 11:30:05 PM
CreationTime           : 3/14/2026 9:30:05 AM
IsFormatValid          : True
DomainController       : CN=ARKHAM-DC01,OU=Domain Controllers,DC=arkham,DC=io
ServerConfiguration    : Microsoft.KeyDistributionService.Cmdlets.KdsServerConfiguration
KeyId                  : 6b591c9e-7676-59cf-6d44-bb745f0040a0
VersionNumber          : 1

2. Create a Security Group

PS C:\Users\Administrator> New-ADGroup -Name "CJH-gMSA-Hosts" `
-GroupScope Global -GroupCategory Security

3. Create the gMSA Identity

Create the actual gMSA account in Active Directory and assign the group permission to access its credentials. Note that the DNS Hostname can be anything in the following command:

PS C:\Users\Administrator> New-ADServiceAccount -Name "cjh_sql_gmsa" `
-DNSHostname "cjh_sql_gmsa.arkham.io" `
-PrincipalsAllowedToRetrieveManagedPassword "CJH-gMSA-Hosts"

4. Check that it the group managed service account was created successfully:

PS C:\Users\Administrator> Get-ADServiceAccount -Identity "cjh_sql_gmsa"
DistinguishedName : CN=cjh_sql_gmsa,CN=Managed Service Accounts,DC=arkham,DC=io
Enabled           : True
Name              : cjh_sql_gmsa
ObjectClass       : msDS-GroupManagedServiceAccount
ObjectGUID        : 890010fb-1b49-47d4-b2dc-552dc64e9bc0
SamAccountName    : cjh_sql_gmsa$
SID               : S-1-5-21-388335747-1197595944-2956950382-1608
UserPrincipalName :

PS C:\Users\Administrator> (Get-ADServiceAccount -Identity "cjh_sql_gmsa" `
-Properties "msDS-GroupMSAMembership").'msDS-GroupMSAMembership'.access
ActiveDirectoryRights : GenericAll
InheritanceType       : None
ObjectType            : 00000000-0000-0000-0000-000000000000
InheritedObjectType   : 00000000-0000-0000-0000-000000000000
ObjectFlags           : None
AccessControlType     : Allow
IdentityReference     : ARKHAM\CJH-gMSA-Hosts
IsInherited           : False
InheritanceFlags      : None
PropagationFlags      : None

5. Add your privileged AD user to the list of Group Members

PS C:\Users\Administrator> Add-ADGroupMember-Identity "CJH-gMSA-Hosts" -Members "cormac"

PS C:\Users\Administrator> Get-ADGroupMember-Identity "CJH-gMSA-Hosts"

distinguishedName : CN=cormac,CN=Users,DC=arkham,DC=io
name: cormac
objectClass : user
objectGUID: b5a2bd65-8832-469b-b278-8fa2fa6fcaea
SamAccountName: cormac
SID : S-1-5-21-388335747-1197595944-2956950382-1602

6. Test the gMSA.

This command should return True. If there is a misconfiguration, this should report back the reason why.

PS C:\Users\Administrator> Test-ADServiceAccount -Identity "cjh_sql_gmsa"
True

The account is now ready to be used when provisioning SQL Server instances via DSM Manager.

Encrypted Backup and Transparent Data Encryption

I already mentioned some of these new SQL Server features in my DSM version 9.1.1 announcement post, but some are worth repeating here as I know a number of customers have requested them. We have now validated Backup Encryption and TDE, Transparent Data Encryption, on DSM provisioned SQL Server databases. Both features are available through the API. Encrypted Backups can be activated by adding a PFX file to the SQL Server instance specification. This PFX (Personal Information Exchange) file is password-protected and contains the SSL/TLS public key certificate and its matching private key. It is added to the spec.serverConfig.certificates of the SQL Server instance, and then referenced via spec.serverConfig.backupEncryptionCertificate. All databases created on the SQL Server instance will now have encrypted backups if this is configured. Note that if you wish to restore an encrypted database to a new SQL Server instance, the same certificate used to encrypt the backup must be imported onto the target SQL Server instance.

Transparent Data Encryption also requires adding a certificate to the specification of the SQL Server instance, spec.serverConfig.certificates, as seen previously. Then, on the user database, the db_owner (database owner) can use TSQL commands to create an encryption key. Finally, the db_owner can turn on encryption by performing ALTER DATABASE .. SET ENCRYPTION ON.

Summary

I haven’t discussed some other important improvements such as support for SQL Server 2025 and on-demand backup for SQL Server databases. However, as mentioned at the outset, you will find the full set of enhancements in the release notes linked above. I am sure you will agree that this release contains plenty of SQL Server enhancements for those customers wishing to provide SQL Server Database as a Service (DBaaS) to their on-premises VMware Cloud Foundation customers.

The post SQL Server enhancements in Data Services Manager version 9.1.1 appeared first on CormacHogan.com.

By default, Net work Security is disabled. The VCFA Provider Admin will need to activate it. Navigate to VCF Services > Data Services. In the Data Services Manager menu, select Advanced. Here you will find the button to activate Network Security.

When the Activate button is clicked, a pop-up will appear explaining what Network Security is setting out to do. It is basically going to implement a set of rules so that only the tenant that requested the database is able to access “from that tenant”. It cannot be access from any other tenant, or from an external ip address outside of VCF. It will do this by implementing a set of gateway and distributed firewall rules on the dedicated DSM organization where the database instance is instantiated.

Now, do note that by default, Transit Gateway (TGW) Firewalls are disabled. Therefore either the DSM Org Admin or the Provider Admin will need to open the DSM Org and enable the Transit Gateway Firewall. A shortcut to this point is to simply click on the “Manage Rules” link found in the Advanced View of the Data Services Manager Network Security section show above. If you fail to enable the TGW firewall step, the gateway rules will not be implemented and will be left in an “In Progress” state, as shown here:

To turn on the Transit Gateway Firewall, select the Settings tab, select the region and turn the firewall on.

Once the Gateway Firewall is enabled, the state should change to Success and any existing DSM provisioned databases should have their ruleset added to the firewalls on the dedicated DSM Org. Below are the transit gateway rules which only allow the configured IP addresses (Supervisor Control Plane VMs and DSM Provider) to communicate to the K8s API server of the database, and only allow the SNAT’ed IP address of the tenant ORG to communicate to the database port.

So far, so good. Now let’s assume that someone outside of that tenant’s org wishes to access the database. Maybe it is an app running somewhere else on the infrastructure. In that case, all you have to do is to add the External IP Address to the database configuration, as shown here:

And once the database reconciles with the new external client ip address, a new Transit Gateway Rule will be in place to allow the connection to succeed. As shown below, this is added to the list of IP addresses that can access the Postgres database but not the list of IP addresses that are allowed to access the database’s K8s API server.

And to verify that we can now connect to the database from that IP address, run the following:

$ ip -f inet address show ens33
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
altname enp2s1
inet 10.13.10.200/24 brd 10.13.10.255 scope global ens33
valid_lft forever preferred_lft forever

$ psql postgresql://pgadmin:43ux4ZwihNYooae00gUn5404ygLr0R@192.168.21.14:5432/ten2-pgdb
psql (16.15 (Ubuntu 16.15-0ubuntu0.24.04.1), server 16.14 (VMware Postgres 16.14.0))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.

ten2-pgdb=# \c
psql (16.15 (Ubuntu 16.15-0ubuntu0.24.04.1), server 16.14 (VMware Postgres 16.14.0))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
You are now connected to database "ten2-pgdb" as user "pgadmin".
ten2-pgdb=#

Everything looks good. The rules are allowing me to connect to the database from an external IP address.

A note about remote replication. If remote replication is configured on a database and the read replica and the primary instances are in different regions, you must manually authorise the network for connectivity. Edit the Primary database “Extra Clients” field like we did earlier and add the SNAT IP of the organization VPC where the read replica instance is created. Now perform the same action on the read replica database, adding the SNAT IP of the organization VPC where the primary instance was created as an “Extra Client”. This will enable data synchronisation and ready it for both read scale out and disaster recovery scenarios.

The post Data Services Manager version 9.1.1 Network Security appeared first on CormacHogan.com.

In this post, I wanted to share some of the setup and initial configuration steps so that you can see what’s involved. I’ll show you how a Provider Admin enables Object Storage on their VCF Private Cloud. Next, we will see how either the Provider Admin or the Org Admin creates the Object Stores. Finally, I’ll take you to the point where tenant users in a VCF Automation organization can begin to provision their own S3 compatible buckets. In a future post, we’ll look at how a Project Admin can configure access control to the object storage buckets for different project users and assign different levels of privileges to the different project users.

Be aware that by the time the service is generally available, the screenshots shown here may have changed. This post is simply to provide an overview, in the knowledge that the steps may be different in future releases.

Prep vCenter and ESXi

For the tech preview, you will need to enable some advanced options on both ESXi and vCenter to enable Object Storage. You won’t need this step when Native Object Storage is generally available, but for the tech preview, you will need to enable these settings manually.

1. ESXi – set /VSAN/VsanVBOSSEnabled to 1

[root@sfo01-w01-r01-esx01:~] esxcfg-advcfg -g /VSAN/VsanVBOSSEnabled
Value of VsanVBOSSEnabled is 0
[root@sfo01-w01-r01-esx01:~] esxcfg-advcfg -s 1 /VSAN/VsanVBOSSEnabled
Value of VsanVBOSSEnabled is 1

2. ESXi – restart vsanmgmtd

[root@sfo01-w01-r01-esx01:~] /etc/init.d/vsanmgmtd restart
Terminating vsanperfsvc with PID 2561654
vsanperfsvc stopped.
vsanperfsvc started.
[root@sfo01-w01-r01-esx01:~] /etc/init.d/vsanmgmtd status
vsanperfsvc is running
[root@sfo01-w01-r01-esx01:~]

3. vCenter – enable Advanced Setting config.vpxd.vsan.vboss.enabled

4. vCenter – restart vsan-health

root@sfo-w01-vc01 [ ~ ]# vmon-cli -r vsan-health
Completed Restart service request.

Download & Install Native Object Store Package

Tech preview customers will be provided with a run-book to step through how to download the necessary Native Object Storage package. You will need a Linux jump host. Once downloaded, the components need to be pushed up to VCF Automation VMSP Platform. Once that step is completed, tech preview customers can proceed with the installation of the Native Object Storage Service.

Note, however, that you will first need to create a Region object in VCF Automation. Native Object Storage is scoped at a region level, and one or more regions need to be selected when Native Object Storage is installed. The following section has some more specifics on Region settings.

To proceed with the installation, the VCFA Provider Admin used a new Native Object Store tile found under Services Management, as shown below. However, this tile is hidden in VCF 9.1.1 and only those customers participating in the tech preview will be provided with the steps to make it visible. Once the backend components for Object Storage are added to VCF, the VCF Automation Service Management tile for Native Object Storage can be used to do the registration and installation of the service at the Supervisor level.

With Native Object Storage installed, a new supervisor namespace called ‘svc-objectstore-xxx’ is created. In this namespace, the Native Object Storage operator is deployed, watching for requests from different VCFA organisations to build object stores.

The operator has two network services associated with it, compatibility-api and controller manager.

Once the install is complete, we can proceed with getting it ready for the tenant organizations.

VCFA Region Settings

I mentioned previously that Native Object Storage is scoped at the region level. Thus, at least one region must exist before starting the installation. The region is comprised of one or more Supervisors, backed by one or more vSphere clusters. There must be a vSAN cluster on at least one of the vSphere clusters to enable Native Object Storage. The main item to ensure is selected in the region settings is the vSAN Object Store Policy in the Storage Class settings. This is the storage class which is used to create object store volumes on vSAN, both for the metadata and for the data.

Note that there is no capability to choose which organization gets access to object storage in the 9.1.1 tech preview. If an organization is using a region quota from a region which is scoped to use the Native Object Storage, then that organization has the ability to create an object store.

Also note that if your region is a multi-zone, for example, the Supervisor is backed by multiple vSphere/vSAN clusters, the object store will only provision to the cluster placed in the first zone in the list. Tech Preview does not allow the provisioning of object storages across multiple zones.

VCFA Organization Settings

Once we have confirmed that the region does indeed include the vSAN Object Store Policy, we can proceed with the creation of the organizations. The organization region quota includes the vSAN Object Storage Policy.

There will probably be many questions about networking when it comes to Native Object Storage. Whilst different configuration are possible, I am going to keep the Object Store and the tenant on the same VPC. When creating this organization, I am selecting a network configuration whereby a number of default configuration items are automatically implemented, including VPC, Transit Gateway (TGW), SNAT and connectivity profile. I will continue to use this default VPC when creating the Object Store, as we will see later.

The final step in the Organization setup is to create a first user. In my case I am creating a first user who has the privileges of an Organisation Administrator. We will use this user later when we log onto the organisation we created.

When the Organisation is created, we can launch the Organisation portal directly from the Provider / System organization, automatically connecting to it as the Provider Admin. This approach does have some restrictions. In particular, whilst it should be possible for a Provider Admin to create Object Stores on behalf of organisation users, the Provider Admin should not be able to have access to buckets or bucket contents on the object store.

Create Object Store

As the Provider Admin in the tenant Organisation, I can select the Build & Deploy tab, and observe that there is now an Object Storage menu item on the left hand side. Click this should take me to the option of creating an object store, as shown below.

Let’s look at the information that must be provided when building an object store. Obviously, give the object store a name and select which region it is to be placed. The region quote from this region, which includes the vSAN Object Storage class will be used to place the volumes for the object store. You must select a VPC. In this case, it is the default VPC for the organisation but it could be a completely different VPC if you wish. The VPC must exist before you start the object store create process; it is not possible to create a new VPC in this workflow like we have with namespace creation. Set the size of the object store (100Gb minimum), and decide if you want to enable secure HTTPS access or not to the endpoints. If you do want secure access (recommended), you will need to provide a wildcard certificate (e.g., *.rainpole.io) and a private key to enable secure access on the object store endpoints (IAM and storage). Lastly, select a performance class (resources) for the object store.

This will initiate the creation of a new Supervisor namespace for the object store, based on the name of the object store. Initially you should observe the ‘telegraf’ and the ‘config manager’ vSphere pods starting up.

After a few moments, this should expand to 5 vSphere pods, and all should enter a running state if no issues are encountered.

A number of network services and endpoints for the object store should also be created in the namespace:

Checking back in VCFA, we should see the object store come online and healthy. Note that the IAM (Identity and Access Management) and storage endpoints should correlate to the LoadBalancer IP address of the Envoy service in the namespace network services view above. You should ensure that the object storage FQDN resolves to that IP address in your DNS.

As you are still connected to the organization as the Provider Admin, you can try to connect to the Object Store. You may need to accept the certificates depending on how they were created, and if your browser trusts the Certificate Authority used to build them.

But even after doing so, you should observe that your Provider Admin is not allowed to see any details regarding how the organization is using the object store. The Provider Admin cannot see any buckets or bucket contents, nor can they see Access Control info or any service accounts. This is by design. Let’s now try this step as an Org Admin.

Access Object Store as Org Admin

Log out as the Provider Admin, change organizations to the tenant organisation where the object store is created and login as the Org Admin using the first user credentials created when the Org was created.

Now when we connect, we can see the initial credentials pop up after accepting the certificates. This looks better than previous when we tried to use the Provider Admin to access the object store.

And finally we are in a position to create buckets and manage access control and additional security credentials to said buckets. In a future post, I will go into more detail regarding access controls, and what access different roles within a project have when it comes to object storage.

Summary

That completes the post. I hope it has given you an idea of how we have architected Native Object Storage, using vSAN, into VCF version 9.1.1. I will be following up with some additional post to go deeper into some of the aspects here, notably how to control who has access to the buckets from within an organisation. We will also look into some CLI (since Native Object Storage supports the standard S3 API, we can do lots of cool things via tools such as the aws cli, for example). Thanks for reading this far. I am sure you will agree that this is a very interesting feature in VCF 9.1.1.

The post A first look at Native Object Storage in VCF 9.1.1 appeared first on CormacHogan.com.

Unified DSM Install Experience direct from VCF Automation

First and foremost, the installation experience for customers integrating their VMware Data Services Manager with VCF Automation (VCFA) has improved dramatically. Those of you who have been following my previous blogs on how to get started with DSM and VCFA will know that a number of context switches were required between different User Interfaces (UIs) – DSM, vSphere Client and VCFA – to get up and running. In VCF 9.1.1, Data Services Manager is a first-class VCF Service, so the deployment of DSM can be initiated directly from VCFA. This includes the rollout of the DSM Provider Appliance VM, the Consumption Operator on the Supervisor, connecting to a vCenter Server, setting up an initial DSM Admin account and selecting a dedicated DSM Organization in VCFA. Once the dedicated DSM Org is created, any namespaces created within this dedicated DSM organisation automatically become DSM Infrastructure Policies. This will speed up the deployment process considerably, and make it far easier for VCFA Provider Admins to do an initial setup. The DSM UI can also be launched directly from VCF Automation for additional configuration tasks. Here is a look at the Data Services Manager interface in VCF Automation. You can see additional enhancements such as the ability to create SQL Servers and setup Active Directory Domains directly from VCFA. Policies now represents what were previously called Data Service Policies, and control which organizations are allowed to access data services and their related resources.

Higher Database Availability with Multi-Zone Support

A Supervisor can contain multiple vSphere Clusters, each of which can be configured as a supervisor zone. Namespaces can now be constructed to span multiple supervisor zones. Namespaces can also be created in the dedicated DSM Organization to be used as infrastructure policies for the provisioning of databases. If a multi-zone namespace-based infrastructure policy is chosen when provisioning a database, the database will be deployed with nodes placed in different zones. This also means that different database objects (e.g., for Postgres, this implies the primary, replica and monitor pods) are placed on different nodes in different zones. The end result is even higher availability for your DSM provisioned databases should a full zone failure occur. I know some customers have been waiting on this feature, so it is nice to see this fully available for DSM and VCF Automation version 9.1.1.

Support for PostgreSQL version 18

DSM can now provision databases running PostgreSQL version 18. We have also retired the older PostgreSQL version 13. Our customers can now provision 5 different major versions of PostgreSQL, from version 14 all the way through to version 18.

Support for additional PostgreSQL Read Replicas

Although this features has been available in MySQL for some time, DSM version 9.1.1 introduces support to add additional read replicas to PostgreSQL databases. Note some changes in terminology though.  Beginning with version 9.1.1, DSM uses the term read replica instead of secondary to refer to remote PostgreSQL database instances that replicate data from a primary database. Read replicas now serve both disaster recovery and read-scaling use cases. Once Remote Replication is enabled on the PostgreSQL primary database, the same workflow used for DR is now used to enable the creation of additional PostgreSQL read replicas. This workflow is identical to the creation of a secondary database that we saw in prior releases. Once the read replica is created, it can be used by read-only workloads. This is a very useful feature for customers who wish to scale their databases for read intensive workloads.

Support for SQL Server 2025 & gMSAs

We announced support for SQL Server 2022 in DSM release 9.1. In this release, we are extending for SQL Server on Linux to include SQL Server 2025, along with recent Cumulative Updates. For VCF Automation customers, the main thing to note is that the SQL Server instance can now be provisioned from VCFA by the Provider Admin. DSM admins no longer need to do the SQL Server instance deployment via the DSM portal. VCFA Organization tenants can continue to provision SQL Server databases as before, so long as their respective Data Service Policy allows their organization to do so. This release also include support for gMSAs (Group Managed Service Accounts). These are domain accounts which are secure by design. They use strong, rotated passwords from Active Directory. These accounts can also manage their own Service Principal Names (SPNs) and allow encrypted network communication (Kerberos) to happen securely.

Support for SQL Server Native Backup Encryption

A much sought after feature next. We’ve heard from customers that they want to be able to encrypt their SQL Server backups. In 9.1.1, this is now available through the API. This feature can be activated by adding a PFX file to the SQL Server instance specification. This PFX (Personal Information Exchange) file is password-protected and contains the SSL/TLS public key certificate and its matching private key. It is added to the spec.serverConfig.certificates of the SQL Server instance, and then referenced via spec.serverConfig.backupEncryptionCertificate. All databases created on the SQL Server instance will now have encrypted backups if this is configured. Note that if you wish to restore an encrypted database to a new SQL Server instance, the same certificate used to encrypt the backup must be imported onto the target SQL Server instance. A nice security feature for your SQL Server databases.

Support for SQL Server Transparent Data Encryption (TDE)

Similarly, another security enhancement now available for SQL Server 9.1.1 is database-level encryption which is achieved by enabling Transparent Data Encryption (TDE). This once again requires adding a certificate to the specification of the SQL Server instance, spec.serverConfig.certificates, as seen previously. Then, on the user database, the database owner can use TSQL commands to create an encryption key. Finally, the db_owner can turn on encryption by performing ALTER DATABASE .. SET ENCRYPTION ON. Another nice security feature for your SQL Server databases.

Complete Disaster Recovery (DR) workflows in VCFA

This is something we have been working towards over the past several releases, but in version 9.1.1, I am pleased to say that we have full DSM database Disaster Recovery (DR) workflows plumbed in to VCF Automation. This means that administrators can initiate actions such as “Block Connections”, “Promote”, “Demote” and “Unblock Connections” directly from the database context in VCF Automation, and not have to context switch out to the DSM Provider UI to perform these tasks.

Data Services Manager Network Security

As you can probably tell, security has been top of mind in this release. Some of these we have seen already, but this one is particularly interesting and applies to all databases. The DSM team has introduced a new network security feature for databases. Once activated, this feature automatically puts in place the necessary NSX gateway firewall rules and distributed firewall rules to ensure that the database is secured. The only ports and communication channels that remain open are a management path between the DSM Provider appliance and the database and a data path from the tenant’s namespace (the tenant who requested the database to be created) and the database. This feature is located in the Advanced Section of Data Services Manager in the VCF Automation Provider view.

If a client needs to connect to the database from an external IP address (outside of the tenant’s namespace and organization), then simply add it to the “Extra IP address” section of the database configuration. The Network Security features will automatically create the necessary rules to allow this client to connect to the database from the external IP address.

Note that the Advanced view is also where you define the default DSM organization. Any namespaces created in this organization become infrastructure policies for the provisioning of databases.

Locking down SSH Access

Continuing the security theme in DSM 9.1.1, there is no SSH interface to DSM appliance by default when provisioned via VCF Automation. The Provider Admin needs to configure SSH access. This is done by creating a new VirtualMachineService of type Load Balancer for the DSM Appliance. This task is done from the Supervisor using its Kubernetes API. This service must be configured to allow SSH on port 22. The root password is auto-created and stored in a K8s secret. This secret must also be retrieved and decoded. This is part of an ongoing effort to make VMware Data Services Manager more secure.

PostgreSQL Extensions

Historically, DSM has always loaded the pg_stat_statements, pgaudit and pg_cron extensions onto its Postgres databases even if customers were not using them. In DSM version 9.1.1, you can now optionally load these libraries using the PostgreSQL shared_preload_libraries mechanism.

Another update in this space is that the set_user extension is now included in DSM’s PostgreSQL distributions. This extension allows switching users and optional privilege escalation and adds logging when unprivileged users escalate themselves.

Additional Metrics Target

In previous releases, it was possible to set up additional Metrics Targets, either VCF Operations or Prometheus. However, you could only send all metrics to one target, and per-namespace metrics to other target. In this release, it is now possible to send all metrics to multiple targets. The only requirement is that when you configure the metrics targets, they need to have a specific naming convention; the first must be called metrics-default and the other metrics-additional:

root@photon-a526d7c1c992 [ ~ ]# kg get metricstargets -A
NAMESPACE    NAME                 STATUS
dsm-system   metrics-additional   Ready
dsm-system   metrics-default      Ready

Summary

As you can see, this is a very impactful release of VMware Data Services Manager. We have closed the gap on providing consistent feature parity for customers no matter which deployment model of DSM is used (standalone or VCFA). We have also added a bunch of new benefits around security and resilience, and also introduced some newer engine versions, reaffirming that VMware Data Services Manager is the best database as a service (DBaaS) solution for VMware Cloud Foundation.

The post Announcing VMware Data Services Manager 9.1.1 appeared first on CormacHogan.com.

Before looking at the email alerts, it is vitally important to ensure that your SMTP configuration is working correctly. When configuring SMTP in Data Services Manager, an attempt is made to send a test email to the DSM Admin account that is currently logged in to DSM and performing the SMTP configuration. If that DSM login is an LDAP/AD user, DSM attempts to fetch the email address from that LDAP/AD user object using attributes such as mail or rfc822mailbox.If that email address is not configured, the DSM Admin will not receive the test email, and won’t receive future Global Alert emails either. So ensure this is working first. If you have configured it correctly, as in the example shown here:

You should receive a notification similar to the following:

Let’s look at some of those alerts, and their corresponding emails now.

Global Alerts

The full list of Global Alerts can be found in the official Data Services Manager documentation. One Global Alert is root password expiry on the DSM appliance. I reproduced that issue to generate the alert and resulting email to the DSM Admin. In fact any user with the role DSM Admin will get the global alert email.

Here is the alert:

And here is the resulting email sent to the DSM Admin(s):

Let’s now look at the Database Alerts.

Database Alerts

As mentioned, Database Alerts go to all DSM Users who have permissions in the namespace where the database is created. One caveat is that the Database Alerts are not going to DSM Admins. This is a caveat that we will address in a future release. It is currently listed as a known issue in the Release Notes.

With that in mind, let’s look at a typical Database Alert. In this example, I remove access to the backup endpoint (S3 bucket) which meant that the Write Ahead Log (WAL) files for the Postgres database in question could not be written. This causes a Database Alert, Wal file health, as shown here:

And this is the email sent to the DSM Users with Permissions in that Namespace:

Additional Alert Email Recipients

There may be other users who are not part of the DSM Users who might need to receive Database Alerts as well. This can be done manually in the DSM UI by navigating to the database in question, selecting the Monitor tab, selecting Notifications and then Editing the Alert Email Notification section and adding a comma-separated list of email addresses. The email that is sent to DSM Users when an database alert occurs, as shown above, will also be sent to these users. Here is an example of where to add the additional emails:

In the Notifications section shown above, you will see the Auto Subscribed Namespace reference. This is how DSM Users which have their permissions associated with a namespace are discovered, and so have the Database Alerts sent to their respective emails. But for VCF Automation, those Supervisor namespace where the databases are provisioned do not have any DSM Users associated. Thus, Database Alerts from databases provisioned in VCF Automation are not sent to any users automatically, not even the VCFA organization users who have been assigned to the Project where the namespaces are created. Thus, to have users receive VCFA provisioned database alerts via email notifications, add those users to the Additional Alert Email Recipients section shown above. Unfortunately, there is no public API to automate this task at the time of writing, so the process is manual for now. The DSM team are working to address both of these limitations.

Summary

I hope this has gone some way towards explaining the different alert types, and how they are sent to email addresses of the different personas within DSM. The important part in all of this is ensuring that the DSM Admins and DSM Users have their emails configured correctly, especially if the users are LDAP or Active Directory users – make sure that the test email is working before looking for any other emails! And if you are a VCF Automation user, database alerts email notifications can be configured via the DSM UI, adding the recipients email address to the appropriate databases.

The post Configuring Email Alerts in Data Services Manager version 9.1 appeared first on CormacHogan.com.

Download the SQL Server Management Pack

In VCF Operations, select Administration > Integrations > Solutions Catalog. The database management packs are not available by default. You will need to download them from the VCF Solutions Catalog.  You can navigate to the VCF Solutions Catalog directly from VCF operations, or you can get to it from https://vcf.broadcom.com/vsc/services. Browse the Management Packs. Here you will find the Management Pack for SQL Server. You will need to sign in to download it. You will also have to create a “Signing Key” and then provide this when you install the Management Pack into VCF Operations. The latest version of the SQL Server Management Pack at the time of writing is 9.0.0.0100.

Install SQL Server Management Pack

In VCF Operations, select Operate / Integrations > Solutions Catalog. You can see the downloaded management packs from there, as shown below. Once the management pack is downloaded into the catalog, install it. You will be prompted for the “Signing Key” created earlier as part of the install.

Create SQL Server Account

Microsoft SQL Server now appears as an available account type after the management pack has been successfully installed in your VCF Operations.

Add a SQL Server account. Provide the necessary information to connect to the SQL Server instance in the account. I am using the mssql-admin credentials that DSM created for the instance.

Validate the Connection as a test:

Soon after adding the account, you should see data getting collected from the SQL Server.

Examine the SQL Server Dashboards

The SQL Server Management Pack includes some very useful dashboards ‘out of the box’. Navigate to dashboards in VCF Operations where you should see MS SQL Server metrics now displayed. This first one details the Query metrics.

You can also drill down into database specific resource consumption:

Note that on the initial install of the management pack, and the initial view of the dashboards, the Microsoft SQL Server dashboard was empty in my environment. It did not show any SQL Server instances. I therefore edited the dashboard to see what the inputs were. The inventory tree was correctly set to MS SQL Server Environment but there were no Objects. After adding the SQL Server instances to the Input data Objects as shown here, and saving the changes to the dashboard, everything started to work (this step may not be necessary in your environments, as the second time I tested the management pack installation and account creation, the Objects auto-filled correctly):

Now the SQL Server instances are visible in the dashboard and can be selected to get more details on the deployment. To get this view, single click on the instance to open the interaction menu, and then select the internal view. The other item takes you back to the Query dashboard seen earlier.

This is a really nice dashboard to have because once you have selected the instance and database, you can begin to examine metrics like Longest Wait Types, which are essentially the internal indicators that record why a query thread has paused execution and what specific resource it is waiting for.

One final note – there does appear to be some discrepancy in building relationships between the VMs and the SQL Server database nodes. This can  be seen in the Related Objects window of the final screenshot above. If VCF Operations is unable to resolve the SQL Server node name, then the relationships are not built in this version of the management pack 9.0.0.0100.  In the adapter logs, located on the VCF Ops 9.1 appliance in /storage/log/vcops/log/adapters/MicrosoftSQLServerAdapter, the logs will show the following:

Failed to add VirtualMachine resource with hostname "msql-vcfa-instance1-2" as parent of SqlAvailabilityGroup 
resource with name "DSMAG": Failed to create relationship with VM names ("msql-vcfa-instance1-2")

The reason is that the SQL Server nodename msql-vcfa-instance-2 may not be the same as the hostname of the virtual machine/OS. In my case, the hostname of the VM was msql-vcfa-instance1-398432-4z2fl-czs9p as shown in the vCenter inventory. So this is why it cannot resolve, and why it cannot create the relationships. Basically, VCF Operations needs to be able to resolve the nodename of the SQL Server. A work-around is to simply add an alias in DNS for the SQL server nodename alongside the VM hostname, (i.e., have entries for both msql-vcfa-instance-2 and msql-vcfa-instance1-398432-4z2fl-czs9p in DNS pointing to the same IP address). This should allow the relationship between VM/OS hostname (plus other vSphere infra) and the SQL Server nodename to work, and the Relationship Objects view should reveal far more detail:

For further information on the SQL Server Management Pack, check out some of the official documentation here and here.

The post Using SQL Server Management Pack in VCF Operations with DSM provisioned SQL Server appeared first on CormacHogan.com.

First, you will need some simple scripts to get you started. If you are unfamiliar with blueprints, feel free to download some of my simple examples from my GitHub repo here. The blueprints that I will use in this post are the “create-mysql-snpshot.blp” and “create-mysql-from-snap.blp”.

Next, we should discuss the personas that can be granted to VCF Automation Organization users. We have Project Admins, Project Advanced Users and Project Users. Only the Project Admins can do Blueprint designs. Project Advanced Users and Project Users do not have access to blueprint designs, but both can consume request blueprints from the catalog. These are the users in my organization:

Create MySQL Snapshot

Let’s start by creating the snapshot of an existing MySQL database. I will login to the organization as user ‘cormac’ who is a project admin. Under Build & Deploy, select Content Hub > Blueprint Design. From the Blueprints > “New From” dropdown, we can begin to import the blueprints downloaded from the GitHub repo mentioned earlier.

Next, provide general information about the blueprint. Select the name, optional description, project, whether to share the blueprint with other projects in the organization, upload the downloaded blueprint, and optionally select an icon for the catalog item.

The blueprint should now be visible.

You can now begin to modify the blueprint to match your own environment. Click on the name of the blueprint to enter ‘edit’ mode:

There are two sections to the blueprint code. First, you are setting up your inputs which will be used to prompt the user to provide information about the MySQL database to snapshot. The second is actually the piece of YAML code which matches the object your are trying to create. For us, this is a MySQL snapshot as defined in the DSM API. You can now fine-tune the code, correcting namespace names and other default values. When you are happy with the code, click on the ‘TEST’ button in the lower left hand corner of the screen. This will launch the blueprint but will only check syntax – it does not do an actual test.

If the test is successful, it will report back something similar to the following:

To add the blueprint to the catalog, click on the VERSION button, and add a description, a change log and check the box to publish the blueprint to the catalog. Now, navigate to the Catalog in the left hand navigation menu, and the item should be available.

Now you can request a deployment of the catalog item. A deployment name is needed so the request can be identified in the instances view. The other items relate to the Supervisor namespace where the MySQL database is deployed from, the name of the database and the name of the snapshot that you wish to create:

Click Submit, and the name of the deployment will be displayed as in progress. If you then click on the name of the deployment (demo), and then the History view, you should be able to observer various events occurring as the snapshot is getting created. This is also a good place to check if the snapshot creation fails as it should report which part of the job the snapshot failed at.

Tasks will also be visible in the vSphere client as the snapshot is getting provisioned.

Eventually, you should be able to observe that the request completed successfully and that the blueprint has been able to take a snapshot of an existing MySQL database.

It is also possible to check on the snapshot through the API or by logging into the DSM appliance as the root user and running the following command (kg is an alias for kubectl which points to the gateway KUBECONFIG):

# kg get mysqlsnapshots -n b03cde-tenant-02-ns-01-5wtsk 
NAME                 CLUSTERNAME    STATUS   CREATED 
mysql-db01-snap-01   mysql-db01     Ready    24m

Clone MySQL Snapshot to new database

We can now start to look at the second part of the post, which is how to clone a snapshot to create a new MySQL database. Once more, we can begin by importing an existing blueprint to do the clone operation. This can once again be retrieved from my GitHub repo, and the process is identical to before.

Fine tune any code, and the default values to match your own environment. Again, items such as namespaces will need to be changed to reflect your setup. the only inputs this time are the name of the new database and the name of the snapshot. Many of the other parameters simply relate to the information that one would need to provide when creating a MySQL database. But you can tune the code for all of these entries. Use the TEST button once again to make sure that the fields are populating as expected.

Use the VERSION button to upload a copy of the blueprint to the catalog when you are happy with the code and the tests are successful.

Verify the item appears in the catalog alongside our snapshot entry:

Once added to the catalog, you can make a request to clone a new MySQL database from an existing snapshot.

Again, you can monitor the clone request in the same way as you monitored the snapshot request. In fact, this can also be monitored from the DSM UI if you wish.

And if everything has gone smoothly, you can click on the deployment (clone-demo) to get further details, similar to what we saw with the snapshot. The new database should also be Ready under the Databases view in the tenant’s namespace, as shown below. The original database which the snapshot was taken from is also visible in this view.

Summary

In Data Services Manager version 9.1, MySQL databases can now leverage the the new snapshot capability in vSAN ESA. Although not available for selection direction in the VCF Automation UI, the blueprint feature allows project admin users to create catalog items which can then be requested by other project users.

Now, you might be thinking that this is all well and good, but it needs vSAN. You would be correct. But have you considered using your existing certified ESX hosts as vSAN ESA nodes by adding NVMe devices? Were you aware that you could do this? If so, and you were planning to do so, we would like to hear from you. Even if you’re not, we would still like to hear from you. You will find a short survey link here.

The post Using VCF Automation 9.1 Blueprints to snapshot and clone DSM provisioned MySQL databases appeared first on CormacHogan.com.