Home Linux & Systems Cybersecurity Cloud & DevOps Networks & Infrastructure SIEM & Monitoring DFIR & Threat Intel Development & Other All categories Projects About Tools

Migrating Cloud Witness to a New Storage Account via Private Endpoint

Leer en espanol
Migrating Cloud Witness to a New Storage Account via Private Endpoint

Table of contents

Context

In production environments with Windows Server Failover Cluster (WSFC), the Cloud Witness-based quorum uses an Azure Storage Account as witness. When the infrastructure is migrated to a new subscription or security is hardened using Private Endpoints, the Cloud Witness must be reconfigured to point to the new Storage Account.

This procedure documents step by step how to perform this migration safely and without service downtime, given that the witness does not actively participate in the cluster I/O except in split-brain situations.

Scenario

ElementCurrent valueTarget value
Storage Accountoldcloudwitness (public)newazurestorage01 (Private Endpoint)
Access typeInternet (public endpoint)Private Endpoint (internal network)
Cluster nodesSQLNODE01, SQLNODE02Unchanged
Cluster nameSQLCLUSTER01Unchanged
PortTCP 443TCP 443

Prerequisites

  • Administrative access to the cluster nodes
  • Access Key of the new Storage Account (Azure Portal > Storage Account > Security + networking > Access keys)
  • Private Endpoint configured on the new Storage Account
  • Firewall rules that allow the communication

Step 1: Verify the current configuration

Before making any change, we document the current state of the quorum:

POWERSHELL
Get-ClusterQuorum
CODE
Cluster              QuorumResource
-------              --------------
SQLCLUSTER01         Cloud Witness

We check the parameters of the current Cloud Witness:

POWERSHELL
Get-ClusterResource | Where-Object {$_.ResourceType -eq "Cloud Witness"} | Get-ClusterParameter
CODE
Object        Name         Value                    Type
------        ----         -----                    ----
Cloud Witness AccountName  oldcloudwitness          String
Cloud Witness EndpointInfo  core.windows.net        String

Step 2: Review firewall rules

For the migration to Private Endpoint, we need to ensure connectivity from both cluster nodes toward the private IPs of the new Storage Account:

SourceDestinationPortProtocol
10.10.1.11 (SQLNODE01)10.20.5.10 (newazurestorage01 PE)443TCP
10.10.1.11 (SQLNODE01)10.20.5.11 (newazurestorage02 PE)443TCP
10.10.1.12 (SQLNODE02)10.20.5.10 (newazurestorage01 PE)443TCP
10.10.1.12 (SQLNODE02)10.20.5.11 (newazurestorage02 PE)443TCP

Note: If several candidate Storage Accounts are available, validate connectivity to all of them before deciding which one to use as the new witness.

Step 3: Validate DNS resolution and connectivity

Verify DNS resolution from the nodes

From each cluster node, we check that the FQDN of the new Storage Account resolves to the Private Endpoint IP (not to the public IP):

POWERSHELL
# Resolution of the current storage account (public)
Resolve-DnsName "oldcloudwitness.blob.core.windows.net"

# Resolution of the new storage accounts (should resolve to a private IP)
Resolve-DnsName "newazurestorage01.blob.core.windows.net"
Resolve-DnsName "newazurestorage02.blob.core.windows.net"

Expected result - The new storage should resolve via privatelink:

CODE
Name                           Type   TTL   Section    NameHost
----                           ----   ---   -------    --------
newazurestorage01.blob.core.wi CNAME  9     Answer     newazurestorage01.privatelink.blob.core.windows.net
ndows.net

Name       : newazurestorage01.privatelink.blob.core.windows.net
QueryType  : A
TTL        : 9
Section    : Answer
IP4Address : 10.20.5.10

If the resolution returns a public IP instead of the private one, review the private DNS zone configuration in Azure (privatelink.blob.core.windows.net).

Connectivity test

We verify that the nodes reach port 443 of the new Storage Account:

POWERSHELL
# Test against new storage account
Test-NetConnection -ComputerName "newazurestorage01.blob.core.windows.net" -Port 443
Test-NetConnection -ComputerName "newazurestorage02.blob.core.windows.net" -Port 443

Expected result:

CODE
ComputerName     : newazurestorage01.blob.core.windows.net
RemoteAddress    : 10.20.5.10
RemotePort       : 443
InterfaceAlias   : Ethernet0
SourceAddress    : 10.10.1.11
TcpTestSucceeded : True

Important: The TcpTestSucceeded field must be True on both nodes for both Storage Accounts. If it is False, review the NSG rules, on-premise firewall or UDR.

Step 4: Configure the new Cloud Witness

Once connectivity is validated, we proceed to reconfigure the Cloud Witness. There are two methods:

Option A: PowerShell (recommended)

POWERSHELL
Set-ClusterQuorum -CloudWitness `
    -AccountName "newazurestorage01" `
    -AccessKey "YOUR_BASE64_ACCESS_KEY_HERE=="

Note: The Access Key is obtained from Azure Portal > Storage Account > Security + networking > Access keys (key1 or key2).

Option B: Failover Cluster Manager (GUI)

  1. Open Failover Cluster Manager
  2. Right-click on the cluster > More Actions > Configure Cluster Quorum Settings...
  3. In the wizard, select Select the quorum witness > Configure a cloud witness
  4. Enter:
  • Azure Storage account name: name of the new Storage Account
  • Azure Storage account key: key1 or key2 of the new SA
  1. Finish the wizard

The cluster automatically creates a blob container named msft-cloud-witness in the new Storage Account.

Step 5: Post-change verification

After applying the configuration, we verify that the cluster has adopted the new Storage Account:

POWERSHELL
Get-ClusterResource "Cloud Witness" | Get-ClusterParameter

Expected result:

CODE
Object        Name         Value                    Type
------        ----         -----                    ----
Cloud Witness AccountName  newazurestorage01        String
Cloud Witness EndpointInfo  core.windows.net        String

Additionally, verify in the Azure portal that the msft-cloud-witness container has been created in the new Storage Account.

Change window

AspectDetail
Estimated duration15-30 minutes (if prerequisites are completed)
Service impactNo downtime - the witness only intervenes in split-brain
RollbackRun Set-ClusterQuorum with the data of the previous SA
RiskLow - changing the witness does not affect the cluster I/O

Security considerations

  • Private Endpoint: By using a Private Endpoint, the traffic between the nodes and the Storage Account does not go out to the Internet, staying within the Azure private network
  • Key rotation: Plan periodic rotation of the Storage Account Access Keys
  • Monitor the witness: Configure alerts in case the Cloud Witness becomes inaccessible (event 1562 in the cluster Event Log)

Conclusions

Migrating the Cloud Witness between Storage Accounts is a low-risk operation that does not cause any service downtime. The key lies in the prior preparation: validating firewall rules, confirming DNS resolution via Private Endpoint and verifying TCP 443 connectivity from all nodes before executing the change.

Comments