You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Choosing the right number of fault domains for Virtual Machine Scale Set
16
16
17
-
The fault domain (FD) configuration for Virtual Machine Scale Sets varies depending on the orchestration mode:
17
+
A fault domain is a fault isolation group within an availability zone or datacenter of hardware nodes that share the same power, networking, cooling, and platform maintenance schedule. Virtual machine (VM) instances that are on different fault domains are not likely to be impacted by the same planned or unplanned outage.
18
+
19
+
You can specify how instances are spread across fault domains within a region or zone. The fault domain configuration options available to you vary depending on the orchestration mode your scale set uses, and whether [your scale set uses availability zones](./virtual-machine-scale-sets-use-availability-zones.md).
18
20
19
21
## Supported fault domain configurations
20
22
21
-
The following table shows the supported `platformFaultDomainCount` values for different orchestration modes and deployment types:
23
+
There are two types of fault domain spreading: max and fixed.
24
+
25
+
The following table summarizes the supported `platformFaultDomainCount` values for different orchestration modes and deployment types:
22
26
23
27
| Orchestration Mode | Deployment Type | Supported Values | Default Value |
Azure spreads the scale set's VM instances across as many fault domains as possible, on a best-effort basis. It might use greater or fewer than five fault domains.
37
+
38
+
Set `platformFaultDomainCount` to a value of `1` for max spreading.
39
+
40
+
When you look at the scale set's VM instances, you only see one fault domain. This is expected behavior. The spreading is implicit.
41
+
42
+
> [!NOTE]
43
+
> We recommend using max spreading (`platformFaultDomainCount = 1`) for most scale sets, because it provides the best spreading in most cases. If you need your instances to be spread across distinct hardware isolation units, we recommend you configure max spreading and use multiple availability zones. Azure spreads the instances across fault domains within each zone.
44
+
45
+
### Fixed spreading
46
+
47
+
When your scale set uses fixed spreading, Azure spreads the VM instances across the specified number of fault domains. If the scale set can't allocate VMs that meet specified fault domain count, the request fails.
48
+
49
+
Set the `platformFaultDomainCount` to a value greater than 1 for fixed spreading.
29
50
30
-
## Uniform Orchestration Mode
31
-
Virtual Machine Scale Sets with Uniform orchestration are created with five fault domains by default in Azure regions with no zones. For regions that support zonal deployment of Virtual Machine Scale Sets and this option is selected, the default value of the fault domain count is `1` for each of the zones. A `platformFaultDomainCount` of `1` in this case implies that the virtual machine (VM) instances belonging to the scale set are spread across many racks on a best effort basis.
51
+
There are specific numbers of fault domains you can select depending on your orchestration mode and deployment type.
32
52
33
-
You can also consider aligning the number of scale set fault domains with the number of Managed Disks fault domains. This alignment can help prevent loss of quorum if an entire Managed Disks fault domain goes down. The FD count can be set to less than or equal to the number of Managed Disks fault domains available in each of the regions. Refer to the [documentation](../virtual-machines/availability-set-overview.md) to learn about the number of Managed Disks fault domains by region.
53
+
## Uniform orchestration mode
34
54
35
-
## Flexible Orchestration Mode
36
-
Virtual Machine Scale Sets with Flexible orchestration support different fault domain configurations depending on the deployment type:
55
+
Scale sets with Uniform orchestration support different fault domain configurations depending on the deployment type:
37
56
38
-
-**Regional deployments**: Support fault domain counts of `1`, `2`, or `3`.
39
-
-**Zonal deployments**: Support only fault domain count of `1`.
57
+
-**Zonal or zone-spanning scale sets:** Use max spreading by default (`platformFaultDomainCount = 1`). You can optionally configure fixed spreading with five fault domains (`platformFaultDomainCount = 5`).
40
58
41
-
For zonal deployments, a `platformFaultDomainCount` of `1` implies that the VM instances belonging to the scale set are spread across many racks within the zone on a best effort basis. Fault domain and update domain information isn't exposed in the Instance View REST API response for Flexible scale sets, unlike Uniform orchestration mode.
59
+
-**Regional (nonzonal) scale sets:** Use fixed spreading with five fault domains by default (`platformFaultDomainCount = 5`). You can optionally configure max spreading (`platformFaultDomainCount = 1`).
42
60
43
-
### Instance View API Behavior
44
-
When using the [Virtual Machines - Instance View REST API](/rest/api/compute/virtualmachines/instanceview) with Flexible orchestration mode:
45
-
- The response doesn't include `faultDomain` and `updateDomain` properties
46
-
- This is by design and differs from Uniform orchestration mode where these properties are returned
47
-
- For regional deployments with multiple fault domains, the VM instances are distributed across the configured fault domains, but this information isn't exposed through the API
48
-
- For zonal deployments, the VM instances are distributed across multiple racks within the zone
61
+
You can also consider aligning the number of scale set fault domains with the number of managed disk fault domains (for example, `platformFaultDomainCount = 2`). This alignment can help prevent loss of quorum if an entire disk fault domain goes down. The fault domain count can be set to less than or equal to the number of managed disks fault domains available in each of the regions. Refer to the [availability sets documentation](../virtual-machines/availability-set-overview.md) to learn about the number of managed disk fault domains by region.
49
62
50
-
## REST API
51
-
For Uniform orchestration mode, you can set the property `properties.platformFaultDomainCount` to `1`, `2`, or `3`. If not set, the property will default to `1`. For Flexible orchestration mode, you can set this property to `1`, `2`, or `3` for regional deployments, and only `1` is supported for zonal deployments. Refer to the REST API documentation for [Virtual Machine Scale Sets](/rest/api/compute/virtualmachinescalesets/createorupdate).
63
+
## Flexible orchestration mode
52
64
53
-
## Azure CLI
65
+
Scale sets with Flexible orchestration support different fault domain configurations depending on the deployment type:
66
+
67
+
-**Zonal or zone-spanning scale sets**: Only support max spreading (`platformFaultDomainCount = 1`).
68
+
69
+
-**Regional (nonzonal) scale sets**: Use max spreading (`platformFaultDomainCount = 1`) by default. You can optionally configure fault domain counts of `2` or `3`.
70
+
71
+
## Fault domains and tools
72
+
73
+
You can configure fault domains by using standard Azure deployment tools and APIs.
74
+
75
+
### REST API
76
+
77
+
Configure the scale set's fault domain spreading by setting the property `properties.platformFaultDomainCount`. Refer to the REST API documentation for [Virtual Machine Scale Sets](/rest/api/compute/virtualmachinescalesets/createorupdate).
78
+
79
+
### Azure CLI
54
80
55
81
> [!IMPORTANT]
56
-
>Starting November 2023, VM scale sets created using PowerShell and Azure CLI will default to Flexible Orchestration Mode if no orchestration mode is specified. For more information about this change and what actions you should take, go to [Breaking Change for VMSS PowerShell/CLI Customers - Microsoft Community Hub](https://techcommunity.microsoft.com/t5/azure-compute-blog/breaking-change-for-vmss-powershell-cli-customers/ba-p/3818295)
57
-
58
-
For Uniform orchestration mode, you can set the parameter `--platform-fault-domain-count` to 1, 2, or 3 (default of 3 if not specified). For Flexible orchestration mode, you can set this parameter to 1, 2, or 3 for regional deployments, but only 1 is supported for zonal deployments. Refer to the Azure CLI documentation for [Virtual Machine Scale Sets](/cli/azure/vmss#az-vmss-create).
59
-
60
-
### Example for Uniform Orchestration Mode
61
-
62
-
```azurecli-interactive
63
-
az vmss create \
64
-
--resource-group myResourceGroup \
65
-
--name myScaleSet \
66
-
--orchestration-mode Uniform \
67
-
--image Ubuntu2204 \
68
-
--admin-username azureuser \
69
-
--platform-fault-domain-count 3\
70
-
--generate-ssh-keys
71
-
```
72
-
73
-
### Example for Flexible Orchestration Mode
74
-
75
-
#### Regional deployment with multiple fault domains
76
-
```azurecli-interactive
77
-
az vmss create \
78
-
--resource-group myResourceGroup \
79
-
--name myScaleSet \
80
-
--orchestration-mode Flexible \
81
-
--image Ubuntu2204 \
82
-
--admin-username azureuser \
83
-
--platform-fault-domain-count 3 \
84
-
--generate-ssh-keys
85
-
```
86
-
87
-
#### Zonal deployment
88
-
```azurecli-interactive
89
-
az vmss create \
90
-
--resource-group myResourceGroup \
91
-
--name myScaleSet \
92
-
--orchestration-mode Flexible \
93
-
--image Ubuntu2204 \
94
-
--admin-username azureuser \
95
-
--zones 1 \
96
-
--generate-ssh-keys
97
-
```
82
+
> VM scale sets created using PowerShell and Azure CLI default to Flexible orchestration mode if no orchestration mode is specified.
83
+
84
+
Configure the scale set's fault domain spreading by setting the `--platform-fault-domain-count` parameter. Refer to the Azure CLI documentation for [Virtual Machine Scale Sets](/cli/azure/vmss#az-vmss-create).
85
+
86
+
The following examples show how to use the Azure CLI to deploy scale sets with varying configurations:
87
+
88
+
-**Zone-spanning Flexible scale set that uses three zones, with max spreading in each zone:**
89
+
90
+
```azurecli-interactive
91
+
az vmss create \
92
+
--resource-group myResourceGroup \
93
+
--name myScaleSet \
94
+
--orchestration-mode Flexible \
95
+
--image Ubuntu2204 \
96
+
--admin-username azureuser \
97
+
--zones 1 2 3 \
98
+
--generate-ssh-keys
99
+
```
100
+
101
+
-**Zonal (single-zone) Flexible scale set with max spreading:**
102
+
103
+
```azurecli-interactive
104
+
az vmss create \
105
+
--resource-group myResourceGroup \
106
+
--name myScaleSet \
107
+
--orchestration-mode Flexible \
108
+
--image Ubuntu2204 \
109
+
--admin-username azureuser \
110
+
--zones 1 \
111
+
--generate-ssh-keys
112
+
```
98
113
99
114
> [!NOTE]
100
-
> For zonal Flexible virtual machine scale set deployments, the fault domain count is automatically set to 1 and can't be configured to a higher value.
115
+
> For zone-spanning and zonal Flexible virtual machine scale set deployments, the fault domain count is automatically set to 1 (max spreading) and can't be configured to a different value.
116
+
117
+
-**Nonzonal Flexible scale set with fixed spreading:**
118
+
119
+
```azurecli-interactive
120
+
az vmss create \
121
+
--resource-group myResourceGroup \
122
+
--name myScaleSet \
123
+
--orchestration-mode Flexible \
124
+
--image Ubuntu2204 \
125
+
--admin-username azureuser \
126
+
--platform-fault-domain-count 3 \
127
+
--generate-ssh-keys
128
+
```
129
+
130
+
-**Nonzonal Uniform scale set with max spreading:**
131
+
132
+
```azurecli-interactive
133
+
az vmss create \
134
+
--resource-group myResourceGroup \
135
+
--name myScaleSet \
136
+
--orchestration-mode Uniform \
137
+
--image Ubuntu2204 \
138
+
--admin-username azureuser \
139
+
--platform-fault-domain-count 1 \
140
+
--generate-ssh-keys
141
+
```
101
142
102
143
It takes a few minutes to create and configure all the scale set resources and VMs.
103
144
145
+
### Instance View API
146
+
147
+
When you use the [Virtual Machines - Instance View REST API](/rest/api/compute/virtualmachines/instanceview) with Flexible orchestration mode scale sets, VM instances are distributed across the configured fault domains, but this information isn't exposed through the API. The API responses don't include fault domain or update domain information. This behavior is by design and differs from Uniform orchestration mode where these properties are returned.
148
+
104
149
## Next steps
150
+
105
151
- Learn more about [availability and redundancy features](../virtual-machines/availability.md) for Azure environments.
Copy file name to clipboardExpand all lines: articles/virtual-machine-scale-sets/virtual-machine-scale-sets-scale-in-policy.md
+11-13Lines changed: 11 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,7 +29,7 @@ The scale-in policy feature provides users a way to configure the order in which
29
29
#### Flexible orchestration
30
30
By default, Virtual Machine Scale Set applies this policy to determine which instance(s) will be scaled in. With the *Default* policy, VMs are selected for scale-in in the following order:
31
31
32
-
1. Balance virtual machines across availability zones (if the scale set is deployed in zonal configuration)
32
+
1. Balance virtual machines across availability zones (if the scale set is deployed in zone-spanning configuration)
33
33
2. Balance virtual machines across fault domains (best effort)
34
34
3. Delete virtual machine with the highest instance ID
35
35
@@ -38,7 +38,7 @@ Users don't need to specify a scale-in policy if they just want the default orde
38
38
#### Uniform orchestration
39
39
By default, Virtual Machine Scale Set applies this policy to determine which instance(s) will be scaled in. With the *Default* policy, VMs are selected for scale-in in the following order:
40
40
41
-
1. Balance virtual machines across availability zones (if the scale set is deployed in zonal configuration)
41
+
1. Balance virtual machines across availability zones (if the scale set is deployed in zone-spanning configuration)
42
42
2. Balance virtual machines across fault domains (best effort)
43
43
3. Delete virtual machine with the highest instance ID
44
44
@@ -48,11 +48,11 @@ Balancing across availability zones or fault domains doesn't move instances acro
48
48
49
49
### NewestVM scale-in policy
50
50
51
-
This policy will delete the newest, or most recently created virtual machine in the scale set, after balancing VMs across availability zones (for zonal deployments). Enabling this policy requires a configuration change on the Virtual Machine Scale Set model.
51
+
This policy will delete the newest, or most recently created virtual machine in the scale set, after balancing VMs across availability zones (for zone-spanning scale sets). Enabling this policy requires a configuration change on the Virtual Machine Scale Set model.
52
52
53
53
### OldestVM scale-in policy
54
54
55
-
This policy will delete the oldest created virtual machine in the scale set, after balancing VMs across availability zones (for zonal deployments). Enabling this policy requires a configuration change on the Virtual Machine Scale Set model.
55
+
This policy will delete the oldest created virtual machine in the scale set, after balancing VMs across availability zones (for zone-spanning scale sets). Enabling this policy requires a configuration change on the Virtual Machine Scale Set model.
56
56
57
57
## Enabling scale-in policy
58
58
@@ -126,7 +126,7 @@ az vmss create \
126
126
--scale-in-policy OldestVM
127
127
```
128
128
129
-
### Using Template
129
+
### Resource Manager template
130
130
131
131
In your template, under “properties”, add the `scaleInPolicy` property:
132
132
@@ -140,7 +140,7 @@ These code blocks specify that the Virtual Machine Scale Set will delete the Old
140
140
141
141
When a Virtual Machine Scale Set isn't zone balanced, the scale set will first delete VMs across the imbalanced zone(s). Within the imbalanced zones, the scale set uses the specified scale-in policy to determine which VM to scale in. In this case, within an imbalanced zone, the scale set will select the Oldest VM in that zone to be deleted.
142
142
143
-
For non-zonal Virtual Machine Scale Set, the policy selects the oldest VM across the scale set for deletion.
143
+
For nonzonal Virtual Machine Scale Set, the policy selects the oldest VM across the scale set for deletion.
144
144
145
145
The same process applies when using the ‘NewestVM’ scale-in policy.
146
146
@@ -196,7 +196,7 @@ az vmss update \
196
196
--scale-in-policy OldestVM
197
197
```
198
198
199
-
### Using Template
199
+
### Resource Manager template
200
200
201
201
In your template, under “properties”, modify the template as below and redeploy:
202
202
@@ -235,7 +235,7 @@ The below examples demonstrate how a Virtual Machine Scale Set selects VMs to be
235
235
| Scale-in | 5, 10 |***6***, 9, 11 | 7, 8 | Choose Zone 2 even though Zone 1 has the oldest VM. Delete VM6 in Zone 1 as it's the oldest VM in that zone. |
236
236
| Scale-in |***5***, 10 | 9, 11 | 7, 8 | Zones are balanced. Delete VM5 in Zone 1 as it's the oldest VM in the scale set. |
237
237
238
-
For non-zonal Virtual Machine Scale Sets, the policy selects the oldest VM across the scale set for deletion. Any “protected” VM is skipped for deletion.
238
+
For nonzonal Virtual Machine Scale Sets, the policy selects the oldest VM across the scale set for deletion. Any “protected” VM is skipped for deletion.
239
239
240
240
### NewestVM scale-in policy
241
241
@@ -249,15 +249,13 @@ For non-zonal Virtual Machine Scale Sets, the policy selects the oldest VM acros
249
249
| Scale-in | 3, 4, ***5***| 2, 6 | 1, 7 | Choose Zone 1 even though Zone 3 has the newest VM. Delete VM5 in Zone 1 as it's the newest VM in that Zone. |
250
250
| Scale-in | 3, 4 | 2, 6 | 1, ***7***| Zones are balanced. Delete VM7 in Zone 3 as it's the newest VM in the scale set. |
251
251
252
-
For non-zonal Virtual Machine Scale Sets, the policy selects the newest VM across the scale set for deletion. Any “protected” VM is skipped for deletion.
252
+
For nonzonal Virtual Machine Scale Sets, the policy selects the newest VM across the scale set for deletion. Any “protected” VM is skipped for deletion.
253
253
254
254
## Troubleshoot
255
255
256
-
1. Failure to enable scaleInPolicy
257
-
If you get a ‘BadRequest’ error with an error message stating "Could not find member 'scaleInPolicy' on object of type 'properties'”, then check the API version used for Virtual Machine Scale Set. API version 2019-03-01 or higher is required for this feature.
256
+
-**Failure to enable scaleInPolicy:** If you get a ‘BadRequest’ error with an error message stating "Could not find member 'scaleInPolicy' on object of type 'properties'”, then check the API version used for Virtual Machine Scale Set. API version 2019-03-01 or higher is required for this feature.
258
257
259
-
2. Wrong selection of VMs for scale-in
260
-
Refer to the examples in this document. If your Virtual Machine Scale Set is a Zonal deployment, scale-in policy is applied first to the imbalanced Zones and then across the scale set once it's zone balanced. If the order of scale-in isn't consistent with the examples documented here, raise a query with the Virtual Machine Scale Set team for troubleshooting.
258
+
-**Wrong selection of VMs for scale-in:** Refer to the examples in this document. If your scale set is zone-spanning, the scale-in policy is applied first to the imbalanced zones and then across the scale set once it's zone balanced. If the order of scale-in isn't consistent with the examples documented here, raise a query with the Virtual Machine Scale Set team for troubleshooting.
0 commit comments