In addition to configuring the environment VoltDB runs in, there are many different characteristics of the database itself you can control. These include mapping network interfaces and ports, selecting and configuring database features, and identifying the database schema, class files, and security settings.
The network settings are defined through the cluster.serviceSpec properties, where you can choose
the individual ports and choose whether to expose them through the networking service. For example, the following YAML file
disables exposure of the admin port and assigns the externalized client port to 31313:
Documentationcluster:
serviceSpec:
type: NodePort
adminPortEnabled: false
clientPortEnabled: true
clientNodePort: 31313The majority of the database configuration options for VoltDB are traditionally defined in an XML configuration file. When using Kubernetes, these options are declared using YAML and Helm properties. The Helm properties follow the same structure as the XML configuration, beginning with "cluster.config". So, for example, where the number of sites per host is defined in XML as:
<deployment>
<cluster sitesperhost="{n}"/>
</deployment>It is defined in Kubernetes as:
cluster:
config:
deployment:
cluster:
sitesperhost: {n}The following sections give examples of defining common database configurations options using YAML. See Section B.6, “VoltDB Database Configuration Options” for a complete list of the Helm properties available for configuring the database.
Volt Active Data provides high availability through K-safety, where copies of each partition are distributed to different nodes in the database cluster. If a node fails, the database can continue to operate because there are still copies of every partition within the cluster. The amount of durability depends on the K factor. So a K factor of one means that the cluster is guaranteed to survive one node (or pod) failing, a factor of two guarantees two nodes, and so on. (See the chapter on Availability in the Using VoltDB manual for more information on how K-safety works.)
You set the K-safety factor using the cluster.config.deployment.cluster.kfactor property when
configuring your database. For example, the following YAML sets the K-safety factor to two:
cluster:
clusterSpec:
replicas: 6
config:
deployment:
cluster:
sitesperhost: 8
kfactor: 2Note that the number of replicas must be at least as large as the K factor plus one (K+1) and K-safety is most effective if the number of replicas times the number of sites per host is a multiple of K+1.
The combination of K-safety and Kubernetes provides an automated, self-healing system where K-safety ensures the cluster survives individual nodes failing and Kubernetes manages the automated recreation of the pods when they fail so the database can be restored to a full complement of nodes as soon as possible. However, to take full advantage of this capability you need to ensure the Kubernetes infrastructure is configured correctly to distribute the Volt servers evenly and that Volt uses information about the configuration to manage the distribution of partitions within the database. The following sections explain how to use Kubernetes configuration options, such as affinity and spread constraints, and Volt placement groups to achieve maximum availability.
K-safety ensures the database cluster can survive at least a certain number of node failures. However, to reduce the risk of larger scale outages, you need to make sure that the Volt servers are distributed in such a way to minimize the impact of external outages. In particular, you want to ensure that each Volt server pod runs on a separate Kubernetes node (so that a Kubernetes node failure cannot impact multiple pods) and that the pods are, as much as possible, evenly distributed among the availability zones in use.
By default, the Volt Operator establishes Kubernetes affinity and anti-affinity rules such that no two Volt server pods can run on the same Kubernetes node. So, normally, you do not need to take any actions to make this happen. However, if you are overriding the Operator's default configurations, you will need to make sure your custom Kubernetes configuration includes this behavior.
When using multiple availability zones, you should also adjust the Kubernetes configuration — specifically
the spread constraints — so that the Volt server pods are evenly distributed among the zones. This makes it
possible to avoid the database failing due to the loss of any one zone that contains an unbalanced and excessive number
of Volt server processes. You can define the distribution of server pods within your Helm configuration using the
cluster.clusterSpec.topologySpreadConstraints property. The following example demonstrates how to
do this, using the label selector to identify the Volt server processes.
cluster:
clusterSpec:
topologySpreadConstraints:
- topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
maxSkew: 1
labelSelector:
matchLabels:
name: voltdb-clusterIf you are running multiple databases within a single namespace, you should consider replacing the last line of the configuration, "name: voltdb-cluster", with an identifier that is specific to the cluster being configured. For example, if the cluster release name is mydb, the last line of the configuration should read "voltdb-cluster-name: mydb-voltdb-cluster".
K-safety guarantees the minimum number of nodes that can fail without stopping the database. Configuring Kubernetes affinity and spread constraints to evenly distribute the database server pods reduces the overall threat of external failures taking down the database. However, to fully maximize the availability, Volt needs to use knowledge about the Kubernetes configuration to intelligently distribute the individual copies of the partitions among those servers.
The cluster may survive more failures than just the minimum guaranteed by K-safety depending on how the partitions are distributed and which nodes fail. Placement groups are a mechanism for providing more context concerning the hardware environment to improve the likelihood of the cluster surviving multiple failures. For example, if you tell Volt certain nodes are in the same region and zone (i.e. in the same placement group), it avoids placing all copies of any partition on those nodes, so if the zone fails, the database can survive.
Because you do not control exactly where each pod is created in Kubernetes, Volt can use its knowledge of the
Kubernetes availability zones and regions[3] to automate the placement groups and minimize the potential of an infrastructure failure taking the
database down with it. You enable cloud native placement groups in Kubernetes by setting the property
cluster.clusterSpec.useCloudNativePlacementGroup to "true". For cloud native placement groups to be
effective, the cluster configuration must meet the following requirements:
The cluster must be distributed over three or more regions or availability zones.
The number of nodes (or replicas) must be a multiple of the number of availability zones.
The number of availability zones must be a multiple of K+1.
For example, the following configuration assumes the cluster is distributed across four availability zones:
cluster:
clusterSpec:
replicas: 8
useCloudNativePlacementGroup: true
config:
deployment:
cluster:
sitesperhost: 8
kfactor: 1Once the database is running, you can use the @Statistics system procedure with the HOST selector to determine where each node is running and what partitions are running on that node. In addition, if one or more nodes go down, the "SAFETOSTOP" column lets you know which of the remaining nodes could safely be stopped without endangering the cluster as a whole.
$ sqlcmd 1> execute @Statistics HOST; TIMESTAMP HOST_ID HOSTNAME PARTITIONS LEADERS PLACEMENTGROUP SAFETOSTOP REGION ZONE ------------- ------- --------------------- ----------------------- ----------- -------------- ---------- ------ ---- 1677777171869 0 mydb-voltdb-cluster-0 24,25,26,27,28,29,30,31 25,27,29,31 east--zone4 true east zone4 1677777171870 1 mydb-voltdb-cluster-1 8,9,10,11,12,13,14,15 8,10,12,14 east--zone1 true east zone1 1677777171870 2 mydb-voltdb-cluster-2 8,9,10,11,12,13,14,15 9,11,13,15 east--zone2 true east zone2 1677777171870 3 mydb-voltdb-cluster-3 16,17,18,19,20,21,22,23 16,18,20,22 east--zone3 true east zone3 1677777171870 4 mydb-voltdb-cluster-4 16,17,18,19,20,21,22,23 17,19,21,23 east--zone4 true east zone4 1677777171870 5 mydb-voltdb-cluster-5 24,25,26,27,28,29,30,31 24,26,28,30 east--zone3 true east zone3 1677777171870 6 mydb-voltdb-cluster-6 0,1,2,3,4,5,6,7 0,2,4,6 east--zone1 true east zone1 1677777171870 7 mydb-voltdb-cluster-7 0,1,2,3,4,5,6,7 1,3,5,7 east--zone2 true east zone2 (Returned 8 rows in 0.01s) TIMESTAMP PLACEMENTGROUP SAFETOSTOP ------------- -------------- ---------- 1677777171882 east--zone3 true 1677777171882 east--zone2 true 1677777171882 east--zone1 true 1677777171882 east--zone4 true
Command logging provides durability of the database content across failures. You can control the level of durability
as well as the length of time required to recover the database by configuring the type of command logging and size of the
logs themselves. In Kubernetes this is done with the cluster.config.deployment.commandlog properties.
The following example enables synchronous command logging and sets the log size to 3,072 megabytes and the frequency to
1,000 transactions:
cluster:
config:
deployment:
commandlog:
enabled: true
synchronous: true
logsize: 3072
frequency:
transactions 1000Export simplifies the integration of the VoltDB database with external databases and systems. You use the export
configuration to define external "targets" the database can write to. In Kubernetes you define export targets using the
cluster.config.deployment.export.configurations property. Note that the
configurations property can accept multiple configuration definitions. In YAML, you specify a list by
prefixing each list element with a hyphen, even if there is only one element. The following example defines one export
target, eventlog, using the file export connector:
cluster:
config:
deployment:
export:
configurations:
- target: eventlog
type: file
properties:
type: csv
nonce: eventlog [3] Placement groups depend on the Kubernetes labels topology.kubernetes.io/region and topology.kubernetes.io/zone, which are defined automatically by most commercial cloud providers. If you are using a custom cloud deployment, you will need to make sure these labels are declared appropriately before enabling cloud native placement groups.