A.3. Database Configuration Options

Documentation

VoltDB Home » Documentation » Administrator's Guide

A.3. Database Configuration Options

Runtime configuration options are set either as part of the configuration file or as command line options when starting the VoltDB server process. These database configuration options are only summarized here. See the Using VoltDB manual for a more detailed explanation. The configuration options include:

  • Sites per host

  • K-Safety

  • Network partition detection

  • Read level consistency

  • Automated snapshots

  • Import and export

  • Command logging

  • Heartbeat

  • Temp table size

  • Query timeout

  • Long-running process warning

A.3.1. Sites per Host

Sites per host specifies the number of unique VoltDB "sites" that are created on each physical database server. The section on "Determining How Many Sites per Host" in the Using VoltDB manual explains how to choose a value for sites per host.

You set the value of sites per host using the sitesperhost attribute of the <cluster> tag in the configuration file.

A.3.2. K-Safety

K-safety defines the level of availability or durability that the database can sustain, by replicating individual partitions to multiple servers. K-safety is described in detail in the "Availability" chapter of the Using VoltDB manual.

You specify the level of K-safety that you want in the configuration file using the kfactor attribute of the <cluster> tag.

A.3.3. Network Partition Detection

Network partition detection protects a VoltDB cluster in environments where the network is susceptible to partial or intermittent failure among the server nodes. Partition detection is described in detail in the "Availability" chapter of the Using VoltDB manual.

Use of network partition detection is strongly recommended for production systems and therefore is enabled by default. You can enable or disable network partition detection in the configuration file using the <partition-detection> tag.

A.3.4. Read Level Consistency

In a highly available cluster (that is, a VoltDB cluster using K-safety), write transactions are executed on all copies of a partition. This guarantees data consistency across the cluster. By default, read transactions are also processed on all copies of the partition, using safe read level consistency.

Since the data is the same on all copies of the partition, it is possible to optimize read transactions by processing the queries on a single copy of the partition only. However, using this fast read level consistency, read transactions can access data modified by write transactions on the local server, before all nodes confirm those write transactions. During a node failure or network partition, it is possible that the locally completed writes could be rolled back as part of the network partition resolution. Of course, this only happens when the read transaction accesses data modified by an immediately preceding write that is not committed on all copies of the partition prior to a network partition.

If you do not need guaranteed read consistency in all cases, you can choose to have the cluster produce faster read transactions by setting the read level consistency to "fast" in the deployment file, so read transactions are processed on only one copy of the partition. To set the read level consistency, use the the <consistency> element in the configuration file and set the readlevel attribute to either "safe" (the default) or "fast". For example:

<consistency readlevel="fast"/>

A.3.5. Automated Snapshots

Automated snapshots provide ongoing protection against possible database failure (due to hardware or software issues) by taking periodic snapshots of the database's contents. Automated snapshots are described in detail in the section on "Scheduling Automated Snapshots" in the Using VoltDB manual.

You enable and configure automated snapshots with the <snapshot> tag in the configuration file.

Snapshot activity involves both processing and disk I/O and so may have a noticeable impact on performance (in terms of throughput and/or latency) on a very busy database. You can control the priority of snapshots activity using the <snapshot/> tag within the <systemsettings> element of the deployment file. The snapshot priority is an integer value between 0 and 10, with 0 being the highest priority and 10 being the lowest. The closer to 10, the longer snapshots take to complete, but the less they can affect ongoing database work.

Note that snapshot priority affects all snapshot activity, including automated snapshots, manual snapshots, and command logging snapshots.

A.3.6. Import and Export

The import and export functions let you automatically import and/or export selected data between your VoltDB database and another database or distributed service at runtime. These features are described in detail in the chapter on "Importing and Exporting Live Data" in the Using VoltDB manual.

You enable and disable import and export using the <import> and <export> tags in the configuration file.

A.3.7. Command Logging

The command logging function saves a record of each transaction as it is initiated. These logs can then be "replayed" to recreate the database's last known state in case of intentional or accidental shutdown. This feature is described in detail in the chapter on "Command Logging and Recovery" in the Using VoltDB manual.

To enable and disable command logging, use the <commandlog> tag in the configuration file.

A.3.8. Heartbeat

The database servers use a "heartbeat" to verify the presence of other nodes in the cluster. If a heartbeat is not received within a specified time limit, that server is assumed to be down and the cluster reconfigures itself with the remaining nodes (assuming it is running with K-safety). This time limit is called the "heartbeat timeout" and is specified as a integer number of seconds.

For most situations, the default value for the timeout (10 seconds) is appropriate. However, if your cluster is operating in an environment that is susceptible to network fluctuations or unpredictable latency, you may want to increase the heartbeat timeout period.

You can set an alternate heartbeat timeout using the <heartbeat> tag in the configuration file.

Note

Be aware that certain Linux system settings can override the VoltDB heartbeat messages. In particular, lowering the setting for TCP_RETRIES2 may result in the system network timeout interrupting VoltDB's heartbeat mechanism and causing timeouts sooner than expected. Values lower than 8 for TCP_RETRIES2 are not recommended.

A.3.9. Temp Table Size

VoltDB uses temporary tables to store intermediate table data while processing transactions. The default temp table size is 100 megabytes. This setting is appropriate for most applications. However, extremely complex queries or many updates to large records could cause the temporary space to exceed the maximum size, resulting in the transaction failing with an error.

In these unusual cases, you may need to increase the temp table size. You can specify a different size for the temp tables using the <temptables> tag in the configuration file and specifying a whole number of megabytes. Note: since the temp tables are allocated as needed, increasing the maximum size can result in a Java out-of-memory error at runtime if the system is memory-constrained. Modifying the temp table size should be done with caution.

A.3.10. Query Timeout

In general, SQL queries execute extremely quickly. But it is possible, usually by accident, to construct a query that takes an unexpectedly long time to execute. This usually happens when the query is overly complex or accesses extremely large tables without the benefit of an appropriate filter or index.

You have the option to set a query timeout limit cluster-wide, for an interactive session, or per transaction. The query limit sets a limit on the length of time any read-only query (or batch of queries in the case of the voltExecuteSQL() method in a stored procedure) is allowed to run. You specify the timeout limit in milliseconds.

To set a cluster-wide query limit you use the <systemsettings> and <query timeout="{limit}"> tags in the configuration file. To set a limit for an interactive session in the sqlcmd utility, you use the --query-timeout flag when invoking sqlcmd. To specify a limit when invoking a specific stored procedure, you use the callProcedureWithTimeout method in place of the callProcedure method.

The cluster-wide limit is set when you initialize the database root directory. By default, the system-wide limit is 10 seconds. You can set a different timeout in the configuration file. Or It can be adjusted using the voltadmin update command to modify the configuration settings while the database is running. If security is enabled, any user can set a lower query limit on a per session or per transaction basis. However, the user must have the ADMIN privilege to set a query limit longer than the cluster-wide setting.

The following example configuration file sets a cluster-wide query timeout value of three seconds:

<systemsettings>
   <query timeout="3000"/>
</systemsettings>

If any query or batch of queries exceeds the query timeout, the query is interrupted and an error returned to the calling application. Note that the limit is applied to read-only ad hoc queries or queries in read-only stored procedures only. In a K-Safe cluster, queries on different copies of a partition may execute at different rates. Consequently the same query may timeout in one copy of the partition but not in another. To avoid possible non-deterministic changes, VoltDB does not apply the time out limit to any queries or procedures that may modify the database contents.

A.3.11. Long-Running Process Warning

You can avoid runaway read-only queries using the query timeout setting. But you cannot stop read-write procedures or other computational tasks, such as automated snapshots. These processes must run to completion. However, you may want to be notified when a process is blocking an execution queue for an extended period of time.

By default, VoltDB writes an informational message into the log file whenever a task runs for more than ten seconds in any of the execution sites. These tasks may be stored procedures, procedure fragments (in the case of multi-partitioned procedures), or operational tasks such as snapshot creation. You can adjust the limit when these messages are written by specifying a value, in milliseconds in the loginfo attribute of the <procedure> tag in the configuration file. For example, the following configuration file entry changes the threshold after which a message is written to the log to three seconds:

<systemsettings>
   <procedure loginfo="3000"/>
</systemsettings>

Note that in a cluster, the informational message is written only to the log of the server that is hosting the affected queue, not to all server logs.