HA Behavior - Avoiding split brain between two scalearc systems
|Release||Document Status||DB Platform||Categories|
How to avoid splitbrain status between two ScaleArc systems?
Split brain is when two ScaleArc Systems in HA configuration think they are both primary or both secondary in the UI. Often this occurs in the following scenario:
1. Create an HA setup between 2 ScaleArc systems. SA1 (primary) and SA2 (secondary).
This can be avoided by removing the entire configuration from SA2 including the network settings. Make sure to remove the system from the network and change it's IP address before turning it back on and adding it to the same network.
Split brain can also occur when a secondary ScaleArc device detects a loss of communication with a primary ScaleArc device when the primary ScaleArc is in fact still operational. The HA deadtime setting controls the amount of time that the secondary will wait when no keepalive traffic is seen before declaring the primary offline. If the secondary declares the primary as offline when the primary is in fact still operational, a split brain situation will occur.
Ensure that the keepalive and deadtime values for your HA configuration is appropriate for your specifc network environment.
It is important to note that the base failover time for ScaleArc is 650ms with an additional 130ms added for each configured cluster. For example, if a ScaleArc HA pair is configured with 10 clusters, the total time needed to failover from primary to secondary would be 1300ms + 650ms for a total of 1950ms, or just under two seconds. Depending on your business and application needs, a keepalive time of 1000ms with a deadtime of 3000ms may be sufficient for failove purposes while allowing for short periods of network packet delay or interruption.
If you are experiencing issues with ScaleArc or with any of it's features, please contact ScaleArc Support. We are available 24x7 by phone at 855 800 7225 or +1 408 412 7315. For general support inquiries, you can also e-mail us at email@example.com.