Apache Geode
CHANGELOG
Configure Apache Geode to Handle Network Partitioning
This section lists configuration considerations relating to network partition detection.
The system uses a combination of member coordinators and system members, designated as lead members, to detect and resolve network partitioning problems.
Network partition detection works in all environments. Using multiple locators mitigates the effect of network partitioning. See Configuring Peer-to-Peer Discovery.
Network partition detection is enabled by default. The default setting in the
gemfire.propertiesfile isenable-network-partition-detection=trueProcesses that do not have network partition detection enabled are not eligible to be the lead member, so their failure will not trigger declaration of a network partition.
All system members should have the same setting for
enable-network-partition-detection. If they do not, the system throws aGemFireConfigExceptionupon startup.The property
enable-network-partition-detectionmust be true if you are using either partitioned or persistent regions. If you create a persistent region andenable-network-partition-detectionto set to false, you will receive the following warning message:Creating persistent region {0}, but enable-network-partition-detection is set to false. Running with network partition detection disabled can lead to an unrecoverable system in the event of a network split."Configure regions you want to protect from network partitioning with a scope setting of
DISTRIBUTED_ACKorGLOBAL. Do not useDISTRIBUTED_NO_ACKscope. This prevents operations from being performed throughout the cluster before a network partition is detected. Note: Geode issues an alert if it detectsDISTRIBUTED_NO_ACKregions when network partition detection is enabled:Region {0} is being created with scope {1} but enable-network-partition-detection is enabled in the distributed system. This can lead to cache inconsistencies if there is a network failure.These other configuration parameters affect or interact with network partitioning detection. Check whether they are appropriate for your installation and modify as needed.
- If you have network partition detection enabled, the threshold percentage value for allowed membership weight loss is automatically configured to 51. You cannot modify this value. Note: The weight loss calculation uses round to nearest. Therefore, a value of 50.51 is rounded to 51 and will cause a network partition.
- Failure detection is initiated if a member’s
ack-wait-threshold(default is 15 seconds) andack-severe-alert-threshold(15 seconds) properties elapse before receiving a response to a message. If you modify theack-wait-thresholdconfiguration value, you should modifyack-severe-alert-thresholdto match the other configuration value. - If the system has clients connecting to it, the clients’
cache.xmlpoolread-timeoutshould be set to at least three times themember-timeoutsetting in the server’sgemfire.propertiesfile. The default poolread-timeoutsetting is 10000 milliseconds. - You can adjust the default weights of members by specifying the system property
gemfire.member-weightupon startup. For example, if you have some VMs that host a needed service, you could assign them a higher weight upon startup.
By default, members that are forced out of the cluster by a network partition event will automatically restart and attempt to reconnect. Data members will attempt to reinitialize the cache. See Handling Forced Cache Disconnection Using Autoreconnect.