VIP Configuration Enhancements

Multiple virtual IP (VIP) groups can be configured for use by the applications being deployed in the K8s cluster. SMI’s cluster deployer logic has been enhanced to check if any IPv4 or IPv6 VIP address has been assigned to more than one VIP group. If the same VIP address has been assigned to multiple VIP groups, the deployment configuration validation will fail.

The following is a sample erroneous VIP groups configuration and a sample of the resulting error message logged through the validation:

Erroneous VIP Configurations and Sample Error Messages

Example Erroneous keepalived Configuration

Example Error Message

show running-config clusters tb1-smi-blr-c3 virtual-ips 
clusters tb1-smi-blr-c3
virtual-ips rep2
vrrp-interface ens224
vrrp-router-id 188
ipv4-addresses 192.168.139.85
mask 24
broadcast 192.168.139.255
device ens224
exit
ipv4-addresses 192.168.139.95
mask 24
broadcast 192.168.139.255
device ens256
exit
hosts controlplane2
priority 99
exit
hosts controlplane3
priority 100
exit
exit
virtual-ips rep3
vrrp-interface ens224
vrrp-router-id 189
ipv4-addresses 192.168.139.85
mask 24
broadcast 192.168.139.255
device ens224
exit

Manual validation:

clusters tb1-smi-blr-c3 actions validate-config run 

2021-04-27 15:21:45.967 ERROR __main__: Duplicate not allowed: ipv4-addresses 192.168.139.85 is assigned across multiple virtual-ips groups
2021-04-27 15:21:45.968 ERROR __main__: virtual-ips groups with same ip-addresses are rep3 and rep2
2021-04-27 15:21:45.968 ERROR __main__: Checks failed in the cluster tb1-smi-blr-c3 are:
2021-04-27 15:21:45.968 ERROR __main__: Check: ntp failed.
2021-04-27 15:21:45.968 ERROR __main__: Check: k8s-node-checks failed.
2021-04-27 15:21:45.968 ERROR __main__: Check: vip-checks failed.
Auto-Validation actions sync run:
clusters tb1-smi-blr-c3 actions sync run 
This will run sync. Are you sure? [no,yes] yes

message Validation errors occurred:
Error: An error occurred validating SSH private key for cluster: tb1-smi-blr-c3
Error: An error occurred validating node proxy for cluster: tb1-smi-blr-c3
Error: An error occurred validating node oam label config for cluster: tb1-smi-blr-c3

The keepalived_config container monitors the configmap vip-config for any changes at regular intervals and if a change is detected the keepalived configuration file is reloaded.

With this enhancement, either all or none of the VIP addresses configured in a VIP group must be present on a node. If only some of the addresses exist on the node, that keepalived process wil be stopped and a new process is automatically started and apply the latest configuration. This ensures that the keepalived processes assign those IP addresses appropriately.

The following is an example of the resulting error message logged through the validation:

kubectl logs keepalived-zqlzp -n smi-vips -c keepalived-config --tail 50 --follow 

container
INFO:root:group name :rep2
INFO:root:Ip address: 192.168.139.85 on interface ens224 found on this device: True
INFO:root:Ip address: 192.168.139.95 on interface ens256 found on this device: False
INFO:root:Error Occurred: All VIPs in /config/keepalived.yaml must be either present or absent in this device
INFO:root:VIP Split brain Scenario: Restarting the keepalived process.

Monitoring Virtual IPs for Multiple Ports

SMI Cluster Deployer supports monitoring the Virtual IP for a single port using the check-port command.


virtual-ips rep2
	check-port 25
	vrrp-interface ens224
	vrrp-router-id 188
	check-interface ens256
exit

Now, the cluster deployer is enhanced to monitor the VIP for multiple ports.

For multiple ports, use check-ports command:


virtual-ips rep2
	check-ports [ 25 80 43 65]
	vrrp-interface ens224
	vrrp-router-id 188
	check-interface ens256
exit
Note

Use either check-port or check-ports during configuration, but not both.