There is a very weird yet genuine issue I have come across on PItoPI Interface nodes.
Brief Overview: PItoPI gets data from a source PI Collective. The PItoPI is configured in a way that node1 is having Primary PI Server as the source and node2 has Secondary PI Server as the source. This PI to PI has warm failover configured.
Issue: The interfaces are undergoing frequent failovers with below logs-
"1. Our communication with PI is good; Our device status is equal to primary. Transition from Backup to Primary based on device status and PI status.
2. Interface in the "Primary" state; "Backup" interface available with communication error to PI
3. Interface transitioning from Primary to Backup; based on activeID value and good heartbeat value from synchronization file
4. Our communication with PI is good;Our device status is equal to primary"
Seems like interfaces are taking it on their ego to become the primary one to send data to PI and not relaxing enough.
After checking I was not able to find any communication drops between PItoPI and PI. Still failover keeps on happening very frequently and we have these logs on a repeat mode.
Please suggest what might be causing this and what can be done here.