Separating ACE from the PI servers is a decidedly good start. ACE is designed to read data from PI using the SDK, and the standard SDK failover works in a much more straight forward fashion when remoted from the PI collective nodes.
Likewise for the ACE outputs. Standard n-way buffering is used to send calculation results to the collective nodes, and so you really want ACE to be on its own machines. Make sure to use the latest version of the PI Buffer Subsystem (and defiantly not the old Buffer Server).
It's worth noting that making ACE itself HA and using ACE with an HA PI Server Collective are independent tasks. We obviously recommend that you do both!
The only other serious considerations have to do with the way one writes ACE code. If you are using the built-in mechanisms to perform all of your reads and writes, then everything pretty much works smoothly. If, however, you create your own PI SDK connections and derived objects then you will have to manage all of the HA aspects yourself.
Pay particular attention to the ways in which MDB is used. MDB is only writeable on the Primary PI Server in the Collective. This has a few implications for how ACE deals with the Primary being down. You can seriously complicate this issue if your ACE code starts writing to MDB! As a best practice, ACE code should not write to MDB. If storage which persists across ACE shutdown/startup is needed - use PI tags, not MDB properties.
Thanks a lot for the directions, and for a very useful resource for others here on vCampus :)
I haven't written my ACE code with HA in mind yet, but I don't think I've done any of the no-no's either. So it should work as it is!
Hey Matt. I think you know some of our architecture here. We are actually running 3 ACE servers against our PI collective. One of them uses the MDB of the collective; the other two each have their own PI servers and are using their MDBs. Since PI and ACE are installed on the same server I'm stuck using bufserv on them. Is there any way around this other than having the PI and ACE on different servers?
The “local” PI server in this case is not a collective, so I think there’s a way that this could work. I’d be tempted to open a TS call on this so that they can walk you through the configuration (or tell you that I’ve lost my mind!)
On a related note, you probably would be better off getting that 3rd ACE instance setup like the other two. No sense in having one of them using the “real” MDB and the other two using their own. The upcoming AF<->MDB replication will make this much easier for you to manage by the way
Here's a couple of related setup tips.
1. Server/connection names.
Because ACE uses both PI-API and PI-SDK connections, you have to set these up right. Let's say you have a collective MyHA consisting of servers MyPI1 and MyPI2. When you first make an SDK connection from your ACE node to MyPI1, the SDK transforms the connection to a "collective" one called MyHA. The "Network Node" in the Connections dialog is the FQDN of whichever PI server you happen to be connected to, e.g. "MyPI1.myorg.co.uk". (This is unless you have the EnableHostNameForFQDN tuning parameter set to 1). This is the name that ACE will use to make PI-API connections for sending data. To get data replicated, the name must be in the BufferedServers list and the ReplicatedServers list in piclient.ini. Therefore, you need to have the API connection names MyPI1.myorg.co.uk and MyPI2.myorg.co.uk in pilogin.ini before you set up buffering/replication. You'll have to put these in by hand. (In case you were wondering, ACE gets the API connection name from the PIACEPoint object by creating a temporary SDK PIPoint object and extracting the server path from it).
Hint, use pidiag -dapi <name> to get the right server ID numbers to put in pilogin.ini.
2. Make ACE Scheduler dependent on pibufss
You probably want the ACE Scheduler to stop when the Pi Buffer Subsystem does, since any data it sends would be lost anyway. Any official OSI advice on this?
I don't know any OSI-approved way to do this. I always use the DOS sc (service configure) command:
sc \\localhost config PIACENetScheduler depend= pibufss
Note all spaces are significant especially the one after depend= , which is a bit counter-intuitive.
3. Control which server is used for PI-MD access (%OSI\ACEClassLibraries\...)
ACE uses the default SDK connection (which generally is the only connection) for this. You can override this with the following Registry string value:
Of course this doesn't affect actual ACE Contexts since they explicitly specify the PI Server/Collective (SDK connection).
All ACE components can modify the Module Database. If connecting to a collective, therefore, they will prefer the Primary node. If this is not available they will connect to a secondary but will not be able to modify the MDB. Therefore it is not possible to do the following when the primary is down:
- Wizard: Cannot create or modify settings for executables, or register them. You can edit the code and debug it.
- Manager: effectively read only. You cannot stop, start or change anything.
- Scheduler: continues to schedule but cannot stop or start anything or change status.
- Class Libraries: cannot write to MDB - just don't try it, OK? Trust me on this.
I can also explain how redundant ACE servers work if you're interested.