13 Replies Latest reply on Oct 20, 2012 5:00 AM by mhalhead

    AF to AF interface

      When will it be finished?  And will it be bi-directional, so AF A is replicating to AF B, but if AF B has changes it will push those back to AF A with a message that AF B sent the update thus AF A doesn't need to send to AF B.  There could of course be an AF C and AF D.  SSB for AF.  Yay or nay?

        • Re: AF to AF interface
          skwan

          Rhys:

           

          We're targeting AF 2.6 to provide AF2AF functionalities in 2013.  At the moment we have not finalized the scope for AF 2.6 but I can tell you that for AF2AF v1, it will be uni-directional.  That doesn't mean you cannot have multiple AF2AF running, but we will not be supporting bi-directional synchronization in v1.  As to SSB for AF, that's probably a different question than AF2AF but at the moment, that's not in scope for AF 2.6.

            • Re: AF to AF interface

              Thanks for the update Steve.

               

              So v1 is really all about pushing a hierarchy from a master to subscribers? Will it push configured analytics attributes?

                • Re: AF to AF interface
                  skwan

                  Rhys:

                   

                  I wouldn't exactly call this a "push", but rather I would describe this as having a source and a destination and the flow is one way.  As to your question on configured analytics attributes, are you asking about the inputs or the outputs?

                    • Re: AF to AF interface
                      mhalhead

                      Hi Steve,

                       

                      Any information on how the template libraries will be handled?

                       

                      We spoke awhile ago regarding AF-to-AF. Since then we've built a small interim solution that work pretty well but does have some issues. The original design was from a single central AF with the element hierarchies being distributed to the operations. The libraries are also replicated; however, these go through a transformation because there is a slight difference in the PI Point configuration on the central server to the site server (PI-to-PI).

                       

                      Since we've built this I've come to the conclusion that we made a mistaken in going central to operations. A better option would have been to make the distributed one the replicate to the central one. But the libraries should be managed centrally.

                        • Re: AF to AF interface
                          pcombellick

                          Michael,

                           

                          Are you suggesting that you would copy the entire central AF database to each plant ?

                           

                          What sort of transformation rules for PI Point datareferences do you require?

                           

                          Do you require synchronization support for AFElements or EventFrames?

                           

                          Do you require synchronization identical metadata to multiple destinations?

                           

                          Do you require metadata or data transfer outside of your company, such as with partner or vendor companies or government agencies?

                           

                          How frequently do you schedule synchronization?

                           

                          Regards,

                           

                          Paul Combellick

                           

                          AF Dev

                            • Re: AF to AF interface
                              mhalhead

                              Hi Paul,

                               

                              No we currently only replicate the portion of the AF hierarchy that is relavent to the operation. So each site only has their portion and not everything. They do however, have a complete copy of all the templates. As I said earlier I would like to reverse this with the site replicating up to the central server. In this scenario a specific node in the hierarchy (and all it's children) will replicate to a specific node on the central AF. The template however, should still come from the central AF.

                               

                              The transformation rules were pretty simple. On the central AF we configured the template to create the tags and add them to the relevant PI-to-PI pointsource. Each PI point attribute has a child attribute that stores the PI Point name on the source PI; this is used for the configuration. When we replicate the libraries we apply a tranformation to the to the PI Point attributes (i.e. any attribute that has a PI Point DR) to alter this behaviour seeing that we don't want to recreate the points on the source system but simply read them. The transforms are "coded" in XSLT.

                               

                              We require replication of the AF Elements at this point. To be honest I haven't applied my mind to the EventFrames component yet; we are doing a number of other things with EF but replication isn't one of them. This may become a requirement in the medium term.

                               

                              Yes we do require replicating identical meta data to multiple destinations; the templates are the best example.

                               

                              We do require sending meta data and data outside of the company to various vendors. Currently this is a bit of a mess; I have more solutions to this than I have vendors. I've had a discussion with Laurent regarding the use of Azure for data transfer between companies. I'm waiting for this solution with baited breath.

                               

                              Currently we do a delta replication every hour; i.e. we only replicate the changes. The changes are determined by storing the last replication cookie. We then do a full replication once a day. The templates/tables and enums are only replicated on a daily basis. Why hourly? Well it just sounded like a good number.

                               

                              If you would like I can provide you with all the source code and the element templates? The code isn't great, there are some bugs in it that need fixing but to be honest I'm waiting for OSIsoft. The replication isn't our core business so I'm resisting the urge to spend to much time and/or money on it.

                               

                              You will notice that I've used the term replicate rather than synchronize. I've done this intentionally as the data flow is in a single direction. Currently if you make a change to the site hierachy or elements the changes will be overwritten; no warnings. i.e. the source is the master and the destination is forced to match this. We make no attempt to try and synchronise the hierarchies; this simpyl becomes too complex. This is one of the reasons I want to reverse the direction of the replications.

                                • Re: AF to AF interface
                                  pcombellick

                                  Michael,

                                   

                                  I agree with your use of the term replicate rather than synchronize.  AF to AF (PIAF Sync) supports moving specific AF objects in one direction, but supports replicating sets of AF objects from AFSystem A to AFSystem B and other AF objects from AFSystem B to AFSystem A.  Modifying a specific AF object on more than one AFSystem is not supported.  (No multi-master replication).

                                   

                                  You describe two interesting features that are not currently implemented or planned: (1) periodic "full" replications and (2) triggering of custom logic, which in your case is via XSL transformation.

                                    • Re: AF to AF interface

                                      I'm just about at the point where I need to start thinking about replicating Event Frames from small systems to a large aggregator.  We currently do this for PI Server data so 100's of systems can be analysed centrally from one spot.  AF hierarchy is going to be pushed from a master to each of the 100's of systems, but then events detected on each of the systems will need to come back up to the central system.

                                       

                                      Not given it anywhere near as much thought as Michael has, but I think we are tackling the same problem.  My hope has always been AF to AF would become an offering quickly to tackle these scenarios.  Now that we have talked about it on here we can keep pushing until it is delivered

                                       

                                      Will AF to AF work over port 5457?  Be interesting to see some early testing of AF to AF over high latency, low bandwidth connections...PI to PI testing is highly important over those connections, as is the push or pull stance.  

                                        • Re: AF to AF interface
                                          pcombellick

                                          Rhys,

                                           

                                          The on-premise version of "AF to AF" uses the AFSDK to communicate with AFServer(s).  As usual, you control the ports that the AF Server is listening on.

                                           

                                          Paul

                                            • Re: AF to AF interface
                                              skwan

                                              Sorry if this is a dupe as apparently hitting reply deleted my first post.

                                               

                                              So what is the desired behavior in a high latency/low bandwidth environment?  What if the network is so bad that before the replication is complete, someone made changes to the source or destination?  Should we lock the source and destination until the replication is complete?  (probably not, can't stop people from working...?)

                                          • Re: AF to AF interface
                                            mhalhead

                                            Paul Combellick

                                            You describe two interesting features that are not currently implemented or planned: (1) periodic "full" replications and (2) triggering of custom logic, which in your case is via XSL transformation.

                                             

                                            Paul of the two features I would say that the transformation is the most important to me. This was actually surprisingly easy to do seeing that we use the xml import/export features of the SDK. Although I must admit that writing XSLT is not fun, I don't do it often enough. An alternative would be to maintain two versions of the library; a central server version and a site server version. There is some benefit to this approach but it does mean double work when making changes.

                                             

                                            The periodic full replication was introduced to handle the scenario where elements were added to the destination system. If you just update with changes from the source you can landed up in a situation where there are additional elements in the destination that aren't in the source. This only impacted on the replication root node in the hierarchy and its children. Anything added outside of that root node is not affected.

                                              • Re: AF to AF interface
                                                pcombellick

                                                Michael,

                                                 

                                                Does your full replication delete the target element tree?  

                                                 

                                                I understand the need for PI Point DR config string mapping.  We have several algorithms under consideration:

                                                 

                                                (0) do nothing

                                                 

                                                (1) change PI Point DR config string to something like %server%\%element%.%attribute%

                                                 

                                                (2) use a mapping table

                                                 

                                                (3) on destination system, set tag name = Attribute Guid from source AF System.

                                                 

                                                I expect that other DR, including linked AFTable have a similar issue when they reference external systems.

                                                 

                                                Replicating custom DR assemblies is not planned.

                                                 

                                                Regards,

                                                 

                                                Paul

                                                  • Re: AF to AF interface
                                                    mhalhead

                                                    Hi Paul,

                                                     

                                                    No the full replication doesn't delete the target tree; if you did you would run the risk of the Element IDs changing and things like notifications and EF wouldn't be happy with this.

                                                     

                                                    For the DR transformation we use a combination of 0 and 1; but because we are using XSLT we have a lot more flexibility. Currently we don't transform any of the other DRs but there is no technical reason we can't. The assemblies do have to be deployed "manually"; We have a simple .bat script that does this.

                                                     

                                                    Regarding Table. Our solution to this is that we replicate the tables to the target system.