10 Replies Latest reply on Jan 12, 2019 6:37 PM by arosenthal

    AF Hierarchies comparison between environments

    srikanth.nadikatla

      Hi ALL,

       

      We have cases where we want to compare PI AF hierarchies from two different environments(elements as well as their attributes). We do have some solutions in place, one way we extract hierarchies from source and destination AF environment using excel PI Builder add-in, and use MS excel/access queries to compare and get the results, other way is we have powershell script which takes paths from source and destination AF environments as input and automatically extracts elements, compares and generates the results. The excel based solution is fast but involves lot of manual work, taking extracts, running queries etc.. powershell solution is fully automated but performance is bad (especially hierarchy extractions). I am wondering if you guys can suggest an optimal way to handle this case with reasonable automation and  performance. Mind you, the hierarchies we want to compare can be huge, as much as 10000 elements (with roughly around 10 attributes per element).

       

      Regards,

      Srikanth

        • Re: AF Hierarchies comparison between environments
          gregor

          Hi Srikanth,

           

          Is your PowerShell script using OSIsoft.PowerShell module or another Developer Technology like AF SDK or PI Web API. Can you tell which operations take the majority of time and share a representative code snippet? There may be potential for improvement.  

            • Re: AF Hierarchies comparison between environments
              srikanth.nadikatla

              Hi Gregor, The Powershell script uses AF SDK not the OSISoft.Powershell module. Do you think there would be significant improvement if we use OSISoft.Powershell module?

                • Re: AF Hierarchies comparison between environments
                  gregor

                  Hi Srikanth,

                   

                  No, I don't expect a significant performance difference between OSIsoft.PowerShell and AF SDK. A performance similar to the one you get with PI Builder should be possible. My assumption is that your script does something in a pretty inefficient manner.

                    • Re: AF Hierarchies comparison between environments
                      srikanth.nadikatla

                      Hi Gregor, I agree there could be room for improvement in powershell script and in ideal case we might achieve PI Builder performance as well. However, It would still be a time consuming process to extract 10k elements and compare, whereas we just need those 100-150 elements which are different in both the environments. So, I am looking for some smart solution for this which can give an indication of the delta directly without one-to-one comparison. Perhaps, Some sort of queries on AF SQL database we can build to achieve this?

                        • Re: AF Hierarchies comparison between environments
                          gregor

                          Hello Srikanth,

                           

                          With your initial post you mention that you like to compare differences between 2 different PI AF Hierarchies. With other words, you have 2 different instances that do not know each other. While I agree it would be cool having a magic something that is able to report the differences between those 2 instances without comparing them, I wouldn't know such magic thing exists.

                           

                          There are options to monitor changes but this requires you to get a reliable status first and this again requires 2 compare the 2 different instances. Please note that direct access to the PIFD database is not supported.

                  • Re: AF Hierarchies comparison between environments
                    Roger Palmen

                    Hi,

                     

                    First off, 2000 elements with 100 attributes is not that much, so the scale of things should not be a problem.

                     

                    My first question would be: what information do you want to get from the comparison? Just that something is different? What exactly is different? What is needed (Create / Update / Delete) to make the systems equal?

                      • Re: AF Hierarchies comparison between environments
                        srikanth.nadikatla

                        Yes, What is different and information required to make them equal after a quick review. A report which gives the delta between the two environments, elements existing in source but missing in destination and vice versa, attributes which are different in source and destination.

                         

                        Also please note, I have modified question in terms of number of elements, this is the number after discussing with actual operations team!

                          • Re: AF Hierarchies comparison between environments
                            Roger Palmen

                            Just looking at the functional side of things, this is just a basic compare:

                            • Retrieve the data from both the source and target systems.
                            • Compare the source against the target: If exists in source but not target: create in target, If exists in target but not source: delete from target. Do this for both elements, attributes and values.

                             

                            A simple way to do a compare might be to have a look at the change date of the element. If the change data in the source is newer than the change date in the target then you need to update the list of attributes. This simplifid approach only works through if regular use of the AF structure does not cause the element to change (does not require checkin).

                        • Re: AF Hierarchies comparison between environments
                          Sidh

                          Hi,

                           

                          This all depends on what level of comparison you want, Element Vs Element can be tried out with SQL or XML comparison, later can be also efficient for a bulk comparison.

                          • Re: AF Hierarchies comparison between environments
                            arosenthal

                            I do this by comparing the XML exports of the two environments. If you look at the XML export, almost every node has a uniquely defined path based on the name of the node (AFElement, AFAttribute, etc.) and the <Name> or <id> nodes (there are some exceptions to this but generally it holds true). Using this knowledge, you could generate a unique path string for every node in each XML document, and then run a comparison to determine which nodes exist in both documents and which ones exist in only one or the other. For the nodes that are common in both documents, you can perform a comparison to see whether the contents of those nodes are identical or if they have differences. From there, it is up to you to decide how you want to report on the differences.

                            It isn't trivial but I find this approach works best because it is highly generalized without having to deal with every class type using AF SDK.