17 Replies Latest reply on Feb 27, 2014 1:48 PM by pcombellick

    AF Asset Modeling

    veljovic

      Hi all,

       

       

       

      I have a general question regarding modeling with Asset Framework: I want to model a complex system with AF and there are many ways to organaze data into

       

      meaningful AF building blocks(e.g. I can organize data w.r.t. parts of the system to which the data is related, or from which measuring place tha data is coming, etc.).

       

      Does this choice influence the performance of the system regarding the search for specific subset of data? If, for example,  all needed data is organized into a single

       

      AFElement, does it facilitate a faster search?

       

       

       

      Regards,

       

      Slobodan

        • Re: AF Asset Modeling

          Hello Slobodan,

           

          Slobodan Veljovic

          Does this choice influence the performance of the system regarding the search for specific subset of data? If, for example,  all needed data is organized into a single

           

          AFElement, does it facilitate a faster search?

           

          A faster search for what? I would assume finding the single AF Element is fast but searching within the AF Attributes may take longer as opposed to having Attributes distributed within a structure of AF Elements.

           

          I however would suggest designing AF structure according to what makes sense to users or with other words best reflects the existing asset hierarchy rather than with regards to performance.

            • Re: AF Asset Modeling
              veljovic

              Hi Gregor,

               

               

               

              thank you for your answer. The search is for AF Atrributes containing specific data grouped into AF Elements.

               

              However, intended users are other modules in the system which need to, e.g.  generate reports based on available data.

               

               

               

              Regards,

               

              Slobodan

                • Re: AF Asset Modeling
                  Rick Davin

                  I would suggest the emphasis of organization should be more by what makes sense in a plant hierarchy, and less about what makes a faster search, which could lead to an awkward, less clear organization,

                   

                  Personally, I organize by parts of a system where data is related, such as belonging to the same piece of equipment aka asset.  That equipment has data coming from different sources, which I can track by assigning an attribute category to identify the source.  I can now search on the entire piece of equipment, namely the element itself, or can filter additionally on the attribute category to narrow down to those things belonging to a given source.

                    • Re: AF Asset Modeling
                      Lonnie Bowling

                      If possible, try to use templates. That will make searching really fast, even it big hierarchies. Also, you can have attributes "templates" repeated in a group of templates and do searches for them. You can have the best of both worlds if you lay out your system in a logical way for humans to use while incorporating templates for fast searching.

                       

                      Lonnie

                        • Re: AF Asset Modeling
                          tseiferth

                          Above are very good suggestions. To fully utilize the searching functionality heavy use of the different types of categories (element, event frame, and attribute) will aid in narrowing down the search results. Also, element templates allow specific types of assets to be found quickly as well. And for elements, element types (boundary, node, flow, measurement, other and none)  are also available in the element search to reduce the returned results.

                            • Re: AF Asset Modeling
                              skwan

                              Be careful you don't create excessively deep hierarchies.  There's no hard limit but if you get too deep, performance may be impacted.  For example, you probably don't want 10 levels deep hierarchies.  It's best to test and compare.

                                • Re: AF Asset Modeling
                                  skwan

                                  Steve Kwan

                                  Be careful you don't create excessively deep hierarchies.  There's no hard limit but if you get too deep, performance may be impacted.  For example, you probably don't want 10 levels deep hierarchies.  It's best to test and compare.

                                   
                                  I need to clarify this.  Depth of hierarchy does not necessarily affect search, especially with the upcoming AF 2.6 which has some significant search improvement, but may impact "AF Asset Modeling".  Again, it's best to test and compare.  Sorry for the confusion.
                                    • Re: AF Asset Modeling
                                      Roger Palmen

                                      I typically create multiple hierarchies. One primary hierarchy, which is the 'owner' of the assets,and is mainly concerned with Asset Identification and decomposition from toplevel (e.g. Plant) down to the individual assets. Then i have a number of secondary models, used in navigation, or in building other functional compositions of assets as that are needed.

                                       

                                      An example of a secondary hierarchy could be a hierarchy by role in the production process, where one facilty can reoccur inmultiple locations for multiple functions.

                                       

                                      Hope this helps and does not confuse you.

                                        • Re: AF Asset Modeling
                                          pcombellick

                                          I would recommend avoiding circular element references as they create trouble for a new feature in AF 2.6.

                                           

                                          Regards,

                                           

                                          Paul

                                           

                                          AF Dev

                                            • Re: AF Asset Modeling
                                              Asle Frantzen

                                              All right, I'll chime in here as well. Lots of experienced AF modellers here

                                               

                                              I always start with an "Equipment repository" element, where I create one grouping element for each asset type - before I create all the assets below those elements. Advantage: assets are created in one location only (easy to maintain), and naturally sorted by their types (easy to browse by type, regardless of their physical/hierarchical locations)

                                               

                                              After this is done, I start with the different hierarchies I need. This would be set up to resemble how the client organization is used to look at their hierarchical data, and contain references into the "Equipment repository". As an example, a T&D client of ours (the transmission system operator of Finland) has two main hierarchies since they have two main tasks in their day to day work. The first is the hierarchy showing all their assets organized by region, substation, equipment type and voltage level. The second shows their reserve power plants sorted by their locations around the county.

                                               

                                              If you want, you can take a look at the presentation I did together with the client, in the T&D Industry event at the EMEA Users conf. in Paris last September: www.osisoft.com/.../item-abstract.aspx

                                               

                                              You'll see examples from the AF hierarchy in there.

                                               

                                              Also, as mentioned by others, plan your hierachy well using templates. And derived templates.

                                               

                                              I usually have one "Equipment base template" which all other asset templates inherit from. Even though it's empty from the start, I like to have it there just to have a simple way of adding new references to all assets at a later time. The more you fiddle around perfecting the templates in the beginning, the more time you'll save later when you need to add more elements.

                                                • Re: AF Asset Modeling
                                                  Roger Palmen

                                                  Paul,

                                                   

                                                  Thanks for the warning about circular references!

                                                   

                                                  But you mainly make me curious about the new feature, and which good reasons there are not to allow circular references (although i have a hunch...datapipes?). In the end, in any larger AFmodel it's difficult to avoid them.

                                                   

                                                  Sounds to me as a typical recursion problem, where an end needs to be defined (both technical and functional) to provide an emergency stop to the recursion.

                                                   

                                                  About templates:

                                                   

                                                  - can i add abstract templates and multiple enheritance to the list of features for AF 3.4? ;-)

                                                   

                                                  - i always find it hard to find a good reason to allow extensions to templates, except for manually managed AF hierarchies with a low level of connectivity to layered applications. Then you just don't rely on templates too much.

                                                    • Re: AF Asset Modeling
                                                      pcombellick

                                                      Roger,

                                                       

                                                      You are correct, it is a recursion problem.  Prior to AF 2.6, anytime an element was referenced by it's path, the database logic had to compute the path on the fly.  This technique times out, with only a few thousand elements or with concurrent requests.  In AF 2.6, the database caches the path(s) to an element in a table that is indexed on the path (or at least the first ~420 characters of the path).  Once the cache is up to date, it is very quick to find an element by it's path.  When upgrading an existing AF system to v2.6, the upgrade logic will calculate and cache these paths, which can take some time on large systems.  (I talked about this "feature" in the vCampus Live presentation with Chris Manhard.)

                                                       

                                                      There are several OSIsoft client tools that prefer referencing AFElements by "path" rather than "ID" (primary key in database speak), which will experience performance improvement.

                                                       

                                                      I don't know what features will be part of AF v2.7, so AF v2.7++ is completely undefined, at least to me.

                                                       

                                                      Regards,

                                                       

                                                      Paul

                                                        • Re: AF Asset Modeling
                                                          skwan

                                                          Roger:

                                                           

                                                          What is your definition of "abstract templates" vs. "multiple inheritance"?  I want to make sure we have the same definition.  Please describe your use case for both.

                                                            • Re: AF Asset Modeling
                                                              Roger Palmen

                                                              Hi Steve,

                                                               

                                                              Abstract:

                                                               

                                                              An AFElementTemplate that cannot be instantiated into an AFelement. This i typically use for the top-levels of the Inheritance chain, where only the lower levels of AFelements are meant to be instantiated. To quote Asle Frantzen: ""Equipment base template" which all other asset templates inherit from"

                                                               

                                                              Multiple Inheritance:

                                                               

                                                              Let's look at the Pump vs compressor example. Both we have in a electrical driven and engine driven versions. This gives 4 combinations on the lowest level: electrical pump, engine pump, electrical compressor, engine compressor. I now typically choose the most dominant (most AFattributes and complexity) chain as the AFtemplates: build a compressor template, and a pump template. Ideally, i want to inherit the electrical drive and engine drive template attributes too. But currently,i have to copy these sets of attributes across an inherited set of templates.

                                                               

                                                              Thus i would want to build an AFElementTemplate that inherits from both Pump/Compressor and Electric/Engine.

                                                               

                                                              A bit messy explanation, but i think this describes the case.

                                                                • Re: AF Asset Modeling
                                                                  Roger Palmen

                                                                  Hi Paul,

                                                                   

                                                                  I'm trying to understand the issue and implications of what you wrote about the path construction. In my head, the Unique Primary key of an AFelement is the ID. A secondary (functional) Unique Key is the combination of the ElementPath and ElementName. As ElementNames must be unique in a single path, but the same name can be used in multiple paths. An Element does not have a parent, but an Element has a list of Child Elements.

                                                                   

                                                                  For any given Element, multiple paths exist, as an Element can be referenced from multiple locations (even without recursion). To create a path from an Element, the logic needs to find all elements where the Element is a child Element. I don't yet see how circular references can cause an issue in building the path.

                                                                   

                                                                  Let's make an example Hierarchy, where each next number is one level down / a child element:

                                                                  1. Root
                                                                  2. Element A
                                                                  3. Element B
                                                                  4. Element A

                                                                  So let's build the Paths:

                                                                  1. n/a
                                                                  2. \
                                                                  3. \A
                                                                  4. \A\B

                                                                  This resolves to two paths for Element A: \ and \A\B. Resolving these should not be a problem in a recursion, but it does need some attention to avoid looping the circle.

                                                                   

                                                                  Or is there something i overlook? I think you guys have giving this tons more though than i have...

                                                                    • Re: AF Asset Modeling
                                                                      pcombellick

                                                                      Roger,

                                                                       

                                                                      An AFElement can have a parent, although in the database it is labeled "fkownerid".

                                                                       

                                                                      Don't forget AFDatabase.ID, EffectiveDate & ObsoleteDate as these are also a part of the effective Path.

                                                                       

                                                                      All AFElements have a primary key which is exposed as UniqueID and is immutable within the database.  AF Server can quickly retrieve an element from the database by it's UniqueID.

                                                                       

                                                                      For the method AFElement.FindElementsByPath(), the database logic uses the path and querydate to lookup the UniqueID.

                                                                       

                                                                      Assume the following hierarchy of elements with a single version each:

                                                                       

                                                                      A has a child B, has a child C, has a child Z.

                                                                       

                                                                      The paths would be:

                                                                       

                                                                      \A

                                                                       

                                                                      \A\B

                                                                       

                                                                      \A\B\C

                                                                       

                                                                      \A\B\C\Z

                                                                       

                                                                      Now, add a reference from Z to A

                                                                       

                                                                      The new path would be:

                                                                       

                                                                      \A\B\C\Z\A

                                                                       

                                                                      You can build this in PSE and it will allow you to traverse down the tree for a long time.

                                                                       

                                                                      At the database logic level, this creates an interesting challenge to compute the set of non-circular paths to each element (don't forget EffectiveDate and ObsoleteDate, which make things more interesting).

                                                                       

                                                                      We (the AFTeam) accumulate customer provided databases for reproducing bugs and regression testing.  I have one SQL database that is the combination of several customer databases.  Though the combined database is not large (~150K elements, there are ~400K element references, about 250K of the references are circular).  In a set operation, I can prove in a few seconds that at least one circular reference exists.  On my test hardware (Dell T7600), It takes about 300 ms to test each element reference to see if it is a circular reference, which means it will take over 24 hours to test all the references in this database and upgrade this database from a previous AF version to v2.6.

                                                                       

                                                                      Since this path cache table has an index on the computed path, the index-able path is effectively limited to less than 450 UNICODE characters.  (SQL Server indexes are limited to 900 bytes.)  AF stores strings as UNICODE, which leaves only 450 UNICODE characters for an index key, though allowing for DatabaseID and EffectiveDate and ObsoleteDate consumes more bytes.  This means that if the average length of your element names is 20 UNICODE characters, you can have a path that is nested about 20 levels deep, which is good, since we use 25 levels as the maximum depth for recursion trapping.

                                                                       

                                                                      Pre-computing all allowable paths and caching in an indexed table, has significant positive impact on AFElement.FindElementsByPath( ).

                                                                       

                                                                      Regards,

                                                                       

                                                                      Paul

                                                                    • Re: AF Asset Modeling
                                                                      Rhys Kirk

                                                                      Abstract & composite templates have been on the backlog for a while, right Steve? 2.7 you said, right?