Does this choice influence the performance of the system regarding the search for specific subset of data? If, for example, all needed data is organized into a single
AFElement, does it facilitate a faster search?
A faster search for what? I would assume finding the single AF Element is fast but searching within the AF Attributes may take longer as opposed to having Attributes distributed within a structure of AF Elements.
I however would suggest designing AF structure according to what makes sense to users or with other words best reflects the existing asset hierarchy rather than with regards to performance.
thank you for your answer. The search is for AF Atrributes containing specific data grouped into AF Elements.
However, intended users are other modules in the system which need to, e.g. generate reports based on available data.
I would suggest the emphasis of organization should be more by what makes sense in a plant hierarchy, and less about what makes a faster search, which could lead to an awkward, less clear organization,
Personally, I organize by parts of a system where data is related, such as belonging to the same piece of equipment aka asset. That equipment has data coming from different sources, which I can track by assigning an attribute category to identify the source. I can now search on the entire piece of equipment, namely the element itself, or can filter additionally on the attribute category to narrow down to those things belonging to a given source.
If possible, try to use templates. That will make searching really fast, even it big hierarchies. Also, you can have attributes "templates" repeated in a group of templates and do searches for them. You can have the best of both worlds if you lay out your system in a logical way for humans to use while incorporating templates for fast searching.
Above are very good suggestions. To fully utilize the searching functionality heavy use of the different types of categories (element, event frame, and attribute) will aid in narrowing down the search results. Also, element templates allow specific types of assets to be found quickly as well. And for elements, element types (boundary, node, flow, measurement, other and none) are also available in the element search to reduce the returned results.
Be careful you don't create excessively deep hierarchies. There's no hard limit but if you get too deep, performance may be impacted. For example, you probably don't want 10 levels deep hierarchies. It's best to test and compare.
Be careful you don't create excessively deep hierarchies. There's no hard limit but if you get too deep, performance may be impacted. For example, you probably don't want 10 levels deep hierarchies. It's best to test and compare.I need to clarify this. Depth of hierarchy does not necessarily affect search, especially with the upcoming AF 2.6 which has some significant search improvement, but may impact "AF Asset Modeling". Again, it's best to test and compare. Sorry for the confusion.
I typically create multiple hierarchies. One primary hierarchy, which is the 'owner' of the assets,and is mainly concerned with Asset Identification and decomposition from toplevel (e.g. Plant) down to the individual assets. Then i have a number of secondary models, used in navigation, or in building other functional compositions of assets as that are needed.
An example of a secondary hierarchy could be a hierarchy by role in the production process, where one facilty can reoccur inmultiple locations for multiple functions.
Hope this helps and does not confuse you.
I would recommend avoiding circular element references as they create trouble for a new feature in AF 2.6.
All right, I'll chime in here as well. Lots of experienced AF modellers here
I always start with an "Equipment repository" element, where I create one grouping element for each asset type - before I create all the assets below those elements. Advantage: assets are created in one location only (easy to maintain), and naturally sorted by their types (easy to browse by type, regardless of their physical/hierarchical locations)
After this is done, I start with the different hierarchies I need. This would be set up to resemble how the client organization is used to look at their hierarchical data, and contain references into the "Equipment repository". As an example, a T&D client of ours (the transmission system operator of Finland) has two main hierarchies since they have two main tasks in their day to day work. The first is the hierarchy showing all their assets organized by region, substation, equipment type and voltage level. The second shows their reserve power plants sorted by their locations around the county.
If you want, you can take a look at the presentation I did together with the client, in the T&D Industry event at the EMEA Users conf. in Paris last September: www.osisoft.com/.../item-abstract.aspx
You'll see examples from the AF hierarchy in there.
Also, as mentioned by others, plan your hierachy well using templates. And derived templates.
I usually have one "Equipment base template" which all other asset templates inherit from. Even though it's empty from the start, I like to have it there just to have a simple way of adding new references to all assets at a later time. The more you fiddle around perfecting the templates in the beginning, the more time you'll save later when you need to add more elements.
Thanks for the warning about circular references!
But you mainly make me curious about the new feature, and which good reasons there are not to allow circular references (although i have a hunch...datapipes?). In the end, in any larger AFmodel it's difficult to avoid them.
Sounds to me as a typical recursion problem, where an end needs to be defined (both technical and functional) to provide an emergency stop to the recursion.
- can i add abstract templates and multiple enheritance to the list of features for AF 3.4? ;-)
- i always find it hard to find a good reason to allow extensions to templates, except for manually managed AF hierarchies with a low level of connectivity to layered applications. Then you just don't rely on templates too much.
You are correct, it is a recursion problem. Prior to AF 2.6, anytime an element was referenced by it's path, the database logic had to compute the path on the fly. This technique times out, with only a few thousand elements or with concurrent requests. In AF 2.6, the database caches the path(s) to an element in a table that is indexed on the path (or at least the first ~420 characters of the path). Once the cache is up to date, it is very quick to find an element by it's path. When upgrading an existing AF system to v2.6, the upgrade logic will calculate and cache these paths, which can take some time on large systems. (I talked about this "feature" in the vCampus Live presentation with Chris Manhard.)
There are several OSIsoft client tools that prefer referencing AFElements by "path" rather than "ID" (primary key in database speak), which will experience performance improvement.
I don't know what features will be part of AF v2.7, so AF v2.7++ is completely undefined, at least to me.
What is your definition of "abstract templates" vs. "multiple inheritance"? I want to make sure we have the same definition. Please describe your use case for both.
An AFElementTemplate that cannot be instantiated into an AFelement. This i typically use for the top-levels of the Inheritance chain, where only the lower levels of AFelements are meant to be instantiated. To quote Asle Frantzen: ""Equipment base template" which all other asset templates inherit from"
Let's look at the Pump vs compressor example. Both we have in a electrical driven and engine driven versions. This gives 4 combinations on the lowest level: electrical pump, engine pump, electrical compressor, engine compressor. I now typically choose the most dominant (most AFattributes and complexity) chain as the AFtemplates: build a compressor template, and a pump template. Ideally, i want to inherit the electrical drive and engine drive template attributes too. But currently,i have to copy these sets of attributes across an inherited set of templates.
Thus i would want to build an AFElementTemplate that inherits from both Pump/Compressor and Electric/Engine.
A bit messy explanation, but i think this describes the case.
I'm trying to understand the issue and implications of what you wrote about the path construction. In my head, the Unique Primary key of an AFelement is the ID. A secondary (functional) Unique Key is the combination of the ElementPath and ElementName. As ElementNames must be unique in a single path, but the same name can be used in multiple paths. An Element does not have a parent, but an Element has a list of Child Elements.
For any given Element, multiple paths exist, as an Element can be referenced from multiple locations (even without recursion). To create a path from an Element, the logic needs to find all elements where the Element is a child Element. I don't yet see how circular references can cause an issue in building the path.
Let's make an example Hierarchy, where each next number is one level down / a child element:
- Element A
- Element B
- Element A
So let's build the Paths:
This resolves to two paths for Element A: \ and \A\B. Resolving these should not be a problem in a recursion, but it does need some attention to avoid looping the circle.
Or is there something i overlook? I think you guys have giving this tons more though than i have...
An AFElement can have a parent, although in the database it is labeled "fkownerid".
Don't forget AFDatabase.ID, EffectiveDate & ObsoleteDate as these are also a part of the effective Path.
All AFElements have a primary key which is exposed as UniqueID and is immutable within the database. AF Server can quickly retrieve an element from the database by it's UniqueID.
For the method AFElement.FindElementsByPath(), the database logic uses the path and querydate to lookup the UniqueID.
Assume the following hierarchy of elements with a single version each:
A has a child B, has a child C, has a child Z.
The paths would be:
Now, add a reference from Z to A
The new path would be:
You can build this in PSE and it will allow you to traverse down the tree for a long time.
At the database logic level, this creates an interesting challenge to compute the set of non-circular paths to each element (don't forget EffectiveDate and ObsoleteDate, which make things more interesting).
We (the AFTeam) accumulate customer provided databases for reproducing bugs and regression testing. I have one SQL database that is the combination of several customer databases. Though the combined database is not large (~150K elements, there are ~400K element references, about 250K of the references are circular). In a set operation, I can prove in a few seconds that at least one circular reference exists. On my test hardware (Dell T7600), It takes about 300 ms to test each element reference to see if it is a circular reference, which means it will take over 24 hours to test all the references in this database and upgrade this database from a previous AF version to v2.6.
Since this path cache table has an index on the computed path, the index-able path is effectively limited to less than 450 UNICODE characters. (SQL Server indexes are limited to 900 bytes.) AF stores strings as UNICODE, which leaves only 450 UNICODE characters for an index key, though allowing for DatabaseID and EffectiveDate and ObsoleteDate consumes more bytes. This means that if the average length of your element names is 20 UNICODE characters, you can have a path that is nested about 20 levels deep, which is good, since we use 25 levels as the maximum depth for recursion trapping.
Pre-computing all allowable paths and caching in an indexed table, has significant positive impact on AFElement.FindElementsByPath( ).
Abstract & composite templates have been on the backlog for a while, right Steve? 2.7 you said, right?