Hello everyone,
I'm actually having some performance issues about updating a high volume of attributes (not tags).
I would like to know what is the reasonable limit about the number of AFValue to send to UpdateValues ?
Right now I have around 1 200 000 attributes value to update at once (it may not be optimal but this is a client demand).
I'm doing 5 different AFListData.UpdateValues in different threads to increase the speed, but I'm getting kicked by the timeout before the update can end (5 minutes timeout).
Let's imagine I can't cut on the number of attributes, and I need to achieve the fastest update possible. What would be the optimal solution ?
Thank you
Hey Jason,
Frequently writing to a lot of static attributes (attributes without a data reference) isn't a great idea.
There is a table in the SQL backend that keeps track of all changes in the AF Server. This table is named FindChanges_AT.
Every OSIsoft AF Client (PI Vision, Asset Analytics, Notifications, etc...) leverages the FindChanges method in the AF SDK to identify what changed in the AF Server in a given time interval.
Now depending on the version the behavior is a little different.
Prior to 2017, 1 change to a static attribute = 1 entry in the FindChanges_AT table. If the table goes above 1M records, you may run into performance issues with ALL OSIsoft AF Clients. The table retains 7 days worth of changes by default.
After 2017, 1 change to a static attribute = 1 entry in the FindChanges_AT table, however we have a SPROC that will go and remove "duplicate entries" on a certain interval (I can't remember off the top of my head the interval time). An entry is considered duplicate if it is the same type of change on the same object. So multiple changes (ValueUpdate) to the same static attribute, after the time interval the SPROC runs on, will equal 1 change only.
If your customer is running a version prior to 2017, this is a VERY VERY bad idea.
If they are running 2017, we have a known issue with the new SPROC we introduced and the rate at which duplicate items are cleaned out is smaller by at least 2 orders of magnitude than in 2017 SP2.
If they are running 2017 R2 or later, you can get away with more frequent writes to static attributes. However, if there are more than 1M unique changes in the FindChanges_AT table... you will run into performance issues. If this customer uses AF a lot (especially event frames), 1M unique changes is pretty easy to reach in 7 days IF there are an excessive amount of numerous static attribute updates.
I think each situation is unique, but from my experience, if I were a customer, I would use more tags in this type of situation. It is an additional up front cost, but if you end up spending a ton of time troubleshooting system wide performance issues (that may be impacting production) and determine you need to use those tags, the cost is so much more.... It's not obvious if the first effects manifest in PI Vision or Asset Analytics...
My rule of thumb, time series data goes in the PI Data Archive. It's super highly performant for this type of data.
Relational data (metadata) goes in AF.
Thanks,
Seb