AnsweredAssumed Answered

PISDK - Slow Update response times using UpdateValues

Question asked by pauljohnson on Oct 25, 2018
Latest reply on Oct 25, 2018 by gregor

Hi there I'm currently experiencing an issue with progressively slow response times when writing historical data to our PI Data Archive.  When the interface starts processing a data file I'm getting about 7 updates per second, however a few minutes into the process this drops to 1 every 2 or 3 seconds.  Each data file comprises a 1008x307 array of data values (307 Tags x 1008 values).   Please note that due to Corporate restrictions and legacy software I am not able to use the AFSDK.

 

The following is trace output from a sample I ran this morning:

 

21|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~Frequency~HZ~S~AI10~
22|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~Frequency~HZ~S~AI10~ succesfully updated
23|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrFundamental~V~S~AI10~
24|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrFundamental~V~S~AI10~ succesfully updated
25|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrTHD~V~S~AI10~
26|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrTHD~V~S~AI10~ succesfully updated
27|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH2~V~S~AI10~
28|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH2~V~S~AI10~ succesfully updated
29|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH3~V~S~AI10~
30|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH3~V~S~AI10~ succesfully updated
31|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH4~V~S~AI10~
32|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH4~V~S~AI10~ succesfully updated
33|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH5~V~S~AI10~
34|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH5~V~S~AI10~ succesfully updated
35|10:11:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH6~V~S~AI10~
36|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH6~V~S~AI10~ succesfully updated
37|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH7~V~S~AI10~
38|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH7~V~S~AI10~ succesfully updated
39|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH8~V~S~AI10~
40|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH8~V~S~AI10~ succesfully updated
41|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH9~V~S~AI10~
42|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH9~V~S~AI10~ succesfully updated
43|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH10~V~S~AI10~
44|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH10~V~S~AI10~ succesfully updated
45|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH11~V~S~AI10~
46|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~VrH11~V~S~AI10~ succesfully updated
47|10:11:58|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~VrH12~V~S~AI10~

...

609|10:18:28|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH39~AMPS~S~AI10~
610|10:18:31|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH39~AMPS~S~AI10~ succesfully updated
611|10:18:31|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH40~AMPS~S~AI10~
612|10:18:33|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH40~AMPS~S~AI10~ succesfully updated
613|10:18:34|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH41~AMPS~S~AI10~
614|10:18:36|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH41~AMPS~S~AI10~ succesfully updated
615|10:18:37|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH42~AMPS~S~AI10~
616|10:18:39|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH42~AMPS~S~AI10~ succesfully updated
617|10:18:40|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH43~AMPS~S~AI10~
618|10:18:42|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH43~AMPS~S~AI10~ succesfully updated
619|10:18:43|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH44~AMPS~S~AI10~
620|10:18:45|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH44~AMPS~S~AI10~ succesfully updated
621|10:18:45|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH45~AMPS~S~AI10~
622|10:18:48|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH45~AMPS~S~AI10~ succesfully updated
623|10:18:49|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH46~AMPS~S~AI10~
624|10:18:51|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH46~AMPS~S~AI10~ succesfully updated
625|10:18:51|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH47~AMPS~S~AI10~
626|10:18:54|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH47~AMPS~S~AI10~ succesfully updated
627|10:18:54|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH48~AMPS~S~AI10~
628|10:18:56|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH48~AMPS~S~AI10~ succesfully updated
629|10:18:57|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH49~AMPS~S~AI10~
630|10:18:59|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH49~AMPS~S~AI10~ succesfully updated
631|10:18:59|EURO\TE602510|Warn|TopsBatchProcessor - Updating Tag 33~AREC3~WF1A~IbH50~AMPS~S~AI10~
632|10:19:02|EURO\TE602510|Warn|TopsBatchProcessor - Tag 33~AREC3~WF1A~IbH50~AMPS~S~AI10~ succesfully updated

 

 

Here is an excerpt of the code that carries out the updates:

 

 

    Do Until topsDevice.Data(_titleRow, colCnt) = ""
        'Get a new TOPSPoint object to store tag attributes outside the PIPoint object (requires iterating through all elements in order to read attributes for a single point)
        _topsPnt = New TopsPoint

        'Lookup the device datapoint in the instrumenttag attribute of the pi point list
        _dataPoint = deviceName & plantId & topsDevice.Data(_titleRow, colCnt).ToString.Replace(plantId, "")
        'Strip all spaces from the datapoint
        _dataPoint = _dataPoint.Replace(" ", "")

        _logger.SendLog("Finding InstrumentTag " & _dataPoint, NLog.LogLevel.Trace, Nothing)
        _topsPnt = FindPoint(_topsPnts, _dataPoint)
        _logger.SendLog("Found InstrumentTag " & _dataPoint, NLog.LogLevel.Trace, Nothing)
        If Not _topsPnt Is Nothing Then
            'Build the array of data values for writing to the archive
            _logger.SendLog("Instrument Tag '" & _dataPoint & "' FOUND in tag database as PointID " & _topsPnt.PointID.ToString & "/" & _topsPnt.TagName, NLog.LogLevel.Trace)
            piValCnt = 0
            For rowCnt = _dataPointRow To topsDevice.Data.GetUpperBound(0) - 1
                timeStamp = CDate(topsDevice.Data(rowCnt, _timeStampCol))
                topsValue.TimeStamp = timeStamp
                topsValue.PiValue = topsDevice.Data(rowCnt, colCnt).ToString.Trim
                piValues.Add(topsValue.TimeStamp, topsValue.PiValue, Nothing)
            Next
            _logger.SendLog("Updating Tag " & _topsPnt.TagName, NLog.LogLevel.Warn) '@@ Temporary code to be removed
            _piServer.PIPoints(_topsPnt.TagName).Data.UpdateValues(piValues, DataMergeConstants.dmReplaceDuplicates)
            _logger.SendLog("Tag " & _topsPnt.TagName & " succesfully updated", NLog.LogLevel.Warn) '@@ Temporary code to be removed
        Else
            _logger.SendLog("Instrument Tag '" & _dataPoint & "' NOT found in tag database.", NLog.LogLevel.Info, _fileCheckTimer, _fileCheckInterval, Nothing)
        End If
        colCnt = colCnt + 1
    Loop 'for the next datapoint

 


 

Can anyone explain what is happening here?

 

Kind Regards

Paul.

 

 

Outcomes