The topic of system latency has come up a couple of times in recent projects. If you really think about it, this is not surprising. The more manufacturing gets integrated, data have to be synchronized and\or orchestrated between different applications. Here are just some examples:

  1. MES: Manufacturing execution system typically connect to a variety of data sources, so the workflow developer needs to know timeout settings for different applications. Connections to the automation system will have a very low latency, but what is the expected latency of the historian?
  2. Analysis: More and more companies move towards real-time analytics. But how fast can you really expect calculations to be updated? This is especially true for Enterprise level systems, that are typically clones from source PI servers by way of PI-to-PI. So you are looking of a data flow of for example:

    Source -> PI Data Archive (local) -> PI-to-PI -> PI Data Archive (region) -> PI-to-PI -> PI Data Archive (enterprise)

    and latency in each steep.

  3. Reports: One example are product release reports, how long do you need to wait to make sure that all data have been collected?

 

The PI time series object provides a time stamp which is typically provided from the source system. This time stamp will bubble up though interfaces and data archives unchanged. This makes sense when you comparing historical data, but it masked the latency in your data.

 

To detect when the data point actually gets queued and the recorded at the data server, PI offers 2 event queue that can be monitored:

 

AFDataPipeType.Snapshot ... to monitor the snapshot queue

AFDataPipeType.Archive ... to monitor the archive queue

 

There are some excellent articles on VCampus how to poll these queues, like:How to use the PIDataPipe or the AFDataPipe

 

You can also use PowerShell scripts, which have the advantage to be lighter application that can be combined with the existing OSIsoft PowerShell library. PowerShell is also available on most server, so you don't need a separate development environment for code changes.

 

The first step is to connect to the PI Server using the AFSDK:

function Connect-PIServer{

 

   [OutputType('OSIsoft.AF.PI.PIServer')]

   param

   (

  [string]

   [Parameter(Mandatory=$true,

   Position=0,

   ValueFromPipeline=$true,

   ValueFromPipelineByPropertyName=$true)]

   $PIServerName

   )

  

   $Library=$env:PIHOME+"\AF\PublicAssemblies\OSIsoft.AFSDK.dll"

   Add-Type -Path $Library


   $PIServer=[OSIsoft.AF.PI.PIServer]::FindPIServer($PIServerName)

   $PIServer.Connect()

   Write-Output($PIServer)

}

 

The function opens a connection to the server and returns the .NET object.

To monitor the queues and write the values will then look like the following:

 

function Get-PointReference{

 

   param

   (

   [PSTypeName('OSIsoft.AF.PI.PIServer')]

   [Parameter(Mandatory=$true,

   Position=0,

   ValueFromPipeline=$true,

   ValueFromPipelineByPropertyName=$true)]

   $PIServer,

  [string]

   [Parameter(Mandatory=$true,

   Position=1,

   ValueFromPipeline=$true,

   ValueFromPipelineByPropertyName=$true)]

   $PIPointName

   )

   $PIPoint=[OSIsoft.AF.PI.PIPoint]::FindPIPoint($PIServer,$PIPointName)

   Write-Output($PIPoint)

}

function Get-QueueValues{

   param

   (

   [PSTypeName('OSIsoft.AF.PI.PIPoint')]

   [Parameter(Mandatory=$true,

   Position=0,

   ValueFromPipeline=$true,

   ValueFromPipelineByPropertyName=$true)]

   $PIPoint,

  [double]

   [Parameter(Mandatory=$true,

   Position=1,

   ValueFromPipeline=$true,

   ValueFromPipelineByPropertyName=$true)]

   $DurationInSeconds

   )


   # get the pi point and cretae NET list

   $PIPointList = New-Object System.Collections.Generic.List[OSIsoft.AF.PI.PIPoint]

   $PIPointList.Add($PIPoint)


   # create the pipeline

   $ArchivePipeline=[OSIsoft.AF.PI.PIDataPipe]::new( [OSIsoft.AF.Data.AFDataPipeType]::Archive)

   $SnapShotPipeline=[OSIsoft.AF.PI.PIDataPipe]::new( [OSIsoft.AF.Data.AFDataPipeType]::Snapshot)


   # add signups

   $ArchivePipeline.AddSignups($PIPointList)

   $SnapShotPipeline.AddSignups($PIPointList)


   # now the polling

   $EndTime=(Get-Date).AddSeconds($DurationInSeconds)

   While((Get-Date) -lt $EndTime){

   $ArchiveEvents = $ArchivePipeline.GetUpdateEvents(1000);

   $SnapShotEvents = $SnapShotPipeline.GetUpdateEvents(1000);

   $RecordedTime=(Get-Date)

   # format output:

   foreach($ArchiveEvent in $ArchiveEvents){

   $AFEvent = New-Object PSObject -Property @{

  Name =  $ArchiveEvent.Value.PIPoint.Name

  Type = "ArchiveEvent"

  Action = $ArchiveEvent.Action

  TimeStamp = $ArchiveEvent.Value.Timestamp.LocalTime.ToString("yyyy-MM-dd HH:mm:ss.fff")

  QueueTime = $RecordedTime.ToString("yyyy-MM-dd HH:mm:ss.fff")

  Value = $ArchiveEvent.Value.Value.ToString()

  }

   $AFEvent.pstypenames.Add('My.DataQueueItem')

   Write-Output($AFEvent)

  }

   foreach($SnapShotEvent in $SnapShotEvents){

   $AFEvent = New-Object PSObject -Property @{

  Name =  $SnapShotEvent.Value.PIPoint.Name

  Type = "SnapShotEvent"

  Action = $SnapShotEvent.Action

  TimeStamp = $SnapShotEvent.Value.Timestamp.LocalTime.ToString("yyyy-MM-dd HH:mm:ss.fff")

  QueueTime = $RecordedTime.ToString("yyyy-MM-dd HH:mm:ss.fff")

  Value = $SnapShotEvent.Value.Value.ToString()

  }

   $AFEvent.pstypenames.Add('My.DataQueueItem')

   Write-Output($AFEvent)

  }

   # 150 ms delay

   Start-Sleep -m 150

  }

   $ArchivePipeline.Dispose()

   $SnapShotPipeline.Dispose()

}

These 2 scripts are all you need to monitor events coming into a single server. The latency is simply the difference between the value's time stamp and the time it is recorded.

 

Measuring the latency between 2 servers - for example a local and an enterprise server - can be done the same way. You just need 2 server objects and then monitor the snapshot (or archive) events.

 

function Get-Server2ServerLatency{

 

   param

   (

   [PSTypeName('OSIsoft.AF.PI.PIPoint')]

   [Parameter(Mandatory=$true, Position=0,

   ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]

   $SourcePoint,

   [PSTypeName('OSIsoft.AF.PI.PIPoint')]

   [Parameter(Mandatory=$true, Position=1,

   ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]

   $TargetPoint,

  [double]

   [Parameter(Mandatory=$true, Position=2,

   ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]

   $DurationInSeconds

   )

   $SourceList = New-Object System.Collections.Generic.List[OSIsoft.AF.PI.PIPoint]

   $SourceList.Add($SourcePoint)

   $TargetList = New-Object System.Collections.Generic.List[OSIsoft.AF.PI.PIPoint]

   $TargetList.Add($TargetPoint)


   # create the pipeline

   $SourcePipeline=[OSIsoft.AF.PI.PIDataPipe]::new( [OSIsoft.AF.Data.AFDataPipeType]::Snapshot)

   $TargetPipeline=[OSIsoft.AF.PI.PIDataPipe]::new( [OSIsoft.AF.Data.AFDataPipeType]::Snapshot)


   # add signups

   $SourcePipeline.AddSignups($SourceList)

   $TargetPipeline.AddSignups($TargetList)


   # now the polling

   $EndTime=(Get-Date).AddSeconds($DurationInSeconds)

   While((Get-Date) -lt $EndTime){

   $SourceEvents = $SourcePipeline.GetUpdateEvents(1000);

   $TargetEvents = $TargetPipeline.GetUpdateEvents(1000);

   $RecordedTime=(Get-Date)

   # format output:

   foreach($SourceEvent in $SourceEvents){

   $AFEvent = New-Object PSObject -Property @{

  Name =  $SourceEvent.Value.PIPoint.Name

  Type = "SourceEvent"

  Action = $SourceEvent.Action

  TimeStamp = $SourceEvent.Value.Timestamp.LocalTime.ToString("yyyy-MM-dd HH:mm:ss.fff")

  QueueTime = $RecordedTime.ToString("yyyy-MM-dd HH:mm:ss.fff")

  Value = $SourceEvent.Value.Value.ToString()

  }

   $AFEvent.pstypenames.Add('My.DataQueueItem')

   Write-Output($AFEvent)

  }

   foreach($TargetEvent in $TargetEvents){

   $AFEvent = New-Object PSObject -Property @{

  Name =  $TargetEvent.Value.PIPoint.Name

  Type = "TargetEvent"

  Action = $TargetEvent.Action

  TimeStamp = $TargetEvent.Value.Timestamp.LocalTime.ToString("yyyy-MM-dd HH:mm:ss.fff")

  QueueTime = $RecordedTime.ToString("yyyy-MM-dd HH:mm:ss.fff")

  Value = $TargetEvent.Value.Value.ToString()

  }

   $AFEvent.pstypenames.Add('My.DataQueueItem')

   Write-Output($AFEvent)

  }

   # 150 ms delay

   Start-Sleep -m 150

  }

   $SourcePipeline.Dispose()

   $TargetPipeline.Dispose()

}

 

Here is a quick test of a PI2PI interface reading and writing to the same server

Get-Server2ServerLatency $srv $srv sinusoid sinusclone 30 | Out-GridView

As you can see the difference between target and source is a bit over 1 sec, which is to be expected since the scan rate is 1 second.