6 Replies Latest reply on Apr 12, 2019 3:45 PM by dkennedy

    Generating test data for a PI Data Archive

    dkennedy

      Hey guys,

       

      I'm using a local OSIPI server to attempt to replicate a client's environment. I'm looking to generate some data to test my AF SDK scripts with. I have a formatted CSV file from the client with representative data attached, but I'm not sure how challenging it would be to convert that back into the data archive. I'd also be happy with generating fake or custom data. Any guidelines for how to do this?

       

      I'm viewing my CSV file in excel, here's roughly how its structured

       

      Header 1Header 2Header 3Header 4
      DescriptionTom's WeightJoe's WeightSarah's Weight
      Unit TypekgPoundkg
      Name1TomJoe Sarah
      Name2BombadilRoganSalazar
      1/1/199023190124
      2/2/199024187123
      3/3/199022225125

       

      The kicker here is I don't need all of this detail, the name of the column (Tom's weight) and the numeric values inside of it are enough. Times are uniform for all measurement's so that could be extrapolated. So this could be as easy as copying all of my numeric values to a new spreadsheet and leading them with column information. However, I don't know much about uploading data to a data archive, any advice you have would be awesome!

        • Re: Generating test data for a PI Data Archive
          VAshok

          Hi Derek,

           

          This mainly depends on the tools available to you, the options

          1) Use PI UFL if this is available.

          https://customers.osisoft.com/s/productcontent?id=a7R1I000000XyWkUAK

          2) Please refer to the following , to generate data to PI Archive using excel

          https://customers.osisoft.com/s/knowledgearticle?knowledgeArticleUrl=1501OSI8

          3) Create a small code snippet using PI SDK/AF SDK

          • Re: Generating test data for a PI Data Archive
            sraposo

            Hi Derek,

             

            If you're familiar with PowerShell, you can create a random interface pretty quickly. For example:

             

            [Reflection.Assembly]::LoadWithPartialName("OSIsoft.AFSDK") | Out-Null
            [OSIsoft.AF.PI.PIServers] $PIServers = New-object OSIsoft.AF.PI.PIServers
            [OSIsoft.AF.PI.PIServer] $PIDA = $PIservers["PIDATAARCHIVE"] 
            [string[]] $PIPointList = Get-Content -Path "Path To File with PI Point Nmaes" #PI Points must exist
            [OSIsoft.AF.Data.AFUpdateOption] $UpdateOption = "Insert"
            [OSIsoft.AF.Data.AFBufferOption] $Buffer = "Buffer"
            $PIPoints = [OSIsoft.AF.PI.PIPoint]::FindPIPoints($PIDA,$PIPointList)
            While($true){ #infinite loop
            {
                 $Time = Get-Date
                 [OSIsoft.AF.Time.AFTime] $Timestamp = New-Object OSIsoft.AF.Time.AFTime($Time)
                 ForEach($P in $PIPoints)
                 {
                      $RandomValue = Get-Random -Minimum 0 -Maximum 100 #Range of values
                      [OSIsoft.AF.Asset.AFValue] $Value = New-object OSIsoft.AF.Asset.AFValue($RandomValue,$Timestamp)
                      $P.UpdateValue($Value,$UpdateOption,$Buffer)
                 }
                 Start-Sleep -Seconds 30 #approximate frequency 
            }
                 
            
            
            
            
            
            
            
            
            
            2 of 2 people found this helpful
            • Re: Generating test data for a PI Data Archive
              jamespr

              In my case, configuring an instance of the UFL interface has been the best solution. The main benefit is the speed with which I can add new data from a text / csv file. If it is possible to create the instance in my opinion it is the most elegant and robust solution. With the ini file you can reuse it for future projects.

              1 of 1 people found this helpful
              • Re: Generating test data for a PI Data Archive
                gregor

                I am kind of surprised nobody recommends using PI Random and PI Ramp Soak Simulator. Both are always included with the PI System. Besides that there are other options for simulated data like the Asset Based PI Example Kits.

                2 of 2 people found this helpful
                • Re: Generating test data for a PI Data Archive
                  dkennedy

                  Wow, thanks for the awesome feedback guys. Lots to check out here!

                   

                  Roger Palmen, the link was hepful, I found a post by Kelsey Duffy (OSIsoft Developer Network: spreadsheet interface, control f for kelsey duffy)

                   

                   

                  I setup my spreadsheet to mimic Kelsey's data structure and modified his script to include my CSV file/data connection. However, when running the actual script I'm seeing the following errors

                   

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                  Add-PIValue : [-12011] Identifier not found in PInttemplate

                  At C:\Projects\SLAV\PI_Data_Importer.ps1:6 char:5

                  +     Add-PIValue -PointName $_.tag -Value $_.value -Time $_.time -Conn ...

                  +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      + CategoryInfo          : InvalidOperation: (Address: pi://i...dentity piadmin:ClientChannel) [Add-PIValue], PIObjectNotFoundException

                      + FullyQualifiedErrorId : AddPIValue,OSIsoft.PowerShell.AddPIValue

                   

                   

                   

                  So I think I'm missing a piece of the puzzle here. Do I need to setup some kind of template for the data within my Pi data archive? If so how would I do that?

                   

                  Thanks!

                   

                  Derek