-
Re: Data extraction
Lal_Babu_ShaikMay 15, 2018 10:35 AM (in response to Arpit)
Hi Arpit,
You have below options
1) PI Config
2) PI OLEDB
3) AF SDK
-
Re: Data extraction
Arpit May 15, 2018 1:01 PM (in response to Lal_Babu_Shaik)Thank you for the reply Lal.
What is the performance impact of using PI Config scripts to query 100k tags?
-
-
Re: Data extraction
rborgesMay 15, 2018 12:11 PM (in response to Arpit)
Let me help you by asking some questions first:
1) Where do you want to send this data to? Depending on the final destination, there are better ways to do it.
2) How frequent is this data? Are we talking about second / sub-second or hourly / daily?
-
Re: Data extraction
Arpit May 15, 2018 12:59 PM (in response to rborges)Thank you for the reply Rafael.
Below are the answers for your question:
1 - I just want to extract the data and provide it to the analysts (flat files will do).
2 - We have hourly data - 24 values/day.Arpit
-
Re: Data extraction
sraposoMay 15, 2018 1:08 PM (in response to Arpit)
Hello Arpit,
I think PowerShell is probably the simplest way to do this. It of course uses the AF SDK, but you don't need to compile an application in Visual Studio, and the cmdlets are quite easy to use.
For reference: https://techsupport.osisoft.com/Documentation/PI-Powershell/title.html
Hope this helps,
Seb
-
Re: Data extraction
Arpit May 16, 2018 9:08 AM (in response to sraposo)Thank you Seb.
I will give Powershell a try, but most likely i will go with AFSDK.
Arpit
-
-
Re: Data extraction
rborgesMay 15, 2018 1:13 PM (in response to Arpit)
1 of 1 people found this helpfulSo we are talking about 876M data points. That's a lot and you should not use PI Config for it. I suggest you two ways to do it: The first one is using AF SDK. Extract data using it is simple and allows you even to already do some data alignment for your analysts. An alternative is to follow Sebastien's suggestion and use our PowerShell tools, that is a wrapper around AF SDK. The second one depends on the platform your analysts use to process the data. If they use R, you can use the PI Web API R Package. We have some other libraries for specific platforms.
-
-
-
Re: Data extraction
tramachandranMay 15, 2018 4:18 PM (in response to Arpit)
For extracting large amounts of data sets from the PI System for use in business applications, the recommended method would be using PI Integrator for Business Analytics
PI Integrator for Business Analytics presents PI System data perfectly suited to business intelligence tools including but not limited to Tableau, Tibco Spotfire, QlikView, and Microsoft Power BI for reporting and analytics.
Other techniques, such as the use of of Developer technologies should always be considered as a secondary method.
If you are interested in PI AF SDK, check out GetLargeRecordedValues - working around ArcMaxCollect and a console application wrapper that uses this for downloading data as csv files Extracting large event counts from the PI Data Archive