7 Replies Latest reply on Apr 18, 2016 3:32 PM by Dan Fishman

maxCount in PI WebAPI calls restricted

Paurav Joshi Apr 18, 2016 2:22 PM

Hello All,

I have come today that maxCount limitation in PI WebAPI call is 150000. I went to check tuning parameter in my PI Data Archive and to my surprise found one parameter maxArcCollect which is also same number 150000 as value .

That raised some doubts for me:

What is significance of this maxArcCollect ? How it affects querying archived data of tags in PI WebAPI calls?
How processbook or PI datalink is fetching data of the same tag of last month period which has 238543 events?
What will be impact on PI Data Archive, if I increased that number of maxArcCollect to 2000000?
If I query tag which has 2000000 archived value after increasing number of maxArcCollect, how will it affects resources (example, RAM, CPU usage) on my machine?

Regards,

Paurav Joshi

Note: This case is useful only for bulk read using stream calls of PI Web API

Message was edited by: Paurav Joshi

Re: maxCount in PI WebAPI calls restricted

Dan Fishman Apr 12, 2016 12:19 PM (in response to Paurav Joshi)

Here is a KB article that describes ArcMaxCollect which is the parameter I believe you are asking about. KB00367 - Changing the ArcMaxCollect parameter: What are the ramifications or possible issues?
Older servers that were upgraded most likely have a default limit of 150K and newer severs have a limit of 1.5 million as the default. This parameter is designed to protect the PI Sever from expensive queries. It is more likely that this data will be on disc and not on the primary archive so your'll see more disc activity and this data will be placed in the file systems cache on the Data Historian server. In general, 2 million events is not an unreasonable setting and you will still give you protection against large queries. An alternative to going up to say 10 million is to chunk your queries. A few other parameters you might want read about: 3224OSI8 - How to limit expensive queries to the PI Server archive
2 of 2 people found this helpful
Like Show 0 Likes(0)

Actions
- Re: maxCount in PI WebAPI calls restricted
  
  Paurav Joshi Apr 12, 2016 12:35 PM (in response to Dan Fishman)
  
  Thanks Dan. This KB articles are very informative .
  
  Curious about case like question 2:
  If my processbook trend or PI datalink fetching more data than maxArcCollect, then how it will be done?
  Will my one call through processbook or datalink automatically converts into two concurrent calls or if otherwise than how?
  
  Cheers,
  Paurav Joshi
  
  Like Show 0 Likes(0)
  
  Actions
  - Re: maxCount in PI WebAPI calls restricted
    
    Dan Fishman Apr 12, 2016 1:51 PM (in response to Paurav Joshi)
    
    Good question! No, PI Datalink and PI Processbook do not automatically convert into two (or more) calls if the limit is hit. You typically do not experience this issue in PI Processbook because it use the SDK method Plotvalues which only retrieves enough information to accurately display the trend. A PI ProcessBook DataSet requires all the archived values so it does not use PlotValues and is subject to ArcHaxCollect.
    
    Like Show 1 Likes(1)
    
    Actions
    - Re: maxCount in PI WebAPI calls restricted
      
      Paurav Joshi Apr 12, 2016 2:29 PM (in response to Dan Fishman)
      
      That was very insightful Dan
      
      I have query that if I am developing custom application using PI Web API and I am fetching archive data of, as an example 6, assets for a month, then the query has to get 20 million around data. If my maxArcCollect is 10 million, then please advise me on how should I make query to get this data?
      Option 1: Increase the maxArcCollect
      Option 2: Divide the single query into two and call them concurrently
      
      Please suggest if we have options except above two.
      
      Regards,
      Paurav Joshi
      
      Like Show 0 Likes(0)
      
      Actions
      - Re: maxCount in PI WebAPI calls restricted
        
        Dan Fishman Apr 12, 2016 4:09 PM (in response to Paurav Joshi)
        
        If you always know the data density and approximately how many events it might make sense to do concurrent chunks to provide some protection to your PI System instead of increasing maxArcCollect. If users often need 10 million events, then maybe increase this. In most cases, the Chunking is slightly difficult to determine appropriate sizes since data density is not known ahead and we don't really have a way to page on this yet.
        1 of 1 people found this helpful
        
        Like Show 0 Likes(0)
        
        Actions
Re: maxCount in PI WebAPI calls restricted

Paurav Joshi Apr 18, 2016 2:21 PM (in response to Paurav Joshi)

Little information need to be added here.
I was facing this issue when I am fetching PI tag data using PI Web API via streams like,
https://{web_api_hostname}/piwebapi/streams/{webid_of_tag}/recorded?starttime=*-30d&endtime=*&maxcount=150000

For bulk read via streamset please follow this thread .

Cheers
Like Show 0 Likes(0)

Actions
- Re: maxCount in PI WebAPI calls restricted
  
  Dan Fishman Apr 18, 2016 3:32 PM (in response to Paurav Joshi)
  
  Take a look at the MaxItemsPerCall configuration attribute and you can read about it here: PI Web API
  1 of 1 people found this helpful
  
  Like Show 0 Likes(0)
  
  Actions

Go to original post