4 Replies Latest reply on Feb 10, 2010 8:20 AM by andreas

    Blobs at PIPoint

    dsabin

      What is the largest size of binary data that can be stored in a point that uses pttypBlob as its normal values?  I am interested in storing some byte arrays that could be up to 100 kilobytes in size.

       

      Thanks!
      Dan

        • Re: Blobs at PIPoint

          The largest size for a Binary Large Object (BLOB) PI Point is 976 bytes (a little less than 1 kylobyte).

           

          You might want to consider

          • storing that data somewhere else than in the PI Server and retrieve it from there
          • use some "multi-plexing" technique - storing multiple successive parts of the array at different timestamps, but very close to each other. Use the same concepts as linked list, where each part of the chain would specify whether or not you reached the end of that array.

            The maximum frequency at which you can store data in PI is 15.26 µs (15.26 microsecond), the equivalent of 65535 values/second. This means the the 105 or so parts of your 100 KB array, wouldn't "occupy" more than 1.7 milli-second (probably plenty enough to leave some blank before the start of the next array...).

          Hope this helps!

          • Re: Blobs at PIPoint
            Daniel Takara

            Hi Daniel Sabin,

             

            An alternative you may want to consider is to use annotations in PIValue objects to store your byte arrays. In this case, the PIPoint itself could be of any type (i.e., could be int16, float32, digital, string, etc, not necessarily blob) and it's the annotations ("appended" to dummy PIValues) that would actually be the byte arrays you want to store.

             

            The PI SDK documentation and the PI Server Reference Guide excerpts below describe in more detail the nature of annotations and a few issues to take into account. The PI SDK documentation also contains sample code to add and retrieve such annotations.

             

            In PI versions previous to 3.4.380, the annotation size was limited by the Event Queue page size, which had a default size of 1MB and a maximum size of 8MB. I am not sure if this limit changed in PI 3.4.380, but, since your byte arrays have a maximum size of 100Kbytes, I guess this should not be any issue for you.

             

            Hope this helps.

             

            PI SDK documentation (v.1.3.6)

            Annotations property (PIValue object)

            Annotations are used to attach an object, such as a comment, electronic signature, or picture, to a value placed in PI. The annotation is a link to a single object, but that object can be a PIAnnotations collection which allows multiple annotations on a single value. PIAnnotations collections are supported in PI-SDK version 1.3.3 and later.

            In retrieval, the user must know what type of annotation is being access to properly resolve the object that is returned.  The PI server does not restrict what type of byte stream is stored as an annotation, as such, when retrieving the annotation, the onus of proper interpretation of the annotation is on the application that receives the annotation.

            Annotations are only supported on PI server versions 3.3.362.55 and higher.

            To set an annotation of an event on the PI server, a PIValue must be submitted with a ValueAttributes collection that contains a NamedValue with the Name "Annotations".  If a NamedValue of Name "Annotations" is present in a ValueAttributes NamedValues collection, the Value of that NamedValue is used as the annotation. The event that is saved in the PI server will have the "Annotated" flag set - an annotation is present. Note, the replacement of a server event with another event, even the same value with a blank annotation, will set the "Substituted" flag on the event, since the event has been altered.

            Annotations are persisted with an event, even if the value of the event is edited. Replacing a previous event that contains an annotation with a value that does not have an annotation will not remove the annotation from the previous value, i.e. the new value will appear with the old annotation.

             

            The removal of an annotation is done by replacing the server event with a PIValue of the same timestamp and an annotation with a blank (zero-length) string. A blank string as an annotation will not create an annotation, but will always remove an annotation if present. A blank annotation will do nothing, if there is no annotation previously present.

             

             

             

            PI Server Reference Guide (published in November 2009)


            Add and Edit Annotations

            Every value in the Snapshot or the Archive may be annotated. An annotation can be of any data type. Annotations are stored in an Annotation file. Each Archive file has a single associated Annotation file, with an .ann extension. You can use piconfig to add and edit text annotations.

            Note: Any operation on annotation translates into an actual I/O, bypassing Archive caching. Thus it is much less efficient than non-annotated events. Be aware of this when using annotations.

              • Re: Blobs at PIPoint

                Daniel Takara

                An alternative you may want to consider is to use annotations in PIValue objects to store your byte arrays
                That's true, I didn't list that one. But this will result in less efficiency compared to storing values in the actual "value" field. As per Daniel (Takara)'s last quote: operations on annotations translate into an actual I/O operation (they go directly to the archiveFile.ann file rather than the archiveFile). I didn't test it myself, but chances are that storing and retrieving those 100 KB of data will be slower when using annotations than when using the regular (and highly optimized) data retrieval mechanisms in PI.

                 

                Something else you might you test is whether using "structured annotations" (through the PISDK.PIAnnotations object) is more efficient thank the legacy "unstructured annotations" approach (a variant/object object)

                 

                Daniel Takara

                In PI versions previous to 3.4.380, the annotation size was limited by the Event Queue page size, which had a default size of 1MB and a maximum size of 8MB. I am not sure if this limit changed in PI 3.4.380
                This limit did not change in PI 3.4.380 - the maximum setting for the Snapshot_EventQueuePageSize tuning parameter is 8192 KB (8 MB).

                  • Re: Blobs at PIPoint
                    andreas

                    To add to Steve's concerns, the number of annotations per archive is limited (default to 65k per archive). This is configurable but from my experience I agree to Steve's comment - it will be much slower to use annotations than the standard archive. The limitation should also give you a hint that annotations are not meant to store frequent data in the PI system.