28 Replies Latest reply on Aug 2, 2010 4:52 PM by cmanhard

    Memory Leak when AFAttribute.GetValue.Value used on Formula

    kpotter

      The following line of code (repeating by timer) causes a memory leak (VB.net 3.5, VS 2008, windows service):

       

      LocalSingle1 = LocalAFAttribute.GetValue.Value

       

      I tried changing the Attribute Data Reference to <None>, and entered a constant, and that stopped the leak.  The leak occurs when the Data Reference is a formula.  In my case, the formula is bringing in Parameters from other elements; some go another layer or 2 back.

       

      Any insight is appreciated.... Thank you.

        • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
          cescamilla

          This is quite interesnting, did you try that inside a loop? does it happen if you call the function a lot on itself? what happens if you dispose the "LocalSingle1" before re-setting?

            • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
              kpotter

              It's not really in a loop, but several similar instances are called once a minute (or whatever frequency I set my timer).  In my efforts to stop the leak, I'm setting practically everything to "Nothing" at the bottom of every Sub or Function, so that should dispose the LocalSingle1.  I'm not sure exactly what you mean by "call the function on itself", but I don't think I'm ever doing that.  I tried using the following Function to pull values from attributes:

              Private Function GetAttributeValueSingle(ByVal EntireElementString As String, ByVal AttributeIndex As Integer) As Single
                    
                      Dim LocalElement As OSIsoft.AF.Asset.AFElement
                      Dim LocalAFValue As OSIsoft.AF.Asset.AFValue

                      GetAttributeValueSingle = 0
                      LocalElement = MyPublicAFDatabase.Elements(EntireElementString)
                      LocalAFValue = LocalElement.Attributes(AttributeIndex).GetValue()
                      If IsNumeric(LocalAFValue.Value) Then
                          GetAttributeValueSingle = LocalAFValue.Value
                      End If

                      LocalAFValue = Nothing
                      LocalElement = Nothing

              End Function

              This function works, but causes a leak WHEN the attribute is a formula.  The formula is not tied into any circular references.

               

              Thank you!

                • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                  cescamilla

                  "call the function on itself"

                   

                   

                   

                  I meant BY itself...

                   

                  something like putting the following in a 10,000 iteration loop

                          If IsNumeric(LocalAFValue.Value) Then
                              GetAttributeValueSingle = LocalAFValue.Value
                          End If

                   

                   

                  If it is causing a memory leak specifically there wouldn't it increase if you repeat this? if not, then there is some context to the memory leak.

                    • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                      David Hearn

                      What are you doing to determine that there is a memory leak? Are you doing a garbage collect before measuring memory usage each time? The .NET runtime will hold on to memory until it determines that a GC is required.

                       

                      How are the other elements that are being referenced by the formula configured? Do they also have a datareference and if so which ones?

                        • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                          kpotter

                          Originally, I never had it by itself, but just several lines like the following thoughout the code:

                          MySingle = MyAttribute.GetValue.Value

                          I tried putting it into its own function, but there was no difference in the leak.

                          In order to pin down this leak, I have only one instance, and I have my timer running this every 3 seconds.  It is not in a loop.  But, I set my timer to 5 seconds instead of the usual 60 so that the memory leaks faster.  I just now tried looping it 10000 times, and that did not make the leak faster.

                          I know of the leak by watching the service's Mem Usage in Task Manager, under Processes.  When I start the service, it starts out around 60,000 K, and climbs at rate of about 3000 K per hour.  When I let it go for few days, it gets to about 460,000 K, and locks up.  After much troubleshooting, I ultimately discovered that by commenting out that 1 line of code, the leak is stopped, and the service Mem Usage stays right around 60,000 K forever.

                          Right now, the leak is occuring while reading in only one Attribute, which is the following formula: A=..\..\Suction|VolumeFlow;UOM=GPM;[ A ];UOM=GPM

                          The Paramter A (VolumeFlow) has the formula: A=MassFlowConst;UOM=lb/min;B=OutputV;[A*B];UOM=SCFM
                          It's 2 incoming Parameters are both set to constants (Data Reference: None)

                          Interestingly, when I edit the VolumeFLow to be a constant, the leak stops.

                            • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                              David Hearn

                              I am going to attempt to reproduce what you are seeing. What version of AF are you using on the client and server?

                                • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                  kpotter

                                  The AF Explorer (client and server) and OSIsoft.AFSDK.dll are all version 2.1.1.3624

                                   

                                  Thank you!

                                    • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                      kpotter

                                      I consolidated all of the attributes into one element, and the leak still exists:

                                       

                                      Attribute being read by code is InputX.  Below is the guts of the xml which you might be able to import.  I can email it to you as well:

                                      <AF Description="" ExportedObject="PPI">
                                        <AFElement>
                                          <Name>TestElement</Name>
                                          <AFAttribute>
                                            <Name>InputX</Name>
                                            <IsConfigurationItem>False</IsConfigurationItem>
                                            <DefaultUOM>GPM</DefaultUOM>
                                            <Type>Single</Type>
                                            <UOM>GPM</UOM>
                                            <Value type="Single">700</Value>
                                            <DataReference>Formula</DataReference>
                                            <ConfigString>A=VolumeFlow;UOM=GPM;[ A ];UOM=GPM</ConfigString>
                                          </AFAttribute>
                                          <AFAttribute>
                                            <Name>MassFlowConst</Name>
                                            <DefaultUOM>kpph</DefaultUOM>
                                            <Type>Single</Type>
                                            <UOM>kpph</UOM>
                                            <Value type="Single">484</Value>
                                          </AFAttribute>
                                          <AFAttribute>
                                            <Name>OutputV</Name>
                                            <Description>Specific Volume</Description>
                                            <IsConfigurationItem>False</IsConfigurationItem>
                                            <DefaultUOM>ft3/lbm</DefaultUOM>
                                            <Type>Single</Type>
                                            <UOM>ft3/lbm</UOM>
                                            <Value type="Single">1.76615677773952E-02</Value>
                                          </AFAttribute>
                                          <AFAttribute>
                                            <Name>OutputY</Name>
                                            <IsConfigurationItem>False</IsConfigurationItem>
                                            <DefaultUOM>ft</DefaultUOM>
                                            <Type>Single</Type>
                                            <UOM>ft</UOM>
                                            <Value type="Single">7032.47998046875</Value>
                                          </AFAttribute>
                                          <AFAttribute>
                                            <Name>VolumeFlow</Name>
                                            <IsConfigurationItem>False</IsConfigurationItem>
                                            <DefaultUOM>GPM</DefaultUOM>
                                            <Type>Single</Type>
                                            <DataReference>Formula</DataReference>
                                            <ConfigString>A=MassFlowConst;UOM=lb/min;B=OutputV;[A*B];UOM=SCFM</ConfigString>
                                          </AFAttribute>
                                        </AFElement>
                                      </AF>
                                        • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                          jstarnes

                                          Since the leak does not occur faster in the loop, it appears something else is holding onto the memory. Thus, would it be possible for you to submit the complete application for further investigation?

                                            • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                              kpotter

                                              Yes, but I need to create a scaled down version first.  On another note, I've discovered that the leak only exists when I have the unit conversions which are present in the xml.  I tried setting all UOMs to <None>, and no leak.

                                                • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula

                                                  Do you have the details/formulas of the UOMs that have been added?

                                                    • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                      kpotter

                                                      Sure, in fact, I can email you my UnitsOfMeasure.xml if you like (kpotter@process-innovations.net)

                                                       

                                                      Class: Volume Flow Rate

                                                       

                                                      Canonical UOM: m3/s

                                                       

                                                      GPM - Reference: m3/s, Simple Factor: 6.30901964E-05

                                                       

                                                      SCFM - Reference: m3/s, Simple Factor: 0.0004719474432

                                                       

                                                       

                                                       

                                                      Class: Mass Flow Rate

                                                       

                                                      Canonical UOM: kg/s

                                                       

                                                      lb/s - Reference: kg/s, Simple Factor: 0.45359237

                                                       

                                                      lb/min - Reference: lb/s, Formula: lb/s = lb/h / 60 ;  lb/h = lb/s * 60

                                                       

                                                      lb/h - Reference: lb/s, Formula: lb/s = lb/h / 3600 ;  lb/h = lb/s * 3600

                                                       

                                                      kpph - Reference: lb/h, Simple Factor:1000

                                                       

                                                       

                                                       

                                                      Class: Specific Volume

                                                       

                                                      Canonical UOM: m3/kg

                                                       

                                                      ft3/lbm - Reference: m3/kg, Simple Factor: 0.0624279605761446

                                                       

                                                       

                                                       

                                                       

                                                       

                                                       

                                                       

                                                       

                                                       

                                                       

                                                        • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula

                                                          Yeah my immediate thought was maybe an error (or similar) of the added UOMs is causing the memory leak.  Otherwise, it must be a general issue with UOM in the AFSDK.

                                                            • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                              kpotter

                                                              I'm almost there.  All I need to know now is how to do this:

                                                               

                                                              MyAttribute.GetValue( ?NO UOMs??? ).Value

                                                               

                                                              How can I specify <None> for the UOMs in this line?

                                                                • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                  David Hearn

                                                                  If you do not specify a UOM for the GetValue call, then it will automatically use the attribute's DefaultUOM.

                                                                   

                                                                  I think your problem is likely with the UOMs defined by a formula. Using a conversion factor is much more efficient than using a formula because a formula must be compiled at runtime. Your lb/h and lb/m UOMs can easily be defined as a conversion factor of 1/3600 (0.00277777) and 1/60 (0.01666666).

                                                                    • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                      kpotter

                                                                      I realize this now, and I am currently editing my UOM database (which I am sure is superior to the default, particularly for the Power Generation industry) to use the simple factors.  HOWEVER, I would LOVE to know how to have the .GetValue(???).Value method IGNORE UOMs (ie. set UOM to <None>) in the code.

                                                                        • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                          David Hearn

                                                                          This is one of the benefits of using AF is to ensure that you always get the value in the correct UOM so you don't have to worry about keeping track of the units yourself. The only way to have AF ignore UOMs is to not configure any UOMs for your attributes.

                                                                            • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                              kpotter

                                                                              "not configuring any UOMs for my attibutes" is, I'm sorry to say, an absurd sacrifice.

                                                                               

                                                                              The UOM system in AF is wonderful, as I've explained to many of my clients.  But, I don't need my code to consider them for my purposes.  In AF, I have the option of selecting "<None>" for UOM.  Since the GetValue.Value method allows me to specify UOM, it should likewise allow me specify "<None>".  Since it doesn't, I now must dumb down about 300 UOM formulas into simple factors, or else I get a memory leak.  Why have UOM formulas if simple factors are just as good?  Obviously, the formulas are a more precise and proper definition, which is why the AF offers that choice.

                                                                               

                                                                              Are you sure that there is no way to set the UOM to <None> on the GetValue.Value method?  Please consider this:

                                                                               

                                                                              Dim MyUOM As OSIsoft.AF.UnitsOfMeasure.UOM
                                                                              Dim MyAttribute As OSIsoft.AF.Asset.AFAttribute

                                                                               

                                                                              After setting vaules, I can do this:

                                                                               

                                                                              MyDouble = MyAttribute.GetValue(MyUOM).Value

                                                                               

                                                                              BUT, I have not yet figured out how to assign a value to MyUOM.  Since <None> is always a choice in AF, I would think that there has to be a way to assign MyUOM to <None>.

                                                                               

                                                                              If you could show me how to assign MyUOM to anything, that would help.

                                                                               

                                                                              Thank you

                                                                                • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                  David Hearn

                                                                                  I was not suggesting that you change your configuration to not specify UOMs, I was just stating that is the only why to not use UOM conversions. Using conversion factors is much better than formulas for performance and memory usage. Formulas should only be used where a conversion factor cannot be used (e.g. conversion between degree Fahrenheit and Celsius).

                                                                                   

                                                                                  Specifying <None> in the AF GUI is the same as specifying null for the UOM. If you specify null for the UOM parameter to the GetValue call, this will cause the GetValue to use the DefaultUOM for the attribute.

                                                                                    • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                      kpotter

                                                                                      Thank you for your help.  However, I'm convinced that you have a bug that needs to be fixed.

                                                                                       

                                                                                      A.  Others will opt for UOM formulas (i.e. divide by 60 or 3600) as I did.  And, I must now warn clients about this pitfall in case they decide to add UOMs

                                                                                       

                                                                                      B.  I might still have a problem whenever I bring in a value from a temperature, which I do quite often.

                                                                                       

                                                                                      Thanks again.

                                                                                        • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                          David Hearn

                                                                                          Actually, conversion between tempeature do not use a formula. They use a conversion factor and offset. The standard UOMs shipped with AF do not use formulas for their conversions. There is one UOM that ships with Sigmafine that uses a formula for SG/API conversions and it is optmized within AF to not actually use a compiled formula.

                                                                                           

                                                                                          Therefore, I don't think you will have any problems if you avoid using formulas in your UOM conversions. We are looking into the use of UOM formulas to determine if there is a problem with memory.

                                                                                            • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                              David Hearn

                                                                                              We have identified one case where memory is lost when using UOM formulas and the PISystem object gets garbage collected. Since the UOM formula is compiled .NET code, it is not GC'd because it is not a managed object. But the only case where we have found this to happen is if you are specifying 'true' to the PISystems constructor or you do not hold a strong reference to the PISystem object and it gets GC'd.

                                                                                               

                                                                                              We are creating a work item to address this for the next release of AF 2.3, but we would like to know if the PISystem object could be getting GC'd in your application. Holding a global strong reference to the PISystem object should prevent this issue if this is the cause in your application.

                                                                                                • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                                  kpotter

                                                                                                  I am declaring my lone PISystem PISystems object publicly:

                                                                                                   

                                                                                                  Public MyPISystem As OSIsoft.AF.PISystem

                                                                                                   

                                                                                                  Public iPISystems As New OSIsoft.AF.PISystems

                                                                                                   

                                                                                                  As such, I wouldn't think that the PISystem object could be getting GC'd

                                                                                                   

                                                                                                   

                                                                                                    • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                                      kpotter

                                                                                                      What if I did the following?:

                                                                                                       

                                                                                                      MyDouble = MyAttribute.GetValue(MyAttribute.DefaultUOM).Value

                                                                                                       

                                                                                                      Would this prevent the issue since it sets UOM to DefaultUOM, thus no conversion???

                                                                                                       

                                                                                                      ....Nevermind, that's the same as the original.

                                                                                                       

                                                                                                       

                                                                                                        • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                                          Rick Davin

                                                                                                          Let me try again.  To not have any UOM conversion, you could try:

                                                                                                           

                                                                                                          1)   MyDouble = MyAttribute.GetValue().Value

                                                                                                           

                                                                                                          Or

                                                                                                           

                                                                                                          2)   Dim MyUOM As OSIsoft.AF.UnitsOfMeasure.UOM = Nothing

                                                                                                           

                                                                                                                  MyDouble = MyAttribute.GetValue(MyUOM).Value

                                                                                                           

                                                                                                           

                                                                                                            • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                                              kpotter

                                                                                                              Rick,

                                                                                                               

                                                                                                              That is a great idea, and it is exactly what I did yesterday.  I compiled my new code, and have it running on 2 separate servers, each with their own databases.   The one with my new UOM system (with no Unit Conversion Formulas) has no memory leak.  The other with my former UOM system (with some formulas such as GPH = GPM / 60) still has a memory leak.   So, that method probably results in the same as the original .GetValue.Value.

                                                                                                               

                                                                                                              So, it looks like there is no possible way to read the numerical value without the code insisting on digging into the UOMs, and leaking memory if it finds a UOM conversion formula.  But thank you, Rick, for the suggestion.

                                                                                                               

                                                                                                              -------------------

                                                                                                               

                                                                                                              Realizing that some might argue that the UOM system can work just as well purely with simple factors (which I have much experience with as a co-developer of an unmentioned, increasingly obsolete, 20th century, black-box product), I would suggest otherwise:

                                                                                                               

                                                                                                              I invested a tremendous effort developing a very sophisticated and proper UOM database based upon international standards and definitions.   I've been so impressed by this UOM system because of it's abiltiy to preserve the proper origins of the conversions instead of having them washed into a huge list of factors.

                                                                                                               

                                                                                                              For example,

                                                                                                               

                                                                                                              1 inch is exacly 0.0254 meters, by definition.  12in = 1ft, 3ft=1yd, 5280ft=1mile, etc.

                                                                                                               

                                                                                                              Then you have velocities such as ft/sec, ft/min, mph, etc.

                                                                                                               

                                                                                                              Then you have volumetric flows such as ft^3/hr, etc.

                                                                                                               

                                                                                                              So, to stop the memory leak, I had to wash away the purity and simplicity in proving that the UOM conversions are proper.

                                                                                                               

                                                                                                              Another thing to consider:

                                                                                                               

                                                                                                              Let's say there is a design specification of 6000 SCFH (ft^3/hr).  An attribute reflecting this parameter is using SCFM.

                                                                                                               

                                                                                                              Will it read 100 SCFM, or 99.9999994?

                                                                                                               

                                                                                                              Sorry for rabbit trail

                                                                                                                • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                                                  cmanhard

                                                                                                                  Ken,

                                                                                                                   

                                                                                                                   

                                                                                                                   

                                                                                                                  Regarding the memory leak, UOM formula conversions are compiled on a per PI System basis.  As long as the PI System is not GC’d, then there will be no leak.  The Public statement alone is not sufficient to guarantee that the “MyPISystem” variable is held onto because it in turn relies on the class that it is a part of to be in continual existence.  Without context of the containing class usage, using a “Shared” variable in VB would be more of a guarantee.  Your performance will in general better if you do not allow the PI system to be GC'd. 

                                                                                                                   

                                                                                                                   

                                                                                                                   

                                                                                                                  Regarding performance of UOM formula vs. using the factor, you should expect to see about 2x slower performance for the raw conversion calculation for formula over the factor once the formula has been compiled.  The speed is in the range of a 1000 conversions per millisecond, so in general, this overhead would be indistinguishable from the overall performance of retrieving external data.  It should not be a concern.  However, the initial compile of each formula will take a bit of time (100 to 200 ms)

                                                                                                                   

                                                                                                                   

                                                                                                                   

                                                                                                                  Regarding the accuracy of the UOM conversion using factor and offset:  AF UOM’s allow you to define their conversion factors by a reference UOM, as opposed to always using a canonical UOM conversion.  You will notice the LENGTH class defines all English units of measure in terms of inches.  This allows the UOM conversion between English units to be specified in whole numbers, and the conversion between English units will not use the metric conversion.  The use of whole number factors reduces the occurrences of round-off error when converting whole numbers. 

                                                                                                                   

                                                                                                                   

                                                                                                                   

                                                                                                                  In any case, all conversions are done in 64 bit floating point, so the amount of precision lost on one calculation will be limited to at most 1e10-15.  The example given of SCFH and SCFM will produce a result of 100 whether the conversion is done through the canonical UOM or the reference UOM.   If you are not seeing that, then it may be the UOM conversion factors you entered are not specified to the full precision offered.

                                                                                                                   

                                                                                                                   

                                                                                                                   

                                                                                                                   

                                                                                                  • Re: Memory Leak when AFAttribute.GetValue.Value used on Formula
                                                                                                    Rick Davin

                                                                                                    To set a UOM to the equivalent of <None> on the GetValue.Value method, you could try this:

                                                                                                     

                                                                                                    Dim MyUOM As OSIsoft.AF.UnitsOfMeasure.UOM = Nothing
                                                                                                    Dim MyAttribute As OSIsoft.AF.Asset.AFAttribute
                                                                                                    MyDouble = MyAttribute.GetValue(MyUOM).Value