5 Replies Latest reply on Mar 22, 2012 4:58 PM by Ahmad Fattahi

    Memory leak using EventPipes through COM - unclosed threads

    mesa.mosselbay

      Hello,

       

      I have an issue with a long-running process which is getting data change events through EventPipes from PI.  After reading up on STA-related memory issues, I moved my PI access code to a single thread.  Every 10 seconds, it checks the state of its event pipes and creates events from what it finds, like so (this uses the Python win32 COM access):

       

          def check_event_pipe(self, source):
              for event_sink, event_pipe in self.event_pipes[source]:
                  eventlist = []
                  try:
                      if event_pipe.Count:
                          events = event_pipe.TakeAll()
                          for event in events:
                              eventlist.append(self._get_event(source, event.EventData.PIValue, event.EventData.PIPoint))
                              del event
                  except Exception as e:
                      if e[2] and "no longer connected" in e[2][2]:
                          logging.info("Error getting events for eventsink %r - connection to server %s lost.  Reconnecting...",event_sink,source)
                          if not self._reconnect_source(source, True):
                              logging.error("Failed to reconnect %s",source)
                          continue
                      else:
                          logging.error("Error getting events for eventsink %r from source %s - %s",event_sink,source,str(e))
                          continue
                  if eventlist:
                      self.event_threadpool.apply_async(event_sink.receive_events, (eventlist,))
      

       As can be seen, all I run is TakeAll on the pipe, and then delete each event object as it is processed.  The _get_event function just pulls value, descriptor, zero, span, EngUnits and checks if it IsGood().  As stated earlier, this is all in the same thread which created the EventPipe.

       

      I was experiencing a memory leak, and noticed that the number of threads assigned to my process was increasing, when I was not creating any new threads.  The stack trace from these threads is:

      ntdll.dll!KiFastSystemCallRet
      kernel32.dll!Sleep+0xf
      pisdk.dll!EventRetrievalThreadRoutine+0x79
      MSVCR80.dll!endthreadex+0x3b
      MSVCR80.dll!endthreadex+0xc7
      ntdll.dll!RtlInitializeExceptionChain+0x63
      ntdll.dll!RtlInitializeExceptionChain+0x36
      

       As can be seen, it is stuck in pisdk.dll function EventRetrievalThreadRoutine.

       

      Is there something I am doing wrong?  How do I clean these threads up?