AnsweredAssumed Answered

Timeout when calling AFDatabase.FindObject(realPath) : the socket connection was aborted.

Question asked by VarbanVarbanov on Jan 3, 2017
Latest reply on Jan 6, 2017 by VarbanVarbanov

Hi all, We are sometimes randomly receiving a timeout when calling the AFDatabase.FindObject(path). The AF Sdk version is 2.7.5. The AF server version is 2.5 (I know especially the server is a bit old version, but I'm interested for a solution in which we do not need to upgrade it , if such exist). We managed to isolate the problem where nothing else is calling AF, so the AF Server is not under heavy stress, however about each 30 seconds this method is called 4 times, roughly at the same time (in parallel). The problem happens rarely and randomly (roughly once an hour). It seems that AF Sdk hangs in infinite loop or so, because we also observe severe cpu increasing on the process on which we call af sdk, and there is no increasing of the cpu usage on the af server process or sql. Below you can find a full call stack, got with WCF tracing. The error in the log happens on operation: "Close Client Base. Contract type: 'IAFServer'". Is it possible that this is some known issue? What should we do to avoid it? Thank you in advance!

 

 

http://msdn.microsoft.com/nb-NO/library/System.ServiceModel.Channels.TcpConnectionResetError.aspxThe socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:05'. The local IP address and port is [fe80::4035:d1cf:48b2:7946%23]:64204. The remote IP address and port is [fe80::4035:d1cf:48b2:7946%23]:5457./LM/W3SVC/3/ROOT/AssetsProviderService-1-131269810262284006System.ServiceModel.CommunicationException, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:05'.  at System.ServiceModel.Channels.SocketConnection.ConvertTransferException(SocketException socketException, TimeSpan timeout, Exception originalException, TransferOperation transferOperation, Boolean aborted, String timeoutErrorString, TransferOperation timeoutErrorTransferOperation, SocketConnection socketConnection, TimeSpan remainingTime)  at System.ServiceModel.Channels.SocketConnection.ConvertSendException(SocketException socketException, TimeSpan remainingTime)  at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout)  at System.ServiceModel.Channels.BufferedConnection.WriteNow(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout, BufferManager bufferManager)  at System.ServiceModel.Channels.BufferedConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout)  at System.ServiceModel.Channels.ConnectionStream.Write(Byte[] buffer, Int32 offset, Int32 count)  at System.Net.Security.NegotiateStream.StartWriting(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)  at System.Net.Security.NegotiateStream.ProcessWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)  at System.Net.Security.NegotiateStream.Write(Byte[] buffer, Int32 offset, Int32 count)  at System.ServiceModel.Channels.StreamConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout)  at System.ServiceModel.Channels.FramingDuplexSessionChannel.CloseOutputSessionCore(TimeSpan timeout)  at System.ServiceModel.Channels.TransportDuplexSessionChannel.CloseOutputSession(TimeSpan timeout)  at System.ServiceModel.Channels.TransportDuplexSessionChannel.OnClose(TimeSpan timeout)  at System.ServiceModel.Channels.CommunicationObject.Close(TimeSpan timeout)  at System.ServiceModel.Channels.ServiceChannel.OnClose(TimeSpan timeout)  at System.ServiceModel.Channels.CommunicationObject.Close(TimeSpan timeout)  at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)  at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)  at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)  at System.ServiceModel.Channels.ServiceChannelProxy.ExecuteMessage(Object target, IMethodCallMessage methodCall)  at System.ServiceModel.Channels.ServiceChannelProxy.InvokeChannel(IMethodCallMessage methodCall)  at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)  at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)  at System.ServiceModel.ICommunicationObject.Close(TimeSpan timeout)  at System.ServiceModel.ClientBase`1.System.ServiceModel.ICommunicationObject.Close(TimeSpan timeout)  at OSIsoft.AF.Support.AFProxy.Disconnect(TimeSpan disconnectTimeout)  at OSIsoft.AF.Support.AFProxy.Connect(AFCollectiveMember member, Boolean userSpecified, Boolean updatePreference, Int32 numRetries, IWin32Window owner, AFConnectionPreference preference)  at OSIsoft.AF.Support.AFProxy.AutoConnect()  at OSIsoft.AF.Support.AFSerialProxy.Call(String rpcName, ProxyDelegate codeBlock)  at OSIsoft.AF.Support.AFSerialProxy.GetElementList(dcElementCollections findType, dcObjectIdentity parentIdentity, dcObjectIdentity filterIdentity, AFTime queryDate, DateTime sinceDate, String startName, Int32 startIndex, AFSortField sortField, AFSortOrder sortOrder, Int32 maxCount, Boolean returnTotalCount, Boolean& fullUpdate, DateTime& lastUpdateTime, Int32& totalCount)  at OSIsoft.AF.Asset.AFElements.LoadObjects(Int32 page, Boolean fullReload)  at OSIsoft.AF.AFCollection`1.LoadPage(Int32 page)  at OSIsoft.AF.AFCollection`1.GetItemsPage(Int32 page, AFGetPageMode mode)  at OSIsoft.AF.AFCollection`1.<GetEnumerator>d__1.MoveNext()  at OSIsoft.AF.AFNamedCollection`1.TryGetValue(String key, Boolean loadFirstPageFromServer, Boolean returnDeletedItems, T& value)  at OSIsoft.AF.AFNamedCollection`1.GetItemTyped(String name, Boolean returnDeletedItems, Boolean handlePathInName, Boolean inLoad)  at OSIsoft.AF.Asset.AFElements.GetItemTyped(String name, Boolean returnDeletedItems, Boolean handlePathInName, Boolean inLoad)  at OSIsoft.AF.AFNamedCollection`1.GetItem(String name, Boolean returnDeletedItems, Boolean handlePathInName, Boolean inLoad)  at OSIsoft.AF.AFCollection`1.FindItemInCollection(AFCollection`1 items, AFPathToken pathToken, AFObject relativeFrom, Boolean handlePathInName)  at OSIsoft.AF.AFCollection`1.FindItemInCollection(AFPathToken pathToken, AFObject relativeFrom, Boolean handlePathInName)  at OSIsoft.AF.AFCollection`1.FindItemsInCollection(AFCollection`1 items, AFPathToken pathToken, AFObject relativeFrom, Boolean handlePathInName, Boolean returnFoundItems, IList`1& foundItems)  at OSIsoft.AF.AFCollection`1.FindItemsInCollection(AFPathToken pathToken, AFObject relativeFrom, Boolean handlePathInName, Boolean returnFoundItems, IList`1& foundItems)  at OSIsoft.AF.AFObject.FindChildObjects(AFPathToken pathToken, Boolean handlePathInName, Boolean returnFoundItems, IList`1& foundItems, IDictionary nodeValues)  at OSIsoft.AF.Support.AFPath.FindSingleSubObject(AFPathToken part, AFObject relativeFrom, AFTime queryDate, Boolean returnFoundItems, IList`1& foundItems)  at OSIsoft.AF.Support.AFPath.FindSubObject(AFETWTraceBuffer buffer, AFPath path)  at OSIsoft.AF.Support.AFPath.FindObject(AFPath path, Boolean enableBulkLoad, String& parentPath, Int32& parentPathDepth)  at OSIsoft.AF.Support.AFPath.FindObject(AFPath path)  at OSIsoft.AF.AFObject.FindObject(String path, AFObject relativeFrom, Nullable`1 queryDate)  at AssetsProvider.Service.AssetsProvider.<>c__DisplayClass10.<GetElmentsByPaths>b__f(String path)  at System.Threading.Tasks.Parallel.<>c__DisplayClass32`2.<PartitionerForEachWorker>b__30()  at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)  at System.Threading.Tasks.Task.<>c__DisplayClass11.<ExecuteSelfReplicating>b__10(Object param0)  at System.Threading.Tasks.Task.Execute()  at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)  at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)  at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)  at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution)  at System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(Task task, Boolean taskWasPreviouslyQueued)  at System.Threading.Tasks.TaskScheduler.TryRunInline(Task task, Boolean taskWasPreviouslyQueued)  at System.Threading.Tasks.Task.InternalRunSynchronously(TaskScheduler scheduler, Boolean waitForCompletion)  at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner`1 source, ParallelOptions parallelOptions, Action`1 simpleBody, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)

Outcomes