ID3242: The security token could not be authenticated or authorized.

Hi, I'm getting the error of "ID3242: The security token could not be authenticated or authorized." when retrieving Invoice record from CRM 2011 ADFS

My application is using the crm helper from the SDK to connect to the CRM.

So this application will want to insert to Invoice, but first will select the Invoice if there is any existing one. Selecting existing Invoice is using the LINQ for CRM and early binding.

The error will show only on large number of records. I have found this after 3k+ or 5k+ records.

Stack trace :

at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.Xrm.Sdk.IOrganizationService.Execute(OrganizationRequest request)
   at Microsoft.Xrm.Sdk.Client.OrganizationServiceProxy.ExecuteCore(OrganizationRequest request)
   at Microsoft.Xrm.Sdk.Client.OrganizationServiceProxy.Execute(OrganizationRequest request)
   at Microsoft.Xrm.Sdk.Client.OrganizationServiceContext.Execute(OrganizationRequest request)
   at Microsoft.Xrm.Sdk.Linq.QueryProvider.RetrieveEntityCollection(OrganizationRequest request, NavigationSource source)
   at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute(QueryExpression qe, Boolean throwIfSequenceIsEmpty, Boolean throwIfSequenceNotSingle, Projection projection, NavigationSource source, List`1 linkLookups, String& pagingCookie, Boolean& moreRecords)
   at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute[TElement](QueryExpression qe, Boolean throwIfSequenceIsEmpty, Boolean throwIfSequenceNotSingle, Projection projection, NavigationSource source, List`1 linkLookups)
   at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute[TElement](Expression expression)
   at Microsoft.Xrm.Sdk.Linq.QueryProvider.System.Linq.IQueryProvider.Execute[TResult](Expression expression)
   at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable`1 source)

Inner Exception
    ---------------
    Type : System.ServiceModel.FaultException, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
    Message : ID3242: The security token could not be authenticated or authorized.
    Source :
    Help link :
    Action :
    Code : System.ServiceModel.FaultCode
    Reason : ID3242: The security token could not be authenticated or authorized.
    Data : System.Collections.ListDictionaryInternal
    TargetSite :
    HResult : -2146233087
    Stack Trace : The stack trace is unavailable.

Anyone have any idea what happen?