Failed Jobs

Enqueued 8 / 0 Scheduled 0 Processing 3 Succeeded 177,343,860 Failed 11,976 Deleted 6,528 Awaiting 0

Failed jobs do not become expired to allow you to re-queue them without any time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete) attribute to delete them automatically.

Id	Job	Failed
#144599950	ExecuteDataCleanedUpQueueOngbauProcessingEventTask.ExecuteJob An exception occurred during processing of a background job. Fewer details...	10/05/2025 18:03:40
#144599950	System.InvalidOperationException Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection() at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func) at Hangfire.SqlServer.SqlServerConnection.GetJobData(String id) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#144590304	ExecuteExportJobTask.ExecuteJob An exception occurred during processing of a background job. More details...	10/05/2025 16:42:54
#144590304	System.InvalidOperationException Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection() at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func) at Hangfire.SqlServer.SqlServerConnection.GetJobData(String id) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#144590303	ExecuteDataQueueProcessingEventTask.ExecuteJob An exception occurred during performance of the job. More details...	10/05/2025 16:42:55
#144590303	System.InvalidOperationException Could not get a value of the job parameter `CurrentCulture`. See inner exception for details. System.InvalidOperationException: Could not get a value of the job parameter `CurrentCulture`. See inner exception for details. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection() at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func) at Hangfire.SqlServer.SqlServerConnection.GetJobParameter(String id, String name) at Hangfire.Server.PerformContext.GetJobParameter[T](String name) --- End of inner exception stack trace --- at Hangfire.Server.PerformContext.GetJobParameter[T](String name) at Hangfire.CaptureCultureAttribute.OnPerforming(PerformingContext filterContext) at Hangfire.Profiling.ProfilerExtensions.InvokeAction[TInstance](InstanceAction`1 tuple) at Hangfire.Profiling.SlowLogProfiler.InvokeMeasured[TInstance,TResult](TInstance instance, Func`2 action, String message) at Hangfire.Profiling.ProfilerExtensions.InvokeMeasured[TInstance](IProfiler profiler, TInstance instance, Action`1 action, String message) at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation)
#144590302	ExecuteDataQueuePendingEventTask.ExecuteJob An exception occurred during performance of the job. More details...	10/05/2025 16:42:54
#144590302	System.InvalidOperationException Could not get a value of the job parameter `CurrentUICulture`. See inner exception for details. System.InvalidOperationException: Could not get a value of the job parameter `CurrentUICulture`. See inner exception for details. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection() at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func) at Hangfire.SqlServer.SqlServerConnection.GetJobParameter(String id, String name) at Hangfire.Server.PerformContext.GetJobParameter[T](String name) --- End of inner exception stack trace --- at Hangfire.Server.PerformContext.GetJobParameter[T](String name) at Hangfire.CaptureCultureAttribute.OnPerforming(PerformingContext filterContext) at Hangfire.Profiling.ProfilerExtensions.InvokeAction[TInstance](InstanceAction`1 tuple) at Hangfire.Profiling.SlowLogProfiler.InvokeMeasured[TInstance,TResult](TInstance instance, Func`2 action, String message) at Hangfire.Profiling.ProfilerExtensions.InvokeMeasured[TInstance](IProfiler profiler, TInstance instance, Action`1 action, String message) at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation)
#144539957	AutomaticInteractionSendAfterEventTask.ExecuteJob An exception occurred during performance of the job. More details...	10/05/2025 09:22:54
#144539957	System.Net.Http.HttpRequestException An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.IO.IOException: The response ended prematurely. at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at IdentityModel.AspNetCore.AccessTokenManagement.ClientAccessTokenHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in D:\a\IdentityModel.AspNetCore\IdentityModel.AspNetCore\src\AccessTokenManagement\ClientAccessToken\ClientAccessTokenHandler.cs:line 46 at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at DPoint.Scheduler.Infrastructure.Tasks.AutomaticInteractionSendAfterEventTask.ExecuteJob() in /src/DPoint.Scheduler.Infrastructure/Tasks/AutomaticInteractionSendAfterEventTask.cs:line 41 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#144535652	ExecuteDataCleanedUpQueueVarnaProcessingEventTask.ExecuteJob An exception occurred during performance of the job. More details...	10/05/2025 08:42:08
#144535652	System.InvalidOperationException Could not get a value of the job parameter `CurrentUICulture`. See inner exception for details. System.InvalidOperationException: Could not get a value of the job parameter `CurrentUICulture`. See inner exception for details. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection() at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func) at Hangfire.SqlServer.SqlServerConnection.GetJobParameter(String id, String name) at Hangfire.Server.PerformContext.GetJobParameter[T](String name) --- End of inner exception stack trace --- at Hangfire.Server.PerformContext.GetJobParameter[T](String name) at Hangfire.CaptureCultureAttribute.OnPerforming(PerformingContext filterContext) at Hangfire.Profiling.ProfilerExtensions.InvokeAction[TInstance](InstanceAction`1 tuple) at Hangfire.Profiling.SlowLogProfiler.InvokeMeasured[TInstance,TResult](TInstance instance, Func`2 action, String message) at Hangfire.Profiling.ProfilerExtensions.InvokeMeasured[TInstance](IProfiler profiler, TInstance instance, Action`1 action, String message) at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation)
#144531508	ExecuteDataCleanedUpQueueDaphacoSapoProcessingEventTask.ExecuteJob An exception occurred during performance of the job. More details...	10/05/2025 08:05:44
#144531508	System.Net.Http.HttpRequestException An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.IO.IOException: The response ended prematurely. at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at IdentityModel.AspNetCore.AccessTokenManagement.ClientAccessTokenHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in D:\a\IdentityModel.AspNetCore\IdentityModel.AspNetCore\src\AccessTokenManagement\ClientAccessToken\ClientAccessTokenHandler.cs:line 46 at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at DPoint.Scheduler.Infrastructure.Tasks.ExecuteDataCleanedUpQueueDaphacoSapoProcessingEventTask.ExecuteJob() in /src/DPoint.Scheduler.Infrastructure/Tasks/ExecuteDataCleanedUpQueueDaphacoSapoProcessingEventTask.cs:line 40 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#144456821	NotifyReportRedeemGiftTask.ExecuteJob An exception occurred during performance of the job. More details...	10/04/2025 22:01:26
#144456821	System.Net.Http.HttpRequestException Response status code does not indicate success: 500 (Internal Server Error). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 500 (Internal Server Error). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at DPoint.Scheduler.Infrastructure.Tasks.NotifyReportRedeemGiftTask.ExecuteJob() in /src/DPoint.Scheduler.Infrastructure/Tasks/NotifyReportRedeemGiftTask.cs:line 39 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)`
#144456798	AutoInActiveLoyaltyProgramTask.ExecuteJob An exception occurred during performance of the job. More details...	10/04/2025 22:01:49
#144456798	System.Threading.Tasks.TaskCanceledException The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. ---> System.TimeoutException: The operation was canceled. ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException (125): Operation canceled --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token) at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at IdentityModel.AspNetCore.AccessTokenManagement.ClientAccessTokenHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in D:\a\IdentityModel.AspNetCore\IdentityModel.AspNetCore\src\AccessTokenManagement\ClientAccessToken\ClientAccessTokenHandler.cs:line 46 at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) --- End of inner exception stack trace --- --- End of inner exception stack trace --- at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at DPoint.Scheduler.Infrastructure.Tasks.AutoInActiveLoyaltyProgramTask.ExecuteJob() in /src/DPoint.Scheduler.Infrastructure/Tasks/AutoInActiveLoyaltyProgramTask.cs:line 36 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#144437012	ClearFileReportTask.ExecuteJob An exception occurred during performance of the job. More details...	10/04/2025 19:01:47
#144437012	System.Threading.Tasks.TaskCanceledException The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. ---> System.TimeoutException: The operation was canceled. ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException (125): Operation canceled --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token) at System.Net.Http.HttpConnection.<CheckUsabilityOnScavenge>g__ReadAheadWithZeroByteReadAsync\|45_0() at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at IdentityModel.AspNetCore.AccessTokenManagement.ClientAccessTokenHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in D:\a\IdentityModel.AspNetCore\IdentityModel.AspNetCore\src\AccessTokenManagement\ClientAccessToken\ClientAccessTokenHandler.cs:line 46 at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) --- End of inner exception stack trace --- --- End of inner exception stack trace --- at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at DPoint.Scheduler.Infrastructure.Tasks.Reports.ClearFileReportTask.ExecuteJob() in /src/DPoint.Scheduler.Infrastructure/Tasks/Reports/ClearFileReportTask.cs:line 41 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

System.InvalidOperationException

System.InvalidOperationException

System.InvalidOperationException

System.InvalidOperationException

System.Net.Http.HttpRequestException

System.InvalidOperationException

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException

System.Threading.Tasks.TaskCanceledException

System.Threading.Tasks.TaskCanceledException