Troubleshooting¶
Common problems, their causes, and how to fix them.
Job Not Running¶
Symptoms: Job stays in Enqueued or Scheduled state indefinitely.
Cause 1: Dispatcher Not Running¶
NexJob jobs are executed by BackgroundService instances. If your app doesn't run as a host, the dispatcher won't start.
Diagnose: Check if your app calls host.Run() or app.Run().
Fix: Ensure the application runs as a host:
// Worker Service
var host = Host.CreateDefaultBuilder(args)
.ConfigureServices(services => services.AddNexJob())
.Build();
await host.RunAsync();
Cause 2: Queue Not Configured¶
The dispatcher only processes queues listed in options.Queues.
Diagnose: Check the job's queue name vs options.Queues.
Fix:
builder.Services.AddNexJob(options =>
{
options.Queues = new[] { "default", "emails" }; // Add the job's queue
});
Cause 3: Scheduled Job Not Due Yet¶
Scheduled jobs wait until ScheduledAt is reached.
Diagnose: Check JobRecord.ScheduledAt vs UtcNow.
Fix: Wait for the scheduled time, or enqueue immediately with EnqueueAsync instead of ScheduleAsync.
Cause 4: Workers Exhausted¶
All worker slots are occupied by long-running jobs.
Diagnose: Check the dashboard for jobs stuck in Processing. Check options.Workers.
Fix: Increase worker count or add [Throttle] to prevent resource exhaustion.
options.Workers = 50; // Increase from default 10
Job Stuck in Queue¶
Symptoms: Job shows Processing but no worker is actually running it (or it never leaves that state).
Cause: Orphaned Worker¶
A previous worker crashed while processing this job, so the job is stuck in Processing.
Diagnose: Check JobRecord.ProcessingStartedAt and JobRecord.HeartbeatAt. If UtcNow - HeartbeatAt > HeartbeatTimeout, the job is orphaned.
Fix: The OrphanedJobWatcherService handles this automatically (default: 5 minutes). Wait for the orphan watcher to re-enqueue it (a job that had already used all its attempts is marked Failed instead), or reduce the timeout:
options.HeartbeatTimeout = TimeSpan.FromMinutes(2); // Faster detection
Deadline Expiring Unexpectedly¶
Symptoms: Jobs marked as Expired before you expected.
Cause: Tight Deadline on Busy Queue¶
deadlineAfter is checked when the dispatcher picks up the job. If the queue is busy, jobs may wait longer than the deadline.
Diagnose: Check the job's ExpiresAt and compare to when it was actually picked up.
Fix: Increase the deadline or add more workers to reduce queue wait time.
await scheduler.EnqueueAsync<MyJob>(
deadlineAfter: TimeSpan.FromMinutes(30), // Increased from 5 minutes
cancellationToken: ct);
Cause: Deadline on Scheduled Job¶
deadlineAfter only works with EnqueueAsync. Scheduled jobs ignore the deadline.
Diagnose: Check if you used ScheduleAsync or ScheduleAtAsync with deadlineAfter.
Fix: There is no built-in deadline for scheduled jobs (ScheduledAt is the earliest time the job may run, not a deadline). If a late run is useless, put the deadline in the job's input and check it when the job starts:
var runAt = DateTimeOffset.UtcNow.AddMinutes(30);
await scheduler.ScheduleAtAsync<MyJob, MyInput>(
new MyInput(NotAfter: runAt.AddMinutes(10)),
runAt,
cancellationToken: ct);
// In the job:
// if (DateTimeOffset.UtcNow > input.NotAfter) return; // too late, skip
Duplicate Jobs¶
Symptoms: Same job executed multiple times with the same input.
Cause: No Idempotency Key¶
Without an idempotencyKey, every EnqueueAsync creates a new job.
Diagnose: Check if EnqueueAsync was called multiple times without idempotencyKey.
Fix: Provide an idempotency key. See Idempotency.
await scheduler.EnqueueAsync<ProcessOrderJob, ProcessOrderInput>(
new ProcessOrderInput(orderId),
idempotencyKey: $"order-{orderId}",
cancellationToken: ct);
Cause: Retry Re-Executing Same Work¶
A job partially completes an external action (e.g., charges a card) then throws. On retry, it charges again.
Diagnose: Check the dead-letter handler or job logs for duplicate external calls.
Fix: Make the job idempotent by checking state before acting.
public async Task ExecuteAsync(PaymentInput input, CancellationToken ct)
{
var alreadyCharged = await _db.Payments.ExistsAsync(input.OrderId, ct);
if (alreadyCharged) return; // Skip if already done
await _db.Payments.ChargeAsync(input.OrderId, input.Amount, ct);
}
Dashboard Not Showing Jobs¶
Symptoms: Dashboard is empty or shows no data.
Cause: Storage Mismatch¶
The dashboard reads from the same storage as the dispatcher. If they use different connection strings, the dashboard sees nothing.
Diagnose: Check that AddNexJob() and the dashboard share the same storage configuration.
Fix: Ensure both use the same provider and connection string.
Cause: No Jobs in Storage¶
Jobs may have been purged by retention policies.
Diagnose: Check RetentionSucceeded, RetentionFailed, RetentionExpired settings.
Fix: Increase retention periods or check for recent jobs:
options.RetentionSucceeded = TimeSpan.FromDays(14); // Keep longer
Cause: Dashboard Route Not Enabled¶
Diagnose: Check if UseNexJobDashboard() or AddNexJobStandaloneDashboard() is called.
Fix:
// ASP.NET Core
app.UseNexJobDashboard();
// Worker Service
builder.Services.AddNexJobStandaloneDashboard();
Cause: Missing Dashboard NuGet Package¶
Symptom: Compiler error CS1061: 'IApplicationBuilder' does not contain a definition for 'UseNexJobDashboard'.
Fix: Install the dedicated package:
# For ASP.NET Core
dotnet add package NexJob.Dashboard
# For Worker Services
dotnet add package NexJob.Dashboard.Standalone
using NexJob.Dashboard; (or using NexJob.Dashboard.Standalone;).
Cause: Missing IMemoryCache Registration¶
Symptom: Runtime exception at startup: InvalidOperationException: No service for type 'Microsoft.Extensions.Caching.Memory.IMemoryCache' has been registered.
Diagnose: The dashboard requires IMemoryCache to aggregate and cache metrics. AddNexJob() (and AddNexJobStandaloneDashboard()) already register it, so you only see this when the dashboard is mounted in a host that never called either of them.
Fix: Call AddNexJob(), or register the cache yourself in Program.cs:
builder.Services.AddMemoryCache();
Next Steps¶
- Idempotency — Prevent duplicates
- Dashboard — Monitor and debug visually
- Best Practices — Avoid these issues proactively