Incident postmortem · INC-2026-002
Talent search returned stale rankings
SEV-3·resolved
Index refresh job skipped a cycle; rankings reflected scores up to 3h old. No incorrect verifications issued.
Started
Feb 3, 2026, 09:41 AM UTC
Resolved
Feb 3, 2026, 11:15 AM UTC
Duration
1h 34m
Root cause
Cron worker was OOM-killed mid-cycle; the next scheduled run started cleanly but the gap was not detected until a recruiter flagged a known-stale profile.
Customer impact
Two recruiter seats viewed top-25 lists for two distinct SkillQuery records that excluded a profile updated within the prior 90 minutes.
Remediation
- Doubled the indexer pod memory limit and set GOMEMLIMIT to match.
- Added freshness telemetry — alert if the last-completed-cycle age > 75 minutes.
- Documented the recovery playbook step for catching up missed indexer cycles.