Scan Honeycomb for customer performance issues and diagnose root causes with AI
Build a read-only daily performance investigation workflow for production tenants. Trigger it on a daily cron schedule around 8:00 AM US Eastern. Query Honeycomb—or another observability platform—over the previous 24 hours to identify tenants with unusually high request latency, background-job delays, or error rates. Exclude low-volume and development/test tenants, rank meaningful threshold breaches by severity, and stop quietly when nothing warrants investigation. Send the most severe findings to an asynchronous AI investigation agent powered by Anthropic—or another capable language model. Have the agent examine relevant telemetry, traces, and anomalies in Honeycomb, then search and read the related application source code in GitHub—or another source-control platform—to identify likely root causes. Keep all investigation tools read-only. Produce a concise structured report for each affected tenant with the flagged signals, severity, suspected root cause, supporting evidence, and a small set of potential fixes ranked by effort and payoff. Do not implement or deploy any fixes. Format the completed report and send it as a direct message through Slack—or your team’s chat or incident-management tool. Send only one message after the investigation finishes, include a clear failure notice if the agent cannot complete its work, and make the destination recipient or channel easy to change. Tools used: Honeycomb, GitHub, Anthropic, Slack
What this prompt builds
This workflow scans Honeycomb daily for customer performance anomalies—high latency, job delays, or error spikes—then dispatches an AI agent to investigate telemetry, traces, and source code in GitHub. It produces a structured report of suspected root causes and ranked fix options delivered via Slack, enabling engineering teams to plan remediation proactively without distraction or customer escalation.
The problem
Engineering teams often learn about customer performance problems only after support escalations, leaving widespread or unreported issues invisible until manual investigation distracts from planned work. This workflow automatically scans Honeycomb each morning for tenants showing high latency, background-job delays, or elevated error rates, then dispatches an AI agent to investigate telemetry, traces, and source code in GitHub. It delivers a structured report to Slack listing affected customers, suspected root causes, supporting evidence, and ranked fix options—so teams can plan remediation proactively at their own pace.
Solution and impact
Teams surface performance problems before customers complain, turning reactive firefighting into planned, data-driven work. The AI agent reads observability signals and application code to propose concrete fixes ranked by effort and impact, reducing investigation overhead and helping engineers prioritize the changes that matter most. Daily automated scans replace ad-hoc digging, freeing engineering time while improving customer experience through faster, more systematic issue resolution.