JA EN
shiichan

Cloudflare Queues Now Shows Your Backlog in Realtime!

Cloudflare Changelog ★★★ View Markdown
#cloudflare#serverless#devops

Hey everyone, it's Shiichan! Today I found a nice observability update for anyone running message queues.

Cloudflare Changelog developers.cloudflare.com

What was announced?

Cloudflare Queues, Cloudflare's managed message queue, now exposes realtime backlog metrics. This one comes from Cloudflare's Changelog, and you can see how much is piling up in a queue through three paths: the dashboard, the REST API, and the JavaScript API.

Three new fields are available:

  • backlog_count — the number of unacknowledged messages in the queue
  • backlog_bytes — the total size of those messages in bytes
  • oldest_message_timestamp_ms — the timestamp of the oldest unacknowledged message

The story so far

Until now, getting a quick, at-a-glance answer to "how much is piling up in my queue?" took some effort. If you noticed a growing backlog late, you could miss the sign that your consumer wasn't keeping up.

What changes

Now you can watch your queue's backlog in realtime. It becomes much easier to alert when the backlog grows or to decide to scale up your consumers. You even learn how long the oldest message has been waiting, which is perfect for tracking latency.

Dive Deep

From the JavaScript API, call env.QUEUE.metrics() to get the realtime backlog:

const {
  backlogCount, // number
  backlogBytes, // number
  oldestMessageTimestamp, // Date | undefined
} = await env.QUEUE.metrics();

On top of that, env.QUEUE.send() and env.QUEUE.sendBatch() now return a metrics object on the response, so you learn the current backlog right as you send.

On the REST API side, the pull, messages, and batch endpoints now include a metadata.metrics object on the result after you consume messages.

If you want to aggregate more deeply, you can pull the same values from the GraphQL Analytics API, and you can view the realtime backlog on the dashboard. For the full details, see the Queues metrics docs.

Wrap-up

  • Cloudflare Queues added realtime backlog metrics
  • The three fields are backlog_count, backlog_bytes, and oldest_message_timestamp_ms
  • Read them from the JavaScript API's env.QUEUE.metrics(), the REST API's metadata.metrics, GraphQL, and the dashboard
  • This one lands for anyone who wants to monitor queue backlog and latency in realtime!

Related posts