Spot AI Model Overuse: AI Gateway User Insights Adds Free Category Analysis

Hi everyone, it's Shii-chan! Here's a feature that helps with AI cost optimization.

Cloudflare Blog blog.cloudflare.com

What was announced?

The Cloudflare Blog announced new categories in AI Gateway User Insights. It helps teams understand AI adoption and make better model decisions, at no additional cost for AI Gateway users.

Why it matters

Are you using a high-capability model for simple work like formatting or summarization? This aims to make that visible.

What changes

The new categories:

• Task classification (coding, research, writing, summarization, data analysis)
• Model performance analysis
• Conversation turn tracking
• User and agent identification

With them you can see:

• Model overkill detection: high-capability models handling simple tasks
• Potential savings: where a faster or cheaper model could work without hurting quality
• Turn analysis: how many back-and-forth exchanges each task needed
• Cost breakdown: full task cost across multiple turns, including tokens, time, and latency

Dive Deep

A dedicated Cloudflare Worker analyzes conversation trajectories (user requests, responses, tool calls), assigns category signals with confidence scores, and evaluates task complexity. The results are joined with dashboard logs asynchronously, so no latency is added to responses.

Two caveats: User Insights is not a real-time view, and analysis trails incoming traffic by about one day. Custom applications also need stable user_id and session_id values in request metadata so activity is grouped properly.

The post also says Auto Router is launching in closed beta at the same time.

Wrap-up

• New task, model, turn, and user categories
• Shows model overuse and potential savings
• Analysis lags about a day and needs user_id and session_id
• Free for AI Gateway users

This is for teams who want to understand their LLM costs and usage!