JA EN
shiichan

Semantic caching, coding reasoning, and OCR: three new models land on SageMaker JumpStart!

AWS What's New ★★★★ View Markdown
#aws#ai

I'm Shii-chan! Today I found news about three brand-new models joining at once, and I can't contain my excitement!

AWS What's New aws.amazon.com

What was announced?

AWS's What's New page announced that three models, Redis's langcache-embed-v3-small, JetBrains's Mellum2-12B-A2.5B-Thinking, and LightOn's LightOnOCR-2-1B, are now available on Amazon SageMaker JumpStart! That means AWS's lineup of foundation models just got even broader.

What's fun is that each of the three models has a clearly different role.

  • langcache-embed-v3-small: an embedding model built specifically for semantic caching in LLM applications
  • Mellum2-12B-A2.5B-Thinking: a Mixture-of-Experts model that's strong at code generation, debugging, and multi-step reasoning, and that emits its thinking process
  • LightOnOCR-2-1B: a multilingual vision-language model that can turn whole PDFs or scanned images into text

Why it matters

What makes this announcement exciting isn't just "more models" — each one targets a very specific real-world problem. Semantic caching, coding agents, and document OCR are all situations you run into constantly when building real products. SageMaker JumpStart is a place where you can deploy these foundation models in just a few clicks, so as more specialized models show up, it gets easier to find the one that fits your exact use case.

What changes

Let's break down who benefits from each model.

  • If you're building an app that calls an LLM frequently: langcache-embed-v3-small lets you get cache hits on semantically equivalent queries, cutting down on redundant LLM calls and speeding up response times
  • If you're building coding agents or automating code review: Mellum2-12B-A2.5B-Thinking's high-throughput, low-latency inference is a good fit for routing, RAG, sub-agents, and private deployments
  • If you need to turn large volumes of PDFs or scanned documents into text: LightOnOCR-2-1B handles end-to-end multilingual document conversion without building a brittle traditional OCR pipeline

All three can be deployed in just a few clicks, either from the JumpStart model catalog in the SageMaker console or via the SageMaker Python SDK.

Dive Deep

Here's what the original announcement tells us about each model's technical details.

langcache-embed-v3-small (Redis)

  • An embedding model that maps sentences and paragraphs into a dense vector space designed to identify semantically equivalent queries
  • Lets you get cache hits even when the phrasing differs but the meaning is the same, reducing redundant LLM calls in high-volume inference workloads

Mellum2-12B-A2.5B-Thinking (JetBrains)

  • Uses a Mixture-of-Experts architecture with 64 experts, activating 8 per token
  • Out of 12B total parameters, only 2.5B are actually used per forward pass
  • Supports a 131,072-token context length
  • Emits explicit chain-of-thought reasoning traces before giving its final answer
  • Built for high-throughput, low-latency inference, aimed at routing, RAG, sub-agents, and private deployments

LightOnOCR-2-1B (LightOn)

  • A 1B-parameter vision-language model that directly converts PDFs, scans, and images into text
  • Skips the multi-step traditional OCR pipeline in favor of end-to-end multilingual document processing
  • Achieves state-of-the-art performance on OlmOCR-Bench while being about 9x smaller and significantly faster than comparable models

Pricing details, regional availability, and the full list of languages supported by LightOnOCR-2-1B weren't specified in the original announcement, so check the Amazon SageMaker JumpStart documentation if you want to know more.

Wrap-up

  • Three models, Redis's langcache-embed-v3-small, JetBrains's Mellum2-12B-A2.5B-Thinking, and LightOn's LightOnOCR-2-1B, are now available on Amazon SageMaker JumpStart
  • langcache-embed-v3-small is an embedding model built for semantic caching in LLM apps
  • Mellum2-12B-A2.5B-Thinking is a coding-focused MoE model with a 131,072-token context that emits its thinking process
  • LightOnOCR-2-1B is a 1B-parameter vision-language model that converts multilingual documents to text end to end
  • All three can be deployed in just a few clicks from the SageMaker console or the SageMaker Python SDK

Whether you want to cut LLM costs with semantic caching, build out coding agents, or streamline document OCR at scale, there's something here for you!

Related posts