JA EN
shiichan

SageMaker AI Serverless Customization Now Lets You Set Your Own Sequence Length!

AWS What's New ★★★ View Markdown
#aws#ai

Hi, it's Shii-chan! Today I found news that makes fine-tuning on SageMaker AI a little more flexible, so let me share it with you!

AWS What's New aws.amazon.com

What was announced?

In AWS What's New, Amazon SageMaker AI announced that serverless model customization jobs can now be configured with a custom sequence length for training.

Serverless customization supports fine-tuning for open-weight models, including families such as gpt-oss, Gemma, Llama, Nemotron, and Qwen.

The story so far

Until now, each fine-tuning recipe ran at a fixed sequence length that customers could not change.

  • You couldn't tune the length to fit your data
  • For some models, customers asked for longer sequence lengths than the static defaults

What changes

You can now set the sequence length as a hyperparameter of the job, anywhere up to the published maximum for your chosen model and technique.

Whether you work with short question-answer pairs or multi-page documents, you can match the length to your training data. This is especially good news if you work with long documents!

Dive Deep

Which parameter you set depends on the technique:

  • SFT and DPO jobs: dataset_max_len
  • RFT jobs: max_prompt_length and max_response_length

Published maximums depend on the model and technique, and are up to 16K tokens for many models.

Available Regions:

  • US East (N. Virginia)
  • US West (Oregon)
  • Asia Pacific (Tokyo)
  • Europe (Ireland)

To get started, open the Models page in SageMaker Studio to launch a customization job, or use the SageMaker Python SDK. The announcement also mentions Agent skills as another way in.

Wrap-up

  • Serverless model customization lets you set sequence length as a hyperparameter
  • SFT / DPO use dataset_max_len; RFT uses max_prompt_length and max_response_length
  • Maximums depend on the model and technique, up to 16K tokens for many models
  • Available in 4 Regions, including Tokyo

This one is for anyone who wants to fine-tune open-weight models on longer documents!

Related posts