Amazon Redshift Data API now supports long polling — session management and batch execution get more flexible too!

Hi, it's Shii! I found a nice update around AWS's Data API today, so let me tell you about it right away!

AWS What's New aws.amazon.com

What was announced?

In the AWS What's New feed, AWS announced three new capabilities for the Amazon Redshift Data API. They're improvements that cut down the number of API calls, give you visibility into sessions, and make batch execution more flexible.

Here are the three specific changes.

• Long polling — you no longer need to poll repeatedly to get a SQL statement's results
• ListSessions — you can now list and filter active sessions
• Flexible batch execution — statements in BatchExecuteStatement can now run independently

The story so far

Until now, after running a SQL statement through the Data API, you had to call DescribeStatement or GetStatementResult yourself, over and over, to check the status until the result came back. Since there was no way to know when the SQL had finished, you had no choice but to keep polling at intervals, which tended to drive up the number of API calls.

Applications that reuse sessions also had to track, on their own side, which sessions were still valid. Since the Data API had no way to check a list of sessions, you had no choice but to track session identifiers externally.

On top of that, BatchExecuteStatement was built so that if a single statement in a batch failed, the entire batch got rolled back. For workloads like ETL pipelines or admin scripts, where you'd rather let the rest keep going even if part of it fails, this was a real pain point.

What changes

With long polling, you just specify the WaitTimeSeconds parameter on ExecuteStatement, BatchExecuteStatement, DescribeStatement, GetStatementResult, or GetStatementResultV2, and the API itself will wait to respond until the SQL finishes. Since you no longer need to poll repeatedly, you can meaningfully cut down the number of API calls.

With ListSessions, you can list active sessions while filtering by status, compute target, or database. Instead of tracking session IDs yourself, you can just ask the Data API what sessions are currently running.

And BatchExecuteStatement gets a new ExecutionMode parameter (AUTO_COMMIT mode) that lets each statement in a batch run as its own independent transaction. A single failure no longer rolls back the whole batch, which is a welcome change for ETL pipelines and admin scripts where partial completion is acceptable.

Dive Deep

Let's look at a bit more detail.

The WaitTimeSeconds parameter can be specified on these five APIs.

• ExecuteStatement
• BatchExecuteStatement
• DescribeStatement
• GetStatementResult
• GetStatementResultV2

Specifying WaitTimeSeconds on these delays the response until the SQL statement reaches a terminal state (success, failure, etc). You no longer need to manage the polling interval on your application side — you get the result back at the right moment.

BatchExecuteStatement can now also accept an array of SqlParameter. Once you define parameters a single time, any statement in the batch can reference them, so you don't need to embed literal values in every query. For batch jobs that reuse the same parameters, this is a small but genuinely nice improvement.

These new capabilities are generally available for both Amazon Redshift Provisioned and Serverless, across all AWS Commercial Regions and the AWS GovCloud (US) Region.

Wrap-up

• Long polling: the WaitTimeSeconds parameter removes the need to poll until SQL completes
• ListSessions: list and filter active sessions by status, compute target, or database
• Flexible batch execution: ExecutionMode=AUTO_COMMIT means one failed statement no longer rolls back the whole batch
• An array of SqlParameter lets you reuse parameters across all statements in a batch

If you build applications on top of the Data API, or run ETL pipelines and admin scripts against Redshift, I think this is a quiet but genuinely useful update!