Lambda Kafka event sources now support OAuth authentication!
Hi everyone, it's Shii-chan! Here's OAuth support for anyone using Kafka with Lambda.
AWS What's New
aws.amazon.com
What was announced?
AWS What's New announced that AWS Lambda now supports OAuth authentication for self-managed Apache Kafka event source mappings (ESM). That covers Kafka clusters you run yourself as well as managed offerings such as Confluent Cloud, Aiven, and Redpanda.
The story so far
Kafka ESM supported only SASL/PLAIN, SASL/SCRAM, and mutual TLS (mTLS) as authentication methods. Customers in regulated industries that require OAuth couldn't use Kafka ESM with their clusters.
What changes
You can now authenticate Lambda Kafka consumers through your enterprise identity provider. With an IdP such as Amazon Cognito or Okta, you can apply the same identity governance and access policies across your Kafka clusters and Lambda consumers.
Dive Deep
Some extra details:
- Kafka ESM provides automatic scaling, error handling, batching, and event filtering, and is used for serverless Kafka consumers in cases like payment processing, fraud detection, and real-time data pipelines
- To use OAuth, create a new Kafka ESM with your authentication configuration, through the AWS Management Console, Lambda API, AWS CLI, AWS CloudFormation, or AWS SAM
- It's available in all AWS commercial Regions where self-managed Kafka ESM is available
Wrap-up
- Self-managed Kafka ESM now supports OAuth authentication
- Managed services like Confluent Cloud, Aiven, and Redpanda are covered
- Authenticate with IdPs such as Cognito or Okta and unify policies
- Configure it when creating a new ESM via the console, API, CLI, CloudFormation, or SAM
It's a great fit if you need OAuth and want Lambda-based Kafka consumers.