Amazon SageMaker Unified Studio Notebooks now support Amazon EMR Serverless with Apache Spark Connect, giving data engineers and analysts more flexibility in choosing their Spark runtime for interactive analytics and data engineering workloads. In addition to Amazon Athena Spark, users can now leverage Amazon EMR Serverless as their Spark runtime, selecting the optimal engine based on their requirements. With this launch, you can run PySpark and Spark SQL on an EMR Serverless Spark Application in Notebook cells. Users can select their Spark runtime from the Notebook side panel, and the selecte
The continuous evolution of cloud data processing and AI/ML development platforms (like SageMaker) necessitates integration with diverse and scalable backend compute options.
This development improves flexibility and cost-efficiency for data professionals and developers who need to run large-scale analytics and data engineering workloads within a unified environment.
Users of Amazon SageMaker Unified Studio Notebooks can now directly utilize EMR Serverless for Apache Spark workloads, expanding their choices beyond Athena Spark.
- · AWS
- · Data Engineers
- · Data Analysts
- · Cloud-native ML Development
- · Traditional on-premise Spark deployments
Increased adoption of serverless Spark for data processing tasks within AWS environments.
Potential for more seamless integration of data preparation and machine learning model training workflows.
Further commoditization of big data infrastructure, shifting focus to application layer innovation rather than underlying compute management.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at AWS What's New