Wednesday, February 8, 2023

Spark SQL in Images

1. Spark's components

2. Spark SQL Architecture

3. SQL Data Types

4. Spark's context objects

5. File Formats Supported By Spark

6. SQL Workflow

7. Catalyst Optimizer

Below steps explain the workflow of the catalyst optimizer: 1. Analyzing a logical plan with the metadata 2. Optimizing the logical plan 3. Creating multiple physical plans 4. Analyzing the plans and finding the most optimal physical plan 5. Converting the physical plan to RDDs
Tags: Spark,Technology,

No comments:

Post a Comment