How is it created
While creating SparkSession, SparkContext will also be created in getOrCreate. TODO - Add more info.
Things in SparkContext
As a function entry for Spark programming, SparkContext wraps lots of key components, including:
- LiveListenerBus: Message center for SparkListenerEvent, like StageSubmitted, StageCompleted, etc. Listeners will receive events and update their status, like UI update.
- SparkEnv: provides serializer, MapOutputTracker to track the output from map, ShuffleManager, BroadcastManager, MemoryManager.
- HeartbeatReceiver - TODO
- SchedulerBackend
- TaskScheduler
- DAGScheduler
- ExecutorAllocationManager - TODO