ApplicationMaster is the class to start YARN Application Master. Here are the functions used to start the application - Driver.

runDriver firstly invokes startUserApplication. An independent thread is started to run the main entry in user's spark job code. Usually, for the job written in Java and Scala, the main function will be invoked in this thread. This thread acts as the spark job Driver. Then RpcEnvs for Spark Application Driver and YARN Application Master will be created. Since in cluster mode, driver and application master are in the same process, they share the same ip and port. YarnRMClient communicates with YARN Resource Manager to get the resource for executors. This is done by YarnAllocator.

results matching ""

    No results matching ""