Why it matters
ApplicationMaster design determines how well an application handles failures. A well-designed AM detects failed task containers, requests replacements, and restarts them from checkpointed state. A poorly-designed one gives up on first failure or, worse, keeps requesting resources it cannot use. The quality of Spark or Tez as YARN citizens comes down largely to their AM quality.
Understanding AM behavior also helps diagnose stuck applications. If your job hangs at 99 percent completion, the AM is usually where the problem lives — either it is not requesting the last container correctly or it is not shutting down cleanly after work completes.
The architecture
The AM has two RPC endpoints it talks to. First is the ApplicationMasterProtocol, used to communicate with the RM: register, request containers, release containers, send heartbeats, and unregister. Second is the ContainerManagementProtocol, used to talk to NodeManagers: launch a container, stop a container, get container status.
Internally, an AM typically has a component that decides what tasks need to run (the planner), a component that requests containers based on task needs (the allocator), a component that launches tasks in allocated containers (the launcher), and a component that tracks task progress and handles failures (the monitor). The details differ per framework but this decomposition is universal.
How it works end to end
The AM starts when the RM allocates its container and instructs an NM to launch it. Once running, the AM registers with the RM by sending an ApplicationMasterRegistrationRequest. It then enters its main loop: send resource requests describing needed containers (memory, vcores, node preferences), receive allocations from the RM, launch tasks in those allocations by calling the NM, and monitor progress.
When a task container completes, the AM either treats it as success and marks the corresponding work done, or treats it as failure and reschedules the task. Failure handling policy is entirely up to the AM: how many retries, whether to blacklist nodes that fail repeatedly, whether a certain failure percentage should fail the whole application.
When all work is done, the AM sends UnregisterApplicationMasterRequest to the RM and exits. The RM marks the application succeeded and reclaims the AM's container. If the AM crashes without unregistering, the RM detects timeout on the heartbeat and either restarts the AM (up to a retry limit) or fails the application.