For Spark instances running on AWS EMR, install the Instana agent on the EC2 instances withing the EMR cluster. If you want automated deployment of the Spark monitoring sensor, the Instana agent must be placed on all nodes in the EMR cluster.
Instana’s Spark Monitoring includes an automatically built summary dashboard that centers around application KPIs – including response time and load. The dashboard also includes key infrastructure configuration and performance metrics, as well as specific Spark processing data metrics. The dashboard allows DevOps and IT Ops to see all relevant Spark data on one screen, making it easy to understand the state of their Spark instances.
Monitoring the health and performance of Apache Spark instances requires both an understanding of Spark, itself, as well as the ability to see the interactions and dependencies between clustered spark instances and the interactions with other microservices (both upstream and downstream). Instana’s Spark monitoring sensor automatically identifies and collects those relevant metrics.