ai.databand.spark

Class DbndSparkListener

java.lang.Object

org.apache.spark.scheduler.SparkListener

ai.databand.spark.DbndSparkListener

All Implemented Interfaces:

org.apache.spark.scheduler.SparkListenerInterface

public class DbndSparkListener
extends org.apache.spark.scheduler.SparkListener

Spark listener to report all spark-generated metrics for stages to Databand. This class is safe to use in any circumstances and won't break client code.

Constructor Summary

Constructors

Constructor and Description
DbndSparkListener()
DbndSparkListener(DbndWrapper dbnd)

Method Summary

Modifier and Type	Method and Description
protected void	extractIoInfo(org.apache.spark.sql.execution.SparkPlanInfo plan) input location path read schema data format
void	onJobStart(org.apache.spark.scheduler.SparkListenerJobStart jobStart)

Methods inherited from class org.apache.spark.scheduler.SparkListener

onApplicationEnd, onApplicationStart, onBlockManagerAdded, onBlockManagerRemoved, onBlockUpdated, onEnvironmentUpdate, onExecutorAdded, onExecutorBlacklisted, onExecutorBlacklistedForStage, onExecutorMetricsUpdate, onExecutorRemoved, onExecutorUnblacklisted, onJobEnd, onNodeBlacklisted, onNodeBlacklistedForStage, onNodeUnblacklisted, onOtherEvent, onSpeculativeTaskSubmitted, onStageCompleted, onStageSubmitted, onTaskEnd, onTaskGettingResult, onTaskStart, onUnpersistRDD

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DbndSparkListener

public DbndSparkListener(DbndWrapper dbnd)

DbndSparkListener

public DbndSparkListener()

Method Detail

extractIoInfo

protected void extractIoInfo(org.apache.spark.sql.execution.SparkPlanInfo plan)

input location path read schema data format

Parameters:

plan -

onJobStart

public void onJobStart(org.apache.spark.scheduler.SparkListenerJobStart jobStart)

Specified by:

onJobStart in interface org.apache.spark.scheduler.SparkListenerInterface

Overrides:

onJobStart in class org.apache.spark.scheduler.SparkListener