Packages

object SparkSingletons extends Logging

Singletons an initialization code related to Spark.

While Spark provides various getOrCreate methods, this allows WASP to initialize Spark in a cleaner way and in just a few places and access the various entry points in a much more straightforward way.

Linear Supertypes
Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparkSingletons
  2. Logging
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  9. def getSQLContext: SQLContext

    Returns the SQLContext singleton, or throws an exception if Spark was not initialized.

    Returns the SQLContext singleton, or throws an exception if Spark was not initialized.

    Annotations
    @throws( ... )
    Exceptions thrown

    IllegalStateException if Spark was not already initialized

  10. def getSparkContext: SparkContext

    Returns the SparkContext singleton, or throws an exception if Spark was not initialized.

    Returns the SparkContext singleton, or throws an exception if Spark was not initialized.

    Annotations
    @throws( ... )
    Exceptions thrown

    IllegalStateException if Spark was not already initialized

  11. def getSparkSession: SparkSession

    Returns the SparkSession singleton, or throws an exception if Spark was not initialized.

    Returns the SparkSession singleton, or throws an exception if Spark was not initialized.

    Annotations
    @throws( ... )
    Exceptions thrown

    IllegalStateException if Spark was not already initialized

  12. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  13. def initializeSpark(sparkConfigModel: SparkConfigModel, telemetryConfig: TelemetryConfigModel, kafkaConfigModel: KafkaConfigModel): Boolean

    Try to initialize the SparkSession in the SparkSingleton with the provided configuration.

    Try to initialize the SparkSession in the SparkSingleton with the provided configuration.

    If it does not exist, it will be created using the settings from sparkConfigModel and true will be returned.

    If the SparkSession already exists, nothing will be done, and false will be returned.

    Annotations
    @throws( ... )
    Exceptions thrown

    IllegalStateException if Spark was already initialized but not by using this method

  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. val logger: WaspLogger
    Attributes
    protected
    Definition Classes
    Logging
  16. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  17. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  18. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  19. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  20. def toString(): String
    Definition Classes
    AnyRef → Any
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  23. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated @deprecated
    Deprecated

    (Since version ) see corresponding Javadoc for more information.

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped