util

Type Members

1. final case class ExasolConfiguration(host: String, port: Int, jdbc_options: String, username: String, password: String, max_nodes: Int, create_table: Boolean, drop_table: Boolean, batch_size: Int) extends Product with Serializable

   

   The configuration parameters for Spark Exasol connector.

   The configuration parameters for Spark Exasol connector.

   These can be user provided when loading or defined in Spark configurations. For example, user provided:

   df = sparkSession
     .read
     .format("exasol")
     .option("host", "127.0.0.1")
     .option("port", "8888")
     // ...
     .load

   From Spark configuration:

   val sparkConf = new SparkConf()
     .setMaster("local[*]")
     .setAppName("spark-exasol-connector")
     .set("spark.exasol.host", "localhost")
     .set("spark.exasol.port", "1234")
     // ...
   
   val sparkSession = SparkSession
     .builder()
     .config(sparkConf)
     .getOrCreate()

   If both are defined, spark configs are used. If nothing is defined, then default values are used.

2. final case class ExasolConnectionManager(config: ExasolConfiguration) extends Product with Serializable

   

   A class that provides and manages Exasol connections.

   A class that provides and manages Exasol connections.

   It is okay to serialize this class to Spark workers, it will create Exasol jdbc connections within each executor JVM.

   config

   An ExasolConfiguration with user provided or runtime configuration parameters