package util
- Alphabetic
- Public
- All
Type Members
-
final
case class
ExasolConfiguration(host: String, port: Int, jdbc_options: String, username: String, password: String, fingerprint: String, max_nodes: Int, create_table: Boolean, drop_table: Boolean, batch_size: Int) extends Product with Serializable
The configuration parameters for Spark Exasol connector.
The configuration parameters for Spark Exasol connector.
These can be user provided when loading or defined in Spark configurations. For example, user provided:
df = sparkSession .read .format("exasol") .option("host", "127.0.0.1") .option("port", "8888") // ... .load
From Spark configuration:
val sparkConf = new SparkConf() .setMaster("local[*]") .setAppName("spark-exasol-connector") .set("spark.exasol.host", "localhost") .set("spark.exasol.port", "1234") // ... val sparkSession = SparkSession .builder() .config(sparkConf) .getOrCreate()
If both are defined, spark configs are used. If nothing is defined, then default values are used.
-
final
case class
ExasolConnectionManager(config: ExasolConfiguration) extends Product with Serializable
A class that provides and manages Exasol connections.
A class that provides and manages Exasol connections.
It is okay to serialize this class to Spark workers, it will create Exasol jdbc connections within each executor JVM.
- config
An ExasolConfiguration with user provided or runtime configuration parameters
Value Members
-
object
Converter extends Logging
A helper object with functions to convert JDBC java.sql.ResultSet into Spark org.apache.spark.sql.Row or vice versa.
A helper object with functions to convert JDBC java.sql.ResultSet into Spark org.apache.spark.sql.Row or vice versa.
Most of the functions here are adapted from
spark/sql/execution/datasources/jdbc/JdbcUtils.scalaclass. -
object
ExasolConfiguration extends Serializable
A companion object that creates
ExasolConfiguration. -
object
ExasolConnectionManager extends Logging with Serializable
The companion object to ExasolConnectionManager.
-
object
Filters
A helper class with functions to create Exasol where clauses from Spark org.apache.spark.sql.sources.Filter-s.
-
object
Types extends Logging
A helper class with mapping functions between Exasol JDBC types and Spark SQL types.