Packages

package utils

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. case class AvroDeserializerExpression(child: Expression, schemaAvroJson: String, darwinConfig: Option[Config], avoidReevaluation: Boolean = true) extends UnaryExpression with ExpectsInputTypes with Product with Serializable

    An Expression which deserializes a binary field encoded in Avro and returns the corresponding representation in Spark.

    An Expression which deserializes a binary field encoded in Avro and returns the corresponding representation in Spark.

    child

    the Expression containing the binary Avro to be deserialized

    schemaAvroJson

    the JSON representation of the Avro schema

    darwinConfig

    the configuration for the AvroSchemaManager

    avoidReevaluation

    this filed forces the Expression to be non-deterministic. Setting this to true measn that this expression is executed only once even though the Optimizer creates several copies of it (eg. it happens usually with CollapseProject if the value returned by the expression is used several times, as with a selection of fields of the struct returned). If you set this flag to false, the expression may be evaluated once for each occurence of ti you see in the physical plan.

  2. case class AvroSerializerExpression extends UnaryExpression with ExpectsInputTypes with TimeZoneAwareExpression with Product with Serializable
  3. case class CompressExpression(codecName: String, conf: HadoopConfiguration, _child: Expression) extends UnaryExpression with ExpectsInputTypes with Product with Serializable
  4. case class DecodeUsingAvro[A](child: Expression, tag: ClassTag[A], schema: String, avroSchemaManager: () ⇒ AvroSchemaManager, fromGenericRecord: (GenericRecord) ⇒ A) extends UnaryExpression with NonSQLExpression with Product with Serializable
  5. case class EncodeUsingAvro[A](child: Expression, schema: String, avroSchemaManager: () ⇒ AvroSchemaManager, toGenericRecord: (A) ⇒ GenericRecord) extends UnaryExpression with NonSQLExpression with Product with Serializable
  6. class HBaseConnection extends GuardedLogging
  7. class HadoopConfiguration extends Serializable

    Serializable wrapper for a Hadoop Configuration

Value Members

  1. object AvroEncoders
  2. object AvroSerializerExpression extends Serializable
  3. object CompressExpression extends Serializable
  4. object DecodeUsingAvro extends Serializable
  5. object EncodeUsingAvro extends Serializable
  6. object MetadataUtils
  7. object Quartz2Utils

    Utilities for Quartz 2 scheduler.

  8. object SchemaFlatteners
  9. object SparkUtils extends Logging with WaspConfiguration with ElasticConfiguration with SparkStreamingConfiguration

    Utilities related to Spark.

  10. object ToolBoxUtils

Ungrouped