final class ScioContextOps extends AnyVal
Enhanced version of ScioContext with BigQuery methods.
- Alphabetic
- By Inheritance
- ScioContextOps
- AnyVal
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new ScioContextOps(self: ScioContext)
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- Any
- final def ##: Int
- Definition Classes
- Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def bigQuerySelect(sqlQuery: Query): SCollection[TableRow]
Get an SCollection for a BigQuery SELECT query.
Get an SCollection for a BigQuery SELECT query. Both Legacy SQL and Standard SQL dialects are supported. By default the query dialect will be automatically detected. To override this behavior, start the query string with
#legacysqlor#standardsql. - def bigQuerySelect(sqlQuery: Query, flattenResults: Boolean): SCollection[TableRow]
Get an SCollection for a BigQuery SELECT query.
Get an SCollection for a BigQuery SELECT query. Both Legacy SQL and Standard SQL dialects are supported. By default the query dialect will be automatically detected. To override this behavior, start the query string with
#legacysqlor#standardsql. - def bigQueryStorage(query: Query): SCollection[TableRow]
Get an SCollection for a BigQuery SELECT query using the storage API.
Get an SCollection for a BigQuery SELECT query using the storage API.
- query
SQL query
- def bigQueryStorage(table: Table, selectedFields: List[String] = BigQueryStorage.ReadParam.DefaultSelectFields, rowRestriction: String = null): SCollection[TableRow]
Get an SCollection for a BigQuery table using the storage API.
Get an SCollection for a BigQuery table using the storage API.
- selectedFields
names of the fields in the table that should be read. If empty, all fields will be read. If the specified field is a nested field, all the sub-fields in the field will be selected.
- rowRestriction
SQL text filtering statement, similar ti a WHERE clause in a query. Currently, we support combinations of predicates that are a comparison between a column and a constant value in SQL statement. Aggregates are not supported. For example:
"a > DATE '2014-09-27' AND (b > 5 AND c LIKE 'date')"
- def bigQueryTable(table: Table): SCollection[TableRow]
Get an SCollection for a BigQuery table.
- def getClass(): Class[_ <: AnyVal]
- Definition Classes
- AnyVal → Any
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def tableRowJsonFile(path: String): SCollection[TableRow]
Get an SCollection for a BigQuery TableRow JSON file.
- def toString(): String
- Definition Classes
- Any
- def typedBigQuery[T <: HasAnnotation](newSource: Option[Source])(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]
- def typedBigQuery[T <: HasAnnotation](newSource: Source)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]
- def typedBigQuery[T <: HasAnnotation]()(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]
- def typedBigQueryStorage[T <: HasAnnotation](table: Table, rowRestriction: String)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]
- def typedBigQueryStorage[T <: HasAnnotation](rowRestriction: String)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]
- def typedBigQueryStorage[T <: HasAnnotation](table: Table)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]
- def typedBigQueryStorage[T <: HasAnnotation]()(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]
Get a typed SCollection for a BigQuery storage API.
Get a typed SCollection for a BigQuery storage API.
Note that
Tmust be annotated with BigQueryType.fromStorage or BigQueryType.fromQuery - def typedBigQueryTable[T](table: Table, parseFn: (SchemaAndRecord) => T)(implicit arg0: Schema[T], arg1: Coder[T], arg2: ClassTag[T]): SCollection[T]
- def typedBigQueryTable[T](table: Table)(implicit arg0: Schema[T], arg1: Coder[T], arg2: ClassTag[T]): SCollection[T]