package proto
Type Members
-
final
class
AddArtifactsRequest extends GeneratedMessageV3 with AddArtifactsRequestOrBuilder
Request to transfer client-local artifacts.
Request to transfer client-local artifacts.
Protobuf type
spark.connect.AddArtifactsRequest - trait AddArtifactsRequestOrBuilder extends MessageOrBuilder
-
final
class
AddArtifactsResponse extends GeneratedMessageV3 with AddArtifactsResponseOrBuilder
Response to adding an artifact. Contains relevant metadata to verify successful transfer of artifact(s).
Response to adding an artifact. Contains relevant metadata to verify successful transfer of artifact(s).
Protobuf type
spark.connect.AddArtifactsResponse - trait AddArtifactsResponseOrBuilder extends MessageOrBuilder
-
final
class
Aggregate extends GeneratedMessageV3 with AggregateOrBuilder
Relation of type [[Aggregate]].
Relation of type [[Aggregate]].
Protobuf type
spark.connect.Aggregate - trait AggregateOrBuilder extends MessageOrBuilder
-
final
class
AnalyzePlanRequest extends GeneratedMessageV3 with AnalyzePlanRequestOrBuilder
Request to perform plan analyze, optionally to explain the plan.
Request to perform plan analyze, optionally to explain the plan.
Protobuf type
spark.connect.AnalyzePlanRequest - trait AnalyzePlanRequestOrBuilder extends MessageOrBuilder
-
final
class
AnalyzePlanResponse extends GeneratedMessageV3 with AnalyzePlanResponseOrBuilder
Response to performing analysis of the query. Contains relevant metadata to be able to reason about the performance.
Response to performing analysis of the query. Contains relevant metadata to be able to reason about the performance.
Protobuf type
spark.connect.AnalyzePlanResponse - trait AnalyzePlanResponseOrBuilder extends MessageOrBuilder
-
final
class
ApplyInPandasWithState extends GeneratedMessageV3 with ApplyInPandasWithStateOrBuilder
Protobuf type
spark.connect.ApplyInPandasWithState - trait ApplyInPandasWithStateOrBuilder extends MessageOrBuilder
-
final
class
ArtifactStatusesRequest extends GeneratedMessageV3 with ArtifactStatusesRequestOrBuilder
Request to get current statuses of artifacts at the server side.
Request to get current statuses of artifacts at the server side.
Protobuf type
spark.connect.ArtifactStatusesRequest - trait ArtifactStatusesRequestOrBuilder extends MessageOrBuilder
-
final
class
ArtifactStatusesResponse extends GeneratedMessageV3 with ArtifactStatusesResponseOrBuilder
Response to checking artifact statuses.
Response to checking artifact statuses.
Protobuf type
spark.connect.ArtifactStatusesResponse - trait ArtifactStatusesResponseOrBuilder extends MessageOrBuilder
- final class Base extends AnyRef
-
final
class
CacheTable extends GeneratedMessageV3 with CacheTableOrBuilder
See `spark.catalog.cacheTable`
See `spark.catalog.cacheTable`
Protobuf type
spark.connect.CacheTable - trait CacheTableOrBuilder extends MessageOrBuilder
-
final
class
CachedLocalRelation extends GeneratedMessageV3 with CachedLocalRelationOrBuilder
A local relation that has been cached already.
A local relation that has been cached already.
Protobuf type
spark.connect.CachedLocalRelation - trait CachedLocalRelationOrBuilder extends MessageOrBuilder
-
final
class
CachedRemoteRelation extends GeneratedMessageV3 with CachedRemoteRelationOrBuilder
Represents a remote relation that has been cached on server.
Represents a remote relation that has been cached on server.
Protobuf type
spark.connect.CachedRemoteRelation - trait CachedRemoteRelationOrBuilder extends MessageOrBuilder
-
final
class
CallFunction extends GeneratedMessageV3 with CallFunctionOrBuilder
Protobuf type
spark.connect.CallFunction - trait CallFunctionOrBuilder extends MessageOrBuilder
-
final
class
Catalog extends GeneratedMessageV3 with CatalogOrBuilder
Catalog messages are marked as unstable.
Catalog messages are marked as unstable.
Protobuf type
spark.connect.Catalog - trait CatalogOrBuilder extends MessageOrBuilder
- final class CatalogOuterClass extends AnyRef
-
final
class
ClearCache extends GeneratedMessageV3 with ClearCacheOrBuilder
See `spark.catalog.clearCache`
See `spark.catalog.clearCache`
Protobuf type
spark.connect.ClearCache - trait ClearCacheOrBuilder extends MessageOrBuilder
-
final
class
CoGroupMap extends GeneratedMessageV3 with CoGroupMapOrBuilder
Protobuf type
spark.connect.CoGroupMap - trait CoGroupMapOrBuilder extends MessageOrBuilder
-
final
class
CollectMetrics extends GeneratedMessageV3 with CollectMetricsOrBuilder
Collect arbitrary (named) metrics from a dataset.
Collect arbitrary (named) metrics from a dataset.
Protobuf type
spark.connect.CollectMetrics - trait CollectMetricsOrBuilder extends MessageOrBuilder
-
final
class
Command extends GeneratedMessageV3 with CommandOrBuilder
A [[Command]] is an operation that is executed by the server that does not directly consume or produce a relational result.
A [[Command]] is an operation that is executed by the server that does not directly consume or produce a relational result.
Protobuf type
spark.connect.Command - trait CommandOrBuilder extends MessageOrBuilder
- final class Commands extends AnyRef
- final class Common extends AnyRef
-
final
class
CommonInlineUserDefinedFunction extends GeneratedMessageV3 with CommonInlineUserDefinedFunctionOrBuilder
Protobuf type
spark.connect.CommonInlineUserDefinedFunction - trait CommonInlineUserDefinedFunctionOrBuilder extends MessageOrBuilder
-
final
class
CommonInlineUserDefinedTableFunction extends GeneratedMessageV3 with CommonInlineUserDefinedTableFunctionOrBuilder
Protobuf type
spark.connect.CommonInlineUserDefinedTableFunction - trait CommonInlineUserDefinedTableFunctionOrBuilder extends MessageOrBuilder
-
final
class
ConfigRequest extends GeneratedMessageV3 with ConfigRequestOrBuilder
Request to update or fetch the configurations.
Request to update or fetch the configurations.
Protobuf type
spark.connect.ConfigRequest - trait ConfigRequestOrBuilder extends MessageOrBuilder
-
final
class
ConfigResponse extends GeneratedMessageV3 with ConfigResponseOrBuilder
Response to the config request.
Response to the config request.
Protobuf type
spark.connect.ConfigResponse - trait ConfigResponseOrBuilder extends MessageOrBuilder
-
final
class
CreateDataFrameViewCommand extends GeneratedMessageV3 with CreateDataFrameViewCommandOrBuilder
A command that can create DataFrame global temp view or local temp view.
A command that can create DataFrame global temp view or local temp view.
Protobuf type
spark.connect.CreateDataFrameViewCommand - trait CreateDataFrameViewCommandOrBuilder extends MessageOrBuilder
-
final
class
CreateExternalTable extends GeneratedMessageV3 with CreateExternalTableOrBuilder
See `spark.catalog.createExternalTable`
See `spark.catalog.createExternalTable`
Protobuf type
spark.connect.CreateExternalTable - trait CreateExternalTableOrBuilder extends MessageOrBuilder
-
final
class
CreateTable extends GeneratedMessageV3 with CreateTableOrBuilder
See `spark.catalog.createTable`
See `spark.catalog.createTable`
Protobuf type
spark.connect.CreateTable - trait CreateTableOrBuilder extends MessageOrBuilder
-
final
class
CurrentCatalog extends GeneratedMessageV3 with CurrentCatalogOrBuilder
See `spark.catalog.currentCatalog`
See `spark.catalog.currentCatalog`
Protobuf type
spark.connect.CurrentCatalog - trait CurrentCatalogOrBuilder extends MessageOrBuilder
-
final
class
CurrentDatabase extends GeneratedMessageV3 with CurrentDatabaseOrBuilder
See `spark.catalog.currentDatabase`
See `spark.catalog.currentDatabase`
Protobuf type
spark.connect.CurrentDatabase - trait CurrentDatabaseOrBuilder extends MessageOrBuilder
-
final
class
DataType extends GeneratedMessageV3 with DataTypeOrBuilder
This message describes the logical [[DataType]] of something. It does not carry the value itself but only describes it.
This message describes the logical [[DataType]] of something. It does not carry the value itself but only describes it.
Protobuf type
spark.connect.DataType - trait DataTypeOrBuilder extends MessageOrBuilder
-
final
class
DatabaseExists extends GeneratedMessageV3 with DatabaseExistsOrBuilder
See `spark.catalog.databaseExists`
See `spark.catalog.databaseExists`
Protobuf type
spark.connect.DatabaseExists - trait DatabaseExistsOrBuilder extends MessageOrBuilder
-
final
class
Deduplicate extends GeneratedMessageV3 with DeduplicateOrBuilder
Relation of type [[Deduplicate]] which have duplicate rows removed, could consider either only the subset of columns or all the columns.
Relation of type [[Deduplicate]] which have duplicate rows removed, could consider either only the subset of columns or all the columns.
Protobuf type
spark.connect.Deduplicate - trait DeduplicateOrBuilder extends MessageOrBuilder
-
final
class
Drop extends GeneratedMessageV3 with DropOrBuilder
Drop specified columns.
Drop specified columns.
Protobuf type
spark.connect.Drop -
final
class
DropGlobalTempView extends GeneratedMessageV3 with DropGlobalTempViewOrBuilder
See `spark.catalog.dropGlobalTempView`
See `spark.catalog.dropGlobalTempView`
Protobuf type
spark.connect.DropGlobalTempView - trait DropGlobalTempViewOrBuilder extends MessageOrBuilder
- trait DropOrBuilder extends MessageOrBuilder
-
final
class
DropTempView extends GeneratedMessageV3 with DropTempViewOrBuilder
See `spark.catalog.dropTempView`
See `spark.catalog.dropTempView`
Protobuf type
spark.connect.DropTempView - trait DropTempViewOrBuilder extends MessageOrBuilder
-
final
class
ExamplePluginCommand extends GeneratedMessageV3 with ExamplePluginCommandOrBuilder
Protobuf type
spark.connect.ExamplePluginCommand - trait ExamplePluginCommandOrBuilder extends MessageOrBuilder
-
final
class
ExamplePluginExpression extends GeneratedMessageV3 with ExamplePluginExpressionOrBuilder
Protobuf type
spark.connect.ExamplePluginExpression - trait ExamplePluginExpressionOrBuilder extends MessageOrBuilder
-
final
class
ExamplePluginRelation extends GeneratedMessageV3 with ExamplePluginRelationOrBuilder
Protobuf type
spark.connect.ExamplePluginRelation - trait ExamplePluginRelationOrBuilder extends MessageOrBuilder
- final class ExamplePlugins extends AnyRef
-
final
class
ExecutePlanRequest extends GeneratedMessageV3 with ExecutePlanRequestOrBuilder
A request to be executed by the service.
A request to be executed by the service.
Protobuf type
spark.connect.ExecutePlanRequest - trait ExecutePlanRequestOrBuilder extends MessageOrBuilder
-
final
class
ExecutePlanResponse extends GeneratedMessageV3 with ExecutePlanResponseOrBuilder
The response of a query, can be one or more for each request. Responses belonging to the same input query, carry the same `session_id`.
The response of a query, can be one or more for each request. Responses belonging to the same input query, carry the same `session_id`.
Protobuf type
spark.connect.ExecutePlanResponse - trait ExecutePlanResponseOrBuilder extends MessageOrBuilder
-
final
class
Expression extends GeneratedMessageV3 with ExpressionOrBuilder
Expression used to refer to fields, functions and similar. This can be used everywhere expressions in SQL appear.
Expression used to refer to fields, functions and similar. This can be used everywhere expressions in SQL appear.
Protobuf type
spark.connect.Expression - trait ExpressionOrBuilder extends MessageOrBuilder
- final class Expressions extends AnyRef
-
final
class
Filter extends GeneratedMessageV3 with FilterOrBuilder
Relation that applies a boolean expression `condition` on each row of `input` to produce the output result.
Relation that applies a boolean expression `condition` on each row of `input` to produce the output result.
Protobuf type
spark.connect.Filter - trait FilterOrBuilder extends MessageOrBuilder
-
final
class
FunctionExists extends GeneratedMessageV3 with FunctionExistsOrBuilder
See `spark.catalog.functionExists`
See `spark.catalog.functionExists`
Protobuf type
spark.connect.FunctionExists - trait FunctionExistsOrBuilder extends MessageOrBuilder
-
final
class
GetDatabase extends GeneratedMessageV3 with GetDatabaseOrBuilder
See `spark.catalog.getDatabase`
See `spark.catalog.getDatabase`
Protobuf type
spark.connect.GetDatabase - trait GetDatabaseOrBuilder extends MessageOrBuilder
-
final
class
GetFunction extends GeneratedMessageV3 with GetFunctionOrBuilder
See `spark.catalog.getFunction`
See `spark.catalog.getFunction`
Protobuf type
spark.connect.GetFunction - trait GetFunctionOrBuilder extends MessageOrBuilder
-
final
class
GetResourcesCommand extends GeneratedMessageV3 with GetResourcesCommandOrBuilder
Command to get the output of 'SparkContext.resources'
Command to get the output of 'SparkContext.resources'
Protobuf type
spark.connect.GetResourcesCommand - trait GetResourcesCommandOrBuilder extends MessageOrBuilder
-
final
class
GetResourcesCommandResult extends GeneratedMessageV3 with GetResourcesCommandResultOrBuilder
Response for command 'GetResourcesCommand'.
Response for command 'GetResourcesCommand'.
Protobuf type
spark.connect.GetResourcesCommandResult - trait GetResourcesCommandResultOrBuilder extends MessageOrBuilder
-
final
class
GetTable extends GeneratedMessageV3 with GetTableOrBuilder
See `spark.catalog.getTable`
See `spark.catalog.getTable`
Protobuf type
spark.connect.GetTable - trait GetTableOrBuilder extends MessageOrBuilder
-
final
class
GroupMap extends GeneratedMessageV3 with GroupMapOrBuilder
Protobuf type
spark.connect.GroupMap - trait GroupMapOrBuilder extends MessageOrBuilder
-
final
class
Hint extends GeneratedMessageV3 with HintOrBuilder
Specify a hint over a relation. Hint should have a name and optional parameters.
Specify a hint over a relation. Hint should have a name and optional parameters.
Protobuf type
spark.connect.Hint - trait HintOrBuilder extends MessageOrBuilder
-
final
class
HtmlString extends GeneratedMessageV3 with HtmlStringOrBuilder
Compose the string representing rows for output. It will invoke 'Dataset.htmlString' to compute the results.
Compose the string representing rows for output. It will invoke 'Dataset.htmlString' to compute the results.
Protobuf type
spark.connect.HtmlString - trait HtmlStringOrBuilder extends MessageOrBuilder
-
final
class
InterruptRequest extends GeneratedMessageV3 with InterruptRequestOrBuilder
Protobuf type
spark.connect.InterruptRequest - trait InterruptRequestOrBuilder extends MessageOrBuilder
-
final
class
InterruptResponse extends GeneratedMessageV3 with InterruptResponseOrBuilder
Protobuf type
spark.connect.InterruptResponse - trait InterruptResponseOrBuilder extends MessageOrBuilder
-
final
class
IsCached extends GeneratedMessageV3 with IsCachedOrBuilder
See `spark.catalog.isCached`
See `spark.catalog.isCached`
Protobuf type
spark.connect.IsCached - trait IsCachedOrBuilder extends MessageOrBuilder
-
final
class
JavaUDF extends GeneratedMessageV3 with JavaUDFOrBuilder
Protobuf type
spark.connect.JavaUDF - trait JavaUDFOrBuilder extends MessageOrBuilder
-
final
class
Join extends GeneratedMessageV3 with JoinOrBuilder
Relation of type [[Join]]. `left` and `right` must be present.
Relation of type [[Join]]. `left` and `right` must be present.
Protobuf type
spark.connect.Join - trait JoinOrBuilder extends MessageOrBuilder
-
final
class
KeyValue extends GeneratedMessageV3 with KeyValueOrBuilder
The key-value pair for the config request and response.
The key-value pair for the config request and response.
Protobuf type
spark.connect.KeyValue - trait KeyValueOrBuilder extends MessageOrBuilder
-
final
class
Limit extends GeneratedMessageV3 with LimitOrBuilder
Relation of type [[Limit]] that is used to `limit` rows from the input relation.
Relation of type [[Limit]] that is used to `limit` rows from the input relation.
Protobuf type
spark.connect.Limit - trait LimitOrBuilder extends MessageOrBuilder
-
final
class
ListCatalogs extends GeneratedMessageV3 with ListCatalogsOrBuilder
See `spark.catalog.listCatalogs`
See `spark.catalog.listCatalogs`
Protobuf type
spark.connect.ListCatalogs - trait ListCatalogsOrBuilder extends MessageOrBuilder
-
final
class
ListColumns extends GeneratedMessageV3 with ListColumnsOrBuilder
See `spark.catalog.listColumns`
See `spark.catalog.listColumns`
Protobuf type
spark.connect.ListColumns - trait ListColumnsOrBuilder extends MessageOrBuilder
-
final
class
ListDatabases extends GeneratedMessageV3 with ListDatabasesOrBuilder
See `spark.catalog.listDatabases`
See `spark.catalog.listDatabases`
Protobuf type
spark.connect.ListDatabases - trait ListDatabasesOrBuilder extends MessageOrBuilder
-
final
class
ListFunctions extends GeneratedMessageV3 with ListFunctionsOrBuilder
See `spark.catalog.listFunctions`
See `spark.catalog.listFunctions`
Protobuf type
spark.connect.ListFunctions - trait ListFunctionsOrBuilder extends MessageOrBuilder
-
final
class
ListTables extends GeneratedMessageV3 with ListTablesOrBuilder
See `spark.catalog.listTables`
See `spark.catalog.listTables`
Protobuf type
spark.connect.ListTables - trait ListTablesOrBuilder extends MessageOrBuilder
-
final
class
LocalRelation extends GeneratedMessageV3 with LocalRelationOrBuilder
A relation that does not need to be qualified by name.
A relation that does not need to be qualified by name.
Protobuf type
spark.connect.LocalRelation - trait LocalRelationOrBuilder extends MessageOrBuilder
-
final
class
MapPartitions extends GeneratedMessageV3 with MapPartitionsOrBuilder
Protobuf type
spark.connect.MapPartitions - trait MapPartitionsOrBuilder extends MessageOrBuilder
-
final
class
NADrop extends GeneratedMessageV3 with NADropOrBuilder
Drop rows containing null values. It will invoke 'Dataset.na.drop' (same as 'DataFrameNaFunctions.drop') to compute the results.
Drop rows containing null values. It will invoke 'Dataset.na.drop' (same as 'DataFrameNaFunctions.drop') to compute the results.
Protobuf type
spark.connect.NADrop - trait NADropOrBuilder extends MessageOrBuilder
-
final
class
NAFill extends GeneratedMessageV3 with NAFillOrBuilder
Replaces null values. It will invoke 'Dataset.na.fill' (same as 'DataFrameNaFunctions.fill') to compute the results. Following 3 parameter combinations are supported: 1, 'values' only contains 1 item, 'cols' is empty: replaces null values in all type-compatible columns. 2, 'values' only contains 1 item, 'cols' is not empty: replaces null values in specified columns. 3, 'values' contains more than 1 items, then 'cols' is required to have the same length: replaces each specified column with corresponding value.
Replaces null values. It will invoke 'Dataset.na.fill' (same as 'DataFrameNaFunctions.fill') to compute the results. Following 3 parameter combinations are supported: 1, 'values' only contains 1 item, 'cols' is empty: replaces null values in all type-compatible columns. 2, 'values' only contains 1 item, 'cols' is not empty: replaces null values in specified columns. 3, 'values' contains more than 1 items, then 'cols' is required to have the same length: replaces each specified column with corresponding value.
Protobuf type
spark.connect.NAFill - trait NAFillOrBuilder extends MessageOrBuilder
-
final
class
NAReplace extends GeneratedMessageV3 with NAReplaceOrBuilder
Replaces old values with the corresponding values. It will invoke 'Dataset.na.replace' (same as 'DataFrameNaFunctions.replace') to compute the results.
Replaces old values with the corresponding values. It will invoke 'Dataset.na.replace' (same as 'DataFrameNaFunctions.replace') to compute the results.
Protobuf type
spark.connect.NAReplace - trait NAReplaceOrBuilder extends MessageOrBuilder
-
final
class
Offset extends GeneratedMessageV3 with OffsetOrBuilder
Relation of type [[Offset]] that is used to read rows staring from the `offset` on the input relation.
Relation of type [[Offset]] that is used to read rows staring from the `offset` on the input relation.
Protobuf type
spark.connect.Offset - trait OffsetOrBuilder extends MessageOrBuilder
-
final
class
Parse extends GeneratedMessageV3 with ParseOrBuilder
Protobuf type
spark.connect.Parse - trait ParseOrBuilder extends MessageOrBuilder
-
final
class
Plan extends GeneratedMessageV3 with PlanOrBuilder
A [[Plan]] is the structure that carries the runtime information for the execution from the client to the server. A [[Plan]] can either be of the type [[Relation]] which is a reference to the underlying logical plan or it can be of the [[Command]] type that is used to execute commands on the server.
A [[Plan]] is the structure that carries the runtime information for the execution from the client to the server. A [[Plan]] can either be of the type [[Relation]] which is a reference to the underlying logical plan or it can be of the [[Command]] type that is used to execute commands on the server.
Protobuf type
spark.connect.Plan - trait PlanOrBuilder extends MessageOrBuilder
-
final
class
Project extends GeneratedMessageV3 with ProjectOrBuilder
Projection of a bag of expressions for a given input relation. The input relation must be specified. The projected expression can be an arbitrary expression.
Projection of a bag of expressions for a given input relation. The input relation must be specified. The projected expression can be an arbitrary expression.
Protobuf type
spark.connect.Project - trait ProjectOrBuilder extends MessageOrBuilder
-
final
class
PythonUDF extends GeneratedMessageV3 with PythonUDFOrBuilder
Protobuf type
spark.connect.PythonUDF - trait PythonUDFOrBuilder extends MessageOrBuilder
-
final
class
PythonUDTF extends GeneratedMessageV3 with PythonUDTFOrBuilder
Protobuf type
spark.connect.PythonUDTF - trait PythonUDTFOrBuilder extends MessageOrBuilder
-
final
class
Range extends GeneratedMessageV3 with RangeOrBuilder
Relation of type [[Range]] that generates a sequence of integers.
Relation of type [[Range]] that generates a sequence of integers.
Protobuf type
spark.connect.Range - trait RangeOrBuilder extends MessageOrBuilder
-
final
class
Read extends GeneratedMessageV3 with ReadOrBuilder
Relation that reads from a file / table or other data source. Does not have additional inputs.
Relation that reads from a file / table or other data source. Does not have additional inputs.
Protobuf type
spark.connect.Read - trait ReadOrBuilder extends MessageOrBuilder
-
final
class
ReattachExecuteRequest extends GeneratedMessageV3 with ReattachExecuteRequestOrBuilder
Protobuf type
spark.connect.ReattachExecuteRequest - trait ReattachExecuteRequestOrBuilder extends MessageOrBuilder
-
final
class
ReattachOptions extends GeneratedMessageV3 with ReattachOptionsOrBuilder
Protobuf type
spark.connect.ReattachOptions - trait ReattachOptionsOrBuilder extends MessageOrBuilder
-
final
class
RecoverPartitions extends GeneratedMessageV3 with RecoverPartitionsOrBuilder
See `spark.catalog.recoverPartitions`
See `spark.catalog.recoverPartitions`
Protobuf type
spark.connect.RecoverPartitions - trait RecoverPartitionsOrBuilder extends MessageOrBuilder
-
final
class
RefreshByPath extends GeneratedMessageV3 with RefreshByPathOrBuilder
See `spark.catalog.refreshByPath`
See `spark.catalog.refreshByPath`
Protobuf type
spark.connect.RefreshByPath - trait RefreshByPathOrBuilder extends MessageOrBuilder
-
final
class
RefreshTable extends GeneratedMessageV3 with RefreshTableOrBuilder
See `spark.catalog.refreshTable`
See `spark.catalog.refreshTable`
Protobuf type
spark.connect.RefreshTable - trait RefreshTableOrBuilder extends MessageOrBuilder
-
final
class
Relation extends GeneratedMessageV3 with RelationOrBuilder
The main [[Relation]] type. Fundamentally, a relation is a typed container that has exactly one explicit relation type set. When adding new relation types, they have to be registered here.
The main [[Relation]] type. Fundamentally, a relation is a typed container that has exactly one explicit relation type set. When adding new relation types, they have to be registered here.
Protobuf type
spark.connect.Relation -
final
class
RelationCommon extends GeneratedMessageV3 with RelationCommonOrBuilder
Common metadata of all relations.
Common metadata of all relations.
Protobuf type
spark.connect.RelationCommon - trait RelationCommonOrBuilder extends MessageOrBuilder
- trait RelationOrBuilder extends MessageOrBuilder
- final class Relations extends AnyRef
-
final
class
ReleaseExecuteRequest extends GeneratedMessageV3 with ReleaseExecuteRequestOrBuilder
Protobuf type
spark.connect.ReleaseExecuteRequest - trait ReleaseExecuteRequestOrBuilder extends MessageOrBuilder
-
final
class
ReleaseExecuteResponse extends GeneratedMessageV3 with ReleaseExecuteResponseOrBuilder
Protobuf type
spark.connect.ReleaseExecuteResponse - trait ReleaseExecuteResponseOrBuilder extends MessageOrBuilder
-
final
class
Repartition extends GeneratedMessageV3 with RepartitionOrBuilder
Relation repartition.
Relation repartition.
Protobuf type
spark.connect.Repartition -
final
class
RepartitionByExpression extends GeneratedMessageV3 with RepartitionByExpressionOrBuilder
Protobuf type
spark.connect.RepartitionByExpression - trait RepartitionByExpressionOrBuilder extends MessageOrBuilder
- trait RepartitionOrBuilder extends MessageOrBuilder
-
final
class
ResourceInformation extends GeneratedMessageV3 with ResourceInformationOrBuilder
ResourceInformation to hold information about a type of Resource. The corresponding class is 'org.apache.spark.resource.ResourceInformation'
ResourceInformation to hold information about a type of Resource. The corresponding class is 'org.apache.spark.resource.ResourceInformation'
Protobuf type
spark.connect.ResourceInformation - trait ResourceInformationOrBuilder extends MessageOrBuilder
-
final
class
SQL extends GeneratedMessageV3 with SQLOrBuilder
Relation that uses a SQL query to generate the output.
Relation that uses a SQL query to generate the output.
Protobuf type
spark.connect.SQL - trait SQLOrBuilder extends MessageOrBuilder
-
final
class
Sample extends GeneratedMessageV3 with SampleOrBuilder
Relation of type [[Sample]] that samples a fraction of the dataset.
Relation of type [[Sample]] that samples a fraction of the dataset.
Protobuf type
spark.connect.Sample - trait SampleOrBuilder extends MessageOrBuilder
-
final
class
ScalarScalaUDF extends GeneratedMessageV3 with ScalarScalaUDFOrBuilder
Protobuf type
spark.connect.ScalarScalaUDF - trait ScalarScalaUDFOrBuilder extends MessageOrBuilder
-
final
class
SetCurrentCatalog extends GeneratedMessageV3 with SetCurrentCatalogOrBuilder
See `spark.catalog.setCurrentCatalog`
See `spark.catalog.setCurrentCatalog`
Protobuf type
spark.connect.SetCurrentCatalog - trait SetCurrentCatalogOrBuilder extends MessageOrBuilder
-
final
class
SetCurrentDatabase extends GeneratedMessageV3 with SetCurrentDatabaseOrBuilder
See `spark.catalog.setCurrentDatabase`
See `spark.catalog.setCurrentDatabase`
Protobuf type
spark.connect.SetCurrentDatabase - trait SetCurrentDatabaseOrBuilder extends MessageOrBuilder
-
final
class
SetOperation extends GeneratedMessageV3 with SetOperationOrBuilder
Relation of type [[SetOperation]]
Relation of type [[SetOperation]]
Protobuf type
spark.connect.SetOperation - trait SetOperationOrBuilder extends MessageOrBuilder
-
final
class
ShowString extends GeneratedMessageV3 with ShowStringOrBuilder
Compose the string representing rows for output. It will invoke 'Dataset.showString' to compute the results.
Compose the string representing rows for output. It will invoke 'Dataset.showString' to compute the results.
Protobuf type
spark.connect.ShowString - trait ShowStringOrBuilder extends MessageOrBuilder
-
final
class
Sort extends GeneratedMessageV3 with SortOrBuilder
Relation of type [[Sort]].
Relation of type [[Sort]].
Protobuf type
spark.connect.Sort - trait SortOrBuilder extends MessageOrBuilder
-
final
class
SparkConnectServiceGrpc extends AnyRef
Main interface for the SparkConnect service.
Main interface for the SparkConnect service.
- Annotations
- @Generated() @GrpcGenerated()
-
final
class
SqlCommand extends GeneratedMessageV3 with SqlCommandOrBuilder
A SQL Command is used to trigger the eager evaluation of SQL commands in Spark. When the SQL provide as part of the message is a command it will be immediately evaluated and the result will be collected and returned as part of a LocalRelation. If the result is not a command, the operation will simply return a SQL Relation. This allows the client to be almost oblivious to the server-side behavior.
A SQL Command is used to trigger the eager evaluation of SQL commands in Spark. When the SQL provide as part of the message is a command it will be immediately evaluated and the result will be collected and returned as part of a LocalRelation. If the result is not a command, the operation will simply return a SQL Relation. This allows the client to be almost oblivious to the server-side behavior.
Protobuf type
spark.connect.SqlCommand - trait SqlCommandOrBuilder extends MessageOrBuilder
-
final
class
StatApproxQuantile extends GeneratedMessageV3 with StatApproxQuantileOrBuilder
Calculates the approximate quantiles of numerical columns of a DataFrame. It will invoke 'Dataset.stat.approxQuantile' (same as 'StatFunctions.approxQuantile') to compute the results.
Calculates the approximate quantiles of numerical columns of a DataFrame. It will invoke 'Dataset.stat.approxQuantile' (same as 'StatFunctions.approxQuantile') to compute the results.
Protobuf type
spark.connect.StatApproxQuantile - trait StatApproxQuantileOrBuilder extends MessageOrBuilder
-
final
class
StatCorr extends GeneratedMessageV3 with StatCorrOrBuilder
Calculates the correlation of two columns of a DataFrame. Currently only supports the Pearson Correlation Coefficient. It will invoke 'Dataset.stat.corr' (same as 'StatFunctions.pearsonCorrelation') to compute the results.
Calculates the correlation of two columns of a DataFrame. Currently only supports the Pearson Correlation Coefficient. It will invoke 'Dataset.stat.corr' (same as 'StatFunctions.pearsonCorrelation') to compute the results.
Protobuf type
spark.connect.StatCorr - trait StatCorrOrBuilder extends MessageOrBuilder
-
final
class
StatCov extends GeneratedMessageV3 with StatCovOrBuilder
Calculate the sample covariance of two numerical columns of a DataFrame. It will invoke 'Dataset.stat.cov' (same as 'StatFunctions.calculateCov') to compute the results.
Calculate the sample covariance of two numerical columns of a DataFrame. It will invoke 'Dataset.stat.cov' (same as 'StatFunctions.calculateCov') to compute the results.
Protobuf type
spark.connect.StatCov - trait StatCovOrBuilder extends MessageOrBuilder
-
final
class
StatCrosstab extends GeneratedMessageV3 with StatCrosstabOrBuilder
Computes a pair-wise frequency table of the given columns. Also known as a contingency table. It will invoke 'Dataset.stat.crosstab' (same as 'StatFunctions.crossTabulate') to compute the results.
Computes a pair-wise frequency table of the given columns. Also known as a contingency table. It will invoke 'Dataset.stat.crosstab' (same as 'StatFunctions.crossTabulate') to compute the results.
Protobuf type
spark.connect.StatCrosstab - trait StatCrosstabOrBuilder extends MessageOrBuilder
-
final
class
StatDescribe extends GeneratedMessageV3 with StatDescribeOrBuilder
Computes basic statistics for numeric and string columns, including count, mean, stddev, min, and max. If no columns are given, this function computes statistics for all numerical or string columns.
Computes basic statistics for numeric and string columns, including count, mean, stddev, min, and max. If no columns are given, this function computes statistics for all numerical or string columns.
Protobuf type
spark.connect.StatDescribe - trait StatDescribeOrBuilder extends MessageOrBuilder
-
final
class
StatFreqItems extends GeneratedMessageV3 with StatFreqItemsOrBuilder
Finding frequent items for columns, possibly with false positives. It will invoke 'Dataset.stat.freqItems' (same as 'StatFunctions.freqItems') to compute the results.
Finding frequent items for columns, possibly with false positives. It will invoke 'Dataset.stat.freqItems' (same as 'StatFunctions.freqItems') to compute the results.
Protobuf type
spark.connect.StatFreqItems - trait StatFreqItemsOrBuilder extends MessageOrBuilder
-
final
class
StatSampleBy extends GeneratedMessageV3 with StatSampleByOrBuilder
Returns a stratified sample without replacement based on the fraction given on each stratum. It will invoke 'Dataset.stat.freqItems' (same as 'StatFunctions.freqItems') to compute the results.
Returns a stratified sample without replacement based on the fraction given on each stratum. It will invoke 'Dataset.stat.freqItems' (same as 'StatFunctions.freqItems') to compute the results.
Protobuf type
spark.connect.StatSampleBy - trait StatSampleByOrBuilder extends MessageOrBuilder
-
final
class
StatSummary extends GeneratedMessageV3 with StatSummaryOrBuilder
Computes specified statistics for numeric and string columns. It will invoke 'Dataset.summary' (same as 'StatFunctions.summary') to compute the results.
Computes specified statistics for numeric and string columns. It will invoke 'Dataset.summary' (same as 'StatFunctions.summary') to compute the results.
Protobuf type
spark.connect.StatSummary - trait StatSummaryOrBuilder extends MessageOrBuilder
-
final
class
StorageLevel extends GeneratedMessageV3 with StorageLevelOrBuilder
StorageLevel for persisting Datasets/Tables.
StorageLevel for persisting Datasets/Tables.
Protobuf type
spark.connect.StorageLevel - trait StorageLevelOrBuilder extends MessageOrBuilder
-
final
class
StreamingForeachFunction extends GeneratedMessageV3 with StreamingForeachFunctionOrBuilder
Protobuf type
spark.connect.StreamingForeachFunction - trait StreamingForeachFunctionOrBuilder extends MessageOrBuilder
-
final
class
StreamingQueryCommand extends GeneratedMessageV3 with StreamingQueryCommandOrBuilder
Commands for a streaming query.
Commands for a streaming query.
Protobuf type
spark.connect.StreamingQueryCommand - trait StreamingQueryCommandOrBuilder extends MessageOrBuilder
-
final
class
StreamingQueryCommandResult extends GeneratedMessageV3 with StreamingQueryCommandResultOrBuilder
Response for commands on a streaming query.
Response for commands on a streaming query.
Protobuf type
spark.connect.StreamingQueryCommandResult - trait StreamingQueryCommandResultOrBuilder extends MessageOrBuilder
-
final
class
StreamingQueryInstanceId extends GeneratedMessageV3 with StreamingQueryInstanceIdOrBuilder
A tuple that uniquely identifies an instance of streaming query run. It consists of `id` that persists across the streaming runs and `run_id` that changes between each run of the streaming query that resumes from the checkpoint.
A tuple that uniquely identifies an instance of streaming query run. It consists of `id` that persists across the streaming runs and `run_id` that changes between each run of the streaming query that resumes from the checkpoint.
Protobuf type
spark.connect.StreamingQueryInstanceId - trait StreamingQueryInstanceIdOrBuilder extends MessageOrBuilder
-
final
class
StreamingQueryManagerCommand extends GeneratedMessageV3 with StreamingQueryManagerCommandOrBuilder
Commands for the streaming query manager.
Commands for the streaming query manager.
Protobuf type
spark.connect.StreamingQueryManagerCommand - trait StreamingQueryManagerCommandOrBuilder extends MessageOrBuilder
-
final
class
StreamingQueryManagerCommandResult extends GeneratedMessageV3 with StreamingQueryManagerCommandResultOrBuilder
Response for commands on the streaming query manager.
Response for commands on the streaming query manager.
Protobuf type
spark.connect.StreamingQueryManagerCommandResult - trait StreamingQueryManagerCommandResultOrBuilder extends MessageOrBuilder
-
final
class
SubqueryAlias extends GeneratedMessageV3 with SubqueryAliasOrBuilder
Relation alias.
Relation alias.
Protobuf type
spark.connect.SubqueryAlias - trait SubqueryAliasOrBuilder extends MessageOrBuilder
-
final
class
TableExists extends GeneratedMessageV3 with TableExistsOrBuilder
See `spark.catalog.tableExists`
See `spark.catalog.tableExists`
Protobuf type
spark.connect.TableExists - trait TableExistsOrBuilder extends MessageOrBuilder
-
final
class
Tail extends GeneratedMessageV3 with TailOrBuilder
Relation of type [[Tail]] that is used to fetch `limit` rows from the last of the input relation.
Relation of type [[Tail]] that is used to fetch `limit` rows from the last of the input relation.
Protobuf type
spark.connect.Tail - trait TailOrBuilder extends MessageOrBuilder
-
final
class
ToDF extends GeneratedMessageV3 with ToDFOrBuilder
Rename columns on the input relation by the same length of names.
Rename columns on the input relation by the same length of names.
Protobuf type
spark.connect.ToDF - trait ToDFOrBuilder extends MessageOrBuilder
-
final
class
ToSchema extends GeneratedMessageV3 with ToSchemaOrBuilder
Protobuf type
spark.connect.ToSchema - trait ToSchemaOrBuilder extends MessageOrBuilder
- final class Types extends AnyRef
-
final
class
UncacheTable extends GeneratedMessageV3 with UncacheTableOrBuilder
See `spark.catalog.uncacheTable`
See `spark.catalog.uncacheTable`
Protobuf type
spark.connect.UncacheTable - trait UncacheTableOrBuilder extends MessageOrBuilder
-
final
class
Unknown extends GeneratedMessageV3 with UnknownOrBuilder
Used for testing purposes only.
Used for testing purposes only.
Protobuf type
spark.connect.Unknown - trait UnknownOrBuilder extends MessageOrBuilder
-
final
class
Unpivot extends GeneratedMessageV3 with UnpivotOrBuilder
Unpivot a DataFrame from wide format to long format, optionally leaving identifier columns set.
Unpivot a DataFrame from wide format to long format, optionally leaving identifier columns set.
Protobuf type
spark.connect.Unpivot - trait UnpivotOrBuilder extends MessageOrBuilder
-
final
class
UserContext extends GeneratedMessageV3 with UserContextOrBuilder
User Context is used to refer to one particular user session that is executing queries in the backend.
User Context is used to refer to one particular user session that is executing queries in the backend.
Protobuf type
spark.connect.UserContext - trait UserContextOrBuilder extends MessageOrBuilder
-
final
class
WithColumns extends GeneratedMessageV3 with WithColumnsOrBuilder
Adding columns or replacing the existing columns that have the same names.
Adding columns or replacing the existing columns that have the same names.
Protobuf type
spark.connect.WithColumns - trait WithColumnsOrBuilder extends MessageOrBuilder
-
final
class
WithColumnsRenamed extends GeneratedMessageV3 with WithColumnsRenamedOrBuilder
Rename columns on the input relation by a map with name to name mapping.
Rename columns on the input relation by a map with name to name mapping.
Protobuf type
spark.connect.WithColumnsRenamed - trait WithColumnsRenamedOrBuilder extends MessageOrBuilder
-
final
class
WithWatermark extends GeneratedMessageV3 with WithWatermarkOrBuilder
Protobuf type
spark.connect.WithWatermark - trait WithWatermarkOrBuilder extends MessageOrBuilder
-
final
class
WriteOperation extends GeneratedMessageV3 with WriteOperationOrBuilder
As writes are not directly handled during analysis and planning, they are modeled as commands.
As writes are not directly handled during analysis and planning, they are modeled as commands.
Protobuf type
spark.connect.WriteOperation - trait WriteOperationOrBuilder extends MessageOrBuilder
-
final
class
WriteOperationV2 extends GeneratedMessageV3 with WriteOperationV2OrBuilder
As writes are not directly handled during analysis and planning, they are modeled as commands.
As writes are not directly handled during analysis and planning, they are modeled as commands.
Protobuf type
spark.connect.WriteOperationV2 - trait WriteOperationV2OrBuilder extends MessageOrBuilder
-
final
class
WriteStreamOperationStart extends GeneratedMessageV3 with WriteStreamOperationStartOrBuilder
Starts write stream operation as streaming query. Query ID and Run ID of the streaming query are returned.
Starts write stream operation as streaming query. Query ID and Run ID of the streaming query are returned.
Protobuf type
spark.connect.WriteStreamOperationStart - trait WriteStreamOperationStartOrBuilder extends MessageOrBuilder
-
final
class
WriteStreamOperationStartResult extends GeneratedMessageV3 with WriteStreamOperationStartResultOrBuilder
Protobuf type
spark.connect.WriteStreamOperationStartResult - trait WriteStreamOperationStartResultOrBuilder extends MessageOrBuilder