Package com.hashicorp.cdktf.providers.google.dataproc_job

Interface DataprocJobPysparkConfig

  • All Superinterfaces:
    software.amazon.jsii.JsiiSerializable
    All Known Implementing Classes:
    DataprocJobPysparkConfig.Jsii$Proxy
    @Generated(value="jsii-pacmak/1.102.0 (build e354887)",
           date="2024-08-31T03:59:20.741Z")
@Stability(Stable)
public interface DataprocJobPysparkConfig
extends software.amazon.jsii.JsiiSerializable

    • Method Detail

      • getMainPythonFileUri

        @Stability(Stable)
@NotNull
String getMainPythonFileUri()
        Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.

        Docs at Terraform Registry: {@link https://registry.terraform.io/providers/hashicorp/google/5.43.1/docs/resources/dataproc_job#main_python_file_uri DataprocJob#main_python_file_uri}

      • getArchiveUris

        @Stability(Stable)
@Nullable
default List<String> getArchiveUris()
        Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.

        Docs at Terraform Registry: {@link https://registry.terraform.io/providers/hashicorp/google/5.43.1/docs/resources/dataproc_job#archive_uris DataprocJob#archive_uris}

      • getArgs

        @Stability(Stable)
@Nullable
default List<String> getArgs()
        Optional.

        The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission Docs at Terraform Registry: {@link https://registry.terraform.io/providers/hashicorp/google/5.43.1/docs/resources/dataproc_job#args DataprocJob#args}

      • getFileUris

        @Stability(Stable)
@Nullable
default List<String> getFileUris()
        Optional.

        HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks Docs at Terraform Registry: {@link https://registry.terraform.io/providers/hashicorp/google/5.43.1/docs/resources/dataproc_job#file_uris DataprocJob#file_uris}

      • getJarFileUris

        @Stability(Stable)
@Nullable
default List<String> getJarFileUris()
        Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.

        Docs at Terraform Registry: {@link https://registry.terraform.io/providers/hashicorp/google/5.43.1/docs/resources/dataproc_job#jar_file_uris DataprocJob#jar_file_uris}

      • getLoggingConfig

        @Stability(Stable)
@Nullable
default DataprocJobPysparkConfigLoggingConfig getLoggingConfig()
        logging_config block.

        Docs at Terraform Registry: {@link https://registry.terraform.io/providers/hashicorp/google/5.43.1/docs/resources/dataproc_job#logging_config DataprocJob#logging_config}

      • getProperties

        @Stability(Stable)
@Nullable
default Map<String,​String> getProperties()
        Optional.

        A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code Docs at Terraform Registry: {@link https://registry.terraform.io/providers/hashicorp/google/5.43.1/docs/resources/dataproc_job#properties DataprocJob#properties}

      • getPythonFileUris

        @Stability(Stable)
@Nullable
default List<String> getPythonFileUris()
        Optional.

        HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip Docs at Terraform Registry: {@link https://registry.terraform.io/providers/hashicorp/google/5.43.1/docs/resources/dataproc_job#python_file_uris DataprocJob#python_file_uris}