Package org.hpccsystems.ws.client

Class HPCCWsDFUClient

  • public class HPCCWsDFUClient
extends BaseHPCCWsClient
    Facilitates Logical File System actions on target HPCC Systems instance. Supported actions include fetching file information, file data, file listings, creating, publishing and deleting files, and more.

    • Constructor Detail

      • HPCCWsDFUClient

        protected HPCCWsDFUClient​(Connection baseConnection)
        Instantiates a new HPCC ws DFU client.
        Parameters:
        baseConnection - the base connection

    • Method Detail

      • getServiceWSDLURL

        public static String getServiceWSDLURL()
        Gets the service WSDLURL.
        Returns:
        the service WSDLURL

      • getServiceWSDLPort

        public static int getServiceWSDLPort()
        Gets the service WSDL port.
        Returns:
        the service WSDL port

      • getDefaultStub

        public org.apache.axis2.client.Stub getDefaultStub()
                                            throws org.apache.axis2.AxisFault
        Gets the default stub.
        Specified by:
        getDefaultStub in class BaseHPCCWsClient
        Returns:
        the default stub
        Throws:
        org.apache.axis2.AxisFault - the axis fault

      • get

        public static HPCCWsDFUClient get​(Connection connection)
        Gets the.
        Parameters:
        connection - the connection
        Returns:
        the HPCC ws DFU client

      • get

        public static HPCCWsDFUClient get​(String protocol,
                                  String targetHost,
                                  String targetPort,
                                  String user,
                                  String pass)
        Gets the.
        Parameters:
        protocol - the protocol
        targetHost - the target host
        targetPort - the target port
        user - the user
        pass - the pass
        Returns:
        the HPCC ws DFU client

      • get

        public static HPCCWsDFUClient get​(String protocol,
                                  String targetHost,
                                  String targetPort,
                                  String user,
                                  String pass,
                                  int timeout)
        Gets the.
        Parameters:
        protocol - the protocol
        targetHost - the target host
        targetPort - the target port
        user - the user
        pass - the pass
        timeout - the timeout
        Returns:
        the HPCC ws DFU client

      • initWsDFUClientStub

        protected void initWsDFUClientStub​(Connection conn)
        Initializes the service's underlying stub Should only be used by constructors.
        Parameters:
        conn - -- All connection settings included

      • getFileInfo

        public DFUInfoWrapper getFileInfo​(String logicalname,
                                  String clustername)
                           throws Exception,
                                  ArrayOfEspExceptionWrapper
        Gets the file info.
        Parameters:
        logicalname - - logical file to get file info for, can start with '~' or not
        clustername - - Optional. If specified, the cluster on which to search for the file
        Returns:
        a DFUInfoResponse object containing the file info
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getFileInfo

        public DFUInfoWrapper getFileInfo​(String logicalname,
                                  String clustername,
                                  boolean jsonTypeInfo,
                                  boolean binTypeInfo)
                           throws Exception,
                                  ArrayOfEspExceptionWrapper
        Gets the file info.
        Parameters:
        logicalname - logical file name, can start with ~
        clustername - optional, if specified the cluster name used in the search
        jsonTypeInfo - want record structure information returned as JSON
        binTypeInfo - want record structure information returned in binary format
        Returns:
        the file info
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getFileData

        public NodeList getFileData​(String logicalname,
                            Long beginrow,
                            Integer numrows,
                            String clustername)
                     throws Exception,
                            ArrayOfEspExceptionWrapper
        Gets the file data.
        Parameters:
        logicalname - - logical file to retrieve data for
        beginrow - - beginning row to retrieve data from
        numrows - - number of rows of data to retrieve
        clustername - - Optional. If specified, the cluster on which to find the logical file.
        Returns:
        an XML Element object holding the '<Row>' elements containing data.
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • createSuperfile

        public org.hpccsystems.ws.client.wrappers.gen.wsdfu.AddtoSuperfileResponseWrapper createSuperfile​(String superfilename,
                                                                                                  String[] subfiles)
                                                                                           throws Exception,
                                                                                                  ArrayOfEspExceptionWrapper
        Create super file
        Parameters:
        superfilename - super file name
        subfiles - sub files
        Returns:
        Add to super file request object
        Throws:
        Exception - general exception
        ArrayOfEspExceptionWrapper - array of esp exception wrapper

      • addToSuperFile

        public org.hpccsystems.ws.client.wrappers.gen.wsdfu.AddtoSuperfileResponseWrapper addToSuperFile​(org.hpccsystems.ws.client.wrappers.gen.wsdfu.AddtoSuperfileRequestWrapper request)
                                                                                          throws Exception,
                                                                                                 ArrayOfEspExceptionWrapper

        addToSuperFile.

        Parameters:
        request - - User provided request wrapper
        Returns:
        add to super file response object
        Throws:
        Exception - general exception
        ArrayOfEspExceptionWrapper - array of esp exception wrapper

      • getFileMetaData

        public List<DFUDataColumnWrapper> getFileMetaData​(String logicalname,
                                                  String clustername)
                                           throws Exception,
                                                  ArrayOfEspExceptionWrapper
        Use this function to retrieve file metadata such as column information, for superfiles the metadata from the first subfile will be returned.
        Parameters:
        logicalname - - Logical filename.
        clustername - - Optional - The cluster the logical filename is associated with.
        Returns:
        ArrayList of DFUDataColumnInfo
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getFileDataColumns

        public List<DFUDataColumnWrapper> getFileDataColumns​(String logicalname,
                                                     String clustername)
                                              throws Exception,
                                                     ArrayOfEspExceptionWrapper

        getFileDataColumns.

        Parameters:
        logicalname - - logical filename to retrieve the dfu data columns for. Currently this method/service call functions for THOR files but will return nothing for CSV/XML/FLAT data files
        clustername - - optional. The cluster the logical filename is associated with.
        Returns:
        ArrayList of DFUDataColumns
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getDatasetFields

        public EclRecordWrapper getDatasetFields​(String datasetname,
                                         String clusterName,
                                         String fieldSeparator)
                                  throws Exception,
                                         ArrayOfEspExceptionWrapper
        This method retrieves a list of columns for a THOR or CSV dataset. -if THOR, this is a straightforward service call to get the columns. -if CSV, depending on whether or not the CSV was sprayed or output by thor, it may or may not - have a record definition associated with it. If it doesn't, the number of fields in the CSV - is calculated by retrieving the first line and splitting it on the separator, and returning - a list of string Field1, Field2, etc. fields (that match the standard CSV field naming conventions.)
        Parameters:
        datasetname - - the name of the dataset to get dataset fields for. Can begin with '~' or not.
        clusterName - - optional. If given, the name of the cluster associated with the dataset.
        fieldSeparator - - optional. if given, and if the file being retrieved is a csv file, and if no field definitions exist for that csv,. this separator will be used rather than the separator defined during spraying when splitting up the first line of the file to define the number of fields
        Returns:
        an ArrayList of DFUDataColumns containing the name and field type.
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getFirstRow

        public String getFirstRow​(String datasetname,
                          String clustername)
                   throws Exception,
                          ArrayOfEspExceptionWrapper
        Returns the first row of data for a dataset.
        Parameters:
        datasetname - - logical filename, with or without '~' at the beginning
        clustername - - optional. name of the cluster the file's associated with
        Returns:
        a String of data representing the first row in the file.
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getLogicalFiles

        public List<DFULogicalFileWrapper> getLogicalFiles​(String filename,
                                                   String cluster,
                                                   int firstN,
                                                   int pageStartFrom,
                                                   int pageSize)
                                            throws Exception,
                                                   ArrayOfEspExceptionWrapper
        Get array of logical files on target HPCC system based on input parameters.
        Parameters:
        filename - the filename
        cluster - --- NO LONGER USED ---
        firstN - the first N
        pageStartFrom - the page start from
        pageSize - the page size
        Returns:
        the logical files
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getDFUData

        @Deprecated
public org.hpccsystems.ws.client.gen.axis2.wsdfu.latest.DFUSearchDataResponse getDFUData​(String openLogicalName,
                                                                                         String cluster,
                                                                                         boolean roxieSelections,
                                                                                         int chooseFile,
                                                                                         int count,
                                                                                         boolean schemaOnly,
                                                                                         long startIndex)
                                                                                  throws Exception,
                                                                                         ArrayOfEspExceptionWrapper
        Deprecated.
        Deprecated - use DFUSearchDataResponseWrapper getDFUData(DFUSearchDataRequestWrapper req) instead Return raw response from WsDFU DFUSearchData method requests.
        Parameters:
        openLogicalName - the open logical name
        cluster - the cluster
        roxieSelections - the roxie selections
        chooseFile - the choose file
        count - the count
        schemaOnly - the schema only
        startIndex - the start index
        Returns:
        the DFU data
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • searchFiles

        public List<DFULogicalFileWrapper> searchFiles​(org.hpccsystems.ws.client.wrappers.gen.wsdfu.DFUQueryRequestWrapper request)
                                        throws Exception,
                                               ArrayOfEspExceptionWrapper
        searchFiles Note, DFUQueryFileType.getText() can be used to populate DFUQueryRequestWrapper.setFileType
        Parameters:
        request - - the search request
        Returns:
        - collection of files matching the request passed in
        Throws:
        Exception - general exception
        ArrayOfEspExceptionWrapper - array of esp exception wrapper

      • getFileAccessBlob

        public String getFileAccessBlob​(org.hpccsystems.ws.client.gen.axis2.wsdfu.v1_39.SecAccessType accesstype,
                                String filename,
                                String clustername,
                                int expiryseconds,
                                String jobid)
                         throws Exception,
                                ArrayOfEspExceptionWrapper
        getFileAccessBlob - HPCC 7.0.x version The response is to be used in conjunction with DAFILESRV's rowservice distributed file read stream. The response grants the holder access to the target file for the duration of 'expiryseconds' seconds since the Accessblob is generated. This version is to be used when targeting a 7.0.x HPCC (ESP and DAFILESRV)
        Parameters:
        accesstype - - the file access level to request to request
        filename - - the name of the target file to be accessed
        clustername - - the name of the target file's HPCC cluster (can be empty)
        expiryseconds - - the number of seconds file access is granted
        jobid - - unique identifier for access token
        Returns:
        - Access artifact to be propagated as part of DAFILESERV file access requests
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getFileAccessBlob

        public String getFileAccessBlob​(String filename,
                                String clustername,
                                int expiryseconds,
                                String jobid)
                         throws Exception,
                                ArrayOfEspExceptionWrapper
        getFileAccessBlob The response is to be used in conjunction with DAFILESRV's rowservice distributed file read/write stream. The response grants the holder access to the target file for the duration of 'expiryseconds' seconds since the Accessblob is generated.
        Parameters:
        filename - - the name of the target file to be accessed
        clustername - - the name of the target file's HPCC cluster (can be empty)
        expiryseconds - - the number of seconds file access is granted
        jobid - - unique identifier for access token
        Returns:
        - Access artifact to be propagated as part of DAFILESERV file access requests
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getFileAccess

        public DFUFileAccessInfoWrapper getFileAccess​(org.hpccsystems.ws.client.gen.axis2.wsdfu.v1_39.SecAccessType accesstype,
                                              String filename,
                                              String clustername,
                                              int expiryseconds,
                                              String jobid,
                                              boolean includejsonTypeInfo,
                                              boolean includebinTypeInfo,
                                              boolean requestfileinfo)
                                       throws Exception,
                                              ArrayOfEspExceptionWrapper
        To be used in conjunction with DAFILESRV's rowservice distributed file read stream. The response wrapper provides access to the 'accessblob' which grants the holder read access to the target file for the duration of 'expiryseconds' seconds. Other access details can be fetched from the wrapper. since the Accessblob is generated.
        Parameters:
        accesstype - HPCC 7.0.x version - the file access level to request to request
        filename - - the name of the target file to be accessed
        clustername - - the name of the target file's HPCC cluster (can be empty)
        expiryseconds - - the number of seconds file access is granted
        jobid - - unique identifier for access token
        includejsonTypeInfo - - flag to request file info in JSON format
        includebinTypeInfo - - flag to request file info in Binary format
        requestfileinfo - the requestfileinfo
        Returns:
        - Access artifact to be propagated as part of DAFILESERV file access requests
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getFileAccess

        public DFUFileAccessInfoWrapper getFileAccess​(String filename,
                                              String clustername,
                                              int expiryseconds,
                                              String jobid)
                                       throws Exception,
                                              ArrayOfEspExceptionWrapper
        To be used in conjunction with DAFILESRV's rowservice distributed file read/write stream. The response wrapper provides access to the 'accessblob' which grants the holder read access to the target file for the duration of 'expiryseconds' seconds. Other access details can be fetched from the wrapper.
        Parameters:
        filename - - the name of the target file to be accessed
        clustername - - the name of the target file's HPCC cluster (can be empty)
        expiryseconds - - the number of seconds file access is granted
        jobid - - unique identifier for access token
        Returns:
        - Access artifact to be propagated as part of DAFILESERV file access requests
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • createFile

        public DFUCreateFileWrapper createFile​(String fileName,
                                       String cluster,
                                       String eclRecordDefinition,
                                       String[] partitionHostMap,
                                       int expirySeconds)
                                throws Exception,
                                       ArrayOfEspExceptionWrapper
        Create a new (unpublished) dfu file. Does not request all file metadata. DAFILESERV fileaccess token is requested
        Parameters:
        fileName - the file name
        cluster - the cluster
        eclRecordDefinition - the ecl record definition
        partitionHostMap - Array declaring the Filepart[i] to Node mapping
        expirySeconds - the expiry seconds
        Returns:
        the DFU create file wrapper
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • createFileAndAcquireAccess

        public DFUCreateFileWrapper createFileAndAcquireAccess​(String fileName,
                                                       String cluster,
                                                       String eclRecordDefinition,
                                                       String[] partitionHostMap,
                                                       int expirySeconds,
                                                       Boolean returnBinTypeInfo,
                                                       Boolean returnJsonTypeInfo,
                                                       org.hpccsystems.ws.client.gen.axis2.wsdfu.v1_39.FileAccessRole accessrole,
                                                       org.hpccsystems.ws.client.gen.axis2.wsdfu.v1_39.SecAccessType accesstype)
                                                throws Exception,
                                                       ArrayOfEspExceptionWrapper
        Create a new (unpublished) dfu file. All file metadata can be requested. DAFILESERV fileaccess token is requested
        Parameters:
        fileName - the file name
        cluster - the cluster
        eclRecordDefinition - the ecl record definition
        partitionHostMap - Array declaring the Filepart[i] to Node mapping
        expirySeconds - the expiry seconds
        returnBinTypeInfo - the return bin type info
        returnJsonTypeInfo - the return json type info
        accessrole - the accessrole
        accesstype - the accesstype
        Returns:
        the DFU create file wrapper
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • createFile

        public DFUCreateFileWrapper createFile​(String fileName,
                                       String cluster,
                                       String eclRecordDefinition,
                                       int expirySeconds)
                                throws Exception,
                                       ArrayOfEspExceptionWrapper
        Create a new (unpublished), uncompressed dfu file. JSON based info will be requested -- appropriate for HPCC post 7.2.0 DAFILESERV fileaccess token is requested
        Parameters:
        fileName - the file name
        cluster - the cluster
        eclRecordDefinition - the ecl record definition
        expirySeconds - the expiry seconds
        Returns:
        the DFU create file wrapper
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • createFile

        public DFUCreateFileWrapper createFile​(String fileName,
                                       String cluster,
                                       String eclRecordDefinition,
                                       int expirySeconds,
                                       Boolean compressed)
                                throws Exception,
                                       ArrayOfEspExceptionWrapper
        Create a new (unpublished) dfu file. JSON based info will be requested -- appropriate for HPCC post 7.2.0 DAFILESERV fileaccess token is requested
        Parameters:
        fileName - the file name
        cluster - the cluster
        eclRecordDefinition - the ecl record definition
        expirySeconds - the expiry seconds
        compressed - the compressed
        Returns:
        the DFU create file wrapper
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • createFile

        public DFUCreateFileWrapper createFile​(String fileName,
                                       String cluster,
                                       String eclRecordDefinition,
                                       int expirySeconds,
                                       Boolean compressed,
                                       DFUFileTypeWrapper filetype,
                                       String requestid)
                                throws Exception,
                                       ArrayOfEspExceptionWrapper
        Create a new (unpublished) dfu file. JSON based info will be requested -- appropriate for HPCC post 7.2.0 DAFILESERV fileaccess token is requested
        Parameters:
        fileName - the file name
        cluster - the cluster
        eclRecordDefinition - the ecl record definition
        expirySeconds - the expiry seconds
        compressed - the compressed
        filetype - - for example DFUFileTypeWrapper.Csv
        requestid - the requestid
        Returns:
        the DFU create file wrapper
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • createFileAndAcquireAccess

        public DFUCreateFileWrapper createFileAndAcquireAccess​(String fileName,
                                                       String cluster,
                                                       String eclRecordDefinition,
                                                       int expirySeconds,
                                                       Boolean compressed,
                                                       DFUFileTypeWrapper type,
                                                       String requestId)
                                                throws Exception,
                                                       ArrayOfEspExceptionWrapper
        Create a new (unpublished) dfu file. JSON based info will be requested -- appropriate for HPCC post 7.2.0 DAFILESERV fileaccess token is requested
        Parameters:
        fileName - the file name
        cluster - the cluster
        eclRecordDefinition - the ecl record definition
        expirySeconds - the expiry seconds
        compressed - the compressed
        type - - for example DFUFileTypeWrapper.Csv
        requestId - the request id
        Returns:
        the DFU create file wrapper
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • publishFile

        public void publishFile​(String fileId,
                        String eclRecordDefinition,
                        long totalRecords,
                        long fileSize)
                 throws Exception,
                        ArrayOfEspExceptionWrapper
        Publishes file created by WsDFUCreateFile. Requires fileId returned from createfile method call
        Parameters:
        fileId - the file id
        eclRecordDefinition - the ecl record definition
        totalRecords - the total records
        fileSize - the file size
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • publishFile

        public void publishFile​(String fileId,
                        String eclRecordDefinition,
                        long totalRecords,
                        long fileSize,
                        Boolean overwrite)
                 throws Exception,
                        ArrayOfEspExceptionWrapper
        Publishes file created by WsDFUCreateFile. Requires fileId returned from createfile method call
        Parameters:
        fileId - the file id
        eclRecordDefinition - the ecl record definition
        totalRecords - the total records
        fileSize - the file size
        overwrite - the overwrite
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • getFileDetails

        public DFUFileDetailWrapper getFileDetails​(String logicalname,
                                           String clustername,
                                           boolean jsonTypeInfo,
                                           boolean binTypeInfo)
                                    throws Exception,
                                           ArrayOfEspExceptionWrapper
        Gets the file details.
        Parameters:
        logicalname - logical file for request, can start with ~
        clustername - optional
        jsonTypeInfo - true if record structure information in JSON format is to be returned
        binTypeInfo - true if record structure information in binary format is to be returned
        Returns:
        DFUInfoResponse object containing the information
        Throws:
        Exception - the exception
        ArrayOfEspExceptionWrapper - the array of esp exception wrapper

      • ping

        public boolean ping()
             throws Exception
        Ping.
        Returns:
        true, if successful
        Throws:
        Exception - the exception