Package org.datavec.api.split

Interface InputSplit

    • Method Detail

      • canWriteToLocation

        boolean canWriteToLocation​(URI location)
        Returns true if the given uri can be written to
        Parameters:
        location - the location to determine
        Returns:

      • addNewLocation

        String addNewLocation()
        Add a new location with the name generated by this input split/

      • addNewLocation

        String addNewLocation​(String location)
        Add a new location to this input split (this may do anything from updating an in memory location to creating a new file)
        Parameters:
        location - the location to add

      • updateSplitLocations

        void updateSplitLocations​(boolean reset)
        Refreshes the split locations if needed in memory. (Think a few file gets added)
        Parameters:
        reset -

      • needsBootstrapForWrite

        boolean needsBootstrapForWrite()
        Returns true if this InputSplit needs bootstrapping for writing. A simple example of needing bootstrapping is for FileSplit where there is only a directory existing, but no file to write to
        Returns:
        true if this input split needs bootstrapping for writing to or not

      • bootStrapForWrite

        void bootStrapForWrite()
        Bootstrap this input split for writing. This is for use with RecordWriter

      • openOutputStreamFor

        OutputStream openOutputStreamFor​(String location)
                          throws Exception
        Open an OutputStream for the given location. Note that the user is responsible for closing the associated output stream.
        Parameters:
        location - the location to open the output stream for
        Returns:
        the output input stream
        Throws:
        Exception

      • openInputStreamFor

        InputStream openInputStreamFor​(String location)
                        throws Exception
        Open an InputStream for the given location. Note that the user is responsible for closing the associated input stream.
        Parameters:
        location - the location to open the input stream for
        Returns:
        the opened input stream
        Throws:
        Exception

      • length

        long length()
        Length of the split
        Returns:

      • locations

        URI[] locations()
        Locations of the splits
        Returns:

      • locationsIterator

        Iterator<URI> locationsIterator()
        Returns:

      • locationsPathIterator

        Iterator<String> locationsPathIterator()
        Returns:

      • reset

        void reset()
        Reset the InputSplit without reinitializing it from scratch. In many cases, this is a no-op. For InputSplits that have randomization: reset should shuffle the order.

      • resetSupported

        boolean resetSupported()
        Returns:
        True if the reset() method is supported (or is a no-op), false otherwise. If false is returned, reset() may throw an exception