Class DataUtils


  • public final class DataUtils
    extends Object
    Utility methods
    • Field Detail

      • ERROR_READING_FAILED

        public static final int ERROR_READING_FAILED
        An error occurred while reading from the file.
        See Also:
        Constant Field Values
      • ERROR_WRITING_FAILED

        public static final int ERROR_WRITING_FAILED
        An error occurred when trying to write to the file.
        See Also:
        Constant Field Values
      • ERROR_INTERNAL

        public static final int ERROR_INTERNAL
        An internal error occurred. This could be a bug, or a memory corruption (for example caused by out of memory).
        See Also:
        Constant Field Values
      • ERROR_CLOSED

        public static final int ERROR_CLOSED
        The object is already closed.
        See Also:
        Constant Field Values
      • ERROR_UNSUPPORTED_FORMAT

        public static final int ERROR_UNSUPPORTED_FORMAT
        The file format is not supported.
        See Also:
        Constant Field Values
      • ERROR_FILE_CORRUPT

        public static final int ERROR_FILE_CORRUPT
        The file is corrupt or (for encrypted files) the encryption key is wrong.
        See Also:
        Constant Field Values
      • ERROR_FILE_LOCKED

        public static final int ERROR_FILE_LOCKED
        The file is locked.
        See Also:
        Constant Field Values
      • ERROR_SERIALIZATION

        public static final int ERROR_SERIALIZATION
        An error occurred when serializing or de-serializing.
        See Also:
        Constant Field Values
      • ERROR_CHUNK_NOT_FOUND

        public static final int ERROR_CHUNK_NOT_FOUND
        The application was trying to read data from a chunk that is no longer available.
        See Also:
        Constant Field Values
      • ERROR_BLOCK_NOT_FOUND

        public static final int ERROR_BLOCK_NOT_FOUND
        The block in the stream store was not found.
        See Also:
        Constant Field Values
      • ERROR_TRANSACTION_CORRUPT

        public static final int ERROR_TRANSACTION_CORRUPT
        The transaction store is corrupt.
        See Also:
        Constant Field Values
      • ERROR_TRANSACTION_LOCKED

        public static final int ERROR_TRANSACTION_LOCKED
        An entry is still locked by another transaction.
        See Also:
        Constant Field Values
      • ERROR_TOO_MANY_OPEN_TRANSACTIONS

        public static final int ERROR_TOO_MANY_OPEN_TRANSACTIONS
        There are too many open transactions.
        See Also:
        Constant Field Values
      • ERROR_TRANSACTION_ILLEGAL_STATE

        public static final int ERROR_TRANSACTION_ILLEGAL_STATE
        The transaction store is in an illegal state (for example, not yet initialized).
        See Also:
        Constant Field Values
      • ERROR_TRANSACTION_TOO_BIG

        public static final int ERROR_TRANSACTION_TOO_BIG
        The transaction contains too many changes.
        See Also:
        Constant Field Values
      • ERROR_TRANSACTIONS_DEADLOCK

        public static final int ERROR_TRANSACTIONS_DEADLOCK
        Deadlock discovered and one of transactions involved chosen as victim and rolled back.
        See Also:
        Constant Field Values
      • PAGE_TYPE_LEAF

        public static final int PAGE_TYPE_LEAF
        The type for leaf page.
        See Also:
        Constant Field Values
      • PAGE_TYPE_NODE

        public static final int PAGE_TYPE_NODE
        The type for node page.
        See Also:
        Constant Field Values
      • PAGE_COMPRESSED

        public static final int PAGE_COMPRESSED
        The bit mask for compressed pages (compression level fast).
        See Also:
        Constant Field Values
      • PAGE_COMPRESSED_HIGH

        public static final int PAGE_COMPRESSED_HIGH
        The bit mask for compressed pages (compression level high).
        See Also:
        Constant Field Values
      • MAX_VAR_INT_LEN

        public static final int MAX_VAR_INT_LEN
        The maximum length of a variable size int.
        See Also:
        Constant Field Values
      • MAX_VAR_LONG_LEN

        public static final int MAX_VAR_LONG_LEN
        The maximum length of a variable size long.
        See Also:
        Constant Field Values
      • COMPRESSED_VAR_INT_MAX

        public static final int COMPRESSED_VAR_INT_MAX
        The maximum integer that needs less space when using variable size encoding (only 3 bytes instead of 4).
        See Also:
        Constant Field Values
      • COMPRESSED_VAR_LONG_MAX

        public static final long COMPRESSED_VAR_LONG_MAX
        The maximum long that needs less space when using variable size encoding (only 7 bytes instead of 8).
        See Also:
        Constant Field Values
      • PAGE_LARGE

        public static final int PAGE_LARGE
        The marker size of a very large page.
        See Also:
        Constant Field Values
      • META_CHUNK

        public static final String META_CHUNK
        The prefix for chunks ("chunk."). This, plus the chunk id (hex encoded) is the key, and the serialized chunk metadata is the value.
        See Also:
        Constant Field Values
      • META_NAME

        public static final String META_NAME
        The prefix for names ("name."). This, plus the name of the map, is the key, and the map id (hey encoded) is the value.
        See Also:
        Constant Field Values
      • META_MAP

        public static final String META_MAP
        The prefix for maps ("map."). This, plus the map id (hex encoded) is the key, and the serialized in the map metadata is the value.
        See Also:
        Constant Field Values
      • META_ROOT

        public static final String META_ROOT
        The prefix for root positions of maps ("root."). This, plus the map id (hex encoded) is the key, and the position (hex encoded) is the value.
        See Also:
        Constant Field Values
    • Constructor Detail

      • DataUtils

        public DataUtils()
    • Method Detail

      • getVarIntLen

        public static int getVarIntLen​(int x)
        Get the length of the variable size int.
        Parameters:
        x - the value
        Returns:
        the length in bytes
      • getVarLongLen

        public static int getVarLongLen​(long x)
        Get the length of the variable size long.
        Parameters:
        x - the value
        Returns:
        the length in bytes
      • readVarInt

        public static int readVarInt​(ByteBuffer buff)
        Read a variable size int.
        Parameters:
        buff - the source buffer
        Returns:
        the value
      • readVarLong

        public static long readVarLong​(ByteBuffer buff)
        Read a variable size long.
        Parameters:
        buff - the source buffer
        Returns:
        the value
      • writeVarInt

        public static void writeVarInt​(OutputStream out,
                                       int x)
                                throws IOException
        Write a variable size int.
        Parameters:
        out - the output stream
        x - the value
        Throws:
        IOException - if some data could not be written
      • writeVarInt

        public static void writeVarInt​(ByteBuffer buff,
                                       int x)
        Write a variable size int.
        Parameters:
        buff - the source buffer
        x - the value
      • writeStringData

        public static void writeStringData​(ByteBuffer buff,
                                           String s,
                                           int len)
        Write characters from a string (without the length).
        Parameters:
        buff - the target buffer (must be large enough)
        s - the string
        len - the number of characters
      • readString

        public static String readString​(ByteBuffer buff)
        Read a string.
        Parameters:
        buff - the source buffer
        Returns:
        the value
      • readString

        public static String readString​(ByteBuffer buff,
                                        int len)
        Read a string.
        Parameters:
        buff - the source buffer
        len - the number of characters
        Returns:
        the value
      • writeVarLong

        public static void writeVarLong​(ByteBuffer buff,
                                        long x)
        Write a variable size long.
        Parameters:
        buff - the target buffer
        x - the value
      • writeVarLong

        public static void writeVarLong​(OutputStream out,
                                        long x)
                                 throws IOException
        Write a variable size long.
        Parameters:
        out - the output stream
        x - the value
        Throws:
        IOException - if some data could not be written
      • copyWithGap

        public static void copyWithGap​(Object src,
                                       Object dst,
                                       int oldSize,
                                       int gapIndex)
        Copy the elements of an array, with a gap.
        Parameters:
        src - the source array
        dst - the target array
        oldSize - the size of the old array
        gapIndex - the index of the gap
      • copyExcept

        public static void copyExcept​(Object src,
                                      Object dst,
                                      int oldSize,
                                      int removeIndex)
        Copy the elements of an array, and remove one element.
        Parameters:
        src - the source array
        dst - the target array
        oldSize - the size of the old array
        removeIndex - the index of the entry to remove
      • readFully

        public static void readFully​(FileChannel file,
                                     long pos,
                                     ByteBuffer dst)
        Read from a file channel until the buffer is full. The buffer is rewind after reading.
        Parameters:
        file - the file channel
        pos - the absolute position within the file
        dst - the byte buffer
        Throws:
        IllegalStateException - if some data could not be read
      • writeFully

        public static void writeFully​(FileChannel file,
                                      long pos,
                                      ByteBuffer src)
        Write to a file channel.
        Parameters:
        file - the file channel
        pos - the absolute position within the file
        src - the source buffer
      • encodeLength

        public static int encodeLength​(int len)
        Convert the length to a length code 0..31. 31 means more than 1 MB.
        Parameters:
        len - the length
        Returns:
        the length code
      • getPageChunkId

        public static int getPageChunkId​(long pos)
        Get the chunk id from the position.
        Parameters:
        pos - the position
        Returns:
        the chunk id
      • getPageMaxLength

        public static int getPageMaxLength​(long pos)
        Get the maximum length for the given page position.
        Parameters:
        pos - the position
        Returns:
        the maximum length
      • decodePageLength

        public static int decodePageLength​(int code)
        Get the maximum length for the given code. For the code 31, PAGE_LARGE is returned.
        Parameters:
        code - encoded page length
        Returns:
        the maximum length
      • getPageOffset

        public static int getPageOffset​(long pos)
        Get the offset from the position.
        Parameters:
        pos - the position
        Returns:
        the offset
      • getPageType

        public static int getPageType​(long pos)
        Get the page type from the position.
        Parameters:
        pos - the position
        Returns:
        the page type (PAGE_TYPE_NODE or PAGE_TYPE_LEAF)
      • isLeafPosition

        public static boolean isLeafPosition​(long pos)
        Determines whether specified file position corresponds to a leaf page
        Parameters:
        pos - the position
        Returns:
        true if it is a leaf, false otherwise
      • isPageSaved

        public static boolean isPageSaved​(long pos)
        Find out if page was saved.
        Parameters:
        pos - the position
        Returns:
        true if page has been saved
      • getPagePos

        public static long getPagePos​(int chunkId,
                                      int offset,
                                      int length,
                                      int type)
        Get the position of this page. The following information is encoded in the position: the chunk id, the offset, the maximum length, and the type (node or leaf).
        Parameters:
        chunkId - the chunk id
        offset - the offset
        length - the length
        type - the page type (1 for node, 0 for leaf)
        Returns:
        the position
      • getCheckValue

        public static short getCheckValue​(int x)
        Calculate a check value for the given integer. A check value is mean to verify the data is consistent with a high probability, but not meant to protect against media failure or deliberate changes.
        Parameters:
        x - the value
        Returns:
        the check value
      • appendMap

        public static StringBuilder appendMap​(StringBuilder buff,
                                              HashMap<String,​?> map)
        Append a map to the string builder, sorted by key.
        Parameters:
        buff - the target buffer
        map - the map
        Returns:
        the string builder
      • appendMap

        public static void appendMap​(StringBuilder buff,
                                     String key,
                                     String value)
        Append a key-value pair to the string builder. Keys may not contain a colon. Values that contain a comma or a double quote are enclosed in double quotes, with special characters escaped using a backslash.
        Parameters:
        buff - the target buffer
        key - the key
        value - the value
      • appendMap

        public static void appendMap​(StringBuilder buff,
                                     String key,
                                     long value)
        Append a key-value pair to the string builder. Keys may not contain a colon.
        Parameters:
        buff - the target buffer
        key - the key
        value - the value
      • appendMap

        public static void appendMap​(StringBuilder buff,
                                     String key,
                                     int value)
        Append a key-value pair to the string builder. Keys may not contain a colon.
        Parameters:
        buff - the target buffer
        key - the key
        value - the value
      • getMapName

        public static String getMapName​(String s)
        Parse a name from key-value pair list.
        Parameters:
        s - the list
        Returns:
        value of name item, or null
        Throws:
        IllegalStateException - if parsing failed
      • getFromMap

        public static String getFromMap​(String s,
                                        String key)
        Parse a specified pair from key-value pair list.
        Parameters:
        s - the list
        key - the name of the key
        Returns:
        value of the specified item, or null
        Throws:
        IllegalStateException - if parsing failed
      • getFletcher32

        public static int getFletcher32​(byte[] bytes,
                                        int offset,
                                        int length)
        Calculate the Fletcher32 checksum.
        Parameters:
        bytes - the bytes
        offset - initial offset
        length - the message length (if odd, 0 is appended)
        Returns:
        the checksum
      • checkArgument

        public static void checkArgument​(boolean test,
                                         String message,
                                         Object... arguments)
        Throw an IllegalArgumentException if the argument is invalid.
        Parameters:
        test - true if the argument is valid
        message - the message
        arguments - the arguments
        Throws:
        IllegalArgumentException - if the argument is invalid
      • newIllegalArgumentException

        public static IllegalArgumentException newIllegalArgumentException​(String message,
                                                                           Object... arguments)
        Create a new IllegalArgumentException.
        Parameters:
        message - the message
        arguments - the arguments
        Returns:
        the exception
      • newUnsupportedOperationException

        public static UnsupportedOperationException newUnsupportedOperationException​(String message)
        Create a new UnsupportedOperationException.
        Parameters:
        message - the message
        Returns:
        the exception
      • newIllegalStateException

        public static IllegalStateException newIllegalStateException​(int errorCode,
                                                                     String message,
                                                                     Object... arguments)
        Create a new IllegalStateException.
        Parameters:
        errorCode - the error code
        message - the message
        arguments - the arguments
        Returns:
        the exception
      • formatMessage

        public static String formatMessage​(int errorCode,
                                           String message,
                                           Object... arguments)
        Format an error message.
        Parameters:
        errorCode - the error code
        message - the message
        arguments - the arguments
        Returns:
        the formatted message
      • getErrorCode

        public static int getErrorCode​(String m)
        Get the error code from an exception message.
        Parameters:
        m - the message
        Returns:
        the error code, or 0 if none
      • readHexLong

        public static long readHexLong​(Map<String,​?> map,
                                       String key,
                                       long defaultValue)
        Read a hex long value from a map.
        Parameters:
        map - the map
        key - the key
        defaultValue - if the value is null
        Returns:
        the parsed value
        Throws:
        IllegalStateException - if parsing fails
      • parseHexLong

        public static long parseHexLong​(String x)
        Parse an unsigned, hex long.
        Parameters:
        x - the string
        Returns:
        the parsed value
        Throws:
        IllegalStateException - if parsing fails
      • parseHexInt

        public static int parseHexInt​(String x)
        Parse an unsigned, hex long.
        Parameters:
        x - the string
        Returns:
        the parsed value
        Throws:
        IllegalStateException - if parsing fails
      • readHexInt

        public static int readHexInt​(Map<String,​?> map,
                                     String key,
                                     int defaultValue)
        Read a hex int value from a map.
        Parameters:
        map - the map
        key - the key
        defaultValue - if the value is null
        Returns:
        the parsed value
        Throws:
        IllegalStateException - if parsing fails
      • getConfigParam

        public static int getConfigParam​(Map<String,​?> config,
                                         String key,
                                         int defaultValue)
        Get the configuration parameter value, or default.
        Parameters:
        config - the configuration
        key - the key
        defaultValue - the default
        Returns:
        the configured value or default