public class RDBDocumentStore extends Object implements DocumentStore
DocumentStore for relational databases.
The code is supposed to be sufficiently generic to run with a variety of database implementations. However, the tables are created when required to simplify testing, and that code specifically supports these databases:
Data for each of the DocumentStore's Collections is stored in its own
database table (with a name matching the collection).
The tables essentially implement key/value storage, where the key usually is
derived from an Oak path, and the value is a serialization of a
Document (or a part of one). Additional fields are used for queries,
debugging, and concurrency control:
| Column | Type | Description |
|---|---|---|
| ID | varchar(512) not null primary key | The document's key (for databases that can not handle 512 character primary keys, such as MySQL, varbinary is possible as well). |
| MODIFIED | bigint | Low-resolution timestamp. |
| HASBINARY | smallint | Flag indicating whether the document has binary properties. |
| DELETEDONCE | smallint | Flag indicating whether the document has been deleted once. |
| MODCOUNT | bigint | Modification counter, used for avoiding overlapping updates. |
| DSIZE | bigint | The approximate size of the document's JSON serialization (for debugging purposes). |
| VERSION | smallint | The schema version the code writing to a row (or inserting it) was aware of (introduced with schema version 1). Not set for rows written by version 0 client code. |
| SDTYPE | smallint | Split Document type. |
| SDMAXREVTIME | bigint | Split document max revision time.. |
| DATA | varchar(16384) | The document's JSON serialization (only used for small document sizes, in which case BDATA (below) is not set), or a sequence of JSON serialized update operations to be applied against the last full serialization. |
| BDATA | blob | The document's JSON serialization (usually GZIPped, only used for "large" documents). |
The names of database tables can be prefixed; the purpose is mainly for testing, as tables can also be dropped automatically when the store is disposed (this only happens for those tables that have been created on demand).
The initial database layout used in OAK 1.0 through 1.6 is version 0.
Version 1 introduces an additional "version" column, which records the schema version of the code writing to the database (upon insert and update). This is in preparation of future layout changes which might introduce new columns.
Version 2 introduces an additional "sdtype" and "sdmaxrevtime".
The code deals with both version 0, version 1 and version 2 table layouts. By default, it tries to create version 2 tables, and also tries to upgrade existing version 0 and 1 tables to version 2.
Note that the database needs to be created/configured to support all Unicode characters in text fields, and to collate by Unicode code point (in DB2: "collate using identity", in PostgreSQL: "C"). THIS IS NOT THE DEFAULT!
For MySQL, the database parameter "max_allowed_packet" needs to be increased to support ~16M blobs.
The code tries to create the tables when they are not present. Likewise, it tries to upgrade to a newer schema when needed.
Users/Administrators who prefer to stay in control over table generation can
create them "manually". RDBHelper can be used to print out the DDL
statements that would have been used for auto-creation.
The cache borrows heavily from the MongoDocumentStore implementation.
The implementation currently supports only three indexed properties: "_bin",
"deletedOnce", and "_modified". Attempts to use a different indexed property
will cause a DocumentStoreException.
| Modifier and Type | Class and Description |
|---|---|
protected static class |
RDBDocumentStore.QueryCondition |
protected class |
RDBDocumentStore.UnsupportedIndexedPropertyException |
| Modifier and Type | Field and Description |
|---|---|
static int |
CHAR2OCTETRATIO |
static String |
COLLISIONSMODCOUNT
Optional counter for changes to "_collisions" map (
NodeDocument.COLLISIONS). |
protected static List<String> |
EMPTY_KEY_PATTERN |
protected static int |
SCHEMA |
protected static boolean |
USECMODCOUNT |
static String |
VERSIONPROP |
| Constructor and Description |
|---|
RDBDocumentStore(DataSource ds,
DocumentNodeStoreBuilder<?> builder)
Creates a RDBDocumentStore instance using the provided
DataSource, DocumentNodeStoreBuilder, and default
RDBOptions. |
RDBDocumentStore(DataSource ds,
DocumentNodeStoreBuilder<?> builder,
RDBOptions options)
Creates a RDBDocumentStore instance using the provided
DataSource, DocumentNodeStoreBuilder, and RDBOptions. |
| Modifier and Type | Method and Description |
|---|---|
static byte[] |
asBytes(@NotNull String data) |
protected <T extends Document> |
convertFromDBObject(@NotNull Collection<T> collection,
@NotNull RDBRow row) |
<T extends Document> |
create(Collection<T> collection,
List<UpdateOp> updateOps)
Try to create a list of documents.
|
<T extends Document> |
createOrUpdate(Collection<T> collection,
List<UpdateOp> updateOps)
Create or unconditionally update a number of documents.
|
<T extends Document> |
createOrUpdate(Collection<T> collection,
UpdateOp update)
Atomically checks if the document exists and updates it, otherwise the
document is created (aka "upsert"), unless the update operation requires
the document to be present (see
UpdateOp.isNew()). |
long |
determineServerTimeDifferenceMillis() |
void |
dispose()
Dispose this instance.
|
protected void |
finalize() |
<T extends Document> |
find(Collection<T> collection,
String id)
Get the document with the given
key. |
<T extends Document> |
find(Collection<T> collection,
String id,
int maxCacheAge)
Get the document with the
key. |
<T extends Document> |
findAndUpdate(Collection<T> collection,
UpdateOp update)
Performs a conditional update (e.g.
|
Iterable<org.apache.jackrabbit.oak.cache.CacheStats> |
getCacheStats() |
String |
getDroppedTables() |
<T extends Document> |
getIfCached(Collection<T> collection,
String id)
Fetches the cached document.
|
Map<String,String> |
getMetadata() |
protected <T extends Document> |
getMinValue(Collection<T> collection,
String field,
String fromKey,
String toKey,
List<String> excludeKeyPatterns,
List<RDBDocumentStore.QueryCondition> conditions) |
protected NodeDocumentCache |
getNodeDocumentCache() |
@NotNull Map<String,String> |
getStats()
Statistics are generated for each table.
|
protected <T extends Document> |
getTable(Collection<T> collection) |
static List<String> |
getTableNames() |
CacheInvalidationStats |
invalidateCache()
Invalidate the document cache.
|
<T extends Document> |
invalidateCache(Collection<T> collection,
String id)
Invalidate the document cache for the given key.
|
CacheInvalidationStats |
invalidateCache(Iterable<String> keys)
Invalidate the document cache but only with entries that match one
of the keys provided.
|
<T extends Document> |
query(Collection<T> collection,
String fromKey,
String toKey,
int limit)
Get a list of documents where the key is greater than a start value and
less than an end value.
|
protected <T extends Document> |
query(Collection<T> collection,
String fromKey,
String toKey,
List<String> excludeKeyPatterns,
List<RDBDocumentStore.QueryCondition> conditions,
int limit) |
<T extends Document> |
query(Collection<T> collection,
String fromKey,
String toKey,
String indexedProperty,
long startValue,
int limit)
Get a list of documents where the key is greater than a start value and
less than an end value and the given "indexed property" is greater
or equals the specified value.
|
protected <T extends Document> |
queryAsIterable(Collection<T> collection,
String fromKey,
String toKey,
List<String> excludeKeyPatterns,
List<RDBDocumentStore.QueryCondition> conditions,
int limit,
String sortBy) |
protected <T extends Document> |
queryCount(Collection<T> collection,
String fromKey,
String toKey,
List<String> excludeKeyPatterns,
List<RDBDocumentStore.QueryCondition> conditions) |
<T extends Document> |
remove(Collection<T> collection,
List<String> ids)
Batch remove documents with given keys.
|
<T extends Document> |
remove(Collection<T> collection,
Map<String,Long> toRemove)
Batch remove documents with given keys and corresponding equal conditions
on
NodeDocument.MODIFIED_IN_SECS values. |
<T extends Document> |
remove(Collection<T> collection,
String id)
Remove a document.
|
<T extends Document> |
remove(Collection<T> collection,
String indexedProperty,
long startValue,
long endValue)
Batch remove documents where the given "indexed property" is within the given
range (exclusive) -
(startValue, endValue). |
void |
setReadWriteMode(String readWriteMode)
Set the level of guarantee for read and write operations, if supported by this backend.
|
void |
setStatsCollector(DocumentStoreStatsCollector stats) |
public static final String COLLISIONSMODCOUNT
NodeDocument.COLLISIONS).public static final int CHAR2OCTETRATIO
protected static final boolean USECMODCOUNT
protected static final int SCHEMA
public static String VERSIONPROP
public RDBDocumentStore(DataSource ds, DocumentNodeStoreBuilder<?> builder, RDBOptions options)
DataSource, DocumentNodeStoreBuilder, and RDBOptions.public RDBDocumentStore(DataSource ds, DocumentNodeStoreBuilder<?> builder)
DataSource, DocumentNodeStoreBuilder, and default
RDBOptions.public <T extends Document> T find(Collection<T> collection, String id)
DocumentStorekey. This is a convenience method
and equivalent to DocumentStore.find(Collection, String, int) with a
maxCacheAge of Integer.MAX_VALUE.
The returned document is immutable.
find in interface DocumentStoreT - the document typecollection - the collectionid - the keypublic <T extends Document> T find(Collection<T> collection, String id, int maxCacheAge)
DocumentStorekey. The implementation may serve the
document from a cache, but the cached document must not be older than
the given maxCacheAge in milliseconds. An implementation must
invalidate a cached document when it detects it is outdated. That is, a
subsequent call to DocumentStore.find(Collection, String) must return the
newer version of the document.
The returned document is immutable.
find in interface DocumentStoreT - the document typecollection - the collectionid - the keymaxCacheAge - the maximum age of the cached document (in ms)@NotNull public <T extends Document> @NotNull List<T> query(Collection<T> collection, String fromKey, String toKey, int limit)
DocumentStoreThe returned documents are sorted by key and are immutable.
query in interface DocumentStoreT - the document typecollection - the collectionfromKey - the start value (excluding)toKey - the end value (excluding)limit - the maximum number of entries to return (starting with the lowest key)@NotNull public <T extends Document> @NotNull List<T> query(Collection<T> collection, String fromKey, String toKey, String indexedProperty, long startValue, int limit)
DocumentStore
The indexed property can either be a Long value, in which case numeric
comparison applies, or a Boolean value, in which case "false" is mapped
to "0" and "true" is mapped to "1".
The returned documents are sorted by key and are immutable.
query in interface DocumentStoreT - the document typecollection - the collectionfromKey - the start value (excluding)toKey - the end value (excluding)indexedProperty - the name of the indexed property (optional)startValue - the minimum value of the indexed propertylimit - the maximum number of entries to return@NotNull protected <T extends Document> @NotNull List<T> query(Collection<T> collection, String fromKey, String toKey, List<String> excludeKeyPatterns, List<RDBDocumentStore.QueryCondition> conditions, int limit)
public <T extends Document> void remove(Collection<T> collection, String id)
DocumentStore
In case of a DocumentStoreException, the document with the given
key may or may not have been removed from the store. It is the
responsibility of the caller to check whether it still exists. The
implementation however ensures that the result of the operation is
properly reflected in the document cache. That is, an implementation
could simply evict the document with the given key.
remove in interface DocumentStoreT - the document typecollection - the collectionid - the keypublic <T extends Document> void remove(Collection<T> collection, List<String> ids)
DocumentStorekeys may have been
removed.
In case of a DocumentStoreException, the documents with the given
keys may or may not have been removed from the store. It may also be
possible that only some have been removed from the store. It is the
responsibility of the caller to check which documents still exist. The
implementation however ensures that the result of the operation is
properly reflected in the document cache. That is, an implementation
could simply evict documents with the given keys from the cache.
remove in interface DocumentStoreT - the document typecollection - the collectionids - list of keyspublic <T extends Document> int remove(Collection<T> collection, Map<String,Long> toRemove)
DocumentStoreNodeDocument.MODIFIED_IN_SECS values. Keys for documents that
do not exist are simply ignored. A document is only removed if the
corresponding condition is met.
In case of a DocumentStoreException, the documents with the given
keys may or may not have been removed from the store. It may also be
possible that only some have been removed from the store. It is the
responsibility of the caller to check which documents still exist. The
implementation however ensures that the result of the operation is
properly reflected in the document cache. That is, an implementation
could simply evict documents with the given keys from the cache.
remove in interface DocumentStoreT - the document typecollection - the collection.toRemove - the keys of the documents to remove with the corresponding
timestamps.public <T extends Document> int remove(Collection<T> collection, String indexedProperty, long startValue, long endValue) throws DocumentStoreException
DocumentStore(startValue, endValue).
The indexed property is a Long value and numeric comparison applies.
In case of a DocumentStoreException, the documents with the given
keys may or may not have been removed from the store. It may also be
possible that only some have been removed from the store. It is the
responsibility of the caller to check which documents still exist. The
implementation however ensures that the result of the operation is
properly reflected in the document cache. That is, an implementation
could simply evict documents with the given keys from the cache.
remove in interface DocumentStoreT - the document typecollection - the collection.indexedProperty - the name of the indexed propertystartValue - the minimum value of the indexed property (exclusive)endValue - the maximum value of the indexed property (exclusive)DocumentStoreException - if the operation failed. E.g. because of
an I/O error.public <T extends Document> boolean create(Collection<T> collection, List<UpdateOp> updateOps)
DocumentStoretrue iff
none of the documents existed before and the create was successful. This
method will return false if one of the documents already exists
in the store. Some documents may still have been created in the store.
An implementation does not have to guarantee an atomic create of all the
documents described in the updateOps. It is the responsibility of
the caller to check, which documents were created and take appropriate
action. The same is true when this method throws
DocumentStoreException (e.g. when a communication error occurs).
In this case only some documents may have been created.create in interface DocumentStoreT - the document typecollection - the collectionupdateOps - the list of documents to add (where UpdateOp.Conditions are not allowed)public <T extends Document> T createOrUpdate(Collection<T> collection, UpdateOp update)
DocumentStoreUpdateOp.isNew()). The returned
document is immutable.
If this method fails with a DocumentStoreException, then the
document may or may not have been created or updated. It is the
responsibility of the caller to check the result e.g. by calling
DocumentStore.find(Collection, String). The implementation however ensures
that the result of the operation is properly reflected in the document
cache. That is, an implementation could simply evict documents with the
given keys from the cache.
createOrUpdate in interface DocumentStoreT - the document typecollection - the collectionupdate - the update operation (where UpdateOp.Conditions are not
allowed)null if it either didn't exist
before, or the UpdateOp required the document to be
present but UpdateOp.isNew() was false.public <T extends Document> List<T> createOrUpdate(Collection<T> collection, List<UpdateOp> updateOps)
DocumentStore
In case of a DocumentStoreException (e.g. when a communication
error occurs) only some changes may have been applied. In this case it is
the responsibility of the caller to check which UpdateOps
were applied and take appropriate action. The implementation however
ensures that the result of the operations are properly reflected in the
document cache. That is, an implementation could simply evict documents
related to the given update operations from the cache.
createOrUpdate in interface DocumentStoreT - the document typecollection - the collectionupdateOps - the update operation listnull values if they didn't exist
before (see DocumentStore.createOrUpdate(Collection, UpdateOp)), where the order
reflects the order in the "updateOps" parameterpublic <T extends Document> T findAndUpdate(Collection<T> collection, UpdateOp update)
DocumentStoreUpdateOp.Condition.Type.EXISTS and only updates the
document if the condition is true. The returned document is
immutable.
In case of a DocumentStoreException (e.g. when a communication
error occurs) the update may or may not have been applied. In this case
it is the responsibility of the caller to check whether the update was
applied and take appropriate action. The implementation however ensures
that the result of the operation is properly reflected in the document
cache. That is, an implementation could simply evict the document related
to the given update operation from the cache.
findAndUpdate in interface DocumentStoreT - the document typecollection - the collectionupdate - the update operation with the conditionnull if the condition is not met or
if the document wasn't foundpublic CacheInvalidationStats invalidateCache()
DocumentStoreDocumentStore.getIfCached(Collection, String)
returns a non-null value for a key.
An implementation is allowed to perform lazy invalidation and only check
whether a document is up-to-date when it is accessed after this method
is called. However, this also includes a call to DocumentStore.getIfCached(Collection, String),
which must only return the document if it was up-to-date at the time
this method was called. Similarly, a call to DocumentStore.find(Collection, String)
must guarantee the returned document reflects all the changes done up to
when invalidateCache() was called.
In some implementations this method can be a NOP because documents can
only be modified through a single instance of a DocumentStore.
invalidateCache in interface DocumentStorenull if none are
available.public CacheInvalidationStats invalidateCache(Iterable<String> keys)
DocumentStoreDocumentStore.invalidateCache() for the general contract of cache
invalidation.invalidateCache in interface DocumentStorekeys - the keys of the documents to invalidate.null if none are
available.public <T extends Document> void invalidateCache(Collection<T> collection, String id)
DocumentStoreDocumentStore.invalidateCache() for the general contract of cache
invalidation.invalidateCache in interface DocumentStorecollection - the collectionid - the keypublic long determineServerTimeDifferenceMillis()
determineServerTimeDifferenceMillis in interface DocumentStorepublic String getDroppedTables()
public void dispose()
DocumentStoredispose in interface DocumentStorepublic <T extends Document> T getIfCached(Collection<T> collection, String id)
DocumentStorenull will be returned. This method is consistent with other find
methods that may return cached documents and will return null
even when the implementation has a negative cache for documents that
do not exist. This method will never return NodeDocument.NULL.getIfCached in interface DocumentStoreT - the document typecollection - the collectionid - the keynull.public Iterable<org.apache.jackrabbit.oak.cache.CacheStats> getCacheStats()
getCacheStats in interface DocumentStorepublic Map<String,String> getMetadata()
getMetadata in interface DocumentStore@NotNull public @NotNull Map<String,String> getStats()
Collection.CLUSTER_NODES is added:
RDBDocumentStoreDB.getAdditionalStatistics(RDBConnectionHandler, String, String)
for details.getStats in interface DocumentStoreprotected void finalize()
throws Throwable
protected <T extends Document> Iterable<T> queryAsIterable(Collection<T> collection, String fromKey, String toKey, List<String> excludeKeyPatterns, List<RDBDocumentStore.QueryCondition> conditions, int limit, String sortBy)
protected <T extends Document> long queryCount(Collection<T> collection, String fromKey, String toKey, List<String> excludeKeyPatterns, List<RDBDocumentStore.QueryCondition> conditions)
protected <T extends Document> long getMinValue(Collection<T> collection, String field, String fromKey, String toKey, List<String> excludeKeyPatterns, List<RDBDocumentStore.QueryCondition> conditions)
@NotNull protected <T extends Document> @NotNull org.apache.jackrabbit.oak.plugins.document.rdb.RDBDocumentStore.RDBTableMetaData getTable(Collection<T> collection)
public static byte[] asBytes(@NotNull
@NotNull String data)
public void setReadWriteMode(String readWriteMode)
DocumentStoresetReadWriteMode in interface DocumentStorereadWriteMode - the read/write modepublic void setStatsCollector(DocumentStoreStatsCollector stats)
@NotNull protected <T extends Document> T convertFromDBObject(@NotNull @NotNull Collection<T> collection, @NotNull @NotNull RDBRow row)
protected NodeDocumentCache getNodeDocumentCache()
Copyright © 2012–2018 The Apache Software Foundation. All rights reserved.