ai.grakn.engine.loader

Class Loader

java.lang.Object

ai.grakn.engine.loader.Loader

Direct Known Subclasses:

BlockingLoader, DistributedLoader

public abstract class Loader
extends Object

RESTLoader to perform bulk loading into the graph

Field Summary

Fields

Modifier and Type	Field and Description
protected int	batchSize
protected AtomicInteger	enqueuedJobs
protected AtomicInteger	errorJobs
protected AtomicInteger	finishedJobs
protected AtomicInteger	loadingJobs
protected Collection<InsertQuery>	queries
protected int	threadsNumber

Constructor Summary

Constructors

Constructor and Description
Loader()

Method Summary

Modifier and Type	Method and Description
void	add(InsertQuery query) Add an insert query to the queue
void	flush() Load any remaining batches in the queue.
int	getBatchSize()
protected void	handleError(Exception e, int i)
void	printLoaderState() Method that logs the current state of loading transactions
protected abstract void	sendQueriesToLoader(Collection<InsertQuery> batch) Method to load data into the graph.
Loader	setBatchSize(int size) Set the size of the each transaction in terms of number of vars.
Loader	setThreadsNumber(int number) Set the number of thread that will execute insert transactions at a time
abstract void	waitToFinish() Wait for all loading to terminate.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

enqueuedJobs

protected AtomicInteger enqueuedJobs

loadingJobs

protected AtomicInteger loadingJobs

finishedJobs

protected AtomicInteger finishedJobs

errorJobs

protected AtomicInteger errorJobs

queries

protected Collection<InsertQuery> queries

batchSize

protected int batchSize

threadsNumber

protected int threadsNumber

Constructor Detail

Loader

public Loader()

Method Detail

sendQueriesToLoader

protected abstract void sendQueriesToLoader(Collection<InsertQuery> batch)

Method to load data into the graph. Implementation depends on the type of the loader.

waitToFinish

public abstract void waitToFinish()

Wait for all loading to terminate.

add

public void add(InsertQuery query)

Add an insert query to the queue

Parameters:

query - insert query to be executed

setBatchSize

public Loader setBatchSize(int size)

Set the size of the each transaction in terms of number of vars.

Parameters:

size - number of vars in each transaction

getBatchSize

public int getBatchSize()

Returns:

the current batch size - minimum number of vars to be loaded in a transaction

setThreadsNumber

public Loader setThreadsNumber(int number)

Set the number of thread that will execute insert transactions at a time

flush

public void flush()

Load any remaining batches in the queue.

printLoaderState

public void printLoaderState()

Method that logs the current state of loading transactions

handleError

protected void handleError(Exception e,
                           int i)