Constructor Detail

• WebDb

  WebDb(ImmutableConfig conf)

Method Detail

• getSpecifiedDataStore

   final DataStore<String, GWebPage> getSpecifiedDataStore()

• setSpecifiedDataStore

   final Unit setSpecifiedDataStore(DataStore<String, GWebPage> specifiedDataStore)

• getDataStore

   final DataStore<String, GWebPage> getDataStore()

• getDataStoreOrNull

   final DataStore<String, GWebPage> getDataStoreOrNull()

• getSchemaName

   final String getSchemaName()

• getConf

   final ImmutableConfig getConf()

• getOrNull

   final WebPage getOrNull(String originalUrl, GWebPage.Field field)

• getOrNull

   final WebPage getOrNull(String originalUrl, Iterable<GWebPage.Field> fields)

• getOrNull

   final WebPage getOrNull(String originalUrl, String field)

• getOrNull

   final WebPage getOrNull(String originalUrl, Boolean norm, Array<String> fields)

  Returns the WebPage corresponding to the given url.

  Parameters:

  originalUrl - the original url of the page, it comes from user input, web page parsing, etc

  fields - the fields required in the WebPage.

• get

   final WebPage get(String originalUrl, GWebPage.Field field)

  Returns the WebPage corresponding to the given url.

  Parameters:

  originalUrl - the original address of the page

• get

   final WebPage get(String originalUrl, Iterable<GWebPage.Field> fields)

• get

   final WebPage get(String originalUrl, String field)

• get

   final WebPage get(String originalUrl, Boolean norm, Array<String> fields)

• exists

   final Boolean exists(String originalUrl, Boolean norm)

• put

  @JvmOverloads() final Boolean put(WebPage page, Boolean replaceIfExists)

• put

  @JvmOverloads() final Boolean put(WebPage page)

• putAll

   final Unit putAll(Iterable<WebPage> pages)

• delete

  @JvmOverloads() final Boolean delete(String originalUrl, Boolean norm)

• delete

  @JvmOverloads() final Boolean delete(String originalUrl)

• truncate

  @JvmOverloads() final Boolean truncate(Boolean force)

• truncate

  @JvmOverloads() final Boolean truncate()

• scan

   final Iterator<WebPage> scan(String urlBase)

  Scan all pages who's url starts with {@param originalUrl}

• scan

   final Iterator<WebPage> scan(String originalUrl, Iterable<GWebPage.Field> fields)

  Scan all pages who's url starts with {@param originalUrl}

  Parameters:

  originalUrl - The base url

• scan

   final Iterator<WebPage> scan(String originalUrl, Array<String> fields)

  Scan all pages who's url starts with {@param originalUrl}

  Parameters:

  originalUrl - The base url

• scan

   final Iterator<WebPage> scan(String originalUrl, Array<String> fields, Filter<String, GWebPage> filter)

  Scan all pages who's url starts with {@param originalUrl}

  Parameters:

  originalUrl - The base url

• query

   final Iterator<WebPage> query(DbQuery query)

  Scan all pages matches the {@param query}

  Parameters:

  query - The query

• flush

   final Unit flush()

• close

   Unit close()