-
public interface UrlCacheThe url cache holds UrlAwares.
A url cache contains several queues for different purpose: reentrant, non-reentrant and n-reentrant. A reentrant queue accepts a url multiple times, a non-reentrant queue accepts a url only once, and a n-reentrant queue accepts a url for n times at most.
Url caches are expected to be very large, the items can be loaded from external source, for example, MongoDB.
-
-
Method Summary
Modifier and Type Method Description abstract UnitremoveDeceased()Remove dead urls. abstract Unitclear()Clear the local cache. UnitdeepClear()Clear both the local cache and external source. abstract StringgetName()The cache name abstract IntegergetPriority()The priority abstract Queue<UrlAware>getNonReentrantQueue()A non-reentrant queue accepts the same url only once abstract Queue<UrlAware>getNReentrantQueue()A non-reentrant queue accepts the same url for n times at most abstract Queue<UrlAware>getReentrantQueue()A reentrant queue accepts the same url multiple times List<Queue<UrlAware>>getQueues()IntegergetSize()IntegergetExternalSize()IntegergetEstimatedExternalSize()IntegergetEstimatedSize()-
-
Method Detail
-
removeDeceased
abstract Unit removeDeceased()
Remove dead urls.
-
getPriority
abstract Integer getPriority()
The priority
-
getNonReentrantQueue
abstract Queue<UrlAware> getNonReentrantQueue()
A non-reentrant queue accepts the same url only once
-
getNReentrantQueue
abstract Queue<UrlAware> getNReentrantQueue()
A non-reentrant queue accepts the same url for n times at most
-
getReentrantQueue
abstract Queue<UrlAware> getReentrantQueue()
A reentrant queue accepts the same url multiple times
-
getExternalSize
Integer getExternalSize()
-
getEstimatedExternalSize
Integer getEstimatedExternalSize()
-
getEstimatedSize
Integer getEstimatedSize()
-
-
-
-