Package ai.platon.pulsar.common.collect
Interface UrlCache
-
- All Implemented Interfaces:
public interface UrlCacheThe url cache holds urls.
A url cache contains several queues for different purpose: reentrant, non-reentrant and n-reentrant. A reentrant queue accepts the same url multiple times, a non-reentrant queue accepts the same url only once, and an n-reentrant queue accepts the same url for n times at most.
The URL cache is expected to be very large and items may be loaded from external sources such as MongoDB.
-
-
Method Summary
Modifier and Type Method Description abstract UnitremoveDeceased()Remove dead urls. abstract Unitclear()Clear the local cache. UnitdeepClear()Clear both the local cache and external source. abstract StringgetName()The cache name abstract IntegergetPriority()The priority abstract Queue<UrlAware>getNonReentrantQueue()A non-reentrant queue accepts the same url only once abstract Queue<UrlAware>getNReentrantQueue()An n-reentrant queue accepts the same url for n times at most abstract Queue<UrlAware>getReentrantQueue()A reentrant queue accepts the same url multiple times List<Queue<UrlAware>>getQueues()IntegergetSize()IntegergetExternalSize()IntegergetEstimatedExternalSize()IntegergetEstimatedSize()-
-
Method Detail
-
removeDeceased
abstract Unit removeDeceased()
Remove dead urls.
-
getPriority
abstract Integer getPriority()
The priority
-
getNonReentrantQueue
abstract Queue<UrlAware> getNonReentrantQueue()
A non-reentrant queue accepts the same url only once
-
getNReentrantQueue
abstract Queue<UrlAware> getNReentrantQueue()
An n-reentrant queue accepts the same url for n times at most
-
getReentrantQueue
abstract Queue<UrlAware> getReentrantQueue()
A reentrant queue accepts the same url multiple times
-
getExternalSize
Integer getExternalSize()
-
getEstimatedExternalSize
Integer getEstimatedExternalSize()
-
getEstimatedSize
Integer getEstimatedSize()
-
-
-
-