-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.collect.UrlPool
public abstract class AbstractUrlPool implements UrlPool
The abstract url pool
-
-
Field Summary
Fields Modifier and Type Field Description private final IntegertotalCountprivate final UrlCachelowestCacheprivate final UrlCachelower5Cacheprivate final UrlCachelower4Cacheprivate final UrlCachelower3Cacheprivate final UrlCachelower2Cacheprivate final UrlCachelowerCacheprivate final UrlCachenormalCacheprivate final UrlCachehigherCacheprivate final UrlCachehigher2Cacheprivate final UrlCachehigher3Cacheprivate final UrlCachehigher4Cacheprivate final UrlCachehigher5Cacheprivate final UrlCachehighestCacheprivate final ImmutableConfigconfprivate final UrlCacherealTimeCacheprivate final Queue<DelayUrl>delayCacheprivate final Map<Integer, UrlCache>orderedCachesprivate final List<UrlCache>unorderedCachesprivate final IntegertotalItems
-
Constructor Summary
Constructors Constructor Description AbstractUrlPool(ImmutableConfig conf)
-
Method Summary
Modifier and Type Method Description IntegergetTotalCount()UrlCachegetLowestCache()UrlCachegetLower5Cache()UrlCachegetLower4Cache()UrlCachegetLower3Cache()UrlCachegetLower2Cache()UrlCachegetLowerCache()UrlCachegetNormalCache()UrlCachegetHigherCache()UrlCachegetHigher2Cache()UrlCachegetHigher3Cache()UrlCachegetHigher4Cache()UrlCachegetHigher5Cache()UrlCachegetHighestCache()final ImmutableConfiggetConf()abstract UrlCachegetRealTimeCache()The real time url cache in which urls have the highest priority of all. abstract Queue<DelayUrl>getDelayCache()An unbounded queue of Delayed urls, in which an element can only be taken when its delay has expired. abstract Map<Integer, UrlCache>getOrderedCaches()The ordered url caches. abstract List<UrlCache>getUnorderedCaches()The unordered url caches, tasks in unordered caches have the lowest priority. IntegergetTotalItems()Booleanadd(String url, Priority13 priority)Add a url to the pool Booleanadd(UrlAware url)Add a url to the pool BooleanaddAll(Iterable<String> urls, Priority13 priority)Add urls to the pool BooleanaddAll(Iterable<UrlAware> urls)Add urls to the pool UnitremoveDeceased()Remove deceased urls, such as URLs that are past the deadline. Unitclear()Clear the pool BooleanhasMore()Check if there is more items in the url pool -
-
Constructor Detail
-
AbstractUrlPool
AbstractUrlPool(ImmutableConfig conf)
-
-
Method Detail
-
getTotalCount
Integer getTotalCount()
-
getLowestCache
UrlCache getLowestCache()
-
getLower5Cache
UrlCache getLower5Cache()
-
getLower4Cache
UrlCache getLower4Cache()
-
getLower3Cache
UrlCache getLower3Cache()
-
getLower2Cache
UrlCache getLower2Cache()
-
getLowerCache
UrlCache getLowerCache()
-
getNormalCache
UrlCache getNormalCache()
-
getHigherCache
UrlCache getHigherCache()
-
getHigher2Cache
UrlCache getHigher2Cache()
-
getHigher3Cache
UrlCache getHigher3Cache()
-
getHigher4Cache
UrlCache getHigher4Cache()
-
getHigher5Cache
UrlCache getHigher5Cache()
-
getHighestCache
UrlCache getHighestCache()
-
getConf
final ImmutableConfig getConf()
-
getRealTimeCache
abstract UrlCache getRealTimeCache()
The real time url cache in which urls have the highest priority of all.
-
getDelayCache
abstract Queue<DelayUrl> getDelayCache()
An unbounded queue of Delayed urls, in which an element can only be taken when its delay has expired.
Delay cache has higher priority than all ordered caches and is usually used for retrying tasks.
-
getOrderedCaches
abstract Map<Integer, UrlCache> getOrderedCaches()
The ordered url caches.
Ordered caches has higher priority than unordered caches.
-
getUnorderedCaches
abstract List<UrlCache> getUnorderedCaches()
The unordered url caches, tasks in unordered caches have the lowest priority.
Unordered caches has the lowest priority of all
-
getTotalItems
Integer getTotalItems()
-
add
Boolean add(String url, Priority13 priority)
Add a url to the pool
-
addAll
Boolean addAll(Iterable<String> urls, Priority13 priority)
Add urls to the pool
-
removeDeceased
Unit removeDeceased()
Remove deceased urls, such as URLs that are past the deadline.
-
-
-
-