-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.collect.collector.DataCollector,ai.platon.pulsar.common.collect.collector.PriorityDataCollector,kotlin.Comparable
public class UrlCacheCollector extends AbstractPriorityDataCollector<UrlAware>
-
-
Field Summary
Fields Modifier and Type Field Description private Stringnameprivate final Integersizeprivate final IntegerexternalSizeprivate final IntegerestimatedExternalSizeprivate final IntegerestimatedSizeprivate final UrlCacheurlCacheprivate final Integercapacityprivate final Integerpriorityprivate final Integeridprivate final Set<String>labelsprivate Stringlangprivate Stringcountryprivate Stringdistrictprivate IntegercollectCountprivate IntegercollectedCountprivate final InstantcreateTimeprivate InstantfirstCollectTimeprivate InstantlastCollectedTimeprivate InstantdeadTimeprivate final DurationcollectTimeprivate final BooleanisDead
-
Constructor Summary
Constructors Constructor Description UrlCacheCollector(UrlCache urlCache)
-
Method Summary
Modifier and Type Method Description StringgetName()The collector name UnitsetName(String name)The collector name IntegergetSize()IntegergetExternalSize()IntegergetEstimatedExternalSize()IntegergetEstimatedSize()final UrlCachegetUrlCache()IntegergetCapacity()The capacity IntegergetPriority()IntegergetId()The collector id Set<String>getLabels()The task labels StringgetLang()Required website language UnitsetLang(String lang)Required website language StringgetCountry()Required website country UnitsetCountry(String country)Required website country StringgetDistrict()Required website district UnitsetDistrict(String district)Required website district IntegergetCollectCount()The total count of collect attempt UnitsetCollectCount(Integer collectCount)The total count of collect attempt IntegergetCollectedCount()The total collected count UnitsetCollectedCount(Integer collectedCount)The total collected count InstantgetCreateTime()InstantgetFirstCollectTime()UnitsetFirstCollectTime(Instant firstCollectTime)InstantgetLastCollectedTime()UnitsetLastCollectedTime(Instant lastCollectedTime)InstantgetDeadTime()UnitsetDeadTime(Instant deadTime)DurationgetCollectTime()BooleangetIsDead()BooleanhasMore()If the fetch cache is a LoadingurlCache, the items can be both in memory or in external source, so even if all queues are empty, hasMore can return true IntegercollectTo(List<UrlAware> sink)List<String>dump()Unitclear()UnitdeepClear()-
Methods inherited from class ai.platon.pulsar.common.collect.collector.AbstractPriorityDataCollector
collectTo, collectTo, collectTo, compareTo -
Methods inherited from class ai.platon.pulsar.common.collect.collector.UrlCacheCollector
toString -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Constructor Detail
-
UrlCacheCollector
UrlCacheCollector(UrlCache urlCache)
-
-
Method Detail
-
getExternalSize
Integer getExternalSize()
-
getEstimatedExternalSize
Integer getEstimatedExternalSize()
-
getEstimatedSize
Integer getEstimatedSize()
-
getUrlCache
final UrlCache getUrlCache()
-
getCapacity
Integer getCapacity()
The capacity
-
getPriority
Integer getPriority()
-
getCountry
String getCountry()
Required website country
-
setCountry
Unit setCountry(String country)
Required website country
-
getDistrict
String getDistrict()
Required website district
-
setDistrict
Unit setDistrict(String district)
Required website district
-
getCollectCount
Integer getCollectCount()
The total count of collect attempt
-
setCollectCount
Unit setCollectCount(Integer collectCount)
The total count of collect attempt
-
getCollectedCount
Integer getCollectedCount()
The total collected count
-
setCollectedCount
Unit setCollectedCount(Integer collectedCount)
The total collected count
-
getCreateTime
Instant getCreateTime()
-
getFirstCollectTime
Instant getFirstCollectTime()
-
setFirstCollectTime
Unit setFirstCollectTime(Instant firstCollectTime)
-
getLastCollectedTime
Instant getLastCollectedTime()
-
setLastCollectedTime
Unit setLastCollectedTime(Instant lastCollectedTime)
-
getDeadTime
Instant getDeadTime()
-
setDeadTime
Unit setDeadTime(Instant deadTime)
-
getCollectTime
Duration getCollectTime()
-
hasMore
@Synchronized() Boolean hasMore()
If the fetch cache is a LoadingurlCache, the items can be both in memory or in external source, so even if all queues are empty, hasMore can return true
-
collectTo
@Synchronized() Integer collectTo(List<UrlAware> sink)
-
dump
@Synchronized() List<String> dump()
-
clear
@Synchronized() Unit clear()
-
deepClear
@Synchronized() Unit deepClear()
-
-
-
-