-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.collect.collector.DataCollector,ai.platon.pulsar.common.collect.collector.PriorityDataCollector,kotlin.Comparable
public class QueueCollector extends AbstractPriorityDataCollector<UrlAware>
-
-
Field Summary
Fields Modifier and Type Field Description private Stringnameprivate final Integersizeprivate StringloadArgsprivate final Queue<UrlAware>queueprivate final Integercapacityprivate final Integerpriorityprivate final Integeridprivate final Set<String>labelsprivate Stringlangprivate Stringcountryprivate Stringdistrictprivate final IntegerexternalSizeprivate final IntegerestimatedExternalSizeprivate final IntegerestimatedSizeprivate IntegercollectCountprivate IntegercollectedCountprivate final InstantcreateTimeprivate InstantfirstCollectTimeprivate InstantlastCollectedTimeprivate InstantdeadTimeprivate final DurationcollectTimeprivate final BooleanisDead
-
Constructor Summary
Constructors Constructor Description QueueCollector(Priority13 priority)QueueCollector(Queue<UrlAware> queue, Integer priority)
-
Method Summary
Modifier and Type Method Description StringgetName()The collector name UnitsetName(String name)The collector name IntegergetSize()final StringgetLoadArgs()final UnitsetLoadArgs(String loadArgs)final Queue<UrlAware>getQueue()IntegergetCapacity()The capacity IntegergetPriority()IntegergetId()The collector id Set<String>getLabels()The task labels StringgetLang()Required website language UnitsetLang(String lang)Required website language StringgetCountry()Required website country UnitsetCountry(String country)Required website country StringgetDistrict()Required website district UnitsetDistrict(String district)Required website district IntegergetExternalSize()IntegergetEstimatedExternalSize()IntegergetEstimatedSize()IntegergetCollectCount()The total count of collect attempt UnitsetCollectCount(Integer collectCount)The total count of collect attempt IntegergetCollectedCount()The total collected count UnitsetCollectedCount(Integer collectedCount)The total collected count InstantgetCreateTime()The create time of this collector InstantgetFirstCollectTime()The first collect time UnitsetFirstCollectTime(Instant firstCollectTime)The first collect time InstantgetLastCollectedTime()The last time to collect an item successfully UnitsetLastCollectedTime(Instant lastCollectedTime)The last time to collect an item successfully InstantgetDeadTime()The dead time of this collector, if the collector is dead, all items should be dropped UnitsetDeadTime(Instant deadTime)The dead time of this collector, if the collector is dead, all items should be dropped DurationgetCollectTime()BooleangetIsDead()BooleanhasMore()IntegercollectTo(List<UrlAware> sink)List<String>dump()Unitclear()-
Methods inherited from class ai.platon.pulsar.common.collect.collector.AbstractPriorityDataCollector
collectTo, collectTo, collectTo, compareTo -
Methods inherited from class ai.platon.pulsar.common.collect.collector.QueueCollector
toString -
Methods inherited from class ai.platon.pulsar.common.collect.collector.AbstractDataCollector
deepClear -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Constructor Detail
-
QueueCollector
QueueCollector(Priority13 priority)
-
-
Method Detail
-
getLoadArgs
final String getLoadArgs()
-
setLoadArgs
final Unit setLoadArgs(String loadArgs)
-
getCapacity
Integer getCapacity()
The capacity
-
getPriority
Integer getPriority()
-
getCountry
String getCountry()
Required website country
-
setCountry
Unit setCountry(String country)
Required website country
-
getDistrict
String getDistrict()
Required website district
-
setDistrict
Unit setDistrict(String district)
Required website district
-
getExternalSize
Integer getExternalSize()
-
getEstimatedExternalSize
Integer getEstimatedExternalSize()
-
getEstimatedSize
Integer getEstimatedSize()
-
getCollectCount
Integer getCollectCount()
The total count of collect attempt
-
setCollectCount
Unit setCollectCount(Integer collectCount)
The total count of collect attempt
-
getCollectedCount
Integer getCollectedCount()
The total collected count
-
setCollectedCount
Unit setCollectedCount(Integer collectedCount)
The total collected count
-
getCreateTime
Instant getCreateTime()
The create time of this collector
-
getFirstCollectTime
Instant getFirstCollectTime()
The first collect time
-
setFirstCollectTime
Unit setFirstCollectTime(Instant firstCollectTime)
The first collect time
-
getLastCollectedTime
Instant getLastCollectedTime()
The last time to collect an item successfully
-
setLastCollectedTime
Unit setLastCollectedTime(Instant lastCollectedTime)
The last time to collect an item successfully
-
getDeadTime
Instant getDeadTime()
The dead time of this collector, if the collector is dead, all items should be dropped
-
setDeadTime
Unit setDeadTime(Instant deadTime)
The dead time of this collector, if the collector is dead, all items should be dropped
-
getCollectTime
Duration getCollectTime()
-
-
-
-