Package ai.platon.pulsar.common.collect
Class DelayExternalUrlLoader
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.collect.ExternalUrlLoader
public abstract class DelayExternalUrlLoader extends AbstractExternalUrlLoader
-
-
Constructor Summary
Constructors Constructor Description DelayExternalUrlLoader(Duration countDelay, Duration loadDelay)
-
Method Summary
Modifier and Type Method Description final BooleanisExpired()final DurationgetCountDelay()final DurationgetLoadDelay()Unitexpire()Unitreset()Force the loading time to expire BooleanhasMore()If there are more items in the source BooleanhasMore(UrlTopic topic)If there are more items in the source abstract IntegerdoCountRemaining()abstract IntegerdoCountRemaining(UrlTopic topic)IntegerestimateRemaining()Estimate the size of remaining items, this operation should be very fast IntegerestimateRemaining(UrlTopic topic)Estimate the size of remaining items, this operation should be very fast IntegercountRemaining()Count remaining size, this operation might be slow for a large url database, use estimateRemaining for fast counting. IntegercountRemaining(UrlTopic topic)Count remaining size with topic, this operation might be slow for a large url database, use estimateRemaining for fast counting. <T extends Any> UnitloadTo(Collection<T> sink, Integer size, UrlTopic topic, Function1<UrlAware, T> transformer)Load items from the source to the sink safely -
Methods inherited from class ai.platon.pulsar.common.collect.AbstractExternalUrlLoader
loadTo, loadToNow, saveAll -
Methods inherited from class ai.platon.pulsar.common.collect.ExternalUrlLoader
deleteAll, loadToNow, save -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Method Detail
-
getCountDelay
final Duration getCountDelay()
-
getLoadDelay
final Duration getLoadDelay()
-
doCountRemaining
abstract Integer doCountRemaining()
-
doCountRemaining
abstract Integer doCountRemaining(UrlTopic topic)
-
estimateRemaining
Integer estimateRemaining()
Estimate the size of remaining items, this operation should be very fast
-
estimateRemaining
Integer estimateRemaining(UrlTopic topic)
Estimate the size of remaining items, this operation should be very fast
-
countRemaining
Integer countRemaining()
Count remaining size, this operation might be slow for a large url database, use estimateRemaining for fast counting.
-
countRemaining
Integer countRemaining(UrlTopic topic)
Count remaining size with topic, this operation might be slow for a large url database, use estimateRemaining for fast counting.
-
-
-
-