All Implemented Interfaces:

kotlin.collections.Iterable
```
public final class UrlFeeder
 implements Iterable<UrlAware>
                    
```
The url feeder collects urls from the url pool and feed them to the crawlers.
The url feed collect urls using DataCollector, each DataCollector collect urls from exactly one UrlCache.
The user can register multiple UrlCaches and DataCollectors for different type of tasks.

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description

Field Summary

Fields
Modifier and Type	Field	Description
`private final ConcurrentLoadingIterable<UrlAware>`	`loadingIterable`
`private final Integer`	`cacheSize`
`private final Collection<PriorityDataCollector<UrlAware>>`	`openCollectors`
`private final List<PriorityDataCollector<UrlAware>>`	`collectors`
`private final String`	`abstract`
`private final String`	`report`
`private final UrlPool`	`urlPool`
`private final Integer`	`lowerCacheSize`
`private final Boolean`	`enableDefaults`

Constructor Summary

Constructors
Constructor	Description
`UrlFeeder(UrlPool urlPool, Integer lowerCacheSize, Boolean enableDefaults)`

Enum Constant Summary

Enum Constants
Enum Constant	Description

Method Summary

Modifier and Type	Method	Description
`final ConcurrentLoadingIterable<UrlAware>`	`getLoadingIterable()`
`final Integer`	`getCacheSize()`
`final Collection<PriorityDataCollector<UrlAware>>`	`getOpenCollectors()`
`final List<PriorityDataCollector<UrlAware>>`	`getCollectors()`
`final String`	`getAbstract()`
`final String`	`getReport()`
`final UrlPool`	`getUrlPool()`
`final Integer`	`getLowerCacheSize()`
`final Boolean`	`getEnableDefaults()`
`final Unit`	`addFirst(UrlAware url)`	Add a hyperlink to the very beginning of the fetch queue, so it will be served first
`final Unit`	`addLast(UrlAware url)`	Add a hyperlink to the end of the fetch queue, so it will be served last
`Iterator<UrlAware>`	`iterator()`
`final Integer`	`estimatedOrder(Integer priority)`	Estimate the order to fetch for the next task to add with priority priority.
`final UrlFeeder`	`addDefaultCollectors()`
`final UrlFeeder`	`addCollector(PriorityDataCollector<UrlAware> collector)`
`final UrlFeeder`	`addCollectors(Iterable<PriorityDataCollector<UrlAware>> collectors)`
`final List<PriorityDataCollector<UrlAware>>`	`findByName(String name)`
`final List<PriorityDataCollector<UrlAware>>`	`findByName(Iterable<String> names)`
`final List<PriorityDataCollector<UrlAware>>`	`findByName(Regex regex)`
`final List<PriorityDataCollector<UrlAware>>`	`findByNameLike(String name)`
`final Boolean`	`remove(DataCollector<UrlAware> collector)`
`final Boolean`	`removeAll(Collection<DataCollector<UrlAware>> collectors)`
`final Unit`	`clear()`

Methods inherited from class kotlin.collections.Iterable
forEach, spliterator
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UrlFeeder

UrlFeeder(UrlPool urlPool, Integer lowerCacheSize, Boolean enableDefaults)

Method Detail

getLoadingIterable

 final ConcurrentLoadingIterable<UrlAware> getLoadingIterable()

getCacheSize
```
 final Integer getCacheSize()
```

getOpenCollectors

 final Collection<PriorityDataCollector<UrlAware>> getOpenCollectors()

getCollectors

 final List<PriorityDataCollector<UrlAware>> getCollectors()

getAbstract
```
 final String getAbstract()
```

getReport
```
 final String getReport()
```

getUrlPool
```
 final UrlPool getUrlPool()
```

getLowerCacheSize
```
 final Integer getLowerCacheSize()
```

getEnableDefaults
```
 final Boolean getEnableDefaults()
```

addFirst
```
 final Unit addFirst(UrlAware url)
```
Add a hyperlink to the very beginning of the fetch queue, so it will be served first

addLast
```
 final Unit addLast(UrlAware url)
```
Add a hyperlink to the end of the fetch queue, so it will be served last

iterator
```
 Iterator<UrlAware> iterator()
```

estimatedOrder
```
 final Integer estimatedOrder(Integer priority)
```
Estimate the order to fetch for the next task to add with priority priority.

addDefaultCollectors
```
 final UrlFeeder addDefaultCollectors()
```

addCollector

 final UrlFeeder addCollector(PriorityDataCollector<UrlAware> collector)

addCollectors

 final UrlFeeder addCollectors(Iterable<PriorityDataCollector<UrlAware>> collectors)

findByName

 final List<PriorityDataCollector<UrlAware>> findByName(String name)

findByName

 final List<PriorityDataCollector<UrlAware>> findByName(Iterable<String> names)

findByName

 final List<PriorityDataCollector<UrlAware>> findByName(Regex regex)

findByNameLike

 final List<PriorityDataCollector<UrlAware>> findByNameLike(String name)

remove

 final Boolean remove(DataCollector<UrlAware> collector)

removeAll

 final Boolean removeAll(Collection<DataCollector<UrlAware>> collectors)

clear
```
 final Unit clear()
```

Class UrlFeeder

Nested Class Summary

Field Summary

Constructor Summary

Enum Constant Summary

Method Summary

Methods inherited from class kotlin.collections.Iterable

Methods inherited from class java.lang.Object

Constructor Detail

UrlFeeder

Method Detail

getLoadingIterable

getCacheSize

getOpenCollectors

getCollectors

getAbstract

getReport

getUrlPool

getLowerCacheSize

getEnableDefaults

addFirst

addLast

iterator

estimatedOrder

addDefaultCollectors

addCollector

addCollectors

findByName

findByName

findByName

findByNameLike

remove

removeAll

clear