default WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.authenticationConfiguration(Consumer<AuthenticationConfiguration.Builder> authenticationConfiguration) |
Configuration information required to connect to websites using authentication.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.authenticationConfiguration(AuthenticationConfiguration authenticationConfiguration) |
Configuration information required to connect to websites using authentication.
|
static WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.builder() |
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.crawlDepth(Integer crawlDepth) |
The 'depth' or number of levels from the seed level to crawl.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.maxContentSizePerPageInMegaBytes(Float maxContentSizePerPageInMegaBytes) |
The maximum size (in MB) of a web page or attachment to crawl.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.maxLinksPerPage(Integer maxLinksPerPage) |
The maximum number of URLs on a web page to include when crawling a website.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.maxUrlsPerMinuteCrawlRate(Integer maxUrlsPerMinuteCrawlRate) |
The maximum number of URLs crawled per website host per minute.
|
default WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.proxyConfiguration(Consumer<ProxyConfiguration.Builder> proxyConfiguration) |
Configuration information required to connect to your internal websites via a web proxy.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.proxyConfiguration(ProxyConfiguration proxyConfiguration) |
Configuration information required to connect to your internal websites via a web proxy.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.toBuilder() |
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.urlExclusionPatterns(String... urlExclusionPatterns) |
A list of regular expression patterns to exclude certain URLs to crawl.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.urlExclusionPatterns(Collection<String> urlExclusionPatterns) |
A list of regular expression patterns to exclude certain URLs to crawl.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.urlInclusionPatterns(String... urlInclusionPatterns) |
A list of regular expression patterns to include certain URLs to crawl.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.urlInclusionPatterns(Collection<String> urlInclusionPatterns) |
A list of regular expression patterns to include certain URLs to crawl.
|
default WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.urls(Consumer<Urls.Builder> urls) |
Specifies the seed or starting point URLs of the websites or the sitemap URLs of the websites you want to
crawl.
|
WebCrawlerConfiguration.Builder |
WebCrawlerConfiguration.Builder.urls(Urls urls) |
Specifies the seed or starting point URLs of the websites or the sitemap URLs of the websites you want to
crawl.
|