A C G H Q R
All Classes All Packages

A

ask(String, String) - Method in interface com.panforge.robotstxt.RobotsTxt

Asks for grant.

C

com.panforge.robotstxt - package com.panforge.robotstxt

Provides support for reading and parsing robots.txt

G

getClause() - Method in interface com.panforge.robotstxt.Grant

Gets clause being engaged.

getCrawlDelay() - Method in interface com.panforge.robotstxt.Grant

Gets crawl delay from the group.

getCrawlDelay() - Method in interface com.panforge.robotstxt.RobotsTxt

Deprecated.

use ask(java.lang.String, java.lang.String) to get Grant from which Grant.getCrawlDelay() might be invoked.

getDisallowList(String) - Method in interface com.panforge.robotstxt.RobotsTxt

Gets a list of disallowed resources.

getHost() - Method in interface com.panforge.robotstxt.RobotsTxt

Gets host.

getSitemaps() - Method in interface com.panforge.robotstxt.RobotsTxt

Gets site maps.

getUserAgents() - Method in interface com.panforge.robotstxt.Grant

Gets user agents from the group.

Grant - Interface in com.panforge.robotstxt

Grant.

H

hasAccess() - Method in interface com.panforge.robotstxt.Grant

Checks if there is an access granted.

Q

query(String, String) - Method in interface com.panforge.robotstxt.RobotsTxt

Checks access to the given HTTP path.

R

read(InputStream) - Static method in interface com.panforge.robotstxt.RobotsTxt

Reads robots.txt available at the URL.

RobotsTxt - Interface in com.panforge.robotstxt

Represents access policy from a single "robots.txt" file.

A C G H Q R
All Classes All Packages