com.trigonic.jrobotx
Class RobotExclusion

java.lang.Object
  com.trigonic.jrobotx.RobotExclusion

public class RobotExclusion
extends java.lang.Object

Constructor Summary

RobotExclusion()

RobotExclusion(URLInputStreamFactory urlInputStreamFactory)

Method Summary

boolean	allows(java.net.URL url, java.lang.String userAgentString) Determine whether the specified URL is allowed for the specified user agent string.
RecordIterator	get(java.net.URL url) Get a robot exclusion RecordIterator for the server in the specified URL, or null if none is available.
Record	get(java.net.URL url, java.lang.String userAgentString) Get a robot exclusion Record for the specified URL, or null if none is available.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RobotExclusion

public RobotExclusion(URLInputStreamFactory urlInputStreamFactory)

RobotExclusion

public RobotExclusion()

Method Detail

get

public RecordIterator get(java.net.URL url)

Get a robot exclusion RecordIterator for the server in the specified URL, or null if none is available. If the protocol is not supported--that is, not HTTP-based--null is returned.

get

public Record get(java.net.URL url,
                  java.lang.String userAgentString)

Get a robot exclusion Record for the specified URL, or null if none is available. This uses get(URL) and iterates through the RecordIterator to find a matching Record.

allows

public boolean allows(java.net.URL url,
                      java.lang.String userAgentString)

Determine whether the specified URL is allowed for the specified user agent string. This uses get(URL, String) and returns whether the matching record allows the path specified in the URL.