Package opennlp.tools.tokenize

Class TokenSample

java.lang.Object
opennlp.tools.tokenize.TokenSample
All Implemented Interfaces:
Serializable
public class TokenSample extends Object implements Serializable
A TokenSample is text with token spans.
See Also:

  • Field Details

  • Constructor Details

    • TokenSample

      public TokenSample(String text, Span[] tokenSpans)
      Initializes the current instance.
      Parameters:
      text - the text which contains the tokens.
      tokenSpans - the spans which mark the begin and end of the tokens.

    • TokenSample

      public TokenSample(Detokenizer detokenizer, String[] tokens)

  • Method Details

    • getText

      public String getText()
      Retrieves the text.

    • getTokenSpans

      public Span[] getTokenSpans()
      Retrieves the token spans.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • parse

      public static TokenSample parse(String sampleString, String separatorChars)

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object