PunctuationSeparator (Deep Java Library 0.24.0 API specification)

java.lang.Object
- ai.djl.modality.nlp.preprocess.PunctuationSeparator

All Implemented Interfaces:

TextProcessor
```
public class PunctuationSeparator
extends java.lang.Object
implements TextProcessor
```
PunctuationSeparator separates punctuation into a separate token.

Constructor Summary

Constructors
Constructor Description

PunctuationSeparator()

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`java.util.List<java.lang.String>`	`preprocess(java.util.List<java.lang.String> tokens)`	Applies the preprocessing defined to the given input tokens.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - PunctuationSeparator
```
public PunctuationSeparator()
```
- Method Detail
  - preprocess
```
public java.util.List<java.lang.String> preprocess(java.util.List<java.lang.String> tokens)
```
    Applies the preprocessing defined to the given input tokens.
    
    Specified by:
    
    preprocess in interface TextProcessor
    
    Parameters:
    
    tokens - the tokens created after the input text is tokenized
    
    Returns:
    
    the preprocessed tokens