stream.io

Class LineStream

java.lang.Object

stream.io.AbstractStream

stream.io.AbstractLineStream

stream.io.LineStream

All Implemented Interfaces:

Source, Stream

public class LineStream
extends AbstractLineStream

This is a very simple stream that just reads from a URL line-by-line. The content of the line is stored in the attribute determined by the getKey() method of this instance. By default the key LINE is used. It also supports the specification of a simple format string that can be used to create a generic parser to populate additional fields of the data item read from the stream. The parser format is:

       %{IP} [%{DATE}] "%{URL}"
 

This will create a parser that is able to read line in the format

       127.0.0.1 [2012/03/14 12:03:48 +0100] "http://example.com/index.html"
 

The outcoming data item will have the attribute IP set to "127.0.0.1" and the DATE attribute set to "2012/03/14 12:03:48 +0100". The URL attribute will be set to http://example.com/index.html.

Author:

Christian Bockermann <christian.bockermann@udo.edu>

Field Summary

Fields

Modifier and Type	Field and Description
(package private) String	format
(package private) String	key
(package private) static org.slf4j.Logger	log
(package private) Parser<Map<String,String>>	parser

Fields inherited from class stream.io.AbstractLineStream

attributes, encoding, reader

Fields inherited from class stream.io.AbstractStream

closed, count, id, in, limit, prefix, seqId, sequenceKey, url

Fields inherited from interface stream.io.Stream

SOURCE_KEY

Constructor Summary

Constructors

Constructor and Description
LineStream(InputStream in)
LineStream(SourceURL url)

Method Summary

Modifier and Type	Method and Description
String	getFormat()
String	getKey()
Data	readNext()
void	setFormat(String format)
void	setKey(String key)

Methods inherited from class stream.io.AbstractLineStream

close, getEncoding, init, readLine, setEncoding

Methods inherited from class stream.io.AbstractStream

getId, getInputStream, getLimit, getPrefix, getSequenceKey, getUrl, read, setId, setLimit, setPrefix, setSequenceKey, setUrl

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

static org.slf4j.Logger log

key

String key

format

String format

parser

Parser<Map<String,String>> parser

Constructor Detail

LineStream

public LineStream(SourceURL url)
           throws Exception

Throws:

Exception

LineStream

public LineStream(InputStream in)
           throws Exception

Throws:

Exception

Method Detail

getKey

public String getKey()

Returns:

the key

setKey

@Parameter(required=false,
           defaultValue="LINE",
           description="")
public void setKey(String key)

Parameters:

key - the key to set

getFormat

public String getFormat()

Returns:

the format

setFormat

@Parameter(required=false,
           description="The format how to parse each line. Elements like %(KEY) will be detected and automatically populated in the resulting items.")
public void setFormat(String format)

Parameters:

format - the format to set

readNext

public Data readNext()
              throws Exception

Specified by:

readNext in class AbstractStream

Throws:

Exception

See Also:

stream.io.AbstractStream#readItem(stream.Data)