weka.core.converters

Class ArffLoader.ArffReader

java.lang.Object

weka.core.converters.ArffLoader.ArffReader

All Implemented Interfaces:

RevisionHandler

Enclosing class:

ArffLoader

public static class ArffLoader.ArffReader
extends java.lang.Object
implements RevisionHandler

Reads data from an ARFF file, either in incremental or batch mode.

Typical code for batch usage:

 BufferedReader reader =
   new BufferedReader(new FileReader("/some/where/file.arff"));
 ArffReader arff = new ArffReader(reader);
 Instances data = arff.getData();
 data.setClassIndex(data.numAttributes() - 1);
 

Typical code for incremental usage:

 BufferedReader reader =
   new BufferedReader(new FileReader("/some/where/file.arff"));
 ArffReader arff = new ArffReader(reader, 1000);
 Instances data = arff.getStructure();
 data.setClassIndex(data.numAttributes() - 1);
 Instance inst;
 while ((inst = arff.readInstance(data)) != null) {
   data.add(inst);
 }
 

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	m_batchMode
protected Instances	m_Data the actual data
protected java.util.List<java.lang.String>	m_enclosures List of (single character) enclosures to use instead of the defaults
protected java.lang.String	m_fieldSeparator Field separator (single character string) to use instead of the defaults
protected int[]	m_IndicesBuffer Buffer of indices for sparse instance
protected int	m_Lines the number of lines read so far
protected boolean	m_retainStringValues Whether the values for string attributes will accumulate in the header when reading incrementally
protected java.util.List<java.lang.Integer>	m_stringAttIndices
protected java.io.StreamTokenizer	m_Tokenizer the tokenizer for reading the stream
protected double[]	m_ValueBuffer Buffer of values for sparse instance

Constructor Summary

Constructors

Constructor and Description
ArffReader(java.io.Reader reader) Reads the data completely from the reader.
ArffReader(java.io.Reader reader, Instances template, int lines, int capacity, boolean batch, java.lang.String... fieldSepAndEnclosures) Initializes the reader without reading the header according to the specified template.
ArffReader(java.io.Reader reader, Instances template, int lines, int capacity, java.lang.String... fieldSepAndEnclosures) Initializes the reader without reading the header according to the specified template.
ArffReader(java.io.Reader reader, Instances template, int lines, java.lang.String... fieldSepAndEnclosures) Reads the data without header according to the specified template.
ArffReader(java.io.Reader reader, int capacity)
ArffReader(java.io.Reader reader, int capacity, boolean batch) Reads only the header and reserves the specified space for instances.

Method Summary

Modifier and Type	Method and Description
protected void	compactify() compactifies the data
protected void	errorMessage(java.lang.String msg) Throws error message with line number and last token read.
protected double	getAttributeWeight() Gets the value of an attribute's weight (if one exists).
Instances	getData() Returns the data that was read
protected void	getFirstToken() Gets next token, skipping empty lines.
protected void	getIndex() Gets index, checking for a premature and of line.
protected Instance	getInstance(Instances structure, boolean flag) Reads a single instance using the tokenizer and returns it.
protected Instance	getInstanceFull(boolean flag) Reads a single instance using the tokenizer and returns it.
protected Instance	getInstanceSparse(boolean flag) Reads a single instance using the tokenizer and returns it.
protected double	getInstanceWeight() Gets the value of an instance's weight (if one exists)
protected void	getLastToken(boolean endOfFileOk) Gets token and checks if its end of line.
int	getLineNo() returns the current line number
protected void	getNextToken() Gets next token, checking for a premature and of line.
boolean	getRetainStringValues() Get whether to retain the values of string attributes in memory (in the header) when reading incrementally.
java.lang.String	getRevision() Returns the revision string.
Instances	getStructure() Returns the header format
protected void	initBuffers() initializes the buffers for sparse instances to be read
protected void	initTokenizer() Initializes the StreamTokenizer used for reading the ARFF file.
protected java.util.ArrayList<Attribute>	parseAttribute(java.util.ArrayList<Attribute> attributes) Parses the attribute declaration.
protected void	readHeader(int capacity) Reads and stores header of an ARFF file.
Instance	readInstance(Instances structure) Reads a single instance using the tokenizer and returns it.
Instance	readInstance(Instances structure, boolean flag) Reads a single instance using the tokenizer and returns it.
protected void	readTillEOL() Reads and skips all tokens before next end of line token.
void	setRetainStringValues(boolean retain) Set whether to retain the values of string attributes in memory (in the header) when reading incrementally.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

m_Tokenizer

protected java.io.StreamTokenizer m_Tokenizer

the tokenizer for reading the stream

m_ValueBuffer

protected double[] m_ValueBuffer

Buffer of values for sparse instance

m_IndicesBuffer

protected int[] m_IndicesBuffer

Buffer of indices for sparse instance

m_stringAttIndices

protected java.util.List<java.lang.Integer> m_stringAttIndices

m_Data

protected Instances m_Data

the actual data

m_Lines

protected int m_Lines

the number of lines read so far

m_batchMode

protected boolean m_batchMode

m_retainStringValues

protected boolean m_retainStringValues

Whether the values for string attributes will accumulate in the header when reading incrementally

m_fieldSeparator

protected java.lang.String m_fieldSeparator

Field separator (single character string) to use instead of the defaults

m_enclosures

protected java.util.List<java.lang.String> m_enclosures

List of (single character) enclosures to use instead of the defaults

Constructor Detail

ArffReader

public ArffReader(java.io.Reader reader)
           throws java.io.IOException

Reads the data completely from the reader. The data can be accessed via the getData() method.

Parameters:

reader - the reader to use

Throws:

java.io.IOException - if something goes wrong

See Also:

getData()

ArffReader

public ArffReader(java.io.Reader reader,
                  int capacity)
           throws java.io.IOException

Throws:

java.io.IOException

ArffReader

public ArffReader(java.io.Reader reader,
                  int capacity,
                  boolean batch)
           throws java.io.IOException

Reads only the header and reserves the specified space for instances. Further instances can be read via readInstance().

Parameters:

reader - the reader to use

capacity - the capacity of the new dataset

batch - true if reading in batch mode

Throws:

java.io.IOException - if something goes wrong

java.io.IOException - if a problem occurs

See Also:

getStructure(), readInstance(Instances)

ArffReader

public ArffReader(java.io.Reader reader,
                  Instances template,
                  int lines,
                  java.lang.String... fieldSepAndEnclosures)
           throws java.io.IOException

Reads the data without header according to the specified template. The data can be accessed via the getData() method.

Parameters:

reader - the reader to use

template - the template header

lines - the lines read so far

fieldSepAndEnclosures - an optional array of Strings containing the field separator and enclosures to use instead of the defaults. The first entry in the array is expected to be the single character field separator to use; the remaining entries (if any) are enclosure characters to use.

Throws:

java.io.IOException - if something goes wrong

See Also:

getData()

ArffReader

public ArffReader(java.io.Reader reader,
                  Instances template,
                  int lines,
                  int capacity,
                  java.lang.String... fieldSepAndEnclosures)
           throws java.io.IOException

Initializes the reader without reading the header according to the specified template. The data must be read via the readInstance() method.

Parameters:

reader - the reader to use

template - the template header

lines - the lines read so far

capacity - the capacity of the new dataset

fieldSepAndEnclosures - an optional array of Strings containing the field separator and enclosures to use instead of the defaults. The first entry in the array is expected to be the single character field separator to use; the remaining entries (if any) are enclosure characters to use.

Throws:

java.io.IOException - if something goes wrong

See Also:

getData()

ArffReader

public ArffReader(java.io.Reader reader,
                  Instances template,
                  int lines,
                  int capacity,
                  boolean batch,
                  java.lang.String... fieldSepAndEnclosures)
           throws java.io.IOException

Initializes the reader without reading the header according to the specified template. The data must be read via the readInstance() method.

Parameters:

reader - the reader to use

template - the template header

lines - the lines read so far

capacity - the capacity of the new dataset

batch - true if the data is going to be read in batch mode

fieldSepAndEnclosures - an optional array of Strings containing the field separator and enclosures to use instead of the defaults. The first entry in the array is expected to be the single character field separator to use; the remaining entries (if any) are enclosure characters to use.

Throws:

java.io.IOException - if something goes wrong

See Also:

getData()

Method Detail

initBuffers

protected void initBuffers()

initializes the buffers for sparse instances to be read

See Also:

m_ValueBuffer, m_IndicesBuffer

compactify

protected void compactify()

compactifies the data

errorMessage

protected void errorMessage(java.lang.String msg)
                     throws java.io.IOException

Throws error message with line number and last token read.

Parameters:

msg - the error message to be thrown

Throws:

java.io.IOException - containing the error message

getLineNo

public int getLineNo()

returns the current line number

Returns:

the current line number

getFirstToken

protected void getFirstToken()
                      throws java.io.IOException

Gets next token, skipping empty lines.

Throws:

java.io.IOException - if reading the next token fails

getIndex

protected void getIndex()
                 throws java.io.IOException

Gets index, checking for a premature and of line.

Throws:

java.io.IOException - if it finds a premature end of line

getLastToken

protected void getLastToken(boolean endOfFileOk)
                     throws java.io.IOException

Gets token and checks if its end of line.

Parameters:

endOfFileOk - whether EOF is OK

Throws:

java.io.IOException - if it doesn't find an end of line

getInstanceWeight

protected double getInstanceWeight()
                            throws java.io.IOException

Gets the value of an instance's weight (if one exists)

Returns:

the value of the instance's weight, or NaN if no weight has been supplied in the file

Throws:

java.io.IOException

getNextToken

protected void getNextToken()
                     throws java.io.IOException

Gets next token, checking for a premature and of line.

Throws:

java.io.IOException - if it finds a premature end of line

initTokenizer

protected void initTokenizer()

Initializes the StreamTokenizer used for reading the ARFF file.

readInstance

public Instance readInstance(Instances structure)
                      throws java.io.IOException

Reads a single instance using the tokenizer and returns it.

Parameters:

structure - the dataset header information, will get updated in case of string or relational attributes

Returns:

null if end of file has been reached

Throws:

java.io.IOException - if the information is not read successfully

readInstance

public Instance readInstance(Instances structure,
                             boolean flag)
                      throws java.io.IOException

Reads a single instance using the tokenizer and returns it.

Parameters:

structure - the dataset header information, will get updated in case of string or relational attributes

flag - if method should test for carriage return after each instance

Returns:

null if end of file has been reached

Throws:

java.io.IOException - if the information is not read successfully

getInstance

protected Instance getInstance(Instances structure,
                               boolean flag)
                        throws java.io.IOException

Reads a single instance using the tokenizer and returns it.

Parameters:

structure - the dataset header information, will get updated in case of string or relational attributes

flag - if method should test for carriage return after each instance

Returns:

null if end of file has been reached

Throws:

java.io.IOException - if the information is not read successfully

getInstanceSparse

protected Instance getInstanceSparse(boolean flag)
                              throws java.io.IOException

Reads a single instance using the tokenizer and returns it.

Parameters:

flag - if method should test for carriage return after each instance

Returns:

null if end of file has been reached

Throws:

java.io.IOException - if the information is not read successfully

getInstanceFull

protected Instance getInstanceFull(boolean flag)
                            throws java.io.IOException

Reads a single instance using the tokenizer and returns it.

Parameters:

flag - if method should test for carriage return after each instance

Returns:

null if end of file has been reached

Throws:

java.io.IOException - if the information is not read successfully

readHeader

protected void readHeader(int capacity)
                   throws java.io.IOException

Reads and stores header of an ARFF file.

Parameters:

capacity - the number of instances to reserve in the data structure

Throws:

java.io.IOException - if the information is not read successfully

parseAttribute

protected java.util.ArrayList<Attribute> parseAttribute(java.util.ArrayList<Attribute> attributes)
                                                 throws java.io.IOException

Parses the attribute declaration.

Parameters:

attributes - the current attributes vector

Returns:

the new attributes vector

Throws:

java.io.IOException - if the information is not read successfully

readTillEOL

protected void readTillEOL()
                    throws java.io.IOException

Reads and skips all tokens before next end of line token.

Throws:

java.io.IOException - in case something goes wrong

getAttributeWeight

protected double getAttributeWeight()
                             throws java.io.IOException

Gets the value of an attribute's weight (if one exists).

Returns:

the value of the attribute's weight, or 1.0 if no weight has been supplied in the file

Throws:

java.io.IOException

getStructure

public Instances getStructure()

Returns the header format

Returns:

the header format

getData

public Instances getData()

Returns the data that was read

Returns:

the data

setRetainStringValues

public void setRetainStringValues(boolean retain)

Set whether to retain the values of string attributes in memory (in the header) when reading incrementally.

Parameters:

retain - true if string values are to be retained in memory when reading incrementally

getRetainStringValues

public boolean getRetainStringValues()

Get whether to retain the values of string attributes in memory (in the header) when reading incrementally.

Returns:

true if string values are to be retained in memory when reading incrementally

getRevision

public java.lang.String getRevision()

Returns the revision string.

Specified by:

getRevision in interface RevisionHandler

Returns:

the revision