weka.gui.beans

Class ClassifierPerformanceEvaluator

java.lang.Object

java.awt.Component

java.awt.Container

javax.swing.JComponent

javax.swing.JPanel

weka.gui.beans.AbstractEvaluator

weka.gui.beans.ClassifierPerformanceEvaluator

All Implemented Interfaces:

java.awt.image.ImageObserver, java.awt.MenuContainer, java.io.Serializable, java.util.EventListener, javax.accessibility.Accessible, BatchClassifierListener, BeanCommon, EventConstraints, UserRequestAcceptor, Visible

public class ClassifierPerformanceEvaluator
extends AbstractEvaluator
implements BatchClassifierListener, java.io.Serializable, UserRequestAcceptor, EventConstraints

A bean that evaluates the performance of batch trained classifiers

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	ClassifierPerformanceEvaluator.AggregateableClassifierErrorsPlotInstances Subclass of ClassifierErrorsPlotInstances to allow plot point sizes to be scaled according to global min/max values.
protected class	ClassifierPerformanceEvaluator.EvaluationTask Inner class for running an evaluation on a split

Nested classes/interfaces inherited from class javax.swing.JPanel

javax.swing.JPanel.AccessibleJPanel

Nested classes/interfaces inherited from class javax.swing.JComponent

javax.swing.JComponent.AccessibleJComponent

Nested classes/interfaces inherited from class java.awt.Container

java.awt.Container.AccessibleAWTContainer

Nested classes/interfaces inherited from class java.awt.Component

java.awt.Component.AccessibleAWTComponent, java.awt.Component.BaselineResizeBehavior, java.awt.Component.BltBufferStrategy, java.awt.Component.FlipBufferStrategy

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	m_errorPlotPointSizeProportionalToMargin
protected int	m_executionSlots Number of threads to use to train models with
protected java.util.concurrent.ThreadPoolExecutor	m_executorPool
protected java.util.List<java.lang.String>	m_metricsList
protected java.lang.String	m_selectedEvalMetrics Evaluation metrics to output
protected java.util.List<ClassifierPerformanceEvaluator.EvaluationTask>	m_tasks

Fields inherited from class weka.gui.beans.AbstractEvaluator

m_listenee, m_logger, m_visual

Fields inherited from class javax.swing.JComponent

listenerList, TOOL_TIP_TEXT_KEY, ui, UNDEFINED_CONDITION, WHEN_ANCESTOR_OF_FOCUSED_COMPONENT, WHEN_FOCUSED, WHEN_IN_FOCUSED_WINDOW

Fields inherited from class java.awt.Component

accessibleContext, BOTTOM_ALIGNMENT, CENTER_ALIGNMENT, LEFT_ALIGNMENT, RIGHT_ALIGNMENT, TOP_ALIGNMENT

Fields inherited from interface java.awt.image.ImageObserver

ABORT, ALLBITS, ERROR, FRAMEBITS, HEIGHT, PROPERTIES, SOMEBITS, WIDTH

Constructor Summary

Constructors

Constructor and Description
ClassifierPerformanceEvaluator()

Method Summary

Modifier and Type	Method and Description
void	acceptClassifier(BatchClassifierEvent ce) Accept a classifier to be evaluated.
void	addTextListener(TextListener cl) Add a text listener
void	addThresholdDataListener(ThresholdDataListener cl) Add a threshold data listener
void	addVisualizableErrorListener(VisualizableErrorListener vel) Add a visualizable error listener
protected static Evaluation	adjustForInputMappedClassifier(Evaluation eval, Classifier classifier, Instances inst, ClassifierErrorsPlotInstances plotInstances)
protected void	aggregateEvalTask(Evaluation eval, Classifier classifier, Instances testData, ClassifierErrorsPlotInstances plotInstances, int setNum, int maxSetNum, java.lang.String evalLabel) Takes an evaluation object from a task and aggregates it with the overall one.
java.util.Enumeration<java.lang.String>	enumerateRequests() Return an enumeration of user activated requests for this bean
java.lang.String	errorPlotPointSizeProportionalToMarginTipText() Get the tip text for this property.
java.lang.String	evaluationMetricsToOutputTipText() Get the tip text for this property.
boolean	eventGeneratable(java.lang.String eventName) Returns true, if at the current time, the named event could be generated.
java.lang.String	executionSlotsTipText() Get the tip text for this property.
java.lang.String	getCustomName() Get the custom (descriptive) name for this bean (if one has been set)
boolean	getErrorPlotPointSizeProportionalToMargin() Get whether the point size on classification error plots should be proportional to the prediction margin.
java.lang.String	getEvaluationMetricsToOutput() Get the evaluation metrics to output (as a comma-separated list).
int	getExecutionSlots() Get the number of execution slots to use.
java.lang.String	globalInfo() Global info for this bean
boolean	isBusy() Returns true if. at this time, the bean is busy with some (i.e. perhaps a worker thread is performing some calculation).
void	performRequest(java.lang.String request) Perform the named request
void	removeTextListener(TextListener cl) Remove a text listener
void	removeThresholdDataListener(ThresholdDataListener cl) Remove a Threshold data listener
void	removeVisualizableErrorListener(VisualizableErrorListener vel) Remove a visualizable error listener
void	setCustomName(java.lang.String name) Set a custom (descriptive) name for this bean
void	setErrorPlotPointSizeProportionalToMargin(boolean e) Set whether the point size on classification error plots should be proportional to the prediction margin.
void	setEvaluationMetricsToOutput(java.lang.String m) Set the evaluation metrics to output (as a comma-separated list).
void	setExecutionSlots(int slots) Set the number of executions slots to use.
void	stop() Try and stop any action
protected void	stringToList(java.lang.String l)

Methods inherited from class weka.gui.beans.AbstractEvaluator

connectionAllowed, connectionAllowed, connectionNotification, disconnectionNotification, getVisual, setLog, setVisual, useDefaultVisual

Methods inherited from class javax.swing.JPanel

getAccessibleContext, getUI, getUIClassID, paramString, setUI, updateUI

Methods inherited from class javax.swing.JComponent

addAncestorListener, addNotify, addVetoableChangeListener, computeVisibleRect, contains, createToolTip, disable, enable, firePropertyChange, firePropertyChange, firePropertyChange, fireVetoableChange, getActionForKeyStroke, getActionMap, getAlignmentX, getAlignmentY, getAncestorListeners, getAutoscrolls, getBaseline, getBaselineResizeBehavior, getBorder, getBounds, getClientProperty, getComponentGraphics, getComponentPopupMenu, getConditionForKeyStroke, getDebugGraphicsOptions, getDefaultLocale, getFontMetrics, getGraphics, getHeight, getInheritsPopupMenu, getInputMap, getInputMap, getInputVerifier, getInsets, getInsets, getListeners, getLocation, getMaximumSize, getMinimumSize, getNextFocusableComponent, getPopupLocation, getPreferredSize, getRegisteredKeyStrokes, getRootPane, getSize, getToolTipLocation, getToolTipText, getToolTipText, getTopLevelAncestor, getTransferHandler, getVerifyInputWhenFocusTarget, getVetoableChangeListeners, getVisibleRect, getWidth, getX, getY, grabFocus, hide, isDoubleBuffered, isLightweightComponent, isManagingFocus, isOpaque, isOptimizedDrawingEnabled, isPaintingForPrint, isPaintingOrigin, isPaintingTile, isRequestFocusEnabled, isValidateRoot, paint, paintBorder, paintChildren, paintComponent, paintImmediately, paintImmediately, print, printAll, printBorder, printChildren, printComponent, processComponentKeyEvent, processKeyBinding, processKeyEvent, processMouseEvent, processMouseMotionEvent, putClientProperty, registerKeyboardAction, registerKeyboardAction, removeAncestorListener, removeNotify, removeVetoableChangeListener, repaint, repaint, requestDefaultFocus, requestFocus, requestFocus, requestFocusInWindow, requestFocusInWindow, resetKeyboardActions, reshape, revalidate, scrollRectToVisible, setActionMap, setAlignmentX, setAlignmentY, setAutoscrolls, setBackground, setBorder, setComponentPopupMenu, setDebugGraphicsOptions, setDefaultLocale, setDoubleBuffered, setEnabled, setFocusTraversalKeys, setFont, setForeground, setInheritsPopupMenu, setInputMap, setInputVerifier, setMaximumSize, setMinimumSize, setNextFocusableComponent, setOpaque, setPreferredSize, setRequestFocusEnabled, setToolTipText, setTransferHandler, setUI, setVerifyInputWhenFocusTarget, setVisible, unregisterKeyboardAction, update

Methods inherited from class java.awt.Container

add, add, add, add, add, addContainerListener, addImpl, addPropertyChangeListener, addPropertyChangeListener, applyComponentOrientation, areFocusTraversalKeysSet, countComponents, deliverEvent, doLayout, findComponentAt, findComponentAt, getComponent, getComponentAt, getComponentAt, getComponentCount, getComponents, getComponentZOrder, getContainerListeners, getFocusTraversalKeys, getFocusTraversalPolicy, getLayout, getMousePosition, insets, invalidate, isAncestorOf, isFocusCycleRoot, isFocusCycleRoot, isFocusTraversalPolicyProvider, isFocusTraversalPolicySet, layout, list, list, locate, minimumSize, paintComponents, preferredSize, printComponents, processContainerEvent, processEvent, remove, remove, removeAll, removeContainerListener, setComponentZOrder, setFocusCycleRoot, setFocusTraversalPolicy, setFocusTraversalPolicyProvider, setLayout, transferFocusDownCycle, validate, validateTree

Methods inherited from class java.awt.Component

action, add, addComponentListener, addFocusListener, addHierarchyBoundsListener, addHierarchyListener, addInputMethodListener, addKeyListener, addMouseListener, addMouseMotionListener, addMouseWheelListener, bounds, checkImage, checkImage, coalesceEvents, contains, createImage, createImage, createVolatileImage, createVolatileImage, disableEvents, dispatchEvent, enable, enableEvents, enableInputMethods, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, getBackground, getBounds, getColorModel, getComponentListeners, getComponentOrientation, getCursor, getDropTarget, getFocusCycleRootAncestor, getFocusListeners, getFocusTraversalKeysEnabled, getFont, getForeground, getGraphicsConfiguration, getHierarchyBoundsListeners, getHierarchyListeners, getIgnoreRepaint, getInputContext, getInputMethodListeners, getInputMethodRequests, getKeyListeners, getLocale, getLocation, getLocationOnScreen, getMouseListeners, getMouseMotionListeners, getMousePosition, getMouseWheelListeners, getName, getParent, getPeer, getPropertyChangeListeners, getPropertyChangeListeners, getSize, getToolkit, getTreeLock, gotFocus, handleEvent, hasFocus, imageUpdate, inside, isBackgroundSet, isCursorSet, isDisplayable, isEnabled, isFocusable, isFocusOwner, isFocusTraversable, isFontSet, isForegroundSet, isLightweight, isMaximumSizeSet, isMinimumSizeSet, isPreferredSizeSet, isShowing, isValid, isVisible, keyDown, keyUp, list, list, list, location, lostFocus, mouseDown, mouseDrag, mouseEnter, mouseExit, mouseMove, mouseUp, move, nextFocus, paintAll, postEvent, prepareImage, prepareImage, processComponentEvent, processFocusEvent, processHierarchyBoundsEvent, processHierarchyEvent, processInputMethodEvent, processMouseWheelEvent, remove, removeComponentListener, removeFocusListener, removeHierarchyBoundsListener, removeHierarchyListener, removeInputMethodListener, removeKeyListener, removeMouseListener, removeMouseMotionListener, removeMouseWheelListener, removePropertyChangeListener, removePropertyChangeListener, repaint, repaint, repaint, resize, resize, setBounds, setBounds, setComponentOrientation, setCursor, setDropTarget, setFocusable, setFocusTraversalKeysEnabled, setIgnoreRepaint, setLocale, setLocation, setLocation, setName, setSize, setSize, show, show, size, toString, transferFocus, transferFocusBackward, transferFocusUpCycle

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

m_executorPool

protected transient java.util.concurrent.ThreadPoolExecutor m_executorPool

m_tasks

protected transient java.util.List<ClassifierPerformanceEvaluator.EvaluationTask> m_tasks

m_errorPlotPointSizeProportionalToMargin

protected boolean m_errorPlotPointSizeProportionalToMargin

m_executionSlots

protected int m_executionSlots

Number of threads to use to train models with

m_selectedEvalMetrics

protected java.lang.String m_selectedEvalMetrics

Evaluation metrics to output

m_metricsList

protected java.util.List<java.lang.String> m_metricsList

Constructor Detail

ClassifierPerformanceEvaluator

public ClassifierPerformanceEvaluator()

Method Detail

stringToList

protected void stringToList(java.lang.String l)

setEvaluationMetricsToOutput

public void setEvaluationMetricsToOutput(java.lang.String m)

Set the evaluation metrics to output (as a comma-separated list).

Parameters:

m - the evaluation metrics to output

getEvaluationMetricsToOutput

public java.lang.String getEvaluationMetricsToOutput()

Get the evaluation metrics to output (as a comma-separated list).

Returns:

the evaluation metrics to output

evaluationMetricsToOutputTipText

public java.lang.String evaluationMetricsToOutputTipText()

Get the tip text for this property.

Returns:

the tip text for this property.

setErrorPlotPointSizeProportionalToMargin

public void setErrorPlotPointSizeProportionalToMargin(boolean e)

Set whether the point size on classification error plots should be proportional to the prediction margin.

Parameters:

e - true if the point size is to be proportional to the margin.

getErrorPlotPointSizeProportionalToMargin

public boolean getErrorPlotPointSizeProportionalToMargin()

Get whether the point size on classification error plots should be proportional to the prediction margin.

Returns:

true if the point size is to be proportional to the margin.

errorPlotPointSizeProportionalToMarginTipText

public java.lang.String errorPlotPointSizeProportionalToMarginTipText()

Get the tip text for this property.

Returns:

the tip text for this property.

getExecutionSlots

public int getExecutionSlots()

Get the number of execution slots to use.

Returns:

the number of execution slots to use

setExecutionSlots

public void setExecutionSlots(int slots)

Set the number of executions slots to use.

Parameters:

slots - the number of execution slots to use

executionSlotsTipText

public java.lang.String executionSlotsTipText()

Get the tip text for this property.

Returns:

the tip text for this property.

setCustomName

public void setCustomName(java.lang.String name)

Set a custom (descriptive) name for this bean

Specified by:

setCustomName in interface BeanCommon

Parameters:

name - the name to use

getCustomName

public java.lang.String getCustomName()

Get the custom (descriptive) name for this bean (if one has been set)

Specified by:

getCustomName in interface BeanCommon

Returns:

the custom name (or the default name)

globalInfo

public java.lang.String globalInfo()

Global info for this bean

Returns:

a String value

adjustForInputMappedClassifier

protected static Evaluation adjustForInputMappedClassifier(Evaluation eval,
                                                           Classifier classifier,
                                                           Instances inst,
                                                           ClassifierErrorsPlotInstances plotInstances)
                                                    throws java.lang.Exception

Throws:

java.lang.Exception

aggregateEvalTask

protected void aggregateEvalTask(Evaluation eval,
                                 Classifier classifier,
                                 Instances testData,
                                 ClassifierErrorsPlotInstances plotInstances,
                                 int setNum,
                                 int maxSetNum,
                                 java.lang.String evalLabel)

Takes an evaluation object from a task and aggregates it with the overall one.

Parameters:

eval - the evaluation object to aggregate

classifier - the classifier used by the task

testData - the testData from the task

plotInstances - the ClassifierErrorsPlotInstances object from the task

setNum - the set number processed by the task

maxSetNum - the maximum number of sets in this batch

evalLabel - the label to associate with the results of this evaluation

acceptClassifier

public void acceptClassifier(BatchClassifierEvent ce)

Accept a classifier to be evaluated.

Specified by:

acceptClassifier in interface BatchClassifierListener

Parameters:

ce - a BatchClassifierEvent value

isBusy

public boolean isBusy()

Returns true if. at this time, the bean is busy with some (i.e. perhaps a worker thread is performing some calculation).

Specified by:

isBusy in interface BeanCommon

Returns:

true if the bean is busy.

stop

public void stop()

Try and stop any action

Specified by:

stop in interface BeanCommon

Specified by:

stop in class AbstractEvaluator

enumerateRequests

public java.util.Enumeration<java.lang.String> enumerateRequests()

Return an enumeration of user activated requests for this bean

Specified by:

enumerateRequests in interface UserRequestAcceptor

Returns:

an Enumeration value

performRequest

public void performRequest(java.lang.String request)

Perform the named request

Specified by:

performRequest in interface UserRequestAcceptor

Parameters:

request - the request to perform

Throws:

java.lang.IllegalArgumentException - if an error occurs

addTextListener

public void addTextListener(TextListener cl)

Add a text listener

Parameters:

cl - a TextListener value

removeTextListener

public void removeTextListener(TextListener cl)

Remove a text listener

Parameters:

cl - a TextListener value

addThresholdDataListener

public void addThresholdDataListener(ThresholdDataListener cl)

Add a threshold data listener

Parameters:

cl - a ThresholdDataListener value

removeThresholdDataListener

public void removeThresholdDataListener(ThresholdDataListener cl)

Remove a Threshold data listener

Parameters:

cl - a ThresholdDataListener value

addVisualizableErrorListener

public void addVisualizableErrorListener(VisualizableErrorListener vel)

Add a visualizable error listener

Parameters:

vel - a VisualizableErrorListener value

removeVisualizableErrorListener

public void removeVisualizableErrorListener(VisualizableErrorListener vel)

Remove a visualizable error listener

Parameters:

vel - a VisualizableErrorListener value

eventGeneratable

public boolean eventGeneratable(java.lang.String eventName)

Returns true, if at the current time, the named event could be generated. Assumes that supplied event names are names of events that could be generated by this bean.

Specified by:

eventGeneratable in interface EventConstraints

Parameters:

eventName - the name of the event in question

Returns:

true if the named event could be generated at this point in time