Class ClassifierSubsetEval

All Implemented Interfaces:
Serializable, ErrorBasedMeritEvaluator, SubsetEvaluator, CapabilitiesHandler, OptionHandler, RevisionHandler

public class ClassifierSubsetEval extends HoldOutSubsetEvaluator implements OptionHandler, ErrorBasedMeritEvaluator
Classifier subset evaluator:

Evaluates attribute subsets on training data or a seperate hold out testing set. Uses a classifier to estimate the 'merit' of a set of attributes.

Valid options are:

 -B <classifier>
  class name of the classifier to use for accuracy estimation.
  Place any classifier options LAST on the command line
  following a "--". eg.:
   -B weka.classifiers.bayes.NaiveBayes ... -- -K
  (default: weka.classifiers.rules.ZeroR)
 -T
  Use the training data to estimate accuracy.
 -H <filename>
  Name of the hold out/test set to 
  estimate accuracy on.
 
 Options specific to scheme weka.classifiers.rules.ZeroR:
 
 -D
  If set, classifier is run in debug mode and
  may output additional info to the console
Version:
$Revision: 5511 $
Author:
Mark Hall (mhall@cs.waikato.ac.nz)
See Also:
  • Constructor Details

    • ClassifierSubsetEval

      public ClassifierSubsetEval()
  • Method Details

    • globalInfo

      public String globalInfo()
      Returns a string describing this attribute evaluator
      Returns:
      a description of the evaluator suitable for displaying in the explorer/experimenter gui
    • listOptions

      public Enumeration listOptions()
      Returns an enumeration describing the available options.
      Specified by:
      listOptions in interface OptionHandler
      Returns:
      an enumeration of all the available options.
    • setOptions

      public void setOptions(String[] options) throws Exception
      Parses a given list of options.

      Valid options are:

       -B <classifier>
        class name of the classifier to use for accuracy estimation.
        Place any classifier options LAST on the command line
        following a "--". eg.:
         -B weka.classifiers.bayes.NaiveBayes ... -- -K
        (default: weka.classifiers.rules.ZeroR)
       -T
        Use the training data to estimate accuracy.
       -H <filename>
        Name of the hold out/test set to 
        estimate accuracy on.
       
       Options specific to scheme weka.classifiers.rules.ZeroR:
       
       -D
        If set, classifier is run in debug mode and
        may output additional info to the console
      Specified by:
      setOptions in interface OptionHandler
      Parameters:
      options - the list of options as an array of strings
      Throws:
      Exception - if an option is not supported
    • classifierTipText

      public String classifierTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • setClassifier

      public void setClassifier(Classifier newClassifier)
      Set the classifier to use for accuracy estimation
      Parameters:
      newClassifier - the Classifier to use.
    • getClassifier

      public Classifier getClassifier()
      Get the classifier used as the base learner.
      Returns:
      the classifier used as the classifier
    • holdOutFileTipText

      public String holdOutFileTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • getHoldOutFile

      public File getHoldOutFile()
      Gets the file that holds hold out/test instances.
      Returns:
      File that contains hold out instances
    • setHoldOutFile

      public void setHoldOutFile(File h)
      Set the file that contains hold out/test instances
      Parameters:
      h - the hold out file
    • useTrainingTipText

      public String useTrainingTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • getUseTraining

      public boolean getUseTraining()
      Get if training data is to be used instead of hold out/test data
      Returns:
      true if training data is to be used instead of hold out data
    • setUseTraining

      public void setUseTraining(boolean t)
      Set if training data is to be used instead of hold out/test data
      Parameters:
      t - true if training data is to be used instead of hold out data
    • getOptions

      public String[] getOptions()
      Gets the current settings of ClassifierSubsetEval
      Specified by:
      getOptions in interface OptionHandler
      Returns:
      an array of strings suitable for passing to setOptions()
    • getCapabilities

      public Capabilities getCapabilities()
      Returns the capabilities of this evaluator.
      Specified by:
      getCapabilities in interface CapabilitiesHandler
      Overrides:
      getCapabilities in class ASEvaluation
      Returns:
      the capabilities of this evaluator
      See Also:
    • buildEvaluator

      public void buildEvaluator(Instances data) throws Exception
      Generates a attribute evaluator. Has to initialize all fields of the evaluator that are not being set via options.
      Specified by:
      buildEvaluator in class ASEvaluation
      Parameters:
      data - set of instances serving as training data
      Throws:
      Exception - if the evaluator has not been generated successfully
    • evaluateSubset

      public double evaluateSubset(BitSet subset) throws Exception
      Evaluates a subset of attributes
      Specified by:
      evaluateSubset in interface SubsetEvaluator
      Parameters:
      subset - a bitset representing the attribute subset to be evaluated
      Returns:
      the error rate
      Throws:
      Exception - if the subset could not be evaluated
    • evaluateSubset

      public double evaluateSubset(BitSet subset, Instances holdOut) throws Exception
      Evaluates a subset of attributes with respect to a set of instances. Calling this function overides any test/hold out instancs set from setHoldOutFile.
      Specified by:
      evaluateSubset in class HoldOutSubsetEvaluator
      Parameters:
      subset - a bitset representing the attribute subset to be evaluated
      holdOut - a set of instances (possibly seperate and distinct from those use to build/train the evaluator) with which to evaluate the merit of the subset
      Returns:
      the "merit" of the subset on the holdOut data
      Throws:
      Exception - if the subset cannot be evaluated
    • evaluateSubset

      public double evaluateSubset(BitSet subset, Instance holdOut, boolean retrain) throws Exception
      Evaluates a subset of attributes with respect to a single instance. Calling this function overides any hold out/test instances set through setHoldOutFile.
      Specified by:
      evaluateSubset in class HoldOutSubsetEvaluator
      Parameters:
      subset - a bitset representing the attribute subset to be evaluated
      holdOut - a single instance (possibly not one of those used to build/train the evaluator) with which to evaluate the merit of the subset
      retrain - true if the classifier should be retrained with respect to the new subset before testing on the holdOut instance.
      Returns:
      the "merit" of the subset on the holdOut instance
      Throws:
      Exception - if the subset cannot be evaluated
    • toString

      public String toString()
      Returns a string describing classifierSubsetEval
      Overrides:
      toString in class Object
      Returns:
      the description as a string
    • getRevision

      public String getRevision()
      Returns the revision string.
      Specified by:
      getRevision in interface RevisionHandler
      Overrides:
      getRevision in class ASEvaluation
      Returns:
      the revision
    • main

      public static void main(String[] args)
      Main method for testing this class.
      Parameters:
      args - the options