Classification (classification
)¶
Logistic Regression¶

class
Orange.classification.
LogisticRegressionLearner
(penalty=’l2’, dual=False, tol=0.0001, C=1.0, fit_intercept=True, intercept_scaling=1, class_weight=None, random_state=None, solver=’liblinear’, max_iter=100, multi_class=’ovr’, verbose=0, n_jobs=1, preprocessors=None)[source]¶ A wrapper for sklearn.linear_model.logistic.LogisticRegression. The following is its documentation:
Logistic Regression (aka logit, MaxEnt) classifier.
In the multiclass case, the training algorithm uses the onevsrest (OvR) scheme if the ‘multi_class’ option is set to ‘ovr’, and uses the cross entropy loss if the ‘multi_class’ option is set to ‘multinomial’. (Currently the ‘multinomial’ option is supported only by the ‘lbfgs’, ‘sag’ and ‘newtoncg’ solvers.)
This class implements regularized logistic regression using the ‘liblinear’ library, ‘newtoncg’, ‘sag’ and ‘lbfgs’ solvers. It can handle both dense and sparse input. Use Cordered arrays or CSR matrices containing 64bit floats for optimal performance; any other input format will be converted (and copied).
The ‘newtoncg’, ‘sag’, and ‘lbfgs’ solvers support only L2 regularization with primal formulation. The ‘liblinear’ solver supports both L1 and L2 regularization, with a dual formulation only for the L2 penalty.
Read more in the User Guide.
Random Forest¶

class
Orange.classification.
RandomForestLearner
(n_estimators=10, criterion=’gini’, max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=’auto’, max_leaf_nodes=None, bootstrap=True, oob_score=False, n_jobs=1, random_state=None, verbose=0, class_weight=None, preprocessors=None)[source]¶ A wrapper for sklearn.ensemble.forest.RandomForestClassifier. The following is its documentation:
A random forest classifier.
A random forest is a meta estimator that fits a number of decision tree classifiers on various subsamples of the dataset and use averaging to improve the predictive accuracy and control overfitting. The subsample size is always the same as the original input sample size but the samples are drawn with replacement if bootstrap=True (default).
Read more in the User Guide.
Simple Random Forest¶

class
Orange.classification.
SimpleRandomForestLearner
(n_estimators=10, min_instances=2, max_depth=1024, max_majority=1.0, skip_prob=’sqrt’, seed=42)[source]¶ A random forest classifier, optimized for speed. Trees in the forest are constructed with
SimpleTreeLearner
classification trees.Parameters: n_estimators : int, optional (default = 10)
Number of trees in the forest.
min_instances : int, optional (default = 2)
Minimal number of data instances in leaves. When growing the three, new nodes are not introduced if they would result in leaves with fewer instances than min_instances. Instance count is weighed.
max_depth : int, optional (default = 1024)
Maximal depth of tree.
max_majority : float, optional (default = 1.0)
Maximal proportion of majority class. When this is exceeded, induction stops (only used for classification).
skip_prob : string, optional (default = “sqrt”)
Data attribute will be skipped with probability
skip_prob
. if float, then skip attribute with this probability.
 if “sqrt”, then skip_prob = 1  sqrt(n_features) / n_features
 if “log2”, then skip_prob = 1  log2(n_features) / n_features
seed : int, optional (default = 42)
Random seed.
Softmax Regression¶

class
Orange.classification.
SoftmaxRegressionLearner
(lambda_=1.0, preprocessors=None, **fmin_args)[source]¶ L2 regularized softmax regression classifier. Uses the LBFGS algorithm to minimize the categorical cross entropy cost with L2 regularization. This model is suitable when dealing with a multiclass classification problem.
When using this learner you should:
 choose a suitable regularization parameter lambda_,
 consider using many logistic regression models (one for each value of the class variable) instead of softmax regression.
Parameters: lambda_ : float, optional (default=1.0)
Regularization parameter. It controls tradeoff between fitting the data and keeping parameters small. Higher values of lambda_ force parameters to be smaller.
preprocessors : list, optional (default=[RemoveNaNClasses(), RemoveNaNColumns(), Impute(), Continuize(), Normalize()])
Preprocessors are applied to data before training or testing. Default preprocessors:
 remove columns with all values as NaN
 replace NaN values with suitable values
 continuize all discrete attributes,
 transform the dataset so that the columns are on a similar scale,
fmin_args : dict, optional
Parameters for LBFGS algorithm.
kNearest Neighbors¶

class
Orange.classification.
KNNLearner
(n_neighbors=5, metric=’euclidean’, weights=’uniform’, algorithm=’auto’, metric_params=None, preprocessors=None)[source]¶ A wrapper for sklearn.neighbors.classification.KNeighborsClassifier. The following is its documentation:
Classifier implementing the knearest neighbors vote.
Read more in the User Guide.
Naive Bayes¶

class
Orange.classification.
NaiveBayesLearner
(preprocessors=None)[source]¶ Naive Bayes classifier. Works only with discrete attributes. By default, continuous attributes are discretized.
Parameters: preprocessors : list, optional (default=”[Orange.preprocess.Discretize]”)
An ordered list of preprocessors applied to data before training or testing.
The following code loads lenses data set (four discrete attributes and discrete class), constructs naive Bayesian learner, uses it on the entire data set to construct a classifier, and then applies classifier to the first three data instances:
>>> import Orange
>>> lenses = Orange.data.Table('lenses')
>>> nb = Orange.classification.NaiveBayesLearner()
>>> classifier = nb(lenses)
>>> classifier(lenses[0:3], True)
array([[ 0.04358755, 0.82671726, 0.12969519],
[ 0.17428279, 0.20342097, 0.62229625],
[ 0.18633359, 0.79518516, 0.01848125]])
For data sets that include continuous attributes,
Support Vector Machines¶

class
Orange.classification.
SVMLearner
(C=1.0, kernel=’rbf’, degree=3, gamma=’auto’, coef0=0.0, shrinking=True, probability=False, tol=0.001, cache_size=200, max_iter=1, preprocessors=None)[source]¶ A wrapper for sklearn.svm.classes.SVC. The following is its documentation:
CSupport Vector Classification.
The implementation is based on libsvm. The fit time complexity is more than quadratic with the number of samples which makes it hard to scale to dataset with more than a couple of 10000 samples.
The multiclass support is handled according to a onevsone scheme.
For details on the precise mathematical formulation of the provided kernel functions and how gamma, coef0 and degree affect each other, see the corresponding section in the narrative documentation: svm_kernels.
Read more in the User Guide.
Linear Support Vector Machines¶

class
Orange.classification.
LinearSVMLearner
(penalty=’l2’, loss=’squared_hinge’, dual=True, tol=0.0001, C=1.0, multi_class=’ovr’, fit_intercept=True, intercept_scaling=True, random_state=None, preprocessors=None)[source]¶ A wrapper for sklearn.svm.classes.LinearSVC. The following is its documentation:
Linear Support Vector Classification.
Similar to SVC with parameter kernel=’linear’, but implemented in terms of liblinear rather than libsvm, so it has more flexibility in the choice of penalties and loss functions and should scale better to large numbers of samples.
This class supports both dense and sparse input and the multiclass support is handled according to a onevstherest scheme.
Read more in the User Guide.
NuSupport Vector Machines¶

class
Orange.classification.
NuSVMLearner
(nu=0.5, kernel=’rbf’, degree=3, gamma=’auto’, coef0=0.0, shrinking=True, probability=False, tol=0.001, cache_size=200, max_iter=1, preprocessors=None)[source]¶ A wrapper for sklearn.svm.classes.NuSVC. The following is its documentation:
NuSupport Vector Classification.
Similar to SVC but uses a parameter to control the number of support vectors.
The implementation is based on libsvm.
Read more in the User Guide.
One Class Support Vector Machines¶

class
Orange.classification.
OneClassSVMLearner
(kernel=’rbf’, degree=3, gamma=’auto’, coef0=0.0, tol=0.001, nu=0.5, shrinking=True, cache_size=200, max_iter=1, preprocessors=None)[source]¶ A wrapper for sklearn.svm.classes.OneClassSVM. The following is its documentation:
Unsupervised Outlier Detection.
Estimate the support of a highdimensional distribution.
The implementation is based on libsvm.
Read more in the User Guide.
Classification Tree¶
Orange includes three implemenations of classification trees. TreeLearner is homegrown and properly handles multinominal and missing values. The one from scikitlearn, SklTreeLearner, is faster. Another homegrown, SimpleTreeLearner, is simpler and stil faster.

class
Orange.classification.
TreeLearner
(*args, binarize=False, max_depth=None, min_samples_leaf=1, min_samples_split=2, sufficient_majority=0.95, **kwargs)[source]¶ Tree inducer with proper handling of nominal attributes and binarization.
The inducer can handle missing values of attributes and target. For discrete attributes with more than two possible values, each value can get a separate branch (binarize=False), or values can be grouped into two groups (binarize=True, default).
The tree growth can be limited by the required number of instances for internal nodes and for leafs, the sufficient proportion of majority class, and by the maximal depth of the tree.
If the tree is not binary, it can contain zerobranches.
 Args:
 binarize (bool):
 if True the inducer will find optimal split into two subsets for values of discrete attributes. If False (default), each value gets its branch.
 min_samples_leaf (float):
 the minimal number of data instances in a leaf
 min_samples_split (float):
 the minimal nubmer of data instances that is split into subgroups
max_depth (int): the maximal depth of the tree
 sufficient_majority (float):
 a majority at which the data is not split further
 Returns:
 instance of OrangeTreeModel
Simple Tree¶

class
Orange.classification.
SimpleTreeLearner
(min_instances=2, max_depth=1024, max_majority=1.0, skip_prob=0.0, bootstrap=False, seed=42)[source]¶ Classification or regression tree learner. Uses gain ratio for classification and mean square error for regression. This learner was developed to speedup random forest construction, but can also be used as a standalone tree learner.
 min_instances : int, optional (default = 2)
 Minimal number of data instances in leaves. When growing the three, new nodes are not introduced if they would result in leaves with fewer instances than min_instances. Instance count is weighed.
 max_depth : int, optional (default = 1024)
 Maximal depth of tree.
 max_majority : float, optional (default = 1.0)
 Maximal proportion of majority class. When this is exceeded, induction stops (only used for classification).
 skip_prob : string, optional (default = 0.0)
Data attribute will be skipped with probability
skip_prob
. if float, then skip attribute with this probability.
 if “sqrt”, then skip_prob = 1  sqrt(n_features) / n_features
 if “log2”, then skip_prob = 1  log2(n_features) / n_features
 bootstrap : data table, optional (default = False)
 A bootstrap data set.
 seed : int, optional (default = 42)
 Random seed.
Majority Classifier¶

class
Orange.classification.
MajorityLearner
(preprocessors=None)[source]¶ A majority classifier. Always returns most frequent class from the training set, regardless of the attribute values from the test data instance. Returns class value distribution if class probabilities are requested. Can be used as a baseline when comparing classifiers.
In the special case of uniform class distribution within the training data, class value is selected randomly. In order to produce consistent results on the same data set, this value is selected based on hash of the class vector.
Elliptic Envelope¶

class
Orange.classification.
EllipticEnvelopeLearner
(store_precision=True, assume_centered=False, support_fraction=None, contamination=0.1, random_state=None, preprocessors=None)[source]¶ A wrapper for sklearn.covariance.outlier_detection.EllipticEnvelope. The following is its documentation:
An object for detecting outliers in a Gaussian distributed dataset.
Read more in the User Guide.
Neural Network¶

class
Orange.classification.
NNClassificationLearner
(hidden_layer_sizes=(100, ), activation=’relu’, solver=’adam’, alpha=0.0001, batch_size=’auto’, learning_rate=’constant’, learning_rate_init=0.001, power_t=0.5, max_iter=200, shuffle=True, random_state=None, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True, early_stopping=False, validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e08, preprocessors=None)[source]¶ A wrapper for sklearn.neural_network.multilayer_perceptron.MLPClassifier. The following is its documentation:
Multilayer Perceptron classifier.
This model optimizes the logloss function using LBFGS or stochastic gradient descent.
New in version 0.18.
CN2 Rule Induction¶
Induction of rules works by finding a rule that covers some learning instances, removing these instances, and repeating this until all instances are covered. Rules are scored by heuristics such as impurity of class distribution of covered instances. The module includes common rulelearning algorithms, and allows for replacing rule search strategies, scoring and other components.

class
Orange.classification.rules.
CN2Learner
(preprocessors=None, base_rules=None)[source]¶ Classic CN2 inducer that constructs a list of ordered rules. To evaluate found hypotheses, entropy measure is used. Returns a CN2Classifier if called with data.
References
[R11] “The CN2 Induction Algorithm”, Peter Clark and Tim Niblett, Machine Learning Journal, 3 (4), pp261283, (1989)

class
Orange.classification.rules.
CN2UnorderedLearner
(preprocessors=None, base_rules=None)[source]¶ Construct a set of unordered rules.
Rules are learnt for each class individually and scored by the relative frequency of the class corrected by the Laplace correction. After adding a rule, only the covered examples of that class are removed.
The code below loads the iris data set (four continuous attributes and a discrete class) and fits the learner.
import Orange data = Orange.data.Table('iris') learner = Orange.classification.CN2UnorderedLearner() # consider up to 10 solution streams at one time learner.rule_finder.search_algorithm.beam_width = 10 # continuous value space is constrained to reduce computation time learner.rule_finder.search_strategy.constrain_continuous = True # found rules must cover at least 15 examples learner.rule_finder.general_validator.min_covered_examples = 15 # found rules may combine at most 2 selectors (conditions) learner.rule_finder.general_validator.max_rule_length = 2 classifier = learner(data)
References
[R33] “Rule Induction with CN2: Some Recent Improvements”, Peter Clark and Robin Boswell, Machine Learning  Proceedings of the 5th European Conference (EWSL91), pp151163, 1991

class
Orange.classification.rules.
CN2SDLearner
(preprocessors=None, base_rules=None)[source]¶ Ordered CN2SD inducer that constructs a list of ordered rules. To evaluate found hypotheses, Weighted relative accuracy measure is used. Returns a CN2SDClassifier if called with data.
In this setting, ordered rule induction refers exclusively to finding best rule conditions and assigning the majority class in the rule head (target class is set to None). To later predict instances, rules will be regarded as unordered.
Notes
A weighted covering algorithm is applied, in which subsequently induced rules also represent interesting and sufficiently large subgroups of the population. Covered positive examples are not deleted from the learning set, rather their weight is reduced.
The algorithm demonstrates how classification rule learning (predictive induction) can be adapted to subgroup discovery, a task at the intersection of predictive and descriptive induction.
References
[R55] “Subgroup Discovery with CN2SD”, Nada Lavrač et al., Journal of Machine Learning Research 5 (2004), 153188, 2004

class
Orange.classification.rules.
CN2SDUnorderedLearner
(preprocessors=None, base_rules=None)[source]¶ Unordered CN2SD inducer that constructs a set of unordered rules. To evaluate found hypotheses, Weighted relative accuracy measure is used. Returns a CN2SDUnorderedClassifier if called with data.
Notes
A weighted covering algorithm is applied, in which subsequently induced rules also represent interesting and sufficiently large subgroups of the population. Covered positive examples are not deleted from the learning set, rather their weight is reduced.
The algorithm demonstrates how classification rule learning (predictive induction) can be adapted to subgroup discovery, a task at the intersection of predictive and descriptive induction.
References
[R77] “Subgroup Discovery with CN2SD”, Nada Lavrač et al., Journal of Machine Learning Research 5 (2004), 153188, 2004