Evaluation and sampling

1. Confusion matrix. Accuracy, error rate, weighted error. Precision, recall, F1 measure.

2. Evaluation measures for a regression task 

3. How to build a test set. N-fold cross validation. 

4. Comparing different settings of classifiers. Learning curve, ROC curve.

5. Comparing different classifiers. Parametric t-test and non-parametric Wilcoxon test.

6. Sampling. Bootstrapping. Undersampling and oversampling