Abstract
[Poster presentation of eponym paper, published in 2017 from my work at CWI] Abstract: Classifiers can provide counts of items per class, but systematic classification errors yield biases (e.g., if a class is often misclassified as another, its size may be under-estimated). To handle classification biases, the statistics and epidemiology domains devised methods for estimating unbiased class sizes (or class probabilities) without identifying which individual items are misclassified. These bias correction methods are applicable to machine learning classifiers, but in some cases yield high result variance and increased biases. We present the applicability and drawbacks of existing methods and extend them with three novel methods. Our Sample-to-Sample method provides
accurate confidence intervals for the bias correction results. Our Maximum Determinant method predicts which classifier yields the least result variance. Our Ratio-to-TP method details the error decomposition in classifier outputs (i.e., how many items classified as class Cy truly belong to Cx, for all possible classes) and has properties of interest for applying the Maximum Determinant method. Our methods are demonstrated empirically, and we discuss the need for establishing theory and guidelines for choosing the methods and classifier to apply.
accurate confidence intervals for the bias correction results. Our Maximum Determinant method predicts which classifier yields the least result variance. Our Ratio-to-TP method details the error decomposition in classifier outputs (i.e., how many items classified as class Cy truly belong to Cx, for all possible classes) and has properties of interest for applying the Maximum Determinant method. Our methods are demonstrated empirically, and we discuss the need for establishing theory and guidelines for choosing the methods and classifier to apply.
Original language | English |
---|---|
Number of pages | 1 |
Publication status | Published - 2018 |
Externally published | Yes |
Event | BNAIC 30th Annual Conference 2018 - Den Bosch, Netherlands Duration: 8 Nov 2018 → 9 Nov 2018 |
Conference
Conference | BNAIC 30th Annual Conference 2018 |
---|---|
Abbreviated title | BNAIC 2018 |
Country | Netherlands |
City | Den Bosch |
Period | 8/11/18 → 9/11/18 |