QuaPy/TODO.txt

Documentation with sphinx
Add quantification_report (akin to classification_report from sklearn) (?)
Add NAE, NRAE
Add "measures for evaluating ordinal"?
Document methods with paper references
The parallel training in svmperf seems not to work (not sure...)
In binary quantification (hp, kindle, imdb) we used F1 in the minority class (which in kindle and hp happens to be the
negative class). This is not covered in this new implementation, in which the binary case is not treated as such, but as
an instance of single-label with 2 labels. Check
Add classnames to LabelledCollection ?
Check the overhead in OneVsAll for SVMperf-based (?)
Add HDy to QuaNet? if so, wrap HDy into OneVsAll in case the dataset is not binary.
Plots (one for binary -- the "diagonal", or for a specific class), another for the error as a funcition of drift.
Add datasets for topic.
Add other methods
Clarify whether QuaNet is an aggregative method or not.
Add datasets from Pérez-Gallego et al. 2017, 2019
Add ensemble models from Pérez-Gallego et al. 2017, 2019
Add plots models like those in Pérez-Gallego et al. 2017 (error boxes)
Add support for CV prediction in ACC and PACC for tpr, fpr
Add medium swap method
Explore the hyperparameter "number of bins" in HDy
Implement HDy for single-label?
Rename EMQ to SLD ?
How many times is the system of equations for ACC and PACC not solved? How many times is it clipped? Do they sum up
    to one always?
Parallelize the kFCV in ACC and PACC
svmperf wrapper added 2020-12-03 16:59:13 +01:00			`Documentation with sphinx`
QuaNet added, two examples of TextClassifiers added (CNN, LSTM) 2020-12-29 20:33:59 +01:00			`Add quantification_report (akin to classification_report from sklearn) (?)`
evaluation by artificial prevalence sampling added. New methods added. New util functions added to quapy.functional and quapy.utils 2020-12-10 19:04:33 +01:00			`Add NAE, NRAE`
			`Add "measures for evaluating ordinal"?`
aggregative methods adapted. Explicit loss minimization methods (SVMQ, SVMKLD, ...) added and with support to binary or single-label. HDy added 2020-12-04 19:32:08 +01:00			`Document methods with paper references`
refactoring aggregative methods as methods that not only implement 'classify' and 'quantify', but that also implement 'aggregate' and that, by default, have a default implementation of 'quantify' as a pipeline of 'classify' and 'aggregate'; this helps speeding up evaluations A LOT, since the documents can be pre-classified and the samples are carried out across pre-classified values (labels, or posterior probabilities), and thus only aggregate is called many times within the artificial sampling protocol 2020-12-11 19:28:17 +01:00			`The parallel training in svmperf seems not to work (not sure...)`
QuaNet added, two examples of TextClassifiers added (CNN, LSTM) 2020-12-29 20:33:59 +01:00			`In binary quantification (hp, kindle, imdb) we used F1 in the minority class (which in kindle and hp happens to be the`
			`negative class). This is not covered in this new implementation, in which the binary case is not treated as such, but as`
			`an instance of single-label with 2 labels. Check`
			`Add classnames to LabelledCollection ?`
			`Check the overhead in OneVsAll for SVMperf-based (?)`
added Ensemble methods (methods ALL, ACC, Ptr, DS from Pérez-Gallego et al 2017 and 2019) and some UCI ML datasets used in those articles (only 5 datasets out of 32 they used) 2021-01-06 14:58:29 +01:00			`Add HDy to QuaNet? if so, wrap HDy into OneVsAll in case the dataset is not binary.`
			`Plots (one for binary -- the "diagonal", or for a specific class), another for the error as a funcition of drift.`
			`Add datasets for topic.`
			`Add other methods`
			`Clarify whether QuaNet is an aggregative method or not.`
			`Add datasets from Pérez-Gallego et al. 2017, 2019`
			`Add ensemble models from Pérez-Gallego et al. 2017, 2019`
			`Add plots models like those in Pérez-Gallego et al. 2017 (error boxes)`
			`Add support for CV prediction in ACC and PACC for tpr, fpr`
			`Add medium swap method`
			`Explore the hyperparameter "number of bins" in HDy`
			`Implement HDy for single-label?`
			`Rename EMQ to SLD ?`
			`How many times is the system of equations for ACC and PACC not solved? How many times is it clipped? Do they sum up`
adding tweet sent quant experiments 2021-01-11 18:31:12 +01:00			`to one always?`
			`Parallelize the kFCV in ACC and PACC`