Merge branch 'devel' of gitea-s2i2s.isti.cnr.it:moreo/QuaPy into devel

I have revised this PR (which was very nice, thanks). I have made some modifications including
improvements in the normalization functions, documentation, and refactoring of qp.functional.

I will leave this in devel until I find the time to "stress-test" the modifications.

Thanks to Pawel Czyz for the nice contribution!

CHANGE_LOG.txt

@@ -1,7 +1,9 @@
 Change Log 0.1.9
 ----------------
 
-<...>
+- Added Bayesian CC method (thanks to Pawel Czyz!). The method is described in detail in the paper
+    Ziegler, Albert, and Paweł Czyż. "Bayesian Quantification with Black-Box Estimators."
+    arXiv preprint arXiv:2302.09159 (2023).
 
 
 Change Log 0.1.8

README.md

@@ -14,7 +14,6 @@ for facilitating the analysis and interpretation of the experimental results.
 ### Last updates:
 
 * Version 0.1.8 is released! major changes can be consulted [here](CHANGE_LOG.txt).
-* A detailed documentation is now available [here](https://hlt-isti.github.io/QuaPy/)
 * The developer API documentation is available [here](https://hlt-isti.github.io/QuaPy/build/html/modules.html)
 
 ### Installation

TODO.txt

@@ -1,95 +0,0 @@
-ensembles seem to be broken; they have an internal model selection which takes the parameters, but since quapy now
-    works with protocols it would need to know the validation set in order to pass something like
-    "protocol: APP(val, etc.)"
-sample_size should not be mandatory when qp.environ['SAMPLE_SIZE'] has been specified
-clean all the cumbersome methods that have to be implemented for new quantifiers (e.g., n_classes_ prop, etc.)
-make truly parallel the GridSearchQ
-make more examples in the "examples" directory
-merge with master, because I had to fix some problems with QuaNet due to an issue notified via GitHub!
-added cross_val_predict in qp.model_selection (i.e., a cross_val_predict for quantification) --would be nice to have
-    it parallelized
-
-check the OneVsAll module(s)
-
-check the set_params de neural.py, because the separation of estimator__<param> is not implemented; see also
-    __check_params_colision
-
-HDy can be customized so that the number of bins is specified, instead of explored within the fit method
-
-Packaging:
-==========================================
-Document methods with paper references
-unit-tests
-clean wiki_examples!
-
-Refactor:
-==========================================
-Unify ThresholdOptimization methods, as an extension of PACC (and not ACC), the fit methods are almost identical and
-    use a prob classifier (take into account that PACC uses pcc internally, whereas the threshold methods use cc
-    instead). The fit method of ACC and PACC has a block for estimating the validation estimates that should be unified
-    as well...
-Refactor protocols. APP and NPP related functionalities are duplicated in functional, LabelledCollection, and evaluation
-
-
-New features:
-==========================================
-Add "measures for evaluating ordinal"?
-Add datasets for topic.
-Do we want to cover cross-lingual quantification natively in QuaPy, or does it make more sense as an application on top?
-
-Current issues:
-==========================================
-Revise the class structure of quantification methods and the methods they inherit... There is some confusion regarding
-    methods isbinary, isprobabilistic, and the like. The attribute "learner_" in aggregative quantifiers is also
-    confusing, since there is a getter and a setter.
-Remove the "deep" in get_params. There is no real compatibility with scikit-learn as for now.
-SVMperf-based learners do not remove temp files in __del__?
-In binary quantification (hp, kindle, imdb) we used F1 in the minority class (which in kindle and hp happens to be the
-negative class). This is not covered in this new implementation, in which the binary case is not treated as such, but as
-an instance of single-label with 2 labels. Check
-Add automatic reindex of class labels in LabelledCollection (currently, class indexes should be ordered and with no gaps)
-OVR I believe is currently tied to aggregative methods. We should provide a general interface also for general quantifiers
-Currently, being "binary" only adds one checker; we should figure out how to impose the check to be automatically performed
-Add random seed management to support replicability (see temp_seed in util.py).
-GridSearchQ is not trully parallelized. It only parallelizes on the predictions.
-In the context of a quantifier (e.g., QuaNet or CC), the parameters of the learner should be prefixed with "estimator__",
-    in QuaNet this is resolved with a __check_params_colision, but this should be improved. It might be cumbersome to
-    impose the "estimator__" prefix for, e.g., quantifiers like CC though... This should be changed everywhere...
-QuaNet needs refactoring. The base quantifiers ACC and PACC receive val_data with instances already transformed. This
-    issue is due to a bad design.
-
-Improvements:
-==========================================
-Explore the hyperparameter "number of bins" in HDy
-Rename EMQ to SLD ?
-Parallelize the kFCV in ACC and PACC?
-Parallelize model selection trainings
-We might want to think of (improving and) adding the class Tabular (it is defined and used on branch tweetsent). A more
-    recent version is in the project ql4facct. This class is meant to generate latex tables from results (highligting
-    best results, computing statistical tests, colouring cells, producing rankings, producing averages, etc.). Trying
-    to generate tables is typically a bad idea, but in this specific case we do have pretty good control of what an
-    experiment looks like. (Do we want to abstract experimental results? this could be useful not only for tables but
-    also for plots).
-Add proper logging system. Currently we use print
-It might be good to simplify the number of methods that have to be implemented for any new Quantifier. At the moment,
-    there are many functions like get_params, set_params, and, specially, @property classes_, which are cumbersome to
-    implement for quick experiments. A possible solution is to impose get_params and set_params only in cases in which
-    the model extends some "ModelSelectable" interface only. The classes_ should have a default implementation.
-
-Checks:
-==========================================
-How many times is the system of equations for ACC and PACC not solved? How many times is it clipped? Do they sum up
-    to one always?
-Re-check how hyperparameters from the quantifier and hyperparameters from the classifier (in aggregative quantifiers)
-    is handled. In scikit-learn the hyperparameters from a wrapper method are indicated directly whereas the hyperparams
-    from the internal learner are prefixed with "estimator__". In QuaPy, combinations having to do with the classifier
-    can be computed at the begining, and then in an internal loop the hyperparams of the quantifier can be explored,
-    passing fit_learner=False.
-Re-check Ensembles. As for now, they are strongly tied to aggregative quantifiers.
-Re-think the environment variables. Maybe add new ones (like, for example, parameters for the plots)
-Do we want to wrap prevalences (currently simple np.ndarray) as a class? This might be convenient for some interfaces
-    (e.g., for specifying artificial prevalences in samplings, for printing them -- currently supported through
-    F.strprev(), etc.). This might however add some overload, and prevent/difficult post processing with numpy.
-Would be nice to get a better integration with sklearn.
-
-

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/build/html/_static/basic.css

@@ -4,7 +4,7 @@
  *
  * Sphinx stylesheet -- basic theme.
  *
- * :copyright: Copyright 2007-2024 by the Sphinx team, see AUTHORS.
+ * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS.
  * :license: BSD, see LICENSE for details.
  *
  */
@@ -237,10 +237,6 @@ a.headerlink {
     visibility: hidden;
 }
 
-a:visited {
-    color: #551A8B;
-}
-
 h1:hover > a.headerlink,
 h2:hover > a.headerlink,
 h3:hover > a.headerlink,
@@ -328,7 +324,6 @@ aside.sidebar {
 p.sidebar-title {
     font-weight: bold;
 }
-
 nav.contents,
 aside.topic,
 div.admonition, div.topic, blockquote {
@@ -336,7 +331,6 @@ div.admonition, div.topic, blockquote {
 }
 
 /* -- topics ---------------------------------------------------------------- */
-
 nav.contents,
 aside.topic,
 div.topic {
@@ -612,7 +606,6 @@ ol.simple p,
 ul.simple p {
     margin-bottom: 0;
 }
-
 aside.footnote > span,
 div.citation > span {
     float: left;
@@ -674,16 +667,6 @@ dd {
     margin-left: 30px;
 }
 
-.sig dd {
-    margin-top: 0px;
-    margin-bottom: 0px;
-}
-
-.sig dl {
-    margin-top: 0px;
-    margin-bottom: 0px;
-}
-
 dl > dd:last-child,
 dl > dd:last-child > :last-child {
     margin-bottom: 0;
@@ -752,14 +735,6 @@ abbr, acronym {
     cursor: help;
 }
 
-.translated {
-    background-color: rgba(207, 255, 207, 0.2)
-}
-
-.untranslated {
-    background-color: rgba(255, 207, 207, 0.2)
-}
-
 /* -- code displays --------------------------------------------------------- */
 
 pre {

docs/build/html/_static/doctools.js

@@ -4,7 +4,7 @@
  *
  * Base JavaScript utilities for all Sphinx HTML documentation.
  *
- * :copyright: Copyright 2007-2024 by the Sphinx team, see AUTHORS.
+ * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS.
  * :license: BSD, see LICENSE for details.
  *
  */

docs/build/html/_static/documentation_options.js

@@ -1,5 +1,6 @@
-const DOCUMENTATION_OPTIONS = {
-    VERSION: '0.1.8',
+var DOCUMENTATION_OPTIONS = {
+    URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'),
+    VERSION: '0.1.9',
     LANGUAGE: 'en',
     COLLAPSE_INDEX: false,
     BUILDER: 'html',

docs/build/html/_static/language_data.js

@@ -5,7 +5,7 @@
  * This script contains the language-specific data used by searchtools.js,
  * namely the list of stopwords, stemmer, scorer and splitter.
  *
- * :copyright: Copyright 2007-2024 by the Sphinx team, see AUTHORS.
+ * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS.
  * :license: BSD, see LICENSE for details.
  *
  */

docs/build/html/_static/searchtools.js

@@ -4,7 +4,7 @@
  *
  * Sphinx JavaScript utilities for the full-text search.
  *
- * :copyright: Copyright 2007-2024 by the Sphinx team, see AUTHORS.
+ * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS.
  * :license: BSD, see LICENSE for details.
  *
  */
@@ -57,12 +57,12 @@ const _removeChildren = (element) => {
 const _escapeRegExp = (string) =>
   string.replace(/[.*+\-?^${}()|[\]\\]/g, "\\$&"); // $& means the whole matched string
 
-const _displayItem = (item, searchTerms, highlightTerms) => {
+const _displayItem = (item, searchTerms) => {
   const docBuilder = DOCUMENTATION_OPTIONS.BUILDER;
+  const docUrlRoot = DOCUMENTATION_OPTIONS.URL_ROOT;
   const docFileSuffix = DOCUMENTATION_OPTIONS.FILE_SUFFIX;
   const docLinkSuffix = DOCUMENTATION_OPTIONS.LINK_SUFFIX;
   const showSearchSummary = DOCUMENTATION_OPTIONS.SHOW_SEARCH_SUMMARY;
-  const contentRoot = document.documentElement.dataset.content_root;
 
   const [docName, title, anchor, descr, score, _filename] = item;
 
@@ -75,24 +75,20 @@ const _displayItem = (item, searchTerms, highlightTerms) => {
     if (dirname.match(/\/index\/$/))
       dirname = dirname.substring(0, dirname.length - 6);
     else if (dirname === "index/") dirname = "";
-    requestUrl = contentRoot + dirname;
+    requestUrl = docUrlRoot + dirname;
     linkUrl = requestUrl;
   } else {
     // normal html builders
-    requestUrl = contentRoot + docName + docFileSuffix;
+    requestUrl = docUrlRoot + docName + docFileSuffix;
     linkUrl = docName + docLinkSuffix;
   }
   let linkEl = listItem.appendChild(document.createElement("a"));
   linkEl.href = linkUrl + anchor;
   linkEl.dataset.score = score;
   linkEl.innerHTML = title;
-  if (descr) {
+  if (descr)
     listItem.appendChild(document.createElement("span")).innerHTML =
       " (" + descr + ")";
-    // highlight search terms in the description
-    if (SPHINX_HIGHLIGHT_ENABLED)  // set in sphinx_highlight.js
-      highlightTerms.forEach((term) => _highlightText(listItem, term, "highlighted"));
-  }
   else if (showSearchSummary)
     fetch(requestUrl)
       .then((responseData) => responseData.text())
@@ -101,9 +97,6 @@ const _displayItem = (item, searchTerms, highlightTerms) => {
           listItem.appendChild(
             Search.makeSearchSummary(data, searchTerms)
           );
-        // highlight search terms in the summary
-        if (SPHINX_HIGHLIGHT_ENABLED)  // set in sphinx_highlight.js
-          highlightTerms.forEach((term) => _highlightText(listItem, term, "highlighted"));
       });
   Search.output.appendChild(listItem);
 };
@@ -122,15 +115,14 @@ const _finishSearch = (resultCount) => {
 const _displayNextItem = (
   results,
   resultCount,
-  searchTerms,
-  highlightTerms,
+  searchTerms
 ) => {
   // results left, load the summary and display it
   // this is intended to be dynamic (don't sub resultsCount)
   if (results.length) {
-    _displayItem(results.pop(), searchTerms, highlightTerms);
+    _displayItem(results.pop(), searchTerms);
     setTimeout(
-      () => _displayNextItem(results, resultCount, searchTerms, highlightTerms),
+      () => _displayNextItem(results, resultCount, searchTerms),
       5
     );
   }
@@ -164,7 +156,7 @@ const Search = {
     const htmlElement = new DOMParser().parseFromString(htmlString, 'text/html');
     htmlElement.querySelectorAll(".headerlink").forEach((el) => { el.remove() });
     const docContent = htmlElement.querySelector('[role="main"]');
-    if (docContent) return docContent.textContent;
+    if (docContent !== undefined) return docContent.textContent;
     console.warn(
       "Content block not found. Sphinx search tries to obtain it via '[role=main]'. Could you check your theme or template."
     );
@@ -288,9 +280,9 @@ const Search = {
     let results = [];
     _removeChildren(document.getElementById("search-progress"));
 
-    const queryLower = query.toLowerCase().trim();
+    const queryLower = query.toLowerCase();
     for (const [title, foundTitles] of Object.entries(allTitles)) {
-      if (title.toLowerCase().trim().includes(queryLower) && (queryLower.length >= title.length/2)) {
+      if (title.toLowerCase().includes(queryLower) && (queryLower.length >= title.length/2)) {
         for (const [file, id] of foundTitles) {
           let score = Math.round(100 * queryLower.length / title.length)
           results.push([
@@ -368,7 +360,7 @@ const Search = {
     // console.info("search results:", Search.lastresults);
 
     // print the results
-    _displayNextItem(results, results.length, searchTerms, highlightTerms);
+    _displayNextItem(results, results.length, searchTerms);
   },
 
   /**

docs/build/html/genindex.html

@@ -1,22 +1,23 @@
 <!DOCTYPE html>
-<html class="writer-html5" lang="en" data-content_root="./">
+<html class="writer-html5" lang="en">
 <head>
   <meta charset="utf-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>Index &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.8 documentation</title>
-      <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=92fd9be5" />
-      <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+  <title>Index &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.9 documentation</title>
+      <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
+      <link rel="stylesheet" type="text/css" href="_static/css/theme.css" />
 
   
   <!--[if lt IE 9]>
     <script src="_static/js/html5shiv.min.js"></script>
   <![endif]-->
   
-        <script src="_static/jquery.js?v=5d32c60e"></script>
-        <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
-        <script src="_static/documentation_options.js?v=22607128"></script>
-        <script src="_static/doctools.js?v=9a2dae69"></script>
-        <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
+        <script src="_static/jquery.js"></script>
+        <script src="_static/underscore.js"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
+        <script src="_static/doctools.js"></script>
+        <script src="_static/sphinx_highlight.js"></script>
     <script src="_static/js/theme.js"></script>
     <link rel="index" title="Index" href="#" />
     <link rel="search" title="Search" href="search.html" /> 
@@ -115,8 +116,6 @@
       <li><a href="quapy.html#quapy.error.acce">acce() (in module quapy.error)</a>
 </li>
       <li><a href="quapy.data.html#quapy.data.preprocessing.IndexTransformer.add_word">add_word() (quapy.data.preprocessing.IndexTransformer method)</a>
-</li>
-      <li><a href="quapy.html#quapy.functional.adjusted_quantification">adjusted_quantification() (in module quapy.functional)</a>
 </li>
       <li><a href="quapy.method.html#quapy.method.aggregative.AdjustedClassifyAndCount">AdjustedClassifyAndCount (in module quapy.method.aggregative)</a>
 </li>
@@ -136,6 +135,8 @@
         <li><a href="quapy.method.html#quapy.method.aggregative.ACC.aggregate">(quapy.method.aggregative.ACC method)</a>
 </li>
         <li><a href="quapy.method.html#quapy.method.aggregative.AggregativeQuantifier.aggregate">(quapy.method.aggregative.AggregativeQuantifier method)</a>
+</li>
+        <li><a href="quapy.method.html#quapy.method.aggregative.BayesianCC.aggregate">(quapy.method.aggregative.BayesianCC method)</a>
 </li>
         <li><a href="quapy.method.html#quapy.method.aggregative.CC.aggregate">(quapy.method.aggregative.CC method)</a>
 </li>
@@ -174,6 +175,8 @@
         <li><a href="quapy.method.html#quapy.method.aggregative.ACC.aggregation_fit">(quapy.method.aggregative.ACC method)</a>
 </li>
         <li><a href="quapy.method.html#quapy.method.aggregative.AggregativeQuantifier.aggregation_fit">(quapy.method.aggregative.AggregativeQuantifier method)</a>
+</li>
+        <li><a href="quapy.method.html#quapy.method.aggregative.BayesianCC.aggregation_fit">(quapy.method.aggregative.BayesianCC method)</a>
 </li>
         <li><a href="quapy.method.html#quapy.method.aggregative.CC.aggregation_fit">(quapy.method.aggregative.CC method)</a>
 </li>
@@ -221,6 +224,8 @@
       <li><a href="quapy.method.html#quapy.method._kdey.KDEBase.BANDWIDTH_METHOD">BANDWIDTH_METHOD (quapy.method._kdey.KDEBase attribute)</a>
 </li>
       <li><a href="quapy.method.html#quapy.method.base.BaseQuantifier">BaseQuantifier (class in quapy.method.base)</a>
+</li>
+      <li><a href="quapy.method.html#quapy.method.aggregative.BayesianCC">BayesianCC (class in quapy.method.aggregative)</a>
 </li>
       <li><a href="quapy.classification.html#quapy.classification.calibration.BCTSCalibration">BCTSCalibration (class in quapy.classification.calibration)</a>
 </li>
@@ -284,11 +289,13 @@
       </ul></li>
       <li><a href="quapy.method.html#quapy.method.aggregative.ClassifyAndCount">ClassifyAndCount (in module quapy.method.aggregative)</a>
 </li>
-  </ul></td>
-  <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="quapy.method.html#quapy.method._neural.QuaNetTrainer.clean_checkpoint">clean_checkpoint() (quapy.method._neural.QuaNetTrainer method)</a>
 </li>
       <li><a href="quapy.method.html#quapy.method._neural.QuaNetTrainer.clean_checkpoint_dir">clean_checkpoint_dir() (quapy.method._neural.QuaNetTrainer method)</a>
+</li>
+  </ul></td>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="quapy.html#quapy.functional.clip">clip() (in module quapy.functional)</a>
 </li>
       <li><a href="quapy.classification.html#quapy.classification.neural.CNNnet">CNNnet (class in quapy.classification.neural)</a>
 </li>
@@ -306,9 +313,13 @@
         <li><a href="quapy.method.html#quapy.method._threshold_optim.X.condition">(quapy.method._threshold_optim.X method)</a>
 </li>
       </ul></li>
+      <li><a href="quapy.html#quapy.functional.condsoftmax">condsoftmax() (in module quapy.functional)</a>
+</li>
       <li><a href="quapy.html#quapy.model_selection.ConfigStatus">ConfigStatus (class in quapy.model_selection)</a>
 </li>
       <li><a href="quapy.data.html#quapy.data.base.LabelledCollection.counts">counts() (quapy.data.base.LabelledCollection method)</a>
+</li>
+      <li><a href="quapy.html#quapy.functional.counts_from_labels">counts_from_labels() (in module quapy.functional)</a>
 </li>
       <li><a href="quapy.html#quapy.util.create_if_not_exist">create_if_not_exist() (in module quapy.util)</a>
 </li>
@@ -472,6 +483,8 @@
         <li><a href="quapy.method.html#quapy.method.non_aggregative.DMx.fit">(quapy.method.non_aggregative.DMx method)</a>
 </li>
         <li><a href="quapy.method.html#quapy.method.non_aggregative.MaximumLikelihoodPrevalenceEstimation.fit">(quapy.method.non_aggregative.MaximumLikelihoodPrevalenceEstimation method)</a>
+</li>
+        <li><a href="quapy.method.html#quapy.method.non_aggregative.ReadMe.fit">(quapy.method.non_aggregative.ReadMe method)</a>
 </li>
         <li><a href="quapy.html#quapy.model_selection.GridSearchQ.fit">(quapy.model_selection.GridSearchQ method)</a>
 </li>
@@ -505,6 +518,8 @@
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="quapy.html#quapy.protocol.OnLabelledCollectionProtocol.get_collator">get_collator() (quapy.protocol.OnLabelledCollectionProtocol class method)</a>
+</li>
+      <li><a href="quapy.method.html#quapy.method.aggregative.BayesianCC.get_conditional_probability_samples">get_conditional_probability_samples() (quapy.method.aggregative.BayesianCC method)</a>
 </li>
       <li><a href="quapy.html#quapy.functional.get_divergence">get_divergence() (in module quapy.functional)</a>
 </li>
@@ -542,6 +557,8 @@
       </ul></li>
   </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="quapy.method.html#quapy.method.aggregative.BayesianCC.get_prevalence_samples">get_prevalence_samples() (quapy.method.aggregative.BayesianCC method)</a>
+</li>
       <li><a href="quapy.method.html#quapy.method.meta.get_probability_distribution">get_probability_distribution() (in module quapy.method.meta)</a>
 </li>
       <li><a href="quapy.html#quapy.util.get_quapy_home">get_quapy_home() (in module quapy.util)</a>
@@ -628,18 +645,20 @@
 <h2 id="L">L</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="quapy.html#quapy.functional.l1_norm">l1_norm() (in module quapy.functional)</a>
+</li>
       <li><a href="quapy.data.html#quapy.data.base.LabelledCollection">LabelledCollection (class in quapy.data.base)</a>
 </li>
       <li><a href="quapy.html#quapy.functional.linear_search">linear_search() (in module quapy.functional)</a>
 </li>
+  </ul></td>
+  <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="quapy.data.html#quapy.data.base.Dataset.load">load() (quapy.data.base.Dataset class method)</a>
 
       <ul>
         <li><a href="quapy.data.html#quapy.data.base.LabelledCollection.load">(quapy.data.base.LabelledCollection class method)</a>
 </li>
       </ul></li>
-  </ul></td>
-  <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="quapy.classification.html#quapy.classification.methods.LowRankLogisticRegression">LowRankLogisticRegression (class in quapy.classification.methods)</a>
 </li>
       <li><a href="quapy.classification.html#quapy.classification.neural.LSTMnet">LSTMnet (class in quapy.classification.neural)</a>
@@ -673,6 +692,8 @@
       <li><a href="quapy.method.html#quapy.method.meta.MedianEstimator">MedianEstimator (class in quapy.method.meta)</a>
 </li>
       <li><a href="quapy.method.html#quapy.method.meta.MedianEstimator2">MedianEstimator2 (class in quapy.method.meta)</a>
+</li>
+      <li><a href="quapy.method.html#quapy.method.aggregative.ACC.METHODS">METHODS (quapy.method.aggregative.ACC attribute)</a>
 </li>
       <li><a href="quapy.html#quapy.error.mkld">mkld() (in module quapy.error)</a>
 </li>
@@ -686,7 +707,7 @@
     module
 
       <ul>
-        <li><a href="generated/quapy.html#module-quapy">quapy</a>, <a href="quapy.html#module-quapy">[1]</a>
+        <li><a href="quapy.html#module-quapy">quapy</a>
 </li>
         <li><a href="quapy.classification.html#module-quapy.classification">quapy.classification</a>
 </li>
@@ -772,6 +793,8 @@
       <li><a href="quapy.classification.html#quapy.classification.neural.NeuralClassifierTrainer">NeuralClassifierTrainer (class in quapy.classification.neural)</a>
 </li>
       <li><a href="quapy.method.html#quapy.method.aggregative.newELM">newELM() (in module quapy.method.aggregative)</a>
+</li>
+      <li><a href="quapy.method.html#quapy.method.aggregative.ACC.newInvariantRatioEstimation">newInvariantRatioEstimation() (quapy.method.aggregative.ACC class method)</a>
 </li>
       <li><a href="quapy.method.html#quapy.method.base.newOneVsAll">newOneVsAll() (in module quapy.method.base)</a>
 </li>
@@ -786,6 +809,8 @@
       <li><a href="quapy.method.html#quapy.method.aggregative.newSVMRAE">newSVMRAE() (in module quapy.method.aggregative)</a>
 </li>
       <li><a href="quapy.html#quapy.error.nkld">nkld() (in module quapy.error)</a>
+</li>
+      <li><a href="quapy.method.html#quapy.method.aggregative.ACC.NORMALIZATIONS">NORMALIZATIONS (quapy.method.aggregative.ACC attribute)</a>
 </li>
       <li><a href="quapy.html#quapy.functional.normalize_prevalence">normalize_prevalence() (in module quapy.functional)</a>
 </li>
@@ -830,6 +855,8 @@
       <li><a href="quapy.method.html#quapy.method.aggregative.PACC">PACC (class in quapy.method.aggregative)</a>
 </li>
       <li><a href="quapy.html#quapy.util.parallel">parallel() (in module quapy.util)</a>
+</li>
+      <li><a href="quapy.html#quapy.util.parallel_unpack">parallel_unpack() (in module quapy.util)</a>
 </li>
       <li><a href="quapy.method.html#quapy.method.aggregative.PCC">PCC (class in quapy.method.aggregative)</a>
 </li>
@@ -880,6 +907,8 @@
       <li><a href="quapy.method.html#quapy.method.aggregative.ProbabilisticAdjustedClassifyAndCount">ProbabilisticAdjustedClassifyAndCount (in module quapy.method.aggregative)</a>
 </li>
       <li><a href="quapy.method.html#quapy.method.aggregative.ProbabilisticClassifyAndCount">ProbabilisticClassifyAndCount (in module quapy.method.aggregative)</a>
+</li>
+      <li><a href="quapy.html#quapy.functional.projection_simplex_sort">projection_simplex_sort() (in module quapy.functional)</a>
 </li>
   </ul></td>
 </tr></table>
@@ -911,6 +940,8 @@
         <li><a href="quapy.method.html#quapy.method.non_aggregative.DMx.quantify">(quapy.method.non_aggregative.DMx method)</a>
 </li>
         <li><a href="quapy.method.html#quapy.method.non_aggregative.MaximumLikelihoodPrevalenceEstimation.quantify">(quapy.method.non_aggregative.MaximumLikelihoodPrevalenceEstimation method)</a>
+</li>
+        <li><a href="quapy.method.html#quapy.method.non_aggregative.ReadMe.quantify">(quapy.method.non_aggregative.ReadMe method)</a>
 </li>
         <li><a href="quapy.html#quapy.model_selection.GridSearchQ.quantify">(quapy.model_selection.GridSearchQ method)</a>
 </li>
@@ -919,7 +950,7 @@
     quapy
 
       <ul>
-        <li><a href="generated/quapy.html#module-quapy">module</a>, <a href="quapy.html#module-quapy">[1]</a>
+        <li><a href="quapy.html#module-quapy">module</a>
 </li>
       </ul></li>
       <li>
@@ -1108,15 +1139,17 @@
       <li><a href="quapy.html#quapy.error.rae">rae() (in module quapy.error)</a>
 </li>
       <li><a href="quapy.html#quapy.protocol.AbstractStochasticSeededProtocol.random_state">random_state (quapy.protocol.AbstractStochasticSeededProtocol property)</a>
+</li>
+      <li><a href="quapy.method.html#quapy.method.non_aggregative.ReadMe">ReadMe (class in quapy.method.non_aggregative)</a>
 </li>
       <li><a href="quapy.classification.html#quapy.classification.calibration.RecalibratedProbabilisticClassifier">RecalibratedProbabilisticClassifier (class in quapy.classification.calibration)</a>
 </li>
       <li><a href="quapy.classification.html#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase">RecalibratedProbabilisticClassifierBase (class in quapy.classification.calibration)</a>
-</li>
-      <li><a href="quapy.data.html#quapy.data.base.Dataset.reduce">reduce() (quapy.data.base.Dataset method)</a>
 </li>
   </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="quapy.data.html#quapy.data.base.Dataset.reduce">reduce() (quapy.data.base.Dataset method)</a>
+</li>
       <li><a href="quapy.data.html#quapy.data.preprocessing.reduce_columns">reduce_columns() (in module quapy.data.preprocessing)</a>
 </li>
       <li><a href="quapy.data.html#quapy.data.reader.reindex_labels">reindex_labels() (in module quapy.data.reader)</a>
@@ -1145,6 +1178,8 @@
         <li><a href="quapy.html#quapy.protocol.UPP.sample">(quapy.protocol.UPP method)</a>
 </li>
       </ul></li>
+      <li><a href="quapy.method.html#quapy.method.aggregative.BayesianCC.sample_from_posterior">sample_from_posterior() (quapy.method.aggregative.BayesianCC method)</a>
+</li>
       <li><a href="quapy.html#quapy.protocol.AbstractStochasticSeededProtocol.samples_parameters">samples_parameters() (quapy.protocol.AbstractStochasticSeededProtocol method)</a>
 
       <ul>
@@ -1193,7 +1228,13 @@
 </li>
       <li><a href="quapy.html#quapy.error.smooth">smooth() (in module quapy.error)</a>
 </li>
-      <li><a href="quapy.method.html#quapy.method.aggregative.ACC.solve_adjustment">solve_adjustment() (quapy.method.aggregative.ACC class method)</a>
+      <li><a href="quapy.html#quapy.functional.softmax">softmax() (in module quapy.functional)</a>
+</li>
+      <li><a href="quapy.html#quapy.functional.solve_adjustment">solve_adjustment() (in module quapy.functional)</a>
+</li>
+      <li><a href="quapy.html#quapy.functional.solve_adjustment_binary">solve_adjustment_binary() (in module quapy.functional)</a>
+</li>
+      <li><a href="quapy.method.html#quapy.method.aggregative.ACC.SOLVERS">SOLVERS (quapy.method.aggregative.ACC attribute)</a>
 </li>
       <li><a href="quapy.data.html#quapy.data.base.LabelledCollection.split_random">split_random() (quapy.data.base.LabelledCollection method)</a>
 </li>
@@ -1210,6 +1251,8 @@
 </li>
       </ul></li>
       <li><a href="quapy.html#quapy.model_selection.Status">Status (class in quapy.model_selection)</a>
+</li>
+      <li><a href="quapy.method.html#quapy.method.non_aggregative.ReadMe.std_constrained_linear_ls">std_constrained_linear_ls() (quapy.method.non_aggregative.ReadMe method)</a>
 </li>
       <li><a href="quapy.html#quapy.functional.strprev">strprev() (in module quapy.functional)</a>
 </li>
@@ -1228,6 +1271,8 @@
       <li><a href="quapy.method.html#quapy.method._threshold_optim.T50">T50 (class in quapy.method._threshold_optim)</a>
 </li>
       <li><a href="quapy.html#quapy.util.temp_seed">temp_seed() (in module quapy.util)</a>
+</li>
+      <li><a href="quapy.html#quapy.functional.ternary_search">ternary_search() (in module quapy.functional)</a>
 </li>
       <li><a href="quapy.data.html#quapy.data.preprocessing.text2tfidf">text2tfidf() (in module quapy.data.preprocessing)</a>
 </li>
@@ -1261,6 +1306,16 @@
   <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="quapy.data.html#quapy.data.base.Dataset.train_test">train_test (quapy.data.base.Dataset property)</a>
 </li>
+      <li><a href="quapy.classification.html#quapy.classification.neural.CNNnet.training">training (quapy.classification.neural.CNNnet attribute)</a>
+
+      <ul>
+        <li><a href="quapy.classification.html#quapy.classification.neural.LSTMnet.training">(quapy.classification.neural.LSTMnet attribute)</a>
+</li>
+        <li><a href="quapy.classification.html#quapy.classification.neural.TextClassifierNet.training">(quapy.classification.neural.TextClassifierNet attribute)</a>
+</li>
+        <li><a href="quapy.method.html#quapy.method._neural.QuaNetModule.training">(quapy.method._neural.QuaNetModule attribute)</a>
+</li>
+      </ul></li>
       <li><a href="quapy.classification.html#quapy.classification.methods.LowRankLogisticRegression.transform">transform() (quapy.classification.methods.LowRankLogisticRegression method)</a>
 
       <ul>

docs/build/html/index.html

@@ -1,24 +1,24 @@
 <!DOCTYPE html>
-<html class="writer-html5" lang="en" data-content_root="./">
+<html class="writer-html5" lang="en">
 <head>
   <meta charset="utf-8" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>Welcome to QuaPy’s documentation! &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.8 documentation</title>
-      <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=92fd9be5" />
-      <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+  <title>Welcome to QuaPy’s documentation! &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.9 documentation</title>
+      <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
+      <link rel="stylesheet" type="text/css" href="_static/css/theme.css" />
 
   
   <!--[if lt IE 9]>
     <script src="_static/js/html5shiv.min.js"></script>
   <![endif]-->
   
-        <script src="_static/jquery.js?v=5d32c60e"></script>
-        <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
-        <script src="_static/documentation_options.js?v=22607128"></script>
-        <script src="_static/doctools.js?v=9a2dae69"></script>
-        <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
-        <script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
+        <script src="_static/jquery.js"></script>
+        <script src="_static/underscore.js"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
+        <script src="_static/doctools.js"></script>
+        <script src="_static/sphinx_highlight.js"></script>
     <script src="_static/js/theme.js"></script>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
@@ -73,21 +73,21 @@
            <div itemprop="articleBody">
              
   <section id="welcome-to-quapy-s-documentation">
-<h1>Welcome to QuaPy’s documentation!<a class="headerlink" href="#welcome-to-quapy-s-documentation" title="Link to this heading"></a></h1>
+<h1>Welcome to QuaPy’s documentation!<a class="headerlink" href="#welcome-to-quapy-s-documentation" title="Permalink to this heading"></a></h1>
 <p>QuaPy is a Python-based open-source framework for quantification.</p>
 <p>This document contains the API of the modules included in QuaPy.</p>
 <section id="installation">
-<h2>Installation<a class="headerlink" href="#installation" title="Link to this heading"></a></h2>
+<h2>Installation<a class="headerlink" href="#installation" title="Permalink to this heading"></a></h2>
 <p><cite>pip install quapy</cite></p>
 </section>
 <section id="github">
-<h2>GitHub<a class="headerlink" href="#github" title="Link to this heading"></a></h2>
+<h2>GitHub<a class="headerlink" href="#github" title="Permalink to this heading"></a></h2>
 <p>QuaPy is hosted in GitHub at <a class="reference external" href="https://github.com/HLT-ISTI/QuaPy">https://github.com/HLT-ISTI/QuaPy</a></p>
 <div class="toctree-wrapper compound">
 </div>
 </section>
 <section id="contents">
-<h2>Contents<a class="headerlink" href="#contents" title="Link to this heading"></a></h2>
+<h2>Contents<a class="headerlink" href="#contents" title="Permalink to this heading"></a></h2>
 <div class="toctree-wrapper compound">
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="modules.html">quapy</a><ul>
@@ -128,12 +128,14 @@
 <li class="toctree-l6"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.CNNnet"><code class="docutils literal notranslate"><span class="pre">CNNnet</span></code></a><ul>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.CNNnet.document_embedding"><code class="docutils literal notranslate"><span class="pre">CNNnet.document_embedding()</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.CNNnet.get_params"><code class="docutils literal notranslate"><span class="pre">CNNnet.get_params()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.CNNnet.training"><code class="docutils literal notranslate"><span class="pre">CNNnet.training</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.CNNnet.vocabulary_size"><code class="docutils literal notranslate"><span class="pre">CNNnet.vocabulary_size</span></code></a></li>
 </ul>
 </li>
 <li class="toctree-l6"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.LSTMnet"><code class="docutils literal notranslate"><span class="pre">LSTMnet</span></code></a><ul>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.LSTMnet.document_embedding"><code class="docutils literal notranslate"><span class="pre">LSTMnet.document_embedding()</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.LSTMnet.get_params"><code class="docutils literal notranslate"><span class="pre">LSTMnet.get_params()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.LSTMnet.training"><code class="docutils literal notranslate"><span class="pre">LSTMnet.training</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.LSTMnet.vocabulary_size"><code class="docutils literal notranslate"><span class="pre">LSTMnet.vocabulary_size</span></code></a></li>
 </ul>
 </li>
@@ -154,6 +156,7 @@
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.TextClassifierNet.forward"><code class="docutils literal notranslate"><span class="pre">TextClassifierNet.forward()</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.TextClassifierNet.get_params"><code class="docutils literal notranslate"><span class="pre">TextClassifierNet.get_params()</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.TextClassifierNet.predict_proba"><code class="docutils literal notranslate"><span class="pre">TextClassifierNet.predict_proba()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.TextClassifierNet.training"><code class="docutils literal notranslate"><span class="pre">TextClassifierNet.training</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.TextClassifierNet.vocabulary_size"><code class="docutils literal notranslate"><span class="pre">TextClassifierNet.vocabulary_size</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.classification.html#quapy.classification.neural.TextClassifierNet.xavier_uniform"><code class="docutils literal notranslate"><span class="pre">TextClassifierNet.xavier_uniform()</span></code></a></li>
 </ul>
@@ -260,10 +263,13 @@
 <li class="toctree-l5"><a class="reference internal" href="quapy.method.html#submodules">Submodules</a></li>
 <li class="toctree-l5"><a class="reference internal" href="quapy.method.html#module-quapy.method.aggregative">quapy.method.aggregative module</a><ul>
 <li class="toctree-l6"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC"><code class="docutils literal notranslate"><span class="pre">ACC</span></code></a><ul>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC.METHODS"><code class="docutils literal notranslate"><span class="pre">ACC.METHODS</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC.NORMALIZATIONS"><code class="docutils literal notranslate"><span class="pre">ACC.NORMALIZATIONS</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC.SOLVERS"><code class="docutils literal notranslate"><span class="pre">ACC.SOLVERS</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC.aggregate"><code class="docutils literal notranslate"><span class="pre">ACC.aggregate()</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC.aggregation_fit"><code class="docutils literal notranslate"><span class="pre">ACC.aggregation_fit()</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC.getPteCondEstim"><code class="docutils literal notranslate"><span class="pre">ACC.getPteCondEstim()</span></code></a></li>
-<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC.solve_adjustment"><code class="docutils literal notranslate"><span class="pre">ACC.solve_adjustment()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.ACC.newInvariantRatioEstimation"><code class="docutils literal notranslate"><span class="pre">ACC.newInvariantRatioEstimation()</span></code></a></li>
 </ul>
 </li>
 <li class="toctree-l6"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.AdjustedClassifyAndCount"><code class="docutils literal notranslate"><span class="pre">AdjustedClassifyAndCount</span></code></a></li>
@@ -289,6 +295,14 @@
 </ul>
 </li>
 <li class="toctree-l6"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.AggregativeSoftQuantifier"><code class="docutils literal notranslate"><span class="pre">AggregativeSoftQuantifier</span></code></a></li>
+<li class="toctree-l6"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BayesianCC"><code class="docutils literal notranslate"><span class="pre">BayesianCC</span></code></a><ul>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BayesianCC.aggregate"><code class="docutils literal notranslate"><span class="pre">BayesianCC.aggregate()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BayesianCC.aggregation_fit"><code class="docutils literal notranslate"><span class="pre">BayesianCC.aggregation_fit()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BayesianCC.get_conditional_probability_samples"><code class="docutils literal notranslate"><span class="pre">BayesianCC.get_conditional_probability_samples()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BayesianCC.get_prevalence_samples"><code class="docutils literal notranslate"><span class="pre">BayesianCC.get_prevalence_samples()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BayesianCC.sample_from_posterior"><code class="docutils literal notranslate"><span class="pre">BayesianCC.sample_from_posterior()</span></code></a></li>
+</ul>
+</li>
 <li class="toctree-l6"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BinaryAggregativeQuantifier"><code class="docutils literal notranslate"><span class="pre">BinaryAggregativeQuantifier</span></code></a><ul>
 <li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BinaryAggregativeQuantifier.fit"><code class="docutils literal notranslate"><span class="pre">BinaryAggregativeQuantifier.fit()</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.aggregative.BinaryAggregativeQuantifier.neg_label"><code class="docutils literal notranslate"><span class="pre">BinaryAggregativeQuantifier.neg_label</span></code></a></li>
@@ -385,6 +399,7 @@
 <li class="toctree-l6"><a class="reference internal" href="quapy.method.html#quapy.method._neural.QuaNetModule"><code class="docutils literal notranslate"><span class="pre">QuaNetModule</span></code></a><ul>
 <li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method._neural.QuaNetModule.device"><code class="docutils literal notranslate"><span class="pre">QuaNetModule.device</span></code></a></li>
 <li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method._neural.QuaNetModule.forward"><code class="docutils literal notranslate"><span class="pre">QuaNetModule.forward()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method._neural.QuaNetModule.training"><code class="docutils literal notranslate"><span class="pre">QuaNetModule.training</span></code></a></li>
 </ul>
 </li>
 <li class="toctree-l6"><a class="reference internal" href="quapy.method.html#quapy.method._neural.QuaNetTrainer"><code class="docutils literal notranslate"><span class="pre">QuaNetTrainer</span></code></a><ul>
@@ -494,6 +509,12 @@
 <li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.non_aggregative.MaximumLikelihoodPrevalenceEstimation.quantify"><code class="docutils literal notranslate"><span class="pre">MaximumLikelihoodPrevalenceEstimation.quantify()</span></code></a></li>
 </ul>
 </li>
+<li class="toctree-l6"><a class="reference internal" href="quapy.method.html#quapy.method.non_aggregative.ReadMe"><code class="docutils literal notranslate"><span class="pre">ReadMe</span></code></a><ul>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.non_aggregative.ReadMe.fit"><code class="docutils literal notranslate"><span class="pre">ReadMe.fit()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.non_aggregative.ReadMe.quantify"><code class="docutils literal notranslate"><span class="pre">ReadMe.quantify()</span></code></a></li>
+<li class="toctree-l7"><a class="reference internal" href="quapy.method.html#quapy.method.non_aggregative.ReadMe.std_constrained_linear_ls"><code class="docutils literal notranslate"><span class="pre">ReadMe.std_constrained_linear_ls()</span></code></a></li>
+</ul>
+</li>
 </ul>
 </li>
 <li class="toctree-l5"><a class="reference internal" href="quapy.method.html#module-quapy.method">Module contents</a></li>
@@ -543,12 +564,15 @@
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#module-quapy.functional">quapy.functional module</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.HellingerDistance"><code class="docutils literal notranslate"><span class="pre">HellingerDistance()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.TopsoeDistance"><code class="docutils literal notranslate"><span class="pre">TopsoeDistance()</span></code></a></li>
-<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.adjusted_quantification"><code class="docutils literal notranslate"><span class="pre">adjusted_quantification()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.argmin_prevalence"><code class="docutils literal notranslate"><span class="pre">argmin_prevalence()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.as_binary_prevalence"><code class="docutils literal notranslate"><span class="pre">as_binary_prevalence()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.check_prevalence_vector"><code class="docutils literal notranslate"><span class="pre">check_prevalence_vector()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.clip"><code class="docutils literal notranslate"><span class="pre">clip()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.condsoftmax"><code class="docutils literal notranslate"><span class="pre">condsoftmax()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.counts_from_labels"><code class="docutils literal notranslate"><span class="pre">counts_from_labels()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.get_divergence"><code class="docutils literal notranslate"><span class="pre">get_divergence()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.get_nprevpoints_approximation"><code class="docutils literal notranslate"><span class="pre">get_nprevpoints_approximation()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.l1_norm"><code class="docutils literal notranslate"><span class="pre">l1_norm()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.linear_search"><code class="docutils literal notranslate"><span class="pre">linear_search()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.normalize_prevalence"><code class="docutils literal notranslate"><span class="pre">normalize_prevalence()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.num_prevalence_combinations"><code class="docutils literal notranslate"><span class="pre">num_prevalence_combinations()</span></code></a></li>
@@ -556,7 +580,12 @@
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.prevalence_from_labels"><code class="docutils literal notranslate"><span class="pre">prevalence_from_labels()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.prevalence_from_probabilities"><code class="docutils literal notranslate"><span class="pre">prevalence_from_probabilities()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.prevalence_linspace"><code class="docutils literal notranslate"><span class="pre">prevalence_linspace()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.projection_simplex_sort"><code class="docutils literal notranslate"><span class="pre">projection_simplex_sort()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.softmax"><code class="docutils literal notranslate"><span class="pre">softmax()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.solve_adjustment"><code class="docutils literal notranslate"><span class="pre">solve_adjustment()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.solve_adjustment_binary"><code class="docutils literal notranslate"><span class="pre">solve_adjustment_binary()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.strprev"><code class="docutils literal notranslate"><span class="pre">strprev()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.ternary_search"><code class="docutils literal notranslate"><span class="pre">ternary_search()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.uniform_prevalence_sampling"><code class="docutils literal notranslate"><span class="pre">uniform_prevalence_sampling()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.functional.uniform_simplex_sampling"><code class="docutils literal notranslate"><span class="pre">uniform_simplex_sampling()</span></code></a></li>
 </ul>
@@ -657,6 +686,7 @@
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.util.get_quapy_home"><code class="docutils literal notranslate"><span class="pre">get_quapy_home()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.util.map_parallel"><code class="docutils literal notranslate"><span class="pre">map_parallel()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.util.parallel"><code class="docutils literal notranslate"><span class="pre">parallel()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.util.parallel_unpack"><code class="docutils literal notranslate"><span class="pre">parallel_unpack()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.util.pickled_resource"><code class="docutils literal notranslate"><span class="pre">pickled_resource()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.util.save_text_file"><code class="docutils literal notranslate"><span class="pre">save_text_file()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="quapy.html#quapy.util.temp_seed"><code class="docutils literal notranslate"><span class="pre">temp_seed()</span></code></a></li>
@@ -673,7 +703,7 @@
 </section>
 </section>
 <section id="indices-and-tables">
-<h1>Indices and tables<a class="headerlink" href="#indices-and-tables" title="Link to this heading"></a></h1>
+<h1>Indices and tables<a class="headerlink" href="#indices-and-tables" title="Permalink to this heading"></a></h1>
 <ul class="simple">
 <li><p><a class="reference internal" href="genindex.html"><span class="std std-ref">Index</span></a></p></li>
 <li><p><a class="reference internal" href="py-modindex.html"><span class="std std-ref">Module Index</span></a></p></li>

docs/build/html/modules.html

@@ -1,24 +1,24 @@
 <!DOCTYPE html>
-<html class="writer-html5" lang="en" data-content_root="./">
+<html class="writer-html5" lang="en">
 <head>
   <meta charset="utf-8" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>quapy &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.8 documentation</title>
-      <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=92fd9be5" />
-      <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+  <title>quapy &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.9 documentation</title>
+      <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
+      <link rel="stylesheet" type="text/css" href="_static/css/theme.css" />
 
   
   <!--[if lt IE 9]>
     <script src="_static/js/html5shiv.min.js"></script>
   <![endif]-->
   
-        <script src="_static/jquery.js?v=5d32c60e"></script>
-        <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
-        <script src="_static/documentation_options.js?v=22607128"></script>
-        <script src="_static/doctools.js?v=9a2dae69"></script>
-        <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
-        <script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
+        <script src="_static/jquery.js"></script>
+        <script src="_static/underscore.js"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
+        <script src="_static/doctools.js"></script>
+        <script src="_static/sphinx_highlight.js"></script>
     <script src="_static/js/theme.js"></script>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
@@ -77,7 +77,7 @@
            <div itemprop="articleBody">
              
   <section id="quapy">
-<h1>quapy<a class="headerlink" href="#quapy" title="Link to this heading"></a></h1>
+<h1>quapy<a class="headerlink" href="#quapy" title="Permalink to this heading"></a></h1>
 <div class="toctree-wrapper compound">
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="quapy.html">quapy package</a><ul>
@@ -153,12 +153,15 @@
 <li class="toctree-l2"><a class="reference internal" href="quapy.html#module-quapy.functional">quapy.functional module</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.HellingerDistance"><code class="docutils literal notranslate"><span class="pre">HellingerDistance()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.TopsoeDistance"><code class="docutils literal notranslate"><span class="pre">TopsoeDistance()</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.adjusted_quantification"><code class="docutils literal notranslate"><span class="pre">adjusted_quantification()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.argmin_prevalence"><code class="docutils literal notranslate"><span class="pre">argmin_prevalence()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.as_binary_prevalence"><code class="docutils literal notranslate"><span class="pre">as_binary_prevalence()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.check_prevalence_vector"><code class="docutils literal notranslate"><span class="pre">check_prevalence_vector()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.clip"><code class="docutils literal notranslate"><span class="pre">clip()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.condsoftmax"><code class="docutils literal notranslate"><span class="pre">condsoftmax()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.counts_from_labels"><code class="docutils literal notranslate"><span class="pre">counts_from_labels()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.get_divergence"><code class="docutils literal notranslate"><span class="pre">get_divergence()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.get_nprevpoints_approximation"><code class="docutils literal notranslate"><span class="pre">get_nprevpoints_approximation()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.l1_norm"><code class="docutils literal notranslate"><span class="pre">l1_norm()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.linear_search"><code class="docutils literal notranslate"><span class="pre">linear_search()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.normalize_prevalence"><code class="docutils literal notranslate"><span class="pre">normalize_prevalence()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.num_prevalence_combinations"><code class="docutils literal notranslate"><span class="pre">num_prevalence_combinations()</span></code></a></li>
@@ -166,7 +169,12 @@
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.prevalence_from_labels"><code class="docutils literal notranslate"><span class="pre">prevalence_from_labels()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.prevalence_from_probabilities"><code class="docutils literal notranslate"><span class="pre">prevalence_from_probabilities()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.prevalence_linspace"><code class="docutils literal notranslate"><span class="pre">prevalence_linspace()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.projection_simplex_sort"><code class="docutils literal notranslate"><span class="pre">projection_simplex_sort()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.softmax"><code class="docutils literal notranslate"><span class="pre">softmax()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.solve_adjustment"><code class="docutils literal notranslate"><span class="pre">solve_adjustment()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.solve_adjustment_binary"><code class="docutils literal notranslate"><span class="pre">solve_adjustment_binary()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.strprev"><code class="docutils literal notranslate"><span class="pre">strprev()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.ternary_search"><code class="docutils literal notranslate"><span class="pre">ternary_search()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.uniform_prevalence_sampling"><code class="docutils literal notranslate"><span class="pre">uniform_prevalence_sampling()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.functional.uniform_simplex_sampling"><code class="docutils literal notranslate"><span class="pre">uniform_simplex_sampling()</span></code></a></li>
 </ul>
@@ -267,6 +275,7 @@
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.util.get_quapy_home"><code class="docutils literal notranslate"><span class="pre">get_quapy_home()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.util.map_parallel"><code class="docutils literal notranslate"><span class="pre">map_parallel()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.util.parallel"><code class="docutils literal notranslate"><span class="pre">parallel()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.util.parallel_unpack"><code class="docutils literal notranslate"><span class="pre">parallel_unpack()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.util.pickled_resource"><code class="docutils literal notranslate"><span class="pre">pickled_resource()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.util.save_text_file"><code class="docutils literal notranslate"><span class="pre">save_text_file()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="quapy.html#quapy.util.temp_seed"><code class="docutils literal notranslate"><span class="pre">temp_seed()</span></code></a></li>

docs/build/html/py-modindex.html

@@ -1,22 +1,23 @@
 <!DOCTYPE html>
-<html class="writer-html5" lang="en" data-content_root="./">
+<html class="writer-html5" lang="en">
 <head>
   <meta charset="utf-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>Python Module Index &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.8 documentation</title>
-      <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=92fd9be5" />
-      <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+  <title>Python Module Index &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.9 documentation</title>
+      <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
+      <link rel="stylesheet" type="text/css" href="_static/css/theme.css" />
 
   
   <!--[if lt IE 9]>
     <script src="_static/js/html5shiv.min.js"></script>
   <![endif]-->
   
-        <script src="_static/jquery.js?v=5d32c60e"></script>
-        <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
-        <script src="_static/documentation_options.js?v=22607128"></script>
-        <script src="_static/doctools.js?v=9a2dae69"></script>
-        <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
+        <script src="_static/jquery.js"></script>
+        <script src="_static/underscore.js"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
+        <script src="_static/doctools.js"></script>
+        <script src="_static/sphinx_highlight.js"></script>
     <script src="_static/js/theme.js"></script>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />

docs/build/html/quapy.classification.html

@@ -1,23 +1,24 @@
 <!DOCTYPE html>
-<html class="writer-html5" lang="en" data-content_root="./">
+<html class="writer-html5" lang="en">
 <head>
   <meta charset="utf-8" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>quapy.classification package &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.8 documentation</title>
-      <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=92fd9be5" />
-      <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+  <title>quapy.classification package &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.9 documentation</title>
+      <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
+      <link rel="stylesheet" type="text/css" href="_static/css/theme.css" />
 
   
   <!--[if lt IE 9]>
     <script src="_static/js/html5shiv.min.js"></script>
   <![endif]-->
   
-        <script src="_static/jquery.js?v=5d32c60e"></script>
-        <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
-        <script src="_static/documentation_options.js?v=22607128"></script>
-        <script src="_static/doctools.js?v=9a2dae69"></script>
-        <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
+        <script src="_static/jquery.js"></script>
+        <script src="_static/underscore.js"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
+        <script src="_static/doctools.js"></script>
+        <script src="_static/sphinx_highlight.js"></script>
     <script src="_static/js/theme.js"></script>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
@@ -95,15 +96,15 @@
            <div itemprop="articleBody">
              
   <section id="quapy-classification-package">
-<h1>quapy.classification package<a class="headerlink" href="#quapy-classification-package" title="Link to this heading"></a></h1>
+<h1>quapy.classification package<a class="headerlink" href="#quapy-classification-package" title="Permalink to this heading"></a></h1>
 <section id="submodules">
-<h2>Submodules<a class="headerlink" href="#submodules" title="Link to this heading"></a></h2>
+<h2>Submodules<a class="headerlink" href="#submodules" title="Permalink to this heading"></a></h2>
 </section>
 <section id="module-quapy.classification.calibration">
-<span id="quapy-classification-calibration-module"></span><h2>quapy.classification.calibration module<a class="headerlink" href="#module-quapy.classification.calibration" title="Link to this heading"></a></h2>
+<span id="quapy-classification-calibration-module"></span><h2>quapy.classification.calibration module<a class="headerlink" href="#module-quapy.classification.calibration" title="Permalink to this heading"></a></h2>
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.calibration.BCTSCalibration">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">BCTSCalibration</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#BCTSCalibration"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.BCTSCalibration" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">BCTSCalibration</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#BCTSCalibration"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.BCTSCalibration" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <a class="reference internal" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase" title="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase"><code class="xref py py-class docutils literal notranslate"><span class="pre">RecalibratedProbabilisticClassifierBase</span></code></a></p>
 <p>Applies the Bias-Corrected Temperature Scaling (BCTS) calibration method from <cite>abstention.calibration</cite>, as defined in
 <a class="reference external" href="http://proceedings.mlr.press/v119/alexandari20a.html">Alexandari et al. paper</a>:</p>
@@ -124,7 +125,7 @@ training set afterwards. Default value is 5.</p></li>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.calibration.NBVSCalibration">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">NBVSCalibration</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#NBVSCalibration"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.NBVSCalibration" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">NBVSCalibration</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#NBVSCalibration"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.NBVSCalibration" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <a class="reference internal" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase" title="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase"><code class="xref py py-class docutils literal notranslate"><span class="pre">RecalibratedProbabilisticClassifierBase</span></code></a></p>
 <p>Applies the No-Bias Vector Scaling (NBVS) calibration method from <cite>abstention.calibration</cite>, as defined in
 <a class="reference external" href="http://proceedings.mlr.press/v119/alexandari20a.html">Alexandari et al. paper</a>:</p>
@@ -145,7 +146,7 @@ training set afterwards. Default value is 5.</p></li>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.calibration.RecalibratedProbabilisticClassifier">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">RecalibratedProbabilisticClassifier</span></span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifier"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifier" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">RecalibratedProbabilisticClassifier</span></span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifier"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifier" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
 <p>Abstract class for (re)calibration method from <cite>abstention.calibration</cite>, as defined in
 <a class="reference external" href="http://proceedings.mlr.press/v119/alexandari20a.html">Alexandari, A., Kundaje, A., &amp; Shrikumar, A. (2020, November). Maximum likelihood with bias-corrected calibration
@@ -154,7 +155,7 @@ is hard-to-beat at label shift adaptation. In International Conference on Machin
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">RecalibratedProbabilisticClassifierBase</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">calibrator</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">RecalibratedProbabilisticClassifierBase</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">calibrator</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">BaseEstimator</span></code>, <a class="reference internal" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifier" title="quapy.classification.calibration.RecalibratedProbabilisticClassifier"><code class="xref py py-class docutils literal notranslate"><span class="pre">RecalibratedProbabilisticClassifier</span></code></a></p>
 <p>Applies a (re)calibration method from <cite>abstention.calibration</cite>, as defined in
 <a class="reference external" href="http://proceedings.mlr.press/v119/alexandari20a.html">Alexandari et al. paper</a>.</p>
@@ -174,7 +175,7 @@ training set afterwards. Default value is 5.</p></li>
 </dl>
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.classes_">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">classes_</span></span><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.classes_" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">classes_</span></span><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.classes_" title="Permalink to this definition"></a></dt>
 <dd><p>Returns the classes on which the classifier has been trained on</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -185,7 +186,7 @@ training set afterwards. Default value is 5.</p></li>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit">
-<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit" title="Permalink to this definition"></a></dt>
 <dd><p>Fits the calibration for the probabilistic classifier.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -202,7 +203,7 @@ training set afterwards. Default value is 5.</p></li>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit_cv">
-<span class="sig-name descname"><span class="pre">fit_cv</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.fit_cv"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit_cv" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">fit_cv</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.fit_cv"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit_cv" title="Permalink to this definition"></a></dt>
 <dd><p>Fits the calibration in a cross-validation manner, i.e., it generates posterior probabilities for all
 training instances via cross-validation, and then retrains the classifier on all training instances.
 The posterior probabilities thus generated are used for calibrating the outputs of the classifier.</p>
@@ -221,7 +222,7 @@ The posterior probabilities thus generated are used for calibrating the outputs
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit_tr_val">
-<span class="sig-name descname"><span class="pre">fit_tr_val</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.fit_tr_val"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit_tr_val" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">fit_tr_val</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.fit_tr_val"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.fit_tr_val" title="Permalink to this definition"></a></dt>
 <dd><p>Fits the calibration in a train/val-split manner, i.e.t, it partitions the training instances into a
 training and a validation set, and then uses the training samples to learn classifier which is then used
 to generate posterior probabilities for the held-out validation data. These posteriors are used to calibrate
@@ -241,7 +242,7 @@ the classifier. The classifier is not retrained on the whole dataset.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.predict">
-<span class="sig-name descname"><span class="pre">predict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.predict"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.predict" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">predict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.predict"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.predict" title="Permalink to this definition"></a></dt>
 <dd><p>Predicts class labels for the data instances in <cite>X</cite></p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -255,7 +256,7 @@ the classifier. The classifier is not retrained on the whole dataset.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.predict_proba">
-<span class="sig-name descname"><span class="pre">predict_proba</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.predict_proba"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.predict_proba" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">predict_proba</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#RecalibratedProbabilisticClassifierBase.predict_proba"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase.predict_proba" title="Permalink to this definition"></a></dt>
 <dd><p>Generates posterior probabilities for the data instances in <cite>X</cite></p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -271,7 +272,7 @@ the classifier. The classifier is not retrained on the whole dataset.</p>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.calibration.TSCalibration">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">TSCalibration</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#TSCalibration"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.TSCalibration" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">TSCalibration</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#TSCalibration"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.TSCalibration" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <a class="reference internal" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase" title="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase"><code class="xref py py-class docutils literal notranslate"><span class="pre">RecalibratedProbabilisticClassifierBase</span></code></a></p>
 <p>Applies the Temperature Scaling (TS) calibration method from <cite>abstention.calibration</cite>, as defined in
 <a class="reference external" href="http://proceedings.mlr.press/v119/alexandari20a.html">Alexandari et al. paper</a>:</p>
@@ -292,7 +293,7 @@ training set afterwards. Default value is 5.</p></li>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.calibration.VSCalibration">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">VSCalibration</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#VSCalibration"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.VSCalibration" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.calibration.</span></span><span class="sig-name descname"><span class="pre">VSCalibration</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">classifier</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/calibration.html#VSCalibration"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.calibration.VSCalibration" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <a class="reference internal" href="#quapy.classification.calibration.RecalibratedProbabilisticClassifierBase" title="quapy.classification.calibration.RecalibratedProbabilisticClassifierBase"><code class="xref py py-class docutils literal notranslate"><span class="pre">RecalibratedProbabilisticClassifierBase</span></code></a></p>
 <p>Applies the Vector Scaling (VS) calibration method from <cite>abstention.calibration</cite>, as defined in
 <a class="reference external" href="http://proceedings.mlr.press/v119/alexandari20a.html">Alexandari et al. paper</a>:</p>
@@ -313,10 +314,10 @@ training set afterwards. Default value is 5.</p></li>
 
 </section>
 <section id="module-quapy.classification.methods">
-<span id="quapy-classification-methods-module"></span><h2>quapy.classification.methods module<a class="headerlink" href="#module-quapy.classification.methods" title="Link to this heading"></a></h2>
+<span id="quapy-classification-methods-module"></span><h2>quapy.classification.methods module<a class="headerlink" href="#module-quapy.classification.methods" title="Permalink to this heading"></a></h2>
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.methods.LowRankLogisticRegression">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.methods.</span></span><span class="sig-name descname"><span class="pre">LowRankLogisticRegression</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">n_components</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.methods.</span></span><span class="sig-name descname"><span class="pre">LowRankLogisticRegression</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">n_components</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">BaseEstimator</span></code></p>
 <p>An example of a classification method (i.e., an object that implements <cite>fit</cite>, <cite>predict</cite>, and <cite>predict_proba</cite>)
 that also generates embedded inputs (i.e., that implements <cite>transform</cite>), as those required for
@@ -335,7 +336,7 @@ while classification is performed using <code class="xref py py-class docutils l
 </dl>
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.methods.LowRankLogisticRegression.fit">
-<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.fit" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.fit" title="Permalink to this definition"></a></dt>
 <dd><p>Fit the model according to the given training data. The fit consists of
 fitting <cite>TruncatedSVD</cite> and then <cite>LogisticRegression</cite> on the low-rank representation.</p>
 <dl class="field-list simple">
@@ -353,7 +354,7 @@ fitting <cite>TruncatedSVD</cite> and then <cite>LogisticRegression</cite> on th
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.methods.LowRankLogisticRegression.get_params">
-<span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.get_params" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.get_params" title="Permalink to this definition"></a></dt>
 <dd><p>Get hyper-parameters for this estimator.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -364,7 +365,7 @@ fitting <cite>TruncatedSVD</cite> and then <cite>LogisticRegression</cite> on th
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.methods.LowRankLogisticRegression.predict">
-<span class="sig-name descname"><span class="pre">predict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.predict"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.predict" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">predict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.predict"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.predict" title="Permalink to this definition"></a></dt>
 <dd><p>Predicts labels for the instances <cite>X</cite> embedded into the low-rank space.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -379,7 +380,7 @@ instances in <cite>X</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.methods.LowRankLogisticRegression.predict_proba">
-<span class="sig-name descname"><span class="pre">predict_proba</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.predict_proba"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.predict_proba" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">predict_proba</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.predict_proba"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.predict_proba" title="Permalink to this definition"></a></dt>
 <dd><p>Predicts posterior probabilities for the instances <cite>X</cite> embedded into the low-rank space.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -393,7 +394,7 @@ instances in <cite>X</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.methods.LowRankLogisticRegression.set_params">
-<span class="sig-name descname"><span class="pre">set_params</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">params</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.set_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.set_params" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">set_params</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">params</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.set_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.set_params" title="Permalink to this definition"></a></dt>
 <dd><p>Set the parameters of this estimator.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -406,7 +407,7 @@ and eventually also <cite>n_components</cite> for <cite>TruncatedSVD</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.methods.LowRankLogisticRegression.transform">
-<span class="sig-name descname"><span class="pre">transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.transform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.transform" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/methods.html#LowRankLogisticRegression.transform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.methods.LowRankLogisticRegression.transform" title="Permalink to this definition"></a></dt>
 <dd><p>Returns the low-rank approximation of <cite>X</cite> with <cite>n_components</cite> dimensions, or <cite>X</cite> unaltered if
 <cite>n_components</cite> &gt;= <cite>X.shape[1]</cite>.</p>
 <dl class="field-list simple">
@@ -423,10 +424,10 @@ and eventually also <cite>n_components</cite> for <cite>TruncatedSVD</cite></p>
 
 </section>
 <section id="module-quapy.classification.neural">
-<span id="quapy-classification-neural-module"></span><h2>quapy.classification.neural module<a class="headerlink" href="#module-quapy.classification.neural" title="Link to this heading"></a></h2>
+<span id="quapy-classification-neural-module"></span><h2>quapy.classification.neural module<a class="headerlink" href="#module-quapy.classification.neural" title="Permalink to this heading"></a></h2>
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.neural.CNNnet">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">CNNnet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">vocabulary_size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_classes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">embedding_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hidden_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">256</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">repr_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">kernel_heights</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">[3,</span> <span class="pre">5,</span> <span class="pre">7]</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">stride</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">padding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">drop_p</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.5</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#CNNnet"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.CNNnet" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">CNNnet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">vocabulary_size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_classes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">embedding_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hidden_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">256</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">repr_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">kernel_heights</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">[3,</span> <span class="pre">5,</span> <span class="pre">7]</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">stride</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">padding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">drop_p</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.5</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#CNNnet"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.CNNnet" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <a class="reference internal" href="#quapy.classification.neural.TextClassifierNet" title="quapy.classification.neural.TextClassifierNet"><code class="xref py py-class docutils literal notranslate"><span class="pre">TextClassifierNet</span></code></a></p>
 <p>An implementation of <a class="reference internal" href="#quapy.classification.neural.TextClassifierNet" title="quapy.classification.neural.TextClassifierNet"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.classification.neural.TextClassifierNet</span></code></a> based on
 Convolutional Neural Networks.</p>
@@ -448,7 +449,7 @@ consecutive tokens that each kernel covers</p></li>
 </dl>
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.CNNnet.document_embedding">
-<span class="sig-name descname"><span class="pre">document_embedding</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">input</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#CNNnet.document_embedding"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.CNNnet.document_embedding" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">document_embedding</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">input</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#CNNnet.document_embedding"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.CNNnet.document_embedding" title="Permalink to this definition"></a></dt>
 <dd><p>Embeds documents (i.e., performs the forward pass up to the
 next-to-last layer).</p>
 <dl class="field-list simple">
@@ -466,7 +467,7 @@ dimensionality of the embedding</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.CNNnet.get_params">
-<span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#CNNnet.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.CNNnet.get_params" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#CNNnet.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.CNNnet.get_params" title="Permalink to this definition"></a></dt>
 <dd><p>Get hyper-parameters for this estimator</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -475,9 +476,14 @@ dimensionality of the embedding</p>
 </dl>
 </dd></dl>
 
+<dl class="py attribute">
+<dt class="sig sig-object py" id="quapy.classification.neural.CNNnet.training">
+<span class="sig-name descname"><span class="pre">training</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><a class="headerlink" href="#quapy.classification.neural.CNNnet.training" title="Permalink to this definition"></a></dt>
+<dd></dd></dl>
+
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.classification.neural.CNNnet.vocabulary_size">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">vocabulary_size</span></span><a class="headerlink" href="#quapy.classification.neural.CNNnet.vocabulary_size" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">vocabulary_size</span></span><a class="headerlink" href="#quapy.classification.neural.CNNnet.vocabulary_size" title="Permalink to this definition"></a></dt>
 <dd><p>Return the size of the vocabulary</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -490,7 +496,7 @@ dimensionality of the embedding</p>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.neural.LSTMnet">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">LSTMnet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">vocabulary_size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_classes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">embedding_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hidden_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">256</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">repr_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lstm_class_nlayers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">drop_p</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.5</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#LSTMnet"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.LSTMnet" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">LSTMnet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">vocabulary_size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_classes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">embedding_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hidden_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">256</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">repr_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lstm_class_nlayers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">drop_p</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.5</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#LSTMnet"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.LSTMnet" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <a class="reference internal" href="#quapy.classification.neural.TextClassifierNet" title="quapy.classification.neural.TextClassifierNet"><code class="xref py py-class docutils literal notranslate"><span class="pre">TextClassifierNet</span></code></a></p>
 <p>An implementation of <a class="reference internal" href="#quapy.classification.neural.TextClassifierNet" title="quapy.classification.neural.TextClassifierNet"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.classification.neural.TextClassifierNet</span></code></a> based on
 Long Short Term Memory networks.</p>
@@ -509,7 +515,7 @@ Long Short Term Memory networks.</p>
 </dl>
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.LSTMnet.document_embedding">
-<span class="sig-name descname"><span class="pre">document_embedding</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#LSTMnet.document_embedding"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.LSTMnet.document_embedding" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">document_embedding</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#LSTMnet.document_embedding"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.LSTMnet.document_embedding" title="Permalink to this definition"></a></dt>
 <dd><p>Embeds documents (i.e., performs the forward pass up to the
 next-to-last layer).</p>
 <dl class="field-list simple">
@@ -527,7 +533,7 @@ dimensionality of the embedding</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.LSTMnet.get_params">
-<span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#LSTMnet.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.LSTMnet.get_params" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#LSTMnet.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.LSTMnet.get_params" title="Permalink to this definition"></a></dt>
 <dd><p>Get hyper-parameters for this estimator</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -536,9 +542,14 @@ dimensionality of the embedding</p>
 </dl>
 </dd></dl>
 
+<dl class="py attribute">
+<dt class="sig sig-object py" id="quapy.classification.neural.LSTMnet.training">
+<span class="sig-name descname"><span class="pre">training</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><a class="headerlink" href="#quapy.classification.neural.LSTMnet.training" title="Permalink to this definition"></a></dt>
+<dd></dd></dl>
+
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.classification.neural.LSTMnet.vocabulary_size">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">vocabulary_size</span></span><a class="headerlink" href="#quapy.classification.neural.LSTMnet.vocabulary_size" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">vocabulary_size</span></span><a class="headerlink" href="#quapy.classification.neural.LSTMnet.vocabulary_size" title="Permalink to this definition"></a></dt>
 <dd><p>Return the size of the vocabulary</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -551,7 +562,7 @@ dimensionality of the embedding</p>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">NeuralClassifierTrainer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">net</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.classification.neural.TextClassifierNet" title="quapy.classification.neural.TextClassifierNet"><span class="pre">TextClassifierNet</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight_decay</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">patience</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">10</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">epochs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">200</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">batch_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">64</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">batch_size_test</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">512</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">padding_length</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">300</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">device</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'cuda'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">checkpointpath</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'../checkpoint/classifier_net.dat'</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">NeuralClassifierTrainer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">net</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.classification.neural.TextClassifierNet" title="quapy.classification.neural.TextClassifierNet"><span class="pre">TextClassifierNet</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight_decay</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">patience</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">10</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">epochs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">200</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">batch_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">64</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">batch_size_test</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">512</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">padding_length</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">300</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">device</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'cuda'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">checkpointpath</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'../checkpoint/classifier_net.dat'</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
 <p>Trains a neural network for text classification.</p>
 <dl class="field-list simple">
@@ -574,7 +585,7 @@ according to the evaluation in the held-out validation split (default ‘../chec
 </dl>
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer.device">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">device</span></span><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.device" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">device</span></span><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.device" title="Permalink to this definition"></a></dt>
 <dd><p>Gets the device in which the network is allocated</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -585,7 +596,7 @@ according to the evaluation in the held-out validation split (default ‘../chec
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer.fit">
-<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">labels</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.3</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.fit" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">labels</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">val_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.3</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.fit" title="Permalink to this definition"></a></dt>
 <dd><p>Fits the model according to the given training data.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -603,7 +614,7 @@ according to the evaluation in the held-out validation split (default ‘../chec
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer.get_params">
-<span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.get_params" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.get_params" title="Permalink to this definition"></a></dt>
 <dd><p>Get hyper-parameters for this estimator</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -614,7 +625,7 @@ according to the evaluation in the held-out validation split (default ‘../chec
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer.predict">
-<span class="sig-name descname"><span class="pre">predict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.predict"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.predict" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">predict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.predict"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.predict" title="Permalink to this definition"></a></dt>
 <dd><p>Predicts labels for the instances</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -629,7 +640,7 @@ instances in <cite>X</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer.predict_proba">
-<span class="sig-name descname"><span class="pre">predict_proba</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.predict_proba"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.predict_proba" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">predict_proba</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.predict_proba"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.predict_proba" title="Permalink to this definition"></a></dt>
 <dd><p>Predicts posterior probabilities for the instances</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -643,7 +654,7 @@ instances in <cite>X</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer.reset_net_params">
-<span class="sig-name descname"><span class="pre">reset_net_params</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">vocab_size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_classes</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.reset_net_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.reset_net_params" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">reset_net_params</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">vocab_size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_classes</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.reset_net_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.reset_net_params" title="Permalink to this definition"></a></dt>
 <dd><p>Reinitialize the network parameters</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -657,7 +668,7 @@ instances in <cite>X</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer.set_params">
-<span class="sig-name descname"><span class="pre">set_params</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">params</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.set_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.set_params" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">set_params</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">params</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.set_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.set_params" title="Permalink to this definition"></a></dt>
 <dd><p>Set the parameters of this trainer and the learner it is training.
 In this current version, parameter names for the trainer and learner should
 be disjoint.</p>
@@ -670,7 +681,7 @@ be disjoint.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.NeuralClassifierTrainer.transform">
-<span class="sig-name descname"><span class="pre">transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.transform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.transform" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#NeuralClassifierTrainer.transform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.NeuralClassifierTrainer.transform" title="Permalink to this definition"></a></dt>
 <dd><p>Returns the embeddings of the instances</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -687,12 +698,12 @@ where <cite>embed_size</cite> is defined by the classification network</p>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">TextClassifierNet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">TextClassifierNet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">Module</span></code></p>
 <p>Abstract Text classifier (<cite>torch.nn.Module</cite>)</p>
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet.dimensions">
-<span class="sig-name descname"><span class="pre">dimensions</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.dimensions"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.dimensions" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">dimensions</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.dimensions"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.dimensions" title="Permalink to this definition"></a></dt>
 <dd><p>Gets the number of dimensions of the embedding space</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -703,7 +714,7 @@ where <cite>embed_size</cite> is defined by the classification network</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet.document_embedding">
-<em class="property"><span class="pre">abstract</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">document_embedding</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.document_embedding"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.document_embedding" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">abstract</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">document_embedding</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.document_embedding"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.document_embedding" title="Permalink to this definition"></a></dt>
 <dd><p>Embeds documents (i.e., performs the forward pass up to the
 next-to-last layer).</p>
 <dl class="field-list simple">
@@ -721,7 +732,7 @@ dimensionality of the embedding</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet.forward">
-<span class="sig-name descname"><span class="pre">forward</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.forward"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.forward" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">forward</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.forward"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.forward" title="Permalink to this definition"></a></dt>
 <dd><p>Performs the forward pass.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -737,7 +748,7 @@ for each of the instances and classes</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet.get_params">
-<em class="property"><span class="pre">abstract</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.get_params" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">abstract</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">get_params</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.get_params"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.get_params" title="Permalink to this definition"></a></dt>
 <dd><p>Get hyper-parameters for this estimator</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -748,7 +759,7 @@ for each of the instances and classes</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet.predict_proba">
-<span class="sig-name descname"><span class="pre">predict_proba</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.predict_proba"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.predict_proba" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">predict_proba</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.predict_proba"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.predict_proba" title="Permalink to this definition"></a></dt>
 <dd><p>Predicts posterior probabilities for the instances in <cite>x</cite></p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -762,9 +773,14 @@ is length of the pad in the batch</p>
 </dl>
 </dd></dl>
 
+<dl class="py attribute">
+<dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet.training">
+<span class="sig-name descname"><span class="pre">training</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.training" title="Permalink to this definition"></a></dt>
+<dd></dd></dl>
+
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet.vocabulary_size">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">vocabulary_size</span></span><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.vocabulary_size" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">vocabulary_size</span></span><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.vocabulary_size" title="Permalink to this definition"></a></dt>
 <dd><p>Return the size of the vocabulary</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -775,7 +791,7 @@ is length of the pad in the batch</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.TextClassifierNet.xavier_uniform">
-<span class="sig-name descname"><span class="pre">xavier_uniform</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.xavier_uniform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.xavier_uniform" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">xavier_uniform</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TextClassifierNet.xavier_uniform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TextClassifierNet.xavier_uniform" title="Permalink to this definition"></a></dt>
 <dd><p>Performs Xavier initialization of the network parameters</p>
 </dd></dl>
 
@@ -783,7 +799,7 @@ is length of the pad in the batch</p>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.neural.TorchDataset">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">TorchDataset</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">labels</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TorchDataset"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TorchDataset" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.neural.</span></span><span class="sig-name descname"><span class="pre">TorchDataset</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">labels</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TorchDataset"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TorchDataset" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">Dataset</span></code></p>
 <p>Transforms labelled instances into a Torch’s <code class="xref py py-class docutils literal notranslate"><span class="pre">torch.utils.data.DataLoader</span></code> object</p>
 <dl class="field-list simple">
@@ -796,7 +812,7 @@ is length of the pad in the batch</p>
 </dl>
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.neural.TorchDataset.asDataloader">
-<span class="sig-name descname"><span class="pre">asDataloader</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">batch_size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">shuffle</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pad_length</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">device</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TorchDataset.asDataloader"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TorchDataset.asDataloader" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">asDataloader</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">batch_size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">shuffle</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pad_length</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">device</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/neural.html#TorchDataset.asDataloader"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.neural.TorchDataset.asDataloader" title="Permalink to this definition"></a></dt>
 <dd><p>Converts the labelled collection into a Torch DataLoader with dynamic padding for
 the batch</p>
 <dl class="field-list simple">
@@ -820,10 +836,10 @@ applied, meaning that if the longest document in the batch is shorter than
 
 </section>
 <section id="module-quapy.classification.svmperf">
-<span id="quapy-classification-svmperf-module"></span><h2>quapy.classification.svmperf module<a class="headerlink" href="#module-quapy.classification.svmperf" title="Link to this heading"></a></h2>
+<span id="quapy-classification-svmperf-module"></span><h2>quapy.classification.svmperf module<a class="headerlink" href="#module-quapy.classification.svmperf" title="Permalink to this heading"></a></h2>
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.classification.svmperf.SVMperf">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.svmperf.</span></span><span class="sig-name descname"><span class="pre">SVMperf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">svmperf_base</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">C</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.01</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">loss</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'01'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">host_folder</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/svmperf.html#SVMperf"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.svmperf.SVMperf" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.classification.svmperf.</span></span><span class="sig-name descname"><span class="pre">SVMperf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">svmperf_base</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">C</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.01</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">loss</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'01'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">host_folder</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/svmperf.html#SVMperf"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.svmperf.SVMperf" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">BaseEstimator</span></code>, <code class="xref py py-class docutils literal notranslate"><span class="pre">ClassifierMixin</span></code></p>
 <p>A wrapper for the <a class="reference external" href="https://www.cs.cornell.edu/people/tj/svm_light/svm_perf.html">SVM-perf package</a> by Thorsten Joachims.
 When using losses for quantification, the source code has to be patched. See
@@ -848,7 +864,7 @@ for further details.</p>
 </dl>
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.svmperf.SVMperf.decision_function">
-<span class="sig-name descname"><span class="pre">decision_function</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/svmperf.html#SVMperf.decision_function"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.svmperf.SVMperf.decision_function" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">decision_function</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/svmperf.html#SVMperf.decision_function"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.svmperf.SVMperf.decision_function" title="Permalink to this definition"></a></dt>
 <dd><p>Evaluate the decision function for the samples in <cite>X</cite>.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -865,7 +881,7 @@ for further details.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.svmperf.SVMperf.fit">
-<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/svmperf.html#SVMperf.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.svmperf.SVMperf.fit" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/svmperf.html#SVMperf.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.svmperf.SVMperf.fit" title="Permalink to this definition"></a></dt>
 <dd><p>Trains the SVM for the multivariate performance loss</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -882,7 +898,7 @@ for further details.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.classification.svmperf.SVMperf.predict">
-<span class="sig-name descname"><span class="pre">predict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/svmperf.html#SVMperf.predict"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.svmperf.SVMperf.predict" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">predict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/classification/svmperf.html#SVMperf.predict"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.classification.svmperf.SVMperf.predict" title="Permalink to this definition"></a></dt>
 <dd><p>Predicts labels for the instances <cite>X</cite></p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -897,14 +913,14 @@ instances in <cite>X</cite></p>
 
 <dl class="py attribute">
 <dt class="sig sig-object py" id="quapy.classification.svmperf.SVMperf.valid_losses">
-<span class="sig-name descname"><span class="pre">valid_losses</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">{'01':</span> <span class="pre">0,</span> <span class="pre">'f1':</span> <span class="pre">1,</span> <span class="pre">'kld':</span> <span class="pre">12,</span> <span class="pre">'mae':</span> <span class="pre">26,</span> <span class="pre">'mrae':</span> <span class="pre">27,</span> <span class="pre">'nkld':</span> <span class="pre">13,</span> <span class="pre">'q':</span> <span class="pre">22,</span> <span class="pre">'qacc':</span> <span class="pre">23,</span> <span class="pre">'qf1':</span> <span class="pre">24,</span> <span class="pre">'qgm':</span> <span class="pre">25}</span></em><a class="headerlink" href="#quapy.classification.svmperf.SVMperf.valid_losses" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">valid_losses</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">{'01':</span> <span class="pre">0,</span> <span class="pre">'f1':</span> <span class="pre">1,</span> <span class="pre">'kld':</span> <span class="pre">12,</span> <span class="pre">'mae':</span> <span class="pre">26,</span> <span class="pre">'mrae':</span> <span class="pre">27,</span> <span class="pre">'nkld':</span> <span class="pre">13,</span> <span class="pre">'q':</span> <span class="pre">22,</span> <span class="pre">'qacc':</span> <span class="pre">23,</span> <span class="pre">'qf1':</span> <span class="pre">24,</span> <span class="pre">'qgm':</span> <span class="pre">25}</span></em><a class="headerlink" href="#quapy.classification.svmperf.SVMperf.valid_losses" title="Permalink to this definition"></a></dt>
 <dd></dd></dl>
 
 </dd></dl>
 
 </section>
 <section id="module-quapy.classification">
-<span id="module-contents"></span><h2>Module contents<a class="headerlink" href="#module-quapy.classification" title="Link to this heading"></a></h2>
+<span id="module-contents"></span><h2>Module contents<a class="headerlink" href="#module-quapy.classification" title="Permalink to this heading"></a></h2>
 </section>
 </section>
 

docs/build/html/quapy.data.html

@@ -1,23 +1,24 @@
 <!DOCTYPE html>
-<html class="writer-html5" lang="en" data-content_root="./">
+<html class="writer-html5" lang="en">
 <head>
   <meta charset="utf-8" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>quapy.data package &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.8 documentation</title>
-      <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=92fd9be5" />
-      <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+  <title>quapy.data package &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.9 documentation</title>
+      <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
+      <link rel="stylesheet" type="text/css" href="_static/css/theme.css" />
 
   
   <!--[if lt IE 9]>
     <script src="_static/js/html5shiv.min.js"></script>
   <![endif]-->
   
-        <script src="_static/jquery.js?v=5d32c60e"></script>
-        <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
-        <script src="_static/documentation_options.js?v=22607128"></script>
-        <script src="_static/doctools.js?v=9a2dae69"></script>
-        <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
+        <script src="_static/jquery.js"></script>
+        <script src="_static/underscore.js"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
+        <script src="_static/doctools.js"></script>
+        <script src="_static/sphinx_highlight.js"></script>
     <script src="_static/js/theme.js"></script>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
@@ -95,15 +96,15 @@
            <div itemprop="articleBody">
              
   <section id="quapy-data-package">
-<h1>quapy.data package<a class="headerlink" href="#quapy-data-package" title="Link to this heading"></a></h1>
+<h1>quapy.data package<a class="headerlink" href="#quapy-data-package" title="Permalink to this heading"></a></h1>
 <section id="submodules">
-<h2>Submodules<a class="headerlink" href="#submodules" title="Link to this heading"></a></h2>
+<h2>Submodules<a class="headerlink" href="#submodules" title="Permalink to this heading"></a></h2>
 </section>
 <section id="module-quapy.data.base">
-<span id="quapy-data-base-module"></span><h2>quapy.data.base module<a class="headerlink" href="#module-quapy.data.base" title="Link to this heading"></a></h2>
+<span id="quapy-data-base-module"></span><h2>quapy.data.base module<a class="headerlink" href="#module-quapy.data.base" title="Permalink to this heading"></a></h2>
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.data.base.</span></span><span class="sig-name descname"><span class="pre">Dataset</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">training</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">test</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">vocabulary</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">dict</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">None</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">''</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.data.base.</span></span><span class="sig-name descname"><span class="pre">Dataset</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">training</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">test</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">vocabulary</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">dict</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">''</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
 <p>Abstraction of training and test <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a> objects.</p>
 <dl class="field-list simple">
@@ -118,7 +119,7 @@
 </dl>
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.SplitStratified">
-<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">SplitStratified</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">collection</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">train_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.6</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.SplitStratified"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.SplitStratified" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">SplitStratified</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">collection</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">train_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.6</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.SplitStratified"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.SplitStratified" title="Permalink to this definition"></a></dt>
 <dd><p>Generates a <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">Dataset</span></code></a> from a stratified split of a <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a> instance.
 See <a class="reference internal" href="#quapy.data.base.LabelledCollection.split_stratified" title="quapy.data.base.LabelledCollection.split_stratified"><code class="xref py py-meth docutils literal notranslate"><span class="pre">LabelledCollection.split_stratified()</span></code></a></p>
 <dl class="field-list simple">
@@ -136,7 +137,7 @@ See <a class="reference internal" href="#quapy.data.base.LabelledCollection.spli
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.binary">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">binary</span></span><a class="headerlink" href="#quapy.data.base.Dataset.binary" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">binary</span></span><a class="headerlink" href="#quapy.data.base.Dataset.binary" title="Permalink to this definition"></a></dt>
 <dd><p>Returns True if the training collection is labelled according to two classes</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -147,7 +148,7 @@ See <a class="reference internal" href="#quapy.data.base.LabelledCollection.spli
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.classes_">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">classes_</span></span><a class="headerlink" href="#quapy.data.base.Dataset.classes_" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">classes_</span></span><a class="headerlink" href="#quapy.data.base.Dataset.classes_" title="Permalink to this definition"></a></dt>
 <dd><p>The classes according to which the training collection is labelled</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -158,7 +159,7 @@ See <a class="reference internal" href="#quapy.data.base.LabelledCollection.spli
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.kFCV">
-<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">kFCV</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">data</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">nfolds</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">nrepeats</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.kFCV"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.kFCV" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">kFCV</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">data</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">nfolds</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">nrepeats</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.kFCV"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.kFCV" title="Permalink to this definition"></a></dt>
 <dd><p>Generator of stratified folds to be used in k-fold cross validation. This function is only a wrapper around
 <a class="reference internal" href="#quapy.data.base.LabelledCollection.kFCV" title="quapy.data.base.LabelledCollection.kFCV"><code class="xref py py-meth docutils literal notranslate"><span class="pre">LabelledCollection.kFCV()</span></code></a> that returns <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">Dataset</span></code></a> instances made of training and test folds.</p>
 <dl class="field-list simple">
@@ -177,7 +178,7 @@ See <a class="reference internal" href="#quapy.data.base.LabelledCollection.spli
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.load">
-<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">load</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">train_path</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">test_path</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">loader_func</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">callable</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">classes</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">loader_kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.load"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.load" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">load</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">train_path</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">test_path</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">loader_func</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">callable</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">classes</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">loader_kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.load"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.load" title="Permalink to this definition"></a></dt>
 <dd><p>Loads a training and a test labelled set of data and convert it into a <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">Dataset</span></code></a> instance.
 The function in charge of reading the instances must be specified. This function can be a custom one, or any of
 the reading functions defined in <a class="reference internal" href="#module-quapy.data.reader" title="quapy.data.reader"><code class="xref py py-mod docutils literal notranslate"><span class="pre">quapy.data.reader</span></code></a> module.</p>
@@ -201,7 +202,7 @@ See <a class="reference internal" href="#quapy.data.base.LabelledCollection.load
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.n_classes">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">n_classes</span></span><a class="headerlink" href="#quapy.data.base.Dataset.n_classes" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">n_classes</span></span><a class="headerlink" href="#quapy.data.base.Dataset.n_classes" title="Permalink to this definition"></a></dt>
 <dd><p>The number of classes according to which the training collection is labelled</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -212,7 +213,7 @@ See <a class="reference internal" href="#quapy.data.base.LabelledCollection.load
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.reduce">
-<span class="sig-name descname"><span class="pre">reduce</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">n_train</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_test</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.reduce"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.reduce" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">reduce</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">n_train</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_test</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.reduce"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.reduce" title="Permalink to this definition"></a></dt>
 <dd><p>Reduce the number of instances in place for quick experiments. Preserves the prevalence of each set.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -229,7 +230,7 @@ See <a class="reference internal" href="#quapy.data.base.LabelledCollection.load
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.stats">
-<span class="sig-name descname"><span class="pre">stats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">show</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.stats"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.stats" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">stats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">show</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#Dataset.stats"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.Dataset.stats" title="Permalink to this definition"></a></dt>
 <dd><p>Returns (and eventually prints) a dictionary with some stats of this dataset. E.g.,:</p>
 <div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">data</span> <span class="o">=</span> <span class="n">qp</span><span class="o">.</span><span class="n">datasets</span><span class="o">.</span><span class="n">fetch_reviews</span><span class="p">(</span><span class="s1">&#39;kindle&#39;</span><span class="p">,</span> <span class="n">tfidf</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">min_df</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
 <span class="gp">&gt;&gt;&gt; </span><span class="n">data</span><span class="o">.</span><span class="n">stats</span><span class="p">()</span>
@@ -252,7 +253,7 @@ the collection), <cite>prevs</cite> (the prevalence values for each class)</p>
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.train_test">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">train_test</span></span><a class="headerlink" href="#quapy.data.base.Dataset.train_test" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">train_test</span></span><a class="headerlink" href="#quapy.data.base.Dataset.train_test" title="Permalink to this definition"></a></dt>
 <dd><p>Alias to <cite>self.training</cite> and <cite>self.test</cite></p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -266,7 +267,7 @@ the collection), <cite>prevs</cite> (the prevalence values for each class)</p>
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.Dataset.vocabulary_size">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">vocabulary_size</span></span><a class="headerlink" href="#quapy.data.base.Dataset.vocabulary_size" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">vocabulary_size</span></span><a class="headerlink" href="#quapy.data.base.Dataset.vocabulary_size" title="Permalink to this definition"></a></dt>
 <dd><p>If the dataset is textual, and the vocabulary was indicated, returns the size of the vocabulary</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -279,7 +280,7 @@ the collection), <cite>prevs</cite> (the prevalence values for each class)</p>
 
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.data.base.</span></span><span class="sig-name descname"><span class="pre">LabelledCollection</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">labels</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">classes</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.data.base.</span></span><span class="sig-name descname"><span class="pre">LabelledCollection</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instances</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">labels</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">classes</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
 <p>A LabelledCollection is a set of objects each with a label attached to each of them.
 This class implements several sampling routines and other utilities.</p>
@@ -296,7 +297,7 @@ from the labels. The classes must be indicated in cases in which some of the lab
 </dl>
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.X">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">X</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.X" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">X</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.X" title="Permalink to this definition"></a></dt>
 <dd><p>An alias to self.instances</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -307,7 +308,7 @@ from the labels. The classes must be indicated in cases in which some of the lab
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.Xp">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">Xp</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.Xp" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">Xp</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.Xp" title="Permalink to this definition"></a></dt>
 <dd><p>Gets the instances and the true prevalence. This is useful when implementing evaluation protocols from
 a <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a> object.</p>
 <dl class="field-list simple">
@@ -319,7 +320,7 @@ a <a class="reference internal" href="#quapy.data.base.LabelledCollection" title
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.Xy">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">Xy</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.Xy" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">Xy</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.Xy" title="Permalink to this definition"></a></dt>
 <dd><p>Gets the instances and labels. This is useful when working with <cite>sklearn</cite> estimators, e.g.:</p>
 <div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">svm</span> <span class="o">=</span> <span class="n">LinearSVC</span><span class="p">()</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="o">*</span><span class="n">my_collection</span><span class="o">.</span><span class="n">Xy</span><span class="p">)</span>
 </pre></div>
@@ -333,7 +334,7 @@ a <a class="reference internal" href="#quapy.data.base.LabelledCollection" title
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.binary">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">binary</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.binary" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">binary</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.binary" title="Permalink to this definition"></a></dt>
 <dd><p>Returns True if the number of classes is 2</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -344,7 +345,7 @@ a <a class="reference internal" href="#quapy.data.base.LabelledCollection" title
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.counts">
-<span class="sig-name descname"><span class="pre">counts</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.counts"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.counts" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">counts</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.counts"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.counts" title="Permalink to this definition"></a></dt>
 <dd><p>Returns the number of instances for each of the classes in the codeframe.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -356,7 +357,7 @@ as listed by <cite>self.classes_</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.join">
-<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">join</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Iterable</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.join"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.join" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">join</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Iterable</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.join"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.join" title="Permalink to this definition"></a></dt>
 <dd><p>Returns a new <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a> as the union of the collections given in input.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -370,7 +371,7 @@ as listed by <cite>self.classes_</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.kFCV">
-<span class="sig-name descname"><span class="pre">kFCV</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">nfolds</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">nrepeats</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.kFCV"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.kFCV" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">kFCV</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">nfolds</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">nrepeats</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.kFCV"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.kFCV" title="Permalink to this definition"></a></dt>
 <dd><p>Generator of stratified folds to be used in k-fold cross validation.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -388,7 +389,7 @@ as listed by <cite>self.classes_</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.load">
-<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">load</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">loader_func</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">callable</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">classes</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">loader_kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.load"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.load" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">load</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">loader_func</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">callable</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">classes</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">loader_kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.load"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.load" title="Permalink to this definition"></a></dt>
 <dd><p>Loads a labelled set of data and convert it into a <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a> instance. The function in charge
 of reading the instances must be specified. This function can be a custom one, or any of the reading functions
 defined in <a class="reference internal" href="#module-quapy.data.reader" title="quapy.data.reader"><code class="xref py py-mod docutils literal notranslate"><span class="pre">quapy.data.reader</span></code></a> module.</p>
@@ -411,7 +412,7 @@ these arguments are used to call <cite>loader_func(path, **loader_kwargs)</cite>
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.n_classes">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">n_classes</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.n_classes" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">n_classes</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.n_classes" title="Permalink to this definition"></a></dt>
 <dd><p>The number of classes</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -422,7 +423,7 @@ these arguments are used to call <cite>loader_func(path, **loader_kwargs)</cite>
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.p">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">p</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.p" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">p</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.p" title="Permalink to this definition"></a></dt>
 <dd><p>An alias to self.prevalence()</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -433,7 +434,7 @@ these arguments are used to call <cite>loader_func(path, **loader_kwargs)</cite>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.prevalence">
-<span class="sig-name descname"><span class="pre">prevalence</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.prevalence"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.prevalence" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">prevalence</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.prevalence"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.prevalence" title="Permalink to this definition"></a></dt>
 <dd><p>Returns the prevalence, or relative frequency, of the classes in the codeframe.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -445,7 +446,7 @@ as listed by <cite>self.classes_</cite></p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.sampling">
-<span class="sig-name descname"><span class="pre">sampling</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">prevs</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">shuffle</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.sampling"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.sampling" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">sampling</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">prevs</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">shuffle</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.sampling"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.sampling" title="Permalink to this definition"></a></dt>
 <dd><p>Return a random sample (an instance of <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a>) of desired size and desired prevalence
 values. For each class, the sampling is drawn without replacement if the requested prevalence is larger than
 the actual prevalence of the class, or with replacement otherwise.</p>
@@ -469,7 +470,7 @@ prevalence == <cite>prevs</cite> if the exact prevalence values can be met as pr
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.sampling_from_index">
-<span class="sig-name descname"><span class="pre">sampling_from_index</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">index</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.sampling_from_index"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.sampling_from_index" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">sampling_from_index</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">index</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.sampling_from_index"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.sampling_from_index" title="Permalink to this definition"></a></dt>
 <dd><p>Returns an instance of <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a> whose elements are sampled from this collection using the
 index.</p>
 <dl class="field-list simple">
@@ -484,7 +485,7 @@ index.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.sampling_index">
-<span class="sig-name descname"><span class="pre">sampling_index</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">prevs</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">shuffle</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.sampling_index"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.sampling_index" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">sampling_index</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">prevs</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">shuffle</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.sampling_index"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.sampling_index" title="Permalink to this definition"></a></dt>
 <dd><p>Returns an index to be used to extract a random sample of desired size and desired prevalence values. If the
 prevalence values are not specified, then returns the index of a uniform sampling.
 For each class, the sampling is drawn with replacement if the requested prevalence is larger than
@@ -508,7 +509,7 @@ it is constrained. E.g., for binary collections, only the prevalence <cite>p</ci
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.split_random">
-<span class="sig-name descname"><span class="pre">split_random</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">train_prop</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.6</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.split_random"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.split_random" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">split_random</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">train_prop</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.6</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.split_random"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.split_random" title="Permalink to this definition"></a></dt>
 <dd><p>Returns two instances of <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a> split randomly from this collection, at desired
 proportion.</p>
 <dl class="field-list simple">
@@ -529,7 +530,7 @@ second one with <cite>1-train_prop</cite> elements</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.split_stratified">
-<span class="sig-name descname"><span class="pre">split_stratified</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">train_prop</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.6</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.split_stratified"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.split_stratified" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">split_stratified</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">train_prop</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.6</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.split_stratified"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.split_stratified" title="Permalink to this definition"></a></dt>
 <dd><p>Returns two instances of <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a> split with stratification from this collection, at desired
 proportion.</p>
 <dl class="field-list simple">
@@ -550,7 +551,7 @@ second one with <cite>1-train_prop</cite> elements</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.stats">
-<span class="sig-name descname"><span class="pre">stats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">show</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.stats"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.stats" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">stats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">show</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.stats"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.stats" title="Permalink to this definition"></a></dt>
 <dd><p>Returns (and eventually prints) a dictionary with some stats of this collection. E.g.,:</p>
 <div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">data</span> <span class="o">=</span> <span class="n">qp</span><span class="o">.</span><span class="n">datasets</span><span class="o">.</span><span class="n">fetch_reviews</span><span class="p">(</span><span class="s1">&#39;kindle&#39;</span><span class="p">,</span> <span class="n">tfidf</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">min_df</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
 <span class="gp">&gt;&gt;&gt; </span><span class="n">data</span><span class="o">.</span><span class="n">training</span><span class="o">.</span><span class="n">stats</span><span class="p">()</span>
@@ -572,7 +573,7 @@ values for each class)</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.uniform_sampling">
-<span class="sig-name descname"><span class="pre">uniform_sampling</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.uniform_sampling"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.uniform_sampling" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">uniform_sampling</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.uniform_sampling"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.uniform_sampling" title="Permalink to this definition"></a></dt>
 <dd><p>Returns a uniform sample (an instance of <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">LabelledCollection</span></code></a>) of desired size. The sampling is drawn
 with replacement if the requested size is greater than the number of instances, or without replacement
 otherwise.</p>
@@ -591,7 +592,7 @@ otherwise.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.uniform_sampling_index">
-<span class="sig-name descname"><span class="pre">uniform_sampling_index</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.uniform_sampling_index"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.uniform_sampling_index" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">uniform_sampling_index</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">random_state</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/base.html#LabelledCollection.uniform_sampling_index"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.base.LabelledCollection.uniform_sampling_index" title="Permalink to this definition"></a></dt>
 <dd><p>Returns an index to be used to extract a uniform sample of desired size. The sampling is drawn
 with replacement if the requested size is greater than the number of instances, or without replacement
 otherwise.</p>
@@ -610,7 +611,7 @@ otherwise.</p>
 
 <dl class="py property">
 <dt class="sig sig-object py" id="quapy.data.base.LabelledCollection.y">
-<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">y</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.y" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">y</span></span><a class="headerlink" href="#quapy.data.base.LabelledCollection.y" title="Permalink to this definition"></a></dt>
 <dd><p>An alias to self.labels</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -623,10 +624,10 @@ otherwise.</p>
 
 </section>
 <section id="module-quapy.data.datasets">
-<span id="quapy-data-datasets-module"></span><h2>quapy.data.datasets module<a class="headerlink" href="#module-quapy.data.datasets" title="Link to this heading"></a></h2>
+<span id="quapy-data-datasets-module"></span><h2>quapy.data.datasets module<a class="headerlink" href="#module-quapy.data.datasets" title="Permalink to this heading"></a></h2>
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.fetch_IFCB">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_IFCB</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">single_sample_train</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">for_model_selection</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_IFCB"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_IFCB" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_IFCB</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">single_sample_train</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">for_model_selection</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_IFCB"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_IFCB" title="Permalink to this definition"></a></dt>
 <dd><p>Loads the IFCB dataset for quantification from <a class="reference external" href="https://zenodo.org/records/10036244">Zenodo</a> (for more
 information on this dataset, please follow the zenodo link).
 This dataset is based on the data available publicly at
@@ -658,7 +659,7 @@ i.e., a sampling protocol that returns a series of samples labelled by prevalenc
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.fetch_UCIBinaryDataset">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_UCIBinaryDataset</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">test_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.3</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_UCIBinaryDataset"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_UCIBinaryDataset" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_UCIBinaryDataset</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">test_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.3</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_UCIBinaryDataset"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_UCIBinaryDataset" title="Permalink to this definition"></a></dt>
 <dd><p>Loads a UCI dataset as an instance of <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.Dataset</span></code></a>, as used in
 <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1566253516300628">Pérez-Gállego, P., Quevedo, J. R., &amp; del Coz, J. J. (2017).
 Using ensembles for problems with characterizable changes in data distribution: A case study on quantification.
@@ -688,7 +689,7 @@ The list of valid dataset names can be accessed in <cite>quapy.data.datasets.UCI
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.fetch_UCIBinaryLabelledCollection">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_UCIBinaryLabelledCollection</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_UCIBinaryLabelledCollection"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_UCIBinaryLabelledCollection" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_UCIBinaryLabelledCollection</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_UCIBinaryLabelledCollection"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_UCIBinaryLabelledCollection" title="Permalink to this definition"></a></dt>
 <dd><p>Loads a UCI collection as an instance of <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.LabelledCollection</span></code></a>, as used in
 <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1566253516300628">Pérez-Gállego, P., Quevedo, J. R., &amp; del Coz, J. J. (2017).
 Using ensembles for problems with characterizable changes in data distribution: A case study on quantification.
@@ -725,7 +726,7 @@ This can be reproduced by using <a class="reference internal" href="#quapy.data.
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.fetch_UCIMulticlassDataset">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_UCIMulticlassDataset</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">test_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.3</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_UCIMulticlassDataset"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_UCIMulticlassDataset" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_UCIMulticlassDataset</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">test_split</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.3</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_UCIMulticlassDataset"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_UCIMulticlassDataset" title="Permalink to this definition"></a></dt>
 <dd><p>Loads a UCI multiclass dataset as an instance of <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.Dataset</span></code></a>.</p>
 <p>The list of available datasets is taken from <a class="reference external" href="https://archive.ics.uci.edu/">https://archive.ics.uci.edu/</a>, following these criteria:
 - It has more than 1000 instances
@@ -758,7 +759,7 @@ This can be reproduced by using <a class="reference internal" href="#quapy.data.
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.fetch_UCIMulticlassLabelledCollection">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_UCIMulticlassLabelledCollection</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_UCIMulticlassLabelledCollection"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_UCIMulticlassLabelledCollection" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_UCIMulticlassLabelledCollection</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><span class="pre">LabelledCollection</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_UCIMulticlassLabelledCollection"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_UCIMulticlassLabelledCollection" title="Permalink to this definition"></a></dt>
 <dd><p>Loads a UCI multiclass collection as an instance of <a class="reference internal" href="#quapy.data.base.LabelledCollection" title="quapy.data.base.LabelledCollection"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.LabelledCollection</span></code></a>.</p>
 <p>The list of available datasets is taken from <a class="reference external" href="https://archive.ics.uci.edu/">https://archive.ics.uci.edu/</a>, following these criteria:
 - It has more than 1000 instances
@@ -791,7 +792,7 @@ This can be reproduced by using <a class="reference internal" href="#quapy.data.
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.fetch_lequa2022">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_lequa2022</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_lequa2022"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_lequa2022" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_lequa2022</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_lequa2022"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_lequa2022" title="Permalink to this definition"></a></dt>
 <dd><p>Loads the official datasets provided for the <a class="reference external" href="https://lequa2022.github.io/index">LeQua</a> competition.
 In brief, there are 4 tasks (T1A, T1B, T2A, T2B) having to do with text quantification
 problems. Tasks T1A and T1B provide documents in vector form, while T2A and T2B provide raw documents instead.
@@ -822,7 +823,7 @@ that return a series of samples stored in a directory which are labelled by prev
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.fetch_reviews">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_reviews</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tfidf</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pickle</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_reviews"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_reviews" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_reviews</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tfidf</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pickle</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_reviews"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_reviews" title="Permalink to this definition"></a></dt>
 <dd><p>Loads a Reviews dataset as a Dataset instance, as used in
 <a class="reference external" href="https://dl.acm.org/doi/abs/10.1145/3269206.3269287">Esuli, A., Moreo, A., and Sebastiani, F. “A recurrent neural network for sentiment quantification.”
 Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018.</a>.
@@ -848,7 +849,7 @@ faster subsequent invokations</p></li>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.fetch_twitter">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_twitter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">for_model_selection</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pickle</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_twitter"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_twitter" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">fetch_twitter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">for_model_selection</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data_home</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pickle</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></span><a class="reference internal" href="_modules/quapy/data/datasets.html#fetch_twitter"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.fetch_twitter" title="Permalink to this definition"></a></dt>
 <dd><p>Loads a Twitter dataset as a <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.Dataset</span></code></a> instance, as used in:
 <a class="reference external" href="https://link.springer.com/content/pdf/10.1007/s13278-016-0327-z.pdf">Gao, W., Sebastiani, F.: From classification to quantification in tweet sentiment analysis.
 Social Network Analysis and Mining6(19), 1–22 (2016)</a>
@@ -879,15 +880,15 @@ faster subsequent invokations</p></li>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.datasets.warn">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">warn</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/datasets.html#warn"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.warn" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.datasets.</span></span><span class="sig-name descname"><span class="pre">warn</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/datasets.html#warn"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.datasets.warn" title="Permalink to this definition"></a></dt>
 <dd></dd></dl>
 
 </section>
 <section id="module-quapy.data.preprocessing">
-<span id="quapy-data-preprocessing-module"></span><h2>quapy.data.preprocessing module<a class="headerlink" href="#module-quapy.data.preprocessing" title="Link to this heading"></a></h2>
+<span id="quapy-data-preprocessing-module"></span><h2>quapy.data.preprocessing module<a class="headerlink" href="#module-quapy.data.preprocessing" title="Permalink to this heading"></a></h2>
 <dl class="py class">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.IndexTransformer">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">IndexTransformer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer" title="Link to this definition"></a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">IndexTransformer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer" title="Permalink to this definition"></a></dt>
 <dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
 <p>This class implements a sklearn’s-style transformer that indexes text as numerical ids for the tokens it
 contains, and that would be generated by sklearn’s
@@ -901,7 +902,7 @@ contains, and that would be generated by sklearn’s
 </dl>
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.IndexTransformer.add_word">
-<span class="sig-name descname"><span class="pre">add_word</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">word</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">id</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">nogaps</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.add_word"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.add_word" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">add_word</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">word</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">id</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">nogaps</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.add_word"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.add_word" title="Permalink to this definition"></a></dt>
 <dd><p>Adds a new token (regardless of whether it has been found in the text or not), with dedicated id.
 Useful to define special tokens for codifying unknown words, or padding tokens.</p>
 <dl class="field-list simple">
@@ -922,7 +923,7 @@ precedent ids stored so far</p></li>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.IndexTransformer.fit">
-<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.fit" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">fit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.fit"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.fit" title="Permalink to this definition"></a></dt>
 <dd><p>Fits the transformer, i.e., decides on the vocabulary, given a list of strings.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -936,7 +937,7 @@ precedent ids stored so far</p></li>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.IndexTransformer.fit_transform">
-<span class="sig-name descname"><span class="pre">fit_transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.fit_transform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.fit_transform" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">fit_transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.fit_transform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.fit_transform" title="Permalink to this definition"></a></dt>
 <dd><p>Fits the transform on <cite>X</cite> and transforms it.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -953,7 +954,7 @@ precedent ids stored so far</p></li>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.IndexTransformer.transform">
-<span class="sig-name descname"><span class="pre">transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.transform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.transform" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">n_jobs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.transform"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.transform" title="Permalink to this definition"></a></dt>
 <dd><p>Transforms the strings in <cite>X</cite> as lists of numerical ids</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -970,7 +971,7 @@ precedent ids stored so far</p></li>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.IndexTransformer.vocabulary_size">
-<span class="sig-name descname"><span class="pre">vocabulary_size</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.vocabulary_size"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.vocabulary_size" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">vocabulary_size</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#IndexTransformer.vocabulary_size"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.IndexTransformer.vocabulary_size" title="Permalink to this definition"></a></dt>
 <dd><p>Gets the length of the vocabulary according to which the document tokens have been indexed</p>
 <dl class="field-list simple">
 <dt class="field-odd">Returns<span class="colon">:</span></dt>
@@ -983,7 +984,7 @@ precedent ids stored so far</p></li>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.index">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">index</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">inplace</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#index"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.index" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">index</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">inplace</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#index"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.index" title="Permalink to this definition"></a></dt>
 <dd><p>Indexes the tokens of a textual <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.Dataset</span></code></a> of string documents.
 To index a document means to replace each different token by a unique numerical index.
 Rare words (i.e., words occurring less than <cite>min_df</cite> times) are replaced by a special token <cite>UNK</cite></p>
@@ -1007,7 +1008,7 @@ are lists of str</p></li>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.reduce_columns">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">reduce_columns</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">inplace</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#reduce_columns"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.reduce_columns" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">reduce_columns</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">inplace</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#reduce_columns"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.reduce_columns" title="Permalink to this definition"></a></dt>
 <dd><p>Reduces the dimensionality of the instances, represented as a <cite>csr_matrix</cite> (or any subtype of
 <cite>scipy.sparse.spmatrix</cite>), of training and test documents by removing the columns of words which are not present
 in at least <cite>min_df</cite> instances in the training set</p>
@@ -1030,7 +1031,7 @@ in the training set have been removed</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.standardize">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">standardize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">inplace</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#standardize"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.standardize" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">standardize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">inplace</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#standardize"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.standardize" title="Permalink to this definition"></a></dt>
 <dd><p>Standardizes the real-valued columns of a <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.Dataset</span></code></a>.
 Standardization, aka z-scoring, of a variable <cite>X</cite> comes down to subtracting the average and normalizing by the
 standard deviation.</p>
@@ -1050,7 +1051,7 @@ standard deviation.</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.preprocessing.text2tfidf">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">text2tfidf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">3</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">sublinear_tf</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">inplace</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#text2tfidf"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.text2tfidf" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.preprocessing.</span></span><span class="sig-name descname"><span class="pre">text2tfidf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><span class="pre">Dataset</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">min_df</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">3</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">sublinear_tf</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">inplace</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/preprocessing.html#text2tfidf"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.preprocessing.text2tfidf" title="Permalink to this definition"></a></dt>
 <dd><p>Transforms a <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.Dataset</span></code></a> of textual instances into a <a class="reference internal" href="#quapy.data.base.Dataset" title="quapy.data.base.Dataset"><code class="xref py py-class docutils literal notranslate"><span class="pre">quapy.data.base.Dataset</span></code></a> of
 tfidf weighted sparse vectors</p>
 <dl class="field-list simple">
@@ -1074,10 +1075,10 @@ current Dataset (if inplace=True) where the instances are stored in a <cite>csr_
 
 </section>
 <section id="module-quapy.data.reader">
-<span id="quapy-data-reader-module"></span><h2>quapy.data.reader module<a class="headerlink" href="#module-quapy.data.reader" title="Link to this heading"></a></h2>
+<span id="quapy-data-reader-module"></span><h2>quapy.data.reader module<a class="headerlink" href="#module-quapy.data.reader" title="Permalink to this heading"></a></h2>
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.reader.binarize">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">binarize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">y</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pos_class</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#binarize"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.binarize" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">binarize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">y</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pos_class</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#binarize"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.binarize" title="Permalink to this definition"></a></dt>
 <dd><p>Binarizes a categorical array-like collection of labels towards the positive class <cite>pos_class</cite>. E.g.,:</p>
 <div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">binarize</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="n">pos_class</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
 <span class="gp">&gt;&gt;&gt; </span><span class="n">array</span><span class="p">([</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">])</span>
@@ -1099,7 +1100,7 @@ current Dataset (if inplace=True) where the instances are stored in a <cite>csr_
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.reader.from_csv">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">from_csv</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'utf-8'</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#from_csv"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.from_csv" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">from_csv</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'utf-8'</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#from_csv"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.from_csv" title="Permalink to this definition"></a></dt>
 <dd><p>Reads a csv file in which columns are separated by ‘,’.
 File format &lt;label&gt;,&lt;feat1&gt;,&lt;feat2&gt;,…,&lt;featn&gt;</p>
 <dl class="field-list simple">
@@ -1117,7 +1118,7 @@ File format &lt;label&gt;,&lt;feat1&gt;,&lt;feat2&gt;,…,&lt;featn&gt;</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.reader.from_sparse">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">from_sparse</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#from_sparse"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.from_sparse" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">from_sparse</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#from_sparse"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.from_sparse" title="Permalink to this definition"></a></dt>
 <dd><p>Reads a labelled collection of real-valued instances expressed in sparse format
 File format &lt;-1 or 0 or 1&gt;[s col(int):val(float)]</p>
 <dl class="field-list simple">
@@ -1132,7 +1133,7 @@ File format &lt;-1 or 0 or 1&gt;[s col(int):val(float)]</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.reader.from_text">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">from_text</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'utf-8'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">class2int</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#from_text"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.from_text" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">from_text</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'utf-8'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">class2int</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#from_text"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.from_text" title="Permalink to this definition"></a></dt>
 <dd><p>Reads a labelled colletion of documents.
 File fomart &lt;0 or 1&gt;        &lt;document&gt;</p>
 <dl class="field-list simple">
@@ -1151,7 +1152,7 @@ File fomart &lt;0 or 1&gt;        &lt;document&gt;</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="quapy.data.reader.reindex_labels">
-<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">reindex_labels</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#reindex_labels"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.reindex_labels" title="Link to this definition"></a></dt>
+<span class="sig-prename descclassname"><span class="pre">quapy.data.reader.</span></span><span class="sig-name descname"><span class="pre">reindex_labels</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">y</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="_modules/quapy/data/reader.html#reindex_labels"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#quapy.data.reader.reindex_labels" title="Permalink to this definition"></a></dt>
 <dd><p>Re-indexes a list of labels as a list of indexes, and returns the classnames corresponding to the indexes.
 E.g.:</p>
 <div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">reindex_labels</span><span class="p">([</span><span class="s1">&#39;B&#39;</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">,</span> <span class="s1">&#39;A&#39;</span><span class="p">,</span> <span class="s1">&#39;C&#39;</span><span class="p">])</span>
@@ -1170,7 +1171,7 @@ E.g.:</p>
 
 </section>
 <section id="module-quapy.data">
-<span id="module-contents"></span><h2>Module contents<a class="headerlink" href="#module-quapy.data" title="Link to this heading"></a></h2>
+<span id="module-contents"></span><h2>Module contents<a class="headerlink" href="#module-quapy.data" title="Permalink to this heading"></a></h2>
 </section>
 </section>
 

docs/build/html/search.html

@@ -1,11 +1,11 @@
 <!DOCTYPE html>
-<html class="writer-html5" lang="en" data-content_root="./">
+<html class="writer-html5" lang="en">
 <head>
   <meta charset="utf-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>Search &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.8 documentation</title>
-      <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=92fd9be5" />
-      <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+  <title>Search &mdash; QuaPy: A Python-based open-source framework for quantification 0.1.9 documentation</title>
+      <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
+      <link rel="stylesheet" type="text/css" href="_static/css/theme.css" />
 
   
     
@@ -13,11 +13,12 @@
     <script src="_static/js/html5shiv.min.js"></script>
   <![endif]-->
   
-        <script src="_static/jquery.js?v=5d32c60e"></script>
-        <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
-        <script src="_static/documentation_options.js?v=22607128"></script>
-        <script src="_static/doctools.js?v=9a2dae69"></script>
-        <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
+        <script src="_static/jquery.js"></script>
+        <script src="_static/underscore.js"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
+        <script src="_static/doctools.js"></script>
+        <script src="_static/sphinx_highlight.js"></script>
     <script src="_static/js/theme.js"></script>
     <script src="_static/searchtools.js"></script>
     <script src="_static/language_data.js"></script>

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/source/conf.py

@@ -0,0 +1,55 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+import pathlib
+import sys
+from os.path import join
+quapy_path = join(pathlib.Path(__file__).parents[2].resolve().as_posix(), 'quapy')
+print(f'quapy path={quapy_path}')
+sys.path.insert(0, quapy_path)
+
+
+project = 'QuaPy: A Python-based open-source framework for quantification'
+copyright = '2024, Alejandro Moreo'
+author = 'Alejandro Moreo'
+
+
+
+import quapy
+
+release = quapy.__version__
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = [
+    'sphinx.ext.duration',
+    'sphinx.ext.doctest',
+    'sphinx.ext.autodoc',
+    'sphinx.ext.autosummary',
+    'sphinx.ext.viewcode',
+    'sphinx.ext.napoleon'
+]
+
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = 'sphinx_rtd_theme'
+# html_theme = 'furo'
+# need to be installed: pip install furo (not working...)
+html_static_path = ['_static']
+
+

docs/source/index.rst

@@ -0,0 +1,41 @@
+.. QuaPy: A Python-based open-source framework for quantification documentation master file, created by
+   sphinx-quickstart on Wed Feb  7 16:26:46 2024.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to QuaPy's documentation!
+==========================================================================================
+
+QuaPy is a Python-based open-source framework for quantification.
+
+This document contains the API of the modules included in QuaPy.
+
+Installation
+------------
+
+`pip install quapy`
+
+GitHub
+------------
+
+QuaPy is hosted in GitHub at `https://github.com/HLT-ISTI/QuaPy <https://github.com/HLT-ISTI/QuaPy>`_
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+Contents
+--------
+
+.. toctree::
+
+   modules
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

docs/source/modules.rst

@@ -0,0 +1,7 @@
+quapy
+=====
+
+.. toctree::
+   :maxdepth: 4
+
+   quapy

docs/source/quapy.classification.rst

@@ -0,0 +1,45 @@
+quapy.classification package
+============================
+
+Submodules
+----------
+
+quapy.classification.calibration module
+---------------------------------------
+
+.. automodule:: quapy.classification.calibration
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.classification.methods module
+-----------------------------------
+
+.. automodule:: quapy.classification.methods
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.classification.neural module
+----------------------------------
+
+.. automodule:: quapy.classification.neural
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.classification.svmperf module
+-----------------------------------
+
+.. automodule:: quapy.classification.svmperf
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: quapy.classification
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/quapy.data.rst

@@ -0,0 +1,46 @@
+quapy.data package
+==================
+
+Submodules
+----------
+
+quapy.data.base module
+----------------------
+
+.. automodule:: quapy.data.base
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.data.datasets module
+--------------------------
+
+.. automodule:: quapy.data.datasets
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+
+quapy.data.preprocessing module
+-------------------------------
+
+.. automodule:: quapy.data.preprocessing
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.data.reader module
+------------------------
+
+.. automodule:: quapy.data.reader
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: quapy.data
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/quapy.method.rst

@@ -0,0 +1,61 @@
+quapy.method package
+====================
+
+Submodules
+----------
+
+quapy.method.aggregative module
+-------------------------------
+
+.. automodule:: quapy.method.aggregative
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+.. automodule:: quapy.method._kdey
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+.. automodule:: quapy.method._neural
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+.. automodule:: quapy.method._threshold_optim
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+
+quapy.method.base module
+------------------------
+
+.. automodule:: quapy.method.base
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.method.meta module
+------------------------
+
+.. automodule:: quapy.method.meta
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.method.non\_aggregative module
+------------------------------------
+
+.. automodule:: quapy.method.non_aggregative
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: quapy.method
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/quapy.rst

@@ -0,0 +1,80 @@
+quapy package
+=============
+
+Subpackages
+-----------
+
+.. toctree::
+   :maxdepth: 4
+
+   quapy.classification
+   quapy.data
+   quapy.method
+
+
+Submodules
+----------
+
+quapy.error module
+------------------
+
+.. automodule:: quapy.error
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.evaluation module
+-----------------------
+
+.. automodule:: quapy.evaluation
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.functional module
+-----------------------
+
+.. automodule:: quapy.functional
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.model\_selection module
+-----------------------------
+
+.. automodule:: quapy.model_selection
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.plot module
+-----------------
+
+.. automodule:: quapy.plot
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.protocol module
+---------------------
+
+.. automodule:: quapy.protocol
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+quapy.util module
+-----------------
+
+.. automodule:: quapy.util
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: quapy
+   :members:
+   :undoc-members:
+   :show-inheritance:

examples/bayesian_quantification.py

@@ -0,0 +1,194 @@
+"""
+.. author:: Paweł Czyż
+
+This example shows how to use Bayesian quantification (https://arxiv.org/abs/2302.09159),
+which is suitable for low-data situations and when the uncertainty of the prevalence estimate is of interest.
+
+For this, we will need to install extra dependencies:
+
+```
+$ pip install quapy[bayesian]
+```
+
+Running the script via:
+
+```
+$ python examples/bayesian_quantification.py
+```
+
+will produce a plot `bayesian_quantification.pdf`.
+
+Due to a low sample size and the fact that classes 2 and 3 are hard to distinguish,
+it is hard to estimate the proportions accurately, what is visible by looking at the posterior samples,
+showing large uncertainty.
+"""
+
+import numpy as np
+import matplotlib.pyplot as plt
+import quapy as qp
+
+from sklearn.ensemble import RandomForestClassifier
+
+from quapy.method.aggregative import BayesianCC, ACC, PACC
+from quapy.data import LabelledCollection, Dataset
+
+
+FIGURE_PATH = "bayesian_quantification.pdf"
+
+
+def simulate_data(rng) -> Dataset:
+    """Generates a simulated data set with three classes."""
+
+    # Number of examples of each class in both data sets
+    n_train = [400, 400, 400]
+    n_test = [40, 25, 15]
+
+    # Mean vectors and shared covariance of P(X|Y) distributions
+    mus = [np.zeros(2), np.array([1, 1.5]), np.array([1.5, 1])]
+    cov = np.eye(2)
+
+    def gen_Xy(centers, sizes):
+        X = np.concatenate([rng.multivariate_normal(mu_i, cov, size_i) for mu_i, size_i in zip(centers, sizes)])
+        y = np.concatenate([[i] * n for i, n in enumerate(sizes)])
+        return X, y
+
+    # Generate the features accordingly
+    train = LabelledCollection(*gen_Xy(centers=mus, sizes=n_train))
+    test  = LabelledCollection(*gen_Xy(centers=mus, sizes=n_test))
+    
+    return Dataset(training=train, test=test)
+
+
+def plot_simulated_data(axs, data: Dataset) -> None:
+    """Plots a simulated data set.
+
+    :param axs: a list of three `plt.Axes` objects, on which the samples will be plotted.
+    :param data: the simulated data set.
+    """
+    train, test = data.train_test
+    xlim = (
+        -0.3 + min(train.X[:, 0].min(), test.X[:, 0].min()),
+        0.3 + max(train.X[:, 0].max(), test.X[:, 0].max())
+    )
+    ylim = (
+        -0.3 + min(train.X[:, 1].min(), test.X[:, 1].min()),
+        0.3 + max(train.X[:, 1].max(), test.X[:, 1].max())
+    )
+
+    for ax in axs:
+        ax.set_xlabel("$X_1$")
+        ax.set_ylabel("$X_2$")
+        ax.set_aspect("equal")
+        ax.set_xlim(*xlim)
+        ax.set_ylim(*ylim)
+        ax.set_xticks([])
+        ax.set_yticks([])
+
+    ax = axs[0]
+    ax.set_title("Training set")
+    for i in range(data.n_classes):
+        ax.scatter(train.X[train.y == i, 0], train.X[train.y == i, 1], c=f"C{i}", s=3, rasterized=True)
+
+    ax = axs[1]
+    ax.set_title("Test set\n(with labels)")
+    for i in range(data.n_classes):
+        ax.scatter(test.X[test.y == i, 0], test.X[test.y == i, 1], c=f"C{i}", s=3, rasterized=True)
+
+    ax = axs[2]
+    ax.set_title("Test set\n(as observed)")
+    ax.scatter(test.X[:, 0], test.X[:, 1], c="C5", s=3, rasterized=True)
+
+
+def plot_true_proportions(ax: plt.Axes, test_prevalence: np.ndarray) -> None:
+    """Plots the true proportions."""
+    n_classes = len(test_prevalence)
+    x_ax = np.arange(n_classes)
+    ax.plot(x_ax, test_prevalence, c="black", linewidth=2, label="True")
+
+    ax.set_xlabel("Class")
+    ax.set_ylabel("Prevalence")
+    ax.set_xticks(x_ax, x_ax + 1)
+    ax.set_yticks([0, 0.25, 0.5, 0.75, 1.0])
+    ax.set_xlim(-0.1, n_classes - 0.9)
+    ax.set_ylim(-0.01, 1.01)
+
+
+def get_random_forest() -> RandomForestClassifier:
+    """An auxiliary factory method to generate a random forest."""
+    return RandomForestClassifier(n_estimators=10, random_state=5)    
+
+
+def _get_estimate(estimator_class, training: LabelledCollection, test: np.ndarray) -> None:
+    """Auxiliary method for running ACC and PACC."""
+    estimator = estimator_class(get_random_forest())
+    estimator.fit(training)
+    return estimator.quantify(test)
+
+
+def train_and_plot_bayesian_quantification(ax: plt.Axes, training: LabelledCollection, test: LabelledCollection) -> None:
+    """Fits Bayesian quantification and plots posterior mean as well as individual samples"""
+    print('training model Bayesian CC...', end='')
+    quantifier = BayesianCC(classifier=get_random_forest())
+    quantifier.fit(training)
+
+    # Obtain mean prediction
+    mean_prediction = quantifier.quantify(test.X)
+    mae = qp.error.mae(test.prevalence(), mean_prediction)
+    x_ax = np.arange(training.n_classes)
+    ax.plot(x_ax, mean_prediction, c="salmon", linewidth=2, linestyle=":", label="Bayesian")
+
+    # Obtain individual samples 
+    samples = quantifier.get_prevalence_samples()
+    for sample in samples[::5, :]:
+        ax.plot(x_ax, sample, c="salmon", alpha=0.1, linewidth=0.3, rasterized=True)
+    print(f'MAE={mae:.4f} [done]')
+
+
+def train_and_plot_acc(ax: plt.Axes, training: LabelledCollection, test: LabelledCollection) -> None:
+    print('training model ACC...', end='')
+    estimate = _get_estimate(ACC, training, test.X)
+    mae = qp.error.mae(test.prevalence(), estimate)
+    ax.plot(np.arange(training.n_classes), estimate, c="darkblue", linewidth=2, linestyle=":", label="ACC")
+    print(f'MAE={mae:.4f} [done]')
+
+
+def train_and_plot_pacc(ax: plt.Axes, training: LabelledCollection, test: LabelledCollection) -> None:
+    print('training model PACC...', end='')
+    estimate = _get_estimate(PACC, training, test.X)
+    mae = qp.error.mae(test.prevalence(), estimate)
+    ax.plot(np.arange(training.n_classes), estimate, c="limegreen", linewidth=2, linestyle=":", label="PACC")
+    print(f'MAE={mae:.4f} [done]')
+
+
+def main() -> None:
+    # --- Simulate data ---
+    print('generating simulated data')
+    rng = np.random.default_rng(42)
+    data = simulate_data(rng)
+    training, test = data.train_test
+
+    # --- Plot simulated data ---
+    fig, axs = plt.subplots(1, 4, figsize=(13, 3), dpi=300)
+    for ax in axs:
+        ax.spines[['top', 'right']].set_visible(False)
+    plot_simulated_data(axs[:3], data)
+
+    # --- Plot quantification results ---
+    ax = axs[3]
+    plot_true_proportions(ax, test_prevalence=test.prevalence())
+
+    train_and_plot_acc(ax, training=training, test=test)
+    train_and_plot_pacc(ax, training=training, test=test)
+    train_and_plot_bayesian_quantification(ax=ax, training=training, test=test)
+    print('[done]')
+
+    ax.legend(bbox_to_anchor=(1.05, 1), loc='upper left', frameon=False)
+
+    print(f'saving plot in path {FIGURE_PATH}...', end='')
+    fig.tight_layout()
+    fig.savefig(FIGURE_PATH)
+    print('[done]')
+
+
+if __name__ == '__main__':
+    main()

examples/custom_quantifier.py

@@ -1,33 +1,79 @@
 import quapy as qp
 from quapy.data import LabelledCollection
-from quapy.method.base import BinaryQuantifier
+from quapy.method.base import BinaryQuantifier, BaseQuantifier
 from quapy.model_selection import GridSearchQ
 from quapy.method.aggregative import AggregativeSoftQuantifier
 from quapy.protocol import APP
 import numpy as np
 from sklearn.linear_model import LogisticRegression
+from time import time
 
 
 # Define a custom quantifier: for this example, we will consider a new quantification algorithm that uses a
 # logistic regressor for generating posterior probabilities, and then applies a custom threshold value to the
 # posteriors. Since the quantifier internally uses a classifier, it is an aggregative quantifier; and since it
-# relies on posterior probabilities, it is a probabilistic-aggregative quantifier. Note also it has an
-# internal hyperparameter (let say, alpha) which is the decision threshold. Let's also assume the quantifier
-# is binary, for simplicity.
+# relies on posterior probabilities, it is a probabilistic-aggregative quantifier (aka AggregativeSoftQuantifier).
+# Note also it has an internal hyperparameter (let say, alpha) which is the decision threshold.
+#
+# Let's also assume the quantifier is binary, for simplicity. Any quantifier (i.e., any subclass of BaseQuantifier)
+# is required to implement the "fit" and "quantify" methods. Aggregative quantifiers are special subtypes of base
+# quantifiers, i.e., are quantifiers that undertake a classification-phase followed by an aggregation-phase. QuaPy
+# already implements most common functionality, and requires the developer to simply implement the "aggregation_fit"
+# and the "aggregation" methods.
+#
+# We are providing two implementations of the same method to illustrate this characteristic of QuaPy. Let us begin
+# with the general case, in which we implement a (base) quantifier
+
+class MyQuantifier(BaseQuantifier):
 
-class MyQuantifier(AggregativeSoftQuantifier, BinaryQuantifier):
     def __init__(self, classifier, alpha=0.5):
         self.alpha = alpha
-        # aggregative quantifiers have an internal self.classifier attribute
         self.classifier = classifier
 
-    def fit(self, data: LabelledCollection, fit_classifier=True):
-        assert fit_classifier, 'this quantifier needs to fit the classifier!'
+    # in general, we would need to implement the method fit(self, data: LabelledCollection, fit_classifier=True,
+    # val_split=None); this would amount to:
+    def fit(self, data: LabelledCollection):
+        assert data.n_classes==2, \
+            'this quantifier is only valid for binary problems [abort]'
         self.classifier.fit(*data.Xy)
         return self
 
-    # in general, we would need to implement the method quantify(self, instances) but, since this method is of
-    # type aggregative, we can simply implement the method aggregate, which has the following interface
+    # in general, we would need to implement the method quantify(self, instances); this would amount to:
+    def quantify(self, instances):
+        assert hasattr(self.classifier, 'predict_proba'), \
+            'the underlying classifier is not probabilistic! [abort]'
+        posterior_probabilities = self.classifier.predict_proba(instances)
+        positive_probabilities = posterior_probabilities[:, 1]
+        crisp_decisions = positive_probabilities > self.alpha
+        pos_prev = crisp_decisions.mean()
+        neg_prev = 1 - pos_prev
+        return np.asarray([neg_prev, pos_prev])
+
+
+# Note that the above implementation contains a lot of boilerplate code. Many parts can be omitted since QuaPy
+# provides implementations for them. Some of these routines (like, for example, training a classifier and generating
+# posterior probabilities) are often carried out in a k-fold cross-validation manner. These, along with many other
+# common routines are already provided by highly-optimized routines in QuaPy. Let's see a much better implementation
+# of the method, now adhering to the AggregativeSoftQuantifier:
+
+class MyAggregativeSoftQuantifier(AggregativeSoftQuantifier, BinaryQuantifier):
+    def __init__(self, classifier, alpha=0.5):
+        # aggregative quantifiers have an internal attribute called self.classifier
+        self.classifier = classifier
+        self.alpha = alpha
+
+    # since this method is of type aggregative, we can simply implement the method aggregation_fit, which
+    # assumes the classifier has already been fitted properly and the predictions for the training set required
+    # to train the aggregation function have been properly generated (i.e., on a validation split, or using a
+    # k-fold cross validation strategy). What remains ahead is to learn an aggregation function. In our case
+    # this amounts to doing... nothing, since our method was pretty basic. BinaryQuantifier also add some
+    # basic functionality for checking binary consistency.
+    def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
+        pass
+
+    # since this method is of type aggregative, we can simply implement the method aggregate (i.e., we should
+    # only describe what to do with the classifier predictions --which in this case are posterior probabilities
+    # because we are inheriting from the "Soft" subtype). This comes down to:
     def aggregate(self, classif_predictions: np.ndarray):
         # the posterior probabilities have already been generated by the quantify method; we only need to
         # specify what to do with them
@@ -38,31 +84,68 @@ class MyQuantifier(AggregativeSoftQuantifier, BinaryQuantifier):
         return np.asarray([neg_prev, pos_prev])
 
 
+# a small example using these two implementations of our method
+
 if __name__ == '__main__':
 
-    qp.environ['SAMPLE_SIZE'] = 100
-
-    # define an instance of our custom quantifier
-    quantifier = MyQuantifier(LogisticRegression(), alpha=0.5)
+    qp.environ['SAMPLE_SIZE'] = 250
 
     # load the IMDb dataset
     train, test = qp.datasets.fetch_reviews('imdb', tfidf=True, min_df=5).train_test
+    train, val = train.split_stratified(train_prop=0.75)  # let's create a validation set for optimizing hyperparams
 
-    # model selection
-    # let us assume we want to explore our hyperparameter alpha along with one hyperparameter of the classifier
-    train, val = train.split_stratified(train_prop=0.75)
-    param_grid = {
-        'alpha': np.linspace(0, 1, 11),         # quantifier-dependent hyperparameter
-        'classifier__C': np.logspace(-2, 2, 5)  # classifier-dependent hyperparameter
-    }
-    quantifier = GridSearchQ(quantifier, param_grid, protocol=APP(val), n_jobs=-1, verbose=True).fit(train)
+    def test_implementation(quantifier):
+        class_name = quantifier.__class__.__name__
+        print(f'\ntesting implementation {class_name}...')
+        # model selection
+        # let us assume we want to explore our hyperparameter alpha along with one hyperparameter of the classifier
+        tinit = time()
+        param_grid = {
+            'alpha': np.linspace(0, 1, 11),         # quantifier-dependent hyperparameter
+            'classifier__C': np.logspace(-2, 2, 5)  # classifier-dependent hyperparameter
+        }
+        gridsearch = GridSearchQ(quantifier, param_grid, protocol=APP(val), n_jobs=-1, verbose=False).fit(train)
+        t_modsel = time() - tinit
+        print(f'\tmodel selection took {t_modsel:.2f}s', flush=True)
 
-    # evaluation
-    mae = qp.evaluation.evaluate(quantifier, protocol=APP(test), error_metric='mae')
+        # evaluation
+        optimized_model = gridsearch.best_model_
+        mae = qp.evaluation.evaluate(
+            optimized_model,
+            protocol=APP(test, repeats=5000, sanity_check=None),  # disable the check, we want to generate many tests!
+            error_metric='mae',
+            verbose=True)
 
-    print(f'MAE = {mae:.4f}')
+        t_eval = time() - t_modsel - tinit
+        print(f'\tevaluation took {t_eval:.2f}s [MAE = {mae:.4f}]')
 
-    # final remarks: this method is only for demonstration purposes and makes little sense in general. The method relies
+    # define an instance of our custom quantifier and test it!
+    quantifier = MyQuantifier(LogisticRegression(), alpha=0.5)
+    test_implementation(quantifier)
+
+    # define an instance of our custom quantifier, with the second implementation, and test it!
+    quantifier = MyAggregativeSoftQuantifier(LogisticRegression(), alpha=0.5)
+    test_implementation(quantifier)
+
+    # the output should look like this:
+    """
+    testing implementation MyQuantifier...
+        model selection took 12.86s
+    predicting: 100%|██████████| 105000/105000 [00:22<00:00, 4626.30it/s]
+        evaluation took 22.75s [MAE = 0.0630]
+    
+    testing implementation MyAggregativeSoftQuantifier...
+        model selection took 3.10s
+    speeding up the prediction for the aggregative quantifier, total classifications 25000 instead of 26250000
+    predicting: 100%|██████████| 105000/105000 [00:04<00:00, 22779.62it/s]
+        evaluation took 4.66s [MAE = 0.0630]
+    """
+    # Note that the first implementation is much slower, both in terms of grid-search optimization and in terms of
+    # evaluation. The reason why is that QuaPy is highly optimized for aggregative quantifiers (by far, the most
+    # popular type of quantification methods), thus significantly speeding up model selection and test routines.
+    # Furthermore, it is simpler to extend an aggregation type since QuaPy implements boilerplate functions for you.
+
+    # Final remarks: this method is only for demonstration purposes and makes little sense in general. The method relies
     # on an hyperparameter alpha for binarizing the posterior probabilities. A much better way for fulfilling this
     # goal would be to calibrate the classifier (LogisticRegression is already reasonably well calibrated) and then
     # simply cut at 0.5.

quapy/functional.py

@@ -1,52 +1,51 @@
 import itertools
+import warnings
 from collections import defaultdict
-from typing import Union, Callable
+from typing import Literal, Union, Callable
+from numpy.typing import ArrayLike
 
 import scipy
 import numpy as np
 
 
-def prevalence_linspace(n_prevalences=21, repeats=1, smooth_limits_epsilon=0.01):
+# ------------------------------------------------------------------------------------------
+# Counter utils
+# ------------------------------------------------------------------------------------------
+
+def counts_from_labels(labels: ArrayLike, classes: ArrayLike) -> np.ndarray:
     """
-    Produces an array of uniformly separated values of prevalence.
-    By default, produces an array of 21 prevalence values, with
-    step 0.05 and with the limits smoothed, i.e.:
-    [0.01, 0.05, 0.10, 0.15, ..., 0.90, 0.95, 0.99]
+    Computes the raw count values from a vector of labels.
 
-    :param n_prevalences: the number of prevalence values to sample from the [0,1] interval (default 21)
-    :param repeats: number of times each prevalence is to be repeated (defaults to 1)
-    :param smooth_limits_epsilon: the quantity to add and subtract to the limits 0 and 1
-    :return: an array of uniformly separated prevalence values
-    """
-    p = np.linspace(0., 1., num=n_prevalences, endpoint=True)
-    p[0] += smooth_limits_epsilon
-    p[-1] -= smooth_limits_epsilon
-    if p[0] > p[1]:
-        raise ValueError(f'the smoothing in the limits is greater than the prevalence step')
-    if repeats > 1:
-        p = np.repeat(p, repeats)
-    return p
-
-
-def prevalence_from_labels(labels, classes):
-    """
-    Computed the prevalence values from a vector of labels.
-
-    :param labels: array-like of shape `(n_instances)` with the label for each instance
+    :param labels: array-like of shape `(n_instances,)` with the label for each instance
     :param classes: the class labels. This is needed in order to correctly compute the prevalence vector even when
         some classes have no examples.
-    :return: an ndarray of shape `(len(classes))` with the class prevalence values
+    :return: ndarray of shape `(len(classes),)` with the raw counts for each class, in the same order
+        as they appear in `classes`
     """
-    if labels.ndim != 1:
+    if np.asarray(labels).ndim != 1:
         raise ValueError(f'param labels does not seem to be a ndarray of label predictions')
     unique, counts = np.unique(labels, return_counts=True)
     by_class = defaultdict(lambda:0, dict(zip(unique, counts)))
-    prevalences = np.asarray([by_class[class_] for class_ in classes], dtype=float)
-    prevalences /= prevalences.sum()
+    counts = np.asarray([by_class[class_] for class_ in classes], dtype=int)
+    return counts
+
+
+def prevalence_from_labels(labels: ArrayLike, classes: ArrayLike):
+    """
+    Computes the prevalence values from a vector of labels.
+
+    :param labels: array-like of shape `(n_instances,)` with the label for each instance
+    :param classes: the class labels. This is needed in order to correctly compute the prevalence vector even when
+        some classes have no examples.
+    :return: ndarray of shape `(len(classes),)` with the class proportions for each class, in the same order
+        as they appear in `classes`
+    """
+    counts = counts_from_labels(labels, classes)
+    prevalences = counts.astype(float) / np.sum(counts)
     return prevalences
 
 
-def prevalence_from_probabilities(posteriors, binarize: bool = False):
+def prevalence_from_probabilities(posteriors: ArrayLike, binarize: bool = False):
     """
     Returns a vector of prevalence values from a matrix of posterior probabilities.
 
@@ -55,8 +54,9 @@ def prevalence_from_probabilities(posteriors, binarize: bool = False):
         converting the vectors of posterior probabilities into class indices, by taking the argmax).
     :return: array of shape `(n_classes,)` containing the prevalence values
     """
+    posteriors = np.asarray(posteriors)
     if posteriors.ndim != 2:
-        raise ValueError(f'param posteriors does not seem to be a ndarray of posteior probabilities')
+        raise ValueError(f'param posteriors does not seem to be a ndarray of posterior probabilities')
     if binarize:
         predictions = np.argmax(posteriors, axis=-1)
         return prevalence_from_labels(predictions, np.arange(posteriors.shape[1]))
@@ -66,25 +66,264 @@ def prevalence_from_probabilities(posteriors, binarize: bool = False):
         return prevalences
 
 
-def as_binary_prevalence(positive_prevalence: Union[float, np.ndarray], clip_if_necessary=False):
+def num_prevalence_combinations(n_prevpoints:int, n_classes:int, n_repeats:int=1) -> int:
+    """
+    Computes the number of valid prevalence combinations in the n_classes-dimensional simplex if `n_prevpoints` equally
+    distant prevalence values are generated and `n_repeats` repetitions are requested.
+    The computation comes down to calculating:
+
+    .. math::
+        \\binom{N+C-1}{C-1} \\times r
+
+    where `N` is `n_prevpoints-1`, i.e., the number of probability mass blocks to allocate, `C` is the number of
+    classes, and `r` is `n_repeats`. This solution comes from the
+    `Stars and Bars <https://brilliant.org/wiki/integer-equations-star-and-bars/>`_ problem.
+
+    :param int n_classes: number of classes
+    :param int n_prevpoints: number of prevalence points.
+    :param int n_repeats: number of repetitions for each prevalence combination
+    :return: The number of possible combinations. For example, if `n_classes`=2, `n_prevpoints`=5, `n_repeats`=1,
+        then the number of possible combinations are 5, i.e.: [0,1], [0.25,0.75], [0.50,0.50], [0.75,0.25],
+        and [1.0,0.0]
+    """
+    N = n_prevpoints-1
+    C = n_classes
+    r = n_repeats
+    return int(scipy.special.binom(N + C - 1, C - 1) * r)
+
+
+def get_nprevpoints_approximation(combinations_budget:int, n_classes:int, n_repeats:int=1) -> int:
+    """
+    Searches for the largest number of (equidistant) prevalence points to define for each of the `n_classes` classes so
+    that the number of valid prevalence values generated as combinations of prevalence points (points in a
+    `n_classes`-dimensional simplex) do not exceed combinations_budget.
+
+    :param int combinations_budget: maximum number of combinations allowed
+    :param int n_classes: number of classes
+    :param int n_repeats: number of repetitions for each prevalence combination
+    :return: the largest number of prevalence points that generate less than combinations_budget valid prevalences
+    """
+    assert n_classes > 0 and n_repeats > 0 and combinations_budget > 0, 'parameters must be positive integers'
+    n_prevpoints = 1
+    while True:
+        combinations = num_prevalence_combinations(n_prevpoints, n_classes, n_repeats)
+        if combinations > combinations_budget:
+            return n_prevpoints-1
+        else:
+            n_prevpoints += 1
+
+
+# ------------------------------------------------------------------------------------------
+# Prevalence vectors
+# ------------------------------------------------------------------------------------------
+
+def as_binary_prevalence(positive_prevalence: Union[float, ArrayLike], clip_if_necessary: bool=False) -> np.ndarray:
     """
     Helper that, given a float representing the prevalence for the positive class, returns a np.ndarray of two
     values representing a binary distribution.
 
-    :param positive_prevalence: prevalence for the positive class
-    :param clip_if_necessary: if True, clips the value in [0,1] in order to guarantee the resulting distribution
+    :param positive_prevalence: float or array-like of floats with the prevalence for the positive class
+    :param bool clip_if_necessary: if True, clips the value in [0,1] in order to guarantee the resulting distribution
         is valid. If False, it then checks that the value is in the valid range, and raises an error if not.
     :return: np.ndarray of shape `(2,)`
     """
+    positive_prevalence = np.asarray(positive_prevalence, float)
     if clip_if_necessary:
         positive_prevalence = np.clip(positive_prevalence, 0, 1)
     else:
-        assert 0 <= positive_prevalence <= 1, 'the value provided is not a valid prevalence for the positive class'
+        assert np.logical_and(0 <= positive_prevalence, positive_prevalence  <= 1).all(), \
+            'the value provided is not a valid prevalence for the positive class'
     return np.asarray([1-positive_prevalence, positive_prevalence]).T
 
 
+def strprev(prevalences: ArrayLike, prec: int=3) -> str:
+    """
+    Returns a string representation for a prevalence vector. E.g.,
 
-def HellingerDistance(P, Q) -> float:
+    >>> strprev([1/3, 2/3], prec=2)
+    >>> '[0.33, 0.67]'
+
+    :param prevalences: array-like of prevalence values
+    :param prec: int, indicates the float precision (number of decimal values to print)
+    :return: string
+    """
+    return '['+ ', '.join([f'{p:.{prec}f}' for p in prevalences]) + ']'
+
+
+def check_prevalence_vector(prevalences: ArrayLike, raise_exception: bool=False, tolerance: float=1e-08, aggr=True):
+    """
+    Checks that `prevalences` is a valid prevalence vector, i.e., it contains values in [0,1] and
+    the values sum up to 1. In other words, verifies that the `prevalences` vectors lies in the
+    probability simplex.
+
+    :param ArrayLike prevalences: the prevalence vector, or vectors, to check
+    :param bool raise_exception: whether to raise an exception if the vector (or any of the vectors) does
+        not lie in the simplex (default False)
+    :param float tolerance: error tolerance for the check `sum(prevalences) - 1 = 0`
+    :param bool aggr: if True (default) returns one single bool (True if all prevalence vectors are valid,
+        False otherwise), if False returns an array of bool, one for each prevalence vector
+    :return: a single bool True if `prevalences` is a vector of prevalence values that lies on the simplex,
+        or False otherwise; alternatively, if `prevalences` is a matrix of shape `(num_vectors, n_classes,)`
+        then it returns one such bool for each prevalence vector
+    """
+    prevalences = np.asarray(prevalences)
+
+    all_positive = prevalences>=0
+    if not all_positive.all():
+        if raise_exception:
+            raise ValueError('some prevalence vectors contain negative numbers; '
+                             'consider using the qp.functional.normalize_prevalence with '
+                             'any method from ["clip", "mapsimplex", "softmax"]')
+
+    all_close_1 = np.isclose(prevalences.sum(axis=-1), 1, atol=tolerance)
+    if not all_close_1.all():
+        if raise_exception:
+            raise ValueError('some prevalence vectors do not sum up to 1; '
+                             'consider using the qp.functional.normalize_prevalence with '
+                             'any method from ["l1", "clip", "mapsimplex", "softmax"]')
+
+    valid = np.logical_and(all_positive.all(axis=-1), all_close_1)
+    if aggr:
+        return valid.all()
+    else:
+        return valid
+
+
+def normalize_prevalence(prevalences: ArrayLike, method='l1'):
+    """
+    Normalizes a vector or matrix of prevalence values. The normalization consists of applying a L1 normalization in
+    cases in which the prevalence values are not all-zeros, and to convert the prevalence values into `1/n_classes` in
+    cases in which all values are zero.
+
+    :param prevalences: array-like of shape `(n_classes,)` or of shape `(n_samples, n_classes,)` with prevalence values
+    :param str method: indicates the normalization method to employ, options are:
+
+        * `l1`: applies L1 normalization (default); a 0 vector is mapped onto the uniform prevalence
+        * `clip`: clip values in [0,1] and then rescales so that the L1 norm is 1
+        * `mapsimplex`: projects vectors onto the probability simplex. This implementation relies on
+          `Mathieu Blondel's projection_simplex_sort <https://gist.github.com/mblondel/6f3b7aaad90606b98f71>`_
+        * `softmax`: applies softmax to all vectors
+        * `condsoftmax`: applies softmax only to invalid prevalence vectors
+
+    :return: a normalized vector or matrix of prevalence values
+    """
+    if method in ['none', None]:
+        return prevalences
+
+    prevalences = np.asarray(prevalences, dtype=float)
+
+    if method=='l1':
+        normalized = l1_norm(prevalences)
+        check_prevalence_vector(normalized, raise_exception=True)
+    elif method=='clip':
+        normalized = clip(prevalences)  # no need to check afterwards
+    elif method=='mapsimplex':
+        normalized = projection_simplex_sort(prevalences)
+    elif method=='softmax':
+        normalized = softmax(prevalences)
+    elif method=='condsoftmax':
+        normalized = condsoftmax(prevalences)
+    else:
+        raise ValueError(f'unknown {method=}, valid ones are ["l1", "clip", "mapsimplex", "softmax", "condsoftmax"]')
+
+    return normalized
+
+
+def l1_norm(prevalences: ArrayLike) -> np.ndarray:
+    """
+    Applies L1 normalization to the `unnormalized_arr` so that it becomes a valid prevalence
+    vector. Zero vectors are mapped onto the uniform distribution. Raises an exception if
+    the resulting vectors are not valid distributions. This may happen when the original
+    prevalence vectors contain negative values. Use the `clip` normalization function
+    instead to avoid this possibility.
+
+    :param prevalences: array-like of shape `(n_classes,)` or of shape `(n_samples, n_classes,)` with prevalence values
+    :return: np.ndarray representing a valid distribution
+    """
+    n_classes = prevalences.shape[-1]
+    accum = prevalences.sum(axis=-1, keepdims=True)
+    prevalences = np.true_divide(prevalences, accum, where=accum > 0)
+    allzeros = accum.flatten() == 0
+    if any(allzeros):
+        if prevalences.ndim == 1:
+            prevalences = np.full(shape=n_classes, fill_value=1. / n_classes)
+        else:
+            prevalences[allzeros] = np.full(shape=n_classes, fill_value=1. / n_classes)
+    return prevalences
+
+
+def clip(prevalences: ArrayLike) -> np.ndarray:
+    """
+    Clips the values in [0,1] and then applies the L1 normalization.
+
+    :param prevalences: array-like of shape `(n_classes,)` or of shape `(n_samples, n_classes,)` with prevalence values
+    :return: np.ndarray representing a valid distribution
+    """
+    clipped = np.clip(prevalences, 0, 1)
+    normalized = l1_norm(clipped)
+    return normalized
+
+
+def projection_simplex_sort(unnormalized_arr: ArrayLike) -> np.ndarray:
+    """Projects a point onto the probability simplex.
+
+    The code is adapted from Mathieu Blondel's BSD-licensed
+    `implementation <https://gist.github.com/mblondel/6f3b7aaad90606b98f71>`_
+    (see function `projection_simplex_sort` in their repo) which is accompanying the paper
+
+    Mathieu Blondel, Akinori Fujino, and Naonori Ueda.
+    Large-scale Multiclass Support Vector Machine Training via Euclidean Projection onto the Simplex,
+    ICPR 2014, `URL <http://www.mblondel.org/publications/mblondel-icpr2014.pdf>`_
+
+    :param `unnormalized_arr`: point in n-dimensional space, shape `(n,)`
+    :return: projection of `unnormalized_arr` onto the (n-1)-dimensional probability simplex, shape `(n,)`
+    """
+    unnormalized_arr = np.asarray(unnormalized_arr)
+    n = len(unnormalized_arr)
+    u = np.sort(unnormalized_arr)[::-1]
+    cssv = np.cumsum(u) - 1.0
+    ind = np.arange(1, n + 1)
+    cond = u - cssv / ind > 0
+    rho = ind[cond][-1]
+    theta = cssv[cond][-1] / float(rho)
+    return np.maximum(unnormalized_arr - theta, 0)
+
+
+def softmax(prevalences: ArrayLike) -> np.ndarray:
+    """
+    Applies the softmax function to all vectors even if the original vectors were valid distributions.
+    If you want to leave valid vectors untouched, use condsoftmax instead.
+
+    :param prevalences: array-like of shape `(n_classes,)` or of shape `(n_samples, n_classes,)` with prevalence values
+    :return: np.ndarray representing a valid distribution
+    """
+    normalized = scipy.special.softmax(prevalences, axis=-1)
+    return normalized
+
+
+def condsoftmax(prevalences: ArrayLike) -> np.ndarray:
+    """
+    Applies the softmax function only to vectors that do not represent valid distributions.
+
+    :param prevalences: array-like of shape `(n_classes,)` or of shape `(n_samples, n_classes,)` with prevalence values
+    :return: np.ndarray representing a valid distribution
+    """
+    invalid_idx = ~ check_prevalence_vector(prevalences, aggr=False, raise_exception=False)
+    if isinstance(invalid_idx, np.bool_) and invalid_idx:
+        # only one vector
+        normalized = scipy.special.softmax(prevalences)
+    else:
+        prevalences = np.copy(prevalences)
+        prevalences[invalid_idx] = scipy.special.softmax(prevalences[invalid_idx], axis=-1)
+        normalized = prevalences
+    return normalized
+
+
+# ------------------------------------------------------------------------------------------
+# Divergences
+# ------------------------------------------------------------------------------------------
+
+def HellingerDistance(P: np.ndarray, Q: np.ndarray) -> float:
     """
     Computes the Hellingher Distance (HD) between (discretized) distributions `P` and `Q`.
     The HD for two discrete distributions of `k` bins is defined as:
@@ -99,7 +338,7 @@ def HellingerDistance(P, Q) -> float:
     return np.sqrt(np.sum((np.sqrt(P) - np.sqrt(Q))**2))
 
 
-def TopsoeDistance(P, Q, epsilon=1e-20):
+def TopsoeDistance(P: np.ndarray, Q: np.ndarray, epsilon: float=1e-20):
     """
     Topsoe distance between two (discretized) distributions `P` and `Q`.
     The Topsoe distance for two discrete distributions of `k` bins is defined as:
@@ -113,9 +352,132 @@ def TopsoeDistance(P, Q, epsilon=1e-20):
     :return: float
     """
     return np.sum(P*np.log((2*P+epsilon)/(P+Q+epsilon)) + Q*np.log((2*Q+epsilon)/(P+Q+epsilon)))
-                  
 
-def uniform_prevalence_sampling(n_classes, size=1):
+
+def get_divergence(divergence: Union[str, Callable]):
+    """
+    Guarantees that the divergence received as argument is a function. That is, if this argument is already
+    a callable, then it is returned, if it is instead a string, then tries to instantiate the corresponding
+    divergence from the string name.
+
+    :param divergence: callable or string indicating the name of the divergence function
+    :return: callable
+    """
+    if isinstance(divergence, str):
+        if divergence=='HD':
+            return HellingerDistance
+        elif divergence=='topsoe':
+            return TopsoeDistance
+        else:
+            raise ValueError(f'unknown divergence {divergence}')
+    elif callable(divergence):
+        return divergence
+    else:
+        raise ValueError(f'argument "divergence" not understood; use a str or a callable function')
+
+
+# ------------------------------------------------------------------------------------------
+# Solvers
+# ------------------------------------------------------------------------------------------
+
+def argmin_prevalence(loss: Callable,
+                      n_classes: int,
+                      method: Literal["optim_minimize", "linear_search", "ternary_search"]='optim_minimize'):
+    """
+    Searches for the prevalence vector that minimizes a loss function.
+
+    :param loss: callable, the function to minimize
+    :param n_classes: int, number of classes
+    :param method: string indicating the search strategy. Possible values are::
+        'optim_minimize': uses scipy.optim
+        'linear_search': carries out a linear search for binary problems in the space [0, 0.01, 0.02, ..., 1]
+        'ternary_search': implements the ternary search (not yet implemented)
+    :return: np.ndarray, a prevalence vector
+    """
+    if method == 'optim_minimize':
+        return optim_minimize(loss, n_classes)
+    elif method == 'linear_search':
+        return linear_search(loss, n_classes)
+    elif method == 'ternary_search':
+        ternary_search(loss, n_classes)
+    else:
+        raise NotImplementedError()
+
+
+def optim_minimize(loss: Callable, n_classes: int):
+    """
+    Searches for the optimal prevalence values, i.e., an `n_classes`-dimensional vector of the (`n_classes`-1)-simplex
+    that yields the smallest lost. This optimization is carried out by means of a constrained search using scipy's
+    SLSQP routine.
+
+    :param loss: (callable) the function to minimize
+    :param n_classes: (int) the number of classes, i.e., the dimensionality of the prevalence vector
+    :return: (ndarray) the best prevalence vector found
+    """
+    from scipy import optimize
+
+    # the initial point is set as the uniform distribution
+    uniform_distribution = np.full(fill_value=1 / n_classes, shape=(n_classes,))
+
+    # solutions are bounded to those contained in the unit-simplex
+    bounds = tuple((0, 1) for _ in range(n_classes))  # values in [0,1]
+    constraints = ({'type': 'eq', 'fun': lambda x: 1 - sum(x)})  # values summing up to 1
+    r = optimize.minimize(loss, x0=uniform_distribution, method='SLSQP', bounds=bounds, constraints=constraints)
+    return r.x
+
+
+def linear_search(loss: Callable, n_classes: int):
+    """
+    Performs a linear search for the best prevalence value in binary problems. The search is carried out by exploring
+    the range [0,1] stepping by 0.01. This search is inefficient, and is added only for completeness (some of the
+    early methods in quantification literature used it, e.g., HDy). A most powerful alternative is `optim_minimize`.
+
+    :param loss: (callable) the function to minimize
+    :param n_classes: (int) the number of classes, i.e., the dimensionality of the prevalence vector
+    :return: (ndarray) the best prevalence vector found
+    """
+    assert n_classes==2, 'linear search is only available for binary problems'
+
+    prev_selected, min_score = None, None
+    for prev in prevalence_linspace(grid_points=100, repeats=1, smooth_limits_epsilon=0.0):
+        score = loss(np.asarray([1 - prev, prev]))
+        if min_score is None or score < min_score:
+            prev_selected, min_score = prev, score
+
+    return np.asarray([1 - prev_selected, prev_selected])
+
+
+def ternary_search(loss: Callable, n_classes: int):
+    raise NotImplementedError()
+
+
+# ------------------------------------------------------------------------------------------
+# Sampling utils
+# ------------------------------------------------------------------------------------------
+
+def prevalence_linspace(grid_points:int=21, repeats:int=1, smooth_limits_epsilon:float=0.01) -> np.ndarray:
+    """
+    Produces an array of uniformly separated values of prevalence.
+    By default, produces an array of 21 prevalence values, with
+    step 0.05 and with the limits smoothed, i.e.:
+    [0.01, 0.05, 0.10, 0.15, ..., 0.90, 0.95, 0.99]
+
+    :param grid_points: the number of prevalence values to sample from the [0,1] interval (default 21)
+    :param repeats: number of times each prevalence is to be repeated (defaults to 1)
+    :param smooth_limits_epsilon: the quantity to add and subtract to the limits 0 and 1
+    :return: an array of uniformly separated prevalence values
+    """
+    p = np.linspace(0., 1., num=grid_points, endpoint=True)
+    p[0] += smooth_limits_epsilon
+    p[-1] -= smooth_limits_epsilon
+    if p[0] > p[1]:
+        raise ValueError(f'the smoothing in the limits is greater than the prevalence step')
+    if repeats > 1:
+        p = np.repeat(p, repeats)
+    return p
+
+
+def uniform_prevalence_sampling(n_classes: int, size: int=1) -> np.ndarray:
     """
     Implements the `Kraemer algorithm <http://www.cs.cmu.edu/~nasmith/papers/smith+tromble.tr04.pdf>`_
     for sampling uniformly at random from the unit simplex. This implementation is adapted from this
@@ -144,21 +506,11 @@ def uniform_prevalence_sampling(n_classes, size=1):
 uniform_simplex_sampling = uniform_prevalence_sampling
 
 
-def strprev(prevalences, prec=3):
-    """
-    Returns a string representation for a prevalence vector. E.g.,
+# ------------------------------------------------------------------------------------------
+# Adjustment
+# ------------------------------------------------------------------------------------------
 
-    >>> strprev([1/3, 2/3], prec=2)
-    >>> '[0.33, 0.67]'
-
-    :param prevalences: a vector of prevalence values
-    :param prec: float precision
-    :return: string
-    """
-    return '['+ ', '.join([f'{p:.{prec}f}' for p in prevalences]) + ']'
-
-
-def adjusted_quantification(prevalence_estim, tpr, fpr, clip=True):
+def solve_adjustment_binary(prevalence_estim: ArrayLike, tpr: float, fpr: float, clip: bool=True):
     """
     Implements the adjustment of ACC and PACC for the binary case. The adjustment for a prevalence estimate of the
     positive class `p` comes down to computing:
@@ -166,10 +518,10 @@ def adjusted_quantification(prevalence_estim, tpr, fpr, clip=True):
     .. math::
         ACC(p) = \\frac{ p - fpr }{ tpr - fpr }
 
-    :param prevalence_estim: float, the estimated value for the positive class
-    :param tpr: float, the true positive rate of the classifier
-    :param fpr: float, the false positive rate of the classifier
-    :param clip: set to True (default) to clip values that might exceed the range [0,1]
+    :param float prevalence_estim: the estimated value for the positive class (`p` in the formula)
+    :param float tpr: the true positive rate of the classifier
+    :param float fpr: the false positive rate of the classifier
+    :param bool clip: set to True (default) to clip values that might exceed the range [0,1]
     :return: float, the adjusted count
     """
 
@@ -182,184 +534,75 @@ def adjusted_quantification(prevalence_estim, tpr, fpr, clip=True):
     return adjusted
 
 
-def normalize_prevalence(prevalences):
+def solve_adjustment(
+    class_conditional_rates: np.ndarray,
+    unadjusted_counts: np.ndarray,
+    method: Literal["inversion", "invariant-ratio"],
+    solver: Literal["exact", "minimize", "exact-raise", "exact-cc"]) -> np.ndarray:
     """
-    Normalize a vector or matrix of prevalence values. The normalization consists of applying a L1 normalization in
-    cases in which the prevalence values are not all-zeros, and to convert the prevalence values into `1/n_classes` in
-    cases in which all values are zero.
+    Function that tries to solve for :math:`p` the equation :math:`q = M p`, where :math:`q` is the vector of
+    `unadjusted counts` (as estimated, e.g., via classify and count) with :math:`q_i` an estimate of
+    :math:`P(\hat{Y}=y_i)`, and where :math:`M` is the matrix of `class-conditional rates` with :math:`M_{ij}` an
+    estimate of :math:`P(\hat{Y}=y_i|Y=y_j)`.
 
-    :param prevalences: array-like of shape `(n_classes,)` or of shape `(n_samples, n_classes,)` with prevalence values
-    :return: a normalized vector or matrix of prevalence values
+    :param class_conditional_rates: array of shape `(n_classes, n_classes,)` with entry `(i,j)` being the estimate
+        of :math:`P(\hat{Y}=y_i|Y=y_j)`, that is, the probability that an instance that belongs to class :math:`y_j`
+        ends up being classified as belonging to class :math:`y_i`
+
+    :param unadjusted_counts: array of shape `(n_classes,)` containing the unadjusted prevalence values (e.g., as
+        estimated by CC or PCC)
+
+    :param str method: indicates the adjustment method to be used. Valid options are:
+
+        * `inversion`: tries to solve the equation :math:`q = M p` as :math:`p = M^{-1} q` where
+          :math:`M^{-1}` is the matrix inversion of :math:`M`. This inversion may not exist in
+          degenerated cases.
+        * `invariant-ratio`: invariant ratio estimator of `Vaz et al. 2018 <https://jmlr.org/papers/v20/18-456.html>`_,
+          which replaces the last equation in :math:`M` with the normalization condition (i.e., that the sum of
+          all prevalence values must equal 1).
+
+    :param str solver: the method to use for solving the system of linear equations. Valid options are:
+
+        * `exact-raise`: tries to solve the system using matrix inversion. Raises an error if the matrix has rank
+          strictly lower than `n_classes`.
+        * `exact-cc`: if the matrix is not full rank, returns :math:`q` (i.e., the unadjusted counts) as the estimates
+        * `exact`: deprecated, defaults to 'exact-cc' (will be removed in future versions)
+        * `minimize`: minimizes a loss, so the solution always exists
     """
-    prevalences = np.asarray(prevalences)
-    n_classes = prevalences.shape[-1]
-    accum = prevalences.sum(axis=-1, keepdims=True)
-    prevalences = np.true_divide(prevalences, accum, where=accum>0)
-    allzeros = accum.flatten()==0
-    if any(allzeros):
-        if prevalences.ndim == 1:
-            prevalences = np.full(shape=n_classes, fill_value=1./n_classes)
-        else:
-            prevalences[accum.flatten()==0] = np.full(shape=n_classes, fill_value=1./n_classes)
-    return prevalences
+    if solver == "exact":
+        warnings.warn(
+            "The 'exact' solver is deprecated. Use 'exact-raise' or 'exact-cc'", DeprecationWarning, stacklevel=2)
+        solver = "exact-cc"
 
+    A = np.asarray(class_conditional_rates, dtype=float)
+    B = np.asarray(unadjusted_counts, dtype=float)
 
-def __num_prevalence_combinations_depr(n_prevpoints:int, n_classes:int, n_repeats:int=1):
-    """
-    Computes the number of prevalence combinations in the n_classes-dimensional simplex if `nprevpoints` equally distant
-    prevalence values are generated and `n_repeats` repetitions are requested.
-
-    :param n_classes: integer, number of classes
-    :param n_prevpoints: integer, number of prevalence points.
-    :param n_repeats: integer, number of repetitions for each prevalence combination
-    :return: The number of possible combinations. For example, if n_classes=2, n_prevpoints=5, n_repeats=1, then the
-        number of possible combinations are 5, i.e.: [0,1], [0.25,0.75], [0.50,0.50], [0.75,0.25], and [1.0,0.0]
-    """
-    __cache={}
-    def __f(nc,np):
-        if (nc,np) in __cache:  # cached result
-            return __cache[(nc,np)]
-        if nc==1:  # stop condition
-            return 1
-        else:  # recursive call
-            x = sum([__f(nc-1, np-i) for i in range(np)])
-            __cache[(nc,np)] = x
-            return x
-    return __f(n_classes, n_prevpoints) * n_repeats
-
-
-def num_prevalence_combinations(n_prevpoints:int, n_classes:int, n_repeats:int=1):
-    """
-    Computes the number of valid prevalence combinations in the n_classes-dimensional simplex if `n_prevpoints` equally
-    distant prevalence values are generated and `n_repeats` repetitions are requested.
-    The computation comes down to calculating:
-
-    .. math::
-        \\binom{N+C-1}{C-1} \\times r
-
-    where `N` is `n_prevpoints-1`, i.e., the number of probability mass blocks to allocate, `C` is the number of
-    classes, and `r` is `n_repeats`. This solution comes from the
-    `Stars and Bars <https://brilliant.org/wiki/integer-equations-star-and-bars/>`_ problem.
-
-    :param n_classes: integer, number of classes
-    :param n_prevpoints: integer, number of prevalence points.
-    :param n_repeats: integer, number of repetitions for each prevalence combination
-    :return: The number of possible combinations. For example, if n_classes=2, n_prevpoints=5, n_repeats=1, then the
-        number of possible combinations are 5, i.e.: [0,1], [0.25,0.75], [0.50,0.50], [0.75,0.25], and [1.0,0.0]
-    """
-    N = n_prevpoints-1
-    C = n_classes
-    r = n_repeats
-    return int(scipy.special.binom(N + C - 1, C - 1) * r)
-
-
-def get_nprevpoints_approximation(combinations_budget:int, n_classes:int, n_repeats:int=1):
-    """
-    Searches for the largest number of (equidistant) prevalence points to define for each of the `n_classes` classes so
-    that the number of valid prevalence values generated as combinations of prevalence points (points in a
-    `n_classes`-dimensional simplex) do not exceed combinations_budget.
-
-    :param combinations_budget: integer, maximum number of combinations allowed
-    :param n_classes: integer, number of classes
-    :param n_repeats: integer, number of repetitions for each prevalence combination
-    :return: the largest number of prevalence points that generate less than combinations_budget valid prevalences
-    """
-    assert n_classes > 0 and n_repeats > 0 and combinations_budget > 0, 'parameters must be positive integers'
-    n_prevpoints = 1
-    while True:
-        combinations = num_prevalence_combinations(n_prevpoints, n_classes, n_repeats)
-        if combinations > combinations_budget:
-            return n_prevpoints-1
-        else:
-            n_prevpoints += 1
-
-
-def check_prevalence_vector(p, raise_exception=False, toleranze=1e-08):
-    """
-    Checks that p is a valid prevalence vector, i.e., that it contains values in [0,1] and that the values sum up to 1.
-
-    :param p: the prevalence vector to check
-    :return: True if `p` is valid, False otherwise
-    """
-    p = np.asarray(p)
-    if not all(p>=0):
-        if raise_exception:
-            raise ValueError('the prevalence vector contains negative numbers')
-        return False
-    if not all(p<=1):
-        if raise_exception:
-            raise ValueError('the prevalence vector contains values >1')
-        return False
-    if not np.isclose(p.sum(), 1, atol=toleranze):
-        if raise_exception:
-            raise ValueError('the prevalence vector does not sum up to 1')
-        return False
-    return True
-
-
-def get_divergence(divergence: Union[str, Callable]):
-    if isinstance(divergence, str):
-        if divergence=='HD':
-            return HellingerDistance
-        elif divergence=='topsoe':
-            return TopsoeDistance
-        else:
-            raise ValueError(f'unknown divergence {divergence}')
-    elif callable(divergence):
-        return divergence
+    if method == "inversion":
+        pass  # We leave A and B unchanged
+    elif method == "invariant-ratio":
+        # Change the last equation to replace it with the normalization condition
+        A[-1, :] = 1.0
+        B[-1] = 1.0
     else:
-        raise ValueError(f'argument "divergence" not understood; use a str or a callable function')
+        raise ValueError(f"unknown {method=}")
 
-
-def argmin_prevalence(loss, n_classes, method='optim_minimize'):
-    if method == 'optim_minimize':
-        return optim_minimize(loss, n_classes)
-    elif method == 'linear_search':
-        return linear_search(loss, n_classes)
-    elif method == 'ternary_search':
-        raise NotImplementedError()
+    if solver == "minimize":
+        def loss(prev):
+            return np.linalg.norm(A @ prev - B)
+        return optim_minimize(loss, n_classes=A.shape[0])
+    elif solver in ["exact-raise", "exact-cc"]:
+        # Solvers based on matrix inversion, so we use try/except block
+        try:
+            return np.linalg.solve(A, B)
+        except np.linalg.LinAlgError:
+            # The matrix is not invertible.
+            # Depending on the solver, we either raise an error
+            # or return the classifier predictions without adjustment
+            if solver == "exact-raise":
+                raise
+            elif solver == "exact-cc":
+                return unadjusted_counts
+            else:
+                raise ValueError(f"Solver {solver} not known.")
     else:
-        raise NotImplementedError()
-
-
-def optim_minimize(loss, n_classes):
-    """
-    Searches for the optimal prevalence values, i.e., an `n_classes`-dimensional vector of the (`n_classes`-1)-simplex
-    that yields the smallest lost. This optimization is carried out by means of a constrained search using scipy's
-    SLSQP routine.
-
-    :param loss: (callable) the function to minimize
-    :param n_classes: (int) the number of classes, i.e., the dimensionality of the prevalence vector
-    :return: (ndarray) the best prevalence vector found
-    """
-    from scipy import optimize
-
-    # the initial point is set as the uniform distribution
-    uniform_distribution = np.full(fill_value=1 / n_classes, shape=(n_classes,))
-
-    # solutions are bounded to those contained in the unit-simplex
-    bounds = tuple((0, 1) for _ in range(n_classes))  # values in [0,1]
-    constraints = ({'type': 'eq', 'fun': lambda x: 1 - sum(x)})  # values summing up to 1
-    r = optimize.minimize(loss, x0=uniform_distribution, method='SLSQP', bounds=bounds, constraints=constraints)
-    return r.x
-
-
-def linear_search(loss, n_classes):
-    """
-    Performs a linear search for the best prevalence value in binary problems. The search is carried out by exploring
-    the range [0,1] stepping by 0.01. This search is inefficient, and is added only for completeness (some of the
-    early methods in quantification literature used it, e.g., HDy). A most powerful alternative is `optim_minimize`.
-
-    :param loss: (callable) the function to minimize
-    :param n_classes: (int) the number of classes, i.e., the dimensionality of the prevalence vector
-    :return: (ndarray) the best prevalence vector found
-    """
-    assert n_classes==2, 'linear search is only available for binary problems'
-
-    prev_selected, min_score = None, None
-    for prev in prevalence_linspace(n_prevalences=100, repeats=1, smooth_limits_epsilon=0.0):
-        score = loss(np.asarray([1 - prev, prev]))
-        if min_score is None or score < min_score:
-            prev_selected, min_score = prev, score
-
-    return np.asarray([1 - prev_selected, prev_selected])
+        raise ValueError(f'unknown {solver=}')

quapy/method/__init__.py

@@ -17,14 +17,39 @@ AGGREGATIVE_METHODS = {
     aggregative.MAX,
     aggregative.MS,
     aggregative.MS2,
+    aggregative.DMy,
     aggregative.KDEyML,
     aggregative.KDEyCS,
     aggregative.KDEyHD,
+    aggregative.BayesianCC
 }
 
+BINARY_METHODS = {
+    aggregative.HDy,
+    aggregative.DyS,
+    aggregative.SMM,
+    aggregative.X,
+    aggregative.T50,
+    aggregative.MAX,
+    aggregative.MS,
+    aggregative.MS2,
+}
+
+MULTICLASS_METHODS = {
+    aggregative.CC,
+    aggregative.ACC,
+    aggregative.PCC,
+    aggregative.PACC,
+    aggregative.EMQ,
+    aggregative.KDEyML,
+    aggregative.KDEyCS,
+    aggregative.KDEyHD,
+    aggregative.BayesianCC
+}
 
 NON_AGGREGATIVE_METHODS = {
-    non_aggregative.MaximumLikelihoodPrevalenceEstimation
+    non_aggregative.MaximumLikelihoodPrevalenceEstimation,
+    non_aggregative.DMx
 }
 
 META_METHODS = {

quapy/method/_bayesian.py

@@ -0,0 +1,79 @@
+"""
+Utility functions for `Bayesian quantification <https://arxiv.org/abs/2302.09159>`_ methods.
+"""
+import numpy as np
+
+try:
+    import jax
+    import jax.numpy as jnp
+    import numpyro
+    import numpyro.distributions as dist
+
+    DEPENDENCIES_INSTALLED = True
+except ImportError:
+    jax = None
+    jnp = None
+    numpyro = None
+    dist = None
+
+    DEPENDENCIES_INSTALLED = False
+
+
+P_TEST_Y: str = "P_test(Y)"
+P_TEST_C: str = "P_test(C)"
+P_C_COND_Y: str = "P(C|Y)"
+
+
+def model(n_c_unlabeled: np.ndarray, n_y_and_c_labeled: np.ndarray) -> None:
+    """
+    Defines a probabilistic model in `NumPyro <https://num.pyro.ai/>`_.
+
+    :param n_c_unlabeled: a `np.ndarray` of shape `(n_predicted_classes,)`
+        with entry `c` being the number of instances predicted as class `c`.
+    :param n_y_and_c_labeled: a `np.ndarray` of shape `(n_classes, n_predicted_classes)`
+        with entry `(y, c)` being the number of instances labeled as class `y` and predicted as class `c`.
+    """
+    n_y_labeled = n_y_and_c_labeled.sum(axis=1)
+
+    K = len(n_c_unlabeled)
+    L = len(n_y_labeled)
+
+    pi_ = numpyro.sample(P_TEST_Y, dist.Dirichlet(jnp.ones(L)))
+    p_c_cond_y = numpyro.sample(P_C_COND_Y, dist.Dirichlet(jnp.ones(K).repeat(L).reshape(L, K)))
+
+    with numpyro.plate('plate', L):
+        numpyro.sample('F_yc', dist.Multinomial(n_y_labeled, p_c_cond_y), obs=n_y_and_c_labeled)
+
+    p_c = numpyro.deterministic(P_TEST_C, jnp.einsum("yc,y->c", p_c_cond_y, pi_))
+    numpyro.sample('N_c', dist.Multinomial(jnp.sum(n_c_unlabeled), p_c), obs=n_c_unlabeled)
+
+
+def sample_posterior(
+    n_c_unlabeled: np.ndarray,
+    n_y_and_c_labeled: np.ndarray,
+    num_warmup: int,
+    num_samples: int,
+    seed: int = 0,
+) -> dict:
+    """
+    Samples from the Bayesian quantification model in NumPyro using the
+    `NUTS <https://arxiv.org/abs/1111.4246>`_ sampler.
+
+    :param n_c_unlabeled: a `np.ndarray` of shape `(n_predicted_classes,)`
+        with entry `c` being the number of instances predicted as class `c`.
+    :param n_y_and_c_labeled: a `np.ndarray` of shape `(n_classes, n_predicted_classes)`
+        with entry `(y, c)` being the number of instances labeled as class `y` and predicted as class `c`.
+    :param num_warmup: the number of warmup steps.
+    :param num_samples: the number of samples to draw.
+    :seed: the random seed.
+    :return: a `dict` with the samples. The keys are the names of the latent variables.
+    """
+    mcmc = numpyro.infer.MCMC(
+        numpyro.infer.NUTS(model),
+        num_warmup=num_warmup,
+        num_samples=num_samples,
+        progress_bar=False
+    )
+    rng_key = jax.random.PRNGKey(seed)
+    mcmc.run(rng_key, n_c_unlabeled=n_c_unlabeled, n_y_and_c_labeled=n_y_and_c_labeled)
+    return mcmc.get_samples()

quapy/method/_kdey.py

@@ -52,7 +52,7 @@ class KDEBase:
         """
         return np.exp(kde.score_samples(X))
 
-    def get_mixture_components(self, X, y, n_classes, bandwidth):
+    def get_mixture_components(self, X, y, classes, bandwidth):
         """
         Returns an array containing the mixture components, i.e., the KDE functions for each class.
 
@@ -62,7 +62,7 @@ class KDEBase:
         :param bandwidth: float, the bandwidth of the kernel
         :return: a list of KernelDensity objects, each fitted with the corresponding class-specific covariates
         """
-        return [self.get_kde_function(X[y == cat], bandwidth) for cat in range(n_classes)]
+        return [self.get_kde_function(X[y == cat], bandwidth) for cat in classes]
 
 
 
@@ -114,7 +114,7 @@ class KDEyML(AggregativeSoftQuantifier, KDEBase):
         self.random_state=random_state
 
     def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
-        self.mix_densities = self.get_mixture_components(*classif_predictions.Xy, data.n_classes, self.bandwidth)
+        self.mix_densities = self.get_mixture_components(*classif_predictions.Xy, data.classes_, self.bandwidth)
         return self
 
     def aggregate(self, posteriors: np.ndarray):
@@ -196,7 +196,7 @@ class KDEyHD(AggregativeSoftQuantifier, KDEBase):
         self.montecarlo_trials = montecarlo_trials
 
     def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
-        self.mix_densities = self.get_mixture_components(*classif_predictions.Xy, data.n_classes, self.bandwidth)
+        self.mix_densities = self.get_mixture_components(*classif_predictions.Xy, data.classes_, self.bandwidth)
 
         N = self.montecarlo_trials
         rs = self.random_state

quapy/method/aggregative.py

@@ -1,6 +1,6 @@
 from abc import ABC, abstractmethod
 from copy import deepcopy
-from typing import Callable, Union
+from typing import Callable, Literal, Union
 import numpy as np
 from abstention.calibration import NoBiasVectorScaling, TempScaling, VectorScaling
 from scipy import optimize
@@ -16,6 +16,8 @@ from quapy.classification.calibration import NBVSCalibration, BCTSCalibration, T
 from quapy.classification.svmperf import SVMperf
 from quapy.data import LabelledCollection
 from quapy.method.base import BaseQuantifier, BinaryQuantifier, OneVsAllGeneric
+from quapy.method import _bayesian
+
 
 
 # Abstract classes
@@ -162,8 +164,8 @@ class AggregativeQuantifier(BaseQuantifier, ABC):
         """
         Trains the aggregation function.
 
-        :param classif_predictions: a LabelledCollection containing the label predictions issued
-            by the classifier
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the predictions issued by the classifier and, as labels, the true labels
         :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
         """
         ...
@@ -249,7 +251,7 @@ class AggregativeQuantifier(BaseQuantifier, ABC):
 
 class AggregativeCrispQuantifier(AggregativeQuantifier, ABC):
     """
-    Abstract class for quantification methods that base their estimations on the aggregation of crips decisions
+    Abstract class for quantification methods that base their estimations on the aggregation of crisp decisions
     as returned by a hard classifier. Aggregative crisp quantifiers thus extend Aggregative
     Quantifiers by implementing specifications about crisp predictions.
     """
@@ -335,7 +337,8 @@ class CC(AggregativeCrispQuantifier):
         """
         Nothing to do here!
 
-        :param classif_predictions: this is actually None
+        :param classif_predictions: not used
+        :param data: not used
         """
         pass
 
@@ -349,110 +352,6 @@ class CC(AggregativeCrispQuantifier):
         return F.prevalence_from_labels(classif_predictions, self.classes_)
 
 
-class ACC(AggregativeCrispQuantifier):
-    """
-    `Adjusted Classify & Count <https://link.springer.com/article/10.1007/s10618-008-0097-y>`_,
-    the "adjusted" variant of :class:`CC`, that corrects the predictions of CC
-    according to the `misclassification rates`.
-
-    :param classifier: a sklearn's Estimator that generates a classifier
-    :param val_split: specifies the data used for generating classifier predictions. This specification
-        can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to
-        be extracted from the training set; or as an integer (default 5), indicating that the predictions
-        are to be generated in a `k`-fold cross-validation manner (with this integer indicating the value
-        for `k`); or as a collection defining the specific set of data to use for validation.
-        Alternatively, this set can be specified at fit time by indicating the exact set of data
-        on which the predictions are to be generated.
-    :param n_jobs: number of parallel workers
-    :param solver: indicates the method to be used for obtaining the final estimates. The choice
-        'exact' comes down to solving the system of linear equations :math:`Ax=B` where `A` is a
-        matrix containing the class-conditional probabilities of the predictions (e.g., the tpr and fpr in 
-        binary) and `B` is the vector of prevalence values estimated via CC, as :math:`x=A^{-1}B`. This solution
-        might not exist for degenerated classifiers, in which case the method defaults to classify and count 
-        (i.e., does not attempt any adjustment).
-        Another option is to search for the prevalence vector that minimizes the L2 norm of :math:`|Ax-B|`. The latter
-        is achieved by indicating solver='minimize'. This one generally works better, and is the default parameter.
-        More details about this can be consulted in `Bunse, M. "On Multi-Class Extensions of Adjusted Classify and
-        Count", on proceedings of the 2nd International Workshop on Learning to Quantify: Methods and Applications
-        (LQ 2022), ECML/PKDD 2022, Grenoble (France) <https://lq-2022.github.io/proceedings/CompleteVolume.pdf>`_.
-    """
-
-    def __init__(self, classifier: BaseEstimator, val_split=5, n_jobs=None, solver='minimize'):
-        self.classifier = classifier
-        self.val_split = val_split
-        self.n_jobs = qp._get_njobs(n_jobs)
-        self.solver = solver
-
-    def _check_init_parameters(self):
-        assert self.solver in ['exact', 'minimize'], "unknown solver; valid ones are 'exact', 'minimize'"
-
-    def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
-        """
-        Estimates the misclassification rates.
-
-        :param classif_predictions: classifier predictions with true labels
-        """
-        pred_labels, true_labels = classif_predictions.Xy
-        self.cc = CC(self.classifier)
-        self.Pte_cond_estim_ = self.getPteCondEstim(self.classifier.classes_, true_labels, pred_labels)
-
-    @classmethod
-    def getPteCondEstim(cls, classes, y, y_):
-        # estimate the matrix with entry (i,j) being the estimate of P(hat_yi|yj), that is, the probability that a
-        # document that belongs to yj ends up being classified as belonging to yi
-        conf = confusion_matrix(y, y_, labels=classes).T
-        conf = conf.astype(float)
-        class_counts = conf.sum(axis=0)
-        for i, _ in enumerate(classes):
-            if class_counts[i] == 0:
-                conf[i, i] = 1
-            else:
-                conf[:, i] /= class_counts[i]
-        return conf
-
-    def aggregate(self, classif_predictions):
-        prevs_estim = self.cc.aggregate(classif_predictions)
-        return ACC.solve_adjustment(self.Pte_cond_estim_, prevs_estim, solver=self.solver)
-
-    @classmethod
-    def solve_adjustment(cls, PteCondEstim, prevs_estim, solver='exact'):
-        """
-        Solves the system linear system :math:`Ax = B` with :math:`A` = `PteCondEstim` and :math:`B` = `prevs_estim`
-
-        :param PteCondEstim: a `np.ndarray` of shape `(n_classes,n_classes,)` with entry `(i,j)` being the estimate
-            of :math:`P(y_i|y_j)`, that is, the probability that an instance that belongs to :math:`y_j` ends up being
-            classified as belonging to :math:`y_i`
-        :param prevs_estim: a `np.ndarray` of shape `(n_classes,)` with the class prevalence estimates
-        :param solver: indicates the method to use for solving the system of linear equations. Valid options are
-             'exact' (tries to solve the system --may fail if the misclassificatin matrix has rank < n_classes) or
-             'optim_minimize' (minimizes a norm --always exists). 
-        :return: an adjusted `np.ndarray` of shape `(n_classes,)` with the corrected class prevalence estimates
-        """
-
-        A = PteCondEstim
-        B = prevs_estim
-
-        if solver == 'exact':
-            # attempts an exact solution of the linear system (may fail)
-
-            try:
-                adjusted_prevs = np.linalg.solve(A, B)
-                adjusted_prevs = np.clip(adjusted_prevs, 0, 1)
-                adjusted_prevs /= adjusted_prevs.sum()
-            except np.linalg.LinAlgError:
-                adjusted_prevs = prevs_estim  # no way to adjust them!
-
-            return adjusted_prevs
-
-        elif solver == 'minimize':
-            # poses the problem as an optimization one, and tries to minimize the norm of the differences
-
-            def loss(prev):
-                return np.linalg.norm(A @ prev - B)
-
-            return F.optim_minimize(loss, n_classes=A.shape[0])
-
-
 class PCC(AggregativeSoftQuantifier):
     """
     `Probabilistic Classify & Count <https://ieeexplore.ieee.org/abstract/document/5694031>`_,
@@ -468,7 +367,8 @@ class PCC(AggregativeSoftQuantifier):
         """
         Nothing to do here!
 
-        :param classif_predictions: this is actually None
+        :param classif_predictions: not used
+        :param data: not used
         """
         pass
 
@@ -476,55 +376,232 @@ class PCC(AggregativeSoftQuantifier):
         return F.prevalence_from_probabilities(classif_posteriors, binarize=False)
 
 
+class ACC(AggregativeCrispQuantifier):
+    """
+    `Adjusted Classify & Count <https://link.springer.com/article/10.1007/s10618-008-0097-y>`_,
+    the "adjusted" variant of :class:`CC`, that corrects the predictions of CC
+    according to the `misclassification rates`.
+
+    :param classifier: a sklearn's Estimator that generates a classifier
+
+    :param val_split: specifies the data used for generating classifier predictions. This specification
+        can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to
+        be extracted from the training set; or as an integer (default 5), indicating that the predictions
+        are to be generated in a `k`-fold cross-validation manner (with this integer indicating the value
+        for `k`); or as a collection defining the specific set of data to use for validation.
+        Alternatively, this set can be specified at fit time by indicating the exact set of data
+        on which the predictions are to be generated.
+
+    :param str method: adjustment method to be used:
+
+        * 'inversion': matrix inversion method based on the matrix equality :math:`P(C)=P(C|Y)P(Y)`,
+          which tries to invert :math:`P(C|Y)` matrix.
+        * 'invariant-ratio': invariant ratio estimator of `Vaz et al. 2018 <https://jmlr.org/papers/v20/18-456.html>`_,
+          which replaces the last equation with the normalization condition.
+
+    :param str solver: indicates the method to use for solving the system of linear equations. Valid options are:
+
+        * 'exact-raise': tries to solve the system using matrix inversion. Raises an error if the matrix has rank
+          strictly less than `n_classes`.
+        * 'exact-cc': if the matrix is not of full rank, returns `p_c` as the estimates, which corresponds to
+          no adjustment (i.e., the classify and count method. See :class:`quapy.method.aggregative.CC`)
+        * 'exact': deprecated, defaults to 'exact-cc'
+        * 'minimize': minimizes the L2 norm of :math:`|Ax-B|`. This one generally works better, and is the
+          default parameter. More details about this can be consulted in `Bunse, M. "On Multi-Class Extensions of
+          Adjusted Classify and Count", on proceedings of the 2nd International Workshop on Learning to Quantify:
+          Methods and Applications (LQ 2022), ECML/PKDD 2022, Grenoble (France)
+          <https://lq-2022.github.io/proceedings/CompleteVolume.pdf>`_.
+
+    :param str norm: the method to use for normalization.
+
+        * `clip`, the values are clipped to the range [0,1] and then L1-normalized.
+        * `mapsimplex` projects vectors onto the probability simplex. This implementation relies on
+          `Mathieu Blondel's projection_simplex_sort <https://gist.github.com/mblondel/6f3b7aaad90606b98f71>`_
+        * `condsoftmax`, applies a softmax normalization only to prevalence vectors that lie outside the simplex
+
+    :param n_jobs: number of parallel workers
+    """
+    def __init__(
+            self,
+            classifier: BaseEstimator,
+            val_split=5,
+            solver: Literal['minimize', 'exact', 'exact-raise', 'exact-cc'] = 'minimize',
+            method: Literal['inversion', 'invariant-ratio'] = 'inversion',
+            norm: Literal['clip', 'mapsimplex', 'condsoftmax'] = 'clip',
+            n_jobs=None,
+    ):
+        self.classifier = classifier
+        self.val_split = val_split
+        self.n_jobs = qp._get_njobs(n_jobs)
+        self.solver = solver
+        self.method = method
+        self.norm = norm
+
+    SOLVERS = ['exact', 'minimize', 'exact-raise', 'exact-cc']
+    METHODS = ['inversion', 'invariant-ratio']
+    NORMALIZATIONS = ['clip', 'mapsimplex', 'condsoftmax', None]
+
+    @classmethod
+    def newInvariantRatioEstimation(cls, classifier: BaseEstimator, val_split=5, n_jobs=None):
+        """
+        Constructs a quantifier that implements the Invariant Ratio Estimator of
+        `Vaz et al. 2018 <https://jmlr.org/papers/v20/18-456.html>`_. This amounts
+        to setting method to 'invariant-ratio' and clipping to 'project'.
+
+        :param classifier: a sklearn's Estimator that generates a classifier
+        :param val_split: specifies the data used for generating classifier predictions. This specification
+        can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to
+        be extracted from the training set; or as an integer (default 5), indicating that the predictions
+        are to be generated in a `k`-fold cross-validation manner (with this integer indicating the value
+        for `k`); or as a collection defining the specific set of data to use for validation.
+        Alternatively, this set can be specified at fit time by indicating the exact set of data
+        on which the predictions are to be generated.
+        :param n_jobs: number of parallel workers
+        :return: an instance of ACC configured so that it implements the Invariant Ratio Estimator
+        """
+        return ACC(classifier, val_split=val_split, method='invariant-ratio', norm='mapsimplex', n_jobs=n_jobs)
+
+    def _check_init_parameters(self):
+        if self.solver not in ACC.SOLVERS:
+            raise ValueError(f"unknown solver; valid ones are {ACC.SOLVERS}")
+        if self.method not in ACC.METHODS:
+            raise ValueError(f"unknown method; valid ones are {ACC.METHODS}")
+        if self.norm not in ACC.NORMALIZATIONS:
+            raise ValueError(f"unknown clipping; valid ones are {ACC.NORMALIZATIONS}")
+
+    def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
+        """
+        Estimates the misclassification rates.
+
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the label predictions issued by the classifier and, as labels, the true labels
+        :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
+        """
+        pred_labels, true_labels = classif_predictions.Xy
+        self.cc = CC(self.classifier)
+        self.Pte_cond_estim_ = ACC.getPteCondEstim(self.classifier.classes_, true_labels, pred_labels)
+
+    @classmethod
+    def getPteCondEstim(cls, classes, y, y_):
+        """
+        Estimate the matrix with entry (i,j) being the estimate of P(hat_yi|yj), that is, the probability that a
+        document that belongs to yj ends up being classified as belonging to yi
+
+        :param classes: array-like with the class names
+        :param y: array-like with the true labels
+        :param y_: array-like with the estimated labels
+        :return: np.ndarray
+        """
+        conf = confusion_matrix(y, y_, labels=classes).T
+        conf = conf.astype(float)
+        class_counts = conf.sum(axis=0)
+        for i, _ in enumerate(classes):
+            if class_counts[i] == 0:
+                conf[i, i] = 1
+            else:
+                conf[:, i] /= class_counts[i]
+        return conf
+
+    def aggregate(self, classif_predictions):
+        prevs_estim = self.cc.aggregate(classif_predictions)
+        estimate = F.solve_adjustment(
+            class_conditional_rates=self.Pte_cond_estim_,
+            unadjusted_counts=prevs_estim,
+            solver=self.solver,
+            method=self.method,
+        )
+        return F.normalize_prevalence(estimate, method=self.norm)
+
+
 class PACC(AggregativeSoftQuantifier):
     """
     `Probabilistic Adjusted Classify & Count <https://ieeexplore.ieee.org/abstract/document/5694031>`_,
     the probabilistic variant of ACC that relies on the posterior probabilities returned by a probabilistic classifier.
 
     :param classifier: a sklearn's Estimator that generates a classifier
+
     :param val_split: specifies the data used for generating classifier predictions. This specification
         can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to
         be extracted from the training set; or as an integer (default 5), indicating that the predictions
         are to be generated in a `k`-fold cross-validation manner (with this integer indicating the value
         for `k`). Alternatively, this set can be specified at fit time by indicating the exact set of data
         on which the predictions are to be generated.
+
+    :param str method: adjustment method to be used:
+
+        * 'inversion': matrix inversion method based on the matrix equality :math:`P(C)=P(C|Y)P(Y)`,
+          which tries to invert `P(C|Y)` matrix.
+        * 'invariant-ratio': invariant ratio estimator of `Vaz et al. <https://jmlr.org/papers/v20/18-456.html>`_,
+          which replaces the last equation with the normalization condition.
+
+    :param str solver: the method to use for solving the system of linear equations. Valid options are:
+
+        * 'exact-raise': tries to solve the system using matrix inversion.
+          Raises an error if the matrix has rank strictly less than `n_classes`.
+        * 'exact-cc': if the matrix is not of full rank, returns `p_c` as the estimates, which
+          corresponds to no adjustment (i.e., the classify and count method. See :class:`quapy.method.aggregative.CC`)
+        * 'exact': deprecated, defaults to 'exact-cc'
+        * 'minimize': minimizes the L2 norm of :math:`|Ax-B|`. This one generally works better, and is the
+          default parameter. More details about this can be consulted in `Bunse, M. "On Multi-Class Extensions
+          of Adjusted Classify and Count", on proceedings of the 2nd International Workshop on Learning to
+          Quantify: Methods and Applications (LQ 2022), ECML/PKDD 2022, Grenoble (France)
+          <https://lq-2022.github.io/proceedings/CompleteVolume.pdf>`_.
+
+    :param str norm: the method to use for normalization.
+
+        * `clip`, the values are clipped to the range [0,1] and then L1-normalized.
+        * `mapsimplex` projects vectors onto the probability simplex. This implementation relies on
+          `Mathieu Blondel's projection_simplex_sort <https://gist.github.com/mblondel/6f3b7aaad90606b98f71>`_
+        * `condsoftmax`, applies a softmax normalization only to prevalence vectors that lie outside the simplex
+
     :param n_jobs: number of parallel workers
-    :param solver: indicates the method to be used for obtaining the final estimates. The choice
-        'exact' comes down to solving the system of linear equations :math:`Ax=B` where `A` is a
-        matrix containing the class-conditional probabilities of the predictions (e.g., the tpr and fpr in
-        binary) and `B` is the vector of prevalence values estimated via CC, as :math:`x=A^{-1}B`. This solution
-        might not exist for degenerated classifiers, in which case the method defaults to classify and count
-        (i.e., does not attempt any adjustment).
-        Another option is to search for the prevalence vector that minimizes the L2 norm of :math:`|Ax-B|`. The latter
-        is achieved by indicating solver='minimize'. This one generally works better, and is the default parameter.
-        More details about this can be consulted in `Bunse, M. "On Multi-Class Extensions of Adjusted Classify and
-        Count", on proceedings of the 2nd International Workshop on Learning to Quantify: Methods and Applications
-        (LQ 2022), ECML/PKDD 2022, Grenoble (France) <https://lq-2022.github.io/proceedings/CompleteVolume.pdf>`_.
-
     """
-
-    def __init__(self, classifier: BaseEstimator, val_split=5, n_jobs=None, solver='minimize'):
+    def __init__(
+            self,
+            classifier: BaseEstimator,
+            val_split=5,
+            solver: Literal['minimize', 'exact', 'exact-raise', 'exact-cc'] = 'minimize',
+            method: Literal['inversion', 'invariant-ratio'] = 'inversion',
+            norm: Literal['clip', 'mapsimplex', 'condsoftmax'] = 'clip',
+            n_jobs=None
+    ):
         self.classifier = classifier
         self.val_split = val_split
         self.n_jobs = qp._get_njobs(n_jobs)
         self.solver = solver
+        self.method = method
+        self.norm = norm
 
     def _check_init_parameters(self):
-        assert self.solver in ['exact', 'minimize'], "unknown solver; valid ones are 'exact', 'minimize'"
+        if self.solver not in ACC.SOLVERS:
+            raise ValueError(f"unknown solver; valid ones are {ACC.SOLVERS}")
+        if self.method not in ACC.METHODS:
+            raise ValueError(f"unknown method; valid ones are {ACC.METHODS}")
+        if self.clipping not in ACC.NORMALIZATIONS:
+            raise ValueError(f"unknown clipping; valid ones are {ACC.NORMALIZATIONS}")
 
     def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
         """
         Estimates the misclassification rates
 
-        :param classif_predictions: classifier soft predictions with true labels
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the posterior probabilities issued by the classifier and, as labels, the true labels
+        :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
         """
         posteriors, true_labels = classif_predictions.Xy
         self.pcc = PCC(self.classifier)
-        self.Pte_cond_estim_ = self.getPteCondEstim(self.classifier.classes_, true_labels, posteriors)
+        self.Pte_cond_estim_ = PACC.getPteCondEstim(self.classifier.classes_, true_labels, posteriors)
 
     def aggregate(self, classif_posteriors):
         prevs_estim = self.pcc.aggregate(classif_posteriors)
-        return ACC.solve_adjustment(self.Pte_cond_estim_, prevs_estim, solver=self.solver)
+
+        estimate = F.solve_adjustment(
+            class_conditional_rates=self.Pte_cond_estim_,
+            unadjusted_counts=prevs_estim,
+            solver=self.solver,
+            method=self.method,
+        )
+        return F.normalize_prevalence(estimate, method=self.norm)
 
     @classmethod
     def getPteCondEstim(cls, classes, y, y_):
@@ -626,6 +703,14 @@ class EMQ(AggregativeSoftQuantifier):
         return posteriors
 
     def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
+        """
+        Trains the aggregation function of EMQ. This comes down to recalibrating the posterior probabilities
+        ir requested.
+
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the posterior probabilities issued by the classifier and, as labels, the true labels
+        :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
+        """
         if self.recalib is not None:
             P, y = classif_predictions.Xy
             if self.recalib == 'nbvs':
@@ -640,6 +725,8 @@ class EMQ(AggregativeSoftQuantifier):
                 raise ValueError('invalid param argument for recalibration method; available ones are '
                                  '"nbvs", "bcts", "ts", and "vs".')
 
+            if not np.issubdtype(y.dtype, np.number):
+                y = np.searchsorted(data.classes_, y)
             self.calibration_function = calibrator(P, np.eye(data.n_classes)[y], posterior_supplied=True)
 
         if self.exact_train_prev:
@@ -681,6 +768,11 @@ class EMQ(AggregativeSoftQuantifier):
         """
         Px = posterior_probabilities
         Ptr = np.copy(tr_prev)
+
+        if np.product(Ptr) == 0:  # some entry is 0; we should smooth the values to avoid 0 division
+            Ptr += epsilon
+            Ptr /= Ptr.sum()
+
         qs = np.copy(Ptr)  # qs (the running estimate) is initialized as the training prevalence
 
         s, converged = 0, False
@@ -705,6 +797,99 @@ class EMQ(AggregativeSoftQuantifier):
         return qs, ps
 
 
+class BayesianCC(AggregativeCrispQuantifier):
+    """
+    `Bayesian quantification <https://arxiv.org/abs/2302.09159>`_ method,
+    which is a variant of :class:`ACC` that calculates the posterior probability distribution
+    over the prevalence vectors, rather than providing a point estimate obtained
+    by matrix inversion.
+
+    Can be used to diagnose degeneracy in the predictions visible when the confusion
+    matrix has high condition number or to quantify uncertainty around the point estimate.
+
+    This method relies on extra dependencies, which have to be installed via:
+    `$ pip install quapy[bayes]`
+
+    :param classifier: a sklearn's Estimator that generates a classifier
+    :param val_split: a float in (0, 1) indicating the proportion of the training data to be used,
+        as a stratified held-out validation set, for generating classifier predictions.
+    :param num_warmup: number of warmup iterations for the MCMC sampler (default 500)
+    :param num_samples: number of samples to draw from the posterior (default 1000)
+    :param mcmc_seed: random seed for the MCMC sampler (default 0)
+    """
+    def __init__(self,
+                 classifier: BaseEstimator,
+                 val_split: float = 0.75,
+                 num_warmup: int = 500,
+                 num_samples: int = 1_000,
+                 mcmc_seed: int = 0):
+
+        if num_warmup <= 0:
+            raise ValueError(f'parameter {num_warmup=} must be a positive integer')
+        if num_samples <= 0:
+            raise ValueError(f'parameter {num_samples=} must be a positive integer')
+
+        if (not isinstance(val_split, float)) or val_split <= 0 or val_split >= 1:
+            raise ValueError(f'val_split must be a float in (0, 1), got {val_split}')
+
+        if _bayesian.DEPENDENCIES_INSTALLED is False:
+            raise ImportError("Auxiliary dependencies are required. Run `$ pip install quapy[bayes]` to install them.")
+
+        self.classifier = classifier
+        self.val_split = val_split
+        self.num_warmup = num_warmup
+        self.num_samples = num_samples
+        self.mcmc_seed = mcmc_seed
+
+        # Array of shape (n_classes, n_predicted_classes,) where entry (y, c) is the number of instances
+        # labeled as class y and predicted as class c.
+        # By default, this array is set to None and later defined as part of the `aggregation_fit` phase
+        self._n_and_c_labeled = None
+
+        # Dictionary with posterior samples, set when `aggregate` is provided.
+        self._samples = None
+
+    def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
+        """
+        Estimates the misclassification rates.
+
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the label predictions issued by the classifier and, as labels, the true labels
+        :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
+        """
+        pred_labels, true_labels = classif_predictions.Xy
+        self._n_and_c_labeled = confusion_matrix(y_true=true_labels, y_pred=pred_labels, labels=self.classifier.classes_)
+
+    def sample_from_posterior(self, classif_predictions):
+        if self._n_and_c_labeled is None:
+            raise ValueError("aggregation_fit must be called before sample_from_posterior")
+
+        n_c_unlabeled = F.counts_from_labels(classif_predictions, self.classifier.classes_)
+
+        self._samples = _bayesian.sample_posterior(
+            n_c_unlabeled=n_c_unlabeled,
+            n_y_and_c_labeled=self._n_and_c_labeled,
+            num_warmup=self.num_warmup,
+            num_samples=self.num_samples,
+            seed=self.mcmc_seed,
+        )
+        return self._samples
+
+    def get_prevalence_samples(self):
+        if self._samples is None:
+            raise ValueError("sample_from_posterior must be called before get_prevalence_samples")
+        return self._samples[_bayesian.P_TEST_Y]
+
+    def get_conditional_probability_samples(self):
+        if self._samples is None:
+            raise ValueError("sample_from_posterior must be called before get_conditional_probability_samples")
+        return self._samples[_bayesian.P_C_COND_Y]
+
+    def aggregate(self, classif_predictions):
+        samples = self.sample_from_posterior(classif_predictions)[_bayesian.P_TEST_Y]
+        return np.asarray(samples.mean(axis=0), dtype=float)
+
+
 class HDy(AggregativeSoftQuantifier, BinaryAggregativeQuantifier):
     """
     `Hellinger Distance y <https://www.sciencedirect.com/science/article/pii/S0020025512004069>`_ (HDy).
@@ -726,14 +911,11 @@ class HDy(AggregativeSoftQuantifier, BinaryAggregativeQuantifier):
 
     def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
         """
-        Trains a HDy quantifier.
+        Trains the aggregation function of HDy.
 
-        :param data: the training set
-        :param fit_classifier: set to False to bypass the training (the learner is assumed to be already fit)
-        :param val_split: either a float in (0,1) indicating the proportion of training instances to use for
-         validation (e.g., 0.3 for using 30% of the training set as validation data), or a
-         :class:`quapy.data.base.LabelledCollection` indicating the validation set itself
-        :return: self
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the posterior probabilities issued by the classifier and, as labels, the true labels
+        :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
         """
         P, y = classif_predictions.Xy
         Px = P[:, self.pos_label]  # takes only the P(y=+1|x)
@@ -750,8 +932,6 @@ class HDy(AggregativeSoftQuantifier, BinaryAggregativeQuantifier):
         self.Pxy1_density = {bins: hist(self.Pxy1, bins) for bins in self.bins}
         self.Pxy0_density = {bins: hist(self.Pxy0, bins) for bins in self.bins}
 
-        return self
-
     def aggregate(self, classif_posteriors):
         # "In this work, the number of bins b used in HDx and HDy was chosen from 10 to 110 in steps of 10,
         # and the final estimated a priori probability was taken as the median of these 11 estimates."
@@ -773,7 +953,7 @@ class HDy(AggregativeSoftQuantifier, BinaryAggregativeQuantifier):
             # at small steps (modern implementations resort to an optimization procedure,
             # see class DistributionMatching)
             prev_selected, min_dist = None, None
-            for prev in F.prevalence_linspace(n_prevalences=101, repeats=1, smooth_limits_epsilon=0.0):
+            for prev in F.prevalence_linspace(grid_points=101, repeats=1, smooth_limits_epsilon=0.0):
                 Px_train = prev * Pxy1_density + (1 - prev) * Pxy0_density
                 hdy = F.HellingerDistance(Px_train, Px_test)
                 if prev_selected is None or hdy < min_dist:
@@ -826,6 +1006,13 @@ class DyS(AggregativeSoftQuantifier, BinaryAggregativeQuantifier):
         return (left + right) / 2
 
     def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
+        """
+        Trains the aggregation function of DyS.
+
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the posterior probabilities issued by the classifier and, as labels, the true labels
+        :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
+        """
         Px, y = classif_predictions.Xy
         Px = Px[:, self.pos_label]  # takes only the P(y=+1|x)
         self.Pxy1 = Px[y == self.pos_label]
@@ -864,6 +1051,13 @@ class SMM(AggregativeSoftQuantifier, BinaryAggregativeQuantifier):
         self.val_split = val_split
       
     def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
+        """
+        Trains the aggregation function of SMM.
+
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the posterior probabilities issued by the classifier and, as labels, the true labels
+        :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
+        """
         Px, y = classif_predictions.Xy
         Px = Px[:, self.pos_label]  # takes only the P(y=+1|x)
         self.Pxy1 = Px[y == self.pos_label]
@@ -937,19 +1131,17 @@ class DMy(AggregativeSoftQuantifier):
 
     def aggregation_fit(self, classif_predictions: LabelledCollection, data: LabelledCollection):
         """
-        Trains the classifier (if requested) and generates the validation distributions out of the training data.
+        Trains the aggregation function of a distribution matching method. This comes down to generating the
+        validation distributions out of the training data.
         The validation distributions have shape `(n, ch, nbins)`, with `n` the number of classes, `ch` the number of
         channels, and `nbins` the number of bins. In particular, let `V` be the validation distributions; then `di=V[i]`
         are the distributions obtained from training data labelled with class `i`; while `dij = di[j]` is the discrete
         distribution of posterior probabilities `P(Y=j|X=x)` for training data labelled with class `i`, and `dij[k]`
         is the fraction of instances with a value in the `k`-th bin.
 
-        :param data: the training set
-        :param fit_classifier: set to False to bypass the training (the learner is assumed to be already fit)
-        :param val_split: either a float in (0,1) indicating the proportion of training instances to use for
-         validation (e.g., 0.3 for using 30% of the training set as validation data), or a LabelledCollection
-         indicating the validation set itself, or an int indicating the number k of folds to be used in kFCV
-         to estimate the parameters
+        :param classif_predictions: a :class:`quapy.data.base.LabelledCollection` containing,
+            as instances, the posterior probabilities issued by the classifier and, as labels, the true labels
+        :param data: a :class:`quapy.data.base.LabelledCollection` consisting of the training data
         """
         posteriors, true_labels = classif_predictions.Xy
         n_classes = len(self.classifier.classes_)
@@ -1330,8 +1522,8 @@ AdjustedClassifyAndCount = ACC
 ProbabilisticClassifyAndCount = PCC
 ProbabilisticAdjustedClassifyAndCount = PACC
 ExpectationMaximizationQuantifier = EMQ
-DistributionMatchingY = DMy
 SLD = EMQ
+DistributionMatchingY = DMy
 HellingerDistanceY = HDy
 MedianSweep = MS
 MedianSweep2 = MS2

quapy/method/non_aggregative.py

@@ -150,6 +150,7 @@ class DMx(BaseQuantifier):
 class ReadMe(BaseQuantifier):
 
     def __init__(self, bootstrap_trials=100, bootstrap_range=100, bagging_trials=100, bagging_range=25, **vectorizer_kwargs):
+        raise NotImplementedError('under development ...')
         self.bootstrap_trials = bootstrap_trials
         self.bootstrap_range = bootstrap_range
         self.bagging_trials = bagging_trials

quapy/model_selection.py

@@ -25,6 +25,7 @@ class Status(Enum):
 
 
 class ConfigStatus:
+
     def __init__(self, params, status, msg=''):
         self.params = params
         self.status = status

quapy/tests/test_evaluation.py

@@ -6,14 +6,17 @@ import quapy as qp
 from sklearn.linear_model import LogisticRegression
 from time import time
 
-from quapy.error import QUANTIFICATION_ERROR_SINGLE, QUANTIFICATION_ERROR, QUANTIFICATION_ERROR_NAMES, \
-    QUANTIFICATION_ERROR_SINGLE_NAMES
+from quapy.error import QUANTIFICATION_ERROR_SINGLE_NAMES
 from quapy.method.aggregative import EMQ, PCC
 from quapy.method.base import BaseQuantifier
 
 
 class EvalTestCase(unittest.TestCase):
+
     def test_eval_speedup(self):
+        """
+        Checks whether the speed-up heuristics used by qp.evaluation work, i.e., actually save time
+        """
 
         data = qp.datasets.fetch_reviews('hp', tfidf=True, min_df=10, pickle=True)
         train, test = data.training, data.test
@@ -55,8 +58,11 @@ class EvalTestCase(unittest.TestCase):
         self.assertEqual(tend_no_optim>(tend_optim/2), True)
 
     def test_evaluation_output(self):
+        """
+        Checks the evaluation functions return correct types for different error_metrics
+        """
 
-        data = qp.datasets.fetch_reviews('hp', tfidf=True, min_df=10, pickle=True)
+        data = qp.datasets.fetch_reviews('hp', tfidf=True, min_df=10, pickle=True).reduce(n_train=100, n_test=100)
         train, test = data.training, data.test
 
         qp.environ['SAMPLE_SIZE']=100
@@ -79,6 +85,5 @@ class EvalTestCase(unittest.TestCase):
             self.assertEqual(scores.mean(), score)
 
 
-
 if __name__ == '__main__':
     unittest.main()

quapy/tests/test_hierarchy.py

@@ -1,19 +1,54 @@
 import unittest
 from sklearn.linear_model import LogisticRegression
+
+from quapy.method import AGGREGATIVE_METHODS, BINARY_METHODS
 from quapy.method.aggregative import *
+import inspect
 
 
 class HierarchyTestCase(unittest.TestCase):
 
     def test_aggregative(self):
         lr = LogisticRegression()
-        for m in [CC(lr), PCC(lr), ACC(lr), PACC(lr)]:
-            self.assertEqual(isinstance(m, AggregativeQuantifier), True)
+        for m in AGGREGATIVE_METHODS:
+            self.assertEqual(isinstance(m(lr), AggregativeQuantifier), True)
+
+    def test_inspect_aggregative(self):
+
+        import quapy.method.aggregative as aggregative
+
+        members = inspect.getmembers(aggregative)
+        classes = set([cls for name, cls in members if inspect.isclass(cls)])
+        quantifiers = [cls for cls in classes if issubclass(cls, BaseQuantifier)]
+        quantifiers = [cls for cls in quantifiers if issubclass(cls, AggregativeQuantifier)]
+        quantifiers = [cls for cls in quantifiers if not inspect.isabstract(cls) ]
+
+        for cls in quantifiers:
+            self.assertIn(cls, AGGREGATIVE_METHODS)
 
     def test_binary(self):
         lr = LogisticRegression()
-        for m in [HDy(lr)]:
-            self.assertEqual(isinstance(m, BinaryQuantifier), True)
+        for m in BINARY_METHODS:
+            self.assertEqual(isinstance(m(lr), BinaryQuantifier), True)
+
+    def test_inspect_binary(self):
+
+        import quapy.method.base as base
+        import quapy.method.aggregative as aggregative
+        import quapy.method.non_aggregative as non_aggregative
+        import quapy.method.meta as meta
+
+        members = inspect.getmembers(base)
+        members+= inspect.getmembers(aggregative)
+        members += inspect.getmembers(non_aggregative)
+        members += inspect.getmembers(meta)
+        classes = set([cls for name, cls in members if inspect.isclass(cls)])
+        quantifiers = [cls for cls in classes if issubclass(cls, BaseQuantifier)]
+        quantifiers = [cls for cls in quantifiers if issubclass(cls, BinaryQuantifier)]
+        quantifiers = [cls for cls in quantifiers if not inspect.isabstract(cls) ]
+
+        for cls in quantifiers:
+            self.assertIn(cls, BINARY_METHODS)
 
     def test_probabilistic(self):
         lr = LogisticRegression()
@@ -27,3 +62,4 @@ class HierarchyTestCase(unittest.TestCase):
 
 if __name__ == '__main__':
     unittest.main()
+

quapy/util.py

@@ -56,6 +56,7 @@ def parallel(func, args, n_jobs, seed=None, asarray=True, backend='loky'):
     :param seed: the numeric seed
     :param asarray: set to True to return a np.ndarray instead of a list
     :param backend: indicates the backend used for handling parallel works
+    :param open_args: if True, then the delayed function is called on *args_i, instead of on args_i
     """
     def func_dec(environ, seed, *args):
         qp.environ = environ.copy()
@@ -74,6 +75,40 @@ def parallel(func, args, n_jobs, seed=None, asarray=True, backend='loky'):
     return out
 
 
+def parallel_unpack(func, args, n_jobs, seed=None, asarray=True, backend='loky'):
+    """
+    A wrapper of multiprocessing:
+
+    >>> Parallel(n_jobs=n_jobs)(
+    >>>      delayed(func)(*args_i) for args_i in args
+    >>> )
+
+    that takes the `quapy.environ` variable as input silently.
+    Seeds the child processes to ensure reproducibility when n_jobs>1.
+
+    :param func: callable
+    :param args: args of func
+    :param seed: the numeric seed
+    :param asarray: set to True to return a np.ndarray instead of a list
+    :param backend: indicates the backend used for handling parallel works
+    """
+
+    def func_dec(environ, seed, *args):
+        qp.environ = environ.copy()
+        qp.environ['N_JOBS'] = 1
+        # set a context with a temporal seed to ensure results are reproducibles in parallel
+        with ExitStack() as stack:
+            if seed is not None:
+                stack.enter_context(qp.util.temp_seed(seed))
+            return func(*args)
+
+    out = Parallel(n_jobs=n_jobs, backend=backend)(
+        delayed(func_dec)(qp.environ, None if seed is None else seed + i, *args_i) for i, args_i in enumerate(args)
+    )
+    if asarray:
+        out = np.asarray(out)
+    return out
+
 @contextlib.contextmanager
 def temp_seed(random_state):
     """

setup.py

@@ -123,10 +123,9 @@ setup(
     #
     # Similar to `install_requires` above, these must be valid existing
     # projects.
-    # extras_require={  # Optional
-    #     'dev': ['check-manifest'],
-    #     'test': ['coverage'],
-    # },
+    extras_require={  # Optional
+       'bayes': ['jax', 'jaxlib', 'numpyro'],
+    },
 
     # If there are data files included in your packages that need to be
     # installed, specify them here.