UCI and OpenML Data Sets for Ordinal Quantification

<p>These four labeled data sets are targeted at ordinal quantification. The goal of quantification is not to predict the label of each individual instance, but the distribution of labels in unlabeled sets of data.</p> <p>With the scripts provided, you can extract CSV files from the UCI machine learning repository and from OpenML. The ordinal class labels stem from a binning of a continuous regression label.</p> <p>We complement this data set with the indices of data items that appear in each sample of our evaluation. Hence, you can precisely replicate our samples by drawing the specified data items. The indices stem from two evaluation protocols that are well suited for ordinal quantification. To this end, each row in the files <em>app_val_indices.csv</em>, <em>app_tst_indices.csv</em>, <em>app-oq_val_indices.csv</em>, and <em>app-oq_tst_indices.csv</em> represents one sample.</p> <p>Our first protocol is the artificial prevalence protocol (APP), where all possible distributions of labels are drawn with an equal probability. The second protocol, APP-OQ, is a variant thereof, where only the smoothest 20% of all APP samples are considered. This variant is targeted at ordinal quantification tasks, where classes are ordered and a similarity of neighboring classes can be assumed.</p> <p><strong>Usage</strong></p> <p>You can extract four CSV files through the provided script <em>extract-oq.jl</em>, which is conveniently wrapped in a <em>Makefile</em>. The <em>Project.toml</em> and <em>Manifest.toml</em> specify the Julia package dependencies, similar to a requirements file in Python.</p> <p><strong>Preliminaries:</strong> You have to have a working Julia installation. We have used Julia v1.6.5 in our experiments.</p> <p><strong>Data Extraction:</strong> In your terminal, you can call either</p> <pre><code>make</code></pre> <p>(recommended), or</p> <pre><code>julia --project="." --eval "using Pkg; Pkg.instantiate()" julia --project="." extract-oq.jl</code></pre> <p><strong>Outcome: </strong>The first row in each CSV file is the header. The first column, named &quot;class_label&quot;, is the ordinal class.</p> <p><strong>Further Reading</strong></p> <p>Implementation of our experiments: <a href="https://github.com/mirkobunse/regularized-oq">https://github.com/mirkobunse/regularized-oq</a></p>