Block-wise sparse matrix-vector product dataset and convolutional neural nets for estimating the run time and energy consumption of the sparse matrix-vector product

<p><strong>Introduction</strong></p> <p><strong>SpMV-CNN</strong> is a set of Convolutional Neural Networks (CNNs) that provide&nbsp;accurate estimations of the performance and energy consumption of the SpMV kernel. The proposed CNN-based models use a block-wise approach to make the CNN&nbsp;architecture independent of the matrix size. These models cat be trained to estimate run time as well as total, package and DRAM energy consumption at different processor frequencies.</p> <p><strong>Prerequisites</strong></p> <p><strong>SpMV-CNN</strong> requires Python3 with the following packages:</p> <pre><code>keras==2.1.6 tensorflow==1.8.0 h5py==2.7.1 matplotlib==2.1.1 scikit-learn==0.19.1 </code></pre> <p><strong>Obtaining the dataset</strong></p> <p>The execution time and energy consumption data corresponding to the SpMV&nbsp;operation on a set of sparse matrices from the SuiteSparse Matrix Collection have been obtained on an Intel Xeon E5-2630 core running at frequencies 1.2,&nbsp;1.6, 2.0, 2.4 GHz. The energy consumption measurements are obtained via the Intel RAPL interface and gathered at three different levels (total, package and DRAM, where total = package + DRAM) for this specific processor.</p> <p>The <code>spmv-cnn-dataset.tgz</code> archive contains the whole dataset, including&nbsp;the following HDF5 files:</p> <pre><code>$ tree . |-- test | |-- f_1200000_b250 | | |-- output_2cubes_sphere_1200000.h5 | | |-- output_apache2_1200000.h5 | | |-- output_bcsstk36_1200000.h5 | | |-- output_cfd1_1200000.h5 | | |-- output_cfd2_1200000.h5 | | |-- output_ct20stif_1200000.h5 | | |-- output_denormal_1200000.h5 | | |-- output_Dubcova2_1200000.h5 | | |-- output_Dubcova3_1200000.h5 | | |-- output_ecology2_1200000.h5 | | |-- output_gyro_1200000.h5 | | |-- output_gyro_k_1200000.h5 | | |-- output_msc10848_1200000.h5 | | |-- output_msc23052_1200000.h5 | | |-- output_nasasrb_1200000.h5 | | |-- output_nd3k_1200000.h5 | | |-- output_offshore_1200000.h5 | | |-- output_oilpan_1200000.h5 | | |-- output_olafu_1200000.h5 | | |-- output_parabolic_fem_1200000.h5 | | |-- output_qa8fm_1200000.h5 | | |-- output_raefsky4_1200000.h5 | | |-- output_s3dkq4m2_1200000.h5 | | |-- output_s3dkt3m2_1200000.h5 | | |-- output_ship_001_1200000.h5 | | |-- output_ship_003_1200000.h5 | | |-- output_shipsec1_1200000.h5 | | |-- output_shipsec5_1200000.h5 | | |-- output_shipsec8_1200000.h5 | | |-- output_smt_1200000.h5 | | |-- output_thermomech_dM_1200000.h5 | | |-- output_thread_1200000.h5 | | `-- output_vanbody_1200000.h5 | |-- f_1600000_b250 | | |-- output_2cubes_sphere_1600000.h5 | | |—- ... | | `-- output_vanbody_1600000.h5 | |-- f_2000000_b250 | | |-- output_2cubes_sphere_2000000.h5 | | |—- ... | | `-- output_vanbody_2000000.h5 | `-- f_2400000_b250 | |-- output_2cubes_sphere_2400000.h5 | |—- ... | `-- output_vanbody_2400000.h5 |-- test_pagerank | |-- f_1200000_b250 | | |-- output_adaptive_1200000.h5 | | |-- output_cit-HepPh_1200000.h5 | | |-- output_delaunay_n22_1200000.h5 | | |-- output_email-Enron_1200000.h5 | | |-- output_email-EuAll_1200000.h5 | | |-- output_europe_osm_1200000.h5 | | |-- output_hugebubbles-00020_1200000.h5 | | |-- output_rgg_n_2_24_s0_1200000.h5 | | |-- output_road_usa_1200000.h5 | | |-- output_Stanford_1200000.h5 | | |-- output_wb-edu_1200000.h5 | | |-- output_web-BerkStan_1200000.h5 | | |-- output_web-Google_1200000.h5 | | |-- output_web-NotreDame_1200000.h5 | | |-- output_wiki-Talk_1200000.h5 | | `-- output_wiki-Vote_1200000.h5 | |-- f_1600000_b250 | | |-- output_adaptive_1600000.h5 | | |—- ... | | `-- output_wiki-Vote_1600000.h5 | |-- f_2000000_b250 | | |-- output_adaptive_2000000.h5 | | |—- ... | | `-- output_wiki-Vote_2000000.h5 | `-- f_2400000_b250 | |-- output_adaptive_2400000.h5 | |—- ... | `-- output_wiki-Vote_2400000.h5 `-- train |-- merged_energy_train_shuffle_f1200000_250.h5 |-- merged_energy_train_shuffle_f1600000_250.h5 |-- merged_energy_train_shuffle_f2000000_250.h5 `-- merged_energy_train_shuffle_f2400000_250.h5 </code></pre> <p>The matrices contained in the merged training files (<code>merged_energy_train_shuffle_fXX00000_250.h5</code>) are the following:</p> <pre><code>$ tree . |-- output_af_0_k101_1200000.h5 |-- output_af_1_k101_1200000.h5 |-- output_af_2_k101_1200000.h5 |-- output_af_3_k101_1200000.h5 |-- output_af_4_k101_1200000.h5 |-- output_af_5_k101_1200000.h5 |-- output_af_shell10_1200000.h5 |-- output_af_shell1_1200000.h5 |-- output_af_shell2_1200000.h5 |-- output_af_shell3_1200000.h5 |-- output_af_shell4_1200000.h5 |-- output_af_shell5_1200000.h5 |-- output_af_shell6_1200000.h5 |-- output_af_shell7_1200000.h5 |-- output_af_shell8_1200000.h5 |-- output_af_shell9_1200000.h5 |-- output_atmosmodd_1200000.h5 |-- output_atmosmodj_1200000.h5 |-- output_atmosmodl_1200000.h5 |-- output_audikw_1_1200000.h5 |-- output_BenElechi1_1200000.h5 |-- output_bmw3_2_1200000.h5 |-- output_bmw7st_1_1200000.h5 |-- output_bmwcra_1_1200000.h5 |-- output_bone010_1200000.h5 |-- output_boneS01_1200000.h5 |-- output_boneS10_1200000.h5 |-- output_bundle_adj_1200000.h5 |-- output_cage14_1200000.h5 |-- output_cage15_1200000.h5 |-- output_circuit5M_1200000.h5 |-- output_circuit5M_dc_1200000.h5 |-- output_CO_1200000.h5 |-- output_consph_1200000.h5 |-- output_CoupCons3D_1200000.h5 |-- output_crankseg_1_1200000.h5 |-- output_crankseg_2_1200000.h5 |-- output_CurlCurl_2_1200000.h5 |-- output_CurlCurl_3_1200000.h5 |-- output_CurlCurl_4_1200000.h5 |-- output_dielFilterV2real_1200000.h5 |-- output_dielFilterV3real_1200000.h5 |-- output_Emilia_923_1200000.h5 |-- output_ESOC_1200000.h5 |-- output_F1_1200000.h5 |-- output_F2_1200000.h5 |-- output_Fault_639_1200000.h5 |-- output_Freescale1_1200000.h5 |-- output_Freescale2_1200000.h5 |-- output_FullChip_1200000.h5 |-- output_G3_circuit_1200000.h5 |-- output_Ga10As10H30_1200000.h5 |-- output_Ga19As19H42_1200000.h5 |-- output_Ga3As3H12_1200000.h5 |-- output_Ga41As41H72_1200000.h5 |-- output_Ge87H76_1200000.h5 |-- output_Ge99H100_1200000.h5 |-- output_Geo_1438_1200000.h5 |-- output_gsm_106857_1200000.h5 |-- output_Hardesty3_1200000.h5 |-- output_hood_1200000.h5 |-- output_Hook_1498_1200000.h5 |-- output_human_gene1_1200000.h5 |-- output_human_gene2_1200000.h5 |-- output_inline_1_1200000.h5 |-- output_JP_1200000.h5 |-- output_kkt_power_1200000.h5 |-- output_ldoor_1200000.h5 |-- output_Long_Coup_dt0_1200000.h5 |-- output_Long_Coup_dt6_1200000.h5 |-- output_mat_104_10000_1200000.h5 |-- output_mat_104_1000_1200000.h5 |-- output_mat_104_5000_1200000.h5 |-- output_mat_112_10000_1200000.h5 |-- output_mat_112_1000_1200000.h5 |-- output_mat_112_5000_1200000.h5 |-- output_mat_120_10000_1200000.h5 |-- output_mat_120_1000_1200000.h5 |-- output_mat_120_5000_1200000.h5 |-- output_mat_128_10000_1200000.h5 |-- output_mat_128_1000_1200000.h5 |-- output_mat_128_5000_1200000.h5 |-- output_mat_16_10000_1200000.h5 |-- output_mat_16_1000_1200000.h5 |-- output_mat_16_5000_1200000.h5 |-- output_mat_24_10000_1200000.h5 |-- output_mat_24_1000_1200000.h5 |-- output_mat_24_5000_1200000.h5 |-- output_mat_32_10000_1200000.h5 |-- output_mat_32_1000_1200000.h5 |-- output_mat_32_5000_1200000.h5 |-- output_mat_40_10000_1200000.h5 |-- output_mat_40_1000_1200000.h5 |-- output_mat_40_5000_1200000.h5 |-- output_mat_48_10000_1200000.h5 |-- output_mat_48_1000_1200000.h5 |-- output_mat_48_5000_1200000.h5 |-- output_mat_56_10000_1200000.h5 |-- output_mat_56_1000_1200000.h5 |-- output_mat_56_5000_1200000.h5 |-- output_mat_64_10000_1200000.h5 |-- output_mat_64_1000_1200000.h5 |-- output_mat_64_5000_1200000.h5 |-- output_mat_72_10000_1200000.h5 |-- output_mat_72_1000_1200000.h5 |-- output_mat_72_5000_1200000.h5 |-- output_mat_80_10000_1200000.h5 |-- output_mat_80_1000_1200000.h5 |-- output_mat_80_5000_1200000.h5 |-- output_mat_8_10000_1200000.h5 |-- output_mat_8_1000_1200000.h5 |-- output_mat_8_5000_1200000.h5 |-- output_mat_88_10000_1200000.h5 |-- output_mat_88_1000_1200000.h5 |-- output_mat_88_5000_1200000.h5 |-- output_mat_96_10000_1200000.h5 |-- output_mat_96_1000_1200000.h5 |-- output_mat_96_5000_1200000.h5 |-- output_memchip_1200000.h5 |-- output_ML_Laplace_1200000.h5 |-- output_mouse_gene_1200000.h5 |-- output_msdoor_1200000.h5 |-- output_m_t1_1200000.h5 |-- output_nd12k_1200000.h5 |-- output_nd24k_1200000.h5 |-- output_nd6k_1200000.h5 |-- output_nlpkkt120_1200000.h5 |-- output_nlpkkt80_1200000.h5 |-- output_PFlow_742_1200000.h5 |-- output_pwtk_1200000.h5 |-- output_rajat31_1200000.h5 |-- output_RM07R_1200000.h5 |-- output_Rucci1_1200000.h5 |-- output_Serena_1200000.h5 |-- output_Si34H36_1200000.h5 |-- output_Si41Ge41H72_1200000.h5 |-- output_Si87H76_1200000.h5 |-- output_SiO2_1200000.h5 |-- output_sls_1200000.h5 |-- output_StocF-1465_1200000.h5 |-- output_TEM152078_1200000.h5 |-- output_TEM181302_1200000.h5 |-- output_thermal2_1200000.h5 |-- output_tmt_sym_1200000.h5 |-- output_torso1_1200000.h5 |-- output_Transport_1200000.h5 |-- output_TSOPF_FS_b300_c2_1200000.h5 |-- output_TSOPF_FS_b300_c3_1200000.h5 |-- output_TSOPF_RS_b2383_1200000.h5 |-- output_TSOPF_RS_b2383_c1_1200000.h5 |-- output_TSOPF_RS_b678_c2_1200000.h5 `-- output_x104_1200000.h5 f_1600000_b250 |-- output_af_0_k101_1600000.h5 |—- ... `-- output_x104_1600000.h5 f_2000000_b250 |-- output_af_0_k101_2000000.h5 |—- ... `-- output_x104_2000000.h5 f_2400000_b250 |-- output_af_0_k101_2400000.h5 |—- ... `-- output_x104_2400000.h5 </code></pre> <p><strong>Creating your own dataset</strong></p> <p>If you wish to create your own training/testing dataset on a different target&nbsp;architecture you need to take the following steps:</p> <ol> <li> <p>Build the SpMV driver:</p> <ol> <li> <p>Go to <code>cd SpMV-driver/src</code></p> </li> <li> <p>Edit makefile and set the PAPI and HDF5&nbsp;install prefixes.</p> </li> <li> <p>Build the driver via <code>make.</code></p> </li> </ol> </li> <li> <p>Run the SpMV driver:&nbsp;</p> <p><code>./driver &lt;arg0&gt; &lt;arg1&gt; ...</code></p> <p>List of driver arguments:</p> <pre><code>matrix = audikw_1.rb # Input matrix in rb format reps = 10000 # Number of repetitions of the operation to avoid overhead block_size_ini = 250 # Minimum block size block_size_end = 1000 # Maximum block size increment = 250 # Increment between block sizes base = 0 # Starting nnz of the matrix freq = [2400000, 2000000, 1600000, 1200000] # Operating frequency sym = 1 # If 1 the matrix is symmetric. If 0 the matrix is no-symmetric.</code></pre> <p>Example:</p> <p><code>numactl --membind 0 taskset -c 0 ./src/driver audikw_1.rb 10000 250 1000 250 0 2400000</code></p> <p>Note that <code>numactl</code> and <code>taskset</code> utilities are used to guarantee both NUMA and&nbsp;process-to-core affinity.</p> </li> <li> <p>Generating the dataset:</p> <ol> <li> <p>Edit the <code>SpMV-driver/run_all.sh</code> and uncomment the line <code>matrices =</code> in order to launch the driver for Train_symmetric / Train_noSymmetric /&nbsp;Test_symmetric / Test_noSymmetric matrices.</p> </li> <li> <p>Edit the 3rd parameter in the command SpMV-driver/run_driver.sh: 1 for&nbsp;symmetric matrices, 2 for unsymmetric matrices.</p> </li> <li> <p>Edit the command in SpMV-driver/run_driver.sh to select the input parameters&nbsp;of the driver as explained before.</p> </li> <li> <p>Run <code>SpMV-driver/run_all.sh</code> to obtain <code>hdf5</code> files that will create the&nbsp;dataset.</p> </li> </ol> </li> <li> <p>Merging the dataset:</p> <p>Run the script</p> <p><code>python3 SpMV-driver/merge_train_matrices.py /path/to/hdf5/matrix/files /output/path</code></p> <p>to obtain a single <code>hdf5</code> file containing all data from individual <code>hdf5</code> files&nbsp;obtained in the previous step. This merged file is the training dataset.</p> </li> </ol> <p><strong>Hyperparameter search</strong></p> <p>The script <code>spmv_cnn_hyperas.py</code> performs the hyperparameter search via the&nbsp;Hyperas tool. This script requires the hdf5 file dataset in the directory&nbsp;<code>dataset/train/</code> and produces both a <code>best_model_*.json</code> and <code>best_run_*.json&nbsp;</code>files in the&nbsp;<code>results/models/</code> directory containing the model structure and&nbsp;hyperparameters of the best performing configuration.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_hyper.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) at which the&nbsp;dataset was generated and <code>Time</code> the modeled metric. According to the labels in&nbsp;the dataset, the hyperparameter search can also be performed with the <code>Energy</code>,&nbsp;<code>EPKG</code> and <code>EDRAM</code> metrics, corresponding to the energy measured by the Intel&nbsp;RAPL counters from our Intel Xeon Haswell core. In our case, however, we only&nbsp;search hyperparameters for the <code>Time</code> and <code>Energy</code> metrics at 2.4 GHz. Other&nbsp;metrics and frequencies inherit the best performing model and settings from the&nbsp;previous configuration.</p> <p><strong>Training</strong></p> <p>The script <code>spmv_cnn_train.py</code> performs the training on the best performing&nbsp;models obtained on the previous step. For that, it uses both the&nbsp;<code>best_model_*.json</code> and <code>best_run_*.json</code> files obtained in the hyperparameter&nbsp;search.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_train.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) and <code>Time</code> the&nbsp;modeled metric. The training should be performed per metric and frequency. The&nbsp;training produces a file that contains the trained weights, so the model is&nbsp;ready for performing inference (testing).</p> <p><strong>Testing</strong></p> <p>The script <code>spmv_cnn_test.py</code> performs the test on the set of testing matrices&nbsp;involved in the SpMV operation.</p> <p>This script can be invoked in the following way:</p> <p><code>python3 spmv_cnn_test.py 2400000 Time</code></p> <p>where <code>2400000</code> is the operating processor frequency (2.4 GHz) and <code>Time</code> the&nbsp;modeled metric. The test should be performed per metric and frequency. The&nbsp;training produces two files in the <code>results/tests/</code> directory:</p> <ul> <li><code>Pred_*.txt</code>: This file contains the real measurements and the predictions&nbsp;obtained by the CNN for the individual vpos blocks of the testing matrices.</li> <li><code>Test_*.txt</code>: This file summarizes the information of <code>Pred_*.txt</code> file,&nbsp;showing the average relative error among the blocks of each test matrix and the&nbsp;total relative error, which is computed by summing up the real measurements and&nbsp;the predictions for all the blocks of a same matrix and computing the relative&nbsp;error upon those values.</li> </ul> <p><em>Note that this testing step and the two previous steps (hyperparameter search&nbsp;and training) can be performed at once using the <code>run.sh</code> script.</em></p> <p><strong>References</strong></p> <p>Publications describing <strong>SpMV-CNN-Model</strong>:</p> <ul> <li>Barreda, M., Dolz, M.F., Casta&ntilde;o, M.A. et al. Performance modeling of the&nbsp;sparse matrix&ndash;vector product via convolutional neural networks. J Supercomputing (2020). <a href="https://doi.org/10.1007/s11227-020-03186-1">https://doi.org/10.1007/s11227-020-03186-1</a></li> </ul> <p><strong>Acknowledgments</strong></p> <p>The <strong>SpMV-CNN-Model</strong> research has been partially supported by:</p> <ul> <li> <p>Project TIN2017-82972-R <strong>&ldquo;Agorithmic Techniques for Energy-Aware and&nbsp;Error-Resilient High Performance Computing&rdquo;</strong> funded by the Spanish Ministry of Economy and Competitiveness (2018-2020).</p> </li> <li> <p>Project CDEIGENT/2017/04 <strong>&ldquo;High Performance Computing for Neural Networks&rdquo;&nbsp;</strong>funded by the Valencian Government.</p> </li> <li> <p>Project UJI-A2019-11 <strong>&ldquo;Energy-Aware High Performance Computing for Deep&nbsp;Neural Networks&rdquo;</strong> funded by the Universitat Jaume I.</p> </li> </ul>