Performance Evolution Matrix: Visualizing Performance Variations along Software Versions

<p># Performance Evolution Matrix<br> This repository contains the artifacts needed to replicate our experiment in the paper &quot;Performance Evolution Matrix&quot;.</p> <p># Video Demo<br> [download](https://github.com/jpsandoval/PerfEvoMatrix/blob/master/MatrixMovie.mp4)<br> <br> # XMLSupport and GraphET Examples</p> <p>To open the XMLSupport and GraphET Examples (which appears in the paper) execute the following commands in a Terminal.</p> <p>**MacOSX.** We do all the experiments in a Mac Book Pro. To open the Matrix execute the following command in the folder where this project was downloaded.&nbsp;</p> <p>```<br> ./Pharo-OSX/Pharo.app/Contents/MacOS/Pharo Matrix.image<br> ```</p> <p>**Windows.**<br> You may also run the experiment in Windows, but depending on the windows version you have installed it may be some some UI bugs.<br> ```<br> cd Pharo-Windows<br> Pharo.exe ../XMLSupportExample.image<br> ```</p> <p>**Open the Visualization.**<br> Please select the following code, then execute it using the green play button (at the top right of the window).<br> ```<br> ToadBuilder xmlSupportExample.<br> ```<br> or&nbsp;<br> ```<br> ToadBuilder graphETExample.<br> ```<br> **Note.** There are two buttons at the panel top left In (zoom in) and Out (zoom out). To move the visualization just drag the moves over the panel.</p> <p># Experiment<br> This subsection describe how to execute the tools, for replicating our experiment.</p> <p>## Baseline<br> The baseline contains the tools and the project-dataset to realize the tasks described in the paper (identifying and understanding performance variations).</p> <p>## Open the Baseline</p> <p>**MacOSX.** We do all the experiments in a Mac Book Pro. To open the Baseline execute the following command in the folder where this project was downloaded.&nbsp;</p> <p>```<br> ./Pharo-OSX/Pharo.app/Contents/MacOS/Pharo Baseline.image<br> ```</p> <p>**Windows.**<br> You may also run the experiment in Windows, but depending on the windows version you have installed it may be some some UI bugs.<br> ```<br> cd Pharo-Windows<br> Pharo.exe ../Baseline.image<br> ```</p> <p>## Open a Project</p> <p>There are three projects under study, depending on the project you wanna use for the task, you may execute one of the following scripts. For executing a script press Cmd-d or right-click and press do it.</p> <p>**Roassal**<br> ```<br> TProfileVersion openRoassal.<br> ```</p> <p>**XML**<br> ```<br> TProfileVersion openXML.<br> ```<br> **Grapher**<br> ```<br> TProfileVersion openGrapher.<br> ```</p> <p>## Baseline Options<br> For each project, we provide a UI which contains all the tools we use as a baseline. Each item in the list is a version of the selected project.</p> <p>&lt;img src=&quot;images/baseline.png&quot; width=&quot;300&quot;&gt;</p> <p>- Browse: open a standard window to inspect the code of the project in the selected version.<br> - Profile: open a window with a call context tree for the selected version.<br> - Source Diff: open a window with the code differences between the selected version and the previous one.<br> - Execution Diff: open a window with the merge call context tree gathered from the selected version and the previous one.&nbsp;</p> <p>**Note.** All these options require you select first a item in the list.</p> <p># Matrix</p> <p>## Open Matrix Image.</p> <p>**MacOSX.** We do all the experiments in a Mac Book Pro. To open the Matrix execute the following command in the folder where this project was downloaded.&nbsp;</p> <p>```<br> ./Pharo-OSX/Pharo.app/Contents/MacOS/Pharo Matrix.image<br> ```</p> <p>**Windows.**<br> You may also run the experiment in Windows, but depending on the windows version you have installed it may be some some UI bugs.<br> ```<br> cd Pharo-Windows<br> Pharo.exe ../Matrix.image<br> ```</p> <p>## Open a project</p> <p>There are three projects under study, depending on the project you wanna use for the task, you may execute one of the following scripts. For executing a script press Cmd-d or right-click and press do it.</p> <p>**Roassal**<br> ```<br> ToadBuilder roassal.<br> ```</p> <p>**XML**<br> ```<br> ToadBuilder xml.<br> ```<br> **Grapher**<br> ```<br> ToadBuilder grapher.<br> ```</p> <p># Data Gathering</p> <p>Before each participant starts a task we execute the following script in Smalltalk. For executing a script press Cmd-d or right-click and press do it. It allows us to track the time that a user starts the experiment and how many mouse clicks, movements.<br> ```<br> UProfiler newSession.<br> UProfiler current start.<br> ```</p> <p>After finishing the task we executed the following script. It stop recording the mouse events and save the stops time.<br> ```<br> UProfiler current end.<br> ```</p> <p>The last script generates a file with the following information: start time, end time, number of clicks, number of mouse movements, and the number of mouse drags (we do not use this last one).<br> ```<br> 11:34:52.5205 am,11:34:56.38016 am,14,75,0</p> <p>```<br> # Quit<br> To close the artifact, just close the window or press click in any free space of the window and select quit.</p>