Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
3
datasets available to search
ShareScore release 0.9.0
Dataset results
3 results for “UI Log”
UIS Log: Synthetic User Interface with Screenshots Log
<p>These data correspond to the set of problems for evaluating the proposal detailed in Martínez-Rojas et al. 2022. The evaluation utilizes a set of synthetic problems that simulate realistic administrative use cases. Each problem includes a UI Log with a synthetic screenshot corresponding to each event, capturing 3 distinct processes (<em>P</em>) marked by varying complexity levels. These levels are defined by the number of activities, the process execution variants, and the visual features influencing decisions between these variants.</p> <p>The implementation of this proposal can be found in the tool available at <a href="https://github.com/RPA-US/screenrpa" target="_new">this GitHub repository</a>, which utilizes the logs of these 3 processes for validation. Here they are described:<br><br></p> <ul> <li><em>P1 Client creation</em>. A process with <strong>5 activities and 2 variants</strong>. The single decision in this process is made based on the existence of an attachment in the reception email.</li> <li><em>P2 Client validation</em>. A process with <strong>7 activities and 2 variants</strong>. The decision is made based on the user’s response to a query.</li> <li><em>P3 Client deletion</em>. A process with <strong>7 activities and 4 variants</strong>. The decisions are made based on two conditions: (1) the existence of pending invoices and (2) the existence of an attachment to justify the payment of the invoices.</li> </ul> <p>These processes all contain a single decision point, although the one in P3 is complex. All processes include</p> <ol> <li>synthetic screen captures for their activities and</li> <li>a sample event log with a single instance for each variant.</li> </ol> <p>To generate the objects for the valuation, we generate event logs of different sizes (|<em>L</em>|) for each of these processes by deriving events from the sample event log. We consider log sizes in the range of {10, 25, 50, 100} events. Note that we consider complete instances in the log and thus, we remove the last instance if it goes beyond |<em>L</em>|.<br>Some of these logs are generated with a balanced number of instances, while others are unbalanced (<em>B</em>?) which present more than 20% of different frequency between the most frequent and less frequent variants. To average the result over a collection of problems, 30 instances are randomly generated for each tuple < <em>P</em>, |<em>L</em>|, <em>B</em>? >.<br>In this dataset there are 3 zips, one for each family. Each family corresponds to a process:</p> <ul> <li><em>Basic </em>corresponds to P1</li> <li><em>Intermediate </em>corresponds to P12</li> <li><em>Advanced </em>corresponds to P3</li> </ul> <p>Within these folders, we find 30 different scenarios (folder), in which the look and feel of the applications present in the screenshots have suffered little variations. Within each of these scenarios, variations are carried out respecting to the data entered in the forms and the images or attachments present in the user interface to generate log instances depending on the characteristics of each process.<br>For each scenario, we find 8 folders with the concrete problem which is defined by Log_size (in {10,25,50,100}) and Balanced (in {Balanced, Unbalanced}). The name of these folders have this format<em>: Family_LogSize_Balanced.</em><br>Inside each problem folder the UI Log and the screen captures can be found.<br><br><strong>References</strong><br><br>Martínez-Rojas, A., Jiménez-Ramírez, A., Enríquez, J. G., & Reijers, H. A. (2022, September). Analyzing variable human actions for robotic process automation. In <em>International Conference on Business Process Management</em> (pp. 75-90). Cham: Springer International Publishing.</p>
BPM Synthetic UI Logs Collection
<p>This data package described in the <em>BPM Demos&Resources</em> publication entitled: "<em>BPM Hub: An Open Collection of UI Logs</em>", consists of synthetic UI logs along with corresponding screenshots. The UI logs closely resemble real-world use cases within the administrative domain. They exhibit varying levels of complexity, measured by the number of activities, process variants, and visual features that influence the outcome of decision points. For its generation, the <a href="https://canela.lsi.us.es/bpmloggenerator/">BPM Log Generator tool</a> has been used, which requires the following initial generation configuration:</p> <p><strong>Initial Generation Configuration</strong></p> <ul> <li>Seed log: Includes a single instance for each process variant and their associated screenshots.</li> <li>Variability configuration: <ul> <li>Case-level: Refers to variations in the content that can be introduced or modified by the user, such as variations in the text inputs, selectable options, checkboxes, etc.</li> <li>Scenario-level: Refers to varying the GUI (Graphical User Interface) components related to the look and feel of the different applications appearing in the process screenshots.</li> </ul> </li> </ul> <p><strong>Data Package Contents</strong></p> <p>The data package comprises three distinct processes, P1, P2, P3, for which their initial configuration is provided, i.e., a tuple of <SeedLog, Case-level variability conf., Scenario-level variability conf.>. They are characterized by the following:</p> <p>P1. Client Creation</p> <ul> <li>Activities: 5</li> <li>Variants: 2</li> <li>Decision point: Revolves around the presence of an attachment in the reception of an email.</li> </ul> <p>P2. Client Deletion. User's presence in the system</p> <ul> <li>Activities: 7</li> <li>Variants: 2</li> <li>Decision point: Based on the result of the user's search in the Customer Management System (CRM), represented by a checkbox.</li> </ul> <p>P3. Client Deletion. Validation of customer payments</p> <ul> <li>Activities: 7</li> <li>Variants: 4</li> <li>Decision: Involves two conditions: <ol> <li>The presence of an attachment justifying the payment of the invoices in the email.</li> <li>The existence of pending invoices in the user CRM profile.</li> </ol> </li> </ul> <p>These problems depict processes with a single decision point, without cycles, and executed sequentially to ensure a non-interleaved execution pattern. Particularly, P3 shows higher complexity as its decision point is determined by two visual characteristics.</p> <p><strong>Generation of UI Logs</strong></p> <p>For each problem, case-level variations have been applied to generate logs with different sizes in the range of {10, 25, 50, 100} events. In cases where the log exceeds the desired size, the last instance is removed to maintain completeness. Each log size has its associated balanced and unbalanced log. Balanced logs have an approximately equal distribution of instances across variants, while unbalanced logs have a frequency difference of more than 20% between the most frequent and least frequent variants.</p> <p><strong>Scenarios</strong></p> <p>To ensure the reliability of the obtained results, 30 scenarios are generated for each tuple <Problem, LogSize, Balanced?>. These scenarios exhibit slight variations at the scenario-level, particularly in the look and feel and user interface of the applications depicted in the screenshots. Each scenario consists of UI logs that correspond to specific problems categorized by log size (10, 25, 50, 100) and balanced? (Balanced, Unbalanced). Folders containing UI logs and their corresponding screenshots are organized in folders named as follows: sc{scenarioId}_size_{LogSize}_{Balanced?}.</p> <p><strong>Additional Artefacts</strong></p> <p>In addition, each problem includes two more artefacts:</p> <ul> <li>initial_generation_configuration folder: Holds the data needed for problem data generation using the [5] tool.</li> <li>decision.json file: Specifies the condition driving the decision made at the decision point.</li> </ul> <p><strong>decision.json</strong></p> <p>The decision.json acts as a testing oracle, serving as a label for validating mined data. It contains two main sections: "UICompos" and "decision". The "UICompos" section includes a key for each activity related to the decision, storing key-value pairs that represent the UI components involved, along with their bounding box coordinates. The "decision" section defines the condition for a case to match a specific variant based on the mentioned UI components.</p>
UI Logs for Screenshot and for Activity Identification
<p>These data correspond to the set of Problems used for the evaluation of the proposal <strong><em>From Screenshots to Process Models: Improving Activity Identification through Screen Text.</em></strong></p> <p>Each problem includes a User-Interaction (UI) Log that contains comprehensive process information. Accompanying the UI Log are directories with 58 screenshots from different instances of processes recorded on Odoo or Gmail interfaces and directories containing all on-screen text extracted from each screenshot using OCR. The UI Log references the screenshots and their corresponding OCR files for each row.</p> <p>Additionally, results for every execution run per Problem are available. The results directories include subdirectories, differentiated by the image/text weight in the execution, containing the expanded base UI Log with columns resulting from feature extraction and clustering. Each Problem has its summary file, providing an overview of all executions and the relevant clustering and evaluation metrics.</p> <p>A precise breakdown and description of each of the directories and files present in this assessment is provided below:</p> <p><strong>input/Problem_X-Y/ </strong>(where X represents the number of the Problem given in the article and Y its full name).</p> <ul> <li><strong>screenshots/:</strong> contains the screenshots from Odoo and Gmail interfaces. The naming pattern is <em>X_img.png</em> where X is the corresponding identifier. Each record contains up to 58 screenshots representing different activities within the instantiation of a business process.</li> <li><strong>ocr_results/:</strong> stores the results of applying OCR techniques to all screenshots for extracting all on-screen text. This directory includes a .txt file for each image, containing the extracted text. The naming pattern is similar to the previous one: <em>X_ocr.txt </em>where X is the identifier. These files are used for executions that use all on-screen text as a source along with the images.</li> <li><strong>log.csv:</strong> .csv file representing the UI Log of the process. It contains the following columns: <ul> <li><strong>id:</strong> identifier corresponding to a screenshot within the UI log.</li> <li><strong>screenshot:</strong> path to the image corresponding to the row in the UI log within <em>screenshots/.</em></li> <li><strong>timestamp:</strong> time marker for defining the sequential order of actions in the UI log</li> <li><strong>header:</strong> text string extracted from the browser’s tab.</li> <li><strong>header_txt:</strong> path to each corresponding text file within <em>ocr_results/.</em></li> <li><strong>ground_truth:</strong> label assigned to identify and group images by activity. Compared with the <em>activity_label</em> column after algorithm application to determine execution’s evaluation metrics.</li> </ul> </li> </ul> <p><strong>output/X_Y/ </strong>(where X represents an abbreviation of the Problem chosen for execution and Y specifies whether all on-screen text, browser’s tab text or image hash was used).</p> <ul> <li><strong>X_Y_a_b/</strong> (where a,b refers to image/text weight for each execution). Each directory represents one specific execution from a Problem. <ul> <li><strong>df.csv:</strong> .csv file representing the expanded UI Log with the experimentation results after the algorithm has been applied. The new columns are described below: <ul> <li><strong>combined_features:</strong> contains the features extracted per image, either individually using hash or CLIP, or, in the latter case, combining them with those from the text source used.</li> <li><strong>activity_label:</strong> represents the group assigned by the algorithm through clustering based on feature comparison.</li> <li><strong>mapped_prediction:</strong> mapping of the cluster assigned by the algorithm to the group distribution established manually by <em>ground_truth.</em></li> </ul> </li> <li><strong>summary.csv: </strong>file which contains a summary of all executions from a Problem. It contains the following information: <ul> <li><strong>exec:</strong> number indicating the row identifier.</li> <li><strong>image_weight:</strong> parameter that determines the importance of image features in the model.</li> <li><strong>text_weight:</strong> parameter that determines the importance of text features in the model.</li> <li><strong>Evaluation metrics:</strong> <ul> <li><strong>Precision:</strong> the ratio of true positives to the sum of true positives and false positives. It measures the accuracy of not labeling different activities as the same.</li> <li><strong>Recall:</strong> the ratio of true positives to the sum of true positives and false negatives. It measures the ability to identify all the activities properly grouped.</li> <li><strong>F1-Score:</strong> a harmonic mean of precision and recall. It balances the relative contribution of precision and recall equally.</li> </ul> </li> <li><strong>Clustering related metrics. </strong>(Calculated for testing purposes; in the article, our approach focuses solely on evaluation metrics). <ul> <li><strong>Silhoutte Coefficient:</strong> measures how similar is an object to its cluster compared to other clusters. Values range from -1 to 1, with higher values indicating better-defined clusters.</li> <li><strong>Davies-Bouldin Index:</strong> assesses cluster quality by comparing the distance between clusters to the size of the clusters themselves. Lower values indicate better-defined clusters.</li> <li><strong>Calinski-Harabasz Index:</strong> evaluate cluster quality by comparing the spread within clusters to the spread between clusters. Higher values mean better-defined clusters.</li> </ul> </li> </ul> </li> </ul> </li> </ul>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.