Connecting The Dots: Low Loss Paths Between Neighbor Embeddings

<h3>This repository contains animations of the figures in&nbsp;<a href="https://github.com/premfi/NE-AutoNEB/blob/main/Thesis.pdf" target="_blank" rel="noopener">Connecting The Dots: Low Loss Paths Between Neighbor Embeddings (2024)</a>.</h3> <h2>Abstract</h2> <p>UMAP and t-SNE belong to the most common dimension reduction techniques for the visualization of high-dimensional data across various domains. Previous work focused largely on finding the best minima of these algorithms. We connect different minima on the loss surface, using the AutoNEB algorithm to optimize for a low loss path.&nbsp;For UMAP, we find paths with a loss continuously within the range of usual fluctuations between different minima. For t-SNE, the loss rises with increasing distance from the initial minima and is not brought down fully even by further optimization.&nbsp;Additionally, we introduce a new way to visualize the loss contribution of individual points, which provides an intuitive overview over the local embedding quality.</p> <h2>Explanation</h2> <p>When running a dimension reduction technique, such as UMAP and t-SNE, multiple times on the same dataset, the results can differ significantly. The differences are especially considerable when the embeddings are initialized randomly.<br>Several largely different embeddings can have a similarly low loss, underscoring that only local minima are found.<br>The goal was to find a connection between these local minima with a continuously relatively low loss. This corresponds to embeddings being transformed into one another while maintaining a relatively good visualization quality throughout. The results of these low-loss transformations are the .gif files uploaded here.</p> <p>The AutoNEB algorithm starts with an initial path between the minima, usually with a high loss, and then iteratively optimizes the points along the path using gradient descent.&nbsp;The default method for creating the initial path is just interpolating linearly between the two embeddings.&nbsp;This method has been used for all of the connections, except for "t-SNE_Fashion-MNIST_preconfigured_*.gif", where the initial path was set manually.</p> <p>For more details please refer to the <a title="Connecting The Dots: Low Loss Paths Between Neighbor Embeddings (2024)" href="https://github.com/premfi/NE-AutoNEB/blob/main/Thesis.pdf" target="_blank" rel="noopener">thesis</a>, especially chapter 3 "Methods".</p> <h2>Files sorted by corresponding Figure</h2> <table> <tbody> <tr> <td>Fig. 4.4</td> <td>UMAP on Fashion-MNIST</td> <td>UMAP_Fashion-MNIST_*.gif</td> </tr> <tr> <td>Fig. 4.5</td> <td>UMAP on C. Elegans</td> <td>UMAP_C.Elegans_*.gif</td> </tr> <tr> <td>Fig, 4.7</td> <td>t-SNE on Fashion-MNIST</td> <td>t-SNE_Fashion-MNIST_*.gif</td> </tr> <tr> <td>Fig. 4.8</td> <td>t-SNE on C. Elegans</td> <td>t-SNE_C.Elegans_*.gif</td> </tr> <tr> <td>Fig. 4.10</td> <td>t-SNE on Fashion-MNIST with preconfigured path</td> <td>t-SNE_Fashion-MNIST_preconfigured_*.gif</td> </tr> </tbody> </table> <h3>Each file exists in a "fast" and "slow" variant:</h3> <p>"*_fast.gif" files linearly interpolate between each existing pair of pivots with 2 additional frames.<br>"*_slow.gif" files linearly interpolate between each existing pair of pivots with 4 additional frames.</p>