Embeddings Augmented by Random Permutations (EARP)

<p>This data set includes the word embeddings used for our CoNLL 2018 paper, &quot;<em>Bringing Order to Neural Word Embeddings with Embeddings Augmented By Random Permutations&quot;</em>.</p> <p>As described in the paper, encoding word order with random permutations leads to substantive improvements in performance across multiple analogical retrieval tasks, most notably for &quot;syntactic&quot; analogies that involve mapping between morphological derivatives. Smaller improvements are also evident in downstream sequence labeling tasks.&nbsp;</p> <p>Both baseline Skipgram-with-negative-sampling and permutation-based vectors are provided in two: formats - the Semantic Vectors (https://github.com/semanticvectors/semanticvectors) binary format (.bin) and the word2vec binary format (<em>.w2v.bin</em>).</p> <p>All models were created with two different sliding window configurations - radius 2 (<em>r2</em>) and radius (<em>r5</em>); as well as with and without subword embeddings (<em>sw</em>). The models are as follows:</p> <p><em>basic, fastText</em>: standard Skipgram-with-negative-sampling, implemented in Semantic Vectors and fastText respectively</p> <p><em>earp_directional</em>: these models use random permutations (EARP) to distinguish between positions before and after the focus term in a sliding window</p> <p><em>earp_positional</em>: these models use random permutations (EARP) to distinguish between each position in a&nbsp;sliding window</p> <p><em>earp_proximity</em>: these models use random permutations (EARP) to generate encodings that are different, but not orthogonal, for neighboring positions in a sliding window.&nbsp;</p> <p><em>earpx variants</em>: these models use exact sliding window position without replacing subsampled terms, as occurs in other models.&nbsp;</p> <p>The word2vec binary editions include&nbsp;random vectors for some characters and terms&nbsp;that were eliminated by Semantic Vectors&#39;&nbsp;Lucene-based tokenization procedure -&nbsp;in particular punctuation marks - as these are useful for certain&nbsp;downstream machine learning tasks.</p>