Freebase Datasets for Robust Evaluation of Knowledge Graph Link Prediction Models

<p><strong>Freebase</strong> is amongst the largest public cross-domain knowledge graphs. It possesses three main data modeling idiosyncrasies. It&nbsp;has a strong <strong>type system</strong>; its properties are purposefully represented in&nbsp;<strong>reverse pairs</strong>; and it uses <strong>mediator objects</strong> to represent multiary relationships. These design choices are important in modeling the real-world. But&nbsp;they also pose nontrivial challenges in research of embedding models for&nbsp;knowledge graph completion, especially when models are developed and&nbsp;evaluated agnostically of these idiosyncrasies. We&nbsp;make&nbsp;available several&nbsp;variants of the Freebase dataset by inclusion and exclusion of these data&nbsp;modeling idiosyncrasies. This is the first-ever publicly available <strong>full-scale</strong> Freebase dataset&nbsp;that has gone through <strong>proper preparation</strong>.&nbsp;</p><p>&nbsp;</p><p>Dataset Details</p><p>The dataset consists of the four variants of Freebase dataset as well as related mapping/support files. For each variant, we made three kinds of files available:</p><ul><li>Subject matter triples file<ul><li><i>fb+/-CVT+/-REV</i>&nbsp;One folder for each variant. In each folder there are 5 files: train.txt, valid.txt, test.txt, entity2id.txt, relation2id.txt Subject matter triples are the triples belong to subject matters domains—domains describing real-world facts.<ul><li>Example of a row in train.txt, valid.txt, and test.txt:&nbsp;<ul><li>2, 192, 0</li></ul></li><li>Example of a row in entity2id.txt:<ul><li>/g/112yfy2xr, 2</li></ul></li><li>Example of a row in relation2id.txt:<ul><li>/music/album/release_type, 192</li></ul></li><li>Explaination<ul><li>"/g/112yfy2xr" and "/m/02lx2r" are the MID of the subject entity and object entity, respectively. "/music/album/release_type" is the realtionship between the two entities. 2, 192, and 0 are the IDs assigned by the authors to the objects.</li></ul></li></ul></li></ul></li><li>Type system file<ul><li><i>freebase_endtypes</i>: Each row maps an edge type to its required subject type and object type.<ul><li>Example<ul><li>92, 47178872, 90</li></ul></li><li>Explanation<ul><li>"92" and "90" are the type id of the subject and object which has the relationship id "47178872".</li></ul></li></ul></li></ul></li><li>Metadata files<ul><li><i>object_types</i>: Each row maps the MID of a Freebase object to a type it belongs to.<ul><li>Example<ul><li>/g/11b41c22g, /type/object/type, /people/person</li></ul></li><li>Explanation<ul><li>The entity with MID "/g/11b41c22g" has a type "/people/person"</li></ul></li></ul></li><li><i>object_names</i>: Each row maps the MID of a Freebase object to its textual label.<ul><li>Example<ul><li>/g/11b78qtr5m, /type/object/name, "Viroliano Tries Jazz"@en</li></ul></li><li>Explanation<ul><li>The entity with MID "/g/11b78qtr5m" has name "Viroliano Tries Jazz" in English.</li></ul></li></ul></li><li><i>object_ids</i>: Each row maps the MID of a Freebase object to its user-friendly identifier.<ul><li>Example<ul><li>/m/05v3y9r, /type/object/id, "/music/live_album/concert"</li></ul></li><li>Explanation<ul><li>The entity with MID "/m/05v3y9r" can be interpreted by human as a music concert live album.</li></ul></li></ul></li><li><i>domains_id_label</i>: Each row maps the MID of a Freebase domain to its label.<ul><li>Example<ul><li>/m/05v4pmy, geology, 77</li></ul></li><li>Explanation<ul><li>The object with MID "/m/05v4pmy" in Freebase is the domain "geology", and has id "77" in our dataset.</li></ul></li></ul></li><li><i>types_id_label</i>: Each row maps the MID of a Freebase type to its label.<ul><li>Example<ul><li>/m/01xljxh, /government/political_party, 147</li></ul></li><li>Explanation<ul><li>The object with MID "/m/01xljxh" in Freebase is the type "/government/political_party", and has id "147" in our dataset.</li></ul></li></ul></li><li><i>entities_id_label</i>: Each row maps the MID of a Freebase entity to its label.<ul><li>Example<ul><li>/g/11b78qtr5m, Viroliano Tries Jazz, 2234</li></ul></li><li>Explanation<ul><li>The entity with MID "/g/11b78qtr5m" in Freebase is "Viroliano Tries Jazz", and has id "2234" in our dataset.</li></ul></li><li><i>properties_id_label</i>: Each row maps the MID of a Freebase property to its label.<ul><li>Example<ul><li>/m/010h8tp2, /comedy/comedy_group/members, 47178867</li></ul></li><li>Explanation<ul><li>The object with MID "/m/010h8tp2" in Freebase is a property(relation/edge), it has label "/comedy/comedy_group/members" and has id "47178867" in our dataset.</li></ul></li></ul></li><li><i>uri_original2simplified</i>&nbsp;and&nbsp;<i>uri_simplified2original</i>: The mapping between original URI and simplified URI and the mapping between simplified URI and original URI repectively.<ul><li>Example<ul><li><i>uri_original2simplified</i><ul><li>"<a href="http://rdf.freebase.com/ns/type.property.unique">http://rdf.freebase.com/ns/type.property.unique</a>": "/type/property/unique"</li></ul></li><li><i>uri_simplified2original</i><ul><li>"/type/property/unique": "<a href="http://rdf.freebase.com/ns/type.property.unique">http://rdf.freebase.com/ns/type.property.unique</a>"</li></ul></li></ul></li><li>Explanation<ul><li>The URI "<a href="http://rdf.freebase.com/ns/type.property.unique">http://rdf.freebase.com/ns/type.property.unique</a>" in the original Freebase RDF dataset is simplified into "/type/property/unique" in our dataset.</li><li>The identifier "/type/property/unique" in our dataset has URI&nbsp;<a href="http://rdf.freebase.com/ns/type.property.unique">http://rdf.freebase.com/ns/type.property.unique</a>&nbsp;in the original Freebase RDF dataset.</li></ul></li></ul></li></ul></li></ul></li></ul>