452,000,000 public Git commits on GitHub (October 2016)

<p>What's inside</p> <p>part-000xx.lzo - LZO archives with the data (refer to "Format").</p> <p>part-000xx.lzo.index - LZO index files so that the archives are splittable in Hadoop.</p> <p>stats.csv.gz - GZIP-ed CSV file with some repository statistics related to the commits.</p> <p>Format</p> <p>part-000xx - text, one line per repository, every line is JSON with the following scheme:</p> <p>{ "r": "repository name", "c": [{ "h": "git hash", "a": "author's email hash", "t": "date and time commit was created", "m": "commit message" }, ...] }</p> <p>Date and time format is <em>mostly</em> Go language's time.Time.String(), I recommend to use dateutil.parse() to parse it with Python.</p> <p>Commit message contains explicit \r and \n symbols in order to be a single line.</p> <p>stats.csv has 4 columns: repository name, number of commits, number of contributors, average length of the commit messages.</p>