1117 Russian cities with city name, region, geographic coordinates and 2020 population estimate

<p>1117 Russian cities with city name, region, geographic coordinates and 2020 population estimate.</p> <p>&nbsp;</p> <p>How to use</p> <pre>from pathlib import Path import requests import pandas as pd url = (&quot;https://raw.githubusercontent.com/&quot; &quot;epogrebnyak/ru-cities/main/assets/towns.csv&quot;) # save file locally p = Path(&quot;towns.csv&quot;) if not p.exists(): content = requests.get(url).text p.write_text(content, encoding=&quot;utf-8&quot;) # read as dataframe df = pd.read_csv(&quot;towns.csv&quot;) print(df.sample(5))</pre> <p>&nbsp;</p> <p>Files:</p> <ul> <li><a href="https://github.com/epogrebnyak/ru-cities/blob/main/assets/towns.csv">towns.csv</a> - city information</li> <li><a href="https://github.com/epogrebnyak/ru-cities/blob/main/assets/regions.csv">regions.csv</a> - list of Russian Federation regions</li> <li><a href="https://github.com/epogrebnyak/ru-cities/blob/main/assets/alt_city_names.json">alt_city_names.json</a> - alternative city names</li> </ul> <p>&nbsp;</p> <p>Сolumns (towns.csv):</p> <p>Basic info:</p> <ul> <li><code>city</code> - city name (several cities have alternative names marked in <code>alt_city_names.json</code>)</li> <li><code>population</code> - city population, thousand people, Rosstat estimate as of 1.1.2020</li> <li><code>lat,lon</code> - city geographic coordinates</li> </ul> <p>Region:</p> <ul> <li><code>region_name</code> - subnational region (oblast, republic, krai or AO)</li> <li><code>region_iso_code</code> - <a href="https://en.wikipedia.org/wiki/ISO_3166-2:RU">ISO 3166 code</a>, eg <code>RU-VLD</code></li> <li><code>federal_district</code>, eg <code>Центральный</code></li> </ul> <p>City codes:</p> <ul> <li><code>okato</code></li> <li><code>oktmo</code></li> <li><code>fias_id</code></li> <li><code>kladr_id</code></li> </ul> <p>&nbsp;</p> <p>Data sources</p> <ul> <li>City list and city population collected from Rosstat publication <a href="https://rosstat.gov.ru/folder/210/document/13206">Регионы России. Основные социально-экономические показатели городов</a> and parsed from publication Microsoft Word files.</li> <li>City list corresponds to <a href="https://ru.wikipedia.org/wiki/%D0%A1%D0%BF%D0%B8%D1%81%D0%BE%D0%BA_%D0%B3%D0%BE%D1%80%D0%BE%D0%B4%D0%BE%D0%B2_%D0%A0%D0%BE%D1%81%D1%81%D0%B8%D0%B8">this Wikipedia article</a>.</li> <li>Alternative dataset is <a href="https://github.com/hflabs/city">wiki-based Dadata city dataset</a> (no population data).</li> </ul> <p>&nbsp;</p> <p>Comments</p> <p>&nbsp;</p> <p>City groups</p> <ul> <li> <p><code>Ханты-Мансийский</code> and <code>Ямало-Ненецкий</code> autonomous regions excluded to avoid duplication as parts of <code>Тюменская область</code>.</p> </li> <li> <p>Several notable towns are classified as administrative part of larger cities (<code>Сестрорецк</code> is a municpality at Saint-Petersburg, <code>Щербинка</code> part of Moscow). They are not and not reported in this dataset.</p> </li> </ul> <p>&nbsp;</p> <p>By individual city</p> <ul> <li><code>Белоозерский</code> not found in Rosstat publication, but <a href="https://github.com/epogrebnyak/ru-cities/issues/5#issuecomment-886179980">should be considered a city as of 1.1.2020</a></li> </ul> <p>&nbsp;</p> <p>Alternative city names</p> <ul> <li> <p>We suppressed letter &quot;ё&quot; <code>city</code> columns in towns.csv - we have <code>Орел</code>, but not <code>Орёл</code>. This affected:</p> <ul> <li><code>Белоозёрский</code></li> <li><code>Королёв</code></li> <li><code>Ликино-Дулёво</code></li> <li><code>Озёры</code></li> <li><code>Щёлково</code></li> <li><code>Орёл</code></li> </ul> </li> <li> <p><code>Дмитриев</code> and <code>Дмитриев-Льговский</code> are the same city.</p> </li> </ul> <p><code>assets/alt_city_names.json</code> contains these names.</p> <p>&nbsp;</p> <p>Tests</p> <pre><code>poetry install poetry run python -m pytest </code></pre> <p>&nbsp;</p> <p>How to replicate dataset</p> <p>&nbsp;</p> <p>1. Base dataset</p> <p>Run:</p> <ul> <li>download data stro rar/get.sh</li> <li>convert <code>Саратовская область.doc</code> to docx</li> <li>run make.py</li> </ul> <p>Creates:</p> <ul> <li><code>_towns.csv</code></li> <li><code>assets/regions.csv</code></li> </ul> <p>&nbsp;</p> <p>2. API calls</p> <p>Note: do not attempt if you do not have to - this runs a while and loads third-party API access.</p> <p>You have the resulting files in repo, so probably does not need to these scripts.</p> <p>Run:</p> <ul> <li><code>cd geocoding</code></li> <li>run coord_dadata.py (needs token)</li> <li>run coord_osm.py</li> </ul> <p>Creates:</p> <ul> <li>coord_dadata.csv</li> <li>coord_osm.csv</li> </ul> <p>&nbsp;</p> <p>3. Merge data</p> <p>Run:</p> <ul> <li>run merge.py</li> </ul> <p>Creates:</p> <ul> <li>assets/towns.csv</li> </ul> <p>&nbsp;</p>