Stress tests
How the NIH count measures behave when one thing changes: the baseline check, what each scenario moves, backtests and edge cases, each in the testing suite's scenario card.
- baseline checks pass: at no change the simulator reproduces each published distribution exactly
- 3 of 3
- See the numbers
- with new and renewal awards if NIH competing awards fell 20% in FY2026
- 16,077 researchers
- See every input
- mean error when each year predicts the next as it stands, for new and renewal awards
- 5.4%
- See the backtest
New and renewal awards
| Scenario | Assumption | Expected | Result | Status |
|---|---|---|---|---|
| BaselineThe simulator reproduces the published numbers | No change, on FY2026's published distribution | Equal on every figure and range | 19,615 researchers, median 1, 90th percentile 2; published 19,615, 1 and 2 | Passed |
| SensitivityAwards fall 30% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 14,244 researchers (−27.4%), median 1 (no change), 90th percentile 1 (down 1) | Ran |
| SensitivityAwards fall 20% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 16,077 researchers (−18.0%), median 1 (no change), 90th percentile 1 (down 1) | Ran |
| SensitivityCapped at 1 | No researcher holds more than 1 new and renewal awards, FY2026 | Same researchers; the top of the range drops | 19,615 researchers (no change), median 1 (no change), 90th percentile 1 (down 1) | Ran |
| SensitivityAwards fall 10% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 17,866 researchers (−8.9%), median 1 (no change), 90th percentile 2 (no change) | Ran |
| SensitivityCapped at 3 | No researcher holds more than 3 new and renewal awards, FY2026 | Same researchers; the top of the range drops | 19,615 researchers (no change), median 1 (no change), 90th percentile 2 (no change) | Ran |
| SensitivityCapped at 2 | No researcher holds more than 2 new and renewal awards, FY2026 | Same researchers; the top of the range drops | 19,615 researchers (no change), median 1 (no change), 90th percentile 2 (no change) | Ran |
| BacktestEach year predicts the next as it stands | FY2016 to FY2026, 8 pairs of neighbouring years with complete records | Small errors where funding is steady | Mean error 5.4% on researchers; median held in 8 of 8 pairs, 90th percentile in 8 | Ran |
| Edge caseIncomplete years | A year with fewer than 60% of the researchers of the years beside it, on average, is marked | Marked on charts; left out of backtests | FY2020 marked and left out | Ran |
| Edge caseExtreme values | The highest values in FY2026 | A few researchers far above the 99th percentile | Highest 22; 99th percentile 3; lowest 1 | Ran |
| FairnessEach scenario, rerun for each group | Groups: career stage, NIH institute, institution type, state | Flag any group hit 1.5 times harder than all researchers | Not run yet: institute and state are on every award record; career stage and institution type need new data | Not run |
| GamingHow the count could be inflated | One project split in two; one person added as co-PI on many awards; a renewal timed into a chosen year | Run before any grade or ranking is built on this measure | Not run yet: no grade or ranking uses this measure | Not run |
R01-equivalent projects
| Scenario | Assumption | Expected | Result | Status |
|---|---|---|---|---|
| BaselineThe simulator reproduces the published numbers | No change, on FY2026's published distribution | Equal on every figure and range | 30,824 researchers, median 1, 90th percentile 2; published 30,824, 1 and 2 | Passed |
| SensitivityCapped at 1 | No researcher holds more than 1 R01-equivalent projects, FY2026 | Same researchers; the top of the range drops | 30,824 researchers (no change), median 1 (no change), 90th percentile 1 (down 1) | Ran |
| SensitivityAwards fall 30% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 23,598 researchers (−23.4%), median 1 (no change), 90th percentile 2 (no change) | Ran |
| SensitivityAwards fall 20% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 26,151 researchers (−15.2%), median 1 (no change), 90th percentile 2 (no change) | Ran |
| SensitivityAwards fall 10% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 28,555 researchers (−7.4%), median 1 (no change), 90th percentile 2 (no change) | Ran |
| SensitivityCapped at 3 | No researcher holds more than 3 R01-equivalent projects, FY2026 | Same researchers; the top of the range drops | 30,824 researchers (no change), median 1 (no change), 90th percentile 2 (no change) | Ran |
| SensitivityCapped at 2 | No researcher holds more than 2 R01-equivalent projects, FY2026 | Same researchers; the top of the range drops | 30,824 researchers (no change), median 1 (no change), 90th percentile 2 (no change) | Ran |
| BacktestEach year predicts the next as it stands | FY2016 to FY2026, 10 pairs of neighbouring years with complete records | Small errors where funding is steady | Mean error 12.8% on researchers; median held in 10 of 10 pairs, 90th percentile in 10 | Ran |
| Edge caseIncomplete years | A year with fewer than 60% of the researchers of the years beside it, on average, is marked | Marked on charts; left out of backtests | No year marked | Ran |
| Edge caseExtreme values | The highest values in FY2026 | A few researchers far above the 99th percentile | Highest 11; 99th percentile 4; lowest 1 | Ran |
| FairnessEach scenario, rerun for each group | Groups: career stage, NIH institute, institution type, state | Flag any group hit 1.5 times harder than all researchers | Not run yet: institute and state are on every award record; career stage and institution type need new data | Not run |
| GamingHow the count could be inflated | One project split in two; one person added as co-PI on many awards; a renewal timed into a chosen year | Run before any grade or ranking is built on this measure | Not run yet: no grade or ranking uses this measure | Not run |
Award records
| Scenario | Assumption | Expected | Result | Status |
|---|---|---|---|---|
| BaselineThe simulator reproduces the published numbers | No change, on FY2026's published distribution | Equal on every figure and range | 54,400 researchers, median 1, 90th percentile 3; published 54,400, 1 and 3 | Passed |
| SensitivityCapped at 1 | No researcher holds more than 1 award records, FY2026 | Same researchers; the top of the range drops | 54,400 researchers (no change), median 1 (no change), 90th percentile 1 (down 2) | Ran |
| SensitivityAwards fall 30% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 42,484 researchers (−21.9%), median 1 (no change), 90th percentile 2 (down 1) | Ran |
| SensitivityCapped at 2 | No researcher holds more than 2 award records, FY2026 | Same researchers; the top of the range drops | 54,400 researchers (no change), median 1 (no change), 90th percentile 2 (down 1) | Ran |
| SensitivityAwards fall 20% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 46,717 researchers (−14.1%), median 1 (no change), 90th percentile 3 (no change) | Ran |
| SensitivityAwards fall 10% | Each counted project is lost on its own with the same chance (binomial thinning), FY2026 | Fewer researchers; the 90th percentile may drop | 50,678 researchers (−6.8%), median 1 (no change), 90th percentile 3 (no change) | Ran |
| SensitivityCapped at 3 | No researcher holds more than 3 award records, FY2026 | Same researchers; the top of the range drops | 54,400 researchers (no change), median 1 (no change), 90th percentile 3 (no change) | Ran |
| BacktestEach year predicts the next as it stands | FY2016 to FY2026, 8 pairs of neighbouring years with complete records | Small errors where funding is steady | Mean error 4.1% on researchers; median held in 8 of 8 pairs, 90th percentile in 7 | Ran |
| Edge caseIncomplete years | A year with fewer than 60% of the researchers of the years beside it, on average, is marked | Marked on charts; left out of backtests | FY2020 marked and left out | Ran |
| Edge caseExtreme values | The highest values in FY2026 | A few researchers far above the 99th percentile | Highest 30; 99th percentile 6; lowest 1 | Ran |
| FairnessEach scenario, rerun for each group | Groups: career stage, NIH institute, institution type, state | Flag any group hit 1.5 times harder than all researchers | Not run yet: institute and state are on every award record; career stage and institution type need new data | Not run |
| GamingHow the count could be inflated | One project split in two; one person added as co-PI on many awards; a renewal timed into a chosen year | Run before any grade or ranking is built on this measure | Not run yet: no grade or ranking uses this measure | Not run |
How the tests are run
The scenarios come from the Scenario lab's grid, read on the server from how many researchers hold each value, so no researcher's value leaves it. A cut keeps each counted project with the same chance, on its own, and gives expected values; a cap is the lower of a researcher's count and the cap.
The baseline check runs first: at no change the grid must reproduce the published researchers, median, 90th percentile and every range of the chart, or the measure's page runs no scenario. Sensitivity ranks the inputs by how far they move the 90th percentile, then the median, then the researchers with a value.
The backtest lets each fiscal year predict the next as it stands, for every pair of neighbouring years whose records look complete; a year holding fewer than 60% of the researchers of the years beside it is left out. Fairness and gaming tests need data or uses this page does not have yet, and say so.
Download every card as CSV, one row per card in the suite's fields: measure, category, name, assumption, metrics affected, expected direction, result and status.
Other data pages
- Use in research8,970 papers from 2021 to 2025 name publication counts, the most named familyAll 47 measures with search, filters, papers per year and each family's search.
- Validity4 of 28 families have 150 or more validation papersUse set against validation papers: the most studied measures, and the well named but little studied.
- Agreement15 pairs among the 6 citation measures, each with its rank correlationHow the six citation measures order researchers: the near-twins and the opposites.
- Trends4 NIH award measures, each by fiscal year from FY2016 to FY2026Researchers, median and 90th percentile each fiscal year, with any year whose records look incomplete marked.
- Data overview35 of 47 measures graded for use and evidenceEach page in brief, how the counts are made, and the census files to download and cite.
ScienceDex app
See every measure for yourself
Your record shows your h-index, relative citation ratio (RCR) and citations by year, free with a ScienceDex account and your ORCID iD. A department plan shows every faculty member's NIH funding and rank by year.