Skip to main content
ConductScienceScienceDex
Testing suite · NIH RePORTER, FY2026

Stress tests

How the NIH count measures behave when one thing changes: the baseline check, what each scenario moves, backtests and edge cases, each in the testing suite's scenario card.

baseline checks pass: at no change the simulator reproduces each published distribution exactly
3 of 3
See the numbers
with new and renewal awards if NIH competing awards fell 20% in FY2026
16,077 researchers
See every input
mean error when each year predicts the next as it stands, for new and renewal awards
5.4%
See the backtest

New and renewal awards

FY2026. Each row is one scenario card; the measure's own page runs the scenarios with a slider. Open its Scenario lab
New and renewal awards: the testing suite's scenario cards
ScenarioAssumptionExpectedResultStatus
BaselineThe simulator reproduces the published numbersNo change, on FY2026's published distributionEqual on every figure and range19,615 researchers, median 1, 90th percentile 2; published 19,615, 1 and 2Passed
SensitivityAwards fall 30%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop14,244 researchers (−27.4%), median 1 (no change), 90th percentile 1 (down 1)Ran
SensitivityAwards fall 20%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop16,077 researchers (−18.0%), median 1 (no change), 90th percentile 1 (down 1)Ran
SensitivityCapped at 1No researcher holds more than 1 new and renewal awards, FY2026Same researchers; the top of the range drops19,615 researchers (no change), median 1 (no change), 90th percentile 1 (down 1)Ran
SensitivityAwards fall 10%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop17,866 researchers (−8.9%), median 1 (no change), 90th percentile 2 (no change)Ran
SensitivityCapped at 3No researcher holds more than 3 new and renewal awards, FY2026Same researchers; the top of the range drops19,615 researchers (no change), median 1 (no change), 90th percentile 2 (no change)Ran
SensitivityCapped at 2No researcher holds more than 2 new and renewal awards, FY2026Same researchers; the top of the range drops19,615 researchers (no change), median 1 (no change), 90th percentile 2 (no change)Ran
BacktestEach year predicts the next as it standsFY2016 to FY2026, 8 pairs of neighbouring years with complete recordsSmall errors where funding is steadyMean error 5.4% on researchers; median held in 8 of 8 pairs, 90th percentile in 8Ran
Edge caseIncomplete yearsA year with fewer than 60% of the researchers of the years beside it, on average, is markedMarked on charts; left out of backtestsFY2020 marked and left outRan
Edge caseExtreme valuesThe highest values in FY2026A few researchers far above the 99th percentileHighest 22; 99th percentile 3; lowest 1Ran
FairnessEach scenario, rerun for each groupGroups: career stage, NIH institute, institution type, stateFlag any group hit 1.5 times harder than all researchersNot run yet: institute and state are on every award record; career stage and institution type need new dataNot run
GamingHow the count could be inflatedOne project split in two; one person added as co-PI on many awards; a renewal timed into a chosen yearRun before any grade or ranking is built on this measureNot run yet: no grade or ranking uses this measureNot run

R01-equivalent projects

FY2026. Each row is one scenario card; the measure's own page runs the scenarios with a slider. Open its Scenario lab
R01-equivalent projects: the testing suite's scenario cards
ScenarioAssumptionExpectedResultStatus
BaselineThe simulator reproduces the published numbersNo change, on FY2026's published distributionEqual on every figure and range30,824 researchers, median 1, 90th percentile 2; published 30,824, 1 and 2Passed
SensitivityCapped at 1No researcher holds more than 1 R01-equivalent projects, FY2026Same researchers; the top of the range drops30,824 researchers (no change), median 1 (no change), 90th percentile 1 (down 1)Ran
SensitivityAwards fall 30%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop23,598 researchers (−23.4%), median 1 (no change), 90th percentile 2 (no change)Ran
SensitivityAwards fall 20%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop26,151 researchers (−15.2%), median 1 (no change), 90th percentile 2 (no change)Ran
SensitivityAwards fall 10%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop28,555 researchers (−7.4%), median 1 (no change), 90th percentile 2 (no change)Ran
SensitivityCapped at 3No researcher holds more than 3 R01-equivalent projects, FY2026Same researchers; the top of the range drops30,824 researchers (no change), median 1 (no change), 90th percentile 2 (no change)Ran
SensitivityCapped at 2No researcher holds more than 2 R01-equivalent projects, FY2026Same researchers; the top of the range drops30,824 researchers (no change), median 1 (no change), 90th percentile 2 (no change)Ran
BacktestEach year predicts the next as it standsFY2016 to FY2026, 10 pairs of neighbouring years with complete recordsSmall errors where funding is steadyMean error 12.8% on researchers; median held in 10 of 10 pairs, 90th percentile in 10Ran
Edge caseIncomplete yearsA year with fewer than 60% of the researchers of the years beside it, on average, is markedMarked on charts; left out of backtestsNo year markedRan
Edge caseExtreme valuesThe highest values in FY2026A few researchers far above the 99th percentileHighest 11; 99th percentile 4; lowest 1Ran
FairnessEach scenario, rerun for each groupGroups: career stage, NIH institute, institution type, stateFlag any group hit 1.5 times harder than all researchersNot run yet: institute and state are on every award record; career stage and institution type need new dataNot run
GamingHow the count could be inflatedOne project split in two; one person added as co-PI on many awards; a renewal timed into a chosen yearRun before any grade or ranking is built on this measureNot run yet: no grade or ranking uses this measureNot run

Award records

FY2026. Each row is one scenario card; the measure's own page runs the scenarios with a slider. Open its Scenario lab
Award records: the testing suite's scenario cards
ScenarioAssumptionExpectedResultStatus
BaselineThe simulator reproduces the published numbersNo change, on FY2026's published distributionEqual on every figure and range54,400 researchers, median 1, 90th percentile 3; published 54,400, 1 and 3Passed
SensitivityCapped at 1No researcher holds more than 1 award records, FY2026Same researchers; the top of the range drops54,400 researchers (no change), median 1 (no change), 90th percentile 1 (down 2)Ran
SensitivityAwards fall 30%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop42,484 researchers (−21.9%), median 1 (no change), 90th percentile 2 (down 1)Ran
SensitivityCapped at 2No researcher holds more than 2 award records, FY2026Same researchers; the top of the range drops54,400 researchers (no change), median 1 (no change), 90th percentile 2 (down 1)Ran
SensitivityAwards fall 20%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop46,717 researchers (−14.1%), median 1 (no change), 90th percentile 3 (no change)Ran
SensitivityAwards fall 10%Each counted project is lost on its own with the same chance (binomial thinning), FY2026Fewer researchers; the 90th percentile may drop50,678 researchers (−6.8%), median 1 (no change), 90th percentile 3 (no change)Ran
SensitivityCapped at 3No researcher holds more than 3 award records, FY2026Same researchers; the top of the range drops54,400 researchers (no change), median 1 (no change), 90th percentile 3 (no change)Ran
BacktestEach year predicts the next as it standsFY2016 to FY2026, 8 pairs of neighbouring years with complete recordsSmall errors where funding is steadyMean error 4.1% on researchers; median held in 8 of 8 pairs, 90th percentile in 7Ran
Edge caseIncomplete yearsA year with fewer than 60% of the researchers of the years beside it, on average, is markedMarked on charts; left out of backtestsFY2020 marked and left outRan
Edge caseExtreme valuesThe highest values in FY2026A few researchers far above the 99th percentileHighest 30; 99th percentile 6; lowest 1Ran
FairnessEach scenario, rerun for each groupGroups: career stage, NIH institute, institution type, stateFlag any group hit 1.5 times harder than all researchersNot run yet: institute and state are on every award record; career stage and institution type need new dataNot run
GamingHow the count could be inflatedOne project split in two; one person added as co-PI on many awards; a renewal timed into a chosen yearRun before any grade or ranking is built on this measureNot run yet: no grade or ranking uses this measureNot run

How the tests are run

The scenarios come from the Scenario lab's grid, read on the server from how many researchers hold each value, so no researcher's value leaves it. A cut keeps each counted project with the same chance, on its own, and gives expected values; a cap is the lower of a researcher's count and the cap.

The baseline check runs first: at no change the grid must reproduce the published researchers, median, 90th percentile and every range of the chart, or the measure's page runs no scenario. Sensitivity ranks the inputs by how far they move the 90th percentile, then the median, then the researchers with a value.

The backtest lets each fiscal year predict the next as it stands, for every pair of neighbouring years whose records look complete; a year holding fewer than 60% of the researchers of the years beside it is left out. Fairness and gaming tests need data or uses this page does not have yet, and say so.

Download every card as CSV, one row per card in the suite's fields: measure, category, name, assumption, metrics affected, expected direction, result and status.

ScienceDex app

See every measure for yourself

Your record shows your h-index, relative citation ratio (RCR) and citations by year, free with a ScienceDex account and your ORCID iD. A department plan shows every faculty member's NIH funding and rank by year.