For reviewers
Would Aluni have been right? We checked against 25 years of records.
We ran Aluni's calculations on one real field (Story County, Iowa) for every season from 2001 to 2025, and scored them against records they never read. Some held up, some did not, and the app changed because of it.
- 2 passed
- 1 reported
- 5 failed
How the test works
Each study asks one question a farmer or a judge would ask of Aluni, picks an independent record of what happened, and sets the pass mark before looking at the result.
- Pass marks set first. Every criterion was written down before the results were seen, and a failure stays a failure.
- Independent truth. Rain gauges, USDA county yields, crop insurance losses by cause, published field trials and satellite images, none of which the app reads.
- Nothing fitted. Every coefficient comes from FAO, USDA or the cited papers, so the test is out of sample by construction.
- Reproducible. The data are public and the scripts are in the repository; anyone can rerun every number on this page.
The eight studies
Each card shows the question, the record used as truth, the pass mark set beforehand, the result and what it means.
-
V1
Is the NASA data at the field factual?
Failed
- Truth
- 59 CoCoRaHS volunteer rain gauges within 30 km, and the Daymet 1 km weather grid.
- Pass mark
- Daily highs and lows within 2.5 °C (RMSE), and season rain within 15 % of the gauges in at least 80 % of the years.
- Result
- Temperature passes: highs within 1.72 °C and lows within 2.37 °C, frost days agree 95 % of the time. NASA POWER rain read low in 18 of 18 seasons (12.5 % on average) and was within 15 % in only 67 % of them; NASA GPM IMERG rain was within 15 % in 83 %.
- What it means
- Temperature and frost are reliable at the pin. For rain, Aluni now reads IMERG instead of POWER.
-
V2
Would the water model have flagged the bad years?
Failed
- Truth
- USDA NASS Story County yields (trend removed) and USDA crop insurance losses by cause, 2001 to 2025.
- Pass mark
- A correlation of at least 0.5 with the yield anomaly, and 2012 among the three driest modelled years.
- Result
- It ranks drought and heat insurance losses well (0.61 for corn, 0.8 for soybeans) and puts 2012 first, but it does not track yields (correlation 0.03 for corn, -0.05 for soybeans): it calls 22 to 24 of 25 seasons stressed. In 2025, when the model cut soybeans by 13 %, Story County soybeans yielded 4.2 % above trend.
- What it means
- The water need is a drought flag, not a harvest forecast, and the app now says so.
-
V3
Does the crop evidence point at the crops that thrive here?
Failed
- Truth
- The crops grown at the pin (USDA Cropland Data Layer, 2008 to 2025) and USDA NASS yields.
- Pass mark
- Corn and soybeans ranked above wheat and sunflower, and FAO GAEZ attainable yield within 30 % of the real yields.
- Result
- FAO GAEZ rates all 8 crops at the top suitability (10000), so it cannot rank them here. Water alone would favour Wheat and Sunflower. GAEZ maize was 20 % above the real yields, soybeans 99 % above.
- What it means
- Suggesting a replacement crop needs yields and prices (a margin), not suitability and water alone.
-
V4
Is the rotation advice backed by evidence?
Passed
- Truth
- A meta-analysis of field trials (Zhao et al. 2022): 670 comparisons from 39 sites in a climate like Ames, at the nitrogen rates farmers use.
- Pass mark
- The 95 % interval of the soybean pre-crop effect on corn yield must exclude zero.
- Result
- Corn after soybeans yields 9.9 % more than after a non-legume (95 % interval 7.2 to 13.2 %); without nitrogen fertiliser the gain is 45.8 %. The farmer's corn and soybean alternation meets every rule the comparator checks.
- What it means
- The rotation advice rests on hundreds of field trials, not on points alone.
-
V5
Do the pest and disease weather rules match real losses?
Reported
- Truth
- USDA crop insurance losses to plant disease, mycotoxin and insects in the twelve counties of central Iowa.
- Pass mark
- Reported, not scored: rank correlation and lead time for each of the 5 rules.
- Result
- Every rule leans the right way (correlations 0.14 to 0.32), but none is significant; the first favourable day comes about 3 months before the main loss month.
- What it means
- The rules say when to scout, not that a crop will be hit. Leaf wetness and scouting records are needed: the strongest case for the field station.
-
V6
How much soil does each rotation lose, and how much does a cover crop save?
Passed
- Truth
- The USDA soil loss equation (RUSLE) on the field's rain, soil erodibility and slope.
- Pass mark
- The app's erosion score must order the rotations the way RUSLE does.
- Result
- The order holds. A rye cover crop saves 13 to 14 % of the soil lost each year, far less than counting bare months suggests, because most erosive rain falls from May to September under young crops. The losses (17 to 24 t/ha a year) rest on assumed residue and cover values.
- What it means
- The ranking is sound; the timeline now counts young crops as open soil.
-
V7
Would choosing the crop by expected margin have paid?
Failed
- Truth
- USDA NASS Story County yields and Iowa prices, 18 seasons, with placeholder costs.
- Pass mark
- A margin rule must earn at least as much as the farmer's strict corn and soybean alternation.
- Result
- The margin rule earned $11,995 per hectare against $12,250 for the farmer, $10,366 for continuous corn and $13,567 with hindsight. One 12-row planter covers 400 ha in all 25 springs; Iowa corn is half planted between 25 Apr and 18 May, not on 1 April.
- What it means
- Rotating clearly pays against continuous corn, but a simple margin rule does not beat an experienced farmer. Aluni supports the farmer's judgement, it does not replace it.
-
V8
Do the timeline's crop and bare months match the field?
Failed
- Truth
- Landsat and Sentinel-2 satellite images of the field, one clear view a month, 2013 to 2025, with the crop of each year.
- Pass mark
- At least 80 % of the months agree, and planting and harvest within one month of the satellite green-up and drying.
- Result
- Only 76 % of 148 months agreed: the canopy closes about 2.2 months after the table's planting month, and November and December were open soil in 26 of 26 views. With the rule the app uses now (real planting months, young crops counted as open soil) agreement rises to 90 %.
- What it means
- The timeline now plants when farmers in the region plant and shows the soil open until the crop covers it.
A second field: does it hold up somewhere else?
One field could be luck. We ran the same eight studies, with the same pass marks, on the team's demo farm: York, York County, Nebraska, which is warmer and drier and where USDA publishes rainfed (non-irrigated) yields.
| Study |
Story County, Iowa |
York County, Nebraska |
| V1 NASA data |
Failed
Highs within 1.72 °C, lows 2.37 °C. POWER rain within 15 % in 67 % of seasons, IMERG in 83 %.
|
Passed
Highs within 1.88 °C, lows 2.49 °C. POWER rain within 15 % in 83 % of seasons, IMERG in 83 %.
|
| V2 Water model vs yields |
Failed
Correlation with yields: 0.03 corn, -0.05 soybeans. Drought losses ranked 0.61 / 0.8.
|
Passed
Correlation with yields: 0.66 corn, 0.77 soybeans. Drought losses ranked 0.84 / 0.76.
|
| V3 Crop evidence |
Failed
GAEZ maize 20 % and soybeans 99 % off the real yields; water alone favours Wheat and Sunflower.
|
Failed
GAEZ maize 21 % and soybeans 14 % off the real yields; water alone favours Wheat and Sunflower.
|
| V4 Rotation |
Passed
Corn after soybeans +9.9 % (7.2 to 13.2 %), 670 trial comparisons from 39 sites.
|
Passed
Corn after soybeans +12.7 % (5.6 to 22 %), 194 trial comparisons from 17 sites.
|
| V5 Pest rules |
Reported
Correlations 0.14 to 0.32; none significant.
|
Reported
Correlations -0.19 to 0.18; none significant.
|
| V6 Erosion |
Passed
A rye cover saves 13 to 14 % of the soil lost.
|
Passed
A rye cover saves 12 to 13 % of the soil lost.
|
| V7 Margin rule |
Failed
Margin rule $11,995 against the farmer's $12,250 per hectare, 18 seasons.
|
Passed
Margin rule $13,335 against the farmer's $12,583 per hectare, 15 seasons.
|
| V8 Timeline |
Failed
76 % of months agree; 90 % with the rule the app uses now.
|
Failed
65 % of months agree; 82 % with the rule the app uses now.
|
What changes with two fields. The water model, which did not track yields in humid Iowa, tracks rainfed yields in drier Nebraska: it works where water is what limits the harvest, and the app should say how much that applies to each field. The margin rule beat the York farmer, who grew corn three years running, but not the Ames farmer, who alternated strictly: it helps most where a rotation slips. Rotation evidence, the erosion ranking and the weak pest rules repeat on both fields. The York rainfed yields stop in 2017 (corn) and 2013 (soybeans), so its sample is smaller.
Buin, Chile, the other demo farm, cannot be scored the same way yet: none of the documented open sources has Chilean rain gauges, county yields or crop insurance losses.
Answers to the questions that matter
Could Aluni have predicted the facts on this field?
Partly.
Temperature and frost from NASA are reliable at both fields; rain from NASA GPM IMERG passes everywhere, NASA POWER rain read low in Iowa. In drier, rainfed Nebraska the water model tracks the yields (correlation 0.66 for corn, 0.77 for soybeans); in humid Iowa it only spots the drought disasters. Aluni shows it as a drought risk, not a harvest figure, because it overstates the size of the losses at both fields.
Can it suggest a better rotation or a replacement crop?
A better rotation, yes. A replacement crop, not yet.
The rotation advice is backed by hundreds of field trials and by the erosion model. Ranking replacement crops needs yields and prices, which the action plan adds as a margin criterion.
Can it help prevent pests and disease?
It says when to scout.
Weather rules point at the right seasons but did not track real losses significantly. Leaf-wetness sensors and scouting records come next.
Can it inform costs, supplies, people, equipment and land productivity?
The calculations work on real data.
Planting capacity against the days fields can be worked, real planting dates, nitrogen balances and margins all run on public data. What is proven is the value of rotating; what is not is that Aluni beats an experienced farmer.
What changed in Aluni because of this
Every failed study named a cause. These are the changes, and the two still open.
-
Done
U-27
The water need reads NASA GPM IMERG rain, which passed against the gauges, instead of NASA POWER rain.
-
Done
U-28
Crops are planted in the month farmers in the region plant them (USDA NASS crop progress) before the FAO-56 table.
-
Done
U-29
The timeline counts a young crop's months as open soil until its canopy closes; satellite agreement rose to 90 %.
-
Done
U-30
The water need is presented as a drought risk, never as a harvest figure.
-
Open
U-31
Erosion as a soil loss in tonnes, with documented residue and cover values and a published reference.
-
Open
U-32
Disease rules checked against leaf-wetness readings and scouting records in the pilot.
Limits
One field and one county; yields and insurance losses are county totals; insured losses are a thin record of disease; costs are placeholders; the soil losses rest on assumed residue and cover values. The pilot fields in the action plan widen the test.
Run it yourself
The data are public; the scripts live in docs/proposal/validation in the repository.
mise exec -- bin/rails runner docs/proposal/validation/fetch.rbmise exec -- ruby docs/proposal/validation/run_all.rbmise exec -- ruby docs/proposal/claims_audit.rb
Results generated on November 15, 2026 for the pin at 42.0, -93.75.