Methods, Performance and Price Impact of USDA Production Forecasts
content
 |
Darrel L. GoodProfessor, Agricultural and Consumer EconomicsPhone: (217) 333-4716 E-mail: d-good@illinois.edu |
Scott H. IrwinProfessor, Agricultural and Consumer EconomicsPhone: (217) 333-6087 E-mail: sirwin@illinois.edu |
|
There appears to be continuing misunderstanding of U.S. Department of Agriculture (USDA) motives, methods and procedures used to make production forecasts for U.S. corn and soybeans. Some in the agricultural community apparently even believe that the USDA manipulates crop forecasts to fulfill some mystical objectives that are contrary to the best interest of farmers. There is a need for a better understanding of the USDA crop production forecasting process.
The purposes of this study are to:
-
Provide an overview of the forecasting process used by the USDA,
-
Examine relationships in the monthly changes in production forecasts,
-
Examine errors in the USDA forecasts, and
-
Examine the price response to USDA forecasts.
USDA Forecasting Process
The USDA uses a highly sophisticated and well-documented procedure to generate its crop production forecasts. For corn and soybeans, production forecasts are released in August, September, October, and November. Final estimates are published in January. The USDA generates crop production forecasts based on estimates of planted and harvested acreage and two types of yield indications, a farmer-reported survey and objective measurements. The acreage figures are obtained from the June Agricultural Survey based on a large farm operator list frame and a separate and independent area frame survey. The farmer and objective yield Aprobability@ surveys use the same sampling, survey and estimation procedures from year to year.
The farmer-reported yield survey is conducted for states with significant corn and soybean production. In 2003, 33 states were surveyed for corn and 29 for soybeans. Farmers included in the yield survey are randomly selected from the list frame of individuals that responded to the June Agricultural Survey. The USDA has determined that farmers tend to make conservative (low) yield predictions, especially early in the season, so the final farmer-reported yield for a given month is adjusted to reflect that fact that farmers consistently are conservative.
The objective yield survey is conducted only for the seven most important production states and is based on an area-frame sampling design, where fields are randomly selected from the total land area used in production of a given crop. Observations are obtained from two independently located plots in each randomly selected field, with yield indications derived from models based on observations over the last five years. Separate monthly models are constructed by maturity stage so forecast adjustments are automatically made for early or late maturing crops. Yield forecasts reflect crop conditions at the beginning of the survey month and assume normal growing conditions for the remainder of the season.
At the end of the season, plots are harvested and yields are calculated based on actual grain weights and harvest losses. In addition, an interview is conducted with the farm operator immediately after harvest to determine harvested acres and actual yield.
USDA Forecast Performance
Changes in monthly corn and soybean production forecasts were examined for the 33-year period covering 1970–2002. The September change for both corn and soybeans was very large in 1983. For a given month, there appears to be little difference in the magnitude of monthly changes over time. In addition, there appears to be no change in the pattern or direction of monthly changes over time. It is not surprising that the size of forecast changes tends to diminish from September to January.
The results, however, indicate that there is a positive relationship in the monthly changes for both corn and soybeans. Correlations in corn are moderate for October versus September changes and January versus November changes and high for November versus October changes. The average correlation across all three comparisons for corn is 0.55, which indicates substantial "smoothing" of changes in corn production forecasts. Relationships are more limited in soybeans, where the highest correlation is 0.50 (November versus October changes). The average correlation across all three comparisons for soybeans is 0.31, indicating a moderate amount of smoothing of soybean production forecasts across months. For the most part, both the direction and the magnitude of changes in monthly USDA corn and soybean production forecasts have been well anticipated by the private sector. This suggests that private analysts are able to anticipate and incorporate any "smoothness" in USDA changes into their own forecasts.
While the pattern of changes in USDA corn and soybean production forecasts has been stable over time and the private sector has anticipated the changes reasonably well, there is still the question of accuracy of the monthly forecasts. The accuracy of the August, September, October and November forecasts is measured against the January estimate. In every case, the average forecast error is near zero, indicating that USDA forecasts do not tend to be consistently too high or too low. There also does not appear to be any trend in the size of forecast errors through the years. Finally, the percentage forecast errors for corn tend to be larger than for soybeans in August, September and October. The percentage forecast errors are roughly the same in November for both crops.
USDA and private forecast errors are about the same magnitude, especially for soybeans. However, there are times when the forecast errors diverge sharply. The average absolute errors for USDA and the private sector are reported in Table 1 for the entire 1970–2002 period and for two sub-periods, 1970–1984 and 1985–2002. In corn, the relative forecasting accuracy of the USDA was superior in every case except August in the latter sub-period. USDA forecasts in corn also improved more quickly than the private market as the growing season progressed. The one trouble spot for the USDA in corn is August forecast accuracy since the mid-1980s. Since that time, private market forecasts have been more accurate by an average of 0.7 percentage points. This reflects a sharp improvement in private sector forecast accuracy relative to the USDA over the last three decades.
The relative forecasting comparisons for soybeans are more surprising. Private market forecasts generally were more accurate than USDA forecasts for August and September, regardless of the time period considered (the one exception: September, 1970–1984). As with corn, private sector accuracy for August forecasts improved much more than the USDA's over the last three decades. As the growing season progressed, the USDA's relative accuracy improved, with the USDA having average absolute percentage errors equal to or smaller than the private market for October and November soybean forecasts.
Price Impact of USDA Forecasts
Theoretically, the price impact of USDA corn and soybean production forecasts should be determined by how well the market anticipates the forecasts. Prices should change in relation to the degree that the market is "surprised" by the new information. The difference between private and USDA forecasts is an estimate of the market surprise for each crop report. A positive surprise number is considered "bearish," and a negative surprise number is considered "bullish." Market surprises tend to be largest in August and smallest in November for both corn and soybeans. There does not appear to be any obvious trends in market surprises across crop years.
Price impact is indicated by the reaction of December corn futures and November soybean futures immediately after the release of the USDA production forecasts. For corn, the largest reaction occurs following the August report, and recent price reactions have been somewhat larger than historical reactions (except for 1973). Reactions have been relatively small in September, larger in October, and very small in November, with a dramatic exception in 1993.
For soybeans, price reactions have had similar magnitudes in August and September. Price reactions in October were relatively small through the 1980s (with the exception of 1970), but have generally been larger in recent years. Price reactions to the November forecasts have been relatively small, and the pattern has changed little over time.
As expected, there is a negative relationship between the direction of the surprise and the direction of price reaction for both corn and soybeans. The relationships are somewhat stronger for corn than soybeans, with the variation of surprises in the USDA production forecasts explaining 37 to 56 percent of the variation of the immediate change in corn prices and 21 to 32 percent of the variation of the immediate change in soybean prices. The strongest reaction is found in November for both crops. A one-percentage-point surprise leads, on average, to a one-percentage-point change in corn futures prices and a three-quarters-of-a-percentage-point change in soybean futures prices. Price impacts provide strong evidence that market participants view USDA corn and soybean production forecasts as important pieces of new information.
The 2003 Experience
At the writing of this report, the USDA had released the August, September, October and November 2003 production forecasts for corn and soybeans. The USDA forecasts and corresponding private market forecasts are presented in Table 2.
The information presented in Table 3 provides historical perspective on the USDA corn and soybean crop forecasts released to date in 2003. Three key indicators are presented: the change in forecast, the market surprise and the resulting price reaction. The value for 2003 is compared to the previous high and low values over 1970–2002. Based upon a comparison of the absolute value in 2003 to the absolute value of previous highs and lows, none of the 2003 values in corn fell outside the historical ranges found in the table.
Results are more striking in soybeans. In two cases (September: market surprise; October: change in forecast), magnitudes were record large. Several other cases were near record highs in terms of magnitude. It is important to note than even in the cases where new records were set in soybeans, the difference between the new record value and old record value is relatively small. It seems reasonable to argue that the experience is not dramatically different than what has been seen before.
Further perspective is provided by the price reaction comparisons found in Table 4. The price reaction for each of the 2003 production forecasts is the same as that presented in Table 3. Predicted price reactions for each month are based on the regression equations estimated from historical data. In each of the eight cases, the direction of price reaction predicted by the regression equations is the same as that actually observed. In six cases, the magnitude of actual price reactions is reasonably close to the predicted magnitude. In the other two cases (August 2003 corn and September 2003 soybeans), the actual magnitude is substantially underestimated, but the large errors in these two cases are within historical experience.
Overall, the results presented in this section indicate that 2003 USDA corn and soybean production forecasts generally are within historical ranges in terms of magnitude of changes, market surprise and price reaction. The September and October soybean forecasts were major market surprises, and the market's price reactions confirmed this, but they were not unprecedented.
For complete results of this study, visit the Website http://www.farmdoc.illinois.edu/agmas/.
Tables & Figures
| Table 1. Average
absolute errors for USDA and private forecasts. |
||||
|---|---|---|---|---|
| CORN | SOYBEANS | |||
| |
USDA FORECAST |
PRIVATE FORECAST |
USDA FORECAST |
PRIVATE FORECAST |
% |
% |
|||
| August | ||||
1970-2002 |
5.3 |
5.4 |
4.7 |
4.2 |
1970-1984 |
5.9 |
7.0 |
4.9 |
4.8 |
1985-2002 |
4.7 |
4.0 |
4.6 |
3. |
| September | ||||
1970-2002 |
4.0 |
4.3 |
3.8 |
3.6 |
1970-1984 |
3.9 |
4.2 |
3.2 |
3.3 |
1985-2002 |
4.1 |
4.5 |
4.3 |
3.9 |
| October | ||||
1970-2002 |
2.4 |
3.1 |
2.4 |
2.6 |
1970-1984 |
2.4 |
3.1 |
2.7 |
2.7 |
1985-2002 |
2.5 |
3.0 |
2.1 |
2.6 |
| November | ||||
1970-2002 |
1.2 |
1.6 |
1.4 |
1.5 |
1970-1984 |
1.3 |
1.8 |
1.8 |
2.1 |
1985-2002 |
1.0 |
1.4 |
1.1 |
1.1 |
| Table 2. USDA
and private forecasts for 2003. |
||||
|---|---|---|---|---|
| CORN | SOYBEANS | |||
| MONTH |
USDA FORECAST |
PRIVATE FORECAST |
USDA FORECAST |
PRIVATE FORECAST |
million bushels |
million bushels |
|||
| August 2003 | 10,064 |
10,293 |
2,862 |
2,943 |
| September 2003 | 9,944 |
9,800 |
2,643 |
2,757 |
| October 2003 | 10,207 |
10,062 |
2,468 |
2,561 |
| November 2003 | 10,278 |
10,327 |
2,452 |
2,451 |
| Table 3. Key
indicators for 2003 forecasts. |
||||||
|---|---|---|---|---|---|---|
| CORN | SOYBEANS | |||||
| RELEASE MONTH |
CHANGE IN FORECAST |
MARKET SURPRISE | PRICE REACTION |
CHANGE IN FORECAST |
MARKET SURPRISE | PRICE REACTION |
% |
% |
|||||
| August | ||||||
2003 |
NA |
-2.3 |
6.4 |
NA |
-2.8 |
3.7 |
Previous High |
NA |
4.5 |
12.7 |
NA |
3.7 |
11.4 |
Previous Low |
NA |
-6.3 |
-6.3 |
NA |
-3.2 |
-6.0 |
| September | ||||||
2003 |
-1.2 |
1.4 |
-0.8 |
-7.7 |
-4.3 |
5.3 |
Previous High |
4.5 |
3.1 |
2.2 |
5.3 |
4.2 |
2.3 |
Previous Low |
-16.2 |
-2.2 |
-3.8 |
-16.7 |
-4.0 |
-5.6 |
| October | ||||||
2003 |
2.6 |
1.4 |
-1.4 |
-6.6 |
-3.8 |
4.4 |
Previous High |
3.7 |
2.4 |
4.5 |
6.4 |
4.4 |
4.9 |
Previous Low |
-5.5 |
-2.6 |
-3.8 |
-4.8 |
-3.5 |
-9.4 |
| November | ||||||
2003 |
0.7 |
-0.5 |
1.0 |
-0.6 |
0.0 |
0.8 |
Previous High |
4.4 |
2.1 |
10.1 |
4.4 |
2.3 |
5.2 |
Previous Low |
-6.6 |
-2.9 |
-2.7 |
-3.5 |
-1.5 |
-2.2 |
| Table 4. Actual
and predicted price reactions. |
||||
|---|---|---|---|---|
| CORN | SOYBEANS | |||
| MONTH |
ACTUAL PRICE REACTION |
PREDICTED PRICE REACTION |
ACTUAL PRICE REACTION |
PREDICTED PRICE REACTION |
% |
% |
|||
| August 2003 | 6.4 |
2.8 | 3.7 | 2.9 |
| September 2003 | -0.8 |
-1.1 |
5.3 |
1.7 |
| October 2003 | -1.4 |
-1.2 |
4.4 |
4.3 |
| November 2003 | 1.0 |
1.2 |
0.8 |
0.2 |