PDF

The primary goal of this study is doing a meta-analysis research on two groups of published studies. First, the ones that focus on the evaluation of the United States Department of Agriculture (USDA) forecasts and second, the ones that evaluate the market reactions to the USDA forecasts. We investigate four questions. 1) How the previously published studies evaluate the accuracy of the USDA forecasts? 2) How they evaluate the market reactions to the USDA forecasts? 3) Is there any heterogeneity in the results of the mentioned studies? 4) Is there any publication bias? About the first question, while some researchers argue that the forecasts are unbiased, most of them maintain that they are biased, inefficient, not optimal, or not rational. About the second question, while a few studies claim that the forecasts are not newsworthy, most of them maintain that they are newsworthy, provide useful information, and cause market reactions. About the third and the fourth questions, based on our findings, there are some clues that the results of the studies are heterogeneous, but we could not find enough evidences of publication bias.

USDA forecasts

• Aulerich, N., Irwin, S., Nelson, C. (2007), “The Impact of Measurement Error on Estimates of the Price Reaction to USDA Crop Reports”, NCCC-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management, Chicago, Illinois, USA. No. 1313-2016-102638.
• Baujat, B., Mahé, C., Pignon, J. P., Hill, C. (2002), “A graphical method for exploring heterogeneity in meta‐analyses: application to a meta‐analysis of 65 trials”, Statistics in medicine, 21(18), 2641-2652.
• Begg, C., Mazumdar, M. (1994), “Operating Characteristics of a Rank Correlation Test for Publication Bias”, Biometrics, 50(4), 1088-1101. Colling, P. L., Irwin, S. H. (1990), “The reaction of live hog futures prices to USDA hogs and pigs’ reports”, American Journal of Agricultural Economics,72(1), 84-94.
• Colling, P. L., Irwin, S. H., Zulauf, C. R. (1992), “Weak-and strong-form rationality tests of market analysts' expectations of USDA hogs and pigs reports”, Review of Agricultural Economics, 14(2), 263-270.
• Colling, P. L., Irwin, S. H., Zulauf, C. R. (1996), “Reaction of Wheat, Corn, and Soybean Futures Prices to USDA" Export Inspections" Reports”, Review of Agricultural Economics, 127-136.
• Colling, P., Irwin, S., Zulauf, C. (1997), “Future price responses to USDA's Cold Storage report”, Agribusiness, 13(4), 393-400.
• Darby, J. L. (2015), Information Content of USDA Rice Reports and Price Reactions of Rice Futures (Doctoral dissertation, University of Arkansas).
• Egelkraut, T. M., Garcia, P., Irwin, S. H., Good, D. L. (2003), “An evaluation of crop forecast accuracy for corn and soybeans: USDA and private information agencies”, Journal of Agricultural and Applied Economics, 35(01), 79-95.
• Egger, M., Smith, G. D., Schneider, M., Minder, C. (1997), “Bias in meta-analysis detected by a simple, graphical test”, British Medical Journal, 315(7109): 629-634.
• Flora, C. (1993), “Revisiting the USDA reports: Context, capital and organization”, American Journal of Alternative Agriculture, 8(4), 155-157.
• Fortenbery, T. R., Sumner, D. A. (1993), “The effects of USDA reports in futures and options markets”, Journal of Futures Markets, 13(2), 157-173.
• Glass, G. V. (1976), “Primary, secondary, and meta-analysis of research”, Educational researcher, 5(10), 3-8.
• Good, D. L, Irwin, S. H. (2005), “Understanding USDA Corn and Soybean Production Forecasts: Methods, Performance and Market Impacts over 1970-2004”, University of Illinois at Urbana Champaign, Department of Agricultural and Consumer Economics, AgMAS Project Research Reports. 1-48.
• Good, D. L., Irwin, S. H. (2006), “Understanding USDA corn and soybean production forecasts: Methods, performance and market impacts over 1970-2005”, IDEAS Working Paper Series from RePEc, 1-49.
• Gunnelson, G., Dobson, W. D., Pamperin, S. (1972), “Analysis of the accuracy of USDA crop forecasts”, American Journal of Agricultural Economics,54(4 Part 1), 639-645.
• Higgins, J. P., Thompson, S. G. (2002), “Quantifying heterogeneity in a meta‐analysis”, Statistics in medicine, 21(11), 1539-1558.
• Hoffman, L. A., Etienne, X. L., Irwin, S. H., Colino, E. V., Toasa, J. I. (2015), “Forecast performance of WASDE price projections for US corn”, Agricultural Economics, 46(S1), 157-171.
• Irwin, S. H., Good, D. L., Gomez, J. K., Isengildina, O. (2001). “The value of USDA outlook information: an investigation using event study analysis”, In NCR Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management, St. Louis MO. No. 1265-2016-101891: 1-40.
• Irwin, S. H., Sanders, D. R., Good, D. L. (2014), “Evaluation of Selected USDA WAOB and NASS Forecasts and Estimates in Corn and Soybeans”, Marketing and Outlook Research Report, 1. 1-140.
• Isengildina-Massa, O., Irwin, S. H., Good, D. L. (2004), “Does the Market Anticipate Smoothing in USDA Crop Production Forecasts?”, In 2004 Annual meeting, August 1-4, Denver, CO (No. 20145). American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association). No. 377-2016-21017: 1-19.
• Isengildina-Massa, O., Irwin, S. H., Good, D. L. (2013a), “Do Big Crops Get Bigger and Small Crops Get Smaller? Further Evidence on Smoothing in US Department of Agriculture Forecasts”, Journal of Agricultural and Applied Economics, 45(01): 95-107.
• Isengildina-Massa, O., Irwin, S., Good, D. (2006), “Are Revisions to USDA Crop Production Forecasts Smoothed?”, American Journal of Agricultural Economics, 88(4), 1091-1104.
• Isengildina-Massa, O., Irwin, S., Good, D., Massa, L. (2011), “Empirical confidence intervals for USDA commodity price forecasts”, Applied Economics, 43(26), 3789-3803.
• Isengildina-Massa, O., Karali, B., Irwin, S. H. (2013b), “When do the USDA forecasters make mistakes?”, Applied Economics, 45(36), 5086-5103.
• Isengildina-Massa, O., Macdonald, Stephen, Xie, Ran. (2012), “A comprehensive evaluation of USDA cotton forecasts”, Journal of Agricultural and Resource Economics: JARE; the Journal of the Western Agricultural Economics Association, 37(1), 98-113.
• Isengildina-Massa, O., Tysinger, D., Gerard, P., MacDonald, S. (2011), “What Can We Learn from our Mistakes? Evaluating the Benefits of Correcting Inefficiencies in USDA Cotton Forecasts”, In 2011 Annual Meeting, February 5-8, 2011, Corpus Christi, Texas (No. 98811). Southern Agricultural Economics Association: 1-20.
• Isengildina-Messa, O., Mattos, F. (2015), “Accuracy-Informativeness Tradeoff for Interval Forecast Comparison”, NCR-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management St. Louis, Missouri. 1-13.
• Jarrell, S. B., Stanley, T. D. (1990), “A meta-analysis of the union-nonunion wage gap”, ILR Review, 44(1), 54-67.
• Karali, B. (2012), “Do USDA Announcements Affect Comovements Across Commodity Futures Returns?”, Journal of Agricultural and Resource Economics,37(1): 77-97.
• Kastens, T. L., Schroeder, T. C., Plain, R. (1998), “Evaluation of extension and USDA price and production forecasts”, Journal of Agricultural and Resource Economics, 244-261.
• Keegan, Charles E., III, Morgan, Todd A., Wagner, Francis G., Cohn, Patricia J., Blatner, Keith A., Spoelma, Timothy P., Shook, Steven R. (2005), “Capacity for utilization of USDA Forest Service, Region I small-diameter timber”, Forest Products Journal, 55(12): 143-147.
• Lee, Robert D, Nieman, David C, Rainwater, Marvin. (1995), “Comparison of Eight Microcomputer Dietary Analysis Programs with the USDA Nutrient Data Base for Standard Reference”, Journal of the American Dietetic Association, 95(8), 858-867.
• Manfredo, M. R., Sanders, D. R. (2004), “The value of public price forecasts: Additional evidence in the live hog market”, Journal of Agribusiness, 22(2): 119-131.
• McKenzie, A. M. (2008), “Pre-harvest price expectations for corn: The information content of USDA reports and new crop futures”, American Journal of Agricultural Economics, 90(2), 351-366.
• Meyer, S. R., Lawrence, J. D. (1988), “Comparing USDA Hogs and Pigs Reports to Subsequent Slaughter: Does Systematic Error Exist?”, University of Missouri--Columbia, Department of Agricultural Economics. No. 1717-2017-519: 19-35.
• No, S. C., Salassi, M. E. (2009), “A sequential rationality test of USDA preliminary price estimates for selected program crops: Rice, soybeans, and wheat”, International Advances in Economic Research, 15(4), 470-482.
• Olkin, I. (1995), “Meta-Analysis: Reconciling the Results of Independent Studies”, Statistics in medicine 14.5-7: 457–472.
• Papendick, R. (1993), “Revisiting the USDA reports: Some additional observations”, American Journal of Alternative Agriculture, 8(4), 154-155.
• Patterson, P. M., Brorsen, B. W. (1993), “USDA Export Sales Report: Is It News?”, Review of Agricultural Economics, 367-378.
• Pruitt, J. R., Tonsor, G. T., Brooks, K. R., Johnson, R. J. (2014), “End user preferences for USDA market information”, Food Policy, 47, 24-33.
• Ray, D. (2008), “USDA top officials vs. USDA data”, Review of African Political Economy, 35(117), 514-516.
• Roberts, M. J., Schimmelpfennig, D. E., Ashley, E., Livingston, M. J., Ash, M. S., Vasavada, U. (2006). The value of plant disease early-warning systems: a case study of USDA's soybean rust coordinated framework, No. 1477-2016-121162: 1-38.
• Sanders, D. R., Manfredo, M. R. (2002), “USDA production forecasts for pork, beef, and broilers: an evaluation”, Journal of Agricultural and Resource Economics, 27(01), 114-127.
• Sanders, D. R., Manfredo, M. R. (2003a), “Keep up the good work? An evaluation of the USDA’s livestock price forecasts”, NCR-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management, NCR-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management, 2003 Conference, St. Louis, Missouri, April 21-22. Sanders, D. R., Manfredo, M. R. (2003b), “USDA livestock price forecasts: A comprehensive evaluation”, Journal of Agricultural and Resource Economics, 316-334.
• Sanders, D. R., Manfredo, M. R. (2005), “A Test of Forecast Consistency Using USDA Livestock Price Forecasts”, In 2005 Conference, April 18-19, 2005, St. Louis, Missouri (No. 19042). NCR-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management. 1-12.
• Sanders, D. R., Manfredo, M. R. (2008), “Multiple horizons and information in USDA production forecasts”, Agribusiness, 24(1), 55-66.
• Schaefer, M. P., Myers, R. J. (1999), Forecasting accuracy, rational expectations, and market efficiency in the US beef cattle industry (Master's thesis, Michigan State University. Dept. of Agricultural Economics). 1-16.
• Schroeder, T., Blair, J., Mintert, J. (1990), “Abnormal returns in livestock futures prices around USDA inventory report releases”, North Central Journal of Agricultural Economics, 12(2), 293-304.
• Shapley, D. (1976), “Crops and Climatic Change: USDA's Forecasts Criticized”, Science, 193(4259), 1222-1224.
• Sterne, Jonathan A.C, Gavaghan, David, Egger, Matthias. (2000), “Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature”, Journal of Clinical Epidemiology, 53(11), 1119-1129.
• Summer, D. A., Mueller, R. A. (1989), “Are harvest forecasts news? USDA announcements and futures market reactions”, American Journal of Agricultural Economics, 71(1), 1-8.
• Viechtbauer, W. (2010), “Conducting meta-analyses in R with the metafor package”, Journal of Statistical Software, 36(3), 1-48.
• Von Bailey, D., Brorsen, B. W. (1998), “Trends in the accuracy of USDA production forecasts for beef and pork”, Journal of Agricultural and Resource Economics, 23(02), 515-525.
• Xiao, J., Lence, S., Hart, C. (2014), “USDA and private analysts’ forecasts of ending stocks: how good are they?” In 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota, USA, No. 170642. 1-25.