Benchmarks and software standards: A case study of GARCH procedures

Article type: Research Article

Authors: McCullough, B.D.^a | Renfro, Charles G.^b

Affiliations: [a] Federal Communications Commission, 445 12th Street SW, Room 2C-134, Washington, DC 20554, USA. E-mail: [email protected] | [b] Alphametrics Corporation, P.O. Box 2566, Bala Cynwood, PA 19004-6566, USA. E-mail: [email protected]

Abstract: This paper addresses the evaluation of nonlinear methods in econometric software, taking GARCH procedures as a case study. In particular, it analyzes seven widely used packages, utilizing a recently developed benchmark. Four of the packages are found to be unsuitable, in most cases because the developer either does not specifically indicate which of the many possible GARCH models is being estimated, or does not accommodate the most common model specified in the applied literature, or both. A principal finding is that implementation of the GARCH procedure varies so widely that two packages ostensibly doing the same thing actually may be estimating substantively different models. This lack of standardization raises several questions concerning the evaluation of software. These include the issues normally associated with the creation of benchmarks, but also the critical role that software plays, and can play, in the development of modern econometrics.

DOI: 10.3233/JEM-1999-0160

Journal: Journal of Economic and Social Measurement, vol. 25, no. 2, pp. 59-71, 1998