Understanding the Sims-Cogley-Nason Approach in A Finite Sample

Kehoe2006 advocates that in evaluating an economic model, the Sims-Cogley-Nason (SCN) approach should be adopted in which empirical impulse responses are compared to those obtained from the identical structural VAR run on model generated data of the same length as actual observations. This paper examines, using Monte Carlo simulation, finite sample properties of the SCN approach. Throughout the paper, we use the simple textbook New-Keynesian model as data generating process, and focus on effects of the identified monetary shocks, derived by structural VAR with short-run identification assumption. We find that when the model violates the identifying restriction and monetary shocks are misidentified, the SCN approach has poor small sample performance. We show that: 1) The estimated impulse responses are biased and uninformative; 2) The parameter estimates derived by matching impulse responses are biased and with large mean square error. Ironically, the very reason calling for the SCN approach - mis-identification, is also the cause for its poor finite sample performance.