Using experimental manipulation of questionnaire design and a Kenyan panel to test for the reliability of reported perceptions of climate change and adaptation

While the use of surveys to understand perception of climate change and adaptation is common in research on agriculture, the reliability of some aspects of the methodology is largely untested. In particular, there is limited evidence on (i) the degree to which measures of perception are sensitive to questionnaire design, (ii) the accuracy of recall methods for climate change, and (iii) the degree to which measures of adaptation based on recall from one-time surveys match the historical record. Using an established panel of farmers from across Kenya and a split sample method, I test both the sensitivity of stated perceptions of climate change to question format and the accuracy of recalled adaptations. In one treatment, farmers face open-ended questions about temperature and rainfall changes while in the other treatment, farmers are offered closed-end questions. Both approaches are common in the voluminous literature on climate change adaptation. Responses are highly sensitive to question format, both in the degree of perceived change and in the types of changes. Stated adaptations are not so sensitive to question format, but still diverge. Stated adaptations do not correspond well to the historical record of farming practices over the 15 years of the panel. Overall, the evidence suggests that researchers and policy-makers should be highly cautious in their use of subjective perceptions of climate change and the use of adaptation measures based on recall data.