READ ME File for Supplemental Electronic Material concerning Determination of Verification RE Thresholds (Section 4) in: "The importance of geophysical context in statistical evaluations of climate reconstruction procedures", by C. Ammann and E. Wahl Climatic Change (2007) DOI 10.1007/s10584-007-9276-x [Article hereafter called "AW"] NOTE: all calculations were coded in the R language, v. 2.1.1 The analysis of verification RE thresholds is divided into three components. 1) Generation of red noise pseudo-proxy time series relating to the Mann, Bradley, Hughes ("MBH", 1998/1999) N. Hemisphere reconstruction. All proxies in each MBH segment and Scenarios 5 and 6 in the companion article to this one (*) (hereafter called "WA") were analyzed for their full autocorrelation structure using R's "acf" algorithm. Then these autocorrelation coefficients were used in R's "hosking.sim" algorithm to generate a red-noise time series corresponding to each proxy.(**) The pseudo-proxy series were scaled to have the same variance as the original MBH proxies, as described in Appendix 2 of WA. The hosking.sim process was repeated 1000 times to generate a Monte Carlo set of pseudo-proxies for each MBH segment and each WA Scenario 5 and 6 examination. The pseudo-proxy generator files and necessary MBH/WA proxy data files to run them are included in the folder entitled "Pseudo-proxy_generators". 2) Input of the pseudo-proxies from (1) into the WA emulation of the MBH reconstruction algorithm. [See the electronic supplement to the WA article for the full WA reconstruction code.] In terms of the code from section (1), this input means using the data in the R variable "Proxy_network_sim" in a 1000x loop with the WA reconstruction code. The outputs of calibration and verification N. Hemisphere RE values for all Monte Carlo reconstruction iterations are collected for each MBH segment and each WA Scenario 5 and 6 examination. Each of these 1000x2 matrices is saved as an ASCII file with the name "Validation.stats_SEGMENT/SCENARIO.dat" for use in step (3). [The placeholder "SEGMENT/SCENARIO" stands for the MBH segment or WA Scenario 5 or 6 examination specific to each .dat file.] The "Validation.stats_SEGMENT/SCENARIO.dat" files are included in the folder entitled "Verification_RE_tests". 3) Use of the calibration and verification RE outputs from (2) in the analysis reported in Section 4 of AW. The "Validation.stats_SEGMENT/SCENARIO.dat" files are called into the "WA-MBH_MASTER_CODE_Verif_RE_tests.txt" file, to perform the verification RE threshold analysis per se. [Note that four sub-files specific to WA Scenario 5 and 6 examinations are called into this master file. Two of these sub-files provide code for cases when the real-world WA verification RE values are positive, and the other two provide code when the real-world WA verification RE value are negative (WA Scenarios 5d and 6a-c for the MBH 1400 proxy networks).] The full output from "WA-MBH_MASTER_CODE_Verif_RE_tests.txt" is included in the file called "WA-MBH_Verif_RE_tests_OUTPUT.txt", for a range of calibration/verification RE ratios from 0.5 to 1.5. In this "OUTPUT" file, the category called "test_statistic..." in each case gives the estimated liklihood from the Monte Carlo analysis that verification RE based on red noise proxies will be BELOW the verification RE actually seen in the real-world WA emulations of MBH and WA Scenarios 5 and 6 -- GIVEN the real-world conditions stated in AW Section 4 that the associated calibration RE for a given Monte Carlo iteration must be > 0 and the calibration/verification RE ratio cannot be less than the level specified. Thus, 1 minus the "test statistic..." values represents the estimated liklihood of commiting a Type I error.(***) The quantile values in the "OUTPUT" file similarly give the verification RE value associated with each percentile threshold under the same conditions.(***) All files mentioned in this section are included in the folder entitled "Verification_RE_tests". NOTES (*) E. Wahl and C. Ammann, "Robustness of the Mann, Bradley, Hughes reconstruction of Northern hemisphere surface temperatures: Examination of criticisms based on the nature and processing of proxy climate evidence", Climatic Change (2007), same volume as this article. (**) The manner of using "acf" and "hosking.sim" is derived from code made available at the website of Peter Huybers , which is adapted from code developed by Stephen McIntyre and Ross McKitrick, as entitled below. The original download was on July 26, 2005, for use in analysis reported in WA Appendix 2. Title for McIntyre and McKitrick code, as cited by Huybers at website noted above is: "** THIS IS SCRIPT TO PRODUCE SIMULATIONS, FIGURES AND STATISTICS FOR # 2004GL021750 # Hockey Sticks, Principal Components and Spurious Significance # Stephen McIntyre and Ross McKitrick" (***) When the associated real-world WA verification RE value is positive, a value of -9999 in the quantile results indicates that the specific percentile of the Monte Carlo iterations did not pass one or both of the basic criteria that calibration RE be > 0 and the calibration/verification RE ratio must be greater than or equal to the level specified. [A stratification such as this is necessary to implement the quantile analysis for the cases that do pass the basic criteria.] When the associated real-world WA verification RE value is negative (Scenarios 5d and 6a-c for the MBH 1400 proxy networks), the "test_statistic..." value also gives the estimated liklihood that red-noise-driven verification RE will be BELOW the verification RE actually seen in the real-world WA emulations. In this situation, the additional basic criteria are not evaluated because the negative verification RE values represent validation failures per se, and thus passing or failing the additional conditions has no decision-relevance in relation to these failures. The values of 9999 (note these are positive numbers) in the quantile results indicate percentiles of the Monte Carlo iterations for which the calibration RE and calibration/verification RE ratio conditions ARE satisfied, and the verification RE values are greater than or equal to the real-world (but nonetheless negative) WA verification values. [A stratification such as this is also necessary to implement the quantile analysis in these cases.]