8 Regression Techniques

Because the reconstructions all rely on finding relationships between modern geomagnetic activity data and simultaneous measurements of near-Earth space, regression techniques are needed to enable extrapolation to before the space age. The discussion in the literature between Svalgaard and Cliver (2005Jump To The Next Citation Point), Lockwood et al. (2006aJump To The Next Citation Point), and Svalgaard and Cliver (2006Jump To The Next Citation Point) highlights the many pitfalls in this area.1 This discussion was on the use of the IDV index. Svalgaard and Cliver (2005Jump To The Next Citation Point) (SC05) used IDV to conclude that B increased by only 25% between the 1900s to the 1950s and that this was in contrast to the more than doubling of B which they argued was inherent in the results of Lockwood et al. (1999aJump To The Next Citation Point). Lockwood et al. (2006a) (LEA06) pointed out that some of this difference was due to the fact that Lockwood et al. (1999aJump To The Next Citation Point) actually reported a doubling in the open solar flux, FS, not B (as shown later by Figure 29View Image, FS is not proportional to B). However, there were several other factors, all of which worked in the same direction and so combined to make the estimate of the drift by SC05 exceptionally low. SC05 employed a simple ordinary linear least squares (OLS) regression which yielded residuals that showed heteroscedasticity, some non-linearity, and a systematic bias and which do not have a Gaussian distribution, thereby violating central assumptions of least-squares regression and showing the derived fit is unreliable. The regression results of SC05 were strongly influenced by outliers, which applied great leverage to their regression fit. More reliable regressions were obtained by LEA06 using least median squares (LMS) regression and, better still, using Bayesian statistics (the BLS procedure employed by REA07). In addition, SC05 attempted to fill in the data gaps in the IMF data using a 27-day recurrence technique, despite the relatively low autocorrelation functions of the IMF at 27 day lags, and LEA06 show that this also caused a slight underestimation of the long-term drift (piecewise removal of the IDV data during IMF data gaps is much more reliable). Lastly SC05 under-estimated the long-term change in their own results.
View Image

Figure 22: Scatter plot and best-fit linear regressions between the IDV index and the IMF strength B. The wide variety of regression slopes demonstrates the effect of the choice of regression method used. The light blue line is the fit by SC05, the other lines are the best-fit regressions by LEA06 for residual minimization methods of: (mauve line) Ordinary Least Squares (OLS); (green line) least median of squares (LMS); (blue line) Major Axis Analysis (MAA); (black line) Bayesian Least Squares (BLS).

The initial response by Svalgaard and Cliver (2006Jump To The Next Citation Point) did not accept these arguments, but as shown in the following sections, a subsequent reconstruction by Svalgaard and Cliver (2010Jump To The Next Citation Point) is in very good agreement with the Lockwood et al. (1999aJump To The Next Citation Point) reconstruction. In fact, the change between the Svalgaard and Cliver (2010Jump To The Next Citation Point) and SC05 reconstructions of IMF B was almost exactly what was called for by the residuals analysis of LEA06. This change was caused by the availability of just four additional annual mean datapoints near the long and low minimum between cycles 23 and 24 for which B was low. The fact that change was needed in response to the addition of just a few more datapoints confirms that the original SC05 fit was not robust.

Because the potential pitfalls in regression techniques can have such a major effect on the reconstructions, it is worth exploring the relative merits of the various linear regression procedures used in this context. Figure 22View Image stresses how much they can differ, showing the scatter plot and the various regressions between annual means of the IDV index and the IMF, B. SC05 used OLS but the slope they derived is slightly lower than LEA06’s implementation of OLS because of their different treatment of data gaps. OLS gives the lowest slope, whereas BLS gives the largest.

The details of the regression procedures (with appropriate references for the statistical techniques) and discussion of their relative merits and pitfalls are given in the paper by LEA06. The advantage of the LMS procedure is that it is not as influenced by outliers that can change the slope of the fit dramatically if they have a high value of the Cook-D leverage factor. The MAA (Major Axis Analysis) procedure is inappropriate in the context of these reconstructions and the BLS procedure is as employed by Rouillard et al. (2007Jump To The Next Citation Point) (REA07). The tests described below show that BLS performed best.

View Image

Figure 23: Analysis by Svalgaard and Cliver (2006Jump To The Next Citation Point) of the fit residuals in the fit of IDV against B by Svalgaard and Cliver (2005Jump To The Next Citation Point). The fit residuals are plotted as a function of the fitted value, which is the correct test for homoscedasticity. Image reproduced by permission from Svalgaard and Cliver (2006Jump To The Next Citation Point), copyright by AGU.

Notably, the OLS procedure used by SC05 gives lowest slope in Figure 22View Image, and so would give the lowest long-term drift in the reconstruction of open solar flux. There are a number of ways of evaluating the quality of a regression fit. One of the most important is to check that the fit residuals are randomly and normally distributed: the fit of IDV against B used by SC05 is analysed in Figures 23View Image, 24View Image, and 25View Image.

Fits should be homoscedastic, i.e., the residuals should not show a trend in their spread. In their reply, Svalgaard and Cliver (2006) quite rightly state that this should be tested by plotting fit residuals against the fitted values. Figure 23View Image shows this residual plot which they claim shows the fit is homoscedastic because the mean of the residuals does not change with the fitted value. However, homoscedasticity requires the spread of fit residuals (not their mean value) does not change with the fitted value, and Figure 23View Image shows the spread does increase with increasing fitted values. Hence the fit is heteroscedastic rather than homoscedastic. In addition, the plot shows a marked tendency towards an inverted-U form which is characteristic of some nonlinearity.

View Image

Figure 24: Quantile-quantile (QQ) plots for (left) OLS and (right) BLS regressions of the IDV index and the IMF strength B. These plots are used to check if the distribution of fit residuals is Gaussian, as assumed by least-squares fitting procedures. The ordered, standardized residuals ΔBi āˆ• σ (where σ2 = Σi ΔB2iāˆ•(n − 2 ) and n is the number of samples) are shown as a function of the corresponding quantiles of a standard normal distribution. Deviations from the line of slope 1 shown reveal departures from a normal distribution of standard distribution σ.

A second test used by LEA06 was quantile-quantile (QQ) plots to test if the residuals were normally distributed, as is assumed by all least squares regressions. The standardized residuals are placed in order by size and plotted against the quantiles for a standard normal distribution. The deviations from the straight line of slope 1 reveal departures from a normal distribution. Figure 24View Image shows that the OLS fit gave considerably larger deviations from a Gaussian distribution of residuals than did the BLS fit and so the BLS method is giving the more valid least-squares regression.

Because they found the SC05 fit was heteroscedastic, potentially nonlinear and failed the QQ test for normally-distributed residuals, LEA06 tested for a trend in the residuals by plotting the fit residuals as a function of the observed values. The results are shown in Figure 23View Image for three regression prcodures. The BLS regression meets the requirement that there is no trend (and LEA06 show the LMS regression does as well) but the MAA and OLS fail this bias test. They underestimate the trend in the data because Bfit is consistently an overestimate of B obs when B obs is small and consistently an underestimate of B obs when it is large. Thus, reconstructions based on this OLS fit will self-evidently underestimate the true range of variation in IMF B.

View Image

Figure 25: Analysis of the fit residuals in the fit of IDV against B using: (left) MAA; (middle) OLS; and (right) (BLS). The fit residuals (Bobs − B fit) are shown as a function of Bobs, where Bobs is the observed value and B fit is the fitted value from the IDV index. The dashed line is the linear regression fit to the points to highlight trends. The OLS and MAA fit residuals show strong trends and so should not be used because Bfit is consistently an overestimate of Bobs when Bobs is small and consistently an underestimate of Bobs when it is large. Hence, these fits seriously underestimate the real trend in the data. In contrast, the BLS regression is free of such a trend.

The better the correlation between two parameters, the more similar will be the results of the various regression procedures and, hence, these tests would become increasingly less important. Given that small changes in the fitted slope will make very large differences to the maximum and minimum values seen in a reconstruction, it is very important that these tests are carried out to ensure that the optimum regression procedure has been used and any one regression fit is valid.

  Go to previous page Scroll to top Go to next page