Reconstruction of geomagnetic activity and near-Earth interplanetary conditions over the past 167 yr - Part 3: improved representation of solar cycle 11

. Svalgaard (2014) has recently pointed out that the calibration of the Helsinki magnetic observatory’s H component variometer was probably in error in published data for the years 1866–1874.5 and that this makes the interdiurnal variation index based on daily means, IDV(1d), (Lockwood et al., 2013a)

magnetic field B at all times before the space age was as similar as possible to that in modern times.Lockwood et al. (2013b) subsequently used this index to reconstruct B with the first full analysis of reconstruction errors, carried out using a Monte Carlo technique.This homogeneous construction was achieved by using just three magnetic observatories, in the same latitude band and the same longitude sector, to generate the IDV(1d) composite and by using a model of the secular change in the geomagnetic field to allow for the drift of the observatories in geomagnetic coordinates.The problem with this approach is that unknown errors in the data from a station used would be reproduced in the composite.As a check against this possibility, Lockwood et al. (2013a) compared the IDV(1d) composite against similarly derived values from other nearby stations.This check was necessarily poorer for the data from the 19th century when fewer stations were available for comparisons.Specifically, agreement was found to be very good in comparisons with data from Wilhelmshaven (commencing 1883), Parc St Maur (commencing 1883) and Ekaterinburg (commencing 1887).In addition, after 1880 agreement was shown to be close with the IDV index, derived from a global network of a variable number of observatories (Svalgaard and Cliver, 2010).Before 1863, the IDV(1d) composite data were compared with observations from St Petersburg and the agreement was again very good.Between 1863 and 1883 the only two data sets available for comparisons were from Greenwich and Bartels' interdiurnal daily mean variability index u which, like IDV, was compiled from a variable mix of stations.Agreement between IDV(1d) and u was good after about 1880 but Lockwood et al. (2013a) noted that before then it was partly based on Greenwich data for which both they and Bartels had noted considerable stability problems and could only be readily corrected for temperature variations in a statistical manner.In addition, Lockwood et al. (2013a) were concerned about the u index before 1872 because it was based on diurnal range proxies and so was not a homogeneously constructed index.As a result, despite good correlations with IDV over a period of overlap, it could have different general responses to solar wind and IMF (interplanetary magnetic field) properties.
This concern was shown to be well founded.Because of the stability issues with the Greenwich data, Bartels gave them half weighting and so his interdiurnal u data largely came from the Colaba Observatory, Bombay (now Mumbai), and daily mean interdiurnal variation data for modern stations at such a low geomagnetic latitude station show a (BV SW ) n dependence on solar wind speed V SW with an estimate of n of −0.4 ± 0.9 (Lockwood et al., 2013a), the uncertainty being at the 95 % significance level, derived by applying Fisher's Z test to the correlograms with n (as described by Lockwood, 2002).A result of a negative n is that u would be depressed in intervals of enhanced solar wind speed and this is particularly clear during the declining phases of solar cycles 9 and 10 when u is compared to IDV(1d) which was designed to depend on the IMF strength only (i.e.n ≈ 0).This was confirmed to be the case for cycle 9 by the IDV(1d) variation derived from the St Petersburg data, IDV(1d) SPE (the three-letter IAGA observatory code for St Petersburg is SPE).In the light of this finding, Lockwood et al. (2013a) concluded that the difference between u and IDV(1d) at the peak of solar cycle 11 could have arisen for a lower than average solar wind speed at that time and so was not necessarily due to an instrumental error in either data series.
However, Svalgaard (2014) noted that, although the diurnal range in annual means of the Helsinki horizontal (H ) component data was similar to that from other stations for most of years for which that station operated (giving usable annual means for 1845-1897), for 8.5 of those years (1866-1874.5) it was lower.Furthermore, using a linear regression of group sunspot number with IDV(1d), values were found to be low at this time, whereas for all other years the agreement was considerably better.He concluded that poor calibration of the "horizontal force" variometer at Helsinki was causing the H component (and hence the IDV(1d) values based on it) to be low in these years.Previously, Nevanlinna (2004) had noted that the range index Ak(H ) HLS , scaled from the three-hourly range k index derived from the H data from Helsinki (the IAGA code for Helsinki is HLS) were lower than the linearly regressed aa range index (derived from a Northern Hemisphere and a Southern Hemisphere station) at this time which could also have been caused by the same calibration problem.Nevanlinna (2004) also noted that Ak(H ) HLS at this time was also systematically lower than Ak(D) HLS (scaled from the station's three-hourly range k index derived from the declination data, D).The H data require a temperature correction and this was implemented by Nevanlinna and Ketola (1993) and, in the absence of any data to check against, a constant sensitivity of the H variometer was applied throughout.These authors also noted that, because the D measurements require no temperature correction and employed a simpler observing geometry, they may be the more reliable of the two in the historical data.From his analysis, Svalgaard (2014) proposed that the Helsinki H , Ak(H ) HLS and IDV(1d) data all need recalibration for the period 1866-1874.5 with an increase of around 30 %. Lockwood et al. (2013a, b) were reluctant to make such a recalibration for a number of reasons: (1) there was considerable variability between Ak(H ) HLS , Ak(D) HLS and aa in all intervals; (2) using sunspot numbers to recalibrate destroys the independence of the geomagnetic and sunspot data sequences; and (3) if the uncertainties in the recalibration were sufficiently large, they could make the IDV(1d) variation appear unwarrantedly similar to the group sunspot data record.This last concern is shown to be well founded in the next section.Subsequent sections study the accuracy and validity of the proposed cycle 11 recalibrations.and IDV(1d) (left-hand panels) on the simultaneous annual means of the international sunspot number, R. For IDV(1d), results for before 1872 are given for both the original index as presented in Lockwood et al. (2013a, b: hereafter "Paper 1" and "Paper 2") (black line) and using the correction derived in the present paper (green line).The top panels (a and b) show the correlations with R n , as a function of the exponent n.The data have been divided into three intervals: 1844-1872, 1873-1945 and 1946-2012 (inclusive), shown by the black, red and blue lines, respectively.The division on 1 January 1946 was implemented because there has been discussion about a potential discontinuity in R around this date (e.g., Svalgaard, 2012), the division on 1 January 1872 occurs because after this date IDV and IDV(1d) agree closely but less so than before then (see Paper 1).It can be seen that after 1872, both indices agree quite closely on the optimum n value and the level of correlation it yields (the red and blue lines).

Ann
The fact that correlations are almost as high for IDV(1d) as for IDV is, at first sight, surprising because IDV is based on a number of stations that varies with date from 1 to over 60, whereas IDV(1d) is based on just one station at any one time.However, IDV uses just one hourly mean value per day, whereas IDV(1d) uses all 24.If we regard data taken by the same station at different Universal Time (UT) as differing only in their UT-dependent calibration, we have a suppression of geophysical and instrument noise by an additional factor in IDV(1d) of 24 0.5 = 4.89, which is only achieved using 24 stations for IDV.Hence although noise suppression in IDV(1d) is better for later years (when more than 24 stations are available), it is poorer in early years.In addition, as discussed in Paper 1, the point of IDV(1d) is that it is, unlike IDV, homogeneously constructed to ensure its responses to solar wind parameter variations were as similar in past epochs to those in the space age as it is possible to make them.
The peak correlations r in Fig. 1 for post-1872 data are all in the range 0.821-0.848and for n in the range 0.5-0.75.The middle panels (c and d) show the variation of the significance S of the difference between the peak correlation and that at general n, as determined by Lockwood (2002).It can be seen that the differences between the S(n) variations are not statistically significant after 1872.However, for neither IDV(1d) or IDV is the same true before 1872.For (uncorrected) IDV(1d), the peak r is 0.806 at n = 0.29 (the black line in Fig. 1a).For IDV, however, the peak r is 0.942 at n = 1.18 (the black line in Fig. 1b).These discrepancies could denote problems with either the geomagnetic indices or with the sunspot number record, or both.However, it is noticeable that the error is in the opposite sense for the two geomagnetic indices, with peak correlation at somewhat lower n for IDV(1d) but considerably higher n for IDV.The black lines in Fig. 1e and f show the probabilities S that the true n is the same before 1872 as after it: for IDV(1d) this peaks at 87 % whereas for IDV it peaks at just 62 %. Figure 1 therefore shows that neither index is behaving consistently before 1872.In the case of IDV, the correlation with R is far higher than for after 1872 (peaking at 0.942) and is for an optimum value of n that is close to unity.Hence it is clear that the sunspot number and the IDV geomagnetic index are no longer independent data before 1872, making IDV far too similar to R in this interval.This may be because the sunspot number has been used to correct and/or calibrate the geomagnetic data, or vice versa, or both.Hence the concerns about using R to correct geomagnetic indices expressed in Paper 2 are well founded.That having been said, the IDV(1d) index before 1872 is also not behaving in the same way with R as it does in later years, indicating that it too is in error.In the present paper, we derive a correction that does not depend on sunspot number, thereby avoiding the inhomogeneity that Fig. 1 shows that the IDV index suffers from.For completeness, the green lines in Fig. 1a, c and d show the results for IDV(1d) after that correction has been applied.In this case, the n giving a peak correlation of 0.68 is not significantly different to that for the intervals after 1872.The peak correlation coefficient ris lower (0.782), which is to be expected given the increased uncertainties in both the geomagnetic and sunspot data for the earliest years.The green line in Fig. 1e shows that the probability that the behaviour with R for the corrected IDV(1d) is different before 1872 to that after 1872 is just 0.003.

Retrospective recalibration techniques
3.1 Using the diurnal range of geomagnetic data from other observatories Svalgaard (2014) makes use of the measured diurnal variation in annual means of the H component to recalibrate the H variometer.He points out that the average diurnal range (the difference between the maximum and minimum values of the average diurnal variation, over an extended period such as a year) is very well correlated with solar activity indices such as group sunspot number.That being the case, the average diurnal range should be the same at all stations (provided they are removed from the influence of auroral currents) and so comparing measured diurnal ranges from different stations allows one to intercalibrate those stations and comparing to group sunspot number provides a second test.However, the term "very well correlated" is subjective and does not quantify the uncertainty inherent in adopting this approach.Correlation coefficients r are typically 0.9 meaning that (1-r 2 ) ≈ 0.2 of the variation at one station is not matched at another and hence calibration using these correlations will typically be accurate to about 20 %.
The red and green lines in Fig. 2 show the annual mean diurnal range of the H component observed by modern-day stations Nurmijarvi and Eskdalemuir ( H NUR and H ESK ).Nurmijarvi is the closest magnetometer to the old Helsinki Observatory and Eskdalemuir and Helsinki are the primary observatories contributing to the IDV(1d) index (the coordinates of all three stations are given in Table 1 of Paper 1).Also shown (in black) is the linearly regressed group sunspot number, R G , as observed by the network coordinated by the Royal Greenwich Observatory (RGO) until 1976 and, thereafter, from the US Air Force (USAF) and US National Oceanic and Atmospheric Administration (NOAA) Solar Optical Observing Network (SOON).These data have been homogenised into a single data set (see Hathaway, 2010).The correlations are indeed good, between H NUR and H ESK it is 0.972 (significant at > 99.999 % level by comparison against the AR1 red-noise model), between H NUR and R G it is 0.897 (significant at the 75 % level) and between H ESK and R G it is 0.935 (significant at the 78 % level).However it can be seen that there are differences between the two diurnal range variations, potentially due to site differences but also due to calibration uncertainties.In particular, agreement between the diurnal ranges and sunspot numbers during solar cycle 20 (peaking around 1970) is notably poor, especially if 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005  group sunspot numbers, R G , are used rather than the international sunspot number, R.
Figure 3 shows the best linear regression fits of the diurnal range variations from a large number of stations (in grey).These were evaluated for a total of 81 stations.However it was found auroral stations correlated badly and so all stations poleward of the corrected geomagnetic latitude = 60 • were excluded.This left 67 stations and of these a further 5 were removed because the correlation with H NUR was below 0.65, indicating possible calibration problems.The green and mauve lines show the variations of the 5th and 95th percentiles of the distribution for each year and it can be seen that there is considerable spread.Some of this arises from intercalibration problems, but there may also be site changes, noise interference and genuine differences in the behaviour at different sites: in particular, sites near the upper limit of = 60 • will be subject to greater auroral effects and this includes Nurmijarvi itself (corrected geomagnetic latitude = 56.91 • ).
The top panel in Fig. 4 presents the total distribution of the fit residuals inherent in the fits shown in Fig. 3, for all years and all 62 stations, as a percentage of the simultaneous H NUR value.The vertical red lines mark the 5th and 95th percentiles which are at percentage deviations of −19 % and +19 %.Thus even for modern stations, using data from one other station gives a calibration that is accurate for any one annual mean to ±19 % (at the 2σ level).This value is consistent with the expectations from the magnitude of the correlation coefficients.For the 19th-century data, there are very few stations available and if their data have been, at any point, intercalibrated using the diurnal range, these stations 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000  will not be providing independent calibration.It is important to note that, because it is at a higher latitude, there is an auroral component to the response of Helsinki, indeed that response is important in the development of IDV(1d) as it gives a dependence on IMF B rather than the BV −0.4 of lower latitude stations (see Paper 1).Thus at least some of the ±19 % uncertainty arises from the latitude of Helsinki.Importantly, Svalgaard's (2014) estimate of a 30 % error during the period 1866-1874.5 is outside the ±19 % uncertainty inherent in the recalibration procedure he adoptedconfirming the Helsinki data are indeed too low in this interval and potentially correcting the error in IDV(1d) also identified in Fig. 1.

Using group sunspot numbers
Svalgaard (2014) also tests diurnal range estimates against group sunspot numbers.However, Fig. 2 shows that although the correlation is high, the inherent errors in this recalibration can also be quite high.The middle panel in Fig. 4 shows the distribution of percentage residuals obtained by fitting the Nurmijarvi and then the Eskdalemuir data with group sunspot number.The largest error seen is again −19 %.Note this is a much smaller sample than in the top panel, using just two data series rather than 63, however, the overall error appears to be similar to that obtained in the previous section.

Using geomagnetic three-hourly-range k values
Figure 5 compares the Ak(H ) series obtained from the range k values from the horizontal field component H measured at Eskdalemuir and Nurmijarvi with the aa C index, also derived from k values.The aa C index is based on the standard aa index of Mayaud (1971Mayaud ( , 1972Mayaud ( , 1980) ) with some corrections devised by Lockwood et al. (2006) by comparison with range k indices and/or the Ap index based on the k indices from a network of midlatitude stations.The biggest difference between aa and aa C is a 2.5 nT shift around 1957 associated with the move of the Northern Hemisphere's aa station from Abinger to Hartland.The correction to aa to yield aa C is discussed further in Paper 4 (Lockwood et al., 2014).The correlation between Ak(H ) NUR and Ak(H ) ESK is 0.948 (significant at the 85.3 % level), between Ak(H ) ESK and aa C it is 0.980 (95 %) and between Ak(H ) NUR and aa C it is 0.975 (91 %).From these levels of correlation we would expect the use of Ak(H ) and aa C to give calibrations that are accurate to the order of 5-10 %.The bottom panel in Fig. 4 shows the distribution of percentage fit residuals of the linear regression of aa C to Ak(H ) NUR and Ak(H ) ESK .As in the other two panels, the vertical red lines mark the 5th and 95th percentiles which are at −9 % and +12 %.This spread is close to half that found using the diurnal range and the group sunspot number.Thus we find comparing with aa to be preferable to comparing diurnal ranges or diurnal range with group sunspot number.
Looking at the 19th-century data from Helsinki, the correlation between Ak(H ) HLS and aa C for the interval of available data (1868-1897, inclusive) is 0.85 (significance > 99.99 %).However, this contains the interval for which Svalgaard (2014) finds the calibrations of the horizontal component instrument to be poorly calibrated (1866-1874.5), and removing these data causes the correlation to rise to 0.92 (however the significance falls to 99 % due to the lower number of data points).The best-fit linear regression for this second fit is (1) Note that for the modern Nurmijarvi data (1953-2011) the corresponding slope is 1.026 with an intercept of zero to within seven decimal figures.The ratio of the regression slopes with aa C for the modern Nurmijarvi data and the historic Helsinki data (0.843/1.026) implies that the sensitivity of the 19th century Helsinki instrument is 82 % of that for the modern Nurmijarvi instrument and this is consistent with the intercalibrations used to generate IDV(1d), as described in Paper 1.The ratio [Ak(H ) HLS ] fit /Ak(H ) HLS from Eq. (1) gives us a correction factor which we can apply to the data from 1868 (when aa C is available) and Fig. 4c indicates that this recalibration can be expected to be accurate to within about ±10 %.The black line bounding the filled orange area is the IDV index, scaled using the regression for the period 1880-2013, as shown in Fig. 12 of Lockwood et al. (2013a).IDV(1d) derived from various early data sets are also shown: from St Petersburg, IDV(1d) SPE (black triangles); from Ekaterinburg, IDV(1d) EKA (cyan stars); from Parc St Maur, IDV(1d) PSM (mauve squares); from Wilhelmshaven, IDV(1d) WLH (yellow diamonds); and from Greenwich, IDV(1d) GRW (white circles).The blue diamonds are derived using the D component observed at Helsinki, IDV(1d) HLS,D .The Greenwich values are compiled using the daily means of H that are uncorrected for temperature variations, with annual IDV(1d) GRW means subsequently corrected (giving the shown uncertainty bands caused by the temperature effects).Solar cycle numbers are shown across the base of the figure.The green-and-black dashed line in Fig. 6 is the variation of IDV(1d) derived Paper 1 from the uncorrected Helsinki magnetometer data.The mauve line shows the data series with the 30 % upward recalibration for the period 1866-1874.5derived by Svalgaard (2014).

The corrected IDV(1d) data series
The green-and-black dashed line in Fig. 6 is the variation of IDV(1d) derived in Paper 1 from the uncorrected Helsinki magnetometer data.The mauve line shows the data series with the 30 % upward recalibration for 1866-1874.5derived by Svalgaard (2014).The green line uses the correction described in Sect.2.3, derived from the observed aa C index for the period 1868-1899, which yields [Ak(H ) HLS ] fit from Eq. (1) and hence the correction factor to H values [Ak(H ) HLS ] fit /Ak(H ) HLS from Eq. (1) (which Fig. 4c shows to be accurate to within ±10 %).This correction cannot be applied before the start of the aa C data in 1868 and for the period 1866-1867, the correction of Svalgaard (2014) 4a is shown to be accurate to within ±20 %.The green and mauve lines are particularly close for the period 1868-1870 which means that this procedure does not introduce a discontinuity into the correction.

Comparison with St Petersburg and declination data
We have recently completed digitising hourly means of H component data in the yearbooks from the St Petersburg observatory (Kupffer, 1853(Kupffer, -1865;;Wild, 1872Wild, -1895;;Rykatchew, 1896Rykatchew, -1898Rykatchew, , 1899Rykatchew, -1908)).The black triangles in Fig. 6 show the annual means derived from daily means of these data, IDV(1d) SPE , presented here for the first time.
Absolute H -component data are available for 1870 and 1873-1906, inclusive.Note that in Fig. 6, no scaling of the IDV(1d) SPE data for 1870 and after has been carried out.After 1883, agreement is very good with other data from Wilhelmshaven, Parc St Maur and Ekaterinburg (IDV(1d) WLH , IDV(1d) PSM , and IDV(1d) EKA -shown by the yellow diamonds, mauve squares and cyan stars, respectively).These data have also not been scaled.The good agreement of all these stations with the Helsinki data (IDV(1d) HLS -green line) is good confirmation that the fit of the calibrations derived in Paper 1 to the Eskdalemuir data (using the Niemegk data) is broadly correct.Note that the earlier St Petersburg data (for up to 1863) has required an upward rescaling of 11 % to fit the amplitude of the variation seen in the Helsinki data.This factor is derived from comparing the Ak(H ) data for Helsinki and St Petersburg (Nevanlinna and Häkkinen, 2010), but we have, at least up to the time of writing, no absolute independent calibration of either the Helsinki or St Petersburg H variometers at this time.
As a further test we have computed IDV(1d) using the declination data from Helsinki.The results for IDV(1d) HLS,D are shown by the blue diamonds in Fig. 6.These data have been scaled by linear regression to the standard IDV(1d) data from Helsinki (which, using the same notation, would be IDV(1d) HLS,H ) to allow for the differences between H and D. Figure 6 shows that the corrected H and D values are give very similar variations at all times.The worst deviations being for the period 1865-1867, for which the correction is most uncertain.
During solar cycles 12 and 13, the various stations are all giving broadly similar results.All the stations imply IDV is slightly too high during the declining phase of cycle 13, but all the data agree well on the first peak of cycle 13 (1892).The station data suggest that the second peak ( 1894) is slightly too large in IDV but slightly too small in IDV(1d).Agreement of all stations and with both IDV and IDV(1d) is better in cycle 12.The correction of the Helsinki data using aa C has slightly reduced the peak of this cycle in IDV(1d) so it now agrees better with the data from the other stations.The white circles show the Greenwich data, and the relatively good agreement found in this cycle encouraged Lockwood et al. (2013a) to think it may be useful, for all its limitations, in cycle 11.Figure 6 shows us that this is not the case and the Greenwich values in cycle 11 are too low.It is not clear if this is because of an over-correction for the stability problems, because of the statistical temperature correction used or because of changes in the observing procedure and/or equipment at Greenwich.
The poor reliability of the Greenwich data in cycle 11, calls into question its use as a check on the variation between cycles 11 and 12.This being the case, the St Petersburg data are the best test and support the slightly deeper minimum in IDV.At the peak of cycle 11, the available IDV(1d) SPE data (1870 and 1873) are closer to the Svalgaard correction than the one implemented here, but given they are intrinsically accurate to about ±20 % and ±10 %, respectively, these differences are not significant and either variation is equally valid.
For cycles 9 and 10, the St Petersburg data provide a test of the Helsinki data.Svalgaard notes the Helsinki H data are consistent with the group sunspot data but, as shown in Fig. 4b, this test is only accurate to 20 %.The St Petersburg data match well after rescaling with a linear regression factor of 1.1 which is well within the uncertainty of the sunspot test.
The St Petersburg data show the major declining phase peak in cycle 9 is real.This peak is absent completely in IDV which, at this time, is based on a best-fit diurnal proxy and not on interdiurnal variation.This strong suppression appears to be due to an inverse dependence of the proxy on solar wind flow speed.This being the case, the diurnal-proxy u data used to extend the IDV into cycles 8-11 (Svalgaard and Cliver, 2010) must be used with great caution and recognising that it does not have the dependence on B alone which is the important characteristic of interdiurnal variation indices.

Revised reconstruction of the near-Earth IMF
We have reapplied the analysis of Lockwood et al. (2013b) (Paper 2) to the corrected IDV(1d) to reconstruct the IMF B from 1845 onwards.The result is shown in Fig. 7.The dark grey band shows the uncertainty associated with the regression and the joins between data from different stations, and so is the corrected version of the error band shown in Fig. 7 of Paper 2. The light grey area in Fig. 7 of the present paper shows this uncertainty combined with that in the raw data from which IDV(1d) is compiled, obtained by comparing Ak(H ) and Ak(D) with data from nearby stations (for the earliest data this is from St Petersburg only).The major difference from the variation presented in Paper 2 is that the peak of cycle 11 is considerably higher such that reconstructions based on IDV(1d) and IDV are no longer different at this time to within the uncertainties.In addition, the minimum before cycle 11 is much less deep, which  1d) and the polynomial fit to B derived in Lockwood et al. (2013b).The dark grey area surrounding this black line is the uncertainty band associated with using this polynomial fit derived using a Monte Carlo technique (see Lockwood et al., 2013b) and the light grey area convolves this uncertainty with the measurement uncertainty.The red line shows the best reconstruction using their linear fit.The green line shows the SC10 reconstruction.Blue dots show the annual means of the observed IMF.
11.However, we note these values are corrected using the diurnal range method and that correction is only accurate to ±20 %.Table 1 gives the full, corrected data sequences for IDV(1d) and IMF B with their associated uncertainty bands.
7 Relationship of near-Earth IMF to group sunspot number Svalgaard and Cliver (2005) note a correlation between the IMF and the square root of the simultaneous group sunspot number (in annual mean data).In this section, we investigate this correlation in the reconstructed IMF shown in Fig. 7.The top panel in Fig. 8 shows correlograms of group sunspot number (to power n), R n G , against the near-Earth IMF B for annual mean data.The square of the correlation coefficient r 2 is shown as a function of n from 0 to 3 for (in red) IMF observations (1964-2012, inclusive), (in black) the reconstructed IMF (1845-2012) from the IDV(1d) index, and (in blue) the modelled IMF .The modelled IMF is based on the modelling of open solar flux (OSF) by Owens and Lockwood (2012) from the continuity equation, using the group sunspot number to quantify the OSF emergence rate (Solanki et al., 2000), and a loss rate that depends on the current sheet tilt (Owens et al., 2011).This is converted into B using the empirical relationship by Lockwood et al. (2009)   G , against the IMF B for annual mean data.The square of the correlation coefficient r 2 is shown as a function of n from 0 to 3 for: (in red) IMF observations (1964-2012, inclusive), (in black) the reconstructed IMF (1845-2012) from the IDV(1d) index, and (in blue) using modelled IMF (1611IMF ( -2012)).(Bottom) The significance S of the difference between the peak correlation and the correlation at a general n.The horizontal dashed grey line is the 90 % significance level.
and the resulting time series is shown in Fig. 1 of Lockwood and Owens (2014).The plot shows that for the modelled and observed IMF data series, the correlations peak at n = 0.4, and for the geomagnetically reconstructed IMF it peaks at n = 0.5.The peak correlation coefficients r and their significance S (computed against the AR-1 auto-regressive "rednoise" model) are r = 0.837 (S = 96.3%) for the IMF data, r = 0.835 (S = 99.98 %) for the geomagnetic reconstruction and r = 0.729 (S > 99.99 %) for the modelled data.Hence for the observations and geomagnetic reconstruction of the IMF, R n G explains about 70 % of the IMF variation, but R n G only explains 53 % of the modelled variation.The bottom panel in Fig. 8 shows the significance S of the difference between the peak correlation and the correlation at a general n using Fisher's Z test as described by Lockwood (2002).The dashed grey line is the 90 % significance level and at this confidence level the uncertainty in the peak n is lower for the longer data series.The length of the data series means that the upper limits to n are 2, 1.5 and 1 for the IMF, geomagnetic and model data series; for all three cases the lower limit is near n = 0.2.Given this large uncertainty in the peak n estimates (at the 90 % confidence level), the difference in the behaviour of the reconstructed IMF and the other two is not significant.the modelled variation after 1845 all show a linear variation.This appears to show a limiting minimum value or "floor" in the IMF of around 4 nT when the group sunspot number R G falls to zero.The only data in Fig. 9 which fall below such a floor is the modelled data for before 1845 and so one might be tempted to dismiss these modelled values as an error, even though the modelling is very successful after 1845 (the cyan squares match the reconstructed values well, giving r = 0.810, S > 99.99).Note that there are unknown uncertainties in the early group sunspot data and these will have added to the scatter in Fig. 9 for the early data.
In this context, it is important to realise that the correlation is only with the simultaneous group sunspot number and that the prior history of the solar activity is not a factor.This is contradicted by a number of pieces of pertinent evidence.Firstly the observed and reconstructed IMF variations clearly show some long-term persistence with the variation over one cycle being an evolution of that seen during the previous cycle.This has been confirmed quantitatively by Lockwood et al. (2011), who show significant autocorrelation (and predictability) over several solar cycles in open solar flux and cosmic ray modulation potential.Another indicator of this persistence is the success of precursor methods, whereby the level of geomagnetic activity, the solar polar field or the near-Earth IMF are used to predict the size of the subsequent maximum in sunspot number (Svalgaard et al., 2005;Lockwood and Owens, 2014).Lastly, we note the importance of the prior history of solar activity in the success of continuity models in explaining the reconstructed OSF variation, as initially demonstrated by Solanki et al. (2000) and as shown by the cyan squares in Fig. 9.This means that it is not just the contemporaneous sunspot number that determines the open solar flux and near-Earth IMF (and related parameters such as the heliospheric modulation parameter), but its prior history also has an influence.It is not immediately obvious over how long a prior period this history matters, but the success of the precursor methods tells us that the polar solar fields half a solar cycle in advance have an influence and Lockwood et al. (2011) show there is some (non-zero) persistence in sunspot numbers up to two solar cycles in advance and a much stronger persistence in open solar flux and heliospheric modulation potential.This provides an explanation of why the behaviours before and after 1844 are so different in the modelled data in Fig. 9: the data after 1844 containing persistently high sunspot numbers (and a grand solar maximum) whereas before 1844 they contain lower sunspot numbers and both the Dalton and the Maunder minima.Here we demonstrate the effect of prior history by using the mean of the group sunspot number over the previous 11 yr, <R G > 11 , in addition to the simultaneous annual value, R G .We also make a small second correction to allow for the known effect of the solar wind speed using the theory of the Parker spiral.We use a simple combination of R G , and <R G > 11 , given by This simple form has been chosen to introduce some dependence on the prior history of the sunspot number (via the term β <R G > m 11 ): it is in no way an optimised form and is used here purely for demonstration purposes.Using the Nelder-Mead search method (Nelder and Mead, 1965;Lagarias et al., 1998), the maximum correlation between the predicted B, B p (given by Eq. 2) and B is obtained for the coefficients α = 0.462, n = 0.454, β = 0.090, m = 0.668 and γ = 0.006.Using these coefficients and Eq.(2) to compute B p , Fig. 10 shows the plot of B as a function of B p for the same data sets as shown in Fig. 9 (using the same symbols).In addition, Fig. 10 makes allowance for the effect on near-Earth IMF of variations of the solar wind speed, V SW , via Parker's spiral theory.This allowance is relevant because increased solar wind speed causes the spiral to unwind and so lowers the near-Earth IMF; as a result, for a given OSF, a higher V SW gives a lower near-Earth B. The factor f (V SW ) used in Fig. 10 gives a correction to a constant solar wind speed of V o and is given by where the garden-hose angle for solar wind speed, V SW , is ϕ = tan −1 (V SW /ωR) and for the solar wind speed, V o , is ϕ o = tan −1 (V o /ωR 1 ), ω is the angular velocity of the solar corona in the frame of the fixed stars and R 1 is one astronomical unit.The average solar wind velocity over recent cycles (470 km s −1 ) is used for V o .This correction for the effect of solar wind speed on the IMF can been made using observed annual means of V SW for 1964 onwards and for 11 +γ (where R G is the simultaneous annual mean group sunspot number and <R G > 11 is the mean value over the preceding 11 yr) with best-fit coefficients α = 0.462, n = 0.454, β = 0.090, m = 0.668 and γ = 0.006.The factor f (V SW ) allows for the effect on IMF B of the solar wind speed V SW using Eq. ( 2), based on Parker's spiral theory.
1868 onwards for the reconstructed data using the solar wind variation derived by Lockwood and Owens (2014), using the combination of the corrected aa index, aa C , and the corrected IDV(1d) index discussed in previous sections of the present paper.For the model, the correction term is unity because the modelled values were generated from OSF predictions and then converted to B assuming constant solar wind speed.
Figure 10 shows that even using this simple form of Eq. (2) the same linear relation between B and B p is shared by all three data sets.The correlation coefficient for the 167 annual means of the geomagnetic reconstruction of B in Fig. 9 is r = 0.835, whereas for that the 144 annual means in Fig. 10 is r = 0.833.Allowing for the number of fit variables, the Fisher Z test gives that the difference between these two correlations is significant at just 4.5 %.For the 49 observed IMF annual means the correlations are 0.837 and 0.843, respectively, a difference that is significant at only the 5.5 % level.However, for the 399 modelled data points the correlation rose from 0.729 to 0.861 which is a significant increase at over the 99.99 % level.Furthermore, all data sets are now giving very similar correlations.Note that the dependence of the term in <R G > 11 is relatively weak with the coefficient β being roughly a fifth of the corresponding coefficient α for R G and the exponents n = 0.454 and m = 0.668, meaning that the term αR n G is typically twice β <R G > m 11 .Thus, making a relatively small and simple allowance for the prior history of the sunspot number explains how the modelled values can be below the apparent floor value before 1845, whilst matching the data and reconstruction www.ann-geophys.net/32/367/2014/Ann.Geophys., 32, 367-381, 2014 after 1845.Making this simple allowance for the prior R G history makes no significant difference to the fits to data after 1845 (for which we have geomagnetic reconstructions) or after 1964 (for which we have in situ observations).This directly demonstrates that the available IMF data and reconstruction do not extend into a wide enough range of conditions to differentiate between the two sunspot number fits and hence allow detection of the dependence on the prior history of R G .Thus one has to look very carefully at the assumptions inherent in a linear regression fit before using it to derive an estimate of any floor value to the near-Earth IMF.

Conclusions
We find that the correction of Svalgaard (2014) to IDV(1d) for solar cycle 11 is largely correct.We do this in three ways, firstly using daily Ak(H ) indices and comparing to the corrected aa index, aa C .Secondly we use the newly digitised St Petersburg data which is conveniently close to Helsinki and so provides a very valuable test.Finally we derive IDV(1d) HLS,D from the declination observations at Helsinki.We show that the Ak(H ) test and recalibration is roughly twice as accurate as methods using the average diurnal range, comparing to other stations or to group sunspot number.The strong dependence of diurnal range-based proxies on solar wind speed means that, as yet, we do not have any digitised geomagnetic data on which we can reconstruct the IMF before the start of the Helsinki data midway through 1844.Thus, accurate reconstruction between 1835 (just 3 yr after Gauss constructed the first magnetometer) and 1844 is not possible until we have found, digitised and calibrated, some other historic magnetometer data.In relation to that task, we urge considerable caution in using average diurnal range and sunspot number to calibrate and correct those data.
Our tests show that such recalibrations are only accurate to ±20 % even on modern magnetometer and group sunspot data and led to Bartels' u index (and hence the IDV index that employed it) becoming incorrectly similar to the sunspot number in the years before 1872.We have shown that there is a consistent behaviour of observed, reconstructed and modelled IMF variations only if the role of prior solar activity over an extended period is considered, and not just the simultaneous value.This analysis indicates that estimates giving a high-level floor to the near-Earth IMF arise from the use of data that does not extend back into the Dalton or Maunder minima and fail to recognise the role of prior solar activity.One important implication of the role of the prior history is that it means predictions of future solar activity have a sound physical basis in addition to the statistical predictability found in heliospheric parameters (such as open solar flux, the interplanetary magnetic field and the cosmic ray modulation potential) over several solar cycles (Lockwood et al., 2010).These predictions have ranged from half a cycle in advance (Svalgaard et al., 2005) to several cycles in advance (Lockwood, 2010;Barnard et al., 2011).The applications of such predictions in the area of space weather are well documented and include satellite orbit prediction, integrated radiation dose prediction for electronics on spacecraft and aircraft, corrosion maintenance planning on long pipelines, and many more.Although the influence of future changes in solar activity on global mean temperatures is predicted to be small (Feulner and Rahmstorf, 2010;Jones et al., 2012) there are interesting indications of some regional influences, particularly in winter (Lockwood et al., 2010;Lockwood, 2012;Woollings et al., 2010), increasing the importance of making predictions of the level of solar activity.

Figure 1
Figure1presents an analysis of the dependence of the annual means of the geomagnetic indices IDV (right-hand panels) and IDV(1d) (left-hand panels) on the simultaneous annual means of the international sunspot number, R. For IDV(1d), results for before 1872 are given for both the original index as presented inLockwood et al. (2013a, b:  hereafter "Paper 1" and "Paper 2") (black line) and using the correction derived in the present paper (green line).The top panels (a and b) show the correlations with R n , as a function of the exponent n.The data have been divided into three intervals:1844-1872, 1873-1945 and 1946-2012 (inclusive), shown by the black, red and blue lines, respectively.The division on 1 January 1946 was implemented because there has been discussion about a potential discontinuity in R around this date (e.g.,Svalgaard, 2012), the division on 1 January 1872 occurs because after this date IDV and IDV(1d) agree closely but less so than before then (see Paper 1).It can be seen that after 1872, both indices agree quite closely on the optimum n value and the level of correlation it yields (the red and blue lines).The fact that correlations are almost as high for IDV(1d) as for IDV is, at first sight, surprising because IDV is based on a number of stations that varies with date from 1 to over 60, whereas IDV(1d) is based on just one station at any one time.However, IDV uses just one hourly mean value per day, whereas IDV(1d) uses all 24.If we regard data taken by the same station at different Universal Time (UT) as differing only in their UT-dependent calibration, we have a suppression of geophysical and instrument noise by an additional factor in IDV(1d) of 24 0.5 = 4.89, which is only achieved using 24 stations for IDV.Hence although noise suppression in IDV(1d) is better for later years (when more than 24 stations are available), it is poorer in early years.In addition, as discussed in Paper 1, the point of IDV(1d) is that it is, unlike IDV, homogeneously constructed to ensure its responses to solar wind parameter variations were as similar in past epochs to those in the space age as it is possible to make them.The peak correlations r in Fig.1for post-1872 data are all in the range 0.821-0.848and for n in the range 0.5-0.75.The middle panels (c and d) show the variation of the significance S of the difference between the peak correlation and that at general n, as determined byLockwood (2002).It can be seen that the differences between the S(n) variations are not statistically significant after 1872.However, for neither IDV(1d) or IDV is the same true before 1872.For (uncorrected) IDV(1d), the peak r is 0.806 at n = 0.29 (the black line in Fig.1a).For IDV, however, the peak r is 0.942 at n = 1.18 (the black line in Fig.1b).These discrepancies could denote problems with either the geomagnetic indices or with the sunspot number record, or both.However, it is noticeable that the error is in the opposite sense for the

Fig. 1 .
Fig. 1.The dependence of the IDV(1d) (corrected and corrected, left panels) and IDV (right panels) on international sunspot number, R, to the power n, as a function of the exponent n.(a) and (b) show the correlation coefficients for the interval 1845-1872 (black line), 1873-1945 (red line) and 1946-2012 (blue line).In the case of IDV(1d) the black line is uncorrected and the green line shows the effect of implementing the correction derived in the present paper.The coloured circle indicates the peak of each correlogram.(c) and (d), the significance S of the differences between the peak correlations and those at general n.(e) and (f), the probability S that the dependence on R before 1872 is the same as after that date (see text for details).

Fig. 2 .
Fig. 2. The annual, mean diurnal range for modern day stations Nurmijarvi (in green) and Eskdalemuir (in red), H NUR and H ESK for the period 1953-2008.Also shown (in black) are the best fit linear regression to H NUR of the annual means of the group sunspot number R G taken from the Royal Greenwich Observatory observations up to 1976 and the USAF/NOAA SOON network thereafter.

Fig. 4 .
Fig. 4. Distributions of percentage, linear-regression fit residuals.(Top panel) For the linear regression fits of H X (where X is the observatory code), to H NUR for the 62 variations shown in Fig. 3. (Middle panel) For the linear regression fits of group sunspot number to the H NUR and H ESK variations.(Bottom panel) For the linear regression fits of the aa range index to Ak(H ) NUR and Ak(H ) ESK .In each panel the vertical, red lines give the 5th and 95th percentiles of the distribution, the values of which are given in red.

Fig. 6 .
Fig. 6.Revised version of Fig. 14 of Lockwood et al. (2013a) (Paper 1).Annual means of the IDV(1d) composite are shown with data originating from ESK in red, NGK in blue and HLS in green, surrounded by a grey area showing the band of uncertainty that arises from the two regression fits.The HLS data shown by the green line have been corrected in cycle 11 as discussed in the text.The black line bounding the filled orange area is the IDV index, scaled using the regression for the period 1880-2013, as shown in Fig.12ofLockwood et al. (2013a).IDV(1d) derived from various early data sets are also shown: from St Petersburg, IDV(1d) SPE (black triangles); from Ekaterinburg, IDV(1d) EKA (cyan stars); from Parc St Maur, IDV(1d) PSM (mauve squares); from Wilhelmshaven, IDV(1d) WLH (yellow diamonds); and from Greenwich, IDV(1d) GRW (white circles).The blue diamonds are derived using the D component observed at Helsinki, IDV(1d) HLS,D .The Greenwich values are compiled using the daily means of H that are uncorrected for temperature variations, with annual IDV(1d) GRW means subsequently corrected (giving the shown uncertainty bands caused by the temperature effects).Solar cycle numbers are shown across the base of the figure.The green-and-black dashed line in Fig.6is the variation of IDV(1d) derived Paper 1 from the uncorrected Helsinki magnetometer data.The mauve line shows the data series with the 30 % upward recalibration for the period 1866-1874.5derived bySvalgaard (2014).
Fig. 7.Corrected version of Fig.7ofLockwood et al. (2013b) which gives reconstructions of the near-Earth IMF, B, from geomagnetic data.The black line uses the corrected geomagnetic activity composite, IDV(1d) and the polynomial fit to B derived inLockwood et al. (2013b).The dark grey area surrounding this black line is the uncertainty band associated with using this polynomial fit derived using a Monte Carlo technique (seeLockwood et al., 2013b) and the light grey area convolves this uncertainty with the measurement uncertainty.The red line shows the best reconstruction using their linear fit.The green line shows the SC10 reconstruction.Blue dots show the annual means of the observed IMF.

Fig. 8 .
Fig. 8. (Top) Correlograms of group sunspot number, (to power n) R nG , against the IMF B for annual mean data.The square of the correlation coefficient r 2 is shown as a function of n from 0 to 3 for: (in red) IMF observations(1964-2012, inclusive), (in black) the reconstructed IMF (1845-2012) from the IDV(1d) index, and (in blue) using modelledIMF (1611IMF ( -2012)).(Bottom) The significance S of the difference between the peak correlation and the correlation at a general n.The horizontal dashed grey line is the 90 % significance level.

Fig. 9 .
Fig. 9. Scatter plot of near-Earth IMF B as a function of R 0.4G for three data series: (red triangles) in situ observations; (black dots) the reconstructed IMF from the IDV(1d) index, as presented here (with uncertainties in B as shown in Fig.7); (cyan squares) the modelled IMF since 1845; and (yellow diamonds) the modelled IMF before 1845.

Fig. 10 .
Fig. 10.A modified version of Fig. 9 with f (V SW )B shown as a function ofB p = α × R n G + β× < R G > m11 +γ (where R G is the simultaneous annual mean group sunspot number and <R G > 11 is the mean value over the preceding 11 yr) with best-fit coefficients α = 0.462, n = 0.454, β = 0.090, m = 0.668 and γ = 0.006.The factor f (V SW ) allows for the effect on IMF B of the solar wind speed V SW using Eq.(2), based on Parker's spiral theory.

www.ann-geophys.net/32/367/2014/ Ann. Geophys., 32, 367-381, 2014
NUR .The green and mauve lines are the 5th and 95th percentiles of the distribution for each year.The 62 stations selected from the total of 81 analysed are those that gave a correlation with H NUR exceeding 0.65 and were at a corrected magnetic latitude below 60 • .
year Fig.3.Annual, mean diurnal range variations from 62 mid-and low-latitude stations (in grey), H X (where X is the observatory code), linearly regressed against the variation for Nurmijarvi (in black), H The variations of the Ak(H ) indices derived from the horizontal field observed at Nurmijarvi, Ak(H ) NUR (green, available until 2001), and Eskdalemuir Ak(H ) ESK (red) and the linearly regressed corrected aa index, aa C (black).

Table 1 .
The full data series with corrected data for the period 1866-1875.Annual IDV(1d) composite and derived IMF B values are given with 2σ uncertainties.