Correcting the geomagnetic IHV index of the Eskdalemuir observatory

. We study here the recently proposed measure of local geomagnetic activity called the IHV (Inter-Hour Variability) index calculated for the Eskdalemuir (ESK) station. It was found earlier that the ESK IHV index depicts an artiﬁ-cial, step-like increase from 1931 to 1932. We show here that this increase is due to the fact that the values of the magnetic ﬁeld components of the ESK observatory stored at the World Data Center are two-hour running averages of hourly data stored in ESK yearbooks. Two-hour averaging greatly reduces the variability of the data which leads to artiﬁcially small values of the IHV index in 1911–1931. We also study the effect of two-hour averaging upon hourly mean and spot values using 1-minute data available for recent years, and calculate the correction factors for the early years, taking into account the weak dependence of correction factors on solar activity. Using these correction factors, we correct the ESK IHV indices in 1912–1931, and revise the estimate of the centennial change based on them. The effect of correction is very signiﬁcant: the centennial increase in the ESK IHV-raw (IHV-cor) index in 1912–2000 changes from 73.9% (134.4%) before correction to 10.3% (25.3%) thereafter, making the centennial increase at ESK quite similar to other mid-latitude stations. Obviously, earlier long-term studies based on ESK IHV values are affected by the correction and need to be revised. These results also strongly suggest that the ESK yearbook data should be digitized and the hourly ESK data at WDC should be replaced by them.


Introduction
Long-term studies of geomagnetic activity often use the aa index (see, for example, Mayaud, 1980) because of its uniquely long time span.However, serious concern has recently been raised on the long-term consistency of the aa index (Svalgaard et al., 2003; Svalgaard et al., 2004; Jarvis,  2005; Lockwood et al., 2006 1 ).Moreover, there exists a fundamental problem in that the aa index (and many other indices) can not be exactly reproduced or properly verified at the present time, e.g., because the original early measurements do not exist in digital format.In order to have a more straightforward, homogeneous and easily verifiable measure of local geomagnetic activity, Svalgaard et al. (2004) introduced the so called IHV (Inter-Hour Variability) index.The IHV index was originally defined as an average of six absolute differences of successive hourly values of the magnetic H component between 19:00-01:00 local time (00:00-06:00 UT for CLH/FRD station).This definition was based on the fairly flat daily curve at the CLH/FRD station in this LT sector, and on the fact that this LT sector is geomagnetically very active.Note that, since the hourly values of the measured magnetic field available in digital format in the World Data Centers cover the whole observation interval for several stations, a homogeneous, long-term IHV index can be easily calculated and straightforwardly verified.
We have recently calculated the IHV indices for a number of stations (Mursula et al., 2004;Mursula and Martini, 2006) whose measurements cover most of the last century.We found that at all stations the local geomagnetic activity described by the IHV index depicts a very similar long-term pattern (rapid increase from early 1900 until 1960, then a dramatic dropout, and a weak increase thereafter), although the effective centennial trends vary significantly from one station to another.Recently, we noted (Mursula and Martini, 2006) that the centennial trends are strongly affected by the fact that many stations first registered hourly spot values in the beginning of the century before turning to register hourly means.Because the hourly spot values have more variability than hourly means, the IHV index, as a measure of variability, overestimates geomagnetic activity in the early years of the last century and, therefore, underestimates the centennial increase.We also calibrated this effect of changed sampling upon the IHV index and showed that the centennial trends are considerably enhanced.
While analyzing the effect of sampling change to the IHV index at the various stations, we found that the IHV index at the mid-latitude Eskdalemuir (ESK) station was unique in showing by far the largest centennial trend of all the seven stations studied.Moreover, we found that the ESK IHV index depicted a sizable, step-like increase from 1931 to 1932.Since the above mentioned sampling change decreases rather than increases the IHV index, this behavior clearly indicated a different problem.Also, because of the size of the step it is clear that the ESK IHV indices would remain inappropriate for any long-term studies without solving and correcting this problem.In this paper we examine and solve this problem in ESK data and ESK IHV index.We give a consistent explanation to the problem and correct the ESK IHV indices so that they will be homogeneous over the whole measurement interval.We also correct our earlier estimate (Mursula et al., 2004) about the centennial trend in the ESK IHV index and compare that with similar trends for other mid-latitude stations.

Stations, data and IHV indices
As mentioned above, we will mostly study here the ESK observatory data.We will also compare ESK IHV indices with the IHV indices from the same two mid-latitude stations (actually, one station and one station pair) that were included in our earlier studies (Mursula et al., 2004;Mursula and Martini, 2006).These stations have the longest records of magnetic observations since early 1900s, and their data is avail-able at hourly resolution in digital format.The codes, coordinates, local midnight UT hours, start years of observations and start years of hourly mean registration (as opposed to hourly spot values) of these stations are depicted in Table 1.
For each station we will calculate two series of IHV indices.The first, to be called the IHV-raw index, is calculated from the measured hourly values of the magnetic H component according to the original IHV recipe mentioned above.(We note in passing that, although the IHV definition was slightly changed from its original definition, using sums rather than averages, and midnight centering rather than 19:00-01:00 LT sector (cf.Svalgaard et al., 2004Svalgaard et al., , 2003)), we prefer to keep here to the original definition for consistency with our earlier results.Nevertheless, the differences between the results based on the original and new definition are small).This is the original IHV index introduced by Svalgaard et al. (2004), also called the IHV-raw index (Mursula et al., 2004;Mursula and Martini, 2006).However, as shown earlier (Mursula et al., 2004), even the IHV index is affected by the long-term variation of the daily curve which, therefore, has to be removed from it.This will be done as follows (for more details, see Mursula et al., 2004).We first calculate the yearly averaged daily curves for each station in order to obtain a proxy for the quiet-time daily variation in each year.Then we calculate the yearly quiet-time IHV value to be called IHV-q from these smooth yearly curves and form the corrected IHV-cor index by subtracting the yearly IHV-q values from the original daily IHV-raw index.

Problem in ESK data set
Calculating ratios of yearly IHV values between two stations we have found (Mursula and Martini, 2006) that such ratios depicted step-like changes of about 30% in certain years.We showed that these changes were due to the change of data sampling at one of the two stations from hourly spot values to hourly means.Because the IHV index is a measure of variability, changing from spot values (with larger variability) to hourly means will reduce the index at the respective station.The years when the sampling was changed at any station are indicated in Table 1.We also introduced a detailed procedure in order to correct the effect of the sampling change upon the Ann.Geophys., 24,[3411][3412][3413][3414][3415][3416][3417][3418][3419]2006 www IHV indices.Note that the correction of sampling change reduces the excessively large IHV values in the beginning of the last century and, thereby, increases the centennial change.
For some stations the change was quite dramatic.E.g., at the CLH/FRD station, the centennial increase changed from 6% to 24%.When calculating the above mentioned IHV values and ratios for the ESK observatory we were faced with a very different behaviour (Mursula and Martini, 2006).While all other stations depicted a roughly 30% decrease in the IHV index since the year of sampling change, the ESK data was found to have an increase of about 60% since 1932.We have depicted the ESK to NGK IHV ratio in Fig. 1. (Note that NGK changed to hourly mean sampling already in 1905 and was, therefore, used as the normalizing station in (Mursula and Martini, 2006).)On the other hand, no clear step downward was found in the ESK to NGK IHV ratio in 1918 when sampling was supposed to be changed at ESK. (We will discuss this later in more detail.) Exactly the same behaviour is found when calculating the ratio between ESK and any other station, indicating that the problem is indeed with the ESK data.Using yearly means before and after 1932 we find that there is a 57.4% increase in the ESK IHV index from the average level prior to 1932 to a higher level thereafter.Obviously, this increase can not be explained by the change of sampling from hourly spot to hourly mean values because this would decrease the variability of data.Rather, the variability of data has increased considerably in 1932.
In accordance with our procedure correcting the sampling change (Mursula and Martini, 2006), the observed 57.4% increase leads to a correction factor of 1/1.574=0.635.We call this the RF-raw (RF-cor in case of IHV-cor) factor (see also Table 2).This is the correction factor with which the early Table 2. Correction factors RF-raw and RF-cor for IHV-raw and IHV-cor from ESK/NGK ratios, CF-raw and CF-q for IHV-raw and IHV-q from the 2ra1h IHV / 1h IHV ratios in 1996 and 1999 and CF-cor from the average correction factor in 1911-1931 for IHV-cor.
RF-raw RF-cor CF-raw CF-raw CF-q CF-q CF-cor 1996 1999 1996 1999 0.635 0.498 0.624 0.646 0.824 0.963 0.536 ESK IHV-raw (IHV-cor, respectively) values should be corrected (divided) in order to take the increase in 1932 into account.However, this is only a technical solution of the problem and does not allow us to understand how the ESK data were really treated in the early years.Therefore, we have examined the recent ESK data in detail in order to understand how the early ESK data were processed, and in order to be able to reproduce the observed step in the ESK IHV index.
We will also implement a more accurate method to obtain calibration factors which take into account changes due to the varying level of solar activity.However, we will find that these more refined factors are quite close to the above mentioned rough estimates of correlation factors (RF-raw and RF-cor).

Solving the ESK data problem
According to the observatory yearbooks there was a change in the instrument orientation in 1932.The magnetometers which had been measuring the X and Y components of the horizontal magnetic field until the end of year 1931, were set to measure the H and D components, respectively, from 1932 onwards.In fact, Clilverd et al. (2005)  archival at the WDC.This possibility can be studied by comparing the originally observed data given in the observatory yearbooks and the WDC data for the same time.Figure 2 also shows that the two-hour averaging notably reduces the variability of the data.Therefore, the IHV indices calculated from the WDC data will be considerably lower in 1911-1931 than they should be according to the originally measured values documented in the yearbooks.We have also calculated the two-hour running averages of the yearbook data for a few days in the same month and depicted those in Fig. 2 together with the simultaneous WDC values, with all data now located on the exact hour.The two agree with each other to a high accuracy, thus further verifying that the WDC data are two-hour running averages of yearbook data.
The idea to use two-hour running averages is most likely related to the early practice of measuring hourly spot values, i.e., momentary values at every sharp hour.Some time later, after the original measurements, there was a need to have a homogeneous, long-term data base consisting of hourly means.Taking an average of two successive spot values could be considered as a rough proxy for an hourly mean for the hour starting at the first spot value.By this method, one could indeed obtain a data base where the time series of each component was fairly homogeneous over the whole lifetime of the station.However, as we will quantify more exactly later, the two-hour running averages of spot values have a slightly smaller variability than the true hourly means, thus affecting the IHV index.

Analysis using recent ESK data
In order to examine in more detail the effect upon the IHV index from the change in data sampling and from the twohour averaging, we have used the more recent ESK data that is sampled once every minute.Using these 1-min data and the hourly means based on them, we have calculated four sets of daily IHV indices.One set (to be called 1min IHV) uses hourly spot values, taking the first minute values each hour, the second set uses hourly means (1h IHV; this is the normal IHV index), the third set uses two-hour running averages of spot values (2ra1min IHV), and the fourth set uses two-hour running averages of hourly means (2ra1h IHV).All these indices were calculated for one year of low solar activity (1996) and high solar activity (1999) in order to be able to estimate the possible changes with solar activity.(This follows our earlier procedure, Mursula and Martini, 2006).
We have listed the yearly means of the four sets of IHV indices for the two years in Table 3.Let us first note on the ordering of the IHV values.The largest IHV indices, those based on the hourly spot values (1min IHV), are some 45% larger than the IHV values based on hourly means (1h IHV).This increase is quite close to the similar results found earlier for the other stations (Mursula and Martini, 2006), especially to the 40% increase at the NGK station.Note also that the 1h IHV values are some 18% larger than the 2ra1min IHV values.This verifies that variability is indeed smaller if one uses two-hour averaging of spot values as a substitute to obtain hourly means.The mathematical reason is that the twohour averages of spot values yield a time series whose data points are more dependent than the hourly means obtained from averaging 60 1-min data points each hour.Finally, of Ann.Geophys., 24,[3411][3412][3413][3414][3415][3416][3417][3418][3419]2006 www.ann-geophys.net/24/3411/2006/course, the smallest IHV values result from the two-hour averages of hourly means.
Although the absolute IHV values differ by about 10-20% between the two years included in the study (between low and high solar activity), the various ratios of two IHV yearly means remain roughly the same.E.g., the ratio between the 1min IHV and 1h IHV values is about 1.47 for 1996 and 1.45 for 1999.Also, the 1h IHV to 2ra1min IHV ratio is about 1.20 for 1996 and 1.16 for 1999.Accordingly, there is only a minor dependence on solar activity (of a few percent only) in such IHV ratios, in agreement with our earlier findings (Mursula and Martini, 2006).
The observed 57.4% increase in the ESK IHV index from 1931 to 1932 cannot be explained if the data before 1932 were two-hour averages of hourly samples because the increase would be only about 18%.However, Table 3 shows that the increase from 2ra1h IHV values to 1h IHV values is 60.2% for 1996 and 54.7% for 1999.Accordingly, the average increase of about 57.5% is almost exactly the same as the increase (57.4%) that was earlier found based on the ESK/NGK IHV ratio.This verifies that (at least most of) the WDC data before 1932 must have been hourly means that had been further averaged to two-hour running averages.

Analysis using ESK yearbooks
It is interesting to note the change of headings in the ESK yearbooks.In first years 1911-1913 the heading "Readings of the (e.g., north) component of terrestrial magnetic force at each hour of Greenwich mean time" was used, which changed to "Readings of the (e.g., north) component of terrestrial magnetic force for each hour of Greenwich mean time" in 1914-1917.A more significant change to "Mean  1911-1931. 1910 1920 1930 1940 1950 1960 1970 1980 1990  values for periods of 60 min centered at the hours of Greenwich mean time" occurred in 1918 which lasted until 1932 since when the heading read "Mean values for periods of 60 minutes ending at the hours of Greenwich mean time".The change from 1917 to 1918 could be understood so that the earlier records are hourly samples, and the later records hourly means.However, as implied by the earlier discussion, this interpretation is erroneous.In order to further study this question, we have digitized the yearbook data for Januaries 1911Januaries -1918Januaries and 1920Januaries -1932. .(The yearbook of 1919 was missing.Digitization of all yearbook data is under way.)We calculated the relative differences between the yearbook and www.ann-geophys.net/24/3411/2006/Ann.Geophys., 24,[3411][3412][3413][3414][3415][3416][3417][3418][3419]2006 WDC IHV-raw values.Figure 3 shows that in 1912-1931 the values fluctuate around the average value of 56% which is very close to the difference of 57% (see Table 3) which is the value of the difference if the yearbook data are hourly means and the WDC data are two-hourly running averages of hourly mean data.The difference in 1932 is zero, indicating that the two data sets are identical since that year.The ratio was largest in 1911, suggesting that the yearbook values in 1911 are hourly spot values.(However, the difference in 1911 is even larger than the expected ratio of about 72% between the hourly spot values and their two-hour running averages.This may be due to the fact that each data point in Fig. 3 only includes one month of data, leading to considerable scatter in the various yearly values, also in 1911.)This analysis further supports the conclusion that the above mentioned change in yearbook heading in 1918 does not mean that the observatory data before and after 1918 are differently sampled and that, most likely, hourly spot values were only registered in 1911.As explicitly noted in the headings in 1918-1931, the hourly means were taken as 60-min averages centered at sharp hours.This was probably chosen on the basis of the earlier practice of measuring spot values at sharp hours and in order to avoid the 30-min shift that would result from using hourly means centered at half-hours.On the other hand, most likely because of this centering on sharp hours, even the hourly mean data were averaged until 1931 by the same two-hour running average method (as earlier for spot values), in order to attain hourly means centered at half-hours.This, again, left the data before 1931 with less variability than plain hourly means.
We would also note that the 1911 yearbook was printed in 1912, i.e., immediately after the observations.However, 1912 observatory book was printed only in 1914.At this time the change in sampling method was widely discussed and most stations started measuring hourly means in 1915, while some had adopted the new practice already slightly earlier (1905earlier ( in NGK, 1914 in SOD; for more information, see Mursula and Martini, 2006).It is possible that, after the first yearbook was printed in 1912, all other yearbooks already included hourly means but the heading remained original, thus erroneous, until 1918.

Correcting the ESK IHV values
We can now correct the ESK IHV values for the fact that the IHV indices prior to 1932 are artificially low because twohour averaging.As correction factors one could use the RF factors introduced above.However, in order to take into account the (weak) dependence on solar activity, we use here correction factors that can be derived by analyzing the ESK data in recent years.
As above, we define the correction factors as the ratio between the annual 2ra1h IHV and 1h IHV values.Using the values depicted in factors for 1996 and 1999 to be 0.6242 and 0.6465 (CF-raw factors in Table 2).Note that their average is up to three digits the same as the above mentioned RF factor obtained from the ESK/NGK ratio.As in our earlier work (Mursula and Martini, 2006), we assume here that the (annual) correction factors are linearly dependent on the (annual) sunspot number.Figure 4 shows the dependence of correction factors on sunspot numbers for 1911-1931 .As mentioned above, the correction factors are only weakly dependent on solar activity.Using a constant correction factor would only lead to a minor (less than 5%) error in correction.Also, because of this reason, the assumption of a linear (as opposed to nonlinear) relationship between correlation factors and solar activity does not produce a notable error.We have depicted in Fig. 5 the ratio of annual IHV indices between ESK and NGK stations (see also Fig. 1), now using both corrected and uncorrected IHV ESK indices.One can see that the large step around 1932 depicted in Fig. 1 has disappeared.This verifies that the adopted correction is appropriate to make the ESK IHV indices a homogeneous long-term measure of geomagnetic activity.
As described above, we have corrected the whole time interval from 1911 to 1931 by the correction factors obtained from the ratio between 2ra1h IHV and 1h IHV values.This procedure applies only for those years when ESK observatory was measuring hourly means.For spot values, we should use correction factors which are obtained from the ratio between 2ra1min IHV and 1h IHV values.As can be seen in Table 3, those correction factors would be only 0.834 (in 1996) and 0.861 (in 1999) One can see that the two IHV series agree remarkably well with each other.This is particularly interesting for the early years 1911-1917 of claimed hourly spot sampling.Indeed, if we used the correction factors assuming hourly spot values, the corresponding ESK IHV curve would have to be lowered by 34%, as indicated in Fig. 6.Thus, clearly, the values in 1912-1917 can not have been hourly spot values, but hourly means.nT/year), while the slope of the corrected IHV-raw is only 0.0034, i.e., roughly one order of magnitude smaller.

Centennial evolution of ESK IHV
We have further quantified the centennial change by calculating, as earlier (Mursula et al., 2004;Mursula and Martini, 2006), the average values of the IHV-raw and IHV-cor indices for ESK and the two other mid-latitude stations of Table 1 during the last  and first  22 years of the previous century.Table 4 depicts the IHV values and their relative centennial increases for these stations.(Note that, because of different start years, the stations cover slightly different fractions of the first 22 years.Therefore, all the relative changes are not readily comparable).As one can see in Fig. 7 and Table 4, the correction of the IHVraw and IHV-cor indices for the two-hour averaging has a dramatic effect also on the centennial increase of geomagnetic activity based on the ESK IHV index.Before correction the centennial increase of the ESK IHV-raw (IHV-cor) index in 1912-2000 was as large as 73.9% (134.4%),i.e., clearly much larger than in any other two mid-latitude stations.After the correction, the centennial increase of the ESK IHVraw (IHV-cor) index is only 10.3% (25.3%).These numbers show dramatically how important the correction of the ESK IHV indices is, especially when using it in long-term studies.
We note that, e.g., Clilverd et al. (2005) used the uncorrected ESK IHV indices as one main data set in their analysis to study the long-term robustness of the aa index.It is expected, based on the results presented in this paper, that their analysis will be considerably modified after the ESK IHV indices are properly corrected for two-hour averaging.We have studied this question by reconstructing the aa index from its correlation with the ESK IHV index.Figure 8 shows the difference between the reconstructed and original aa indices using uncorrected and corrected ESK IHV indices.The two differences depict a very different pattern.While the differences for uncorrected ESK IHV show both positive www.ann-geophys.net/24/3411/2006/Ann.Geophys., 24,[3411][3412][3413][3414][3415][3416][3417][3418][3419]2006 old geomagnetic data.They also strongly suggest that the ESK yearbook data should be digitized and the hourly ESK data at WDC should be replaced by them.
Figure 2 presents an example of the hourly H components for one day (29 January 1911) from the 1911 yearbook and the WDC data.The WDC data is placed on the half hour (in the middle of the hour) while the yearbook values are located on the exact hour.The figure shows that the WDC values in 1911 are indeed averages of the two neighboring yearbook hourly values.

Fig. 3 .
Fig. 3. Relative differences between the IHV-raw values calculated from the yearbooks and WDC data for Januaries in 1911-1932 (yearbook data in 1919 were missing).

Fig. 5 .
Fig. 5. Ratio of the corrected annual IHV-raw values between ESK and NGK (thick line with dots).For comparison the uncorrected ratio is also shown (thin line with stars).The step from 1931 to 1932 is seen before but not after correction.

Figure 7
Figure 7 depicts the corrected and uncorrected yearly IHVraw and IHV-cor indices for ESK observatory in 1912-2000, together with their best fitting lines.(We omit 1911 in the subsequent long-term analysis because a separate correction factor would be needed for this year, as discussed above.)It is easy to see the dramatic effect of the correction upon the centennial trend.The slope of the best fitting line for the uncorrected IHV-raw index is about 0.030 (in units of

Table 1 .
Information on stations used.Magnetic coordinates are calculated using the IGRF 2000 model.MN hour indicates the local mid-night hour in UT, and HMS start stands for the year when hourly mean sampling started.At ESK HMS probably started in 1912.
Top: The ESK H component according to the observatory yearbook (line with black dots) and according to the WDC data base (line with circles) in 29 January 1911.The WDC data points are shifted by half an hour forward to demonstrate the fact that they are two-hour averages of the neighboring yearbook values.
dominates in H and the IHV indices calculated from X are quite close to those calculated from H.Since any additional averaging of the hourly values would strongly reduce variability in the data, we may suspect that the ESK data in the early years until 1931 were altered before www.ann-geophys.net/24/3411/2006/Ann.Geophys., 24, 3411-3419, 2006 Bottom: The hourly ESK H values according to the WDC data (big open circles), together with the two-hour running means of the yearbook data (line with small dots) for a few days in January, 1911.Coinciding dots and circles give further evidence that the WDC data are running averages of the yearbook data.

Table 3 .
Annual IHV-raw values for ESK observatory in 1996 and 1999 using four differently treated data sets.
Table 3 we can calculate the correction

Table 4 .
Mean IHV-raw and IHV-cor values for ESK (calculated  for 1912-2000)and the two other mid-latitude stations at the beginning (from start until 1922) and at the endof the last century, and their relative increases.