Revised Dst and the Epicycles of Magnetic Disturbance: 19582007,‖ Annales Geophysicae 27

Abstract. A revised version of the storm-time disturbance index Dst is calculated using hourly-mean magnetic-observatory data from four standard observatories and collected over the years 1958–2007. The calculation algorithm is a revision of that established by Sugiura et al., and which is now used by the Kyoto World Data Center for routine production of Dst. The most important new development is for the removal of solar-quiet variation. This is done through time and frequency-domain band-stop filtering – selectively removing specific Fourier terms approximating stationary periodic variation driven by the Earth's rotation, the Moon's orbit, the Earth's orbit around the Sun, and their mutual coupling. The resulting non-stationary disturbance time series are weighted by observatory-site geomagnetic latitude and then averaged together across longitudes to give what we call Dst5807-4SH. Comparisons are made with the standard Kyoto Dst. Various biases, especially for residual solar-quiet variation, are identified in the Kyoto Dst, and occasional storm-time errors in the Kyoto Dst are noted. Using Dst5807-4SH, storms are ranked for maximum storm-time intensity, and we show that storm-occurrence frequency follows a power-law distribution with an exponential cutoff. The epicycles of magnetic disturbance are explored: we (1) map low-latitude local-time disturbance asymmetry, (2) confirm the 27-day storm-recurrence phenomenon using autocorrelation, (3) investigate the coupled semi-annual-diurnal variation of magnetic activity and the proposed explanatory equinoctial and Russell-McPherron hypotheses, and (4) illustrate the well-known solar-cycle modulation of storm-occurrence likelihood. Since Dst5807-4SH is useful for a variety of space physics and solid-Earth applications, it is made freely available to the scientific community.


Introduction
The dynamic behavior of the Earth's magnetosphere is governed by the Sun and the solar wind (e.g.Cowley, 1995;Russell, 2000).Embedded in the solar wind is the interplanetary magnetic field, and with its connection onto the geomagnetic field, especially with the arrival at Earth of coronal-mass ejections, the field lines of the magnetosphere are progressively advected from the dayside magnetopause, across the noon-midnight meridian of the polar cap, and into the nightside magnetotail (e.g.Kamide, 1988;Hughes, 1995;Kennel, 1995).This process of magnetospheric convection can be highly time-dependent, giving disturbance to the geomagnetic field that we call a magnetic storm (e.g.Lui, 2000) prominently manifest in the magnetograms of ground-based observatories (e.g.McPherron, 1995).The most obvious storm-time signature in data from low-latitude magnetic observatories is a general reduction in the intensity of the horizontal component of the magnetic field (see the early papers by Broun, 1861;van Bemmelen, 1900;Chapman, 1927).This is usually inferred to be due to a magnetospheric electric current that encircles the Earth in the equatorial plane and which generally flows in the westward direction (e.g.Daglis et al., 1999), although it is also widely recognized that this is a simplified depiction of what is really only a part of a complex current continuum.The degree of diminution of the lowlatitude, horizontal intensity is a fundamental measure of the size of a magnetic storm (e.g.Gonzalez et al., 1994;Kozyra and Liemohn, 2003), and it may be expressed in terms of the energy content of an equivalent magnetospheric ring current (Dessler and Parker, 1959;Sckopke, 1966), An early and ambitious program for measuring globalscale storm-time magnetic disturbance was undertaken by Vestine et al. (1947).Their efforts were originally motivated by a need to avoid magnetically-disturbed data when mapping the main field, but digression was made to better describe magnetic disturbance for purposes of basic scientific understanding.Subsequently, and as part of the International Geophysical Year, Sugiura (1964) initiated the development of what is now the standard method for J. J. Love and J. L. Gannon: Revised D st and the epicycles of magnetic disturbance : 1958-2007 measuring low-latitude disturbance, the D st index.Over the years, the method for calculating D st has been revised (Sugiura and Hendricks, 1967;Sugiura and Poros, 1971;Sugiura and Kamei, 1991), but the basic philosophy has remained the same: Subtract a time-dependent quiescent baseline from low-latitude magnetic-observatory data, and average the remaining disturbance fields from several observatories.Unfortunately, estimating the quiet-time baseline is not easy, and this has motivated alternative proposals for measuring low-latitude disturbance (e.g.Kertz, 1964;Svalgaard and Cliver, 2007).Despite these difficulties, D st has become an almost indispensable diagnostic of space weather, and, as a service to the scientific community, the index is continuously calculated by the Kyoto World Data Center.There are also proposals for making the prediction of D st into a routine operational service, minutes or hours in advance of its realization (e.g.McPherron and O'Brien, 2001;Lundstedt et al., 2002;Temerin and Li, 2002).
Recently, and in something of an academic context, Karinen and Mursula (2005) constructed a long and continuous D st time series, one that overlaps with that initiated by Sugiura et al.Although they generally follow the standard calculation method, Karinen and Mursula also correct some obvious defects in the standard formulas.Retrospective analyses are important for continued progress, but numerous problems remain.It is well known, for example, that the Kyoto D st time series contains significant solar-quiet variation (e.g.Mayaud, 1980, Sect. 8.5), and the standard calculation method includes ad hoc treatments of the observatory data, some of which date back to a time when computers and numerical methods were not nearly as advanced as they are today.In light of all of this, it is clear that a complete re-evaluation of the standard method for D st calculation is worthwhile, examining whether or not the index has the properties it is supposed to have, and considering new algorithms for its routine production.
Here, we document the development of a new algorithm for extracting the disturbance field recorded in magneticobservatory time series.Of importance to our method is the conceptual distinction between the qualities of stationarity and non-stationarity as they apply to time series of finite length (e.g.Priestley, 1988;Kantz and Schreiber, 1997;Bendat and Piersol, 2000).Roughly speaking, if a time series is longer than all of the timescales characterizing the physical system being recorded, and if running measures, such as mean, variance, and spectral content, are constant over the duration of the time series, then the time series can be considered to be stationary.Otherwise, the time series is either non-stationary, or it contains non-stationary ingredients, or it is too short for stationary qualities to be well measured.The later applies to the secular variation of the Earth's magnetic field.When viewed over very long periods of time, hundreds of thousand of years or longer, secular variation might actually be stationary, but over the relatively brief historical time during which an observatory is operated, secular variation appears as a slowly-drifting non-stationary baseline ingredient in magnetic-field time series.
In the frequency domain the observatory data display a prominent set of stationary Fourier harmonics having periods equal to integer fractions of the Earth's rotational period, the Moon's orbital period, the Earth's orbital period, and the cross-coupling of harmonics with those periods.These harmonics dominate what is commonly described as solar-quiet variation.The same periodic terms are seen in ocean tidegauge data (e.g.Cartwright, 1999;Pugh, 2004).Although most of the underlying physics is different, the classical analysis of ocean tides can serve as inspiration for the analysis of solar-quiet magnetic variation.Working in the frequency domain, oceanographers use a process of band-pass and band-stop filtering to separate stationary tidal variation from non-stationary storm-surge variation.A modified treatment, one that accommodates the magnetic-storm bias towards diminished horizontal-field intensity, permits the separation of magnetic-observatory time series into stationary quiet variation and non-stationary disturbance variation.
Magnetic observatory time series display many types of non-stationary disturbance, some of which are semi-cyclical or semi-periodic in nature.For example, buried under the average deterministic evolution of many magnetic storms there is a local-time diurnal variation caused by the Earth's rotation beneath the active magnetosphere.This is seen in the time series of an individual observatory as a transient periodic signal that is only present during storms.Over longer timescales, there is tendency for a storm to occur 27 days after another storm, a phenomenon related to solar rotation that might be analyzed in terms of recurrence probability.A coupling of the well-known semi-annual modulation of stormoccurrence likelihood and the universal-time modulation of magnetic activity is thought to be related to the orientation of the geomagnetic dipole relative to the Sun, but a precise explanation is controversial.And, of course, with the waxing and waning of sunspots over the course of the 10 or 11-year solar-cycle, the likelihood of storm occurrence is also modulated.
In studying these and other disturbance signals, it is often assumed that the D st index has had solar-quiet variation tidily removed, a necessity for resolving some of the smallamplitude signals of magnetic disturbance.We investigate the validity of this assumption.We carefully extract disturbance time series from low-latitude magnetic-observatory data, and we use them to construct a new, continuous, and self-consistent D st time series that records a 50-year history of magnetic storms from 1958 to 2007.We document the statistics of storms, and we explore the superimposed epicycles of non-stationary global magnetic disturbance using a combination of the individual observatory disturbance time series and the revised D st time series.

Observatory data
In calculating D st , hourly-mean data from a loose longitudinal necklace of low-latitude observatories are used.In principle, data from many different observatories could be used.In his original analysis, Sugiura (1964) used data from 8 different stations.But these days, 4 observatories contribute data to the standard (Kyoto) D st index.This is sufficient to provide a reasonable longitudinal-average measure of low-latitude magnetic disturbance (e.g.Mendes et al., 2006).Ordered in longitude, the standard observatories are Hermanus (HER) South Africa, Kakioka (KAK) Japan, Honolulu (HON) Hawaii, and San Juan (SJG) Puerto Rico; see summary in Table 1; for reference, OBS = {HER, KAK, HON, SJG}. (1) These observatories are distinguished by the relative reliability of their operation, and, today, by the promptness with which their data are reported.The D st index routinely produced by the Kyoto World Data Center is a one-hour index that is continuous from 1957, the commencement of the IGY.Magnetic observatories are specially designed and carefully operated facilities that provide accurate data over long periods of time (e.g.Jankowski and Sucksdorff, 1996;Love, 2007).Since observatories started operating in the 19th century, their data have been acquired by different means, defined according to different conventions, and reported in different formats.Fortunately, for OBS and for the years considered here , the quality and organization of the data are very high.Prior to about the 1980s, variational time series from OBS were collected by analog-photographic systems, but since then, variational time series have been collected using digital systems.In parallel with continuouslyoperating data acquisition systems, additional measurements needed for calibration are made from a reference pier on each observatory site.Through processing, the variational data are combined with the calibration data to give time series that have long-term stability and accuracy, usually much better than 5 nT.In the course of an observatory's operation, if procedures are changed or if the reference pier is moved, then this shows up in the time series as an abrupt offset.In most cases, these operational changes are documented, and large offsets are obvious upon inspection of the time series.Still, researchers must always be cognizant of these issues, especially when long observatory time series are being analyzed.
The older hourly-mean observatory data were obtained from visual measurement of the analog records; the modern hourly-mean data were formally constructed from 1-min digital data.The two averaging methods yield data of comparable quality.We obtained most of the data from the World Data Centers in Copenhagen (now Edinburgh) and Kyoto; but some of the most recent data were obtained from the Intermagnet organization's archive.The data are reported in either Cartesian components (X north, Y east, Z down) or horizontal-polar components (H horizontal intensity, D declination, Z down).Conversion between the two coordinate systems is simple.Time stamps have been consistently assigned on the universal-time half hour (00:30, 01:30, etc.).
Yearbooks provide valuable records of observatory operation and other types of metadata.For all of the years 1958-2007 the Hermanus and Kakioka observatories have published yearbooks.Unfortunately, this practice was discontinued for the American observatories in the 1960s (somewhat affecting our use of data from Honolulu and San Juan).When necessary (and when they exist), we have consulted them in order to better understand the data, especially when we have suspected a problem with the time series.
The horizontal-intensity component of the observatory time series is used in calculating D st .We represent each individual datum as H i and its corresponding time stamp as t i .With perfect continuity, for 50 years of time there would be N d =438288 data per observatory.For OBS the percentages of missing data are small, Table 1; there are literally no missing data from the KAK time series.Different levels of continuity are reflective of the levels of support that different observatory agencies can afford for each observatory, primarily in terms of on-site personnel working hours and the maintenance of redundant acquisition systems (if there are any).Through inspection we found several obviously erroneous data; we are aware of a total of 6 documented pier offsets; we identified 2 very small offsets for HON of unknown origin and 2 rather serious offsets for SJG that appear www.ann-geophys.net/27/3101/2009/Ann.Geophys., 27,2009 to have been accidentally introduced while the data were in archive, after submission by the US Geological Survey.

Observatory disturbance time series
Our method for calculating D st is physically motivated, conceptually simple, and consistent with the sentiment outlined by Sugiura et al. and subsequently documented by Mayaud (1980).The primary difference between our method and the standard method is that our removal of solar-quiet variation from the observatory time series involves a layered filtering process in both the time and frequency domains, while the filtering method Sugiura et al. is essentially a time-domain procedure.In this section we describe how we extract a disturbance time series for each of OBS, and we examine the properties of the time series.

The many signals in observatory time series
It is important to recognize that the geomagnetic field measured at an observatory is generated by source electric currents that are both within the Earth and above the Earth's surface.They all contribute to a superposition that is the magnetic field measured at an observatory.Although the internal-source/external-source divide is conventional, it is also somewhat artificial.So, for example, telluric currents in the lithosphere and mantle are induced by magnetic-field variations having their source in the ionosphere and magnetosphere.Therefore, internal and external time series are partially coupled together.Still, it is useful to adopt a simple vocabulary that describes, as much as possible, the various phenomena of interest.Toward that end, we assume that magnetic-observatory data record a combination of signals that can be described by the following sum: (2) compare with Sugiura (1964, his Eq. 1).C is permanent crustal magnetization (e.g.Purucker and Whaler, 2007); SV is the main field and its secular variation generated by the dynamo in the Earth's core (e.g.Jackson and Finlay, 2007); Sq is solar-quiet variation that has its primary source in ionospheric electric currents (e.g.Campbell, 1989), but where magnetospheric and induced telluric currents contribute as well (e.g.Constable, 2007); and SC is any long-term, cyclical or secular variation associated with the solar-cycle (e.g.Clúa de Gonzalez et al., 1993;Macmillan and Droujinina, 2007).The disturbance time series Dist is dominated by magnetic storms, and it is the focus of our analysis.In terms of mathematical adjectives, it is transient and non-stationary magnetic variation occurring intermittently over timescales ranging from hours to days and possibly out to weeks.Dist is very distinct from the long-term decadal SV and SC and shorterterm, but stationary, harmonic variation Sq.For OBS, the primary source of disturbance is the magnetospheric ring current.Observatories that are situated either on or very close to the magnetic equator, or observatories situated underneath or close to the auroral zone, record disturbance that is dominated by ionospheric currents.For this reason, observatories used in D st studies must be on magnetic latitudes that are neither too low nor too high.
For all observatories, however, the local disturbance field has, as part of its source, internal telluric currents that are sustained by external field variation.It is well known that induced magnetic fields are most prominently manifest in the vertical magnetic-vector component (e.g.Parkinson, 1983, his Fig. 104), with a minor proportion manifest in the horizontal component used for D st calculation.Häkkinen et al. (2002) estimate that telluric induction contributes about 25% of D st .This might be a concern to researchers intent on using D st for detailed analyses of the evolution of individual magnetic storms.In our analysis, starting from Sect.6.2, we distill results from the recording of many storms, isolating signals that are either an average of all the data or common to all the data.That this is possible might be reflective of a certain incoherence in the induced telluric signals -perhaps it represents noise superimposed on the ring-current signal.These are, of course, the difficulties of reality, but the standard formulation of D st does not make an explicit separation of the telluric signal from the magnetospheric signal (Sugiura, 1964, p. 44), and, so, neither do we.As the reader might imagine, there is abundant opportunity for further refinement of the formulation of D st .

Estimating the internal-field time series
In detail, consider, first, the constant and most slowlyvarying parts of the observatory time series.The dynamo in the Earth's core generates the geomagnetic main field, and secular variation of the main field is the result of convective fluid motion in the core.This is seen in observatory data as a slow drift in the magnetic vector over periods of decades (e.g.Courtillot and Le Mouël, 1988).In restricting ourselves to time series from just a few observatories, it is impossible to distinguish a permanent, and therefore constant, crustal field from the time-averaged part of the main field.Therefore, we treat the core and permanent-crustal fields together as a total internal field, (3) Since the internal-field time series constitutes a baseline about which the external field varies, in estimating I (t) it is sensible to use a subset of the observatory time series for which external-field variation is subdued.For this, the International Quiet Days (e.g.Joselyn, 1989), as identified by the GeoForschungsZentrum in Potsdam, are often used.We have found it advantageous to use our own algorithm.We select quiet days using a 24-h sliding window, within which we measure the average of absolute hour-to-hour differences, where the quantity is not calculated if more than half of the data are missing.For each month, the five smallest δH i values determine the 5 quietest days, which we note do not necessarily correspond to whole Universal-Time days, nor do they necessarily correspond exactly to International Quiet Days, although there is often significant overlap.
We approximate the internal-field time series at each observatory by Chebyshev polynomials of the first kind, T k (t).We make this choice because of the rapid rate with which Chebyshev expansions converge when approximating smooth functions (e.g.Conte and de Boor, 1980).Thus, for each observatory and for the entire period from 1958 to 2007, the internally-sustained signal is modeled as where the coefficients c k are determined by a least-squares fitting to the observatory data recording the 5 quietest days of each month.We have chosen a truncation level of k≤10, which we have found through experiment gives a good fit to the long-term secular variation at each observatory; results are not, however, particularly sensitive to this choice.In estimating the secular variation using the traditional method of Sugiura et al., quadratic polynomials are fitted to a 5-year sliding window of data.This is, in principle, a reasonable approach, but the total number of parameters that would de-scribe 50 years of secular variation far exceeds the 10 we use.
Because we seek a parsimonious description of the data, we prefer our method.In Fig. 1 we show I (t) fitted to HER data.

Examination of external-field variation
The difference between the data and our model of the internally-sustained time series gives us an estimate of the externally-sustained time series, In Figs.
2 and 3 we show E for quiet and stormy periods in 2003.In separating the various time-series ingredients, we have found it convenient and computationally efficient to work in both the time and frequency domains.Transformation back and forth between the two domains requires that that the observatory data be continuous, without any gaps.
But actual time series usually have gaps, because data are missing.We will sometimes introduce gaps when we intentionally remove segments of data we identify as corresponding to magnetically active periods.In either case, it is important that the filling of gaps with interpolated values be done in such a was as to cause minimal change to estimates of the spectral content of the remaining data.We use simple interpolation to fill short data gaps of one hour (isolated losses), according to For longer data gaps, in order to preserve some semblance of quiet-time diurnal variation, for each missing hourly value we interpolate across days.So, for example, to fill in the i th datum, we find the last (first) day without missing data m (n) number of days preceding (following) the day with missing data.We then interpolate between data corresponding to the same time-of-day according to the formula On its own, this formula works reasonably well for interpolating across long gaps, but there is sometimes a slight discontinuity at the end points of the interpolated time segment.
To correct this, we apply a linear transformation to Eq. ( 8)tilting the interpolated segment so that it smoothly matches the data on either side of the gap.An example of interpolation over a long data gap is shown in Fig. 2d.
After interpolating over data gaps, we apply a fast-Fourier transform ("realft", Press et al., 1992) to the discrete time series.This is done to 2 N f inputted data, where N d <2 N f , and where zero-value padding is used for the difference 2 N f −N d .The Fourier transform is represented as where each pair of Fourier coefficients (e ι , e ι+1 ), having indices (1, 2), (3, 4), (5, 6), ..., correspond to sine and cosine functions with discrete frequencies of (0), (24/N f ), (48/N f ), ... (cycles/day).We calculate power spectra by adding the square of each pair of Fourier coefficients.Obviously, the inverse Fourier transformation, brings the data back into the time domain.
Close inspection of the observatory power spectra reveals the intricate nature of stationary solar-quiet variation.In Fig. 4a we show a panoramic view of the spectrum for HON data.The diurnal harmonics with periods of 1 day, 1/2 day, 1/3 day, etc., are clearly identified as spectral peaks, as are the harmonics of annual and semi-annual variation.But the diurnal spectral peaks are split.In Fig. 4b we see that the daily peak is surrounded by a structured comb of neighboring spectral peaks.Similar observations apply to the half-day peak, Fig. 4c, and other diurnal peaks as well.
What causes this spectral-line splitting?The answer is related to the combination of periodic forces that sustain solarquiet variation.First, consider the driving forces separately.Differential heating of the atmosphere drives high-altitude winds that sustain the ionospheric dynamo.Diurnal driving of day-night differential heating is caused by the rotation of the Earth under the Sun, and north-south differential heating is caused by the tilt (obliquity) of the Earth's rotational axis and the annual orbit of the Earth around the Sun.
Therefore, solar-quiet variation, which is dominated by ionospheric dynamo action, will contain Fourier harmonics with periods corresponding to the solar day (d, 1.00...d) and the solar tropical year (a, 365.24...d).Differential solar irradiance causes differential ionization, controlling the efficiency of the ionospheric dynamo and affecting the amplitudes of the diurnal (d) and annual (a) harmonics.Gravitational atmospheric tides have harmonics corresponding to the lunar synodic month (m, 29.53...d), as well as diurnal (d) and annual (a) harmonics (e.g.Fejer, 1964).There are a host of other contributing factors, including the tilt of the ring current relative to the Earth's rotational axis (d) (e.g.Olsen, 1996) and the tilt of the ring current relative to the ecliptic (a) (e.g.Malin and Is ¸ikara, 1976).It is important to recognize, however, that drivers having identical periods contribute to a superposition of harmonic variation.Without some auxiliary source of information, it is impossible to untangle their sum.
But it does not stop there.The driving forces of solar-quiet variation are all coupled together.So, for example, while the rotation of the Earth under the Sun drives diurnal variation, the amplitude of this variation is modulated as the Earth orbits the Sun.This modulation ties together all the diurnal (d) and annual (a) harmonics.Similarly, the amplitude of diurnal variation is modulated by the orbit of the Moon around the Earth, tying all diurnal (d) and monthly (m) harmonics.And, as might be expected, the amplitude of the monthly modulated diurnal variation is itself modulated by the orbit of the Earth around the Sun, tying all diurnal (d), monthly (m), and annual (a) harmonics.Therefore, solar-quiet variation can be represented by a three-dimensional Fourier series of the form (11) Expansion ( 11) can be rewritten in the form (Bracewell, 1978, modulation theorem, p. 108).This demonstrates that cross-harmonic coupling between the drivers of solar-quiet variation results in a multitude of discrete harmonics (d, m, a, d+m, d+a, d+m+a) that can be resolved in terms of a one-dimensional Fourier series.It is these discrete harmonics, each of identifiable period, that are revealed in Fig. 4 (see also De Meyer, 1980;Olsen, 1997).With all of this cross-coupling, the solar-quiet time series only appears to be complicated.In fact, the multi-harmonic content of solar-quiet variation has a rather compact set of explanations.
For the longest periods shown in Fig. 4a, at ∼100 days there is a leveling out of the power spectrum, and for periods >1000 days there is an enhancement of spectral energy.Some of this long-period energy could be residual unmodeled secular variation, and some of it might be a vague hint of www.ann-geophys.net/27/3101/2009/Ann.Geophys., 27, 3101-3131, 2009 the ∼10.5 year solar cycle, but where discrete harmonics are not well resolved.There would be merit to pursuing an analysis covering a period of time that is considerably longer than that considered here.This might reveal harmonic cross coupling between solar-cycle harmonics (s) and the diurnal (d), monthly (m), and annual (a) harmonics -with solar-cycle modulation of solar-quiet variation (Ellis, 1880;Moos, 1910;Chapman and Bartels, 1962, Ch. 7.3).And since solar-quiet variation at an observatory is a function of the global form of the main geomagnetic field, future analysis might even accommodate the coupling of SV and Sq.For now, however, we use the simpler representations given by Eqs. ( 11) and ( 12).
see Sugiura and Kamei (1991, their Eq. 3).Without stated evidence, Sugiura asserts that the monthly (m) harmonics are negligible.Harmonic decompositions involving diurnal and monthly harmonics were used much earlier by Moos (1910), and they are discussed in the classic book by Chapman and Bartels (1962, Ch. 23.6), among other places.Of course, compared to some of these earlier studies, today, with modern computers, we can analyze long observatory time series, either in whole or in part, and having many constituent harmonics.
With respect to the external-field time series E, then, we identify all stationary periodic variation driven by the Earth's rotation, the Moon's orbit, and the Earth's orbit as being solar-quiet variation.Conversely, we identify disturbance Dist and solar-cycle related phenomena SC as any temporary deviation from this stationary periodic variation; see Eq. ( 6).Making a distinction in the time series between the Dist and SC ingredients is straightforward -the differences of timescale are considerable.The solar-cycle's modulation of quiet variation and the solar-cycle's modulation of stormoccurrence probability each occur gradually over the course of ∼10.5 years or so.On the other hand, magnetic disturbance is dominated by storms, and the evolution of each individual storm occurs over the much shorter timescales of hours and days.This is true, regardless of when or how often they occur.In other words, there is very little overlap in the spectral content of storms and the periods over which storm occurrence is modulated.There are a number of subtle issues here, some of which are related to the different qualities of stationary and non-stationary time series.We will return to them when we examine the spectral properties of our D st time series and when we map the epicycle modulation of storm-time disturbance.

Solar-quiet, solar-cycle, and disturbance variation
To extract a disturbance time series from observatory data we first construct a model time series of solar-quiet variation.This involves a layered set of data manipulations in the time and frequency domains.But first, recall that storms are characterized by reductions in the horizontal-field intensity measured at low-latitude observatories.Therefore, even though we have not yet removed solar-quiet and solar-cycle variation from the external-field time series, E, we can still use it to identify periods of time corresponding to large magnetic storms.It is helpful to remove the most active periods from the external-field time series.This is done for each individual observatory time series by first ranking the hourly −E i values.Starting with the largest value, which we denote as occurring at time t i , we open up a window of time that begins (ends) at least 12 h before (after) t i -the duration of the time window is at least 25 h in length, and its actual duration determined by a simple threshold criterion for maximum |E| within a sliding 12-h span of time.This enables us to define an active duration the commences before (ends after) maximum −E i .We then remove all points from this identified active duration and substitute interpolated values according to Eq. ( 8).The remaining −E i values are then ranked again, and the process of disturbance identification and interpolation is repeated until a termination threshold is reached.The resulting disturbance-interpolated time series Q has had active periods removed, and it is, therefore, close to being a quiet time series.The identification of active periods and filling them in with interpolated values is illustrated qualitatively in Figs. 2 and 3.
Next, we filter Q to obtain a more accurate representation of solar-quiet variation.This is done in the frequency domain, and so we apply a Fourier transformation, We band-pass filter B the coefficients q ι that correspond to narrow windows centered on each of the frequencies applying the filtering to a finite number of harmonic terms in the expansions given by Eqs. ( 11) and ( 12); see Fig. 4. For example, uncoupled diurnal terms (d) are kept out to degree 12, only one monthly term (m) is kept, of degree 2, uncoupled annual terms (a) are kept out to degree 7, etc.We adjust the width of the passing windows for each harmonic term so that, roughly speaking, the widest windows correspond to those frequencies having the greatest power.This accommodation is necessary because of normal leakage between adjacent frequency bins that results from the numerical application of the fast-Fourier transformation and because of spectral-peak broadening due to harmonic coupling with the unmodeled solar-cycle.We tune truncation levels and window widths iteratively, using before-and-after comparisons of the filtered and unfiltered power spectra and time series.As with just about any nontrivial treatment of data, it is obvious that there is some subjectivity to this process.We have sought to minimize the spectral content of the solar-quiet variation without excessive filtering.This is consistent with our taste for minimal manipulation of the data, but it also means that a small amount of solar-quiet variation might contaminate the disturbance time series of each observatory.Of course, the qualities of the final time series and power spectra will demonstrate the success (or failure) of our work.
Our treatment for what we collectively call solar-cycle variation is comparatively straightforward.We low-pass filter the disturbance-interpolated coefficients, keeping all coefficients corresponding to frequencies below 1/3 yr −1 , where H denotes the Heaviside step function.Inverse-Fourier transformation back to the time domain, gives us the sought-after model solar-quiet and solar-cycle time series, and the disturbance time series is obtained by subtraction in the time domain, Example segments of the solar-quiet and disturbance time series are shown in Figs. 2 and 3.
In contrast to our method for estimating Sq is the standard method used for calculating the Kyoto D st 1 .Summarizing Sugiura and Kamei (1991): For each observatory and for each month, data from a five-day quota of quiet days are identified.Since these days might occur during the gradual recovery phase of a storm, a linear trend is subtracted from each day's data.The data for all five quiet days are then averaged.A two-dimensional Fourier series, Eq. ( 13), is fitted to twelve months of these averaged quiet data.This fit serves as the estimated Sq, and it is subtracted from the time series of each observatory.Karinen and Mursula (2006) have studied the details of the standard method for estimating Sq and have found that it results in semi-annual biases of up to 12 nT.From our standpoint, we find the standard method to be cumbersome, what with the necessary detrending of each quiet day.We are also concerned that the method might not have been developed with checks on the Fourier content of the resulting disturbance time series.We will return to this issue in Sect.4, where we examine the power spectra for both the Kyoto D st and our new version of the index. 1 The current standard method for estimating D st (Sugiura and Kamei, 1991) is an adjustment of a method given by Sugiura (1964).

Examination of local magnetic disturbance
We begin our examination of the disturbance time series at each observatory in the frequency domain.In Fig. 4 we compare the HON power spectrum for the disturbance Dist time series with that for external-field time series E and the disturbance-interpolated time series Q. Note, especially, the significant reduction of the diurnal (d) spectral peaks by 3 or 4 orders of magnitude in the Dist time series.Other spectral peaks, such as annual harmonics (a), and, even, prominent peaks corresponding to annual modulation of lunar modulation of diurnal variation (d+m+a), are either neatly removed or significantly reduced.There is, however, a small forest of spectral peaks having periods of about a month and half a month that do not accurately correspond to lunar harmonics (m).These are caused by localized active regions on the Sun's surface and semi-persistent emission of high-speed streams of solar plasma -together they modulate geomagnetic activity.Since the Sun is not a solid body, it does not have a discrete rotational frequency (Howard, 1984), and so many peaks corresponding to periods of approximately one month are seen in the geomagnetic spectrum (Roberts, 1984).As far as we are concerned, this type of geomagnetic activity is not solar-quiet variation -it is disturbance -and we have not removed it from the Dist time series.
Turning now to the time domain, example segments of the Sq time series are shown in Figs. 2 and 3; note the slowly evolving form of the modeled solar-quiet variation and its unique character for each of OBS.Note also the generally successful removal of Sq from the Dist time series at each observatory.Details are of interest.In Fig. 2 at about days 259-262 and especially in the HON data (c) a prominent example of transient diurnal variation is seen.We believe that this is caused by a temporary enhancement of solar activity that has not, itself, initiated a storm.As the Earth rotates under the Sun, then, there is a perturbation to the ionization of the ionosphere, resulting in a short-lived change in the amplitude of diurnal variation.Undoubtedly, some researchers will prefer to classify this as solar-quiet variation.We assert, however, that such a sentiment would reflect a lack of clarity on what, exactly, solar-quiet variation is and what it is not.Again, we designate disturbance as any temporary non-stationary departure from the stationary solar-quiet variation.That can include a wide variety of magnetic activity, both in terms of magnitude and type, and it can be the result of numerous causes (Sugiura, 1964, p. 9;Mayaud, 1980, p. 119).
We normalize the disturbance field from each observatory n by its magnetic latitude λ n B , This transforms the local horizontal intensity into an equivalent equatorial intensity under the assumption of a uniform planar-current source.Over the 50 years considered here, the   23), for some large magnetic storms.Note the prominent diurnal variation in D st and abrupt offsets during storms.magnetic latitude of each observatory, as defined by the geomagnetic dipole, changes, but not very much: For Hermanus the magnetic latitude in 1958 was −33.38 • and in 2007 it was −33.99 • .So, accounting for this secular variation is not our highest priority.We do it because we can.We use a set of standard main-field models for estimating magnetic latitude for each observatory over time: gufm (Jackson et al., 2000) and IGRF (Macmillan and Maus, 2005).In Table 1 we summarize the statistics of the disturbance time series for each of OBS.Means µ of the Dlat time series are small and negative.This reflects the fact that each disturbance time series records extended periods of quiescence that are only occasionally punctuated by a storm.The means (standard deviations σ ) of the weighted disturbance fields are consistent to within about 1.00 (0.79) nT, less than the 1.00 nT resolution of the data.

Revised D st and the Kyoto standard
Having obtained the disturbance time series for each observatory, we calculate D st by averaging over longitude for each moment in time, summation in n is understood to be over the N=4 of OBS, and where we make appropriate accommodation for occasional missing values coming during data gaps.It is worth mentioning that in the original formulation for D st , Sugiura (1964, p. 13) used the following weighting of observatory disturbance fields: Upon seeing this equation, one might reasonably ask: Why should the observatory magnetic latitudes be averaged before taking the cosine?Subsequently, Sugiura and Kamei (1991, their Eq.5) changed the weighting to and this remains the weighting used today by the Kyoto World Data Center.But upon seeing this equation, one might reasonably ask: Why should the weighting for observatory magnetic latitudes be averaged separately from the observatory disturbance fields?Mursula et al. (2008) have analyzed the effects these unusual normalizations have on D st , and so we do not pursue the matter any further.We use the normalization given in Eq. ( 20).Henceforth, we refer to our particular version of the storm-time disturbance index as D 5807-4SH st , where 58 and 07 denote the beginning and end years of the model, and 4SH denotes the usage of the four standard observatories OBS and hourly data.
Comparison with the standard Kyoto D st is essential.In Fig. 5 we present a panoramic view of the two indices for 1958-2007 and in Fig. 6 we present more detailed views for 40-day time segments.In many respects the two indices appear to be very similar, the Pearson correlation coefficient is 0.9596, and they both show the expected modulation of storm-occurrence probability driven by the solar cycle.Close inspection, however, reveals important differences, and these can be clearly seen in the difference The Kyoto index is, on average, 8.60 nT lower than D 5807-4SH st , and the root-mean-square difference is 11.01 nT.In Fig. 5 we see that the D st time series has significant semi-annual variation, and in Fig. 6 we see that it has significant diurnal variation.
To better understand the origin of these differences, in Fig. 7 we compare the spectral power of the two D st time series.Here we see, as expected, that global magnetic disturbance has a broad range of Fourier components (e.g.Klimas et al., 2000;Balasis et al., 2006).But we also see that the Kyoto D st has prominent energy in the harmonics of solarquiet variation (e.g.Takalo et al., 1995) -something that is virtually non-existent in D 5807-4SH st .The semi-annual and diurnal signals in the Kyoto D st have been identified by others (Cliver et al., 2001;Saroso et al., 1993), and explanations have been proposed for their presence (e.g.Takalo and Mursula, 2001;Mursula et al., 2008).We believe that the actual explanation is very mundane: stationary solar-quiet variation has not been effectively subtracted from the disturbance time series during calculation of the Kyoto D st .Some confirmation of this assertion is obtained from Fig. 8, where we plot average spectra from the ten quietest and ten most active years, each defined in terms of the root-mean-square of D st over the duration 1958-2007.Note that the Kyoto D st spectrum for the quiet years shows prominent power in the diurnal harmonics while the quiet-years spectrum for D 5807-4SH st does not.
But what is perhaps even more worrisome are differences between D 5807-4SH st and the Kyoto D st that occur during some magnetic storms.In Fig. 5 storm-time differences appear as spikes in the D st time series.Figs. 6 shows these differences in detail.For the storm of February 1986 (b, days 38 and 39) significant differences of up to about 70 nT persist for the entirety of the storm's main phase.For the great storm of March 1989 (c, days 72 and 73), there are two abrupt offsets, each of about 70 nT, and each coming on top of a long-term negative bias.Oddly enough, for other storms (not shown) there is much less difference between the two indices.We have no explanation for this mixture of differences.

Storm statistics
Having developed a revised version of D st , we next investigate some its implications.How does the overall statistical record of magnetic storms recorded by D 5807-4SH st compare with the Kyoto D st ?

Storm size ranking
For some applications a relative ranking of historical storm sizes is needed.In Table 2 we report the 30 largest storms for 1958-2007, ranked by maximum negative D st , denoted as −D st M for each storm.The great storm of March 1989 (e.g.Allen et al., 1989) is at the top of the list: for D 5807-4SH st (Kyoto) the storm-time maximum intensity −D st M is 574 nT (589 nT).The relative ranking of other storms depends, to some extent, on which D st index is used.For example, the February 1986 storm is the 23nd (15th) largest of the past 50 years, with a −D st M of 278 (307) nT.

Storm size versus occurrence frequency
For many physical phenomena, the probability of the occurrence of an event is a power-law function of event size (e.g.Newman, 2005) -a prominent geophysical example is the distribution of the number of earthquakes versus earthquake moment (e.g.Turcotte, 1997).A power-law distribution is scale-invariant, making it reasonable to extrapolate estimates of probability across a range of event sizes.For the compilation of storm-time maximum intensities −D st M , we show in Fig. 9a histograms of the occurrence-rate number density and in (b) the corresponding cumulative frequency versus  24) and ( 25) and the parameters in Table 3. rank -for both D 5807-4SH st and the Kyoto D st indices.If a simple power-law scaling existed here, then the data should follow straight lines when plotted against log-log axes.That they do not tells us that no simple power-law scaling exists for −D st M occurrence statistics.In other words, storm-size probability is not scale-invariant.
This could have been expected.Because there exists a practical upper bound on storm size (Vasyliūnas, 2001), the occurrence frequency of very large magnetic storms is limited by what is physically possible.In such circumstances, an appropriate model consists of a power-law that is modified by a high −D st M exponential cut-off (e.g.Clauset et al., 2009):  24) and ( 25) and Fig. 22.
Eqs. ( 24) and ( 25) where A is amplitude; −D N is a normalizing factor: α is the scaling exponent; and −D L is a limiting factor that when small reduces the probable occurrence of very large storms.We choose to consider storms for which −D st M >−D N =33 nT, in which case Eq. ( 24) has only three independent parameters: (A, α, D L ).The corresponding cumulative frequency of exceedance, measuring the expected occurrence rate of storms with peak intensity greater than Here, the incomplete gamma function (e.g.Spanier and Oldham, 1987)  that can accept 1−α<−1 ("gcf", Press et al., 1992).Fits are shown in Fig. 9 and parameters are summarized in Table 3. Equation ( 25) conveniently summarizes the expected occurrence rate of storm-time maximum intensity.Examples for different exceedance levels are given in Table 3. Obviously, interest will tend towards the occurrence rate for the largest storms, like that of September 1859, the largest storm ever recorded by magnetic observatories: −D st M ≈1600 nT (Tsurutani et al., 2002).Extrapolating our results, the expected occurrence rate for a storm exceeding this size is 7 or 8 per million years.This is not very frequent, but is it a reasonable estimate?Without the compilation of D st statistics from a longer time span, it is difficult to say.But one thing is for certain: the storm of September 1859 was a very rare event, perhaps more so than has been generally appreciated.

The epicycles of magnetic disturbance
Next, we investigate some of the prominent semi-cyclical and semi-periodic signals of non-stationary magnetic disturbance.Some of these are well-established in the published literature, and so it is natural to wonder how they are revealed in D 5807-4SH st and the individual observatory disturbance time series.

Site-to-site similarities and differences
We begin this section with a confirmation of the global nature of storm-time disturbance and checking for any systematic site-specific anomalies.In Fig. 10a-f we show siteto-site correlation of disturbance by plotting data from one observatory versus data from another.The general consistency, measured in terms of positive correlation coefficients, confirms that most of each observatory's disturbance field is recording a global phenomenon, something that can be represented in terms of an equivalent magnetospheric ring current.Some, and possibly most, of the dispersion here is due to storm-time asymmetry in the ring current.But if each observatory was actually providing an unbiased measure of the ring current, then we might expect a more consistent degree of dispersion among the plotted pairs of observatories; instead, there are visually-obvious differences.The correlation is highest between the KAK and HON disturbance time series (correlation coefficient: 8923) and lowest between the KAK and SJG time series (0.7799).This difference might be due to induced telluric currents in the lithosphere and mantle.The complexity of near-surface geology has a correspondingly complicated electrical conductivity (e.g.Jones, 1992).Therefore, telluric currents induced by rapid external magnetic-field variation can give induced magnetic disturbance fields with measurable site-to-site differences.Magnetotelluric studies usually involve detailed analyses of fullvector magnetic data in the frequency domain (e.g. Simpson and Bahr, 2005), including measurement of electric currents.Both are beyond the scope of the analysis presented here.
Another site-to-site comparison can be made of the anomalous disturbance time series, something we define in terms of the deviation from D st , In Fig. 10g-l we show site-to-site correlation of the anomalous disturbance time series by plotting data from one observatory versus another.Interestingly, in some cases there is obvious anti-correlation.We do not necessarily expect a high degree of anti-correlation, but it is noteworthy that it is greatest between the observatory pairs of HER-HON (−0.3769) and KAK-SJG (−0.6625) -pairs that are on almost opposite geographic longitudes: HER-HON: (19.22 • , 202.00 • ), KAK-SJG: (140.19 • , 293.85 • ).The relationship revealed here is one of spatial orthogonality.What is its origin?To help answer this question, in the next section we construct a local-time map of magnetic disturbance.

Local-time diurnal disturbance map
Previous researchers have studied spatial averages of lowlatitude magnetic disturbance, seeking an expression in terms of a local-time map.Some of these analyses have focused on individual storms (e.g.Akasofu and Chapman, 1964), while others have been based on data recording many storms (Chapman and Bartels,1962, Chs. 6.8 and 9.3;Cummings, 1966;Häkkinen et al., 2003).These and earlier works (Sabine, 1856;Moos, 1910;Bartels, 1932) have revealed that magnetic disturbance at each observatory exhibits a diurnal variation, with greatest (least) storm-time disturbance at dusk (dawn).We seek to depict this storm-time disturbance in a general form, one that accommodates storms of any size.
Towards that end, we examine the local time dependence of the disturbance time series for each observatory.For example, the Hermanus observatory is on a geographic longitude of 19.22 • .Therefore, the first HER hourly mean of each universal time-of-day, centered, as is standard, on 00:30, corresponds closely to an hourly mean with a local time of 01:30.We group the disturbance data from OBS into 24 local-time bins.In Fig. 11 we plot the disturbance data versus D st for local midnight, dawn, noon, and dusk; we do this for each observatory separately as well as for all the data combined.Next, we use a least-squares algorithm to find a factor, δ, that represents the proportionality between local latitudeweighted disturbance Dlat and global disturbance D st , for each observatory n and for each local-time bin (Kamide and Fukushima, 1971, p. 277;Francia et al., 2004, p. 3700).
The best-fitting proportionality factors are plotted in Fig. 11.Note the general consistency among OBS for each local-time slice.For local midnight and noon the disturbance field is very nearly one-to-one proportional to D st .On the other  26), from one observatory plotted versus data from another, along with linear correlation coefficients.In order to better reveal the shapes of the correlations we have plotted, in sequence, 100.0% of the data, followed by 10.0%, 1.0%, and 0.1% of the data.Note the general correlation seen in the disturbance data and the anti-correlation seen in anomalous disturbance seen in observatories from near-opposite longitudes.
hand, for local dawn the disturbance field is about 20% less than D st , and for local dusk it is about 18% greater than D st .
In Fig. 12 we present the proportionality factor, averaged for all OBS, as a D st -scaleable local-time disturbance map.This makes the dawn-dusk asymmetry very clear.where local time θ h is measured in continuous decimal hours.
We remark that the mapped values of δ show a very systematic form across the 24 h of local time -there is very little statistical jitter.
The map of disturbance asymmetry can, if one wishes, be interpreted in terms of a dusk-centered partial ring current, but a couple of points are worth considering.The first is whether or not partial ring currents can actually be inferred from ground-based magnetometer data (e.g.Fukushima and Kamide, 1973).Theoretical studies (Harel et al., 1981; st for one-hour bins near local midnight (00:30), dawn (06:30), noon (12:30), and dusk (18:30) for OBS of (a, b, c, d) Hermanus (HER), (e, f, g, h) Kakioka (KAK), (i, j, k, l) Honolulu (HON), and (m, n, o, p) San Juan (SJG), and (q, r, s, t) for all OBS together for 1958-2007.Plotted in blue is the line of proportionality δ between Dlat and D st , and given is the numerical proportionality factor itself.Note the general consistency of results here, and the specific tendency for one-to-one proportionality at midnight and noon, the smaller proportionality at dawn, and the larger proportionality at dusk.Crooker and Siscoe, 1981) suggest that local-time disturbance asymmetry might be the result of field-aligned Birkeland currents and their connecting partial-ring currents, something that has also been studied using ground-based declination data (Iyemori, 1990).Curiously, some satellite studies confirm a dawn-dusk asymmetry in the disturbance field (Langel and Sweeney, 1971;Suzuki and Fukushima, 1984), while others find a midnight-centered partial ring  27) is shown as a (blue) histogram and δ(θ h ) from Eq. ( 28) is shown as a smooth (red) curve.Note the noon-midnight (dawn-dusk) symmetry (asymmetry).
current (e.g.C:son Brandt et al., 2002;Le et al., 2004).We are agnostic on interpretations of this discrepancy, but we assert that maps of ground magnetic disturbance, like Fig. 12, need to be explained, and, of course, we acknowledge that current interpretations might be revised in the future.
The second point that should be considered concerns the symmetry between noon and midnight.Before producing this map we expected to find some asymmetry.Indeed, it is now standard to correct D st with a term that approximates the magnetopause current contribution to ground-based stormtime magnetic disturbance, what is usually called D * st (e.g.Burton et al., 1975).In the equatorial plane of the magnetosphere, the magnetopause current is eastwards, contrary to the westward direction of the ring current.This should contribute a dayside enhancement of the disturbance field, opposite to the disturbance depression given by the ring current.On the nightside, the equatorial tail current is westwards, parallel to the direction of the ring current.This should contribute a nightside reinforcement of the disturbance depression given by the ring current.Turner et al. (2000) estimate that the tail current might even contribute about 25% of D st .Therefore, we might expect a noon-midnight asymmetry in our local-time disturbance map, with less (more) depression on the day (night) side.Why don't we see this asymmetry?Perhaps the answer is related to the fact that we are mapping the cumulative disturbance over many magnetic storms, while magnetopause currents tend to be most important during storm commencement and tail currents tend to be most important during storm main-phase.We feel that this is insufficient and possibly irrelevant.For the magnetic signatures of these particular storm phases to be erased from our map there would need to be, at one point or another, dayside (nightside) currents that are parallel (antiparallel) to the ring-current.We find this difficult to imagine.We think it more likely that an explanation is related to the relative sizes of the magnetopause and the magnetotail, which are both large compared to the Earth.Therefore, currents in the magnetopause and magnetotail cannot generate significant magnetic-field gradients across the the dimension of the Earth, where, obviously, the observatories are measuring the field.The result is the observed noon-midnight symmetry in disturbance.

Modulation of storm probability by solar rotation
The occasional tendency for one magnetic storm to occur 27 or so days after another was discovered long ago from analysis of observatory data (Broun, 1876;Chree and Stagg, 1928;Chapman and Bartels, 1962, Ch. 12).Early on it was recognized that the cause of this recurrence was related to the Sun's ∼27-day rotation (Maunder, 1905;Bartels, 1932), but the nature of underlying physical connection remained mysterious until the discovery in the 1970s of coronal holes and their emitted high-speed streams of plasma (Neupert and Pizzo, 1974).Prominent examples of the recurrence phenomenon are shown in Fig. 13, where we see pairs of storms separated by about 27 days, and even, in one case (b) a triple occurrence of storms.Some of the storms shown in this figure (a,  c) are among the largest of the past 50 years, an observation that is at odds with the commonly held perception that corotating interaction regions tend to be responsible for small storms.
As we have already noted, the ∼27-day recurrence of storms shows up only faintly in the power spectra of observatory data.An alternative measure, useful for searching a time series for recurrent statistical phenomena, is autocorrelation.The Wiener-Khinchine theorem (e.g.Bendat and Piersol, 2000) tells us that power spectral density and autocorrelation are equivalent, provided the time series being analyzed is stationary.Otherwise, the two methods can give qualitatively different results.Therefore, examination of D st auto-correlation is worthwhile, since storms can occur randomly in time, even if they might also occur in seriallyrelated pairs.We calculate the discrete quantity where N D is the number of data in the D st time series, and where the index j can be expressed in terms of a time lag τ .In Fig. 14 we show the auto-correlation results, clearly seen are prominent peaks for time lags of 27, 54, 81 ... days, confirming the modulation of storm-occurrence probability with periods corresponding to synodic solar rotation.This recurrence is seen for both D 5807-4SH st and the Kyoto D st .Other than the peaks associated with solar rotation, auto-correlation for D 5807-4SH st shows little additional coherent repetitive signal.On the other hand, for the Kyoto D st broad peaks centered on semi-annual and annual time lags are seen; the square root of the peak-to-peak amplitude is about 8 nT.This is another indication of the Kyoto index has had an incomplete removal of semi-annual solar-quiet variation.

Semi-annual-diurnal modulation of disturbance
Another discovery made long ago using magnetic observatory data is the semi-annual modulation of storm-occurrence probability.Storms are most likely to occur at about the time of equinox, and least likely to occur at about the time of solstice (Sabine, 1856;Cortie, 1912;Chapman and Bar-tels, 1962, Ch. 11.9).Subsequently, it was discovered that there is also a universal time-of-day modulation of magnetic activity (Bartels, 1925;McIntosh, 1959), with maximum activity tending to occur at about 10:00-11:00 UT and 22:00-23:00 UT.Two hypotheses have been proposed to explain the combination of these observations.The first hypothesis is the so-called equinoctial hypothesis of Bartels.Here it is supposed that magnetic activity is modulated by the angle that the geomagnetic dipole axis makes with the Sun-Earth line.As an example, a function that resembles would correspond to activity that is subdued (enhanced) when is small (large)2 .Given the tilt of the Earth's   31).Red (Blue) denotes enhanced (subdued) predicted magnetic activity.The symmetry ( 32) is easily seen here to be a property of both hypotheses.rotational axis relative to the ecliptic and the tilt of the geomagnetic dipole axis relative to the rotational axis, can be parameterized in terms of universal time-of-year t Y and time-of-day t D .There is no specific physical mechanism associated with the equinoctial hypothesis, and as such, it is phenomenological.
The second and best-known hypothesis is due to Russell and McPherron (1973).Here it is supposed that magnetic activity is controlled by connection of the interplanetary magnetic field onto the geomagnetic field (Dungey, 1961), and this is most likely when, in solar-magnetospheric coordi-nates, the interplanetary field is southward oriented.Because the Parker (1958) spiral tends to entrain interplanetary field lines into the solar-equatorial plane, geomagnetic activity should be modulated by a function of the form where is the angle that the sub-solar geomagnetic field makes with the solar equator.Activity is enhanced (subdued) when is small (large), and this can also be parameterized in terms of universal time-of-year and time-of-day.For both the equinoctial and Russell-McPherron hypotheses, a couple issues are worth careful consideration.The first concerns the predicted periodicities and the reported observation of those periodicities.Semi-annual and diurnal harmonics appear prominently in solar-quiet variation, see, once again, Fig. 4. And, in constructing D 5807-4SH st , we have removed a stationary Sq time series containing semi-annual and diurnal harmonics (and their coupled relatives) from the observatory data, Eqs. ( 11) and ( 12).The Kyoto D st is constructed with a partial removal of most of the same harmonics, Eq. ( 13).What, then, is the meaning of semi-annual and diurnal periods identified in the Kyoto D st time series when those same periods have been partially removed?
The second issue concerns analytical methods of data analysis.Over the semi-annual timescale, magnetic storms can be envisioned as discrete events realized from a stochastic process, and their occurrence can be treated statistically.Over the diurnal timescale, storms evolve continuously and this usually motivates a deterministic treatment.How can a single hypothesis be interpreted in terms of such different descriptions of nature?Most published discussions of the equinoctial hypothesis have focussed on continuous evolution of activity across both semi-annual and diurnal timescales (Bartels, 1925, his Fig. 3;Svalgaard, 1977, his Fig. 20), although in a few cases the semi-annual statistics of storm occurrences have also discussed (e.g.Svalgaard et al., 2002, their Table 1).The original analysis of Russell and McPherron (1973) was mostly focussed on semi-annual statistics (their Fig. 2), and, indeed, Mayaud (1974) objected that the Russell-McPherron hypothesis cannot explain continuous diurnal magnetic activity.Our concern here is that a conflation of continuous modulation with modulation of probability risks obscuring the important distinction between the qualities of stationarity and non-stationarity.
In Fig. 15 we show the predicted relative activity amplitudes for the equinoctial and Russell-McPherron hypotheses across all phases of universal time-of-year and time-of-day; compare with Russell and McPherron (1973, their Fig. 5) and Svalgaard (1977, his Fig. 21).Note that both hypotheses have the symmetry but that there are also significant differences.The equinoctial (Russell-McPherron) hypothesis predicts distinctive lows (highs) superimposed on a general baseline of magnetic activity; Cliver et al. (2000) use the metaphor of "valley digging" ("mountain building") to describe these differences.
Figure 15 is a conventional presentation, but it is also useful to consider the separate daily and annual averages, From Fig. 16 we see that these averages have amplitudes with different functional forms, but both hypotheses have simultaneous maxima, at the equinoxes and magnetic dawndusk, and simultaneous minima, at the solstices and magnetic noon-midnight.Therefore, when only one of either the annual or diurnal dimensions is considered, the two hypotheses have a perfectly parallel phase relationship.Tests of the two hypotheses will be most discriminating when examination is made over all phases of time-of-year and time-of-day, such as mapped in Fig. 15.When comparing predicted variation with that actually realized, it is conventional to make simple averages of many years of magnetic-index data.In Fig. 17 34).Red (Blue) denotes enhanced (subdued) predicted magnetic activity; the contouring range is the same for each case and the symmetry (32) is enforced in order to reduce statistical jitter.
the symmetry (32).We have also normalized the contouring range so that it is the same for each plotted quantity.In (a) for D 5807-4SH st the average value is −7.65 nT, and the amplitude of the variation about the mean is small, about 3 nTthe resolution of the raw data is 1.00 nT.These observations are yet another indication that we have successfully removed almost all solar-quiet variation from the observatory time series used to construct D 5807-4SH st .The situation is very different for the Kyoto D st .In Fig. 17b the average value is −16.25 nT; recall that the Kyoto D st has larger negative bias than D 5807-4SH st .A more important difference is the 8 nT of variation about the mean and the prominent presence of a semi-annual signal.This presentation for the Kyoto D st is essentially a reproduction of a result given by Cliver et al. (2000, their Plt.2), which they assert supports the equinoctial hypothesis.It is important to note, however, that the data give metaphorical valleys that are not very low, and mountains that are not very high.The small topography is the result of the D st time series recording extended periods of magnetic quiescence -magnetic storms are relatively rare and of short duration -and so, upon averaging, the variation has a small amplitude, much smaller than might be expected if averaging was restricted to (say) large storms.Simple averaging of observatory time series is not the best way to reveal transient non-stationary disturbance.
The pattern exhibited in Fig. 17b appears to be residual solar-quiet variation in the Kyoto D st .If this is true, then Fig. 17b is not an accurate depiction of storm-time magnetic disturbance.This conclusion is supported by Fig. 17c, where we plot the latitude-weighted average of the four stationary Sq time series from each of OBS; Equation ( 34) is not, of course, a sensible quantity for studying solar-quiet variation, but it is useful for estimating the effects of unremoved Sq in the Kyoto D st time series.The correlation between Fig. 17b and c is not perfect, nor do we expect it to be -the Kyoto D st has had some, but not all, solar-quiet variation removed.Still, substantial correlation is obvious: the two signals are of the same sign, the same semiannual phase, and approximately of the same peak-to-peak amplitude.It may sound obvious, but investigators should be cautious when a signal they identify as being due to magnetic disturbance resembles solar-quiet variation.Our worry is that some of the topography mapped for investigations of the equinoctial and Russell-McPherron hypotheses is contaminated by solar-quiet variation.This is one of the reasons why we have undertaken a complete re-examination of the D st index, starting from the raw data and carrying all the way through each step of the constructive method.We choose to interpret the equinoctial and Russell-McPherron hypotheses in terms of the modulation of storm-occurrence probability 3 .For comparison of the hypotheses with data, we use storm-time, maximum intensi-3 We acknowledge that it is worthwhile to investigate the stationary and continuous, semi-annual and diurnal modulation of magnetic "activity", at least insofar as it is possible to define such a signal.For such work the am index has often been used, but D st (Kyoto, 5807-4SH, or othwise) is not an appropriate index.In this study, where D st is the focus, we choose to investigate the equinoctial and Russell-McPherron hypotheses in terms of the modulated probability of storm occurrence.ties −D st M , Sect.5.1, which we sort in time.This takes us far away from the simple averages formed from all of the data which, for the reasons stated, we dislike.Our approach is similar to that made by others in their analysis of semiannual statistics (Newton 1948, his p. 62, andRussell andMcPherron 1973, their Fig. 2), except that here we extend the approach to diurnal statistics as well.
In Fig. 18 we show separate universal (a) time-of-year and (b) time-of-day statistics for three different exceedance levels of storm-time maximum intensity.Generally speaking, the phases of the occurrence statistics are consistent with both the equinoctial and Russell-McPherron hypotheses 4 , Fig. 16. 4 We are focussed on the predicted phases of the equinoctial and The largest storms (a, −D st M >100 nT) are most likely to occur at or around late April, just after the vernal equinox, and late October, just after the autumnal equinox.With respect to diurnal dependencies, for the largest storms (d) maximum intensity tends to occur at 06:00-12:00, prior to magnetic dawn, and 18:00-24:00 UT, prior to magnetic dusk.Note that as the threshold for −D st M is increased (decreased), the amplitude of the semi-annual and semi-diurnal modulation increases (decreases).This is consistent with a property of the D 5807-4SH st time series which we have previously identified: the inclusion of all data shows little or no modulation, but subsets of the time series can show prominent periodicities and statistical modulation.We are reminded of the important distinction between a time series that is stationary and fragments of the same time series that are not.
In terms of the generalized phase map, one simultaneously relating both semi-annual and diurnal variation, Fig. 19 shows binned statistics obtained from D 5807-4SH st for three different −D st M exceedance levels.Unfortunately, a coherent pattern is not obvious, and we do not find any significant correlation between these topographies and the predictions shown in Fig. 15.We cannot, therefore, draw any conclusion about the validity of either the equinoctial or Russell-McPherron hypotheses.Some readers might find this to be disappointing, but they should not be surprised -if the data were sufficient to make a clear distinction between the two hypotheses, then that distinction would probably have already been identified by researchers, and there would be no need for us to discuss rival theories.If a coherent pattern is to be found in terms of statistical counts of −D st M , then it might have to wait for the inclusion of additional data.For this and other reasons, we are pursuing the detailed construction of a 100-year D st time series.

Solar cycle modulation of storm probability
The longest characteristic timescale considered in this analysis is the ∼10.5-year, solar-cycle.It is a well-known observation that concurrent with the semi-periodic waxing and waning in time of sunspots there is a modulation of stormoccurrence probability (Ellis, 1899;Chapman and Bartels, 1962, Ch. 11.11).This is, of course, one of the most important relationships in the subject of space weather.Its discovery raised "terrestrial magnetism to the dignity of a cosmical science" (Sabine, 1856, p. 362).For completeness and as a summary, in Fig. 20 we show the solar-cycle modulation Russell-McPherron hypotheses.We are not focussed on the details of the predicted amplitude profiles of each hypothesis.In our opinion, the physics of storm initiation and evolution depend on too many variables, such as solar-wind velocity and interplanetary magnetic-field polarity and strength, to justify detailed analysis of amplitudes for either hypothesis when only observatory time series are being used.Other works (e.g.Berthelier, 1976;O'Brien and McPherron, 2002) take important steps in directions different from those taken here.of storm-occurrence statistics, expressed in terms of monthly root-mean-square D st .

Conclusions
Having presented a new algorithm for calculating D st and having examined results for 1958-2007, let us conclude with some general points and observations.Beginning with our motivation: Why revise a standard magnetic index?It has now been over half a century since the D st index was first invented.Since then, the science of "terrestrial magnetism", the methods of time-series analysis, and the technology of computers have all advanced considerably, and the data time series from magnetic observatories have grown longer and longer.Retrospective analyses, such as that undertaken here, can exploit this progress.It should not be too surprising when these retrospective analyses identify difficulties and problems with previous work and its subsequent interpretation.This is a natural part of the scientific process, and it is the way in which progress is made.
In developing our new algorithm for D st and in applying it to historical magnetic-observatory data, we have been reminded of the importance of (1) using the original source data, (2) inspecting the data and derived results in both the time and frequency domains, and (3) carefully considering the distinction between stationary and non-stationary timeseries ingredients.The combination of the new D st index and the individual time series from each observatory reveals patterns in the global magnetic disturbance field.Some of these patterns are well understood, but others remind us that we still have a lot to learn about the magnetosphere, magnetic storms, and the Earth's relationship with the Sun.

Fig. 1 .
Fig. 1.Horizontal-intensity data from Hermanus (HER) for 1958-2007.Shown are the data Obs H i , the Quiet subset of the data, and the internally-sustained time series I (t)=C+SV (t) corresponding to permanent crustal magnetization and the main field and its secular variation.

Fig. 2 .
Fig. 2. Horizontal-intensity time series illustrating step-by-step construction of the disturbance time series Dist from the observatories of (a) Hermanus (HER), (b) Kakioka (KAK), (c) Honolulu (HON), and (d) San Juan (SJG) for relatively quiet days from 18 August-26 September 2003.Shown are the externally-sustained time series E i =H i −I (t i ), the Quiet and Active signal subsets, the disturbanceinterpolated time series Q i , the solar-quiet variation Sq i , and the disturbance time series Dist i .Note the interpolation over the data gap in the SJG data.

Fig. 3 .
Fig. 3. Horizontal-intensity time series illustrating step-by-step construction of the disturbance time series Dist from the observatories of (a) Hermanus (HER), (b) Kakioka (KAK), (c) Honolulu (HON), and (d) San Juan (SJG) for relatively active days from 26 October-30 November 2003.Shown are the externally-sustained time series E i =H i −I (t i ), the Quiet and Active signal subsets, the disturbanceinterpolated time series Q i , the solar-quiet variation Sq i , and the disturbance time series Dist i .

Fig. 4 .
Fig. 4. Relative power spectra for horizontal-intensity data from Honolulu (HON) for 1958-2007.Shown are spectra for (a) periods of 0.08-8000 days, (b) periods in the neighborhood of 1 day, and (c) periods in the neighborhood of 0.5 days; in each case for the external-field E time series, the disturbance-interpolated time series Q, the solar-quiet variation Sq(t), and the residual disturbance time series Dist i .Labels show spectral peaks corresponding to diurnal (d), monthly (m), and annual (a) harmonics, and their cross-harmonic coupling.
ι d,m,a sq ι d,m,a e iι d ω d t e iι m ω m t e iι a ω a t .

Fig. 5 .
Fig. 5. Panoramic view of D 5807-4SH st and the Kyoto D st for 1958-2007, together with the difference D st defined by Eq. (23).Note the prominent semi-annual variation in D st and the occasional abrupt differences that occur during storms.

Fig. 6 .
Fig. 6.D 5807-4SH st and the Kyoto D st , together with the difference D st defined by Eq. (23), for some large magnetic storms.Note the prominent diurnal variation in D st and abrupt offsets during storms.

Fig. 7 .
Fig. 7. Relative power spectra for D 5807-4SH st and the Kyoto D st for 1958-2007.Shown are (a) the spectrum over 0.08-8000 days, (b) for periods in the neighborhood of 1 day, and (c) for periods in the neighborhood of 0.5 days.Note the prominent spectral peaks in the Kyoto D st corresponding to diurnal (d), monthly (m), and annual (a) harmonics, and their cross-harmonic coupling.www.ann-geophys.net/27/3101/2009/Ann.Geophys., 27, 3101-3131, 2009

Fig. 9 .
Fig. 9. Storm-time maximum-intensity statistics for both D 5807-4SH st and the Kyoto D st showing (a) number density and (b) cumulative density.Also show are fits given by Eqs.(24) and (25) and the parameters in Table3.

Fig. 10 .
Fig.10.For 1958For  -2007 (a-f)  (a-f)  disturbance data Dlat from one observatory plotted versus data from another, along with linear correlation coefficients, (g-l) anomalous disturbance Anom, Eq. (26), from one observatory plotted versus data from another, along with linear correlation coefficients.In order to better reveal the shapes of the correlations we have plotted, in sequence, 100.0% of the data, followed by 10.0%, 1.0%, and 0.1% of the data.Note the general correlation seen in the disturbance data and the anti-correlation seen in anomalous disturbance seen in observatories from near-opposite longitudes.
Using another least-squares algorithm, we fit the 24 local-time proportionality factors δ to a truncated Fourier series.The following smooth function accurately describes the local-time, disturbance map: δ(θ h ) = 0.9995 +(28)

Fig. 14 .
Fig. 14.Auto-correlation, Eq. (29), for both D 5807-4SH st and the Kyoto D st .Note the storm recurrence peaks corresponding to a synodic solar rotational period (27.27 d), two synodic solar rotations (54.54 d), etc.Note also the prominent, but broad, semi-annual and annual peaks for the Kyoto D st time series.

Fig. 16 .
Fig. 16.Polar-coordinate phase maps showing for the equinoctial and Russell-McPherron hypotheses (a) the semi-annual modulation of the daily averages and (b) the daily modulation of the annual average, Eq. (33).Also shown are (a) the vernal equinox and (b) the magnetic meridian determined by the axis of the geomagnetic dipole.
we show binned averages for D 5807-4SH st and the Kyoto D st , each as a function of time-of-year and time-of-day.In each case we have enforced

Fig. 17 .
Fig. 17.Binned averages for (a) D 5807-4SH st , (b) the Kyoto D st , and (c) the latitude-weighted Sq, Eq. (34).Red (Blue) denotes enhanced (subdued) predicted magnetic activity; the contouring range is the same for each case and the symmetry (32) is enforced in order to reduce statistical jitter.

Fig. 18 .
Fig. 18.Polar-coordinate binned statistical counts of the number of occurrences of −D st M or universal (a-c) time-of-year and (b) time-ofday, in each case for the indicated exceedance levels.Also shown are the vernal equinox and the magnetic meridian.

Fig. 19 .
Fig. 19.Binned statistical counts of the number of occurrences of −D st M , in each case as a function of storm-time maximum intensity.The symmetry (32) is enforced.

Fig. 20 .
Fig. 20.Time-spiral plot with a period of 10.5 years showing monthly root-mean-square D st and the solar-cycle waxing and waning of global magnetic activity from 1958-2007.

Table 1 .
Summary of the observatories (OBS) and their data for D5807-4SH Sugiura et al. clearly appreciated the importance of harmonic cross-coupling in describing solar-quiet variation.Their calculation of D st assumes an Sq model consisting of a two-dimensional Fourier series having diurnal and annual harmonics, (d) and (a): ι d,a sq ι d,a e iι d ω d t e iι a ω a t or ι d,a sq ι d,a e i(ι d ω d +ι a ω a )t ;

Table 2 .
Largest magnetic storms for 1958-2007 ranked by maximum −D st .

Table 3 .
Summary of model parameters for Eqs. (