A method to identify aperiodic disturbances in the ionosphere

In this paper, variations in the ionospheric F2 layer’s critical frequency are decomposed into their periodic and aperiodic components. The latter include disturbances caused both by geophysical impacts on the ionosphere and random noise. The spectral whitening method (SWM), a signal-processing technique used in statistical estimation and/or detection, was used to identify aperiodic components in the ionosphere. The whitening algorithm adopted herein is used to divide the Fourier transform of the observed data series by a real envelope function. As a result, periodic components are suppressed and aperiodic components emerge as the dominant contributors. Application to a synthetic data set based on significant simulated periodic features of ionospheric observations containing artificial (and, hence, controllable) disturbances was used to validate the SWM for identification of aperiodic components. Although the random noise was somewhat enhanced by postprocessing, the artificial disturbances could still be clearly identified. The SWM was then applied to real ionospheric observations. It was found to be more sensitive than the oftenused monthly median method to identify geomagnetic effects. In addition, disturbances detected by the SWM were characterized by a Gaussian-type probability density function over all timescales, which further simplifies statistical analysis and suggests that the disturbances thus identified can be compared regardless of timescale.


Introduction
The variability of the critical frequency of the ionospheric F2 layer (foF2) is a proxy for the complex behavior of the ionosphere.In the frequency domain, periodicity is the most significant feature in the foF2's power spectrum, as indicated by the vertical dash-dotted lines in Fig. 1a.The black curve is similar to that in Fig. 1 of Liu et al. (2011), based on observations at Canberra (149.0 • E, 35.3 • S) obtained between 1950 and 2007, and with a time resolution of one hour.In general, the intensity of the power spectrum increases with decreasing frequency.However, aperiodic variations do not exhibit any prevailing frequency.They do not show any periodicity, and they cannot be read off directly from the frequency domain.Unlike periodic signals, which are easy to both distinguish (e.g., through filtering) and model (e.g., Zhao et al., 2005), aperiodic variations in the ionosphere are usually discussed case by case; in addition, they are difficult to distinguish automatically.This difficulty has its origin in the complexity of the ionosphere's behavior, which is modulated by solar radiation, the solar wind, and its geomagnetic consequences, the neutral atmosphere, as well as by electrodynamical processes (Rishbeth and Mendillo, 2001).These modulations reveal themselves partially as periodic signals and also as aperiodic disturbances in the ionosphere.
However, the characterization of background, disturbance, and noise is usually question oriented; i.e., it is dependent on the issue at hand in a given study.From the perspective of data processing, it is more convenient to deal with explicit features than with implicit mechanisms.In the present paper, we decompose the foF2's variations into periodic and aperiodic components.The aperiodic components could be either random noise or disturbances caused by geophysical impacts (such as geomagnetic storms) on the ionosphere.Here, the noise features are referred to as persistent, fast random dithers with small amplitudes, superposed onto the periodic components.Other aperiodic components are manifested as deviations with significant amplitudes and/or significant durations, which can usually be attributed to external geophysical impacts.In this paper, we refer to periodic components  as the background, to random noise as noise, and to other aperiodic components as aperiodic disturbances.
The spectral whitening method (SWM) is a technique often used for statistical estimation and detection.It is useful for either decorrelating a data sequence or controlling the spectral shape (Eldar and Oppenheim, 2003).The SWM is adopted to remove the periodic components from the foF2 variations by flattening the Fourier spectrum and therefore to identify the aperiodic components (particularly the aperiodic disturbances).

Methodology
To illustrate this SWM approach and to test its validity, a simulated signal series, g(t) (where time t is expressed in hours; Fig. 2a), was generated from (1) all of the major periodic components highlighted in Fig. 1a; and (2) a total of 100 (although this number can be changed arbitrarily) segments of artificial disturbances, g d , of random amplitudes and random durations (but less than 72 h), consistent with typical ionospheric disturbances caused by geomagnetic storms (cf.Zhao et al., 2008, and references therein), distributed randomly in time and connected by random noise (Fig. 2b).The data set length, the time period, and even the data loss characteristics were also set to be identical to those pertaining to the Canberra foF2 data set.Thus, where ξ i represents the ith frequency of the main periodic components indicated by the vertical dash-dotted line in Fig. 1a (i runs from 0 to 7, where 0 represents the DC component); A i and φ i are the amplitude and phase of the ith periodic component, where φ i is set randomly.A i is initially set by reference to the Fourier transform of the Canberra foF2 data set.Due to the randomness of φ i and g d , adjustments to A i were applied to improve the similarity between the spectrums of g(t) (Fig. 1b) and the Canberra foF2 (Fig. 1a).The spikes at higher frequencies than that of the terdiurnal component in Fig. 1a were not included in the synthetic data, as these components are the high-frequency harmonics of the diurnal variation, which are usually not considered.It appears that aperiodic disturbances can be distinguished only when they are not buried in periodic signals.It is impossible, and (fortunately) unnecessary, to simultaneously filter out all periodic components.It is sufficient to detect aperiodic components under conditions where every periodic component has been reduced to, or below, the level of the aperiodic components.That is, all discernible spikes, humps, or slopes in the spectrum (Fig. 1) related to periodic components need to be polished, so that there is no higher power intensity at any frequency.Visually, this corresponds to a power spectrum that is flattened.Spectral flattening can be achieved through application of the SWM, although the whitening algorithm is not unique (Eldar and Oppenheim, 2003).In this paper, we whiten the foF2 data sequence by dividing the Fourier transform of the observed data series by a real envelope function.
Considering a transformation from g(t) to g d (t), and where P env (ξ ) is the upper envelope of the power spectrum of g(t).P env (ξ ) can be calculated based on the strict definition of the envelope of a curve family (see Bruce and Giblin, 1992), which is rather complicated.However, it is possible to obtain a good approximation of the envelope value on the basis of the maximum value in a given data window.P 0 is the value that occurs most often in the data set of P env (ξ ); i.e., it is the mode of P env (ξ ).The transformation above consists of four steps.(1) A Fourier transformation, shown as the bracketed part in Eq. ( 2), is applied to the original data, while a periodic extension is imposed on the original time series to avoid boundary effects affecting the transformation (Gonzalez-Velasco, 1995).(2) The Fourier power spectrum is divided by its upper envelope P env (ξ ), so that the maximum intensity becomes unity.The spectrum is thus flattened and displays the spectral form of white noise (i.e., it has been whitened).As the aperiodic components are, in fact, not discernible in the spectrum, they are hardly affected by the whitening process.(3) The whitened spectrum is multiplied by P 0 to ensure that the most commonly occurring values of the spectrum's intensity remain the same before and after whitening.In other words, most components approximately retain the same amplitudes before and after the transformation.The spectrum thus whitened is shown in grey in Fig. 1b.(4) An inverse Fourier transformation is applied to the whitened spectrum and a new time series, g * d (t), is derived.The whitening process degrades the periodic components to noise and, therefore, additional random dithers are introduced.To reduce the enhanced noise, a three-point running average is applied to smooth g * d (t) and the final result, g d (t), is obtained (Fig. 2c).It is easily found that g d (t) can reproduce most features of g d (t) (compare Fig. 2b and c, as well as e and f), and the correlation coefficient of g d (t) and g d attains values of up to 0.89, which is an improvement over the correlation coefficient found below for the monthly median method (MMM).The major difference is the relatively larger amplitude of the irregular dithers in g d (t).The derived background, i.e., g(t) − g d (t), is nearly the same as g(t) − g d (t), except for small-amplitude random deviations (Fig. 2h).Thus, the SWM introduced here is capable of identifying aperiodic components among periodic signals.
The mean or median value for a given time period (window) is usually used as a proxy for the ionospheric background variation, in particular to describe its general tendency.Contrary to the universality of the mean or median method, the window width applied may differ, depending on research interests, or be question oriented.However, if no mechanism-specific considerations are imposed and a general window width is needed instead, periods of 1 month or 27 days are usually acceptable.For example, the monthly median foF2 at a certain time means the median value of all the foF2 values at the same time on different days during this month.In addition, median foF2 measures are preferred when data outliers could affect the mean value, which is the reason why, in ionospheric data analysis, the monthly median foF2 value at a given local time and its difference from the original value are usually adopted as the background value and its perturbation, respectively (Piggott and Rawer, 1972;Perrone and Di Franceschi, 1998) time window chosen covers the month prior to the observation time (i.e., we use a backward-running median) so as to simulate real-time data processing.The MMM is a good approach when attempting to distinguish periodic components from aperiodic disturbances when the ionosphere is sufficiently stable such that all foF2 values are very close to the median value.However, its efficacy will decrease when dayto-day ionospheric variations are significant (Fig. 2h).For the simulated data set, the MMM can detect man-made perturbations, but there is still a systematic offset, which is caused by the differences between the monthly median value and the real background (Fig. 2g).For the entire synthetic data set, the MMM is used to achieve a correlation coefficient of 0.71 between the original and derived disturbances.This is good but of lower significance than the SWM's correlation coefficient of 0.89.Moreover, the variance of the deviation between the original and derived disturbances is 1.06 as derived using the SWM, which is much lower than the MMMbased value of 2.88 (Fig. 3).In fact, if we shift the median widow of the MMM day by day from one month before the observation time to one month after, the correlation coefficient varies from 0.57 to 0.73.This indicates that the SWM can reproduce the aperiodic disturbances more precisely and is, therefore, generally better than the MMM at identifying aperiodic disturbances from periodic components.
In the context of other Fourier-transformation filtering processes, such as band-pass filtering, the transformation kernel is a window function that is used to strongly constrain the intensities around the frequencies of interest (Gonzalez-Velasco, 1995).To identify aperiodic components, it is theoretically necessary to simultaneously filter out all periodic signals, which is normally impossible unless the frequencies subject to filtering are all known and discrete.However, it is not necessary to do so, because if all periodic components are reduced to, or below, the level of the aperiodic components, the latter are easily discernible, and this is the function of our SWM application.The SWM can be considered to be a type of Fourier-transformation filter with a special kernel, P env /P 0 (ξ ).This kernel suppresses all known and unknown periodic components with power intensities in excess of P 0 to the level of P 0 , so that they are rendered unimportant and other components (i.e., the aperiodic disturbances and noise) emerge.Meanwhile, the most common spectrum intensity (P 0 ) is unchanged, so that the signals before and after processing, i.e., the original data and aperiodic disturbances (as well as the noise) are consistent in amplitude.However, note that the suppressed periodic features have been reduced and have become part of the noise, thus contributing to the irregular variations.This implies that the SWM is not suitable for the detection of disturbances at frequencies around the original sampling frequency.Meanwhile, the SWM cannot deal with perturbations characterized by durations that are similar to the observation duration.Despite these shortcomings, successful application to the synthetic data set validates the SWM's efficacy in distinguishing aperiodic disturbances from periodic components.
Note that the MMM is often used, but it is not always the most appropriate of the median or mean methods that use different window widths to distinguish between the background and perturbations.A specific window width set by the requirements pertaining to the issue under investigation usually leads to better results than a fixed width of 1 month.However, this means that one must determine the most suitable width case by case.As shown here, the SWM is question independent, and can be applied universally to any time sequence to which a Fourier transformation can be applied.

Application to the ionosphere
We next applied the SWM to the Canberra foF2 data set .The original foF2 and the SWM-derived aperiodic disturbances (denoted f D ), as well as the background variation, f B = foF2 − f D , are compared in Fig. 4. As expected, f B represents the general foF2 variation, while f D (which varies around zero) demonstrates the foF2 fluctuations.As all periodic components have been reduced to such a level that no specific frequency is significantly stronger than any other, the significant fluctuations in f D are related to the aperiodic components, as suggested by the synthetic data.Given the proven success of the SWM in identifying aperiodic disturbances, it is interesting to explore the additional information the SWM can offer compared to other methods such as the MMM.We denote the monthly median of the foF2 as f r and define df = foF2 − f r .To simplify the descriptions, the backgrounds and disturbances derived on the basis of the SWM and MMM are referred to as SWM background (f B ), SWM disturbance (f D ), MMM background (f r ), and MMM disturbance (df ).
The general trends exhibited by both f D and df are similar (Fig. 4c and d).However, notable differences become apparent when the figure is enlarged (Fig. 4e and f).As the MMM background is composed of the median foF2 values measured during the month before the reference date, it is stable and smooth, i.e., similar to that in the simulation, although foF2 variations occur much more frequently than their simulated counterparts.In comparison, the SWM background shows more features than either the MMM background or the simulated SWM results (Fig. 2h).These differences are reflected in the behavior of the perturbations.During the shaded periods in Fig. 4e and f, the periods when either f r or f B deviate significantly from the observed foF2 are reflected by the corresponding trends in f D and df , respectively.During Period (I), f D and df are similar, but for Periods (II) and (III), f D is larger than df .The Dst index (Fig. 4f, thick solid curve) indicates that during Period (II) a geomagnetic storm occurred.The drop in the foF2 is the ionospheric response to the storm.The change in f D (= foF2 − f B ) is more significant than that in df (= foF2 − f r ; Fig. 4f) or, equivalently, f D is more sensitive to external disturbances than df .As geomagnetic storms mainly contribute to aperiodic perturbations in the foF2 (in addition to contributing weak, recurrent components), the period of solar rotation is only marginally discernible in the foF2 power spectrum.The significance of the period of solar rotation is much lower than that of other spectra (Fig. 1a).In addition, the SWM can suppress the narrow hump around the period of solar rotation in the spectrum and blur the imprints of recurrent geomagnetic  storms (Kamide and Chian, 2007), but it leaves that of aperiodic storms in the SWM disturbance.
To reach a statistical conclusion regarding the sensitivity of the SWM to known source(s) of perturbation (i.e., the geomagnetic activity), all periods with Dst values continuously below −20 nT for at least 24 h were investigated as potential geomagnetic-activity periods (GAPs).They are  Matuura, 1972;Fuller-Rowell et al., 1994) was recognized as an ionospheric disturbance.All disturbances occurring during the same GAP were compared, and the f D with the maximum absolute value is referred to as f Dm .The ionospheric disturbance during this GAP was recorded as a positive or negative disturbance if f Dm > 0 or f Dm < 0, respectively.The corresponding df with the maximum absolute value during the same GAP is denoted df m .The difference between df m and f Dm (i.e., f Dm − df m ) is plotted as a function of the corresponding Dst m in Fig. 5. Positive (negative) disturbances are plotted as grey (black) dots.In nearly 72 % of cases, the amplitude of f Dm is greater than that of df m (i.e., f Dm > df m for positive cases and f Dm < df m for negative cases) during GAPs (Fig. 5; Table 1).The difference can be up to 5 MHz, i.e., approximately 1/3 of the maximum observed foF2 (ca.15 MHz).Obviously, the SWM disturbance is more sensitive to strong geomagnetic activity.Figure 5 shows that negative ionospheric disturbances generally occur more often than positive perturbations.For Dst < −100 nT, the stronger the geomagnetic activity is, the more negative the ionospheric disturbances become (Fig. 5; Table 1).This result is in accordance with previously obtained statistical conclusions about positive and negative phases of ionospheric storms in similar regions (Prölss, 1993(Prölss, , 1995;;Tsagouri, 2000).It is not strange that the aperiodic disturbances discussed here share some features with ionospheric storms, because such storms are, in fact, intense disturbances (e.g., Rush, 1976).

Discussion
The MMM, which blurs the day-to-day differences within a given month, is in fact a high-stop filtering approach.The SWM, on the other hand, does not cause any frequency to become notably stronger than any other frequency, so that all features with distinct periodic components will be retained in the SWM background and, hence, periodic components cannot be found in the SWM disturbance.Equivalently, there is no typical frequency in the SWM disturbance.The fact that the SWM disturbance is not associated with any typical frequency results in some interesting characteristics in the probability domain.The probability density function (PDF; Billingsley, 1979) of the SWM disturbances at different local times was compared with its best fit to a Gaussian function, and their correlation coefficient was always close to unity (Table 2).This indicates that the SWM disturbance's PDF is Gaussian, which is due to the spectral-whitening process.For comparison, the PDF of the MMM disturbance differs significantly from a Gaussian distribution and varies with observation time (Table 2).
As the Gaussian distribution has been thoroughly discussed and the definitions of many elementary statistical parameters involve Gaussian distributions, the Gaussian-type PDF of the SWM disturbance offers significant benefits to further study.For example, the width of the Gaussian-type PDF under different conditions is exactly its variance, which is a measure of the spread of a set of numbers.It is the most convenient parameter to describe the degree of disturbance of the variable of interest (Kagan and Shepp, 1998).Meanwhile, the PDF width can be adopted as a scale factor to normalize all disturbances identified with a Gaussian-type PDF, so that, according to the central limit theorem (Rice, 1995), all normalized disturbances can be compared with each other, irrespective of their original physical meaning.Therefore, the comparability of the disturbances thus separated, as well as the dependence of f D on Dst revealed in Fig. 5, offers the possibility to define an ionospheric disturbance index that is independent of both time and physical mechanism, and which can be linked directly to other geophysical indices such as the Dst index.However, defining such an index is beyond the scope of this paper and will be discussed in a subsequent article.In addition, a Gaussian-type PDF is easily treated and advantageous in terms of mathematical and computational simplicity.

Conclusions
In this paper, the SWM was introduced to identify ionospheric aperiodic disturbances.Although the SWM could enhance the noise and is unsuitable for the identification of perturbations characterized by very short (i.e., close to the sampling period) or very long (i.e., comparable with the data set length) timescales, its validation based on application of the SWM to a synthetic data set (which simulated the main features of ionosphere observations) demonstrates that the SWM can identify aperiodic disturbances in the ionosphere.Similarly to the commonly used MMM, the SWM can be applied to all ionospheric data sets, but it yields a better background than the MMM.The main advantage of this characteristic is that the SWM-derived disturbances in the ionosphere are more sensitive to external geophysical perturbations than their MMM-derived counterparts.Meanwhile, disturbances identified by the SWM have a Gaussian-type PDF, which not only simplifies further statistical analysis by virtue of the well-known characteristics of Gaussian-type PDFs, but also simplifies comparisons of the disturbances at different times, regardless of the underlying physical processes.This comparability at different times offers the possibility to define an ionospheric disturbance index in future studies.

( a )
Power spectrum of the foF2 at Canberra during the period 1950-2007 (black).ant periods are indicated by vertical dash-dotted lines.The flattened spectrum is shown in e power intensities at all frequencies have been suppressed to below a given value, so that le frequency is dominant.(b) Spectrum (black) of a simulated dataset and its flattened (grey).

Figure 1 .
Figure 1.(a) Power spectrum of the foF2 at Canberra during the period 1950-2007 (black).Significant periods are indicated by vertical dash-dotted lines.The flattened spectrum is shown in grey; the power intensities at all frequencies have been suppressed to below a given value, so that no single frequency is dominant.(b) Spectrum (black) of a simulated data set and its flattened spectrum (grey).
Fig. 2. (a) Simulated time series.(b) Artificial disturbances of random amplitudes and random 339 durations distributed randomly in time with additional random noise.(c) Spectrum of the flattened 340 signal.(d) Disturbances identified by the MMM.(e), (f), and (g) Magnified period corresponding 341 to panels (b), (c), and (d), respectively.(h) Comparison of the original, SWM, and MMM 342 backgrounds.343 344

Figure 3 .
Figure 3.Comparison of the original disturbance (D) and the MMM disturbance (df ), as well as of D and the SWM disturbance (f D ). σ (df -D) and σ (f D -D) are the variances of df -D and f D -D, respectively.CC: correlation coefficient of the parameters on both axes.

Fig. 3 .
Fig. 3. Comparison of the original disturbance (D) and the MMM-disturbance (df), as well as of D 347 and the SWM-disturbance (f D ). σ(df-D) and σ(f D -D) are the variances of df-D and f D -D, 348 respectively.CC: correlation coefficient of the parameters on both axes.349 350 358

Fig. 5 .Figure 5 .
Fig. 5. Difference between the maximum absolute values of df and f D (i.e., f Dm -df m ) compared 359 with the minimum Dst (Dst m ) during periods when Dst is continuously less than -20 nT for at least 360 24 hours.Negative and positive disturbances according to f D are represented by black and grey 361 dots, respectively.362 363

Table 2 .
Correlation coefficients (CCs) of the PDFs of SWM/MMM-derived disturbances and their best fits to a Gaussian function.