Performance of the IRI-2016 over Santa Maria, a Brazilian low-latitude station located in the central region of the South American Magnetic Anomaly (SAMA)

In this work we analyze the ionograms obtained by the recent digisonde installed in Santa Maria (29.7 S, 53.7W, dip angle=−37), Brazil, to calculate the monthly averages of the F2 layer critical frequency (foF2), its peak height (hmF2), and the E-region critical frequency (foE) acquired during geomagnetically quiet days from September 2017 to August 2018. The monthly averages are compared to the 2016 version of the International Reference Ionosphere (IRI) model predictions in order to study its performance close to the center of the South America Magnetic Anomaly (SAMA), which is a region particularly important for high-frequency (HF) ground-to-satellite navigation signals. The foF2 estimated with the Consultative Committee International Radio (CCIR) and International Union of Radio Science (URSI) options makes good predictions throughout the year, whereas, for hmF2, it is recommended to use the SHU-2015 option instead of the other available options (AMTB2013 and BSE-1979). The IRI-2016 model outputs for foE and the observations presented very good agreement.


Introduction
The growing importance of space technologies through satellites for a large variety of applications such as science, Earth observation, meteorology, communications, security, and defence, puts forward the need to improve our ability of ionospheric modelling. For instance, the drag force on satellites in low-Earth orbit (LEO, generally defined as an orbit below an altitude of approximately 2,000 kilometer) increases when the solar activity is at its greatest over the 11-year solar cycle, 25 which may cause uncontrolled re-entry and degrade the predictions of satellite positions (Horne et al., 2013). During space weather conditions as defined by Denardini et al. (2016), elevated flux levels of high energetic particles may precipitate in the ionosphere in regions of anomalously weak geomagnetic field strength such as the South America Magnetic Anomaly (SAMA). Besides enhancing the ionization distribution and conductivities (Moro et al., 2013(Moro et al., , 2012, the energetic particles create high background counts which render satellite sensors unusable in this region Heirtzler, 2002). 30 Operators who control satellites in LEO may need to know with a high degree of accuracy when and where to turn satellites on and off to minimize the risk of detector saturation (Jones et al., 2017). Ionospheric modelling is also important for ground assets since it is essential to predict the ionospheric behavior for successful radio communication. Since drastic ionospheric variations can affect the performance of radio-based systems, such prediction may identify the periods, the path regions and the sections of high-frequency bands that will allow or disrupt the use of the radio transmissions (Ezquer et al., 2008). 35 One of the most widely used ionospheric models is the International Reference Ionosphere (IRI), which became the official International Standardization Organization (ISO) standard for the ionosphere since April 2014 (Bilitza et al., 2017). IRI is a joint project of the Committee on Space Research (COSPAR) and International Union of Space Science (URSI). It is derived from ionospheric observations collected by ground and in situ measurements such as the worldwide network of ionosondes, incoherent scatter radars, several compilations of rocket measurements, and satellite data. The model describes monthly 40 averages of the electron density, electron and ion temperature, total electron content (TEC), and ion composition as a function of height, location, and local time. Several major milestone editions of IRI have been released by the IRI Working Group since the 1970s in order to constantly revising the model to remain it up to date and accurate as possible (Rawer et al., 1978a(Rawer et al., , 1978b(Rawer et al., , 1981Bilitza, 1990Bilitza, , 2001Bilitza and Rawer, 1996;Bilitza and Reinisch, 2008;Bilitza et al., 2014Bilitza et al., , 2017. The latest version is known as IRI-2016 and has important improvements over the and 2007versions (IRI-2012and 45 IRI-2007. The most important update is the inclusion of two new model options for the F2 layer peak height, hmF2. These two options allow the users to model the hmF2 directly and no longer depend on the propagation factor M(3000)F2 described by Bilitza et al. (1979). Besides, IRI-2016 has a better representation of topside ion densities during very low and high solar activities. The details about the IRI model are available in the following homepage: http://irimodel.org/. 50 Among several parameters, IRI can predict the F2 layer critical frequency (foF2), hmF2, and the E-region critical frequency (foE) for a given time and location. The correct understanding of these parameters is particularly important for space technologies. The critical frequencies are two key parameters when calculating the electron densities of the ionosphere at F2 (NmF2) and E-region heights. Moreover, foF2 is related to the maximum usable frequency for the radio waves reflection and TEC that is significant for the phase delay of High Frequency (HF) ground-to-satellite navigation signals (Fuller-Rowell et 55 al., 2000). On the other hand, hmF2 receives much of the attention since it gives the highest stratification of the upper ionosphere.
In the literature, several papers have reported many comparative studies around the globe between the ionospheric parameters measured by ionosondes and different versions of the IRI to study its performance. In South America, Ezquer et al. (2008) analysed NmF2 over Tucumán (26.9º S, 66.4º W, dip angle = -26º), Argentina, during the low and high solar 60 activity years 1965 and 1970, respectively, and the moderate solar activity years 1967 and 1972. Bertoni et al. (2006) used Batista and Abdu (2004) compared the parameters foF2, hmF2, and B0 measured by two digital ionosondes over São Luís (2.6º S, 44.2º W, dip angle = -4.3º, 65 magnetic equator), and Cachoeira Paulista (22.7º S, 45º W, dip angle = -33.5º, close to the southern crest of the Equatorial Ionization Anomaly -EIA) with the IRI-2007 for high and low solar activity periods. Moro et al. (2016)  Port Stanley station (51.6º S, 57.9º W, dip angle = -49.8º), Argentina. Therefore, validate the IRI-2016 in a region under the influence of the SAMA is particularly important for HF communication and radio-based space systems as described before, besides contributing with IRI Working Group evaluating the goodness of the model in the low latitude Brazilian region.

Observed Data, Modelling, and Method of Analysis
The SMK29 is set to transmit radio waves continuously into the ionosphere from 1 MHz and increases the frequency up to 90 20 MHz with the sweep rate of 25 kHz for each round. The train of echoes to form an ionogram is transmitted/received with a 5 minutes temporal resolution. All recorded ionograms are initially auto-scaled by the Automatic Real-Time Ionogram  Table 1. The average of the solar emission at a wavelength of 10.7 cm from September 2017 to August 2018 is only (71.6 ± 3.5) × 10 -22 Wm -2 Hz -1 , and the sunspot number range from 1 to 18, characterizing the low solar 100 activity period.
Monthly average values of the observed foF2, foE, and hmF2 parameters are calculated from the daily hourly values. The IRI-2016 predictions of foF2, foE, and hmF2 are computed for the same geophysical conditions to compare with the observational data and to evaluate the discrepancies and goodness of the model. The Relative Deviation (RD) of the predicted values concerning to the observed values for modelling the foF2 using the Consultative Committee on International Radio (CCIR) 105 coefficient (CCIR, 1967) had been computed through the Eq. (1).
The term foF2 CCIR stands for the monthly average of the foF2 modelled by the CCIR sub-routine, while the term foF2 Observed is 110 the monthly average of foF2 measured by the SMK29. Besides the comparison between the observed foF2 with CCIR, the sub-routine URSI (Rush et al., 1989) is also tested and, therefore, Eq. (1) is also used considering foF2 URSI instead of foF2 CCIR. The foF2 storm model (Araujo-Pradere et al., 2002) was turned off in the IRI-2016 options since we are interested in the quiet time conditions. The RD is also evaluated for hmF2 and foE using Eq. (1). For hmF2, the comparison is made using the three currently available options for determining IRI-hmF2: AMTB2013 (Altadill et al., 2013), SHU-2015 (Shubin, 2015), and 115 BSE-1979(Bilitza et al., 1979, called AMTB, SHU and BSE, respectively, hereafter. The AMTB model is based on ionospheric data deduced from ionograms recorded by 26 Digisondes embracing latitudes from 65ºN to 52ºS and the longitude sector from 120ºW to 170ºE. The data cover different levels of solar activity from 1998 to 2006. The spherical harmonic technique was applied in AMTB to model the quiet pattern of the hmF2 at a global scale. The SHU model is based on the ionospheric radio-occultation data collected by CHAMP (from 2001 to 2008), GRACE (from 2007 to 2011) and 120 COSMIC (from 2006 to 2012) satellite missions and ionospheric sounding data collected by 62 Digisondes from 1987 to 2012. SHU uses the spherical harmonics decomposition to model hmF2. Finally, the older BSE uses the correlation between hmF2 and propagation factor M(3000)F2 which in turn is defined by the ratio between the highest frequency that, refracted by the ionosphere, can be detected at a distance of 3,000 km (M(3000)) and foF2. At last, the foE comparison is made using IRI-foE developed by Muggleton (1973a, 1973b) for CCIR (1973) with a modified zenith angle introduced by Rawer 125 and Bilitza (1990) to improve the nighttime variations. Finally, to evaluate the performance of IRI-2016, a correlation analysis is performed between the modelled parameters and the observational data.
In some cases, the results are discussed considering the seasonal differences between the observed and modelled parameters.  Fig. 1(b) and URSI predictions in Fig. 1(c), foF2 CCIR and foF2 URSI, respectively, it is observed a very similar diurnal and seasonal variation patterns as seen in the observed values. However, a first look at the foF2 CCIR-RD in Fig. 1(d) and foF2 URSI-RD in Fig. 1(e) in the bottom panels reveals that the coefficient outputs grossly underestimate/overestimate the foF2 in 150 some hours and months as indicated below.
The foF2 CCIR-RD in Fig. 1(d) ranges from -20 % (underestimation) to 50 % (overestimation). The higher underestimations are observed in September and October from 9:00 UT to 16:00 UT, and later from November to February between 20:00 UT and 22:30 UT. There is also an underestimation of 20 % from April to August at around 10:00 UT. On the other hand, the overestimations are most significant during nighttime hours at almost all months from 23:00 UT to 08:00 UT. The foF2 URSI-RD 155 varies from -15 % to more than 50 %. The most negative deviations are observed only in two small portions of the contour plot in Fig. 1(e), which is around 21:00 UT in October, and from 18:00 UT to 22:00 UT in December. However, significant positive deviations higher than 50 % are seen around 9:00 UT from March to July, and in the nighttime hours around 23:00 UT from February to April. From these results, it seems that the URSI (CCIR) sub-routine overestimate (underestimate) foF2 more than the CCIR (URSI). 160 A more detailed analysis has to be performed to further investigate the level of reliability of each IRI sub-routine. Since the data is not significantly drawn from a normally distributed population at the 0.05 % level, the quantitative estimate can be achieved by analysing the statistical relationship between IRI-foF2 and observed values using the Spearman correlation coefficient (r). The significance of the calculated r-value is examined with a confidence level of 95 % between the hourly values modelled and observed data. The scatter plots of modelled IRI-foF2 using CCIR and URSI coefficients versus the 165 observational data are shown in Fig. 2. The results of the calculated r are 0.97 for both IRI coefficients. It is shown an almost perfect positive correlation.
The contour plots of the monthly averaged hmF2 (in km) observed by the SMK29 and modelled by AMTB, SHU, and BSE sub-routines and the RD (in percent) versus universal time (UT, vertical axis) and month (from September 2017 to August 2018, horizontal axis) are shown in Fig. 3. The color-coded bar on the right-hand side of the upper panels represent hmF2 170 ranging from 180 km to 360 km. In the lower panels, the color-coded bar refers to the RD and ranges ± 50 % for the three plots. The observed hmF2 values in Fig. 3(a) show that the F2-layer is higher during nighttime hours achieving 340 km from September to December from 1:00 UT to around 03:00 UT. There is also more intense hmF2 values between 300 km and 320  Fig. 3(f), and hmF2 BSE-RD in Fig. 3(g). A visual comparative analysis shows that the SHU agrees better with the observations since the RD encompasses, in general, ±10 % most of the time. The same is not true for AMTB 180 and BSE predictions.
The hmF2 AMTB-RD ranges from -10 % to 43 %. The main differences are related to the overestimation of hmF2 most of the time in September, October, and from March to August as represented by the hottest color of the palette. It differs especially near the sunrise period from 7:00 UT to 11:00 UT in the June equinox. The hmF2 SHU-RD varies from -20 % to 20 %. In general, the SHU outputs differ only ±10 % from the observation results revealing very good agreement with the observations. Regarding 185 hmF2 BSE-RD, it ranges from -24 % to 20 %. There are some small periods near sunrise (sunset) that hmF2 is overestimated (underestimated), but in general, BSE also represents well the observations. As shown by the results presented in Fig. 3, SHU and BSE perform better than AMTB in modelling hmF2. This result is also confirmed by the statistical relationship through the Spearman r values shown in Fig. 4. Modelling the hmF2 with the SHU coefficients presents the best scenario with the r = 0.86, as shown in Fig. 4(b). Despite the AMTB in Fig. 4(a) presents the lower correlation (r = 0.72), it is important to 190 note that it is still significant.

Performance of IRI-foE
The contour plots of the monthly averaged foE (in MHz) observed by the SMK29 and modelled by IRI-2016 and the estimated RD (in percent) versus universal time (UT, vertical axis) and month (from September 2017 to August 2018, horizontal axis) are shown in Fig. 5. The foE values are represented by the color-coded bar on the right-hand side, ranging 195 from 1 MHz to 3.5 MHz for the critical frequency in the upper panels, and ± 20 % for the RD in the lower panel.
The observed foE in Fig. 5(a) shows a regular diurnal variation, increasing from sunrise to a peak in the afternoon to around 3.5 MHz, and falling until sunset. The low electron density at night makes it difficult to detect the E-region by the Digisonde.
The most intense values around 3.5 MHz are seen during September equinox and December solstice months. The agreement between IRI-foE and observations is very good as shown in Fig. 5(b). The maxima values seen in IRI occur longer than the 200 observations, however. It is shifted two months (April and May) and it starts earlier (July). The foEIRI-RD in Fig. 5(c) are positive (overestimation) up to 5 % only, confirming the good IRI-2016 performance in modelling foE almost all the time over Santa Maria. There are some considerable differences in a short time in the sunrise and sunset hours. These are critical periods which may be caused by distortions in the E-region traces due to horizontal gradients in the ionosphere making it difficult to be modelled by IRI, as can be expected by the users. The r-value obtained between the modelled and observed 205 values is the highest in this work, showing a very strong positive correlation, as shown in Fig. 6.

Discussion
The focus of this work is to use the foF2, foE and hmF2 measured by the recent Digisonde installed in Santa Maria, Brazil, to test the performance of the IRI-2016 in the low-latitude ionosphere situated close to the center of the SAMA during the geomagnetically quiet days from September 2017 to August 2018. The results presented in Figs. 1 and 2 show that the foF2 210 predictions obtained with CCIR and URSI coefficients are very similar in a month by month analysis. However, CCIR (URSI) fails underestimating (overestimating) foF2 in specific nighttime hours. When the whole period of data is considered, both coefficients gave r equal to 0.97. The correlation is an indication that the model accurately predicts the diurnal and seasonal trends of foF2 over Santa Maria. In general, the IRI user may choose anyone sub-routine to model foF2.
The results obtained in this work closely follow the earlier work of Ezquer et al. (2008), who had compared the CCIR and 215 URSI coefficients with the ionosonde data in Tucumán. They report that, in general, both coefficients give comparable values. However, they also report disagreements among predictions and measurements reaching values of RD close to 50 %.
In the Brazilian sector, Batista and Abdu (2004) in a similar comparative study pointed out that the agreements between the URSI values and the observed foF2 in São Luís were always better as compared to the CCIR coefficients. They also showed that the foF2 after sunset is overestimated for the equatorial station of São Luís. It seems that over the Brazilian territory the 220 right choice between CCIR and URSI in modelling foF2 depends on the location of the users. In the Brazilian equatorial region, CCIR performs better, while in the SAMA region there are no appreciated differences between both. In China,  found that the CCIR performs better than URSI during post-sunset under low solar activity or in the EIA region.
For other time and outside the EIA region over China CCIR shows no large difference in performance as compared to URSI.
Figs. 3 and 4 show that the SHU option for modelling the hmF2 performs better over Santa Maria, followed by BSE and the 225 AMTB is the worst. The r-value of AMTB is 0.72, the lowest observed in the present study. It is even lower than the older BSE coefficient used in the previews versions of the IRI model. Overall, the AMTB (BSE) overestimate (underestimate) the observed values. Therefore, it is recommended the usage of SHU option when modelling the hmF2 over Santa Maria. These results agree with the finds of Zhao et al. (2017), who also recommend the use of SHU option over China region when using IRI-2016 to model hmF2. Since this is the first evaluation of the three IRI-hmF2 options in the Brazilian sector to the author`s 230 knowledge, there is no comparison between our work with others Brazilian equatorial or low latitude regions, and it is suggested as a future study.
Finally, the comparative results presented in Figs. 5 and 6 show that the IRI-predicted foE values are in excellent agreement with observations in Santa Maria. The calculated r-value is 0.99. The strong correlation may be explained by the fact that the E region ionization is subject to solar radiation control, and therefore IRI predicts the E region solar ionization fairly 235 accurately everywhere in the globe since there is no plasma transport in the E region.

Conclusions
The present work uses the foF2, foE and hmF2 parameters acquired by a recent Digisonde installed in Santa Maria, Brazil, close to the center of the SAMA, to test the performance of the IRI-2016. Only data collected under quiet conditions from September 2017 to August 2018 are used to eliminate the effects of geomagnetic disturbances. Monthly average values of 240 the observed ionospheric parameters are calculated from the daily hourly values and compared with the IRI-2016 predictions for the same geophysical conditions. The Relative Deviation (RD) had been computed using the CCIR and URSI coefficients to estimate the IRI-foF2 performance. The IRI-hmF2 predictions are evaluated using the RD estimated using the three options AMTB, SHU, and BSE. The IRI-foE performance is also tested. The main findings of the study are as follows: a) CCIR and URSI predictions represent the diurnal and seasonal variation patterns of the observed values. foF2 CCIR-RD 245 ranges from -20 % (underestimation) to 50 % (overestimation). The higher underestimations are observed in September and October from 9:00 UT to 16:00 UT, and later from November to February between 20:00 UT and 22:30 UT. There is also an underestimation of 20 % from April to August at around 10:00 UT. The overestimations are most significant during nighttime hours at almost all months from 23:00 UT to 08:00 UT. The foF2 URSI-RD varies from -15 % to more than 50 %. The most negative deviations are observed at around 21:00 UT in October, and from 250 18:00 UT to 22:00 UT in December. Significant positive deviations higher than 50 % are seen around 9:00 UT from March to July, and in the nighttime hours around 23:00 UT from February to April. b) SHU agrees better with the observations than AMTB and BSE for modelling hmF2. The hmF2 AMTB-RD ranges from -10 % to 43 %. The main differences are related to the overestimation of hmF2 most of the time in September, October, and from March to August. It differs especially near the sunrise period from 7:00 UT to 11:00 UT in June 255 equinox. The hmF2 SHU-RD varies from -20 % to 20 % and, in general, differ only ±10 % from the observation. results revealing very good agreement with the observations. hmF2BSE-RD ranges from -24 % to 20 %. There are some small periods near sunrise (sunset) that hmF2 is overestimated (underestimated), but in general, BSE also represents well the observations. c) The agreement between IRI-foE and observations are very high. However, the maxima values seen in IRI occur 260 longer than the observations and it is shifted two months (April and May) and it starts earlier (July). The foEIRI-RD webpage from INPE Space Weather Program in the following link: http://www2.inpe.br/climaespacial/portal/en/. The authors acknowledge the support of the Federal University of Santa Maria (UFSM) Central Administration.