Comparison of quite time ionospheric total electron content from IRI-2016 model and GPS observations

Abstract. Earth’s ionosphere is an important medium of radio wave propagation in modern times. However, the effective use of ionosphere depends on the understanding of its spatio-temporal variability. Towards this end, a number of ground and space-based monitoring facilities have been set up over the years. This is also complemented by model-based studies. However, assessment of the performance of the ionospheric models in capturing observations needs to be conducted. In this work, the performance of IRI-2016 model in simulating total electron content (TEC) observed by network of global position 5 System (GPS) is evaluated based on RMSE, bias, correlation and categorical metrics such as Quantile Probability of Detection (QPOD), Quantile False Alarm Ratio (QFAR), Quantile Categorical Miss (QCM), and Quantile Critical Success Index(QCSI). IRI-2016 model simulations are evaluated against GPS-TEC observations during the solar minima 2008 and maxima 2013. Higher correlation, low RMSE and bias between the modeled and measured TEC values are observed during solar minima than solar maxima. The IRI-2016 model TEC agrees with GPS-TEC strongly over higher latitudes than over tropics in general 10 and EIA crest regions in particular as demonstrated by low RMSE and bias. However, the phases of modeled and simulated TEC agree strongly over the rest of the globe with the exception of the polar regions as indicated by high correlation during all solar activities. Moreover, the performance of the model in capturing extreme values over magnetic equator, midand highlatitudes is poor. This has been noted from a decrease in QPOD, QCSI and an increase in QCM and QFAR over most of the globe with an increase in the threshold percentile values of TEC to be simulated from 10% to 90% during both solar minimum 15 and maximum periods. The performance of IRI-2016 in correctly simulating observed low (as low as 10 percentile) and high (high than 90 percentile) TEC over EIA crest regions is reasonably good given that IRI-2016 is a climatological model despite large RMSE and positive model bias. Therefore, this study reveals the strength of the IRI-2016 model, which was concealed due to large RMSE and positive bias, in correctly simulating the observed TEC distribution during all seasons and solar activities for the first time. However, it is also worth noting that the performance of IRI-2016 model is relatively poor in 2


Introduction
Radio wave has turned into indispensable and spectacular means in the progress of space satellite communication and navigation.Earth's ionosphere is an essential medium for the propagation of radio wave signals (Mengistu andAbraha, 2014, 2010;AghaKouchak et al., 2011).
Therefore, this paper focus on the comprehensive global validation of IRI-2016 model on monthly, seasonal and annual TEC variations based on observed TEC by network of ground-GPS receivers run by International Global Navigation Satellite System(GNSS) Service (IGS) using the common statistical metrics and the quantile-based categorical metrics.To our knowledge, there is no comprehensive and global evaluation of IRI-2016 that includes detailed analysis at the tails of TEC distribution.
The paper is organized such that Section 2 highlights data and methodologies employed for validation of IRI-2016 TEC data against IGS GPS TEC.Section 3 covers results and discussion while Section 4 provides conclusion.
2 Data and methodology 2.1 Data

GPS
The TEC data extracted at a grid resolution of 5 0 latitude by 5 0 longitude from IGS, hereafter referred to as GPS-TEC for solar minimum 2008 and solar maximum 2013 are used.The data is processed using 422 ground-based GPS receivers (see Fig. 1 and Table 1) from 32 GPS satellites for the solar minima 2008 and solar maxima 2013 as shown in Table 1.TEC is measured by GPS signals through integration of the electron density profile.The differential phase, ∆Φ, of the two waves on L1 and L2 bands of dual frequency GPS can be used to determine TEC according to procedure described by many authors (Bossler et al., 1980;Melbourne et al., 1994;Morgan and Johnston, 1995;Axelrad et al., 1996;Komjathy, 1997;Schreiner et al., 1999;Parssinen et al., 1999;Hajj et al., 2000;Woo, 2000;Hajj et al., 2002;Oloufa et al., 2003;Borghetti et al., 2006;Hoffmann and Jacobi, 2006;Hernandez et al., 2011;Pradhananga and Teizer, 2013).
GPS data is filtered using Dst data such that days with geomagnetic storms are excluded from the comparison since it has been indicated in several other studies that IRI models are insensitive to the storm option and fails to reproduce observed TEC on storm days (e.g., Asmare et al., 2014;Tariku, 2015).TEC data is simulated using IRI-2016 as function of universal time and geographical grids that matches the spatio-temporal grids of observed IGS GPS-TEC for the two selected years.The model is configured such that the URSI and NeQuick2 options for F-peak model and for the top-side profile estimation have been considered in this study.Furthermore, the newly added Shubin-Cosmic model for hmF2 and ABT-2009 option for the bottom side thickness shape parameter are considered.Moreover, the storm related models were set to off.The Shubin-Cosmic model was developed with a large amount of radio occulation (RO) data from CHAMP, GRACE and COSMIC and with hmF2 data from 62 digisondes for the years 1987-2012 from the Digital Ionogram Data Base [http://ulcar.uml.edu/DIDBase/](Shubin et al., 2013;Shubin, 2015;Bilitza et al., 2017, and references therein).Moreover, the historical development of IRI and details of the recent IRI-2016 model are given by Bilitza et al. (2017).The Dst index represents the axially symmetric disturbance magnetic field at the dipole equator on the Earth's surface.Major disturbances in Dst are negative, namely decreases in the geomagnetic field.Therefore, days with Dst greater than -30 nT are assumed to be quite days and therefore included in the comparison of IRI-2016 and GPS-TECs.The two-hourly Dst data is obtained from http://www.wdc.kugi.kyoto-u.ac.jp/dstdir/ for the two years.systematic error (Bias) and pattern correlation between them for the selected years.RMSE, which is the square root of the mean of all errors, indicates the deviation between simulated and observed data.It is given as or in terms of individual standard deviations (variances) of the simulations (σ S ), observations (σ O ) as well as bias and correlation (R) between the two data sets: The systematic error (Bias) discloses the mean difference between the simulated (IRI-TEC) and measured (GPS-TEC) data: where S i and O i are simulated and observed total electron content values respectively and n is the total number of data points for comparison.
The correlation coefficient is derived from the covariance of the simulated and observed variables divided by the product of their standard deviations.Moreover the correlation coefficient (R) is an implication of how much both the spatial and temporal patterns in the IRI-2016 Model match the IGS-GPS observations (Murphy, 1998;Taylor, 2001;Daniel., 2006;Ochoa et al., 2014): 2.2.2 Categorical Statistics: Quantile Probability of Detection (QPOD), Quantile False Alarm Ratio (QFAR), Quantile Categorical Miss (QCM) ,Quantile Critical Success Index (QCSI) Categorical statistics employed in this study aim to evaluate the extent to which the simulation captures the distribution of the observed GPS-TEC above certain selected thresholds.As IRI model is empirical model based mainly on past observations, it is natural to expect that its performance at the extreme ends of the observed distribution may suffer from inaccuracies.However, the extent of this discrepancy at the extreme ends of the observed TEC distribution is not fully assessed.Therefore, categorical where H and M stand for hit and miss rates respectively.H and M are given in terms of t, OBS i and SIM i as follows: A perfect detection signifies that the miss rate is zero implying that QPOD equals 1.In contrast a model with no skill has zero hit rate which suggests a QPOD value of zero.Therefore, QPOD attains a value of 0 for no skill and 1 for perfect score (Behrangi et al., 2011;AghaKouchak and Mehran, 2013).
The QFAR quantifies TEC above the selected threshold detected by simulation but not available in observations.The QFAR covers from 0 to 1; 0 signify perfect score (Brown et al., 2004;AghaKouchak and Mehran, 2013): where F stands for false alarm rate and is given as The QCM may be defined as 1 -POD which ranges from 0 to 1, with 0 being the perfect score.QCM can be given specifically in terms of hit and miss rate as: The QCSI combines various features of the QPOD and QFAR, to determine the total skill of the simulation relative to observation as a function of H, F and miss rate (M): The QCSI ranges from 0 (no skill) to 1 (perfect skill) (Davis et al., 2009;AghaKouchak and Mehran, 2013).For example, a QCSI of 0.7 indicates that the simulation detects 70% of observed TEC above certain percentiles.
Both of these numerical (continuous) and categorical statistics are used to assess the model skill in capturing the individual observations for each calendar months and seasons for the solar minimum and maximum years.to 11.9 TECU.In contrast, in 2013, the RMSE outside tropics ranges from 0.5 to 11.9 TECU and it varies from 4.3 to 23.3 TECU over tropics in 2013 implying IRI-2016 model exhibited poor performance in capturing observed GPS-TEC over the same region during the 2013 solar maximum period as demonstrated by very high RMSE (Fig. 3).The difference between the model and GPS TEC over the EIA crest regions is much higher than the rest of the globe as noted from high RMSE during both solar activity years (Figs.2-3).Moreover, the summer hemispheres experience higher RMSE than winter hemisphere with high (low) RMSE from May to August over northern (southern) hemisphere and vice verse from October to January during solar maximum period of 2013 (Fig. 3).There is no similar apparent hemispheric differences with seasons during solar minimum 2008 period.
The IRI-2016 TEC is low biased up to -7 TECU over most of the globe with respect to GPS-TEC during 2008.A positive bias is notable over EIA crest regions throughout the whole period in 2008.However, maximum positive bias in IRI TEC with respect to GPS TEC is observed over EIA crest regions from August to December (Fig. 2).The bias along EIA crest region shows longitudinal variation with most of the peaks located in African and American longitude sectors.The IRI TEC over the Asian longitude sector is low biased along the EIA crest region during most of the months (Fig. 2).This is consistent with previous investigation by Kenpankho et al. (2011) who found that IRI-2007 underestimates GPS-TEC with a maximum difference of 15 TECU during day times and a minimum variation of 5 TECU during night times over an equatorial region in Thailand.Grynyshyna-Poliuga et al. (2015) have also shown that the TEC derived from the IRI-2012 model over mid-latitude station, Warsaw, was generally low biased with respect to the GPS-TEC.The maximum differences are about 10 TECU during the daytime and 2 TECU during the nighttime.As noted by other authors (e.g., Akala et al., 2015, and references therein) contribution from the plasmasphere above 2000 km in GPS-TEC might have contributed to the discrepancy.For example, Akala et al. (2015) have found that the contribution of Plasmasphere electron content to GPS-TEC is maximum during the December solstice and minimum during the June solstice.Moreover, the authors noted that plasmaspheric TEC contribution to GPS-TEC is varying with respect to solar activity.The discrepancy between the IRI-2016 TEC simulations and GPS-TEC can not be fully attributed to the plasmasphere TEC since positive biases in IRI-2016 TEC are evident along some longitude sectors of EIA crest regions.This positive bias with maximum (up to 16.5 TECU) during day and minimum during night time was also reported by Wan et al. (2017) at four stations in China covering the EIA crest region.However, this modest discrepancy is not the case in 2013 during solar maximum period when wide spread negative bias in IRI TEC of up to -18 TECU in May in the northern hemisphere, and during November and December in the southern hemisphere was observed.Moreover, negative bias was also prevalent over most of northern hemisphere from June to August and over most of southern hemisphere from October to December and in January.In contrast the period from July to September is characterized by positive bias in IRI TEC over most of southern hemisphere (Fig. 3).The weak performance of IRI model during solar maximum is also reported by Other studies with focus on high latitudes have shown similar weakness in IRI model.For example, IRI is also shown to significantly underestimate the magnitude of solar cycle variations in TEC and underestimate monthly median TEC at high solar activity by as much as 15 TECU (Themens and Jayachandran, 2016) which are greatest during the equinoxes and significant during summer periods but are lowest during winter median TEC.These asymmetries suggest that the IRI-2016 has weakness to capture enhanced TEC during summer of each hemisphere when the sun is overhead in each hemisphere at time of maximum solar activity.This is partly inherent in its nature as IRI is based on mean observations to develop empirical formulations.
In contrast to simulation of the actual magnitude of TEC (as demonstrated by high negative bias), IRI-2016 performs well in capturing the phase of variation of TEC irrespective of seasons and the nature of solar activity as demonstrated by high correlation with GPS-TEC during both periods (2008 and 2013) (see Figs. 2-3).However, IRI-2016 exhibits weak performance over high latitudes (low correlation of up to 0.17 or negative correlation in some months) as compared to tropics and midlatitude with correlation as high as 0.98.In addition there is also evidence of poorer correlation between simulated IRI TEC and observed GPS TEC over high latitudes in south-western hemisphere during solar maximum 2013 than solar minimum 2008 periods (see Figs. 2-3).   of the model during summer of each hemisphere at these longitudes.However data points from EIA crest regions along these longitudes have also contributed to the high bias and RMSE as well as low correlations.The performance of the model in capturing daytime TEC is better than nighttime TEC (not shown).The presence of large scatter at lower and higher ends of the distribution is common during both solar activity years.At both ends, IRI-TEC is low biased (See Fig. 4).The level of weakness in IRI-2016 at these parts of TEC distribution is further assessed in Sections 3.2-3.3using quantile-based categorical metrics.

Comparison of IRI-2016 simulation and GPS-TEC observations on a seasonal basis
In previous sections, the comparisons were based either on individual data within a given calendar month (Section 3.1.1)or the whole year (Section 3.1.2).However, as we have noted in these sections, there is indication that the model performance is in Fig. 5 for 2008 and in Fig. 6 for 2013 period respectively.
In 2008 (solar minimum period), the RMSE is generally higher during March and September equinoctial months than other seasons over tropics (Fig. 5).However, pronounced positive biases mainly along EIA crest regions are noted during September   3.2 Categorical Statistics: QPOD, QFAR, QCSI, QCM

Categorical comparison of IRI-2016 simulation and GPS-TEC observations
As noted in Section 3.1.2with the scatter plots at lower and upper ends of TEC distribution, most of the deviations from observations arise at these ends.Therefore, there has been efforts to understand these discrepancies.For instance, Venkata Ratnam distribution.Therefore, QPOD, QFAR, QCSI and QCM are employed in this section to assess the performance of IRI-2016 against GPS-TEC observations.Fig. 7 shows QPOD, QFAR, QCM, and QCSI for TEC values exceeding 10, 25, 75 and 90 percentiles for 2008.The notable feature in Fig. 7 is the decrease in QPOD as the percentile increases from (Fig. 7a) to (Fig. 7m) over mid-and polar latitudes suggesting that the model skill decreases as the high extremes are dominant parts of TEC values in the evaluation of the metrics.However, these trends in quantile metrics are reversed with increase in threshold percentiles from 10% to 25% implying model performs also weakly at the low extreme.Similar changes in values of metrics that measure model skill is observed during 2013 (see Fig. 8).Consistent with this, QCM (Fig. 7c, 7g, 7k and 7o) exhibits increasing trend with percentile increase as expected.In contrast, QCSI increases global as the percentile changes from 10% (Fig. 7d) to 25% (Fig. 7h) globally.Moreover, QCSI begins to decrease with a change from 25% (Fig. 7h) to 90% (Fig. 7p) globally.The difference in pattern between QPOD and QCSI is attributed to the fact that the false alarm rate has increased with increase in percentile threshold from 25% (Fig. 7f) to 90%(Fig.7n).In particular, the increase in false alarm rate of detection of observations by the simulation over the EIA crest region is quite evident with a shift from 10 th percentile to 90 th percentile in 2008.This false model skill has been removed in QCSI as opposed to QPOD which shows high model skill.Similar patterns of metrics that measure model skill is observed during 2013 (Fig. 8).However, the rate of decrease from 25 to 90 percentile is significantly higher than those in 2008 in particular over southern hemisphere.For example QCSI drops from 1 (perfect skill) for 10 percentile to about 0.4 for 90 percentile over tropics covering EIA crest region.This is consistent with results of Section 3.1 that shows weakness in the IRI-2016 model during enhanced solar activity.Therefore we noticed here generally the IRI-2016 model has better agreement with GPS during solar minima 2008 than solar maxima 2013 at the extreme margins of TEC distribution.
There are also some major notable characteristics that can be highlighted from individual categorical metrics.QPOD varies from 0.4 to 0.8 over tropics at 10 th percentiles with maximum mainly along the northern EIA crest regions during 2008 solar minimum year (Fig. 7a).At 25 th percentile (Fig. 7e), the change in QPOD from 10 th percentile is notable only over the northern EIA crest region (increase from 0.8 to 1).However, at the 75 th percentile (Fig. 7i), significant drop in QPOD ( below 0.2) over northern mid and high latitudes, magnetic equator along Asian sector and southern polar region is observed.At the same time, the model skill in capturing observed TEC over southern mid-latitude and tropical ionosphere is enhanced with QPOD values in the range of 0.7 to 1.This skill of the model at 90 th percentile has persisted over EIA crest regions with drop in QPOD elsewhere (Fig. 7m).The overall patterns of QCM ( i.e., low over tropics except magnetic equator versus high over high latitudes) and QCSI( i.e., high over tropics except magnetic equator versus low over high latitudes) at the 75 th and 90 th percentile threshold levels are consistent with QPOD and QFAR changes at this threshold level.The difference in spatial patterns between these metrics is hardly apparent except at transition zones between mid-and high latitudes during 2008 solar minimum year.In contrast to 2008, the model performed very well in capturing observed TEC above 10 th percentile (Fig. 8a) in 2013 over most of the ionosphere as evidenced by high QPOD above 0.8.However, the model skill continued to be poor over southern parts of North America and Europe ionosphere as well as over areas from Australia to South Africa.The model skill improved with increase in percentile thresholds (10% (Fig. 8a) to 25% (Fig. 8e)) globally in 2013.At 75 th (Fig. 8i) and 90 th (Fig. 8m) percentiles, the skill of the model deteriorated over tropics with the exception of a few longitude sectors along northern EIA crest region from its performance at 25 th percentile.The model is also deteriorated further over magnetic equator, midand high latitudes (Fig. 8).Other metrics (i.e., QFAR, QCM and QCSI) portray similar features.remained the same at 25 th percentile (Fig. 9b).However, QPOD increased to values exceeding 60% over the rest of tropics and mid-latitudes.IRI-2016 captures more than 85% of observed TEC exceeding 75 th and 90 th percentiles over the EIA crest region and exhibits a steady drop (to less than 20%) in skill over the rest of the globe.Specifically, the change in QPOD along geomagnetic equator from values exceeding 80% at the 10 th percentile to values less than 20% at the 75 th percentile is signif-  improves to a value exceeding 80% over EIA crest regions while it exhibits decrease over both northern and southern mid-and high latitudes.The changes over the southern mid and high latitudes during this season at the 75 th and 90 th percentiles appear to be a mirror reflection of June solstice in the northern hemisphere.This similarity in model detection skill during the two solstices is also apparent at the lower ends of TEC distribution.Unlike June solstice, the higher skill score covered most of the southern hemisphere(see Fig. 9m, left).
In 2013 during solar maximum year, the QPOD characteristics are similar to that of 2008 for all the seasons but with notable improvement at the lower ends for the two equinoctial seasons (see Fig.  Fig. 9d,l,right).This suggests that the observed TEC distribution has slightly shifted towards higher values relative to 2008 as a whole which is consistent with the high solar activity.This conclusion follows from the fact that any improvement in model performance arises from the nature of the observed TEC distribution rather than the model itself since the model configuration remained the same.This changes in skill is also apparent within the same year from one season to the other as noted in previous paragraph.Table 5 summarizes the changes in QPOD with season
Statistics: RMSE, Bias and Correlation Monthly comparison of TEC from IRI-2016 Model and GPS measurements are evaluated with root mean square error (RMSE), Ann. Geophys.Discuss., https://doi.org/10.5194/angeo-2019-44Manuscript under review for journal Ann.Geophys.Discussion started: 26 March 2019 c Author(s) 2019.CC BY 4.0 License.statistics such as QPOD, QFAR, QCM, and QCSI are employed to evaluate the performance of IRI-2016 model in simulating the whole spectrum of observed distributions from low to high extreme TECs.The QPOD defines part of the observations (OBS) above selected percentile threshold (t) identified accurately by the simulation (SIM).It is given by Fig.2shows RMSE, Bias and R between IRI-2016 TEC simulations and GPS-TEC observations for all the 12 months during solar minimum 2008 period.Fig.3shows also RMSE, Bias and correlation as Fig.2but for the solar maximum period 2013 using same color scale.The RMSE in Fig.2is in the range of 0.5 to 11.9 TECU during 2008.The range of RMSE in 2013 (0.5 to 23.3 TECU) is much higher than that of 2008.Moreover, the RMSE of IRI TEC with respect to GPS TEC during 2008 is within 0.5 to 4.3 TECU over most of the globe with the exception of tropics which exhibits RMSE in the range of 4.3

Fig. 4
Fig.4shows scatter plots of TECs from IRI-2016 versus IGS GPS at 4 selected longitudes.It has been already noted in Section 3.1.1that performance of the IRI-2016 model degrades with high solar activity and summer season of each hemisphere.The IRI-2016 TEC at the four selected longitudes is low biased (1.7 to 1.9 TECU) against GPS-TEC during 2008 (Fig.4, four left panels).In contrast, this bias has increased to values ranging from 3.7 to 4.5 TECU (four right panels of Fig.4) during 2013 consistent with analysis on the monthly time scale for the whole globe.The pattern observed in bias is similar for RMSE which increased from lowest value of 1.9 TECU in 2008 to highest value of 5.6 TECU in 2013.Similarly, the correlation dropped from highest 0.97 in 2008 to lowest 0.93 in 2013.Much of the discrepancies are attributed to weaker performance
et al. (2017) have included relative TEC deviation index, monthly variations in the grand mean of ionospheric TEC, TEC in-5 tensity, the upper and lower quartiles in their comparison of GPS-TEC with IRI-2007 and IRI-2012 predicted TECs.Although the inclusion of lower and upper quartiles is a step in the right direction to understand the discrepancy in these parts of the distribution, much of the observed differences lie in the extreme ends within the quartiles.Therefore, application of quantile categorical statistics is necessary for more insight into the problem as indicated in Section 3.2.Since much of the data used to constrain the IRI-model represent mainly the mean values, the model is likely to under perform at the extreme ends of TEC 10 15 Ann.Geophys.Discuss., https://doi.org/10.5194/angeo-2019-44Manuscript under review for journal Ann.Geophys.Discussion started: 26 March 2019 c Author(s) 2019.CC BY 4.0 License.
9a-b and Fig. 9i-j, right).In contrast to 2008, the model detection skill at the 75 th and 90 th percentiles has weakened over the EIA crest regions.Instead, improved performance of IRI-2016 model can be seen over most of southern and northern hemispheres during December and June solstices respectively (Fig. 9g-h and Fig. 9o-p, right).Unlike the solstices, during March and September equinoctial months, the performance at the 75 th percentile is good across broader areas along EIA crest regions and hemispherically symmetric (Fig. 9g,h, right).At the 90 th percentile, the model performance is very bad over most parts of the globe during March Equinox and reasonably good over northern mid-latitude during September Equinox (
. Discuss., https://doi.org/10.5194/angeo-2019-44Manuscript under review for journal Ann.Geophys.Discussion started: 26 March 2019 c Author(s) 2019.CC BY 4.0 License.and lack of plasmaspheric TEC in the IRI model according to several past studies.It is also evident that the IRI-2016 model captures the phase of TEC variation with great accuracy as revealed by high correlations over most of the globe.The scatter plots at selected longitude sectors indicate that IRI-2016 model is low biased at both low and high tails of the TEC distribution suggesting that IRI-2016 is capable of satisfactorily simulating the mean TEC globally.The extent of the IRI model weakness and strength at the extreme portions of observed TEC are assessed using categorical statistical metrics such as QPOD, QCSI, QFAR and QCM using 10 th and 25 th percentiles as lower margin and 75 th and 90 th percentiles as upper margins of the TEC distribution for the two distinct solar activity periods.The performance for the whole annual time series and seasonal time series were evaluated using these thresholds.The model has generally reasonable skill at the low ends of TEC distribution over most of the globe.This skill weakens at high ends of the TEC distribution over much of the globe except EIA crest regions during both solar activity years.There is also hemispheric symmetry during June and December solstices with poorer performance over the summer hemisphere at the high extremes of observed TEC.This feature is consistent with high RMSE and low bias in model during summer as compared to winter time.Similarly, the robust skill at low ends of observed TEC distribution can be attributed to the fact that low TECs that constitute the low portion of TEC distribution are mainly observed during night time while those at the high ends of the distribution occur during daytime.In summary, the IRI-2016 model as a climatological empirical model have simulated significant portion of observed TEC with better accuracy during both solar activity periods and different seasons.The model performance at the extreme ends of the distribution is also remarkably good.In particular, the IRI-model skill in detecting observed TEC over EIA crest regions at the extreme ends is robust despite high RMSE for the whole TEC distribution.Therefore, this encouraging IRI-2016 model performance at the extreme parts of observed TEC distribution suggests the importance of further work to improve the model so that it can be used for real time operational forecasting.Competing interests.There is no conflict of interests.Ann.Geophys.Discuss., https://doi.org/10.5194/angeo-2019-44Manuscript under review for journal Ann.Geophys.Discussion started: 26 March 2019 c Author(s) 2019.CC BY 4.0 License.

Table 1 .
GPS stations and satellites across the world.
Table 2 shows the global average performance of IRI-2016 with respect to GPS TEC observations in 2008 and 2013.The table also includes global maximum and minimum RMSE, bias and correlation.The lowest RMSE of 0.48 TECU and 0.74 TECU in July are observed during 2008 and 2013 respectively.Similarly, highest RMSE of 11.63 in October and 24.27 TECUs in November are determined during 2008 and 2013 respectively.The global average RMSE varies from 2.16 TECU in July to 3.64 TECU in March during solar minimum period of 2008.In contrast, in 2013, the mean RMSE varies from lowest value of 4.78 TECU in July to highest value of 9.39 TECU in December indicating the model performance is poorer in 2013 than in 2008 with significant seasonal variation in the IRI-2016 model skill.The globally averaged bias varies from -2.23 TECU in April to -0.76 TECU in October in 2008.However, it varies from 0.04 TECU in September to -6.4 TECU in December in 2013 (see Table2).The spatial mean of seasonal correlation varies from 0.78 in January and December to 0.91 in September in 2008 while it varies from 0.80 to 0.88 in 2013 through January to December.

Table 2 .
Statistical parameters for monthly comparisons of solar minima 2008 and maxima 2013.
3.1.2Comparison of IRI-2016 simulation and GPS-TEC observations at selected longitude sectors

Table 3 .
Statistical parameters for seasonal variations of solar minima 2008 and maxima 2013.
Solstice (See Table3).The correlation is within a range of -0.1 to 0.98 with a spatial mean value of 0.8 to 9.2 in 2008.The lowest range of correlation (0.75 to 0.98) is during September Equinox in 2008.Similarly the lowest range of correlation

Table 4 .
Categorical comparison of statistical parameters for solar minima 2008 and maxima 2013.
Table4summarizes global minimum, maximum, and mean of QPOD, QFAR, QCM and QCSI at all percentile levels for the two periods under study.The lowest skill as demonstrated by QPOD and QCSI of zero is noted at the 75 th and 90 th percentiles in 2008 and 2013.The highest QFAR and QCM of one are attained at 90 th percentile during 2008 and at 75 th percentile during 2013.The mean QPOD varies from 0.29 at 90 th to 0.71 at 10 th percentiles during 2008.In contrast, QPOD varies from 0.13 at 90 th to 0.89 at 10 th percentiles.These features are also exhibited by QCSI as it varies from global mean of 0.19 and 0.10 at 90 th to 0.71 and 0.87 at 10 th percentiles during 2008 and 2013 respectively.The global mean QFAR increases from 0 at 10 th to 0.36 and 0.43 at 90 th percentiles during 2008 and 2013 respectively.Similarly, global mean QCM increases from 0.29 and 0.11 at 10 th to 0.71 and 0.87 at the 90 th percentiles in the same order (see Table4).

Table 6 .
Statistical parameters of QCM for all seasonal variations of solar minima 2008 and maxima 2013.