D.: The association of coronal mass ejections with their effects near the earth

Abstract. To this day, the prediction of space weather effects near the Earth suffers from a fundamental problem: The radial propagation speed of "halo" CMEs (i.e. CMEs pointed along the Sun-Earth-line that are known to be the main drivers of space weather disturbances) towards the Earth cannot be measured directly because of the unfavorable geometry. From inspecting many limb CMEs observed by the LASCO coronagraphs on SOHO we found that there is usually a good correlation between the radial speed and the lateral expansion speed Vexp of CME clouds. This latter quantity can also be determined for earthward-pointed halo CMEs. Thus, Vexp may serve as a proxy for the otherwise inaccessible radial speed of halo CMEs. We studied this connection using data from both ends: solar data and interplanetary data obtained near the Earth, for a period from January 1997 to 15 April 2001. The data were primarily provided by the LASCO coronagraphs, plus additional information from the EIT instrument on SOHO. Solar wind data from the plasma instruments on the SOHO, ACE and Wind spacecraft were used to identify the arrivals of ICME signatures. Here, we use "ICME" as a generic term for all CME effects in interplanetary space, thus comprising not only ejecta themselves but also shocks as well. Among 181 front side or limb full or partial halo CMEs recorded by LASCO, on the one hand, and 187 ICME events registered near the Earth, on the other hand, we found 91 cases where CMEs were uniquely associated with ICME signatures in front of the Earth. Eighty ICMEs were associated with a shock, and for 75 of them both the halo expansion speed Vexp and the travel time Ttr of the shock could be determined. The function Ttr=203-20.77*ln (Vexp) fits the data best. This empirical formula can be used for predicting further ICME arrivals, with a 95% error margin of about one day. Note, though, that in 15% of comparable cases, a full or partial halo CME does not cause any ICME signature at Earth at all; every fourth partial halo CME and every sixth limb halo CME does not hit the Earth (false alarms). Furthermore, every fifth transient shock or ICME or isolated geomagnetic storm is not caused by an identifiable partial or full halo CME on the front side (missing alarms).


Introduction
Our modern hi-tech society has become increasingly vulnerable to disturbances from outside our Earth system, in particular to those initiated by explosive events on the Sun.The economic consequences are enormous (see, e.g.Siscoe, 2000).That's one reason why space weather and its predictability have recently attained major attention, not only with the involved scientists but also with the general public.Another reason is the new quality of observational data that have been obtained over the last decade from a new generation of space-based instruments.They have allowed major advances, among others, in the understanding of the processes involved near the Sun, in interplanetary space, and in the near-Earth environment (see, e.g. the review by Crooker, 2000).
Unfortunately, the preciseness of predictions of space weather effects is still poor.Solar energetic transients, i.e. flares and coronal mass ejections (CMEs), occur rather spontaneously, and we have not yet identified unique signatures that would indicate an imminent explosion and its probable onset time, location, and strength.The underlying physics is not yet sufficiently well understood.Solar energetic particles, accelerated to near-relativistic energies during major solar storms arrive at the Earth's orbit within minutes (see, e.g.Garcia, 2004) and may, among other things, severely endanger astronauts on the way to the Moon or Mars.But we have no appropriate warning tool yet! 1 Figure 1.The X17 flare and the halo CME of 28 October 28, 2003, as seen by the SOHO instruments EIT (a), LASCO C2 (b) and C3 (c).This most dramatic event caused major disturbances of space weather and affected the Earth system in various ways: charged particles were accelerated to near-relativistic energies, such that they could penetrate spacecraft skins and disturbed, e.g., the CCD cameras on SOHO (see c) and other satellite instrumentation, an extremely fast interplanetary shock wave was initiated that reached the Earth only 19 hours later, a severe geomagnetic storm (D st -363 nT) was launched, with bright aurora even all over Europe and the US.That was about 7 hours later than predicted using the tool described in this paper.(c).This most dramatic event caused major disturbances in space weather and affected the Earth's system in various ways: charged particles were accelerated to near-relativistic energies, such that they could penetrate spacecraft skins and disturbed, for example, the CCD cameras on SOHO (see c) and other satellite instrumentation; an extremely fast interplanetary shock wave was initiated that reached the Earth only 19 h later; a severe geomagnetic storm (D st −363 nT) was launched, with bright aurora even all over Europe and the US.This was about 5 h earlier than predicted using the tool described in this paper.
Even once a notable outbreak has actually been observed, it is still hard to predict whether the ejected gas clouds will reach the Earth at all, at which time and what their effects will be.(The interplanetary counterparts of CMEs are often called "ICMEs".In this paper, we use this term in a more generic way, such that it comprises all CME effects in interplanetary space, i.e. not only the ejecta themselves but transient shocks as well.)This is the main aspect to be addressed in this paper.
Of crucial importance is to determine the direction in which an eruption is originally pointed, since only one out of ten CME events hits the Earth (see, e.g.Webb andHoward, 1994 andSt. Cyr et al., 2000).Space-based coronagraphs keep providing spectacular views of erupting gas clouds (see, e.g. the Web page http://star.mpae.gwdg.de/release/movieg.html) but they show just their projections onto the plane of the sky and do not allow to infer the ejection direction.CMEs pointed along the Earth-Sun line appear as "halos" around the occultor disk in a coronagraph's field-of-view (Howard et al., 1982).Complementary disk observations are required for deciding whether a halo CME is pointed towards or away from the Earth.The value of such concerted observations has been demonstrated ever since the set of modern solar telescopes on the Solar and Heliospheric Observatory (SOHO) spacecraft went into operation in early 1996 (Plunkett et al., 1998;Thompson et al., 1998).The disk images taken by the EUV Imaging Telescope (EIT, Delaboudinière et al., 1995;Delannée et al., 2000) at a sufficiently high time cadence allow almost continuous detection of flare explosions and filament eruptions.Simultaneously, the instruments C1 (no longer operating since June 1998), C2, and C3 of the Large Angle and Spectrometric Coronagraph (LASCO, Brueckner et al., 1995) observe the corona above the solar limb in a range from 1.1 R s from Sun center out to 32 R s , with unprecedented sensitivity.In Fig. 1 we show a typical example of a halo CME as observed by EIT, C2 and C3.Such images and animated sequences plus data from other instruments both in space and on the ground are available on the Internet almost in real time (see, e.g.http://sohowww.estec.esa.nl).Many professional and amateur space weather analysts are routinely taking advantage of their sites (e.g.http://www.sec.noaa.gov/today.html, http://sidc.oma.be,http://www.estec.esa.nl/wmwww/wma/spweather/, http://www.spaceweather.com/,http://dxlc.com/solar/).
Of particular value are the messages the LASCO operations team issues for any halo CME as soon as it appears (http://lasco-www.nrl.navy.mil/halocme.html).Some characteristic data, such as exact timing, angular span, front speed (also called "plane of the sky speed") and position angle of the fastest feature, any potential association with flares and other events seen by EIT, are also given, and images and movies are made accessible through the Internet: ftp://ares.nrl.navy.mil/pub/lasco/halo.Detailed information on all CMEs, including halos, is listed under http://cdaw.gsfc.nasa.gov/CMElist/.
The mere knowledge of a halo's general direction means an enormous step forward.The "away" events that are usually not geo-efficient can now be revealed right away.However, for the "toward" events the arrival times at Earth remain hard to predict, since the line-of-sight speed of a halo CME cannot be measured directly.
The data catalogs show that the CME front speeds vary widely between 200 km/s and more than 2000 km/s, i.e. by a factor of 10.However, for those cases with a uniquely recognizable arrival of ICME effects at Earth, the travel speeds vary by not more than a factor of 3 (see, e.g.Cane et al., 2000;Gopalswamy et al., 2000).Apparently, the faster ICMEs are significantly decelerated (Woo et al., 1985, see also, e.g.Watari and Detman, 1998, and references therein), and the slower ones are post-accelerated in the ambient solar wind flow (Lindsay et al., 1999b).Brueckner et al. (1998), based on a number of 8 cases then known, had even concluded that the travel time of most ICMEs from the Sun to the Earth (measured from the first appearance in C2 images to the beginning of the maximum K p index of an associated geomagnetic storm) always amounts to about 80 h, regardless of the halos' behavior close to the Sun.Brueckner's "80 h rule", as the most simple prediction tool, appears to work pretty well in many cases, in particular near activity minimum.Several researchers have tried to find relationships between CME properties and ICME signatures, "with an eye towards space weather forecasting", as Lindsay et al. (1999b) phrased it.Gopalswamy et al. (2000) determined from coronagraph images the speed of the fastest moving halo CME feature, i.e. its speed V P S projected on the plane of the sky, and compared it with the transit time of the associated ICME towards 1 AU, as defined by the in-situ appearance of intrinsic ICME signatures, i.e. magnetic clouds and low plasma beta.It is here where Gopalswamy et al. (2000) differ from other authors who used the onset of associated geomagnetic effects (e.g.Brueckner et al., 1998;Wang et al., 2002a).Assuming a global, effective acceleration/deceleration representing the solar wind ICME interaction, Gopalswamy et al. (2000) derived a simple relation between the initial speed V P S of a CME and its propagation.The authors consider this a first step towards a future predictive tool, although the travel times derived using their scheme deviate considerably from the observed travel times.Further studies (e.g.Webb et al., 2000;Cane et al., 2000;Gopalswamy et al., 2001c;Vršnak and Gopalswamy, 2002;Michalek et al., 2002;González-Esparza et al., 2003a,b;Yurchyshyn et al., 2003) tried to improve the prediction accuracy, but they all keep suffering from the problem of deducing the proper propagation speed of ICMEs.In a recent analysis Burkepile et al. (2004) studied that projection effect by selecting a subgroup of 111 limb CMEs observed by the Solar Maximum Mission (SMM).Not only do these limb CMEs have, on average, greater speeds than the average values obtained from all SMM CMEs (see Hundhausen, 1993), they also come from lower latitudes and have smaller cone angles.All this provides strong evidence that projection effects systematically influence the apparent CME properties.E90), for illustration of th expansion speed V exp and the plane of the sky speed V PS .In this case we km/s, and V PS = 1118 km/s, A shock arrived at the Earth 55.6 hours la earlier than we predicted (case E20 in Table 2).A severe magnetic storm 14 hours after shock arrival.The images are running differences between L The halo CME of 5 November 1998 (E90), for illustration of the different meaning of speed V exp and the plane of the sky speed V PS .In this case we measured V exp = 1116 V PS = 1118 km/s, A shock arrived at the Earth 55.6 hours later, i.e., about 7 hours an we predicted (case E20 in Table 2).A severe magnetic storm (D st -142 nT) peaked after shock arrival.The images are running differences between LASCO C3 images.1998 (E90), for illustration of the different meaning of expansion speed V exp and the plane of the sky speed V P S .In this case we measured V exp =1116 km/s, and V P S =1118 km/s, A shock arrived at the Earth 55.6 h later, i.e. about 7 h earlier than we predicted (case E20 in Table 2).A severe magnetic storm (D st −142 nT) peaked 14 h after shock arrival.The images are running differences between LASCO C3 images.
Figure 2 illustrates the problem: The plane of the sky speed V P S as defined in the figure is containing almost inevitably a component resulting from a projection effect.In the case of the 11 November 1998 halo CMEs, the ejection is apparently pointed to the northwest.This is consistent with the location of the associated M8 flare at N18 W21.Needless to say, an observer located somewhere else in the heliosphere would have determined a different value of V P S for the same CME, in contrast to V exp , the speed of lateral expansion.
The work by Plunkett et al. (1998) made use of the observation that the outer boundaries of most limb CMEs form a cone of constant spread with a median angular extent of about 50 • .This was confirmed for many more limb CMEs by St.Cyr et al. (2000).Adopting this value and knowing the projection speed V P S of a halo's outer rim they inferred a value around 600 km/s for its true frontal speed.Based on similar considerations, Zhao et al. (2002) derived their "cone model" of halo CMEs.They tried to fit 3 free parameters (angular width, latitude and longitude of cone axis) such that an observational image sequence is met best.This way, they inferred geometric and kinematic properties for the 12 May 1997 CME.However, they had to admit that "there are limitations for some halo CMEs".Michalek et al. (2003), also using a cone model, derived the cones' parameters by taking into account the moments of first appearance of the CME above opposite limbs on the occultor edge.No conclusive results were obtained, though, which is due -in the authors' opinion -to several shortcomings, for example, possible acceleration and expansion of CMEs, missing symmetry of CMEs.
In some cases, interplanetary shocks on their way through the heliosphere generate type II radio emission at the local plasma frequency and its harmonics.This provides a means of remotely observing and even tracking ICMEs (LeBlanc et al., 2001, see also Reiner et al., 1998 2003b andreferences therein).This technique has been used for studying ICME dynamics and interactions (Reiner et al., 2003b;Gopalswamy et al., 2001aGopalswamy et al., ,b, 2002a,b;,b;Lara et al., 2003), but usually after the fact.For near-real-time predictions the type II radio emission is only of limited value.
The basic task remains to find and measure early on the optimum proxy data for the otherwise inaccessible propagation speed of a halo CME towards an observer.In analogy, one would have to measure the speed of a car approaching headon.This is usually achieved by RADAR techniques, but nothing similar works out in the case of head-on CMEs.Alternately, one could measure how fast the car's image grows in a telescope's field-of-view.The apparent lateral expansion speed is a unique measure of the unknown line-of-sight speed.In fact, Fred Hoyle (1957) in his book "The Black Cloud" let a clever scientist calculate the arrival time of a deadly cloud approaching the Earth from deep space using the apparent expansion alone.Upon this warning, mankind was able in time to take provisions for survival.Based on this idea, we developed a similar, purely empirical technique.It makes use of the following well-established observational facts: 1.The cone angles of CME expansion and, more generally, the shapes of the expanding CMEs are usually well maintained (Plunkett et al., 1998).The CME shapes remain "self-similar".We remember that Low (1982) had already noticed the "often observed coherence of the large-scale structure of the moving transient", which could be explained by what he called "self-similar evolution".He studied the expulsion of a CME quantitatively on the basis of an ideal, polytropic, hydromagnetic description and found self-similar solutions that describe the main flows of white-light CMEs as they are observed (see also Low, 1984;Gibson and Low, 1998;and Low, 2001).
2. We found it surprising in our inspection of hundreds of CMEs that the shapes of the vast majority of CMEs appear to be consistent with a nearly perfect circular cross section.Indeed, halo CMEs moving exactly along the Sun-Earth line exhibit generally a circular and smooth shape.This observation is rather surprising in that CMEs are resulting from the eruption of basically 2-D elongated filament structures (for further discussion see the review by Schwenn, 1986).Thus, the apparent lateral CME expansion speed can be considered independent of the viewing direction.
The expansion speed V exp is the only parameter which can uniquely be measured for any CME, be it on the limb or pointed along the Sun-Earth line, on the front or back side; V exp would be an ideal proxy for deriving the unknown radial CME speed V rad and allowing predictions of ICME arrivals at Earth, provided there is a correlation between V exp and V rad .It is the intention of this paper to find and establish such a correlation that then may be applied as a practical tool for space weather forecasting.Therefore, we determined the characteristics of all relevant CMEs observed between January 1997 and 15 April 2001.We searched for signatures of these ejections using in-situ plasma data obtained from spacecraft near the Earth.For those cases where a unique association could safely be made we noted the total travel time from the Sun to Earth.We found that there is indeed a marked correlation between the expansion speed V exp of halo CMEs and their travel time to the Earth.
A preliminary version of this tool has already been presented some time ago (Schwenn et al., 2001;Dal Lago et al., 2003a) and was applied tentatively by several experts in the forecasting community, with good success even for extreme events, e.g. the "Halloween events" (see Fig. 1).However, in the vast literature on the subject there exist discrepancies between associations and interpretations even of identical events.Also, there is major disagreement between various authors about the reliability of predictions, i.e. the significance of missing and false alarms.After all, we decided to redo and double-check the whole analysis and search for statistically sound results.This is the outline of the paper: -In Sect. 2 we give some useful background information on the status of our current understanding of the whole scenario (for more detailed general reviews see, e.g.Webb, 2000, 2002, and the discussion in Cliver and Hudson, 2002).This section provides some definitions and explanations of relevant terms and processes we are addressing in this paper.The experienced reader may wish to skip this section -In Sect. 3 we revisit observations from the SOLWIND coronagraph and the Helios 1 space probe.Favorable orbital constellations allowed determining both: the radial speed of certain limb CMEs and the travel time of the respective ICMEs to an in-situ observer.This exercise tells us which quality of travel time predictions can be expected for the optimum cases where the CME head-on speed near the Sun can be measured precisely.
-In Sect. 4 we study a number of limb CMEs observed by LASCO where the radial speed and the expansion speed could both be measured.If the correlation between those two quantities is good enough, we can then use one of them as proxy data for the other one.
-In Sect. 5 we describe and illustrate how we searched through available data recently obtained and listed all relevant signatures of CMEs, on the one hand, and ICME signatures, on the other hand.
-In Sect.6 we searched for correlations between the two types of events.We went in both directions: from CMEs at the Sun to their potential ICME signatures in front of the Earth, and the other way round: from ICMEs at the Earth back to their potential sources at the Sun.
-In Sect.7 we inspect the 91 cases with a unique association of solar and near-Earth events and derive an empirical function for the expected ICME travel time vs. the measured expansion speed.
-In Sect.8 we discuss the various remaining uncertainties for reliable predictions.
-In Sect.9 the main results are briefly summarized.The concluding remarks address some unsolved problems for future predictions.

Useful background information on CMEs, ICMEs, shocks and geomagnetic disturbances
Geomagnetic disturbances and storms are closely related to the interplanetary magnetic field (IMF) and its fluctuations, both in magnitude and direction.In particular, a southward pointing IMF (B z negative) will allow magnetic reconnection with the northward pointing Earth's field to occur.As a consequence, geomagnetic disturbances and even severe geomagnetic storms will be initiated (Rostoker and Fälthammar, 1967, for reviews see, e.g.Tsurutani and Gonzalez, 1997, or Gonzalez et al., 1994, 1999).So the main question arises: which effects cause B z to turn south?
In the Skylab era of the 1970s the two fundamentally different sources of B z south swings were identified.These swings will be described below.
B z south swings: sources on the non-active Sun Solar wind high-speed streams are dominated by largeamplitude transverse Alfvénic fluctuations causing major excursions of both the proton flow and the IMF vector on time scales of minutes to hours (Belcher and Davis, 1971, see also Marsch, 1990).These high-speed streams were found to emerge from coronal holes, which at solar activity minimum are covering the Sun's polar caps, with some stable extensions to equatorial latitudes.They are representatives of the non-active Sun.They corotate with the Sun, often for several months.Once these high-speed streams reach the Earth, the occasional southward deflections of the IMF, due to the Alfvén turbulence, stir medium level geomagnetic activity (see Tsurutani and Gonzalez, 1987).Bartels (1932), had postulated "M-regions" on the Sun as sources of these geomagnetic effects.The close association between high-speed streams and M-regions had already been noted in the earliest solar wind observations from the Mariner 2 space probe in 1962 (Snyder et al., 1963).Tsurutani and Gonzalez (1987) and Tsurutani et al. (2004b) inspected the effects of highspeed streams on geomagnetism in terms of "high-intensity, long-duration, continuous AE activity (HILDCAA) events".The compression and deflection of the plasma flow in the corotating interaction regions (CIRs) in front of high-speed streams may also lead to substantial B z south components and thus contribute to geomagnetic activity in the rhythm of the solar rotation (Schwenn, 1983;Tsurutani et al., 2004b).It does not matter whether the steepening at the CIRs has already led to the formation of corotating shocks or shock pairs at the CIRs, a process which only rarely occurs inside the Earth's orbit (see Schwenn, 1990).

B z south swings: sources on the active Sun
The major geomagnetic storms are linked to solar activity and come in a very irregular fashion.Since Carrington's famous flare observation in 1859, which he correctly associated with the subsequent giant geomagnetic storm (for the original report, see Meadows, 1970, see also Tsurutani et al., 2003) the close association between these two phenomena seemed to be well established.More than one hundred years later the existence of CMEs and their even more pronounced significance for the Earth was revealed (Gosling et al., 1974, for a recent review see Gopalswamy, 2004).It is important to remember the definition of a CME (Hundhausen et al., 1984, see also Schwenn, 1996): We define a coronal mass ejection (CME) to be an observable change in coronal structure that 1) occurs on a time scale of a few minutes and several hours and 2) involves the appearance (and outward motion) of a new, discrete, bright, white-light feature in the coronagraph field-of-view."Note that this definition does not specify the origin of the "feature", nor its nature, be it the ejecta themselves or the effects driven by them.
CMEs cause gigantic plasma clouds (often called ICMEs for "Interplanetary" counterparts of CMEs) to leave the Sun, which then drive large-scale density waves out into space.They eventually steepen to form collisionless shock waves, similar to the bow shock in front of the Earth's magnetosphere.These density waves are surprisingly difficult to detect optically even with the modern, most sensitive coronagraphs (Vourlidas et al., 2003).On the other hand, shock signatures are the most prominent ones announcing in-situ the arrival of an ICME. Figure 3 gives a good example of a fast ICME event as seen in-situ by the Helios 1 space probe.
The sheath plasma (see, e.g.Tsurutani et al., 1988) behind the shock front results from compression, deflection, and heating of the ambient solar wind by the ensuing ejecta.The sheath may contain substantial distortions of the IMF due to field line draping (McComas et al., 1989) around the ejecta cloud pressing from behind.
For about one-third of all shocks driven by ICMEs, the succeeding plasma shows an in-situ observer the topology of magnetic clouds (Burlaga et al., 1981, see reviews by, e.g.Gosling, 1990;Burlaga et al., 1991, Osherovich andBurlaga, 1997).Smooth rotation of the field vector in a plane vertical to the propagation direction, mostly combined with very low plasma beta, i.e. low plasma densities and strong IMF with low variance give evidence of a flux rope topology (Marubashi et al., 1986;Bothmer and Schwenn, 1998) of these magnetic clouds.This is consistent with the concept of magnetic reconnection (might be better to call it "disconnection") processes in coronal loop systems in the course of prominence eruptions at the Sun (Priest, 1988).It is true though that the boundaries of magnetic clouds are often difficult to identify (Goldstein et al., 1998;Wei et al., 2003).
Most of these ICME signatures can be found in the event shown in Fig. 3. Usually, only a fraction of the criteria for identifying ejecta is encountered in individual events, and to this day a trained expert's eye is needed to tell what is ejecta and what not.The situation is additionally complicated by the class of very slow CMEs found to take off more like balloons rather than as fast projectiles (Srivastava et al., 1999a,b).After many hours of slow rise, they finally float along in the ambient solar wind.Naturally, they do not drive a shock wave.Only in rare cases do a few of their ejecta signatures (e.g.composition anomalies, magnetic cloud topology) remain and disclose their origin.
The compressed sheath plasma behind shocks and the ejecta clouds may both cause substantial deviations of the magnetic field direction from the usual Parker spiral, including strong, out-of-the-ecliptic components.In either case, a southward pointing IMF may result, with well-known consequences on the Earth's geomagnetism (Tsurutani et al., 1988;Gonzalez et al., 1999;Huttunen et al., 2002).
It is important to keep in mind that the sources of magnetic field deflections in the sheath plasma and the ejecta are of fundamentally different origin: -The field line deflection in the sheath due to draping depends on the orientation of the ejecta relative to the heliospheric current sheet and to the observer sitting, say, near the Earth's bow shock.Thus, the field orientations in the sheath vary dramatically from case to case.In some events a southward component never occurs, while in others it lasts for several hours.The compressed, high-density sheath plasma puts the magnetosphere under additional pressure, and in conjunction with a southward B z the resulting geomagnetic storms may become particularly severe.
-The magnetic topology inside ejecta clouds is not yet well understood.It is unclear where the filament lies within an erupting CME and how it is transformed into what becomes the ICME later.Even so, the filament's pre-eruption orientation is often reflected in the ICME configuration.In accordance with the filament's original orientation, the field vector inside the ICME might point to the south at first, then rotate to the east (or west), and finally to the north (SEN and SWN topologies as named by Bothmer and Schwenn, 1998).Figure 3 is an example of this particular type.In the following activity cycle, due to the reversed magnetic polarity of the Sun, the northward swing will occur first, (NES and NWS).With every other activity cycle, B z south occurs predominantly at the front or the rear of the clouds, respectively.This applies to filaments with axes close to the ecliptic plane (Bothmer and Schwenn, 1998), as are usually encountered around solar activity minimum.At times of increased solar activity, perpendicular axis orientations are also possible, leading to the corresponding topologies (ESW, ENW, WSE, WNE).Note that half of these latter cloud topologies at high solar activity never have a southward B z at all.Consequently, those ICMEs do not cause any geomagnetic disturbances.This might explain in part the lowered occurrence rate of geomagnetic storms around maximum activity, as suggested by Mulligan et al. (1998).For the set of magnetic clouds that occurred between 1997 and 2002, Huttunen et al. (2004) found that the majority of magnetic clouds with perpendicular axis orientation occurred in 1997 and 1998,   rising phase of solar activity.Since orientation and helicity of filaments before eruption is often reflected in the topology of the resulting magnetic clouds (Bothmer and Schwenn, 1998;Yurchyshyn et al., 2001), we can use this knowledge for optimizing the prediction of geomagnetic effects (Zhao andHoeksema, 1997, 1998;McAllister et al., 2001;Zhao et al., 2001).
Every CME launched with a speed exceeding 400 km/s was found to eventually drive a shock wave, which can be observed in-situ, provided that the observer is located within the angular span of the CME (Schwenn, 1983;Sheeley et al., 1983Sheeley et al., , 1985)).In reverse, every shock wave observed in space (except the corotating ones) can uniquely be associated with an appropriately pointed CME at the Sun.This implies that there is a causal chain linking CMEs to geomagnetic effects.
No similar statement can be made for flares.Indeed, there are many CMEs (with geoeffects) without associated flares, and there are flares without associated CMEs (and without geoeffects).The long-standing "flare myth" was finally abolished (see Gosling, 1993).However, for the very big and most dangerous events like the one Carrington happened to witness, strong X-ray flares and large CMEs usually occur in a close timely context ( Švestka, 2001).It is now commonly thought that both flares and CMEs are just the symptoms of a common underlying "magnetic disease" of the Sun (Harrison, 2003).Zhang et al. (2001Zhang et al. ( , 2004) ) described the initiation of CMEs in a three-phase scenario: the initiation phase, the impulsive acceleration phase and the propagation phase.The initiation phase (taking some tens of minutes) always occurs before the onset of an associated flare, and the impulsive phase coincides well with the flare's rise phase.The acceleration ceases with the peak of soft X-ray flares.Right at the launch time of several CMEs, Kaufmann et al. (2003) discovered rapid solar spikes at submillimeter wavelengths that might be representative of an early signature of CME onset.In order to disentangle the various processes around CME initiation new observations with significantly better resolution, both spatially and in time, and even supported by spectroscopic diagnostics, are needed, as was demonstrated by Innes et al. (2001) and Balmaceda et al. (2003).It is interesting to notice that some of the theoretical CME models begin to postulate different phases of acceleration (see, e.g.Chen and Krall, 2003).
Some authors claim that there are two kinds of coronal mass ejections (e.g.Sheeley et al., 1999;Srivastava et al., 2000, see also Švestka, 2001;Moon et al., 2002): 1) gradual CMEs, with balloon-like shapes, accelerating slowly and over large distances to speeds in the range 300 to 600 km/s, and 2) Impulsive CMEs, often associated with flares, accelerated already low down to extreme speeds (sometimes more than 2000 km/s).It is not yet clear whether these are really fundamentally different processes or whether they represent just the extrema of an otherwise continuous spectrum of CME properties.These differences will be reflected in the properties of the related ICMEs and their effects.Tsurutani et al. (2004a) analyzed particularly slow magnetic clouds and found them to be surprisingly geoeffective.A good example is the famous event on 6 January 1997: A comparatively slow, unsuspiciously looking, faint partial halo CME caused a "problem storm" at the Earth 85 h later, with enormous effects, as described in a series of papers (Zhao and Hoeksema, 1997;Burlaga et al., 1998;Webb et al., 1998).On the other hand, the very fast ICMEs are often found to be responsible for the most intense geomagnetic storms (Srivastava and Venkatakrishnan, 2002;Gonzalez et al., 2004;Yurchyshyn et al., 2004), apparently because they build up extreme ram pressure on the Earth's magnetosphere.
The number of CMEs observed at the Sun is about 3 per day at maximum solar activity (St.Cyr et al., 2000).Note though that Gopalswamy (2004) found a higher rate since their count included the extremely faint, narrow and slow CMEs that become visible due to the very high sensitivity and the enormous dynamic range of the LASCO instruments.The number of shocks passing an observer located, say, in front of the Earth, is about 0.3 per day (Webb and Howard, 1994).In other words, an observer sees only one out of ten shocks released at the Sun.The average shock shell covers about one-tenth of the full solid angle 4 Pi.Thus, the average shock cone angle as seen from the Sun's center amounts to about 100 • .This is significantly larger than the average angular size of the CMEs of about 45 • (Howard et al., 1985;Hundhausen, 1993;St.Cyr et al., 2000).The conclusion is that shock fronts extend much further out in space than their drivers, the ejecta clouds, as had been suggested earlier by Borrini et al. (1982).This explains why a large fraction of shocks hitting the Earth are exhibiting just sheath plasma, with no ejecta following them.In any case, major geomagnetic storms may be driven.
These are all fairly empirical descriptions of the observed facts.However, the initiation and evolution of CMEs and the resulting propagation of the ejecta clouds through the heliosphere have also been a key subject for theoreticians and modelers ever since.There is vast literature on various models, some of them being quite controversial.The statement by Riley and Crooker (2004) describes the present status quite well: "These models include a rich variety of physics and have been quite successful in reproducing a wide range of observational signatures.However, as the level of sophistication increases, so does the difficulty in interpreting the results".In fact, it is fair to say that at present there is not yet a unified understanding of all processes involved, and we are still searching for the decisive observational facts.For the response of the Earth's system to those interplanetary processes the situation is not better.While the details of their interaction with the magnetosphere are still under study, some empirical relationships continue to be of great use, for example, the famous "Burton formula" (Burton et al., 1975, see also Lindsay et al., 1999a) that seems to be capable of predicting geomagnetic response from the sole knowledge of interplanetary parameters in front of the Earth.

Radial propagation of CMEs and travel time to 1 AU
In this section, we investigate how well the travel time of an ICME to an in-situ observer is related to the head-on CME speed near the Sun in the ideal case where both quantities can be precisely determined, for example, by a coronagraph near the Earth and an in-situ observer located at about 90 • offset in longitude relative to the Sun-Earth line.
Such favorable orbital constellations occurred in the later years of the Helios mission.The two Helios probes happened to cruise for substantial time periods near the plane of the sky (as seen from the Earth) at variables distances to the Sun (0.29 to 1 AU).Simultaneously, the SOLWIND coronagraph on the Earth-orbiting P78-1 satellite was watching many limb CMEs.Several of them were apparently aimed directly at Helios (see Schwenn, 1983;Sheeley et al., 1983).In 49 cases a unique association between SOLWIND limb CMEs and Helios in-situ ICMEs could be established (Fig. 3 shows one of these cases).The combined observations led to the conclusion that there is always a unique association between fast CMEs (faster than about 400 km/s) and in-situ shocks, provided the observer is located within the angular span of the CME (Schwenn, 1983;Sheeley et al., 1983Sheeley et al., , 1985)).Further, every interplanetary shock (unless it is a corotating CIR shock) is the product of a CME.Sheeley et al. (1985) found only one shock without a discernible CME source.Note further that 2 of their 49 "safe" and 6 out of their 18 "possible" associations involved CME and shock observations above opposite limbs.This confirms that in extreme cases shock fronts can reach far around the Sun and thus disturb major parts of the whole heliosphere.For several of these events, Woo et al. (1985), using Doppler scintillation observations, could show that substantial deceleration of shocks (with v>1000 km/s) takes place near the Sun and that the amount of deceleration increases with shock speed.
We have now revisited and extended these studies and correlated the measured CME front speeds with the travel times of the associated shock waves to the Helios probe.Using the data listed in Table 1 of Sheeley et al. (1985), we derived the plot shown in Fig. 4 (unfortunately, expansion speeds were not measured at the time).For determining a "normalized" projected travel time to 1 AU we simply assumed that the shocks would travel from the Sun through Helios, all the way to 1 AU at the same average speed.The expected trend is clearly visible: the faster the CME, the shorter the travel time to Helios 1.But there is substantial scatter around the "ideal line".While the fastest shocks arrive about as expected, the large group of very slow starters (<500 km/s) arrive substantially earlier than expected, and the majority of CMEs in the group 750 to 1000 km/s arrives later than expected.This confirms generally what has been stated above (Gopalswamy, 2000(Gopalswamy, , 2001c;;González-Esparza et al., 2003a,b), that slow CMEs are post-accelerated by the ambient solar wind, and the fast ones are decelerated.
However, the large scatter in Fig. 4 comes as a surprise, and we cannot yet offer a unique explanation.The options are: 1.The CME speeds may not have been measured sufficiently well.This could be due to the low sensitivity of SOLWIND, its limited field-of-view (up to 10 R s ) and to bad time cadence.In particular, the slow CMEs might be underestimated.We think that some of the data points in the slowest group should be shifted towards higher values of CME speeds.
2. Helios 1 was certainly not always hit by the fastest parts of the ICMEs.As shock fronts are large curved structures they will arrive later at observers situated in the ICME flanks, compared to the more frontal cases.
For half of all interplanetary shocks no ejecta signatures were found by Helios, thus indicating flank passes through the ICMEs.Thus, some travel time values ought to be corrected towards smaller numbers.
3. The shock travel time must not be confused with the ejecta travel time.The ejecta follow their shocks a few hours later.Thus, the "real" ejecta travel times would be longer.
4. On their way through space, ICMEs travel through very different types of ambient solar wind.The solar wind conditions vary dramatically on short time scales as well as during the solar cycle.Thus, major deviations from simple kinematic models commonly used for predictions will occur.Only those cases were selected where a unique association could be achieved, in particular when Helios 1 was with +/-30 0 near the plane of the sky, and when the angular span of the CME included the Helios orbital plane.The travel time was derived from the moment of first appearance in a coronagraph image and the shock arrival time at Helios 1.The projected travel times to 1 AU as given in the figure were determined by simply assuming that the shocks would travel from the Sun through Helios and all the way to 1 AU at the same average speed .All data were adopted from Table 1 of Sheeley et al. (1985).The dotted line denotes the "ideal" line, i.e., where the travel time would exactly correspond to the CME radial speed near the Sun.
Fig. 4. The travel time of limb CME fronts from the Sun to the location of Helios 1 (at about 90 • off the Earth-Sun line) as a function of the CME radial speed V rad obtained from the SOLWIND coronagraph.Only those cases were selected where a unique association could be achieved, in particular when Helios 1 was with ±30 • near the plane of the sky, and when the angular span of the CME included the Helios orbital plane.The travel time was derived from the moment of first appearance in a coronagraph image and the shock arrival time at Helios 1.The projected travel times to 1 AU, as given in the figure, were determined by simply assuming that the shocks would travel from the Sun through Helios, all the way to 1 AU at the same average speed .All data were adopted from Table 1 of Sheeley et al. (1985).The dotted line denotes the "ideal" line, i.e.where the travel time would exactly correspond to the CME radial speed near the Sun.
Under 3) we encounter a pretty controversial issue (see Cane et al., 2000;Cane and Richardson, 2003b;and Gopalswamy et al., 2003).It would of course be highly desirable and much more consistent if we could locate and then correlate identical features on both ends, i.e. in CME images and in in-situ data.This is what Gopalswamy et al. (2003) kept insisting on.But the driver gas often cannot be identified uniquely or just misses the observer.Further, we cannot yet determine how the familiar three-part structure of most CMEs (bright outer loop, dark void, bright and structured kernel (Hundhausen, 1988) is transformed into the familiar two-part structure of ICMEs (sheath behind the shock wave, followed by driver gas/magnetic cloud) as illustrated in Fig. 3. Which is which?Before future work will solve this problem, we decided to put our present study on a purely empirical basis: we take the most uniquely observable quantities on both ends and look for correlations between them.Near the Sun, we take the first appearance in SOLWIND/LASCO-C2 images of the bright outer CME loop as reference time, and in space we look for any signs (including shocks) indicating approaching ICMEs.Note, by the way, that shocks are easily and uniquely detectable, in great contrast to the ejecta clouds.
If argument 4) is in fact a cause for the large scatter in Fig. 4, then we can already see that determining the radial  Fig. 6.The drawings illustrate how the expansion speed V exp is defined.It can be determined uniquely for all types of CMEs, be it limb, partial halo or full halo CMEs, while the apparent plane of the sky speed V P S often contains an unknown speed component towards the observer.
CME speed near the Sun alone will not guarantee good predictions.The propagation conditions have to be taken into account as well.

As a conclusion of this section we note:
-There is indeed a correlation between the radial front speed V rad of limb CMEs and the travel time towards an in-situ observer.
-Slow CMEs are post-accelerated, fast ones are decelerated.
-The scatter is surprisingly high.

The expansion speed serves as proxy for the radial propagation speed of CMEs
Now we will study a number of limb CMEs observed by LASCO where the radial speed V rad (which in these is identical to V P S ) and the expansion speed V exp could both be measured, in order to check the value of the latter one as proxy data for the other one.Upon inspecting many hundreds of CMEs observed from SOHO we confirmed the observation by Plunkett et al. (1998) that for limb events the cone angles of expansion and, more generally, the shapes of the expanding CMEs were strikingly maintained (by the term "cone angle" we mean the angle between the outer edges of opposing flanks of limb CMEs).The CME shapes remained "self-similar" throughout the LASCO field-of-view.In other words, the ratio between lateral expansion and radial propagation appears to be constant for most CMEs.A typical example is shown in Fig. 5.The way of determining the radial speed V rad and the expansion V exp speed of a CME is illustrated in Fig. 6 (from Dal Lago et al., 2003a).We selected a representative subset of limb CMEs (57 in total), where EIT images showed a uniquely associated erupting feature within 30 • in longitude to the solar limb and within a reasonable time window of a few hours.Further, sufficient coverage in C3 images was required.For those events we determined both the radial speed V rad (of the fastest feature projected onto the plane of the sky) and the expansion speed V exp (measured across the full CME in the direction perpendicular to V rad ).We measured them when they had reached constant values, i.e. usually at around 10 R s .
Figure 7 shows the result: There is indeed a fairly good correlation between the two quantities.A linear fit through the data yields with a correlation coefficient of 0.86.Apparently, the correlation shown in Fig. 7 holds for the slow CMEs as well as for the fast ones, for the narrow ones as well as for the wide ones.The correlation even holds in the extreme cases where a cloud expands faster than it moves as a whole; the front motion would then mainly be due to the expansion alone, where the cone angle amounts to 180 • and V rad would be about equal to V exp , in fairly good agreement with Eq. (1).
As an example the limb CME of 20 April 1998, shown in Fig. 8, can be considered: This was an extremely fast event right behind the west limb with V rad =1944 km/s and V exp =1930 km/s and a cone angle of 170 • .In fact, the effects from this extraordinary limb event even hit the Earth 79.1 h later and caused a moderate geomagnetic storm (D st −59 nT).
In order to understand this remarkable correlation we tried some modeling of CME evolution.We assumed various plausible geometries based on the "ice cream cone model", as first described by Fisher and Munro (1984) and applied again, e.g. by Zhao et al. (2002) and Michalek et al. (2003).We modeled the CME front surface as a section of an expanding sphere and calculated the ratio V exp /V rad as a function of the cone angle α. Figure 9 (upper panels) illustrates the 3 geometries we used: the front sphere could either be a sphere section with the Sun in the center (Model A), or a half sphere sitting on top of a cone (Model B), or a sphere dropped into a cone (like a real ice cream ball, such that the ball surface touches the sphere tangentially, Model C).We assumed the cone angle to be constant and the sphere surface to remain self-similar throughout the expansion.It turns out that in the cone angle range from 40 • to 80 • , where most CMEs belong, model A yields V exp /V rad from 1.2 to 0.85, compared to the experimental value of 0.88.The other models give slightly higher values.On the other hand, for a cone angle near 180 • Model C works best.One can easily verify that for this geometry model C yields V exp =V rad (see also the example of Fig. 8).In view of the rather crude assumptions about CMEs' shape and evolution this is not a bad agreement between our finding and the models.
We further extended these models to cases where the CMEs are pointed away from the plane of the sky by an angle φ (see Fig. 9, lower panels).We calculated the values for the apparent V P S which now no longer equals V rad as it does for limb CMEs.However, unless φ becomes larger than the cone angle α, V P S and V rad remain equal or almost equal, in any model.When φ approaches 90 • , i.e. for the central halo constellation, all 3 models converge to V P S =V exp /2, regardless of the cone angle, as expected.In other words, for a given CME the apparent V P S value varies substantially with the viewing angle φ, while the V exp value does not.This is why V exp is superior to V P S for inferring the radial speed V rad of halo CMEs.
After all we can state: -For limb CMEs the radial front speed V rad is almost the same as the expansion speed V exp .The front surface is assumed to be a sphere section, either with the Sun in the center (A), or a half sphere sitting on top of a cone (B), or a sphere dropped into a cone (like a real ice cream ball, such that the ball surface touches the sphere tangentially, C).The cone angle a (i.e., the full angle between between the outer edges of opposing flanks of CMEs) and the radial speed V rad are assumed to be constant with time.Thus, the linear dimensions are proportional to the speeds V exp and V rad .The upper panels apply to limb CMEs, viewed from the top or from the side.For the lower panels, the CMEs are off-pointed by an angle f with respect to the plane of the sky.
Fig. 9. Three plausible models of CME geometries used for studying the ratio between V exp and V rad and their relation to the plane of the sky speed V P S .The front surface is assumed to be a sphere section, either with the Sun in the center (A), or a half sphere sitting on top of a cone (B), or a sphere dropped into a cone (like a real ice cream ball, such that the ball surface touches the sphere tangentially, C).The cone angle α (i.e. the full angle between between the outer edges of opposing flanks of CMEs) and the radial speed V rad are assumed to be constant with time.Thus, the linear dimensions are proportional to the speeds V exp and V rad .The upper panels apply to limb CMEs, viewed from the top or from the side.For the lower panels, the CMEs are off-pointed by an angle φ with respect to the plane of the sky.
-The plane of the sky speed V P S can differ from V exp and the wanted V rad value by a factor of 2, depending on the CME direction.
-The expansion speed V exp is a fairly reliable proxy for the radial speed V rad for all types of CMEs.

ICMEs near the Earth
We searched the LASCO data from January 1997 to 15 April 2001 for all CMEs that might be considered candidates for producing effects near the Earth, i.e. halo and partial halos.Independently, we searched all in-situ data obtained near the Earth for signatures of ICMEs.

CMEs observed near the Sun
All LASCO data were searched for full or partial halo CMEs.
Visual inspection was required since automated recognition schemes that are under preparation (e.g.Robbrecht and Berghmans, 2004) are not yet available.A CME is termed a "full" halo if a feature (remember the CME definition) appears in at least one image all around the occultor.A "partial" halo has an angular span of at least 120 • .The distinction between full and partial halos often suffers from the fact that the CME brightness and structure vary strongly with the position angle.The features we see might be the ejecta themselves, or the compressed and deflected material related to a shock out ahead of the CME, or the superposition of 2 or more separate CMEs.
We included all these events in the study since they might have a significant speed component along the Sun-Earth-line and may thus be relevant for space weather issues.In the remainder of this paper, the term "halo" will always mean both: full and partial halos.Typical examples for the CME types are shown in Fig. 6.For the search, we inspected the reports issued by the LASCO operations team at GSFC under ftp://ares.nrl.navy.mil/pub/lasco/halo.They helped us to avoid overlooking some of the fainter CMEs, and they gave us important information on the halos' first appearance in C2 images, apparent direction, plane-of-the-sky speed, and correlations with disk events as seen by EIT.This latter information, in particular, allowed us to earmark off-pointed (backside) CMEs early on.Further, we studied the CME catalog under http://cdaw.gsfc.nasa.gov/CMElist/.In order to clarify differences between the catalogs and to derive unique and consistent evaluations, we inspected ourselves every single CME image sequence.This explains why in some cases we arrived at different interpretations.
Table 1 gives the numbers of front side halos (F), backside halos (B), limb CMEs (L) and unclear cases (U) where we could not decide about the source location.The limb events are those with an apparent eruption within 30 • of the limb of the front side disk.The unclear cases are those where we could not uniquely determine whether the halos were frontor backside events.Note that the numbers of backside halos is not representative anyway, since more such events occurred but were not registered and evaluated in the context of this paper.We skipped some of the events listed as halos in the http://cdaw.gsfc.nasa.gov/CMElist/ catalog because they were so faint that we could not confirm them ourselves, even when we applied our most sophisticated technique as described by Dal Lago et al. ( 2003b).The column "Other" events addresses CMEs that we considered relevant although their angular width was less than 120 • .Their importance in context with this work showed up later.Further, there are 4 more "other" cases were SOHO data were not available, but there are other good reasons to classify them as full front side halo CMEs.
For each event we determined the characteristic data and noted them in our data bank (interested readers are referred to the webpage http://star.mpae.gwdg.de/cmeeffects/).After all, we recorded a total of 248 relevant CME events that occurred in the time period between January 1997 and 15 April 2001.Neglecting the 41 backside and 26 unclear events, we ended up with a total of 181 CME candidates for further analysis.The list of 181 relevant CMEs includes 18 non-halo CMEs, i.e.CMEs with an extension of less than 120 • and the 4 front side CMEs for which there exist no SOHO data.The actual relevance of these events for this study was discovered from other sources.
For each CME event we evaluated and listed in the data bank the following entries: -The time of first appearance in C2 is taken as reference onset time, for practical reasons: the actual onset is not always uniquely discernible in data from EIT or Yohkoh, because there is often too much activity around.Experience tells us that only those events that make it into the C2 field-of-view become relevant for our study.The time difference between the CME onset in EIT and its appearance in C2 is of the order of one hour, i.e. small compared to the travel times to 1 AU of some 80 h.
-The plane of the sky speed V P S values were also copied from the same catalog (the linear fit speed values).Note that they were derived for the fastest feature in the images sequences (St.Cyr et al., 2000).The time-height diagrams usually show an increasing speed in the C2 field-of-view, but in the C3 field the speeds appear to be constant from about 10 R s on.In cases where a linear fit obviously does not work, we chose the second order fit at 20 R s .If C3 data were not available, we inserted the C2 value with a minus sign added as an earmark.
-The expansion speed V exp was determined according to Fig. 6.Usually, halos appear with more or less elliptical cross sections.Even a CME with a perfectly circular cross section would exhibit an elliptical shape if it is not pointed exactly along the Earth-Sun line.This is why V exp has to be determined from the expansion of the brightest and fastest features perpendicular to the projected propagation direction.We plotted the lateral expansion as a function of time (similar to a height time plot) and derived a value for V exp when it had reached a constant value, i.e. for halo widths of some 15 R s .

ICMEs recorded by spacecraft in front of the Earth
In order to study the potential correlations between CMEs and their effects on the Earth, we searched for the arrival signatures of ICMEs at the location in front of the Earth's bow shock, i.e. at the SOHO, WIND or ACE spacecraft, all cruising around the L1 point.What is the optimum ICME signature?In about half of all cases, the ejecta themselves do not hit the Earth, although their associated shocks may drive substantial geomagnetic storms (Gosling et al., 1991), and even if so, the ejecta are often not uniquely discernible.Some authors (e.g.Gopalswamy et al., 2000) used magnetic cloud signatures and low plasma beta as markers.These signatures appear not to be unique, and many events will probably be missed.In other studies, the onset times of geomagnetic storms or the D st peak times were taken as further markers (e.g.Brueckner et al., 1998;Zhang et al., 2003).We do not consider this to be a very appropriate method, since a storm may be caused either by the sheath plasma ahead of the ejecta or hours later by the ejecta themselves, or by a combination of both, or even not at all.For those cases where it could be determined, the delay time between ICME arrival and D st peak time was found to vary between 3 and 40 h, with an average value of 18 h (Russell and Mulligan, 2002;Zhang et al., 2003).
What suffers least from such shortcomings is the shock signature as seen in plasma and IMF data taken outside the Earth's bow shock.This signature sticks out so clearly that it can hardly be overlooked or misinterpreted.Even a computer can be taught to identify shocks (e.g.http://umtof.umd.edu/pm/shockspotter.html).
Of course are we aware of the fact that transient shocks are NOT parts of the ICMEs themselves.It is true that shocks are driven by ICMEs, but they move within the ambient solar wind, which determines their propagation properties.These properties vary dramatically from event to event, due to the 3-D structure of the solar wind streams and the IMF.On the other hand, as shown above, the correlation between fast CMEs and the occurrence of interplanetary shocks is safely established (Schwenn, 1983;Sheeley et al., 1985).
Our intention here is to derive an empirical tool based on the most unique available signatures on both ends: the CMEs observed early on by coronagraphs, and the ICME signatures observed in front of the Earth.Therefore, we searched through all in-situ plasma and field data that are available on the Web.They are provided by the SWE and MFI instruments on the WIND spacecraft, by the SWEPAM and MAG instruments on the ACE spacecraft, and by the MTOF-CELIAS proton monitor on SOHO.This way, a 100% complete data coverage over the study interval from January 1997 to 15 April 2001 could be achieved.
The occurrence of a shock wave can be recognized in insitu plasma data by a noticeable, abrupt, and simultaneous increase of speed, density, temperature, and magnetic field magnitude (see Fig. 3)."Abrupt" means between adjacent data points, usually taken a few seconds or minutes apart.The knowledge of the magnetic field increase is considered mandatory.
By visual inspection using these criteria a total of 147 shocks were identified, (plus 4 CIR shocks in front of corotating high-speed streams).
For each shock we made these entries in a shock catalog: -the time of arrival, preferentially from ACE data, -the solar wind speed and density values on both sides of the shock that allowed calculating a rough local shock speed V sh according to Eq. ( 6.6) in Hundhausen (1972).
Recognizing ejecta not accompanied by shocks is not uniquely possible.The signatures are too hard to discern.
Occasionally, a trained eye may discover a candidate event by chance, or triggered by a halo alarm issued, for example, by the LASCO operations team (see http://lasco-www.nrl.navy.mil/halocme.html).Huttunen et al. (2002) show a good example (our case E7) in their Fig. 2. We inserted 40 such events into our catalog: 22 of them we regard as probable magnetic clouds (M events) and 18 others as suspicious density blobs (B events) without magnetic cloud signature.We do not consider that list complete or in any sense representative, since no systematic search could be performed because of the severe ambiguities.
During this search, we noted other quantities for future use, for example, certain ICME signatures, and the K p and D st values of associated geomagnetic storms, if applicable.This lead us to include all at least moderate geomagnetic storm events, i.e. when D st fell below −50 nT, regardless of shock occurrence (as one of the referees pointed out, one should use the change of D st relative to pre-storm levels for storm strength definition rather than the absolute values.We will do that in succeeding papers when the storm effects will be primarily studied).This way, the major geoefficient magnetic clouds were located, plus another 4 geoeffective CIRs without corotating shocks.
We compared our list with the various lists compiled by other authors and found ourselves in a big mess.To illustrate the problem, let us mention only one example: Cane and Richardson (2003a) studied an overlapping time period (May 1996 to November 2002).They searched for ICMEs using the following criteria: 1) abnormal proton temperatures, 2) unusual magnetic field deflections, 3) occurrence of shocks, 4) solar energetic particle increases and galactic cosmic ray decreases, 5) bi-directional solar wind electron strahls.We note that several of "our" unique ICME events are missing in the Cane and Richardson list: e.g. on 27 February 1997 (our event E3), 28 March, 1997 (E4), 20 May 1997 (E10), and many others.Some cases in their list we could not confirm as ICMEs in our definition, e.g. on 16 February 1997 and on 17 September, 1997.For many others the "disturbance time" does not agree with ours.We double-checked our own findings and confirmed or corrected them, respectively.Major discrepancies remain, also with the studies of other authors, for reasons we cannot explain.We find the confirmation of old wisdom, i.e. that one has to be very careful in performing statistical evaluations of large databases For the present study each single case was at least doublechecked by more than one person, and with great certainty we can state: -On the one end, we registered in the study interval a total of 181 CME events that appeared to be relevant for our correlation study.
-On the other end, we registered 147 shock waves near 1 AU (4 CIR shocks not included), plus 40 ejecta clouds not accompanied by a shock.

Correlations between observations of CMEs and ICMEs
The most complicated part of this study was to associate the events from the two catalogs with each other.We went both ways: 1.For any front side halo CME we searched for a response in in-situ plasma data.
2. For any observed shock/ICME we searched for a potential solar source.

From halo CMEs to ICMEs
From the given onset times of all 181 relevant CMEs in Table 1 we searched the plasma data of the following 120 h for shock entries in our shock catalog or for other ICME signatures in the original plasma data sets.The size of the time window was chosen somewhat arbitrarily.It corresponds to an average travel speed of 350 km/s, a value near the lower limit for the speed of shocks and ICMEs.Further, with increasing travel time associations become more and more questionable.The slowest ICME that we could uniquely associate took 107 h from the Sun to the Earth.
A detailed classification and summary evaluation is given in Table 2. Our classification criteria need some explanations: -A "unique ICME association" is given if the time window following an isolated CME contained one single shock or ICME signature.
-Cases with multiple events on either end were treated with special attention.There were several time periods where CMEs followed each other in such a cadence that the later ones were sufficiently faster to probably overtake the earlier ones.Such "cannibalism", as Gopalswamy et al. (2001a) called it, caused increased plasma turbulence at the interaction sites, notable by increased levels of continuum-like radio wave emission and increased acceleration of solar energetic particles (Gopalswamy et al., 2002a,b).Further, in-situ data at 1 AU show a mix-up of structures: shocks, discontinuities, clouds, etc., that cannot uniquely be disentangled and associated with individual CMEs (e.g. for the periods 15 and 16 September 2000 or 23 to 25 November 2000, for details see Burlaga et al., 2002).We put such events into the "possible IP" category.We will discuss this topic in more detail in Sect.8.1.
-For the cases of "no ICME association" we doublechecked the data sets and made sure that within the 120 h time window following the CME in question there was actually no shock/ICME sign discernible.
We split the 181 cases into the categories "front side cases" and "limb cases", and each of them into the subcategories "full", "partial" or "<120 • " cases.
From inspecting Table 2 we conclude: -A unique association between CMEs on the Sun and subsequent ICMEs observed near the Earth was found in 50.3% ( 91) out of all 181 cases.If we exclude the non-halo events, we find a unique association for 53.4% (87) of 163 front side or limb halo CMEs.
-In 59.3% (54) out of the 91 unique cases the CMEs were full halos, in a further 35.2%(32) of the cases the CMEs were partial halos.-Among the 152 uniquely or possibly associated cases, there were 13 non-halo CMEs involved (9 front and 4 limb CMEs, respectively), i.e. a total of 8.6%.Forecasters focused on halo CMEs would have ignored these CMEs: missing alarms.
-For 85.3% ( 139) cases of all 163 halos a unique or possible association with shock/ICMEs at 1 AU could be established.
-14.7% ( 24) of all 163 halo CMEs did NOT cause any notable effect near the Earth.They simply missed the Earth.Forecasters focused on halo CMEs would have issued false alarms for every sixth halo CME.
-7.4% ( 7) out all 94 full halo CMEs (6 out of 20 front side and 1 out of 3 limb events) went by without any effects.The most relevant CMEs are the full front side halos, suggesting that 6.5% (6) would lead to a false alarm.
-Out of all 69 partial front side CMEs a fraction of 24.8% (17) missed the Earth.Out of 23 limb halo CMEs a fraction of 17.3% (4) missed the Earth.
The last 3 points should make forecasters rather cautious.
Taking full or partial halo CMEs as indicators, they would have issued 163 warnings.In 139 cases they would have been right, probably.The remaining 24 alarms (14.7%) would have been 24 false alarms, resulting from halos (7 full and 17 partial) that did not reach the Earth.We will inspect this issue further in Sect.8.

From ICMEs at 1 AU back to CMEs
From each of the 147 listed shocks and the 40 cloud or blob events observed in-situ near the Earth we inspected the halo CME list, and further the CME catalog in http://cdaw.gsfc.nasa.gov/CMElist/, and in case of doubt the complete LASCO/EIT data set.A time window of 120 h ahead of each single ICME was selected.On applying the same criteria as for generating Table 2, we identified a total of 191 relevant event pairs.In Table 3 we show the results: -There were 80 event pairs where a halo CME was uniquely associated with a shock near the Earth, for 11 more halo CMEs ICME signatures without shocks could be found.
-Of course, these 91 cases of clear association are identical to those in Table 2.
-61 more shock and 29 ICME events are possibly/probably also caused by CMEs, but unique association was not possible.This could be due to the occurrence of multiple CMEs or multiple/unclear ICME signatures in the respective time windows.For another 33 cases (not listed in Table 3 but in the data bank http://star.mpae.gwdg.de/cmeeffects/), we could not even name an ICME candidate, due to missing or just unclear data.And yet, a CME/ICME association cannot be ruled out and, consequently, must be considered as "possible".
-The numbers of unique and possible associations are comparable.This leads to serious constraints on prediction reliability.We will inspect this aspect in more detail in Sect.8.
-54.44% (80) of all 147 ICME shocks had definitely a known CME source on the front side of the Sun, another 41.5% (61) probably as well.This means that 95.9% of all transient shocks can be associated with a certain or at least possible CME source.
-This also means that 4.1% (6) of all 147 transient shocks did NOT have a front side CME source.These events hit the Earth completely unpredicted: missing alarms.
One out of the 6 unpredicted ICME shocks, the one on 28 October 2000, 09:01 (E251) led even to a major geomagnetic storm.It is true that on 25 October, 08:20, a full front side halo CME had occurred.This CME, in our interpretation, caused a shock early on 28 October 05:32 (with a succeeding storm, D st −127 nT).For the second shock and its associated storm (D st −119 nT) there is no source found.The other 5 events caused only mild disturbances (D st >−50 nT).Not included in any of the tables are the two moderate storms that occurred completely out of the blue, without any signatures of a CME at the Sun, nor of an ICME near the Earth, nor a CIR: 9 June 1997 (D st −84 nT, E13) and 24 April 2000 (D st −65 nT, E201).The only features these events have in common with other storm events are extended periods of southward pointing B z .We can only speculate about the potential source of such events.One might think of non-halo CMEs, or ICMEs starting from behind the limb and with travel times much longer than 120 h, and overlooked ICME signatures.
For comparison, in the Sheeley et al. (1985) study there were 2% of shocks (i.e. 1 case out of 47) lacking a CME source.For the recent period from 1996 to 2000, Zhang et al. (2003) mentioned 4 major storm events with unknown source regions.We agree with their conclusion, that there may have been very slow and gradual CMEs emerging from the east limb or behind, but we think that "their" events should go into the category "possible CME source".In fact, we found a total of 29 such cases within the group of 90 "possible associations with CMEs" from Table 3.We did not differentiate between those with too few and those with too many faint or remote CME candidates.The cases with too few candidate CMEs are hard to differentiate from those with definitely no CME source, simply because there is almost always "something" going on, and there is hardly ever a 120-h time window where the occurrence of a CME can definitely be negated.
We show a list of the 6 non-associated shocks (from Table 2) and the 2 storms without CME sources in Table 4, because we think they deserve future attention.These are examples of really unpredictable geomagnetic disturbances, i.e. missing alarms.Note further, that for the other 29 "possible" cases just mentioned (i.e.those with an unclear though possible CME source) no positive prediction would have been issued either, since out of the many ordinary, small, slow, narrow CMEs going on most of the time, none was attracting attention as a potential threat to Earth.They were all found after the fact, i.e. when an unexpected impact at Earth had already occurred.Thus, we must conclude that 37 (19.6%)out of the 187 shock or ICMEs and the 2 storm events near Earth would not have been predicted.

The unique associations
The 91 uniquely associated event pairs mentioned in Tables 2 and 3 are listed with their characteristic data in a table to be found in http://star.mpae.gwdg.de/cmeeffects/, in order to allow other authors to compare and cross-check their data with ours.The 91 unique cases include 80 shock events (for 72 front side and 8 limb CMEs).Four of these 91 CMEs had an angular width of less than 120 • and would not be considered halo CMEs.For 11 safe associations, no shocks were observed but othe ICME signatures (magnetic cloud topolo-gies (M), extreme density enhancements (B) etc.).
For 75 of the 80 halo/shock pairs consistent data pairs for V exp /shock travel time could be determined.We will use them in Sect.7.
In summary, we state: -In about 85% of full or partial front side halo CMEs an ICME effect at the Earth can be expected.
-The remaining 15% of full or partial front side halo CME events would miss the Earth: false alarms.
-4.1% of all transient shocks do not have a front side CME source: missing alarms.
-In total, 19.6% out of 189 ICME or storm events near Earth would not have been predicted: missing alarms.

Travel times of uniquely associated CME/shock event pairs
In order to derive a sound empirical correlation between halo CME expansion speeds and the travel times of their associated shocks, we selected those 75 cases where we had found not only unique CME-shock correlations but could also measure all required quantities.In Fig. 10 we show the travel times plotted vs. the halo expansion speed V exp .The open circle symbols denote the 10 limb events among the 75 unique cases.The dashed line indicates the expected travel time, calculated by assuming a constant radial propagation speed (kinematic approach) inferred from the observed expansion speed according to Eq. ( 1).The solid line in Fig. 10 is a least-square fit curve to the 75 main data points.The functional form of this fit curve is based on a straight forward deceleration model assuming viscous drag (see, e.g.Vršnak, 2001a).Here the local deceleration is proportional to the speed difference to the ambient medium.Such a model leads to an asymptotic convergence of the CME speed to the ambient solar wind as described, e.g. by Vršnak (2001a,b), Vršnak and Gopalswamy (2002) and Nakagawa and Matsuoka (2003).With that assumption, we find the optimum fit function for the travel time T tr to be with T tr in hours and V exp in km/s.In Fig. 10 we note: -The functional form of our fit curve meets the measured data points fairly well, despite the large scatter.
-The scatter is indeed substantial.The standard deviation from the fit curve is 14 h.For defining a 95% safety margin we assume 2 standard deviations, i.e. a little more than one day.
-With respect to the functional dependence and the amount of scatter, Fig. 10 looks very similar to Fig. 4 where we had compared radial speeds of limb CMEs with the shock travel times towards the Helios probes.
-All shocks from halos with expansion speeds above 1000 km/s are arriving hours later than expected from the kinematic approach, indicating substantial deceleration.
-The shocks and ICMEs from slowly expanding halos (V exp <500 km/s, equivalent to V rad <450 km/s) arrive earlier than expected.They must have been postaccelerated.
-The 10 uniquely associated limb CMEs (denoted by open circles) show a basically similar behavior.
-The 11 non-shock ICME cases (denoted by green bullets) fall very close to the kinematic approach curve.
For the sake of obtaining a simple relation such as Eq. ( 2), we had assumed the ambient speed to be zero.For non-zero values of the ambient speed, the situation for the fast CMEs does not change much, but for the slow events the effective travel time would increase to some extent.Thus, the fit function would become somewhat steeper, but considering the large scatter of the data points we disregarded this refinement.
There has been a lot of discussion in the literature about the optimum CME acceleration/deceleration model.Some authors, on the basis of coronagraph observations, had assumed an "effective" acceleration to be constant over the Sun-Earth distance (Gopalswamy et al., 2000).To generalize this model, Gopalswamy et al. (2001c) and Michalek et al. (2002) assumed that the effective acceleration ceases at some distance about half way between the Sun and the Earth.Such a model appears to be supported by Reiner et al. (2003a), who measured the frequency drifts of radio emissions from ICME shock fronts and determined radial speed profiles out to and beyond 1 AU.In contrast, the models by González-Esparza et al. (2003a,b) predict that a fast CME moves initially in the inner heliosphere with a quasi-constant 10 seen by the ACE spacecraft near Earth.The 2 full front side halo CMEs had originated from the same region on the Sun within 2.5 hours.The two CMEs had interacted with each other already close to the Sun.At Earth arrived only one shock (7 hours later than we had predicted).Almost a day later arrived a magnetic feature that might be considered a magnetic cloud, although some typical signatures are missing.Bz turned south for about 12 hours, thus causing a moderate magnetic storm with Dst -62 nT.
Figure 10.The ICME travel times plotted vs the halo expansion speed Vexp for the 75 usable cases of unique CME-shock correlations.The travel time Ttr is defined by the CME's first appearance in C2 images and the shock arrival at 1 AU.The solid blue line is a least square fit curve to the 75 data points, the fit function being Ttr = 203 -20.77 * ln (Vexp).The standard deviation from the fit curve is 14 hours.The two dotted lines denote a 95 % certainty margin of 2 standard deviations.The thin dashed line marks the calculated travel time for a constant radial propagation speed (kinematic approach) inferred from the observed expansion speed according to relation (1).The green dots denote ICMEs without shock signatures, i.e. magnetic clouds (M) and plasma blobs (B).These points were not used for the fit.Fig. 10.The ICME travel times plotted vs. the halo expansion speed V exp for the 75 usable cases of unique CME-shock correlations.The travel time T tr is defined by the CME's first appearance in C2 images and the shock arrival at 1 AU.The solid blue line is a least-square fit curve to the 75 data points, with the fit function being T tr =203-20.77*ln (V exp ).The standard deviation from the fit curve is 14 h.The two dotted lines denote a 95% certainty margin of 2 standard deviations.The thin dashed line marks the calculated travel time for a constant radial propagation speed (kinematic approach) inferred from the observed expansion speed according to Eq. ( 1).The green dots denote ICMEs without shock signatures, i.e. magnetic clouds (M) and plasma blobs (B).These points were not used for the fit.velocity until a "critical distance" is reached at which the CME velocity begins to decrease, approaching the speed of the ambient solar wind.This ongoing discussion has led Cargill (2004) to investigate the aerodynamic force acting on ICMEs by applying appropriate MHD simulations.He "assumes a drag force proportional to the square of the relative velocity, as has been used in hydrodynamic".In that he differs from Vršnak, 2001a who assumed a viscous drag force proportional to the relative velocity.However, as Owens and Cargill (2004) pointed out, "that this leads to little difference".The models of Cargill (2004) show that the effective deceleration depends largely on the ICME density relative to the ambient solar wind density.These quantities vary dramatically from event to event, and that may be the main reason for the large scatter of our data points and those of other authors.
We should not assign too much meaning to the empirical models unless we know better the actual propagation environment of the individual CMEs.Note that even a simple linear fit would not do much worse than any other, more sophisticated fit (see, e.g.Fig. 5 in Michalek et al., 2002).This is what Zhang et al. (2003) did when they correlated the halo CME front speeds with the ICME travel time (for those 26 cases where they could discern an ICME at all) and simply fitted a straight line through the data.For the time being, we would like to join in with Reiner et al. (2003a), who, in the spirit of Galileo Galilei, proposed "that a number of CMEs be dropped from La Torre di Pisa and their drag force be directly measured".
Previous studies had always been based on the plane of the sky speed V P S of the fastest CME features.We applied this method to our 75 safe associations and found significantly more scatter.The reason is of course that the plane of the sky speed is not representative of the real radial speed because of projection effects.
In our Fig. 10 we had added the 11 correlated CME events without shocks (the magnetic clouds or plasma "blobs").We determined their travel time and added them as green dots.As the figure shows, these points are surprisingly well lined up with the dashed expectation curve: For comparison, we measured as well the travel times of all other identified ejecta that were driving a shock.Then, the result shows slightly less scatter than in Fig. 10 and is very similar to that shown by Gopalswamy et al. (2001c), who calculated a representative prediction curve (assuming a constant deceleration to 0.67 AU) to the 47 data points in their Fig. 8.As a "mean error" they find a value of 10.7 h.Similar to Cane and Richardson (2003b), we consider this value quite misleading since 7 points (15%) lie outside a ±18 h margin.The 95% safety margin would be at more than 24 h, i.e. very similar to ours.Note though that their evaluation is based on "real" ejecta and their travel times only.Gopalswamy et al. (2001c) tried a correction of the V P S values to account for the projection effect.They made use of the ejection source location, but they state that there was no real improvement.Michalek et al. (2002Michalek et al. ( , 2003) ) developed a method for correcting V P S in which they determined the moments of the first appearance of the CME above opposite limbs above the occultor.As a result, they derived "space velocities" of halo CMEs and correlated them with ICME travel times.They claim that they "reduced the average prediction error from ∼10 to ∼5 h".Unfortunately, the data shown by Michalek et al. (2002) do not support their claim.Note further, that this method can only be applied to non-symmetric halos.
When putting our results and conclusions in context with those of other authors, we often note major differences, even when the same data where used.When putting event lists side by side, it turns out that there are always major discrepancies in both definitions and interpretations.There are several "mixed bags" being used, as Gopalswamy et al. (2003) phrased it when criticizing Cane et al. (2000), (see also reply by Cane and Richardson, 2003b).Some authors look for identifiable ejecta, others concentrate on "disturbances" associated with CMEs, others search for geomagnetic response (even using different indices: K p or D st ), others do statistics.Accordingly, the results and conclusions are differing as well.We leave it to the reader to study the vast literature on the subject.In addition to the papers already quoted, we mention only a few recent ones: Berdichevsky et al. (2002); Vilmer et al. (2003); Wang et al. (2002b); Webb et al. (2000); Yermolaev andYermolaev (2002, 2003); Zhang et al. (2003), Zhao and Webb (2003); Huttunen et al. (2004).
Based on our own study, we consider the following summary statement of this section as "safe", hoping that future forecasters can benefit: -For an isolated, undisturbed front side halo CME the shock/ICME arrival time at the Earth can be derived using the empirical Eq. ( 2).There is a 95% probability that the shock will arrive within one day around that predicted time.

Remaining uncertainties for reliable predictions
For practical applications as a prediction tool Eq. ( 2) based on the expansion speed, was found to work rather successfully, as was confirmed for several events that occurred after our analysis interval, i.e. 15 after April 2001.However, there is still an embarrassing number of cases of complete failure: missing alarms and false alarms.
The 91 uniquely associated event pairs were selected with great care, in order to lead us to safe conclusions and to provide a sound prediction tool.However, our complete list of 304 entries includes another 124 "possibly associated" shock/ICME events, i.e. cases of CMEs where an ICME signature could not be ruled out, and the other way round.
In 37 cases, an ICME signature could not be uniquely identified.It might be hidden in turbulent plasma data, or there were missing or just confused CME data.Anyway, a CME/ICME correlation could not be ruled out.For deriving our tool, these cases of uncertain "backward associations" had to be excluded.In practice, this ambiguity should not worry a forecaster who is usually interested in finding "forward associations", i.e. he wants to uniquely predict the arrival of an ICME once a proper CME has actually occurred.
In the other 87 cases (61 front side or limb halos, 50 of them being full or partial halos, 5 backside halos, 21 of unknown type) a CME/ICME association may also exist.We just cannot tell for sure which one out of several potential CMEs had been the actual cause.
A forecaster would deal preferentially with the 50 partial and full halos.They were classified as "possibly associated" since: 1.For 13 cases among them, we could or did not measure the halo expansion speed (too faint, missing or confusing data, no relevance etc.), a general problem every forecaster has to cope with as well.We would advise him to issue only a cautious general warning.There is no doubt that this type of CME causes, in practice, the number of missing alarms increase.
2. In 12 cases we could not claim a safe association; the signatures on both ends seemed to be too vague.However, we were probably too conservative.A brave forecaster using Eq. ( 2) would in fact have done a pretty good job: his predictions would have been fulfilled rather well, and only one further false alarm would have been issued.
3. Twenty-seven of the 50 halo CMEs were multiple events, i.e. they came in too rapidly.This hinders unique associations, or makes them even impossible, when interactions between the ICMEs occur.In the following Sect.9 we will look a bit closer into this issue.

Multiple CMEs
Around the maximum of solar activity, there occur several CMEs per day (St.Cyr et al., 2000;Gopalswamy, 2004).Sometimes they even emerge from the same region on the Sun within hours.Thus, mutual interactions between CMEs can be expected to happen rather often such that their propagation is substantially disturbed.We give a few examples: -From 23 to 26 November in 2000 there occurred a series of 8 full and 2 partial halos (E258 to E267), with expansion speeds between 590 and about 3000 km/s.Within reasonable time windows, no more than 3 shocks hit the Earth, and only 2 moderate and 1 severe geomagnetic storm were caused.Even the extremely fast halo of 25 November, 01:31 (E264), went by almost unnoticed, probably because the ejecta ran into a rather turbulent ambient medium that had been mixed up by the previous 5 CMEs.Note, though, that a shock prediction using our tool based on the first CME alone would not have been off by more than 3 h!This series of events was studied by Wang et al. (2002b), who tried to apply a simple magnetic flux rope model to this multiple magnetic cloud situation. -

Partial and limb halo CMEs
In Table 2 we found that 93.4% (85) out of 91 full front side halo CMEs produced a definite or possible ICME that hit the Earth.This means, as well, that 6.6% were missing the Earth.
For partial and limb halos the situation is more complicated.
Their main propagation direction is not pointing toward the Earth, and yet the Earth may be hit by the CMEs' flanks.
According to Sheeley et al. (1985) the main requirement for a CME to hit the Earth is that its envelope includes the Sun-Earth line.Usually, limb CMEs with their mean cone angles of some 50 • do not fulfill this condition.However, there are also a few CMEs with cone angles of about 180 • , and even a slight propagation component towards the Earth lets them appear as full halos, On the other hand, there are CMEs with comparatively narrow cone angles that emerge from the front side and are off-pointed from the Sun-Earth line.If their apparent (i.e.projected) cone angle includes the Sun-Earth line then the chances are high that they will also hit the Earth.This is generally true for CMEs with apparent cone angles of more than 120 • that we define as "partial" halo CMEs.Note, though, that we had found unique associations in four cases with even smaller cone angles (see Table 2 at http://star.mpae.gwdg.de/cmeeffects/, E92, E196, E219, E222).We wonder which predictions were issued at the time!Inspecting the 49 cases of front side partial halos (limb events not included), we found that 14 of them (29%) had definitely no effect near Earth.For the remaining 35 cases we found 23 certain and 12 potential associations.Inspecting now the 23 limb events (partial and full), we find 4 cases (17.4%) without association, plus 10 certain and 9 possibly associated events.In Fig. 10 we had circled the 10 wellassociated limb events, clearly showing that their travel time to 1 AU is similarly related to the expansion speed as for the front side halos, with a slight trend to larger travel times.Thus, for limb CMEs the same Eq.( 2) can be used for prediction of the travel time to Earth, maybe with a few hours added.We were not successful in finding signatures in the solar observations indicating early on which one of the big limb CMEs would hit or miss the Earth.In fact, on comparing the event types we were surprised by their similarity.
The four limb halos listed in Table 5 did not show any effect at 1 AU.
The famous case E32 (on 6 November 1997) is particularly interesting.It was associated with an X9.4 flare at about S18 • W 60 • .This event had been studied in much detail by Maia et al. (1999).The CME was considered partial at first, but in the catalog it appears as full halo.It was very fast and produced high fluxes of energetic particles penetrating the SOHO cameras.However, no associated ICME ever arrived at the Earth.The cases E201 and E248 probably occurred closely behind the west limb, with E279 slightly in front of the east limb.
For comparison, Table 6 shows the list of 10 wellassociated limb halo CMEs.Most of these events occurred on the visible disk though very close to the east or west limb.Upon comparing the events in the Tables 5 and 6, one hardly notices a difference that might explain why one group of events was clearly hitting the Earth, while the other one missed it.In both groups there are both rather fast and slow events, with similar angular spans and with sources near the limb.In further studies we will have to investigate more details, for example, the history and magnetic topology of the source regions, in order to solve this intriguing puzzle.

Summary
This study makes use of all available observations concerning CMEs near the Sun and their potential interplanetary signatures near the Earth.Data were taken over more than 4 years around the past solar activity maximum, without major data gaps.We went both ways: For each of the 181 relevant CME events observed by LASCO we looked for potential ICME signatures as obtained by several in-situ spacecraft in front of the Earth's bow shock, and for each of the 147 transient shock waves near 1 AU plus 40 ejecta clouds not accompanied by a shock we searched for potential CME sources.A data set with 304 entries covering each single event or event pair was created (http://star.mpae.gwdg.de/cme effects/).
The total number of cases appears to be sufficient for obtaining statistically meaningful correlative results.In 91 cases a unique association between both types of events could be established; in a further 124 cases a possible association cannot be excluded.
The main results are: -From coronagraph observations of limb CMEs and simultaneous in-situ observations from the Helios probes traveling above the respective limbs we learned that there is a good correlation between the radial front speed V rad of limb CMEs and their travel time towards an in-situ observer.
-However, even for that ideal constellation the scatter is surprisingly high, which is probably due to the action of the highly structured and variable interplanetary medium.
-The expansion speed V exp is a fairly reliable proxy for the radial speed V rad for all types of CMEs.This was found from a correlative study of many limb CMEs observed by LASCO for which both quantities could easily be determined.
-On correlating 181 CMEs, on the one side, and 187 ICMEs, on the other side, we found in about 85% of front side halo CMEs, an ICME effect at the Earth can be expected.
-15% of front side halo CME events (7.4% of all front side full halo CMEs) miss the Earth: false alarms.
-Every fourth partial halo CME and every sixth limb halo CME does not hit the Earth.
-In 8.6% of all unique or possible associations nonhalo CMEs were involved.Forecasters focused on halo CMEs would have ignored these CMEs: missing alarms.
-6 of the 147 transient shocks do not have a front side CME source and would not have been predicted.In total, almost 20% of ICME or storm events would not have been predicted: missing alarms.
-At times of high solar activity, CMEs often occur shortly one after the other such that they interact and merge with each other.Their effects at the Earth become highly unpredictable.
-For 75 out of 91 safe associations between CMEs and ICMEs we could determine both the expansion speed V exp near the Sun and the travel time of the associated shock towards the Earth.An empirical Eq. ( 2) was derived which now allows one to predict (for an isolated, undisturbed front side halo CME) the shock/ICME arrival time at the Earth.There is a 95% probability that the shock will arrive within one day around that predicted time, unless it is one of the 15% of ICMEs that never hit the Earth.
We should keep in mind that these are all statistical results, all evaluated after the fact.We are afraid to admit that they are of limited value for the lonely space weather forecaster on duty who is watching near-real-time solar disk and coronagraph images, aiming to issue in time reasonable warnings.
For illustration, we mention only McAllister et al. (2001), who put themselves into that position and describe in detail their "mixed level of success".At the end of a lengthy and very careful analysis of all available information on location, orientation and surrounding magnetic field configuration around a particular halo CME they find substantial differences of their predictions to the actual events near Earth.They frankly admit their "inability to predict whether an encounter will be head-on with the apex of a flux rope loop or, as deduced in this case, a passage through one of its legs."This is where we are.
There is no doubt that the combined action of both spaceborne coronagraphs EUV-disk imagers and in-situ solar wind monitors, all operating 24 h per day, has brought space weather analysis and predictions to an unprecedented quality and precision.We think that the remaining rate of 20% missing alarms and 15% false alarms is not bad compared to what was available before.This is what we need for further improving the situation: -The present spacecraft fleet will not live forever.Replacement missions are urgently needed for SOHO, ACE, and Wind, in order to maintain a continuous survey of the Sun and interplanetary space.
-Stereoscopic views of CMEs approaching Earth will allow for more precise predictions.We are looking forward to the launch of the STEREO spacecraft.As a complement, solar wind monitors should be stationed in front of the Earth at various heliospheric distances, as proposed, e.g. by Lindsay et al. (1999a,b).
-Extensive computer models should be brought into operation that allow one to predict CME propagation through the interplanetary medium in a realistic way in near-real time.There are several promising computer codes under development, with very encouraging results (e.g.Riley et al., 2001;Linker et al., 2003;Mozer and Briggs, 2003;Cho et al., 2003;Fry et al., 2003;Manchester et al., 2004;Odstrcil et al., 2004;Riley and Crooker, 2004).
-Basic research must go on such that finally the onset of solar transient events can be predicted (with respect to location, onset time, strength and significance) well before they actually occur.
One last word of caution: According to our study, space weather forecasters can now make fairly good predictions about whether or not a shock or ICME will hit the Earth.However, predicting potential geoeffects still remains much more difficult and relies mainly on empirical techniques (e.g.Gonzalez and Tsurutani, 1987).The magnetic topology of magnetic clouds to be expected can probably be modeled soon on the basis of their source filament topologies.In order to do so, continuous high quality recordings of the magnetic topology around active regions and filaments are needed (as pointed out by Webb et al., 2000b) that allow one to infer magnetic features of ICMEs (leading to geoeffects) should they be launched eventually.Note, though, that about half of all ICMEs do not exhibit a magnetic cloud structure when reaching Earth.Their potential geoefficiency arises from the magnetic field topology of the sheath plasma piled up in front of and around the ejecta cloud.At present, we cannot yet predict the strength and duration of B z excursions in a shock sheath.The detailed topology depends on the relative locations of the Earth, the ICME, and the heliospheric current sheet.Real progress will hinge on the development of models that include the propagation of structured ICMEs through a structured 3-D heliosphere.

Fig. 1 .
Fig. 1.The X17 flare and the halo CME of 28 October 2003, as seen by the SOHO instruments EIT (a), LASCO C2 (b) and C3(c).This most dramatic event caused major disturbances in space weather and affected the Earth's system in various ways: charged particles were accelerated to near-relativistic energies, such that they could penetrate spacecraft skins and disturbed, for example, the CCD cameras on SOHO (see c) and other satellite instrumentation; an extremely fast interplanetary shock wave was initiated that reached the Earth only 19 h later; a severe geomagnetic storm (D st

Figure 2 .
Figure 2. The halo CME of 5 November 19 expansion speed V exp and the plane of the s km/s, and V PS = 1118 km/s, A shock arriv earlier than we predicted (case E20 in Tabl 14 hours after shock arrival.The images are

Fig. 2 .
Fig.2.The halo CME of 5 November 1998 (E90), for illustration of the different meaning of expansion speed V exp and the plane of the sky speed V P S .In this case we measured V exp =1116 km/s, and V P S =1118 km/s, A shock arrived at the Earth 55.6 h later, i.e. about 7 h earlier than we predicted (case E20 in Table2).A severe magnetic storm (D st −142 nT) peaked 14 h after shock arrival.The images are running differences between LASCO C3 images.

Figure 3 .
Figure 3.The events associated with the interplane tary shock wave of 19 July 1980, as observed by the Helios 1 spaceprobe from a position at 0.53 AU distance to the Sun and at 92 0 west of the Earth-Sun line, i.e. right above the Sun's west limb as seen from the Earth.The 6 bottom panels show (from top) the IMF north-south (?B) and east-west (f B) direction, the field magnitude, and the solar wind speed, temperature and density.The upper panel shows the azimuthal flow angle f E of suprathermal electrons (221 eV), with the anti-Sun direction at 0 0 .This is one of the cases for which Sheeley et al. (1985) could find a uniquely correlated CME. Figure courtesy Kevin Ivory, MPS.

Fig. 3 .
Fig. 3.The events associated with the interplanetary shock wave of 19 July 1980, as observed by the Helios 1 spaceprobe from a position at 0.53 AU distance to the Sun and at 92 • west of the Earth-Sun line, i.e. right above the Sun's west limb as seen from the Earth.The 6 bottom panels show (from top) the IMF north-south (θ B ) and east-west (φ B ) direction, the field magnitude, and the solar wind speed, temperature and density.The upper panel shows the azimuthal flow angle φ E of suprathermal electrons (221 eV), with the anti-Sun direction at 0 • .This is one of the cases for which Sheeley et al. (1985) could find a uniquely correlated CME. Figure courtesy of Kevin Ivory, MPS.

Figure 4 .
Figure4.The travel time of limb CME fronts from the Sun to the location of Helios 1 (at about 90 0 off the Earth-Sun line) as function of the CME radial speed V rad obtained from the SOLWIND coronagraph.Only those cases were selected where a unique association could be achieved, in particular when Helios 1 was with +/-30 0 near the plane of the sky, and when the angular span of the CME included the Helios orbital plane.The travel time was derived from the moment of first appearance in a coronagraph image and the shock arrival time at Helios 1.The projected travel times to 1 AU as given in the figure were determined by simply assuming that the shocks would travel from the Sun through Helios and all the way to 1 AU at the same average speed .All data were adopted from Table1ofSheeley et al. (1985).The dotted line denotes the "ideal" line, i.e., where the travel time would exactly correspond to the CME radial speed near the Sun.

Figure 5 .
Figure 5. Self-similarity of CMEs: the limb CME of 19 October 1997 is a typical example showing how well the opening cone angle and the general shape of a CME are maintained, at least up to 32 Rs.The term "cone angle" denotes the angle between the outer edges of opposing flanks of limb CMEs.It would amount to 65 0 in this case.The images are running differences between LASCO-C3 images.

Fig. 5 .Figure 6 .
Fig. 5. Self-similarity of CMEs: the limb CME of 19 October 1997 is a typical example showing how well the opening cone angle and the general shape of a CME are maintained, at least up to 32 R s .The term "cone angle" denotes the angle between the outer edges of opposing flanks of limb CMEs.It would amount to 65 • in this case.The images are running differences between LASCO-C3 images.

Figure 7 .
Figure 7.The correlation between radial CME speed Vrad and the lateral expansion speed Vexp for limb CMEs observed by LASCO between January 1997 and April 15, 2001.

Fig. 7 .
Fig. 7.The correlation between radial CME speed V rad and the lateral expansion speed V exp for limb CMEs observed by LASCO between January 1997 and 15 April 2001.

8Figure 8 .
Figure 8.The extremely fast partial halo limb CME of 20 April 1998 (E58) was associated with a M1.6 flare not far behind the west limb.The images are running differences between LASCO C3 images.The expansion angle was about 180 0 .The expansion speed was 1930 km/s, the plane of the sky speed was 1944 km/s.A shock arrived at the Earth 79.1 hours later, i.e., about 24 hours later than we predicted.A moderate magnetic storm (Dst -59 nT) peaked 13 hours after shock arrival.

Fig. 8 . 9 Figure 9
Fig. 8.The extremely fast partial halo limb CME of 20 April 1998 (E58) was associated with a M1.6 flare not far behind the west limb.The images are running differences between LASCO C3 images.The expansion angle was about 180 • .The expansion speed was 1930 km/s; the plane of the sky speed was 1944 km/s.A shock arrived at the Earth 79.1 h later, i.e. about 24 h later than we predicted.A moderate magnetic storm (D st −59 nT) peaked 13 h after shock arrival.

Table 1 .
Summary of all halo CME observations by LASCO from January 1997 to 15 April 2001.

Table 2 .
Association of CME events (back side events excluded) with ICME observations at 1 AU.

Table 3 .
Association of ICME events with their potential CME sources.

Table 4 .
The 6 shock and 2 storm events definitely not associated with any CME on the front side or near the limbs within a 120-h time window.The meaning of the columns: (A) number of events in catalog at http://star.mpae.gwdg.de/cmeeffects/, (B) S stands for shock occurrence, n stands for definite absence of shock or ICME, (C) time of shock arrival, (D) local shock speed, in km/s, (E) K p maximum in succeeding storm, (F) time of K p maximum, (G) D st minimum in succeeding storm, in nT, (H) time of D st minimum.
The analysis of these multiple events allows us to give some advice to the brave forecaster.He would of course be alerted by the first halo CME and make his prediction, not knowing what might follow.If a second halo actually occurs within 24 h, the prediction for the first one would still remain valid, unless the second CME is at least twice as fast as the first one.In this case, he should revise his prediction, based on the expansion speed of the second CME, but with a warning sign attached.For the second and any subsequent CME running into the aftermaths of the other ones, no quantitative prediction can be trusted.
-On 15 and 16 September 2000 there occurred 3 full and 1 partial halo within 18 h (E238 to E241).The last one was particularly fast (V exp =1509 km/s).Nevertheless,

Table 5 .
Limb halo CMEs without any association to ICMEs near Earth.The meaning of the columns: (A) number of event in catalog at http://star.mpae.gwdg.de/cmeeffects/, (B) time of first appearance of CME front in LASCO-C2 field-of-view, (C) angular width of CME, (D) position angle of fastest CME feature (measured counterclockwise from North Pole), (E) plane of the sky speed V P S (in km/s) of fastest CME feature, measured between 10 and 20 R s using C3, (F) expansion speed V exp (in km/s).

Table 6 .
Well-associated limb halo CMEs.The meaning of the columns: (A) number of events in catalog at http://star.mpae.gwdg.de/cme effects/, (B) time of first appearance of CME front in LASCO-C2 field-of-view, (C) angular width of CME, (D) position angle of fastest CME feature (measured counterclockwise from North Pole), (E) plane of the sky speed V P S (in km/s) of fastest CME feature, measured between 10 and 20 R s using C3, (F) expansion speed V exp (in km/s), (G) detection of shock (S), plasma blob (B), (H) arrival time of shock or ICME at 1 AU (in ACE, Wind or SOHO data), (I) local shock speed (in km/s).No entry in case of no shock, (J) travel time (in hours) between Sun and 1 AU, determined from columns B and H.