2 weeks Ago By Laura Dobberstein
Servers that store data collected by two NASA solar observation satellites are down – and the space agency doesn't know when they'll resume operations – after a four-inch chilled water pipe burst at the facility that houses them. The pipe popped last Tuesday at Stanford's Joint Science Operations Center, which houses infrastructure used to handle data from the Solar Dynamics Observatory (SDO) and the Region Imaging Spectrograph (IRIS). The Center (JSOC) handles data from two of SDO's three scientific instruments: the Helioseismic and Magnetic Imager (HMI) instrument and Atmospheric Imaging Array (AIA).
Several inches of water pooled in the JSOC server room. By the next day it had been drained, but equipment needed to be dried. Ironically, hanging it in the sunshine was not an option.
"An initial assessment is that the water affected many systems but did not provide a time for return to service," explained a mission blog post. JSOC referred to the damage as "severe" and a complete recovery is not expected until next year. No estimate was provided on how much rice might be required.
The machine comprises two linked systems. System "M" runs 23,296 Intel Xeon Max 948 cores spread across 208 nodes and packs 26.6TB of memory.
System "P" uses the Xeon Platinum 8480+ and 40.96TB of memory. Both of the processors used are Sapphire Rapids parts that Intel launched in 2023.
The supercomputer will be used for simulations that assist theoretical astronomers to do things like modeling how stars are born. – Simon Sharwood NASA and the University haven't detailed what got wet, but the SDO in its entirety delivers around 42TB of data each month. That number that may seem trivial now, but it was more significant when the observatory launched in 2010 and could require plenty of rack space if kit hasn't been upgraded.
"Science data processing for HMI, AIA, and IRIS will be down for an extended length of time, as will access to the archived data at JSOC," it added. Although data is unavailable, it is not lost. It is still being collected – just not processed.
HMI data, for example is currently being stored in New Mexico. But in the meantime, realtime data needed for space weather, space environment data products, and mission planning is inaccessible. As IRIS makes its near-realtime data available online , data from before November 22 – four days before the pipe burst – is available.
Incomplete AIA and HMI data can also be accessed from various sources. The SDO is a space-based observatory that studies the Sun, focusing on its magnetic field, activity cycles, and impacts on the solar system. It is particularly focused on understanding the connection between the Sun and Earth.
IRIS launched in 2013, and studies the flow of plasma into the corona and heliospere of our local star. It is also intended to advance the understanding of the Sun-Earth relationship. ®.
