Supporting the launch of Rongowai

Rongowai (Rongo – to sense; wai – water) is an international collaborative mission hosting a next-generation Global Navigation Satellite System Reflectometry (GNSS-R) payload on board a commercial Q300 aircraft to collect climate data about New Zealand during flight. The mission is part of NASA’s Cyclone Global Navigation Satellite System (CYGNSS) mission, and along with Air New Zealand, includes the University of Auckland, the University of Michigan responsible for building the device, Ministry of Business, Innovation, and Employment (MBIE), New Zealand Space Agency, Ohio State University and University of Canterbury. The Centre for eResearch at the University of Auckland plays a crucial role in the mission; communicating with the remote payload whenever it lands, ensuring that science and diagnostic data is downloaded, and subsequently processing the data into usable science products fit for research and eventually the wider community.

In the build up to the Rongowai “launch” date of the 13th September 2022 data pipelines were built and tested to ensure that the payload could communicate with the Science Payload Operation Centre (SPOC) infrastructure, and small webtools created to help the Air New Zealand installation crew monitor the status of the payload. After a tense hour on the morning of launch date, with the wider team tracking the maiden flight of Rongowai from Christchurch to Nelson using the mission’s live-flight tracker, the team breathed a sigh of relief and a cheer of celebration as the first flight data of Rongowai was automatically downloaded upon arrival in Nelson!

Automatic data retrieval from the NGRx

Rongowai flies roughly between 5-10 times a day, every day, from the early hours of the morning to through to midnight. It relies on a cellular modem to communicate with the SPOC when the plane has landed and flies to almost 20 airports around New Zealand each with varying levels of cellular reception. As such automated communication with Rongowai is required to ensure smooth operational status of the mission. When the payload lands it establishes a reverse SSH tunnel to the SPOC using its cellular modem, which in turn triggers a series of data transfer scripts to download binary data from the payload. The scripts are designed to be robust enough to handle partial or interrupted downloads which can be caused by poor cellular connection or by aircraft events such as a craft powering down or taking off on its next flight. Included in the downloaded data are log files related to the onboard operations of Rongowai, and also payload engineering data used to monitor a wide range of values including component voltages, currents, temperatures, and more. Engineering data is automatically ingested into a near-live health dashboard for the Rongowai team to easily monitor the operational status of Rongowai, as well as the archiving of binary data to long term storage in case historical data is required in the future.

Automatic generation of Level-0 and Level-1 data products

Both engineering and science data are downloaded from the payload in the form of binary data to ensure that the data is as small as possible when storing on board the remote payload and downloading over constantly changing cellular bandwidths. As a result the raw data retrieved from the Rongowai payload needs to be decoded from raw binary into more science-ready formats before it can be easily used.

Nightly scripts are automatically run to convert the raw binary files into a series of decoded Level-0 netCDF4 files that contain all the engineering and science data pertaining to each of the flights from that day. These files are then used to generate a new Level-1 netCDF4 file from which science algorithms can be developed and research performed. Both Level-0 and Level-1 files are archived to long term storage.

The raw binary data has roughly a 10x decompression rate into netCDF4 files, a subtype of HDF5 files, and a typical file size for both Level-0 and Level-1 products is almost 300MB per 30 minutes of flying. With a typical flight time of 1 hour, and ~5 flights a day, Rongowai is set to produce ~6GB of binary, Level-0, and Level-1 data per day!