Add variables to Hi L1B DE product#2749
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the IMAP-Hi L1B Direct Event (DE) product generation to include two new epoch-dimension variables that carry (1) a floating-point ESA step MET timestamp and (2) a CCSDS packet-level quality flag bitmask.
Changes:
- Add
esa_step_metderived fromesa_step_seconds+esa_step_milliseconds. - Add
ccsds_qfbitmask withPACKET_FULL(bit 0) set for packets containing 664 events. - Extend variable attribute configuration and add unit tests for the new helpers.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| imap_processing/hi/hi_l1b.py | Adds de_esa_step_met() and de_ccsds_qf() and injects their outputs into the L1B DE dataset. |
| imap_processing/quality_flags.py | Introduces ImapHiL1bDeFlags IntFlag for DE packet-level quality bits. |
| imap_processing/cdf/config/imap_hi_variable_attrs.yaml | Adds attribute definitions for hi_de_esa_step_met and hi_de_ccsds_qf. |
| imap_processing/tests/hi/test_hi_l1b.py | Updates dataset variable count expectations and adds unit tests for the new functions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
| # Filter out fill/out-of-range indices (e.g., uint16 FILLVAL 65535) | ||
| valid_mask = (ccsds_indices >= 0) & (ccsds_indices < n_packets) | ||
|
|
||
| # If there are no valid events, all packets keep default quality flag 0 |
There was a problem hiding this comment.
I would think that if there were no valid events you would want everything flagged? Not with ImapHiL1bDeFlags.PACKET_FULL but another one?
There was a problem hiding this comment.
I could add another flag to indicate an empty packet but currently, the only flag I have defined indicates that the CCSDS packet contained the maximum number of events. You are correct that a packet having no packets probably indicates a problem but that may be caught elsewhere.
d4ad77d
into
IMAP-Science-Operations-Center:dev
Change Summary
Overview
Adds two new variables to the Hi L1B Direct Event product
New Variables
- MET time when the ESA was stepped, computed from esa_step_seconds + esa_step_milliseconds / 1000
- Provides a convenient floating-point representation of the ESA step timestamp
- Quality flag bitmask for each CCSDS packet
- Bit 0 (PACKET_FULL): Set when packet contains 664 events (maximum capacity)
Implementation Details
Files Changed
Closes: #2746