Metadata-Conditioned Dark-Frame Modelling and HSDR-ScanFusion Denoising
View Paper on Small Sat Website ↗
Sung Jegal's (the paper's presenter) Github ↗
Collaborated with the lead author and the rest of the team to create diagnostics and reproduced performance figures of other models. This ensured the model trained correctly and enabled direct benchmarking against other industry methods.
1. Explanation of Findings
Reasoning
Hyperspectral sensors are very sensitive to heat, especially when they are mounted on satellites which
lack mass for the heat to go. This is especially a
problem for the small satellite that the FINCH mission would be using, which doesn't have the mass or
energy budget for active cooling or mechanical shutters.
For this reason, a software calibration tool is used instead, which doesn't require any mass, and is a
post-processing step after the information is sent back to Earth.
Method
A physics calibration tool is used instead that estimates the amount of dark noise through simulation
(dark noise is caused
by the heat of the sensor generating electrons within the sensor, which gets detected as an actual
signal). This is
used to create a large amount of synthetic data for the neural network to train on.
The neural network is then trained with some of the metadata from the capture, including temperature,
exposure time, gain (this is how the collected electrons were scaled when they were transformed into
digital numbers for the computer) and read-noise proxy (this is the standard deviation of the baseline
readout noise).
Finally, the model outputs a predicted dark frame which matches the exact dimensions of the hyperspectral
cube to perform direct subtraction with. This removes some of the dark noise before the other model can
remove as much as possible of the remaining dark noise. After this step, a secondary network
(HSDR-ScanFusion) restores the final image.
Results
Here are the metrics being used:
- Peak Signal-to-Noise Ratio (PSNR): Measures how close the image is to the ground truth (the accurate data). The higher the PSNR, the stronger the signal (the important part) is compared to the noise (the unimportant part)
- Structural Similarity Index (SSIM): Measures the similarity between two images. The closer this value is to 1 the better (that would mean the denoised image is exactly the same as the ground truth).
- Spectral Angle Mapper: How well the shape of the spectral signature is preserved between bands. If this is too high, it is impossible to distinguish between different materials (which is important for FINCH as it was meant to detect crop residue).
- During training, the model was not provided any frames of the sensor at 10 degrees Celcius. In the real hardware tests at this unseen temperature, the model achieved a Peak Signal-to-Noise Ratio of 40.17 dB and a Structural Similarity Index of 0.982. Essentially, this means it was able to significantly decrease the noise.
- The simulated synthetic dark frames matched the physical reality of the real camera.
- The model improved the uncorrected image's PSNR from 19.43 to 38.73 dB, meaning it was significantly more accurate to the ground truth (note that the PSNR scale is logarithmic)
- The spectral angle error decreased from 12.06 degrees to 3.74 degrees. This would make it significantly easier to distinguish between different materials (the whole point of the FINCH mission)
2. Conference Gallery
Images of the presentation at the SmallSat Conference. The presenter of the paper is Sung Jegal.
