GRASS logo

NAME

i.dsen2cr.opencl - Removes clouds from Sentinel-2 L1C imagery with the DSen2-CR network and Sentinel-1 SAR-optical fusion on an OpenCL device.

KEYWORDS

imagery, cloud removal, Sentinel, SAR, fusion, deep learning, GPU, OpenCL

SYNOPSIS

i.dsen2cr.opencl
i.dsen2cr.opencl --help
i.dsen2cr.opencl coastal=name blue=name green=name red=name rededge1=name rededge2=name rededge3=name nir=name nir08=name nir09=name cirrus=name swir16=name swir22=name [vv=name] [vh=name] output=basename model=name [scale=float] [offset=float] [tile=integer] [device=string] [platform=string] [--overwrite] [--help] [--verbose] [--quiet] [--ui]

Flags:

--overwrite
Allow output files to overwrite existing files
--help
Print usage summary
--verbose
Verbose module output
--quiet
Quiet module output
--ui
Force launching GUI dialog

Parameters:

coastal=name [required]
Sentinel-2 B01 (coastal aerosol, 60 m)
blue=name [required]
Sentinel-2 B02 (blue, 10 m)
green=name [required]
Sentinel-2 B03 (green, 10 m)
red=name [required]
Sentinel-2 B04 (red, 10 m)
rededge1=name [required]
Sentinel-2 B05 (red edge 1, 20 m)
rededge2=name [required]
Sentinel-2 B06 (red edge 2, 20 m)
rededge3=name [required]
Sentinel-2 B07 (red edge 3, 20 m)
nir=name [required]
Sentinel-2 B08 (NIR, 10 m)
nir08=name [required]
Sentinel-2 B8A (narrow NIR, 20 m)
nir09=name [required]
Sentinel-2 B09 (water vapour, 60 m)
cirrus=name [required]
Sentinel-2 B10 (SWIR cirrus, 60 m)
swir16=name [required]
Sentinel-2 B11 (SWIR 1.6 um, 20 m)
swir22=name [required]
Sentinel-2 B12 (SWIR 2.2 um, 20 m)
vv=name
Sentinel-1 VV backscatter in dB
Required by the models trained with SAR input
vh=name
Sentinel-1 VH backscatter in dB
Required by the models trained with SAR input
output=basename [required]
Basename for output raster maps, suffixed with the band name (e.g. _B02)
model=name [required]
DSen2-CR weights (safetensors)
Converted from a published DSen2-CR checkpoint with dsen2cr_convert.py
scale=float
Scale factor of the optical input values
Reflectance = (value + offset) / scale; use 1 for inputs already in reflectance
Default: 10000
offset=float
Offset added to the optical input values before scaling
E.g. -1000 for products of processing baseline 04.00 or later; outputs are returned in the input units
Default: 0
tile=integer
Side of the output tile in cells
Each tile is computed with a halo of 2 * blocks + 2 cells on every side; larger tiles waste less work on halos but need more device memory
Options: 16-8192
Default: 512
device=string
OpenCL device type
Options: auto, gpu, cpu
Default: auto
platform=string
Only use OpenCL platforms whose name contains this string (case-insensitive)

Table of contents

DESCRIPTION

i.dsen2cr.opencl removes clouds and cloud shadows from Sentinel-2 Level-1C images with DSen2-CR, the deep residual network of Meraner et al. (2020). The network fuses the cloudy optical image with a Sentinel-1 SAR image of the same area, which sees through clouds, and predicts the cloud-free reflectance of all 13 Sentinel-2 bands. It runs on an OpenCL device (GPU or CPU) and reads the published model weights, converted once to a safetensors file. Neither Python nor a deep learning framework is needed at run time.

The 13 Sentinel-2 bands are given through their own options, named after the STAC common band names: coastal (B01), blue (B02), green (B03), red (B04), rededge1 (B05), rededge2 (B06), rededge3 (B07), nir (B08), nir08 (B8A), nir09 (B09), cirrus (B10), swir16 (B11) and swir22 (B12). The Sentinel-1 backscatter is given with vv and vh.

The output maps are named after output, suffixed with the band name, e.g. output_B02. They cover the current computational region, are in the units of the optical input and get the colour table of the matching input band. The network predicts every cell, cloudy or not: trained with the cloud-adaptive regularized loss (CARL), it keeps clear cells close to the input and reconstructs the cloudy ones.

Input data

DSen2-CR is trained on the SEN12MS-CR dataset, and its inputs must match that data:

The network normalises its inputs as in training: optical digital numbers are clipped to [0, 10000] and divided by 2000, VV is clipped to [-25, 0] dB and VH to [-32.5, 0] dB, and both are mapped to [0, 2].

Models

The authors published four checkpoints (Keras HDF5 files, see the DSen2-CR repository): the full model trained on the CARL loss (recommended), the same network trained on a plain L1 loss, and the two variants without SAR input. Convert a checkpoint once with dsen2cr_convert.py, which needs the Python package h5py (or PyTorch, for a state dict of the PyTorch port):

python3 dsen2cr_convert.py model_SARcarl.h5 dsen2cr_sar_carl.safetensors

The script is installed with the module in etc/i.dsen2cr.opencl/ of the GRASS installation (or of $GRASS_ADDON_BASE for an addon installed with g.extension). The safetensors file is then given with model.

The model tells which inputs are needed: vv and vh are required by the SAR models and refused by the optical-only ones, so a band is never silently ignored. The number of residual blocks and of feature channels is read from the file (16 and 256 in the published models).

NOTES

Tiles

The region is processed in square tiles of tile x tile cells. Each tile is computed with a halo of 2 x blocks + 2 cells (34 cells for the published models), which is the reach of the network. The tile cores are therefore exactly what the network gives over the whole region at once, with no seams. Larger tiles waste less computation in the halos but need more device memory: about 1 GiB for 512-cell tiles (the default) and 3.6 GiB for 1024-cell tiles with the published models. If the tiles do not fit, the module stops and asks for a smaller tile.

Null cells

A cell that is null in any input band is null in every output band. For the network input, null cells are filled with the mean of their band over the region, as the reference implementation does with NaN values. The fill only affects valid cells within 34 cells of the null ones.

Performance

DSen2-CR is computationally heavy: about 39 million floating point operations per cell for the published models. On an AMD Radeon Pro WX 7100 (Mesa Clover OpenCL) a 1024 x 1024 region takes about 30 seconds with tile=1024, and a full Sentinel-2 granule (10980 x 10980 cells) takes about 55 minutes. CPU devices are roughly 100 times slower (PoCL on a 4-core Intel Core i7-1165G7: 45 seconds for a 64 x 64 region) and only suited to small regions.

Accuracy

The OpenCL implementation was checked against a PyTorch forward pass (in double precision) of the published CARL model: the outputs agree to within 0.015 digital numbers in every band.

On real data (Kanpur, India: Sentinel-2 L1C of 7 May 2025 with 32 % opaque cloud, Sentinel-1 of 4 May, clear Sentinel-2 of 10 May as reference, 2522 x 2506 cells, 250 seconds on the WX 7100), the mean absolute difference to the clear scene under opaque cloud drops from 2000-3300 digital numbers in the input to 90-630 in the output, in every band. Under thick cloud the reconstruction is smooth and darker than the reference, and cloud shadows are kept: the SAR carries the structure (roads, field patterns) but not all of the optical detail. B09 (water vapour) comes out about 300 digital numbers too high everywhere, an atmospheric band the network fills in from its training data.

EXAMPLES

Cloud removal with SAR

All bands imported at 10 m (e.g. with i.sentinel.import and r.resamp.interp), Sentinel-1 backscatter in dB on the same grid:

g.region raster=T31TCJ_B02
i.dsen2cr.opencl coastal=T31TCJ_B01 blue=T31TCJ_B02 green=T31TCJ_B03 \
    red=T31TCJ_B04 rededge1=T31TCJ_B05 rededge2=T31TCJ_B06 \
    rededge3=T31TCJ_B07 nir=T31TCJ_B08 nir08=T31TCJ_B8A \
    nir09=T31TCJ_B09 cirrus=T31TCJ_B10 swir16=T31TCJ_B11 \
    swir22=T31TCJ_B12 vv=S1_VV_dB vh=S1_VH_dB offset=-1000 \
    model=$HOME/models/dsen2cr/dsen2cr_sar_carl.safetensors \
    output=T31TCJ_cr tile=1024
d.rgb red=T31TCJ_cr_B04 green=T31TCJ_cr_B03 blue=T31TCJ_cr_B02

Keeping the clear cells

To replace only the cloudy cells, combine the result with a cloud mask (here clouds: 1 for cloud and shadow, null elsewhere, e.g. from i.sentinel.mask):

r.mapcalc "B04_filled = if(isnull(clouds), T31TCJ_B04, T31TCJ_cr_B04)"

REFERENCES

Meraner, A., Ebel, P., Zhu, X. X., Schmitt, M., 2020. Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion. ISPRS Journal of Photogrammetry and Remote Sensing 166, 333-346. doi:10.1016/j.isprsjprs.2020.05.013

Ebel, P., Meraner, A., Schmitt, M., Zhu, X. X., 2020. Multisensor data fusion for cloud removal in global and all-season Sentinel-2 imagery. IEEE Transactions on Geoscience and Remote Sensing. doi:10.1109/TGRS.2020.3024744 (the SEN12MS-CR dataset)

DSen2-CR source code and checkpoints: github.com/ameraner/dsen2-cr

SEE ALSO

i.omnicloudmask, i.sen2sr.opencl, i.sentinel.import, i.sentinel.mask, r.fillnulls

AUTHORS

Yann Chemin

SOURCE CODE

Available at: i.dsen2cr.opencl source code (history)

Accessed: Sunday Nov 15 10:56:42 2026


Main index | Imagery index | Topics index | Keywords index | Graphical index | Full index

© 2003-2026 GRASS Development Team, GRASS 8.5.1dev Reference Manual