GRASS logo

NAME

i.sam.opencl - Segments a 3-band imagery group with the Segment Anything Model (SAM) on an OpenCL device and writes the segments as vector areas.

KEYWORDS

imagery, segmentation, object recognition, deep learning, GPU, OpenCL, vectorization

SYNOPSIS

i.sam.opencl
i.sam.opencl --help
i.sam.opencl [-n] group=name [subgroup=name] [checkpoint=name] [weights_dir=name] [model=string] output=name [raster=name] [percentile=lower,upper] [points_per_side=integer] [iou_threshold=float] [stability_threshold=float] [stability_offset=float] [nms_threshold=float] [crop_layers=integer] [crop_points_downscale=integer] [minsize=integer] [maxsize=integer] [overlap=string] [simplify=float] [tile_size=integer] [tile_overlap=integer] [batch=integer] [device=string] [platform=string] [profile=string] [precision=string] [--overwrite] [--help] [--verbose] [--quiet] [--ui]

Flags:

-n
Do not download a missing checkpoint into the weights folder
--overwrite
Allow output files to overwrite existing files
--help
Print usage summary
--verbose
Verbose module output
--quiet
Quiet module output
--ui
Force launching GUI dialog

Parameters:

group=name [required]
Name of input imagery group with exactly 3 raster maps (R, G, B)
subgroup=name
Name of subgroup with exactly 3 raster maps to use instead of the whole group
checkpoint=name
SAM checkpoint file
Original PyTorch checkpoint, e.g. sam_vit_h_4b8939.pth (vit_h, vit_l or vit_b)
weights_dir=name
Folder of SAM checkpoints
Folder of .pth checkpoints named with vit_h, vit_l or vit_b, where missing ones are downloaded; default: $SAM_MODELS_DIR/sam1, else ~/models/sam/sam1
model=string
SAM model to use from the weights folder
Default: the best one present (vit_h, then vit_l, then vit_b), or vit_h if there is none
Options: vit_h, vit_l, vit_b
vit_h: ViT-H, 2.4 GiB, best quality
vit_l: ViT-L, 1.2 GiB
vit_b: ViT-B, 358 MiB, fastest
output=name [required]
Name for output vector map of segments
raster=name
Name for optional output raster map of segment categories
percentile=lower,upper
Percentiles of each band stretched to the 0-255 range fed to SAM
Options: 0-100
Default: 2,98
points_per_side=integer
Number of prompt points along each side of a crop
Options: 1-128
Default: 32
iou_threshold=float
Minimum mask quality predicted by the model
Options: 0-1
Default: 0.88
stability_threshold=float
Minimum mask stability under a shift of the logit cutoff
Options: 0-1
Default: 0.95
stability_offset=float
Logit cutoff shift used to compute the stability score
Default: 1.0
nms_threshold=float
Box IoU above which duplicate masks are suppressed
Options: 0-1
Default: 0.7
crop_layers=integer
Number of zoomed-in crop layers (layer n has 4^n crops)
Options: 0-3
Default: 0
crop_points_downscale=integer
Divide points_per_side by this factor in each crop layer
Options: 1-8
Default: 1
minsize=integer
Minimum segment size in cells
Smaller holes and islands are removed from masks and smaller pieces merged into their neighbours
Options: 0-
Default: 100
maxsize=integer
Maximum mask size in cells
Larger masks, typically a background mask spanning the whole scene, are discarded
Options: 1-
overlap=string
Mask kept where masks overlap
Options: small, large, score
Default: small
small: smallest mask (finest partition)
large: largest mask (coarsest partition)
score: mask with the highest predicted IoU
simplify=float
Boundary simplification tolerance in map units
Douglas-Peucker tolerance; default is one cell, 0 keeps the raster staircase
Options: 0-
tile_size=integer
Side of the tiles, in cells, a larger region is split into
Options: 64-
Default: 1024
tile_overlap=integer
Overlap between tiles in cells
Options: 0-
Default: 256
batch=integer
Number of point prompts decoded at once on the device
Options: 1-256
Default: 32
device=string
OpenCL device type
Options: auto, gpu, cpu
Default: auto
platform=string
Use only OpenCL platforms whose name contains this text
profile=string
Device profile choosing buffers and kernels
Options: auto, cpu, igpu, dgpu, egpu
Default: auto
auto: Detect from the device
cpu: Host-memory device (PoCL): mapped buffers, CPU kernels
igpu: GPU sharing host memory: mapped transfers
dgpu: GPU with its own memory on internal PCIe
egpu: GPU behind a slow link (Thunderbolt, remote): pinned staging
precision=string
Storage type of the weight matrices
Options: fp32, fp16
Default: fp32
fp32: Single precision
fp16: Half precision storage, single precision arithmetic

Table of contents

DESCRIPTION

i.sam.opencl segments a 3-band imagery group with the Segment Anything Model (SAM) and writes the segments as a clean vector area map. The whole network, image encoder and mask decoder, runs on an OpenCL device (GPU or CPU), with weights read directly from the original PyTorch checkpoint file: neither Python nor PyTorch is needed at run time.

The module implements SAM's automatic mask generation: the model is prompted with a regular grid of points, and the masks it returns are kept according to their predicted quality and stability, de-duplicated and cleaned. The overlapping masks are then turned into a non-overlapping, topologically clean vector map with one attribute row per segment.

The group (or its subgroup) must contain exactly three raster maps, which are used as the red, green and blue channels in the order they are listed in the group (see i.group). Each band is stretched linearly between its lower and upper percentile to the 0-255 range SAM was trained on. Cells that are NULL in any band are not segmented. Processing is done in the current computational region.

Checkpoint

The checkpoint is one of the original model files published with SAM; the model variant is detected from the tensor shapes:

The weights are too large to be installed with the module. Without a checkpoint file, the module looks for one in the weights folder: weights_dir, by default the central folder $SAM_MODELS_DIR/sam1, or ~/models/sam/sam1 when the environment variable SAM_MODELS_DIR is not set. If the model is not there, its checkpoint is downloaded into that folder (created if needed) on first use and reused afterwards: the model chosen with model, or ViT-H when the folder holds no checkpoint at all. The download is checked against the published size, resumes where an interrupted one stopped, and waits for another run downloading the same file. The -n flag forbids downloading: a missing checkpoint is then an error.

A checkpoint is recognised in the weights folder by vit_h, vit_l or vit_b in its .pth file name, so renamed copies such as sam_vit_h_checkpoint.pth are found too. The best model present is used (ViT-H, then ViT-L, then ViT-B), or the one chosen with model; the module checks that the file really holds that model. If a folder holds several files of the chosen model, the first in alphabetical order is used, with a warning.

Mask generation

The region is processed in square crops. A region no larger than tile_size cells is one crop, a larger region is split into tiles of tile_size cells overlapping by tile_overlap cells. With crop_layers greater than 0, each tile is additionally segmented in zoomed-in sub-crops (4 in layer 1, 16 in layer 2, ...), prompted with points_per_side divided by crop_points_downscale per layer.

Each crop is resized so that its longest side is 1024 pixels and passed through the image encoder. The mask decoder is then prompted with a grid of points_per_side x points_per_side points, batch prompts at a time, returning three candidate masks per point. A mask is kept if its predicted IoU exceeds iou_threshold and its stability score (the IoU of the mask thresholded at plus and minus stability_offset) is at least stability_threshold. Duplicates whose bounding boxes overlap by more than nms_threshold (IoU) are suppressed, keeping the best one. Masks touching an inner tile or crop edge are discarded, since the neighbouring tile sees them whole. Holes and islands smaller than minsize cells are finally removed from each mask.

These steps and their defaults are those of the reference SamAutomaticMaskGenerator; on identical input, i.sam.opencl returns the same masks.

Vectorization and cleaning

SAM masks overlap: a field and the parcels inside it can both be masks. The overlap rule chooses which mask a cell belongs to: the smallest covering mask (small, finest partition, the default), the largest (large) or the one with the highest predicted IoU (score). Masks larger than maxsize cells are discarded first; this is the way to drop the single mask SAM often returns for the whole background of a scene.

The resulting label raster is cleaned: every connected piece smaller than minsize cells, including small unsegmented gaps, is merged into the neighbouring piece it shares the longest border with. Boundaries are traced along cell edges into shared topological boundaries, simplified with the Douglas-Peucker algorithm using the simplify tolerance (one cell by default; 0 keeps the raster staircase), and the topology is cleaned (boundaries broken at intersections, duplicates removed, pseudo-nodes merged, slivers smaller than one cell removed). Each segment gets one centroid, placed at the cell farthest from its border.

Areas without a centroid are gaps that no mask covers. The optional raster output holds the segment categories of the cleaned label raster before simplification.

Attribute table

ColumnDescription
catSegment category
maskNumber of the SAM mask the segment comes from
predicted_iouMask quality predicted by SAM
stabilityStability score of the mask
cellsSegment size in cells
areaArea of the simplified polygon in square map units

A mask split into several pieces by overlapping masks yields several segments with the same mask number.

NOTES

OpenCL device

With device=auto, the first GPU is used, else the first CPU device. When several OpenCL platforms expose the same device (for example Mesa's Clover and rusticl drivers for AMD GPUs), the first platform on which the kernels compile is used; platform restricts the choice to platforms whose name contains the given text. Mesa's rusticl only exposes a GPU when enabled in the environment, e.g. RUSTICL_ENABLE=radeonsi. A CPU OpenCL implementation such as PoCL gives the same result but is about two orders of magnitude slower.

The ViT-H weights take 2.4 GiB of device memory and the work buffers about 1.3 GiB with the default batch of 32 prompts; a smaller batch reduces memory use. Buffers are kept below the device's maximum allocation size.

Device profile and precision

The profile chooses how buffers are allocated and which matrix product kernels run, to suit the device:

With profile=auto the profile is detected: cpu for a CPU device, igpu for a GPU that reports unified memory, egpu for a GPU whose PCI function Linux marks as removable (when the driver reports its PCI address), dgpu otherwise. The chosen profile and the reason are printed. A profile can be forced to compare them; every profile gives the same results, only the speed differs.

precision=fp16 stores the weight matrices of the linear layers (most of the model's memory) as IEEE half-precision numbers, read with the core OpenCL vload_half functions, so no half-precision extension is needed; the arithmetic stays in single precision. It halves their memory and the bytes read by the matrix products, which matters on devices limited by memory bandwidth (integrated GPUs, CPUs). Rounding the weights to half precision changes the masks slightly; check the result against precision=fp32 (the default) on a sample before relying on it.

Performance

On an AMD Radeon Pro WX 7100 (Polaris, 16 GiB, Mesa rusticl), with the ViT-H model, loading the model takes 1-2 s, the image encoder 3.8 s per crop and the mask decoder 3.7 s per 1024 prompts (the default 32 x 32 grid). A single-crop region is thus segmented in about 8 s. The number of crops, and so the run time, grows with the region size divided by tile_size, and with crop_layers.

Choosing the resolution

SAM sees every crop at 1024 x 1024 pixels. Objects should span at least ten or so pixels at that scale: set the region resolution and tile_size accordingly. A smaller tile_size zooms in (crops are upsampled to 1024 pixels), which finds smaller objects at the cost of more crops. For agricultural parcels in 2.5 to 10 m imagery, the defaults work well.

Debugging aids

When the environment variable I_SAM_OPENCL_DUMP names a directory, the 8-bit crops, the encoder inputs, the image embeddings and the list of masks are written there. The script tests/reference_compare.py in the source tree compares them with the PyTorch reference implementation. I_SAM_OPENCL_PROFILE=1 prints the time spent in each OpenCL kernel. Each line gives the event time and the wall-clock time (some drivers, such as rusticl on Intel iris, report wrong event times), and for the matrix products the achieved GFLOP/s, which shows whether a kernel is limited by compute or by memory bandwidth. I_SAM_OPENCL_DOWNLOAD_URL replaces the download location (https://dl.fbaipublicfiles.com/segment_anything), for a local mirror.

EXAMPLES

Landsat scene (North Carolina sample dataset)

Segment a true-colour composite of Landsat 7 bands 3, 2 and 1, dropping the background mask (the region has 475 x 527 cells):
g.region raster=lsat7_2002_30 -p
i.group group=lsat_rgb input=lsat7_2002_30,lsat7_2002_20,lsat7_2002_10
i.sam.opencl group=lsat_rgb checkpoint=sam_vit_h_4b8939.pth \
    output=lsat_segments maxsize=50000
v.db.select lsat_segments | head
i.sam.opencl example
Segments of the Landsat 7 true-colour composite (yellow).

Weights folder

Use the fastest model for a quick look; it is downloaded into ~/models/sam/sam1 the first time:
i.sam.opencl group=lsat_rgb model=vit_b output=lsat_quick maxsize=50000

Agricultural parcels

Delineate fields in a 2.5 m VNIR image with bands 1-3 grouped as RGB, keeping only segments of at least 0.1 ha (160 cells) and a label raster alongside the vector map:
i.group group=vnir_rgb input=vnir.1,vnir.2,vnir.3
g.region raster=vnir.1 -p
i.sam.opencl group=vnir_rgb checkpoint=sam_vit_h_4b8939.pth \
    output=fields raster=fields minsize=160

REFERENCES

Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Dollar, P., Girshick, R., 2023. Segment Anything. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 4015-4026. arXiv:2304.02643

SEE ALSO

i.group, i.segment, r.to.vect, v.clean, v.generalize

AUTHORS

Yann Chemin

SOURCE CODE

Available at: i.sam.opencl source code (history)

Accessed: Sunday Nov 15 10:57:14 2026


Main index | Imagery index | Topics index | Keywords index | Graphical index | Full index

© 2003-2026 GRASS Development Team, GRASS 8.5.1dev Reference Manual