i.hyper.endmembers identifies the most spectrally "pure"/extreme pixels (endmembers) in a hyperspectral 3D raster cube — the classic first step of a linear spectral unmixing workflow — and reports their locations and spectra. Unlike i.pysptools.unmix, which works on a 2D imagery group, this module reads directly from a GRASS 3D raster (as produced by i.hyper.import), and follows the same band-metadata and region-handling conventions as the rest of the i.hyper module family.
Endmember extraction needs the whole scene loaded at once (unlike per-pixel classification, it searches for extremes across the full image), so processing is not chunked; keep the computational region to your actual area of interest, or sample it with sample_size (below).
PPI and FIPPI runtime scales with ppi_skewers and the number of valid pixels. Lower ppi_skewers for a quicker, less statistically robust estimate. (This is dominated by dense matrix multiplication, so it also benefits substantially from an optimized BLAS — e.g. OpenBLAS rather than the plain reference BLAS some distributions ship by default.)
wl_2265_19), holding that endmember's reflectance at that
band.m.proj, so this works from any
source CRS, not just geographic locations.<prefix>_1,
<prefix>_2, ...).At least one of output / output_file is required.
Band wavelength/FWHM/validity metadata is read the same way as
i.hyper.spectroscopy: from i.hyper.import's
grid3/<map>/hyper.json sidecar if present, else
r3.info -h history, else per-band r.support
metadata. Use -b to restrict processing to bands marked valid
(valid=1) in that metadata, and min_wavelength/
max_wavelength to restrict to a spectral sub-range (e.g. to exclude
noisy detector edges or strong water-vapor absorption windows), consistent
with i.hyper.geology's convention.
-b/min_wavelength/max_wavelength rely on the
cube's own metadata (or the caller) already knowing which bands are
unreliable. That is not always true: a real EMIT scene's per-band
valid=1 metadata did not flag two bands at ~1357nm and
~1417nm (classic strong water-vapor-absorption wavelengths) even though
61.7% and 28.1% of valid pixels there held a physically impossible
out-of-range (outside 0.0-1.0) reflectance value — a
sensor/atmospheric-correction artifact, not a measurement.
This matters more for extraction than it might seem: ATGP starts from the pixel with the single largest spectral vector norm, and N-FINDR/PPI/FIPPI all likewise select or refine endmembers by norm or simplex volume. An out-of-range value in even one band inflates that pixel's norm and can change which pixel looks most "extreme" — and because endmember algorithms specifically hunt for rare/extreme pixels, even a low-frequency artifact (one real case found here: just 0.14% of pixels in a third band) is disproportionately likely to land on exactly the pixels selected as endmembers, not average out harmlessly. Verified directly on this scene: excluding the two worst bands changed 2 of 6 ATGP-selected pixels, including the very first (max-norm) pick.
max_invalid_fraction (default 0.0) guards against this independent of -b/wavelength range/upstream metadata: after loading the cube, any band where more than this fraction of valid pixels has a value outside 0.0-1.0 is excluded entirely — from extraction, identification, and plotting — with a warning naming the wavelength and the fraction affected. The default of 0 (exclude on even a single contaminated pixel) is deliberately strict, for the reason above; raise it (e.g. to 0.01) if your sensor is known to have harmless, genuinely sparse per-pixel noise unrelated to a systematic band-level artifact. Set to 1.0 to disable entirely.
Every 0.0-1.0 test in this module — max_invalid_fraction above, and the identification query sent to i.hyper.speclookup, which drops any value outside that range — assumes the cube holds reflectance as a fraction. Surface-reflectance products commonly ship scaled by 1000 or 10000 instead. Extraction itself does not care, since a common factor cancels out of every norm and simplex volume, but those two tests do: against a cube stored x10000 every band looks entirely out of range and the module stops with "All bands were excluded by max_invalid_fraction".
reflectance_scale (default 0) is the divisor that converts stored values to reflectance. At 0 it is detected from the cube's own value range and reported, using the same detector and the same thresholds as i.hyper.spectroscopy's option of the same name, so one cube resolves to the same scale in both modules. Give it explicitly to override a wrong detection.
By default the module loads every pixel of the computational region. It does that with r3.to.rast, one 2D raster per band, which reads the whole scene once per band. Measured on the C2RD Pourrieres drone mosaic (8471x7284x408): a single band slice takes 10-15 s, and loading the cube through all 408 of them took 1 h 24 min — at a 0.3 m region, already decimated tenfold, because at the native 0.03 m grid the loaded cube would be 61.7 M pixels x 408 bands, about 100 GB as float32.
sample_size=n instead reads n randomly chosen whole spectra with r3.what, which fetches only the voxels asked for, with no per-band pass over the scene and memory of only n x bands x 4 bytes. On the same cube, 50000 sampled spectra at native 0.03 m took 367 s, against the 1 h 24 min that loading the same cube took at a tenth of that resolution, and the whole run — sampling, N-FINDR for 20 endmembers and 20 identification queries — took 6 min 12 s. The sampled cells are drawn without replacement from the region's own cell grid and depend on random_seed, so a run is reproducible; coordinates and spectra of the selected endmembers are identical to what the whole-region path reports for the same pixel.
The cost is set by the cube's tile geometry, not by the band count. That mosaic is tiled 31x26x10 FCELL, so one 408-band spectrum spans 41 tiles and pulls about 1.3 MB to deliver 408 values. Scattered points share almost no tiles — 50000 points over 61.7 M pixels is one per 1234, where a tile column covers 806 — so the per-point cost is flat in the sample size and the reads add up to several times the size of the cube. Measured on this cube:
5000 points scattered over the scene 39.0 s 7.80 ms/point 5000 points inside one 300x300 block 2.8 s 0.57 ms/point
Locality is therefore worth a factor of 14, and a sample restricted to a
smaller region costs far less per spectrum than a sample spread over a large
one. Enlarging the raster3d tile cache
(RASTER3D_DEFAULT_CACHE_SIZE/RASTER3D_MAX_CACHE_SIZE)
does not help the scattered case, where the misses are compulsory rather
than capacity misses: it measured slightly slower (45.5 s against
39.0 s), while on the clustered case it gave 1.5 s against 2.8 s.
What a sample costs is coverage, not accuracy. Endmember extraction hunts for the few most extreme spectra in the scene, so a material covering a thousandth of the region has about a 1 in 1000 chance per sampled pixel of being seen at all: if a known material is missing from the result, raise sample_size rather than lowering it. Sampling is refused together with abundance_prefix, since unmixing writes an abundance for every pixel of the region and so needs the whole cube.
A sample whose size reaches the region's pixel count degenerates to querying every pixel one by one, which is slower than the whole-region path; the module warns and carries on.
Extraction finds the most extreme spectra in the scene, which is not the same as the most pure. A pixel straddling two materials, or sitting on a shadow edge, is extreme in exactly the way N-FINDR and ATGP reward, and the spectrum it yields is a mixture no library will match well.
homogeneity=raster samples a 2D raster of local spectral
variability at each selected pixel and attaches the value to every output as
a homogeneity column (vector) or field (CSV/JSON), reporting it
alongside the raster's own region mean:
Local variability from <mosaic_sam5> (region mean 0.305): Endmember 1: homogeneity=0.012 (0.04x region mean) Endmember 2: homogeneity=0.596 (1.95x region mean)
An endmember well below the region mean sits in a uniform neighbourhood and its spectrum is a plausible pure material; one well above it should be treated as a mixture or an edge, whatever its library match says.
Any 2D raster is accepted and the value is attached exactly as stored: this module does not know which way round the given measure runs. A variance, standard deviation or mean-SAM map from i.variance.opencl measures variability, so there the low values mark the homogeneous pixels, as in the example above. A null at an endmember location is reported as null and written as an empty field, never as zero. The raster is checked to exist before the cube is read, so a typo costs nothing.
-i identifies every extracted endmember against the shared local spectral library built by i.hyper.lib_ecosis/i.hyper.lib_usgs/ i.hyper.lib_relab, by calling i.hyper.speclookup directly — one real GRASS module invocation covering all endmembers at once (each endmember becomes one query row in a temporary CSV, and i.hyper.speclookup already ranks one result set per row), not a reimplementation of its matching logic. This is the natural second step after extraction: N-FINDR/ATGP/PPI/FIPPI tell you where the spectrally distinct materials in a scene are; -i gives each one a candidate real-world identity.
Before being written to that query CSV, each endmember's own values are
checked against the physical reflectance range: any value outside
0.0-1.0 (e.g. a water-vapor-absorption-band retrieval artifact —
the same kind of spike the plot's fixed y-axis clips from view) is left
blank in that row rather than passed through as-is or written as a
literal nan. A blank CSV cell is skipped cleanly by
i.hyper.speclookup's own reader (float('') raises,
same as any other unparseable value), simply dropping that one band from
that endmember's query — whereas writing the literal text
nan would not be caught (Python's float()
parses the strings "nan"/"inf" successfully
instead of raising, the same pitfall already fixed in the harvesters' own
ingestion), and passing the raw out-of-range value through uncorrected
would silently distort the SAM/correlation/Euclidean computation for
every candidate compared against that endmember. This is not a cosmetic
fix: on a real EMIT scene, a handful of such artifact values changed
which library record several endmembers matched best, in one case
improving the best match from sam=55.7 (a poor fit) to
sam=9.4 (a much better one) once the corrupting values were
excluded from the comparison.
http://127.0.0.1:8977) instead of scanning a local
spec_library= tree, passed straight through to
i.hyper.speclookup server=. The server holds the whole
library resident as one normalised matrix and answers an exact search,
so identification costs milliseconds per endmember whatever the
library's size — which is what makes a quarter-million-record
library usable here at all. spec_server= and spec_library=
are alternatives: giving both warns and uses the server, rather than
leaving it ambiguous which one answered. Note that the served index
carries each record's identity but not its own spectrum, so -p
plots the extracted endmembers without the matched reference
overplotted.usgs_splib07's mineral chapters for a
geology scene, or relab's Rock/Mineral
categories) — the same filters i.hyper.speclookup itself
takes, and for the same reason: matching against the full
~266,000-record library takes minutes, matching against one relevant
source/category takes seconds. With spec_server= the
filters cost nothing either way, so they serve to express what the
scene can plausibly contain rather than to save time.mineral, soil, rock,
vegetation, water, snow_ice,
man_made, organic_compound,
mixture, other). Needs spec_server=,
since only the served index carries one normalised category per
record.sam, matching
i.hyper.speclookup's own default): Spectral Angle Mapper is
usually the right choice here specifically because it is insensitive
to a uniform brightness/albedo offset between a real scene endmember
(subject to atmospheric and BRDF effects) and a clean lab/field
reference spectrum.Each identified endmember is labeled with the best match's source
database, dataset, record ID, title,
organization, the similarity score (accuracy) and
number of overlapping bands used to compute it, and the
confidence margin to the runner-up (precision) — attached to
the vector map's attribute table (match_source,
match_dataset, match_record,
match_title, match_organization,
match_score, match_overlap_bands,
match_margin) and, more fully, to output_file (the
same fields plus match_method and the matched record's
complete match_extra_metadata JSON — mineral/plant
name, formula, collection locality, and everything else the source
harvester captured). An endmember with no library candidate above
spec_min_overlap_bands gets a blank/null match rather than a
fabricated one.
-p renders every extracted endmember's spectrum as a matplotlib PNG (solid line, one color per endmember). The y-axis is always fixed to 0.0-1.0, the physical range of a reflectance value — any excursion outside it in the underlying data (e.g. a water-vapor-absorption-band retrieval artifact) is a known data-quality issue and is simply clipped from view rather than being allowed to rescale the whole plot. The x-axis is clipped to the endmember's own wavelength range, even when a matched library record natively covers a much wider range — e.g. a Nicolet FTIR reference extending to 200,000+ nm would otherwise squeeze the whole comparison into an unreadable sliver. If -i was also given, each endmember's identified library match is overplotted in the same color as a dashed line, and the legend names the match's title, record ID, source, and similarity score alongside each endmember.
The PNG is written to plot_dir (default: the current GRASS
location's own directory, $GISDBASE/$LOCATION_NAME/ —
so it lands alongside whatever project the input raster belongs to
without needing to specify one explicitly) as
<output>_endmembers.png, or
<input>_endmembers.png if no vector output was
requested. Add -w to also open it in an interactive window (needs
a display; the file is still written either way). -p alone
(without output/output_file) is a valid, plot-only
invocation.
Column/field names are derived from each band's central wavelength, in
wavelength_unit (nm by default), sanitized into valid SQL identifiers
(e.g. 2265.19 nm becomes wl_2265_19).
Endmember coordinates are cell centers of whichever pixel was selected, computed directly from the active computational region — they are exact regardless of the region's resolution or extent relative to the raster's native footprint.
The examples below use an EMIT scene imported with i.hyper.import
into its own location, emit_slovakia (raster_3d map
emit_20240730, 244 bands, hyper.json sidecar
metadata). Its native footprint covers nearly all of Slovakia (1727 x 2746
pixels, north=49.10 south=48.17 east=20.53 west=19.04) — endmember
extraction needs the whole active region loaded into memory at once (see
above), so the first step in any real session is always to restrict
the region to the actual area of interest, not run against the full native
extent:
g.region n=48.70 s=48.60 e=19.80 w=19.70 res=0:00:01.952037 g.region -g
projection=3 zone=0 n=48.7 s=48.6 w=19.7 e=19.8 nsres=0.000543478260869573 ewres=0.000543478260869573 rows=184 cols=184 cells=33856
i.hyper.endmembers input=emit_20240730 output=slovakia_endmembers \
output_file=slovakia_endmembers.csv -l n_endmembers=6 \
extraction_method=NFINDR
Extracting 244 bands (184x184 pixels)… Extracting 6 endmembers using NFINDR… Writing endmember vector map → slovakia_endmembers… Writing endmember spectra (csv) → slovakia_endmembers.csv… Done: 6 endmember(s) extracted with NFINDR.
The vector map has one point per endmember and one attribute column per band, named by its central wavelength:
v.info -c map=slovakia_endmembers
INTEGER|cat DOUBLE PRECISION|x DOUBLE PRECISION|y DOUBLE PRECISION|wl_381_0056 DOUBLE PRECISION|wl_388_4092 DOUBLE PRECISION|wl_395_8158 ...
And the CSV (-l puts longitude/latitude first, reprojected via
m.proj from this location's native CRS — here already
geographic, so the values are unchanged, but the same flag reprojects
correctly from a projected/UTM location too):
lon,lat,wl_381_0056,wl_388_4092,wl_395_8158,wl_403_2254,... 19.70027174,48.62038043,0.00179211446,0.00183079974,0.00186951912,0.00190832582,...
i.hyper.endmembers input=emit_20240730 output=slovakia_ppi \
n_endmembers=5 extraction_method=PPI \
abundance_prefix=slovakia_abund unmixing_method=FCLS random_seed=42
Extracting 244 bands (184x184 pixels)… Extracting 5 endmembers using PPI… Writing endmember vector map → slovakia_ppi… Unmixing abundances using FCLS… Done: 5 endmember(s) extracted with PPI.
This writes slovakia_abund_1 .. slovakia_abund_5,
one abundance raster per endmember, each valued 0-1 and summing to 1 across
the five rasters at every pixel (FCLS's non-negativity + sum-to-one
constraints):
r.univar map=slovakia_abund_1 -g
n=33856 min=0 max=1 mean=0.0485496587901701
i.hyper.endmembers input=emit_20240730 output=slovakia_endmembers_b \
output_file=slovakia_endmembers.json output_format=json \
n_endmembers=8 extraction_method=ATGP \
-b min_wavelength=450 max_wavelength=2400
Extracting 221 bands (184x184 pixels)… Extracting 8 endmembers using ATGP… Writing endmember vector map → slovakia_endmembers_b… Writing endmember spectra (json) → slovakia_endmembers.json… Done: 8 endmember(s) extracted with ATGP.
221 of the cube's 244 bands fall in the requested 450-2400nm range; no -l here, so the JSON's first two fields are the native projected X,Y (this location happens to be geographic, so numerically these look like lon/lat too, but for a projected/UTM location they would not be):
[
{
"x": 19.78233695652174,
"y": 48.674728260869564,
"wl_455_1703": 0.0431,
"wl_462_5989": 0.0448,
...
},
...
]
i.hyper.endmembers input=emit_20240730 output=slovakia_endmembers_id \
output_file=slovakia_endmembers_id.csv -l -i n_endmembers=6 \
extraction_method=NFINDR spec_source_database=usgs_splib07 \
spec_dataset_id=ChapterV_Vegetation,ChapterM_Minerals,ChapterS_SoilsAndMixtures
Extracting 244 bands (184x184 pixels)… Extracting 6 endmembers using NFINDR… Identifying endmembers against the shared spectral library (i.hyper.speclookup)… Identified 6 of 6 endmember(s) (method=sam). Endmember 1: Vegetation (splib07a_LeafySpurge_Spurge-A2-Jun98_ASDFRa_AREF, usgs_splib07) -- sam=5.18419, margin=0.02959 (240 overlapping bands) Endmember 2: Vegetation (splib07a_Marsh_SCAM50%..._CRMS121v32_ASDFRa_AREF, usgs_splib07) -- sam=3.71498, margin=0.2749 (234 overlapping bands) Endmember 3: Vegetation (splib07a_LeafySpurge_Spurge-A1-Oct97_ASDFRa_AREF, usgs_splib07) -- sam=3.4407, margin=0.03746 (240 overlapping bands) Endmember 4: Vegetation (splib07a_Marsh_SCAM50%..._CRMS121v32_ASDFRa_AREF, usgs_splib07) -- sam=1.37321, margin=0.1085 (234 overlapping bands) Endmember 5: Vegetation (splib07a_Oak_QUDU_CA01-QUDU-1_bush_1_ASDFRa_AREF, usgs_splib07) -- sam=1.7603, margin=0.1107 (236 overlapping bands) Endmember 6: Vegetation (splib07a_Grass-Smoothbrome_YNP-SB-1_AVIRISb_RTGC, usgs_splib07) -- sam=3.79253, margin=0.2401 (237 overlapping bands) Writing endmember vector map → slovakia_endmembers_id… Writing endmember spectra (csv) → slovakia_endmembers_id.csv… Done: 6 endmember(s) extracted with NFINDR.
Genuinely sensible for this Slovakia forest scene: every endmember matches best against USGS's Vegetation chapter, not Minerals or Soils, despite all three being searched. The vector map's attribute table carries the identity and confidence fields directly:
v.db.select map=slovakia_endmembers_id columns=cat,match_title,match_record,match_score,match_overlap_bands,match_margin
cat|match_title|match_record|match_score|match_overlap_bands|match_margin 1|Vegetation|splib07a_LeafySpurge_Spurge-A2-Jun98_ASDFRa_AREF|5.184195|240|0.029587 2|Vegetation|splib07a_Marsh_SCAM50%..._CRMS121v32_ASDFRa_AREF|3.714981|234|0.274929 ...
and the CSV additionally carries the full match_extra_metadata
for the top match, e.g. for endmember 1:
{"documentation_format": "PLANT", "sample_id": "Spurge-A2-Jun98",
"plant_type": "Invasive herbaceous perennial weed", "plant": "Leafy spurge",
"latin_name": "Euphorbia esula L.", "collection_locality": "Golden, Colorado", ...}
Using the emit_dubai project's own
AlMarwanProjectSite saved region and its
emit_20260623 cube (255 bands):
g.region region=AlMarwanProjectSite
i.hyper.endmembers input=emit_20260623 output=dubai_endmembers \
-i -p -b n_endmembers=6 extraction_method=NFINDR \
spec_source_database=usgs_splib07
Extracting 255 bands (350x706 pixels)…
WARNING: Excluding band at 1357.2nm from endmember extraction: 61.7% of
valid pixels have an out-of-range (outside 0.0-1.0) reflectance
value (max_invalid_fraction=0).
WARNING: Excluding band at 1416.8nm from endmember extraction: 28.1% of
valid pixels have an out-of-range (outside 0.0-1.0) reflectance
value (max_invalid_fraction=0).
WARNING: Excluding band at 1439.1nm from endmember extraction: 0.1% of
valid pixels have an out-of-range (outside 0.0-1.0) reflectance
value (max_invalid_fraction=0).
Extracting 6 endmembers using NFINDR…
Identifying endmembers against the shared spectral library (i.hyper.speclookup)…
Identified 6 of 6 endmember(s) (method=sam).
Endmember 1: Vegetation (splib07a_Rangeland_L02-045_S01%_G04%_ASDFRa_AREF, usgs_splib07) -- sam=4.0643, margin=0.06403 (247 overlapping bands)
Endmember 2: Organic Compounds (splib07a_Xanthine_SA-X0626_90K_NIC4aa_RREF, usgs_splib07) -- sam=11.6209, margin=0.5861 (152 overlapping bands)
Endmember 3: Liquids (splib07a_Water+Montmor_SWy-2+5.01g-l_ASDFRa_AREF, usgs_splib07) -- sam=12.8549, margin=3.96 (252 overlapping bands)
Endmember 4: Vegetation (splib07a_Marsh_sediment_DWV3-0511_dry_ASDFRa_AREF, usgs_splib07) -- sam=5.36816, margin=0.4425 (242 overlapping bands)
Endmember 5: Organic Compounds (splib07a_Peptidogylcan_SA-69554_ASDFRa_AREF, usgs_splib07) -- sam=5.96867, margin=0.3165 (252 overlapping bands)
Endmember 6: Minerals (splib07a_Vesuvianite_HS446.1B_Idocras_ASDFRa_AREF, usgs_splib07) -- sam=2.86591, margin=0.01122 (252 overlapping bands)
Wrote endmember spectra plot → /home/yann/grassdata/emit_dubai/dubai_endmembers_spectra.png
Wrote reference spectra plot → /home/yann/grassdata/emit_dubai/dubai_endmembers_reference_spectra.png
Writing endmember vector map → dubai_endmembers…
Done: 6 endmember(s) extracted with NFINDR.
No output_file/plot_dir given here, so both PNGs land in
the project's own directory automatically
($GISDBASE/$LOCATION_NAME/ = ~/grassdata/emit_dubai/),
named entirely by the module from the vector output name.
dubai_endmembers_spectra.png shows the six extracted
endmembers together, numbered; dubai_endmembers_reference_spectra.png
shows their six identified USGS matches together, same numbers/colors,
each labeled with its full identity and score.
This run also demonstrates why max_invalid_fraction's strict default (0) matters at the extraction stage, not just identification: with the two worst contaminated bands (and one marginal one, at just 0.14% of pixels) excluded before ATGP/N-FINDR ever run, different pixels get selected as endmembers than in an earlier run where those bands were left in — confirmed directly by comparing the raw ATGP pixel indices with and without the bands excluded (2 of 6 changed, including the very first max-norm pick). The endmember spectra plot is also visibly cleaner: no more multiple-hundred-percent reflectance spike near 1350-1450nm, since the pixels now selected were never chosen partly because of that artifact in the first place.
The C2RD Pourrieres mosaic is 8471x7284x408 at 0.03 m; the whole-region path measured 1 h 24 min of band extraction even at a decimated 0.3 m region, and would need about 100 GB of memory at the native one. Sampling 50000 whole spectra reads the same cube at full resolution in 367 s:
g.region raster_3d=mosaic_final
i.hyper.endmembers input=mosaic_final output=c2rd_endmembers \
output_file=c2rd_endmembers.csv n_endmembers=20 \
extraction_method=NFINDR sample_size=50000 random_seed=42 \
max_invalid_fraction=0.005 reflectance_scale=10000 \
spec_server=http://127.0.0.1:8977 spec_top_n=5 -i -b
Sampling 50000 random pixels of 7284x8471 (408 bands)…
Sampled 27752 valid of 50000 queried pixels (408 bands).
Reflectance scale detected: 10000 (stored value / 10000 = reflectance)
WARNING: Dropping 238 of 27752 valid pixels that are null in at least one
band: the extraction algorithms cannot take a null. Bands with the
most nulls: 1252.3nm (0.33%), 1247.2nm (0.33%), 1257.4nm (0.33%)
Extracting 20 endmembers using NFINDR…
Identified 20 of 20 endmember(s) (method=sam).
About 41% of that mosaic's bounding box is nodata, which is why 50000 queried pixels yield 27752 spectra. The run took 6 min 12 s in total, of which 367 s was the sampling itself.
Spatial variability from i.variance.opencl cross-checks the result, fed straight into homogeneity so that each endmember carries it:
i.variance.opencl input_3d=mosaic_final method=sam_mean size=5 \
output=mosaic_sam5
i.hyper.endmembers input=mosaic_final output=c2rd_endmembers \
n_endmembers=20 sample_size=50000 random_seed=42 \
reflectance_scale=10000 homogeneity=mosaic_sam5
An endmember whose homogeneity is far above the region mean
sits in a heterogeneous neighbourhood and is a mixed pixel or an edge, not a
pure material.