This tutorial is designed to introduce you to advanced concepts and procedures for analysis of imaging spectrometer data or hyperspectral images . We will use 1995 Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) data from Cuprite, Nevada, to investigate the unique properties of hyperspectral data and how spectral information can be used to identify mineralogy. We will evaluate "Effort" "polished" spectra vs ATREM-calibrated data, and review the Spectral Angle Mapper classification. We will compare apparent reflectance spectra and continuum-removed spectra. We will also compare apparent reflectance images and continuum-removed images and evaluate Spectral Feature Fitting results. This tutorial is designed to be completed in two to four hours.
You must have the ENVI TUTORIALS & DATA CD-ROM mounted on your system to access the files used by this tutorial, or copy the files to your disk.
The files used in this tutorial are contained in the C95AVSUB subdirectory of the ENVIDATA directory on the ENVI TUTORIALS & DATA CD-ROM.
The files listed below, along with their associated .hdr files, are required to run this exercise. Optional files listed below may also be used if more detailed calibration comparisons are desired. All image data files have been converted to integer format by multiplying the reflectance values by 1000 because of disk space considerations. A value of 1000 therefore represents apparent reflectance of 1.0.
CUP95_AT.INT Cuprite ATREM calibrated apparent reflectance data. 50 bands (integer).
CUP95EFF.INT Cuprite Effort-Corrected ATREM calibrated apparent reflectance data. 50 bands (integer).
JPL1.SLI JPL Spectral Library in ENVI format.
USGS_MIN.SLI USGS Spectral Library in ENVI format.
CUP95_AV.ROI Saved ROI locations.
CUPSAMEM.ASC Optional file of ROI mean spectra.
CUPSAM1.IMG SAM classification image using ROI image spectra endmembers.
CUPRUL1.IMG SAM rule image using ROI image spectra endmembers.
CUPSAM2.IMG SAM classification image using Spectral Library endmembers.
CUPRUL2.IMG SAM rule images using Spectral Library endmembers.
CUP95_CR.DAT Continuum-removed data (floating point)
CUP95SFF.DAT Spectral Feature Fitting Results
CUP95SFR.DAT Spectral Feature Fitting Band-Math Results
"Effort", the "Empirical Flat Field Optimized Reflectance Transformation" is a correction used to remove residual "saw-tooth" instrument (or calibration-introduced) noise and atmospheric effects from ATREM-calibrated AVIRIS data. While this correction is not currently available in ENVI, it is the type of custom correction that might be designed for specific data sets to improve overall quality of spectra. It provides the best reflectance spectra available from AVIRIS data, so the Effort data are used in later exercises in this tutorial. Effort is a relatively automated improvement on the Flat-Field Calibration method (Boardman, 1996). The Effort correction selects those AVIRIS spectra that match a low order polynomial estimate in a least-squares sense as representative featureless spectra. These spectra are averaged and a mild gain factor is determined to remove systematic, coherent noise, present in every spectrum, including small residual atmospheric effects near 2.0 mm range attributable to CO2. We will compare 1995 AVIRIS data calibrated to reflectance using ATREM and the corresponding Effort-corrected ATREM data.
Moving the Zoom Window indicator box in this image will cause the cursor location to be updated in the ATREM Main Display window. Both spectral profiles will show the spectrum for the current pixel.
Stack the spectra if desired by selecting Options->Stack Data.
Note how similar the two spectra are, but how very small coherent noise "wiggles" have been removed from the Effort spectrum. Also note the correction of residual CO2 at 2.01 mm.
Note how Effort changes the spectrum.
To close all of the files used in the previous section, select File->Close All Files.
The associated display windows will also be closed.
To close all of the spectral plots, select Basic Tools->Display Controls->Close All Plot Windows.
Next, we will use both image and laboratory spectra to classify the AVIRIS data using the "Spectral Angle Mapper" (SAM). We will go through the endmember selection process for SAM, but will not actually run the algorithm. We will examine previously calculated classification results to answer specific questions about the strengths and weaknesses of the SAM classification.
The Spectral Angle Mapper (SAM) is an automated method for comparing image spectra to individual spectra or a spectral library (Boardman, unpublished data; CSES, 1992; Kruse et al ., 1993a). SAM assumes that the data have been reduced to apparent reflectance (true reflectance multiplied by some unknown gain factor controlled by topography and shadows). The algorithm determines the similarity between two spectra by calculating the "spectral angle" between them, treating them as vectors in a space with dimensionality equal to the number of bands ( nb ). A simplified explanation of this can be given by considering a reference spectrum and an unknown spectrum from two-band data. The two different materials will be represented in the 2-D scatter plot by a point for each given illumination, or as a line (vector) for all possible illuminations
Because it uses only the "direction" of the spectra, and not their "length," the method is insensitive to the unknown gain factor, and all possible illuminations are treated equally. Poorly illuminated pixels will fall closer to the origin. The "color" of a material is defined by the direction of its unit vector. Notice that the angle between the vectors is the same regardless of the length. The length of the vector relates only to how fully the pixel is illuminated.
The SAM algorithm generalizes this geometric interpretation to nb -dimensional space. SAM determines the similarity of an unknown spectrum t to a reference spectrum r , by applying the following equation (CSES, 1992):
which also can be written as:
where nb equals the number of bands in the image.
For each reference spectrum chosen in the analysis of a hyperspectral image, the spectral angle a is determined for every image spectrum (pixel). This value, in radians, is assigned to the corresponding pixel in the output SAM image, one output image for each reference spectrum. The derived spectral angle maps form a new data cube with the number of bands equal to the number of reference spectra used in the mapping. Gray-level thresholding is typically used to empirically determine those areas that most closely match the reference spectrum while retaining spatial coherence.
The SAM algorithm implemented in ENVI takes as input a number of "training classes" or reference spectra from ASCII files, ROIs, or spectral libraries. It calculates the angular distance between each spectrum in the image and the reference spectra or "endmembers" in n -dimensions. The result is a classification image showing the best SAM match at each pixel and a "rule" image for each endmember showing the actual angular distance in radians between each spectrum in the image and the reference spectrum. Darker pixels in the rule images represent smaller spectral angles, and thus spectra that are more similar to the reference spectrum. The rule images can be used for subsequent classifications using different thresholds to decide which pixels are included in the SAM classification image.
You may also make this selection from the Spectral Tools Menu.
The Enter ASCII Plot Filename dialog will appear and available files will be listed.
This will leave you with the mean spectra Zeolite, Calcite, Alunite (2.16), Kaolinite, Illite/Muscovite, Silica (Bright), and Buddingtonite.
Normally, you would click "OK" to start the classification, but because classification can take some time, preprocessed results are provided for this exercise.
The classification image is one band with coded values for each class (for example, alunite is coded as "1").
When opened, the classified image will appear in the Available Bands List dialog.
The classes will automatically be color coded as follows:
Mineral | Color |
---|---|
Zeolites | White |
Calcite | Green |
Alunite | Yellow |
Kaolinite | Red |
Illite/Muscovite | Dark Green |
Silica | Blue |
Buddingtonite | Maroon |
The rule image has one band for each endmember classified, with the pixel values representing the spectral angle in radians. Lower spectral angles (darker pixels) represent better spectral matches to the endmember spectrum.
When opened, one band for each endmember will appear in the Available Bands List dialog.
You may use a rule image color composites or image animation if desired to compare individual rule images.
In this exercise we will once again go through the endmember selection process, but we won't actually perform the SAM classification. Previously saved SAM results will be used for comparisons. If you have time, you can perform your own SAM classification using spectral library endmembers.
You may also access this function from the ENVI Spectral Tools menu.
Note that the spectra are automatically resampled to the AVIRIS wavelengths and resolution using the AVIRIS band positions and FWHM and listed in the Endmember Collection dialog.
If you have time, you can generate your own SAM classification by clicking "OK".
The classification image is one band with coded values for each class (for example, alunite is coded as "1"). When opened, the classified image will appear in the Available Bands List dialog.
The classes will automatically be color coded as follows:
Mineral | Color |
---|---|
Zeolites | White |
Calcite | Green |
Alunite | Yellow |
Kaolinite | Red |
Illite/Muscovite | Dark Green |
Silica | Blue |
Buddingtonite | Maroon |
Again, the rule image has one band for each endmember classified, with the pixel values representing the spectral angle in radians. Lower spectral angles (darker pixels) represent better spectral matches to the endmember spectrum. When opened, one band for each endmember will appear in the Available Bands List dialog.
Based upon the results of the two SAM classifications, answer the following questions:
If you have extra time at the end of the exercise, try generating new classified images based on different thresholds in the rule images.
Thresholds can also be defined using ENVI's interactive density slice tool, by selecting Functions->Color Mapping->Density Slice in the Main window.
The Rule Image Classifier Parameters dialog will appear.
All of the pixels with values lower than the minimum will be classified. Lower thresholds result in fewer pixels being classified.
After a short wait, the new classification image will be listed in the Available Bands List.
To close all of the files used in this portion of the exercise, select File->Close All Files.
To close all of the spectral plots, select Basic Tools->Display Controls->Close All Plot Windows.
Spectral Feature Fitting (SFF) is an absorption-feature-based method for matching image spectra to reference endmembers, similar to methods developed at the U. S. Geological Survey, Denver (Clark et al., 1990, 1991, 1992; Clark and Swayze, 1995).
Most methods used for analysis of hyperspectral data still do not directly identify specific materials. They only indicate how similar the material is to another known material or how unique it is with respect to other materials. Techniques for direct identification of materials, however, via extraction of specific spectral features from field and laboratory reflectance spectra have been in use for many years (Green and Craig, 1985; Kruse et al., 1985; Yamaguchi and Lyon, 1986; Clark et al., 1987). Recently, these techniques have been applied to imaging spectrometer data, primarily for geologic applications (Kruse et al., 1988; Kruse, 1988; Kruse, 1990; Clark et al., 1990, 1991, 1992; Clark and Crowley, 1992; Kruse et al. 1993b, 1993c; Kruse and Lefkoff, 1993, Swayze et al., 1995).
All of these methods require that data be reduced to reflectance and that a continuum be removed from the reflectance data prior to analysis. A continuum is a mathematical function used to isolate a particular absorption feature for analysis (Clark and Roush, 1984; Kruse et al, 1985; Green and Craig, 1985). It corresponds to a background signal unrelated to specific absorption features of interest. Spectra are normalized to a common reference using a continuum formed by defining high points of the spectrum (local maxima) and fitting straight line segments between these points. The continuum is removed by dividing it into the original spectrum (Figure 4).
Spectral feature fitting requires that reference endmembers be selected from either the image or a spectral library, that both the reference and unknown spectra have the continuum removed, and that each reference endmember spectrum be scaled to match the unknown spectrum. A "Scale" image is produced for each endmember selected for analysis by first subtracting the continuum-removed spectra from one, thus inverting them and making the continuum zero. A single multiplicative scaling factor is then determined that makes the reference spectrum match the unknown spectrum. Assuming that a reasonable spectral range has been selected, a large scaling factor is equivalent to a deep spectral feature, while a small scaling factor indicates a weak spectral feature.
.A least-squares-fit is then calculated band-by-band between each reference endmember and the unknown spectrum. The total root-mean-square (RMS) error is used to form an RMS image for each endmember. An optional ratio image of Scale/RMS provides a "Fit" image that is a measure of how well the unknown spectrum matches the reference spectrum on a pixel-by-pixel basis.
For the purposes of this exercise, you will not create your own continuum-removed data, they have been pre-calculated.
If you want to perform this operation on your own:
This image will have the same number of spectral bands as the input image.
If not creating your own continuum-removed file:
This is the continuum-removed data (floating point 0 - 1.0 range) derived from the 1995 AVIRIS Effort-calibrated apparent reflectance as described above.
This is the 1995 AVIRIS ATREM-calibrated apparent reflectance data with the Effort correction applied.
Moving the Zoom Window indicator box in this image will cause the cursor location to be updated in the Effort Main Display window. Both spectral profiles will show the spectrum for the current pixel.
Note how the continuum-removed spectrum normalizes and enhances spectral features.
Note how continuum-removal affects the spectrum.
Note the correspondence between dark areas on the continuum-removed image and red-to-purple areas on the color composite. The dark areas are areas with absorption bands near 2.2 mm.
Note by moving the Zoom window to the dark areas and examining the spectra that these correspond to absorption features near 2.34 mm in both the continuum-removed and Effort spectra.
Click on one of the Effort bands in the Available Bands List and then click "Close File" at the bottom of the dialog.
The associated display windows and spectral plot will also be closed.
For the purposes of this exercise, you will not create your own SFF Scale or RMS data, they have been pre-calculated.
If you want to perform this operation on your own:
The output image will have two images for each endmember, a Scale image and an RMS image.
If you are not going to create your own SFF file:
This contains the Scale and RMS images derived from spectral feature fitting of a library of image endmember spectra from the 1995 AVIRIS Effort-calibrated apparent reflectance data. Library spectra could also have been used, but the image spectra were used to allow direct comparison with other methods that use image spectra.
Note that although similar as stretched images, the values for the two images are very different.
Low RMS values correspond to good spectral matches.
Note the highlighted pixels in the RMS Kaolinite.
For the purposes of this exercise, you will not create your own SFF Fit Images, they have been pre-calculated.
If you want to perform this operation on your own:
The output image will have one Fit image for each endmember.
If not creating your own SFF file:
This contains the images that are the ratio of Scale/RMS for each of the endmembers.
The bright pixels represent the best fit to the reference endmember spectrum of Kaolinite.
Note how the Scale and RMS image interact to produce the Fit result.
When you have finished your ENVI session, click "Quit" or "Exit" on the ENVI Main Menu, then type exit at the IDL command prompt.
If you are using ENVI RT, quitting ENVI will take you back to your operating system.
Clark, R. N., and Roush, T. L., 1984, Reflectance spectroscopy: Quantitative analysis techniques for remote sensing applications: Journal of Geophysical Research, v. 89, no. B7, pp. 6329-6340.
Clark, R. N., King, T. V. V., and Gorelick, N. S., 1987, Automatic continuum analysis of reflectance spectra: in Proceedings, Third AIS workshop, 2-4 June, 1987, JPL Publication 87-30, Jet Propulsion Laboratory, Pasadena, California, p. 138-142.
Clark, R. N., Swayze, G. A., Gallagher, A., King, T. V. V., and Calvin, W. M., 1993, The U. S. Geological Survey Digital Spectral Library: Version 1: 0.2 to 3.0 mm: U. S. Geological Survey, Open File Report 93-592, 1340 p.
Clark, R. N., Gallagher, A. J., and Swayze, G. A., 1990, Material absorption band depth mapping of imaging spectrometer data using the complete band shape least-squares algorithm simultaneously fit to multiple spectral features from multiple materials: in Proceedings of the Third Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Workshop, JPL Publication 90-54, p. 176 - 186.
Clark, R. N., Swayze, G. A., Gallagher, A., Gorelick, N., and Kruse, F. A., 1991, Mapping with imaging spectrometer data using the complete band shape least-squares algorithm simultaneously fit to multiple spectral features from multiple materials: in Proceedings, 3rd Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) workshop, JPL Publication 91-28, p. 2-3.
Clark, R. N., Swayze, G. A., and Gallagher, A., 1992, Mapping the mineralogy and lithology of Canyonlands, Utah with imaging spectrometer data and the multiple spectral feature mapping algorithm: in Summaries of the Third Annual JPL Airborne Geoscience Workshop, JPL Publication 92-14, v 1, p. 11-13.
Center for the Study of Earth from Space (CSES), 1992, SIPS User's Guide, The Spectral Image Processing System, v. 1.1, University of Colorado, Boulder,74 p.
Crowley, J. K., and Clark, R. N., 1992, AVIRIS study of Death Valley evaporite deposits using least-squares band-fitting methods: in Summaries of the Third Annual JPL Airborne Geoscience Workshop, JPL Publication 92-14, v 1, p. 29-31.
Clark, R. N., and Swayze, G. A., 1995, Mapping minerals, amorphous materials, environmental materials, vegetation, water, ice, and snow, and other materials: The USGS Tricorder Algorithm: in Summaries of the Fifth Annual JPL Airborne Earth Science Workshop, JPL Publication 95-1, p. 39 - 40.
Green, A. A., and Craig, M. D., 1985, Analysis of aircraft spectrometer data with logarithmic residuals: in Proceedings, AIS workshop, 8-10 April, 1985, JPL Publication 85-41, Jet Propulsion Laboratory, Pasadena, California, p. 111-119.
Kruse, F. A., Raines, G. L., and Watson, K., 1985, Analytical techniques for extracting geologic information from multichannel airborne spectroradiometer and airborne imaging spectrometer data: in Proceedings, International Symposium on Remote Sensing of Environment, Thematic Conference on Remote Sensing for Exploration Geology, 4th, Environmental Research Institute of Michigan, Ann Arbor, p. 309-324.
Kruse, F. A., Calvin, W. M., and Seznec, O., 1988, Automated extraction of absorption features from Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and Geophysical Environmental Research imaging spectrometer (GERIS) data: In Proceedings of the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) performance evaluation workshop, JPL Publication 88-38, p. 62-75.
Kruse, F. A., 1988, Use of Airborne Imaging Spectrometer data to map minerals associated with hydrothermally altered rocks in the northern Grapevine Mountains, Nevada and California: Remote Sensing of Environment, v. 24, no. 1, pp. 31-51.
Kruse, F. A., 1990, Artificial Intelligence for Analysis of Imaging Spectrometer Data: Proceedings, ISPRS Commission VII, Working Group 2: "Analysis of High Spectral Resolution Imaging Data", Victoria, B. C., Canada, 17-21 September, 1990, p. 59-68.
Kruse, F. A., Lefkoff, A. B., Boardman, J. W., Heidebrecht, K. B., Shapiro, A. T., Barloon, J. P., and Goetz, A. F. H., 1993a, The spectral image processing system (SIPS) - Interactive visualization and analysis of imaging spectrometer data: Remote Sensing of Environment, v. 44, p. 145 - 163.
Kruse, F. A., and Lefkoff, A. B., 1993b, Knowledge-based geologic mapping with imaging spectrometers: Remote Sensing Reviews, Special Issue on NASA Innovative Research Program (IRP) results, v. 8, p. 3 - 28.
Kruse, F. A., Lefkoff, A. B., and Dietz, J. B., 1993c, Expert System-Based Mineral Mapping in northern Death Valley, California/Nevada using the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS): Remote Sensing of Environment, Special issue on AVIRIS, May-June 1993, v. 44, p. 309 - 336.
Swayze, g. A., and Clark, R. N., 1995, Spectral identification of minerals using imaging spectrometry data: evaluating the effects of signal to noise and spectral resolution using the Tricorder Algorithm: in Summaries of the Fifth Annual JPL airborne Earth Science Workshop, JPL Publication 95-1, p. 157 - 158.
Yamaguchi, Yasushi, and Lyon, R. J. P., 1986, Identification of clay minerals by feature coding of near-infrared spectra: in Proceedings, International Symposium on Remote Sensing of Environment, Fifth Thematic Conference, "Remote Sensing for Exploration Geology", Reno, Nevada, 29 September- 2 October, 1986, Environmental Research Institute of Michigan, Ann Arbor, p. 627-636.
Selected Mapping Methods Using Hyperspectral Data