Snapshot hyperspectral imaging using wide dilation networks
- PDF / 1,482,375 Bytes
- 11 Pages / 595.276 x 790.866 pts Page_size
- 14 Downloads / 208 Views
ORIGINAL PAPER
Snapshot hyperspectral imaging using wide dilation networks Mikko E. Toivonen1
· Chang Rajani1 · Arto Klami1
Received: 4 September 2019 / Revised: 16 September 2020 / Accepted: 29 September 2020 © The Author(s) 2020
Abstract Hyperspectral (HS) cameras record the spectrum at multiple wavelengths for each pixel in an image, and are used, e.g., for quality control and agricultural remote sensing. We introduce a fast, cost-efficient and mobile method of taking HS images using a regular digital camera equipped with a passive diffraction grating filter, using machine learning for constructing the HS image. The grating distorts the image by effectively mapping the spectral information into spatial dislocations, which we convert into a HS image by a convolutional neural network utilizing novel wide dilation convolutions that accurately model optical properties of diffraction. We demonstrate high-quality HS reconstruction using a model trained on only 271 pairs of diffraction grating and ground truth HS images. Keywords Hyperspectral imaging · Deep learning · Convolutional neural networks
1 Introduction In hyperspectral imaging, one wishes to capture an image that provides for each pixel the spectrum at a continuous range of wavelengths [1,2]. Since many materials have a unique spectral signature, one can use HS images to, for example, segment images according to materials [3]. This makes HS images useful in wide range of tasks in science and industry, such as satellite surveying [4,5], food quality assurance [6], gas and oil exploration [7], and various medical applications [8]. Special devices called hyperspectral cameras are used to take HS images. These devices generally operate by scanning the scene either spatially (spatial scanning) or spectrally (spectral scanning) [9], and capture tens to hundreds of spectral channels to preserve the shape of the spectrum, as opposed to multispectral cameras that record fewer, possibly disjoint, spectral channels [3]. Capturing a single image in good lighting conditions might take tens of seconds using a scanning method, since the camera needs to capture each spa-
B
Mikko E. Toivonen [email protected] Chang Rajani [email protected] Arto Klami [email protected]
1
Department of Computer Science, University of Helsinki, Helsinki, Finland
tial or spectral dimension separately. Furthermore, the spatial resolution at which these cameras operate is typically low— for example, the Specim IQ, a portable HS camera, yields images of size 512 × 512 [2], and more refined stationary models yield images of 1–2 MP. These specialized devices are also expensive, currently costing in the order of tens of thousands of euros or US dollars. In contrast to the scanning approach, snapshot imaging techniques capture the entire hyperspectral cube at once. They are based, for example, on prism and beam-splitter constructs [10], per-pixel filters at the image sensor [11], or tunable narrow-band optical filters [12]. These methods have the advantage of short capture time, but
Data Loading...
