Image Fusion

Part I Theories 1 Introduction ... 3 1.1 Synergy ... 3 1.2 Fusion Categories ... 5 1.3 Image Fusion Process ... 5 1.4 Pixel Level Algorithms ... 6 1.4.1 Averaging ... 7 1.4.2 Multi-scale ... 7 1.4.3 Region ... 7 1.5 Organization ... 7 1.6 Further Reading ... 7 2 Image Sensors ... 9 2.1 Digital Camera ... 9 2.2 Optical System ... 10 2.2.1 Perspective projection ... 10 2.2.2 Orthographic projection ... 11 2.3 Recording Systems ... 11 2.3.1 Noise ... 11 2.4 Sampling ... 11 2.4.1 Quantization... 12 2.4.2 Bayer ... 12 2.5 Spatial vs. Spectral Resolution ... 13 2.5.1 Spatial Resolution ... 14 2.5.2 Spectral Resolution ... 15 2.6 Further Reading ... 15 3 Image Registration ... 17 3.1 Introduction ... 17 3.2 Pairwise Transformation ... 18 3.3 Hierarchical Registration ... 19 3.4 Mosaic Image ... 20 3.4.1 Stitching ... 22 3.5 Similarity Measures ... 23 3.6 Software ... 23 3.7 Further Reading ... 24 4 Radiometric Calibration ... 25 4.1 Histogram matching ... 25 4.2 Robust Equalization ... 26 4.3 Midway Image Equalization ... 26 4.4 Ranking ... 27 4.5 Thresholding ... 28 4.6 Segmentation ... 28 4.7 Software ... 29 4.8 Further Reading ... 29 5 Pixel Average Fusion ... 31 5.1 Pixel Averaging ... 31 5.2 Principal Components Analysis (PCA) ... 32 5.3 ICA ... 34 5.4 NMF ... 34 6 Wavelets ... 35 6.1 Introduction ... 35 6.2 Discrete Wavelet Transform (DWT) ... 36 6.3 Shift-Invariant Discrete Wavelet Transform (SiDWT) ... 36 6.4 The a' trous algorithm ... 36 6.4.1 Substitutive wavelet fusion scheme ... 37 6.4.2 Additive wavelet fusion scheme ... 38 6.5 Hybrid methods ... 38 6.6 AWL Method ... 39 6.7 Spectral Response ... 39 6.8 Window Spectral Response (WiSpeR) Method ... 40 6.8.1 Sensor Spectral Response ... 40 6.8.2 Physical Emission From Observed Object ... 41 6.8.3 Window Spectral Response (WiSpeR) Algorithm ... 42 6.9 Spectral Quality Assessment ... 43 6.10 Conclusion ... 43 7 Ensemble Learning ... 45 7.1 Ensemble Learning Methods ... 45 7.2 Multiple Image Transformations Ik ... 45 7.3 Multiple Subspace Transformations ... 46 7.3.1 2D-PCA ... 46 7.4 Multiple Color Spaces ... 47 7.4.1 Multiple Classifiers ... 47 7.5 Ensemble Thresholding ... 47 7.6 Multiple classifiers ... 48 7.7 Bagging ... 49 7.8 Ensemble Atlas based Segmentation ... 49 7.9 Atlas-based classifiers ... 50 II Technical Part 8 Mutual Information ... 53 8.1 Mutual information ... 53 8.1.1 Mutual Information of Small Sub-Images ... 55 8.1.2 The effect of Noise on Mutual Information ... 55 8.1.3 Kernel Density Estimation ... 55 9 Optical Flow... 59 9.1 Introduction ... 59 9.2 Lucas-Kanade ... 60 9.3 3-D Motion, Projected Motion and Optical Flow ... 61 9.4 The Motion Estimation Problem... 62 9.5 Complications in Motion Estimation ... 63 9.5.1 Occlusion ... 63 9.5.2 The aperture problem... 63 9.5.3 Aggregating constraints to remove ambiguities ... 64 9.5.4 Difficulties associated with multiple motions ... 64 9.5.5 Motion estimation as an ill-posed problem ... 65 9.6 Common motion models ... 66 10 Image Interpolation ... 69 10.1 Introduction ... 69 10.1.1 Nearest Neighbor Interpolation ... 71 10.1.2 Trilinear Interpolation ... 72 10.1.3 PVI ... 72 11 Image Thresholding ... 73 11.1 Global Thresholding ... 73 11.1.1 Entropy-based Image Thresholding ... 74 11.2 Local Thresholding ... 76 12 Image Key Points ... 77 12.1 Introduction ... 77 12.2 Scale-invariant feature transform ... 77 12.3 Speeded-Up Robust Feature ... 78 13 Image Similarity Measures ... 79 13.1 Introduction ... 79 13.1.1 Cross-Correlation Coefficient ... 80 13.1.2 Increment Sign Correlation Coefficient ... 80 13.2 Mutual information ... 80 13.3 Probabilistic Distance Measures ... 81 13.4 Binary Image Similarity Measure ... 81 13.5 Hausdorff Metric ... 83 13.6 Robust Image Similarity Measures ... 83 13.7 Robust Image Correlation Coefficients ... 84 13.8 Interval Scales in Image Fusion ... 87 14 Vignetting ... 89 14.1 Introduction ... 89 14.2 Vignetting Correction ... 90 15 Color Image Spaces ... 91 15.1 Introduction ... 91 16 Markov Random Fields ... 93 16.1 Introduction ... 93 16.2 Bayesian Method ... 94 16.3 Markov Random Fields ... 94 16.4 Energy Function ... 96 17 RANSAC ... 99 17.1 Introduction ... 99 17.2 Software ... 100 18 Image Quality ... 101 18.1 Introduction ... 101 18.2 Evaluation of Image Fusion Algorithms ... 101 18.3 Statistical Quality Measures ... 102 III Applications Part 19 Pan-Sharpening ... 107 19.1 IHS Fusion Technique ... 108 19.2 Spectral Distortion ... 110 19.3 The method of Choi et al... 111 19.3.1 Spectral Distortion ... 112 19.4 The method of Tu et al... 112 19.4.1 Spectral distortion ... 113 19.5 IKONOS Image Fusion ... 113 19.6 Wavelets ... 114 20 Video Surveillance ... 117 20.1 Affine alignment ... 117 21 Change Detection ... 119 21.1 Introduction ... 119 21.1.1 Simple differencing ... 120 21.1.2 Significance and hypothesis tests ... 120 21.2 Background modeling ... 120 21.2.1 Change mask consistency ... 121 21.3 Order Consistent Change detection ... 122 22 Biometric Technologies ... 123 22.1 Introduction ... 123 22.2 Multimodal biometrics ... 123 22.2.1 Fingerprints ... 124 22.2.2 Signatures ... 124 22.2.3 Faces ... 124 22.2.4 Iris and retina ... 125 22.2.5 Gait biometrics ... 125 22.2.6 Other biometrics ... 126 23 Super-Resolution ... 127 23.1 Introduction ... 127 23.2 Bayesian Framework ... 128 23.3 Modeling and Image Formation ... 129 23.4 ML Estimator ... 130 23.5 MAP Estimators ... 130