186 Pages
English

Illumination invariant interest point detection for vision based recognition tasks [Elektronische Ressource] / Flore Faille

Gain access to the library to view online
Learn more

Description

Illumination invariant interest pointdetection for vision based recognitiontasksFlore FaillePh.D. ThesisLehrstuhl fu¨r Realzeit-ComputersystemeIllumination invariant interest point detection forvision based recognition tasksFlore FailleVollst¨andiger Abdruck der von der Fakult¨at fu¨r Elektrotechnik und Informationstechnik derTechnischen Universit¨at Mun¨ chen zur Erlangung des akademischen Grades einesDoktor-Ingenieurs (Dr.-Ing.)genehmigten Dissertation.Vorsitzender: Univ.-Prof. Dr.-Ing. habil. G. RigollPru¨fer der Dissertation: 1. Univ.-Prof. Dr.-Ing. G. F¨arber2. Univ.-Prof. Dr.-Ing. E. SteinbachDie Dissertation wurde am 25. September 2006 bei der Technischen Universit¨at Mun¨ cheneingereicht und durch die Fakult¨at fur¨ Elektrotechnik und Informationstechnik am 17. Januar2007 angenommen.ContentsList of Figures viList of Tables ixList of Symbols x1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Main contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Outline of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 State of the art and related work 62.1 Image formation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 Interest point detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.2.1 State of the art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.2.2 The Harris detector . . . .

Subjects

Informations

Published by
Published 01 January 2007
Reads 21
Language English
Document size 12 MB

Illumination invariant interest point
detection for vision based recognition
tasks
Flore Faille
Ph.D. ThesisLehrstuhl fu¨r Realzeit-Computersysteme
Illumination invariant interest point detection for
vision based recognition tasks
Flore Faille
Vollst¨andiger Abdruck der von der Fakult¨at fu¨r Elektrotechnik und Informationstechnik der
Technischen Universit¨at Mun¨ chen zur Erlangung des akademischen Grades eines
Doktor-Ingenieurs (Dr.-Ing.)
genehmigten Dissertation.
Vorsitzender: Univ.-Prof. Dr.-Ing. habil. G. Rigoll
Pru¨fer der Dissertation: 1. Univ.-Prof. Dr.-Ing. G. F¨arber
2. Univ.-Prof. Dr.-Ing. E. Steinbach
Die Dissertation wurde am 25. September 2006 bei der Technischen Universit¨at Mun¨ chen
eingereicht und durch die Fakult¨at fur¨ Elektrotechnik und Informationstechnik am 17. Januar
2007 angenommen.Contents
List of Figures vi
List of Tables ix
List of Symbols x
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Main contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Outline of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 State of the art and related work 6
2.1 Image formation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Interest point detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 State of the art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.2 The Harris detector . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.3 Extension of the Harris detector for colour images . . . . . . . . . . 14
2.3 Handling illumination variations . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.1 Grey value image processing . . . . . . . . . . . . . . . . . . . . . . 16
2.3.2 Colour image processing . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3 Illumination invariant interest point detection for grey value images 25
3.1 Illumination influence on the Harris detector . . . . . . . . . . . . . . . . . 25
3.2 Local normalisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3 Homomorphic Harris detector . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4 Local threshold adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.5 Local clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.6 Handling of saturated areas . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.7 Comparison framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.7.1 Quantitative evaluation of the detection stability . . . . . . . . . . 47
3.7.2 Compared interest point detectors . . . . . . . . . . . . . . . . . . . 49
3.7.3 Image data set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.8 Detector evaluation and comparison . . . . . . . . . . . . . . . . . . . . . . 53
3.8.1 Simple illumination changes . . . . . . . . . . . . . . . . . . . . . . 53
3.8.2 Complex illumination changes . . . . . . . . . . . . . . . . . . . . . 55
3.8.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
iiiContents
3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4 Illumination invariant interest point detection for colour images 63
4.1 Colour image acquisition and demosaicing . . . . . . . . . . . . . . . . . . 63
4.1.1 Presentation of the demosaicing algorithms . . . . . . . . . . . . . . 66
4.1.2 Comparison of the algorithms . . . . . . . . . . . . . . 69
4.1.3 Selection of the most appropriate demosaicing method . . . . . . . 73
4.2 Image formation model and Harris detector for colour images . . . . . . . . 75
4.3 Robust invariant interest point detector . . . . . . . . . . . . . . . . . . . . 78
4.4 Homomorphic colour interest point detector . . . . . . . . . . . . . . . . . 81
4.5 M space interest point detector . . . . . . . . . . . . . . . . . . . . . . . . 84
4.6 Preprocessing for the M space detector . . . . . . . . . . . . . . . . . . . . 91
4.7 Comparison framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.7.1 Compared interest point detectors . . . . . . . . . . . . . . . . . . . 95
4.7.2 Image data set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.8 Detector evaluation and comparison . . . . . . . . . . . . . . . . . . . . . . 97
4.8.1 Simple illumination changes . . . . . . . . . . . . . . . . . . . . . . 97
4.8.2 Complex illumination changes . . . . . . . . . . . . . . . . . . . . . 100
4.8.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5 Application to a recognition task 108
5.1 System overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.2 Interest point characterisation . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.2.1 Choice of the descriptor algorithm . . . . . . . . . . . . . . . . . . . 110
5.2.2 SIFT descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.3 Stereo reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.3.1 Principles of stereo vision . . . . . . . . . . . . . . . . . . . . . . . 115
5.3.2 Finding correspondences . . . . . . . . . . . . . . . . . . . . . . . . 117
5.3.3 3D reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.4 Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.4.1 Descriptor similarity constraint . . . . . . . . . . . . . . . . . . . . 123
5.4.2 Geometric constraints . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.4.3 Match list constraints. . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.5 Recognition and localisation . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.5.1 Choice of the recognition and localisation algorithm . . . . . . . . . 131
5.5.2 Filling the accumulators . . . . . . . . . . . . . . . . . . . . . . . . 131
5.5.3 Taking uncertainties into account . . . . . . . . . . . . . . . . . . . 134
5.5.4 Interpreting the accumulators . . . . . . . . . . . . . . . . . . . . . 136
5.6 Evaluation framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5.6.1 Evaluation criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5.6.2 Compared interest point detectors . . . . . . . . . . . . . . . . . . . 141
5.6.3 Object database and test images . . . . . . . . . . . . . . . . . . . 142
5.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
ivContents
5.7.1 Recognition and localisation quality . . . . . . . . . . . . . . . . . . 145
5.7.2 Detector suitability for recognition and localisation . . . . . . . . . 151
5.7.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
5.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
6 Conclusion 157
6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
6.2 Further research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Bibliography 161
vList of Figures
1.1 Overview of a general object recognition system . . . . . . . . . . . . . . . 2
2.1 The different elements of the image formation model. . . . . . . . . . . . . 7
2.2 Border handling for convolution. . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Detection example with the Harris detector. . . . . . . . . . . . . . . . . . 15
2.4le with the Harrisr for colour images. . . . . . . . 15
3.1 Detection example for the HD on the same scene under two different illu-
minations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Suppression of the illumination influence on the derivatives with energy
normalisation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3 Detection example for the N–HD on the same scene under two different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.4 Suppressionoftheilluminationinfluenceonthederivativeswithhomomor-
phic processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.5 Detection example for the H–HD on the same scene under two different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.6 BehaviouroftheratioCF/CF inanimageseriesofasceneunderdifferent
illuminations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.7 Grey value image and corresponding local standard deviation of CF . . . . 35
3.8 Mean and standard deviation of the noise on|CF| . . . . . . . . . . . . . . 36
3.9 Mean and standard deviation of the noise on ln(|CF|) . . . . . . . . . . . . 37
3.10 Detection of the textured areas with the local standard deviation of ln(|CF|) 38
3.11 Histogram of the noise standard deviation on ln(|CF|). . . . . . . . . . . . 39
3.12 Detection example for the AT–HD on the same scene under two different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.13 Discretisation for the local thresholding. . . . . . . . . . . . . . . . . . . . 41
3.14 Histogram of ln(|CF|) in image patches . . . . . . . . . . . . . . . . . . . . 41
3.15 Results of the ISODATA algorithm on neighbourhoods of ln(|CF|) . . . . . 43
3.16 Detection example for the LI–HD on the same scene under two different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.17 Detection and handling of the saturated areas in the image . . . . . . . . . 46
3.18 Two images of the series with shutter time variations. . . . . . . . . . . . . 51
3.19 Sample images of the series with complex illumination variations. . . . . . 52
3.20 Evaluation results for the series with shutter time variations. . . . . . . . . 54
3.21 Evaluation results for the nesquik series . . . . . . . . . . . . . . . . . . . . 55
viList of Figures
3.22 Evaluation results for the nesquik series depending on the complexity mea-
sure CM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.23 Evaluation results for the paper and the calendar series . . . . . . . . . . . 58
3.24 Evan results for the rabbit and for the shelves series . . . . . . . . . . 60
4.1 Bayer CFA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Typical demosaicing artifacts. . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.3 Influence of demosaicing on the colour gradient. . . . . . . . . . . . . . . . 65
4.4 Kernel for median filtering. . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.5 Interpolation of the G value at a sampled R pixel with ACPI. G values at
sampled B pixels are interpolated similarly. . . . . . . . . . . . . . . . . . . 68
4.6 WACPI directions and neighbourhood shown in fig. 4.7. . . . . . . . . . . . 68
4.7 Weight and contribution of the left direction (see fig. 4.6) to the interpola-
tion of the G value at pixel position R6. . . . . . . . . . . . . . . . . . . . 69
4.8 Five images of the Kodak colour image database. . . . . . . . . . . . . . . 69
4.9 Demosaicing results for a colourful image part . . . . . . . . . . . . . . . . 70
4.10icing results for a textured image part . . . . . . . . . . . . . . . . 71
4.11 Influence of white balancing on demosaicing results . . . . . . . . . . . . . 74
4.12 Detection example for the C–HD on the same scene under two different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.13 Detection example for the RI–HD on the same scene under two different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.14 Suppression of the illumination influence on colour derivatives with homo-
morphic processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.15 Detection example for the HC–HD on the same scene under two different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.16 Suppression of the illumination influence on colour derivatives . . . . . . . 87
4.17 Influence of the preprocessing on the m space . . . . . . . . . . . . . . . . 88
4.18 Detection example for the MS–HD on the same scene under two different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.19 Neighbourhoods used for the Nagao filter. . . . . . . . . . . . . . . . . . . 93
4.20bourhoods used for the simplified Nagao filter. . . . . . . . . . . . . . 93
4.21 Results of the preprocessing methods on an enlarged image detail. . . . . . 94
4.22 Influence of the preprocessing methods on the m space gradient. . . . . . . 95
4.23 Two images of the series with shutter time variations. . . . . . . . . . . . . 97
4.24 Sample images of the series with complex illumination variations. . . . . . 98
4.25 Evaluation results for the series with increasing shutter time. . . . . . . . . 100
4.26 Evan results for the shelves series . . . . . . . . . . . . . . . . . . . . 101
4.27 Evaluation results for the box series . . . . . . . . . . . . . . . . . . . . . . 102
4.28 Evan results for the giraffe and box2 series . . . . . . . . . . . . . . . 103
4.29 Evaluation results for the rabbit and snoopy series . . . . . . . . . . . . . . 105
5.1 Degrees of freedom of the recognition system . . . . . . . . . . . . . . . . . 109
5.2 Overview of the recognition system . . . . . . . . . . . . . . . . . . . . . . 110
5.3 SIFT descriptor overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
viiList of Figures
5.4 Perspective camera model. . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.5 Stereo vision and epipolar geometry. . . . . . . . . . . . . . . . . . . . . . 116
5.6 Search areas for stereo vision . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.7 Correspondences between two stereo images . . . . . . . . . . . . . . . . . 119
5.8 3D Reconstruction of a scene point. . . . . . . . . . . . . . . . . . . . . . . 120
5.9 Histograms of the descriptor distances between different interest points
and between interest points showing the same scene point under different
illuminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.10 Selection of the threshold D . . . . . . . . . . . . . . . . . . . . . . . . . 125lim
5.11 Threshold selection for the descriptors based on homomorphic grey value
descriptors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.12 Influence of the symmetry constraint on the matching process. . . . . . . . 130
5.13 Used coordinate systems and estimated localisation parameters . . . . . . . 132
5.14 Example of accumulator filling . . . . . . . . . . . . . . . . . . . . . . . . . 137
5.15 ofulator filling for images of two different objects . . . . . 138
5.16 Images used to create the database. . . . . . . . . . . . . . . . . . . . . . . 143
5.17 The five different poses for the test images of object 10. . . . . . . . . . . . 144
5.18 Different illuminations for the test images of object 10. . . . . . . . . . . . 145
5.19t test images of an object not contained in the database. . . . . . . 145
5.20 Recognition rate and localisation accuracy for the different detectors. . . . 148
5.21 Number of votes for the best pose hypothesis for the different detectors. . . 149
5.22 Matching consistency and deviation for the different detectors. . . . . . . . 150
5.23 Detector suitability for recognition and localisation. . . . . . . . . . . . . . 153
viii