32 Pages
Gain access to the library to view online
Learn more

Bach Mairal Ponce Sapiro


Gain access to the library to view online
Learn more
32 Pages


Dictionary Learning ICCV 2009 Bach, Mairal, Ponce, Sapiro

  • random vector

  • learning sparsity

  • energy minimization

  • prototype signal

  • vector ?

  • being learned

  • given measurements

  • contains very



Published by
Reads 24
Language English
Document size 1 MB


Dictionary Learning
ICCV 2009
Bach, Mairal, Ponce, SapiroRestoration by Energy Minimization
Restoration/representation algorithms are often related to the minimization
of an energy function of the form
1 2
f x   x  y  Pr x 
y : Given measurements Relation to
Prior or regularization
measurementsx : Unknown to be recovered
 Bayesian type of approach
 What is the prior? What is the image model? Thomas Bayes
1702 - 1761
Learning Sparsity 2The Sparseland Model for Images
 Every column in
D (dictionary) is
a prototype signal M K (Atom).
 The vector N 
contains very few
(say L) non-zeros.
A sparse A fixed Dictionary x
& random
vectorD α
Learning Sparsity 3What Should the Dictionary D Be?
1 2 0
  argmin D   y s.t.   L ˆx  D ˆ ˆ
D should be chosen such that it sparsifies the representations
Learn D :
One approach to choose D is from a
Multiscale Learningknown set of transforms (Steerable
wavelet, Curvelet, Contourlets,
Color Image Examples
Bandlets, …)
Task / sensing adapted
Internal structure
Learning Sparsity 4What is being learned?
Learning Sparsity 5Learning D to reconstruct
X D A
P 2 0
Min D   x s.t. j,   L jj j Field & Olshausen (‘96)2 0 Engan et. al. (‘99)D,A j 1 Lewicki & Sejnowski (‘00)
Each example has a Each example is Cotter et. al. (‘03)
Gribonval et. al. (‘04)sparse representation with a linear combination
Aharon, Elad, & Bruckstein (‘04)
no more than L atomsof atoms from D Aharon, Elad, & Bruckstein (‘05)
Ng et al. (‘07)
Mairal, Sapiro, Elad (‘08)
Learning Sparsity 6The K –SVD Algorithm – General
Aharon, Elad, & Bruckstein (`04)
Initialize DD
Sparse Coding
Orthogonal Matching TPursuit (or L1)
Column-by-Column by
SVD computation over
the relevant examples
Learning Sparsity 7Non-uniform noise
Learning Sparsity 8Show me the pictures
Learning Sparsity 9Change the Metric in the OMP
Learning Sparsity 10