Курсы английского
<<  Create the Internet of Your Things: The Microsoft Vision for IoT NARUC Staff Subcommittee Accounting and Finance Accounting for Emission Allowances  >>
Improving the Fisher Kernel for Large-Scale Image Classi
Improving the Fisher Kernel for Large-Scale Image Classi
From generative modeling to features
From generative modeling to features
Simplest example
Simplest example
Fisher vector idea
Fisher vector idea
Fisher vector idea
Fisher vector idea
Fisher vector for image classification
Fisher vector for image classification
Relation to BoW
Relation to BoW
Whitening the data
Whitening the data
Classification with Fisher kernels
Classification with Fisher kernels
Improvements to Fisher Kernels
Improvements to Fisher Kernels
Improvement: power normalization
Improvement: power normalization
Improvement 3: spatial pyramids
Improvement 3: spatial pyramids
Results: Pascal 2007
Results: Pascal 2007
Results: Caltech 256
Results: Caltech 256
PASCAL + additional training data
PASCAL + additional training data
Conclusion
Conclusion

Презентация на тему: «Improving the Fisher Kernel for Large-Scale Image Classi?cation». Автор: Lempitsky. Файл: «Improving the Fisher Kernel for Large-Scale Image Classi?cation.ppt». Размер zip-архива: 852 КБ.

Improving the Fisher Kernel for Large-Scale Image Classi?cation

содержание презентации «Improving the Fisher Kernel for Large-Scale Image Classi?cation.ppt»
СлайдТекст
1 Improving the Fisher Kernel for Large-Scale Image Classi

Improving the Fisher Kernel for Large-Scale Image Classi

cation

Florent Perronnin, Jorge Sanchez, and Thomas Mensink, ECCV 2010

VGG reading group, January 2011, presented by V. Lempitsky

2 From generative modeling to features

From generative modeling to features

dataset

Discriminative classfier model

Input sample

Generative model

fitting

Parameters of the fit

3 Simplest example

Simplest example

Dataset of vectors

Discriminative classfier model

Input vector

K-means

Codebook

fitting

Codebooks Sparse or dense component analysis Deep belief networks Color GMMs ....

Closest codeword

4 Fisher vector idea

Fisher vector idea

Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classi?ers. NIPS’99

Generative model

Discriminative classfier model

Input sample

Parameters of the fit

fitting

Information loss (generative models are always inaccurate!)

Can we retain some of the lost information without building better generative model?

Main idea: retain information about the fitting error for the best fit.

Same best fit, but different fitting errors!

5 Fisher vector idea

Fisher vector idea

X

?

(?1,?2)

Fisher vector:

Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classi?ers. NIPS’99

Generative model

Discriminative classfier model

Input sample

Fisher vector

fitting

Main idea: retain information about the fitting error of the best fit.

6 Fisher vector for image classification

Fisher vector for image classification

F. Peronnin and C. Dance // CVPR 2007

Assuming independence between the observed T features Encoding each visual feature (e.g. SIFT) extracted from image to a Fisher vector Using N-component gaussian mixture models with diagonalized covariance matrices:

N dimensions

128N dimensions

128N dimensions

7 Relation to BoW

Relation to BoW

BoW

Extra info

F. Peronnin and C. Dance // CVPR 2007

N dimensions

128N dimensions

128N dimensions

8 Whitening the data

Whitening the data

Fisher matrix (covariance matrix for Fisher vectors):

Whitening the data (setting the covariance to identity):

Fisher matrix is hard to estimate. Approximations needed:

[Peronnin and Dance//CVPR07] suggest a diagonal approximation to Fisher matrix:

9 Classification with Fisher kernels

Classification with Fisher kernels

Use whitened Fisher vectors as an input to e.g. linear SVM Small codebooks (e.g. 100 words) are sufficient Encoding runs faster than BoW with large codebooks (although with approximate NN this is not so straightforward!) Slightly better accuracy than “plain, linear BoW”

F. Peronnin and C. Dance // CVPR 2007

10 Improvements to Fisher Kernels

Improvements to Fisher Kernels

=0

Perronnin, Jorge Sanchez, and Thomas Mensink, ECCV 2010

Overall very similar to how people improve regular BoW classification

Idea 1: normalization of Fisher vectors. Justification:

our GMM

probability distribution of VW in an image

Assume:

Image specific “content”

Then:

Thus:

Observation: image non-specific “content” affects the length of the vector, but not direction

Conclusion: normalize to remove the effect of non-specific “content” ...also L2-normalization ensures K(x,x) = 1 and improves BoV [Vedaldi et al. ICCV’09]

11 Improvement: power normalization

Improvement: power normalization

? =0.5 i.e. square root works well c.f. for example [Vedaldi and Zisserman// CVPR10] or [Peronnin et al.//CVPR10] on the use of square root and Hellinger’s kernel for BoW

12 Improvement 3: spatial pyramids

Improvement 3: spatial pyramids

Fully standard spatial pyramids [Lazebnik et al.] with sum-pooling

13 Results: Pascal 2007

Results: Pascal 2007

Details: regular grid, multiple scales, SIFT and local RGB color layout, both reduced to 64 dimensions via PCA

14 Results: Caltech 256

Results: Caltech 256

15 PASCAL + additional training data

PASCAL + additional training data

Flickr groups up to 25000 per class ImageNet up to 25000 per class

16 Conclusion

Conclusion

Fisher kernels – good way to exploit your generative model Fisher kernels based on GMMs in SIFT space lead to state-of-the-art results (on par with the most recent BoW with soft assignments) Main advantage of FK over BoW are smaller dictionaries ...although FV are less sparse than BoV Peronnin et al. trained their system within a day for 20 classes for 350K images on 1 CPU

«Improving the Fisher Kernel for Large-Scale Image Classi?cation»
http://900igr.net/prezentacija/anglijskij-jazyk/improving-the-fisher-kernel-for-large-scale-image-classication-159354.html
cсылка на страницу
Урок

Английский язык

29 тем
Слайды
900igr.net > Презентации по английскому языку > Курсы английского > Improving the Fisher Kernel for Large-Scale Image Classi?cation