14 Pages
English

Semi supervised learning of facial attributes in video

Gain access to the library to view online
Learn more

Description

Niveau: Supérieur, Doctorat, Bac+8
Semi-supervised learning of facial attributes in video Neva Cherniavsky1, Ivan Laptev1, Josef Sivic1, Andrew Zisserman1,2 1INRIA, WILLOW, Laboratoire d'Informatique de l'Ecole Normale Superieure, ENS/INRIA/CNRS UMR 8548 2Dept. of Engineering Science, University of Oxford Abstract. In this work we investigate a weakly-supervised approach to learning facial attributes of humans in video. Given a small set of images labeled with attributes and a much larger unlabeled set of video tracks, we train a classifier to recognize these attributes in video data. We make two contributions. First, we show that training on video data improves classification performance over training on images alone. Second, and more significantly, we show that tracks in video provide a natural mech- anism for generalizing training data – in this case to new poses, light- ing conditions and expressions. The advantage of our method is demon- strated on the classification of gender and age attributes in the movie “Love, Actually”. We show that the semi-supervised approach adds a significant performance boost, for example for gender increasing average precision from 0.75 on static images alone to 0.85. 1 Introduction Classification of people according to their attributes is an area of active research, both as a first step in the larger problem of image search and classification on identity [1, 2], and as a goal in and of itself.

  • distinct faces

  • still image

  • face tracks

  • false con- nections

  • labeled

  • query examples

  • facial feature


Subjects

Informations

Published by
Reads 23
Language English
Document size 1 MB

Exrait

Semi-supervisedlearningoffacialattributesinvideoNevaCherniavsky1,IvanLaptev1,JosefSivic1,AndrewZisserman1,21INRIA,WILLOW,Laboratoired’Informatiquedel’EcoleNormaleSupe´rieure,ENS/INRIA/CNRSUMR85482Dept.ofEngineeringScience,UniversityofOxfordAbstract.Inthisworkweinvestigateaweakly-supervisedapproachtolearningfacialattributesofhumansinvideo.Givenasmallsetofimageslabeledwithattributesandamuchlargerunlabeledsetofvideotracks,wetrainaclassifiertorecognizetheseattributesinvideodata.Wemaketwocontributions.First,weshowthattrainingonvideodataimprovesclassificationperformanceovertrainingonimagesalone.Second,andmoresignificantly,weshowthattracksinvideoprovideanaturalmech-anismforgeneralizingtrainingdata–inthiscasetonewposes,light-ingconditionsandexpressions.Theadvantageofourmethodisdemon-stratedontheclassificationofgenderandageattributesinthemovie“Love,Actually”.Weshowthatthesemi-supervisedapproachaddsasignificantperformanceboost,forexampleforgenderincreasingaverageprecisionfrom0.75onstaticimagesaloneto0.85.1IntroductionClassificationofpeopleaccordingtotheirattributesisanareaofactiveresearch,bothasafirststepinthelargerproblemofimagesearchandclassificationonidentity[1,2],andasagoalinandofitself.Forexample,culturalsociologistsareinterestedinmeasuringtheevolutionovertimeofthecharacterizationofgen-der[3]inTVandmovies.Videoanalysisforthesepurposescurrentlyrequireshoursoftediousmanuallabeling,renderinglarge-scaleexperimentsinfeasible.Automatingthedetectionandclassificationofhumantraitsinvideowillpoten-tiallyincreasethequantityanddiversityofexperimentaldata.Ourgoalinthispaperistolearnandclassifyhumanattributesinvideo.Theideaweexploreisthatvideotracksprovideavirtuallyfreeandlimitlesssourceoftrainingdata,sincemanyhumanattributes,e.g.gender,race,age,haircolour,areunchangedoverthecourseofatrack.Forexample,ifwecancorrectlydeterminethegenderofafaceinavideoface-track,wecanthenapplythatlabeltotherestofframeswithinthetrack,includingfacesthatwouldnormallybedifficulttoclassify.Wecanthustakeadvantageofthefullvariationinposesandviewpoints.Itmightbethoughtthatvideoscouldbeclassifiedbytrainingaclassifieronphotosoffaces,forexamplefromflickr.However,asweshowquantitatively,trainingonstillimagedatadoesnotgeneralizewelltovideodata.Although