DOC

An

By Evelyn Hayes,2014-07-23 03:24
14 views 0
An

    An

Vbl_34.No.7ACTAAUTOMATICASINICAJuly,2008

    ;AnImprovedOpticalFlowMethodforImage

    ;RegistrationwithLarge?-scaleMovements

    ;XIONGJingYiLUOYuPinTANGGuang-Rong1

    ;AbstractInthispaper,animprovedopticalflowmethodforimageregistrationisproposed.Itisnovelinthewaythatitimproves

    ;theopticalflowmethodwithaninitialmotionestimator:extendedphasecorrelationtechnique(EPCT),usingmeritsofthelatterto

    ;compensatedeficienciesoftheformer.Inamoredetailedmanner.itcanbesaidthattheopticalflowmethodcanreachthesub-pixel

    ;accuracyandcalculatecomplexdistortionpatternslikechirpingandtiltingbutisweakwithlarge-scalemovements.BecauseEPCT

    ;coversmeasurementsoflargetranslationsandrotationswithpixellevelaccuracyandisefficientinthecalculatingload,itcanbe

    ;treatedasagoodinitialmotionestimatorforopticalflowmethod.Testshaveprovedthatthisimprovedmethodwillsignificantly

    ;enhancetheregistrationperformance,especially,forimageswithlargescalemovement

    sandrobustagainstrandomnoises.

    ;KeywordsImageregistration,improvedopticalflowmethod,motionestimator,extendedphasecorrelationtechnique(EPCT)

    ;Imageregistration.alundamentimageprocessmg

    ;problem,1stheProcessotoverlayingasequenceotimages

    ;ofthesamescenetakenatdifferenttimes,fromdifierent

    ;viewpoints,and/orbydifierentsensors.Duringthispro-

    ;cess.coordinatetransformationsarecalculatedsothatim

    ;agestakenfromthesamestaticscenearerelated.Image

    ;registrationhasbeenwidelyusedasanintermediatestep

    ;inmanyresearchfields,suchassuperresolution,panorama

    ;mosaics,medicalphotosanalysis,etc.

    ;Generallyspeaking,twocategoriesofregistrationalgo

    ;rithmsexist:feature.basedandnonfeaturebased.Inthe

    ;formercategory,therearealgorithmsusinglow-levelfea-

    ;tureslikeedgesandcorners,andhigh.1evelfeatures,such

    ;asidentifiedobjects.orrelationsbetweenfeaturest.While

    ;inthenonfeature-basedcategory.therearealgorithmsus

    ;ingfrequencydoraaininformation[2J.anddifferentialopti

    ;calflowequationmethodt.

    ;Fortheopticalflow.peoplefinditisonthebasisofthe

    ;Taylorexpansionsanddifferentiaitheory,andthus,itis

    ;weaktoestimatelarge-scalemovementsbetweenimages. ;ItwasproposedbyGibsoin1950,andthattheoptical ;flowmethodissensitivetonoisesbecauseitisontheba- ;sisofdifferentialtechnology.Somefilters(bothhighpass

    ;andlow-pass1areusedtoreducethisbadeffect.More

    ;over,Gaussianpyramidsareusedtoconstructdifierent ;levelsofresolutionandincrementallyaccumulatethetrack ;ofeachlevelusingthelawofcompositiont.Thisisuse- ;fulforenhancingtheabilitytoestimatelargescalemove

    ;ments.butitisstillnotenough.Someideashavebeen ;reportedtofurtherimprovetheregistrationbyusingan ;initiaimotionestimatorprovidingaroughinputforfol

    ;lowingcalculation[07I.Someworkshavebeendonebased

    ;onthis,likelog-polarmappingreliedonnonlinearleast ;squareiterativeoptimizationalgorithminl8I,however,this ;methodhasaheavycalculatingload.Andphasecorrela

    ;tionmethodreliedonnonrigidopticalflowestimationin ;91,whichisonlyspecifiedinindocyaninegreenangiogra- ;DllvfICGA)funduswithoutrotatioas.

    ;0nthebasisofthepreviouswork,anewseparatemo- ;tionpreestimator,EPCT(Extendphasecorrelationtech

    ;nique1.isregardedasamoresuitablechoiceinthispaper. ;Theproposedapproachtakesgreatadvantageoftheprop

    ;ertiesofEPCT.measurementsoflargetranslationsandto- ;tations,andcalculatingefficiency,sothattheincreaseof ;workloadisalineargrowthoftheimagesize.Although ;EPCTonlyreachespixellevelaccuracyandcannotdeal ;ReceivedMaly14,2007;inr~visedformAugust9,2007 ;1.DepartmentofAutomation,TsinghuaUniversity,Beijing ;100084.P.R.China

    ;DOI:l0.3724/SP.J.1004.2008.00760

    ;withmorecomplexdistortionslikekeystoningorchirping, ;thesedeficiencieswillberefinedbytheopticalflowmethod ;later.

    ;Anotherworkdoneisdefiningawaytogaugetheregis- ;trationaccuracyandtorevisethelooptimeinopticaiflow ;fromfixedtoself-adaptiveaccordingtoaccuracy.Thus,it ;willsavetimeandeffortincalculation.

    ;Thispaperisorganizedasfollows.Section1outlinesthe ;principlesoftheprojectivemodel,opticalflowmethod,as ;wellasEPCT.then.describeshowtoimproveopticalflow ;methodbyusingEPCTasaprocessortoestimateaninitial ;input.Section2presentssomeregistrationresultsusingthe ;improvedopticalflowmethodandcomparestheresultsof

    ;EPCTwiththatoftheoriginalopticalflowmethod.In ;Section3,conclusionsandsomeresearchperspectivesare ;given.

    ;1Principles

    ;1.1Projectivemodel

    ;Theprojectivemodelischosentodescribemovements

    parameterequationpair ;betweentwoimages.Itisaneight

    ;dealingwithrotation,translation,scaling,chirping,and ;tilting.

    ;z,=

    ;?n6x?77,7Y,=?n

    ;6x()十十lr71

    ;Itcanberewrittenasamari)(

    ;m2

    ;(2)

    ;wherex/,Y=v/w.

    ;Inthismodel,m2and?77,5standfortranslationsinhori

    ;zontalandverticaldirectionsseparately,?77,0,777,1,?77,3,and

    ;m4forscalingandrotation,and?77,6and?77,7areincharge ;ofchirpingandkeystoningeffects.Anexampleislistedin ;Fig.1(seenextpage)fordetailsofeveryparameter. ;1.2Opticalflowmethod

    ;Intheopticalflowassumption,foreachpoint(x,Y)in ;flamet,therewillbeacorrespondingpointiname+?,

    ;whjchmeans

    ;E(z,,t)=E+?z,+?,t+?t)(3)

    ;Applyraylorexpansionstotherightsideof(3),?regetthe

    ;t?dimensionalopticalfl0wconstraint:

    ;++=0c4d.d.d.,

    ;

    ;No.7XIONGJing-Yieta1.:AnImprovedOpticalFlowMethodforImagesRegistration’?’

    761

    ;?

    ;??(e)(f)

    ;Fig

    ;,

    ;.1Relationshipbetweenparametersandimagemotions ;(caThe.rname,m=f;;(b)Horizontal

    ;;(d)Chirpingeffect,m=

    ;,m=

    ;

    ;m=00780Rotation001,=1.1:f.

    ;II

    ;m=.

;)

    ;Definecostfunctiona8

    ;.=

    ;?(ue+e+et)(5)

    ,=YY,e=,e=,and ;whereu=

    ;et=.

    ;Minimizethecostfunctionsothatitsatisfiestheoptical ;flowconstraintasmuchaspossible.Thesmallerthevalue ;of0w,thebettertheregistration.Moredetailscanbe ;foundin[5].

    ;BecauseopticalflowmethodisonthebasisofTaylor ;expansions,anddxandaresubstitutedby,_and

    ;

    ;incalculation,itisweakinestimatinglarge-scale ;movements.Insomecircumstance,itmightevenleadto ;failures.Ifthereisamotionestimationfortheopticalflow ;methodasaninitialinputtoturnthelarger-scalemove- ;mentsintosmaller-scalemovements,itwillbebeneficialfor ;registrationeffect.Thisinitialestimationdoesnotneedto ;beprecisebecauseopticalflowisgoodataccuracyandwill ;refineitsresults,andthelesscosttheprrocessor,the

    ;betterthealgorithm.Agoodchoiceforthispreprocessor ;isEPCTaccordingtoourpreviousanalysisintheintro

    ;ductionsection.

    ;1.3EPCT

    ;Assumethat,2,Y)isatranslatedandrotatedreplica ;of,1,Y),suchthat

    ;,2,)=fl

    ;

    ;(xc

    ;s

    ;o

    ;in

    ;s

    ;6l0

    ;g0

    ;+

    ;+y

    ;co

    ;sin

    ;s

    ;g

    ;6l0

    ;0--

;

    ;x

    ;

    ;0

    ;),(6)

    ;sin0+cos0一珈)

    ;AccordingtothepropertiesofFouriertransform,(6)can ;betransferredfromspacedomaintothefrequencydomain ;as

    ;?,cos.9+0sin90,一?sin90+sin90)(7)R+,一?+)

    ;LetM1andM2bethemagnitudesofEland.From

    ;(7),becausethemagnitudeofe-J(?0+nYo)is1,wehave

    ;(,)M1COS90+sin90,--~sin90+COS90)(8)

    ;MappingthemagnitudefromCartesiancoordinatesto ;polarcoordinates,wehave

    ;M1(P,)=M2(P,90)(9)

    ;whereP,//+,=tan().

    ;Aftertheoperationsmentionedabove,thisproblemhas ;becomeatypicaloneforPCTinpolarcoordinatesbywhich ;90iscalculated.Byrotatingoneoftheframesaccording ;to90,therewillbeonlytranslationsbetweenimages.PCT ;shouldbeappliedagaininCartesiancoordinatessothat ;translationsinverticalandhorizontaldirectionscallbecal- ;culated.

    ;1.4Theimprovedopticalflowalgorithm

    ;SinceEPCTandopticalflowmethodhavebeendis

    ;cussedinprevioussubsections,nowletuscometotheim

    ;provedversionofopticalflowmethod.First.EPCTisused ;togettherotationandtranslationbetweenimages.from ;which90andx0,y0areestimated.Becausethematching ;imagesareofthesamesize,noscalingeffectissupposed. ;Chirpingandkeystoningeffectsarealsounknownatthe ;momentsom6and;7arebothsettobe0.Then.an

    ;

    ;72ACTAAUToMATICASINICAVl01.34

    ;initialpixel--accuratevalueoftheprojectivemodelmatrix ;is

    ;.iCOS00sin00x0l

    ;m=I’sin00cos00y0f(10)

    ;001J

    ;Itiscoarsebutgoodenoughfortheopticalflowmethod ;asaninitialinput,asc.mparedwithm=[],whichmeansthematchingimagesareidentica1.Fr

    omexperi-

    ;mentstheopticalflowmethodissensitivetotheinitialm,

    ;andabetterestimationwillgreatlyimprovetheultimate ;result.SousingtheresultofEPCTinsteadofaniden- ;titymatrixwillsignificantlyenhancetheregistration.An ;overviewofthisimprovedalgorithmisshowninFig.2. ;IEPCTf

    ;,0

    ;.

    ;_

    ;r

    ;ICalculateIitjalm0

    ;|

    ;m0

    ;IOpticalfl.wmeth.dI

    ;ultimate

    ;r

    ;lM.ai.lIJ

    ;Initia1

    ;motion

    ;estimator

    ;Fig.2Overviewoftheregistrationalgorithm

    ;2Implementationandexperimental

    ;results

    ;2.1Implementation

    ;First,inthepreprocessorofEPCTwhere00andx0,y0 ;areestimated,theinitialinputforopticalflowmethodis ;calculatedasin(101.

    ;Second,tomaketheopticalflowmethodmoreeffective,a ;two-layerloopstructureisapplied,asshowninFig.3.Sup- ;posethesizeofimagesbeingregisteredisM×N.Thefirst ;layerisGaussianpyramidsofreferenceandcurrentframes ;fromcoarsetofine(usuallythree-level,withthesizesof ;imagesM.×N,×iN,M×N,respectively),andthesec

    ;ondlayerisrepeatingtheopticalflowcalculatingprocess ;Initialestimation

    ;Layerl:

    ;Gaussian

    ;pyramid

    ;construction

    ;ineachGaussianpyramidlevel,usingtheprojectivemodel ;parametersobtainedlasttimeastheinitialinputfornext ;loop.

    ;Tomakethelooptimesofthesecondlayerself-adaptive, ;tosavetimeandworkload,twocriteriaareappliedasfo1. ;1ows.

;Criterion1.

    ;1oopTimem~>Q

    ;Criterion2.

    ;

    ;O

    ;

    ;M

    ;—— ;S

    ;—— ;E

    ;

    ;( ;t

    ;

    ;h ;i

    ;

    ;s

    ;

    ;_t ;

    ;i

    ;

    ;m

    ;—— ;e

    ;

    ;) ;-

    ;

    ;O—— ;M

    ;—— ;S ;E

    ;. ;( ;la ;s

    ;

    ;t

    ;

    ;_

;

    ;ti

    ;m

    ;

    )< ;e

    ;0MSE(1ast_time,

    ;forconsecutivetimes

    ;InCriteria1and2,Q,,andareparametersdefined ;byuseraccordingtotheaccuracytheyrequire.Andover- ;lappedmeansquareerror(OMSE)iscalculatedas ;OMSE:

    ;:.

    ;(,)?r

    ;N

    ;(11)

    ;In(11),Fistheoverlappedpartofimages,r(x,Y)and ;c(x,Y)arethepixel-valuesofpoint(,Y)inmosaicofthe ;referenceandcurrentframe,respectively,andthereareto

    ;tallyNpointsexistingintheoverlappedpart. ;EitherCriterion1or2willmaketheloopinthesec

    ;ondlayerstopandturntonextGaussianpyramidlevelin ;layerone.Comparedwithfixedloop,therearetwoadvan

    ;tagesforusingthesetwocriteriastomakeloopstopping ;decisions:

    ;11Differentimagesmighthavediversemotionestimation ;convergencespeeds.Thus,self-adaptabilitywillenhance ;thecalculationefficiency.

    ;2)ThroughmakingadjustmentstoQ,,and,itiseasy ;tosetdifferentregistrationaccuracy.Tosomeextent,by ;enlargingQand,ordecreasing,registrationaccuracy ;willbeboosted.

    ;2.2Experimentalresults

    ;Theperformanceofthisimprovedopticalflowmethod ;istestedondifferentsetsofimages.Randomnoisesare ;alsoaddedtotestthealgorithmsrobustness.Twoexper- ;imentalresultsareshown.Fig.4isonthesimulateddata, ;whereasFig.5isontherealdata.

    ;Fig.3Two-layerloopstructureofopticalflowcalculation ;Layer2:

    ;optical

    ;flow

    ;calculation

    ;

    ;No.7XIONGJing-Yieta1.:AnImprovedOpticalFlowMethodforImagesRegistration76

3

    ;(a)

    ;(b)(C)

    ;F.4Grids((a)Thetworectanglesmarkedarethereference ;andcurrentimages,respactively;(b)Themosaicresultof ;opticalflowmethod;(C)Themosaicresultofimprovedoptical ;flowmethod)

    ;Theregistrationdifferencebetweenopticalflowmethod ;andtheimprovedversionisshowninFig.4.InFig.4(a), ;anartificialmovementof100pixelsismadebothvertically ;andhorizontallybetweenthereferenceandcurrentimages ;markedbytherectangles.Itiseasilyseenthatopticalflow ;registrationofFig.4(b)isanerror,andtheimprovedop

    ;ticalflowofFig.4fC)ismuchbetter.Actually,themotion ;estimationfromFig.4(C)is100.0224pixelsinbothdirec

    ;tioas,whichisanexcellentestimation.

    ;InFig.5,(C)fromtraditionalopticalflowmethodis ;clearlyafailure.And(d1fromEPCTisproperlyright, ;yetsomeslightmismatchesexist,markedbyablackrect

    ;angle,andmoredetailsCanbecheckedinanexpanded ;versionfe).Thebestmosaicisff)withoutanyperceptible ;mismatch,fordetailsseefg1.Makeacarefulcomparison ;between(e)and(g),itiseasytofindthat(g),theregistra- ;tionresultofimprovedopticalflow,ismuchbetterthan(e) ;fromopticalflow.In(h),somerandomnoiseisaddedto ;thecurrentandreferenceframes,andtheimprovedmethod ;worksprettywellaswehavepreviouslyanalyzed. ;SomequantitativeevalUationsareshowninles1and ;2.

    ;Table1OMSEofregistrationinFig.5

    ;OMSEiscalculatedas(11),andthevalueofOMSE ;measurestheoveralldifferenceofoverlappedpartsofref- ;erenceandcurrentflamesintheirmosaic.Thesmallerthe ;values,thebettertheregistrationresults,andviceversa. ;Table2TimeconsumingofregistrationinFig.5 ;(a)(b)

    ;(c)

    ;(d)

    ;(f)

    ;(e)

    ;(g)

    ;(h)

    ;Fig.5Snow(a)Thecurrentframe;(b)Thereferenceframe: ;(C)Resultofopticalflowmethod;(d)ResultofEPCT;(e)

    ;Expandedversionofthepartmarkedbyablackrectanglein ;(d);(f)Resultofimprovedopticalflow;(g)Expandedversion ;ofthepartmarkedbyablackrectanglein(f);(h)Resultof ;improvedopticalflowaddedwithsaltandpeppernoise) ;

    ;764ACTAAUTOMATICASINICAVl01.34

    ;Infixedlooptimescategory,thetimeisfixedto10, ;whereasinserf-adaptivelooptimes,=10,:0.1,and ;=2.Registrationresultsareidenticalinthesetwositua- ;tions,buttimeconsumingofthelatterisnearlyhalfofthe ;former.

    ;Table3TimeconsumingdistributionindifferentpartinFig.5 ;(Notethatself-adaptiveopticalflowisabbreviatedasSAOF.) ;Comprehensivelyspeaking,fromexperimentresults ;listedaboveandsomeothertestswhichhavenotbeen ;listedinthispaper.itisimpliedthattheimprovedopti- ;calflowmethodusingEPCTasapreprocessordoesrefine ;themosaiasinsomecases,comparedwithEPCTortradi. ;tionalopticalflowmethod,especially,forthosewithlarge- ;scalemovements.Moreover,thetwo-layerself-adaptive ;loopstructurewillsignificantlyreducecalculationtimeand ;thus,enhanceefficiency.

    ;Somerandomnoisesareaddedtoimagestocheckifthe ;improvedalgorithmkeepsworkingwel1.Anditdoes.The ;reasonforthismerithasbeenexplainedintheintroduction. ;3Conclusion

    ;Inthiswork,animprovedopticalflowmethod,using ;EPCTasapreprocessorandatwo-layerstructureinthe ;opticalflowcalculation,ispresentedtodealwithimagereg- ;istration,especially,forthosewithlarge-scalemovements. ;Thisimprovedalgorithmhasthemeritsofbothmethods. ;Therearetwostepsinthecalculatingprocess.First,EPCT ;isusedtohandlelarge.scalemovementstogetacoarseesti- ;mationofprojectivemodelparameters.Afterthat,optical ;flowisusedtorefinetheresuItfromthepreprocessor,from ;pixeltosubpixelaccuracy.

    ;Experimentsshowthatitiseffectivetothoseimages ;withlarge-scalemovements,whichcannotbehandledby ;traditionalopticalflow,andtoarobustalgorithmagainst ;randomnoises.However,itisalsofoundthatforsome ;imagescontainingtoolarge-scalemovementsorfewclear ;edgesinthecontents,thisimprovedmethodreachesapoor ;result(althoughthesalneforEPCToropticalflow).Some ;investigationshouldbedonetoimprovethealgorithmto

Report this document

For any questions or suggestions please email
cust-service@docsford.com