DOC

Network-aware perceptual error concealment method for H264 video with side information

By Norma Owens,2014-02-18 03:07
8 views 0
Network-aware perceptual error concealment method for H264 video with side informationH,for,H264,video,with,side,error,Error,Video,h264

    Network-aware perceptual error

    concealment method for H.264 video with

    side information

    J.Cent.SouthUniv.Techno1.(2010)17:816-823

    D01:10.1O07/sl1771010-0561x

    Networkawareperceptualerrorconcealmentmethodfor

    H.264videowithsideinformation

    MAHan-jie(马汉杰),CHENYao-wu(陈耀武)

    Springer

    InstituteofAdvancedDigitalTechnologiesandInstrumentation,ZhejiangUniversity,Hangzhou310027,China

    ?CentralSouthUniversityPressandSpringerVerlagBerlinHeidelberg2010

    Abstract:InordertoimprovethevideoqualityoftransmissionwithdatalOSS.aspatia1andtempora1errorconcealmentmethodwas

    proposed,whichconsideredboththestateinformationofthenetworkandtheperceptualweightofthevideocontent.Theproposed

    methoddynamicallychangedthereliabilityweightoftheneighboringmacroblock.whichwasusedtoconcea1thelostmacroblocks

    accordingtothepacketlOSSrateofthecurrentchanne1state.Theperceptualweightmapwasutilizedassideinformationtodo

    weightedpixelinterpolationandside

    matchbasedmotioncompensationforspatialandtemporalerrorconcealment. respectively.

    Andtheperceptualweightoftheneighboringmacroblockswasadaptivelymodifiedaccordingtotheperceptualweightofthe1ost

    macroblocks.ComparedwiththemethodusedinH.264jointmodel,experimentresultsshow

thattheproposedmethodperforms

    wel1bothinsubjectivevideoqualityandobjectivevideoquality,andincreasestheaveragepe

    aksigna1.to.noiseratio(PSNR)ofthe

    wholeframebyabout0.4dBwhenthevideobitstreamsaretransmittedwithpacketslOSS.

    Keywords:errorconcealment;reliabilityweight;perceptualweightmap;weightedpixelinte

    rpolation;motioncompensation;side

    information;H.264

    1IntrOductiOn

    Theapplicationsofvideocommunications,

    especiallyoverthewirelesschannels,arerapidly increasingandinthemeanwhiletherequirementsfor goodvideoqualitytransmissionarealsogrowing.The diculty1iesinthatthedatapacketsmaybelostor damagedduetoeithertraccongestioninthecurrent

    Internetorbiterrorsinthewirelessnetworkscausedby impairmentsofphysicalchannels[1].Moreover,theuse ofpredictivecodingandvariablelengthcodinginvideo compressionmakesthetransmittingdatamoresensitive todataerrors.Inordertosolvethisproblem,many techniqueswereproposed,suchasforwarderror correcting(FEC),automaticrepeatrequest(ARQ),error resilientencodinganderrorconcealment.

    Errorconcealmentisalwaysappliedatthedecoder sideandcarrieslessoverheadbandwidthandhasno delay.Itrecoversthe1ostinformationbyutilizingthe smoothingpropertyofimageorvideosignalsandthe spatial/temporalinterpolation[23].ZENGandLIU[4

    proposedaspatialerrorconcealmentmethodbasedon geometricstructure,CHENetal[5proposedanerror

    concealmentmethodusingacontext.basedmodel,and

    YEeta161presentedacontent.basederrorconcealment methodbyblockclassification.Inordertoimprovethe performanceoferrorconcealment,errorresilienttools, suchasscalablecoding[7],multidescriptioncoding8,

    referencepictureselectionf9]andredundantpictures [10],werecombinedinsomeotherapplications. Sideinformationisalsoaveryusefultoolto improvetheerrorconcealment.Itcanbeembeddedinto thevideoinformationwithoutmuchmoreoverhcadand delay[11-13],andcanalsobecombinedwithothererror resilienttools14].

    ThelatestvideocompressionstandardH.264 performswellinthecompressionratio.Althoughthere aremanyerrorresilienttools.itstil1needserror concealmentatthedecoderwhentherearesomedata losspresented[15].Inordertoimprovethevideoquality ofmislatestvideocompressionstandard.manyvideo concealmenttoolswerealsoproposed.WANGeta116

    proposedanerrorconcealmentmethodbasedonpixel interpolationandmotionestimationwhichwas employedintheH.264jointmode1.XUandZH0U[17] refinedtheerrorconcealmentusingsubblocksand

    directionaledges.BUetal[18proposedanerror

    concealmentmethodwithsignificantmacroblock determinationandprotection.CHUNGeta1191

    proposedanerrorconcealmentfortheintra.framewith bit.errordetectionanddirectionalconcealment. Foundationitem:Project(2006C11200)supportedbytheScienceandTechnologyProjectof

    ZhejiangProvinceofChina

Receiveddate:20091215:Accepteddate:2010-0401

    c0rresp0ndingauthor:CHENYao?wu,PhD,Professor;Tel:+8613605813934;E

    mail:cyw(mail.bme.zju.edu.cn

    J.Cent.SouthUniv.Techno1.(20101l7:816-823 Theabovemethodsdidnottakethecurrentchannel stateinformationintoconsideration.Inotherwords,they didnotconsiderthereliabilityofthereconstructed macroblockstoconcealthelostdata.Inthiswork.a network.awareperceptualerrorconcealmentmethod withsideinformationwasproposed.Dependingonthe currentchannelstateinformation.theproposedmethod dynamicallychangedthereliabilityweightofthe reconstructedmacroblocksaccordingtothepacketloss rate.Thecontentoftheencodedvideoflamewasalso takenintoconsiderationandtheperceptualweightmap ofthevideoframewasutilizedasthesideinformation. Basedonthesetwofactors,theweightinterpolationand side.matchbasedmotioncompensationwasusedto performspatialandtemporalerrorconcealmentinorder toreconstructthelostvideodata.respectively. 817

    2Proposedspatial

    concealment

    andtemporalerror

    Thegeneralflowdiagramoftheproposedspatial andtemporalerrorconcealmentisshowninFig.1. Assumingthatthesideinformationcanbeobtainedfrom theotherprioritychannels[11],embeddingformationin thediscretecosinetransform(DCT)coefficients[12],or fromtheuserdefineddatainthebit.strcamssuchasthe

    supplementalenhancementinformation(SEI)messages intheH.264standard20].Thechannelstateinformation, suchaspacketlOSSrate.canalsobeobtainedfromthe physicallayerordynamicallymeasuredthroughthe realtimecontrolprotocol(RTCP).WhendatalOSS presentsinthevideobitstream.thevideodecoderwil1 detectthebiterrorsintheparsingbitstreamstep.Then, thespatialandtempora1errorconcealmentsareutilized usingboththesideinformationandthechannelstate informationtoconcealthelostdataandimprovethe videoqualityofthereconstructedpicturesinthedecoder buffer.

    Receiver

    Side

    information

    ==============

    Channelstate

    information

    Video

    decoder

    Decoder

    buffler

    Spatialand

    temporalelror

    concealment

    Fig.1Generalflowdiagramofproposedspatialandtemporal errorconcealment

    2.1Reliabilityweightofreconstructedmaeroblacks WhendataIOSSoccursinabitstreamslice.therest macroblocksoftheslicewil1alsobedamageddueto

spatialpredictionandvariablelengthcoding.Inaddition

    thedata1OSSwillaffcctthemacroblocksinsubsequent frameswhentemporalpredictionisused.Theabove spatialandtemporalerrorpropagationsareshownin Fig.2.

    Fig.2Diagramofdatalosspropagation

    pagation

    Toreducetheimpactofdata1OSSonthevisua1 information,smoothnessbasedspatialandtemporalerror concealmentsareemployedbasedonpixelinterpolation andmotioncompensation[21].Theseerrorconcealment algorithmsutilizethereconstructedneighboring macroblocksinthecurrentflameortherefeFenceframe, includingcorrectlyreceivedandconcealedmacroblocks, tobuildthecurrentlostordamagedmacroblockshown inFig.3.

    RRRRRRRR

    RRRRRRCC

    CCC一?LLLL

    LLLLLL}RR

    RRRRRRRR

    RRRRRRRR

    RRRRRRRR

    Fig.3Macroblockstatusmapwithdataloss

    Received

    Concealed

    Lost

    Currentlost

    ewitherrors

    Butnotalltheneighboringmacroblocksshouldbe

    treatedwiththesamereliabilltyweightwiththedatalOSS. Whenamacroblockdatumiscorrectlyreceived.itcan bereconstructedcorrectlyandshouldbetotallyreliable. Butwhenamacroblockdatumislost.itwillbe

    ''guessed''usingthereconstructedneighboring macroblocks.Then.thedistortionbroughtinforthe "guess"isusuallynotaccurateenough.Hence.the concealedmacroblocksshouldnotbetreatedwiththe samereliabilitvasthecorrectlyreceivedmacroblocks beingtreated.Moreover,iftheconcealedmacroblocks areusedtoreconstructthefollowing1ostmacroblocks. thedistortionwillpropagateandthereliabilityofthe subsequentlyreconstructedoneswillalsobereduced. Fromtheaboveanalysis,itcanbeseenthatthe reliabilityofthereconstructedmacroblockshasan inverseratiotothedistortion.Thedistortioncausedby datalosscanbepredictedusingpacketlossprobabilityP f22]andthenthereliabiIityofthereconstructed macroblockscanbemeasuredaccordinglyasshownin 818J.

    Cent.SouthUniv.Techno1.(2010117:816823

    importantfocusinvideocommunicationandthemotion areasarealwaysreceivedmoreattention.Humanface andhandsarethemostattractivepartsofthehuman contentinvideocommunication.Foveaalsoaffects humanperceptualattentionandpeopleusuallyfocuson thecenterregionofthevideoscene.Theperceptual weightmapsofthetestsequencesareshowninFig.5. Eq.(1).Moreover,inthesmoothnessbasederror concealment.thereconstructionprocessofthelostor

    damagedmacroblocksdependsontheneighboring macroblocks.Thus,thereliabilityweightofthe concealedmacroblockshouldbemeasuredaccordingto thereliabilityoftheneighboringmacroblocks.Assuming thatthecorrectlyreceivedmacroblockhasthereliability weightof1,thelostmacroblockwillhavethereliabiljtV weightof0,andthenthemeasurementofthereliability weightoftheconcealedmacroblockcanbedefinedas (f,)=Marg{()}.(1P)(1)

    mE?mb

    wheredenotesthereliabilityweightoftheconcealed macroblock;(f,J)denotesthecoordinationofthecurrent macroblockinthecurrentvideoflame;gdenotesthe averageof,R();^bdenotestheneighbor

    macroblocksandPdenotesthepacketlossprobability. 2.2Sideinformationofperceptualweightmap Whenamacroblockisdamagedbydataloss.

    smoothnessbasederrorconcealmentusingneighboring macroblocksisalwaysemployedtoreconstructthe1ost macroblock161.AsshowninFig.4,thespatialerror concealmentusesdistancebasedweightedpixel interpolation.andthetemporalerrorconcealmentuses side.matchbasedmotioncompensation.Butthese algorithmsdonotconsiderthecontentofthevideoflame, sotheydonotperformwel1inthedetailsofthepicture andinthesubjectivevideoquality.

    -

    J

    'l?

ftIr1tt

    _J

    _

    

    L

    "

    

    

    Metop

    

    riII

    Mvri11I2

    -

    (a)(b)

    Fig.4Spatialandtemporalerrorconcealment(ddenotespixel

    distancesandMvdenotesmotionvectors):(a)Spatial; (b)Temporal

    Toimprovetheperformanceoftheerror

    concealmentatthedecoder,manykindsofside informationarealwaysutilized,suchaserror concealmentmode1f121,importantmacroblockdata[13

    andmotionvectors14].Inthiswork,theperceptual weightmaDOftheencodedvideocontentwasusedasthe sideinformation.Theperceptualweightmapis calculatedattheencodersideusingaperceptual attentionmodelwhichconsidersthethreefactors together:motion.skintoneandfovea.Motionisan (cJ

    Fig.5Perceptualweightmaps:(a)Carphonesequence;(b) Foremansequence;(c)Salesmansequence

    Whentheerrorconcealmentemploysneighboring macroblockstoreconstructthelostmacroblocks.the perceptualweightshouldbetakenintoconsideration. Whentheconcealedmacroblockhasahighperceptual weight,theneighboringmacroblockwithhighperceptual weightshouldbegivenahighreconstructedweightand theonewithlowperceptualweightshouldbegivena 1owreconstructedweightinpixelinterpolationor block-matchcalculation,andviceversa.Letm0d representthemodifiedperceptualweightofthe neighboringmacroblockaccordingtotheperceptual weightofthecurrentconcealedmacroblock,thenitcan bedefinedas

    Wp

    

    rood(m)=exp((2)

    wheremdenotestheindexoftheneighboring macroblock;denotestheperceptualweightofthe neighboringmacroblock;condenotestheperceptual weightofthecurrentconcealedmacroblockand erctIla1denotesthemaximumperceptualweightofthe currentvideoframe.

    J.Cent.SouthUniv.Techno1.(2010)17:816-823819 2.3Spatialandtemporalerrorconcealment Inordertoimprovetheperformanceoftheerror concealmentandreducethecomputationofthe algorithm,someappropriatemacroblocksshouldbe pickedfromalltheneighboringmacroblocksbasedon thereliabilityweight.AccordingtoRef.[16],thepicking orderusingneighboringmacroblockswiththesame

Report this document

For any questions or suggestions please email
cust-service@docsford.com