Optimal
ChineseJournalofChemicalEngineering,16(2)235—240(2008)
;OptimalIterativeLearningControlforBatchProcessesBasedon ;LinearTime.varyingPerturbationModel
;xIoNGZhihua(熊智华),zHANGJieandDONGJin(董进)
;DepartmentofAutomation,TsinghuaUniversity,Beijing100084,China
;ScIio01ofChemicalEngineering—andAdvancedMaterials,UniversityofNewcastle,Ne
wcastleuponTyne,NE17RU,
;SupplyChainManagement&Logistics,IBMChinaResearchLab,Beijing100094,C
hina
;1INTRoDUCTIoN
;Batchprocessesaresuitableforthemanufactur. ;ingofhighvalueaddedproducts,suchas,special ;polymers,specialchemicals,andpharmaceuticals【1】.
;Therepetitivenatureofbatchprocessoperationsa1. ;1owsthattheinformationofpreviousbatchrunscan ;beusedtoimprovetheoperationofthenextbatch.Of ;lateiterativelearningcontrol(ILC1hasbeenusedin ;thebatch..to..batchcontrolofbatchprocessestodi.. ;rectlyupdatetheinputtrajectory【1,2】.Refinementof
;controlsignalsbasedonILCcansignificantlyenhance ;theperformanceoftrackingcontrolsystems. ;Bristoweta1.【3】havepresentedasurveyofthe
;majorresultsbasedonlinearmodelsintheILCanalY. ;sisanddesignoverthepasttwodecades.AstheILC ;iswelldevelopedinlinearmodels,mostofthe ;ILC.based.batch.to.batchcontrolschemesarebased ;onsomekindoflinearmodel,forexample,thelinear ;time—invariantsystem[41.Nonlinearmodel—basedILC
;schemeshavealsorecentlybeenproposed.Xuand ;Hou【5】havereviewedtherecentadvancesinILC, ;basedontheLyapunovmethodsofthenonlinearsys. ;tems.OptimalILCisoneoftheimportantmethodsfor ;designinganiterativelearninglaw.inwhichtheILC ;lawisderivedfromaquadraticobjectivefunction【3】.
;Owenseta1.【6,71haveproposedanoptimalILC,
;basedontheoptimizationprinciple,bycombiningthe ;RiccatifeedbackcontrolandthetypicalILC ;feed—forwardcontro1.Gaoeta1.【8.9】havestudiedthe
;optimalILCfurtherforaninjectionmoldprocessand
;thenproposedageneraldesignframeworkfortheILC ;ofabatchprocessbasedonatwo.dimensionalf2D1 ;system.Leeeta1.inseveralrelatedarticles【2,10—14】
;haveproposedthequadraticcriterion.basedILC ;(O.ILC)approachfortrackingthecontrolfortem. ;peratureofbatchprocesses,basedonalinear ;time.varying(LTV)trackingerrortransitionmode1. ;Leeeta1.【l1】combinetheadvantagesofILCand
;modelpredictivecontrolfMPC)intoasingleframe. ;work.AbatchMPCfBMPC)techniqueanditsexten. ;sionfortrackingcontrolareproposedbyincorporat? ;ingthecapabilityofrea1.timefeedbackcontrolinto ;thequadraticcriterion.basedILCfQ.ILC).Thepro. ;posedapproachisalsoappliedtothetrackingcontrol ;fortemperatureofbatchprocesses.
;ILCcanupdatethecontroltrajectoryforthenext ;batchrunbyusingtheinformationfromtheprevious
;batchrunssothattheoutputtrajectoryconvergesas—
;ymptoticallytothedesiredreferencetrajectory. ;Therefore,theconvergenceofaniterativelearning ;lawisanimportantissueinthedesignandapplication ;OfILC.Intheauthors’previousstudies【15,161,an
;ILCstrategyforthetrackingcontrolofproductquality ;inbatchprocesseswasproposedbasedonanLTVP ;mode1.Inpractice.thePmodelofproductquality ;canbeobtainedbylinearizingthenonlinearmode1. ;withrespecttothenominaltrajectories.Furthermore,to ;addresstheproblemofmode1.plantmismatches.the ;modelpredictionerrorsinthepreviousbatchrunare ;addeddirectlytothemodelpredictionsinthecurrent ;batchrun.Onthebasisofseveralsimulationresults,it ;showsthatthetrackingerrorconvergesnominallyas ;thebatchnumbertendstoinfinite.Inthisarticle,the ;authorspresentarigoroustheoremtoverifythat,when ;thereisnomodelingerror,aperfecttrackingperform—
;ancecanbeobtained,inthesensethatboththetracking ;errorsandthechangesofthecontrolpolicyconvergeto ;zeroasthebatchindexnumbertendstoinfinite. ;2BATCH.To.BATCHITERATIVELEARN.
;INGCoNTRoL
;Thebatch?wiseLTVPmodel—basedILCdeveloped
;Received2006—11—30.accepted200710—28.
;SupportedbytheNationalNaturalScienceFoundationofChina(60404012,60674064),U
KEPSRC(GR/N13319and
;GR一
/R10875):theNationalHighTechnologyResearchandDevelopmentProgramofChina(200
7AA04Z193),NewStarof
;ScienceandTechnologyofBeijingCity(2006A62).andIBMChinaResearchLab2007UR
—Program.
mail:zhxiong@tsinghua.edu.cn ;T0whomcorrespondenceshouldbeaddressed.E—
;
;Chin.J.Chem.Eng.,Vo1.16,No.2,April2008 ;byXiongandZhang【15】isreviewedinthissection.
;Batchprocesseswherethebatchrunlength(t3isfixed ;andconsistsofNsamplingintervalsareconsidered, ;whereallbatchesrunfromthesameinitialconditions. ;Theproblemofthebatch.to..batchcontrolistoma.. ;nipulatethewholecontrolprofilesothattheproduct ;qualityvariablesfollowthespecificdesiredreference ;trajectories.Itwouldbeconvenienttoconsidera ;batch—wisestaticfunctionrelatingthecontrolprofile ;tOtheproductqualityprofileoverthewholebatch ;duration.Itcanbewritteninmatrixformas ;=
;F(Yo,)
;wherethesubscriptkdenotesthebatchindex, ;:
;[(1),(2),…,(?)]T(?n,,z?1),isthe
;productqualityvariableandcanbeobtainedonlineor ;offline,Y0istheinitialvalue,=【瓦(1),瓦(2),…,
;瓦(N一1)I(?R,m:1inthisstudy)isthema—
;nipulatedvariable.andF(?)representsthenonlinear ;staticfunctions.respectively.Subtractingthetime—
;varyingnominaltrajectoriesfromtheprocessopera—
;tiontrajectoriesremovesamajorityoftheprocess ;nonlinearityandallowslinearmodelingmethodsto ;performwellontheresultingperturbationvariables【17].
;2.1LTVPmodel
;Bylinearizingthenonlinearbatchprocessmodel ;describedbyEq.(1)withrespecttocontrolsequence ;aroundthenominaltrajectories,thefollowingcanbe ;obtained
;=rs+
;(Yo,)
;aE,
;(一)+d
;whereUisthenominalcontroltrajectory,risthe ;nominalproductqualitytrajectory,and(0)=Y0,
;anddkisasequenceofmodelerrorsbecauseoflin—
;earization(i.e.,becauseofneglectingthehigheroder ;terms).respectively.Thenthebatch—wiseLTVPmodel
;canbeobtainedas
;=GU+d(3)
;whereG:l(v)/~vJ,uu—u,
;and=—rsaredefinedasperturbationvariables ;ofcontrolandproductqualityvariables,andy(0)=0, ;respectively.Gsisbatch—wiselineartime—varying,in
;thesensethatitvarieswithU,whichusuallyvaries ;frombatchtobatch.Onaccountofthecausality(1.e., ;theproductqualityvariablesattimetareonlyfunc—
;tionsofallcontrolactionsuptotimef),thestructureof ;Gisrestrictedtothefollowinglower-block—triangular
;form:
;G=
;g100
;g2og21
;0
;0
;gNOgN1’..gNN一1
;(4)
;whereg?R.ThebatchwiseLTVPmodelGscan
;U
;befoundbylinearizinganonlinearmodelalongthe ;nominaltrajectoriesorthroughdirectidentification ;fromtheprocessoperationaldata.Availablemethods ;foridentifyingGrangefromsimplestaticlinearre—
;gression,suchas,theleastsquaresanditsvariants ;(e.g.,partialleastsquares,PLS)[15,18],tomore ;elaborateoptimaldynamicestimationmethodssuchas ;theextendedKalmanfiltering(EKF1【2,12】.
;2.2Optimaliterativelearningcontrol
;Themodelpredictioninthekthbatchrunisob—
;tainedas
;=
;GU(5)
;whereGistheestimatedLTVPmodelofG.The ;modelpredictionerrorisdefinedas=—.In
;thisstudy,thepredictionerroroftheperturbation ;modelisassumedtobeboundbyacertainsmallposi—
;tiveconstantB,’l,thatis,II<.Theprediction
;errorboundBisameasuretorepresentthedeviation ;offrom.ThehigherthevalueofB,’lis,the
;poorertheidentifiedmodelis.Aftercompletionofthe ;kthbatchrun,theproductqualitiesaremeasuredor ;analyzedofnineandthepredictionerrorcanbecalcu—
;lated.
;Toaddresstheproblemofmode1..plantmis.. ;matchesfortheLTVPmode1.theauthorshaveutilized ;themodelerrorsoftheimmediatepreviousbatchrun, ;tomodifythemodelpredictions.Basedonthepredic—
;tionerrorsofthekthbatchrun.themodifiedpredic ;tionoftheperturbationmodelinthe(k+1)thbatchrun ;isobtainedas
;=+l+
;Correspondingly,themodifiedpredictionerrorisdefined ;as反+1=+l—+l,anditholds+1=+1一.
;ConsideringthattheobjectiveofILCistotrack ;thedesiredreferencetrajectoriesofproductquality, ;thetrackingerrorsoftheprocess,model,andmodified ;modelpredictionaredefinedrespectivelyas ;e=—,=一,ek:—(7)
;where=—,andarethespecifiedrefer—
;encetrajectoriesandareassumedtobesetreasonably ;here.
;Fromallthesedefinitions,therelationships ;amongthesethreetrackingerrorsare
;
;Chin.J.Chem.Eng.,Vo1.16,No.2,April2008 ;ee一
;ee一一
;1
;fromwhichthefollowingiterativerelationships ;thebatchindexkcanbeobtained[15】:
;+=
;(！)一(+一)
;=一
;G(+,一u)
;+l=+(+l一)一(一一1)
;=一
;G(+一u)一
;e+l:e一(u+1一u)一+l
;ek+1:+l！(一e)
;=e—G(+一U)
;(8)
;along
;(10)
;(11)
;(12)
;Giventheabove..mentionedbatch..wiseerror ;transitionmode1.theobjectiveoftheILCistodesign ;alearningalgorithmtomanipulatethecontrolpolicy ;sothattheproductqualitiesfollowthespecificdesired ;referencetrajectoriesfrombatchtobatch.Bythe
equivalenceprinciple[101,theauthorscon— ;certainty—
;sidersolvingthefollowingoptimalquadraticobjec—
;tivefunction,basedonthetrackingerrorofthemodi. ;fiedmodelpredictionuponthecompletionofthekth ;batchrun.toupdatethecontroltrajectoryforthe ;(k+1)thbatchrun
;J(U)=+唁++一
;=1Q+l+(+一uk)R(U+一U)
;(13)
;whereDandgarepositivedefinitivematrices.The ;weightingmatricesQandgaffectthecontributionsof ;thetrackingerrorsandcontrolchangesinEq.f131and ;mustbeselectedcarefully.Arelativelylargeweight,g, ;onthecontrolchange.willleadtomoreconservative ;adjustmentsofcontrolactionsandtoaslowercon. ;vergence.Arelativelysmallgwillleadtolargead. ;justmentsofcontrolactions,butthiscanleadtoun—
;convergence,asalargechangeincontrolactionsmay ;resultinthelinearizedmodelbeinginvalid.Qandg ;areusuallyselectedasdiagonalmatricesanditisalso ;possibletoweighttrackingerrorsandcontrolactions ;atdifferenttimestagesdifferently.IfQandgarese. ;1ectedasdiagonalmatrices.itcanbeobservedinEq. ;f131matthecontrolperformanceactuallydependson ;therelativevalueoftheweightingmatrices.Forthe ;sakeofsimplicity,andRareselectedinthisstudy ;asQ=I?andg=-I?.
;FromEq.f121andthroughstraightforwardma ;nipulation,thefollowingILClawisobtained[15】
;U+1:U+Ke
;whereKisthelearningrate
;K=IGQG+glGQ(15)l
—J
;ItcanbeobservedintheILClawstatedearlier ;thatthechangeofcontroltrajectory(U+l—U)is
;directlyupdatedbytheactualtrackingerroreofthe ;process,ratherthanthetrackingerrorekofthemodi-
;fledmodelprediction.
;3CoNVERGENCEoFTRACKINGERROR
;Inthepreviousstudies[15,16],theILClawstated ;earlierhasbeenpresentedandsimulationstudieshave ;alsoshownthatthetrackingerrore}nominallycon—
;vergesaccordingtothebatchindexk.HoweveLthe ;convergenceoftheILClawstatedherehasnotbeen
;analyzedtheoretically.Inthisarticle,arigoroustheo—
;remhasbeenusedtoprovethatthetrackingerrore}, ;undertheoptimalILClaw,Eq.(14),canconvergeto ;zero.
;TheoremLet{e}}bethetrackingerrorse—
;quencebasedontheLTVPmodelundertheoptimal ;ILClaw,Eq.(14).Ifthereisnomodelingerror(.e., ;G=G)andthekernelofGiszero,then
;lime=0(16)
;ProofAsU+11stheoptimalcontroltrajectory
;oftheoptimizationproblemEq.(13),thefollowing ;inequalityistrue:
;J(U川)?J(Uk+1)
;=
;唁+】】UkUk眩
;:
;I1一G(u+一u)一l巴+Iv:+一ukl1
;(17)
;where+lisanycontroltrajectoryofthe(k+1)th ;batchrnn.Becausethereisnomodelingerror,thatis, ;G=G,itfollowsthat
;专.厂0=(一e)一(一1一e一1)=
;Andif:+1ischosentobeUt,thenitholds ;J(U+)?(+:):『1唁(19)
;AccordingtoEq.(13),italsofollowsthat ;J(V+)=f+唁+llv+一uk喉?『1+唁(20)
;FromEq.(19)andEq.(20),thefollowingcanbeob—
;tained
;唁?J(V川)?
;Thisimpliesthatthelimitofimexists.Conse—
;—
;‘oo
;quently,.1imJ(U川)alsoexistsandequalsto
;lim},thatis,
;l
;—
;im
;..
;liell=l—
;im
;..
;J(U+)(22)
;
;Chin.J.Chem.Eng.,Vo1.16,No.2,April2008 ;thenaccordingtoEq.(13),itfollowsthat
[.,()一. ;恕+厂熙
;牌J(+?)一牌=o(23)七—o.七—o.”
;AsRisapositivedefinitivematrix,Eq. ;(23)resultsin
;.1im(U一U)=0(24)
;AccordingtoEq.(11),Eq.(14),andEq.(18)(.e., ;=
;0),itwasfoundthat
;川卜)(25)
;=
;【,一Kd)(v一u)
;Becausethefollowingequationholds
=,！Qd+RQd ;,一
;f261
;=
;[GQd+R]R
;Eq.(25)canberewrittenas
;(U川一U):RTQd+R]Ke
;=
;RGQek+l(27)
;FromEq.(24)andEq.(27),itfollowsthat ;.
;1imRG川=0(28)
;BecausethekernelofmodelGiszeroandRandQ ;arepositivedefinitematricesintheobjectivefunction,
;Eq.(28)means
;lime=0(29)
;Thiscompletestheproofofthetheorem. ;Theoretically,thistheoremprovesthatwhenthere ;isnomodelingerror,perfecttrackingwillbeobtained, ;thatise0and(U?一U)0askoo.
;However,forproductqualitytrackingcontrolin ;anonlinearbatchprocess,model—plantmismatchal—
;waysexistsbetweenanestimatedLTVPmodelandan ;actualprocess.Inpractice.theIVPmodelcanbe ;foundbylinearizinganonlinearmodelalongthe
;nominaltrajectoriesorthroughdirectidentification ;fromtheoperationaldataoftheprocess【16],inwhich
;thelinearizedmodelingerrorcannotbeneglected. ;Therefore,becauseofthemodelingerror,thetracking ;errorwillchangetoe77,askoo,where77isa ;smallpositiveconstant.
;Becausethepredictionerroroftheperturbation ;modelexists(.e.,?0)andisbound(i…e
;}ek}<Sm),themodifiedpredictionerror+1isalso ;boundby2B,asfollows
;l反+.l=lek+.一ekl<l+Il+lekl<2(30)
;SubstitutingtheILCupdatelaw,Eq.(14),toEq.(11), ;itfollowsthat
;1一?()一反+(31)
;=
;[,一GK]e一+1
;Becauseoftheboundederror+l’thetrackingerror
;ekwillconvergeatasmallvalue,thatis, ;.
;1ime:77.
?o. ;K—
;Italsomeansthatthetrackingperformancewillde—
;pendontheaccuracyoftheIVPmode1.
;4SIMULATIoNoNABATCHREACToR
;Thisexampleisatypicalnonlinearbatchreactor, ;withtemperatureasthecontrolvariable【19].Theop—
;erationobjectiveofthereactoristomaximizethe ;product(B)afterafixedreactiontime(fF1.Oh).The ;reactionschemeisA—LB—一?C.andtheequa—
;tionsdescribingthebatchprocessare”
;d
;Q
;Xl
;=-
;k1exp(一E./”T~ef)(32)
;dt=k1exp(一Ell”)--k2exp(一/”Tref)X2
;(33)
;wherex1andx2representthedimensionlessconcen—
;trationsofAandBrespectively,”=T/.fisthedi—
;mensionlesstemperatureofthereactorandTrefisthe ;reference~mPerature.Thepar帅etersaresetat
;k1:4.Oxl()J,k2=6.2xl05,El:2.5x10,E,=5.OxlO, ;andTref:348,respectively.Theinitialconditionsare ;x1(O):1andx2(O)=0,andthereactortemperatureis
;constrainedto298K?r?398K.
;Inchisstudy.themechanisticmodelstatedearlier, ;Eq.(32)andEq.(33),isassumedtobenotavailable. ;Becausetheobjectiveofthereactoristomaximizethe ;product(),anLTVPmodelisbuilttomodeltherela- ;tionshiPbetweenY=x.and.Thedesiredproduct ;referencetrajectoryhasbeenselectedfromthe
.TWelvebatchesofprocessoperations, ;literature【20】
;underdifferenttemperatureprofiles.weresimulated ;fromthemechanisticmodelandusedashistorical ;processdatasets.These12temperatureprofileswere ;generatedrandomlybydeviatingfromthenominal ;temperaturetrajectory,U.Subsequently,basedon ;thesehistoricalprocessdatasetsandtheselectedU ;and,theparametersoftheLTVPmodelGwere ;identifiedusingtheleast-squareregressionmethod【151.
;HerethebatchlengthisdividedintoNequal ;stagesandtwovaluesofNarestudied.N=10and ;N=5.Asmentionedearlier,theweightingmatrices ;QandRintheILClawaresetasQ=,and
;R=?,…Thusthecontrolperformanceactually
;dependsontherelativevalue2roftheweightingma—
;tricesQandR.Alarger2rontheinputchangewill ;leadtomoreconservativeadjustmentsandslower ;convergence.Severalvaluesof2rarestudiedand ;comparedintermsofthetrackingperformance,such ;as,rootmeanofsquareerrors(RMSE),andfinally,Q ;andRareselectedasQ=,andR=0.011
;ResultsofthetrackingperformanceofILCunder ;twodifferentschemesareshowninFig.1,inwhich ;theRMSEofthetrackingerrorofproductqualitye}’
;atdifferenttimestagesareshown.Itcanbeseenfterabouteightbatchruns.AlthoughtI1epa-
;rameterstobeestimatedwhenN:10aremorethan ;whenN=5.them?