AIGC与大模型赋能机器人智能控制

AIGC与大模型赋能机器人智能控制 穆尧-香港大学-在读博士 DataFunSummit#2023 目录 CONTENT 01 IntroductionofAIGC 此部分内容作为文字排版占位显示 （建议使用主题字体） IntroductionofEmbodiedAI 03 此部分内容作为文字排版占位显示 （建议使用主题字体） 02 Adaptdiffuser 此部分内容作为文字排版占位显示 （建议使用主题字体） EmbodiedGPT 04 此部分内容作为文字排版占位显示 （建议使用主题字体） 01 AIGC简介 DataFunSummit#2023 VAEMultilayersVAE NormalizingFlow GAN:Needstolearnadiscriminator,trainingprocessunstable Normalizingflow:reversiblefunctionwithlimitedexpressivepower RLApplication:GenerativeAdversarialImitationLearning RLApplication:Flow-basedRecurrentBeliefStateLearningforPOMDPs(ICML2022) HKUMMLAB,TheUniversityofHongKong,HongKong 02 Adaptdiffuser DataFunSummit#2023 Guidancefunctionstransformsanunconditionaltrajectorymodelintoaconditionalpolicyfordiversetasks. LimitedbythedistributionofTrainingofflinedatasetandischallengingtoadaptnewtasks/goalsandnewenvironments TestTrain Solutions: Learnanself-evolvingdiffuserfordiversetasks/goalswithoutrealinteractionwithenvironments Learnametadiffusertogeneralizeacrosstask Differentstartandgoalpoint DifferentGoalofManipulationTasks 03 具身智能简介 DataFunSummit#2023 1.EmbodiedCognitiveSystemsfromaFirst-PersonPerspective2.AchievingHighlyAutonomousDecision-PlanningCapabilities3.AchievingGoal-conditionedPhysicalInteractionwiththeWorld EmbodiedAIreferstotheintegrationofartificialintelligencesystemswithphysicalbodiesorroboticplatforms,allowingthemtoperceiveandinteractwiththephysicalworldinamannersimilartohumans. •1）BuildingaHumanManipulationVideo-TextDatasetEgoCOTwithaMultimodalCognitiveChain,AssociatingVisualInformationwithSub-GoalsinManipulationTasks. Openthedrawer task:openadrawer plans:graspthehandlewiththegripperandpullthehandle actions: 1.grasp(handle,gripper) 1.pull(handle) Pickupthecup task:pickupacuponthetable plans:graspthehandleofthecupwiththegripperandliftitupactions: 1.grasp(handleofthecup,gripper) 2.liftup(cup) 2)IntroducingaVisual-LanguagePretrainingMethodbasedonaMultimodalCognitiveChain. 3)ExtractingHighlyRelevantFeaturesofCurrentVisualObservationsandPlanning'sSpecificSub-GoalsusingSelf-AttentionMechanism,EnablingtheModeltoLearnLow-LevelControlwithMinimalDemonstrationData. DemosofEmbodiedGPT 感谢观看