Future 3D scene flow · evaluation
OOD phone videos — examples
Example predictions on out-of-distribution phone videos of human interactions.
MotionForesight does not take in any language/action input, just the observed video context. The text descriptions go as additional inputs to the baselines where specified.
Video comparisons
opening the microwave door
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the desk phone handset
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the marker from the whiteboard tray
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
closing the washing machine lid
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
opening the countertop oven
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
opening the kitchen drawer
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
opening the cabinet door
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
opening the refrigerator door
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the spray bottle
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the box
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
opening the dryer door
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the paper bag
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
opening the door by the round knob
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the coffee cup
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the object from the table
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the backpack
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
rotating the rolled-up umbrella
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
opening the door by the lever handle
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
picking up the shoe
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks
moving the plush toy
Ground truth MotionForesight MolmoMotion (lang) MolmoMotion (no lang) Video-gen (lang): tracks Video-gen (no lang): tracks