Future 3D scene flow · evaluation

OOD phone videos — examples

Example predictions on out-of-distribution phone videos of human interactions.

MotionForesight does not take in any language/action input, just the observed video context. The text descriptions go as additional inputs to the baselines where specified.

Video comparisons

opening the microwave door
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the desk phone handset
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the marker from the whiteboard tray
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
closing the washing machine lid
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
opening the countertop oven
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
opening the kitchen drawer
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
opening the cabinet door
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
opening the refrigerator door
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the spray bottle
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the box
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
opening the dryer door
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the paper bag
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
opening the door by the round knob
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the coffee cup
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the object from the table
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the backpack
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
rotating the rolled-up umbrella
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
opening the door by the lever handle
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
picking up the shoe
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks
moving the plush toy
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): tracks
Video-gen (no lang): tracks