Future 3D scene flow · evaluation

SSv2 validation — examples

Example video predictions on held-out SSv2 clips.

MotionForesight does not take in any language/action input, just the observed video context. The text descriptions go as additional inputs to the baselines where specified.

← MotionForesightSSv2 valOOD phone

Video comparisons

letting duct tape roll down a slanted surface
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
taking a mobile phone out of a box
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
moving color pencils down
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
pretending to throw bulb holder with wire
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
opening door
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
pushing scissor from right to left
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
spinning a disc so it continues spinning
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
throwing a pen in the air and letting it fall
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
moving mobile phone and a box closer to each other
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
pretending to scoop uppuma up with flat ladle with hole
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
spinning a bottle that quickly stops spinning
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
putting a cup upright on the table
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
turning flip flop upside down
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
turning eraser upside down
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
holding a pencil case
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
poking a tire so lightly that it doesn't or almost doesn't move
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
pushing quick installation guide book so that it slightly moves
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
putting pen onto shoe
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
putting trash into trash can
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks
taking cd from printer
Ground truth
MotionForesight
MolmoMotion (lang)
MolmoMotion (no lang)
Video-gen (lang): generated
Video-gen (lang): tracks
Video-gen (no lang): generated
Video-gen (no lang): tracks