Hacker News
A research prototype, Foresight, investigated whether AI could improve reasoning by generating future video predictions and comparing them to reality, akin to human mental simulation. The project found current models incapable of this but proposes it as a benchmark for future AI development.
AI 生成摘要
名為Foresight的研究原型，旨在探索AI是否能透過生成未來影片預測並與現實比對，來提升其推理能力，類似於人類的心智模擬。該專案發現目前模型尚無法達成此目標，但建議將此作為未來AI發展的基準。
We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Research prototype exploring AI video prediction for improved reasoning
Can AI benefit from "imagining" the future before making decisions?
This research project explored whether AI systems could improve their reasoning by generating video predictions and checking them against reality—like how humans often visualize outcomes before acting.
Result: Not yet. Current models aren't capable of this, but we've documented what works, what doesn't, and propose tracking this as a benchmark for future AI systems.
When you're about to pour coffee, you might briefly imagine the liquid filling the cup. If you imagined it overflowing, you'd pour less. This "mental simulation" helps you make better decisions.
We asked: Can AI do something similar?
The plan was:
If this worked, AI systems could catch their own mistakes by noticing when their predictions look wrong—just like you'd notice if your mental image of pouring coffee showed it going sideways instead of down.
We tried 7 different approaches to make the language model predict what happens next in a video:
All of them performed worse than simply copying the current frame as the "prediction." The language model understands what's in an image, but it cannot predict what will change.
Even when we used a video model to generate predictions (which looked reasonable), comparing them to reality using perceptual metrics (LPIPS) didn't help. Surprisingly, wrong predictions often looked MORE similar to reality than correct ones.
This means you can't use "does it look right?" to catch mistakes—the visual appearance doesn't indicate whether the prediction is semantically correct.
Despite the negative results, we made useful discoveries:
We're releasing this as a benchmark to track when this approach becomes viable. As video models improve, these capabilities may emerge.
Tasks to track:
Why track this? Video generation is improving rapidly. The capabilities we found lacking in 2026 may emerge in future systems. A standardized benchmark helps identify when "visual imagination" becomes useful for AI reasoning.
To reproduce the experiments, you'll need accounts with these services:
The experiments use the Something-Something v2 dataset for action prediction. This must be downloaded manually:
The dataset contains ~220K videos of humans performing 174 different actions (pushing, pulling, dropping, etc.).
If you use this work, please cite:
Research prototype - released for academic use. See paper for full methodology.
Research prototype exploring AI video prediction for improved reasoning
There was an error while loading. Please reload this page.
0
2
There was an error while loading. Please reload this page.