Object Representations for Learning and Reasoning

Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS)

December 11, 2020, Virtual Workshop

Join via the livestream 🎥 · RocketChat · @ORLR_Workshop · #ORLR2020 · Join our community Slack!

---

Abstract

Physical construction---the ability to compose objects, subject to physical dynamics, to serve some function---is fundamental to human intelligence. In this talk, I will introduce a suite of challenging physical construction tasks inspired by how children play with blocks, such as matching a target configuration, stacking blocks to connect objects together, and creating shelter-like structures over target objects. I'll discuss a new way of creating object-centric, structured agents using graph representations, and show that such agents both outperform those which use less structured representations and generalize better beyond their training when asked to reason about larger scenes. Overall, I argue that agents which combine structured representations and reasoning with powerful learning are a key path toward agents that possess rich intuitive physics, scene understanding, and planning.