Projection mapping is broadcasting video onto a 3d surface in a way that is spacially aware.

In Walkaway, they used this to highlight things in their spaces that need to be done, like dishes out of place. My wife describes it as bringing her editor’s red squiggles into real life. DynamicLand is the closest thing I’ve heard of with regards to replicating this system.

Commercially, there was an interesting tool called Lightform which handled a lot of the computer vision portions of modeling the 3D space. Sadly, they went out of business in Aug 2022. They seemed to use a projector to throw a grid of squares onto a surface, then detecting the distortion of it to build a depth map.

TouchDesigner also has support for projection mapping. This is a tool primarily for artists to build experiences.