Previewing Photosynth at Siggraph in Boston

Today at Siggraph conference in Boston, Microsoft Live Labs offered a sneak preview of Photosynth, one of those technologies that makes you stare at the screen for a while and wonder whether you’re looking at the future or the opening scene of a particularly nerdy dystopian film.

The basic idea is simple enough, at least until the mathematics arrive and start breaking furniture. Feed the system a large collection of digital photographs and it analyzes them for distinctive features, figures out which images overlap, calculates their positions in three-dimensional space, and assembles the whole mess into a navigable 3D model.

Microsoft naturally isn’t saying exactly how the magic works. Trade secrets and all that. My guess is that they’re using something along the lines of SIFT (Scale Invariant Feature Transform) to identify features regardless of orientation, then constructing a point cloud from the camera rays sweeping through the scene.

Attendees at Siggraph conference in New Orleans 2000.

The demos look suspiciously nice. The obvious catch is that you probably need a hell of a lot of photographs before the result stops looking like an archaeological reconstruction assembled by a drunken committee. Things become even more entertaining if Photosynth eventually turns into a public service where anyone can upload images. Unless there’s some kind of moderation, humanity will do what humanity always does when given a powerful new technology: immediately upload something stupid.

One picture is taken at noon. Another at midnight. One is overexposed. Another was shot through a dirty bus window. Half the façade has been renovated since Tuesday. And inevitably there’s a tourist standing directly in front of the interesting bit, grinning triumphantly as though personally responsible for the construction of Notre-Dame. How well does the algorithm cope with that?

Give us a system capable of reconstructing the physical world from millions of photographs and somebody will feed it forty-seven pictures of his cat.

What will happen if all photos of the world are inserted in Photosynth and then linked with Google Earth? At that point you’re no longer browsing photographs. You’re assembling a crude digital copy of the planet. Which, depending on your disposition, is either enormously exciting or a perfectly reasonable moment to start worrying.