New AI technology enables 3D editing of real-life objects

If you ever wanted to make changes to a public sculpture, it might now be possible. Researchers have developed an artificial intelligence (AI) technology that enables people to create a 3D model of an object on their smartphone using 2D photos that can be edited. They will be able to make realistic changes to the model or view the object from any angle.

Researchers at Simon Fraser University (SFU) in Canada have developed the new AI technology through which people will be able to take 3D captures of real-life objects and edit their shapes and appearance as they wish using regular photos.

The study was presented recenely at the Conference on Neural Information Processing Systems (NeurIPS) in New Orleans, US. Researchers unveiled a new technique called Proximity Attention Point Rendering (PAPR) that can turn a set of 2D photos of an object into a cloud of 3D points that represent the object’s shape and appearance, the university’s press statement revealed.

Each point of the model enables users to control the object. They can drag a point to change the object’s shape and in turn, its appearance. The 3D model can also be viewed from any angle. The users can also turn into a realistic 2D photo that shows the edited object.

“The remarkable success of machine learning in areas like computer vision and natural language is inspiring researchers to investigate how traditional 3D graphics pipelines can be re-engineered with the same deep learning-based building blocks that were responsible for the runaway AI success stories of late,” study author Ke Li said in the statement.

According to Li, this showcases the many possibilities for consumer technology. 3D may become as common a medium for visual communication and expression as 2D is today, he added in the statement.

With the rapid expansion of AI, developers across the world are experimenting with its many capabilities. For instance, in July 2023, Stability AI, the company behind Stable Diffusion, one of the popular generative AI tools, rolled out Stable Doodle to turn doodles into high-quality original images. With the rapid expansion of AI, it’s becoming the center of many technological leaps.