OpenAI is developing a text-to-model platform that can generate 3D images in just seconds 

Shap-E could be very important for industries such as interior design and architecture.  

After OpenAI revolutionized graphics creation with DALL-E and input with ChatGPT, OpenAI aims to continue dominating the field of 3D modeling.

In a recent OpenAI article, researchers Heewoo Jun and Alex Nichol described the development of Shap-E, a 3D text model that greatly simplifies the generation of 3D assets. It has the potential to change the status quo in many industries including architecture, interior design and gaming.

_{Selection of 3D samples generated by Shap-E. Photo: OpenAI.}

Though still in the early stages of research and development, Shap-E will allow users to type in a text prompt that creates a 3D model, ones that can potentially be printed. Examples the researchers posted images of included “a traffic cone,” “a chair that looks like a tree,” and “an airplane that looks like a banana.”

At present, producing 3D models requires considerable expertise in industry-specific software programs, such as 3ds Max, Autodesk Maya, and Blender.

“We present Shap-E, a conditional generative model for 3D assets,” Jun and Nichol wrote in the paper Shap-E: Generating Conditional 3D Implicit Functions. “When trained on a large dataset of paired 3D and text data, our resulting models are capable of generating complex and diverse 3D assets in a matter of seconds.”

Shape-E is OpenAI’s second foray into 3D modelling and follows Point-E, the release of which in late 2022 coincided with ChatGPT, which monopolized media and consumer attention. Another reason for Point-E’s somewhat lackluster launch was the haphazard results it produced. While Shap-E’s renderings are yet to reach the quality of industry competitors, its speed is alarming with the open-source software requiring 13 seconds to produce an image from a text prompt.

_{Comparison of Point-E and Shap-E images. Photo: OpenAI.}

In addition to speed, Shap-E’s renderings have softer edges, clearer shadows, are less pixilated than its predecessor, and don’t rely on a reference image. The researchers said it “reaches comparable or better sample qualities despite modeling a higher dimension.”

At present, OpenAI continue to work on Shap-E with researchers noting that the rough results can be smoothed out using other 3D generative programs, though further finessing may require OpenAI working with larger, labeled 3D datasets. For now, 3D model enthusiasts can access files and instructions on Shap-E’s GitHub open-source page.