Nvidia’s new tool can turn 3D scenes into AI images

Nvidia launched a new tool that lets developers generate AI images by first creating them in 3D. The awkwardly named Nvidia AI Blueprint for 3D-guided generative AI is available to download today for computers equipped with an Nvidia RTX 4080 GPU or higher, and works by bridging Blender’s 3D modelling software with Black Forest Lab’s FLUX.1 image generator.

Users can draft a scene in Blender using 3D objects like buildings, plants, animals, and vehicles, which is then used as a reference to create a 2D image. The idea is that manually adjusting the viewing position or where certain objects should be placed will provide more control over finer details when generating 2D images compared to using text descriptions alone.

For example, if you have a very specific image of a city in your mind — the shape and height of the buildings, how many trees or cars are displayed, and even the angle you’re viewing it at — you can use Nvidia’s workflow to manually create a rough version of that in Blender. The FLUX.1 image generator will then use that as a base for the images it creates, which could be less frustrating than repeatedly tweaking text-based descriptions until you get something vaguely similar to what you wanted. The 3D objects plotted out in Blender don’t even need to be highly detailed, as the tool is only using them as a layout guide.

Nvidia describes its blueprints as “pre-defined, customizable AI workflows” that are designed to help developers build generative AI apps. In this case, Nvidia says its AI Blueprint for 3D-guided generative AI provides step-by-step documentation, sample assets, and a preconfigured environment to guide users through using the tool.

The workflow isn’t entirely unique — Adobe teased a similar tool during its MAX event in October called “Project Concept” that also lets users map out 3D scenes that can be used to direct image generation. Adobe’s variant is still in its experimental stages, however, and may never be released to the wider public.