MatAtlas

Abstract

We present MatAtlas, a method for consistent text-guided 3D model texturing. Following recent progress we leverage a large scale text-to-image generation model (e.g., Stable Diffusion) as a prior to texture a 3D model. We carefully design an RGB texturing pipeline that leverages a grid pattern diffusion, driven by depth and edges. By proposing a multi-step texture refinement process, we significantly improve the quality and 3D consistency of the texturing output. To further address the problem of baked-in lighting, we move beyond RGB colors and pursue assigning parametric materials to the assets. Given the high-quality initial RGB texture, we propose a novel material retrieval method capitalized on Large Language Models (LLM), enabling editabiliy and relightability. We evaluate our method on a wide variety of geometries and show that our method significantly outperform prior arts. We also analyze the role of each component through a detailed ablation study.

a sports shoe, minimalistic, neutral color palette, light and airy, natural materials

a red and silver metallic headphone

a wood kiosk, stylized 3D Art props, handpainted, colorful, simplified, paint stroke

Material assignment results on Source 3D Assets

For each of the following 3D models from the Souce 3D Assets dataset, we show the generated texture, the material part segmentation, and the result obtained by retreiving materials for each part. In the last column, for each material group provided by the asset, we provide a link to the retreived Substance material.

an american footbal

generated texture	segmentation	material assignment

urban boots

generated texture	segmentation	material assignment

a baseball glove

generated texture	segmentation	material assignment

a hiking sport shoes

generated texture	segmentation	material assignment

an interior plant ficus

generated texture	segmentation	material assignment

a beverage carton set

generated texture	segmentation	material assignment

an umbrella closed

generated texture	segmentation	material assignment

a sewing machine

generated texture	segmentation	material assignment

an armchair

generated texture	segmentation	material assignment

a small duffle bag

generated texture	segmentation	material assignment

a notebook

generated texture	segmentation	material assignment

a modern bentwood chair

generated texture	segmentation	material assignment

a cardboard box bag

generated texture	segmentation	material assignment

a high urban bench

generated texture	segmentation	material assignment

Material assignment results on Objaverse models

For each of the following 3D models from the Objaverse dataset, we show the generated texture, the material part segmentation, and the result obtained by retreiving materials for each part.

a bible

generated texture	segmentation	material assignment

a hourglass

generated texture	segmentation	material assignment

Comparison of textured results with state-of-the-art methods on Objaverse

We provide additional comparisons between our texture generation method and baselines.

a lizard

ours	Text2Tex	TEXTure

a cappuccino

ours	Text2Tex	TEXTure

a candle

ours	Text2Tex	TEXTure

a barrel

ours	Text2Tex	TEXTure

a apple

ours	Text2Tex	TEXTure

a binoculars

ours	Text2Tex	TEXTure

a sunflower

ours	Text2Tex	TEXTure

a bookcase

ours	Text2Tex	TEXTure

Relighting assets with different illumination conditions

Our method generates relightable assets that can be lit with different illumination conditions.

a camping tent

an interior ficus plant

More Results

Below we provide more examples of our texture generation results.

a burrito

ours	Text2Tex	TEXTure

a apricot

ours	Text2Tex	TEXTure

a tricycle

ours	Text2Tex	TEXTure

a skateboard

ours	Text2Tex	TEXTure

Acknowledgements

We thank Aaron Hertzmann for sharing his vast expertise on suggestive contours, and sharing his code snipped for the ridge detection filter.