*equal contribution


arXiv

Abstract

We present MatAtlas, a method for consistent text-guided 3D model texturing. Following recent progress we leverage a large scale text-to-image generation model (e.g., Stable Diffusion) as a prior to texture a 3D model. We carefully design an RGB texturing pipeline that leverages a grid pattern diffusion, driven by depth and edges. By proposing a multi-step texture refinement process, we significantly improve the quality and 3D consistency of the texturing output. To further address the problem of baked-in lighting, we move beyond RGB colors and pursue assigning parametric materials to the assets. Given the high-quality initial RGB texture, we propose a novel material retrieval method capitalized on Large Language Models (LLM), enabling editabiliy and relightability. We evaluate our method on a wide variety of geometries and show that our method significantly outperform prior arts. We also analyze the role of each component through a detailed ablation study.



a sports shoe, minimalistic, neutral color palette, light and airy, natural materials
a red and silver metallic headphone
a wood kiosk, stylized 3D Art props, handpainted, colorful, simplified, paint stroke


Material assignment results on Source 3D Assets

For each of the following 3D models from the Souce 3D Assets dataset, we show the generated texture, the material part segmentation, and the result obtained by retreiving materials for each part. In the last column, for each material group provided by the asset, we provide a link to the retreived Substance material.

an american footbal
generated texture
segmentation
material assignment

urban boots
generated texture
segmentation
material assignment

a baseball glove
generated texture
segmentation
material assignment

a hiking sport shoes
generated texture
segmentation
material assignment

an interior plant ficus
generated texture
segmentation
material assignment

a beverage carton set
generated texture
segmentation
material assignment

an umbrella closed
generated texture
segmentation
material assignment

a sewing machine
generated texture
segmentation
material assignment

an armchair
generated texture
segmentation
material assignment

a small duffle bag
generated texture
segmentation
material assignment

a notebook
generated texture
segmentation
material assignment

a modern bentwood chair
generated texture
segmentation
material assignment

a cardboard box bag
generated texture
segmentation
material assignment

a high urban bench
generated texture
segmentation
material assignment


Material assignment results on Objaverse models

For each of the following 3D models from the Objaverse dataset, we show the generated texture, the material part segmentation, and the result obtained by retreiving materials for each part.

a bible
generated texture
segmentation
material assignment

a hourglass
generated texture
segmentation
material assignment

Comparison of textured results with state-of-the-art methods on Objaverse

We provide additional comparisons between our texture generation method and baselines.

a lizard
ours
Text2Tex
TEXTure

a cappuccino
ours
Text2Tex
TEXTure

a candle
ours
Text2Tex
TEXTure

a barrel
ours
Text2Tex
TEXTure

a apple
ours
Text2Tex
TEXTure

a binoculars
ours
Text2Tex
TEXTure

a sunflower
ours
Text2Tex
TEXTure

a bookcase
ours
Text2Tex
TEXTure

Relighting assets with different illumination conditions

Our method generates relightable assets that can be lit with different illumination conditions.

a camping tent

an interior ficus plant


More Results

Below we provide more examples of our texture generation results.

  • a burrito
    ours
    Text2Tex
    TEXTure
  • a apricot
    ours
    Text2Tex
    TEXTure
  • a tricycle
    ours
    Text2Tex
    TEXTure
  • a skateboard
    ours
    Text2Tex
    TEXTure

Acknowledgements

We thank Aaron Hertzmann for sharing his vast expertise on suggestive contours, and sharing his code snipped for the ridge detection filter.