Better file formats and standards for representing 3D structures like USD and glTF have played a crucial role in the progress of the metaverse and digital twins. However, there has been much less agreement on the representation of materials. Vendors, artists, and companies mitigate these issues today by staying within a given ecosystem of design tools and rendering engines or generating multiple versions.

Now, the 3D industry and industrial designers are exploring ways to promote interoperability of materials between tools. This could allow creators or companies to create a virtual representation of new fabric, upholstery, styling, shoes, or paint and accurately render it across various tools and 3D worlds throughout.

There are actually two complementary materials interoperability challenges. First, each rendering engine takes a different approach to capturing and rendering the physical appearance of materials under various lighting conditions. Second, there are multiple ways to represent the physical properties of materials, such as how they fold, cover, feel, blow in the wind, or resist fire.

It could take a while for the industry to converge on any format. Several file formats have emerged to help exchange materials between tools and virtual worlds, including U3M, AXF, MDB, MTL, KMP, and SBS, each with their strengths and weaknesses. It may be that industry-specific formats dominate within their respective domains, while others are used across domains.

a realistic look

Companies that create 3D assets for games and entertainment are exploring how better material processing techniques, such as physical-based rendering (PBR), can improve the appearance of virtual worlds. “People often think of a material as fabric, but the 3D industry talks about materials as something visual,” Elliott Round, co-founder and CTO of M-XR, a 3D production platform, told VentureBeat.

Most people are familiar with how primary paints like red, yellow, and blue are combined to create a variety of colors. Materials go a step further with additional texture maps that represent other properties such as albedo, metallicity, roughness, opacity, and diffusion. This is where it gets complicated. “Different rendering engines have different amounts of material properties,” Round explained. “Some will have five parameters, while others may have ten, so they may all work slightly differently. That is something that we hope to solve with other companies to unify 3D a little better.”

The industry has traditionally faced computational and memory limitations for the accurate representation of materials. But now, these restrictions are starting to go away with better computers and algorithms. “I hope we get to a position where we no longer have to cut corners and pirate materials because there are fewer restrictions,” Round said. “It could be unified like in the real world.”

His company has been developing tools and techniques to rapidly capture the visual properties of real-world objects in virtual worlds. They started by using tools like photogrammetry and structured light scanning to capture 3D objects. “All of these approaches give you really good 3D geometry, but none of them will give you material information. And that’s possibly what’s key to photorealism,” Round explained. This includes things like how light reflects off an object and whether it is scattered, absorbed, or transmitted.

His team also explored various types of sample scanners that are often used in the textile industry. These types of scanners from companies like Vizoo and X-Rite can capture visual properties of the material by scanning fabric samples or pieces of paper. Artists and business applications can later apply them to 3D objects. Round said these scans are really good but don’t work particularly well for scanning a whole object, prompting research into better whole-object capture techniques. Epic recently invested in M-XR to help scale these tools for 3D creators.

a realistic feel

Companies that manufacture physical materials, such as textiles, upholstery, and clothing, face additional material challenges. They also need to capture the physical feel of things using various tools and approaches. For example, Bru Textiles, a Belgian textile giant, spent four years developing a workflow to capture visually and physically accurate textile digital twins for its new Twinbru service. Twinbru partnership development manager Jo De Ridder told VentureBeat: “[The digital twin] it is a 100% replica of the physical tissue, both physically and spec-wise.”

This helps design firms create realistic prototypes, such as a new hotel lobby, and quickly explore variations for clients. In the past, they had to approach the look through a swatch book and create a mock-up that didn’t always look the same as the finished product. “Having digital twins shortens the supply chain, reduces complexity and increases accuracy,” De Ridder said.

However, it is a complex process. The Twinbru team spent years developing and optimizing the workflow to capture the visual and physical properties and turn them into digital twins. They used a combination of X-Rite and Vizoo scanners to capture AXF and U3M files that represent visual aspects of the fabrics. In addition, they worked with Labotex to capture the physical properties of the textile in a SAP database that is converted to the appropriate physical engine format. They have created digital twins from the web available for Nvidia Omniverse, Chaos Cosmos, ArchiUp and Swatchbook.

Better industry collaboration could help streamline similar workflows for other companies that make and work with textiles, paints, fabrics and other materials. A 2020 Digital Fabric Physics Interoperability Survey conducted by the 3D Retail Coalition concluded that it is now possible to measure five physical fabric attributes once and accurately translate them into the equivalent physical values ​​for multiple 3D apparel software solutions. These include bending, stretch/elongation, shear/diagonal stretch, weight, and thickness.

Industry leaders are also beginning to collaborate on open standards. For example, Browzwear, which makes 3D fashion design software, has been collaborating with Vizoo to drive adoption of the Unified 3D Material (U3M) standard in the fashion industry. A great advantage compared to other formats is that it can capture both the visual information and the physical properties of the fabric.

“I truly believe that the evolution of the metaverse to the point of mass adoption requires materials and textures to be accurately represented,” Avihay Feld, CEO of Browzwear, told VentureBeat. “Synthetic visions involving digital twins as frozen snapshots of the physical world are a good start. Digital twins as an evolving image of reality that is in sync with reality are even better.”

He argues that it’s unclear where the metaverse is headed, but it’s easy to imagine two possibilities. One is a metaverse that departs from reality, where virtual worlds defy the laws of physics. The other is a metaverse that mimics reality so that users have experiences analogous to those possible in the real world.

He believes that a true representation of visual and physical properties will be essential in this second case. Having realistic things within the virtual world will make it more immersive and engaging, but it will also allow the metaverse to support a variety of use cases. One of the main ones is the trade, not of strictly digital items, but of real-life objects. In this second case, having true digital twins, from the perspective of visualizing textures and simulating the physics of an object, will be essential.

“These two possibilities may co-exist, but without the actual experiences, the metaverse is likely to remain a fantasy world for techies rather than the transformative new universe it has the potential to be.” Feld said.