Digital Twins are great, but they have some real pain points :

• cost of scanning
• cost of modelling
• massive data

I believe Machine Learning will be used to solve those practical issues.

New ML algorithms will automatically process photos and lidar scan data into 3D models and tag that data – radically reducing the cost to create Digital Twins and increase the value and usefulness they provide.

Digital Twin : Lite 3D models + extra tags

At its heart a Digital Twin is an accurate 3D model, with extra information to make it useful as a proxy for the real world. There are two basic methods to make a 3D model of a building, public space or industrial plant :

• LIDAR laser scan creating billions of dots in 3D space – a pointcloud
• photogrammetry – taking hundreds of photos from a drone or camera and generating a fine 3D textured mesh

There are some bottlenecks with those current methods :

• LIDAR scanners are expensive, require skill and software to operate [ 30k device, 5k/yr software ]
• LIDAR scanners produce a firehose of data – pointcloiuds are often 20GB to 200GB in size
• turning a LIDAR scan into a 3D CAD model is a manual labor-intensive process
• Photogrammetry usually requires a large number of photos for good results : 400+ photos
• Photogrammetry generates a fine mesh which has a relatively large data footprint

To actually use that data, we need lite 3D CAD models that contain the essentials in an efficient format. For example, a flat wall is a simple quad plane, a pipe represented by a cylinder – not a million points or hundreds of triangles.

We might want a color or texture or tags that describe the wall or pipe, such as its thickness, material, whether it carries cold water steam or oil, a part inventory number, geolocation etc. The BIM .ifc classes are a formal standard for adding some of this information, and simple tag keywords can also go a long way to making the data more usable / searchable.

Machine Learning to the rescue

I believe we are on the brink of having Machine Learning algorithms that can solve those problems and make Digital Twins more cost effective and widespread – but it does require investment and a dedicated engineering effort to apply current techniques to build these new solutions.

ML solutions can be built to automate the creation of these lite 3D models :

• auto generate 3D CAD models from pointcloud lidar scans
• auto generate 3D CAD models from fine mesh photogrammetry ‘scans’
• auto generate 3D CAD models from 360 panorama photo ‘scans’, for cases where “cm accuracy” is adequate
• auto tag the 3D CAD model geometry, so the data is easy to query by humans, programs and LLM or Generative AI

Given the rapid advancements in ML, I think we will see these solutions emerge in the next year or so. It seems inevitable and when it happens it will lower the cost of Digital Twins, generate a lot of value and unlock a wide range of uses and applications.

Bringing down the cost will make 3D Digital Twin models of the real world become ubiquitous, it will be easy to share lite 3D models over the internet. Imagine :

• if streetview became a full 3D experience, where you can walk into and around any public building
• 3D and 2D detailed floor maps of every shopping center, campus, transit station, airport/mrt
• 3D / 2D maps of current state of every construction site
• remote shared view : design walkthroughs of buildings, safety reviews and work plan for construction sites
• VR/AR/3D web shopping, virtual meetings, design reviews, realestate sales and gallery showroom tours
• fully capture every pipe, rail, I-beam, cable, conduit of industrial plants, warehouses, production lines and engine rooms
• robots trained on lifelike simulations of the 3D environment
• Architects, renovators, tradespeople and designers having accurate CAD models quickly and at low cost

We can build this

These new Machine Learning algorithms can be developed quite cheaply given the value they unlock.

The above ML components could potentially be developed by 4 small teams of say 3 engineers on each project for 6 months. A ballpark cost of 4 x 3 x 0.5 x 100k = 600k USD to get to the MVP Minimim-Viable-Product stage. Afther that, they are likely to attract funded trials by mid-size companies for their Digital Twin projects, driving the commercialization of the technology.

We know the engineering is tractable, here are some of my own experiments that prove out the concepts :

• auto detect geometry in pointclouds : walls/edges & pipes
• model directly over 360 panoramas : pipe centerlines & textured walls

A quick google tells me that the worldwide market for Digital Twin tech is around 20Bn USD growing at 40% per year .. with 5Bn per year being spent on manual scan-to-CAD. Given the economic value, my question is – why arent we developing this technology full-steam ahead ?

It seems such an obvious and practical use of the new ML techniques and GPU compute – building this missing technology will unlock a vast set of useful applications, bring 3D to the web and reduce the cost of Digital Twins. We need a rich visionary to bet on the future and write a term sheet to get it done.