The field of construction is well placed to benefit from the advent of machine learning and artificial intelligence (AI). As part of the BIM 360 Project IQ Team at Autodesk, I’ve had the privilege to participate in Autodesk’s foray into machine learning for construction.
This article summarizes developments in this space, and covers some ways in which one can prepare to maximize value from this technology, including a broad survey of some of the applications of AI and machine learning in construction, and the potential impact. These processes are making changes across various areas, including risk management, schedule management, subcontractor management, construction site environment monitoring, and safety, to name a few.
What Do We Mean by AI?
The public perception of artificial intelligence usually ranges between the two extremes of having it rule the world to it being dismissed as fantasy with no place in a serious conversation. In reality, the truth lies somewhere in the middle where AI is very far from being a form of super-intelligence but a branch of study that has found tremendous application and is a big driving factor of applications in today’s technology.
Traditionally, defining AI has always been a challenge. ‘Artificial’ is the easier part of the definition where it can simply mean ‘not naturally occurring.’ ‘Intelligence’ on the other hand has led researchers down several rabbit holes. In general, AI refers to a broad field of science encompassing a range of subjects from computer science and psychology to philosophy and linguistics. It is primarily concerned with getting computers to do tasks that would normally require human intelligence. This series of articles provides deeper reading into understanding the definition and history of artificial intelligence.
There are now many fields of work within the broader scope of AI, but here I’d like to define two of the more popular areas — machine learning and deep learning. Machine learning is one such subset that deals with writing algorithms that allow computers to learn from data without being explicitly programmed. If, for example, you want to write an algorithm to identify spam in emails, you will have to train the algorithm by exposing it to many examples of emails that are manually tagged as spam or not-spam. The algorithm “learns” to identify patterns, like occurrence of certain words or combinations of words, that determine the chance of an email being spam.
Deep learning can be considered a set of specialized techniques under the umbrella of machine learning that have really developed more recently. They are based on neural networks, a type of machine learning algorithm that simulates the neurons in the human brain. Deep learning has allowed several breakthroughs to be made in the fields of image and language processing allowing the possibility of advanced applications such as the home assistants and self-driving cars.
Contributing Factors
AI, as an academic field, has been around for a long time, with the first conference on the subject held in 1956. But it has been more in the spotlight over the last decade. This can be attributed to several factors that have themselves improved a lot recently. All of AI requires a lot of data to be present to power the insights that can be derived from it and there has been a tremendous increase in the amount of data generated over the last few years. According to an IBM blog, a few years ago, 90% of the data was then created in the last two years. I guess that can be stretched to about 95% today. Along with data, the computing power available to analyze this data has increased exponentially every year while the cost of the computing power reduced. Today, almost all the data resides in the cloud, and given the availability of resources to process this data, we have seen an outburst of applications that are focusing on making better decisions based on insights received from this analysis.
Applications of AI
A good way to begin to understand the range of AI applications is to examine how ubiquitous it has become. We get several emails every day and most of us spend more time on it than we would like. However, have you noticed how few the number of outrageous spam emails have become? Five years ago, you would have a received at least one email each week claiming that you have won the lottery. Spam detection is one of the older and more well-known applications of machine learning. By looking at thousands of emails, computer programs have been able to “learn” what a spam email usually look like. It can understand that the sender is not someone likely to email you, or from the text of the email that the content is most likely fraudulent. This is just one example of text processing.
Another area of development is the analysis of images. Problems in this space can be reduced to two broad classes, identifying what the image represents and then identifying where exactly an object is in an image. For example, given a photo, is there a cat in the photo? The second challenge would be whether you can detect where exactly is the cat in the image?
With the development of deep learning, there have been several breakthroughs in this space. Moving from the detection of cats, the new iPhones come with face detection-based locking functionality. The algorithms are able to distinguish the finer details of a photo and do it quick enough that they can be used to make real-time decisions. The algorithms were detecting if photos had a cat in them but now they can actually tell you if it is the same cat in every photo.
When these image recognition and detection algorithms are combined with other forms of AI, you get fascinating applications like self-driving cars. They are able to sense their surrounding and use that to navigate the environment. It is a complex task of understanding different objects in the environment, understanding the differences in how they behave or operate, the rules they imply in the environment. The image below is the example of a view that a self-driving car would use to navigate. The system is able to distinguish between people, cars, stationary objects. It has to understand the meaning of a red light and a one-way sign. It is able to estimate the distance between the objects and use this to make decisions. This article provides a more in-depth and interactive reading on how they are made and the technology behind them.
Human minds are capable of dealing with about three to four different data points when making a decision. This is another area that artificial intelligence has made headway. While human minds are only capable of dealing with three to four dimensions, AI algorithms don’t have a limit. When you are binge watching a show that Netflix recommended, their recommendation engine probably considered a lot of different things; what’s your age, gender, what do other people of this profile like to see, what kind of shows have you been watching, the reviews for the show, the engagement of other users with this show and many such other attributes.
The common requirement across all of these examples is that these algorithms need a lot of data to learn from. AI provides a recommendation or a solution based on the data it is provided with and therefore is only as good as the quality of that data. In order to be able to really utilize what this technology offers, we have to put thought into organizing what data we collect and managing it across all products. A common saying in AI circles is, “garbage in, garbage out.”
What Changes Is AI Bringing to Construction?
Technology for construction has seen a lot of investment over the last few years. A huge portion of that investment has gone towards digitization of different parts of the construction workflow. BIM models have changed the way that buildings are designed, the project management and issue management processes have moved to the cloud, operations management is also becoming more “sensorized” and automated. With the availability of data, AI-based applications have been finding more utility in construction.
Generative Design
Generative design is a form-finding process that can mimic nature’s evolutionary approach to design. Computer scientists have found ways to aid the building design process. It usually starts with clearly specifying the design goals and then exploring innumerable possible permutations of a solution to find the best option. It becomes clearer with an example.
An Autodesk team in Toronto has moved into a new building which was designed using a new design process and this also makes for a good story. Researchers used generative design to assist them in finding the ideal design for the building that met all their needs. The process began with understanding all the parameters that would be important to the residents of the office building — adjacency preference, work style preference, buzz, productivity, daylight, and views to outside.
This input was then fed into a computer system that understood these design parameters along with the requirements of the physical location. The algorithm then produced several designs which fit all these needs that the architect can pick from to match the style and other needs. Since this process was very quick, it was easy to make the design experience iterative and work and reform the final design based on several conversations. As this article explains in more depth, in addition to solving vexing practical challenges, generative design could also raise the efficiency and economy of the overall development process by producing designs that improve scheduling and coordination between multiple stakeholders.
Risk Mitigation
Risk evaluation and mitigation happens every day on a construction site. There are hundreds of subcontractors working on different trades simultaneously; there are thousands of issues which get created and managed and everything is changing constantly. The BIM 360 IQ project focused on understanding the challenges that construction managers, project managers and superintendents, deal with on a daily basis to manage these issues and the ways that the process might be improved with AI. After talking to several construction superintendents, visiting their jobsites, and then looking at the data that they were generating, we identified that prioritization of issues by risk would provide an actionable way to improve efficiency.
Using AI, especially construction language analysis, it is possible to automatically assign priority to issues. The algorithms are able to understand and predict complex things like whether an issue would cause a potential water infiltration if not addressed. The system makes use of the descriptions that many quality managers across projects have observed when they monitor their projects.
For example, if a quality manager observes an issue about the flashing outside a window being incomplete and records it BIM 360 Field, as is usually the practice, then the AI algorithms run through this data and automatically flag it as a potential water issue. This can then be brought to the superintendent’s attention when they review all the issues on the dashboard. This system is currently available on pilot and available to anyone using BIM 360 products.
The system also goes one step further where it encapsulates all the risk from the issues into the subcontractors who are responsible for it. It considers various factors about the subcontractors such as their past behavior of issue management, current workload, the importance of the issues for which they are responsible for. The algorithm is then able to assign a “risk score” to each subcontractor in the project, a metric to indicate the amount of risk that they are currently exposing the project to, so that the construction managers can better prioritize their time to work closer with these teams.
Safety
Construction safety is the number one priority across all jobsites. The BIM 360 IQ focused on understanding the behavior and context around safety issues and then bringing it to the attention of safety managers. The IQ application automatically scans all safety issues on a jobsite and attaches a tag to them indicating whether it could lead to a potential fatality. OSHA shows that about 67% of all construction related fatalities in 2015 were due to issues related to the “fatal four” — fall, struck by, caught between, and electrocution. The IQ algorithms classify safety issues that are precursors to the fatal four.
The application also provides insight into what the hazard causing the potential incident actually was and shows a distribution across 39 different hazards.
This allows safety managers to understand where exactly they should focus their planning and training efforts and to be more observant for specific problems when they do their safety walk.
Today, there are plenty of photos and videos taken daily on a jobsite. Every construction worker has a phone with a camera and it is almost standard practice to take a photo for every issue they create. Drones have become more common and they are often used for aerial shoots, and for more advanced activities like measuring progress. Go-pros and smart helmets are becoming more common as well. Given the plethora of photos, most technology applications have yet to catch up and there hadn’t been a good solution for managing the photos or utilizing them for better insight.
Smartvid.io is a technology start-up addressing exactly this solution. They provide a platform that integrates with different other technology vendors to bring all your images to one place. However, they go one step further, they use AI to understand what is there in the image. Just as we saw earlier in the example on self-driving cars, it is possible to isolate and understand the various objects in an image. Smartvid.io calls them “smart tags” and they allow for a better system to categorize and search your photos.
What Will Autodesk Do in the Future for AI in AEC?
The BIM 360 IQ product for quality was the first AI product for construction and ever since we’ve been trying to push the envelope further every year. We’ve built applications that focus on challenges in both construction quality and safety. Our next effort is to take a similar approach to project management and leverage AI to enhance the process.
Data Platform
Across the construction industry, there are several technology vendors providing solutions to manage their data but they are often incompatible with each other. The power of AI-based solutions can be unlocked best when all data sources can connect to each other. To address this need, Autodesk is also working towards building a data platform that would allow third-party integrations. This would enable different construction companies to bring all their data to one platform that would also have the capabilities of a common analytics layer. Autodesk is bringing in other data sources such as ERP data and project management data into this platform as well as partnering with other data companies in construction such as Smartvid.io, Triax Technologies, SmartBid, and others.
Anand Rajagopal is a data scientist at Autodesk where he is part of the product team for BIM 360. In his current role, he works closely with customers to build, test, and deploy machine learning models that are specific to construction.