3DXTM Inspection Platform for Rotor Blade Inspection...2019/10/03 · whole portfolio of wind turbines. The reporting components of the platform will deliver full transparency regarding

Sulzer Schmid

3DXTM Inspection Platform

for

Rotor Blade Inspection

Sulzer & Schmid Laboratories AG

Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]

www.SulzerSchmid.com

©2019 Sulzer & Schmid Laboratories AG, All Rights Reserved

Page 2 of 18

Table of Contents

1. Introduction .......................................................................................... 3

2. The Inspection Process ........................................................................... 4

3. 3DXTM Inspection Platform ...................................................................... 5

3.1 The Platform Concept ..................................................................... 5

3.2 Platform Benefits ........................................................................... 6

3.3 Operational Flexibility & Freedom ..................................................... 7

3.4 Data Capture ................................................................................ 8

3.4.1 3DXTM Job Portal ................................................................... 10

3.4.2 3DXTM Smart Box & Auto Drone .............................................. 11

3.4.3 3DXTM Ground Station............................................................ 12

3.4.4 3DXTM Control Station ............................................................ 12

3.4.5 3DXTM Cloud Data Repository ................................................. 12

3.5 Data Processing ............................................................................ 13

3.5.1 Pre-Annotation ..................................................................... 13

3.5.2 Annotation ........................................................................... 14

3.5.3 Review ................................................................................ 15

3.5.4 Protection of Data and Data Integrity ...................................... 15

3.6 Data Exploration ........................................................................... 16

3.7 Integration into 3rd Party Proprietary Systems .................................. 17

3.8 Additional Services........................................................................ 18

3.8.1 Data Storage ........................................................................ 18

3.8.2 Data Interfaces ..................................................................... 18

3.8.3 Data Cleansing ..................................................................... 18

http://www.sulzerschmid.com/


Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 3 of 18

1. Introduction

Sulzer & Schmid Laboratories AG (Sulzer Schmid) is a Swiss technology company and innovator

of next-generation technology for the inspection of

wind turbine rotor blades and towers. Our digital

end-to-end process employs autonomous drones

and incorporates cutting-edge image analysis tools.

Reports are generated at the push of a button and

are highly customizable. All the data is stored and

made accessible via secure cloud services and is available for further analysis.

Sulzer Schmid can be deployed globally. More than 5’000 wind turbines have been inspected with

the 3DXTM Inspection Platform (as of 2019).

Sulzer Schmid has been applying the following principles to the development of its 3DXTM Inspec-

tion Platform:

• End-to-end digital inspection process

• High degree of automation along the entire workflow

• Reproducible results & 100% blade coverage

• Quality according to real-world requirements

• Data is equally suitable for manual analysis by blade specialist and automated analysis

by Artificial Intelligence algorithms.

• Cost-effective process and broad operating boundaries (temperature, wind)

More than 5’000 wind turbines

across Europe, US and Canada

have been inspected with the

3DXTM Inspection Platform (as of

2019)



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 4 of 18

2. The Inspection Process

Every visual inspection of a wind turbine is composed of the following three major steps:

Sulzer Schmid has developed several software components, services and hardware to support the

inspection workflow in a seamless and fully digital fashion. This powerful technology toolbox is

commonly referred to as 3DXTM Inspection Platform.

Each step of the inspection process is supported by software and services that are described in

the following.

Customers have many options as to how they want to use the Sulzer Schmid technology compo-

nents. Some may choose to leverage the entire platform while others may prefer to only use

certain components.

In addition, customers have different options when it comes to selecting the staff who operate

the platform components. Customers may want to outsource the entire inspection process, or

they may prefer to only outsource drone operations but keep data analysis and review in house.

The 3DXTM Inspection Platform can support a wide variety of scenarios and use cases.

Figure 1: Flexible Business Model Figure 2: Modular Components



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 5 of 18

3. 3DXTM Inspection Platform

Sulzer Schmid 3DXTM technology supports the entire workflow from

planning inspections to creating inspection reports and making repair

decisions.

In order to support this process, Sulzer Schmid has developed sev-

eral software, hardware and cloud service components that are

jointly referred to as the 3DXTM Inspection Platform.

3.1 The Platform Concept

The 3DXTM Inspection Platform supports several vital functions that all relate to the inspection of

wind turbines. The platform itself is comprised of software, hardware and service components.

The benefits of using these plat-

form components are numerous.

One of the most compelling ad-

vantages is that the platform as-

sures the quality and integrity of

the data.

No matter who the drone pilot is,

the data will always be captured in

the same way.

No matter from which part of the

world the data is coming from, the

data will always be consistent and

compatible.

Results can be compared across

the entire fleet and powerful statis-

tics can be computed across the

whole portfolio of wind turbines.

The reporting components of the platform will deliver full transparency regarding the condition

of the turbines.



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 6 of 18

3.2 Platform Benefits

Leveraging the 3DXTM Inspection Platform will render great benefits along several axis:



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 7 of 18

3.3 Operational Flexibility & Freedom

The 3DXTM Inspection Platform is a technology that has been designed and developed for ease of

use. Anybody can be trained to use it: from drone operators to rotor blade experts.

Since wind turbine inspections are conducted across the globe in many countries, it becomes very

difficult to find one single partner who can provide operational drone services in each of these

countries.

Therefore, several operational partners can be selected - each having a certain geographical foot-

print. The technology will assure that the inspection results are consistent.

Sulzer Schmid will engage with such companies to discuss the prerequisites and setup a training

program for their staff. While the requirements for a 3DXTM operators are minimal, affinity to

technology /IT and previous experience with drones is a clear advantage. Operational partners

are supported by Sulzer Schmid regional basis throughout their entire inspection campaigns. In

certain territories, Sulzer Schmid has already established relationships with operational partners.

Blade experts don’t need to be local as the inspection data is served up from the cloud storage

for review. However, the blade experts need to have firsthand experience inspecting or repairing

blades to the extent to be able to categorize damages and assign severities appropriately.

This is a model that has been proven on several occasions in the context of larger inspection

campaigns.

Figure 3: Different operational partners can be assigned different territories for data acquisi-

tion with autonomous drones



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 8 of 18

3.4 Data Capture

The first step of every inspection is to document the entire surface of the rotor blades (and op-

tionally the tower) with high resolution images. Sulzer Schmid has developed fully autonomous

drone technology that is used to collect data in a consistent and repeatable fashion – irrespective

of drone pilot skills.

Operating a Sulzer Schmid drone is

much like operating any other au-

tomated tool. Within a couple of

days, novice operators can be

trained to use the drone and its

software and will be fully produc-

tive. The combination of an auton-

omous drone with powerful soft-

ware, assures maximum safety

and efficiency during operation.

No longer does it matter how

skilled the drone operator is. If the

field team has received basic train-

ing and if they follow the well-es-

tablished procedures, the results

will be of high quality and high consistency – over and over again.

The data acquisition is carried out by field

teams of two who travel in a light vehicle

(pickup truck, hatchback, etc.) and are

equipped with a 3DXTM Smart Box that con-

tains all the necessary hardware. A field

team consists of one drone operator who

holds a valid pilot license and one turbine

operator who is qualified to stop the turbine and index the rotor.

Figure 4: Reproducible images

No longer does it matter how skilled the

drone operator is … the results will be

of high quality and high consistency –

repeatedly and from anywhere in the

world.



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 9 of 18

As soon as the team arrives at the turbine site, a standard process will unfold. This process is

identical for each inspection and is supported by software and checklists for the team.

The entire process takes approximately 45 minutes (from arriving on site until departure). The

turbine is stopped for 30-40 minutes. During this process, the autonomous drone will capture

high resolution images of all three blades (entire blade surface).

Wind turbine inspection orders can be comprised of a dozen inspections or can represent a large

campaign of several thousand inspections. Sulzer Schmid provides a powerful infrastructure to

comply with orders of any size that can be spread across many countries and several continents.

All this is part of the 3DXTM Inspection Platform.

In the following, we describe all the sub-systems that are used for Data Acquisition.

Stop WTG

Prepare drone, verify WTG

location

Autonomous flight

Index blade(Y-position)

Data import /data verification

Repeat for blades

2,3

.

Figure 5: Standard inspection process with autonomous drone



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 10 of 18

3.4.1 3DXTM Job Portal

Everything starts with an inspection order. An order can consist of a dozen wind turbines or

thousands spread across many countries.

The job portal serves to capture data about these turbines:

• Location (Latitude, Longitude)

• Turbine Type (Brand, Type, Rotor Diameter, Tower Height)

• Serial Numbers (if available)

The 3DXTM Job Portal is a simple to use website that allows entering such data manually or that

also supports importing data from various electronic data sources (Excel, csv, etc.). Setting up

an entire wind park or an entire fleet of turbines is an easy process. Already during this very first

step, data is validated whenever possible.

Depending on the size of an order and the desired completion time, orders can be broken down

into several jobs. These jobs are then assigned to the field teams who will go out to the wind

parks to acquire the visual data with the autonomous 3DXTM drones. Teams can simply download

job files from the portal. These job files contain all the necessary information to execute the drone

inspections in the field.

Each user of the job portal will receive login credentials that control what data can be accessed,

viewed or modified.



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 11 of 18

3.4.2 3DXTM Smart Box & Auto Drone

The 3DXTM Smart Box that contains all the necessary hardware that is used by the field teams.

Sulzer Schmid has incorporated off-the-shelve drones by DJI (the world’s leading drone manu-

facturer). These

drones are equipped

with a custom payload

that contains the high-

resolution camera, all

the necessary sensory

equipment and the au-

tonomous navigation

controller.

In addition to the

drone, each Smart Box

contains:

• A Windows 10 laptop computer with the 3DXTM Ground Station and 3DXTM Control Station

• 3DXTM Payload

• Powered USB hub

• RTK base antenna

• Handheld laser measurement device

• GPS locator tool

• Basic set of spare parts

Figure 6: 3DXTM HD SmartBox with AutoDrone



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 12 of 18

3.4.3 3DXTM Ground Station

The 3DXTM Ground Station software is used by the field team to execute the autonomous drone

missions. The software runs on a stand-

ard Windows 10 computer that is part of

the Smart Box. The Ground Station is

processing the job files that are down-

loaded from the job portal and it transfers

all the necessary data to the drone before

takeoff.

3.4.4 3DXTM Control Station

3DXTM Control Station software greatly simplifies the work of the field teams. It facilitates the

management of all the data and the transfer of such data to the 3DXTM Cloud Data Repository.

The software runs on the same Windows 10 laptop as the Ground Station. Data can be transferred

whenever the team has access to an Internet connection (for example during the night from a

hotel room). Control Station provides for a maximum of data integrity and is easy to use.

3.4.5 3DXTM Cloud Data Repository

The 3DXTM Cloud Data Repository is a highly secure data vault. It leverages the Microsoft Azure

cloud infrastructure which provides many advanced security features and highly granular control

over user credentials. Data from all the field teams is transferred to the repository from where it

can be accessed for data analysis. Different policies can be applied as far as data storage is

concerned.

3DXTM Cloud Data Repository is highly scalable. It can receive large amounts of data from many

field teams that operate at the same time in different parts of the world.

Figure 7: 3DXTM Ground Station



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 13 of 18

3.5 Data Processing

Once the data has been uploaded to the 3DXTM Cloud Data Repository, the data analysis process

can start. This can be as early as the next day after flying the drone.

Sulzer Schmid has developed powerful data analysis tools to cope with the large amounts of data.

Data analysis breaks down into three steps:

• Pre-Annotation (manual or automated via AI algorithms)

• Annotation

• Review

The infrastructure supports an unlimited number of users that can be located anywhere in the

world.

3.5.1 Pre-Annotation

During pre-annotation, a technician uses the 3DXTM Blade Station software to quickly scan the

images that belong to the three blades of a wind turbine. The process mimics a virtual flight along

the blade – much like the drone did. These trained technicians mark anomalies in the data. They

Figure 8: Data Analysis Process

Analyze blades

Run postprocessing

Generate inspection report

Save annotations

Quality assurance (verification)

Repeat for blades

2,3



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 14 of 18

do not have to be composite materials experts as they will not make the final classification of the

defects. Their primary job is to reduce the amount of data that the blade experts must look at.

The pre-annotation can be conducted anywhere in the world. Customers have the choice to use

the cost-effective Sulzer Schmid pre-annotation team (located in India) or they can use their own

staff.

Sulzer Schmid has partnered with a renowned AI (Artificial Intelligence) company to automate

pre-annotation. As soon as the results of the AI algorithms reach the quality of the manual as-

sessment (spring of 2020), we will offer both options (manual and automated).

3.5.2 Annotation

Once the pre-annotation step is completed, a blade expert will examine all the anomalies that

were found during pre-annotation. Blade experts can be located anywhere in the world as they

are using a web application (3DXTM Analysis Portal) that offers all the functionality to examine

anomalies in detail.

Customers have the choice between using their own blade experts or by contracting this as a

service from one of the Sulzer Schmid blade expert partners.

Figure 9: 3DXTM Data Analysis Portal



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 15 of 18

After the assessment by the blade expert (typically 15 minutes per turbine), all the findings are

stored in a digital format and are available for reporting and further analysis.

Traditionally, blade experts produce single turbine inspection reports as pdf documents. The cre-

ation of such reports is fully supported by the 3DXTM Blade Station software and by the 3DXTM

Data Analysis Portal. Reports can be customized to fit the required report format of the customer.

In addition to generating inspection reports, blade experts can create special tags for the man-

agement of blade repairs.

3.5.3 Review

The final step during data analysis is an optional review. Some customers choose to let senior

blade specialists review the inspection reports before they are released and distributed. Reviewers

can access all the findings by the blade experts by logging into the 3DXTM Analysis Portal. As they

are assigned a special role, they can make modifications and changes before they approve an

inspection.

3.5.4 Protection of Data and Data Integrity

As mentioned before, data analysis can be spread across various roles and locations. The 3DXTM

Data Analysis infrastructure provides many safeguards to assure data integrity and to optimize

the workflow. For example, if a blade expert wants to work on a dataset, the system will first

check if the blade expert is authorized to access such data.



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 16 of 18

3.6 Data Exploration

Traditionally, inspections were completed with the generation of an inspection report (pdf docu-

ment) for each wind turbine. These reports were used to assess the need for repairs and were

then archived.

The 3DXTM Inspection Platform offers entirely new capabilities since all findings are stored in a

machine-readable, digital format.

Once the data has been analyzed, it can be presented and consumed in various ways. In addition

to single turbine reports, users who subscribe to Data Exploration Tools such as the 3DXTM Blade

Explorer, can navigate the inspection data of their entire fleet in an intuitive fashion via a browser

application. Here are just a few examples of questions that can be answered with a few mouse

clicks:

• Show me all class 4 & 5 damages of wind park XYZ

• Produce an excel data export of all the damages that need immediate repair

• Examine different wind park locations within my fleet with respect to the probability for

lightning damages of class 3 or higher

In addition to providing a lot of powerful data query functionality, the 3DXTM Blade Explorer can

be white labeled. This might be interesting for companies who want to present such data to their

end customers.

All data access is controlled in a highly secure fashion and in compliance with the corresponding

business agreements.



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 17 of 18

3.7 Integration into 3rd Party Proprietary Systems

Already today, Sulzer Schmid has developed data interfaces of the 3DXTM to selected proprietary

systems of customers and partners. Upon request and based on the business arrangements,

Sulzer Schmid will custom develop such data adapters to facilitate the data transfer between

the 3DXTM Inspection Platform of Sulzer Schmid and a target 3rd party system.

Figure 10: Data Interface



Industriestrasse

8618 Oetwil am See

SWITZERLAND

[email protected]



Page 18 of 18

3.8 Additional Services

In addition to the three main functional clusters (data acquisition, data analysis, data exploration),

Sulzer Schmid offers a few complimentary services that relate to its 3DXTM Inspection Platform.

3.8.1 Data Storage

Data is at the core of each inspection and represents considerable value. Especially as turbines

are re-inspected year after year, historical data can provide extremely relevant insights in topics

such as damage propagation and repair history. Sulzer Schmid has developed an appropriate and

secure infrastructure to preserve such data for as long as customers desire.

3.8.2 Data Interfaces

Sulzer Schmid recognizes the need to support the exchange of data with 3rd Party IT systems.

This can concern the transfer of raw data at the end of the data acquisition process in cases where

customers want to use a proprietary data analysis toolchain or it can concern the transfer of

damage annotation data, or the output of different analysis queries.

Due to the open architecture of the 3DXTM Inspection Platform, data connectors can be provided

and customized to the customer’s needs.

3.8.3 Data Cleansing

When field teams are sent out to a wind park to acquire inspection images, it can happen that

the coordinates for certain WTG’s are misplaced or that the associated turbine data is wrong. This

always creates problems causing friction and lost time during data acquisition. Especially for larger

inspection campaigns, it is recommended that these data problems are resolved (data cleansing)

and fed back into the customer’s database. Data cleansing services are offered on a case-by-case

basis.


Documents

3DXTM Inspection Platform for Rotor Blade Inspection...2019/10/03 · whole portfolio of wind turbines. The reporting components of the platform will deliver full transparency regarding