An original performance evaluation method for automated failure detection solution using SCADA data analysis

Alexis LEBRANCHU 1 ,2, Sylvie CHARBONNIER1, Christophe BERENGUER1, Frédéric PREVOST2
1Univ. Grenoble Alpes & CNRS, GIPSA-lab, Grenoble, France, 2Valemo S.A.S, Bègles, France



The need for renewable energy has led to an increase of the number of new WTs (wind turbines) erected each year. To monitor and assess the health condition of their installation, O&M (Operation & Maintenance) managers need new effective tools that can synthesize automatic monitoring or health indicators and that are scalable, ie compatible with a large number of WTs. Several solutions have already emerged like CMS (condition monitoring system) with vibration analysis or, more recently, solutions involving processing of SCADA (Supervisory Control and Data Acquisition) data. Those are designed to detect failure in advance, in order to adapt maintenance strategy and reduce downtime and OPEX (Operational Expenditure) as a consequence. The emphasis is mostly put on detecting the failure in its early stages. Consequently, when assessing the performance of new methods, the test is very often focused only on what happened during identified failure period in an offline database. Moreover, there is always a trade-off between the detection performances and the number of false alarms when setting up an automated detection threshold. The work performed during the Phd (A.Lebranchu, 2016) concludes that, evaluating automated monitoring solutions this way does not completely reflect the needs and constrains of industrials. Indeed, this testing procedure returns only little information about the indicators behaviour during the periods without failure, which represents most of the lifetime of a WT. This is why we propose an original method for evaluating the performance of monitoring methods using 4 criterions, applied on monitoring solutions using SCADA data.




The 4 performance evaluation criterions aim to translate O&M manager needs into quantitative and objective metrics: the time between detection and failure appearance, the number of useless on-site interventions triggered by false alarms, the persistence and the availability of the health indicator. The performance is evaluated using two "receiver operating curves": one comparing the time separating detection by the indicator and the failure versus the number of useless on site interventions and another linking the persistence and the number of useless on site intervention. The availability is calculated as the percentage of useful health indicators samples in the testing database. The introduction of useless on site intervention is a brand new criteria that is important for on-shore and is becoming crucial for off-shore WTs.


The evaluation method has been tested on several failure detection solutions using SCADA data on a database with historical data collected during 4 years on 6 turbines, including 8 failures involving the temperature of the generator. The results clearly show that a simple visual observation of the indicator during failure is not enough to clearly understand its behaviour in an operational environment. We also show that some methods allow the detection of the failure at least one week in advance with 10 useless interventions on the whole database. It is reasonable for a single farm but too high for a larger fleet. The results presented from an onshore wind farm are even more relevant for offshore.


New monitoring solutions presented in the literature need to be tested on real case scenario and their performance need to be evaluated with an objective method that aims to reflect all the constraints and need of wind farm operators. We created 4 simple criterions with 2 performance assessment curves that allow any future user to have a better understanding of the behaviour of a new method on failure period and on a normal behaviour period. The results show that even if they very are promising the currently proposed methods using SCADA data still need development in order to be automatically interpreted and still require a highly-qualified expert to make the decision whether or not it is necessary to take action, on large fleet of WTs.


As end user, delegates team will learn about the use of automated monitoring solutions and its consequence on their day to day job. They will learn about the potential of monitoring methods using SCADA for condition-based and predictive maintenance and how to use original performance assessment tools to compare monitoring methods in order to choose the most adapted to their own situation.