Name: ANTONIO LUIZ DA SILVA LOCA
Type: MSc dissertation
Publication date: 04/06/2020
Advisor:

Name	Role
THOMAS WALTER RAUBER	Advisor *

Examining board:

Name	Role
FLÁVIO MIGUEL VAREJÃO	Internal Examiner *
FRANCISCO DE ASSIS BOLDT	External Examiner *
THOMAS WALTER RAUBER	Advisor *

Summary: This work presents a systematic procedure to fairly compare experimental performance
values for machine learning approaches for fault diagnosis based on vibration signals. In
the vast majority of related scientific publications, the estimate of accuracy and similar
performance criteria are the only quality parameters presented. The methodology that
was used to generate the results of these publications is predominantly biased, based
on validation methods that are too simple. In addition, all methods, in general, recycle
identical patterns to estimate the best hyperparameters, introducing additional overfitting
in the results. To repair this problem, the conditions used in the training, validation
and test division were critically analyzed and an algorithm was proposed that allows a
well-defined comparison of the experimental results. To illustrate the work’s ideas, the
Case Western Reserve University Bearing Data benchmark is used as a case study. Four
distinct classifiers are compared experimentally, under more difficult generalization tasks
using the proposed evaluation structure: K-Nearest Neighbors, Support Vector Machine,
Random Forest and One-dimensional Convolutional Neural Network. An extensive review
of the literature suggests that most research work at Case Western Reserve University
Bearing Data uses similar standards for training and testing, making classification an easy
task

Access to document