On the Efforts in Training Deep Networks: Reducing Data-related and Computational Costs

Name: Rodrigo Ferreira Berriel
Type: PhD thesis
Publication date: 28/09/2021
Advisor:

Namesort descending Role
Thiago Oliveira dos Santos Advisor *

Examining board:

Namesort descending Role
Alberto Ferreira De Souza Internal Examiner *
Claudine Santos Badue Gonçalves Internal Examiner *
Francisco de Assis Boldt External Examiner *
Jurandy Gomes de Almeida Junior External Examiner *
Thiago Oliveira dos Santos Advisor *

Summary: Nowadays, deep neural networks (DNN) are ubiquitous. Many of our daily problems have been solved or alleviated with the help of such models. It is important to note, though, that the success of these models is built on top of some drawbacks: they are data hungry and computationally intensive. In order to achieve the performance required for real-world usage, it is often necessary to train these DNNs in massive data sets. These data sets, in turn, require the acquisition and annotation of a lot of samples. Both these processes, data acquisition and annotation, are expensive and time-demanding, i.e., they require a lot of investment in sensors, time, and people in a series of sometimes tedious tasks that are error-prone. In this thesis, we explore ways of reducing the effort on training deep neural networks, especially towards the aforementioned problems. We investigate how to leverage data readily available online to reduce the data acquisition cost. Moreover, we propose to fuse data from multiple online services to reduce the data annotation cost. Furthermore, we also study ways of reducing the computational cost on inference time, particularly on the perspective of multi-domain learning. Our results show many applications of automatic large-scale data acquisition and annotation that proved to be useful for training deep convolutional networks for both classification and regression real-world problems. In addition, we proposed a model that has a low computation complexity, requires lower storage, and has a low memory footprint compared to the alternatives, demonstrating an effective way of reducing the computational requirements of deep networks. Finally, our proposals showed that the efforts in training deep networks can be reduced even further, particularly when it comes to data-related and computational costs.

Access to document

Acesso à informação
Transparência Pública

© 2013 Universidade Federal do Espírito Santo. Todos os direitos reservados.
Av. Fernando Ferrari, 514 - Goiabeiras, Vitória - ES | CEP 29075-910