Dissertação

Improving Differentiable Neural Architecture Search with Sparse Connections and Model Pruning EVALUATED

Although these models have shown impressive results in many tasks, the design of neural architectures can take a considerable amount of time. Neural Architecture Search (NAS) aims to solve the problem of manual tuning and experimenting, by automating these tasks. Previously proposed NAS methods have explored reinforcement learning or evolutionary-based algorithms, among other strategies. Differentiable NAS is one of the most promising approaches, particularly in terms of not requiring too many computational resources. Algorithms such as DARTS have shown good results on several benchmarks. Yet, differentiable NAS has several drawbacks which causes instability in the final results and collapse to degenerate architectures. We propose several extensions to the original DARTS algorithm to address the issues. By promoting sparsity while searching for the final structure, and by also changing the optimization algorithm, we can achieve a better performance. The experimental results on different benchmark datasets and search spaces also show that our approach can lead to more stable results with faster convergence.
Neural Architecture Search, DARTS, Sparse Models, Deep Learning

novembro 26, 2021, 14:30

Publicação

Obra sujeita a Direitos de Autor

Orientação

ORIENTADOR

Bruno Emanuel Da Graça Martins

Departamento de Engenharia Electrotécnica e de Computadores (DEEC)

Professor Associado

ORIENTADOR

Paolo Romano

Departamento de Engenharia Informática (DEI)

Professor Associado