Differentiable Architecture Search (DARTS) is a recent approach to discover state-of-the-art neural network architectures. Usually, this type of activity requires an unacceptable human effort, thus the interest in developing algorithmic solutions to automate the manual process of architecture design is rapidly grown in last years.

Within this automated technique, an hard-thresholding operator is usually applied at the end of the optimization process to select among many possible choices of connections. The aim of the thesis is to experiment with optimization algorithms that may allow to obtain a sparse solution of the problem by a smooth, continuous procedure.

Sparse Differentiable Architecture Search