Andy Liu

Sparsifying autoencoders

Started Nov. 3, 2023

Abstract or project description

As the field of machine learning becomes more publicly commonplace, the creation of simple neural networks has become much more accessible to newer programmers. Models with few layers and singular activation functions are simple to create, applicable to many tasks, and serve as an introduction to machine learning. However, mathematically and conceptually simple formulas can cause undesirable results when implemented without modification. In this project, we test the hypothesis that a direct implementation of the commonly used L1 regularization error function is incompatible with encouraging sparsity in an autoencoder model. Additionally, we tested a modification of the regularization function and demonstrate that the mean absolute value function as a loss penalty predictably results in convergence on undesirable states.