Semi-Orthogonal Non-negative Matrix Factorization with Sparse Constraint in Topic Modelling

Abstract

This study introduces a novel method for topic modeling by ombiningsemi-orthogonal matrix factorization with sparse constraints to improve interpretability, coherence, and scalability. Traditional techniques such as Latent Dirichlet Allocation (LDA) and Nonnegative Matrix Factorization (NMF) often face challenges in maintaining these qualities, especially with high-dimensional data. To address these issues, we propose the Spectral Proximal Method (SPM), an optimization approach that uses proximal variable metric updates with spectral diagonal scaling. SPM enforces both l1-norm sparsity and semi-orthogonality to generate diverse and interpretable topics. The algorithm uses non-convex alternating minimization, with initialisation based on NMF to enhance computational efficiency.