Enhancing Text Mining Efficiency Using an Effective Topic Modeling Approach

Abstract

In this digital age, the ever-increasing production of data extends to texts, forcing many applications to transition from manual to automated solutions. Topic analysis and document clustering are the two significant hurdles in the field of Text mining. This research aims to extract topics from documents and cluster them based on these topics by implementing a Distributed database technique. The study investigates popular topic extraction models such as Latent Dirichlet Allocation (LDA) and Bond Energy Algorithm (BEA), which generate vector representations and make semantic clusters across documents. The research evaluates the clustering performance of documents using various metrics suitable for the task. The ultimate goal is to group similar documents in the same cluster to improve document organization and retrieval. The findings suggest that combining topic modeling and vertical partitioning can enhance clustering accuracy while simultaneously reducing computational and storage expenses.