A comprehensive software aging analysis in LLMs-based systems

Abstract

Large language models (LLMs) are increasingly popular in academia and industry due to their wide applicability across various domains. With their rising use in daily tasks, ensuring their reliability is crucial for both specific tasks and broader societal impact. Failures in LLMs can lead to serious consequences such as interruptions in services, disruptions in workflow, and delays in task completion. Despite significant efforts to understand LLMs from different perspectives, there has been a lack of focus on their continuous execution over long periods to identify signs of software aging. In this study, we experimentally investigate software aging in LLM-based systems using Pythia, OPT, and GPT Neo as the LLM models. Through statistical analysis of measurement data, we identify suspicious trends of software aging associated with memory usage under various workloads. These trends are further confirmed by the Mann-Kendall test. Additionally, our process analysis reveals potential suspicious processes that may contribute to memory degradation.