An increasing number of software systems today are very-large-scale software systems (VLSS) with decentralized control; support for multiple platforms; inherently conflicting requirements; continuous evolution and deployment; as well as heterogeneous, inconsistent, and changing elements (Maier 1998; Northrop et al. 2006; Boehm 2006; Sullivan and Kazman 2008; Dahmann and Baldwin 2008). Such systems are often developed by globally distributed teams and communities. VLSS are often based on system-of-systems architectures comprising multiple heterogeneous systems, which evolve over many years to meet customer, market, and technology requirements. Conventional software engineering has been compared to “building a house”, while the development of VLSS has been compared to the “development of cities” (Northrop et al. 2006).
In a VLSS context methods for monitoring and evolution play an essential role:
Monitoring of VLSS. Ensuring the compliance of VLSS with their requirements is a necessity during evolution as organizations are not willing to use software for business-critical operations if the relationship to its requirements is vague (Robinson 2006). However, due to the size and complexity of VLSS a top-down approach to engineering and maintenance is no longer possible. For instance, the system is no longer under the control of a single stakeholder who determines a consistent set of requirements (Hall 2002). Approaches are required to monitor the behavior of VLSS and to support determining the root cause of problems. In particular, monitoring the performance and resource consumption of software systems is crucial after a changes to track down potential bottlenecks and software anomalies. This is particularly challenging in VLSS as systems might use different technologies, and user transactions can span several platform boundaries. Performance monitoring also has to cooperate with the underlying levels of system software such as the operating system, the virtual machine, or the hypervisor to obtain performance data.
Evolution of VLSS. Industrial VLSS represent a significant investment which needs to be preserved. Many systems have a lifetime of 10-30 years and face continuous evolution. This is specifically challenging in VLSS, which comprise multiple diverse software systems that often evolve independently and exist in a many different versions and variants. In software ecosystems, a community of external developers builds solutions based on a common set of components. In such situations upgrading a system or parts of it at reasonable costs can become almost impossible as both the original system and the deployed system may have evolved independently since the original deployment.
The paradigm shift from “building houses” to “building cities” requires new approaches and tools supporting monitoring and evolution. The figure shows the general themes and the specific research areas of the laboratory. We develop monitoring and evolution techniques for three different stages of VLSS development — operation and production, staging and simulation, development.
Operation and production. We develop methods and tools for monitoring systems at runtime to collect data about system performance and system behavior. For instance, Module 1 has been developing an infrastructure for event-based monitoring of heterogeneous systems in SoS. Module 3 develops non-intrusive performance and memory monitoring techniques inside (Java) virtual machines.
Staging and simulation. Verification and validation in a VLSS context is challenging due to complex dependencies between the systems comprising the VLSS and their environment. Support for staging and simulating VLSS after changes becomes vital to ensure compliance with requirements after evolving components in a VLSS. In Module 1 we are developing requirements-based monitoring techniques that interact with existing system simulators to assess VLSS after modifications. Another area of research are capture and replay techniques to assess deviations from the expected VLSS behavior after changes.
Development. VLSS are often developed in software ecosystems in a globally distributed manner by internal and external developers based on core software platforms. In such settings monitoring ongoing development is essential to understand which features have been changed and to assess the impact of changes. In Module 2, the focus is on monitoring technical development tasks (e.g., using code analysis and change tracking) to support VLSS evolution. This work will be integrated with multi-level feature models in software ecosystems and techniques for model-to-code consistency.