headingimage Publications

Towards a taxonomy of software change
J Buckley, T Mens, M Zenger, A Rashid, G Kniesel

Abstract

Previous taxonomies of software change have focused on the purpose of the change (i.e., the why) rather than the underlying mechanisms. This paper proposes a taxonomy of software change based on characterizing the mechanisms of change and the factors that influence these mechanisms. The ultimate goal of this taxonomy is to provide a framework that positions concrete tools, formalisms and methods within the domain of software evolution. Such a framework would considerably ease comparison between the various mechanisms of change. It would also allow practitioners to identify and evaluate the relevant tools, methods and formalisms for a particular change scenario. As an initial step towards this taxonomy, the paper presents a framework that can be used to characterize software change support tools and to identify the factors that impact on the use of these tools. The framework is evaluated by applying it to three different change support tools and by comparing these tools based on this analysis

 

Towards a taxonomy of software evolution
T Mens, J Buckley, M Zenger, A Rashid

Abstract

Previous taxonomies of software evolution have focused on the purpose of the change rather than the underlying mechanisms. This paper proposes a taxonomy of software evolution based on the characterizing mechanisms of change and the factors that influence these mechanisms. The taxonomy is organized into the following logical groupings: temporal properties, objects of change, system properties, and change support. The ultimate goal of this taxonomy is to provide a framework that positions concrete tools, formalisms and methods within the domain of software evolution. Such a framework would considerably ease comparison between these tools, formalisms and methods. It would also allow practitioners to evaluate their potential use in particular change scenarios. As an initial step towards this taxonomy, the paper presents a framework that can be used to characterize software change support tools and to identify the factors that impact on the use of these tools. The framework is evaluated by applying it to three different change support tools and by comparing these tools based on this analysis.

 

An empirical analysis of information retrieval based concept location techniques in software comprehension
B Cleary, C Exton, J Buckley, M English

Abstract

Concept location, the problem of associating human oriented concepts with their counterpart solution domain concepts, is a fundamental problem that lies at the heart of software comprehension. Recent research has attempted to alleviate the impact of the concept location problem through the application of methods drawn from the information retrieval (IR) community. Here we present a new approach based on a complimentary IR method which also has a sound basis in cognitive theory. We compare our approach to related work through an experiment and present our conclusions. This research adapts and expands upon existing language modelling frameworks in IR for use in concept location, in software systems. In doing so it is novel in that it leverages implicit information available in system documentation. Surprisingly, empirical evaluation of this approach showed little performance benefit overall and several possible explanations are forwarded for this finding.

 

Bloom’s taxonomy: a framework for assessing programmers’ knowledge of software systems
J Buckley, C Exton

Abstract

Programmers must attain knowledge about a system before they can perform specific software maintenance tasks on it. Traditionally, computer scientists have described the activity of attaining this knowledge as ‘software comprehension’. However, if we look at the educational literature, attainable knowledge has been described with much finer granularity. Bloom’s (1956) taxonomy identifies six separate levels of knowledge within the cognitive domain, one of which refers to a (more constricted) definition of comprehension. Several other levels in Bloom’s taxonomy seem to correlate more directly to specific software maintenance tasks. This article reviews Bloom’s taxonomy as a richer descriptive framework for programmers’ knowledge of code and illustrates how various software maintenance tasks map to knowledge levels in this hierarchy. A pilot study is presented showing how participants’ knowledge of software may differ at various levels of this hierarchy.

 

Spatial measures of software complexity
CR Douce, PJ Layzell, J Buckley

Abstract

This paper introduces a set of simple software complexity metrics that has been inspired by developments within cognitive psychology. Complexity measures are constructed by analysing the distance between components of a program. The greater the distance between program fragments, the greater the resulting spatial complexity of a program. Suggestions are made as to how spatial complexity measures can be tailored to individual programmer teams. Using these metrics, the complexity of a software system can be adjusted using subjective measures of programmer experience and knowledge. A related set of simple object-oriented metrics based around the same principles of are also suggested. Finally, a number of further research possibilities are suggested. Index Terms : Software metrics, software complexity, psychological complexity, spatial reasoning, object-oriented programming, human-factors in software engineering, programmer experience, software maintenance.

 

An industrial case study of architecture conformance
J Rosik, A Le Gear, J Buckley, M Ali Babar

Abstract

A software designer often has little control over, or means of checking, whether his design is being adhered to, once the implementation begins. This ‘architectural drift’, where the original design of the system and the as-implemented design of the system diverge, can cause serious problems for evolution, maintenance and the comprehensibility of a system if it remains undocumented or uncorrected. The earlier such discrepancies can be identified, the better.

This paper reports on a case study designed to evaluate a proposed approach for monitoring architectural drift during software development. The proposed approach differs significantly from existing design conformance approaches in that it has been designed for application within a forward engineering context. The findings show the utility of the approach in identifying architectural drift but also illustrate a number of improvements that could be made to the technique to further promote architecture conformance.

 

Inference-based and expectation-based processing in program comprehension
MP O’Brien, J Buckley

Abstract

This paper formally distinguishes between two variants of top-down comprehension (as originally described by Brooks (1983) and Soloway (1984)). The first is inference-based comprehension, where the programmer derives meaning from cliched implementations in the code. The second is expectation-based comprehension, where the programmer has pre-generated expectations of the code’s meaning. The paper describes the distinguishing features of the two variants, and uses these characteristics as the basis for an empirical study. This study establishes their existence, and identifies their relationship with programmers’ domain and coding standards familiarity.

 

Assessing architectural drift in commercial software development: a case study
J Rosik, A Le Gear, J Buckley, MA Babar, D Connolly

Abstract

Objectives: Software architecture is perceived as one of the most important artefacts created during a system’s design. However, implementations often diverge from their intended architectures: a phenomenon called architectural drift. The objective of this research is to assess the occurrence of architectural drift in the context of de novosoftware development, to characterize it, and to evaluate whether its detection leads to inconsistency removal. Method: Anin vivo, longitudinal case study was performed during the development of a commercial software system, where an approach based on Reflexion Modelling was employed to detect architectural drift. Observation and think-aloud data, captured during the system’s development, were assessed for the presence and types of architectural drift. When divergences were identified, the data were further analysed to see if identification led to the removal of these divergences. Results: The analysed system diverged from the intended architecture, during the initial implementation of the system. Surprisingly however, this work showed that Reflexion Modelling served to conceal some of the inconsistencies, a finding that directly contradicts the high regard that this technique enjoys as an architectural evaluation tool. Finally, the analysis illustrated that detection of inconsistencies was insufficient to prompt their removal, in the small, informal team context studied. Conclusions: Although the utility of the approach for detecting inconsistencies was demonstrated in most cases, it also served to hide several inconsistencies and did not act as a trigger for their removal. Hence additional efforts must be taken to lessen architectural drift and several improvements in this regard are suggested.

 

An empirical investigation of the comprehensibility of requirements specifications View Document
D Carew, C Exton, J Buckley

Abstract

It is a commonly held view by software engineers that informal requirements specifications are easier to comprehend than formal requirements specifications. Moreover, the training time required to gain a sufficient level of understanding informal notations is unknown. This paper presents an empirical study carried out to compare the comprehensibility of two specifications, a formal specification and an informal (or semi-formal) specification, in an attempt to quantify the amount of training needed to understand formal methods. The two specifications used implemented the same logic, namely a portion of the Irish Electoral System. The “informal” specification was taken directly from the legal definition of the count rules for Irish elections, and the formal specification was an implementation of the same in CafeOBJ. Both Quantitative and Qualitative data was collected. Although participants had received twenty-five hours training in formal methods, the results show that the informal specification was more comprehendible than the formal specification.

 

Modelling the information-seeking behaviour of programmers–an emperical approach
MP O’Brien, J Buckley

Abstract

Information seeking has been recognised as a core subtask in software maintenance. This paper reviews, merges, and adapts existing information seeking models for different domains to propose a non-linear information-seeking model for programmers involved in software maintenance. Talk-aloud data from two empirical studies of industrial programmers carrying out real software maintenance tasks is presented, to illustrate and validate (in part) this model.

 

Exception based inference based and bottom-up software comprehension
MP O’Brien, J Buckley, TM Shaft

Abstract

The software comprehension process has been conceptualized as being either ‘top-down’ or ‘bottom-up’ in nature. We formally distinguish between two comprehension processes that have previously been grouped together as ‘top-down’. The first is ‘expectation-based’ comprehension, where the programmer has pregenerated expectations of the code’s meaning. The second is ‘inference-based’ comprehension, where the programmer derives meaning from cliched implementations in the code. ´ We identify the distinguishing features of the two variants, and use these characteristics as the basis for an empirical study. This study establishes the existence of the above-mentioned processes, in conjunction with ‘bottom-up’ comprehension. It also illustrates the relationship between these processes and programmers’ application domain familiarity

 

Empirically studying software practitioners – bridging the gap between theory and practice
MP O’Brien, J Buckley, C Extonnbsp

Abstract

It is the view of many computer scientists that the standard of empirical software engineering research leaves scope for improvement. However, there is also an increasing awareness in the software engineering community that empirical studies are a vital aspect in the process of improving methods and tools, for software development and maintenance. This paper presents a review of the empirical work carried out to date in the area of program comprehension and illustrates that most of the evidence from these studies derives from lab-based experiments, thus implying a degree of artificial control. The paper argues that, in order to address the methodological shortfalls of the experimental paradigm, more qualitative methods need to be applied to accompany and support these quantitative studies, thus broadening the sources of data and increasing the ‘body of evidence’.

 

Unanticipated Software Evolution
G Kniesel, J Noppen, T Mens, J Buckley

Abstract

This workshop was dedicated to research towards better support for unanticipated software evolution (USE) in development tools, programming languages, component models and related runtime infrastructures.The report gives an overview of the submitted papers and summarizes the essence of discussions during plenary sessions and in working groups.

 

An open-source analysis schema for identifying software comprehension processes
MP O’Brien, TM Shaft, J Buckley

Abstract

This paper presents an analysis schema for talk-aloud protocols, which distinguishes between bottomup and two variants of top-down comprehension (as originally described by Brooks and Soloway et al). The first is ‘expectation-based’ comprehension, where the programmer has pre-generated hypotheses of the code’s meaning before studying the code segments. The second variant is ‘inference-based’ comprehension, where the programmer derives hypotheses from clichéd implementations in the code. The schema is placed in the context of other talk-aloud analysis schemas. It is described in detail, allowing other researchers replicate the protocol and assess the generality of its findings. A brief description of an experiment, which uses this schema, is also included.

 


A Context-Aware Analysis Scheme for Bloom’s Taxonomy
T Kelly, J Buckley

Abstract

A large body of empirical work in the software comprehension area has focused on the cognitive processes that programmers undertake. However, as yet, little work exists on developing and assessing an encompassing framework within which one can compare comprehension studies with one another. Several authors have proposed that Bloom’s taxonomy could provide such a framework and a lexical-analysis schema has been trialled to classify the data from empirical studies into this taxonomy. The schema is simple to apply but may result in ambiguity and reductionism. This paper proposes an alternative context-aware analysis schema. While such a schema undoubtedly consumes more effort, its value is illustrated by means of a pilot study, where its application is compared to that of the lexical-analysis schema

 

Challenges of evolving sequential to parallel code: an exploratory review
A Meade, J Buckley, JJ Collins

Abstract

Large legacy systems that have been in use for several decades need to evolve in order to take advantage of new technological advances. One such development is the emergence of multi-core processors and parallel platforms. However, the evolution of code written for single-core platforms into code that can take advantage of multi-core technology is challenging. The aim of this research is to explore the challenges that parallel programmers face in the evolution of existing software to exploit multicore and parallel architectures. A review of the current literature was conducted and ten frequently reported challenges were identified. It is important to raise awareness of potential challenges that practitioners may face when evolving sequential code to exploit multicore platforms in order to be better prepared for future evolution. The research community can use these results to develop a research agenda in order to design and develop solutions to address these challenges.

 

Characterizing real-time reflexion-based architecture recovery: an in-vivo multi-case study
N Ali, J Rosik, J Buckley

Abstract

Architecting software systems is an integral part of the software development lifecycle. However, often the implementation of the resultant software ends up diverging from the designed architecture due to factors such as time pressures on the development team during implementation/evolution, or the lack of architectural awareness on the part of (possibly new) programmers. In such circumstances, the quality requirements addressed by the as-designed architecture are likely to be unaddressed by the as-implemented system. This paper reports on in-vivo case studies of the ACTool, a tool which supports real-time Reflexion Modeling for architecture recovery and on-going consistency. It describes our experience conducting architectural recovery sessions on three deployed, commercial software systems in two companies with the tool, as a first step towards ongoing architecture consistency in these systems. Our findings provide the first in-depth characterization of real-time Reflexion-based architectural recovery in practice, highlighting the architectural recovery agendas at play, the modeling approaches employed, the mapping approaches employed and characterizing the inconsistencies encountered. Our findings also discuss the usefulness of the ACTool for these companies.

 

Achieving a reuse perspective within a component recovery process: an industrial scale case study
A Le Gear, J Buckley, B Cleary, JJ Collins, K O’Dea

Abstract

Identifying elements of existing software that are reused within a system may provide maintainers with valuable insights during system evolution. This paper evaluates an extension of software reconnaissance that can be used to analyze reuse across features in a system, as part of a component recovery process. We illustrate and evaluate retrieval of reuse information in this fashion using a large, commercial ERP and warehousing application. Results suggest that the approach scales well in terms of reuse information across features in existing software, providing maintainers with a valuable new perspective on the software system in question.

 

JITTAC: a just-in-time tool for architectural consistency
J Buckley, S Mooney, J Rosik, N Ali

Abstract

Architectural drift is a widely cited problem in software engineering, where the implementation of a software system diverges from the designed architecture over time causing architecture inconsistencies. Previous work suggests that this architectural drift is, in part, due to programmers lack of architecture awareness as they develop code. JITTAC is a tool that uses a real-time Reflexion Modeling approach to inform programmers of the architectural consequences of their programming actions as, and often just before, they perform them. Thus, it provides developers with Just-In-Time architectural awareness towards promoting consistency between the as-designed architecture and the as-implemented system. JITTAC also allows programmers to give real-time feedback on introduced inconsistencies to the architect. This facilitates programmer-driven architectural change, when validated by the architect, and allows for more timely team-awareness of the actual architectural consistency of the system. Thus, it is anticipated that the tool will decrease architectural inconsistency over time and improve both developers and architect’s knowledge of their software’s architecture. The JITTAC demo is available to watch under the ‘Tools’ tab on this site.

 

Design requirements for an architecture consistency tool
J Rosik, J Buckley, M Ali Babar

Abstract

Software architecture and its related documentation are acknowledged as some of the most important artefacts created during system design. However, often the implemented system diverges, over time, from the designed architecture. This phenomenon is called architectural drift and is either a result of inconsistent evolution of the system, or a failure to keep the architectural documentation up to date. A case study, performed at IBM, over two years showed how architectural drift can occur in small development teams over time. It suggested that even when approaches are in place to identify architectural drift, they may prove insufficient for subsequent removal of the drift, and some possible reasons for this were derived. Consequently, this document outlines the resultant design requirements for an approach to inhibit architectural drift, primarily by identifying it as, or before, it is introduced.