Evaluating the impact of CASE: an empirical comparison of retrospective and cross-sectional survey approaches
- PDF / 151,151 Bytes
- 10 Pages / 594 x 797 pts Page_size
- 16 Downloads / 181 Views
1997 Operational Research Society Ltd. All rights reserved 0960-085X/97 $12.00
Evaluating the impact of CASE: an empirical comparison of retrospective and cross-sectional survey approaches RT Coupe and NM Onodu Operational Research Group, School of Management and Mechanical Engineering, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK The purpose of this paper is to evaluate two methods of assessing the productivity and quality impact of Computer Aided Software Engineering (CASE) and Fourth Generation Language (4GL) technologies: (1) by the retrospective method; and (2) the cross-sectional method. Both methods involve the use of questionnaire surveys. Developers’ perceptions depend on the context in which they are expressed and this includes expectations about the effectiveness of a given software product. Consequently, it is generally not reliable to base inferences about the relative merits of CASE and 4GLs on a cross-sectional comparison of two separate samples of users. The retrospective method that requires each respondent to directly compare different products is shown to be more reliable. However, there may be scope to employ cross-sectional comparisons of the findings from different samples where both sets of respondents use the same reference point for their judgements, and where numerical rather than verbal rating scales are used to measure perceptions.
Introduction Managers are typically cautious when taking decisions about the adoption of new development software. This is because the likely gain to the organization must be traded against the cost of its purchase, and of training or securing new personnel. These gains are likely to include improved developer productivity, and better quality applications software that, in turn, may enhance operational efficiency and market competitiveness. While potential purchasers can examine a new software product and evaluate the results of bench testing, they must generally rely on information from other sources to find out how far productivity and quality are likely to improve when new software is actually being used in normal operational circumstances by typical developers and end-users. One important source is to assess the opinions of developers already using the product, using questionnaire surveys. Two approaches, the retrospective method and the cross-sectional method (Coupe, 1994), may be used to measure developers’ views. Most published studies of the impact of Computer Aided Software Engineering (CASE) have used the retrospective survey method, which involves developers comparing CASE and the software technologies previously used for software development. However, these studies do not always make the bases for comparison clear, that is, whether CASE is being compared with the Fourth or Third Generation Language (4GL, 3GL) techCorrespondence: Dr RT Coupe
nology that the respondent previously used, even though the productivity achieved with these technologies is likely to inherently differ. This makes it difficult to reliably assess the producti
Data Loading...
