2005.html

<!-- This document was automatically generated with bibtex2html 1.96
     (see http://www.lri.fr/~filliatr/bibtex2html/),
     with the following command:
     bibtex2html -dl -nodoc -nobibsource -nokeys -nokeywords -nofooter 2005.bib  -->




<p><a name="csdl2-04-11"></a>

Philip&nbsp;M. Johnson, Hongbing Kou, Michael&nbsp;G. Paulding, Qin Zhang, Aaron Kagawa,
  and Takuya Yamashita.
 Improving software development management through software project
  telemetry.
 <em>IEEE Software</em>, August 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2004/04-11/04-11.pdf">.pdf</a>&nbsp;]
<blockquote><font size="-1">

Software project telemetry is a new approach to software project
management in which sensors are attached to development environment tools
to unobtrusively monitor the process and products of development. This
sensor data is abstracted into high-level perspectives on development
trends called Telemetry Reports, which provide project members with
insights useful for local, in-process decision making.  This paper presents
the essential characteristics of software project telemetry, contrasts it
to other approaches such as predictive models based upon historical
software project data, describes a reference framework implementation of
software project telemetry called Hackystat, and presents our lessons
learned so far.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-04-16"></a>

Qin Zhang.
 Improving software development management with software project
  telemetry.
 Ph.D. Thesis Proposal CSDL-04-16, Department of Information and
  Computer Sciences, University of Hawaii, Honolulu, Hawaii 96822, October
  2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2004/04-16/04-16.pdf">.pdf</a>&nbsp;]
<blockquote><font size="-1">

Software development is slow, expensive and error prone, often resulting in
products with a large number of defects which cause serious problems in
usability, reliability and performance. To combat this problem, software
measurement provides a systematic and empirically-guided approach to
control and improve development processes and final products. Experience
has shown excellent results so long as measurement programs are
conscientiously implemented and followed. However, due to the high cost
associated with metrics collection and difficulties in metrics
decision-making, many organizations fail to benefit from measurement
programs.

In this dissertation, I propose a new measurement approach -
software project telemetry. It addresses the "metrics collection
cost problem" through highly automated measurement machinery - sensors
are used to collect metrics automatically and unobtrusively. It addresses
the "metrics decision-making problem" through intuitive high-level visual
perspectives on software development that support in-process,
empirically-guided project management and process improvement. Unlike
traditional metrics approaches which are primarily based on historical
project databases and focused on model-based project comparison, software
project telemetry emphasizes project dynamics and in-process control. It
combines both the precision of traditional project management techniques
and the flexibility promoted by agile community.

The main claim of this dissertation is that software project telemetry
provides an effective approach to (1) automated metrics collection, and (2)
in-process, empirically-guided software development process problem
detection and analysis. Three case studies will be
conducted to evaluate the claim in different software development
environments:

(1) A pilot case study with student users in software engineering classes to
(a) test drive the software project telemetry system in preparation for the
next two full-scale case studies, and (b) gather the students' opinions
when the adoption of the technology is mandated by their instructor.

(2) A case study in CSDL to (a) use software project telemetry to
investigate and improve its build process, and (b) evaluate the technology
at the same time in CSDL (an environment typical of traditional software
development with close collaboration and centralized decision-making).

(3) A case study at Ikayzo with open-source project developers
(geologically-dispersed volunteer work and decentralized decision-making)
to gather their opinions about software project telemetry.

The time frame of this research is as follows. The implementation of the
software project telemetry system is complete and deployed. I have finished
the first pilot case study. I will start both the second and third case
studies from October 2005, and they will last 4 - 6 months. I wish to
defend my research in May or August 2006 if everything goes according to
plan.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-04-22"></a>

Philip&nbsp;M. Johnson and Michael&nbsp;G. Paulding.
 Understanding HPCS development through automated process and
  product measurement with Hackystat.
 In <em>Second Workshop on Productivity and Performance in High-End
  Computing (P-PHEC)</em>, February 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2004/04-22/04-22.pdf">.pdf</a>&nbsp;]
<blockquote><font size="-1">

The high performance computing (HPC) community is increasingly
aware that traditional low-level, execution-time measures for assessing
high-end computers, such as flops/second, are not adequate for
understanding the actual productivity of such systems. In response,
researchers and practitioners are exploring new measures and assessment
procedures that take a more wholistic approach to high performance
productivity. In this paper, we present an approach to understanding and
assessing development-time aspects of HPC productivity. It involves the use
of Hackystat for automatic, non-intrusive collection and analysis of six
measures: Active Time, Most Active File,
Command Line Invocations, Parallel and Serial Lines of Code, Milestone
Test Success, and Performance. We illustrate the use and interpretation of
these measures through a case study of small-scale HPC software development.
Our results show that these measures provide useful insight into development-time
productivity issues, and suggest promising additions to and enhancements of the
existing measures.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-05-01"></a>

Aaron Kagawa.
 Priority ranked inspection: Supporting effective inspection in
  resource-limited organizations.
 M.S. Thesis CSDL-05-01, Department of Information and Computer
  Sciences, University of Hawaii, Honolulu, Hawaii 96822, August 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2005/05-01/05-01.pdf">.pdf</a>&nbsp;]
<blockquote><font size="-1">

Imagine that your project manager has budgeted 200 person-hours for the
next month to inspect newly created source code. Unfortunately, in order to
inspect all of the documents adequately, you estimate that it will take 400
person-hours. However, your manager refuses to increase the budgeted
resources for the inspections. How do you decide which documents to inspect
and which documents to skip? Unfortunately, the classic definition of
inspection does not provide any advice on how to handle this situation. For
example, the notion of entry criteria used in Software Inspection
determines when documents are ready for inspection rather
than if it is needed at all.

My research has investigated how to prioritize inspection resources and
apply them to areas of the system that need them more. It is commonly
assumed that defects are not uniformly distributed across all documents in
a system, a relatively small subset of a system accounts for a relatively
large proportion of defects. If inspection resources are
limited, then it will be more effective to identify and inspect the
defect-prone areas.

To accomplish this research, I have created an inspection process called
Priority Ranked Inspection (PRI). PRI uses software product and development
process measures to distinguish documents that are &ldquo;more in need of
inspection&rdquo; (MINI) from those &ldquo;less in need of inspection&rdquo; (LINI). Some
of the product and process measures include: user-reported defects, unit
test coverage, active time, and number of changes. I hypothesize that the
inspection of MINI documents will generate more defects with a higher
severity than inspecting LINI documents.

My research employed a very simple exploratory study, which includes
inspecting MINI and LINI software code and checking to see if MINI code
inspections generate more defects than LINI code inspections. The results
of the study provide supporting evidence that MINI documents do contain
more high-severity defects than LINI documents. In addition, there is some
evidence that PRI can provide developers with more information to help
determine what documents they should select for inspection.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-05-02"></a>

Philip&nbsp;M. Johnson, Brian&nbsp;T. Pentland, Victor&nbsp;R. Basili, and Martha&nbsp;S. Feldman.
 Cedar - cyberinfrastructure for empirical data analysis and reuse.
 Technical Report CSDL-05-02, Department of Information and Computer
  Sciences, University of Hawaii, Honolulu, Hawaii 96822, May 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2005/05-02/05-02.pdf">.pdf</a>&nbsp;]
<blockquote><font size="-1">

This document presents the project description for a proposal to the National Science Foundation
program on Next Generation Cybertools.  It discusses an approach to integrating qualitative
and quantitative empirical data, approaches to privacy policies, and data management issues
to support collection, analysis, and dissemination of this data.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-05-03"></a>

Hongbing Kou.
 Studying micro-processes in software development stream.
 Technical Report CSDL-05-03, Department of Information and Computer
  Sciences, University of Hawaii, Honolulu, Hawaii 96822, July 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2005/05-03/05-03.pdf">.pdf</a>&nbsp;]
<blockquote><font size="-1">

  In this paper we propose a new streaming technique to study software
  development. As we observed software development consists of a series of
  activities such as edit, compilation, testing, debug and deployment etc.
  All these activities contribute to development stream, which is a
  collection of software development activities in time order. Development
  stream can help us replay and reveal software development process at a
  later time without too much hassle. We developed a system called Zorro to
  generate and analyze development stream at Collaborative Software
  Development Laboratory in University of Hawaii. It is built on the top of
  Hackystat, an in-process automatic metric collection
  system developed in the CSDL.  Hackystat sensors continuously collect
  development activities and send them to a centralized data store for
  processing. Zorro reads in all data of a project and constructs stream
  from them. Tokenizers are chained together to divide development stream
  into episodes (micro iteration) for classification with rule engine. In
  this paper we demonstrate the analysis on Test-Driven Development (TDD)
  with this framework.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-05-05"></a>

Philip&nbsp;M. Johnson.
 A continuous, evidence-based approach to discovery and assessment of
  software engineering best practices.
 Technical Report CSDL-05-05, Department of Information and Computer
  Sciences, University of Hawaii, Honolulu, Hawaii 96822, June 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2005/05-05/05-05.pdf">.pdf</a>&nbsp;]
<blockquote><font size="-1">

This document presents the project description for a proposal to the National Science Foundation.
It discusses an approach that integrates Hackystat, Software Project Telemetry, Software
Development Stream Analysis, Pattern Discovery, and Evidence-based software engineering to
support evaluation of best practices.  Both classroom and industrial case studies are proposed
to support evaluation of the techniques.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-05-06"></a>

Philip&nbsp;M. Johnson.
 Readings in empirical evaluation for budding software engineering
  researchers.
 Technical Report CSDL-05-06, Department of Information and Computer
  Sciences, University of Hawaii, Honolulu, Hawaii 96822, July 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2005/05-06/05-06.html">.html</a>&nbsp;]
<blockquote><font size="-1">

Provides links to resources for empirical software engineering evaluation.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-05-07"></a>

Philip&nbsp;M. Johnson.
 Telemetry plate lunch contest results.
 Technical Report CSDL-05-07, Department of Information and Computer
  Sciences, University of Hawaii, Honolulu, Hawaii 96822, July 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2005/05-07/05-07.html">.html</a>&nbsp;]
<blockquote><font size="-1">

The "Telemetry Plate Lunch Contest" was a contest to support investigation of the use
of multi-axis telemetry charts in Hackystat. This document describes the winning submissions.

</font></blockquote>
<p>
</p>

<p><a name="csdl2-05-09"></a>

Christoph Lofi.
 Continuous GQM: An automated framework for the goal-question-metric
  paradigm.
 M.S. Thesis CSDL-05-09, Department of Software Engineering,
  Fachbereich Informatik, Universitat Kaiserslautern, Germany, August 2005.
[&nbsp;<a href="http://csdl.ics.hawaii.edu/techreports/2005/05-09/05-09.pdf">.pdf</a>&nbsp;]
<blockquote><font size="-1">

Measurement is an important aspect of Software Engineering as it is the
foundation of predictable and controllable software project
execution. Measurement is essential for assessing actual project progress,
establishing baselines and validating the effects of improvement or
controlling actions.

The work performed in this thesis is based on Hackystat, a fully automated
measurement framework for software engineering processes and
products. Hackystat is designed to unobtrusively measure a wide range of
metrics relevant to software development and collect them in a centralized
data repository.  Unfortunately, it is not easy to interpret, analyze and
visualize the vast data collected by Hackystat in such way that it can
effectively be used for software project control.

A potential solution to that problem is to integrate Hackystat with the GQM
(Goal / Question / Metric) Paradigm, a popular approach for goal-oriented,
systematic definition of measurement programs for software-engineering
processes and products.  This integration should allow the goal-oriented
use of the metric data collected by Hackystat and increase its usefulness
for project control.  During the course of this work, this extension to
Hackystat which is later called hackyCGQM is implemented. As a result,
hackyCGQM enables Hackystat to be used as a Software Project Control Center
(SPCC) by providing purposeful high-level representations of the
measurement data.

Another interesting side-effect of the combination of Hackystat and
hackyCGQM is that this system is able to perform fully automated
measurement and analysis cycles. This leads to the development of cGQM, a
specialized method for fully automated, GQM based measurement programs.  As
a summary, hackyCGQM seeks to implement a completely automated GQMbased
measurement framework. This high degree of automation is made possible by
limiting the implemented measurement programs to metrics which can be
measured automatically, thus sacrificing the ability to use arbitrary
metrics.

</font></blockquote>
<p>
</p>