Assessment
ASSESSMENT OF SOFTWARE SYSTEM DOCUMENTATION PROCESS
To carry out the assessments we have use an assessment questionnaire, whose
purpose is to determine where an organization’s documentation process stands
relative to the model. The assessment questions are derived directly from
the model and its key practices. There are one or more questions
for each key practice. The process maturity level (for the first
three versions) and the degree of key practice satisfaction are determined
from the questionnaire responses. For Version 1 the key practices were
rated as Not Satisfied, Partially Satisfied, or Fully Satisfied.
This rating system seemed ambiguous and too coarse as there was a wide
latitude in the degree of key practice satisfaction especially when a practice
was classified as Partially Satisfied. For example, does Partially
Satisfied mean the key practice is satisfied seldom or often or half
of the time? Not Satisfied and Fully Satisfied were
also ambiguous. Does Not Satisfied mean never or most of the
time it is not satisfied? Does Fully Satisfied mean always
or most of the time? To eliminate these ambiguities, from Version
2 on there was a change from three to five degrees of satisfaction as follows:
Very
High, High, Medium, Low and Very Low.
Each person on a project team completed the assessment questionnaire,
The degree of key practice satisfaction was generally determined by the
average of the responses to the questions associated with that key practices.
Finally, an assessment report is generated from the questionnaire responses.
The report contains an executive summary with the maturity level, a documentation
process maturity profile and an improvement action plan. Besides
the maturity level (only for the first three versions), the executive summary
lists the key practices that were not satisfied, those that need improvement,
and those that were missing. The process maturity profile indicates
the degree of satisfaction of each key practice. Specific actions
to improve existing practices or to address missing practices to move to
the next higher level are described in the improvement action plan.
For full details about the evolution of the assessment procedure click
here
Interpreting the Assessment Report
Recall that the assessment questionnaire is based on the practices in the
model and assessment report is based on the compilation of the responses
to the questionnaire. Hence the assessment report indicates for the model
which documentation practices are being done satisfactorily, which need
improvement, and which are missing in the organization's documentation
process. For the first three versions, It also assigns a maturity level
(1, 2, 3, or 4) and presents the challenges to move to the next higher
maturity level. It has been our experience that in presenting the results
the maturity level tends to overshadow the detailed information about practices
that are being done satisfactorily, need improvement or are missing and
challenges to the move to the next level. What we found to be most beneficial
from the assessment report is the action plan and the ensuing meaningful
discussion among software engineers and managers about documentation and
support for assessment that it generates. Recognition of the importance
of documentation and motivation to do something about it are important
first steps in any process improvement.
A final comment. The qualitative assessments in the report are relative
to the model. This is no different than the assessments for the SEI models.
They are relative to the SEI model.
What to Do After the Assessment Report?
An important question is: what to do about the recommendations in
the assessment report? An obvious answer is to generate an
action plan to implement the recommendations. However, generating
an action plan is a considerable undertaking as important decisions need
to be made, questions raised, issues addressed and support gained.
We have devised a process framework for generating the necessary information
and activities in the formulation of an action plan. For full details
of the proposed framework including a case study click
here.
Last Updated: Thursday, April 13, 2000
by Curtis R. Cook
& Marcello
Visconti
© 2000 OSU Computer
Science Department & UTFSM
Departamento de Informatica
Back to main page