Stage
3 - Recommending and Reporting
Once data has been collected, it is necessary to apply the analysis
of the data to the process of scoring the related evaluation rubrics. This
step, plus those of creating findings and recommendations and then
creating and disseminating the evaluation report constitute the
third and final stage of the district evaluation process.
The district evaluation committee should work to holistically
score the rubrics. Traditionally, Sun Associates facilitates
this process and often have created our own set of scores to compare
with and challenge the committee's scores. The scoring process
takes the form of a structured (facilitated) discussion of the
data compared to the rubrics. In most cases, this takes
at least a half-day committee meeting. Some districts find
it useful to create scores that break down by building or level
(e.g., elementary, middle, or high). Whether or not this is done
is a decision that must be made by the district before such
scoring occurs since such breakdowns (if any) must relate to the
overall purpose and design of the district's evaluation project.
At this point in the work, it is also now reasonable to create
a series of findings and related recommendations. In fact, findings are
summaries of the key data findings. These findings are usually
directly related to the evaluation questions and are thus organized. In
some cases, there are findings that are opportune and bear noting
even though they are not related to the questions. When
this happens, the opportune findings need to be reported as such
and clearly noted as being in addition to that information that
was being directly sought by the evaluation.
Recommendations are the responses to the evaluation
findings and subsequent rubric scores. Here, the point is
for the committee to detail what it believes is necessary to address
the evaluation findings and to insure/drive positive future
progress. This last point is by no means minor. The
making of recommendations is often a laborious process that has
strong political overtones. Once again, the credibility - and
thus value -- of these recommendations will be enhanced by the
broad stakeholder representation on the committee and by the knowledgeable
resource/support provided by an external evaluator.
The final process step -- reporting -- is important in that it
reinforces the basic point of the evaluation, that is to provide
a structured assessment for fine-tuning and improving progress
towards meeting the district's technology implementation goals. If
findings are not reported, then the assessment has no value. Evaluation's
benefit really occurs when its results are published widely throughout
the district. Such publishing expands and informs
the discussion and reflection processes that started in the evaluation
committee meetings.
What Happens Next?
The release and presentation of the "final" evaluation report
is not the end of the evaluation process. In order for evaluation
to have a formative impact on a district's technology implementation
effort, the first year's evaluation process must be continuously
applied. Ideally, the district will re-evaluate its progress
annually. A good example of a district which has taken this
annual approach, with positive results, is found in Fayette County,
KY. Visit their website directly
to read their annual
reports and review their evaluation process.
Our experience is that when a district takes this truly
formative, annual, approach to its technology evaluation there
is an increasing level of buy-in and relevance afford to the evaluation
with each cycle of the evaluation process. For one thing,
when evaluation becomes a recognized part of the technology planning
and implementation process, some of the fear and anxiety of evaluation/assessment
subsides. Also,
routine evaluation indicates to the broad district community that
technology is an important aspect of the district's expectations
for teaching and learning. In short, technology use, expectations
for use, and assessment of progress toward meeting those expectations
becomes a systemic part of the district's educational environment. This
is, after all, the ultimate goal for instructional technology,
and we have indeed found that formative technology evaluation is
a significant step toward achieving that goal.
|
