Stage 2 - Data Collection

The data collection stage of an evaluation project includes the two basic process steps of data acquisition and data analysis.

Data collection makes use of tools and techniques such as surveys, observations, interviews/focus groups, review of teacher/student work, and public meetings. All data is keyed to the performance indicators created by the district evaluation committee. For example, if an indicator of high achievement in teacher use of technology is that teachers will use electronic mail to communicate with peers outside of the district, then data is needed which shows the amount as well as qualitative substance of teacher email communications. This might include technical logs (e.g., how often do teachers access their email accounts); teacher surveys to determine how often email is used and for what; and teacher interviews to determine the value placed upon email communication. All of this quantitative and qualitative data is used to determine a level of overall achievement in the indicator rubric. A similar logic would be used to measure achievement with any set of indicators.

Here, it is worth mentioning that while data collection might take place at the individual level of performance, individual data it should never be reported. The mission of a district-wide evaluation is to determine the progress of the district as a group of individuals in meeting its goals. Nothing will undermine an evaluation project faster than the perception that it is measuring or ranking individuals. If individual assessments are important, these should be developed and administered separate from your district technology evaluation.

There are a variety of tools useful for data collection. You might find it useful to view some sample data collection tools such as an online teacher survey, classroom observation template, and teacher focus group questions. These, and other samples are found on our Tools page.

Next Steps

While data collection is the most time-consuming part of evaluation, it is actually not the most important. Rather, all the activity (and expense!) of data collection is validated by how that data is used. Applying your data to the rubrics developed in Stage 1 is the focus of Stage 3 - Recommending and Reporting.

If we already survey our teachers, do we need to create separate data collection instruments?

We have found that most districts already collect a variety of data on student and teacher technology use.   Unfortunately, this data does not often relate to any specific questions or areas of investigation in so far as understanding the relationship between technology and student achievement.   Data collected in absence of questions is simply busy work.   Therefore, the point of data collection in the evaluations we facilitate is to collect data which relates to the district's developed indicators.  

Further, we find that involving an external evaluator in the data collection work (or at least in the initial design of the tools and instruments) will provide a considerably more accurate picture of how teachers and students really use and perceive technology.   Often existing district tools are designed to simply affirm existing policies and beliefs and thus do little to actually collect new information that challenges existing practices.