viernes, 23 de octubre de 2009

Data collection instruments

The data collected for this action research project was large in scope and it was collected using data collection instruments that included a pre-test, a post-test, a pre-survey, a post-survey, students’ portfolios which contained the students’ work and results in the different inquiry-based tasks, and teacher field notes.

The sources of data for this action research project were the students and me, the science teacher. In order to get data from students, I applied tests and surveys and asked students to keep portfolios that included the whole collection of the students’ work along the intervention (Hopkins, 2008). The data from me, the teacher researcher, came especially from field notes that I took throughout the research project.

The pre-test and the post-test were administered and scored in a consistent manner. Both tests were designed in such a way that the questions, the test conditions, and the scoring procedures and interpretations were administered and scored in a predetermined, standard manner so that they were valid and relevant. The tests were composed of seven multiple-choice questions and three open questions about the most important issues related to the systems of the human body. In order to score the tests, criterion-referenced score interpretation was used in which interpretations compared test-takers to a criterion or a formal definition of the scientific content, regardless of the scores of other examinees (Brown, 1990).

The content validity of these two standard achievement tests was evaluated by making sure that the test items matched the instructional objectives proposed in the science program for fourth grade and the content contained in the unit about the systems of the human body in the students’ Scott Foresman Science textbooks for this grade. Internal consistency of the tests was high as the questions were taken from the Scott Foresman Science series for grade four which means that they were written by experienced science teachers, pretested, and selected on the basis of the results of a quantitative item analysis (Foresman, 2004)..

The pre-survey and the post-survey that were used to collect data about students’ progress in terms of inquiry, learning styles and learner autonomy tended to be strong on validity or the degree to which the study accurately reflected or assessed these specific concepts that I was attempting to measure (Burns, 1999), and also tended to be strong on reliability or the extent to which these instruments yielded the same result on repeated trials.

The survey format put a strain on validity since the students’ real feelings were hard to grasp in terms of such dichotomies as "totally agree/disagree," "neutral,” “disagree,” “totally disagree” etc., these were only approximate indicators of what I had in mind when I created the survey questions. Reliability, on the other hand, was a clearer matter. The two surveys presented all students with a standardized stimulus, and so they went a long way toward eliminating unreliability in my observations (Schwalbach, 2003). Careful wording, format, content, etc., also reduced significantly the students’ own unreliability.

In general, the surveys were reliable because they measured things consistently and they were valid since they measured what they said they were measuring.

The field notes used allowed me to make a written account of the inquiry-based lessons about the systems of the human body that were taught to this group of forth grade students while they were taking place. The notes were taken as soon as the events were happening in these science sessions so that the information collected was fresh and not distorted (Jimenez, 2006). The field notes were in the form of a record of work (Burns, 1999) of the students in the science lessons in which information such as date, time, class, objectives, work done, way the work was done by the students, homework, participation, things that worked well, and things that did not work very well were all included. This record-keeping provided valuable data about the intervention to draw conclusions about how the inquiry-based learning unit increased the level of autonomy awareness in this group of forth grade students thus contributing to validity and relevance.

When the students’ portfolio was created, I made sure that it was appropriate and fair for the students as this was essential in terms of validity and reliability. Validity and reliability were considered for both the individual pieces and the entire portfolio and it was evaluated for these traits since the portfolio was a collection of students’ work and assessments. Both the audience and the purpose of the portfolio played a major role in determining content validity (Freeman, 1998). This instrument had high content validity because it was developed for a specific science class working toward a certain purpose and specific objectives.

To evaluate the content validity I asked myself if the portfolio matched the instructional purpose and objectives of the inquiry-based project and whether the portfolio assessed what I set out to assess from the beginning. To ensure content validity, I set the purpose of the portfolio in line with project objectives, matched the contents of the portfolio to the purpose, and established clear criteria in relation to the original objectives (McFarland, 1997). In general, the students’ portfolios had high content validity because they integrated instruction and assessment as the work that students produced in the classroom and at home showed improvement in terms of autonomy awareness.

1 comentario:

  1. Hi there, nice post and a good source of information. It really shows that you're an expert in this field. I'm looking for some data collection methods in this site. Thanks for sharing. Keep it up!