Technology’s Impact on Student Achievement

There is growing evidence that technology is having a positive impact on student achievement in FCPS.  There was some evidence of impact in the 99/00 DTA.  Since then, the district has implemented a number of new initiatives that appear to be increasing that impact.  Based on the criteria that Sun Associates established for measuring student achievement impact, the district’s score improved from 2.0 to 2.9 on a scale of 1 to 4.  The score was arrived at by reviewing student impact data from the survey, observations, and focus groups and then matching those to the same rubrics that were used last year.  See the rubrics in appendix 1.  According to Sun, any score 2.0 or above should be considered “passing.”  So this increase is quite good in comparison with the 99/00 score. 

Figure 4.  Student Impact 

The rubric score provides an overall measure of impact.  However, it is equally important to measure the effectiveness of specific uses of technology.  The effectiveness of technology is not determined by what you have, but by how it us used.  Technology is a tool – much like a book, a pen, a notebook, a globe, or a meter stick.  Those tools do not affect student achievement by themselves.  Student achievement is affected by how those tools are used by students and teachers.  

With that in mind, we have undertaken to measure how technology is used by students and teachers in regard to specific applications.  One such application is a program called Read 180, for grades 4-12.  The purpose of Read 180 is to help 60 of the lowest performing students in a school to improve in the areas of: 

• Reading decoding,

• Reading fluency,

• Creating mental images of text that is being read,

• Vocabulary,

• Relating text to content areas, and

• Motivation to read.

It combines individualized computer-based instruction with group instruction and independent reading.  With the help of a Technology Literacy Challenge Fund grant, Read 180 was first piloted in ten Fayette County Schools in 2000.  It has been incrementally expanded during the 2000/2001 and 2001/2002 school years, using other funds.  Currently, the following schools are using the program: 

All Read 180 students are evaluated using the Scholastic Reading Inventory (SRI). Test data from the 2000/2001 school year show improvement of Read 180 students relative to the general population of students at the district and school levels. The following chart shows that Read 180 students are gaining on SRI scores at just over twice the rate of typical students. 

Read 180 research in other school districts has revealed improvements in attendance, social behavior, and attitudes toward school.  Those results are commonly found among technology-based instructional programs.  Anecdotal evidence about Read 180 within our district supports those findings. 

Examples of individual school test scores among Read 180 students are as follows: 

To determine the long-term effects of the program and how well students’ reading skills carried over into other classes, data were collected on a sample of students who participated in Read 180 at Cardinal Valley Elementary last year, and then went on to Leestown Middle School this year.  Prior to improving their reading achievement with Read 180, those students would have been expected to make mostly failing grades.  But follow-up data showed that they were making mostly B’s and C’s.  As seen in the following chart, their SRI scores dropped slightly over the summer, as do most students’, but then quickly rose again.  Their August SRI NCE Scores averaged 33, and their January SRI NCE Scores averaged 39.

Figure 5.  Read 180 Follow-Up  

Considering the positive results in reading achievement realized so far, we hope to implement Read 180 in more schools in the future.  More extensive data on national results are available at the following website: http://teacher.scholastic.com/read180/discover/research/results.htm#94.  Those data suggest that our students will continue to make significant progress in reading. 

The purpose of Read 180 is to help students in later grades catch up with reading deficiencies that developed in earlier grades.  But schools are making efforts to improve reading instruction in earlier grades, to prevent those problems later on.  One successful technology-based approach is a new program called Reading for Meaning (RFM).  It is designed to improve five specific comprehension skills: 

1. Main idea

2. Inference

3. Sequence

4. Cause and effect

5. Compare and contrast

The program is unique in that it is web-based, instead of software based.  That is, the lessons can be accessed via the Internet from any computer without having to install software or deal with software-related technical problems.  It is password protected, so only schools that have paid a subscription fee have access.  It is also unique in that it was designed to be used with large groups, small groups, and individual students.  In large group instruction, students have the opportunity to interact with each other, so the program can be more engaging and promotes positive interactions.

RFM was piloted in several elementary schools this year, and some results have been impressive.  For example, in one school, 2^nd and 3^rd graders used the program for two months, and improved their SRI Lexile scores from 443 to 547.  That took them from a “basic” to “proficient” range.  Their NCE scores improved from 44 to 51, so they went from below average to average.  Results in some other schools have been good, but not as good as those.  We will continue to pilot the program and pinpoint factors that will make it more consistently effective.  Since the initial Internet-based version came out, a new CD-ROM based version was released and the district has negotiated a very low price of $20 per copy.  So RFM is potentially cost effective as well as instructionally effective.

The Kentucky Board of Education has adopted a recommendation that all students in grades 4-12 have email addresses by 2003 and that Internet access be increased.  Based on survey data gathered in Fayette County, it seems that the increased level of access can have a positive impact on students.  For example, on one survey of middle school students, it was found that 47% of parents had used the Internet to contact their child’s teacher.  76% of surveyed middle school students had Internet access at home and 58% had accessed their school’s website from home or somewhere else besides school.  There is clearly a greater level of access among upper income families than lower income families, but the gap seems to be closing.  For example, Bryan Station High School freshmen were surveyed and it was found that 72% had Internet access at home and 73% of their parents used email at home or at work.  All of this has positive implications for the district because it shows that schools have a new means at their disposal for communicating with parents and students outside of school.  Based on the DTA survey, only 37% of teachers reported they had used email with students, so there is clearly potential for growth in that regard. 

In a review of research on the use of computers for testing, Russell and Haney (2000) found that students using computers may write better than students writing by hand.  According to Russell and Haney: 

Eighth grade students from two middle schools in Worcester, MA, were randomly assigned to groups.  Within each subject area, each group was given the same test items, with one group answering on paper and the other on computer.  In addition, data on students’ keyboarding speed and prior computer use were collected.  Finally, all answers written by hand were transcribed to computer text. 

Large differences were evident on the language arts tests.  For students who could keyboard moderately well (20 words per minute or more), performance on computer was much better than on paper.  Overall, the difference represented more progress than the average student makes in an entire year and could raise a student’s score on the MCAS from the “needs improvement” to the “passing” level. 

Recalling that nearly ten million students took some type of state-sponsored written test last year and that nearly half of the students nationwide use word processors in school, these results suggest that state paper-and-pencil tests may be underestimating the abilities of some five million students annually. 

That study has implications for writing among students in FCPS, especially considering the potential for improving our students’ writing skills and test scores.  As with any study, there are certain limitations.  First, it dealt only with 8^th graders, so it is difficult to say whether the same would hold true for other grade levels.  Also, it dealt with the state assessment in Massachusetts, which is not the same as the state assessment in Kentucky.  Notwithstanding those limitations, it is clear that Kentucky students are required to write significant amounts for our state assessment.  Also, the Kentucky Department of Education recommends beginning keyboarding instruction in the 3^rd grade, so students can be better prepared for the demands of writing at an early age. 

With all that in mind, FCPS began a more intensive push to improve keyboarding skills this year, starting in elementary schools.  Keyboarding proficiency in important for our students, as a prerequisite for writing with computers.  A summary of the district’s approach to keyboarding instruction can be found at: http://teach.fcps.net/keyboarding.htm.  In one sample of 4^th graders, students averaged 7 words per minute (wpm) at the beginning of the school year and improved to 13.9 wpm by February.  If they continue to improve at that rate, they could be expected to reach the goal of 20 wpm before they enter middle school.  Unfortunately, keyboarding scores from most schools have not been as positive.  That seems to be due to at least two factors: (1) lack of consistency in keyboarding from school to school and (2) lack of opportunities for students to continue practicing keyboarding after their initial instruction.  Considering our schools’ low writing scores and the potential positive impact that computers can have on writing, it is clear that schools need to place more emphasis on keyboarding and computer-based writing. 

Another way to view technology in terms of student achievement is by exploring the relationship between schools’ performance on CATS and their use of technology.  At the University of Kentucky, Allard and White (2000) conducted a study to compare CATS scores with data from library media reports.  They examined certain characteristics found in the media reports, and how prevalent those characteristics were among the schools scoring in the top 20% and the bottom 20% on CATS.  Parts of that report deal specifically with technology.  Related to technology, Allard and White found that: 

• Schools in the top are much more likely to have Student Technology Leadership Programs than those in the bottom.

• Generally, the top schools at all levels share the characteristic that they have a higher level of technology to offer students than the bottom schools.

For Fayette County’s DTA, we conducted a similar comparison.  Using CATS scores provided by KDE, all of the schools in the district were placed in rank order according to the amount of improvement or loss they demonstrated in CATS from 99/00 to 00/01.  Those are the most recent scores available.  The top half of the list may be thought of as “high improvement schools” and the bottom half as “low improvement schools.”  Technology-related characteristics of schools in both groups were identified, and they were all characteristics that were in effect during 99/00 and 00/01. 

The first factor is “participation in the PET Project.”  PET Project is an intensive two-day professional development in which a team of teachers from a school collaborates with Technology Resource Teachers on developing a standards-based unit of study that incorporates significant amounts of technology.  The focus of the PET Project is on ensuring that the teachers develop an instructionally sound unit and then infuse that unit with what are believed to be effective uses of technology.  One requirement of PET Projects this year is that they must place a significant emphasis on literacy and helping to close the achievement gap.  The following table shows how schools compared: 

Table 5. PET Project Participation 

High improvement schools were more than twice as likely as low improvement schools to participate in the PET Project.  That seems to indicate that schools that intentionally concentrate on integrating technology into their instruction are more prone to show improvement in CATS. 

High schools have not participated in the PET Project as much as middle and elementary schools.  Yet, data from this DTA show that they are lagging behind their middle and elementary school counterparts in classroom use of technology.  One reason for the lower level of participation in the PET Project may be that high school teachers have less opportunity to collaborate with each other, so the collaborative nature of PET is not as useful to them.  So, plans are in the works to offer a slightly different variation of PET Project for high school teachers next year.  The focus will still be on unit development and having teachers work together along side TRTs and curriculum specialists, but the unit they develop will be designed for use in just one classroom.  Also, the project will concentrate on social studies teachers, so there will be a more narrow focus with more extensive support for participating teachers. 

The second factor is the presence of a Technology Resource Teacher (TRT).  The district employs a cadre of 12 instructional TRTs who are each assigned to several schools.  Data on the TRTs indicates that they are doing an outstanding job of providing instructional technology support to teachers.  As of March 2002, TRTs had provided 112 workshops for teachers during the 01/02 school year.  Their average rating for those workshops was 4.8 on a scale of 1 (worst) to 5 (best).  In addition to those workshops, they work directly with teachers in their classrooms during the school day.  That approach of professional development that is “on the job, during the course of the school day, in the midst of working with students” was cited by Commissioner Gene Wilhoit as the most valuable form of professional development (Wilhoite, 2001). 

In the most recent consolidated plans from schools, the vast majority of schools requested TRTs.  The district is making a strong effort to provide those services, but some schools have also found ways to fund a full-time TRT position out of their own staffing allocation.  The following table shows how those schools compared. 

Table 6. TRTs in Schools 

High improvement schools were more than twice as likely to have an instructional TRT than low improvement schools.  That seems to indicate that schools with a full-time instructionally oriented TRT are more prone to show improvement in CATS. 

In summary, it is important to note that this comparison only shows a relationship.  It does not show causation.  Taken individually, these data on student impact may not be very meaningful.  But taken collectively they do seem to demonstrate that (1) effective use of technology goes hand in hand with student achievement, and (2) our schools are making notable improvement in their use of technology.