e-Learning for Educators Research Studies Results
Spring 2010

Download the Power Point Presentation:
FULL RESEARCH RESULTS

The findings from the research studies conducted within the e-Learning for Educators (efE) grant provide strong evidence that high-quality¹ online professional development courses can positively affect teacher content knowledge, teaching practices and student achievement.

Introduction

In fall 2005, Alabama Public Television received a Ready to Teach grant from the US Department of Education.  The grant set aggressive goals that would, if met by the ten collaborative state partnerships, result in a sustainable, systematic delivery of online professional development (OPD) for teachers, particularly those in high-needs (Title I) schools. The e-Learning for Educators research efforts focused on assessing the effects of OPD on three areas: (1) improvements in teacher content knowledge; (2) improvements in instructional practice; and (3) subsequent improvements in academic achievement for students. Assessing improvements required the assistance of a team of researchers and evaluators from inTASC, Boston College’s Technology Assessment and Study Collaborative Center.  Education Development Center (EDC) developed twelve ELA and math online courses used for the research series and provided the training for facilitators who delivered the workshops to teachers participating in the research study.  

Project Research

Boston College (BC) led the rigorous research studies that have been conducted by the e-Learning for Educators (efE) project. The efE research studies were aligned with the goals and priorities of both the Ready to Teach grant and the US Department of Education, some of which are referenced in the previous paragraphs. The efE studies sought to (1) address the limitations and shortcomings noted about prior research of online education in general and online professional development particularly, and (2) focus on teacher outcomes as well as student outcomes. Prior studies had limitations such as small sample sizes and lack of random assignment.  In addition to the shortcomings of existing research, few longitudinal (long-term) studies existed.  For the most part, previous studies assessed “one shot” online courses that simply measured pre-post course content knowledge for one course or were conducted within higher education (university/college) settings rather than K-12, and none focused on how K-12 teacher OPD might affect student achievement.

Study Sample, Organization and Process

To begin the implementation of the rigorous research effort, a series of three online professional development courses for each grade level were collaboratively developed and reviewed among the project partners and by experts in the content areas. The course development process was led by EDC a recognized leader in the field of online course development. Together EDC course developers and BC researchers created a series of customized instruments to assess participant learning in each online course.

Beginning in spring 2007 and continuing through summer 2009, 369 teachers and 21,217 students participated in the research studies. Originally, the project sought to recruit entire schools and conduct the research with a school model; however, recruitment for the school model proved to be extremely difficult so the project shifted to an individual teacher recruitment model. To encourage completion of the three sequential courses and related data collection, teachers were paid small stipends. Additionally, teachers were offered graduate credit associated with the OPD workshops and teachers randomly assigned to the control group were allowed to participate in the workshops once the research study completed.

With teachers from 21 states participating, the scale of participation was large and the quantity of data collected and analyzed was extremely large. For example, more than 21,500 open-ended responses were analyzed in addition to hundreds of multiple-choice questions. There were no existing systems that could capture the types and amount of data that this research effort required so Boston College designed an online data collection system specifically for the e-Learning research effort. Ultimately, teachers completed pre and post surveys, as well as administered student surveys and tests over a period of three semesters.

The study sample size (n) varied according to grade level. In total, data from 315 teachers and their students were used to examine the effect that the online professional development courses had on teacher knowledge, teacher practices, and student achievement. The distribution of teachers and students are listed in Table 1. Despite attempts to retain all teachers (and their students) in the studies, some teachers withdrew from the studies due to circumstances beyond their control such as change in teaching assignment (grade, school, or content area) and other personal circumstances.

The 4th and 7th Grade English Language Arts (ELA) research focused on teacher and student outcomes in the areas of vocabulary, writing and reading/comprehension. The 5th Grade Math research focused on teacher and student outcomes in fractions, algebraic thinking and measurement. The 8th Grade Math research focused on teacher and student outcomes in proportional reasoning, geometric measurement, and functions.

Table 1: Study Participation Group Size by Grade and Assignment (n = number of participants)     

Grade/Subject	Teachers		Students
	Experimental Group (n)	Control Group (n)	Experimental Group (n)	Control Group
4th Gr. ELA	49	61	766	922
5th Gr. Math	34	45	648	790
7th Gr. ELA	35	49	831	1225
8th Gr. Math	28	43	799	1090
Total	133	182	3109	4330

Table 2: Research Study Courses: Grade, Subject, And Topic

Grade/ Subject	Course 1	Course 2	Course 3
4th ELA	Best Practices for Vocabulary  Instruction in the  Elementary   Classroom	Promoting Reading  Comprehension  Skills in the  Elementary Classroom	Teaching Writing in  the Elementary  Classroom
5th Math	Using Models to  Understand  Fractions	Algebraic Thinking in  Elementary School	The Complexities of  Measurement
7th ELA	Best Practices for Vocabulary  Instruction in the  Middle School Classroom	Promoting Reading  Comprehension  Skills in the Middle  School Classroom	Teaching Writing in  the Middle School  Classroom
8th Math	Proportional  Reasoning	A Conceptual Introduction to  Function: Using  Visual  Models	Geometric  Measurement

Data Analyses

Data were collected from all teachers and students in the studies; however, only data from teachers and students who completed both pre and post course surveys were analyzed. Data were subjected to appropriate statistical tests (ANOVA, ANCOVA, HLM²) to determine both statistical impact and the practical educational impact (effect size). These statistical analyses examine whether a measurable effect on teachers or students was happenstance or is a result of taking the OPD courses.

Within each grade level, data focusing on two separate topics have been collected: (1) teachers’ and students’ knowledge in two instructional content areas (math, English), and (2) their practices (the teacher’s instructional methods, strategies and techniques in teaching specific content; students’ practices in computation, writing, etc.)

Study Findings  

These study results demonstrate that the e-Learning courses and process can have a positive effect on teacher knowledge, teacher instructional practices, and, most importantly, student achievement. Overall, both teachers and students made statistically significant³ improvements. In other words, there is some evidence of a cause-and-effect change due to the OPD instruction; improvements did not occur by chance.

Teachers that completed the series of three OPD courses made strong improvements in content knowledge and classroom teaching practices.  Improved content knowledge and improved teaching resulted in students in some grade levels having improved scores on tests in the content areas, such as proportional reasoning or vocabulary, among others. Table 3 depicts the composite results of all workshops from each grade level.

This document does not present information about the strength of effects; however, there were notable impacts for teachers in all grade levels. The impact on teachers was strongest in 5th grade math but there were also very positive effects on some 8th grade math content areas such as proportional reasoning and geometric measurement  – two areas that are frequently noted as being very difficult to teach and very difficult for students to master.  Further data analysis is ongoing to determine further results and to disaggregate data among high needs schools versus all schools or non-high needs schools, etc.

Table 3: Composite Results - Statistically Significant Changes

	4th Grade English Language Arts	5th Grade Math	7th Grade English Language Arts	8th Grade Math
Composite Results for Students	+	+	+	*
Composite Results for Teachers	+	+	+	+

+ indicates statistically significant results.
*  indicates that results vary depending on the statistical analysis; some tests show      statistically significant results.

Summary and Future Implications

The e-Learning for Educators project partners and management team are very pleased with these research study results to date. We anticipate that further findings will also be positive.

The research studies’ design, duration and audiences are groundbreaking and should provide definitive evidence that OPD courses can help improve teaching and more importantly, student learning. These results should inform educational decision makers and guide policymakers as they consider increasingly scarce public and private investments in programs, projects, and activities that will yield the most effective results. These findings and other evaluation results will serve as evidence of success as
e-Learning project partners seek state budget allocations and line-item program funding, innovation grants, and development contributions that will extend this work to serve additional teachers in the ten efE project states and elsewhere.

We expect that these data and findings will be widely published in peer-reviewed research journals, educational magazines, and in state education association newsletters. As further analyses and additional results become available, they will be publicized. Final results will be included with the final report of the e-Learning for Educators project at the close of the grant.

For additional information about the research studies or the e-Learning for Educators project, please contact Lynne Meeks at (205) 380-5158 or at lmeeks@aptv.org.