Engineering schools at Northeastern University, Tufts University, Worcester Polytechnic Institute, and Boston University formed a collaborative to increase the number of girls who develop an interest in science, technology, engineering and mathematics (STEM) fields as well as in STEM careers during the middle school years. The study was funded by the National Science Foundation and researchers from Wellesley Centers for Women served as the external evaluators. The 4 Schools for Women in Engineering (WIE) strategy was to form all-female teams to train teachers and to serve as in-class resources for them. STEM Teams developed gender equitable engineering units to be used in 8th grade science classrooms and helped middle school teachers implement the new units for the engineering strand of the Massachusetts Frameworks. The strategy was implemented in eight public school districts in the greater Boston area. Click here to download the full final report (PDF) of this project.

Evaluation Components

The evaluation focused on how teachers and STEM team members experienced the program and what the impact of the intervention was on students. Teachers’ and STEM Team members’ perceptions were assessed through open-ended anonymous questionnaires completed after the implementation of the intervention. The impact of the intervention on students was assessed through comparing attitudes toward engineering, mathematics, science fields and careers before and after the intervention.

1. Teacher Outcomes.
For the teachers the exposure to expertise from actual engineers, graduate and undergraduate students, and industry representatives coupled with the training that was such an integral part of this intervention. Teachers reported that it made a significant difference in both their understanding of engineering and how to teach this subject effectively. For their students, the benefits teachers highlighted were exposure to role models, glimpses into the real world of work for engineers, and a curriculum that made it possible for all students to achieve up to a benchmark of proficiency rather than a competitive classroom environment where only a few students are engaged in the projects.

2. Impact on STEM Team Members
Each STEM Team was composed of a coordinator, teachers, faculty members, undergraduate and graduate students, and industry representatives. STEM Team members reported they enjoyed the regular meetings that served to keep the teams focused and remarked on the synergy generated by the diversity in team membership. The ability to expose middle school students to a wide variety of practicing engineers was also mentioned as a strength of the program. Several respondents recognized the value of having undergraduate students in the classroom, which not only made some projects possible but was also key in establishing good rapport with the 8th grade students. Some team members also noted the challenges of recruiting and training undergraduates for the classroom. All in all, the study showed that it is feasible to bring together individuals from different domains to work together toward enhancing the teaching of STEM fields, especially the new engineering strand.

3. Impact on Students
The evaluation was carried out by comparing answers students gave to the same questionnaire administered in the beginning of the school year (pre-test) and then at the end of the school year (post-test). The impact of the intervention is reported in terms of general program impact averaged over all students and also in terms of its effect on individual students. The former is important for understanding program level effects. The latter provides information on the ways in which individual students were influenced by the intervention.

Program Level Outcomes

On the whole, students indicated that they found the classroom activities somewhat or very interesting, and this was true both for students who were not particularly interested in pursuing careers in math, science and engineering as well as for those who were. Attitudes toward science were more positive after the intervention, attitudes toward engineering were less positive, while attitudes toward math did not change. The fact that the science attitudes showed a significant increase over the course of the semester speaks to the overall efficacy of the curriculum, which was presented in science classrooms.

Several explanations can address why the engineering attitudes were less positive following the intervention than before it. First, it may be that the use of attitudinal measures does not adequately reflect the actual learning that has taken place in math, science and engineering and that testing actual knowledge may result in more favorable findings. Another explanation is that for some students, a less favorable attitude toward engineering after the intervention may reflect a more realistic appraisal of their thinking about engineering after they became better acquainted with it through the intervention.

Overall, when changes in girls’ attitudes’ toward STEM fields and careers were compared to those in boys’, more girls reported a positive attitude change toward careers in engineering and chemistry than did boys. Also, for girls there was a context effect in that it mattered what school they attended or who their teacher was. For girls, which school they attended (and thereby which teacher taught them, because only one teacher participated in most schools) made a difference in whether the intervention was associated with greater attitude changes toward both STEM fields and STEM careers, whereas it made no difference for boys. These school/teacher context effects must be interpreted with extreme caution as they are based on very small samples.

Individual Level Outcomes

In spite of the fact that students’ overall scores on attitudes toward engineering did not become more positive after the intervention, there were a large number of individual students whose scores increased – that is, their attitudes toward STEM fields or careers became more positive – between pre-and post-test. We examined the impact of the intervention on individual students using regression analyses which yield information on what variables best predict whether a student’s scores will increase or decrease. The results show that students who started out with lower positive attitudes toward both STEM fields and STEM careers made more gains after the intervention than students who started out with more positive attitudes. The increase in positive attitudes among those who started out with less favorable attitudes may be due to a statistical artifact, that of “regression to the mean,” or a “ceiling effect.” However, in other STEM interventions, there have been findings of larger effects among participants who were positively predisposed to the focus of the intervention. Therefore, at least some of the positive impact of the intervention in this study is likely to be due to the impact of becoming exposed to ideas and experiences by students who would not choose to expose themselves to STEM but became exposed as part of a school curriculum.

The intervention had a greater impact on female students’ attitudes toward engineering as a field of study as well as choosing engineering as a career than it did on male students’ attitudes.

We interpret the finding that the intervention had the most influence on girls’ attitudes (both positive and negative) toward engineering as a field and a career in terms of girls’ openness to new information on engineering. We believe that this openness is due to not having much information on engineering before the intervention rather than to a general openness. Indeed, girls’ attitudes were influenced at the same level as boys’ on biology and chemistry as fields of study and careers, which are better known fields. We believe that the intervention was able to influence girls more because, due to a lack of previous experience or knowledge, girls had fewer preconceived notions about engineering. This interpretation is bolstered by the finding that before the intervention fewer girls reported knowing engineers than did boys.

Conclusions

The evaluation shows that the 4 Schools for WIE project had the intended outcomes:
• Capacity building among 8th grade science teachers’ for increasing their confidence to teach engineering;
• Demonstration of the feasibility of forming well-functioning STEM teams that include middle school teachers, academics, graduate and undergraduate students, and industry representatives;
• Program level increase in positive attitudes toward science as a field of study (but not in engineering nor mathematics);
• Individual level impact of the STEM Team strategy on girls’ attitudes toward engineering both as a field of study and as a career to a greater extent than on boys’ attitudes.

The intervention was associated with greater improvement in the attitudes of students who started out with relatively low scores on attitudes toward STEM fields and careers than those who started out with more positive attitudes, which was true for both girls and boys. This suggests that the intervention has to potential to benefit those who hold less positive views of STEM rather than merely making students who already hold positive attitudes even more positive.

All in all, the STEM team intervention succeeded in its goal to improve teachers’ capacity to teach the new engineering strand in the 8th grade science curriculum. It succeeded partially in terms of improving students’ attitudes toward STEM fields: all students’ attitudes toward science as a field of study improved, but not toward engineering or mathematics. Perhaps most significantly, it resulted in more girls than boys coming to view engineering and chemistry as possible careers for themselves and had a larger impact on girls than it did on boys. Moreover, the context of school or teacher made more of a difference in changes in girls’ attitudes toward engineering and mathematics as a field of study and engineering as a career.