Hear Our Voices (HOV), a girls-only program directed by the Computer Clubhouse at the Boston Museum of Science, was designed to provide girls with access to technology resources, female mentors and role models, a community of peers, and a positive learning environment. The program’s goal was to improve girls’ attitudes toward computers so that computers can have a positive impact on their lives. Working on challenging programs, having a positive attitude toward computers, and viewing computers to be relevant to their future were viewed as indicators of girls’ positive approach to computers and technology.

To this end, with funding from the National Science Foundation and researchers from the Wellesley Centers for Women as external evaluators, the Flagship Computer Clubhouse at the Boston Museum of Science put out a competitive Request for Proposals to other Computer Clubhouses to launch 10 HOV programs each year for two years. The funding was used primarily to support the hiring and training of staff to develop locally-based HOV programs at the Clubhouses that also met the program requirements of HOV. Prominent among the program requirements were hiring an HOV coordinator who is employed a minimum of 10 hours a week, attends an off-site orientation meeting, and participates in monthly teleconferences with the other coordinators and the program manager; providing a minimum of 100 hours of girls-only time over the course of a year that incorporates programming that is likely to interest girls; recruiting girls from underserved communities; recruiting and supervising mentors to work with the girls; participating in the Computer Clubhouse Network via the Computer Clubhouse Village Intranet site; and developing a plan to sustain HOV after the initial federal funding period. With professional development and programming support from the Flagship organization, each participating Clubhouse was to tailor their HOV program to meet local needs during the hours set aside for girls-only programming.

In 2003, the initial year of operation of HOV, 10 programs were funded based on their demonstrated understanding of the gender equity issues, soundness of their plan to recruit girls and female mentors, and sensitivity to programming choices designed to appeal to girls. The girls-only programs funded in 2003 served as pilot sites for developing and field testing data collection instruments for the evaluation.

In the second year of operations the Flagship Clubhouse let out an RFP to select 10 additional sites. With the continuation of funding for the original 10 sites, 20 girls-only programs were to operate in the fall of 2004. However, due to circumstances that were unrelated to HOV, several sites had to cease operations, reducing the total number of sites available for evaluation to 15. An additional site was dropped from the evaluation because of irregularities in the data submitted, leaving 14 sites to be evaluated.

Evaluation Design

An evaluation was planned to document (1) the process of the operation of HOV sites over time that captured a record of the anticipated and unanticipated occurrences in the operation of HOV at the different sites; (2) site level outcomes that reflect program implementation elements that contributed to the success of HOV; and (3) the impact of HOV on individual girls’ attitudes toward computers, including a positive impact on their lives through computers. The process and outcome evaluations relied on Quarterly Summary Reports submitted by HOV coordinators. The outcome evaluation also made use of site visits and survey data collected from the girls between September and October 2004 (Time 1) and approximately seven months later between April and May 2005 (Time 2). The surveys were the main source of information for the impact evaluation.

Description of the Sample of Girl Members of HOV

The evaluation employed data collected from 151 members from 14 sites on whom we had both Time 1 and Time 2 data. The girls ranged in age from 5 to 18, the majority was between the ages of 8 and 15 (86.8%); the average age was 11.4. The grade range was from K-1 to college freshmen; the average was 6th grade. The largest racial/ethnic group of surveyed girl members was Black/African American (38%), followed by Latina (31%), Biracial/Multi-ethnic (16%), and Caucasian/White (11%). While the racial/ethnic distribution of the full sample contained extensive diversity, individual sites were more homogeneous as a function of the different concentrations of racial and ethnic groups in their geographic location.

Evaluation Results

Process evaluation. All HOV sites had success stories to share about the girls’ progress as well as numerous creative projects to describe. While in most instances the girls got along with each other remarkably well, a few sites reported initial girl-to-girl conflict which eventually subsided. A major challenge in the program’s implementation was in the area of mentor retention and lack of successful partnering with local agencies and educational institutions. This was especially pronounced at sites that were short-staffed. Issues of sustainability of girls-only programming were addressed by a few sites only. Finally, sites where Clubhouse staff served as HOV coordinators not only reported reduced turnover but also increased transfer of learning from the girls-only program to the parent Clubhouse.

Outcome evaluation. Girls who attended sites that operated according to the standards set by the designers of the girls-only program received the most benefit. Being located in a resource-rich environment of college and universities and women’s professional organizations with which to partner appeared to made it easier to operate according to the original program specifications. Such resources as colleges and universities or professional women’s organizations were not readily available at some rural sites or urban communities where it was more difficult to recruit mentors and carry out activities related to education and career planning. Thus, in resource-rich environments HOV coordinators had greater opportunities to operate girls-only programs in a manner that meets the standards set by the designers of HOV. However, being in a resource-rich environment did not guarantee success. The program staff needed to actively access the resources available. High staff and mentor turnover mitigated against success even in resource-rich environments.

Impact evaluation. With respect to attitudinal changes, on average, girls developed a more positive attitude toward computers after participating in the program. Having relevant positive attitudes to start with made it more likely that a girl will develop even more positive attitudes as a result of participating in the program. On the other hand, girls who did not arrive with the relevant positive attitudes could also benefit from the girls-only program. (1) If they perceived mentors as available and useful, they were likely to develop positive attitudes toward computers and (2) if they developed positive attitudes toward computers in the program, they were likely to view computers to be relevant to their future. Regarding perceiving computers to be relevant to the future, on average, participating in the program did not make a difference. However, for individual girls, those who entered the program with a view that computers are useful for their future ended up more convinced of this. Even if they didn’t already have this view in Time 1, girls who developed a positive attitude toward computers also came to perceive that computers will be helpful for their future.

Recommendations

• Operating a girls-only program according to the blueprint that is spelled out in the Program Requirements is highly recommended; programs that follow these standards are more likely to produce a positive impact than programs that do not implement all the program’s requirements.

• Agencies interested in implementing an informal computer enrichment program for girls need to ascertain that coordinators have the necessary resources to implement all elements of the program.

• Mentors are one of the most important elements of HOV success. Adult mentor recruitment, training, and retention can be challenging in environments where there are no colleges or universities, nor professional women’s associations. In these cases, the parent organization where the girls-only program is housed should lend a hand for mentor recruitment or provide mentors from among its staff.

• Mentor training can be a valuable retention tool as well as make mentors more effective with the girls. The initial training of HOV coordinators should devote time to practice the training of mentors and should provide coordinators with a training manual that includes information on girls’ development and the Clubhouse educational philosophy, in addition to information on software.

• Building partnerships with schools, after-school programs, and girl-serving agencies can be useful for recruiting girls.

• To make connections between computers and college and computers and careers, it is important to proactively and directly show girls the relevance of computers to their future. Organizations that can be a source of mentors from the professions can also help in the implementation of these activities.

• Many girls place a lot of importance on peer relationships. Provide opportunities for strengthening relationships and team building.

• When working with girls who do not initially show much enthusiasm for computers, starting out with where the girls are is an appropriate pedagogical approach. The critical element here is to infuse the use of increasingly more complex technology into content areas in which some of the girls already have an interest (e.g., fashion, making cosmetics, writing poetry, or making music). After a certain level of comfort is achieved, the coordinator can guide the members to venture into a myriad of other topics in which technology is a valuable resource.