Looking Under the Hood: What Happens When We Send 8th Graders to Israel?

Alex Pomson

Each year, close to one hundred Jewish day schools in North America run trips to Israel for students during the final months of eighth grade. In community, Conservative and Reform day school sectors, more than 70% of schools run such trips. While most students are expected to pay their way, few trips depart without philanthropic intervention or financial subvention directly from school budgets. With such widespread practice, it is remarkable that the ROI (return on investment) provided by these trips has, until recently, never been examined.
To make the situation more puzzling, in some quarters these trips are actually viewed with great skepticism. Educators often see the programs as little more than a means to entice students to remain enrolled in school until the end of the middle-school. Because the trips invariably occur during the last few weeks of school, others see them as a kind of graduation ritual, equivalent to an elaborate prom. And because the participants are relatively young, there is often doubt about the kind of impact these programs produce, by comparison with longer trips or experiences that serve teens or older populations.
This is where evaluation can make a significant contribution, helping school leaders understand the ways in which these trips do or do not make a difference for the participants and how best to ensure that these trips are a part of rather than apart from a school’s commitment to Israel education and engagement. When done well, program evaluation provides a chance for educators and program providers to look carefully at their programs and their practices, to test assumptions about why things appear the way they do. Most usefully, evaluation is an opportunity to facilitate informed and constructive conversation among practitioners, as well as between practitioners and those who fund their work.
Two years ago, our team at Rosov Consulting had such an opportunity. While working with Jack Wertheimer on behalf of the AVI CHAI Foundation on the Hearts and Minds: Israel in North American Jewish Day School project, we were approached by the Jewish Agency for Israel. The Agency recognized that many of the two thousand 8th grade students who were participating in the AVI CHAI study, and had already completed a student survey, would soon participate in school trips to Israel. They proposed a follow-up study with a sample of students after their return. This study would make it possible to explore a question that until then had not been researched: if and how middle-school students’ self-understanding and their connections to Israel are changed by participating in short-term educational programs in Israel.
What we learned from the study of the participating schools can be useful to all schools that run such programs. Indeed, the study models what learning can be set in motion by evaluation work in general.
The Study 
In total, 13 middle schools participated in the study: six affiliated with RAVSAK; four with the Solomon Schechter network; three schools identified themselves as modern Orthodox. In total, 227 students responded to both pre- and post-trip surveys, equivalent to 86% of all of the students who participated on trips from these schools. The students were surveyed before their departure as part of the larger study of 95 schools, and then again between two and four weeks after their return to the United States. The students were asked many of the same questions on both occasions to see whether their attitudes and understanding had shifted as a result of their time in Israel. The trips in which they participated ran for between 10 and 23 days.
What We Learned 
Taken together, our pre/post research design revealed that short, ten-day to three-week trips to Israel are related to important outcomes. In cognitive terms, the trips crystalized the ways in which young people thought about Israel and the world. They helped certain ideas about Israel fall into place. For example, there was an intensification in the students’ images of Israel as a “home away from home” and as “a place where teenagers have more freedom to do what they want.” More traditional ideas or spiritual images of Israel didn’t seem to shift. The students came to identify more strongly with ideas or attitudes they had not previously considered: for example, that Israel can be “a warm and friendly place.” And there were clearer and more consistent connections between their ideas about, for example, the meaning of Jewish peoplehood and their connection to the State of Israel.
The trips also influenced students’ affective relationships to Israel, especially for those who previously were not inclined to identify with Israel. The scale of the shift observed was moderate, but still surprisingly large given the brief length of time that participants spent in Israel.
The trips had a significant impact on some students’ understanding of contemporary Israel. Most strongly influenced were students’ thoughts about what it is like for people to live in the country, and especially what it’s like to be a teenager in Israel. In our experience, these personal themes are quite different from those, such as Israel’s place in Jewish history or in religious life, that are more heavily emphasized by day schools over the course of many years of Israel education. In this respect what students learned in Israel was quite different from what they learned about Israel in their classrooms. 
Not all students responded in the same way to their Israel experience. Those who were interested in Jewish matters before the trip were more likely to be engaged by the intellectual and historical ideas they encountered during the trip. Those less connected to Jewish life and Jewish concerns before the trip were more likely to return with a greater sense of Israel as a fun place to spend time and with a greater sense of association with those who live there. Some, as many as a quarter of the students, returned to America with a weaker sense of identification with others Jews and with Israelis.
Lastly, it seems that program time devoted to reflection and discussion were most closely correlated with the changes we observed. We should note that the participants themselves were not fully aware of these effects; they did not appreciate the impact of time spent in such discussions, especially when compared to the programs’ more dramatic components. Our pre/post analysis revealed that these elements were critical to the educational process.
From Answers to Questions
For those schools that offer Israel trips to their students, these data provide important information to help school and trip leaders reflect on their effects. These data also help answer questions school leaders have about their curriculum. From our perspective this study is no less valuable for the questions it poses than for the answers it offers. We see this study as a strong example of how evaluation can stimulate profound questions about practice. Here are just a few.
1. If, as we found, these trips provide a different view of Israel from that which is provided inside of schools, what do we make of that? On the one hand, there is cause for concern in a possible disconnect between the content of trips and the educational programming that precedes them over the course of many years. On the other hand, the programs might be playing an important complementary role, compensating for what is generally absent from regular programming in schools. At the very least, are schools even aware of this disconnect?
2. What happens if students continue on to high school, and the Israel about which they learn is once again an abstract or mythologized one that bears little relation to the one they have experienced in Israel themselves? Will the impacts of these short programs quickly dissipate without reinforcement, or will students start to feel a dissonance between the contemporary Israel they encountered and the abstract Israel they learn about in school?
3. We wonder why young people react in such different ways to their time in Israel. Why do some come home feeling more distant from Israel and from Israelis? Is it because of something they experience during their time in the program, or is it owing to a mismatch between their expectations ahead of the trip and their experiences in the country?
Evaluation, we believe, is not only about providing answers. It is also about posing questions and about stimulating a desire to think carefully and clearly. Some might see this study’s value in what it demonstrates about the ROI in middle school Israel trips; that’s something undoubtedly supported by our findings. Here we emphasize something different: how evaluation helps surface questions and insights that promise to intensify and improve the outcomes produced by trips.


Dr. Alex Pomson is the director of research and evaluation at Rosov Consulting. [email protected]

Return to the issue home page:
Image
HaYidion Taking Measure Fall 2015
Taking Measure
Fall 2015