THRIVING FUTURES 2100
A RESEARCH PROJECT.
The 5 Year Thriving Futures 2100 Research Project :
To Evaluate and improve the Global CarePac programs' impact on a diverse demographic of students' continued development, engagement and active participation in regenerative and connected communities through annual iterations of courses with the goal of creating an effective universe referent citizenship education curriculum that is regeneratively effective through connecting academic, community, and natural cultures and ecosystems
DESIGN BASED IMPLEMENTATION RESEARCH
The Global CarePac project operates as a Networked Improvement Community (NIC) conducting
Design Based Implementation Research (DBIR).
Regional Hubs:
-
BC, Canada
-
Catalunya, Spain
-
Thailand
In designing improvements for Global CarePac's programs and the increased effectiveness of the communities, the NIC seeks participation from a diverse and inclusive group of educational and community stakeholders .
NIC representatives from each of the regional hubs:
-
Elder & initiating teams
-
Regional collaborative leadership teams
-
Teacher training univerisity departments
-
Ecosocial university departments
-
Elementary, middle & high schools
-
Indigenous communites
-
Ecosocial organizations and businesses
“The idea of educational evaluation is deceptively simple. It involves the systematic collection and analysis of data needed to make decisions and identify effects of educational initiatives” (Earl & Timperley, 2015, p. 10).
However, as Gamble (2008) says:
Initiatives that are innovative are often in a state of continuous development and adaptation, and they frequently unfold in a changing and unpredictable environment. The destination is often a notion rather than a crisp image, and the path forward may be unclear. (p. 13)
All stakeholders should be involved in the evaluative thinking process, including communities, parents, and students themselves as key participants and decision-makers. When all the groups who have a commitment to and interest in the innovation bring their diverse perspectives and intentions to the evaluation, the evaluation is likely to be more authentic and all stakeholders are more likely to understand, share, and support decisions (Cousins & Earl, 1992).
THE USE OF RESEARCH EVIDENCE (URE)
Study Design Methodologies
The NIC Research teams' developing research design will consist of both a Quasi-Experiment and a Social Network Analysis (SNA):
-
The Quasi-Experiment will measure (in quantifiable terms) the average and differentiated demographic impact of both the Connection and Regeneration online program and place based experiences on student graduates' future and continuing participation in Global CarePac's prosocial commons communities both virtual and place based compared to the participation of students who have not completed the courses and graduates' other choices relating to regenerative practices. Complimentary research will help explain the reasons for outcome effects.
-
The SNA research will extracts nodes of students, mentors and NIC community members and the amount and strength of ties of continuing communication and relationships both during and beyond students' course participation (i.e., evidence of a relationship, connection, action or interaction).
Data collection Methodologies
-
Interviews
-
Surveys
-
Observation and Discourse Analysis
-
Document Analysis
The Quasi-Experiment will use research data consisting of:
-
Surveys will be used to gain quantifiable data on students' future and continuing community and program participation
-
qualitative methods such as interviews, observations, and document analysis will examine the examine processes and mechanisms.
SNA will use research data consisting of:
-
a raw count of the number of interactions from communication data
-
a reported frequency of interaction on a survey
-
a value of an interaction stated by an interviewee.
The methods used for collecting information from stakeholders will include “document analysis; narrative, stories, and vignettes; surveys, focus groups, and interviews, just-in-time responses using digital technologies and social media” (Earl & Timperley, 2015, p. 24). As a part of their data collecting protocol for evaluation, the team will also make and use videos of teacher training intern student programs of both teaching moments and practices as well as personal evaluation interviews with participants as feedback. MSC techniques will also be used (see below). For final project evaluation, information will be collected of continued ecosocial education community participation of alumni beyond individual course completion
Most Significant Change Technique (MSC)
MSC technique is a form of participatory measurement and evaluation. It involves stakeholders collecting
stories about significant change directly from families and individuals. It is participatory because many
project stakeholders are involved both in deciding the sorts of change to be recorded and in analysing
the data. MSC can be used to help understand the impact on individuals in an authentic and powerful manner.
Phase 3. Data Analysis, interpretation, and reporting:
The evaluation report will be built considering the facts, values, and a Boundary Analysis as represented as three sides of an analysis triangle shown in Figure 3.
Facts and data collected will be analyzed through gender equality, environmental, and marginalized voices themes. The insights and observations need to be converted into knowledge that is both insightful, useful, and relevant.
Figure 3 Systemic Triangulation. Adapted from Reynolds (2015). Adapted from ‘Evaluation Guidance Series Inclusive Systemic Evaluation for Gender Equality, Environments and Marginalized Voices. ISE4GEMs: A new approach for the SDG era’, by Stephens, A., Lewis, E. D., & Reddy, S. M., 2018, p. 110
The goal of the analysis is to arrive at knowledge, “the kind of knowledge that can be transferred and further developed across contexts (Earl & Timperley, 2015, p. 32). The contexts in the case of each program will be preparation for the following program leading ultimately at the end of the year cycle to building capacity for future years' programs.
Figure 4 Communicating evaluation results by conceptualizing systems change using the forest ecocycle analogy. Adapted from Zimmerman, Lindberg, & Plsek (2001). Adapted From ‘Evaluation Guidance Series Inclusive Systemic Evaluation for Gender Equality, Environments
A forest eco-cycle practice model
Here, aspects of the programs being evaluated can be analyzed as belonging to one of four quadrants: Birth, Maturation, Creative Destruction, and Renewal (Figure 4). This model allows for an iterative series of PDSA evaluation led improvements.
In this model, the biological ecocycle metaphor is shown as being an infinity loop. “The infinity loop depicts a living systems scenario with no beginning or end. The movement from the lower-left Quadrant I to the upper right Quadrant II follows an ‘S’ curve” (Stephens et al., 2018, p. 140). It is on this ‘S curve that a focus of strategic planning to improve the efficiency of programs and interventions leading to mature and improved outcomes would be best applied.
Phase 4. Capacity Development
Capacity development goals for this project involve working as a networked improvement community to build and develop theories of knowledge and theories of change and ascertaining how the theories of knowledge and change can be used to scale up mutual program success at more locations through further lateral capacity building with communities and educational organizations.
Figures 1 & 2 lay out the four stages of the evaluation learning and action cycles.
The evaluation cycle will be completed after each regional camp, at the end of each Intern training cycle and at the end of each of the five years of the project.
BOUNDARIES
ITERATIVE PDSA CYCLES
Figure 2 The actual boundary of evaluation. Adapted from ‘Evaluation Guidance Series Inclusive Systemic Evaluation for Gender Equality, Environments and Marginalized Voices. ISE4GEMs: A new approach for the SDG era’ by Stephens, A., Lewis, E. D., & Reddy, S. M., 2018, p. 68.