Mahatma Gandhi once described culture as something that “resides in the hearts and in the soul” of a group. Gandhi’s implication was this: Culture is an integral force within a group, a subconscious pulse of its existence. Although Gandhi was referring to the culture of a nation, we can draw parallels to the culture of assessment within higher education.
For the past decade, the phrase building a culture of assessment, first coined by Lakos and Phipps, (2004) has been used to describe the ideal atmosphere of evaluation and accountability in higher education. The concept of assessment as an ingrained cultural norm has also been adopted by the Council for the Accreditation of Educator Preparation (CAEP) who now uses a similar term “culture of evidence” to describe a central tendency of its accreditation process (CAEP, 2015 p.6). To CAEP, a culture of evidence exists when a program incorporates “intentional use of documentation, multiple and valid measures, and analysis provided as a support for and proof of an educator preparation provider’s (EPP) claims related to CAEP’s standards” (p. 121).
The dichotomy facing EPPs is this – provide “proof” of what should exist in the “heart and soul” of a group. As part of a culture, assessment should permeate so deeply it goes almost unnoticed. Within the program, it simply becomes “What we do.” At the same time, signs and symbols of assessment should be quite conspicuous to anyone external to the group.
Ironically, the outward evidence of assessment readily noticed by external parties is nearly subliminal to those internal to the program. When accreditation team members, potential students, P-12 partners, and other outside constituents visit or interact with an EPP, they should see visible signs and hear conversations indicative of assessment. These outward signs serve as proof of an inherent culture of assessment so ingrained it exists at a nearly subconscious level by the faculty, staff, and students in the program.
How can EPPs address both the external and internal components necessary for building a culture of assessment? EPPs can create highly visible displays of data and a means to clearly document their assessment efforts, as well as provide evidence of program outcomes. To external parties, these displays provide clear and convincing evidence not just of program outcomes but also of the culture of assessment. At the same time, EPPs should intentionally weave assessment into routine meetings and systematic processes in order to deepen the culture assessment. To internal parties, assessment and evidence gathering moves beyond responding to accountability structures and simply becomes the standard way in which the program conducts its work.
Using the What WE DO cycle represented in Figure 1, EPPs can take intentional action to create a culture of assessment by 1) embedding it in the heart and soul of the program and 2) providing visible proof of it. The following example outlines this process within the University of Nebraska at Omaha (UNO) which enrolls approximately 1200 in its undergraduate EPP.
Figure 1
The “What” portion of the cycle identifies learning outcomes. At UNO the “What” was articulated when faculty were required to identify three key learnings for each course in the EPP. Faculty translated these outcomes to language found in the InTASC Standards (Council of Chief State School Officers, 2011). This step identified redundancies and gaps within the program. The crucial conversations necessary for establishing the key learnings were woven into regularly scheduled departmental and committee meetings. Once the key learnings were solidified, they became the basis for a program wall. The program wall was displayed in a public area and visualized the key learnings and related standards for each course. This became highly visible evidence of the program’s intended outcomes.
Data related to these outcomes became the Weighted Evidence (“WE”) portion of the cycle. Recognizing the validity and reliability of assessments necessary to meet external (CAEP) standards, the UNO program intentionally built internal capacity for this process. To do so, UNO offered assessment stipends incentivizing faculty to create rubrics and collect data in low stakes environments. As a result, faculty became more familiar with what constituted high-quality rubrics and what could be interpreted as valid and reliable evidence. Assessment was also added as a standard item on departmental and curriculum committee meeting agendas. Assessment became part of the routine, thus part of the culture, for those internal to the program. At these meetings, program faculty began to identify and create the assessments to collect evidence (WE) of the key learnings.
Assessment discussions were part of the stipend requirements and led to important conversations. The rubric conversations were powerful and showed us the importance of the conversation.” Common themes emerged from these conversations. For example, faculty discovered areas of inconsistency in regard to expectations and terminology. This led to changes to increase the reliability of their rubrics and clarify academic language. Because this process was first done voluntarily within course assessments, and outside of the accreditation process, collecting evidence and strengthening rubrics became embedded in the day-to-day practices of the faculty.
Finally, the data ownership (“DO”) portion of the cycle included two elements critical to the culture of assessment: access and ownership. First, the program developed a systematic and manageable method to share data with faculty as well as program chairs and coordinators. Once they had access to the data, UNO faculty used a process known as the “Big Ds” – Data, Discussions, and Decisions, to take ownership of the data. This process documented discussions and subsequent decisions based on data from the WE as well as other course-level assessments. Internally, the process became routine within the meeting structures of the department. External audiences also had access as evidence from the key assessments were displayed on a virtual version of the program wall. Based on this and other information, they too could use data to inform their decisions regarding program quality.
If an EPP truly creates and sustains a culture of assessment, visible and measurable proof as required by CAEP must be present. At the same time, this culture will, as Gandhi described, have a nearly imperceptible presence in the program’s heart and soul. UNO’s “What WE DO” model demonstrates one approach to achieve these goals.
Written by:
Connie Schaffer, Assessment Coordinator
College of Education, University of Nebraska at Omaha
Lela Nix, Assessment Graduate Assistant
Teacher Education Department, College of Education, University of Nebraska at Omaha
Learn more when you join the University of Nebraska at Omaha at this year’s Assessment Conference where they are featured presenters. To learn more about this year’s program, visit www.livetextconference.com. Hear what your peers are saying about the LiveText Conference… click here.
References
Council for the Accreditation of Educator Preparation (2015). CAEP Accreditation Handbook, Washington DC: Author.
Council of Chief State School Officers (2011). Interstate Teacher Assessment and Support Consortium (InTASC) Model Core Teaching Standards: A Resource for State Dialogue. Washington, DC: Author.
Lakos, A. & Phipps, S. (2004). Creating a culture of assessment: A catalyst for organizational change. Portal-Libraries and the Academy, 4(3), 345 – 361. UCLA: Retrieved from: https://escholarship.org/uc/item/0843106w.
Share
The post LTAC Speaker Spotlight: Building a Culture of Assessment – An Unlikely Partnership between Mahatma Gandhi and CAEP appeared first on LiveText.