Evaluation/Research position at LEAD Center

Looking for an evaluation/researcher position with the Community of Practice?
The LEAD Center is recruiting (in partnership with Gear Learning) to fill this position.

GAMES AND LEARNING RESEARCHER at UW–Madison<http://jobs.hr.wisc.edu/cw/en-us/job/495223/games-and-learning-researcher>

My Journey as an Aspiring Culturally Responsive Evaluator with Stafford Hood!

My Journey as an Aspiring

Culturally Responsive Evaluator

Stafford Hood

Professor, Curriculum & Instruction University of Illinois
College of Education

Graduate student brown bag (i.e., bring your own lunch)
Friday, April 7, 2016 • Noon – 1:30 pm
UW Madison, Ed Sciences Building, Room 259, 1025 West Johnson Street

Sponsored by: 

Dr. Bowman to Co-Present LEAD Brown Bag on 3/30

Dr. Nicole Bowman

LEAD and the Evaluation Community of Practice are co-sponsoring a brown bag on AEA Diversity Initiatives, on March 30th

The flyer is below, please share!

*Download the PDF of the LEAD flyer.

Cultivating Self-in-Context as Responsive Evaluators

Brown Bag Tomorrow!

Cultivating Self-in-Context as Responsive Evaluators: Engaging Boundaries, Borderlands and Border-Crossings

Hazel Symonette, Ph.D.

We increase prospects for operating at our evaluator best when we intentionally embrace a contextually-responsive action researcher stance. This involves systematic data-grounded inquiry as an evidence-framing dialogue with SELF as evaluator–vis a vis one’s stakeholders and the requirements of the evaluation agenda and contexts. For excellence and ethical praxis, evaluation practices should be broadly diversity-grounded and equity-minded. They should be socially-responsive, socially-responsible and socially just as informed by the American Evaluation Association’s Guiding Principles and the Joint Committee on Standards for Educational Evaluation’s Program Evaluation Standards.  My  holistic systematic inquiry and reflective practice framework promotes empathically scanning, tracking and unpacking WHO? factors in context using an Integral Quadrant model: who is served by whom with whom as embedded in situational, relational, temporal and spatial/geographic contexts. This Activist Assessment & Evaluation approach is a resource for enhancing Interpersonal Validity vis a vis responsive programmatic design and development.

LEAD Into 2017

As we get near to 2017, BPC would like to share these reminders of what leadership looks like. Thoughts from UW LEAD Center.

LEAD, by design…
We work collaboratively with faculty and staff who are Principal Investigators (PIs) and are in need of consultation or services to propose or conduct evaluation of a grant. We build logic models and design evaluation plans that are aligned with a project’s specific aims and objectives to improve learning in postsecondary institutions.

LEAD, in the right direction…
When funded, we provide you with formative evaluation to ensure that you are meeting your goals and to help you improve the implementation of your project.

LEAD, with evidence…
We collect and analyze data to assess the efficacy of your program and to provide you and your funding agency with the results from your evaluation and a summative report.

LEAD, with integrity…
We provide an objective view of your project’s implementation, and ensure the ethical treatment of human subjects, confidentiality of your data, and the highest standards of practice in evaluation.

LEAD, so others will follow…
We help you to disseminate evaluation findings and if applicable, publish them broadly so that others can learn from you.

LEAD, the way…
We evaluate locally funded projects, as well as those funded by the federal government and private foundations or trusts.

Formative Evaluation by the LEAD Center

We provide formative evaluation to ensure you meet your goals and improve the implementation of your project.

LEAD in the Right Direction…
When funded, we provide you with formative evaluation to ensure that you are meeting your goals and to help you improve the implementation of your project.

*Learn more

No More PRANKS-Giving: How the Evaluation Community Can Start Rebuilding Relations with Indigenous Communities

Koolamalsi njoos (Greetings, colleagues and friends):image1

As the newest Center for Culturally Responsive Evaluation and Assessment (CREA) Affiliate Researcher, I represent the oldest and least known communities on Turtle Island and Mother Earth: Indigenous people.  My Indigenous ancestors and relatives are on nii nooxw waak nii numoxoomus (my father Peter Bowman’s and my grandfather Morris Bowman’s) side of the family. Neekaawa Lunaapeewook waak Mohiikaneewi (they are Lenaape-Munsee and Mohican Indians).  Mohican (not the “last of” thank you very much) and Lenni-Lunaape Munsee (we were “renamed” “Delaware Indians” as we were “relocated” by the Europeans post-contact) are my Indigenous family and community members who arrived on this continent thousands of years prior to European contact in the 1400s, according to our oral traditions. For the reader’s convenience, I will continue the rest of this discussion in the language the colonial people forced upon Indigenous people when they arrived at and conquered Turtle Island (English).

We need to stop Pranksgiving, and this time of year gives us a chance to rethink Columbus and beyond!  So, as we embark on yet another “Thanksgiving” and Native American history “month” I’d like to offer a different, historical Indigenous perspective, “re-writing and re-righting” (Smith, 1999) the western narrative that the American story began when Columbus or Henry Hudson got here, and adding to or correcting the discourse and printed content which often excludes, is ignorant of, or provides a romanticized and incorrect version of Indigenous people’s perspectives, experiences, and contributions.  We were here before European contact, we’re still here surviving and thriving, and we will continue correcting the information regarding our Indigenous communities’ contributions to contemporary contexts.  This includes the wisdom and content that Indigenous people provide to modern-day educational, academic, governance, and other disciplines, topics, and contexts.

I am proud to be part of an evaluation community at CREA that is leading the way to encompass the history, values, perspectives, strengths, and contributions that our collective and strong diversity represents.  Some of these evaluation roots date back decades and some trace back centuries.  Others are unknown and/or are being re-awakened, recovered, and revived as we strive to understand our origin stories and the cultural, historical, and intellectual traditions of our Indigenous relatives and ancestors.  Undoubtedly, our awakening process is as critical to what we create as evaluators: designs, studies, publications, presentations, trainings, and transformative change that are sustained by the communities we humbly serve.  The journey as well as all the people, places, projects, and insights along the way provide a rich context in which we situationally understand more about ourselves, our traditions, our communities, and our broader academic context.

When the historical, contextual, technical, and content knowledge of Indigenous and underrepresented people is not woven into the more contemporary evaluation fabric, we are in conflict and ignoring the very mission, vision, values, and goals that our professional evaluation community (AEA) is supposed to uphold.  If we are to improve, increase, promote, and support activities that make for more effective evaluators and evaluations then as a profession, we must acknowledge, accept, and address the severe lack of equity, inclusion, ethical consideration, and legal protections critically missing from most evaluation activities, curriculum, studies, and discourse.

Current controversy provides a significant example for reflection and consideration.  Within both Indigenous and academic communities, I am often asked about the conflict surrounding the Dakota Access Pipeline (DAPL) and the #NoDAPL counter-movement launched by human, water, and earth rights protectors from Indigenous and non-Indigenous communities.  #NoDAPL seeks to block a $3.8B, 1,100-mile fracked-oil pipeline currently under construction from the Bakken shale fields of North Dakota to Peoria, Illinois. DAPL is slated to cross Lakota Treaty Territory at the Standing Rock Sioux Reservation where it would be laid underneath the Missouri River, the longest river on the continent, endangering a source of fresh water for the Standing Rock Sioux and 8 million people living downstream. DAPL would also impact many sites that are sacred to the Standing Rock Sioux and other Indigenous nations.

As a culturally responsive community, and as competent and skilled evaluators, we all must understand the cultural, contextual, historical, and legal implications of DAPL and similar projects with Indigenous communities and Tribal nations.  This is one reason why the Indigenous origin stories must be part of the broader evaluation narrative so our “intellectual” evaluative foundations are aware, accurate, informed, inclusive, comprehensive, responsive, and responsible for situating our practice within a much longer and mostly unknown, destructive history than our contemporary publications, agencies, and evaluators acknowledge. Claiming expertise, ignorance, or other falsehoods is not acceptable if we profess to be a global evaluation community.

Like many projects – research, evaluation, and otherwise – conducted with Indigenous people, DAPL has deep, destructive, and illegal historical roots.  The project is in violation of treaty rights afforded the Standing Rock Sioux as a sovereign nation, reflecting the lack of recognition of Tribes as sovereign nations that stretches back to the earliest treaties enacted in 1774 between colonists and Indigenous Tribal nations.  How many research and evaluation projects have been enacted with Indigenous communities to “extract” knowledge and data without due consideration for Indigenous ownership of that knowledge and data?  Far too many!  And this must stop, for ethical, moral, legal, and professional reasons.

Reflecting on the DAPL project reminds us that our broader evaluation community is also about power, networks, and resources.  The perspectives and experiences of the “others” or “have nots” is greatly underreported in our evaluation, academic, education, and other contexts.  We only need to look at who gets funded, published, promoted, elected, or represented on key commissions, editorial review boards, or attends “invite only” events to see that while we talk about inclusion, equity, and diversity, there is little evidence of it in our “community” of practice.  What business interests, amount of dedicated resources, and professional practices are most prevalent in our evaluation community?  It depends on whose conference and what context you are in.  But the interests, practices and viewpoints of Indigenous and other marginalized groups are consistently and significantly absent!

DAPL and the #NoDAPL movement also remind us that our broader evaluation community privileges the written word and values certain evaluation voices – largely white and male – more than others.  A content analysis of AEA journals and evaluation published literature from the 1970s to the present (n=3,305 articles) demonstrates racialized and cultural incongruence issues that are consistently seen between program participants and evaluators stemming from white privilege, wealth inequities, and inequitable distribution of wealth along cultural lines in the United States.  Similar inequities appear when we consider whose voices are heard and respected in discussion around the DAPL project.  For example, DAPL was originally routed through Bismarck, ND, but was redirected in response to the voices of the powerful and political concerns of a municipality over drinking water.  However, voices against the inequitable, unethical, and unlawful treatment of a sovereign Tribal nation remain unheard, not covered by mainstream media as human or environmental concerns.  I welcome emerging voices that tell different stories in different ways – voices like those of Carl Sack, a graduate student at the University of Wisconsin-Madison who is using critical cartography as a culturally-responsive and scientifically-sound method to provide a counter-narrative to the lack of media coverage of the historic, Indigenous and #NoDAPL perspective.



Figure 1.  Sack, C. (2016). The Black Snake in Sioux country: showing the Dakota Access Pipeline reroute through unceded treaty lands and its consequences.   Retrieved from https://northlandia.wordpress.com/2016/11/01/a-nodapl-map/

Where are the social justice evaluators and researchers?  Who will give voice to the historic and contemporary experiences and stories of the people, land, and destruction based on forced removal and contemporary trauma and violence stemming from DAPL? How is our evaluation community holding up the mission of “generating knowledge about effective human action” when we can’t even make a joint statement about #NoDAPL, let alone send a commissioned group of evaluators to ND?  Surely, the Canadian Evaluation Society and the Truth and Reconciliation Commission give us an example of how that is done!   Inclusion and action are the responsibility of everyone who claims to be a principled evaluator within the larger the evaluation profession.  Yes, YOU are being called to action because knowing better should mean doing better!  Standing Rock is just another contemporary example of centuries of injustices, destruction, trauma, and death dealt out to Indigenous people where most of the world stood by.  I am asking you at #NoDAPL and in other contexts to stand with us as global academic #warriors in solidarity as together as transformative evaluators.  In turn, you too will be transformed.

The bottom line is that I feel at home with CREA.  This is the interdependent place where we engage as family to draw enlightenment, continue to grow, and gather support from all races, ethnicities, contexts, cultures, socio-economic, gender, orientation, different abilities, and other diversity components as strengths to stand with us on the front lines.  Processing our ancient pain or contemporary trauma continues to be largely minimized, ignored, denied, and/or aggressively blocked by most post-contact people, institutions, literature, and systems.  Truly this can be paralyzing at times, and is a problem compounded by the low numbers of Indigenous academics who can add our communities’ voices to the mainstream narrative. So we need support, solidarity, and collegial partnerships.  CREA provides the place for speaking, sharing, and celebrating all that our community brings to evaluation; it is only through the stories, sacrifices, struggles, teachings and strengths of our ancestors/relatives coupled consciously and courageously with our CREA community that many of us Indigenous academics can continue.

In closing, I offer my perspective in the most humble and sacred way.  My words and vulnerabilities are my perspectives alone and are shared within context knowing that some of my Indigenous relatives died on the longest Trail of Tears journeys, were raped and killed by colonists, and continue to be treated as sub-humans suffering, traumatized, and dying on the lands that we were forced to move onto.  Yet we find ways to survive contribute and thrive.  As Waapalaneexkweew (Flying Eagle Woman; Accompanied by the Four Eagles), it is my responsibility to use my life to keep telling our origin and contemporary stories until the day Creator takes me back to the spirit world.  And from my deeply grateful heart, I’m so honored that we’ve created more academic, educational, and community spaces to share the truths of Indigenous perspectives that will continue transform us towards a more equitable and empowered future. I feel your strength, my beautiful CREA family, and I know Indian Country does too.  Anushiik (thank you) for being in it with us for the long haul.  We are one Tribe walking the Red Road, so see you soon.

Sincerely, Critically, and Indigenously Yours,

Waapalaneexkweew and my English name is Nicole Bowman (Mohican/Lunaape), PhD

Evaluator & Researcher, University of Wisconsin-Madison (www.wcer.wisc.edu/)

President, Bowman Performance Consulting, Shawano WI (www.bpcwi.com)

If you’d like more information to contextualize my comments, I suggest the following:


Bigelow & Peterson (Eds.). (2003). Rethinking Columbus. Milwaukee, WI:  Rethinking Schools.

Deloria, V. (1988). Custer died for your sins: An Indian manifesto; with new preface. Norman: University of Oklahoma.

Deloria, V. (1995). Red earth, white lies: Native Americans and the myth of scientific fact. New York: Scribner.

Hood, S., Hopson, R. K., & Frierson, H. T. (2015). Continuing the journey to reposition culture and cultural context in evaluation theory and practice.  Greenwich CT:  Information Age Publishing.

Tuhiwai Smith, L. (1999). Decolonizing methodologies: research and Indigenous peoples. Dunedin, New Zealand: Zed Books Limited.

Dr. Bowman Spotlighted on Wisconsin Center for Education Research Newsletter!

uw newsletter circle bowmanDr. Bowman was spotlighted on Wisconsin Center for Education Research’s newsletter. She was congratulated for her appointment to LEAD Center.

About WCER

The Wisconsin Center for Education Research (WCER) at the University of Wisconsin–Madison’s highly ranked public School of Education is one of the first and most productive education research centers in the world. It has assisted scholars and practitioners develop, submit, conduct and share grant-funded education research for more than 50 years.

WCER’s mission is to improve educational outcomes for diverse student populations, impact education practice positively and foster collaborations among academic disciplines and practitioners.

WCER is an incubator for advances in education policy and practice. It provides services and resources for UW–Madison researchers, graduate students and their collaborators, as well as educators, fundersand communities locally and around the world.

*Learn more at http://www.wcer.wisc.edu/

About LEAD

At The LEAD Center, We advance the quality of teaching and learning by evaluating the effectiveness and impact of educational innovations, policies, and practices within higher education. Rigorous, theory-driven methods and interdisciplinary collaborations anchor our approach to evaluation, leading to the adaptation and dissemination of evidence-based strategies in service to student learning.

*Learn more at http://lead.wceruw.org/

LEAD in the Right Direction

We provide formative evaluation to ensure you meet your goals and improve the implementation of your project.

LEAD in the Right Direction…
When funded, we provide you with formative evaluation to ensure that you are meeting your goals and to help you improve the implementation of your project.

*Learn more

What will LEAD do Next?

Enjoy viewing past projects by LEAD. Stay tuned to see what LEAD does next!


Advanced Fellowships in Women’s Health HUB Site-VA
Annual Biomedical Research Conference for Minority Students (ABRCMS)
Biology Scholars Program
Fair Play
Language, Understanding, Cognition, Intelligence and Data Science (LUCID)
Mentor Training
Wisconsin Alliance for Minority Participation
Women in Science and Engineering Leadership Institute (WISELI)

Delta’s Academic Excellence Initiative, 2011-2015

Project site: University of Wisconsin-Madison
Evaluators: Sara Kraemer

Delta’s Academic Excellence Initiative (AEI) is a University of Wisconsin-Madison program whose aim is to close the opportunity gap in undergraduate courses through changes in course instructor practice and pedagogy. Course instructors attend an eight week immersion to learn about key opportunity gap and diversity issues at U.S. college campuses broadly and UW-Madison campuses specifically, followed by developing and implementing research-based course interventions. In our formative evaluation study, Sara Kraemer has been a member of the AEI team, administering instructor-based surveys and working one-on-one with faculty to evaluate their student data and develop target areas of support. AEI has partnered with the UW-Madison’s HOPE Lab to provide summative impact analysis on student achievement and faculty knowledge and attitudes about opportunity gaps and diversity at UW-Madison.

Wisconsin Collaboratory for Enhanced Learning (WisCEL), 2014-2015

Project site: University of Wisconsin-Madison
Evaluator: Sara Kraemer

The Wisconsin Collaboratory for Enhanced Learning (WisCEL) facilitates innovative teaching and student-centered active learning by providing instructional support and resources in unique technology-enhanced learning spaces on the UW-Madison campus.

Sara Kraemer conducted a mixed-methods evaluation study of a Math 112, an introductory mathematics course that fulfills mathematics requirements for undergraduate students. The study evaluated the impact of the WisCEL on student engagement, student achievement, and perceptions about the efficacy of the WisCEL learning experience in Math 112. Formative evaluation included a student survey, a focus group with TAs, and observations of Math 112 classes to asses student engagement. Summative evaluation included a statistical analysis of student achievement scores (i.e., end of course grades, drop rates) of Math 112 courses before they were taught in WisCEL compared to Math 112 coursers after they were taught in WisCEL. Summative results revealed that the WisCEL classroom had a positive impact on student achievement scores (as a whole and across demographic groups) – students were more likely to receive higher grades and have lower rates of drop out in the WisCEL classroom than in the non-WisCEL classroom. The findings were triangulated with the survey and focus groups results to assess how the classroom instruction interacts with the features of the WisCEL space.

WisCEL program managers are using the evaluation findings for program improvement, a proof-of-concept to campus stakeholders, validation of the program’s efficacy, and assessment of transferability of evaluation methods to other WisCEL courses.

Crossroads Project: Intersecting Workshops, Learning Communities, and Research in Biology to Promote Student Success in STEM, NSF-IUSE Program (PI: Loretta Brancaccio-Taras), 2015-2018

Evaluator: Sara Kraemer
Project Site: Kingsborough Community College, Brooklyn, New York

The Crossroads Project at Kingsborough Community College (KCC) is designed to strengthen the skills, retention, and graduation rates of science, technology, engineering, and mathematics (STEM) students. The project targets biology majors, and is specifically designed to support introductory biology students in success in the course, as well as progression through the biology or other STEM majors through three major efforts. The three major efforts are: (1) Strategies for Student Success in STEM Workshop, (2) Community of Biology Learners sessions, and (3) Campus as Research Lab. The workshops and sessions are aimed to enhance student learning through development of study skills in STEM and biology and create cohesion and community through structured group-engagement outside of the classroom. The Campus as Research Lab provides an opportunity for students to work like scientists with KCC faculty, by engaging in the scientific inquiry process and conducting basic experiments.

The evaluation study of the Crossroads Project is a mixed-methods design that includes both formative and summative analysis. For the Strategies for Student Success in STEM Workshop, the summative analysis is comprised of statistical analysis of end of course grades for biology students and the formative analysis is an end of course survey to assess student perceptions of the impact of their experience on their academic success. For the Community of Biology Learners, a formative evaluation content analysis of student answers to assignments as well as a written instructor’s log of teaching practices. For the Campus as Research Lab, the evaluation component is a qualitative content analysis of student writing course assignments and an end of course student survey.