In May 2024, the Office of Faculty ADVANCEment invited all faculty at Clemson University to take part in an updated version of the climate survey, designed to provide data on faculty characteristics, attitudes, and behaviors. Following recent blog posts related to the analysis of numerical answers, we are now reporting on a few additional aspects linked to the analysis of the qualitative data as well as the open-ended questions. The analysis was performed by Alena Höfrová, in partial fulfillment of the requirements for the degree Doctor of Philosophy in International Family and Community Studies. here, we presents a short summary of the overall document.
A mixed-methods outcome evaluation of a NSF ADVANCE project examining the effect of gender on organizational climate and faculty experience with inclusion
The primary purpose of this case study dissertation was to evaluate the outcomes of the NSF TIGERS ADVANCE project on institutional transformation and faculty experience with inclusion at Clemson University. Specifically, the first objective is to determine how the NSF TIGERS ADVANCE project contributed to improved organizational climate by evaluating changes in measures of organizational identification, inclusion climate perceptions, job satisfaction, and intention to leave. The second objective is to investigate potential differences in faculty experiences with inclusion based on their participation in the NSF TIGERS ADVANCE project. The third objective is to examine whether gender differences exist in faculty perception of various organizational climate variables and faculty experiences with inclusion. This study seeks to contribute to the existing literature by assessing whether the NSF TIGERS ADVANCE project achieved its goal of reducing gender inequality and improving the status of women faculty at Clemson University. This research employed both quantitative and qualitative research methodologies, using data from the NSF TIGERS ADVANCE Organizational Climate Surveys administered in 2017 and 2024.
Clemson University was awarded the NSF ADVANCE Institutional Transformation grant in 2016. This project aimed to reduce gender inequalities and enhance the status of female faculty in STEM disciplines at Clemson by utilizing the theory of change approach. The initiative involved all university faculty to create one organizational identity to drive the university’s success rather than promoting smaller in-group identities. The project aimed at achieving five goals to systematically transformation the institution, each contributing to the reduction of inequalities and the creation of a more supportive climate for female faculty. The goals were operationalized through specific activities, initiatives, events, and workshops:
- Transform the culture and improve campus climate
- Increase representation of women in STEM disciplines
- Ensure equitable workload distribution
- Enhance faculty mentoring and leadership development
- Implement family-friendly policies and policy change
The 7-year program concluded in the summer of 2023, involving a total of 1,949 unique participants in 126 NSF TIGERS ADVANCE activities, initiatives, events, and workshops. This diverse group comprised 770 faculty, 478 graduate students, 383 undergraduates, 182 staff members, 12 from university leadership, and 111 “other” participants, including librarians and visitors. Among the participating faculty, 434 (56%) were male, while 334 (43%) were female faculty. In 2022, Clemson University took a significant step by institutionalizing the TIGERS ADVANCE project through the establishment of the Clemson University Faculty ADVANCEment Office. This strategic decision underscores the program’s recognized value and the university’s commitment to its ongoing success in supporting faculty.
Analysis of the qualitative data
A total of 252 participants responded to the open-ended question, “Please describe your experience with inclusion at Clemson University over the last year”, representing a response rate of 48%. In addition to these responses, 7 participants referred to inclusion at the university level when answering the question about the definition of inclusion, while 10 participants mentioned university climate in their responses to the departmental inclusion question. In total, 269 participants referenced university-level inclusion in the survey. Specifically, 144 (53%) of those who provided information about university-level inclusion were female faculty, 104 (39%) were male faculty, 136 (51%) had participated in at least one of the TIGERS ADVANCE activities, and 104 (39%) had not participated in any of the TIGERS ADVANCE activities. A total of 250 participants responded to the open-ended question, “Please describe your experience with inclusion in your department/work unit over the last year”, representing a response rate of 46%. Additionally, 4 participants referred to departmental inclusion when answering the question about the definition of inclusion, and 11 respondents mentioned departmental inclusion in their answers to the university inclusion question. In total, 265 participants referenced departmental inclusion in the survey. Of those who provided information about departmental inclusion, 134 (54%) were female faculty, 96 (36%) were male faculty, 135 (51%) had participated in at least one of the TIGERS ADVANCE activities, and 101 (38%) had not participated in any of the TIGERS ADVANCE activities.
Institutional and departmental inclusion sentiment
Two main themes, institutional inclusion sentiment and departmental inclusion sentiment, were identified from the responses provided to the open-ended questions. Each theme was further categorized into positive, negative, and mixed sentiment sub-themes using evaluation coding. Notably, institutional inclusion was perceived negatively by most participants (109), while departmental inclusion was viewed more positively, with 92 participants expressing positive sentiments. At the institutional level, changes that occurred over time were perceived mostly as negative (26 references), while at the departmental level, participants mostly (25 references) acknowledged that departments are making inclusive efforts, but there are still issues that need to be addressed. The most frequently cited stakeholder positive influence on institutional and departmental inclusion came from other faculty, identified as a key influence on institutional inclusion by 9 participants and on departmental inclusion by 21 participants. The most frequently perceived effective effort for institutional inclusion was the institutional commitment to inclusion, mentioned by 11 participants. The practice of feeling heard and valued within their departments was highlighted as an effective approach by 28 participants. Finally, participants mentioned inclusive climate more often in relation to their departments (46 respondents) than to the university (33 participants). Further analysis of participants’ negative and mixed sentiment references revealed perceptions of systemic barriers at both the institutional and departmental levels that limit the development of an inclusive climate. At the institutional level, faculty rank bias emerged as the most prevalent bias (12 references), with similar sentiments expressed at the departmental level (14 references).
While additional information can be found in the corresponding dissertation, the quantitative results indicated that the institutional climate at the university did not change between 2017 and 2024, and that the NSF TIGERS ADVANCE project did not contribute to changes in organizational climate. However, qualitative findings revealed notable differences between NSF TIGERS ADVANCE participants and non-participants. Participants reported more negative sentiments toward inclusion, greater concerns regarding the impact of university administration and the political climate on inclusion, and higher rates of negative observed experiences especially at the institutional level. Conversely, they were more likely to perceive a positive departmental climate and reported more positive direct experiences of inclusion within both the institution and their departments compared to non-participants. Additionally, participants viewed institutional changes more negatively than non-participants, though they recognized their departments’ efforts toward fostering a more inclusive climate, acknowledging room for improvement. In terms of gender differences, the quantitative data revealed gender differences in the organizational climate, specifically, female faculty showed higher levels of racial campus climate and burnout, but also unexpectedly higher levels of inclusive climate. Qualitative data verify some of these findings, showing that female faculty expressed more negative sentiments regarding inclusion, as well as concerns about bias and discrimination at the institutional level. Nonetheless, female faculty characterized their departmental climate as more inclusive and identified effective practices promoting inclusivity more frequently than their male counterparts.
Conclusions
Numerous studies have examined the effects of specific NSF ADVANCE program initiatives (Bilimoria & Liang, 2012; Plummer, 2006; Rosser & Chameau, 2006; Stewart et al., 2004), revealing a gap in understanding the broader impact of these initiatives on campus climates. This dissertation responds to that gap by providing a comprehensive evaluation of the NSF TIGERS ADVANCE project and its impact on organizational climate at Clemson University. The purpose of this case study dissertation was to evaluate the outcomes of the NSF TIGERS ADVANCE project on institutional transformation and faculty experiences with inclusion. The study sought to examine changes in organizational climate, determine the project’s contributions to these changes, and investigate differences in perceptions based on participation in the project and gender. While the quantitative results indicated no statistically significant change in organizational climate over the years, qualitative data revealed that faculty were aware of various changes occurring within the university. Many participants reported negative changes at the institutional level while acknowledging efforts at the departmental level that still have room for improvement. Notably, there were pronounced differences in perceptions of organizational climate change based on faculty participation in the TIGERS ADVANCE program and gender. Furthermore, the quantitative data indicated that the project did not significantly contribute to changes in organizational climate. However, qualitative results revealed notable differences between participants in the NSF TIGERS ADVANCE program and non-participants. Participants demonstrated greater awareness of issues and barriers related to inclusion, as well as effective inclusion practices. They expressed more negative sentiments toward inclusion at both the institutional and departmental levels, in contrast to non-participants, who exhibited a more positive sentiment regarding departmental inclusion. Additionally, participants reported more positive direct experiences with inclusion at both levels. Finally, the quantitative data revealed significant gender differences in perceptions of organizational climate, with female faculty reporting higher levels of racial tension and burnout, alongside perceptions of a more inclusive climate. Qualitative data further highlighted differences in inclusion experiences among genders, reflecting some of the same findings as the quantitative results. In contrast to the quantitative data, which suggested a more favorable outlook among female faculty regarding inclusion, qualitative responses indicated a more negative sentiment toward both institutional and departmental inclusion. Interestingly, while female faculty described the departmental climate as more inclusive compared to their male counterparts, they were also more likely to reference issues of race and ethnicity, identifying more instances of bias and discrimination. Notably, no faculty members provided insights about burnout in their qualitative responses, regardless of gender. These findings underscore the necessity of a mixed-methods approach in evaluating institutional transformation, as qualitative data provided nuanced perspectives that quantitative measures alone could not capture. By employing such an approach, this study offers valuable insights for institutional administration, highlighting current faculty perceptions of organizational climate, effective practices and policies, as well as barriers to inclusion efforts. Moving forward, future research should not only focus on the effectiveness of specific initiatives but also explore the intersectionality of gender, race, and other identities within institutional climates. This approach will facilitate a deeper understanding of the factors that contribute to institutional transformation within higher education institutions.
In conclusion, this study not only contributes to the existing literature on the NSF ADVANCE program evaluation but also serves as a model for other institutions on how to evaluate their efforts toward creating an equitable and inclusive climate for all faculty.
Want to know more?
If you have any questions regarding this post, please contact:
DR. CARLOS D. GARCIA
Faculty Fellow, Best Practices in Faculty Reviews
Office of Faculty ADVANCEment
cdgarci@clemson.edu
Please use this form to provide feedback, propose stories, or nominate a colleague to be featured (including self-nominations).
