Page 25 - Southern PPI Final Report-5 16 19

Colleges and universities have adopted the use of rubrics as a way of standardizing grading and
improving the transparency of the grading process. Just as rubrics are useful in the classroom,
rubrics are also very useful for program prioritization—for much of the same reason. We use
rubrics to ensure that a standard method is consistently applied. It is used in the review of
academic and administrative programs. Studies show that by using both criteria and rubrics,
committee members are less likely to criticize the process and are more likely to support the
results, especially when they have a hand in developing them.

There were two parts to the development and use of rubrics. First, we went through a process
of ranking the criteria relative to each other. The purpose of this exercise was to get a sense of
what priority order committee members would place criteria. The weights associated with each
criterion will be an essential part of the evaluation process that is conducted at a later time. It
becomes the key basis for sorting and categorizing programs in the evaluation stage.

Once again, the committees were separated by administrative and academic groups. The
purpose of dividing them this way was to capture any unique considerations that might differ
between the two groups.

I5O led the teams through a process of ranking the criteria. Each member was given a ranking
sheet and asked to rank the 10 criteria from most important to least important. The I5O team
gathered the individual feedback and the resulting information is shown below. In advance of
the formal tallying of the ranking, members were asked to identify their top three criteria. They
were then asked to indicate their ranking by placing sticky dots beside the top three criteria
which were reproduced on charts at the front of the meeting room. The purpose of this interim
step was to give everyone an advanced preview of the criteria that each team considered
particularly important. As can be imagined, there was vigorous discussion about the top three
items, but the exchange was quite beneficial and is a critical part of increasing group buy-in of
the process.

Criteria Number and Description 1st 2nd Final

1 Essentiality & Importance to SUBR 13% 11% 12%
2 External Demand for the Program/Unit 11% 13% 12%
3 Internal Demand for the Program/Unit 13% 11% 12%
4 Quality of Program/Unit Inputs and Processes 9% 12% 11%
5 Quality of Program/Unit Outputs/Outcomes 12% 11% 11%
6 Size, Scope and Productivity Program/Unit 8% 9% 8%
7 Revenue Generated by Program/Unit 8% 7% 7%
8 Cost and Other Related Expenses to Program/Unit 6% 7% 7%
9 Impact, Justification, and Overall Essentially or Value of 14% 12% 13%
Program/Unit
10 Opportunity Analysis of Program/Unit 6% 7% 7%

The next part of the process was to establish rubrics for each criterion. These rubrics are used
to define what each criterion means. While the specific rubric items that defined the criteria
were proposed to the team (based on the literature), the members had the opportunity to adjust
and to fine-tune the specific items and language. Once the rubric items were agreed upon, they

25 | P a ge S O U T H E R N U N IVE R S IT Y