Rubrics and Portfolios

Part 1

Objectives

  1. Describe the purposes for the use of rubrics.
  2. Develop a rubric that could be used to evaluate students in your course or program.
  3. Describe how a portfolio could be developed for use as a student evaluation tool.
  4. Describe how to assess a student portfolio.

Rubrics

Part 2

Evaluation/Assessment Methods & Tools

A. Rubrics

Introduction

For some learning tasks where simple “completion” of a number of requirements is the prime objective, a checklist can be an adequate assessment tool.  However, if the learning requires more complicated performance, a rubric may be of greater utility. Examples of tasks in which rubrics can be valuable assessment tools include projects, writings, portfolios, and presentations. If the type of student performance requires interpretation, a rubric can be used to identify and characterize the ranges of performance that might be observed.  If evaluation of a student is conducted by several different people and interpretation of performance is required, a rubric will help all involved focus on the specific aspects of the performance being evaluated.  A rubric can also be an excellent learning tool for students because it clearly defines for them performance expectations.

Definition

A rubric is an assessment tool used to evaluate a range of student performance across several different categories or criteria.  For each category or criterion, a rubric defines the specific attributes that will be used to score or judge the student’s performance and to differentiate between different levels of performance.  Rubrics take time to develop since one has to: a) identify the criteria or categories of performance, b) determine how many levels of performance will be characterized, and c) describe clearly the different levels of performance for each category.

Categories of Performance

The categories or criteria of performance are usually given in the rows of a rubric.  They should comprise all the essential elements that must be present for the student’s work or performance to be of high quality.  For example, developing a rubric for a case study write-up might include the following categories of performance:

Category

       

Background

       

Problem Identification and Issues

       

Response or Strategy

       

Recommendations

       

Writing Quality

       

Format

       

Levels of Performance

Once the categories or criteria are identified (rows), one must then decide on how many levels of performance will be described (columns).  Keep in mind that the levels of performance should be designed to differentiate among the range of truly outstanding and truly inadequate work.  To indicate each level of performance, one could simply identify scores such as 0, 1, 2, 3, or 4.  Although you might understand what level of learning occurred by assigning a score of a “1” or a “4,” no one else would, and this approach would not be fair to the students or enhance their learning. To reduce ambiguity and to assist students in their learning, one must think through the different levels of performance for each category and describe the key features that represent each level.

In the example above, note that the first four categories address the content of a case study write-up.  The last two categories address writing mechanics and organization, both of which support the ultimate usability of the case study. Both the content of the case study and how it is written and organized constitute the purpose of the learning task. However, different levels of performance may occur across the task.  Defining these different levels should be based on the specific characteristics that distinguish excellent from deficient performance.  This is somewhat easier when you, the instructor, have experience with the learning task.  Constructing a rubric for a new assessment that you have not had much experience with will be a greater challenge.  In this situation, share your draft with colleagues for suggestions and improvements; this will save you a lot of headaches.

When creating a rubric, you also have to decide what constitutes a grade or point reward for each level of performance. In Example 1, a letter grade was assigned to each level.  As shown in Example 2, one could also use a point system and the performance across all categories would be used to assign the grade, based on the total number of points.

Example 1

Category A B C Unacceptable

Background

Helps reader to understand the case; all important details included

Appropriate range but lacks details in some areas

Inadequate range and missing important details

Insufficient range and detail provided

Problem identification and issues

Identified the critical problem and component issues

Identified the essential problem but needs clarification

Identified the wrong problem or missed some key component issues

Did not include problem identification or issues

Response or strategy

Clear list of responses; match of responses to problem

Mix of appropriate strategies but not prioritized

Some inappropriate responses to problem

Did not supply a response to problem or responses did not address problem

Recommendations

Prioritized list that addressed all areas of the problem

Appropriate list but too long or too broad

Some items  outside the problem or not backed by the facts of the case

Did not include or not appropriate for problem

Writing Quality

Concise, consistent writing

Some inconsistencies across document

Needs significant editing

Inconsistent, rambling, unable to comprehend

Format

Included all sections and formatted consistently

Included all sections but some format inconsistencies

Used a different format but did not justify in cover letter

Did not include required sections or use a format

Example 2

Category Excellent (3) Good (2) Marginal (1) Unacceptable (0)

Background

Helps reader to understand the case; all important details included

Appropriate range but lacks details in some areas

Inadequate range and missing important details

Insufficient range and detail provided

Problem identification and issues

Identified the critical problem and component issues

Identified the essential problem but needs clarification

Identified the wrong problem or missed some key component issues

Did not include problem identification or issues

Response or strategy

Clear list of responses; match of responses to problem

Mix of appropriate strategies but not prioritized

Some inappropriate responses to problem

Did not supply a response to problem or responses did not address problem

Recommendations

Prioritized list that addressed all areas of the problem

Appropriate list but too long or too broad

Some items  outside the problem or not backed by the facts of the case

Did not include or not appropriate for problem

Writing Quality

Concise, consistent writing

Some inconsistencies across document

Needs significant editing

Inconsistent, rambling, unable to comprehend

Format

Included all sections and formatted consistently

Included all sections but some format inconsistencies

Used a different format but did not justify in cover letter

Did not include required sections or use a format

Total Points = _____    Grading Scale: A = 16 to 18 points; B = 10 to 15 points; C = 5 to 9 points; Fail = 0 to 4 points

You will note that interpretation may still be necessary in many of the cells, but a well-developed rubric reduces interpretation or subjectivity in evaluating student performance. Generally, a rubric must be actually used in order to identify its weaknesses.  A rubric should be evaluated by others, including peers and students, and modifications made as needed for improvement.

Additional Tips for Rubric Development and Use

  • Rubrics can be used to track student performance across time (e.g., across the semester, across an academic program).  This makes the rubric an excellent tool to use as part of a comprehensive program assessment plan.
  • Rubrics should be shared with students up front.  Make sure students understand the categories and the expected levels of performance that represent high quality.
  • Obtain student feedback to continue to improve the rubric categories, the ranges or levels of performance, and the descriptions of each level.
  • When developing rubrics for specific tasks or assignments, include categories or criteria that reflect the specific knowledge or knowledge applications required for successful completion of the tasks or assignments.
  • Consider including criteria in the rubric that reflect important aspects of the process needed to successfully complete the task or assignment.  For example, if critical evaluation of the literature is necessary in order to complete a task or assignment, add a criterion that reflects the extent to which students could perform that critical evaluation.

Part 3

Rubrics Exercise

In this exercise, you will build your own rubric.  Use the table below as a template for each of the four tasks described.

Task 1 – Rubric Purpose

  • Identify a learning task that might be suitable for a rubric form of evaluation and describe its use in your course.  Write these into the table.
  • What does the rubric intend to evaluate?

Task 2 – Performance Categories

  • Identify the different categories of performance that comprise the learning task and list them in the rubric rows.

Task 3 – Levels of Performance

  • Determine the number of different performance levels that you would like for the categories (there are 4 in the template, but feel free to alter this if you feel it best).  For each performance level, write in the rubric brief statements that will distinguish clearly among the different levels of performance in each category.

Task 4 – Rubric Evaluation

  • How will student performance equate into a grade?  Label each level of performance (columns) to reflect this. 
  • Have you identified adequate and complete performance categories? What’s missing or what can be consolidated?
  • Have you written out appropriate statements for levels of performance?  Would they truly differentiate excellence or expert performance from poor or novice performance? This is not always as easy as it looks.  If you can, try out your rubric and see how well it works.

Assessment for:

Description of use:

Levels of Performance

Category

       
 
 
       
 
 
       
 
 
       
 
 
       
 
 
       
 
 
       

Portfolios

Part 4

B. Portfolios

Introduction

Bill Cosby described in a comedy routine the purpose for the house as “a place for my stuff.” In professional circles, file cabinets, desks, bins, hard discs, shelves, and boxes are also places for our stuff. But unless we know what is in them, these places for our stuff do not help us out  much.

Definition

What is a portfolio?  A portfolio is a purposeful collection of work that demonstrates one’s competence or range of effort.  They can be all-inclusive, e.g., the portfolio includes all the relevant work created by a student in a specific course or a program, or they can be selective, e.g., the portfolio includes selected items of a student’s work.  Self-reflection by students about their work included is often a part of a portfolio and can be of value to both the instructor and the learner.  A portfolio should not be just a haphazard or random collection of materials in a folder.  Without a clear purpose and an organizing strategy, a collection of work is just “stuff” and not a portfolio.

Purposes in Academia

There are several potential uses for portfolios.  They can be used to promote learning as well as to assess learning.  When used to promote learning, student reflection about the contents and their use of judgment in identifying materials to include are very important components. Portfolios can be used for a variety of types of assessments, including student assessment, faculty assessment, and program assessment.  Some of the uses of portfolios for each of these types of assessment are shown in the Table.  Faculty often use portfolios to document their teaching efforts as well as tools or innovations used to promote student learning.  Similar types of portfolios can be used to document service efforts, particularly for non-tenure or clinical track faculty who have significant service responsibilities, or to document research efforts for faculty with significant research/scholarship expectations.

Table - Uses for Portfolios in Assessment

Student Portfolios

Faculty Portfolios

Academic Programs

  • Course products
  • Academic program products
  • Job applications
  • Tenure-track documentation
  • Teaching documentation
  • Research documentation
  • Service documentation
  • Certification exhibits
  • Student achievements
  • Teaching accomplishments
  • Research archives

Portfolios can be used as a comprehensive form of student assessment in a course in which specific course work or products are collected and compiled for review by the professor or outside reviewers.  In some programs, a formal “portfolio review” is necessary for student advancement in that program of study.  In classroom use, points or a percentage of the final grade would be assigned for the portfolio; these might include the presentation of the portfolio in addition to the portfolio contents themselves. To assess the portfolio and determine the points or percentage grade that the student will receive, a rubric might be necessary to evaluate the student’s work using various desired criteria and performance expectations.

For program assessment, faculty or perhaps an assessment committee can examine portfolios of average or outstanding students, or a random selection of students, to gain evidence about the extent of student mastery of desired program learning outcomes.  Similar to its use for individual student assessment, a rubric can be useful for compiling data about actual student performance on desired program outcomes.   The information gained by faculty can them be used to gauge the success of their program in accomplishing its educational mission and to identify areas of the curriculum that should be enhanced or modified.

Organizing Strategies

Once the decision is made to use a portfolio for assessment, consideration must be given to how the portfolio is to be prepared and organized.  There are two main types of storage options, analog and digital.  Examples of each include:

  • Analog storage: piles, file folders, bins, 3-ring notebooks
  • Digital storage: personal hard disc, ZIP disks, CD-ROMs, DVDs, remote computer server (web site)

The artifact categories, or categories to be used for organizing the work or products created, depends on the purpose for the portfolio.  The following examples illustrate some portfolio organization strategies:

  • A portfolio developed for a position application should be organized around categories identified in the position announcement, such as education, skills, job experience, capabilities, and work samples.
  • A portfolio developed for a course could be organized around learning task requirements, such as tests, papers, presentations, problem sets, etc.
  • A portfolio used by faculty in a tenure-track portfolio should be organized using Teaching, Research/Scholarship, and Service as major categories.
  • A portfolio used by a program to assess student learning could be organized around formal program student learning outcomes or competencies, or it could be organized using categories determined by the student, such as courses, presentations, papers, thesis and/or dissertation work, or service activity.

Artifact documentation, or how the work or products created will be labeled or documented within the portfolio, can take two forms:  paper or digital.  Examples of each are shown below: 

  • Paper forms: tab system, labels, annotations explaining artifact.
  • Digital forms: folder names, file name formats, front-end hyperlinks to files

Finally, consideration should be given to the type of digital front-end software to be used for digital portfolios.  Digital portfolios can consist of storage media and folders.  Digital portfolios can also include a file that consists of hyperlinks “pointing” to the actual files on the storage media, whether a web site or local digital media.

Considerations in Using Portfolios for Assessment

There are several questions that should be asked when the use of a portfolio for learning assessment is being planned.  These can be summarized as follows:

  • What specific learning outcomes will be assessed?
  • Will all the student works be included, or only a selection of their work?  If just a selection of their work, by what criteria will the selections be made? By whom? What type(s) of work will be included?
  • What will be the role(s) of the student in the portfolio process?
  • What will be the role(s) of the faculty member(s) in the portfolio process?
  • How will the portfolios be stored and organized?  By whom?
  • Who will compile and maintain the portfolios?
  • Is privacy an issue?  If so, how will this be managed?
  • How often will the portfolios be reviewed?  By whom?
  • What specific criteria will be used for portfolio evaluation?  Will a rubric be developed?  Will the criteria or rubric be used for all the works in the portfolio, or for a selection of works? 
  • How will points or a grade be assigned to the criteria used?
  • Will self-reflection by students be incorporated as part of the portfolio?

In summary, a portfolio can be a very useful tool for a variety of assessment purposes.  Careful thought and planning must go into a portfolio to ensure that it is organized and used appropriately and effectively.

Part 5

Portfolio Exercise

First, identify a course or assessment need in your program for which a portfolio might be of value.  Next, work through the following questions and activities:

Question 1 – Portfolio Purpose?

  • Identify the purpose for your portfolio.
  • What do you want to have happen as a result of use of this portfolio?
  • Who will review the portfolio?  When?

Question 2 – Category System?

  • How should the portfolio be organized?  Will it be all-inclusive or selective?
  • What are the categories for the portfolio?
  • How do these categories support the purpose of the portfolio?
  • How will the materials in these categories be evaluated?

Question 3 – Storage-Archival System?

  • Will a physical system or a digital system or both be used?
  • How should the portfolio materials be stored?
  • How should the materials in the portfolio be indexed (e.g., tab system)?
  • What artifacts (works, products) will need explanations or annotations in the portfolio to be understood?

Question 4 – Digital Storage? (For this question, assume digital storage will be used)

  • What will be the names for folders and how will you organize folders and sub-folders (if needed)?
  • What will be your format for file names?
  • Where will the portfolio files/folders be stored and what back-up provisions should be made?
  • What front-end software will you use?
  • What will the front-end file look like? (hyperlinking structure and page layout)