Building an Assessment Program at Columbia University Libraries: Year One

Author(s): Jennifer Rutner, Assessment & Marketing Librarian (jenrutner@columbia.edu)
Jeff Carroll, Assistant Director for Collection Development (jc677@columbia.edu)
Conference: Library Assessment Conference, 2008
Date: 08/04/08

Download a PDF of the poster presentation from the conference.



In 2006, Columbia University Libraries (CUL) created the Program Coordinator for Marketing & Assessment position. This poster session details the work that has been done since then to build an effective, sustainable, practical assessment program at CUL.

CUL Assessment Plan, 2007 - 2009 (PDF)

The Assessment & Marketing Librarian gathered a small working group, the Assessment Team, in the fall of 2006 to write the CUL Assessment Plan. A relatively representative group, it consisted of the following staff:

Jeff Carroll, Collection Development
Joanna DiPasquale, Libraries Digital Programs
Jennifer Rutner, Assessment & Marketing
Bill Sees, Access Services
Jane Winland, Reference Services
Iris Wolley, Technical Services

  •  The Research & Writing Process
    • Environmental Scan
      The group conducted a literature review, and researched other library assessment plans available online. A brief bibliography is included in the Assessment Plan.
    • Staff Interviews
      The Assessment Team conducted a series of interviews with library staff to better understand the current assessment environment at the Libraries, identify needs and opportunities. The questions asked were based on the work of Steve Hiller and Jim Self.

      • 1. What type of assessment have you conducted in the past year (5 years)?
      2. Why? (What’s the motivation for these efforts?)
      3. Are any of these assessment efforts routine? Annual?
      4. Why? (Do you find value in routinely collecting this information?)
      5. Where is the collected data stored?
      6. How do you use the data that you collect? (be specific)
      7. Who oversees the assessment efforts in your library?
      8. Do you have plans for any new assessment programs in the coming year?
      9. Do you feel you have ample staff support, administrative support, and resources to conduct necessary assessment?
      10. What assessment would you like to do?
      11. What are the biggest barriers to conducting assessment in your library?
      12. What has been the most valuable assessment conducted in your library?
      13. What are your expectations for an Assessment Plan?

    • Recent Assessment Data
      The Assessment Team reviewed data from 2003 and 2006 LibQual+ surveys to identify areas in need of further assessment. Reference staff had also conducted a series of "User Input Interviews" with students and faculty, as part of the strategic planning process. Staff had an overwhelmingly positive experience conducting the interviews, and the Assessment Plan aimed to build on the momentum.
    • CUL Strategic Plan
      The Columbia University Libraries Strategic Plan, 2006 - 2009 was completed in the Fall of 2006, just as work was beginning on the Assessment Plan. The Assessment Team worked to align the goals of the assessment program with the Libraries strategic goals.
    • Culture of Assessment Survey
      An electronic survey was sent to professional staff, examining the culture of assessment at the Libraries. This survey was developed by Amos Lakos, and adapted for our needs. The Assessment Team was able to identify a few "red flag" areas that indicated where the culture of assessment was weakest.
  • Approval
    The Columbia University Libraries Management Committee and University Librarian approved the Assessment Plan in February 2007, authorizing the formation of the Assessment Working Group (AWG).
  • Implemention
    The Assessment Working Group is charged with implementing and updating the Assessment Plan. Thus far, as expected, some priorities have changed. The second CUL Assessment Plan will be developed in conjunction with the next CUL Strategic Plan.
  • Goals
    The assessment program will work to foster a culture of assessment at the Libraries, enable data-driven decision making at all staff levels, promote information transparency in the work place, improve service quality throughout the Libraries.

 

Assessment Working Group (AWG)

  • Charge
    "Initiate, support, and monitor a sustainable Assessment Program at CUL. Assist in identifying and maintaining assessment priorities. Carry out assessment projects. Ensure that appropriate training and support is provided to staff. Publish results of assessment initiatives as appropriate. Maintain the Assessment Center on SWIFT (staff intranet). Advocate for a culture of assessment at the Libraries."
  • Membership structure & selection process
    Expectations for membership were outlined when the AWG was created and are intended to define it as a working group rather than a standard committee. Members are expected to contribute work time to AWG projects, attend meetings, engage in professional development and training in areas of assessment, and act as advocates for assessment within their departments.

    The AWG consists of the Assessment & Marketing Librarian (chair) and four permanent members: The Director of Access Services, the Assistant Director of Collection Development, and a member of the Libraries Digital Program Division. There are two representative members, who rotate every two years, from the Reference Coordinating Committee and the Access Services Coordinating Committee. The Libraries Manager for Staff Training is an ex officio member of the AWG.

    Two spots on the AWG are open to any full-time member of staff. Applications were sought, and AWG permanent members reviewed the applications. This was a tremendously valuable approach to membership that allowed the AWG to diversify its perspectives and choose staff who were committed to assessment work.

  • Reporting
    The Assessment Working Group is an independent committee. The Director of Access Services acts as the liaison to the Libraries/Information Services Management Committee.

Successes

  • Several successful assessment projects have led to results at the Libraries, most notably the Digital Social Science Center User Needs Assessment, Geology Library Renovation Questionnaire, and the Virtual Reference Assessment Projects.
  • Learning what is not assessment: clarifying CUL's definition of assessment, and what projects are appropriate for the Assessment Working Group.
  • Assessment Forums: A speaker series developed for CUL staff, held three times a year. Speakers from libraries in the New York metropolitan area are invited to share their assessment success stories. Speakers have included librarians from New York University, Baruch College, Barnard College, and the Columbia Center for New Media Teaching & Learning. Assessment Forums have inspired requests for assessment projects from CUL staff.
  • Assessment-related goals have been included in the annual goals of many staff members at CUL.
  • Documentation to facilitate the management of assessment projects has been developed and is in pilot-phase. (Based on the work of the Cornell University Library Research and Assessment Unit.)
  • Assessment Working Group members have focused on staff development via a "book club," where we explore assessment methodologies used in other libraries.
  • All AWG members were trained in IRB procedures.

Challenges

  • Time & Timing: Assessment is most successful when done early and iteratively.
  • Working with various Libraries units, committees and task forces continues to challenge the project management cycle established for assessment projects. Client project management styles and expectations impact the success of assessment projects dramatically.
  • Assessment inspires a change in the organizational culture. Some library units are naturally more comfortable with assessment methodologies and applying results than others.
  • CUL has a complex data environment that will require an intensively collaborative, long-term, approach to building a CUL Data Center.
  • Maintaining a commitment to gathering and using quality data. This is an area where assessment staff are still building expertise, and clients are often frustrated by the perceived restraints of assuring validity. "Quick & dirty" is often appealing, but is it ever appropriate?
  • Refining the processes and procedures surrounding assessment projects. The AWG is learning from it's mistakes and successes.

Year Two

  • LibQual+ 2009: Involve Libraries/Information Services Management Committee in the process; enhance data analysis to meet the information needs of staff; increase response rate dramatically (from 2006); develop a system-wide response to the results and action plan; communicate results to the Columbia University community.
  • Continue to define the role of the AWG.
  • Continue to build the Data Center.
  • Create public website to communicate assessment results.
  • Involve more staff in assessment skill-building.
  • Continue to build expertise in assessment methodologies.
  • Assess assessment!